From nobody Sun Aug 18 20:44:09 2024 X-Original-To: dev-commits-ports-all@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4Wn74T6q69z5THXr; Sun, 18 Aug 2024 20:44:09 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "R11" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4Wn74T4lBjz4DnG; Sun, 18 Aug 2024 20:44:09 +0000 (UTC) (envelope-from git@FreeBSD.org) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1724013849; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=fhOIlXnZEBsfLb5ZuWRr3KiGaLy85JQk2UEtg+VNN/8=; b=cDkqU5jvwmPULiwQ01AxPFGbUhlsjvdjvLtwVfS9pW29decCkGR6Fc2cWVCVu/lwF8CH3A JvesewZCbzFQpnmIMmVl06zRv8DwI5xnZrQsrYkIXfa0nDDUCXBb7nUjWXgyhqa5V/CMj4 yDPqAiQKIQHPqnc6Lnx+HDyV1C01fXbgg9yBDgUGdXmfnsiyaIDvRpy+dZy0j9/W2tgEM8 DH4GvWMRgdPM2e/01R/HInXn3oVw5gk/CSdh4KzWIKnV2vnzsSX59gxU7yapwNyTz08r6w KMJA/UMmrXVdcXAUNMYXkLbinuz8eATu56+JbOE7nnCp0+cKEYpvgtqOeZyLSg== ARC-Seal: i=1; s=dkim; d=freebsd.org; t=1724013849; a=rsa-sha256; cv=none; b=DnyDis4/LXTp136n2tllzAg8dy4w6+2RCwnM35kPqZFJaW9YKqIojrv+e17TrmsFdqzlZG 4N2iXzxwwt28wqHqs1Dg0IyNEdYl/5aqUf9dHx4zKNyF/dylb6rwU23GODXDTFwTZEAn8v fNfo9fmtbZV+dC7bQ27M65qOJMDxKAk4fdcUKaH4xmNHiXuPVnnKxS0Gxqnh08YXLzwyor hgAQB3B9DgRnLoa7zJWJqm3/0HiRpz+OqcXx1ABF6VqbC+lyQm/rZP4105srAJNAmIMvLK ytHwg1PJAhxEtbJ5alORHown7fnkXqei/KbQqd6n8imsXzu2HvBRNCgv4Xd7Hw== ARC-Authentication-Results: i=1; mx1.freebsd.org; none ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1724013849; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=fhOIlXnZEBsfLb5ZuWRr3KiGaLy85JQk2UEtg+VNN/8=; b=YZyEetFwL62ayj+Dx4npLaG5n8zLq9LHCmBfVFFifNqk1YoSQwIU0R0EnxSyU65M9JXKlX FzjBUIk2IX/GbFshVjwogutKhC3qhaXq4t1wpNE2PoJAVp4ZzFZ5nwzCDn55fSGNGIpuqn aw8v3NDm1WPH6Uj6OEfgMX7aql3V5B9atrXfvLm/TuCtkHFVDF2ilDddoQ2svm3V0xfZZR QzaT7E/V25q/4TZyWjDuhZ/yf1vmL1dBS/05Mntbg1GXwhcdYblvSfIE3yLtwSkMo7zxGE fFH3g65eVP9hgUpRTZMxMQstO9GEZuuZaAbHkC/rHICQQq1sjdbnXSX0Mtab4A== Received: from gitrepo.freebsd.org (gitrepo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:5]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 4Wn74T3G6gznwR; Sun, 18 Aug 2024 20:44:09 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from gitrepo.freebsd.org ([127.0.1.44]) by gitrepo.freebsd.org (8.18.1/8.18.1) with ESMTP id 47IKi9EZ083148; Sun, 18 Aug 2024 20:44:09 GMT (envelope-from git@gitrepo.freebsd.org) Received: (from git@localhost) by gitrepo.freebsd.org (8.18.1/8.18.1/Submit) id 47IKi92X083145; Sun, 18 Aug 2024 20:44:09 GMT (envelope-from git) Date: Sun, 18 Aug 2024 20:44:09 GMT Message-Id: <202408182044.47IKi92X083145@gitrepo.freebsd.org> To: ports-committers@FreeBSD.org, dev-commits-ports-all@FreeBSD.org, dev-commits-ports-main@FreeBSD.org From: Yuri Victorovich Subject: git: 406cb7fcfffc - main - misc/ollama: update 0.3.4 =?utf-8?Q?=E2=86=92?= 0.3.6 List-Id: Commit messages for all branches of the ports repository List-Archive: https://lists.freebsd.org/archives/dev-commits-ports-all List-Help: List-Post: List-Subscribe: List-Unsubscribe: X-BeenThere: dev-commits-ports-all@freebsd.org Sender: owner-dev-commits-ports-all@FreeBSD.org MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit X-Git-Committer: yuri X-Git-Repository: ports X-Git-Refname: refs/heads/main X-Git-Reftype: branch X-Git-Commit: 406cb7fcfffcd69ac3e614f887f2d705127b50e4 Auto-Submitted: auto-generated The branch main has been updated by yuri: URL: https://cgit.FreeBSD.org/ports/commit/?id=406cb7fcfffcd69ac3e614f887f2d705127b50e4 commit 406cb7fcfffcd69ac3e614f887f2d705127b50e4 Author: Yuri Victorovich AuthorDate: 2024-08-18 20:43:52 +0000 Commit: Yuri Victorovich CommitDate: 2024-08-18 20:44:06 +0000 misc/ollama: update 0.3.4 → 0.3.6 --- misc/ollama/Makefile | 14 ++++++++--- misc/ollama/distinfo | 16 ++++++------ misc/ollama/files/ollama-limit-layers | 13 ++++++++++ misc/ollama/files/patch-FreeBSD-compatibility | 29 +++++++++++++++++++--- ...lama.cpp_ggml_src_vulkan-shaders_CMakeLists.txt | 5 ++-- misc/ollama/pkg-message | 7 ++++++ 6 files changed, 67 insertions(+), 17 deletions(-) diff --git a/misc/ollama/Makefile b/misc/ollama/Makefile index 1995935ddfae..be457b2db084 100644 --- a/misc/ollama/Makefile +++ b/misc/ollama/Makefile @@ -1,7 +1,6 @@ PORTNAME= ollama DISTVERSIONPREFIX= v -DISTVERSION= 0.3.4 -PORTREVISION= 4 +DISTVERSION= 0.3.6 CATEGORIES= misc # machine-learning MAINTAINER= yuri@FreeBSD.org @@ -28,11 +27,13 @@ GO_MODULE= github.com/${PORTNAME}/${PORTNAME} GO_TARGET= . USE_GITHUB= nodefault -GH_TUPLE= ggerganov:llama.cpp:6eeaeba:llama_cpp/llm/llama.cpp +GH_TUPLE= ggerganov:llama.cpp:1e6f6554aa11fa10160a5fda689e736c3c34169f:llama_cpp/llm/llama.cpp \ + blabber:go-freebsd-sysctl:503969f:go_sysctl/vendor.x/github.com/blabber/go-freebsd-sysctl MAKE_ENV= PATH=${PATH}:${WRKSRC}/llm/build/bsd/x86_64_static/bin # workaround to find vulkan-shaders-gen -PLIST_FILES= bin/${PORTNAME} +PLIST_FILES= bin/${PORTNAME} \ + bin/ollama-limit-layers post-patch: # workaround for https://github.com/ollama/ollama/issues/6259 (use of extenral libllama.so) @${REINPLACE_CMD} \ @@ -42,6 +43,8 @@ post-patch: # workaround for https://github.com/ollama/ollama/issues/6259 (use o s| ggml | ${WRKSRC}/llm/build/bsd/x86_64_static/ggml/src/libggml.a |; \ ' \ ${WRKSRC}/llm/ext_server/CMakeLists.txt + # move vendor.x to vendor + @(cd ${WRKSRC}/vendor.x && ${TAR} cf - .) | (cd ${WRKSRC}/vendor && ${TAR} xf -) pre-build: @${CP} ${WRKSRC}/app/store/store_linux.go ${WRKSRC}/app/store/store_bsd.go @@ -49,4 +52,7 @@ pre-build: ${SETENVI} ${WRK_ENV} ${MAKE_ENV} ${GO_ENV} GOMAXPROCS=${MAKE_JOBS_NUMBER} GOPROXY=off ${GO_CMD} generate ${GO_BUILDFLAGS} \ ./... +post-install: # pending https://github.com/ollama/ollama/issues/6407 + ${INSTALL_SCRIPT} ${FILESDIR}/ollama-limit-layers ${STAGEDIR}${PREFIX}/bin + .include diff --git a/misc/ollama/distinfo b/misc/ollama/distinfo index c436195aa415..86cae5d113d3 100644 --- a/misc/ollama/distinfo +++ b/misc/ollama/distinfo @@ -1,7 +1,9 @@ -TIMESTAMP = 1722929347 -SHA256 (go/misc_ollama/ollama-v0.3.4/v0.3.4.mod) = ec7b6ce8fd820503671d269fd43ea5ae7275b1ff095c77d6422ac8540ccb410b -SIZE (go/misc_ollama/ollama-v0.3.4/v0.3.4.mod) = 2992 -SHA256 (go/misc_ollama/ollama-v0.3.4/v0.3.4.zip) = d9bc3d3901272f8fbc9b57ce1af1caa2209127978bb5f72257b6a3fb84161969 -SIZE (go/misc_ollama/ollama-v0.3.4/v0.3.4.zip) = 1832031 -SHA256 (go/misc_ollama/ollama-v0.3.4/ggerganov-llama.cpp-6eeaeba_GH0.tar.gz) = f749f494dc5e672926179524cf28c834151edba5acae18e132c954df76ffdf33 -SIZE (go/misc_ollama/ollama-v0.3.4/ggerganov-llama.cpp-6eeaeba_GH0.tar.gz) = 19003544 +TIMESTAMP = 1724010094 +SHA256 (go/misc_ollama/ollama-v0.3.6/v0.3.6.mod) = 16c078d8f0b29f84598fb04e3979acf86da41eb41bf4ff8363548e490f38b54e +SIZE (go/misc_ollama/ollama-v0.3.6/v0.3.6.mod) = 2992 +SHA256 (go/misc_ollama/ollama-v0.3.6/v0.3.6.zip) = 94d2376c9555dd03a76cb093f3213e8155811874db7eab6aff2941d0e75dce07 +SIZE (go/misc_ollama/ollama-v0.3.6/v0.3.6.zip) = 1842735 +SHA256 (go/misc_ollama/ollama-v0.3.6/ggerganov-llama.cpp-1e6f6554aa11fa10160a5fda689e736c3c34169f_GH0.tar.gz) = b2b3137f734fc9a202fac710986f1de837e0ae69a0b532d4cbadb90748f4cb73 +SIZE (go/misc_ollama/ollama-v0.3.6/ggerganov-llama.cpp-1e6f6554aa11fa10160a5fda689e736c3c34169f_GH0.tar.gz) = 19016896 +SHA256 (go/misc_ollama/ollama-v0.3.6/blabber-go-freebsd-sysctl-503969f_GH0.tar.gz) = 1f497780d88f6f056b8d8f59b6aa129566c8041d16d1cda89f212accf88f3165 +SIZE (go/misc_ollama/ollama-v0.3.6/blabber-go-freebsd-sysctl-503969f_GH0.tar.gz) = 2117 diff --git a/misc/ollama/files/ollama-limit-layers b/misc/ollama/files/ollama-limit-layers new file mode 100755 index 000000000000..6713a9f9b02b --- /dev/null +++ b/misc/ollama/files/ollama-limit-layers @@ -0,0 +1,13 @@ +#!/bin/sh + +MODEL=$1 +NUM_GPU=$2 + +if [ -z "$MODEL" ] || [ -z "$NUM_GPU" ]; then + echo "Usage: $0 " +fi + +ollama show --modelfile $MODEL > Modelfile && +echo "PARAMETER num_gpu $NUM_GPU" >> Modelfile && +ollama create "$MODEL-num_gpu$NUM_GPU" -f Modelfile && +echo "model variant $MODEL-num_gpu$NUM_GPU was created" diff --git a/misc/ollama/files/patch-FreeBSD-compatibility b/misc/ollama/files/patch-FreeBSD-compatibility index ca206363f042..046ca9b3417c 100644 --- a/misc/ollama/files/patch-FreeBSD-compatibility +++ b/misc/ollama/files/patch-FreeBSD-compatibility @@ -1,14 +1,19 @@ -- patch based on https://github.com/ollama/ollama/issues/1102#issuecomment-2270042340 new file mode 100644 ---- gpu/gpu_bsd.go.orig 2024-08-06 07:41:26 UTC +--- gpu/gpu_bsd.go.orig 2024-08-18 20:03:12 UTC +++ gpu/gpu_bsd.go -@@ -0,0 +1,101 @@ +@@ -0,0 +1,122 @@ +//go:build dragonfly || freebsd || netbsd || openbsd + +package gpu + +import "github.com/ollama/ollama/format" ++//import sysctl "github.com/lorenzosaino/go-sysctl" // sysctl: this is Linux-only, see https://github.com/lorenzosaino/go-sysctl/issues/7 ++import sysctl "github.com/blabber/go-freebsd-sysctl/sysctl" // sysctl: this is FreeBSD-only basic library ++import ( ++ "log/slog" ++) + +/* +#cgo CFLAGS: -I/usr/local/include @@ -98,8 +103,24 @@ new file mode 100644 +} + +func GetCPUMem() (memInfo, error) { -+ size := C.sysconf(C._SC_PHYS_PAGES) * C.sysconf(C._SC_PAGE_SIZE) -+ return memInfo{TotalMemory: uint64(size)}, nil ++ // all involved sysctl variables ++ sysctl_vm_page_size, _ := sysctl.GetInt64("vm.stats.vm.v_page_size") // memory page size ++ sysctl_hw_physmem, _ := sysctl.GetInt64("hw.physmem") // physical memory in bytes ++ sysctl_vm_free_count, _ := sysctl.GetInt64("vm.stats.vm.v_free_count") // free page count ++ sysctl_vm_swap_total, _ := sysctl.GetInt64("vm.swap_total") // total swap size in bytes ++ ++ // individual values ++ total_memory := uint64(sysctl_hw_physmem) ++ free_memory := uint64(sysctl_vm_free_count) * uint64(sysctl_vm_page_size) ++ free_swap := uint64(sysctl_vm_swap_total) // wrong to use the total swap size here, should be vm.swap_free, see https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=280909 ++ ++ slog.Debug("gpu_bsd.go::GetCPUMem::GetCPUMem", "total_memory", total_memory, "free_memory", free_memory, "free_swap", free_swap) ++ ++ return memInfo{ ++ TotalMemory: uint64(total_memory), ++ FreeMemory: uint64(free_memory), ++ FreeSwap: uint64(free_swap), ++ }, nil +} + +func (l GpuInfoList) GetVisibleDevicesEnv() (string, string) { diff --git a/misc/ollama/files/patch-llm_llama.cpp_ggml_src_vulkan-shaders_CMakeLists.txt b/misc/ollama/files/patch-llm_llama.cpp_ggml_src_vulkan-shaders_CMakeLists.txt index 3ada7c2debc2..a9ef6bec163e 100644 --- a/misc/ollama/files/patch-llm_llama.cpp_ggml_src_vulkan-shaders_CMakeLists.txt +++ b/misc/ollama/files/patch-llm_llama.cpp_ggml_src_vulkan-shaders_CMakeLists.txt @@ -1,9 +1,10 @@ ---- llm/llama.cpp/ggml/src/vulkan-shaders/CMakeLists.txt.orig 2024-08-08 21:55:59 UTC +--- llm/llama.cpp/ggml/src/vulkan-shaders/CMakeLists.txt.orig 2024-08-06 15:33:39 UTC +++ llm/llama.cpp/ggml/src/vulkan-shaders/CMakeLists.txt -@@ -1,5 +1,6 @@ add_executable(${TARGET} vulkan-shaders-gen.cpp) +@@ -2,6 +2,7 @@ add_executable(${TARGET} vulkan-shaders-gen.cpp) set(TARGET vulkan-shaders-gen) add_executable(${TARGET} vulkan-shaders-gen.cpp) +target_link_libraries(${TARGET} PRIVATE pthread) install(TARGETS ${TARGET} RUNTIME) target_compile_features(${TARGET} PRIVATE cxx_std_11) + target_link_libraries(vulkan-shaders-gen PUBLIC Threads::Threads) diff --git a/misc/ollama/pkg-message b/misc/ollama/pkg-message index 90096ac82cef..95b0d4e58448 100644 --- a/misc/ollama/pkg-message +++ b/misc/ollama/pkg-message @@ -15,6 +15,13 @@ You will be able to interact with it in plain English. Please see https://ollama.com/library for the list of all supported models. +The command "ollama list" lists all models downloaded +into your system. + +When the model fails to load into your GPU, please use +the provided ollama-limit-layers script to create +model flavors with different num_gpu parameters. + ollama uses many gigbytes of disk space in your home directory, because advanced AI models are often very large. Pease symlink ~/.ollama to a large disk if needed.