From nobody Mon Aug 19 01:12:11 2024 X-Original-To: dev-commits-ports-main@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4WnF1l4qqgz5TgM0; Mon, 19 Aug 2024 01:12:11 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "R11" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4WnF1l4CB7z4gRh; Mon, 19 Aug 2024 01:12:11 +0000 (UTC) (envelope-from git@FreeBSD.org) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1724029931; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=FD7MUU6Hk7pHI2+YVWNcsQKHy/j6RCw7F9/9Q2DOYag=; b=K/AKQQ0PDgJJckzAVyEGyFb2ugtf3HE0KSFi/Ht2vKPcOOSqz05AvqsV0fjbgYeHdYWEyI WP7kwpexPoN4xzaUCAQUYYXDbluRvRK6lc/8IKp1YoTNv0Q4a6LTZ15N8PK2VS3uAzs+ea yfoOsbG7bPWuOHe3p/tVUWx7/atIkLe3f/6PWVoKzc9SyLOW5eOS9nOGqTkImfCQtVN6wP zVhMt7x1tY7s+1bz8g8+nnpErrmdYPFoqu78ATAKGS2sCmXUPFrZPMHYgn1QDV5fMlqQ5A CVB3u7e2i+82oH7QVk+h6oZsSHNvAF/Cm4G0gJqZvLL/a//61efS97uf8LuW9A== ARC-Seal: i=1; s=dkim; d=freebsd.org; t=1724029931; a=rsa-sha256; cv=none; b=e//EEy54E1ut+6opvn7OwUIBnEI5zVrVV+3kLtKNiXcyF7hu3nTGSMdCaFqTxLExJA7gC3 modThraBvf/al8ohzjgtYCVfzWFpDBn6nflMa0vWNcFFPueK6tRl6OhwFzylhkjXYNWb2r t7Ap5A/UZAsBBTzsB/lwj717E0iTrt4Bk0PBhfU6ialP1m9sMAL1dMnyKnnobc6aD7thyO tOluG+axS+WEPjjkASUUZFZf34Q9kQTQI5Q+i07dFmi0DSpF1t3dh6kmlUMNskP9rv+JNT 5SU4un5zrQ1e0hzkRL7mUWwDYVE2tU5mEWOrqhD/iQplA+r2x01cxTP93R9F3A== ARC-Authentication-Results: i=1; mx1.freebsd.org; none ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1724029931; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=FD7MUU6Hk7pHI2+YVWNcsQKHy/j6RCw7F9/9Q2DOYag=; b=ncuynm6SqtyzaO3UCLEefG2qSZVH1jxt25crqu6V6CUA337bJdNy+52wW6d9Na+JMWnxPR AmVphVmARtHrzZEnfZRt0CyfDixxyxj/4ddhBSPOrdG/eusMpI+HwrtUWvw0ZtOCnUR/1O KcsSORT80Y8H/ZgUdeaqQEbWCg4qalmmBpnKIs0q58av00K1KTJmbjiWXiPap6y/YPZt2V vjbyFKwbSl5AQaDhDuRjidQCuXtlZ2UF7ss1OeiwzMZGNQ8MEf88/oPiDfB8WOGzFbgBBt ppCXtcskmyYUliyn0VRrjYC9SAGYtolbImGnxAy6Iuz9iXsAbbU2BI7oSKRJsw== Received: from gitrepo.freebsd.org (gitrepo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:5]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 4WnF1l3pGgzvrq; Mon, 19 Aug 2024 01:12:11 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from gitrepo.freebsd.org ([127.0.1.44]) by gitrepo.freebsd.org (8.18.1/8.18.1) with ESMTP id 47J1CBlu038069; Mon, 19 Aug 2024 01:12:11 GMT (envelope-from git@gitrepo.freebsd.org) Received: (from git@localhost) by gitrepo.freebsd.org (8.18.1/8.18.1/Submit) id 47J1CBC6038066; Mon, 19 Aug 2024 01:12:11 GMT (envelope-from git) Date: Mon, 19 Aug 2024 01:12:11 GMT Message-Id: <202408190112.47J1CBC6038066@gitrepo.freebsd.org> To: ports-committers@FreeBSD.org, dev-commits-ports-all@FreeBSD.org, dev-commits-ports-main@FreeBSD.org From: Yuri Victorovich Subject: git: ee1259996679 - main - misc/ollama: Improve pkg-message List-Id: Commits to the main branch of the FreeBSD ports repository List-Archive: https://lists.freebsd.org/archives/dev-commits-ports-main List-Help: List-Post: List-Subscribe: List-Unsubscribe: X-BeenThere: dev-commits-ports-main@freebsd.org Sender: owner-dev-commits-ports-main@FreeBSD.org MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit X-Git-Committer: yuri X-Git-Repository: ports X-Git-Refname: refs/heads/main X-Git-Reftype: branch X-Git-Commit: ee1259996679f14ac8b59a10d10614d840d89460 Auto-Submitted: auto-generated The branch main has been updated by yuri: URL: https://cgit.FreeBSD.org/ports/commit/?id=ee1259996679f14ac8b59a10d10614d840d89460 commit ee1259996679f14ac8b59a10d10614d840d89460 Author: Yuri Victorovich AuthorDate: 2024-08-19 01:11:36 +0000 Commit: Yuri Victorovich CommitDate: 2024-08-19 01:12:09 +0000 misc/ollama: Improve pkg-message --- misc/ollama/Makefile | 5 +++-- misc/ollama/files/{ollama-limit-layers => ollama-limit-gpu-layers} | 2 ++ misc/ollama/pkg-message | 6 +++++- 3 files changed, 10 insertions(+), 3 deletions(-) diff --git a/misc/ollama/Makefile b/misc/ollama/Makefile index be457b2db084..724bc72feceb 100644 --- a/misc/ollama/Makefile +++ b/misc/ollama/Makefile @@ -1,6 +1,7 @@ PORTNAME= ollama DISTVERSIONPREFIX= v DISTVERSION= 0.3.6 +PORTREVISION= 1 CATEGORIES= misc # machine-learning MAINTAINER= yuri@FreeBSD.org @@ -33,7 +34,7 @@ GH_TUPLE= ggerganov:llama.cpp:1e6f6554aa11fa10160a5fda689e736c3c34169f:llama_cpp MAKE_ENV= PATH=${PATH}:${WRKSRC}/llm/build/bsd/x86_64_static/bin # workaround to find vulkan-shaders-gen PLIST_FILES= bin/${PORTNAME} \ - bin/ollama-limit-layers + bin/ollama-limit-gpu-layers post-patch: # workaround for https://github.com/ollama/ollama/issues/6259 (use of extenral libllama.so) @${REINPLACE_CMD} \ @@ -53,6 +54,6 @@ pre-build: ./... post-install: # pending https://github.com/ollama/ollama/issues/6407 - ${INSTALL_SCRIPT} ${FILESDIR}/ollama-limit-layers ${STAGEDIR}${PREFIX}/bin + ${INSTALL_SCRIPT} ${FILESDIR}/ollama-limit-gpu-layers ${STAGEDIR}${PREFIX}/bin .include diff --git a/misc/ollama/files/ollama-limit-layers b/misc/ollama/files/ollama-limit-gpu-layers similarity index 93% rename from misc/ollama/files/ollama-limit-layers rename to misc/ollama/files/ollama-limit-gpu-layers index 6713a9f9b02b..f17cb5622d0b 100755 --- a/misc/ollama/files/ollama-limit-layers +++ b/misc/ollama/files/ollama-limit-gpu-layers @@ -5,9 +5,11 @@ NUM_GPU=$2 if [ -z "$MODEL" ] || [ -z "$NUM_GPU" ]; then echo "Usage: $0 " + exit 1 fi ollama show --modelfile $MODEL > Modelfile && echo "PARAMETER num_gpu $NUM_GPU" >> Modelfile && ollama create "$MODEL-num_gpu$NUM_GPU" -f Modelfile && +rm Modelfile && echo "model variant $MODEL-num_gpu$NUM_GPU was created" diff --git a/misc/ollama/pkg-message b/misc/ollama/pkg-message index 95b0d4e58448..936a3494000a 100644 --- a/misc/ollama/pkg-message +++ b/misc/ollama/pkg-message @@ -12,6 +12,10 @@ To run ollama, plese open 2 terminals. This will download and run the AI model "mistral". You will be able to interact with it in plain English. +If the above fails, you can try to run 'ollama start' +with theenvironment variable OLLAMA_NUM_PARALLEL=1. +This is known to help with smaller models. + Please see https://ollama.com/library for the list of all supported models. @@ -19,7 +23,7 @@ The command "ollama list" lists all models downloaded into your system. When the model fails to load into your GPU, please use -the provided ollama-limit-layers script to create +the provided ollama-limit-gpu-layers script to create model flavors with different num_gpu parameters. ollama uses many gigbytes of disk space in your home directory,