From nobody Mon Aug 12 06:21:29 2024 X-Original-To: dev-commits-ports-main@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4Wj4Cs56gNz5TFPK; Mon, 12 Aug 2024 06:21:29 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "R11" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4Wj4Cs1tfKz4nmp; Mon, 12 Aug 2024 06:21:29 +0000 (UTC) (envelope-from git@FreeBSD.org) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1723443689; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=zmJXULjB1bAXNXJGCp/PCrfHukAaO9XwjE5/P9IaEFo=; b=BZud4NFeonp1xgZG+aJM/g06A/EdtjeFmdKR2kzhpFEWIBs9VLEFOvNhS5WLsUHA8FzVUh mf24+XII0tEJb74yB+5VjL+yeeYWVn9fZ3IWA0DjApNT2Dcre3aGPzfNqzEv0YBEWWIyy5 bxTWoLYY++cwZbyk8jHOHuNgaykGdooR/XOCjp6hBt+vutokj4NdIxnr/0oJARClXzXVQo 0E7sTaZ40Yj7W01VLLmNI1glFRHL9NzubUktWa3IUJ9LneFwcB+0ibCAw/RrG3N+R6UM3N G8afHuHNBfPE/yOgY1oT2jdj6zDxdvsPA+HoH48kkgb+IXJjSAbImNkZu1Ocqw== ARC-Seal: i=1; s=dkim; d=freebsd.org; t=1723443689; a=rsa-sha256; cv=none; b=ERqd9mhnmSv2GYfaEM6UNTTBn02TMCFGVDodlshyYbqSQ4Mmj3JKWNWFvtQwGfYBiG2Kii YtS1tsWLTMUYCr2nYUwhzVHJfCyk1/b42pywkfLpVdlESMrC2ZoKClgndFDMbysruVy2pu XG+mqw0YcDgImWZrttlYYijAtJUzmGiyuNmHIbgvsP5+Ko6ril78Z6kZq/+A/EDQXi15le PZFibXNXhtq/3GsT+PDLgc/IaXByOsbu8NXeINg0/kEK6CbFtI+ipVG0KfqMOmquQGIIvO yX3S+mUOR9WNu+3H87KR0tLWc4qp/VWtgXBZ2xobP9jlvkwQUteMQMPwDjTzxw== ARC-Authentication-Results: i=1; mx1.freebsd.org; none ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1723443689; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=zmJXULjB1bAXNXJGCp/PCrfHukAaO9XwjE5/P9IaEFo=; b=TCcHCSqqCKeY7of2WqmiGPxqUfUGAY4QFe2/xngA9cJoG6IwFurtIZSX75FhdxxecB8caX 5REn7lRQ9zJ0eRBSQQKjfl33idsHxFWcrrIhgn+STGle58d3BhoQHxBHs+MozS2NwpOp+z XpnhX5vqnhRMKKVms0xaLpZFkyMDMHPFwPTaPvEcgDTBPVzjT1StGunIpxI/JQovOmfbFy JHXi81PEuJsoKdlLcrKufa9ibVnU5t2s8cfNXhKd01Oi5dgL93SyaYSczx1RfU3msceiNM /BxbFra3beUtTmqAZ/RF1wW6Qwg1wTOSxMVIjhhPU331Hyb6GxeXXqpUD25Uvw== Received: from gitrepo.freebsd.org (gitrepo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:5]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 4Wj4Cs1VgZzGKd; Mon, 12 Aug 2024 06:21:29 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from gitrepo.freebsd.org ([127.0.1.44]) by gitrepo.freebsd.org (8.18.1/8.18.1) with ESMTP id 47C6LTgj073577; Mon, 12 Aug 2024 06:21:29 GMT (envelope-from git@gitrepo.freebsd.org) Received: (from git@localhost) by gitrepo.freebsd.org (8.18.1/8.18.1/Submit) id 47C6LTD0073574; Mon, 12 Aug 2024 06:21:29 GMT (envelope-from git) Date: Mon, 12 Aug 2024 06:21:29 GMT Message-Id: <202408120621.47C6LTD0073574@gitrepo.freebsd.org> To: ports-committers@FreeBSD.org, dev-commits-ports-all@FreeBSD.org, dev-commits-ports-main@FreeBSD.org From: Yuri Victorovich Subject: git: 7d0b9dcc61f2 - main - misc/koboldcpp: New port: Simple way to run GGML and GGUF AI models with a KoboldAI WebUI List-Id: Commits to the main branch of the FreeBSD ports repository List-Archive: https://lists.freebsd.org/archives/dev-commits-ports-main List-Help: List-Post: List-Subscribe: List-Unsubscribe: X-BeenThere: dev-commits-ports-main@freebsd.org Sender: owner-dev-commits-ports-main@FreeBSD.org MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit X-Git-Committer: yuri X-Git-Repository: ports X-Git-Refname: refs/heads/main X-Git-Reftype: branch X-Git-Commit: 7d0b9dcc61f252eefbf75b7079ae31e54f4f35ed Auto-Submitted: auto-generated The branch main has been updated by yuri: URL: https://cgit.FreeBSD.org/ports/commit/?id=7d0b9dcc61f252eefbf75b7079ae31e54f4f35ed commit 7d0b9dcc61f252eefbf75b7079ae31e54f4f35ed Author: Yuri Victorovich AuthorDate: 2024-08-12 06:03:02 +0000 Commit: Yuri Victorovich CommitDate: 2024-08-12 06:21:25 +0000 misc/koboldcpp: New port: Simple way to run GGML and GGUF AI models with a KoboldAI WebUI --- misc/Makefile | 1 + misc/koboldcpp/Makefile | 47 +++++++++++++++++++++++++++++++++++++++++++++++ misc/koboldcpp/distinfo | 3 +++ misc/koboldcpp/pkg-descr | 8 ++++++++ misc/koboldcpp/pkg-plist | 15 +++++++++++++++ 5 files changed, 74 insertions(+) diff --git a/misc/Makefile b/misc/Makefile index 7fb74f1ef16b..1ddda70e5b02 100644 --- a/misc/Makefile +++ b/misc/Makefile @@ -216,6 +216,7 @@ SUBDIR += kf6-purpose SUBDIR += kgeography SUBDIR += klettres + SUBDIR += koboldcpp SUBDIR += kookbook SUBDIR += ktouch SUBDIR += kwordquiz diff --git a/misc/koboldcpp/Makefile b/misc/koboldcpp/Makefile new file mode 100644 index 000000000000..d3472c36e95b --- /dev/null +++ b/misc/koboldcpp/Makefile @@ -0,0 +1,47 @@ +PORTNAME= koboldcpp +DISTVERSIONPREFIX= v +DISTVERSION= 1.72 +CATEGORIES= misc # machine-learning + +MAINTAINER= yuri@FreeBSD.org +COMMENT= Simple way to run GGML and GGUF AI models with a KoboldAI WebUI +WWW= https://github.com/lostruins/koboldcpp + +LICENSE= AGPLv3 +LICENSE_FILE= ${WRKSRC}/LICENSE.md + +LIB_DEPENDS= libopenblas.so:math/openblas \ + libvulkan.so:graphics/vulkan-loader + +RUN_DEPENDS= ${PYTHON_PKGNAMEPREFIX}customtkinter>0:x11-toolkits/py-customtkinter@${PY_FLAVOR} + +USES= gmake localbase:ldflags python shebangfix + +USE_GITHUB= yes +GH_ACCOUNT= LostRuins + +SHEBANG_FILES= koboldcpp.py + +MAKE_ARGS= LLAMA_OPENBLAS=1 LLAMA_VULKAN=1 LLAMA_PORTABLE=1 LDFLAGS="${LDFLAGS}" + +ALL_TARGET= + +do-install: + # dir + ${MKDIR} ${STAGEDIR}${DATADIR} + # prog + ${INSTALL_DATA} ${WRKSRC}/koboldcpp.py ${STAGEDIR}${DATADIR}/koboldcpp.py + ${CHMOD} +x ${STAGEDIR}${DATADIR}/koboldcpp.py + ${RLN} ${STAGEDIR}${DATADIR}/koboldcpp.py ${STAGEDIR}${PREFIX}/bin/koboldcpp + # libraries +.for lib in koboldcpp_default.so koboldcpp_failsafe.so koboldcpp_noavx2.so \ + koboldcpp_openblas.so koboldcpp_vulkan.so koboldcpp_vulkan_noavx2.so + ${INSTALL_LIB} ${WRKSRC}/${lib} ${STAGEDIR}${DATADIR} +.endfor + # embd files +.for embd in klite.embd kcpp_docs.embd rwkv_vocab.embd rwkv_world_vocab.embd \ + kcpp_sdui.embd taesd.embd taesd_xl.embd + ${INSTALL_DATA} ${WRKSRC}/${embd} ${STAGEDIR}${DATADIR} +.endfor + +.include diff --git a/misc/koboldcpp/distinfo b/misc/koboldcpp/distinfo new file mode 100644 index 000000000000..ddceb1356f08 --- /dev/null +++ b/misc/koboldcpp/distinfo @@ -0,0 +1,3 @@ +TIMESTAMP = 1723347219 +SHA256 (LostRuins-koboldcpp-v1.72_GH0.tar.gz) = 1159980d2cda0a9dfde9024454034b7b163e483395b78f6a8d41b58883476bd5 +SIZE (LostRuins-koboldcpp-v1.72_GH0.tar.gz) = 28440368 diff --git a/misc/koboldcpp/pkg-descr b/misc/koboldcpp/pkg-descr new file mode 100644 index 000000000000..cfcd1f317314 --- /dev/null +++ b/misc/koboldcpp/pkg-descr @@ -0,0 +1,8 @@ +KoboldCpp is an easy-to-use AI text-generation software for GGML and GGUF +models, inspired by the original KoboldAI. It's a single self-contained +distributable from Concedo, that builds off llama.cpp, and adds a versatile +KoboldAI API endpoint, additional format support, Stable Diffusion image +generation, speech-to-text, backward compatibility, as well as a fancy UI with +persistent stories, editing tools, save formats, memory, world info, author's +note, characters, scenarios and everything KoboldAI and KoboldAI Lite have to +offer. diff --git a/misc/koboldcpp/pkg-plist b/misc/koboldcpp/pkg-plist new file mode 100644 index 000000000000..8efacd4bc51e --- /dev/null +++ b/misc/koboldcpp/pkg-plist @@ -0,0 +1,15 @@ +bin/koboldcpp +%%DATADIR%%/kcpp_docs.embd +%%DATADIR%%/kcpp_sdui.embd +%%DATADIR%%/klite.embd +%%DATADIR%%/koboldcpp.py +%%DATADIR%%/koboldcpp_default.so +%%DATADIR%%/koboldcpp_failsafe.so +%%DATADIR%%/koboldcpp_noavx2.so +%%DATADIR%%/koboldcpp_openblas.so +%%DATADIR%%/koboldcpp_vulkan.so +%%DATADIR%%/koboldcpp_vulkan_noavx2.so +%%DATADIR%%/rwkv_vocab.embd +%%DATADIR%%/rwkv_world_vocab.embd +%%DATADIR%%/taesd.embd +%%DATADIR%%/taesd_xl.embd