From nobody Mon Feb 12 08:34:26 2024 X-Original-To: dev-commits-ports-all@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4TYHnH3Yjmz59K75; Mon, 12 Feb 2024 08:34:27 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4TYHnH0zZ8z4CqY; Mon, 12 Feb 2024 08:34:27 +0000 (UTC) (envelope-from git@FreeBSD.org) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1707726867; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=1wX101rBGJBM5IZDxELPG6G2r0fa/eWFUZ49MWOa1nU=; b=f/LF/1rRjA8lHQS1mHlYYmZwqP7d5GgosKOlOda7ErqLLEM/wOPcC3dLEgfV5ZQDK02GEG Ul8yueZxb4V7a0fCz9UpElxKdR0zs3324l8kmvvMF5pFmSKkWno0lAKTyRllLpwM9+Faww OFxYfo/l/csXK2zZvI4oO6w/28t8TgACGtCE3aBDbXYXohvejKJHeaXPnjiDG5FnpPPktn Jn4Zt1nNouvEWZzctCCOCVB2eZGH8jnp9WnDUGuphe2We4cYVjC+ga3N1LwLVkFTXpu078 yKTJ6r1iK6jd+m8sPHm8KSvr6ZBCAbjhqMm3vEMK29Ju1xrW0+qbWYDw1pPfDg== ARC-Seal: i=1; s=dkim; d=freebsd.org; t=1707726867; a=rsa-sha256; cv=none; b=NjRiqUFSrSo/UDLW5q0ROosxKN8j1BueYAS2uOi3lgYZYr1/meLc/JF4DLCLUVB2Z3urOM oPj/SWSU+//ZyUFq9wogc/NHVbPK8cXI27kJxDbOvnukFkFea0v8YF+yjdMU1YXBq8Od7r mpFX9xd+zSRk/flS0dqIwldgC7iUkaYE6hgVdOkysUVlbBJI8rDWkKr26WRf7kaFaQkP44 belsvtkXaZ6vgQLFPA2a4dDVBgeM+ERC5dkmJhbCmzdNohwHtzdVmpGZWifG8LPh/PE0MR cTj8N9DWGdEhzvMpqVqoNCkIVYVE/Z8Vtc9B7vuLrPzNGOidhRZ0OXQmObXbMg== ARC-Authentication-Results: i=1; mx1.freebsd.org; none ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1707726867; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=1wX101rBGJBM5IZDxELPG6G2r0fa/eWFUZ49MWOa1nU=; b=ULTAJno83jeXDCiPlFqmZdYwyoRX7LsUcjRT4EpluBra7oQHNcadd7x5q3XUVlZjSJsWwU /4CaFVSteNxPe8yNb59CahOygpri3du7HA2Mz90ezrGIIblzVP0hxSiJbxGFDjg0qd1PZb S5zK/zhGRLmm/76DbwjzqsLV4B1w/0texkxNp8GN4/+KsL1yn+Ysho/iIq4hEHDguyFa26 YE79oe70u73Uk4qKLgjG4wQYYu4/dJJSHBRS2ZHooHxsLFNJc+JZjGDJzhwBLxwljCCqIF UO9nfZ9oze+OtNVigQAUvHA0t+E4GSICU4ZSvjeFH0/lANiWCjQPju1Vf5cNxg== Received: from gitrepo.freebsd.org (gitrepo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:5]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 4TYHnH01hjzKKK; Mon, 12 Feb 2024 08:34:27 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from gitrepo.freebsd.org ([127.0.1.44]) by gitrepo.freebsd.org (8.17.1/8.17.1) with ESMTP id 41C8YQBb088678; Mon, 12 Feb 2024 08:34:26 GMT (envelope-from git@gitrepo.freebsd.org) Received: (from git@localhost) by gitrepo.freebsd.org (8.17.1/8.17.1/Submit) id 41C8YQsN088675; Mon, 12 Feb 2024 08:34:26 GMT (envelope-from git) Date: Mon, 12 Feb 2024 08:34:26 GMT Message-Id: <202402120834.41C8YQsN088675@gitrepo.freebsd.org> To: ports-committers@FreeBSD.org, dev-commits-ports-all@FreeBSD.org, dev-commits-ports-main@FreeBSD.org From: Hiroki Tagato Subject: git: 730828c62763 - main - misc/py-litellm: add port: Call all LLM APIs using the OpenAI format List-Id: Commit messages for all branches of the ports repository List-Archive: https://lists.freebsd.org/archives/dev-commits-ports-all List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-dev-commits-ports-all@freebsd.org X-BeenThere: dev-commits-ports-all@freebsd.org MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit X-Git-Committer: tagattie X-Git-Repository: ports X-Git-Refname: refs/heads/main X-Git-Reftype: branch X-Git-Commit: 730828c627631142966c84d4c2943defaad86e4e Auto-Submitted: auto-generated The branch main has been updated by tagattie: URL: https://cgit.FreeBSD.org/ports/commit/?id=730828c627631142966c84d4c2943defaad86e4e commit 730828c627631142966c84d4c2943defaad86e4e Author: Hiroki Tagato AuthorDate: 2024-02-12 08:30:35 +0000 Commit: Hiroki Tagato CommitDate: 2024-02-12 08:34:14 +0000 misc/py-litellm: add port: Call all LLM APIs using the OpenAI format Call all LLM APIs using the OpenAI format [Bedrock, Huggingface, VertexAI, TogetherAI, Azure, OpenAI, etc.] LiteLLM manages: - Translate inputs to provider's completion, embedding, and image_generation endpoints - Consistent output, text responses will always be available at ['choices'][0]['message']['content'] - Retry/fallback logic across multiple deployments (e.g. Azure/OpenAI) - Router - Track spend & set budgets per project OpenAI Proxy Server WWW: https://github.com/BerriAI/litellm --- misc/Makefile | 1 + misc/py-litellm/Makefile | 46 ++++++++++++++++++++++ misc/py-litellm/distinfo | 3 ++ misc/py-litellm/files/patch-litellm_proxy_start.sh | 8 ++++ misc/py-litellm/pkg-descr | 11 ++++++ 5 files changed, 69 insertions(+) diff --git a/misc/Makefile b/misc/Makefile index 20142149858b..8f664ef1613b 100644 --- a/misc/Makefile +++ b/misc/Makefile @@ -435,6 +435,7 @@ SUBDIR += py-lazrs SUBDIR += py-lightgbm SUBDIR += py-lightning-utilities + SUBDIR += py-litellm SUBDIR += py-log_symbols SUBDIR += py-mffpy SUBDIR += py-mmcv diff --git a/misc/py-litellm/Makefile b/misc/py-litellm/Makefile new file mode 100644 index 000000000000..5702d6b60bf2 --- /dev/null +++ b/misc/py-litellm/Makefile @@ -0,0 +1,46 @@ +PORTNAME= litellm +DISTVERSION= 1.23.9 +CATEGORIES= misc python +MASTER_SITES= PYPI +PKGNAMEPREFIX= ${PYTHON_PKGNAMEPREFIX} + +MAINTAINER= tagattie@FreeBSD.org +COMMENT= Call all LLM APIs using the OpenAI format +WWW= https://github.com/BerriAI/litellm + +LICENSE= MIT +LICENSE_FILE= ${WRKSRC}/LICENSE + +BUILD_DEPENDS= ${PYTHON_PKGNAMEPREFIX}poetry-core>0:devel/py-poetry-core@${PY_FLAVOR} \ + ${PYTHON_PKGNAMEPREFIX}wheel>0:devel/py-wheel@${PY_FLAVOR} +RUN_DEPENDS= ${PYTHON_PKGNAMEPREFIX}openai>=1.0.0:misc/py-openai@${PY_FLAVOR} \ + ${PYTHON_PKGNAMEPREFIX}python-dotenv>=0.2.0:www/py-python-dotenv@${PY_FLAVOR} \ + ${PYTHON_PKGNAMEPREFIX}tiktoken>=0.4.0:textproc/py-tiktoken@${PY_FLAVOR} \ + ${PYTHON_PKGNAMEPREFIX}importlib-metadata>=6.8.0:devel/py-importlib-metadata@${PY_FLAVOR} \ + ${PYTHON_PKGNAMEPREFIX}tokenizers>0:textproc/py-tokenizers@${PY_FLAVOR} \ + ${PYTHON_PKGNAMEPREFIX}click>0:devel/py-click@${PY_FLAVOR} \ + ${PYTHON_PKGNAMEPREFIX}Jinja2>=3.1.2<4.0.0:devel/py-Jinja2@${PY_FLAVOR} \ + ${PYTHON_PKGNAMEPREFIX}aiohttp>0:www/py-aiohttp@${PY_FLAVOR} \ + ${PYTHON_PKGNAMEPREFIX}requests>=2.31.0<3.0.0:www/py-requests@${PY_FLAVOR} + +USES= python shebangfix +USE_PYTHON= autoplist pep517 + +REINPLACE_ARGS= -i '' +NO_ARCH= yes + +PORTDOCS= README.md + +OPTIONS_DEFINE= DOCS + +post-patch: + @${REINPLACE_CMD} -e 's|%%PYTHON_CMD%%|${PYTHON_CMD}|' \ + ${WRKSRC}/litellm/proxy/start.sh + @${FIND} ${WRKSRC}/litellm/proxy -type f \ + \( -name '*.orig' -o -name '*.bak' \) -delete + +post-install-DOCS-on: + @${MKDIR} ${STAGEDIR}${DOCSDIR} + ${INSTALL_MAN} ${PORTDOCS:S|^|${WRKSRC}/|} ${STAGEDIR}${DOCSDIR} + +.include diff --git a/misc/py-litellm/distinfo b/misc/py-litellm/distinfo new file mode 100644 index 000000000000..d494a3abcdc4 --- /dev/null +++ b/misc/py-litellm/distinfo @@ -0,0 +1,3 @@ +TIMESTAMP = 1707722656 +SHA256 (litellm-1.23.9.tar.gz) = 0c1e0e56f4d1d9c8a33da09d6736bde9b21a8ea324db8c05cc3de65c6b4fad7d +SIZE (litellm-1.23.9.tar.gz) = 3139242 diff --git a/misc/py-litellm/files/patch-litellm_proxy_start.sh b/misc/py-litellm/files/patch-litellm_proxy_start.sh new file mode 100644 index 000000000000..f1ce771fdaeb --- /dev/null +++ b/misc/py-litellm/files/patch-litellm_proxy_start.sh @@ -0,0 +1,8 @@ +--- litellm/proxy/start.sh.orig 2024-02-11 03:13:21 UTC ++++ litellm/proxy/start.sh +@@ -1,2 +1,2 @@ +-#!/bin/bash +-python3 proxy_cli.py +\ No newline at end of file ++#!/bin/sh ++%%PYTHON_CMD%% proxy_cli.py diff --git a/misc/py-litellm/pkg-descr b/misc/py-litellm/pkg-descr new file mode 100644 index 000000000000..bf534cc9750c --- /dev/null +++ b/misc/py-litellm/pkg-descr @@ -0,0 +1,11 @@ +Call all LLM APIs using the OpenAI format [Bedrock, Huggingface, +VertexAI, TogetherAI, Azure, OpenAI, etc.] + +LiteLLM manages: +- Translate inputs to provider's completion, embedding, and + image_generation endpoints +- Consistent output, text responses will always be available at + ['choices'][0]['message']['content'] +- Retry/fallback logic across multiple deployments (e.g. Azure/OpenAI) + - Router +- Track spend & set budgets per project OpenAI Proxy Server