Skip to main content
Open In ColabOpen on GitHub

ChatLiteLLM 与 ChatLiteLLMRouter

LiteLLM 是一个简化了调用 Anthropic、Azure、Huggingface、Replicate 等服务的库。

本Notebook 涵盖了如何开始使用 Langchain + LiteLLM I/O 库。

此集成包含两个主要类：

  • ChatLiteLLM：LiteLLM 的主要 Langchain封装，用于基本操作 (docs)。
  • ChatLiteLLMRouter：一个 ChatLiteLLM 的封装，利用了 LiteLLM 的 Router 功能 (docs)。

目录​

  1. 概述
  2. 设置
  3. 凭证
  4. 安装
  5. 实例化
  6. 调用
  7. 异步和流式功能
  8. API 参考

概览​

集成详情​

类名 (Class)包名 (Package)本地 (Local)可序列化 (Serializable)JS 支持 (JS support)包下载量 (Package downloads)包最新版本 (Package latest)
ChatLiteLLMlangchain-litellm❌❌❌PyPI - DownloadsPyPI - Version
ChatLiteLLMRouterlangchain-litellm❌❌❌PyPI - DownloadsPyPI - Version

模型功能​

工具调用结构化输出JSON 模式 (JSON mode)图像输入 (Image input)音频输入 (Audio input)视频输入 (Video input)Token 级流式输出原生异步Token 使用量Logprobs
✅❌❌❌❌❌✅✅✅❌

设置​

要访问 ChatLiteLLM 和 ChatLiteLLMRouter 模型，您需要安装 langchain-litellm 包，并创建一个 OpenAI、Anthropic、Azure、Replicate、OpenRouter、Hugging Face、Together AI 或 Cohere 账户。然后，获取您的 API 密钥并将其导出为环境变量。

凭据​

您需要选择想要使用的 LLM 提供商，并与他们注册以获取其 API 密钥。

示例 - Anthropic​

前往 https://console.anthropic.com/ 注册 Anthropic 并生成 API 密钥。完成此操作后，设置 ANTHROPIC_API_KEY 环境变量。

示例 - OpenAI​

前往 https://platform.openai.com/api-keys 注册 OpenAI 并生成 API 密钥。完成此操作后，设置 OPENAI_API_KEY 环境变量。

## Set ENV variables
import os

os.environ["OPENAI_API_KEY"] = "your-openai-key"
os.environ["ANTHROPIC_API_KEY"] = "your-anthropic-key"

安装​

LangChain LiteLLM 集成可在 langchain-litellm 包中使用：

%pip install -qU langchain-litellm

实例化​

ChatLiteLLM​

您可以通过提供一个 LiteLLM 支持的模型名称 来实例化 ChatLiteLLM 模型。

from langchain_litellm import ChatLiteLLM

llm = ChatLiteLLM(model="gpt-4.1-nano", temperature=0.1)

ChatLiteLLMRouter​

您还可以通过如此处指定的定义模型列表来利用 LiteLLM 的路由功能。

from langchain_litellm import ChatLiteLLMRouter
from litellm import Router

model_list = [
    {
        "model_name": "gpt-4.1",
        "litellm_params": {
            "model": "azure/gpt-4.1",
            "api_key": "<your-api-key>",
            "api_version": "2024-10-21",
            "api_base": "https://<your-endpoint>.openai.azure.com/",
        },
    },
    {
        "model_name": "gpt-4o",
        "litellm_params": {
            "model": "azure/gpt-4o",
            "api_key": "<your-api-key>",
            "api_version": "2024-10-21",
            "api_base": "https://<your-endpoint>.openai.azure.com/",
        },
    },
]
litellm_router = Router(model_list=model_list)
llm = ChatLiteLLMRouter(router=litellm_router, model_name="gpt-4.1", temperature=0.1)

调用​

无论是实例化了 ChatLiteLLM 还是 ChatLiteLLMRouter，您现在都可以通过 Langchain 的 API 使用 ChatModel。

response = await llm.ainvoke(
    "Classify the text into neutral, negative or positive. Text: I think the food was okay. Sentiment:"
)
print(response)
content='Neutral' additional_kwargs={} response_metadata={'token_usage': Usage(completion_tokens=2, prompt_tokens=30, total_tokens=32, completion_tokens_details=CompletionTokensDetailsWrapper(accepted_prediction_tokens=0, audio_tokens=0, reasoning_tokens=0, rejected_prediction_tokens=0, text_tokens=None), prompt_tokens_details=PromptTokensDetailsWrapper(audio_tokens=0, cached_tokens=0, text_tokens=None, image_tokens=None)), 'model': 'gpt-3.5-turbo', 'finish_reason': 'stop', 'model_name': 'gpt-3.5-turbo'} id='run-ab6a3b21-eae8-4c27-acb2-add65a38221a-0' usage_metadata={'input_tokens': 30, 'output_tokens': 2, 'total_tokens': 32}

异步和流式功能​

ChatLiteLLM 和 ChatLiteLLMRouter 还支持异步和流式功能：

async for token in llm.astream("Hello, please explain how antibiotics work"):
    print(token.text(), end="")
Antibiotics are medications that fight bacterial infections in the body. They work by targeting specific bacteria and either killing them or preventing their growth and reproduction.

There are several different mechanisms by which antibiotics work. Some antibiotics work by disrupting the cell walls of bacteria, causing them to burst and die. Others interfere with the protein synthesis of bacteria, preventing them from growing and reproducing. Some antibiotics target the DNA or RNA of bacteria, disrupting their ability to replicate.

It is important to note that antibiotics only work against bacterial infections and not viral infections. It is also crucial to take antibiotics as prescribed by a healthcare professional and to complete the full course of treatment, even if symptoms improve before the medication is finished. This helps to prevent antibiotic resistance, where bacteria become resistant to the effects of antibiotics.

API 参考​

有关 ChatLiteLLM 和 ChatLiteLLMRouter 所有功能和配置的详细文档，请访问 API 参考：https://github.com/Akshay-Dongare/langchain-litellm