ChatLiteLLM 和 ChatLiteLLMRouter

LiteLLM 是一个简化调用 Anthropic、Azure、Huggingface、Replicate 等的库。

本笔记本介绍了如何开始使用 Langchain 和 LiteLLM I/O 库。

此集成包含两个主要类：

ChatLiteLLM: 用于 LiteLLM 基本用法的主 Langchain 包装器（文档）。
ChatLiteLLMRouter: 一个利用 LiteLLM 路由器的 ChatLiteLLM 封装器（文档）。

概览

集成详情

类	包	本地	可序列化的	JS 支持	软件包下载	最新包裹
ChatLiteLLM	langchain-litellm	❌	❌	❌
ChatLiteLLMRouter	langchain-litellm	❌	❌	❌

模型特性

工具调用	结构化输出	JSON模式	图像输入	音频输入	视频输入	令牌级流式传输	原生异步	令牌使用量	对数概率
✅	❌	❌	❌	❌	❌	✅	✅	✅	❌

设置

要访问ChatLiteLLM和ChatLiteLLMRouter模型，您需要安装langchain-litellm包，并创建一个OpenAI、Anthropic、Azure、Replicate、OpenRouter、Hugging Face、Together AI或Cohere帐户。然后，您需要获取API密钥并将其导出为环境变量。

凭据

你必须选择所需的LLM提供商，并注册以获取他们的API密钥。

示例 - Anthropic

前往 https://console.anthropic.com/ 注册Anthropic并生成API密钥。完成后，请设置ANTHROPIC_API_KEY环境变量。

示例 - OpenAI

前往 https://platform.openai.com/api-keys 注册OpenAI并生成API密钥。完成后，请设置OPENAI_API_KEY环境变量。

## Set ENV variables
import os

os.environ["OPENAI_API_KEY"] = "your-openai-key"
os.environ["ANTHROPIC_API_KEY"] = "your-anthropic-key"

安装

LangChain LiteLLM 集成在 langchain-litellm 包中可用：

%pip install -qU langchain-litellm

实例化

ChatLiteLLM

您可以通过提供一个 model 名称来实例化 ChatLiteLLM 模型，该名称受 LiteLLM 支持。

from langchain_litellm import ChatLiteLLM

llm = ChatLiteLLM(model="gpt-4.1-nano", temperature=0.1)

ChatLiteLLMRouter

您还可以通过按此处指定的方式定义模型列表，来利用LiteLLM的路由功能。

from langchain_litellm import ChatLiteLLMRouter
from litellm import Router

model_list = [
    {
        "model_name": "gpt-4.1",
        "litellm_params": {
            "model": "azure/gpt-4.1",
            "api_key": "<your-api-key>",
            "api_version": "2024-10-21",
            "api_base": "https://<your-endpoint>.openai.azure.com/",
        },
    },
    {
        "model_name": "gpt-4o",
        "litellm_params": {
            "model": "azure/gpt-4o",
            "api_key": "<your-api-key>",
            "api_version": "2024-10-21",
            "api_base": "https://<your-endpoint>.openai.azure.com/",
        },
    },
]
litellm_router = Router(model_list=model_list)
llm = ChatLiteLLMRouter(router=litellm_router, model_name="gpt-4.1", temperature=0.1)

调用

无论你实例化的是 ChatLiteLLM 还是 ChatLiteLLMRouter，你现在都可以通过Langchain的API使用ChatModel。

response = await llm.ainvoke(
    "Classify the text into neutral, negative or positive. Text: I think the food was okay. Sentiment:"
)
print(response)

content='Neutral' additional_kwargs={} response_metadata={'token_usage': Usage(completion_tokens=2, prompt_tokens=30, total_tokens=32, completion_tokens_details=CompletionTokensDetailsWrapper(accepted_prediction_tokens=0, audio_tokens=0, reasoning_tokens=0, rejected_prediction_tokens=0, text_tokens=None), prompt_tokens_details=PromptTokensDetailsWrapper(audio_tokens=0, cached_tokens=0, text_tokens=None, image_tokens=None)), 'model': 'gpt-3.5-turbo', 'finish_reason': 'stop', 'model_name': 'gpt-3.5-turbo'} id='run-ab6a3b21-eae8-4c27-acb2-add65a38221a-0' usage_metadata={'input_tokens': 30, 'output_tokens': 2, 'total_tokens': 32}

异步和流功能

ChatLiteLLM 和 ChatLiteLLMRouter 还支持异步和流功能：

async for token in llm.astream("Hello, please explain how antibiotics work"):
    print(token.text(), end="")

Antibiotics are medications that fight bacterial infections in the body. They work by targeting specific bacteria and either killing them or preventing their growth and reproduction.

There are several different mechanisms by which antibiotics work. Some antibiotics work by disrupting the cell walls of bacteria, causing them to burst and die. Others interfere with the protein synthesis of bacteria, preventing them from growing and reproducing. Some antibiotics target the DNA or RNA of bacteria, disrupting their ability to replicate.

It is important to note that antibiotics only work against bacterial infections and not viral infections. It is also crucial to take antibiotics as prescribed by a healthcare professional and to complete the full course of treatment, even if symptoms improve before the medication is finished. This helps to prevent antibiotic resistance, where bacteria become resistant to the effects of antibiotics.

API 参考

有关所有 ChatLiteLLM 和 ChatLiteLLMRouter 功能及配置的详细文档，请访问API参考： https://github.com/Akshay-Dongare/langchain-litellm

聊天模型概念指南
聊天模型操操作指南

目录​

概览​

集成详情​

模型特性​

设置​

凭据​

示例 - Anthropic​

示例 - OpenAI​

安装​

实例化​

ChatLiteLLM​

ChatLiteLLMRouter​

调用​

异步和流功能​

API 参考​

相关​

目录

概览