记录多模态轨迹

LangSmith 支持将图像作为追踪（traces）的一部分进行记录和渲染。目前，此功能仅适用于多模态大语言模型（LLM）运行。

如需记录图像，请在 Python 中使用 wrap_openai，或在 TypeScript 中使用 wrapOpenAI，并将图像 URL 或 base64 编码的图像作为输入的一部分传入。

Python

from openai import OpenAI
from langsmith.wrappers import wrap_openai

client = wrap_openai(OpenAI())

response = client.chat.completions.create(
model="gpt-4-turbo",
messages=[
  {
    "role": "user",
    "content": [
      {"type": "text", "text": "What’s in this image?"},
      {
        "type": "image_url",
        "image_url": {
          "url": "https://upload.wikimedia.org/wikipedia/commons/thumb/d/dd/Gfp-wisconsin-madison-the-nature-boardwalk.jpg/2560px-Gfp-wisconsin-madison-the-nature-boardwalk.jpg",
        },
      },
    ],
  }
],
)
print(response.choices[0])

TypeScript

import OpenAI from "openai";
import { wrapOpenAI } from "langsmith/wrappers";

// Wrap the OpenAI client to automatically log traces
const wrappedClient = wrapOpenAI(new OpenAI());

const response = await wrappedClient.chat.completions.create({
  model: "gpt-4-turbo",
  messages: [
    {
      role: "user",
      content: [
        { type: "text", text: "What’s in this image?" },
        {
          type: "image_url",
          image_url: {
            "url": "https://upload.wikimedia.org/wikipedia/commons/thumb/d/dd/Gfp-wisconsin-madison-the-nature-boardwalk.jpg/2560px-Gfp-wisconsin-madison-the-nature-boardwalk.jpg",
          },
        },
      ],
    },
  ],
});
console.log(response.choices[0]);

该图像将作为追踪记录的一部分，在 LangSmith 用户界面中渲染。