使用聊天/嵌入響應用法

概覽

Spring AI 透過在 Usage 介面中引入 getNativeUsage() 方法並提供 DefaultUsage 實現，增強了模型用量處理功能。此更改簡化了不同 AI 模型跟蹤和報告其用量指標的方式，同時保持了框架的一致性。

主要更改

Usage 介面增強

Usage 介面現在包含一個新方法

Object getNativeUsage();

此方法允許訪問特定於模型的原生用量資料，從而在需要時實現更詳細的用量跟蹤。

與 ChatModel 一起使用

以下是使用 OpenAI ChatModel 跟蹤用量的完整示例

@SpringBootConfiguration
public class Configuration {

        @Bean
        public OpenAiApi chatCompletionApi() {
            return OpenAiApi.builder()
                .apiKey(System.getenv("OPENAI_API_KEY"))
                .build();
        }

        @Bean
        public OpenAiChatModel openAiClient(OpenAiApi openAiApi) {
            return OpenAiChatModel.builder()
                .openAiApi(openAiApi)
                .build();
        }

    }

@Service
public class ChatService {

    private final OpenAiChatModel chatModel;

    public ChatService(OpenAiChatModel chatModel) {
        this.chatModel = chatModel;
    }

    public void demonstrateUsage() {
        // Create a chat prompt
        Prompt prompt = new Prompt("What is the weather like today?");

        // Get the chat response
        ChatResponse response = this.chatModel.call(prompt);

        // Access the usage information
        Usage usage = response.getMetadata().getUsage();

        // Get standard usage metrics
        System.out.println("Prompt Tokens: " + usage.getPromptTokens());
        System.out.println("Completion Tokens: " + usage.getCompletionTokens());
        System.out.println("Total Tokens: " + usage.getTotalTokens());

        // Access native OpenAI usage data with detailed token information
        if (usage.getNativeUsage() instanceof org.springframework.ai.openai.api.OpenAiApi.Usage) {
            org.springframework.ai.openai.api.OpenAiApi.Usage nativeUsage =
                (org.springframework.ai.openai.api.OpenAiApi.Usage) usage.getNativeUsage();

            // Detailed prompt token information
            System.out.println("Prompt Tokens Details:");
            System.out.println("- Audio Tokens: " + nativeUsage.promptTokensDetails().audioTokens());
            System.out.println("- Cached Tokens: " + nativeUsage.promptTokensDetails().cachedTokens());

            // Detailed completion token information
            System.out.println("Completion Tokens Details:");
            System.out.println("- Reasoning Tokens: " + nativeUsage.completionTokenDetails().reasoningTokens());
            System.out.println("- Accepted Prediction Tokens: " + nativeUsage.completionTokenDetails().acceptedPredictionTokens());
            System.out.println("- Audio Tokens: " + nativeUsage.completionTokenDetails().audioTokens());
            System.out.println("- Rejected Prediction Tokens: " + nativeUsage.completionTokenDetails().rejectedPredictionTokens());
        }
    }
}

與 ChatClient 一起使用

如果您使用的是 ChatClient，您可以透過 ChatResponse 物件訪問用量資訊

// Create a chat prompt
Prompt prompt = new Prompt("What is the weather like today?");

// Create a chat client
ChatClient chatClient = ChatClient.create(chatModel);

// Get the chat response
ChatResponse response = chatClient.prompt(prompt)
        .call()
        .chatResponse();

// Access the usage information
Usage usage = response.getMetadata().getUsage();

優勢

標準化：提供一致的方式來處理不同 AI 模型的用量 靈活性：透過原生用量特性支援特定於模型的用量資料簡化：透過預設實現減少樣板程式碼 可擴充套件性：易於針對特定模型需求進行擴充套件，同時保持相容性

型別安全注意事項

處理原生用量資料時，請仔細考慮型別轉換

// Safe way to access native usage
if (usage.getNativeUsage() instanceof org.springframework.ai.openai.api.OpenAiApi.Usage) {
    org.springframework.ai.openai.api.OpenAiApi.Usage nativeUsage =
        (org.springframework.ai.openai.api.OpenAiApi.Usage) usage.getNativeUsage();
    // Work with native usage data
}