文字轉語音 (TTS) API

Spring AI 透過 TextToSpeechModel 和 StreamingTextToSpeechModel 介面提供了統一的文字轉語音 (TTS) API。這使您能夠編寫可在不同 TTS 提供商之間通用的可移植程式碼。

支援的提供商

通用介面

所有 TTS 提供商都實現以下共享介面

TextToSpeechModel

TextToSpeechModel 介面提供將文字轉換為語音的方法

public interface TextToSpeechModel extends Model<TextToSpeechPrompt, TextToSpeechResponse>, StreamingTextToSpeechModel {

    /**
     * Converts text to speech with default options.
     */
    default byte[] call(String text) {
        // Default implementation
    }

    /**
     * Converts text to speech with custom options.
     */
    TextToSpeechResponse call(TextToSpeechPrompt prompt);

    /**
     * Returns the default options for this model.
     */
    default TextToSpeechOptions getDefaultOptions() {
        // Default implementation
    }
}

StreamingTextToSpeechModel

StreamingTextToSpeechModel 介面提供即時流式傳輸音訊的方法

@FunctionalInterface
public interface StreamingTextToSpeechModel extends StreamingModel<TextToSpeechPrompt, TextToSpeechResponse> {

    /**
     * Streams text-to-speech responses with metadata.
     */
    Flux<TextToSpeechResponse> stream(TextToSpeechPrompt prompt);

    /**
     * Streams audio bytes for the given text.
     */
    default Flux<byte[]> stream(String text) {
        // Default implementation
    }
}

TextToSpeechPrompt

TextToSpeechPrompt 類封裝了輸入文字和選項

TextToSpeechPrompt prompt = new TextToSpeechPrompt(
    "Hello, this is a text-to-speech example.",
    options
);

TextToSpeechResponse

TextToSpeechResponse 類包含生成的音訊和元資料

TextToSpeechResponse response = model.call(prompt);
byte[] audioBytes = response.getResult().getOutput();
TextToSpeechResponseMetadata metadata = response.getMetadata();

編寫與提供商無關的程式碼

共享 TTS 介面的關鍵優勢之一是能夠編寫與任何 TTS 提供商相容的程式碼，無需修改。實際的提供商（OpenAI、ElevenLabs 等）由您的 Spring Boot 配置決定，允許您在不更改應用程式程式碼的情況下切換提供商。

基本服務示例

共享介面允許您編寫與任何 TTS 提供商相容的程式碼

@Service
public class NarrationService {

    private final TextToSpeechModel textToSpeechModel;

    public NarrationService(TextToSpeechModel textToSpeechModel) {
        this.textToSpeechModel = textToSpeechModel;
    }

    public byte[] narrate(String text) {
        // Works with any TTS provider
        return textToSpeechModel.call(text);
    }

    public byte[] narrateWithOptions(String text, TextToSpeechOptions options) {
        TextToSpeechPrompt prompt = new TextToSpeechPrompt(text, options);
        TextToSpeechResponse response = textToSpeechModel.call(prompt);
        return response.getResult().getOutput();
    }
}

此服務與 OpenAI、ElevenLabs 或任何其他 TTS 提供商無縫協作，實際實現由您的 Spring Boot 配置決定。

高階示例：多提供商支援

您可以構建同時支援多個 TTS 提供商的應用程式

@Service
public class MultiProviderNarrationService {

    private final Map<String, TextToSpeechModel> providers;

    public MultiProviderNarrationService(List<TextToSpeechModel> models) {
        // Spring will inject all available TextToSpeechModel beans
        this.providers = models.stream()
            .collect(Collectors.toMap(
                model -> model.getClass().getSimpleName(),
                model -> model
            ));
    }

    public byte[] narrateWithProvider(String text, String providerName) {
        TextToSpeechModel model = providers.get(providerName);
        if (model == null) {
            throw new IllegalArgumentException("Unknown provider: " + providerName);
        }
        return model.call(text);
    }

    public Set<String> getAvailableProviders() {
        return providers.keySet();
    }
}

流式音訊示例

共享介面還支援流式傳輸以實現即時音訊生成

@Service
public class StreamingNarrationService {

    private final TextToSpeechModel textToSpeechModel;

    public StreamingNarrationService(TextToSpeechModel textToSpeechModel) {
        this.textToSpeechModel = textToSpeechModel;
    }

    public Flux<byte[]> streamNarration(String text) {
        // TextToSpeechModel extends StreamingTextToSpeechModel
        return textToSpeechModel.stream(text);
    }

    public Flux<TextToSpeechResponse> streamWithMetadata(String text, TextToSpeechOptions options) {
        TextToSpeechPrompt prompt = new TextToSpeechPrompt(text, options);
        return textToSpeechModel.stream(prompt);
    }
}

REST 控制器示例

使用與提供商無關的 TTS 構建 REST API

@RestController
@RequestMapping("/api/tts")
public class TextToSpeechController {

    private final TextToSpeechModel textToSpeechModel;

    public TextToSpeechController(TextToSpeechModel textToSpeechModel) {
        this.textToSpeechModel = textToSpeechModel;
    }

    @PostMapping(value = "/synthesize", produces = "audio/mpeg")
    public ResponseEntity<byte[]> synthesize(@RequestBody SynthesisRequest request) {
        byte[] audio = textToSpeechModel.call(request.text());
        return ResponseEntity.ok()
            .contentType(MediaType.parseMediaType("audio/mpeg"))
            .header("Content-Disposition", "attachment; filename=\"speech.mp3\"")
            .body(audio);
    }

    @GetMapping(value = "/stream", produces = MediaType.APPLICATION_OCTET_STREAM_VALUE)
    public Flux<byte[]> streamSynthesis(@RequestParam String text) {
        return textToSpeechModel.stream(text);
    }

    record SynthesisRequest(String text) {}
}

基於配置的提供商選擇

使用 Spring 配置檔案或屬性在提供商之間切換

# application-openai.yml
spring:
  ai:
    model:
      audio:
        speech: openai
    openai:
      api-key: ${OPENAI_API_KEY}
      audio:
        speech:
          options:
            model: gpt-4o-mini-tts
            voice: alloy

# application-elevenlabs.yml
spring:
  ai:
    model:
      audio:
        speech: elevenlabs
    elevenlabs:
      api-key: ${ELEVENLABS_API_KEY}
      tts:
        options:
          model-id: eleven_turbo_v2_5
          voice-id: your_voice_id

然後啟用所需的提供商

# Use OpenAI
java -jar app.jar --spring.profiles.active=openai

# Use ElevenLabs
java -jar app.jar --spring.profiles.active=elevenlabs

使用可移植選項

為了實現最大的可移植性，請僅使用通用的 TextToSpeechOptions 介面方法

@Service
public class PortableNarrationService {

    private final TextToSpeechModel textToSpeechModel;

    public PortableNarrationService(TextToSpeechModel textToSpeechModel) {
        this.textToSpeechModel = textToSpeechModel;
    }

    public byte[] createPortableNarration(String text) {
        // Use provider's default options for maximum portability
        TextToSpeechOptions defaultOptions = textToSpeechModel.getDefaultOptions();
        TextToSpeechPrompt prompt = new TextToSpeechPrompt(text, defaultOptions);
        TextToSpeechResponse response = textToSpeechModel.call(prompt);
        return response.getResult().getOutput();
    }
}

使用提供商特定功能

當您需要提供商特定功能時，您仍然可以使用它們，同時保持程式碼庫的可移植性

@Service
public class FlexibleNarrationService {

    private final TextToSpeechModel textToSpeechModel;

    public FlexibleNarrationService(TextToSpeechModel textToSpeechModel) {
        this.textToSpeechModel = textToSpeechModel;
    }

    public byte[] narrate(String text, TextToSpeechOptions baseOptions) {
        TextToSpeechOptions options = baseOptions;

        // Apply provider-specific optimizations if available
        if (textToSpeechModel instanceof OpenAiAudioSpeechModel) {
            options = OpenAiAudioSpeechOptions.builder()
                .from(baseOptions)
                .model("gpt-4o-tts")  // OpenAI-specific: use high-quality model
                .speed(1.0)
                .build();
        } else if (textToSpeechModel instanceof ElevenLabsTextToSpeechModel) {
            // ElevenLabs-specific options could go here
        }

        TextToSpeechPrompt prompt = new TextToSpeechPrompt(text, options);
        TextToSpeechResponse response = textToSpeechModel.call(prompt);
        return response.getResult().getOutput();
    }
}

可移植程式碼的最佳實踐

  1. 依賴介面：始終注入 TextToSpeechModel 而不是具體的實現

  2. 使用通用選項：堅持使用 TextToSpeechOptions 介面方法以實現最大的可移植性

  3. 優雅地處理元資料：不同的提供商返回不同的元資料；通用地處理它

  4. 使用多個提供商進行測試：確保您的程式碼至少與兩個 TTS 提供商相容

  5. 文件化提供商假設：如果您依賴特定提供商的行為，請清楚地記錄下來

提供商特定功能

雖然共享介面提供了可移植性，但每個提供商還透過提供商特定的選項類（例如 OpenAiAudioSpeechOptions、ElevenLabsSpeechOptions）提供特定功能。這些類實現了 TextToSpeechOptions 介面，同時添加了提供商特定的功能。

© . This site is unofficial and not affiliated with VMware.