Speech timing

Generate complete or streaming audio with word timing for synchronized text highlighting and seeking.

Generate speech with word/token timestamps using a BreezeBlue TTS 2 model. Choose a complete response, a background job, or a stream. Times are in seconds relative to the delivered audio; character alignment is not provided.

Delivery	Endpoint	Response
Complete	POST /v1/text-to-speech/{voice_id}/with-timestamps	One JSON object with audio_base64, content_type, and the complete word_timestamps list
Background	POST /v1/text-to-speech/{voice_id}/with-timestamps?delivery=async	A job ID; poll the job for word_timestamps and an audio download URL
Streaming	POST /v1/text-to-speech/{voice_id}/stream/with-timestamps	NDJSON audio and timing updates, followed by the complete word list

Complete audio with timing

The synchronous endpoint accepts the ordinary TTS request fields and output formats, including sample-rate profiles such as wav_48000. Default output is MP3. Word timestamps are always included. timestamp_mode applies only to streaming. The history-item-id response header identifies the saved audio; history audio downloads contain audio bytes, not the JSON envelope.

Python

import base64
from pathlib import Path
from breeze_blue import BreezeBlue

client = BreezeBlue()
result = client.text_to_speech.convert_with_timestamps(
    "VOICE_ID", text="Hello world.", model_id="breeze-tts-2-multilingual",
    language_code="en", output_format="wav",
)
Path("speech.wav").write_bytes(base64.b64decode(result["audio_base64"]))
print(result.get("word_timestamps"))

TypeScript

import { writeFile } from "node:fs/promises";
import { BreezeBlueClient } from "@breeze.blue/sdk";

const client = new BreezeBlueClient();
const result = await client.textToSpeech.convertWithTimestamps("VOICE_ID", {
  text: "Hello world.", modelId: "breeze-tts-2-multilingual", languageCode: "en",
}, { outputFormat: "wav" });
await writeFile("speech.wav", Buffer.from(result.audioBase64, "base64"));
console.log(result.wordTimestamps);

CLI

breeze tts 'Hello world.' --voice VOICE_ID \
  --model breeze-tts-2-multilingual --language en \
  --no-stream --with-timestamps --format wav --file speech.wav --agent

The CLI saves decoded audio to speech.wav and the complete list to speech.wav.timestamps.json. JSON output has streamed: false, word_timestamps, and timestamps_file.

HTTP

curl 'https://api.breeze.blue/v1/text-to-speech/VOICE_ID/with-timestamps?output_format=wav' \
  -H "xi-api-key: $BREEZE_API_KEY" -H 'Content-Type: application/json' \
  -d '{"text":"Hello world.","model_id":"breeze-tts-2-multilingual","language_code":"en"}'

Background audio with timing

Set delivery=async to return HTTP 202 with a generation_job_id immediately. Poll GET /v1/generation-jobs/{generation_job_id} until status is ready, then read word_timestamps and download the audio from download_url. Stop polling on failed or cancelled; use a deadline. Pending jobs and ordinary audio jobs omit word_timestamps.

Python

import time
from breeze_blue import BreezeBlue

client = BreezeBlue()
job = client.text_to_speech.create_job_with_timestamps(
    "VOICE_ID", text="Hello world.", model_id="breeze-tts-2-multilingual",
    language_code="en", output_format="wav",
)
deadline = time.monotonic() + 120
while time.monotonic() < deadline:
    result = client.generation_jobs.get(job["generation_job_id"])
    if result["status"] == "ready":
        print(result.get("word_timestamps"))
        break
    if result["status"] in ("failed", "cancelled"):
        raise RuntimeError(result.get("error"))
    time.sleep(2)
else:
    raise TimeoutError("Speech job is still pending")
# Download audio with client.generation_jobs.download_audio(job["generation_job_id"]).

TypeScript

import { BreezeBlueClient } from "@breeze.blue/sdk";

const client = new BreezeBlueClient();
const job = await client.textToSpeech.createJobWithTimestamps("VOICE_ID", {
  text: "Hello world.", modelId: "breeze-tts-2-multilingual", languageCode: "en",
}, { outputFormat: "wav" });
const deadline = Date.now() + 120_000;
while (true) {
  if (Date.now() >= deadline) throw new Error("Speech job is still pending");
  const result = await client.generationJobs.get(job.generationJobId);
  if (result.status === "ready") {
    console.log(result.wordTimestamps);
    break;
  }
  if (["failed", "cancelled"].includes(result.status)) {
    throw new Error(JSON.stringify(result.error));
  }
  await new Promise(resolve => setTimeout(resolve, 2000));
}
// Download audio with client.generationJobs.downloadAudio(job.generationJobId).

CLI

breeze tts 'Hello world.' --voice VOICE_ID --model breeze-tts-2-multilingual --language en --async --with-timestamps --format wav --agent
breeze jobs wait JOB_ID --file speech.wav --agent

Replace JOB_ID with the returned generation_job_id. Waiting with --file saves audio and speech.wav.timestamps.json. To download a completed job later, use breeze jobs download JOB_ID --file speech.wav --with-timestamps --agent. JSON output includes word_timestamps and timestamps_file.

HTTP

curl 'https://api.breeze.blue/v1/text-to-speech/VOICE_ID/with-timestamps?delivery=async&output_format=wav' -H "xi-api-key: $BREEZE_API_KEY" -H 'Content-Type: application/json' -d '{"text":"Hello world.","model_id":"breeze-tts-2-multilingual","language_code":"en"}'
curl 'https://api.breeze.blue/v1/generation-jobs/JOB_ID' -H "xi-api-key: $BREEZE_API_KEY"

Streaming audio with timing

Use POST /v1/text-to-speech/{voice_id}/stream/with-timestamps with a BreezeBlue TTS 2 model. It accepts the usual text, voice, model, language, and voice settings. Audio is returned as base64 inside newline-delimited JSON (NDJSON), with word/token timestamps. The ordinary /stream endpoint continues to return audio bytes.

curl 'https://api.breeze.blue/v1/text-to-speech/VOICE_ID/stream/with-timestamps?output_format=pcm' \
  -H "xi-api-key: $BREEZE_API_KEY" \
  -H 'Content-Type: application/json' \
  -d '{"text":"Hello world.","model_id":"breeze-tts-2-multilingual","language_code":"en","timestamp_mode":"chunk"}'

timestamp_mode is chunk (default) or lookahead. Chunk mode sends boundaries as audio is generated; the same word may span several updates. Lookahead mode waits for later audio to confirm a boundary, so timing can arrive after its audio. It does not promise a fixed wall-clock delay.

Consume the stream

Python

import base64
from breeze_blue import BreezeBlue

client = BreezeBlue()
words = {}
with client.text_to_speech.stream_with_timestamps(
    "VOICE_ID", text="Hello world.", output_format="wav",
    model_id="breeze-tts-2-multilingual", language_code="en",
) as stream, open("speech.wav", "wb") as audio:
    for chunk in stream:
        audio.write(base64.b64decode(chunk["audio_base64"]))
        for word in chunk["word_timestamps"]:
            previous = words.get(word["index"], word)
            words[word["index"]] = {
                **word,
                "start": min(previous["start"], word["start"]),
                "end": max(previous["end"], word["end"]),
            }

TypeScript

import { BreezeBlueClient, type WordTimestamp } from "@breeze.blue/sdk";

const client = new BreezeBlueClient();
const words = new Map<number, WordTimestamp>();
const stream = await client.textToSpeech.streamWithTimestamps("VOICE_ID", {
  text: "Hello world.", modelId: "breeze-tts-2-multilingual",
  languageCode: "en", timestampMode: "lookahead",
});
for await (const chunk of stream) {
  // Decode chunk.audioBase64 and enqueue the bytes for your audio player.
  for (const word of chunk.wordTimestamps) {
    const previous = words.get(word.index) ?? word;
    words.set(word.index, { ...word,
      start: Math.min(previous.start, word.start),
      end: Math.max(previous.end, word.end),
    });
  }
}

CLI

breeze tts 'Hello world.' --voice VOICE_ID \
  --model breeze-tts-2-multilingual --language en \
  --with-timestamps --timestamp-mode lookahead \
  --format wav --file speech.wav --no-play --agent

The CLI writes decoded audio to speech.wav and merged timing to speech.wav.timestamps.json. JSON output includes word_timestamps and timestamps_file. Use --no-stream --with-timestamps for complete JSON delivery. --timestamp-mode applies only to streaming; use --async --with-timestamps for a background job.

SDK iterators raise on stream errors and close the connection when iteration stops. In Python, use the context manager to close even when leaving early. Each call consumes the same speech-generation credits as ordinary TTS.

Timing and display

A word entry has this shape:

{"index":0,"word":"Hello","start":0.2,"end":0.6}

• Times refer to the complete output audio, including requested speed and pauses inserted between long-text segments. They are not offsets inside an HTTP chunk.
• Streaming audio output formats are pcm (mono 24 kHz PCM16 little-endian), wav, and mp3; complete responses support the ordinary synchronous TTS formats. Decode audio_base64 before playback. MP3 encoders can buffer, so a record’s bytes need not align with its word boundaries.
• Preserve the authored text, punctuation, whitespace, and audio tags. Chinese and other CJK languages may return tokens containing multiple characters. This API does not provide character timestamps.
• Highlight a word using the player’s audio clock when start <= currentTime < end. Seek to its start when the reader selects it. Do not use network arrival time as playback time.
• A streaming record can have an empty audio_base64 or an empty word_timestamps array. HTTP transport chunks can split a JSON line or a UTF-8 character; use an incremental parser.
• The final successful streaming record contains the complete merged word list and empty audio. Merge updates by index, keeping the earliest start and latest end, or use this final list for saved playback.
• In a stream, after HTTP 200, a failure can appear as {"error":{"code":"UPSTREAM_GENERATION_ERROR","message":"..."}}. A disconnect, parsing failure, or error record means the output is incomplete.

The API follows the normal authentication, voice visibility, generation concurrency, history, and billing rules. Old history audio is not retroactively aligned.

Related endpoints and tools

• Convert with timestamps
• Convert text to speech
• Stream speech with word timestamps
• Output formats
• CLI text to speech