How to measure latency in a voice AI application.

Which latency are we talking about?

Name where every number starts and ends. A model call’s latency and a caller’s wait are different numbers.

Measurement	Starts	Ends
Caller-perceived reply delay	The caller stops speaking	The first audible reply at the caller’s ear
Turn detection	End of speech in the captured audio	The system commits the caller’s turn
Model first output	The model request starts	The first usable sentence reaches your code
TTS first byte	The speech request starts	The first audio bytes arrive where you measure
Playback start	The first usable audio arrives	The player actually starts

These stages overlap when you stream, so separate medians don’t add up to the median turn. Measure the whole path, then use the parts to explain it.

What does the Fourier TTS request log show?

Every speech request, finished or not, lands in a log:

curl "https://sandbox.voice.miraiminds.co/v2/tts/requests?limit=50" \
  -H "Authorization: Bearer $MIRA_API_KEY"

Each row has first_byte_ms (our time to first audio), total_ms, bytes, format, transport and a status of completed, failed, truncated or rejected. If those look normal and callers still hear gaps, the audio is stuck after it leaves us: on your link, in your event loop, or in pacing to the phone line.

A wav response arrives in one piece at the end, so its first byte comes last; use pcm, mulaw or alaw. Save the X-Mira-Model header with each test, so a model update doesn’t pass for a change in your app.

What does the Realtime API report for each turn?

Every response.done event carries the turn’s numbers in response.metadata, as strings:

Key	Meaning
mirai_v2v_ms	End of the caller’s speech to the first audio of the reply, on Mirai’s side
mirai_stt_ms	Speech recognition’s first result for the turn
mirai_llm_ms	The model’s first output
mirai_tts_ms	Speech synthesis’s first audio
mirai_stt_provider, mirai_llm_provider, mirai_tts_provider	mira, or the fallback that served that step
mirai_fallback	"true" when any step of the turn ran on a fallback

Unmeasured times are left out. Add your network time to mirai_v2v_ms to get what the caller hears, and set fallback turns aside before you compare. pipecat-mirai passes these to an on_turn_metrics handler; see the Realtime guide.

How do I build a fair test?

Give every option the same test set and the same pass mark: short acknowledgements, long answers, Hindi-English switching and your product’s own phrases, over the connection you’ll use in production.

1. Define where each timer starts and stops.
2. Record the client location, network, format, sample rate, model build, voice and concurrency.
3. Run the whole pipeline, player or receiving phone included.
4. Keep failures and timeouts in the results.
5. Report the distribution and sample size, next to task completion and intelligibility.
6. Repeat with the same setup after any real change.

Our end-to-end harness times the model’s first token to first audio at the phone, from wherever you run it.

Why does a fast voice API still feel slow?

Because much of the wait happens outside the TTS call:

• The app waits for the model’s whole answer before sending text to speech, then for all the audio before playing it.
• Each sentence opens a new HTTPS connection, and the handshake costs 0.4 to 1 second from India. Keep one WebSocket per call, or one keep-alive client.
• 48 kHz PCM needs 768 kbit/s per stream, so six calls need a steady 4.6 Mbit/s. A phone line carries 8 kHz anyway, so ask for ulaw_8000.
• The phone provider holds only about 40 ms of audio, so any longer stall on your server breaks the voice up. The docs suggest staying about 0.4 seconds ahead and holding about 150 ms before each sentence.
• After the caller interrupts, queued audio keeps playing. Cancel the reply, clear the provider’s buffer, and don’t let a late response restart the turn.

Smaller text chunks start speech sooner, but can cost the voice its phrasing. Change one setting at a time, and check quality with timing.

What should a published benchmark include?

The method, next to the number: workload, language, client and server locations, format, concurrency, sample size, timing boundaries and how failures were counted. Say what the test leaves out. A best-case sample isn’t a guarantee, and a design diagram isn’t a measurement.

Our voice measurement journal applies the same rule to releases: compare what customers would hear after a change with what they hear today.