A visitor asks a question and waits. After two seconds, silence feels awkward. After five, it feels broken. When we first measured our host Riya, it took about 10 seconds from the end of a question to the first word of her voice. Today she answers in about 3. Here is where the time went, and what fixed it.
Where the seconds go
Before a talking avatar says anything, five things happen in a row:
- Speech recognition turns the visitor's voice into text (skipped if they type).
- The AI model reads its instructions, the conversation and the new question, then starts writing an answer.
- The voice turns the first sentence of that answer into audio.
- The network carries all of it between the visitor's browser and the servers, often more than once.
- The browser starts the audio, and the avatar moves her lips.
The total is the sum, so you have to measure each step separately: guessing leads you to optimise the wrong step. Log the time at each hand-off: question received, first token from the AI, first sentence complete, audio ready, audio playing.
Fix 1: let the AI reuse its prompt
What we found: the AI was reading about 4,000 tokens of instructions for every question: her persona, her style rules and the business's knowledge. Reading takes time before the first word comes out, and on our setup it was the biggest single delay.
What worked:
- Put everything that never changes first, and keep it byte-for-byte identical: persona, rules, knowledge. Most AI serving systems, local and cloud, can reuse an already-processed prefix (prompt caching), but only if it's exactly the same and at the start. The conversation and the new question go after it.
- Cut the prompt itself. We shortened her persona by about a fifth without losing anything she needed. Shorter instructions are faster on every single question.
- Don't let other AI calls evict the cache. Riya also updates a small memory of each visitor with a separate AI call. Running it right after every reply pushed the chat prompt out of the cache, so the next question started from scratch. We now run it after 30 seconds without a message, when the visitor has gone quiet.
Fix 2: speak the first sentence early
What we found: waiting for the whole answer before starting the voice adds the full generation time to every reply.
What worked:
- Stream the AI's answer and split it into sentences as it arrives.
- Send the first complete sentence to the voice immediately, while the AI is still writing the rest. Each later sentence is voiced while the previous one plays.
- Keep the first sentence short. If the AI opens with a long sentence, cut it at the first comma or clause, so her voice starts sooner. We also ask the AI to open with a short sentence.
Fix 3: never wait for a voice download
What we found: a voice model that runs in the visitor's browser is free to run and private, but it has to download first: tens to hundreds of megabytes. Its speed also depends on the device. In our tests, the same 10 seconds of speech took 1.7 s to synthesise with WebGPU on a desktop, and 9.6 s with WebAssembly, which is too slow for phones.
What worked: use a server voice while the browser voice downloads, so the first reply is never silent, and use the server voice on phones. Keep showing progress ("Preparing voice… 40%") rather than nothing.
Fix 4: make the network boring
- Fetch related things in parallel. The mouth animation for a sentence is requested as soon as the audio's headers arrive, while the audio itself is still downloading.
- Check that the audio arrived whole. Tunnels and proxies sometimes end a download early. A cut-off sentence sounds worse than a short wait, so compare the bytes received with the size announced, and fetch it again if it's short.
- Keep servers warm. A model or server that sleeps between visitors adds a cold start to the first question.
What the visitor sees while waiting
Even 3 seconds feels long if nothing happens. While the answer is being prepared, the avatar should look like she's listening and thinking: a nod when the question ends, a glance away, a small breath. Real people don't answer instantly either, and a face that reacts makes the same delay feel shorter.
A checklist
- Log each step's time, and find the biggest one before changing anything.
- Keep the unchanging part of the prompt first and identical, and keep it short.
- Keep background AI calls from evicting the chat's cache.
- Stream the answer, voice the first sentence immediately, and keep it short.
- Don't make visitors wait for a model download; use a server voice on phones.
- Fetch in parallel, verify audio, keep servers warm.
- Animate listening and thinking while the answer is on its way.
Response time is one of the first things to ask any AI avatar vendor about. Measure from the end of the question to the first word of the voice. See the other questions in how to add an AI avatar to your website, or watch Riya answer.