How should you red-team a voice agent before launch?

1. Map the complete voice pipeline

Log every representation after audio enters the system: raw-audio ID, ASR text, cleaned or normalized text, safety-layer input, agent input, tool arguments, final text, and TTS output. Preserve a version and timestamp whenever one stage rewrites another.

A dangerous failure is not merely a bad transcript. It is the safety layer checking version A while the action layer executes version B. Two individually plausible records do not prove that both components understood the same request.

2. Build an audio-condition matrix, not one noise test

Test at least environmental, demographic, and linguistic axes: quiet and noisy rooms, near and far speech, accents and speaking rates, primary and mixed languages. Add phone codecs, packet loss, echo, background media, overlapping speakers, and synthetic speech.

WildASR reports that robustness does not transfer reliably across languages and conditions, and that degraded inputs can produce plausible but unspoken content. An acceptable aggregate word-error rate can still hide costly mistakes in names, amounts, negation, or action verbs.

• Keep meaning fixed while varying accent, rate, volume, codec, and noise.
• Keep audio fixed while comparing ASR systems, normalization, and safety policies.
• Track both full-transcript error and errors on entities, amounts, negation, and intent.
• Place injected audio at the beginning, middle, and end of a sequence.

3. Continue the attack across turns

A refusal on turn one is not a safe conversation. Let the attacker establish harmless context, change the goal gradually, create urgency, cite earlier statements, interrupt the agent, and revise instructions after speech or a tool call has begun.

After every turn, inspect confirmed facts, unconfirmed assumptions, completed actions, and pending actions. After an interruption, do not treat unplayed agent speech as information the caller heard and accepted.

4. Add an independent pre-action gate

Before payments, account changes, privacy-sensitive retrieval, bookings, or outbound calls, revalidate the normalized intent, critical entities, permissions, and confirmation state. The safety decision and the action validator should reference one immutable request ID and input version.

When ASR confidence is low, transcript versions disagree, or a critical entity changes, the safe default is to clarify, pause, or escalate—not to guess and continue.

5. Grade real outcomes and turn production failures into regressions

A refusal string is only an intermediate signal. Check whether a tool was called, arguments exceeded permission, a database or calendar changed, sensitive information entered logs or speech, and the system stopped safely after uncertainty.

After launch, label traces by audio condition, ASR error, state error, safety miss, and unauthorized action. Minimize every new failure into a repeatable case and add it to the release gate instead of merely generating more random personas.