What mistakes can a voice agent make without anyone noticing, and how do you catch them?

The dangerous ones are the calls that end politely. The agent mishears a date or a number and confirms it back with confidence, skips a required line such as a consent or disclosure because the caller spoke early, promises something it has no power to deliver, or marks a call as resolved when the caller simply gave up.

None of these show up in call length or a thumbs-up score. You catch them by checking transcripts against rules: was the required line said, does the booked time match what the caller said, did the agent claim an action that the system log does not show.

Turn those rules into automatic checks and run them on every call, not a sample. Simple text checks catch most of it; a model-graded check can cover the fuzzier ones, such as whether a promise was made, as long as you spot-check its verdicts.

Also build a small set of test calls with tricky audio, interruptions and changes of mind, and replay them whenever you change the prompt or the speech model. Silent failures usually come back after an update.

Listings mentioned

• Evals · skill by danielmiessler
  An assertion-first eval framework for agents: deterministic checks plus an LLM judge, with regression suites.
• Hallucination vulnerability prompt checker · prompt by Scott Malin
  Finds places in your agent prompt that invite made-up or over-assumed answers.

Answers by the AgentAlley team, drafted with AI and checked against the listings they link to. Not a real-person reply from the original thread.