Evaluate voice AI corrections and conversational repair

Distinguish correction from a new request

“Make that two” should be understood in context. “Actually, not spicy” changes a prior preference. “Wait, I need to check” is not permission to finalize the request. Your test should specify which detail is active and what the guest intends to change before the session begins.

Keep the observable goal conversational: did the system acknowledge the changed detail and communicate the current state? Do not infer a successful kitchen ticket from a spoken confirmation. That downstream state needs a separate acceptance test.

Use a ladder of difficulty

1. Simple correction: replace one quantity or time after the system repeats it.
2. Mid-turn interruption: start the correction while the system is speaking.
3. Partial reversal: cancel one detail while retaining the rest of the request.
4. Ambiguity: use “that one” where more than one item could be meant and expect a clarifying question.
5. Abandonment: ask to stop or to speak with staff, then observe whether the system respects the request.

Score repair burden separately

A case may eventually succeed after three unnecessary repetitions. Record that as a passed goal with a high clarification burden, rather than letting the final outcome erase the experience. Define a clarification turn consistently: a system question or repetition needed to resolve uncertain or mistaken understanding.

A planned required choice is not automatically a repair. For example, asking for a size when the guest has not given one is a normal ordering step. Reviewers should distinguish required information collection from correcting something already supplied.

Make the human exit observable

The guest should know whether a person is being contacted, whether a callback or message is offered, and what information will carry forward. Do not mark an offered transfer as a completed human handoff without observing the receiving endpoint. If the test environment cannot reach staff, mark that part blocked or untested.

Loman’s current plan information describes interruption handling and transfer capabilities. Those are reasons to include repair and escalation cases in a phone evaluation, not evidence that every case in your restaurant has already passed.

A repair case with an explicit expected result

Illustrative script: “Two house specials, please. Actually, make that one, and I need a person before I finish.” Expected interaction: the active quantity becomes one, the agent does not finalize against the caller’s instruction, and the staff-request path is explained truthfully.

Write down whether the receiving endpoint answered; an offered transfer alone is not a completed handoff. Consult Loman’s plan and transfer descriptions for stated scope and keep the observation separate from that product statement.