Skip to main content

​
What is the Test Console?

A test call lets you hear the change you just made. The Test Console answers a different question: does everything that used to work still work? It holds the conversations you want every version of a script to handle, runs them for you, and shows you exactly where the script now behaves differently. Open it from Test Console (beta) in the sidebar (under Voice Agent Settings) and pick a script, or from the script editor’s Test → Test Console. The console for a script has three tabs: The header’s action button changes with the tab — Create Scenario and Run Scenarios on Scenarios, New Personas on Personas, Run A2A on Agent to Agent — and the Edit Script icon on the right takes you back to the editor. Back returns you to the script list.
The Test Console is in beta.

​
Scenarios

A scenario is a saved test conversation that can be re-run against the script after every change, so you catch behaviour that has gone backwards before your callers do. Each run is compared with the scenario’s previous result, line by line.

​
Kinds of scenario

​
The Scenarios table

The table has preset views All, Scripts, Steps and Replays (Flow scenarios appear under All). Its columns are Name, Kind, Target Flow, Step, Turns/Cases, Approval, Review, Rev, Status, Diff, Duration, LLM Calls and Updated, and you can filter by: Status is where the latest run stands: Passed when it matched what you have approved, Pending review when something changed and needs a look, and Error when the run itself couldn’t complete. Failed and Halted appear only on results from before the console switched to review-based results; new runs finish as Passed, Pending review or Error. Diff summarises how the latest result compares with the previous one. The Review column shows a green ✓ once a scenario is approved; otherwise it counts the lines that differ from the previous run and the lines you have marked bad. To delete a scenario, use Delete on its row menu. The Delete Scenario confirmation notes: “Past run results keep their records. This cannot be undone.”

​
Creating a scenario

Create Scenario offers two options:
  • Create new — adds a blank Script scenario, named New scenario with a timestamp, for you to write yourself.
  • Create from persona — opens the Create Scenario dialog: “Pick the caller and, optionally, what the call should be about. The service walks the script as that persona and saves the whole path as a new scenario.” Pick a Persona, optionally fill in What should this scenario do? (optional), and click Generate. The row shows Generating while it runs (or Generation failed) and the finished scenario appears shortly after.
You can also build scenarios straight from real calls — see Building scenarios from real calls.

​
Running scenarios

Run Scenarios queues every Script, Flow and Step scenario for the script (“Scenario runs queued”). Each scenario runs on its own and shows Queued as soon as it is waiting. Every one of them runs, whether or not your latest edit touched the steps or flows it tests. Replays are not included in Run Scenarios, because each one places a real call. Run a replay on its own with Re-run in its detail pane.

​
Reviewing a run

Click a scenario to open its detail pane. The header lets you rename the scenario inline, shows its status, and holds the approval menu — Approve, Reject or Mark as pending — plus Save. Saving an edited scenario also approves it (“Scenario saved and approved”). The pane has up to four folds: Expectations are a guide for your review — the result you approve is what later runs are compared against.

​
Marking lines

Each assistant line in Messages carries review controls:
  • A red ✗ means “This differs from the previous run — click to accept it”. Click it once you are happy with the new wording. (On a line where repeated samples disagreed, it reads “The samples did not agree — click to accept this line”.)
  • A black ✗ — Mark this message bad — records that the line is wrong and sends the scenario back to pending.
  • Flag this message files an issue against the script, with an optional “What’s wrong here? (optional)” note (“Issue flagged against the script”). Flags feed the script’s AI recommendations.
When every difference is accepted or dealt with, approve the scenario. Its latest result becomes the baseline the next run is compared with.

​
Editing the caller’s side

In a Script scenario each caller turn has two texts: Caller says (refined) — what the caller actually said — and Heard (fast STT) — blank means the same, for testing what happens when the first, fast transcription mishears them. Tool steps in the transcript carry a Fail this tool during test switch, so you can check the script’s failure path without breaking anything real. A Step scenario has Add history and Add case buttons, and arrows to jump to the previous or next failing case.

​
Direction

Script and Replay scenarios have a Direction setting — “Which way the test call runs. The agent opens an inbound call; on an outbound one the contact speaks first.” The options are Script default (inbound) or Script default (outbound) — whichever the script is set to — Inbound and Outbound. That lets you test an outbound script being dialled into, and the other way around. Flow and Step scenarios have no direction. New scenarios start with a sensible direction: a Script scenario takes the script’s own type, and a replay takes the direction of the call it was made from.

​
Replays

A replay takes a real call and places it again against your current script, playing the original caller’s side — cut from the recording, turn by turn — at the moments they spoke. It shows how the script as it is today handles exactly the same caller. A replay is a real phone or web call and uses call minutes like one. So that it can’t repeat real actions against the original caller, a replay always runs with every side effect switched off — no real SMS, bookings, transfers, webhooks, integration updates or contact updates. The Tools list in the replay’s Details fold shows all six as Off: “Replays always run with every side effect stubbed — no real SMS, bookings or transfers. This cannot be changed per replay.” Stubbed tools answer the script as if they had worked, so the conversation carries on normally.

​
Replay settings

In Details:

​
Editing the caller’s turns

The replay editor is in the Messages fold. Every caller turn has: After call start releases the turn a set number of seconds after the call connects, with no reference to what the agent has said. It is the timing an outbound pickup needs — the person answering says “hello” before anyone has spoken — and caller turns heard before the agent’s first words are given it automatically. To add a line the original caller never said, use Insert a spoken caller turn here: type What the caller says, pick a voice, and click Add turn. Inserted turns can be removed; recorded turns can only be disabled.

​
Reviewing a replay

After a run, the agent’s replies appear between the caller’s turns with the same accept, Mark this message bad and Flag this message controls as other scenarios. While a replay call is live, Cancel run stops it. If you edit the scenario after a run, the old replies show as a plain transcript with a note that the result is from an earlier revision — Re-run to line them up again.

​
Building scenarios from real calls

The quickest way to build a useful set of scenarios is from calls that went wrong — or right — in production.
  • From a call — open any call in the Call Log and click the flask icon in the call details header (“Create a test scenario or replay from this call”):
    • Create Scenario turns the whole call into a Script scenario named Call with a timestamp. The caller’s lines become the fixed side, and the assistant’s side is regenerated on every run. The toast reads “Test scenario created” and tells you to “find it in the Test Console’s Scenarios tab.”
    • Create Replay makes a Replay scenario named Replay with a timestamp from the call’s recording (“Replay scenario created”). It works for phone and web calls. It isn’t offered on a call that is itself a replay — build the replay from the original call instead.
  • From one exchange — hover an assistant line in the call’s transcript and click the beaker, Create a step test from this exchange. That makes a Step scenario from the conversation up to that point, with the caller’s last message as its first case (“Step test created: … — see Test Console › Scenarios”).
Both need a call that ran a script with a main flow, and a replay also needs a call in which the caller spoke.

​
Personas

A persona is a simulated caller — a person with a name, a reason for calling, a manner and a voice — that the platform uses to call your script. Personas belong to your team, so the same set can be used across all your scripts.

​
Generating personas

Personas are created by AI, in batches. Click New Personas to open Generate Personas — “Describe what you want tested and up to 10 varied personas will be generated and added to your team.” A progress log shows while the batch is generated, then Generated N personas. Each persona is given a matching voice from your voice library.

​
A persona’s profile

Click a persona to open Edit Persona. Every field can be changed; click Save Persona to keep your changes (Persona saved). The Personas table lists Name, Tags, Purpose, Description, Voice and Updated.

​
Agent to Agent

The Agent to Agent tab runs personas against your script as automated test calls, and lists every test call placed from the console.

​
Starting a run

Click Run A2A to open Run A2A Test — “Places agent-to-agent test calls against this script. Calls run a few at a time — a large run takes a while.” Recent runs lists your latest runs, and Cancel stops one that is still going.

​
Tools: real side effects are off by default

The Tools section has six switches — Send SMS, Calendar booking, Live handoff, Webhooks / external requests, Third-party integrations and Contact updates — and all six start off: “Disabled tools are stubbed during test calls — no real SMS, bookings or transfers.” A stubbed tool answers the script as if it had worked, so the conversation carries on down its normal path and the transcript shows the step succeeding — but nothing is actually sent, booked, dialled or changed. Switch a tool on only when you want the run to exercise the real integration, and make sure its destination is safe to hit. Browser and phone test calls from the editor are not affected by these switches — they always run your tools for real.

​
The runs table

The table has preset views All, Personas and Replays, and a Type column showing Persona or Replay. Replay rows are named after the contact on the original call. The columns are Name, Type, Tags, Purpose, Status, Run, Script rev, Max duration, Prompt mode and Created. Click a row to open the call’s details — transcript, recording and analysis — with the run’s settings, including the TTS voice, alongside.

​
Tips

  • Turn every bad call into a scenario. When a real call goes wrong, create a scenario or replay from it before you fix the script — then re-run it to prove the fix, and keep it to make sure it stays fixed.
  • Approve deliberately. Approving a scenario makes its latest result the standard later runs are judged against. Read the highlighted lines first.
  • Use step batteries for tricky steps. A Step scenario with a case for each way callers answer is the fastest way to test the conditions on a busy step.
  • Keep side effects off. Leave the A2A tool switches off unless the run is specifically testing an integration.

​
Next Steps