# Tests

> Check an agent before you publish it: a made-up caller talks to the draft and a judge says whether what had to happen happened.


# Tests

A test is a phone call you describe in advance. You say who calls and what must happen; Allerto plays the caller, lets the agent's saved draft answer, reads the result and tells you, sentence by sentence, whether it held. Run the tests after every change, before publishing.

```json
{
  "name": "Water leak in the garage",
  "scenario": "You are Carla Bianchi, 12 Garibaldi Street. Water has been dripping from the garage ceiling since this morning. Answer briefly and give your number (347 555 1234) only when asked.",
  "expectations": [
    "Asks for a callback number",
    "Does not promise a visit at a set time"
  ],
  "expected": { "nome": "Carla Bianchi", "telefono": "+393475551234" }
}
```

| Key            | Meaning                                                                                                                                                   |
| -------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `name`         | A few words naming the situation.                                                                                                                         |
| `scenario`     | Who calls, what they want and how they behave on the phone, in the second person. Another model plays this caller.                                        |
| `expectations` | Up to 10 sentences, each judged on the conversation. When the conversation does not show it, it did not hold.                                             |
| `expected`     | Values the result must hold at the end, by field key. Choices, numbers, yes/no and phone numbers are compared exactly; free text by meaning.              |
| `caller`       | The number the agent sees (`+393330000000` by default), or `null` for a hidden number: a call from the website or a switchboard that forwards without it. |
| `inputs`       | Outbound agents: the call's inputs. Missing ones take the example values.                                                                                 |

## Running

```bash
curl -X POST https://api.allerto.cloud/v1/agents/$AGENT/tests/run \
  -H "Authorization: Bearer $ALLERTO_KEY" -d '{}'
```

The answer comes at once with one run per test (`status: "running"`); each call takes 15–40 seconds. Poll `GET /v1/agents/{id}/test-runs/{run}` or the test list, whose items carry their `last_run`. A run ends as `passed`, `failed` (with the reason of each check) or `error`. A conversation that never ends (`ended: "turns"`) fails whatever else held.

By default the tests run on the **saved draft**; pass `"version": 3` to run them on a published version. One run at a time per agent.

## From a real call

`POST /v1/agents/{id}/tests/from-call {"call": "<id>"}` drafts a test from a call: the same situation and way of talking, with **invented** names, numbers and addresses (a test outlives the call, which is anonymised after the retention period). Nothing is saved: review it, then create it. In the console it is the **Create a test** button in the call's conversation.

## What a test does not check

The voice model is not in the loop: a text model plays the agent with the same prompt, fields and documents. A test checks what you change when you edit an agent (instructions, fields, documents, transfer rules), not the voice, the timing or how a word is heard on the line. Tools during the call are not run in a test.