Response Checks
Response checks review every reply an agent writes before the customer sees it. You choose what to look for, such as a price the agent made up, a card number, a leaked API key, or an answer that breaks your policy. You also choose what happens when a check finds a problem: record it, remove the matched text, block the reply, or ask the agent to try again.
What it's for
An agent's instructions tell it what to do. Response checks make sure it did. Use them when a wrong answer is costly:
- A support agent quotes a refund amount or order number that doesn't exist.
- A tool returns a customer's Social Security number and the agent reads it back.
- Someone talks the agent into repeating its own instructions.
- A sales agent mentions a competitor or promises something you don't offer.
Checks run on every reply, in every channel: web chat, embedded widgets, SMS, phone calls, and replies returned through the API. Text the agent writes before it calls a tool is checked as well.
Navigating here
From the dashboard sidebar, open Agentic Workflows → Agents, open a Simple agent, and scroll to the Response checks card. It lists each check with its type, its action, and an on/off switch. Add check creates a new one. Under the list are the agent-wide settings, a one-line note on how replies are released, and the Test checks panel.
Key concepts
How checks run
When the agent finishes a reply, or each sentence of it (see How replies are released), every enabled check looks at the text at the same time. Adding more checks doesn't make them run one after another, so the slowest check sets the wait, not the total.
Cost depends on the check type:
- AI review asks a model to judge the reply, so it uses model credits like any other model call. If the review runs on your company's own model connection, it uses your key instead.
- Every other check type is free. They're rules that run instantly and never call a model.
Check types
Facts must come from the conversation
This check pulls the facts out of each reply and makes sure each one came from somewhere real. It looks for:
- numbers and amounts
- dates
- links
- email addresses
- phone numbers
- IDs, such as order or account numbers
- quoted phrases of three or more words
A fact counts as supported if it appears in what the customer said, in a tool's request or result, in retrieved knowledge, or in the agent's instructions and prompt variables. Formatting differences don't matter: $1,200.00 matches 1200, March 4, 2026 matches 2026-03-04, and (555) 010-2030 matches 5550102030. Single digits and list numbers ("1.", "2.") are ignored.
Example: a customer asks about a late order. The order-lookup tool returns order A-88213 with a total of $42.50. If the agent replies "Your order A-88213 for $45.00 ships Friday," the check catches $45.00, because that amount never appeared in the conversation, a tool result, your knowledge, or the instructions. A-88213 passes because the tool returned it.
You can limit the check to some kinds of facts. For example, check only numbers and IDs if the agent paraphrases quotes on purpose. Add allowed values for facts the agent may always state even though they aren't in the conversation, like your support phone number, your website, or "30 days" from your return policy.
Pattern rule
A pattern rule looks for text that matches a regular expression you write, such as an internal ticket format (TCK-\d{6}) or a product code you never want quoted. Options:
- Flags: case-insensitive, multi-line, and the other standard flags.
- Required context keywords: a match counts only if one of these words appears within 80 characters of it. For example, match nine digits only near "account" or "member ID", so ordinary numbers don't trigger the rule.
- Card-number check: a match counts only if it passes the checksum that real card numbers pass. This screens out random digit runs.
Sensitive data detectors
These are ready-made detectors. Turn on the ones you need:
| Detector | What it finds |
|---|---|
| Social Security numbers | US Social Security numbers. |
| Card numbers | Payment card numbers that pass the card checksum. |
| Bank account & routing numbers | US bank account and routing numbers. |
| Driver's license numbers | US driver's license numbers. |
| Email addresses | Any email address. |
| Phone numbers | Any phone number. |
| IP addresses | IP addresses. |
| IBANs | International bank account numbers. |
| Dates of birth | Dates written as a date of birth. |
| API keys, tokens & private keys | Credentials from common cloud, developer, payment, and AI providers, signed tokens, and private key blocks. |
| Private values the customer entered | Any value the customer typed in for a private argument in this conversation, or a value from a secret, repeated word for word. |
| The agent's own instructions | A long run of words copied from the agent's instructions. The default is 12 words in a row, and you can change it. |
| Internal links & addresses | Links to local machines, private network addresses, and internal-only domains, plus any domains you list. |
Private values. When a tool asks the customer for a private value, like a card number or PIN, the customer's answer goes straight to the tool. The AI never sees it. And if a tool's result repeats that value (for example, a payment tool that echoes the card number back), the value is now masked before the AI reads the result. So the agent can't say a private value it was never shown. The Private values the customer entered detector is a backstop for the rare case where the customer also typed the same value into the normal chat.
Script rule
A script rule is a short AutessaScript that decides whether the reply passes. Use it for logic a pattern can't express, like a competitor list kept in one place or a rule that depends on what the customer asked.
The script can read three variables:
| Variable | What it holds |
|---|---|
variables.response | The reply text being checked. |
variables.segment | The sentence being released right now. When the whole reply is checked at once, this is the same as variables.response. |
variables.history | The conversation so far, as a JSON string: an array of { "role": "user" | "assistant" | "tool", "name"?: string, "content": string }. |
It must report its result with:
response.setResponse({ pass: true | false, reason: "optional explanation", matches: ["optional", "exact text"] });
pass:trueif the reply is fine,falseif it should trigger the check's action.reason: a short explanation. It shows in the check results, and when the action retries, it's passed to the agent as feedback.matches: exact pieces ofvariables.responseto report, or to remove when the action is Remove the matched text.
Script rules run on their own, without platform calls or network access, and must finish within 2 seconds. A script that throws an error, runs too long, or never calls response.setResponse counts as failed.
Example: block any mention of a competitor.
const competitors = ['Acme Support', 'HelpCo', 'DeskPro'];
const text = variables.response.toLowerCase();
const found = competitors.filter(name => text.includes(name.toLowerCase()));
response.setResponse({
pass: found.length === 0,
reason: found.length ? 'Mentions a competitor: ' + found.join(', ') : '',
matches: found.map(name => {
const start = text.indexOf(name.toLowerCase());
return variables.response.substring(start, start + name.length);
})
});
AI review
An AI review asks a model to judge the reply against criteria you write in plain language:
- Good criteria: things every reply must do, such as "Answers the customer's actual question" or "Offers to connect a human when the customer is upset."
- Bad criteria: things no reply may do, such as "Gives legal or medical advice" or "Promises a delivery date."
Criteria can use prompt and environment variables with @@{variable} placeholders. The reviewer sees the reply, the tool results behind it, and the last few exchanges of the conversation (3 by default, and you can change it). Pick the Model the reviewer uses. A small model is usually enough and keeps the cost down.
AI review is the right choice for judgment calls: tone, policy, completeness. For anything a rule can catch, like a number, a pattern, or a detector, a rule is faster, free, and gives the same answer every time.
Actions
Each check has one action, which decides what happens when it fails:
| Action | What happens | Available on |
|---|---|---|
| Flag only | The reply is sent unchanged, and the problem is recorded in the conversation history. | All check types |
| Remove the matched text | The matched text is replaced with [REDACTED] and the rest of the reply is sent. | All except AI review |
| Block the reply | The customer gets the fallback message instead of the reply. | All check types |
| Retry, then block | The agent is told what was wrong and writes a new reply. If it still fails after the maximum number of retries, the customer gets the fallback message. | All check types |
| Retry, then send | Same as above, but after the last retry the final reply is sent anyway. | All check types |
When several checks fail on one reply, the strongest action wins. A retry comes first while retries remain, then a block, then removal. Flags are always recorded.
Two settings apply to the whole agent:
- Fallback message: what the customer sees when a reply is blocked. If you leave it empty, it's "Sorry, I can't share that. Is there anything else I can help you with?"
- Max retries: how many new replies the agent may write before a retry action gives up. The default is 2.
How replies are released
How a reply reaches the customer depends on the checks you've turned on. There's no separate setting. The note under the check list tells you which way the agent works now:
| Your checks | How replies are released | What the customer notices |
|---|---|---|
| No checks, or every check is Flag only | Replies are sent right away, and checks only record problems. | Nothing. Checks run alongside the reply and add no delay. |
| Checks that remove or block, with no AI review and no retry | Replies are checked sentence by sentence before they're sent. | Almost nothing. Each sentence waits only a few milliseconds, so streaming and voice still feel live. |
| Any AI review that doesn't just flag, or any Retry action | Each reply is held until every check passes. | The reply appears all at once, after the agent has finished writing it and the slowest check is done. |
Holding the whole reply is the only way to retry. You can't take back a sentence the customer already read or heard. That's why AI review and retries switch the agent to the hold mode. If speed matters more than retrying, use Block the reply or Remove the matched text with rule-based checks, and keep the sentence-by-sentence mode.
On phone and voice calls, each sentence is spoken only after it passes. A blocked sentence is never spoken. The caller hears the fallback message, and the agent stops that reply.
When a check can't run
A check can fail to run because its pattern is invalid, its script errors or runs too long, or its AI review doesn't answer in time. When that happens, the check counts as failed, and its action applies, so a broken check never lets a reply through unchecked. The one exception is Flag only: a flag check that can't run just records the error, and the reply goes out normally. The error shows in the check results so you can fix it.
How to add a check
- Open the agent and scroll to Response checks.
- Click Add check and pick a type: Facts must come from the conversation, Pattern rule, Sensitive data detectors, Script rule, or AI review.
- Give the check a name you'll recognize in conversation history, such as "No made-up prices."
- Fill in the type's settings: fact types and allowed values, the pattern, the detectors, the script, or the criteria and model.
- Choose the Action.
- Save the check, then set the Fallback message and Max retries if you use block or retry actions.
- Save the agent, try the checks in the Test window, then publish. Customers get the checks once the version that has them is published. See Versions.
Start new checks on Flag only for a while. Review what they catch in Agent History, then switch them to remove, block, or retry once you trust them.
How to test checks
- In the Response checks card, open Test checks.
- Paste a sample reply, the text you want to check.
- Optionally paste sample sources: the tool results, knowledge, or customer messages the reply should be based on. The facts check compares against these and the agent's instructions.
- Click Test. You'll see each check's result and reason, what it matched, the decision (for example Redacted or Blocked), and the exact text the customer would receive. If the decision is a retry, the panel says the reply would be retried.
AI review checks really run in the test panel, so each test uses model credits. To see checks in a live conversation, use the agent's Test window: the final message shows a checks chip with the decision. See Testing.
Where results show
Every checked reply gets a Response checks card in the conversation viewer in Agent History. It shows:
- one row per check, with its result and reason
- a badge for what happened to the reply: Passed, Flagged, Redacted, Blocked, Retried, or Sent after retries
- the original draft, in a collapsible section, when the customer received something different
Check results are for your team. They're never shown to the customer and aren't returned by the client conversation API.
Critic agents are now Simple agents with an AI review
The Critic agent type has been retired. Its good and bad criteria now live in an AI review check on a Simple agent, which can also be combined with every other check type.
Existing Critic agents keep working unchanged. Their criteria, reviewer model, retry count, and context setting show up as an AI review check with Retry, then send, which is how Critic agents behaved. The next time you save one, it's saved as a Simple agent with that check. New Critic agents can't be created.
One thing does change, and it's an improvement: a draft that fails the review is no longer streamed to the customer before it's judged. Because the AI review holds each reply, the customer only sees the reply that's released.