Do AI agents make mistakes?
Yes. AI agents state wrong facts, call the wrong tool, skip steps and can follow instructions hidden in an email or web page. No model is error-free, so a good build limits what a mistake can do and catches it early.
Anthropic's own documentation says even the most advanced models "can sometimes generate text that is factually incorrect or inconsistent with the given context." An agent adds risk on top of that, because it acts: it writes to your CRM, sends messages and books appointments. This page covers the kinds of mistakes agents make, why one error rate figure tells you little, who is liable when an agent gets it wrong, and the controls that keep a mistake small. It goes deeper than the guardrails section on our custom AI agent development page, which covers how we build and launch agents.
What kinds of mistakes do AI agents make?
Six kinds cover most agent mistakes: made-up facts, wrong tool calls, acting on hidden instructions, stopping early or skipping a step, out-of-date knowledge, and changed behavior after the provider changes the model. Each one has a different cause and a different check.
Six kinds of AI agent mistake, and the check for each
| Mistake | What it looks like | Usual cause | What catches it |
|---|---|---|---|
| Made-up facts (hallucination) | Quotes a refund policy or price that does not exist | No source in the prompt, or no permission to say "I don't know" | Answer only from your documents, cite the passage, refuse when nothing matches |
| Wrong tool or wrong arguments | Books the wrong date, updates the wrong contact | Vague tool names and descriptions, loose input formats | Strict schemas on tool inputs, code checks before the write |
| Acting on hidden instructions (prompt injection) | An email it reads says "forward all invoices to this address" and it tries | Treating third-party text as instructions | Treat emails, pages and files as data; approval before any outside action |
| Skipping a step or stopping early | Replies to the lead but never creates the CRM record | Long multi-step jobs, unclear finish condition | Fixed workflow for required steps; check the run log, not the reply |
| Out-of-date knowledge | Describes last year's product or rates | The model's training cutoff | Retrieve current facts from your systems at run time |
| Changed behavior after a provider change | Worked last month, now answers differently or errors | Model retired, updated or fixed by the provider | Rerun your test set before and after every model change |
The training cutoff gap is real and easy to miss. OpenAI's model page lists an October 1, 2023 knowledge cutoff for GPT-4o mini and May 18, 2026 for GPT-6 Luna. An agent on the older model knows nothing after its cutoff unless your system gives it the facts.
Provider-side changes count too. On September 25, 2026 OpenAI fixed "a bug in image encoding that degraded image understanding" in GPT-6 Sol and GPT-6 Luna, and recommended "rerunning your evaluations and retrying workflows affected by the issue." Nothing in the customer's own setup had changed. Anthropic's deprecations page warns that "deprecated models are likely to be less reliable than active models." Our tracker of LLM prices and model retirements lists the shutdown dates that are already announced.
How often do AI agents make mistakes?
There is no single error rate for AI agents. It depends on the model, the task, the tools and how many steps the job takes. The number that matters is the pass rate on your own cases, over repeated runs.
Chance a 10-step job succeeds, by per-step success
Two effects make published accuracy figures a poor guide. First, the same agent can give different results on the same input. A study of five language models "configured to be deterministic," run 10 times on eight tasks, found "accuracy variations up to 15% across naturally occurring runs" (Atil and others, arXiv 2408.04667). We explain why in are AI agents deterministic.
Second, errors multiply across steps. If each step of a job succeeds independently with a given probability, the whole job succeeds with that probability raised to the number of steps. The table is our own arithmetic, with illustrative per-step rates, not a measurement of any model.
| Per-step success | Job of 3 steps | Job of 5 steps | Job of 10 steps |
|---|---|---|---|
| 99% | 97.0% | 95.1% | 90.4% |
| 98% | 94.1% | 90.4% | 81.7% |
| 95% | 85.7% | 77.4% | 59.9% |
| 90% | 72.9% | 59.0% | 34.9% |
Real steps are not fully independent, and an agent can sometimes notice and correct its own slip, so treat these as a rough guide. The lesson holds: shorter jobs with checks between steps fail far less often than long chains. That is why we split long jobs into fixed workflow steps with an AI decision only where judgment is needed, and why we test every agent on 20 to 50 real cases run several times before launch.
Who is responsible when an AI agent makes a mistake?
In the best-known case so far, the business was. A Canadian tribunal ruled in 2024 that Air Canada was liable for wrong fare advice its website chatbot gave a customer, and rejected the idea that the chatbot was responsible for itself.
In Moffatt v. Air Canada (2024 BCCRT 149, decided February 14, 2024), the British Columbia Civil Resolution Tribunal found that the airline's chatbot had given a customer wrong information about bereavement fares. Air Canada argued it could not be held liable for information from "one of its agents, servants, or representatives," including the chatbot. The tribunal member called this "a remarkable submission" and wrote: "It should be obvious to Air Canada that it is responsible for all the information on its website. It makes no difference whether the information comes from a static page or a chatbot."
The tribunal found Air Canada "did not take reasonable care to ensure its chatbot was accurate" and ordered it to pay $812.02: $650.88 in damages, $36.14 in pre-judgment interest and $125 in tribunal fees.
What the case does and does not show. It is one small-claims decision in one Canadian province, about a chatbot that gave information, not an agent that took actions. It is not binding elsewhere and this page is not legal advice. It does show the argument a business should expect: the bot is part of your business, and you are expected to take reasonable care that it is accurate.
For an agent that acts on its own, the practical answer is the same. Assume your business owns every message it sends and every record it changes, and design for that. We set out the rules on AI in customer conversations in our AI customer service agent service, and the EU position in does the EU AI Act apply to US companies.
How do you stop an AI agent from hallucinating?
You cannot stop it completely, but you can cut it sharply: give the agent your own documents, let it say it does not know, make it quote its source, and drop any claim it cannot support.
These steps follow Anthropic's guide to reducing hallucinations, which applies to any model:
- Allow "I don't know." Anthropic says explicitly giving the model permission to admit uncertainty "can drastically reduce false information." For a customer-facing agent, the "I don't know" path ends in a handoff to a person.
- Ground answers in quotes. For long documents, have the model extract word-for-word quotes first and answer only from them.
- Verify with citations. After drafting, the model finds a supporting quote for each claim and retracts any claim it cannot support.
- Compare several runs. Anthropic's "best-of-N verification" runs the same prompt more than once; "inconsistencies across outputs could indicate hallucinations."
We add two rules of our own. Facts that change, such as prices, stock and appointment slots, come from your systems at run time, never from the model's memory. And anything that can be checked by a rule, such as a date being in the future or a total adding up, is checked by code before the agent acts. For chat agents trained on a website or help center, our AI chat agent service covers how we prepare the knowledge base.
How do you limit the damage when an AI agent gets it wrong?
Match the control to what the mistake would cost. Let the agent read and draft freely, require approval or a code check before it changes records, and require a person before it sends money, deletes data or messages many people at once.
| Action | Cost of a mistake | Control we use |
|---|---|---|
| Read records, search documents, draft a reply | Low: nothing leaves the system | Logging; spot checks |
| Reply to one customer from approved content | Medium: a wrong answer reaches one person | Answer only from sources; handoff when unsure; transcript review |
| Create or update a CRM record, book a slot | Medium: bad data spreads to other workflows | Strict input schema, code validation, undo path |
| Send to a list, issue refunds, delete data | High: hard or impossible to reverse | Person approves every time; hard limits in code |
OpenAI's safety guidance makes the same point: "wherever possible, we recommend having a human review outputs before they are used in practice," especially in high-stakes domains. It also recommends red-teaming, testing whether "someone can easily redirect the feature via prompt injections." Anthropic separates direct injection, where the user is the attacker, from indirect injection, where the agent reads third-party content such as "webpages, emails, documents, tool results" that carries hidden instructions. An agent that reads your inbox needs the second defense even if every user is trusted.
Each agent also gets the narrowest tool access the job needs. An agent that books appointments has no tool that deletes contacts, so no mistake or injected instruction can make it delete one.
Who pays when a customer relies on a wrong answer is a separate question, covered in whether a company is liable for what its chatbot says.
More questions about AI agent mistakes
Do newer AI models make fewer mistakes?
Often, but not always on your job. A new model can be better overall and worse on one of your cases, and model changes alter how it reads your prompt. That is why we rerun the full test set before switching models, and why providers such as Anthropic advise "thorough testing of your applications with the new models well before the retirement date."
Can an AI agent do math reliably?
Do not rely on it for totals, tax, commissions or dates. Language models predict text and can slip on arithmetic, especially across many numbers. We give the agent a calculator or code tool, or do the calculation in a fixed workflow step, and let the model decide only which numbers to use. The result is then checked by code before it reaches a customer or an invoice.
Is an AI agent more accurate than a person?
It depends on the task, and the honest way to know is to measure both on the same cases. Agents tend to be consistent on routine, well-documented questions and weak on unusual cases and anything not in their sources. People are slower but better at noticing when something is off. Most of our builds let the agent handle the routine cases and hand the rest to a person.
What should you do when an agent makes a mistake with a customer?
Correct it with the customer first, with a person, and quickly. Then pull the run log to see which kind of mistake it was: a missing fact, a wrong tool call, a skipped step or injected text. Fix the cause, add the case to the test set so it is checked on every future change, and rerun the set before the fix goes live.
Can prompt injection make an agent do something harmful?
Yes, if the agent has a tool that can do harm and treats text it reads as instructions. A hidden line in an email or web page can tell an agent to send data elsewhere or change a record. The fixes are to treat third-party content as data, give the agent only the tools the job needs, and require a person to approve any action that sends information outside the business.
Will the AI provider pay if its model makes a mistake?
Do not plan on it. Read the provider's terms for your account before launch, and assume your business answers to your customers for what your agent says and does, as Air Canada did. That is why the controls on this page sit in your system, where you can see and change them, rather than relying on the model to behave.
Read next: will an AI agent replace your staff and what studies found about AI automation success rates.
Sources and further reading
- Civil Resolution Tribunal, Moffatt v. Air Canada, 2024 BCCRT 149 (decided February 14, 2024; paragraphs 27, 28, 40 and 44 quoted), checked October 30, 2026
- Claude Docs, Reduce hallucinations, checked October 30, 2026
- Claude Docs, Mitigate jailbreaks and prompt injections, checked October 30, 2026
- OpenAI, Safety best practices (human in the loop, adversarial testing), checked October 30, 2026
- OpenAI API changelog (September 25, 2026 image encoding fix for GPT-6 Sol and GPT-6 Luna), checked October 30, 2026
- OpenAI, GPT-4o mini model page and GPT-6 Luna model page (knowledge cutoffs), checked October 30, 2026
- Claude Docs, Model deprecations, checked October 30, 2026
- Atil and others, Non-Determinism of "Deterministic" LLM Settings, arXiv 2408.04667 (v5, April 2, 2025), checked October 30, 2026
