Every AI agent ships with a confidence problem. The model doesn't know what it doesn't know — it generates plausible-sounding text regardless of whether the underlying facts are correct. That's fine when the stakes are low. It becomes catastrophic when your agent is advising patients, quoting prices, summarizing contracts, or executing actions based on information it fabricated.

What makes hallucinations uniquely dangerous compared to other failure modes:

1. They're silent. A broken API call returns an error you can catch. A hallucination returns a perfectly formatted, confident-sounding response — indistinguishable from a correct one to anyone not checking the source. Users trust it. They act on it. The damage is done before anyone realizes something went wrong.

2. They compound downstream. Agents don't just generate text — they use outputs to take actions. An agent that hallucinates "the contract renewal date is March 15" might then schedule calendar events, draft reminder emails, and log a CRM entry — all based on a date it invented. By the time someone catches the error, six downstream systems have been corrupted.

3. The failure rate is non-zero even for good models. Claude, GPT-4, and Gemini all hallucinate. The rate varies by task domain, prompt structure, and context length — but it never hits zero. "We're using a state-of-the-art model" is not a mitigation. Testing is.

Not all hallucinations are equal in risk or testability. Understanding which type you're dealing with determines which test technique applies. Testing only for one type and missing the others is how teams ship hallucination-prone agents with a false sense of confidence.

There are three testing techniques that cover the hallucination surface area. Each targets different failure modes. You need all three — using only one is like testing a car's brakes and skipping the steering.

Technique 1: Ground-Truth Comparison

Create a test set of questions where you have verified correct answers. Feed these to the agent and compare the output against your ground truth. This catches factual errors and fabricated citations directly. The key requirement: your ground truth must be authoritative and current. Testing against stale reference data produces false negatives.

Technique 2: Consistency Testing

Ask the same question multiple times, with paraphrased phrasing or at different points in the conversation. A reliable agent gives consistent answers. An agent prone to hallucination gives different answers — sometimes correct, sometimes not — because it's generating plausible text rather than retrieving reliable knowledge. High variance on factual questions is a red flag regardless of whether any individual answer is correct.

Technique 3: Adversarial Probing

Ask the agent about things it cannot know, things outside its training data, and things that don't exist. The correct behavior is to express uncertainty or decline. An agent that confidently answers questions about real-time data it has no access to, events after its knowledge cutoff, or wholly fabricated entities is hallucination-prone and cannot be trusted on factual questions where you can't independently verify the answer.

A hallucination test suite is a collection of inputs with known correct outputs, boundary tests, and adversarial probes — run systematically before every deploy. Here's how to build one that actually catches regressions.

Step 1: Identify your agent's factual surface area. List every category of factual claim your agent makes or could make. For a customer support agent: product specs, pricing, policy terms, shipping times. For a medical agent: dosages, drug interactions, diagnostic criteria. For a legal agent: statutes, case citations, filing deadlines. Each category needs its own test cases.

Step 2: Build a ground-truth test bank. For each factual category, create 10-20 questions with verified correct answers. Include: the canonical correct answer, acceptable paraphrases, and values that would be incorrect. Mark tests that require freshness (current prices, live dates) separately — these need to be updated when the underlying data changes.

Step 3: Add citation tests if your agent cites sources. If your agent ever references studies, URLs, cases, or documents — test specifically that those references exist. Scrape the cited URLs. Verify the cited studies appear in the relevant databases. Make this a hard gate: any fabricated citation is a test failure, not a warning.

Step 4: Add consistency tests for high-stakes facts. For every fact your agent states that users will act on, run consistency tests: ask the same question three times with different phrasing. Compute the variance. If the agent gives meaningfully different answers across phrasings, flag it. Reliable facts should be stated consistently — hallucinated ones won't be.

Step 5: Add adversarial probes for your domain. Craft 10-15 questions your agent definitely cannot answer correctly from its training data: current prices, real-time events, internal private data, recent events after its knowledge cutoff, and at least 2-3 completely fictional entities. Every confident answer to these probes is a hallucination.

Pre-deployment testing catches the hallucinations you thought to test for. Production monitoring catches the ones you didn't. LLM behavior drifts over time — model updates, prompt changes, new user inputs outside the tested distribution — and hallucination rates change with it. A monitoring strategy that catches this drift is the difference between knowing your agent degraded and finding out from a user complaint three weeks later.

Drift detection via canary queries. Maintain a set of 20-30 factual questions with known correct answers. Run these against your live agent daily. Track the accuracy rate over time. When it drops — after a model update, a prompt change, or a data pipeline shift — you'll see it in the canary query accuracy before users see it in production responses.

User feedback loops. The most direct signal is users telling you the agent got something wrong. This requires making it easy: a thumbs-down on any response, a "flag this answer" option, or an explicit "was this helpful/accurate?" prompt after high-stakes responses. Log every negative signal with the full conversation context — that's your production hallucination sample set for the next test suite iteration.

Output classification at scale. For agents generating structured outputs (prices, dates, recommendations), run a secondary validation pass on a sample of production outputs. Compare extracted values against source data. Flag responses where the agent's stated facts don't match the data it has access to. This catches hallucinations even when users don't report them.

Uncertainty signal monitoring. Track how often your agent expresses uncertainty ("I'm not sure," "you should verify this," "I don't have access to real-time data"). If this rate drops suddenly after a prompt change, it may mean the agent became more confident — including confidently wrong. Declining uncertainty signals can precede a hallucination rate increase.

Run all 8 before launch. Re-run after prompt changes, model version updates, and knowledge base updates. Any failure is a production reliability risk — not a "known limitation," not "acceptable for now." Fix it or scope the agent's claims to what it can actually verify.