I have watched more AI agent projects fail than succeed. I have contributed to some of those failures. After enough of them, the failure modes stop being surprising and start being predictable, which means they are preventable.
The agents that fail all share a common characteristic: they trust the LLM to manage things the LLM is not good at managing. The agents that succeed all share a different characteristic: they are boring. Explicit state. Deterministic tools. Human-readable logs. Observable behavior. No magic.
Here is the failure taxonomy and the boring architecture that avoids it.
Failure Mode 1: The Hallucinated Tool Call
The agent is given tools and asked to use them to complete a task. The LLM decides to call a tool that does not exist, or calls a real tool with arguments that do not match the schema, or calls a tool that exists but is not appropriate for the current step. The failure is silent: the error gets swallowed, the agent retries with a different (equally wrong) approach, and after several turns produces a confident-sounding response that describes work it did not actually do.
This failure is common in frameworks that use aggressive retry logic with opaque error handling. The agent does not fail loudly; it fails quietly and then pretends it succeeded.
The fix: Every tool call must validate input schema before execution and must return a structured result that includes success/failure status. On failure, the agent stops and escalates rather than retrying indefinitely. Indefinite retry without human intervention is how a three-turn task becomes a thirty-turn disaster that exhausts your token budget.
Failure Mode 2: State Drift
The agent tracks conversation state in the LLM's context window. Over a long multi-turn conversation, the LLM's understanding of the current state drifts from the actual state. It "forgets" that step 3 was completed and tries to redo it. It "remembers" a decision from ten turns ago that was subsequently reversed. It contradicts itself across turns in ways that a human reviewer would catch but the automation does not.
This failure is especially insidious because the agent often produces coherent-sounding output while its internal state model is wrong. The output looks fine. The underlying actions are incorrect.
The fix: State is never in the context window. State lives in a database. Every turn, the agent reads current state from the database, takes its action, and writes updated state back. The context window gets the current state at the start of each turn, not a history of prior turns. This means you can restart the agent from any point, inspect state at any moment, and recover from crashes without losing progress.
class AgentState(BaseModel):
task_id: str
current_step: int
completed_steps: list[str]
pending_actions: list[dict]
data_collected: dict
status: Literal["in_progress", "waiting_hitl", "completed", "failed"]
error: str | None = None
# Every turn reads and writes state explicitly
def run_agent_turn(task_id: str, user_input: str | None = None) -> AgentState:
state = db.get_agent_state(task_id)
# Build context from current state (not conversation history)
context = build_context_from_state(state)
# LLM decides next action based on current state
action = classify_next_action(context, user_input)
# Execute action deterministically
result = execute_tool(action)
# Update state explicitly
state = update_state(state, action, result)
db.save_agent_state(task_id, state)
return state
Failure Mode 3: The Confidence Mirage
The LLM produces a response that sounds certain. The response is wrong. Nobody detects it because the system has no mechanism for checking LLM output against ground truth before it is acted on.
This failure is the one that causes real business damage. An invoice amount is extracted incorrectly but with high expressed confidence. A contract clause is misread but the summary sounds accurate. A customer data record is created with a transposition error that propagates into downstream systems.
The fix: Every LLM output that drives a downstream action must pass through a validation layer before the action executes. The validation layer checks schema, business rules, and, where possible, cross-references with authoritative sources (if the LLM says the invoice total is $47,000 and the line items sum to $52,000, one of those is wrong). Confidence scores from the model are useful signals but are not sufficient as sole validation.
Failure Mode 4: Unobservable Behavior
The agent runs. Something goes wrong. Nobody knows what happened. The logs show "task completed" but the output is wrong. There is no record of what the agent tried, which tools it called, what data it processed, or where in the flow the error occurred.
This failure is not a model failure; it is an infrastructure failure. And it is far more common than it should be. I have seen production agents where the only debugging tool is re-running the task and hoping you can observe the failure in real time.
The fix: Every agent action, tool call, input, output, and state transition is logged with a structured event. Every event includes: timestamp, task ID, step ID, tool name, input hash, output hash, confidence score, and execution time. The log is queryable. When something goes wrong, you can reconstruct exactly what happened.
# Event schema for agent observability
agent_event:
timestamp: "2026-05-11T14:23:07Z"
task_id: "task-8849a3"
turn: 4
step_id: "extract_vendor_name"
tool: "vision_extract"
input_hash: "sha256:a3f9c..."
output:
vendor_name: "Karachi Steel Suppliers"
confidence: 0.94
raw_text: "KARACHI STEEL SUPPLIERS LTD."
duration_ms: 1847
status: "success"
validation_result: "passed"
Failure Mode 5: The Goal Drift Problem
The agent is given an objective. Over multiple turns, it interprets the objective more and more broadly. It starts taking actions that are technically within the objective but were not intended. An agent asked to "update the client record with the new address" also updates the billing address, the shipping address, and the emergency contact address because they all seemed related to "address."
This is a fundamental property of large language models: they generalize. Generalization is useful for understanding ambiguous instructions from humans. It is dangerous in automated systems executing operations with real consequences.
The fix: Every action requires explicit authorization against an allow-list of permitted operations for the current task. The agent cannot take an action that was not in the task's authorized action set, regardless of what the LLM thinks is appropriate.
The Boring Architecture
An AI agent that avoids all five failure modes looks like this:
User/System Request
|
v
Task Creation (explicit goal, authorized actions, initial state)
|
v
For each turn:
1. Load current state from DB
2. Build context (state + task definition, NOT conversation history)
3. LLM classifies next action (constrained output schema)
4. Validate action against allow-list
5. Execute tool (single responsibility, deterministic)
6. Validate output (schema + business rules)
7. Update state in DB
8. Log event (structured, queryable)
9. Check: is task complete? needs HITL? failed?
This is not elegant. It is not a clever use of emergent LLM capabilities. It is plumbing. It is explicit. It is recoverable. It runs reliably in production because every component does exactly one thing, every failure is visible, and every state is inspectable.
What I Got Wrong
I built an agent for a Sydney recruitment agency that used conversation history as its state. It worked well for tasks under ten turns. At twenty turns, it would occasionally decide it was farther along in the process than it was and skip steps. At thirty turns, it would contradict decisions made early in the conversation.
I rewrote the state model to use explicit database-backed state. The rewrite took three days. It has not had a state-related incident since. Three days of rewrite versus six months of intermittent failures and manual cleanup: that is the cost of the wrong architecture.
Production Reality
The boring architecture is slower to build. It requires designing the state schema, the event schema, the allow-list, and the validation layer before you write a single line of agent logic. Most teams skip this because the demo version works without it.
The demo version also fails in production. The boring architecture does not. That is the only metric that matters after the demo.