How to build an AI agent: the loop and four things that break it

How do you build an AI agent? The mechanism is a loop, and it takes about twenty lines. Everything hard about agents is in the failure handling that goes around it.

This walks through one working agent end to end, then the four things that break it in production.

What separates an agent from a chatbot

A chatbot answers. An agent acts, checks the result, and decides what to do next, repeating until the task is done or it gives up.

That loop is the whole idea. The model doesn’t execute anything itself; it names a tool and supplies arguments, your code runs it, and the result goes back into the conversation.

The model never touches your systems. It writes down what it wants done, and your code decides whether to do it.

That separation is also your entire security boundary, which is why the tool definitions matter more than the prompt.

How to build an AI agent: the loop, concretely

Send the task and the list of available tools. The model replies either with an answer or with a tool call. If it’s a tool call, run it, append the result, and send the whole thing back. Repeat.

Three things must be bounded from the first version, not added later: a maximum number of iterations, a maximum spend, and a timeout. An agent without those is a program that can bill you indefinitely.

Defining tools is where quality comes from

Most agent failures I’d expect you to hit are tool design problems wearing a model costume.

Weak tool	Better tool
run_query(sql)	find_orders(customer_id, date_from, date_to)
Free-text parameters	Enumerated options the model must choose between
Returns raw dumps	Returns a summary plus a count
Fails with a stack trace	Fails with a sentence saying what to try instead

Constraining the output format helps as much as constraining the input. Use your provider’s structured-output or tool-calling mode rather than asking for JSON politely, because that enforces the schema during decoding instead of hoping the model complies.

Error messages are the underrated half. The model reads them, so an error that says which parameter was wrong and what values are valid gets corrected on the next iteration. A stack trace gets a retry of the same mistake.

Failure one: errors compound

This is the arithmetic that decides whether your agent is viable, and it’s unforgiving.

A step that succeeds 95% of the time gives you about 60% across ten steps and roughly 36% across twenty. Nothing about model quality changes the shape of that curve, as our piece on why agents stall in production covers.

So keep chains short. Five reliable steps beat twenty hopeful ones, and a checkpoint where a person confirms before anything irreversible happens is worth more than a smarter model.

Failure two: no memory between runs

Every request starts empty. Anything the agent should know across sessions has to be stored by you and resent, and a bigger context window doesn’t help because the window is per-request.

Resending everything also gets expensive fast, since context is billed on every iteration. Our piece on what an AI feature costs works through why long loops bill quadratically.

Store a compact summary rather than a transcript. What the agent decided and why is worth keeping; the full back-and-forth usually isn’t.

Where the agent needs facts rather than history, retrieval is the better tool. The original RAG paper framed this as combining “pre-trained parametric and non-parametric memory”, and an index you update beats stuffing documents into every iteration.

Position matters too. Stanford’s Lost in the Middle found accuracy “significantly degrades when models must access relevant information in the middle of long contexts”, so a growing transcript buries the instruction that matters most.

Failure three: it can’t tell when to stop

A capable agent retries when a step fails. A well-designed one notices the retry isn’t working and stops, and most current systems do the first without the second.

The result is a loop burning tokens on a task that became impossible three steps ago. Detect repetition explicitly: if the same tool is called with the same arguments twice, break and escalate.

Newer models are better here. DeepSeek’s R1 paper reported “self-reflection, verification, and dynamic strategy adaptation” emerging from reinforcement learning alone, which is exactly the missing capability. It’s improvement rather than a solution.

Failure four: it does something nobody asked for

This one stopped being hypothetical this month. Reporting from Reuters, Bloomberg and CNN said AI agents from at least two major labs breached external corporate systems without being instructed to.

That’s what put OpenAI, Anthropic, Google and Meta in front of the White House to agree a testing framework, which CNBC reported focuses specifically on whether models can execute cyberattacks.

SiliconANGLE reported the labs were invited to review the framework rather than simply accept it, which tells you how early this discipline still is.

The lesson for your build is narrow permissions. Give the agent the minimum access the task requires, make destructive actions require confirmation, and log every tool call with its arguments.

Prompting the loop

Two instructions do most of the work, and both come from ordinary prompting practice.

Ask for a plan before the first tool call, since Google Research showed in Chain-of-Thought Prompting that working through steps improves accuracy substantially. And give it an explicit exit: if the task can’t be completed, say so rather than continuing.

Both are covered further in our prompting guide, and they matter more in a loop than in a single call because mistakes propagate.

When not to build one

If the steps are known in advance, write a script and call the model where you need language handled. A deterministic pipeline with one model call is more reliable and considerably cheaper than an agent choosing its own path.

Agents earn their complexity when the path genuinely varies by input and you can’t enumerate the branches. That’s a narrower set of problems than the current enthusiasm suggests.

Start with the smallest useful version: three tools, five iterations, a spend cap, and a human checkpoint before anything irreversible. Add autonomy only where the logs show it was earned.