Building AI Agents: The Loop, the Tools and Everything Around Them

“Agent” now gets used for almost anything: a chatbot with a search button, a script that calls a model three times, a system that works on a codebase for an hour and opens a pull request.

The core idea is small. An agent is a model that uses tools in a loop until a job is done. The hard part is everything around the loop: what the model can do, what it sees on each step, when it stops, what it’s allowed to touch, and how you know it works.

#What an agent is

OpenAI’s guide defines agents as systems that independently accomplish tasks on your behalf. A model that writes a reply or classifies a ticket uses an LLM, but nothing in it decides what happens next, so it isn’t an agent.

An agent has three parts:

• A model that reasons and decides.
• Tools it can call to fetch information or take actions.
• Instructions that define the job and the rules.

The loop ties them together. The model looks at the goal and everything that has happened so far, and picks an action. Your code runs it and appends the result. The model looks again. This repeats until the model says it’s finished or something stops it.

#Workflows and agents

Systems that use models and tools come in two kinds:

• In a workflow, your code decides the path. The model is called at fixed points: classify this, then draft that, then check it.
• In an agent, the model decides the path: which tool to call, in what order, and when to stop.

Real systems sit on a spectrum between the two:

Each row down gains flexibility and loses predictability. Pick the lowest row that does the job.

#When to build one

Agents are expensive. Every step is a model call with its own cost and latency, and errors compound: a forty-step run has forty chances to go wrong, and an early mistake shapes every step after it.

An agent is worth that cost when the task involves one of these:

1. Judgement with many exceptions, like deciding whether to approve a refund.
2. Rules too tangled to maintain, like a vendor security review with a sprawling checklist.
3. Unstructured input: documents, free text, conversation.

Fraud detection shows the difference. A rules engine flags transactions that break preset rules. An agent can weigh the context of a transaction and catch suspicious patterns that no rule describes.

Start with the simplest version that works. Often that’s a single model call with good retrieval. If that isn’t enough, try a fixed workflow. Move to an agent when the steps can’t be written down in advance.

#The loop in code

The loop is short enough to write yourself, and writing it once makes every framework easier to understand. Here it is in TypeScript, with the model call left provider-neutral:

type ToolCall = { id: string; name: string; input: unknown };
type Reply = { text: string; toolCalls: ToolCall[] };

type Toolbox = {
  definitions: { name: string; description: string; input_schema: object }[]; // what the model sees
  run(name: string, input: unknown): Promise<string>; // what actually executes a call
};

async function runAgent(goal: string, tools: Toolbox, maxSteps = 30): Promise<string> {
  const messages: Message[] = [{ role: "user", content: goal }];

  for (let step = 0; step < maxSteps; step++) {
    const reply: Reply = await callModel({ system: INSTRUCTIONS, messages, tools: tools.definitions });
    messages.push({ role: "assistant", content: reply });

    // No tool calls means the model is finished, and its text is the answer.
    if (reply.toolCalls.length === 0) return reply.text;

    for (const call of reply.toolCalls) {
      const result = await tools
        .run(call.name, call.input)
        .catch((err: Error) => "Error: " + err.message); // errors go back to the model
      messages.push({ role: "tool", toolCallId: call.id, content: result });
    }
  }

  throw new Error("Step limit reached before the task was done");
}

Production agents add a lot around this, but four things matter from the first version:

• A stop condition. Here the run ends when the model replies without calling a tool. Other common choices are a dedicated “final answer” tool, a structured output, or an error.
• Budgets. Cap steps, tokens, time and cost per run, and end the run with a clear reason when a limit is hit.
• Errors as results. When a tool fails, return the error to the model as that tool’s result. A specific message, such as “No order 4417 for this customer; their orders are 4415 and 4419”, gives the model what it needs to correct itself on the next step.
• A way out. The agent should be able to say it can’t finish or needs a human, and the loop should treat that as a valid outcome.

weft, the open-source cloud coding agent I built, runs this same loop: a turn ends when the model replies in plain text, which also lets the answer stream on every provider, and a token budget caps every turn.

#Five workflow patterns

Five common workflow patterns keep your code in charge of the path. Check whether one fits before building a full agent.

• Prompt chaining. Split the task into fixed steps, each one a model call working on the previous output, with code checks in between. Each call has an easier job, so accuracy goes up, at the cost of some latency.
• Routing. One call classifies the input, and each class goes to its own prompt, tools or model. Simple questions can go to a small, fast model and hard ones to a larger one.
• Parallelization. Run calls at the same time and combine the results in code. Either split the work into independent parts (sectioning) or run the same task several times and vote (voting).
• Orchestrator–workers. A model breaks the task into subtasks, hands each one to a worker call, then combines the results. The subtasks are decided at run time, after the orchestrator has seen the input.
• Evaluator–optimizer. One call produces a draft, another critiques it against clear criteria, and the first revises. It needs a real standard to check against.

These combine freely: a router can send one branch to a chain and another to an agent.

#Tools

The model can only act through its tools, and it knows them only through their names, descriptions and input schemas. That layer is the agent-computer interface. Tools come in three kinds:

#Design for the model

A model has a limited context window and pays for every token it reads, so a tool that mirrors one API endpoint is often the wrong shape. Better tools:

• Fit a task. A tool that searches contacts serves a model better than one that lists all of them. A tool that books a meeting, finding a free slot itself, beats three separate calls to list users, list events and create an event.
• Return readable fields. Models reason better about names and file types than about UUIDs and MIME types.
• Offer a level of detail. A parameter that switches between concise and detailed output lets the agent skim first and read more only when it needs to.
• Bound every response with pagination, filters and truncation. Claude Code, for example, caps a tool response at 25,000 tokens by default.
• Share a prefix with related tools, such as asana_search and jira_search, so an agent with dozens of tools can tell them apart.

#Make mistakes hard

Manufacturing calls this poka-yoke: design the tool so the wrong use is hard. While building an agent for the SWE-bench benchmark, Anthropic saw the model break relative file paths after changing directory. Requiring absolute paths fixed it completely.

The same idea applies to every parameter: a name like user_id over user, enums over free text, and edits that match exact text over line numbers that shift as the file changes.

#Descriptions are prompts

Write a tool description the way you’d brief a new colleague: what it does, when to use it, what it returns, and how much. Compare:

// Before: the model has to guess.
{
  name: "get",
  description: "Gets data",
  input_schema: { type: "object", properties: { q: { type: "string" } } },
}

// After: what it does, when to use it, and what comes back.
{
  name: "orders_search",
  description:
    "Search one customer's orders by status or date. Use it before answering any " +
    "question about a delivery or a refund. Returns up to 20 orders, newest first, " +
    "each with order_number, status, total and placed_at. Pass cursor for the next page.",
  input_schema: {
    type: "object",
    properties: {
      customer_email: { type: "string", description: "Exact email on the account" },
      status: { type: "string", enum: ["placed", "shipped", "delivered", "refunded"] },
      placed_after: { type: "string", description: "ISO date, e.g. 2026-10-01" },
      cursor: { type: "string", description: "From the previous page, if there was one" },
    },
    required: ["customer_email"],
  },
}

How many tools an agent has matters less than how much they overlap. OpenAI has seen agents use more than fifteen tools well when each does something clearly different, and others get confused by fewer than ten when their jobs overlap. If the model keeps picking the wrong tool, make the tools more distinct before splitting the work across agents.

Test tools the way you test code: run the agent on realistic tasks and look at where it misuses them. You don’t have to fix the descriptions by hand: give a model the transcripts of those failed runs and ask it to rewrite the descriptions so the same mistakes don’t happen again.

#MCP and A2A

Writing every integration separately for every agent doesn’t scale. The Model Context Protocol (MCP), released by Anthropic as an open standard in late 2024, gives integrations one shape. An MCP server wraps a system like GitHub or a database and exposes, over JSON-RPC:

• Tools the model can call.
• Resources: data an application can load into context.
• Prompts: reusable templates.

Any MCP client, from a chat app to your own loop, can then use any server without custom code.

MCP standardizes how tools connect. It says nothing about whether a tool is well designed. And since a server’s descriptions and results go straight into the model’s context, an untrusted server can inject instructions. Vet third-party servers the way you’d vet third-party code.

A2A (Agent2Agent), introduced by Google in 2025, covers a different need: agents from different teams or vendors finding each other and passing work between them. MCP connects agents to tools; A2A connects agents to each other.

#Context engineering

Everything the model sees on a step is its context: instructions, tool definitions, the conversation so far, every tool result. Context engineering is choosing the smallest set of useful tokens for each step.

#The window is a budget

A large context window doesn’t mean you should fill it. The more tokens a model holds, the worse it gets at finding and using any one of them, an effect called context rot. Attention compares every token with every other token, so each token you add spreads the model’s attention a little thinner. Treat the window as a budget: every token should earn its place.

Agents use up that budget quickly, because their context grows on its own. Every tool call adds its result to the history, and after a few dozen steps most of the window can be old output the model no longer needs: files it has already read, logs from a test run it has already fixed.

#The system prompt

A useful way to judge a system prompt is its altitude: how close to the ground the instructions are. Take a support agent that handles refunds:

• Too low, the prompt is a pile of if-then rules: “If the customer says refund and the order is under $50, issue it. If they say return and it’s been over 30 days, refuse…” It handles the cases you listed and breaks on the first one you didn’t.
• Too high, the prompt is vague: “Be helpful and resolve customer issues.” It assumes the model knows your policies, and it doesn’t.
• At the right altitude, the prompt gives the goal, the few rules that really are fixed, and the reason behind them: “Issue refunds under $50 yourself. Send anything larger to a human, because it needs a manager’s approval. If you don’t have the order number, ask for it first.” That’s specific enough to steer the model, and gives it enough of the reasoning to handle cases nobody wrote down.

Three habits help:

• Start from documents you already have: support scripts, runbooks, policies.
• Break them into steps that each map to an action or output: ask for the order number, call the lookup, confirm the amount.
• Spell out the edge cases: missing information, unexpected questions.

For examples, a few varied, typical cases teach more than a long list of edge cases.

#Load context just in time

Let the agent keep references (file paths, URLs, saved queries) and load the content with tools when it needs it. Claude Code works like this: it loads a project’s CLAUDE.md file up front, then searches the repository and opens only the files it needs.

#Long tasks

When a task outgrows one window, three techniques help:

• Compaction. Near the limit, summarize the run so far (decisions made, open problems, remaining work) and continue in a fresh window that starts from the summary. The lightest form is clearing old tool results, which are rarely needed word for word.
• Structured notes. The agent keeps a notes file (or a database record) outside the context window, writes its progress there as it works, and reads it back when it needs it. Old messages can be summarized or dropped, but the notes stay exact. Anthropic describes Claude playing Pokémon over thousands of game steps this way: it wrote down things like how many steps it had spent training on a route and how many levels a Pokémon still needed to reach its target, mapped the areas it had explored, and noted which strategies worked in battle. After its context was reset, it read those notes and carried on from where it stopped. A coding agent can do the same with a progress file or to-do list in the repository.
• Sub-agents. A sub-agent explores in its own clean window, possibly using tens of thousands of tokens, and returns a summary of often 1,000 to 2,000 tokens. The main agent only reads the summary.

#Memory

Memory is context that lasts beyond a single step or session. The CoALA paper (Sumers et al., 2023) borrows four types from cognitive science:

Working memory is managed with the techniques above. Long-term memory is something the agent reads and writes through tools. The design questions are what to save, when to write it, how to find it again and when it expires.

Memory the agent writes on purpose, as files or database rows, is easy to inspect and correct. Memory can also cause harm: an outdated fact can outrank a current one, a wrong conclusion saved once keeps resurfacing, and one user’s data must never reach another user’s context. Give every stored memory an owner, a source, and a way to be corrected or deleted.

#Planning and reasoning

How an agent chooses its next step shapes everything else. Four approaches are widely used:

• ReAct (Yao et al., 2022) alternates reasoning and acting: think, call a tool, read the result, think again. The loop above has this shape.
• Plan, then execute. The model writes a plan first and revises it as results come in. A plan is quick to review, which makes it a good point for a human check.
• Reflection. In Reflexion (Shinn et al., 2023), an agent writes a short critique after a failed attempt and keeps it for the next one. Within a run, the same idea means asking the model what went wrong before it retries.
• To-do lists. The agent keeps an explicit checklist in context or in a file and ticks items off, which keeps long tasks on track and shows the user progress.

Reasoning models do some of this planning and reflection before they answer. Budgets, checkpoints and a record of each run still belong in your code.

#Multi-agent systems

OpenAI recommends getting the most out of one agent first. Split when its instructions fill with conditionals that are hard to maintain, or when it keeps choosing the wrong tool after the tools have been made distinct. Two shapes are common:

• Manager. One agent stays in charge and talks to the user. It calls specialist agents the way it calls any tool and combines what they return.
• Handoff. Agents are peers. A triage agent works out what the user needs and passes the whole conversation to a specialist, which takes over. This suits customer service, where the specialist should own the conversation.

#For and against

The strongest public case for multi-agent is Anthropic’s research system. A lead agent plans the research and starts sub-agents that search in parallel, each in its own context window. On Anthropic’s internal research eval, it beat a single agent by 90.2%. It also used far more tokens: Anthropic measured agents at about 4× the tokens of a chat, and multi-agent systems at about 15×. In their analysis of the BrowseComp benchmark, token usage alone explained 80% of the variance in performance, so a large part of the gain is bought with tokens.

The strongest case against is from Cognition, the team behind the Devin coding agent. Every action an agent takes carries implicit decisions, and sub-agents that can’t see each other’s work make conflicting ones. In their example, “build a Flappy Bird clone” is split into a background and a bird; one sub-agent draws a Super Mario background, and the other builds a bird that doesn’t match it. They recommend a single-threaded agent with a shared, compressed history.

Anthropic’s post draws the same boundary. Multi-agent suits broad, parallel, read-heavy work like research. It works poorly when every agent needs the same context or the parts depend on each other, as in most coding.

#Guardrails and safety

An agent reads content you didn’t write and acts on your behalf. Guardrails work as layers: each check catches some failures, and several independent checks together are much harder to get past.

Tool safeguards matter most for agents. Rate every tool low, medium or high risk by what it can break: read or write, reversible or not, which permissions it needs, whether money moves. Low-risk tools just run. High-risk ones, like cancelling orders, large refunds, payments, deletions or outgoing messages, wait for extra checks or a human until the agent has earned trust. Escalate to a human, too, when the agent keeps failing at the same step.

#Prompt injection

A model can’t reliably tell instructions apart from data. A web page, email, code comment or tool result can contain text written to look like an instruction, and a model that reads it may follow it. No classifier catches every such attempt.

Simon Willison’s lethal trifecta describes when this turns dangerous. An agent is exposed when it has all three of:

1. Access to private data.
2. Exposure to untrusted content.
3. A way to send data out, which can be as small as loading an image from a URL.

With all three, an attacker’s text can make the agent read your data and send it to them. A filter that blocks 95% of attempts doesn’t solve this; as Willison points out, in security 95% is a failing grade. The reliable fix is structural: never give one agent all three. An agent that reads untrusted content shouldn’t be able to send data out, and an agent that browses shouldn’t hold private data.

The rest is standard security, applied strictly:

• Least privilege. Only the tools and data the task needs, with short-lived credentials scoped to it.
• Secrets stay out of context. The tool code can hold an API key; the model never needs to see it.
• Code runs in a sandbox. Anything that executes commands belongs in an isolated, disposable environment with tight network rules. I wrote separately about how production sandboxes for coding agents are built.

#Evals

Evals tell you whether a prompt change helped, whether a cheaper model is good enough, and whether a fix broke something else.

Start small and real. Twenty to fifty tasks taken from real failures is enough to start. Each task is an input plus a clear definition of success. Run each one several times, because an agent can behave differently on every run.

Grade the outcome. Checking that the agent called specific tools in a fixed order penalizes valid approaches nobody anticipated. Check the result: was the refund issued for the right amount, do the tests pass?

Match the grader to the check:

Measure consistency. pass@k is the chance that at least one of k attempts succeeds, which suits tools where a person picks the best result. pass^k, introduced by the τ-bench benchmark, is the chance that all k attempts succeed. For an agent that serves customers, pass^k is closer to what users experience, and it drops quickly as k grows.

Keep two suites. Capability evals should be hard, with low pass rates that leave room to improve. Regression evals should almost always pass, and flag anything that gets worse.

Read the transcripts. A failure can mean the agent made a mistake, or that the grader rejected a correct answer. In one case Anthropic describes, a model’s score on a benchmark went from 42% to 95% once rigid grading and ambiguous tasks were fixed. Treat a score as unverified until someone has read the runs behind it.

#Running in production

Trace everything. Record each model call with its full context, each tool call with its inputs, outputs and timing, and the tokens and cost of every step. When an agent does something strange, the trace shows what it was given and what it did.

Make runs durable. A long run can be cut off halfway by a model API timeout or a worker restarting during a deploy. Keep run state outside the process so the run can resume from where it stopped, put timeouts on every external call, and retry with backoff.

Make actions idempotent. If a step is retried, a tool that charges a card or sends an email must not do it twice. Give each action an idempotency key and check whether it already happened.

Control cost and latency:

• Choose models per step. OpenAI recommends building the prototype on the most capable model to set a baseline with your evals, then swapping in smaller models wherever the evals show they’re good enough. Routing and classification often work fine on a small model.
• Use prompt caching. Many providers charge much less for a prompt prefix they’ve seen recently. Put stable content (instructions, tool definitions) first and append to the history instead of rewriting it, so the prefix stays identical between steps.
• Run independent work in parallel. Running sub-agents side by side, each calling several tools at once, cut the research time of the multi-agent system above by up to 90% on complex queries.
• Stream progress, so users can follow a long run as it happens.

Deploy with care. A deploy can land in the middle of a long run. Anthropic shifts traffic between versions gradually, so runs already in progress finish on the code they started with. Run the regression evals before every prompt or model change, the way you’d run tests before merging code.

#Frameworks

Frameworks and SDKs (OpenAI’s Agents SDK, LangGraph, the Claude Agent SDK, Google’s ADK, Mastra and others) save real work on streaming, tool-call parsing, retries, tracing and multi-agent wiring.

Start with the model API directly; many patterns take only a few lines. Once you’ve written the loop yourself, you know what a framework does for you and can debug it when it misbehaves. If you adopt one, make sure you can inspect the exact requests it sends to the model and how it handles errors and limits.

#Closing

The model decides each step. Whether those decisions are good depends on the engineering around it: tools that are hard to misuse, a context window kept clean, permissions kept tight, and evals you can trust.

#Further reading

• OpenAI, A practical guide to building agents
• Anthropic, Building effective agents
• Anthropic, Writing effective tools for agents
• Anthropic, Effective context engineering for AI agents
• Anthropic, How we built our multi-agent research system
• Anthropic, Demystifying evals for AI agents
• Cognition, Don’t build multi-agents
• Simon Willison, The lethal trifecta for AI agents
• Lilian Weng, LLM Powered Autonomous Agents
• Yao et al., ReAct: Synergizing Reasoning and Acting in Language Models
• Shinn et al., Reflexion: Language Agents with Verbal Reinforcement Learning
• Sumers et al., Cognitive Architectures for Language Agents
• Yao et al., τ-bench: A Benchmark for Tool-Agent-User Interaction
• Model Context Protocol