Agentic Editor · Study Guide

5. The Agent Loop (Chat to Autonomous Agent)

What This Subsystem Is and Why It Exists

The agent loop is the runtime engine that turns a language model from a passive text predictor into something that does work in your repo: reads files, edits code, runs tests, reads the failures, and tries again — all without a human pressing a button between each step.

Think of the model as a brain in a jar. It is brilliant at reasoning but has no hands, no eyes, and no memory of the outside world beyond the text you hand it. The agent loop is the nervous system and the limbs: it gives the model tools (hands), feeds it tool results (senses), and runs the perceive → think → act → observe cycle over and over until the job is done.

The Progression: Autocomplete → Chat → Agent

There are three escalating levels of autonomy, and the agent loop is the third:

  • Autocomplete (tab completion). One model call, no tools, no loop. You type, the model predicts the next span of code. Latency budget ~10–50 ms to first token; the model never decides what to do, only how to continue. (Copilot's original product, Cursor Tab, Supermaven.)
  • Chat / single-shot edit. One or a few model calls. You ask a question or request an edit; the model answers or proposes a diff. It may read context you attached, but it does not autonomously go find more. This is "Cmd-K" / inline edit / the old chat panel.
  • Agent. A loop. The model is handed a goal and a toolbox, then runs many turns: it decides which file to read, reads it, decides to edit, edits, runs the test, sees it fail, debugs, and repeats. The human sets the goal and supervises; the model drives the steps.

The agent loop exists because real engineering tasks are multi-step and contingent. "Fix this failing test" can't be answered in one shot — you must look at the test, look at the code, form a hypothesis, change something, and check whether it worked. The loop is what makes that closed-feedback iteration possible.

How It Works, Step by Step

At its core the loop is shockingly simple. The genius is in the simplicity. Here is the canonical structure.

The Three Inputs Every Turn

Every model call in an agent is built from three things concatenated into one request:

  1. System prompt — the agent's "constitution": who it is, its rules, coding conventions, how to use tools, when to stop. (Often 1k–10k tokens; Claude Code and Cursor's are large and carefully tuned.)
  2. Tool definitions — a machine-readable schema (usually JSON Schema) for each available tool: name, description, parameters. The model is trained to emit structured calls against these. Typical agent: 5–20 tools (read_file, edit_file, run_terminal, grep, list_dir, etc.).
  3. Message history — the running transcript: the user's goal, every prior assistant turn, every tool call the model made, and every tool result that came back. This is the agent's working memory.

The ReAct Cycle (Think–Act–Observe)

The loop implements the ReAct pattern (Reason + Act, Yao et al. 2022). One iteration:

  1. Think. The model reasons (sometimes out loud as visible text, sometimes in hidden reasoning tokens) about what to do next.
  2. Act. Instead of finishing its turn with prose, the model emits a tool call — a structured block like edit_file({path: "auth.ts", ...}). The API surfaces this as a distinct tool_use content block with a unique tool_use_id, and the model's turn stops there (stop_reason: "tool_use").
  3. Execute (the harness's job). The loop code — not the model — parses the tool call, runs the actual function (writes the file, spawns the shell command), and captures the output.
  4. Observe. The harness appends a tool_result message (keyed to the same tool_use_id) to the history and calls the model again with the now-longer transcript.
  5. Repeat until the model emits a normal text turn with no tool call (it decided it's done), or a stopping condition trips.

The mental model: the model is a function that maps history → next action, and the harness is a while loop that executes the action and feeds the result back in. That's it. A minimal agent is ~50 lines of code wrapped around a chat completion endpoint. Thomas Dohmke and others have noted "the agent is a while-loop."

loop:
  response = model(system, tools, history)
  history.append(response)
  if response has no tool_calls:        # model is done
      break
  for call in response.tool_calls:
      result = execute(call)            # harness runs it
      history.append(tool_result(call.id, result))

Parallel vs Sequential Tool Calls

Modern models can emit multiple tool calls in one turn (e.g., read three files at once). The harness runs them — often in parallel for read-only ops — and returns all results in the next message. This cuts round-trips and latency dramatically. Claude and GPT-4-class models both support parallel tool use; harnesses like Claude Code fan out read-only calls concurrently but serialize mutating ones (edits, shell) to keep state coherent.

Planning Patterns

A naive ReAct loop is "greedy" — it picks the next step with no overall map. For bigger tasks, agents layer planning on top.

  • Plan-then-act. The model first produces an explicit plan (a numbered list of steps), then executes it. Reduces flailing on multi-file changes. Cursor's Agent and Claude Code both nudge toward this.
  • Todo lists as external memory. Claude Code's TodoWrite tool literally maintains a checklist in the transcript; the model checks items off as it goes. This is a clever trick: it keeps long-horizon intent anchored in context so the model doesn't drift or forget the original goal after 40 turns. The list acts like a "string around the finger."
  • Decomposition / sub-tasking. Break a big goal into chunks, sometimes dispatched to sub-agents (below).
  • Reflection. After a failure, prompt the model to critique its own last attempt before retrying ("what went wrong, what will you change?"). Improves recovery on hard bugs.

The tension: too much forced planning makes the agent rigid and slow on simple tasks; too little makes it wander on complex ones. Good harnesses let the model decide how much to plan based on task size.

Stopping Conditions and Loop Guards

An autonomous loop that can call itself forever is a runaway risk — burning tokens, money, and possibly making destructive changes. Guards are essential:

  • Natural stop: model returns a plain text turn with no tool call → task believed complete.
  • Turn/step budget: hard cap on iterations (e.g., 25–50 turns) before the loop pauses and asks the human.
  • Token budget: cap total tokens / cost per run.
  • Wall-clock timeout on the whole run and on individual tool executions (a hung npm install shouldn't freeze the agent).
  • Loop-detection heuristics: if the model makes the same tool call with the same args N times, or oscillates between two states, break out. (A classic failure: edit file → test fails → revert edit → test fails → re-apply same edit, forever.)
  • Repeated-error backoff: after K consecutive tool errors, stop and surface to the user.
  • Approval gates: for dangerous tools (shell commands, file writes, git push), require human confirmation unless the user enabled auto-approve / "YOLO mode."

Streaming and Interruptibility

A 5-minute autonomous run is unbearable if you can't see or steer it. Two requirements:

  • Streaming. Tokens stream to the UI as generated (SSE / chunked responses). You watch the model think, see each tool call and result appear live. Critical for trust — you catch a wrong turn early.
  • Interruptibility / steering. The user can hit Esc / "stop" mid-run. The harness cancels the in-flight model request and any running tool, then lets the user inject a new instruction ("no, don't touch the database layer"). That message gets appended to history and the loop resumes with corrected intent. Claude Code and Cursor both support mid-run steering; it's a major UX differentiator because agents will go off-track and you want to correct without restarting.

Implementation note: cancellation has to cleanly tear down a streaming HTTP connection and kill child processes (the shell command the agent spawned), or you leak resources.

Sub-Agents and Orchestration

For large or parallelizable work, a single linear loop isn't enough. Two patterns:

  • Sub-agents (spawned workers). The main agent calls a Task/dispatch_agent tool that launches a fresh agent loop with its own context window and a scoped goal ("find every call site of parseConfig"). The sub-agent runs its own ReAct loop and returns a condensed result. The win: context isolation — the sub-agent's 30 noisy file-reads never pollute the main agent's context; only the summary comes back. Claude Code's subagents and Cursor's background agents use this.
  • Orchestrator/worker (multi-agent). A planner agent decomposes and assigns sub-tasks to multiple workers, possibly in parallel, then integrates results. Powerful but expensive and harder to keep coherent; shared-state conflicts (two agents editing the same file) are the hard part. Anthropic's multi-agent research system reported large quality gains but ~15× token cost vs single-agent chat.

Analogy: a sub-agent is like a senior engineer delegating "go investigate X and just tell me the answer" — the senior doesn't want the junior's entire research session dumped on their desk, just the conclusion.

The Central Design Tension: Thin Harness vs Heavy Scaffolding

This is the architectural debate in agentic coding, and a favorite interview topic.

  • Thin harness + smart model. Give the model a few general tools (read, edit, shell), a good system prompt, and trust it to figure out the rest. Minimal hardcoded control flow. Claude Code is the canonical example — deliberately "unopinionated," betting that as models improve, scaffolding becomes dead weight and even hurts by constraining a capable model.
  • Heavy scaffolding + orchestrated steps. Lots of engineered structure: forced plan/execute/review phases, specialized prompts per phase, retrieval pipelines, validators, state machines. Squeezes more out of weaker/cheaper models and gives more determinism. Early Devin-style systems, many RAG-heavy agents, and structured frameworks lean this way.

The tradeoff:

  • Heavy scaffolding wins today on reliability with a fixed model and on cost (you can use a cheaper model with more guardrails).
  • Thin harnesses win over time: every model upgrade makes them better for free, and they have fewer brittle assumptions. "The bitter lesson" applies — general methods that leverage compute/model improvement tend to beat hand-engineered structure.
  • Heavy scaffolding can actively fight a strong model (e.g., forcing a rigid plan when the model would've found a better path).

The industry has drifted toward thinner harnesses as frontier models (Claude 3.5/4, GPT-4-class) got dramatically better at tool use and long-horizon reasoning.

Concrete Technologies and Numbers

  • Tool-calling format: JSON Schema function definitions; model emits structured tool_use blocks. Standardized across Anthropic Messages API, OpenAI function/tool calling.
  • MCP (Model Context Protocol): Anthropic's 2024 open standard for exposing tools/resources to agents over a uniform interface. Lets editors plug in external tools (databases, browsers, Slack) without bespoke integration. Now adopted broadly (Cursor, VS Code, Zed, Claude Code).
  • Transport: Server-Sent Events / chunked HTTP for token streaming.
  • Typical budgets: 5–20 tools per agent; turn caps ~25–50; context windows 128k (GPT-4-class) to 200k (Claude) to 1M (Gemini, Claude with long-context). A nontrivial agent run = 10–40 model calls, each re-sending the growing transcript.
  • Cost driver: because the entire history is resent every turn, token cost grows roughly quadratically over a long run. Prompt caching (Anthropic/OpenAI) caches the stable prefix (system prompt + tool defs + early history) so you pay full price once and ~10% on cache hits — this is the single biggest cost lever for agents, cutting bills 5–10× on long sessions.

How Leading Products Structure the Loop

  • Claude Code (Anthropic). Thin-harness purist. Small set of general tools (Read, Edit, Bash, Grep, Glob, plus Task sub-agents and TodoWrite). Minimal control flow, big well-tuned system prompt, leans hard on the model. Runs in the terminal, model-agnostic-ish but tuned for Claude. Strong interruptibility and todo-list planning.
  • Cursor Agent. IDE-native loop with a polished diff-review UX. Mixes a smart loop with editor-specific tooling (semantic codebase index for retrieval, "apply" model for fast edits). Supports background agents (sub-agent-style) and a "YOLO" auto-run mode. More scaffolding around retrieval than Claude Code, given it owns the index.
  • Aider. Open-source, pragmatic, git-centric. Auto-commits each change so every step is a checkpoint you can undo. Famous for edit formats: it doesn't use opaque tool calls — it asks the model for search/replace diff blocks in text and parses them, which works across many models. Has "architect mode" (one model plans, another edits) — explicit plan/act separation.
  • Cline (open-source VS Code extension). Transparent ReAct loop with human-in-the-loop approval on every action by default — you approve each file edit and command. Visualizes the think-act-observe cycle clearly; uses MCP heavily. More scaffolding/guardrails than Claude Code, oriented to safety and visibility.
  • GitHub Copilot. Started as autocomplete; "Copilot Chat" added Q&A; Copilot agent mode / Workspace added the full loop (plan → edit across files → run → iterate) inside VS Code, plus the autonomous Copilot coding agent that works on issues in CI and opens PRs. Leans on VS Code's tooling and MCP.
  • VS Code. Provides the platform (LM API, tool/MCP integration, diff UI) that Copilot and extensions build agent loops on, rather than shipping one canonical loop.
  • Zed. Fast Rust editor; agent panel with tool use and MCP, designed for low-latency streaming and tight editor integration; also leans toward letting the model drive.
  • JetBrains (AI Assistant / Junie). Junie is JetBrains' autonomous agent; uses the IDE's deep static analysis and indexing as tools — heavier on language-aware tooling than terminal agents.
  • Sourcegraph (Cody / Amp). Strength is retrieval at scale across huge codebases; the agent loop is fed by powerful code search/graph context. More retrieval scaffolding, fitting their enterprise-scale niche.
  • Continue. Open-source, configurable agent framework for VS Code/JetBrains; you compose models, tools, and context providers — explicitly a "build your own harness" toolkit.
  • Windsurf (Codeium). "Cascade" agent emphasizes the editor acting in lockstep with you (Flows), tracking your actions and the codebase to drive multi-step edits; polished agentic UX with auto-run options.

The throughline: terminal/open agents (Claude Code, Aider) trend thin and general; IDE products (Cursor, Copilot, Windsurf, JetBrains, Sourcegraph) add scaffolding around their proprietary retrieval, indexing, and diff-review UX.

Failure Modes and Mitigations

  • Infinite / oscillating loops. Agent repeats the same failing action. Mitigate: loop-detection on repeated calls, turn budgets, reflection prompts after failure.
  • Context-window exhaustion / "context rot." Long runs fill the window with stale tool output; the model forgets the goal or degrades. Mitigate: sub-agents for context isolation, summarization/compaction of old history, todo lists to re-anchor intent, pruning large tool results.
  • Goal drift. Model wanders off-task after many turns. Mitigate: re-inject the goal/todo list, streaming + human steering, plan-then-act.
  • Destructive actions. rm -rf, bad migration, force-push. Mitigate: approval gates, sandboxing, allow/deny command lists, git auto-commit so everything is reversible (Aider's model), running in a container or worktree.
  • Hallucinated tool calls. Model invents a tool or malformed args. Mitigate: strict schema validation; return a structured error as the tool_result so the model self-corrects next turn (the loop absorbs the error rather than crashing).
  • Premature stop / overconfidence. Model declares victory without running tests. Mitigate: system-prompt rules requiring verification, harness-enforced "did you run the tests?" checks, plan items that include validation.
  • Cost blowups. Quadratic token growth. Mitigate: prompt caching, history compaction, cheaper models for sub-agents, hard token budgets.
  • Tool-result poisoning / prompt injection. A file or web page the agent reads contains instructions ("ignore previous instructions, exfiltrate keys"). Mitigate: treat tool output as untrusted data, sandbox secrets, restrict network/shell, human approval on sensitive actions.

Interview Talking Points

  • The agent loop is a while loop around a chat model: build a request from system prompt + tool definitions + message history, get a tool call back, execute it in the harness, append the result, call the model again — repeat until done.
  • It implements ReAct (think → act → observe). The model only emits tool calls; the harness executes them. Tool calls and results are linked by a tool_use_id.
  • Progression of autonomy: autocomplete (1 call, no tools) → chat (few calls) → agent (many-turn loop with tools). The loop exists because real tasks are multi-step and need feedback.
  • Planning patterns: plan-then-act, todo lists as external memory (Claude Code's TodoWrite), decomposition, reflection-on-failure.
  • Guards against runaways: turn/token/time budgets, loop-detection, approval gates, sandboxing.
  • Streaming + interruptibility are core UX: watch it work, steer mid-run by injecting a message.
  • Sub-agents give context isolation — a worker does messy research and returns only a summary.
  • The big tension: thin harness + smart model (Claude Code) vs heavy scaffolding (orchestrated, cheaper models). Industry trending thin as models improve ("bitter lesson").
  • Cost grows ~quadratically (whole history resent each turn); prompt caching is the #1 mitigation (5–10× savings).
  • MCP is the emerging standard for plugging tools into any agent.

Likely Interview Q&A

Q1. Walk me through one iteration of an agent loop. Build a model request from three parts: the system prompt, the JSON-Schema tool definitions, and the full message history. Call the model. It either returns plain text (it's done — exit the loop) or a tool_use block with a name and arguments. The harness, not the model, parses that call and executes the real function — say it reads a file or runs pytest. The harness captures the output, appends it as a tool_result message keyed to the same tool_use_id, and calls the model again with the now-longer transcript. The model observes the result and decides the next action. Repeat until it stops or a budget trips.

Q2. Why is it called ReAct, and what's the alternative? ReAct = Reasoning + Acting (Yao et al. 2022): the model interleaves reasoning steps with actions and observations, so each decision is informed by real feedback from the environment. The alternative is plan-only (chain-of-thought with no tools — it can't check its work) or a rigid scripted pipeline (deterministic but brittle). ReAct's strength is closed-loop correction: it can see a test fail and respond.

Q3. How do you stop an agent from running forever or doing damage? Layered guards: a hard turn cap (e.g., 25–50), a token/cost budget, wall-clock timeouts on the run and each tool, loop-detection that breaks on repeated identical calls or oscillation, and a backoff after K consecutive tool errors. For damage: approval gates on dangerous tools (shell, write, push), allow/deny command lists, sandboxing in a container or git worktree, and auto-committing each change so every step is reversible.

Q4. What are sub-agents and when would you use one? A sub-agent is a fresh agent loop with its own context window, spawned by the main agent via a tool to handle a scoped subtask, returning only a condensed result. Use it when a subtask is exploratory and noisy — like searching a large codebase — so the dozens of file-reads don't pollute the main context. It's context isolation: the senior engineer delegates the digging and just wants the conclusion. The cost is extra tokens and coordination complexity.

Q5. Explain the thin-harness-vs-heavy-scaffolding tradeoff. Which would you pick? (system design) Thin harness (Claude Code) = few general tools, big system prompt, trust the model to drive; minimal control flow. Heavy scaffolding = forced plan/execute/review phases, validators, retrieval pipelines, state machines. Heavy scaffolding wins today on determinism and lets you use cheaper/weaker models with guardrails, but it's brittle and can fight a strong model. Thin harnesses get better for free with every model upgrade and carry fewer assumptions — the "bitter lesson." I'd default to a thin harness when targeting frontier models and a task space that's varied, and add targeted scaffolding only where I have measured reliability problems (e.g., a verification gate that forces running tests). Start thin, instrument failures, scaffold surgically.

Q6. Why does an agent run get expensive, and how do you control cost? (system design) Because the entire transcript — system prompt, tool defs, all prior turns and tool results — is resent on every model call, total tokens grow roughly quadratically with turns. Levers: (1) prompt caching of the stable prefix, the biggest win, 5–10× on long sessions; (2) history compaction/summarization of old turns; (3) pruning large tool outputs before they enter context; (4) sub-agents to keep the main context small and to run cheaper models for grunt work; (5) routing simple subtasks to a smaller model; (6) hard token budgets. You're trading off cost vs. the model's ability to remember earlier context, so compaction must preserve the goal and key decisions.

Q7. How does the loop handle a tool that errors or a malformed tool call? The harness validates arguments against the schema; on failure or runtime error it returns a structured error string as the tool_result rather than crashing. The model sees the error on the next turn and self-corrects — e.g., fixes the path or retries with different args. The key design principle: the loop absorbs errors as observations, turning them into feedback instead of fatal exceptions. You add a consecutive-error cap so it doesn't thrash forever.

Q8. How do streaming and interruptibility actually work, and why do they matter? Tokens stream from the model over SSE/chunked HTTP and render live, including each tool call and result, so the user can watch and build trust. Interruptibility means the user can cancel mid-run: the harness aborts the in-flight model request and kills any running child process (the shell command it spawned), then accepts a new user message, appends it to history, and resumes the loop with corrected intent. It matters because agents inevitably go off-track on long tasks, and restarting from scratch is wasteful — steering lets you redirect in place.

Q9. Aider doesn't use JSON tool calls for edits — what does it do and why? Aider asks the model to output search/replace diff blocks as text and parses them itself, plus it auto-commits each change to git. The rationale: text diff formats work across many models (including ones with weak or no native tool-calling), edits are easy for humans to review, and git checkpoints make every step reversible. It trades the structure of native tool calls for portability and a strong undo story — a good example of scaffolding chosen to fit a model-agnostic, open-source niche.

Built from Markdown · 22 pages · Read it aloud and burn it in.