Agents in Production: Why They Loop

The loop

Theory

An agent is a model, using tools, in a loop. You send a goal and a list of tools; the model replies with an answer or a tool call; your code runs the tool and sends the result back, and the model decides again. The loop ends when the model replies with no tool call, so the model decides when the work is finished. Nothing in that loop says stop after ten tries unless you add it.

Practical example: The whole loop

Your code is the only adult in the room: a loop with no limit is a billing meter with no off switch.

def call_model(messages):  # stub: returns a tool call or a final answer
    if len(messages) < 3:
        return {"tool": "search_files", "args": {"q": "report"}}
    return {"tool": None, "text": "Done."}

def run_tool(reply):  # stub: your code runs the tool
    return "no files found"

messages = [{"role": "user", "content": "Find the report."}]
while True:  # note: nothing here says stop after ten tries
    reply = call_model(messages)
    if reply["tool"] is None:  # no tool call: the model decided it is done
        break
    messages.append({"role": "tool", "content": run_tool(reply)})
print(reply["text"])

Stuck loops

Theory

Agents often loop because the last result didn't change the model's mind: it sees the same history plus one more failure, so the same move looks reasonable. Other loops are sneakier: ping-pong, where step A undoes step B, or a query rephrased each time so no two calls match exactly. Nothing throws an exception; agent frameworks even ship a dedicated error for running out of steps before hitting a stop condition.

Practical example: Same query, same nothing

The agent searches for a file and gets nothing, so it decides to search again with the same query. Your logs just show a very busy agent.

The token bill

Theory

The model has no memory between calls, so every turn you resend the whole conversation: the prompt, every tool call and every tool result so far. Context only grows, and each turn sends more than the one before. Big tool results are the real culprit: one verbose log dump can add thousands of tokens, permanently.

Practical example: Twenty turns of a failed task

If each turn adds a thousand tokens, turn ten sends about ten thousand and twenty turns add up to about two hundred ten thousand input tokens (illustrative, ignoring prompt caching, which makes repeated history cheaper). First fix: keep tool outputs small.

added_per_turn = 1000  # illustrative; no prompt caching
turns = 20

sent = [added_per_turn * t for t in range(1, turns + 1)]  # each turn resends all
print(sent[9], sent[19])  # turn 10: 10000, turn 20: 20000
print(sum(sent))          # 210000 input tokens in total

Compounding errors

Theory

One small mistake early, and every later step builds on it. The agent doesn't know which step was wrong, and it can treat its own earlier guess as fact. Shorter chains win: fewer steps, clearer tools, and a check between steps.

Practical example: Ninety percent, ten times

If each step succeeds ninety percent of the time, ten steps in a row are all right only about thirty-five percent of the time (assuming independent steps), so roughly two in three runs contain a mistake. Verify with the world, not the model: check that the file exists.

step_success = 0.9  # each step right 90 percent of the time
steps = 10

all_right = step_success ** steps  # assumes independent steps
print(round(all_right, 2), round(1 - all_right, 2))  # 0.35 0.65

Tool errors

Theory

When a tool fails, the error goes back to the model as the next message, so your error text decides what happens next. A stack trace or just the word failed leaves the model guessing, and a common guess is the same call again. Write errors for a stranger with no context, and mark them as errors, since tool-calling APIs can flag a result as an error so it isn't mistaken for data.

Practical example: An error the model can act on

Instead of failed, return: rate limit hit, retry after sixty seconds. A denied action is feedback too: say no, and why, so the model picks another path.

Hard limits

Theory

Rule one: a maximum number of steps, always, in production; agent-building guides list it as a standard stopping condition. Many agent SDKs have this setting, but defaults differ by framework and version and some have no limit unless you set one. Rule two: a token or dollar budget per run; spend is checked after each response, so a run can overshoot by about one call. Rule three: timeouts on every tool call and on the whole run.

Practical example: A step cap that fails loudly

Count the turns; at the cap, stop, return what you have plus a clear status like out of turns, and save progress so the run can resume.

MAX_STEPS = 10

def call_model(messages):  # stub: an agent that never decides it is done
    return {"tool": "search_files", "args": {"q": "report"}}

messages, status = [], "done"
for step in range(MAX_STEPS):
    reply = call_model(messages)
    if reply["tool"] is None:
        break
    messages.append({"role": "tool", "content": "no files found"})
else:
    status = "out_of_turns"  # fail loudly, keep the progress
print(status, len(messages))

Smart guards

Theory

Limits stop the loop only after you've paid for it; smarter guards catch it earlier. Loop detection fingerprints each tool call by name plus arguments and reacts when the same fingerprint repeats. Idempotent tools, where calling twice has the same effect as calling once, make retries safe: give each action a unique key, and the server returns the first result on a repeat.

Practical example: Nudge, then escalate

On a repeat, tell the model it made this exact call before. If it still repeats, stop the run and hand it to a human with the trace attached.

import json
from collections import Counter
seen = Counter()
def check_repeat(tool, args, limit=3):
    fingerprint = tool + json.dumps(args, sort_keys=True)  # name plus arguments
    seen[fingerprint] += 1
    if seen[fingerprint] >= limit:
        return "escalate"  # stop the run, hand it to a human with the trace
    if seen[fingerprint] > 1:
        return "You made this exact call before. Try something different, or give up."
    return None
for _ in range(3):
    print(check_repeat("search_files", {"q": "report"}))

Humans and traces

Theory

Some actions need a human: sending money, deleting data, emailing your whole customer list; agents can pause for human feedback at checkpoints. Add a kill switch, a flag the loop checks every turn that stops every run without a deploy. Trace every turn (input, tool call, result, tokens, time) and alert on too many steps, unusual cost per run, or the same tool called again and again.

Practical example: A simple permission rule

Read-only tools run free. Write tools ask. Irreversible tools ask twice. Save the bad traces: every looping run is a free test case for your evals.