A PaperBridge long-tail question
Why do long-running AI agents need a harness, not just a longer prompt?
A prompt tells the model what should happen and how to reason about the next step. A harness keeps that contract true across tool calls, context compaction, and failure recovery. A minimum harness stores the goal, current state, allowed tools, permissions, budgets, stop conditions, executable acceptance checks, and recovery points outside the model context, then reads real environment results after every action. A longer prompt may improve one decision, but it cannot by itself persist state, block unauthorized actions, prove that a tool succeeded, or resume the next session from the correct checkpoint.
1. Separate the jobs of the prompt and the harness
The prompt is part of the model input: it describes a role, objective, rules, examples, and current context. The harness is the execution system around the model: it constructs context, exposes tools, validates arguments, reads environment results, persists state, enforces permissions and budgets, and decides whether to continue, retry, escalate, or stop.
They are not competing choices. A mature harness normally includes prompt templates, but it also promotes critical boundaries into code, schemas, tests, and state machines. A rule written only in the prompt still depends on the model remembering and applying it at every step; a mechanical guard can reject the action when the model does not.
2. Persist progress outside the context window
Long tasks cross context compaction, process restarts, human handoffs, and changes in external state. Do not leave completed work, remaining work, and verified assumptions only in chat history. Store structured task state, decision records, artifact inventories, unresolved items, and the latest passing checks.
Anthropic's long-running experiment reports two representative failures: agents tried to do too much before context ran out and left partial work, while later sessions saw existing progress and declared completion too early. Initialization, progress files, incremental handoff, and a clean workspace are forms of reliable external memory for the next session.
- Keep stable goals and acceptance criteria separate from the changing current plan.
- Record actions, observed outcomes, unresolved errors, and the next smallest step.
- Make checkpoints machine-readable instead of relying on one free-form summary.
- On resume, reconstruct critical state and verify the environment before trusting the old plan.
3. Design tools and observations as an agent interface
An agent needs more than access to tools. It needs clear arguments, interpretable returns, and recoverable errors. SWE-agent's Agent-Computer Interface research shows that command and feedback design materially changes how well a model can navigate repositories, edit files, and run tests.
A successful tool response is not the same as a successful task. The harness should read the database, file, browser, or API state after an action so the next step uses observed reality rather than the model's story about what just happened. High-risk tools also need schemas, least privilege, idempotency, previews, and pre-commit checks.
4. Turn “done” into executable acceptance
A request such as “finish this project” leaves completion scope to the model. A stronger contract defines completion as tests passing, target artifacts existing, pages responding, database state matching, budgets staying within limits, and prohibited effects not occurring. Use a person or an independent evaluator only for quality that cannot be checked mechanically.
The harness should also define stop conditions: how many failures trigger a new approach, when user input is required, which actions need confirmation, and which state requires a safe exit. The agent should neither retry indefinitely because tokens remain nor stop because it produced text that resembles an answer.
5. Evaluate the model–harness configuration
Under a fixed protocol, Harness-Bench observes substantial model–harness variation and identifies execution-alignment failures where plausible reasoning becomes detached from tool feedback, workspace state, or verifiable output contracts. WildClawBench likewise reports that changing the harness alone moved one model by as much as 18 points.
Production evaluation should therefore pin the model version, harness version, tools, permissions, budgets, and recovery policy. Report completion, validator pass rate, unauthorized actions, retries, cost, latency, and human intervention rather than crediting every success to the model and every failure to the prompt.
Minimum agent-harness execution contract
- Define the goal, non-goals, executable acceptance checks, and prohibited actions.
- Persist task state, decisions, artifacts, and the next step outside model context.
- Give tools schemas, permissions, error semantics, idempotency, and outcome reads.
- Add previews, confirmation, and an independent pre-commit gate for risky actions.
- Set token, time, tool-call, cost, and retry budgets.
- Define failure recovery, human escalation, safe stopping, and cross-session handoff.
- Evaluate repeated model–harness runs with complete traces and final environment state.
Primary research and official documentation
These sources support the facts. Workflow and comparison guidance is PaperBridge's synthesis of research, official documentation, and engineering practice.