Six parts of an open-source agent harness, and the check for each

Instructions and skills: the rules an agent reads before it acts

A harness is everything around the model, and the shorthand is Agent = Model + Harness. LangChain's anatomy of an agent harness puts it plainly: the model contains the intelligence and the harness makes that intelligence useful. Databricks measured how much that matters: "on public benchmarks, the same model can place significantly higher or lower depending entirely on how the harness is built." In one run, Databricks paired GPT-5.5 with the OfficeQA Pro Agent Harness and scored 52.63%, against 36.10% with GPT-5.4. Databricks

Instructions are the feedforward end of the harness. They anticipate what an agent will get wrong and steer it before it starts. The decision this week is what sits in the always-loaded instruction file and what moves into a skill loaded on demand. Keep the file a map. A three-person team built a one-million-line codebase over five months with roughly 1,500 merged pull requests and no code written by hand. The file that traveled with every turn was an AGENTS.md of about 100 lines, pointing at a structured docs/ directory. Milvus The procedures lived where the task could find them.

The check is mechanical. Assert that every path named in the instruction file resolves in the repo, then run one scripted session that asks for a rule you know exists. If the agent cannot name the file that holds the rule, the guide is not in the harness yet.

Tools and permissions: what the agent may do, and who says yes

Tools are where a harness stops being a document. The decision is per tool and per argument: run it, ask first, or refuse it. Approval gates you set. Off until you set them. Per-resource permissions for people and agents. Roles, groups, and an audit trail.

Two questions settle the work. Which tools does the agent get on day one, and which may it never call? For a granted tool, which arguments change the answer? A shell tool allowed to read a log file changes meaning when it can also reach the network, so the permission belongs on the narrower thing you can state.

The check is a refusal. For each class of denied call, run it in a test session and assert the refusal. Then read the audit entry for the allowed version of the same call and confirm it names the same tool and argument. A permission rule that no test has ever tripped is a sentence in a file, and the first person to discover it is the one reviewing the incident.

The sandbox and workspace: where the work happens and what survives

One isolated sandbox per session. Each session has its own isolated machine and branch. An agent session is an isolated sandbox, an ephemeral branch of the repo with its own /workspace, and only what you commit and push survives it. The box is a place to work, and anything worth keeping leaves it as a commit.

The decision is what the sandbox starts with and what has to leave it. Name the checkout, the caches, the credentials in the environment, and the hosts the work needs. Then decide what the session must push before it ends, because the branch outlives the machine and the machine does not outlive the session.

The check is quick. Boot a session and write a file without committing. Tear the box down, boot another session on the same branch, and assert the uncommitted file is gone while the pushed commit is present. A workspace that fails this is a scratch disk with a git remote attached. Every hour of agent work on it is one outage away from nothing.

Memory and state: what persists, what is thrown away, what gets compacted

Memory is the component that decides what the next session knows. The decision is which facts belong in files the repo owns, such as the memory/ directory at the root of the repo layout, and which stay in a transcript that ends with the box.

The second decision is compaction. As the context window fills, agents wrap up tasks prematurely. Cognition capped usage at 200K tokens while enabling the 1M-token beta, so the model believed it had runway. Milvus The number matters less than the principle: a harness that owns the compaction point can choose what survives it.

The check is a restart with a question attached. End a session mid-task, start a new one with the same prompt, and assert that it answers a fact only memory holds and names the file. Then read the diff: if memory lives in the repo, every write to it is a commit, and an empty diff after a session that learned something is a failed check.

Feedback sensors: the checks that catch what the agent cannot see

Sensors observe after the agent acts and help it self-correct. The useful decision is how to split them by cost. Computational controls are deterministic and fast, run by the CPU: tests, linters, type checkers, structural analysis. Inferential controls are semantic and slower, more expensive, more non-deterministic: AI code review, an LLM as judge. Martin Fowler Run the cheap ones on every change and the expensive ones on a sample.

Put the fix in the error. The OpenAI field report describes custom linters that enforced a single dependency direction and carried the fix instruction in the error message, which is the difference between a sensor an agent can act on and one it retries blindly. Software Improvement Group

Keep the writer away from the judge. When agents assess their own output, they score it high, and the known fix is to separate the generator from the evaluator, as in a GAN. A review sensor staffed by the agent that wrote the code is a sensor with a known bias built in.

The check is a seeded defect. Break a dependency direction, or misspell a symbol, and assert the sensor fails and prints the fix. Then count what the sensor missed across a handful of seeded defects. The two failure modes are documented and worth a test each. An agent that repeats the same mistake has feedback and no guides. An agent that encodes rules but never learns whether they worked has guides and no sensors.

Orchestration and model routing: who does the work, on which model

Orchestration is subagent spawning, handoffs and model routing, and it is the component that decides where a task splits. The decision is concrete: which steps get a subagent with its own context window, where the handoff happens, and which model each step uses. A subagent earns its cost when the step needs a clean context window. A second opinion from the same context does not need one.

Session work reaches main through a change request. Merge is default-deny for agents. Orchestration ends at that gate. That makes the end of the pipeline the easiest part to test, and the part most worth testing.

The check reads two records. For a task you ran last week, pull the route log and assert each step went to the model you chose rather than the default. Then have the agent attempt a merge on its own session branch. Assert the refusal, and that the change request stays open for a person. Routing that nobody reads is a preference, and a merge gate nobody has tried is a hope.

Build behavior behind a capability, not behind a harness check

Every component so far can be built in a way that ties it to one harness. The rule that keeps a harness swappable is small: new behavior goes behind a capability, never behind a check on which harness is running. The harness boundary README states it for the runtime, where harness.ts is the only module that imports a concrete adapter and two adapters are registered. There is no per-request switching either: one box, one harness, for the life of the box.

That constraint is what makes the swap boring enough to do. With session log restore on, a box that boots a different harness restores the log and records the switch in harness.history. A preflight endpoint answers before it happens: whether the switch can happen, whether the log restores, what the session uses that the target handles worse, and everything the target lacks.

The decision for a team building its own harness is the same at every new feature. Is this a capability the host exposes, or a branch on which harness is running? The second answer is how a harness becomes a fork. A fork is the component you cannot test against anything else.

The check boots the same session log on the other adapter and asserts the capability still answers. Then it calls the preflight and reads the losses before committing to the switch. A harness that cannot pass both is one you find out about during a migration, with the review time already spent.