Your repo is most of an open-source coding agent harness

What the harness adds to a coding agent

An agent arrives with a harness whether or not anyone designed one. The vendor ships a loop, a set of default tools and a permission prompt, and that is enough to edit a file. The question harness engineering asks is narrower: what does a coding agent need that the loop does not provide?

For code, most of the answer already sits in the repo. Your test suite, your lint rules, your type checker and the reviewer reading the diff catch work going wrong. A team running those has done the expensive half. What it usually lacks is the property that makes the work compound: a failure that happened once cannot come back unnoticed.

Martin Fowler's split is the one to build against. Guides are feedforward controls, and they steer the agent before it acts. Sensors are feedback controls, and they inspect what the agent produced after the act. Every recurring failure belongs in one of those two columns, and the repair is a file the next session reads.

The guide set: what the agent reads before it types

Three guides carry a codebase, and only the first rides along on every turn. The conventions file stays short and points outward. In the field report from OpenAI's team, three people built a one-million-line codebase over five months. They merged roughly 1,500 pull requests and wrote no code by hand. The file that traveled with every request was an AGENTS.md of about 100 lines, acting as a map to a structured docs/ directory (Milvus, Software Improvement Group). That is the shape to copy. A conventions file of a thousand lines gets skimmed; a table of contents gets followed, and the depth waits in the pages it names.

The bootstrap skill holds the commands: how to install, which test to run first, where the fixtures live, what green looks like in this repo. It loads when a task needs it instead of occupying every turn.

The structural test is the guide a person cannot enforce by remembering. One dependency direction, a closed set of exports, a module boundary: write it as a test and every future session inherits the rule. The same team enforced a single dependency direction with custom linters rather than a document, and the linter is what made the rule true.

Check the set by naming a rule from each guide in a scripted session. Ask which file holds it, and ask whether every path in the conventions file resolves. An agent that cannot point at the file has not read the guide, because the guide is not in the harness yet.

The sensor set: what runs after the agent types

The second distinction that matters is cost, and it decides the order. Computational sensors are deterministic and fast, run by the CPU: tests, linters, type checkers, structural analysis. Inferential sensors read for meaning: semantic review, an AI code review, a model as judge. They are slower, more expensive and less predictable, and the changes that need judgment are where they earn their cost.

Two habits keep the set usable. Put the correction in the failure. A linter error that says "import from src/lib/events, never from src/features" ends the retry loop, while "unexpected token" starts one. And keep the writer away from the judge. Agents score their own output high when asked to assess it. The known fix is a separate evaluator, the way a GAN separates generator from discriminator (Milvus). A reviewer agent reading the diff its sibling wrote is a sensor with a bias built into it.

Where each check runs

Order the sensors by cost per run and put the cheap ones closest to the edit. A type checker that answers in a second can be run by the agent while it still holds the context to fix what it broke. A review that answers in ten minutes reaches a person who has moved on to another task.

A threshold counts as harness code only once a check enforces it. A page that should feel fast is a wish. A start-time budget asserted in the test run is a number that fails the build on a busy Tuesday. Bundle size, test duration and allowed warning count work the same way. Write the number in the check, and let the check be where the number is argued about.

Three additions repay a first sprint: a structural test for the rule agents keep breaking, a lint rule whose message carries the fix, and a gate that refuses an agent's own change request. Each one runs at a different stage, and none of them needs a new tool.

The gate: the last sensor

The review gate is the sensor that stops a bad change instead of describing it. In Kortix, session work reaches main through a change request. Merge is default-deny for agents. Approval gates you set. Off until you set them.

Isolation makes the gate the only route. One isolated sandbox per session. Each session has its own isolated machine and branch. Work that never leaves that branch cannot reach the code everyone builds on, and the change request is the one door left open.

The gate is also where the harness itself changes. Your agents, the skills they share, your company memory, and every connector live in one git repo, versioned and diffable (Kortix on GitHub). A shorter conventions file, a new structural test, a narrowed grant: each arrives as a diff someone reads before the next session inherits it.

Test the gate with a seeded defect. Introduce the dependency your structural test forbids, assert the check fails, and assert the change request stays open with no merge. A gate nobody has tripped is a setting nobody has verified.

When one half is missing

Two symptoms cover most broken harnesses. An agent that makes the same mistake in three sessions has sensors and no guides. The repair belongs in the conventions file or a structural test. An agent that follows rules nobody has checked has guides and no sensors, so the repair is a seeded defect that proves each rule fires. Fowler names both halves as the field's failure mode: feedback alone keeps repeating mistakes, and feedforward alone never learns whether the rules worked.

Keep a log while you fix it. One line per failed task, with the command, the error and the rule that would have caught it. That log is the backlog for the guide set, and a bad week turns into two tests and a paragraph.