Harness engineering: every failure becomes a file in an open-source repo

What harness engineering means

Harness engineering is the practice of designing the full environment an AI agent operates within: the rules it follows, the checks it must pass, and the feedback loops that prevent the same mistake from recurring (Software Improvement Group). The shorthand from LangChain is Agent = Model + Harness: the model contains the intelligence and the harness makes that intelligence useful (LangChain).

Benchmarks show how much of the result the harness carries. "On public benchmarks, the same model can place significantly higher or lower depending entirely on how the harness is built," Databricks writes in its introduction to the AI harness. Databricks measured the gap on OfficeQA: paired with the OfficeQA Pro Agent Harness, GPT-5.5 scored 52.63%, up from 36.10% with GPT-5.4.

Where the term came from

Mitchell Hashimoto introduced the term in a personal blog post in February 2026. He described a habit. Every time the agent made a mistake, he engineered a permanent fix into the agent's environment (SIG, Milvus).

That origin explains why the discipline looks less like a purchasing decision and more like bookkeeping. A failure arrives, you find the rule or the check that would have caught it, and you add that rule to the environment. A fix outlives the conversation it came from, and the same failure stops costing you a second afternoon. The work compounds when the environment is stored somewhere the next session reads.

The three layers

"Prompt engineering optimizes what you say to the model; context engineering manages what the model can see; harness engineering designs the world the agent operates in" (Milvus). A better prompt can win one turn. A harness decides whether the tenth turn repeats the first mistake, which is why a bigger context window does not repair a missing check.

Inside a harness

LangChain's anatomy lists the parts: system prompts; tools, skills, MCPs and their descriptions; bundled infrastructure such as the filesystem, sandbox and browser; orchestration logic for subagent spawning, handoffs and model routing; and hooks or middleware that make execution deterministic (LangChain).

Martin Fowler sorts the controls in a way an engineer can act on. Guides are feedforward controls that anticipate the agent's behavior and steer it before it acts. Sensors are feedback controls that observe after the agent acts and help it self-correct (Fowler, April 2026). Both come in two flavors. Computational controls are deterministic and fast, run by the CPU: tests, linters, type checkers, structural analysis. Inferential controls do semantic analysis, AI code review or an LLM as judge, and they are slower, more expensive and more non-deterministic.

Fowler also names the failure mode that follows from building only one half: "Separately, you get either an agent that keeps repeating the same mistakes (feedback-only) or an agent that encodes rules but never finds out whether they worked (feed-forward-only)." A working harness carries both halves, and the computational half is the cheaper one to build.

What a harness engineer does in a week

The job is triage with a longer memory. A recurring failure becomes a guide, a sensor or a permission. A guide is a line in the prompt or a skill that tells the agent how this repo does the thing. A sensor is a check that fails loudly when the agent gets it wrong. A permission narrows what the agent can touch so the failure cannot happen at all.

OpenAI's field report is the clearest public example. A three-person team built a one-million-line codebase over five months with roughly 1,500 merged pull requests, writing no code by hand. They shrank AGENTS.md to about 100 lines acting as a map to a structured docs/ directory, and enforced a single dependency direction with custom linters that carry the fix instruction in the error message (Milvus, SIG).

Two smaller findings explain why the loop needs care. When agents assess their own output, they score it high, so the generator and the evaluator should be separate, as in a GAN. And as the context window fills, agents wrap up tasks prematurely; Cognition capped usage at 200K tokens while enabling the 1M-token beta so the model believed it had runway (Milvus).

Learning it from your own failures

Reading about harnesses is cheap. Growing one takes a loop you can run this week.

1. Read the harness you already have. Open the agent files, the permission grants and the checks your CI runs, and write down what each one is for.
2. Keep a failure log. One line per failed task, with the command, the error and the rule you think was missing.
3. Attach a deterministic check to the fix. A test, a linter rule or a schema check beats a sentence in a prompt because it fails on its own.
4. Re-run the failing task. If the check does not fire, the check is wrong.
5. Commit the rule with the check. A harness change that lives only in a chat window is a setting; one in the repo is a decision the next session inherits.

The last step is where an open harness starts to matter, because the loop only pays off when the rules, the checks and the permissions sit in files a person can read and review. That is also the difference between tuning a product and engineering an environment, and the three layers compared show where each one stops.

A harness you can version

The project is the harness. Your agents, the skills they share, your company memory, and every connector live in one git repo, versioned, diffable, and shared by the whole company (Kortix on GitHub).

An agent in Kortix is an agent file. The frontmatter carries the description, the mode and the permission, and the body is the prompt (agents/kortix.md). The session that file runs in is "an isolated sandbox — an ephemeral branch of this repo, your own /workspace", and "only what you commit + push survives". A rule added to the file therefore arrives as a change someone can read, not as a preference buried in a console.

The project manifest follows the same split. kortix.yaml holds one agents: map per project, and it is governance only: connectors, secrets, kortix_permissions, skills, workspace and enabled, plus file, the path of the agent's .md. Behavior lives in the .md. Grants are deny-by-default, so an omitted connectors, secrets, kortix_permissions or skills grant resolves to "none" and a scoped specialist starts locked down (kortix.yaml).

Isolation is part of the same design. One isolated sandbox per session. Each session has its own isolated machine and branch.

Review is part of the harness too. Session work reaches main through a change request. Merge is default-deny for agents. That makes the change request the place where a new rule is argued about, and it keeps a bad guide or a loose permission from spreading across every session on its own.

Permissions stay deliberate. Approval gates you set. Off until you set them. A team that never sets them runs the default policy.

Harness the company is a different thing

Search the term and you will also find Harness, a software-delivery platform. Its co-founder and CEO is Jyoti Bansal, who co-founded it in 2017 (harness.io). That company sells delivery tooling. Harness engineering is a discipline, not a vendor, and the shared word is the whole overlap.