How long does it take to build a harness?

An afternoon gets you through the first four steps, and a week of real work through the rest. The build is small because each step is one file; the time goes into waiting for failures worth fixing, not into writing configuration.

Do you need a framework?

No. A harness is a few files in the repo your agent already runs from: an agent file, a short guide, a manifest, a check and a change request. A framework helps when you want to swap the runtime underneath, which is a different problem from getting the rules written down.

Can you build one on top of an existing agent?

Yes, and that is the cheaper start. Keep the agent you run today, add AGENTS.md and one deterministic check, then move the repository into a project that gives each session its own branch so a bad run costs nothing.

How to build

How to build an open-source agent harness, one file at a time

An agent harness grows from the failures you actually hit. A harness designed in a document before the first one is a guess. Each step in this build lands one file in your repo, with the command that creates it and the check that proves it worked. Kortix is the open-source AI Operating System.

Try Kortix

Where the seven files land

kortix.yaml

#   agents/<name>.md          one file per agent (`agents.<name>.file`)#   skills/<name>/SKILL.md    skills, loaded by every harness#   memory/                   the project brain (the `kortix-memory` skill)#   tools/<name>.ts           project tools, loaded by every harness (`tools:` below)#   harnesses/opencode/       files only OpenCode reads (`opencode.config_dir`)#   harnesses/pi/             files only pi reads (`pi.config_dir`); create it when needed

Source: https://github.com/kortix-ai/suna/blob/dev/packages/starter/templates/base/kortix.yaml

The starter layout puts agents, skills, memory and tools at the root, so every harness reads the same files.

1. Pick the job that already fails, and give it one file

A harness written before the first failure is a guess about what the agent will get wrong. Pick one job it does badly today, in one repo. Start there with a single file.

kortix init scaffolds the project: kortix.yaml, an agents/ directory, and one general agent file. Copy that file to agents/invoice-triage.md and cut the body down to the job, because the body is the prompt. The frontmatter above it carries the description, the mode and the permission (agents/kortix.md). Then declare the file in the manifest under its own key, file: agents/invoice-triage.md, so the platform knows which .md is which agent.

curl -fsSL https://kortix.com/install | bash
kortix init

What to check after this step: start a session on the new agent and ask what it produces and what it refuses. If the answer is still the general worker's, the job never reached the file, and every later step is built on the wrong agent.

2. Write the guide as a map, not a manual

The instruction file travels with every turn, so it should be short and mostly pointers. Conventions that apply to every task stay in it. Each procedure one kind of task needs moves to its own file under docs/.

A three-person team built a one-million-line codebase over five months, with roughly 1,500 merged pull requests and no code written by hand. It shrank its AGENTS.md to about 100 lines acting as a map to a structured docs/ directory (Software Improvement Group). The file stayed readable because it stopped carrying the procedures.

What to check: every path the map names has to resolve, so run test -f over the list. Then start a session and ask for a convention you know exists. If the agent cannot name the file that holds it, the guide is not reachable yet.

3. Give the session its own machine and its own branch

A harness that can damage the trunk is one nobody lets run unattended. The manifest grants the workspace per agent. workspace: { project_repository: true } gives new sessions a checkout of the project repository, on the session's own branch (kortix.yaml). One isolated sandbox per session. Each session has its own isolated machine and branch.

kortix ship
kortix sessions new --prompt "Reproduce the failing invoice total test and stop there"
kortix cr ls

What to check: break a file inside the session, then confirm main has not moved. kortix cr ls stays empty until that session pushes and opens one. That is the property you want, because a bad run then costs the time it took and nothing else.

4. Declare the tools, and let the omissions deny

The manifest's grants are deny-by-default: an omitted connectors, secrets, kortix_permissions or skills grant resolves to none, so a scoped specialist starts locked down (kortix.yaml). Write the three or four grants the job needs and leave the rest out. tools: takes an allow list or an exclusion list, so the narrow version is a short list of names rather than a wide list with holes cut in it.

The rule to hold while you write it: the agent gets the tools its job needs. One more is a decision someone makes on purpose. A shell tool granted so the agent can read a log file also reaches the network, and the permission is wider than the reason you wrote down.

What to check: have the session call a connector you did not grant, and read the refusal. Then add that one grant and watch the same call succeed. A denial nobody has triggered is a line in a file, and the person who tests it first is usually reading an incident.

5. Attach the first deterministic sensor, with the fix in its message

Guides steer the agent before it acts. Sensors check what it produced after (Martin Fowler). Start with the cheap half. Computational controls are deterministic and fast, run by the CPU: tests, linters, type checkers, structural analysis. Inferential controls, such as AI code review or an LLM as judge, are slower, more expensive and more non-deterministic, and they belong on a sample, not on every change.

The file is a script, scripts/check-imports.sh, wired as a pre-commit hook so it runs on the way in. Put the remedy in the failure text. The custom linters in the OpenAI field report enforce one dependency direction and carry the fix instruction in the error message (SIG). That difference separates a check an agent can act on from one it retries blind.

kortix sessions new --prompt "Write scripts/check-imports.sh, wire it as a pre-commit hook, and prove it fails on a seeded violation"

What to check: seed the violation yourself and read the output. The message names the file, the rule and the next action, and the hook refuses the commit. A message that prints a rule number and nothing else is written for a person who already knows the rule.

6. Put the review gate before main

The last control is the one that keeps a bad guide from spreading. Session work reaches main through a change request. Merge is default-deny for agents. The file for this step is .github/CODEOWNERS, which names the people whose review a path requires. The gate gets a name in the repo as well as in a policy page.

kortix cr ls

What to check: open one change request from a session. It should sit there unmerged until a person reads it, and the session should not be able to merge its own. Then read the diff as a reviewer would. The harness change and the check that proves it belong in the same request. Split across two, the next session inherits a rule with no evidence.

7. Keep the failure log, and re-run the task that failed

Every harness change starts as a failure, and the log is where the failure stops repeating. docs/failures.md takes one row per failure: the command, the error, the file that fixed it, the commit. A row without a commit is a note to yourself. It dies with the session.

kortix sessions new --prompt "Re-run the invoice total fix on a fresh branch and report which check fired"
kortix cr ls

What to check: run the same task again in a fresh session, and watch for the sensor to fire before the agent repeats the mistake. Then read the log row and follow it to the commit. If the re-run produces the same failure, the fix went into the conversation and never into the environment.

Two mistakes that cost the most

The first is the instruction file that grows into a manual. Every incident adds a paragraph until the file is too long to load comfortably, and the rules that matter sit in the middle where nobody reads them. Keep the map short and push each procedure into the file that needs it.

The second is a harness check where a capability belongs. New behaviour in Kortix is built behind a capability, never behind a check on which harness is running (the harness boundary). That rule keeps the layout neutral: agents, skills, memory and tools sit at the root, and the files only one harness reads stay in harnesses/<name>/. A branch on the harness name is how a harness turns into a fork. A fork is the one component you cannot test against anything else.

Questions that come before the first step

How long does it take to build a harness?: An afternoon gets you through the first four steps, and a week of real work through the rest. The build is small because each step is one file; the time goes into waiting for failures worth fixing, not into writing configuration.
Do you need a framework?: No. A harness is a few files in the repo your agent already runs from: an agent file, a short guide, a manifest, a check and a change request. A framework helps when you want to swap the runtime underneath, which is a different problem from getting the rules written down.
Can you build one on top of an existing agent?: Yes, and that is the cheaper start. Keep the agent you run today, add AGENTS.md and one deterministic check, then move the repository into a project that gives each session its own branch so a bad run costs nothing.

Grow the harness in a repo you own

Install the CLI, scaffold one agent, and let the first failure write the first rule.

Try Kortix Read the docs