Understanding AI agent harnesses: From agent loops to production runtimes

An AI agent is more than a language model with permission to call a shell command. The model proposes actions, but another layer decides which tools exist, what context the model receives, what the agent may change, how failures are handled, and when the task is finished.

That layer is the agent harness.

This article explains what a harness does, why it matters, and how four current agent systems approach it: Codex CLI, Pi, OpenCode, and Hermes Agent.

What is a harness, really?

Viv Trivedy’s one-liner, quoted by Addy Osmani in Agent Harness Engineering, does most of the work:

Agent = Model + Harness. If you’re not the model, you’re the harness.

A harness is every piece of code, configuration, and execution logic that isn’t the model itself. A raw model is not an agent. It becomes one once a harness gives it state, tool execution, feedback loops, and enforceable constraints.

That definition gives us a clean boundary for the rest of this article. The model generates decisions and language. Everything that turns those outputs into a controlled, stateful system belongs to the harness.

TL;DR

An agent harness is the runtime around a language model. It manages the model and tool loop, context, permissions, memory, retries, persistence, subagents, and termination.

All four projects discussed here include a harness:

Codex CLI provides a polished, configurable coding harness.
Pi provides a compact and deeply extensible agent runtime.
OpenCode provides a governed multi-agent development platform.
Hermes Agent provides a persistent general-purpose runtime with memory, skills, gateways, scheduling, and delegation.

The important distinction is not whether a project “has a harness.” It is what the harness guarantees, what it merely asks the model to do, and what still depends on operating-system isolation.

Why the harness matters more than the model name

Developers often compare agents by model provider, benchmark score, or context-window size. Those details matter, but they do not explain why two agents using similar models can behave very differently.

A language model can suggest an action. By itself, it does not reliably:

read and edit files;
run tests and inspect the result;
preserve task state;
request approval before a destructive operation;
recover after an interrupted tool call;
compress a long conversation;
delegate work with reduced privileges;
decide whether the requested change is actually complete.

The harness supplies those capabilities and coordinates them over time.

A useful mental model is:

Component	Role
Model	Reasoning engine
Tools	External actions and observations
Memory	Durable facts and reusable procedures
Harness	Control loop, runtime state, policy, and recovery

In practice, the harness often has more influence on reliability and safety than a small difference between model versions.

The minimal agent loop

Most harnesses start with the same basic loop:

User goal
  -> Build prompt and context
  -> Call the model
  -> Model returns a final answer? Return it
  -> Model requests a tool? Check policy and run it
  -> Append the tool result
  -> Call the model again

The loop is easy to sketch. The difficult engineering sits around it.

A production harness has to answer questions such as:

Which instructions enter the prompt, and in what order?
Which tools are visible to the current agent?
Which actions require approval?
How are malformed tool arguments repaired or rejected?
What happens when a stream stops halfway through a tool call?
How does the agent continue when the context window fills up?
What state survives a crash or restart?
Can a child agent gain more authority than its parent?
What evidence is required before the task is marked complete?

A serious harness is not just a while tool_calls loop. It is a stateful runtime with policy and recovery behavior.

What a complete harness contains

Agent state and control flow

The harness tracks whether the agent is reasoning, running a tool, waiting for approval, retrying a request, compacting context, or finishing. Mature systems make those states explicit and persist the important transitions.

Tool registry and execution

Tools need schemas, argument validation, dispatch, progress reporting, error normalization, cancellation, and output limits. Some harnesses also execute independent tools in parallel while serializing file mutations.

Tool visibility and tool permission are separate concerns. Hiding a tool from the prompt reduces accidental use, but the runtime should still reject an unauthorized direct call.

Context management

Every model call competes for a finite token budget. The harness has to select among:

system instructions;
project rules;
conversation history;
tool definitions;
tool output;
retrieved memory;
loaded skills;
files and images;
environment metadata.

When the request becomes too large, the harness may summarize old messages, prune tool output, remove old images, or retain only a recent tail.

Session persistence

Long-running work needs more than an in-memory chat array. A harness may store messages, tool states, model changes, branches, summaries, token usage, file patches, and recovery checkpoints in JSONL, SQLite, or another durable format.

Permissions and isolation

Approval rules can restrict shell commands, edits, external directories, MCP tools, network requests, or subagents. These controls reduce mistakes, but they are not an operating-system sandbox.

If a shell tool runs as your user, an approved command still has your user’s filesystem, process, credential, and network access. Untrusted work needs stronger boundaries such as a container, VM, restricted account, or remote sandbox.

Memory and skills

Memory stores durable facts, preferences, and stable conventions. Skills store reusable procedures. Temporary task progress belongs in neither.

This separation matters. If an agent writes every intermediate conclusion into long-term memory, one bad assumption can contaminate future sessions.

Subagents

Subagents can explore a repository, review a patch, or research separate questions in parallel. A well-designed harness controls their context, model, tools, budget, concurrency, result size, and privilege inheritance.

Observability and recovery

A useful harness makes tool calls, retries, token use, cost, patches, stop reasons, and policy decisions inspectable. A durable harness can also reconstruct an incomplete operation and decide what can be resumed safely.

Codex CLI: the harness is already included

Codex CLI is not just a terminal client for a coding model. It includes the runtime that coordinates the user, model, and local tools.

Its built-in harness covers:

model and tool-call iterations;
repository inspection and file editing;
local command execution;
conversation and context management;
planning;
sandbox and approval modes;
retries and failure handling;
project instructions, skills, MCP tools, and subagents.

For normal use, you do not need to add LangChain, LangGraph, or another harness. Running Codex already means running a model inside the Codex harness.

Most customization happens at the configuration layer:

AGENTS.md defines project-specific working rules.
~/.codex/config.toml controls models, profiles, sandboxing, and approvals.
Skills package repeatable workflows.
MCP adds external tools and context.

The sensible starting point is to configure the existing harness. Build your own only when you need a different execution model, policy engine, scheduler, product interface, or recovery design.

Pi: the hackable harness

Pi explicitly describes itself as an agent harness project. Its repository separates the model layer, generic runtime, coding agent, terminal UI, and telemetry:

packages/ai            Unified model-provider layer
packages/agent         Agent runtime and durable harness work
packages/coding-agent  Coding session, prompts, tools, and CLI
packages/tui           Terminal UI
packages/telemetry     Vendor-neutral telemetry contracts

A useful source-reading order is:

packages/coding-agent/src/core/sdk.ts
packages/coding-agent/src/core/agent-session.ts
packages/agent/src/harness/agent-harness.ts
packages/agent/src/harness/reducer.ts
packages/coding-agent/src/core/tools/
packages/coding-agent/src/core/extensions/
packages/coding-agent/src/core/compaction/

createAgentSession() in sdk.ts shows how the model runtime, tools, settings, session manager, resource loader, and extensions are assembled. AgentSession shows the practical control flow for input, streaming, persistence, retries, context compression, tool hooks, and queued follow-up messages.

Pi’s newer durable harness work is particularly interesting. Its reducer reconstructs lane state from append-only records and entries. It validates incomplete operations, pending tools, deferred work, queued messages, and the effective model configuration. That design looks closer to event-sourced recovery than to a conventional chat transcript.

Where Pi is strong

The package structure is relatively small and readable.
Extensions can hook prompts, providers, tools, sessions, and the TUI.
Extensions can replace built-in tools such as read, edit, and bash.
JSONL sessions are inspectable and support branches, forks, labels, and compaction checkpoints.
The runtime is a good base for experiments and bespoke execution backends.

Where Pi is weak

Pi intentionally does not provide a comprehensive built-in permission system or OS sandbox. It runs with the authority of the process that launched it. Extensions are trusted TypeScript modules with the same authority.

Project trust controls whether project-local settings and extensions are loaded. It does not make model-generated commands safe. Pi recommends external isolation such as Docker, OpenShell, or a Gondolin micro-VM for untrusted work.

Pi’s multi-agent support is also less productized than OpenCode’s or Hermes’s. You can build delegation through extensions, nested model calls, or separate processes, but you are responsible for scheduling, budgets, result aggregation, and isolation.

OpenCode: a governed multi-agent development platform

OpenCode is more opinionated. Its harness is organized around sessions, agent profiles, tools, permissions, plugins, MCP, and a server shared by multiple clients:

session/      Agent loop, streaming, messages, retry, and compaction
agent/        Agent definitions and profiles
tool/         Built-in tools and subagent delegation
permission/   Permission evaluation and approval requests
plugin/       Plugin lifecycle and hooks
mcp/          MCP integration
server/       Runtime APIs for clients

The central loop in session/prompt.ts reconstructs the effective history, handles pending subtasks and compaction work, resolves tools, builds system instructions, invokes the model, and continues when tool results need another model turn.

session/processor.ts handles streamed text and reasoning, tool-call states, token usage, cost, file patches, retry, interruption, and cleanup. Tool calls become durable message parts with states such as pending, running, completed, and error.

Ordered permissions

OpenCode has ordered allow, ask, and deny rules. A rule can match file operations, raw shell commands, URLs, external directories, skills, MCP tools, and target subagent IDs.

The last matching rule wins. Teams can set broad global policy and then refine it for a specific agent.

This is useful governance, but it is still application-level policy. Shell rules match command text, while the command runs with the host user’s authority. Approval reduces accidental damage; it does not contain an approved command.

Native agents and child sessions

OpenCode includes primary agents such as Build and Plan, subagents such as General and Explore, and hidden maintenance agents for compaction, titles, and summaries.

A primary agent can launch a child session in the foreground or background. The child uses its own configured permissions. That flexibility comes with a sharp edge: the child is not automatically limited to a subset of the parent’s authority. Configuration must prevent privilege expansion.

Where OpenCode is strong

Native permission and approval rules
Built-in agent profiles and child sessions
Integrated MCP, plugins, skills, web tools, and multiple clients
Durable streaming state, snapshots, costs, and file patches
A strong out-of-the-box workflow for governed software development

Where OpenCode is weak

More architectural and configuration complexity
Higher cost to understand or replace the runtime
A permission model that can be mistaken for a sandbox
Child-agent boundaries that require careful configuration

Hermes Agent: a persistent general-purpose harness

Hermes Agent goes beyond coding. It is designed for a long-running personal or server-hosted agent that can operate through a terminal, messaging gateway, API, editor protocol, batch runner, or scheduled job.

The core loop is AIAgent.run_conversation() in run_agent.py. Around it, Hermes integrates:

several model providers and API modes;
credential pools and fallback models;
more than 70 registered tools;
MCP and deferred tool discovery;
persistent sessions and full-text search;
memory providers and user modeling;
skills and skill lifecycle tracking;
parallel subagents;
dangerous-command approval;
local, Docker, SSH, cluster, and cloud-sandbox backends;
messaging gateways and Cron automation.

A robust but complex loop

Hermes handles context preflight checks, compression, request timeouts, stale-stream detection, credential rotation, provider fallback, malformed-response recovery, tool batching, interruption, token accounting, and persistence on exit paths.

Built-in, plugin, and MCP tools pass through a shared registry and handle_function_call(). They use the same schema filtering, argument coercion, hooks, approvals, dispatch, and error cleanup.

When the tool catalog becomes too large, Hermes can expose search and describe tools instead of placing every schema in the prompt. The model discovers a tool only when it needs it.

The learning loop is model-mediated

Hermes is often described as self-improving. The supporting infrastructure is real:

the memory tool stores durable facts;
skill_manage creates and patches reusable procedures;
usage records support skill curation;
memory is retrieved before a turn and synchronized afterward;
ownership and safety rules limit autonomous skill maintenance.

The learning decision, however, is mainly prompt-driven. The harness tells the model to save difficult workflows as skills, and the model decides whether to call skill_manage. There is no universal runtime rule that always creates a skill after a fixed number of tool calls.

Hermes therefore has a model-mediated learning loop, not guaranteed online training. It can accumulate useful knowledge, but the quality depends on model compliance, review, and the accuracy of stored memories.

Parallel subagents

Hermes subagents receive a fresh conversation, independent task state, their own iteration budget, and a restricted intersection of available tools.

Shared-memory writes, user clarification, messaging, Cron creation, and recursive delegation are disabled by default. The harness summarizes child results before returning them to the parent and stores full output separately when needed.

The isolation is logical unless the execution backend supplies a stronger boundary. Parent and child can still share the same workspace.

Safety model

Hermes has a layered command-approval engine. It normalizes and partially parses shell commands, detects dangerous patterns, supports session and permanent approvals, accepts user-defined deny rules, and includes an unskippable blocklist for extreme destructive commands.

That is more useful than a generic confirmation dialog, but it cannot prove arbitrary shell code safe. Complex shell behavior, plugins, alternative execution paths, and approved commands can exceed what pattern matching understands. Untrusted execution still belongs in Docker, a VM, or a remote sandbox.

Where Hermes is strong

Broad integrated feature set
Long-running personal agents and remote messaging
Native memory, skills, scheduled jobs, and parallel delegation
Several execution backends and practical approval controls
Sophisticated context compression and provider recovery

Where Hermes is weak

High implementation and operational complexity
A larger attack surface across gateways, skills, MCP, credentials, and automation
Learning that depends partly on prompt compliance
Possible memory pollution from incorrect conclusions
Lossy compression of old logs, images, and edge cases
Static scanning and command matching that cannot replace isolation

Comparing Pi, OpenCode, and Hermes

Dimension	Pi	OpenCode	Hermes Agent
Primary identity	Hackable agent core and coding CLI	Governed multi-agent coding platform	Persistent general-purpose agent runtime
Permission model	Extensions or external isolation	Ordered allow, ask, and deny rules	Command approval, deny rules, and hard blocks
Subagents	Usually custom-built	Native child sessions and profiles	Native parallel delegation with restricted tools
Extensibility	Deep hooks and tool replacement	Plugins, MCP, skills, and agent configuration	Registry, plugins, MCP, toolsets, and memory providers
Persistence	JSONL tree with branches	Structured messages, parts, snapshots, and events	SQLite sessions, search, memory, and trajectories
Context management	Compaction and branch summaries	Maintenance agents and compaction tasks	Layered compression, fallback summaries, and anti-thrashing
Best fit	Harness research and bespoke agent products	Team development and policy governance	Long-running automation and cross-channel agents
Main risk	Trusted extensions have full process authority	Policy complexity and false confidence in containment	Large attack surface and soft learning guarantees

Codex CLI sits next to this comparison as the polished coding product whose harness is mostly configured rather than rebuilt.

Three kinds of guarantees

The most useful habit when reading agent source code is to classify each advertised feature into one of three categories.

Runtime-enforced behavior

The harness deterministically applies the rule. Examples include denying a matching permission rule, limiting steps, rejecting malformed arguments, or blocking a hard-coded destructive command.

Model-mediated behavior

The harness provides a tool and asks the model to use it. Examples include saving a preference, creating a skill after a difficult task, or performing a final self-review. These behaviors may work well, but they are not guaranteed.

Environment-enforced behavior

The operating system or infrastructure creates the boundary. Examples include read-only mounts, restricted Unix users, network policy, containers, VMs, and remote sandboxes.

Do not confuse these layers:

A system prompt is not a policy engine. A policy engine is not a sandbox. A sandbox does not make mounted credentials safe.

How to read any harness source code

A repeatable source-reading method is more useful than memorizing one project’s directory structure.

Confirm the official repository and branch. Agent projects move quickly, and old forks often describe obsolete designs.
Find the user entry point. Locate the CLI command, HTTP handler, gateway callback, or editor adapter.
Trace one complete turn. Follow input, session creation, prompt construction, model streaming, tool dispatch, persistence, and termination.
Inspect the tool boundary. Find schema generation, validation, permission checks, cancellation, error cleanup, and output limits.
Inspect context and persistence. Determine what reaches the model, what survives, how compression works, and how interrupted work is reconstructed.
Inspect privilege inheritance. Check what plugins and subagents inherit, share, and can create.
Separate claims from enforcement. If a feature exists only in a system-prompt string, call it model-mediated.
Verify security documentation against code. Look for fail-open paths, bypasses, approval scope, and the exact point where host authority is exercised.

Design lessons for your own harness

Keep provider-specific message formats out of the orchestration core.
Give every tool call a stable ID, explicit state, and durable result.
Treat context as a budget shared by instructions, history, tools, memory, files, and images.
Persist intent before executing side effects when recovery matters.
Give child agents an explicit subset or intersection of parent capabilities.
Use infrastructure isolation for untrusted work instead of relying on approval prompts.
Bound iterations, concurrency, output size, wall-clock time, and cost.
Store durable facts, reusable procedures, and current task state separately.
Record provenance for generated memories and skills so they can be corrected or rolled back.
Make logs, events, costs, patches, and policy decisions part of the runtime design.

Final takeaway

The harness is where a language model becomes operational software. The model may choose the next action, but the harness decides which actions exist, what context is available, what authority is granted, what state survives, how failure is recovered, and when the work is done.

Codex CLI shows the value of a polished built-in coding harness. Pi shows how a compact runtime can remain understandable and hackable. OpenCode demonstrates policy-driven multi-agent development. Hermes demonstrates how memory, skills, gateways, scheduling, and delegation can turn a harness into a persistent agent platform.

The best harness is not the one with the longest feature list. It is the one whose guarantees match the job.