Google Cloud AI Research has open-sourced Harness², a framework that improves an AI agent's work without changing the model itself. After a single refinement step, the open GLM-5 rose on WorkBuddy-Bench from 64.2 to 75.7 points: +11.5 with the same weights. The code is released under the Apache 2.0 license, the paper appeared on arXiv on October 9, and the project was widely covered on October 18.
What a harness is and why it matters
An AI agent has two layers. The first is the model, which does the thinking. The second is the harness: the system prompt, rules and guardrails, memory, hints for tools, skills, scripts and subagents. Claude Code and Codex are harnesses around models; we covered how agents are built in What Is an AI Agent.
Usually a person writes the harness and edits it by hand. Harness² does it automatically: all eight parts of the harness can be edited, not just the prompt text.
The eight harness parts Harness² can change. Diagram from the project repository.
How the Propose–Probe–Compose loop works
For each new task, the system takes three steps:
- Propose. An editor suggests harness edits: general ones (for example, "always verify the result") and ones specific to the task type.
- Probe. Each edit is run on the same task, and the agent's behavior is compared.
- Compose. The final harness is assembled only from the edits that held up in the runs.
No ground-truth answer is needed: the system compares the agent's behavior across runs rather than checking the result against a reference. That makes the method usable for work tasks that have no answer key. The superscript in the name stands for a second level: lessons from each task are saved in a shared log and help on the next ones.
One step of the loop and experience carried between tasks. Diagram from the project repository.
What the measurements show
The authors tested the method on three professional benchmarks: WorkBuddy-Bench (office, web and code), JobBench (work tasks) and LAB (legal work). Below is the WorkBuddy-Bench average with the OpenCode executor, means over three runs. "k=1" means one refinement step.
| Method | GLM-5 | Gemini 3.5 Flash |
|---|---|---|
| Original harness | 64.2 | 72.4 |
| ReasoningBank | 65.6 | 73.4 |
| Meta-Harness | 69.7 | 76.4 |
| Harness² (k=1) | 75.7 | 77.5 |
| Harness² (k=5, parallel) | - | 80.8 |
What the table shows:
- The weaker model gains more. GLM-5 grew by 11.5 points, Gemini 3.5 Flash by 5.1. With the refined harness GLM-5 (75.7) passed Gemini with its standard one (72.4), but Gemini also reached 77.5 after refinement, so GLM-5 did not catch up in the end.
- The method beats the alternatives. Both alternatives, Meta-Harness (a method from a Stanford and MIT paper) and ReasoningBank, gave a smaller gain on both models.
- More steps, higher score. At k=5 Gemini 3.5 Flash reached 80.8.
On LAB, GLM-5's partial pass rate rose from 59.6 to 71.4% (+11.8 percentage points). The gain on JobBench is more modest: up to 4.5 points.
Example: how two interventions lifted a score from 43 to 78
In one analyzed JobBench task, Gemini 3.5 Flash failed not for lack of knowledge. It did not reconcile data across the source files and did not reopen the finished result to check it. The task score was 43.1.
Harness² made two interventions (four edits in total):
- a general rule: before working, list all inputs, tie every figure to its source and check the finished files with a script;
- a task skill: rules for reconciling schedule, venue, labor and budget files, plus hints on number formats.
After that, the score for the same task was 77.8. This is a single analyzed case, not an average: across the whole of JobBench the gain is much smaller.
Where it is less tidy
- The agent does not write the edits itself. By default the editor is a separate strong model, Claude Opus 4.8 under OpenCode. In a separate experiment the same model served as editor and executor, and all five models tested, from GLM-5 to Claude Opus 5, improved on every benchmark. Opus 5 gained +3.3 on WorkBuddy-Bench, +5.5 on JobBench and +3.0 on LAB; GLM-5 gained +13.7 on LAB. That experiment used two steps (k=2), and only for it can you say the agent edits itself.
- The cost of refinement is not disclosed. The authors write that both configurations lie on the compute-quality Pareto frontier, counting the cost of both runs and edits. There are no tokens or dollars in the paper. After refinement, tool calls fell by 11% on WorkBuddy-Bench but rose by 20% and 21% on JobBench and LAB. The tool error rate fell on all three benchmarks, for example on JobBench from 14.4% to 9.1%.
- There are no guarantees. The authors note themselves that the signal from probe runs depends on randomness and the models' shared blind spots, and does not prove that an edit will work on another task.
- This is a research project. The repository (about 9,000 GitHub stars) states that it is not an officially supported Google product. Running it takes Linux, Python 3.12, the Google Cloud CLI and models through Vertex AI, plus Docker for WorkBuddy-Bench.
What this means for you
If you do not build agents, the takeaway is simple: the quality of an AI's work depends not only on the model but on the instructions and checks it is given. In the JobBench example, the deciding factor was not new knowledge but a requirement to double-check the result. The same works in an ordinary chat: ask the model, after answering, to check the figures against the source data and list what it could not verify.
If you build agents, Harness² is a ready way to test how much automatic harness refinement gives on your tasks. But mind the cost: every step means extra runs and editor sessions.
Where to try
GLM-5, Gemini 3.5 Flash and Claude Opus 4.8, the models used in the experiments, are available in GPTunneL in one chat. Harness² itself does not run with us: it is code for your own server. But you can compare how different models handle your task before and after you add a result check to the instructions, right in the chat. We covered agents more broadly in the article on Claude Code.
Sources
- google-research/harness2, GitHub - code, Apache 2.0 license, component description, run requirements, the "not an official Google product" note.
- Harness²: Recursive Agent Harnessing for an Open World, arXiv, October 9, 2026 - the paper, results tables, the JobBench case, limitations.
- harness2.github.io, project page - WorkBuddy-Bench results for Gemini 3.5 Flash and comparison with competing methods.