October 28, 2026 · The Harbor Team
The world is transitioning from hand-crafted software to learned software. As the development lifecycle moves away from design & implementation and towards evaluation & optimization, the role of humans increasingly becomes reward shaping. Those rewards dictate the optimization loops that will eventually power all digital technology development. However, designing rewards is difficult. Accurately measuring outcomes requires actively protecting against unexpected, undesired, and adversarial behavior. Preventing reward hacking is necessary to ensure that the evaluation & optimization loop is “hill-climbing” the right hill. This new development paradigm requires new tools and best practices. We’re shipping a batch of new features to mitigate reward hacking. Harbor is built to make minimal assumptions for maximal expressiveness, so some of these features are built into Harbor, some are opt-in tools, and some are example tasks.What counts as a reward hack
Reward hacking occurs when an agent receives a reward that doesn’t reflect the intended completion of the task, by exploiting the environment, verifier, or underlying infrastructure. Three types of reward hacks in Harbor are:
Mitigating these reward hacks requires robust infrastructure and tasks. There is no silver bullet, but the features below increase infra guarantees, enable better task creation, and improve reward-hacking-detection tooling.
Three chances to mitigate reward hacking
We’ve addressed reward hacking in three stages: when a task is written, while it runs, and after it finishes.Before the run
harbor initnew task defaultsharbor lintstatic checksharbor checkrubric review.harborignoreunit test verifiersMultiple solutionsavoid overfittingNon-root agentmanage permissionsBefore the run
Separate verifiers by default
Tasks created withharbor init now use a separate verifier. In shared mode, the verifier runs in the same container as the agent. In separate mode, Harbor copies task-specified artifacts into a fresh verifier sandbox, so the agent never touches the tests or the reward output. Existing tasks retain their current behavior.
Separate verifiers have a limitation: agent performance must be embedded into stored artifacts, so in-memory state like running processes has to be materialized to files before the verifier can test it (see the separate verifier conversion guide). Shared environments are still available for verifiers that need live state.
Catch weak spots with harbor lint
Reward hacking opportunities often appear due to dependency drift or poorly configured tasks. harbor lint statically checks task configs and environment definitions for low-quality, unstable, or broken implementations.
A few example rules:
Review the whole task with harbor check
harbor check drops the agent into a sandbox containing the task source files and has the agent grade the task against a rubric. By default, an 11-point Terminal-Bench-inspired rubric is used, but task creators can override the rubric using --rubric.
Support dev files during task development
Tasks sometimes need development files, like data-generation scripts or unit tests for the verifier. Until now, anything intests/ or solution/ was uploaded into the live sandbox during the corresponding phase, which disincentivized their usage. Verifier unit tests in particular are one of the best ways to catch a weak verifier before an agent does.
We’re introducing a new standard, .harborignore, so authors can version control dev files alongside the task while keeping them out of the sandbox and off Harbor Hub.
Each ignore file keeps files out of a different place:
Ship multiple reference solutions
Oracle solutions are helpful when designing and debugging a verifier. However, a single solution can also lead to overfitting. We’ve provided an example task to show how to ship several solutions, to ensure the verifier isn’t overfit to one of them.Run the agent as a non-root user
hello-world-non-root-user shows a task where the agent runs as an unprivileged user, so it can’t touch root-owned grading files. Harbor does not initialize new tasks with an agent user because managing user permissions varies widely across tasks and should be opted into.
During the run
Grade untrusted code as an untrusted user
Many verifiers grade a submission by running the agent’s code, and if that code runs inside the grader’s process, it can tamper with the grader or write its own reward. The Harbor team maintains an optional packageharbor-rewardkit that simplifies verifier creation.
Rewardkit’s new Worker runs submitted code as an unprivileged user that can’t read the tests or write the reward. Worker also kills every process the code started before returning the output. See the secure-python-worker example for a complete task.
Clear stale reward files
In some cases, an agent using a shared environment could simply write a/logs/verifier/reward.json or reward.txt that isn’t overwritten by the verifier. Harbor now deletes reward.txt and reward.json at the start of verification.
Sanitize artifacts before the verifier reads them
With a separate verifier, artifacts are the only thing that crosses from the agent’s sandbox to the verifier’s. Harbor now:- Drops symlinks that point outside the artifact, so the agent can’t link its output to the reference solution
- Skips FIFOs, sockets and device files, which could hang the grader
- Clears setuid and setgid bits, so the agent can’t plant a privileged program
No automatic output mounts
Local environments like Docker used to bind-mount log and artifact directories to the host automatically, including/logs/verifier in the agent’s container, so the agent could still write a reward file the verifier would read (#3741). Those mounts are now opt-in, and logs and artifacts are collected without them.
Simplifying network restrictions
Agents sometimes cheat when given unrestricted internet access. Tasks and users can configure network restrictions, but allowlisting agent-specific endpoints (e.g.api.openai.com) historically required manual per-job configuration.
Agents can now declare default inference endpoints per provider. When a task is restricted, Harbor automatically allows those hosts, or the base_url you configured.
When the agent’s network is restricted, Harbor also turns off built-in web search for agents that have it, such as Claude Code, Codex, Gemini CLI and OpenCode.
Minimizing eval awareness
An agent that can tell it’s in an eval harness may behave differently. Harbor-branded paths, environment variables, generated agent instructions and identities, and tool directories now use neutral names.A smart model will probably be able to tell it’s being evaluated either way, but it’s good regardless to minimize the production-to-evaluation gap.
After the run
Browse trajectories with harbor view
harbor view starts a local web viewer for your jobs directory. Open any trial to step through its trajectory alongside its reward, cost and timing, and compare results across a job.
Share trajectories on Harbor Hub
Hacks are easier to catch with more eyes on them (as we’ve seen with the recent increase in attention on Terminal-Bench 🙂). Users can upload job results to Harbor Hub to share with others.Flag suspicious trials with harbor analyze
Reading every trajectory doesn’t scale to thousands of trials. harbor analyze puts an agent into a sandbox with the task’s source files, the trial’s logs and its trajectory, and has it classify the trial along two dimensions:
- Reward hacking: did the agent earn its reward by solving the problem, or by gaming the verifier?
- Task specification: was the instruction clear enough that a failure is the agent’s fault?
Publish leaderboards backed by trajectories
Leaderboards should be auditable and reproducible. You can create leaderboards on Harbor Hub, where each row can link to the trials behind it, so anyone reading the leaderboard can view the corresponding trajectories (and the exact config that produced the trial).Update tasks & results often
No task is perfect. Whenharbor analyze or the community surfaces a hack, the task should be fixed, and that’s just as important as designing it well the first time.
Benchmarks should be continuously developed and maintained. But benchmarks exist for comparison, and changing a dataset means reconciling existing results with the new tasks, which can be costly. Harbor makes this cheap with semantic versioning: bump a task’s patch version and existing trials are reused, bump the minor version to regrade them against the new verifier, and bump the major version to rerun them.