Tasks created with
harbor task init or harbor init --task configure separate
verification. Omitting both verifier.environment_mode and
[verifier.environment] uses shared verification.environment_mode = "shared" under [verifier], omit [verifier.environment], and install verifier dependencies in the agent image.
Configuration
There are two ways to configure a separate verifier environment:- Set
environment_mode = "separate"under[verifier]intask.toml. - Add
[verifier.environment]totask.toml.
Option 1: Reuse the agent’s environment configuration
Setenvironment_mode = "separate" under [verifier]:
tests/ to /tests/. The agent’s filesystem changes are not inherited.
Option 2: Use a verifier-specific environment
Add[verifier.environment] to use a different image or resources. This implicitly enables separate mode:
[environment], including network policy. It cannot be combined with environment_mode = "shared".
Mode resolution
Existing tasks that omit bothverifier.environment_mode and verifier.environment continue to use shared verification.
Image selection
Harbor selects the first available definition:
Resource settings come from
[verifier.environment] when provided, otherwise [environment]. An inherited agent image never overrides a verifier build definition.
Dedicated verifier image
New tasks includetests/Dockerfile for dependencies and bundled tests (one per step for multi-step tasks). Keep test.sh focused on running tests and writing rewards.
To reuse the agent environment definition, remove the verifier Dockerfile and install verifier dependencies in the agent image.
Dedicated verifier images must provide /tests/test.sh (or /tests/test.bat on Windows). Harbor does not upload tests into these images at runtime.
tests/Dockerfile can bundle the grading files:
.dockerignore for build exclusions;
.harborignore filters uploads and publication.
Artifact transfer
When a separate verifier runs, Harbor copies into the verifier env:/logs/artifacts/(agent publish directory)- Paths listed in task-, trial-, and step-level
artifactsfields
source paths in the verifier environment, not their host destination paths. For example, { source = "/app/report.json", destination = "report.json" } is restored to /app/report.json in the verifier; destination only controls where it is saved on the host. See Artifacts.
/logs/agent/ and /logs/verifier/ are not copied unless declared as artifacts — e.g. trajectory grading:
Sanitize artifacts
If artifacts were uploaded unchanged, an agent could cheat the verifier:- Setuid binary: the agent ships a setuid program. Uploaded as root, it runs as root when the verifier executes it and writes its own
/logs/verifier/reward.json. - Symlink: the agent makes
/app/output.pnga symlink to/tests/solution.png, so the verifier compares the reference solution against itself.
data filter, and uploads the result. The filter clears setuid/setgid bits, skips special files and symlinks that point outside the artifact, and normalizes permissions. Skipped entries are logged as a warning. Artifact contents are still untrusted input.
Multi-step tasks
Each step can override verifier mode under[steps.verifier]. Mixed shared/separate per step is supported:
[steps.verifier].environment_mode when set; else [steps.verifier.environment] present implies "separate"; else trial-level. Network rules: Network policies.
Image precedence is step image, step tests build definition, task verifier image, task tests build definition, then agent environment. A step tests/ directory without a build definition does not override a task verifier image. With the agent-environment fallback, Harbor uploads base tests, then overlays step tests.
Tests are validated against each step’s effective verifier OS, so a Linux agent can be graded on Windows (and vice versa) when the matching test.sh / test.bat exists.
