> ## Documentation Index
> Fetch the complete documentation index at: https://docs.harborframework.com/llms.txt
> Use this file to discover all available pages before exploring further.

# Verifier

> Task verification.

The `tests/test.sh` script is the entrypoint for the verifier. Harbor uploads `tests/` to `/tests/` after the agent runs, unless a [dedicated verifier image or build definition](#separate-verifier-environment) provides the tests. `tests/test.sh` is executed in the task's working directory and is responsible for verifying the task's completion. It must produce a numerical reward at `/logs/verifier/reward.txt` or `/logs/verifier/reward.json`.

## Required script

| OS | File |
| - | - |
| Linux | `tests/test.sh` |
| Windows | `tests/test.bat` |

The script should:

1. Install test dependencies (if needed)
2. Verify that the agent satisfied the [instruction](/tasks/instruction)
3. Write a **reward** under `/logs/verifier/`

We recommend using **absolute paths** inside the script to avoid `cwd` surprises.

## Reward files

| File | Format |
| - | - |
| `/logs/verifier/reward.json` | JSON key-value object of labeled numeric metrics |
| `/logs/verifier/reward.txt` | Single number, usually `1` or `0` |

If both are present, Harbor prefers `reward.json`.

### Example

```bash title="tests/test.sh" theme={"system"}
#!/bin/bash
set -euo pipefail

uvx pytest /tests/test_outputs.py

if [ $? -eq 0 ]; then
  echo 1 > /logs/verifier/reward.txt
else
  echo 0 > /logs/verifier/reward.txt
fi
```

For multi-metric or LLM-based grading, consider using [RewardKit](/rewardkit/quick-start) or writing a structured `reward.json` file.

## Separate verifier environment

Harbor tasks can elect to use a separate sandbox for verification, which improves the security boundary between the agent and the verifier.

To use a separate verifier sandbox, set `environment_mode = "separate"` under `[verifier]` in `task.toml` or include a `[verifier.environment]` section with the same schema as `[environment]`.

Harbor prefers `[verifier.environment].docker_image`, then a build definition in `tests/` (`Dockerfile` or `docker-compose.yaml`), over the agent environment. These dedicated verifier images must include `/tests/test.sh` or `/tests/test.bat`; Harbor does not upload tests into them at runtime.

Without a dedicated verifier image or build definition, Harbor starts a fresh copy of the agent environment and uploads `tests/` to `/tests/`. The agent's filesystem changes are not inherited. For multi-step tasks, step verifier definitions take precedence over task-level definitions.

Artifacts declared in the `[artifacts]` section of the `task.toml` are copied into the verifier sandbox at the same location as they are in the agent sandbox.

See [Separate verifier](/tasks/separate-verifier) for more details.

### Example

```toml title="task.toml" theme={"system"}
[verifier]
environment_mode = "separate"

[verifier.environment]
cpus = 2
```

```dockerfile title="tests/Dockerfile" theme={"system"}
FROM ubuntu:24.04

WORKDIR /app

COPY test.sh /tests/test.sh
```

## Regrading

Harbor can rerun verifiers for existing trials if the task uses a separate verifier environment. This is useful when iterating on a verifier.

See [Regrade](/jobs/regrade) for details.

## Passing environment variables to the verifier

Environment variables can be passed to the verifier using the `[verifier.env]` section in `task.toml`.

```toml title="task.toml" theme={"system"}
[verifier.env]
ANTHROPIC_API_KEY = "${ANTHROPIC_API_KEY}"
MODEL_NAME = "claude-haiku-4-5"
```

The `${VAR}` syntax is used to read from the host environment. Harbor will ask the user to confirm before passing the environment variables to the verifier.

## LLM- or agent-as-a-judge

Task authors can implement `test.sh` to use whatever verification approach they like, including LLM- or agent-as-a-judge.

To avoid boilerplate, consider using [RewardKit](/rewardkit/quick-start) to define your judge or programmatic criteria.

## `rewardkit`

The Harbor team maintains a package called [`harbor-rewardkit`](/rewardkit/quick-start). It is the easiest way to define and run common verifiers, including programmatic criteria and LLM- or agent-as-a-judge.


This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.