> ## Documentation Index
> Fetch the complete documentation index at: https://docs.harborframework.com/llms.txt
> Use this file to discover all available pages before exploring further.

# Custom verifiers

> Customize the job verification process.

During a trial, after the agent runs, the `Verifier` object uploads the `tests/` directory to the sandbox and executes `/tests/test.sh`.

The `Verifier` object is the default implementation of the `BaseVerifier`. If a user wants to orchestrate verification differently, or run some of the verifier logic on the host machine, they can implement a custom `BaseVerifier`.

| | `tests/` w/default `Verifier` | Custom `BaseVerifier` |
| - | - | - |
| Logic | Task files, typically `tests/test.sh` | An importable Python class |
| Execution | Inside the verifier sandbox | `verify()` runs in the Harbor process and can call sandbox operations through `self.environment` |
| Harbor handles | Uploading tests, running the script, downloading verifier logs, and parsing reward files | Constructing the verifier and calling `verify()` |
| Reward output | `/logs/verifier/reward.txt` or `/logs/verifier/reward.json` | A returned `VerifierResult` |
| Configuration | Included with the task | `verifier.import_path` and optional `verifier.kwargs` in the job config |
| Best for | Portable, task-specific verification | Custom orchestration or host-side checks |

<Note>
  A custom verifier replaces the default workflow. It must perform any required uploads, commands, and reward calculation itself.
</Note>

For example, this verifier checks whether the agent created a required file in the sandbox:

```python title="custom_verifier.py" theme={"system"}
from typing import Any

from harbor.models.verifier.result import VerifierResult
from harbor.verifier.base import BaseVerifier


class OutputVerifier(BaseVerifier):
    def __init__(
        self, *, required_path: str = "/app/output.json", **kwargs: Any
    ) -> None:
        super().__init__(**kwargs)
        self.required_path = required_path

    async def verify(self) -> VerifierResult:
        exists = await self.environment.is_file(self.required_path)
        return VerifierResult(rewards={"reward": int(exists)})
```

The file must be importable by the Harbor process. Select the class with `verifier.import_path`; values in `verifier.kwargs` are passed to its constructor.

<Tabs>
  <Tab title="CLI">
    ```bash theme={"system"}
    harbor run -p "<path/to/dataset>" -a "<agent>" -m "<model>" \
      --verifier custom_verifier:OutputVerifier \
      --verifier-kwarg required_path=/app/output.json
    ```
  </Tab>

  <Tab title="Config">
    ```yaml theme={"system"}
    verifier:
      import_path: custom_verifier:OutputVerifier
      kwargs:
        required_path: /app/output.json
    ```
  </Tab>
</Tabs>

See [Configs](/jobs/configs) for the complete verifier configuration schema.


This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.