Skip to main content
Regrade is recommended when only the verifier changes and the task uses a separate verifier. The agent phase is not rerun.

​
Regrade a job

-p accepts a task directory or a directory of tasks. Harbor matches tasks to source trials by task name. Repeat -p to provide multiple locations.

​
Regrade a trial

The updated task must have the same name as the source trial’s task.

​
Regrade from Harbor Hub

Use a Hub job or trial UUID instead of a local path.

​
Requirements

  • The source is a completed trial with readable result.json and artifacts/manifest.json files.
  • The updated task uses a separate verifier.
  • Every artifact required by the updated verifier was collected by the source trial and still exists.
Using a separate verifier for the source task is a good practice because it already grades from recorded artifacts. It is not required: a shared-verifier source works if it recorded every input needed by the updated verifier. Multi-step trials are regraded step by step and must keep the same steps as the updated task. Legacy artifact manifests do not record exclusion filters. When the updated task declares exclusions, Harbor warns and continues regrading these records. Changes to explicitly recorded filters still prevent regrading.

​
How it works

For each source trial, Harbor:
  1. Copies the recorded outputs into a new trial.
  2. Builds the updated verifier environment.
  3. Runs verification and writes a new result.
No agent environment is started, so regrading needs no agent credentials and incurs no new agent cost.

​
Options

Job regrades also support -n, --n-concurrent and --job-name. Trial regrades support --trial-name.