Skip to main content
Regrade is recommended when only the verifier changes and the task uses a separate verifier. The agent phase is not rerun.

​
Regrade a job

-p accepts a task directory or a directory of tasks. Harbor matches tasks to source trials by task name. Repeat -p to provide multiple locations.

​
Regrade a trial

The updated task must have the same name as the source trial’s task.

​
Regrade from Harbor Hub

Use a Hub job or trial UUID instead of a local path.

​
Requirements

  • The source is a completed, single-step trial with readable result.json and artifacts/manifest.json files.
  • The updated task uses a separate verifier.
  • Every artifact required by the updated verifier was collected by the source trial and still exists.
Using a separate verifier for the source task is a good practice because it already grades from recorded artifacts. It is not required: a shared-verifier source works if it recorded every input needed by the updated verifier. Multi-step trials are not supported.

​
How it works

For each source trial, Harbor:
  1. Copies the recorded outputs into a new trial.
  2. Builds the updated verifier environment.
  3. Runs verification and writes a new result.
No agent environment is started, so regrading needs no agent credentials and incurs no new agent cost.

​
Options

Job regrades also support -n, --n-concurrent and --job-name. Trial regrades support --trial-name.