harbor run flags while supporting multiple agents, datasets, and tasks in one reusable JSON or YAML file.
--print-config to inspect the resolved config without running the job:
All top-level fields are optional. Harbor applies the defaults shown below. Fields marked as required are required only when their containing object is present.
Job
string
default:"current timestamp"
Job name. When omitted, Harbor uses the current time in
YYYY-MM-DD__HH-MM-SS format.string
default:"\"jobs\""
Directory where Harbor stores job results.
integer
default:"1"
Number of attempts for every task and agent combination.
boolean
default:"false"
Run agent setup only, skipping the agent phase and disabling verification.
number
default:"1.0"
Multiplier applied to task timeouts unless a phase-specific multiplier overrides it.
number | null
default:"null"
Multiplier applied to the agent execution timeout.
number | null
default:"null"
Multiplier applied to the verifier timeout.
number | null
default:"null"
Multiplier applied to the agent setup timeout.
number | null
default:"null"
Multiplier applied to the environment build timeout.
boolean
default:"false"
Enable debug logging.
integer
default:"4"
Maximum number of concurrent trials. Must be at least 1. Per-agent concurrency limits cannot exceed this value.
boolean
default:"false"
Suppress individual trial progress displays.
RetryConfig
default:"{}"
Retry and exponential-backoff configuration.
EnvironmentConfig
default:"{}"
Shared environment-provider configuration applied to every trial.
VerifierConfig
default:"{}"
Shared verifier configuration applied to every trial.
list[MetricConfig]
default:"[]"
Job-level metrics appended to each dataset’s metrics.
list[AgentConfig]
default:"[{\"name\":\"oracle\"}]"
Agents evaluated by the job.
UserAgentConfig | null
default:"null"
Optional simulated-user agent and bridge applied to every trial.
list[DatasetConfig]
default:"[]"
Dataset sources expanded into tasks.
list[TaskConfig]
default:"[]"
Individual task sources.
list[string | ArtifactConfig]
default:"[]"
Environment paths collected after each trial. A string is shorthand for an artifact with only
source set.list[string]
default:"[]"
Files appended to each task instruction, before
extra_instructions.list[string]
default:"[]"
Inline text appended to each task instruction after
extra_instruction_paths.list[SourceJobConfig]
default:"[]"
Source jobs used for regrading. When set, Harbor derives one new trial from each matching source trial instead of expanding tasks, agents, and attempts normally.
Retry
integer
default:"0"
Maximum retry attempts. Must be at least 0.
set[string] | null
default:"null"
Exception class names eligible for retries.
null includes every exception not excluded below.set[string] | null
default:"built-in non-retryable exceptions"
Exception class names that are never retried. Exclusions take precedence over inclusions. The defaults are
AgentTimeoutError, VerifierTimeoutError, RewardFileNotFoundError, RewardFileEmptyError, VerifierOutputParseError, ApiUsageLimitError, AgentSafetyRefusalError, AgentAuthenticationError, and ModelNotFoundError.number
default:"1.0"
Multiplier for exponential-backoff wait times.
number
default:"1.0"
Minimum delay between retries in seconds.
number
default:"60.0"
Maximum delay between retries in seconds.
Agents
string | null
default:"\"oracle\""
Pre-integrated agent name. Use
import_path for a custom agent.string | null
default:"null"
Custom agent import path in
module.path:ClassName format.string | null
default:"null"
Model identifier passed to the agent.
integer | null
default:"null"
Per-agent limit on concurrent
agent.run() phases. Must be at least 1 and cannot exceed n_concurrent_trials.string | null
default:"null"
Shared concurrency-pool name. Agents in the same group must set the same
n_concurrent value.list[string]
default:"[]"
Local skill directories, Git URLs, or
org/name[@ref] skill sources.number | null
default:"null"
Replace the task’s agent timeout with this value in seconds.
number | null
default:"null"
Replace the agent setup timeout with this value in seconds.
number | null
default:"null"
Upper bound for the effective agent timeout in seconds.
boolean
default:"false"
Resume the agent’s native session between steps of a multi-step task. Requires agent resume support.
string | null
default:"null"
Load a native
.jsonl or ATIF .json trajectory before the first step. Requires the corresponding agent capability.list[string]
default:"[]"
Hostnames, IP addresses, or CIDR ranges added to the allowlist during
agent.run() only.list[string]
default:"[]"
Glob patterns selecting agent log files to download.
list[string]
default:"[]"
Glob patterns excluded from downloaded agent logs after applying
include_logs.object
default:"{}"
Integration-specific keyword arguments passed to the agent constructor.
object[string, string]
default:"{}"
Environment variables exposed only during the agent phase.
list[MCPServerConfig]
default:"[]"
MCP servers made available to the agent.
MCP servers
string
Required server name.
"stdio" | "sse" | "streamable-http"
default:"\"sse\""
MCP transport. The legacy value
http is normalized to streamable-http.string | null
default:"null"
Server URL. Required for
sse and streamable-http transports.string | null
default:"null"
Executable command. Required for the
stdio transport.list[string]
default:"[]"
Arguments passed to a
stdio server command.Simulated user
user_agent supports every field in agents[], plus the fields below. The bridge field is required when user_agent is configured.
string | null
default:"null"
Path to a file defining the simulated user’s persona.
string | null
default:"null"
Path to the Jinja2 prompt template used by the simulated user.
BridgeConfig
Required bridge connecting the simulated user to the primary agent.
"acp"
Required bridge implementation. Currently only
acp is supported.string | null
default:"null"
Optional replacement for the bridge instructions.
object
default:"{}"
Bridge-specific keyword arguments.
Environment
EnvironmentType | null
default:"\"docker\""
Pre-integrated environment provider:
docker, podman, daytona, e2b, modal, runloop, langsmith, ec2, gke, ack, openshift, novita, apple-container, singularity, islo, tensorlake, cwsandbox, wandb, use-computer, cua-cloud, blaxel, opensandbox, beam, skypilot, hf-sandbox, hyperbrowser, or vercel. See pre-integrated sandboxes.string | null
default:"null"
Custom environment import path in
module.path:ClassName format.boolean
default:"false"
Rebuild the environment even when a cached build exists.
boolean
default:"true"
Delete the environment after the trial finishes.
"auto" | "limit" | "request" | "guarantee" | "ignore"
default:"\"auto\""
How the provider enforces the task’s CPU value.
"auto" | "limit" | "request" | "guarantee" | "ignore"
default:"\"auto\""
How the provider enforces the task’s memory value.
integer | null
default:"null"
Replace the task’s CPU value at runtime.
integer | null
default:"null"
Replace the task’s memory value in MB.
integer | null
default:"null"
Replace the task’s storage value in MB.
integer | null
default:"null"
Replace the task’s GPU count.
TpuSpec | null
default:"null"
Replace the task’s TPU specification.
boolean
default:"false"
Deprecated. This field is accepted but has no effect and is excluded when serializing configs.
list[ServiceVolumeConfig] | null
default:"null"
Docker Compose long-syntax volume mounts applied to the environment container.
list[string]
default:"[]"
Additional Docker Compose overlay files.
object[string, string]
default:"{}"
Baseline environment variables exposed inside the sandbox.
object
default:"{}"
Provider-specific keyword arguments passed to the environment constructor.
list[string]
default:"[]"
Hostnames, IP addresses, or CIDR ranges added to the environment network baseline.
TPU override
string
Required TPU alias or canonical GKE accelerator label, such as
v6e or tpu-v6e-slice.string
Required topology in
NxM or NxMxK format, such as 2x4.Mounts
"bind" | "volume" | "image"
Required mount type.
string
Required host path, named volume, or image source.
string
Required target path inside the container.
true
default:"omitted"
Set to
true for a read-only mount.false
default:"omitted"
Set to
false to prevent Docker Compose from creating a missing bind source path."z" | "Z"
default:"omitted"
Optional SELinux relabeling mode for a bind mount.
string
default:"omitted"
Optional subpath within a named volume.
string
default:"omitted"
Optional subpath within an image mount.
Verifier
number | null
default:"null"
Replace the task’s verifier timeout with this value in seconds.
number | null
default:"null"
Upper bound for the effective verifier timeout in seconds.
list[string]
default:"[]"
Glob patterns selecting verifier log files to download. The reward file is always downloaded.
list[string]
default:"[]"
Glob patterns excluded from verifier logs after applying
include_logs.object[string, string]
default:"{}"
Environment variables exposed only during the verifier phase.
string | null
default:"null"
Custom verifier import path in
module.path:ClassName format.object
default:"{}"
Keyword arguments passed to the custom verifier.
boolean
default:"false"
Skip verification. Automatically set to
true when install_only is enabled.Datasets
string | null
default:"null"
Local dataset directory. With
repo, this selects a repo-relative implicit dataset directory.string | null
default:"null"
Harbor Hub dataset in
org/name format, or a bare dataset name for a custom or Git-repository registry.string | null
default:"null"
Version selected from a JSON registry or a named Git-repository registry.
string | null
default:"null"
Tag, revision, or digest selected for a Harbor Hub dataset.
string | null
default:"null"
URL of a custom
registry.json file.string | null
default:"null"
Path to a custom
registry.json. With repo, this path is relative to the repository.string | null
default:"null"
Git repository shorthand or URL, optionally pinned with
@ref.boolean
default:"false"
Overwrite cached remote tasks.
string | null
default:"null"
Directory used to cache downloaded tasks.
list[string] | null
default:"null"
Glob patterns selecting tasks by name.
list[string] | null
default:"null"
Glob patterns excluding tasks after applying
task_names.integer | null
default:"null"
Maximum number of tasks after inclusion and exclusion filters are applied.
repo, set either path or name, but not both. With repo, path selects an implicit dataset while name selects a named registry dataset. version and ref cannot both be set.
See datasets, custom registries, and Git repository datasets.
Tasks
string | null
default:"null"
Local task directory, or the path within a Git repository when
git_url is set.string | null
default:"null"
Git repository containing the task.
string | null
default:"null"
Git commit containing the task. Requires
git_url.string | null
default:"null"
Harbor Hub task name in
org/name format.string | null
default:"null"
Harbor Hub task tag, revision, or digest. Requires
name.boolean
default:"false"
Overwrite the cached remote task.
string | null
default:"null"
Directory used to cache the downloaded task.
string | null
default:"null"
Optional source label used when grouping tasks and metrics.
path or name, but not both. A Git task uses path with git_url; a Harbor Hub task uses name with an optional ref.
See the task overview.
Metrics
"sum" | "min" | "max" | "mean" | "uv-script"
default:"\"mean\""
Metric implementation used to aggregate task rewards.
object
default:"{}"
Keyword arguments passed to the metric implementation.
uv-script requires script_path.Artifacts
string
Required environment path to collect. It cannot contain
.. path components.string | null
default:"null"
Optional path under the trial’s artifact directory. Harbor derives it from
source when omitted.list[string]
default:"[]"
Patterns excluded when downloading a directory artifact.
string | null
default:"null"
Docker Compose service to collect from.
null and main target the agent container.Regrade sources
"regrade"
Required derivation action. Currently only
regrade is supported."local" | "hub"
Required source location.
string | null
default:"null"
Source job UUID. Required for a Hub source and optional for a local source.
string | null
default:"null"
Source job directory. Required for a local source and invalid for a Hub source.
install_only. See Regrade a job.