> ## Documentation Index
> Fetch the complete documentation index at: https://docs.harborframework.com/llms.txt
> Use this file to discover all available pages before exploring further.

# CLI

> Launch remote rollouts and browse jobs, trials, and secrets from the harbor CLI

Please ensure you have either logged in with `harbor auth login` or minted an API key
(see [Creating a Harbor Hub API Key](/core-concepts/hosted-harbor/api-key))

## Launching a Job

The command `harbor run --launch` will attempt to launch a job under your personal organization

The `--launch` flag can be combined with a singular agent/model combination (e.g. `harbor run --launch -a claude-code -m anthropic/fable-5 -d harbor/hello-world`).
Agent/model sweeps must be launched using a config file (e.g. `harbor run --launch -c config.yaml`)

```yaml theme={"system"}
# config.yaml
job_name: opus-vs-codex-sweep
organization: my-org          # omit to use your personal org
credential_mode: gateway      # or "direct"

n_attempts: 3                 # 1-10
n_concurrent_trials: 32       # 1-1000

# Every agent runs against every task. Trials = n_attempts x tasks x agents.
agents:
  - name: claude-code
    model_name: anthropic/claude-opus-4-1
    secrets: [ANTHROPIC_API_KEY]

  - name: claude-code
    model_name: anthropic/claude-sonnet-4-5
    secrets: [ANTHROPIC_API_KEY]

  - name: codex
    model_name: openai/gpt-5
    secrets: [OPENAI_API_KEY, HF_TOKEN]
    n_concurrent: 8           # per-agent sub-limit under n_concurrent_trials
    env:
      RUST_LOG: info          # nonsensitive only - secrets go in `secrets`

  - name: oracle              # needs no inference credential
    secrets: []

datasets:
  - name: terminal-bench/terminal-bench-2-1
    ref: "6"                  # `ref` or `version`, never both
    n_tasks: 25
    exclude_task_names:
      - "flaky-*"

tasks:
  - name: harbor/hello-world
    ref: latest

# Supplied for this job only, read from your local environment at launch.
job_secrets:
  HF_TOKEN:
    from_env: HF_TOKEN

retry:
  max_retries: 2
  include_exceptions:
    - EnvironmentStartError
```

### Injecting Secrets

There are two methods of injecting secrets into the hosted job: flags `--stored-secret <ENV_VAR_NAME>` and `--one-off-secret ENV_VAR_NAME=secret_value`,
and config fields `job_secrets` and `secrets`.

When using flags, `--stored-secret` will pull `<ENV_VAR_NAME>` from the organization's stored credentials on the hub.
`--one-off-secret` will encrypt the secret value and ensure that the value is not persisted
in any job config files. The secret will be decrypted only when necessary and marked revoked
after the job finishes. Flags apply secrets to every agent/model combination in the job config.

In other words, if I launch with the following command: `harbor run --launch -c simple_config.yaml --stored-secret TEST_VAR`
where `simple_config.yaml` is

```yaml theme={"system"}
job_name: simple-config
organization: my-org          # omit to use your personal org

agents:
  - name: claude-code
    model_name: anthropic/claude-opus-4-1
    secrets: [ANTHROPIC_API_KEY]

  - name: claude-code
    model_name: anthropic/claude-sonnet-4-5
    secrets: [ANTHROPIC_API_KEY]

tasks:
  - name: harbor/hello-world
    ref: latest
```

The final resolved config would be as follows:

```yaml theme={"system"}
job_name: simple-config
organization: my-org

agents:
  - name: claude-code
    model_name: anthropic/claude-opus-4-1
    secrets: [ANTHROPIC_API_KEY, TEST_VAR] # notice TEST_VAR is used in both agents

  - name: claude-code
    model_name: anthropic/claude-sonnet-4-5
    secrets: [ANTHROPIC_API_KEY, TEST_VAR] # notice TEST_VAR is used in both agents

tasks:
  - name: harbor/hello-world
    ref: latest
```

| Flag                | Value                 | Effect                                                                                                               |
| ------------------- | --------------------- | -------------------------------------------------------------------------------------------------------------------- |
| `--dry-run`         | —                     | Validate without queuing. Resolves tasks, agents, and the owning org, checks selections, and reports the trial count |
| `--org`             | `TEXT`                | Organization that should own the hosted job. Defaults to your personal org                                           |
| `--credential-mode` | `gateway` \| `direct` | `gateway` proxies the key through the Hub, `direct` hands the real key to the agent                                  |
| `--stored-secret`   | `NAME`                | Select a secret already stored in the owning org. Repeatable                                                         |
| `--one-off-secret`  | `NAME[=VALUE]`        | Supply a secret for this job only. Bare `NAME` reads it from your environment. Repeatable                            |
| `--env-file`        | `PATH`                | Load a `.env`. Every name in it is selected, as with `--one-off-secret`                                              |
| `--registry-secret` | `HOST=NAME_OR_ID`     | Pin a stored pull secret for a private image host. Repeatable                                                        |
| `--no-secrets`      | —                     | Launch with no credentials at all                                                                                    |

`--launch` and `--upload` are mutually exclusive, so the upload-side flags do not apply to a
hosted launch: `--upload`, `--public` / `--private`, `--share-org`, and `--share-user`.

## Listing Jobs

Using the command `harbor hub job list` you can print out a list of all jobs visible to your user on the hub.

| Flag         | Value                     | Effect                                                                  |
| ------------ | ------------------------- | ----------------------------------------------------------------------- |
| `--scope`    | `my` \| `shared` \| `all` | Visibility scope. Defaults to `my`, the jobs your user owns             |
| `--search`   | `TEXT`                    | Only jobs whose names contain the substring                             |
| `--agent`    | `NAME`                    | Filter by agent name. Repeatable, and multiple values match any of them |
| `--provider` | `NAME`                    | Filter by model provider. Repeatable                                    |
| `--model`    | `NAME`                    | Filter by model. Repeatable                                             |

## Job Overview

Use the command `harbor hub job show JOB_ID JOB_ID_2 ...` to print an overview similar to the one provided on the hub.

The overview reports the number of trials, errors, and retries, the average return on each
metric, the cost in USD, and token usage.

## Per-Task Job Breakdown

Use the command `harbor hub job tasks JOB_ID` to see a per-task breakdown of a job.

| Flag                               | Value  | Effect                                       |
| ---------------------------------- | ------ | -------------------------------------------- |
| `--search`                         | `TEXT` | Only tasks whose names contain the substring |
| `--agent`, `--provider`, `--model` | `NAME` | As for [`job list`](#listing-jobs)           |

## Trials Overview

Use the command `harbor hub job trials JOB_ID JOB_ID_2 ...` to list trials across one or more jobs.
In interactive terminals, you can switch pages to audit all trials without running a second command.

| Flag                               | Value                                                 | Effect                                               |
| ---------------------------------- | ----------------------------------------------------- | ---------------------------------------------------- |
| `--search`                         | `TEXT`                                                | Only trials whose names contain the substring        |
| `--agent`, `--provider`, `--model` | `NAME`                                                | As for [`job list`](#listing-jobs)                   |
| `--limit`                          | `N`                                                   | Page size. Defaults to 100 here                      |
| `--failed-only`                    | —                                                     | Only trials that errored or failed                   |
| `--include-retries`                | —                                                     | Include retry history, not just the latest execution |
| `--sort-by`                        | `started_at` \| `task_name` \| `name` \| `error_type` | Sort column                                          |
| `--sort-order`                     | `asc` \| `desc`                                       | Sort direction                                       |

## Comparing 2 or More Jobs

Use command `harbor hub job compare JOB_ID JOB_ID_2 ...` to get the side-by-side grid for performance on tasks

## Job Visibility

Use command `harbor hub job shares JOB_ID` to see who a job is shared with. This returns orgs and users

## Deleting Jobs

Use command `harbor hub job delete JOB_ID JOB_ID_2 ...` to permanently delete jobs you own from the hub, including all of their trials and shares.

The command prompts for confirmation before deleting anything; pass `--yes` / `-y` to skip the prompt (required when scripting or piping).

Only the job's owner can delete a job. Jobs linked to a leaderboard submission and hosted jobs that are still running cannot be deleted.

## Hosted Job Status

Use command `harbor hub job status JOB_ID` to get your job status. Returns counts of pending, running, failed, and completed trials.

## Trial Show

Use command `harbor hub trial show TRIAL_ID` to show a single trial's metadata

## Downloading a Trial

Use command `harbor hub trial download TRIAL_ID` to download a particular trial

| Flag                 | Value  | Effect                                                                        |
| -------------------- | ------ | ----------------------------------------------------------------------------- |
| `--output-dir`, `-o` | `PATH` | Directory to materialize the trial into. Defaults to `./trials`               |
| `--overwrite`        | —      | Replace an existing trial directory                                           |
| `--trajectory`       | —      | Download only `trajectory.json`. Errors if the trial has no stored trajectory |

## Retrying a Hosted Trial

For a trial in a job that was launched remotely, you can use the command `harbor hub trial retry TRIAL_ID` to retry those trials.

These flags are cumulative filters. `harbor hub trial retry --job JOB_ID --failed-only`
selects every trial in the job, then narrows to the ones that failed.

| Flag            | Value    | Selects                                  |
| --------------- | -------- | ---------------------------------------- |
| `--job`         | `JOB_ID` | Every trial in that job                  |
| `--search`      | `TEXT`   | Trials whose names contain the substring |
| `--agent`       | `TEXT`   | Trials run with that agent               |
| `--provider`    | `TEXT`   | Trials run with that provider            |
| `--model`       | `TEXT`   | Trials run with that model               |
| `--task`        | `TEXT`   | Trials run against that task             |
| `--exception`   | `TEXT`   | Trials that failed with that exception   |
| `--failed-only` | —        | Only trials that failed                  |
| `--yes`         | —        | Skip confirmation                        |

## Canceling a Hosted Trial

To cancel a hosted trial, use command `harbor hub trial cancel TRIAL_ID`

| Flag       | Value    | Effect                                                                |
| ---------- | -------- | --------------------------------------------------------------------- |
| `--job`    | `JOB_ID` | Cancel every trial in the job. Same as `harbor hub job cancel JOB_ID` |
| `--all`    | —        | Cancel every currently running trial                                  |
| `--reason` | `TEXT`   | Record a reason for the cancellation                                  |
| `--yes`    | —        | Skip confirmation                                                     |

The selection filters from [`trial retry`](#retrying-a-hosted-trial) apply here too.

## Adding Secrets

Use command `harbor hub secrets add NAME` to upload a secret to the Hub.

| Flag          | Value    | Effect                                                              |
| ------------- | -------- | ------------------------------------------------------------------- |
| `--org`       | `ORG_ID` | Organization to store the secret on. Defaults to your personal org  |
| `--job`       | `JOB_ID` | Scope the secret to a single job instead of account-wide            |
| `--from-env`  | —        | Read the value from the local environment variable of the same name |
| `--yes`, `-y` | —        | Supersede an existing secret without prompting                      |

## Listing Secrets

Use command `harbor hub secrets list` to list the names and metadata of your uploaded secrets.

| Flag                | Value    | Effect                                 |
| ------------------- | -------- | -------------------------------------- |
| `--org`             | `ORG_ID` | Organization to list secrets for       |
| `--job`             | `JOB_ID` | List secrets scoped to that job        |
| `--include-revoked` | —        | Include secrets that have been revoked |

## Deleting Secrets

Use command `harbor hub secrets delete` to revoke a secret.

| Flag      | Value    | Effect                                                                             |
| --------- | -------- | ---------------------------------------------------------------------------------- |
| `--org`   | `ORG_ID` | Revoke a secret in that organization                                               |
| `--job`   | `JOB_ID` | Revoke a secret scoped to that job                                                 |
| `--purge` | —        | Delete the record outright, so it no longer appears under `list --include-revoked` |

## Adding an Image Registry Secret

Use command `harbor hub secrets registry add` to add an image registry secret. Also supports `secrets registry list` and `secrets registry delete`.

| Flag          | Value  | Effect                                                              |
| ------------- | ------ | ------------------------------------------------------------------- |
| `--name`      | `TEXT` | Display name used to select the credential later                    |
| `--from-file` | `PATH` | Read the credential from a file, such as a GAR service account JSON |
| `--yes`, `-y` | —      | Supersede an existing credential without prompting                  |

## Shared Flags

The listing commands above also accept these:

| Flag            | Value | Effect                                                 |
| --------------- | ----- | ------------------------------------------------------ |
| `--quiet`, `-q` | —     | Print only IDs, for piping into `xargs`                |
| `--no-trunc`    | —     | Show full cell content, wrapping instead of truncating |
| `--no-headers`  | —     | Omit the header row                                    |
| `--page`        | `N`   | Fetch one specific page, disabling interactive paging  |
| `--json`        | —     | Return the raw API response as JSON                    |