error_policy.py is the single source of truth for trial errors. Every exception type maps to one cause, and the cause fixes three decisions that every part of Harbor follows: whether the trial is retried, whether the verifier still runs (grade), and whether the reward is valid. A valid reward is one that counts in benchmark and RL results, whether its value is 0 or 1. A trial with no error always has a valid reward. A trial with an error usually does not, but errors caused by the agent itself, such as running out of time, still produce a valid reward.
Exception types
Generated from_POLICY_GROUPS in error_policy.py. Exception types not listed are unclassified.
retry.include_exceptions and retry.exclude_exceptions in the job config override the retry column per job.