Three release names. Two contracts.
Explore the audit contract each Terminal-Bench profile resolves to, its task and verifier requirements, and the source evidence maintained by Tbench AI.
Contracts, coverage and traceable evidence.
Explore the audit requirements for each profile and the recorded reference inventory. Issued audit outcomes are available through the public evidence register.
Result evidence identifies the checks performed, required coverage, source revision and date.
- 01 · Audit contract
- Resolves each task manifest to its versioned schema, isolation rules, and required stage gates.
- 02 · Public evidence register
- Issued marks verify live at /verify with recorded coverage, stance, methodology version, and expiry.
- 03 · Licence-indexed corpus
- Tracks novelty against public Harbor task directories while preserving each repository licence boundary.
What a submission is held to.
Terminal-Bench 3
shippedThe shipped contract: open agent egress, a sealed verifier, witness re-execution.
Read the contractTerminal-Bench 4
aliasThe shipped release. Same task format and egress contract as TB3, so it resolves to TB3.
Read the contractTerminal-Bench 5
Additional difficulty-evidence declarations for the next-generation audit profile.
Read the contract
One shared assurance standard.
TB3, TB4 and TB5 share these requirements. Each requirement links to its implementation source.
The verifier runs apart from the agent
Harbor makes a separate verifier sandbox opt-in. The audit treats it as required, because a verifier sharing a filesystem with the thing it is grading has no boundary left to test.
S11 in src/infinity/checks/separate.pyComplete required coverage before issuance
The report separates completed checks from skipped and errored stages. Every required check must complete before a mark can be issued.
grade() in platform/web/lib/certification.tsThe result names the revision it was graded under
Each result retains the methodology revision used for its audit, so later changes remain distinct from the original evidence.
METHODOLOGY_VERSION = 1.0 in platform/web/lib/brand.ts
A source-indexed reference inventory.
Infinity uses the public Harbor reference inventory for novelty checks. It contains 1135 recorded markers; 404 adapters and CI fixtures are excluded, leaving 731 task directories. Each exclusion has a recorded reason.
- Task directories
- 731
- Harbor manifest format
- 490
- Legacy manifest format
- 241
- Excluded, with reasons
- 404
| Repository | Licence | Tasks |
|---|---|---|
terminal-bench | Apache-2.0 | 156 |
terminal-bench-2 | Apache-2.0 | 89 |
terminal-bench-2-1 | Apache-2.0 | 89 |
terminal-bench-1 | Apache-2.0 | 241 |
terminal-bench-science | Apache-2.0 | 72 |
harbor | Apache-2.0 | 0 |
terminal-bench-challenges | No licence | 3 |
harbor-index | No licence | 80 |
benchmark-template | No licence | 1 |
The survey stores reference metadata — paths, counts and URLs. Task content remains under its source repository’s licence. Regenerate the inventory with python corpus/tb5/inventory.py.
Evidence-led profile selection.
The reference snapshot records next-generation intent from pull-request descriptions. The entries below preserve the declared task, author, date and schema; they are reference records rather than a distributed TB5 task set.
| Task | Author | Opened | Declared schema |
|---|---|---|---|
budgeted-replica-reconciliation | Numan5837 | 2026-09-14 | 2.0 |
orchestral-transcription | linhaowei1 | 2026-09-14 | 1.2 |
The submissions use different manifest schemas. Profile selection follows declared difficulty evidence across schema versions.
Audit a task against the contract it claims.
One task or a complete set. The profile is resolved at load time from the task’s own declaration. Issuance requires complete required-stage coverage.