Skip to content
Tbench AI
Terminal-Bench · audit contract · resolves to tb3

Terminal-Bench 3

The contract the audit defaults to.

TB3 is the default audit profile. It defines the conditions for the task, execution environment and verifier, with open agent egress and separate verifier execution.

Status
shipped
Agent egress
Open by contract
What this profile requires

The conditions a task is audited under.

Read from src/infinity/profile.py, which resolves the profile once at load time so every check reads it from the bundle rather than each deciding for itself.

Agent egress
Open. The benchmark grants the agent the internet by contract, so open egress is the specification rather than a defect to report.
Verifier
Separate container, held to no network regardless of what the agent may reach. Harbor makes the separate verifier opt-in; the audit treats it as required.
Difficulty evidence
Not required. The difficulty check is silent on TB3.
Profile scope

TB3 evaluates task and verifier assurance. Model difficulty qualification is a separate evaluation and is not established by this profile.

Shared with every profile

Shared assurance requirements.

  • The verifier runs apart from the agent

    Harbor makes a separate verifier sandbox opt-in. The audit treats it as required, because a verifier sharing a filesystem with the thing it is grading has no boundary left to test.

    S11 in src/infinity/checks/separate.py

  • Complete required coverage before issuance

    The report separates completed checks from skipped and errored stages. Every required check must complete before a mark can be issued.

    grade() in platform/web/lib/certification.ts

  • The result names the revision it was graded under

    Each result retains the methodology revision used for its audit, so later changes remain distinct from the original evidence.

    METHODOLOGY_VERSION = 1.0 in platform/web/lib/brand.ts

Where to check all of this

Inspect the implementation sources.

Inspect the source definitions for the profile, checks and reference inventory.

  • src/infinity/profile.pyProfile.TB3, the open-egress set, and the alias table
  • src/infinity/checks/separate.pyS11, the separate-verifier-container check
  • platform/backend/app/services/verification.pyThe twelve task stages and their prerequisites
Start here

Submit the task. Get its recorded check evidence.

Tbench AI audits the task, environment, reference solution and verifier. Every completed audit retains its evidence report.

Apply for an accountVerify a result