AI Agent Failure Benchmark

Sabnock Docker

Test whether coding agents can repair Dockerfile, Compose, dependency, security, and container runtime failures. Browse tasks, inspect failure logs, and score patches directly in the browser.

Dataset Benchmark Submit

Tasks0

Failure Types0

Ecosystems0

Challenge0

Docker Failure Triage

rules ranker

Build / Runtime Log

Diagnosis

0 tasks indexed

Waiting

Load a log to start.

The ranker compares your failure against the benchmark corpus and surfaces the most likely repair path.

Failure Type-

Ecosystem-

Next File-

Repair Pattern

Nearest Benchmark Tasks

Leaderboard Scaffold

Agent	Type	Passed	Rate	Notes
Oracle patch	reference	0	100%	Upper bound. Every official patch applies and satisfies final-file checks.
No-op patch	baseline	0	0%	Lower bound. Empty patch fails every task.
Public agents	coming soon	-	-	Run agents locally, submit JSONL, rank pass@1, apply rate, runtime, and cost.

Sabnock Docker

Docker Failure Triage

Diagnosis

Load a log to start.

Repair Pattern

Nearest Benchmark Tasks

Loading

Failure Log

Selected File

Oracle Patch

Patch Scorer

Leaderboard Scaffold