Recorded runsGitHub
Evidence

Measured, not believed

Every verdict is judged against simulator ground truth, where the robot actually went, never against its own odometry alone. These are the gate's own reports from real runs in Gazebo Sim, headless, on Oct 26, 2026.

Locked thresholds

MeasurementPassMarginalFail
Distance error≤ 5%5–10%> 10%
Heading error≤ 5°5–10°> 10°

Printed on every report and never quietly loosened.

How a run is graded

The twin is commanded to drive 1 m and turn 90°. Drive and turn are graded against the simulator's true pose. Odometry is graded against that same truth, so a robot whose sensors are wrong about itself can't pass. When something is off, the gate names the spec field to look at first.

Gate reports

Two clean passes and two failures the gate caught. Each report shows every check, the raw measurements, the ground-truth path, and any suspects the gate named.

Indoor Delivery Rover

Described in words, never drawn

Pass

This robot existed only as one paragraph of plain English at 4:52 PM. Muse Spark turned it into a validated spec, the forge built its twin, and the gate measured it: PASS, with odometry matching ground truth to 0.002% over the drive. The full chain, one flow.

Drive vs ground truthPass
0.398%
0pass ≤5%fail >10%15
Turn vs ground truthPass
3.653°
0pass ≤5°fail >10°15
Odometry distance vs truthPass
0.002%
0pass ≤5%fail >10%15
Odometry heading vs truthPass
1.088°
0pass ≤5°fail >10°15
Drive: truth / odometry1.004 m / 1.004 m
Turn: truth / odometry93.7° / 92.6°

White dot: start. Amber dot: where the robot actually ended up (ground truth).

No suspects. Odometry and ground truth agree inside tolerance on every check.
TurtleBot3 Burger

As specified

Pass

A differential-drive robot built to the public ROBOTIS spec. Nothing injected, nothing hidden.

Drive vs ground truthPass
0.968%
0pass ≤5%fail >10%15
Turn vs ground truthPass
2.028°
0pass ≤5°fail >10°15
Odometry distance vs truthPass
0.042%
0pass ≤5%fail >10%15
Odometry heading vs truthPass
4.548°
0pass ≤5°fail >10°15
Drive: truth / odometry1.010 m / 1.009 m
Turn: truth / odometry88.0° / 92.5°

White dot: start. Amber dot: where the robot actually ended up (ground truth).

No suspects. Odometry and ground truth agree inside tolerance on every check.
TurtleBot3 Burger

Wheels 10% smaller than the paperwork

Fail

Fault injection: the physical wheels were scaled to 0.9x while odometry kept believing the spec. The robot's own sensors could not see it. The gate could.

Drive vs ground truthMarginal
7.617%
0pass ≤5%fail >10%15
Turn vs ground truthMarginal
6.766°
0pass ≤5°fail >10°15
Odometry distance vs truthFail
11.168%
0pass ≤5%fail >10%15
Odometry heading vs truthFail
11.041°
0pass ≤5°fail >10°15
Drive: truth / odometry0.924 m / 1.027 m
Turn: truth / odometry83.2° / 94.3°
Injected faultphysical wheels × 0.9 (odometry keeps the spec size)

White dot: start. Amber dot: where the robot actually ended up (ground truth).

Suspect: wheel.diameter_m
odometry distance disagrees with ground truth while heading agrees — a scale error: the wheel radius odometry uses does not match the physical wheels
Suspect: wheel_separation (robot width / wheel track)
odometry heading disagrees with ground truth — the track width odometry integrates over does not match the physical robot (or the wheels scrub/slip in turns)
Pathfinder skid-steer

The turn that wasn't

Fail

Odometry reported a near-perfect drive. In the turn, its wheels scrubbed: it believed 94.6 degrees, the ground truth was 30.6. Caught only because the gate measures truth, not belief.

Drive vs ground truthPass
1.550%
0pass ≤5%fail >10%15
Turn vs ground truthFail
59.368°
0pass ≤5°fail >10°15
Odometry distance vs truthPass
0.000%
0pass ≤5%fail >10%15
Odometry heading vs truthFail
64.003°
0pass ≤5°fail >10°15
Drive: truth / odometry1.016 m / 1.015 m
Turn: truth / odometry30.6° / 94.6°

White dot: start. Amber dot: where the robot actually ended up (ground truth).

Suspect: wheel_separation (robot width / wheel track)
odometry heading disagrees with ground truth — the track width odometry integrates over does not match the physical robot (or the wheels scrub/slip in turns)

Robodex verdicts

The current verdict for each robot and its latest recorded run. Failures stay on the record exactly as measured.

RobotVerdictRunsLatest driveLatest turn
AndinoFail20.347%9.104°
TurtleBot3 BurgerPass20.968%2.028°
TurtleBot3 Waffle PiNot gated0——
PathfinderPass21.375%2.205°
PX4 X500Not gated0——
Crazyflie 2.1Not gated0——
Unitree Go2Not gated0——
Mini PupperNot gated0——
Unitree G1Not gated0——
ROBOTIS OP3Not gated0——
OpenManipulator-XNot gated0——

Static stability checks

A different kind of evidence, kept separate from the gate reports above: for the legged robots, a computation from the spec — the support polygon its feet make, and how far the robot can tip before its centre of mass passes over the edge. Nothing here was simulated or driven, and a static check never changes a gate verdict: every robot below is still Not gated. Method, inputs and assumptions are printed in each report.

RobotStatic checkTip angleWeakest marginDirectionCheckedReport
Unitree Go2Static check: STABLE — tip margin 32°31.8°136 mmto the right2026-10-26report
Mini PupperStatic check: STABLE — tip margin 28°27.9°48 mmto the right2026-10-26report
Unitree G1Static check: UNSTABLE — tip margin 3°2.7°34 mmbackward2026-10-26report
ROBOTIS OP3Static check: UNSTABLE — tip margin 3°2.7°13 mmbackward2026-10-26report

Harness checks

A third kind of evidence, kept separate from gate runs and static checks: electrical computations over each robot's documented wiring graph — voltage drop, ampacity, logic levels, rail budgets, connector matching. The headline is never a blanket pass; it is counts. A check that could not run says which data it is missing, and that gap list is printed in full on each robot's page: it is the shopping list of datasheets still to hunt.

RobotHarness checksCleanWarnErrPartialNot checkedDateReport
AndinoHarness checks: 0 clean · 0 warn · 0 err · 34 not checked0000342026-10-26report
TurtleBot3 BurgerHarness checks: 0 clean · 0 warn · 0 err · 19 not checked0000192026-10-26report
TurtleBot3 Waffle PiHarness checks: 0 clean · 0 warn · 0 err · 23 not checked0000232026-10-26report
PathfinderHarness checks: 0 clean · 0 warn · 0 err · 21 not checked0000212026-10-26report
PX4 X500Harness checks: 0 clean · 0 warn · 0 err · 17 not checked0000172026-10-26report
Crazyflie 2.1Harness checks: 0 clean · 0 warn · 0 err · 8 not checked000082026-10-26report
Mini PupperHarness checks: 0 clean · 0 warn · 0 err · 8 not checked000082026-10-26report
ROBOTIS OP3Harness checks: 0 clean · 0 warn · 0 err · 10 not checked0000102026-10-26report
OpenManipulator-XHarness checks: 0 clean · 0 warn · 0 err · 10 not checked0000102026-10-26report