Trajectory replays: Mapping and multi-session localization runs from the benchmark

Watch CROSS and 6 other SLAM systems map a session, then try to localize later sessions in the stored map, recorded in another season, at night or after the furniture moved.

Replay viewer

Scene one map, other sessions localize in it

  • 01 ROVER campus map: day, 2024-09-25 · query sessions: summer, autumn, winter, spring, dusk, night, night with lights
  • 02 OpenLORIS corridor map: session 1 · query sessions 2 to 5
  • 03 OpenLORIS market map: session 1 · query sessions 2 and 3
  • 04 OpenLORIS home map: session 1 · query sessions 2 to 5
  • 05 OpenLORIS café map: session 1 · query session 2
  • 06 OpenLORIS office map: session 1 · query sessions 2 to 7

Query session map: Day (2024-09-25)

Sequence one session, mapped from scratch

  • 01 ROVER campus in winter outdoor robot, 295 m, low winter sun
  • 02 ROVER campus at night outdoor robot, 280 m, night
  • 03 KITTI 00 car, 3.7 km, loops
  • 04 KITTI 05 car, 2.2 km, loops
  • 05 OpenLORIS corridor indoor robot, 222 m
  • 06 OpenLORIS market indoor robot, 150 m
  • 07 OpenLORIS home indoor robot, 40 m

ROVER campus

Map: Day (2024-09-25) → query: Night (2024-09-24)

    Sensor and inputs the same for every system shown

    Sensor
    Inputs
    Success radius

    Systems

    Final recall within 5 m

      Select a system to show or hide it.

      All runs on this query session, as a table

      Grouped by sensor. A colored edge marks the runs in the viewer.

      How to read

      What the replays draw, and how the benchmark scores it

      01Localization in a stored map

      Each system that can save a map builds one from the map session. The query session, another recording of the same place, then runs once from its first frame, with no initial pose.

      The route is colored by one system's state: localized within the radius, wrong (beyond it), or no pose in the map yet. As in the benchmark, only frames the map covers and where the robot moves are scored. Final recall is the share of them localized within the radius.

      • One streamSystems that cannot save a map run the map session and the query session as one stream. MASt3R-SLAM, VGGT-SLAM 2.0 and AMB3R-SLAM log their poses after their final optimization, which favors them.
      • Upper boundLingBot-Map has no relocalization step, so every query pose counts: its numbers are upper bounds.

      02Mapping one session

      Each system maps the session on its own. The replay draws its final trajectory, after all loop closures, along the session's timeline.

      The trajectory is aligned to the ground truth (dashed) as the benchmark scores it: rigidly (SE(3)), or with scale (Sim(3)) for the monocular baselines, which get no metric input. CROSS's monocular runs estimate metric scale and are aligned rigidly, which is stricter. ATE is the root-mean-square position error of that trajectory.

      • Partial runsA run that tracked only part of the session shows only that part and stops where its estimate ends (×). Below 80% of the session it counts as failed.
      • RevisitsCROSS's path is its permanent map keyframes, the poses the benchmark scores. While it re-drives a stretch it has mapped it keeps only temporary keyframes, so its path has a gap there: the replay leaves the gap open and holds CROSS's marker, hollow, on its last keyframe.
      • TimingThe published paths carry no timestamps, so the replay times each estimate by matching it, in order, to the ground truth.

      03Same sensor, same inputs

      Each view compares runs with the same sensor and the same kind of input: the images alone, or the images with an IMU or with odometry.

      Camera only is the fairest comparison: every system gets the same images and nothing else. + IMU adds the platform's IMU (visual-inertial). + odometry adds the dataset's odometry, or a Basalt stereo VIO stream for the configurations that use one; where a system has several configurations, its row in the Systems list picks one.

      • ROVERThe dataset odometry is simulated, as in the benchmark: ground-truth increments with noise (SNR 10).
      • KITTIDead reckoning of the car's OXTS (GPS/INS) velocities and yaw rate, without the GPS positions, so it is close to the ground truth.
      • OpenLORISThe robot's own wheel odometry.

      Data: the published benchmark results of 2026-10-25 (commit 2b560ce), 12,491 runs in all. Every run replayed here is in the full results (select a table cell to see its runs), and the benchmark ranks the systems; the rules are in the protocol.