Benchmark: Visual multi-session SLAM under lighting, season and scene change
CROSS gives the best visual multi-session relocalization in this
benchmark: from a cold start in a map from another session, it finds its place far more often than
open-source SLAM systems run on the same sessions. It also ranks first in localization and mapping.
The figures show CROSS with a stereo camera and odometry (stereo_wheel), the
recommended mode for these sensors. RTAB-Map is the other system that takes the same odometry: wheel odometry on
OpenLORIS, close to the ground truth elsewhere (INS velocities on KITTI, ground truth plus noise on ROVER and
SimChange). The leaderboard below ranks each system's best configuration instead.
Leaderboard
Each system's best configuration, ranked; filter by your robot's sensors
Sensor
Inputs
View
Relocalization success T3
Relocalization (T3): relocalization success, higher is better
Small type: the strict radius, 1 m indoors and 3 m outdoors.
Upper bound (hatched bar): LingBot-Map has no relocalization step, so all its query poses count.
k/N failed: k of N query sessions failed (a crash, a timeout or a failed map); their trials count as failed.
Trials scored: 1,998–2,009 for CROSS, ORB-SLAM3 and RTAB-Map; 409 for VGGT-SLAM 2.0; 315 for MASt3R-SLAM, LingBot-Map and AMB3R-SLAM. Systems that cannot store a map re-run the whole map session for every trial, so they are scored on a subset.
Localization in a stored map T2
Localization in a stored map (T2): localization recall, higher is better
ORB-SLAM3RGB-DCamera onlyKITTI –OpenLORIS 0.72 m (7/22 failed)ROVER 0.95 m (3/8 failed)
–no KITTI run
–
0.72 m7/22 failed
0.95 m3/8 failed
–not ranked
ORB-SLAM3MonoCamera onlyKITTI 16.0 m (1/10 failed)OpenLORIS 3.19 m (13/22 failed)ROVER – (8/8 failed)
–22/40 failed
16.0 m1/10 failed
3.19 m13/22 failed
–8/8 failed
–
–not ranked
ORB-SLAM3visual-inertialRGB-D+ IMUKITTI –OpenLORIS 3.39 m (6/22 failed)ROVER 33.5 m (3/8 failed)
–no KITTI run
–
3.39 m6/22 failed
33.5 m3/8 failed
–not ranked
ORB-SLAM3visual-inertialStereo+ IMUKITTI 3.92 m (3/10 failed)OpenLORIS 0.54 m (4/22 failed)ROVER 76.4 m (2/8 failed)
8.39%9/40 failed
3.92 m3/10 failed
0.54 m4/22 failed
76.4 m2/8 failed
–not ranked
ORB-SLAM3visual-inertialMono+ IMUKITTI – (10/10 failed)OpenLORIS 1.94 m (14/22 failed)ROVER 11.3 m (7/8 failed)
–31/40 failed
–10/10 failed
1.94 m14/22 failed
11.3 m7/8 failed
–
–not ranked
RTAB-Mapdataset odometryRGB-D+ odometryKITTI –OpenLORIS 1.41 mROVER 1.96 m
–no KITTI run
–
1.41 m
1.96 m
–not ranked
RTAB-MapBasalt stereo VIORGB-D+ odometryKITTI –OpenLORIS 1.41 mROVER 2.34 m
–no KITTI run
–
1.41 m
2.34 m
–not ranked
RTAB-MapBasalt stereo VIOStereo+ odometryKITTI 3.57 mOpenLORIS 1.61 mROVER 2.19 m
0.80%
3.57 m
1.61 m
2.19 m
–not ranked
RTAB-Mapvisual odometryRGB-DCamera onlyKITTI –OpenLORIS 5.16 m (1/22 failed)ROVER 4.09 m (1/8 failed)
–no KITTI run
–
5.16 m1/22 failed
4.09 m1/8 failed
–not ranked
RTAB-Mapvisual odometryStereoCamera onlyKITTI 6.29 mOpenLORIS 4.51 m (1/22 failed)ROVER 1.23 m (1/8 failed)
1.49%2/40 failed
6.29 m
4.51 m1/22 failed
1.23 m1/8 failed
–not ranked
RTAB-Mapvisual-inertial odometryRGB-D+ IMUKITTI –OpenLORIS 5.07 m (1/22 failed)ROVER 3.39 m (1/8 failed)
–no KITTI run
–
5.07 m1/22 failed
3.39 m1/8 failed
–not ranked
RTAB-Mapvisual-inertial odometryStereo+ IMUKITTI 6.64 mOpenLORIS 4.49 m (1/22 failed)ROVER 0.82 m (1/8 failed)
1.44%2/40 failed
6.64 m
4.49 m1/22 failed
0.82 m1/8 failed
–not ranked
AMB3R-SLAMStereoCamera onlyKITTI 59.5 mOpenLORIS 2.68 mROVER 12.2 m
3.06%
59.5 m
2.68 m
12.2 m
–not ranked
AMB3R-SLAMRGB-DCamera onlyKITTI –OpenLORIS 2.61 mROVER 4.33 m
–no KITTI run
–
2.61 m
4.33 m
k/N failed: k of N sequences failed: tracking lost (under 80% of the frames covered), a crash or a timeout. The mean leaves them out, so these rows are not ranked.
Scale error: how far a monocular map's scale is from metric. Monocular systems without metric input are aligned with Sim(3), so their ATE leaves this error out.
RGB-D on KITTI: KITTI has no depth camera. CROSS uses depth from stereo matching there; the baselines do not run KITTI's RGB-D setup, so their RGB-D rows have no overall value.
Numbers from the published benchmark of 2026-10-25 (commit
2b560ce): 6,368 of its
12,491 runs score these tables. Per-scene tables, the other runs, trajectories and
failure cases: full results.
What we measure
Three tasks, and how every system is run
Three tasks under one protocol:
the same recordings and radii for every system, and the same odometry for every system that takes it.
T3 Relocalization
Can it find its place from a cold start?
Each query session is cut into 10 s trials: 100 frames at 10 Hz, a new trial every 50 frames. A
trial loads the stored map and starts without knowing its pose. It succeeds when its final pose lies within the
radius of the ground truth.
Indoors1 m · 2 m
Outdoors3 m · 5 m
T2 Localization in a stored map
How much of a query session is localized?
A query session (another recording of the same place) runs once against the stored map, from
its first frame and with no initial pose. Recall is the share of its frames that have a pose in the map within the
radius; a frame without one counts as a miss.
Indoors1 m · 2 m
Outdoors3 m · 5 m
T1 Mapping accuracy
How accurate is the map of one session?
One session is mapped from scratch. The error is the RMSE of the final trajectory after loop
closure (ATE), aligned to the ground truth and divided by the length of the path. For CROSS the trajectory is its
map keyframes, which are sparser than the other systems' keyframes.
DatasetsKITTI, OpenLORIS, ROVER
Failure< 80% of frames covered
Each dataset weighs the same in an overall score: the mean of OpenLORIS, ROVER and
SimChange v3 for localization and relocalization (v3 replaced v2 on 25 October 2026; v2's tables are in the
full results), and of KITTI, OpenLORIS and ROVER for mapping, where the error per path
length lets kilometer drives and room-scale sessions compare. Localization and relocalization score only the frames
that the map covers and in which the robot moves. Bold: the radius of the NeurIPS 2026 paper,
in large type in the tables.
How every system is run
Same inputs
Every system gets the same images at 10 Hz in real-time order. Every system that takes odometry gets the same
stream: the dataset's odometry or a Basalt stereo VIO (OpenLORIS keeps its wheel odometry in the VIO rows).
Shipped defaults
Each system runs its published defaults for the sensor, with no per-dataset tuning; the protocol lists the
declared deviations. One is not listed there: outdoors (KITTI, ROVER), the CROSS row with Depth Anything 3 and
visual odometry uses a different odometry-noise setting.
Map reuse
CROSS saves and loads its map, ORB-SLAM3 loads and merges its atlas, and RTAB-Map loads its database in
localization mode. MASt3R-SLAM, VGGT-SLAM 2.0, LingBot-Map and AMB3R-SLAM cannot store a map, so the map session
and the query run as one stream, which keeps the map session's live state.
Long-term relocalization with a probabilistic topological map: a keyframe graph with
verified loop closures, and the camera pose tracked as a mixture of hypotheses until the evidence separates them.
Relative poses come from VGGT-Omega (with any camera), PnP on depth (RGB-D) or the Depth Anything 3 back end
(mono). MIT-licensed code; the VGGT-Omega and Depth Anything 3 weights are for non-commercial use only, and the
RGB-D modes with PnP use neither.
Sensors
RGB-D, stereo or mono · camera only, + IMU or + odometry