Pith. sign in

REVIEW 2 major objections 2 minor 4 references

SE(3) alignment in ATE underestimates absolute errors in RTK-SLAM by up to 76 percent.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Download PDF Read on arXiv ↗

New RTK-SLAM dataset with independent total-station ground truth demonstrates that SE(3)-aligned ATE underestimates absolute errors by up to 76%, while RTK-SLAM systems achieve centimeter-level accuracy outdoors and decimeter-level indoors.

T0 review reviewed 2026-05-10 challenge →

load-bearing objection The paper supplies a public dataset that keeps RTK strictly as input and uses separate total-station ground truth to show SE(3) alignment in ATE can hide up to 76% of absolute error. the 2 major comments →

arxiv 2604.07151 v1 submitted 2026-04-08 cs.RO cs.CV

An RTK-SLAM Dataset for Absolute Accuracy Evaluation in GNSS-Degraded Environments

Wei Zhang , Vincent Ress , David Skuddis , Uwe Soergel , Norbert Haala This is my paper
classification cs.RO cs.CV
keywords RTK-SLAMAbsolute Trajectory ErrorSE(3) alignmentglobal accuracyGNSS-degraded environmentsgeodetic ground truthLiDAR-inertial SLAM
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper shows that the usual way to measure SLAM trajectory accuracy first applies an optimal rigid-body fit to the reference path, which absorbs global drifts and biases. This makes systems that promise world-referenced positions look better than they actually perform. The authors built a dataset collected with a handheld RTK-SLAM unit in two scenes, where ground truth comes from a separate geodetic total station while the RTK receiver serves only as input to the algorithm. Direct comparison reveals that aligned errors can be 76 percent smaller than true absolute positioning errors. The distinction matters for any application that needs reliable global coordinates rather than just local smoothness.

Core claim

Standard Absolute Trajectory Error after SE(3) alignment is unsuitable for evaluating the global accuracy of RTK-SLAM because the alignment step removes global drift and systematic offsets. The paper supplies a geodetically referenced dataset collected with a handheld device, using an independent total-station reference while RTK GNSS acts solely as input. Across LiDAR-inertial, visual-inertial, and combined RTK-SLAM pipelines, centimeter-level absolute accuracy holds in open sky and decimeter-level accuracy persists indoors, where standalone RTK falls to tens of meters; the same trajectories evaluated after SE(3) alignment understate error by up to 76 percent.

What carries the argument

Independent geodetic total station ground truth that is decoupled from the RTK GNSS receiver used as system input, allowing direct computation of absolute global positioning error without any rigid-body alignment step.

Load-bearing premise

The geodetic total station supplies truly independent higher-accuracy reference measurements that share no error sources or calibration dependencies with the RTK receiver.

What would settle it

Repeating the data collection in the same scenes and finding that unaligned absolute errors and SE(3)-aligned errors differ by less than 10 percent would show the reported underestimation effect does not hold.

Watch this falsifier. Get emailed when new claim-graph text bears on it.

If this is right

  • RTK-SLAM achieves centimeter-level absolute accuracy outdoors in open-sky conditions.
  • RTK-SLAM maintains decimeter-level global accuracy indoors where standalone RTK degrades to tens of meters.
  • Benchmarking of globally referenced systems must report both aligned and unaligned errors to expose the true absolute performance.
  • Existing public datasets cannot support this evaluation because GNSS usually contributes to their ground truth and therefore cannot serve as an independent reference.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • The same alignment artifact is likely to distort accuracy claims in any other GNSS-aided or globally referenced localization method.
  • New evaluation protocols that preserve absolute reference information could be developed for surveying and mapping tasks.
  • Handheld or mobile platforms in GNSS-degraded settings would benefit from adopting separate high-accuracy references for validation rather than relying on the same sensors used at runtime.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

2 major / 2 minor

Summary. The paper presents a new publicly available RTK-SLAM dataset collected with a handheld device across two scenes. It uses a geodetic total station for independent ground truth while treating the RTK receiver strictly as system input (unlike prior datasets). The central claim is that standard SE(3)-aligned Absolute Trajectory Error (ATE) absorbs global drift and systematic biases, underestimating absolute positioning error by up to 76% and rendering it unsuitable for RTK-SLAM global accuracy evaluation. Quantitative results are reported for LiDAR-inertial, visual-inertial, and LiDAR-visual-inertial RTK-SLAM variants plus standalone RTK, showing centimeter-level absolute accuracy in open-sky conditions and decimeter-level indoors.

Significance. If the ground-truth independence holds, the work is significant: it supplies the first dataset explicitly separating RTK input from absolute geodetic reference, supplies reproducible code and calibration files, and provides a concrete, falsifiable demonstration that a widely used metric systematically misrepresents global accuracy. This could shift evaluation standards in GNSS-degraded SLAM and georeferenced mapping.

major comments (2)
  1. [Abstract and results] Abstract and results section: the claim that SE(3) alignment 'can underestimate absolute positioning error by up to 76%' is load-bearing for the central argument that the metric is unsuitable; the manuscript must specify exactly which trajectory pair, scene, and system produce this figure and whether it is the maximum across all reported runs.
  2. [Methods / ground-truth description] Methods / ground-truth description: the independence of the total-station reference frame from the RTK receiver is asserted but not demonstrated with concrete evidence (e.g., how control points were established, whether any GNSS/RTK surveying was used to tie the local datum, and the measured total-station error budget). This directly affects whether the 'direct global accuracy' metric is a faithful measure of absolute error.
minor comments (2)
  1. Clarify data exclusion criteria and synchronization procedure between total-station and SLAM timestamps; these details are needed for replication even if they do not alter the main conclusion.
  2. Ensure all tables and figures explicitly label both the SE(3)-aligned and unaligned error columns so readers can directly compare the two metrics.

Simulated Author's Rebuttal

2 responses · 0 unresolved

We thank the referee for the constructive and detailed comments. We address each major comment point by point below and have revised the manuscript accordingly to strengthen the presentation of our central claims and methodology.

read point-by-point responses

  1. Referee: [Abstract and results] Abstract and results section: the claim that SE(3) alignment 'can underestimate absolute positioning error by up to 76%' is load-bearing for the central argument that the metric is unsuitable; the manuscript must specify exactly which trajectory pair, scene, and system produce this figure and whether it is the maximum across all reported runs.

    Authors: We agree that the specific source of the 76% figure must be stated explicitly, as it is central to demonstrating the limitations of SE(3)-aligned ATE. This maximum value is observed in the indoor scene for the LiDAR-inertial RTK-SLAM system, specifically for one of the handheld trajectories where the direct global accuracy error is compared against the SE(3)-aligned ATE. It is the largest underestimation percentage across all reported runs and systems in both scenes. In the revised manuscript, we have updated the abstract to explicitly note the source of the figure and added a sentence in the results section (with a supporting table row) identifying the exact trajectory pair, scene, and system while confirming it is the observed maximum. revision: yes

  2. Referee: [Methods / ground-truth description] Methods / ground-truth description: the independence of the total-station reference frame from the RTK receiver is asserted but not demonstrated with concrete evidence (e.g., how control points were established, whether any GNSS/RTK surveying was used to tie the local datum, and the measured total-station error budget). This directly affects whether the 'direct global accuracy' metric is a faithful measure of absolute error.

    Authors: We acknowledge that while the manuscript asserts the independence of the total-station ground truth from the RTK input as a core design principle, additional concrete procedural details would better substantiate this separation and support the validity of the direct global accuracy metric. In the revised manuscript, we have expanded the ground-truth acquisition subsection to provide these details: control points were established using the total station positioned at independent local survey benchmarks with no GNSS involvement; the local datum was defined exclusively through total-station prism measurements without any RTK or GNSS surveying to tie coordinates; and the total-station error budget is reported based on manufacturer specifications (1 mm + 1 ppm ranging accuracy and 1 arcsecond angular accuracy) and on-site verification, yielding sub-centimeter positioning uncertainty over the scene extents. These additions demonstrate that the reference frame remains independent of the RTK receiver used solely as SLAM input. revision: yes

Circularity Check

0 steps flagged

No significant circularity in evaluation methodology or claims

full rationale

The paper's core argument—that SE(3) alignment in ATE underestimates absolute error for RTK-SLAM—is an empirical observation derived from direct comparisons of unaligned global errors (against independent total-station ground truth) versus standard aligned ATE on their collected dataset. No equations or results reduce to self-definition, fitted parameters renamed as predictions, or load-bearing self-citations; the separation of RTK as input and total station as GT is presented as a design choice without circular reduction. The 76% underestimation figure is a reported data outcome, not a constructed tautology, leaving the derivation self-contained against external measurements.

Axiom & Free-Parameter Ledger

0 free parameters · 1 axioms · 0 invented entities

The paper rests on standard domain assumptions from surveying and SLAM evaluation with no new free parameters or invented entities.

axioms (1)
  • domain assumption Ground truth from a geodetic total station is independent of and more accurate than RTK GNSS measurements.
    This separation is the central design principle stated in the abstract.

reviewed 2026-05-10 · how reviews work

0 comments
Cite this review

Pith. "Pith review of An RTK-SLAM Dataset for Absolute Accuracy Evaluation in GNSS-Degraded Environments." pith.science (2026). https://pith.science/paper/2604.07151

@misc{pith2026260407151,
  author       = {Pith},
  title        = {Pith review of: An RTK-SLAM Dataset for Absolute Accuracy Evaluation in GNSS-Degraded Environments},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/2604.07151}},
  note         = {Machine review of arXiv:2604.07151}
}
Share X Bluesky LinkedIn Reddit HN
read the original abstract

RTK-SLAM systems integrate simultaneous localization and mapping (SLAM) with real-time kinematic (RTK) GNSS positioning, promising both relative consistency and globally referenced coordinates for efficient georeferenced surveying. A critical and underappreciated issue is that the standard evaluation metric, Absolute Trajectory Error (ATE), first fits an optimal rigid-body transformation between the estimated trajectory and reference before computing errors. This so-called SE(3) alignment absorbs global drift and systematic errors, making trajectories appear more accurate than they are in practice, and is unsuitable for evaluating the global accuracy of RTK-SLAM. We present a geodetically referenced dataset and evaluation methodology that expose this gap. A key design principle is that the RTK receiver is used solely as a system input, while ground truth is established independently via a geodetic total station. This separation is absent from all existing datasets, where GNSS typically serves as (part of) the ground truth. The dataset is collected with a handheld RTK-SLAM device, comprising two scenes. We evaluate LiDAR-inertial, visual-inertial, and LiDAR-visual-inertial RTK-SLAM systems alongside standalone RTK, reporting direct global accuracy and SE(3)-aligned relative accuracy to make the gap explicit. Results show that SE(3) alignment can underestimate absolute positioning error by up to 76\%. RTK-SLAM achieves centimeter-level absolute accuracy in open-sky conditions and maintains decimeter-level global accuracy indoors, where standalone RTK degrades to tens of meters. The dataset, calibration files, and evaluation scripts are publicly available at https://rtk-slam-dataset.github.io/.

Figures

Figures reproduced from arXiv: 2604.07151 by David Skuddis, Norbert Haala, Uwe Soergel, Vincent Ress, Wei Zhang.

Figure 1
Figure 1. Figure 1: Top: Equipment setup (left) and overview of checkpoints overlaid on the SLAM map of the Stadtgarten scene (right). Orange marked checkpoints are under open sky, while cyan-marked checkpoints are under GNSS obstruction (e.g. buildings, trees, underpass). Bottom: Absolute 3D error per checkpoint for Stadtgarten Seq. 1 using FAST-LIO-SAM method. Standalone RTK errors grow to tens of meters in GNSS-degraded zo… view at source ↗
Figure 2
Figure 2. Figure 2: Sensor platform with coordinate frames (red = [PITH_FULL_IMAGE:figures/full_fig_p003_2.png] view at source ↗
Figure 3
Figure 3. Figure 3: Example camera images overlaid with projected [PITH_FULL_IMAGE:figures/full_fig_p004_3.png] view at source ↗
Figure 4
Figure 4. Figure 4: Trajectory comparisons for all four sequences on satellite imagery. Rows (a)–(b): Stadtgarten Seq. 1–2; rows (c)–(d): [PITH_FULL_IMAGE:figures/full_fig_p006_4.png] view at source ↗
Figure 5
Figure 5. Figure 5: 3D absolute positioning error (log scale) as a function [PITH_FULL_IMAGE:figures/full_fig_p007_5.png] view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Lean theorems connected to this paper

Citations machine-checked in the Pith Canon. Every link opens the source theorem in the public Lean library.

What do these tags mean?
matches
The paper's claim is directly supported by a theorem in the formal canon.
supports
The theorem supports part of the paper's argument, but the paper may add assumptions or extra steps.
extends
The paper goes beyond the formal theorem; the theorem is a base layer rather than the whole result.
uses
The paper appears to rely on the theorem as machinery.
contradicts
The paper's claim conflicts with a theorem or certificate in the canon.
unclear
Pith found a possible connection, but the passage is too broad, indirect, or ambiguous to say the theorem truly supports the claim.

Reference graph

Works this paper leans on

4 extracted references · 4 canonical work pages

  1. [1]

    Performance Analysis of the IOPES Seamless Indoor- Outdoor Positioning Approach.Int. Arch. Photogramm. Remote Sens. Spatial Inf. Sci., XLIII-B4-2021, 229–235. Boche, S., Jung, J., Laina, S. B., Leutenegger, S., 2025. OKVIS2-X: Open keyframe-based visual-inertial SLAM con- figurable with dense depth or LiDAR, and GNSS.IEEE Trans- actions on Robotics. Burri...

    work page arXiv 2021

  2. [2]

    ORB-HD, 2026

    Mcd: Diverse large-scale multi-campus dataset for robot perception.Proceedings of the IEEE/CVF Conference on Com- puter Vision and Pattern Recognition, 22304–22313. ORB-HD, 2026. deface: Video anonymization by face detec- tion. GitHub repository. Qin, T., Cao, S., Pan, J., Shen, S., 2025. A General Optimisation-Based Framework for Global Pose Estimation W...

    work page 2026

  3. [3]

    Trybała, P., Kasza, D., Wajs, J., Remondino, F., 2023

    A benchmark for the evaluation of RGB-D SLAM sys- tems.Proceedings of the IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), IEEE, 573–580. Trybała, P., Kasza, D., Wajs, J., Remondino, F., 2023. Com- parison of Low-Cost Handheld LiDAR-Based SLAM Systems for Mapping Underground Tunnels.Int. Arch. Photogramm. Re- mote Sens. Spatial ...

    work page 2023

  4. [4]

    IEEE/ASME Transactions on Mechatronics, 30(2), 1212–1223

    GIVL-SLAM: a robust and high-precision SLAM sys- tem by tightly coupled GNSS RTK, inertial, vision, and LiDAR. IEEE/ASME Transactions on Mechatronics, 30(2), 1212–1223. Wei, H., Jiao, J., Hu, X., Yu, J., Xie, X., Wu, J., Zhu, Y ., Liu, Y ., Wang, L., Liu, M., 2025. FusionPortableV2: A unified multi- sensor dataset for generalized SLAM across diverse platf...

    work page 2025

This paper was first reviewed by grok-4.3 on May 10, 2026.