Pith. sign in

REVIEW 4 major objections 6 minor 23 references

A Monocular SLAM-based Multi-User Positioning System with Image Occlusion in Augmented Reality

T0 review · 4 major / 6 minor · reviewed 2026-08-12 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read A detected tabletop in each user's room can replace shared maps as the alignment anchor for multi-user AR.

desk verdict A clearly-built AR integration whose headline multi-user alignment is never actually measured; the single-user scale calibration works, but the across-user avatar placement is only shown in screenshots. read the letter →

arxiv 2411.10940 v1 pith:ZPIXSB5F submitted 2024-11-17 cs.HC cs.CV

Wei-Hsiang Lien , Benedictus Kent Chandra , Robin Fischer , Ya-Hui Tang , Shiann-Jang Wang , Wei-En Hsu , Li-Chen Fu This is my paper · ORCID
classification cs.HCcs.CV
keywords augmentedrealitymulti-userpositioningORB-SLAM2planeestimationscalecalibrationmonoculardepthocclusionrenderingvirtualavatars
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper claims that users in different physical rooms can share one augmented-reality space without sharing SLAM maps, by making each user's detected table plane the common reference frame. Each device runs monocular ORB-SLAM2, performs a one-time scale calibration with a printed marker, and computes the plane of its own table from the resulting sparse map. Users exchange only camera-to-plane poses and table-boundary points through a central server, and every device reconstructs the other users' avatars by applying those relative poses to its own plane. A server-side monocular depth model supplies occlusion, so real objects can correctly hide virtual avatars. After calibration, the reported trajectory error against a commercial visual-tracking baseline is 0.0065 m translational RMSE and 0.6302 degrees rotational RMSE.

What carries the argument

The load-bearing mechanism is the plane-as-reference transform chain. Each user's table plane gives a local coordinate frame whose origin is the centroid of RANSAC inlier map points and whose y-axis is the plane normal; the plane boundary is the convex hull of those inliers (computed by the Graham algorithm). The scale conversion $\mathrm{Scale}=D_\mathrm{physic}/\|P_1-P_2\|$ turns one marker observation into global metric units, and the two transformations above convert any camera pose from one user's SLAM frame into another user's frame through the plane frames. The same plane frames also define the common placement area, since the system intersects the convex polygons of all users' table boundaries to find the region where a virtual object stays on every table. Occlusion works by a separate depth channel: the server-side monocular depth model produces a per-frame depth map, and Unity's fragment shader compares it with rendered object depth, drawing background pixels wherever the real scene is closer.

What would settle it

Repeat the calibration with the marker held at oblique angles (for example, 30 and 45 degrees) or with part of it obscured, then walk a measured path and compare the reconstructed trajectory against a tape-measured ground truth; if the translational RMSE grows substantially beyond the reported 0.0065 m, the scale calibration's viewpoint sensitivity is confirmed.

Watch

Extended reading notes

Core claim

On the author's account, the central discovery is that a planar surface, a tabletop, can act as a shared anchor for coordinate alignment, even though every user's SLAM map is local. Plane estimation uses RANSAC to remove outlier map points and SVD-based least-squares fitting to obtain the table plane; the plane's x-axis is oriented toward the user, so virtual objects face them. A scale factor, $\mathrm{Scale}=D_\mathrm{physic}/\|P_1-P_2\|$, is computed once from two matched ORB points on a known-size marker and converts the unitless monocular map into metric units. The relative pose of a user's camera with respect to their plane, \({}^P T_{C_A} = {}^P T_{S_A}\, {}^{S_A}T_{C_A}\), is broadcast, and the receiving user places the avatar via \({}^ {S_B} T_{C_A} = {}^{S_B}T_P\, {}^P T_{C_A}\). Shared virtual objects are confined to the intersection of the convex hulls of all users' table boundaries, and the MiDaS monocular depth estimator is run on a server to drive occlusion in the fragment shader.

Load-bearing premise

The whole system's position accuracy rests on a one-time calibration that assumes the printed marker is viewed fronto-parallel and that the two most similar ORB matches it chooses are correct; if either fails, every table boundary, virtual object, and avatar is shifted by the same scale error.

Editorial extensions

If this is right

  • After a single scale calibration, the localization output tracks a commercial visual-tracking baseline with 0.0065 m translational RMSE and 0.6302 degrees rotational RMSE.
  • No SLAM map, keyframe, or point cloud ever has to be sent between users; only camera-to-plane poses and plane-boundary points cross the network.
  • A common virtual object stays within every user's table because its allowed placement region is the intersection of all detected table-plane convex hulls.
  • The collaboration mode rotates a user's plane frame by the user's ID-dependent angle, so participants can be arranged around the table as if they were at 180 degrees (two users) or other angular spacings.
  • The paper itself expects the system to scale poorly as the number of users grows, since the coordination server aggregates all pose and boundary data centrally.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The same plane-as-anchor trick should transfer to any detectable planar surface, wall, whiteboard, floor, or poster, so the table is likely a convenience rather than a requirement, as long as all users can detect the same class of surface.
  • Because the scale factor is computed from only two ORB correspondences and one assumed fronto-parallel view of the marker, a natural robustness test is to replace it with a multi-correspondence least-squares or homography-based scale estimate; the paper does not quantify how oblique views degrade the 6.5 mm result.
  • Since the shared placement area is an intersection of convex hulls, adding users with smaller tables shrinks the common region, which may explain the paper's own scalability warning; larger or similar-sized tables would keep the shared area usable.
  • The depth server costs about 80 ms per frame on the depth thread, so occlusion and pose refresh around 12 fps; an on-device or distilled depth model would be the obvious next step for smoother AR experiences.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 6 minor

Summary. The paper proposes a multi-user AR positioning system built on monocular ORB-SLAM2. Each user runs ORB-SLAM2 on RGB images from an AR glass or smartphone; a one-time marker-based calibration resolves monocular scale uncertainty (Section 3.2, Eqs. 1-2); RANSAC/SVD plane estimation recovers a table plane with finite boundaries (Section 3.3); a TCP coordination server exchanges camera-to-plane relative poses so that each user can render other users' avatars in their own SLAM coordinate frame (Section 3.4, Eqs. 3-5); and a MiDaS-based depth server supplies per-frame depth for occlusion rendering (Section 3.5). Quantitative results are reported for scale-calibrated single-user pose accuracy against Vuforia (Tables 1-2, RMSE 0.0065 m and 0.6302 degrees) and for runtime (Table 4), with qualitative demonstrations of plane intersection and occlusion (Figs. 10-11).

Significance. If the multi-user alignment were properly validated, the paper would offer a practical, lightweight alternative to map-merging approaches for remote AR collaboration, and the use of a detected table plane as a common reference frame is a sensible design choice. The system integrates open-source monocular SLAM with a commercial AR headset, clearly decomposes the software architecture into localization, plane estimation, coordination, and depth modules, and reports per-module runtimes, which is useful for practitioners. The contribution is diminished, however, by the fact that the headline multi-user positioning claim is not measured: the only pose-error evaluation is single-user scale calibration against Vuforia, and the multi-user alignment from Eqs. (3)-(5) is supported only by screenshots. The significance of the paper therefore hinges on additional experiments that have not yet been reported.

major comments (4)
  1. [Section 4.2, Tables 1-2] The quantitative evaluation measures single-user camera pose error after scale calibration, but the title, abstract, and introduction claim a multi-user positioning system. The multi-user avatar alignment produced by Eqs. (3)-(5) is never evaluated with any error metric, trial count, confidence interval, or comparison against a reference. Please add an experiment in which two or more users in separate physical spaces are tracked, the rendered avatar position/orientation in one user's SLAM frame is compared with a measured ground-truth relative pose, and the results are reported over multiple trials and trajectories.
  2. [Section 3.2, Eqs. (1)-(2)] The scale factor is computed from exactly two ORB correspondences under the assumption that the marker is viewed fronto-parallel, so that pixel distances scale linearly to physical distances. No sensitivity analysis is provided for perspective distortion, matching errors, non-coplanar correspondences, or invalid 3D map points. Because this single scalar multiplies every subsequent plane boundary, object placement, and avatar position, the paper should quantify how calibration repeatability and viewpoint variation affect the reported 0.0065 m RMSE, or justify why a two-point fronto-parallel model is sufficient.
  3. [Section 3.3 and Section 3.4.1] The plane coordinate system's x-axis is defined by projecting the camera's position onto the estimated plane. This makes the plane frame viewpoint-dependent: two users standing at different positions around their tables will define different x-axes, so 'coinciding all the plane's coordinate systems' is not a physically meaningful alignment unless users stand in corresponding locations. The paper should either define plane axes from table geometry independent of camera position, or explicitly analyze how the viewpoint-dependent x-axis affects the avatar placement in Eq. (5).
  4. [Section 3.4.2, Eq. (6)] Collaboration Mode assigns user positions by an arbitrary rotation theta = 360 deg / N times i rather than by any measured physical arrangement. Consequently, the opposite-side placement shown in Fig. 6 is imposed by the system, not validated as an accurate cross-space alignment. Please clarify that Eq. (6) is a deliberate user-interface choice, and exclude this mode from any claim of measured physical alignment unless a separate metric is reported.
minor comments (6)
  1. [Throughout] Please correct typos such as 'Wphysic' and 'Hphysic' (should be 'physical'), and clean up author/affiliation formatting artifacts such as 'T aiwan' and 'Y a-Hui T ang'.
  2. [Eq. (1)] The text says p1 and p2 are 'correspondences in the marker,' but the formula uses pixel distance in the marker image; clarify whether p1 and p2 are 2D marker coordinates or scene-image keypoints, and define all symbols explicitly.
  3. [Section 3.4.1, Eqs. (3)-(5)] The notation SA P T, P SA T, SA CA T, and P CA T is visually confusing; consider using a consistent convention such as T_{a,b} for a transform from frame b to frame a, with all frames defined in a table.
  4. [Tables 1-2] Report the number of trials, trajectory length, number of frames, and confidence intervals for the RMSE values; also use a consistent numerical format and degree symbol in the table headers.
  5. [Section 4.4, Fig. 11] The occlusion evaluation is qualitative; please state whether any quantitative depth error metrics (e.g., RMSE or absolute relative error against known ground truth) were considered, and clarify how 'satisfactory' is defined in the model-selection discussion.
  6. [Section 4.5, Table 4] The total depth-thread time is listed as 0.0803 s, but the tabulated components are 0.053 s and 0.027 s, which sum to 0.080 s; check this value and the four-decimal '0.07211' in the coordination thread for consistency.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: scale calibration is validated against an independent tracker, and the plane-frame alignment is an explicit coordinate composition rather than a fitted prediction.

full rationale

The only step that could resemble a fitted-input-called-prediction is the one-time scale calibration in Eqs. (1)-(2), where a global scale factor is computed from a known-size marker. However, the reported translational RMSE after scaling (Section 4.2) is evaluated against Vuforia's camera pose, which is an independent commercial tracker, so the paper runs a legitimate calibration-and-evaluation loop rather than predicting a quantity that is determined by construction. The multi-user alignment in Eqs. (3)-(5) is an explicit coordinate-frame composition: each user's plane pose is estimated from SLAM map points, and the relative camera-to-plane pose is transformed into the other user's SLAM frame. The paper states 'Note that here we coincide all the plane's coordinate systems for convenience of subsequent design of user coordination,' making the plane-frame coincidence an announced design convention rather than a hidden measured result. The only self-citation, the prior ICVR paper [22], appears in a related-work list and is not load-bearing for the system's derivation or evaluation. Concerns about the missing quantitative multi-user alignment metric and the sensitivity of the two-correspondence scale calibration to viewing angle are validation and robustness limitations, not circularity. Because no central claim reduces to its own inputs by construction, the appropriate finding is no significant circularity.

Assumptions & free parameters 1 free parameters · 6 assumptions · 0 invented entities

The central claim depends on a fitted global scale factor and on several unstated geometrical and domain assumptions, including fronto-parallel marker viewing, single-scale correction, table presence, Vuforia ground truth, coincident plane frames, and MiDaS depth comparability. No new entities are introduced.

free parameters (1)
  • Global scale factor = Computed via Eq. (2) from marker physical size and SLAM map point distance
    The one-time scale calibration in Section 3.2 fits a single scalar to convert the monocular SLAM map to metric units. All plane boundaries, avatar positions, and object placements inherit this scale.
assumptions (6)
  • domain assumption Every user has a physical table that is detected as a plane in the SLAM map.
    Section 3.4.1 states 'Both UserA and UserB should have a physical table in their environment'; Section 3.3 assumes the table plane is not featureless.
  • domain assumption The detected table planes from all users are treated as coincident in the shared virtual space.
    Section 3.4.1 says 'we coincide all the plane's coordinate systems for convenience'; this is the core assumption that makes the relative pose exchange in Eqs. 3-5 correct.
  • ad hoc to paper The marker is viewed fronto-parallel, so pixel distances scale linearly to physical distances.
    Eq. (1) uses Wphysic/Wpixel times pixel distance without a perspective correction; this assumption is not stated.
  • domain assumption Vuforia provides accurate ground-truth camera poses.
    Section 4.2 uses Vuforia as the reference for pose error; its own error is not characterized.
  • domain assumption A single global scale factor corrects the monocular SLAM scale for the entire trajectory.
    Section 3.2 performs calibration once, but monocular SLAM is known to exhibit scale drift.
  • domain assumption MiDaS depth maps are directly comparable to Unity virtual-scene depth.
    Section 3.5 compares depth map values with virtual object depth; MiDaS produces affine-invariant depth that generally needs alignment to scene scale.

how reviews work

0 comments
Cite this review

Pith. "Pith review of A Monocular SLAM-based Multi-User Positioning System with Image Occlusion in Augmented Reality." pith.science (2026). https://pith.science/paper/ZPIXSB5F

@misc{pith2026241110940,
  author       = {Pith},
  title        = {Pith review of: A Monocular SLAM-based Multi-User Positioning System with Image Occlusion in Augmented Reality},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/ZPIXSB5F}},
  note         = {Machine review of arXiv:2411.10940}
}
read the original abstract

In recent years, with the rapid development of augmented reality (AR) technology, there is an increasing demand for multi-user collaborative experiences. Unlike for single-user experiences, ensuring the spatial localization of every user and maintaining synchronization and consistency of positioning and orientation across multiple users is a significant challenge. In this paper, we propose a multi-user localization system based on ORB-SLAM2 using monocular RGB images as a development platform based on the Unity 3D game engine. This system not only performs user localization but also places a common virtual object on a planar surface (such as table) in the environment so that every user holds a proper perspective view of the object. These generated virtual objects serve as reference points for multi-user position synchronization. The positioning information is passed among every user's AR devices via a central server, based on which the relative position and movement of other users in the space of a specific user are presented via virtual avatars all with respect to these virtual objects. In addition, we use deep learning techniques to estimate the depth map of an image from a single RGB image to solve occlusion problems in AR applications, making virtual objects appear more natural in AR scenes.

Figures

Figures reproduced from arXiv: 2411.10940 by the authors.

Figure 1
Figure 1. Work collaboratively in augmented reality [ [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 2
Figure 2. System overview of the multi-user positioning system. [PITH_FULL_IMAGE:figures/full_fig_p003_2.png] view at source ↗
Figure 3
Figure 3. The process of scale calibration. in the environment, we utilize the map generated by the SLAM process. Assuming that the table plane is not featureless, we extract the 3D map points from the current frame, we can identify the plane that best fits this point cloud. However, the point cloud may contain both inlier points belonging to the desired plane and outlier points that do not lie on the plane. These outliers ca… view at source ↗
Figures from the paper (8 more)
Figure 4
Figure 4. Figure 4: Client-server framework between Unity and coordination server. [PITH_FULL_IMAGE:figures/full_fig_p005_4.png]
Figure 5
Figure 5. Figure 5: Classroom Mode: Two Users are facing the virtual cat on [PITH_FULL_IMAGE:figures/full_fig_p005_5.png]
Figure 6
Figure 6. Figure 6: Collaboration Mode: After rotating UserB’s plane coordi [PITH_FULL_IMAGE:figures/full_fig_p005_6.png]
Figure 7
Figure 7. Figure 7: The server-client architecture for monocular depth estima [PITH_FULL_IMAGE:figures/full_fig_p006_7.png]
Figure 8
Figure 8. Figure 8: The estimated pose trajectory of our system before scaling [PITH_FULL_IMAGE:figures/full_fig_p007_8.png]
Figure 9
Figure 9. Figure 9: The estimated pose trajectory of our system after scaling [PITH_FULL_IMAGE:figures/full_fig_p007_9.png]
Figure 10
Figure 10. Figure 10: Visualization of Virtual plane computation in a Multi-User [PITH_FULL_IMAGE:figures/full_fig_p007_10.png]
Figure 11
Figure 11. Figure 11: Depth maps and occlusion results of different models. The [PITH_FULL_IMAGE:figures/full_fig_p008_11.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

23 extracted references · 20 canonical work pages

  1. [1]

    https://www.chemanager-online.com/en/news/ bridging-gap-between-virtual-planning-and-reality

    Bridging the gap between virtual planning and real- ity. https://www.chemanager-online.com/en/news/ bridging-gap-between-virtual-planning-and-reality . Accessed: 2023-06-30. 2

    work page 2023

  2. [2]

    ”jorjin j7ef plus ar glasses.”

    J.technologies. ”jorjin j7ef plus ar glasses.”. https://www.jorjin. com/. Accessed: 2023-06-30. 2, 6

    work page 2023

  3. [3]

    https://developer.vuforia.com/

    Vuforia. https://developer.vuforia.com/. Accessed: 2023-06-

    work page 2023

  4. [4]

    P. An. A modification of graham’s algorithm for determining the con- vex hull of a finite planar set. Annales Mathematicae et Informaticae , 34, 2007. 4, 6

    work page 2007

  5. [5]

    V . E. Brimkov, S. Kafer, M. Szczepankiewicz, and J. Terhaar. On inter- section graphs of convex polygons. In Combinatorial Image Analysis, pp. 25–36. Springer International Publishing, 2014. 6

    work page 2014

  6. [6]

    Campos, R

    C. Campos, R. Elvira, J. J. G. Rodr ´ıguez, J. M. M. Montiel, and J. D. Tard´os. Orb-slam3: An accurate open-source library for visual, visual-inertial, and multimap slam. IEEE Transactions on Robotics , 37(6):1874–1890, 2021. 2

    work page 2021

  7. [7]

    H. A. Daoud, A. Q. M. Sabri, C. K. Loo, and A. M. Mansoor. Slamm: Visual monocular slam with continuous mapping using multiple maps. PLoS ONE, 13, 2018. 2

    work page 2018

  8. [8]

    Dhakal, X

    A. Dhakal, X. Ran, Y . Wang, J. Chen, and K. K. Ramakrishnan. Slam- share: Visual simultaneous localization and mapping for real-time multi-user augmented reality. InProceedings of the 18th International Conference on Emerging Networking EXperiments and Technologies , pp. 293–306. Association for Computing Machinery, 2022. 2

    work page 2022

Show all 23 references
  1. [9]

    M. A. Fischler and R. C. Bolles. Random sample consensus: A paradigm for model fitting with applications to image analysis and automated cartography. Communications of the ACM, 24(6):381–395, June 1981. 4

    1981

  2. [10]

    Karrer, P

    M. Karrer, P. Schmuck, and M. Chli. Cvi-slam—collaborative visual- inertial slam. IEEE Robotics and Automation Letters , 3(4):2762– 2769, 2018. 2

    2018

  3. [11]

    Keshavarzi, A

    M. Keshavarzi, A. Y . Yang, W. Ko, and L. Caldas. Optimization and manipulation of contextual mutual spaces for multi-user virtual and augmented reality interaction. In 2020 IEEE Conference on Virtual Reality and 3D User Interfaces (VR) , pp. 353–362, 2020. doi: 10. 1109/VR462...

    2020

  4. [12]

    Kolesnikov, A

    A. Kolesnikov, A. Dosovitskiy, D. Weissenborn, G. Heigold, J. Uszko- reit, L. Beyer, M. Minderer, M. Dehghani, N. Houlsby, S. Gelly, T. Un- terthiner, and X. Zhai. An image is worth 16x16 words: Transformers for image recognition at scale, 2021. 8

    2021

  5. [13]

    Lajoie, B

    P. Lajoie, B. Ramtoula, F. Wu, and G. Beltrame. Towards collabora- tive simultaneous localization and mapping: a survey of the current research landscape. CoRR, abs/2108.08325, 2021. 1, 2

    2021 arXiv

  6. [14]

    Miller, E

    J. Miller, E. Soltanaghai, R. Duvall, J. Chen, V . Bhat, N. Pereira, and A. Rowe. Multi-user augmented reality with infrastructure-free col- laborative localization. 10 2021. 2

    2021

  7. [15]

    Mur-Artal, J

    R. Mur-Artal, J. M. M. Montiel, and J. D. Tard ´os. Orb-slam: A ver- satile and accurate monocular slam system. IEEE Transactions on Robotics, 31(5):1147–1163, 2015. 2

    2015

  8. [16]

    Mur-Artal and J

    R. Mur-Artal and J. D. Tard ´os. Orb-slam2: An open-source slam sys- tem for monocular, stereo, and rgb-d cameras. IEEE Transactions on Robotics, 33(5):1255–1262, 2017. 1, 2, 8

    2017

  9. [17]

    T. Qin, P. Li, and S. Shen. Vins-mono: A robust and versatile monoc- ular visual-inertial state estimator. IEEE Transactions on Robotics , 34(4):1004–1020, 2018. 2

    2018

  10. [18]

    X. Ran, C. Slocum, Y .-Z. Tsai, K. Apicharttrisorn, M. Gorlatova, and J. Chen. Multi-user augmented reality with communication efficient and spatially consistent virtual objects. In Proceedings of the 16th International Conference on Emerging Networking EXperiments and Technol...

  11. [19]

    Ranftl, A

    R. Ranftl, A. Bochkovskiy, and V . Koltun. Vision transformers for dense prediction. In 2021 IEEE/CVF International Conference on Computer Vision (ICCV), pp. 12159–12168, 2021. 7

    2021

  12. [20]

    Ranftl, K

    R. Ranftl, K. Lasinger, D. Hafner, K. Schindler, and V . Koltun. To- wards robust monocular depth estimation: Mixing datasets for zero- shot cross-dataset transfer.IEEE Transactions on Pattern Analysis and Machine Intelligence, 44(3):1623–1637, 2022. 6, 7, 8

    2022

  13. [21]

    Stotko, S

    P. Stotko, S. Krumpen, M. B. Hullin, M. Weinmann, and R. Klein. Slamcast: Large-scale, real-time 3d reconstruction and streaming for immersive multi-client live telepresence. IEEE Transactions on Visu- alization and Computer Graphics , 25(5):2102–2112, May 2019. doi: 10.1109/t...

    2019

  14. [22]

    B.-H. Wang, F. Wijaya, R. Fischer, Y .-H. Tang, S.-J. Wang, W.-E. Hsu, and L.-C. Fu. A scene understanding and positioning system from RGB images for tele-meeting application in augmented reality. In 2023 9th International Conference on Virtual Reality (ICVR) , pp. 106–114, 20...

    2023

  15. [23]

    Wang and Y

    J. Wang and Y . Qi. A multi-user collaborative ar system for industrial applications. Sensors, 22(4), 2022. 2

    2022

Pith tools

Reviewed August 12, 2026 · model on record in the stance chip above.