pith. sign in

arxiv: 2502.21029 · v2 · submitted 2025-02-28 · 💻 cs.RO · cs.LG

Sixth-Sense: Self-Supervised Learning of Spatial Awareness of Humans from a Planar Lidar

Simone Arreghini , Nicholas Carlotti , Mirko Nava , Antonio Paolillo , Alessandro Giusti This is my paper

Pith reviewed 2026-05-23 01:56 UTC · model grok-4.3

classification 💻 cs.RO cs.LG
keywords self-supervised learninghuman detectionplanar LiDAR2D pose estimationomnidirectional awarenessservice roboticsRGB-D supervisionspatial awareness
0
0 comments X

The pith

Self-supervised training lets 1D planar LiDAR detect humans omnidirectionally using RGB-D labels.

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper demonstrates a method to train a neural network on readings from an inexpensive 1D LiDAR so that it can locate nearby people in all directions and estimate their distance and facing angle. Supervision comes automatically from an RGB-D camera during 70 minutes of robot-driven data collection, avoiding manual annotation. If the approach holds, ordinary service robots could gain full spatial awareness of approaching users without needing costly 3D sensors or wide-view cameras. Reported performance on unseen environments reaches 71 percent precision and 80 percent recall, with average errors of 13 cm in distance and 44 degrees in orientation.

Core claim

By treating RGB-D detections as automatic labels during autonomous collection of 70 minutes of 1D LiDAR scans, the authors produce a model that outputs human detections together with 2D pose estimates. When evaluated against ground truth in new environments the model reaches 71 percent precision, 80 percent recall, 13 cm mean distance error and 44 degree mean orientation error. This supplies wide-field human awareness to robots that otherwise rely only on narrow cameras or basic planar LiDARs.

What carries the argument

The self-supervised pipeline that converts RGB-D camera detections into training labels for a neural network mapping 1D LiDAR scans to human 2D positions and orientations.

If this is right

  • Robots equipped only with cheap 1D LiDAR gain the ability to detect people approaching from any direction.
  • Service robots can plan safer navigation paths and initiate interactions based on omnidirectional human pose estimates.
  • The same data-collection procedure works across multiple public environments without retraining from scratch.
  • Final deployment uses only the LiDAR sensor, avoiding continuous camera use after training.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • The same labeling trick could transfer to other low-cost sensors such as ultrasonic arrays or single-beam rangefinders.
  • Combining the learned LiDAR detector with existing narrow-FOV cameras might yield higher reliability at modest extra cost.
  • Longer autonomous collection runs or active exploration strategies could further reduce the 13 cm and 44 degree errors.

Load-bearing premise

RGB-D camera detections remain accurate and unbiased enough to serve as ground truth labels for the LiDAR model in varied environments and human poses.

What would settle it

A controlled test in which the LiDAR model is evaluated in scenes where the RGB-D camera systematically misses or misplaces humans due to lighting changes or occlusion, checking whether detection metrics drop sharply below the reported levels.

Figures

Figures reproduced from arXiv: 2502.21029 by Alessandro Giusti, Antonio Paolillo, Mirko Nava, Nicholas Carlotti, Simone Arreghini.

Figure 1
Figure 1. Figure 1: Our approach uses a human detector from the narrow FOV [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 3
Figure 3. Figure 3: Training data is autonomously collected by the robot in a [PITH_FULL_IMAGE:figures/full_fig_p002_3.png] view at source ↗
Figure 2
Figure 2. Figure 2: Walking people and static structures as perceived by the LiDAR: [PITH_FULL_IMAGE:figures/full_fig_p002_2.png] view at source ↗
Figure 4
Figure 4. Figure 4: Our model uses a temporal window of n LiDAR scans to predict the presence p of nearby people, their distance d, and relative bearing o (represented by sine and cosine). Dilated circular convolutions handle omni￾directional scans and yield a 43◦ receptive field. A masked MSE loss is only enforced on predictions that overlap with the camera FOV (red shaded area) [PITH_FULL_IMAGE:figures/full_fig_p003_4.png] view at source ↗
Figure 5
Figure 5. Figure 5: Left: Precision-Recall curve for detection. Right: relative bearing error [PITH_FULL_IMAGE:figures/full_fig_p003_5.png] view at source ↗
Figure 6
Figure 6. Figure 6: Results on the test set: third-person view of two frames (left) and corresponding top view (right) depicting the [PITH_FULL_IMAGE:figures/full_fig_p004_6.png] view at source ↗
read the original abstract

Reliable localization of people is fundamental for service and social robots that must operate in close interaction with humans. State-of-the-art human detectors often rely on RGB-D cameras or costly 3D LiDARs. However, most commercial robots are equipped with cameras with a narrow field of view, leaving them unaware of users approaching from other directions, or inexpensive 1D LiDARs whose readings are hard to interpret. To address these limitations, we propose a self-supervised approach to detect humans and estimate their 2D pose from 1D LiDAR data, using detections from an RGB-D camera as supervision. Trained on 70 minutes of autonomously collected data, our model detects humans omnidirectionally in unseen environments with 71% precision, 80% recall, and mean absolute errors of 13cm in distance and 44{\deg} in orientation, measured against ground truth data. Beyond raw detection accuracy, this capability is relevant for robots operating in shared public spaces, where omnidirectional awareness of nearby people is crucial for safe navigation, appropriate approach behavior, and timely human-robot interaction initiation using low-cost, privacy-preserving sensing. Deployment in two additional public environments further suggests that the approach can serve as a practical wide-FOV awareness layer for socially aware service robotics.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit. Tearing a paper down is the easy half of reading it; the pith above is the substance, this is the friction.

Referee Report

2 major / 2 minor

Summary. The paper proposes a self-supervised method to learn human detection and 2D pose estimation from planar 1D LiDAR scans, using RGB-D camera detections as weak supervision. Trained on 70 minutes of autonomously collected data without manual labels, the model is claimed to achieve omnidirectional detection in unseen environments with 71% precision, 80% recall, and mean absolute errors of 13 cm (distance) and 44° (orientation) against ground truth, enabling low-cost wide-FOV awareness for social robots.

Significance. If the reported metrics rest on independent ground truth, the work offers a practical route to omnidirectional human awareness on inexpensive 1D-LiDAR-equipped platforms, which is relevant for service robotics in shared spaces. The self-supervised autonomous data collection and emphasis on privacy-preserving sensing are positive aspects that could support reproducible follow-on work.

major comments (2)
  1. [Evaluation / Results section] Evaluation / Results section (and abstract): the reported metrics (71% precision, 80% recall, 13 cm / 44° MAE) are stated as measured against ground truth data, yet the manuscript supplies no description of how this ground truth was acquired independently of the RGB-D detections used for supervision. If evaluation labels derive from the same narrow-FOV RGB-D pipeline or post-processing thereof, the quantitative claims reduce to an internal consistency check rather than external validation of LiDAR-only performance; this directly undermines the central empirical claim of reliable omnidirectional detection in unseen environments.
  2. [Methods / Data collection subsection] Methods / Data collection subsection: the validation protocol, error bars, data exclusion criteria, and train/test split details for the unseen-environment experiments are not provided, leaving the soundness of the 71%/80% figures and MAE values without visible support.
minor comments (2)
  1. [Abstract / Introduction] The abstract and introduction would benefit from a brief statement on the LiDAR model architecture (e.g., input representation and output parameterization) to allow readers to assess the self-supervised pipeline without immediately consulting the methods.
  2. [Figures] Figure captions for qualitative results should explicitly note whether displayed detections are from the LiDAR model or the RGB-D supervisor.

Simulated Author's Rebuttal

2 responses · 0 unresolved

Thank you for the constructive review. We address the two major comments below and will revise the manuscript accordingly to improve clarity on evaluation methodology.

read point-by-point responses

  1. Referee: [Evaluation / Results section] Evaluation / Results section (and abstract): the reported metrics (71% precision, 80% recall, 13 cm / 44° MAE) are stated as measured against ground truth data, yet the manuscript supplies no description of how this ground truth was acquired independently of the RGB-D detections used for supervision. If evaluation labels derive from the same narrow-FOV RGB-D pipeline or post-processing thereof, the quantitative claims reduce to an internal consistency check rather than external validation of LiDAR-only performance; this directly undermines the central empirical claim of reliable omnidirectional detection in unseen environments.

    Authors: We agree that the manuscript currently lacks an explicit description of how the ground truth was acquired independently. We will revise the Evaluation / Results section (and update the abstract if needed) to include a clear account of the independent ground truth collection process, distinct from the RGB-D supervision used in training. This addition will substantiate that the metrics reflect external validation of the LiDAR-only model. revision: yes

  2. Referee: [Methods / Data collection subsection] Methods / Data collection subsection: the validation protocol, error bars, data exclusion criteria, and train/test split details for the unseen-environment experiments are not provided, leaving the soundness of the 71%/80% figures and MAE values without visible support.

    Authors: We acknowledge that these protocol details are absent from the current manuscript. We will expand the Methods / Data collection subsection in the revision to specify the train/test splits for the unseen environments, data exclusion criteria, error bar computation, and full validation protocol, thereby providing the necessary support for the reported metrics. revision: yes

Circularity Check

0 steps flagged

No circularity: empirical ML pipeline with external supervision and independent GT

full rationale

The paper presents an empirical self-supervised training procedure that uses RGB-D detections solely as training labels for a LiDAR model. Reported performance numbers are stated as measured against separate ground truth data. No equations, derivations, fitted parameters renamed as predictions, or self-citation chains appear in the provided text. The work contains no mathematical first-principles claims that could reduce to their own inputs by construction. This is the normal case of a self-contained empirical result.

Axiom & Free-Parameter Ledger

0 free parameters · 1 axioms · 0 invented entities

The central claim rests on the assumption that RGB-D detections form an unbiased and sufficiently complete supervision signal for 1D LiDAR. No free parameters, invented entities, or non-standard axioms are described in the abstract.

axioms (1)
  • domain assumption RGB-D camera detections provide reliable and unbiased labels for training the LiDAR model
    The self-supervised pipeline uses these detections as supervision; if they contain systematic errors the downstream metrics would be affected.

pith-pipeline@v0.9.0 · 5777 in / 1307 out tokens · 42329 ms · 2026-05-23T01:56:45.944257+00:00 · methodology

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

21 extracted references · 21 canonical work pages

  1. [1]

    Exploring the educational potential of robotics in schools: A systematic review,

    F. B. V . Benitti, “Exploring the educational potential of robotics in schools: A systematic review,” Comp. & Education , vol. 58, no. 3, pp. 978–988, 2012

    work page 2012

  2. [2]

    Social robots in hospitals: a systematic review,

    C. S. Gonz ´alez-Gonz´alez, V . Violant-Holz, and R. M. Gil-Iranzo, “Social robots in hospitals: a systematic review,” Appl. Sci., vol. 11, no. 13, p. 5976, 2021

    work page 2021

  3. [3]

    Service robots in hotels: understanding the service quality perceptions of human-robot interaction,

    M. M. O. Youngjoon Choi, Miju Choi and S. S. Kim, “Service robots in hotels: understanding the service quality perceptions of human-robot interaction,” J. of Hospitality Marketing & Management , vol. 29, no. 6, pp. 613–635, 2020

    work page 2020

  4. [4]

    An RGB-D based social behavior interpretation system for a humanoid social robot,

    A. Zaraki, M. Giuliani, M. B. Dehkordi, D. Mazzei, A. D’ursi, and D. De Rossi, “An RGB-D based social behavior interpretation system for a humanoid social robot,” in RSI/ISM Int. Conf. on Robot. and Mechatronics, 2014, pp. 185–190

    work page 2014

  5. [5]

    Predicting the intention to interact with a service robot: the role of gaze cues,

    S. Arreghini, G. Abbate, A. Giusti, and A. Paolillo, “Predicting the intention to interact with a service robot: the role of gaze cues,” in IEEE Int. Conf. Robot. and Autom. , 2024, pp. –

    work page 2024

  6. [6]

    Jrdb: A dataset and benchmark of egocentric robot visual perception of humans in built environments,

    R. Martin-Martin, M. Patel, H. Rezatofighi, A. Shenoi, J. Gwak, E. Frankel, A. Sadeghian, and S. Savarese, “Jrdb: A dataset and benchmark of egocentric robot visual perception of humans in built environments,” IEEE Trans. on Pattern Anal. and Machine Intell. , vol. 45, no. 6, pp. 6748–6765, 2021

    work page 2021

  7. [7]

    TIAGo: the modular robot that adapts to different research needs,

    J. Pages, L. Marchionni, and F. Ferro, “TIAGo: the modular robot that adapts to different research needs,” in International workshop on robot modularity, IROS, vol. 290, 2016

    work page 2016

  8. [8]

    A survey on the design and evolution of social robots—past, present and future,

    H. Mahdi, S. A. Akgun, S. Saleh, and K. Dautenhahn, “A survey on the design and evolution of social robots—past, present and future,” Elsevier Robotics and Autonomous Systems , vol. 156, p. 104193, 2022

    work page 2022

  9. [9]

    Human detection in surveillance videos and its applications-a review,

    M. Paul, S. M. Haque, and S. Chakraborty, “Human detection in surveillance videos and its applications-a review,” Springer Journal on Advances in Signal Processing , vol. 2013, no. 1, pp. 1–16, 2013

    work page 2013

  10. [10]

    Skeleton tracking accuracy and precision evaluation of kinect v1, kinect v2, and the azure kinect,

    M. T ¨olgyessy, M. Dekan, and L. Chovanec, “Skeleton tracking accuracy and precision evaluation of kinect v1, kinect v2, and the azure kinect,” MDPI Applied Sciences , vol. 11, no. 12, p. 5756, 2021

    work page 2021

  11. [11]

    Where am i? creating spatial awareness in unmanned ground robots using slam: A survey,

    N. K. Dhiman, D. Deodhare, and D. Khemani, “Where am i? creating spatial awareness in unmanned ground robots using slam: A survey,” Springer Sadhana, vol. 40, pp. 1385–1433, 2015

    work page 2015

  12. [12]

    The peripheral perception of movement,

    M. D. Vernon, “The peripheral perception of movement,” Cambridge University Press British Journal of Psychology , vol. 23, no. 3, p. 209, 1933

    work page 1933

  13. [13]

    Online learning for 3d lidar-based human detection: experimental analysis of point cloud clustering and classification methods,

    Z. Yan, T. Duckett, and N. Bellotto, “Online learning for 3d lidar-based human detection: experimental analysis of point cloud clustering and classification methods,” Springer Autonomous Robots , vol. 44, no. 2, pp. 147–164, 2020

    work page 2020

  14. [14]

    Cnn-based human detection using a 3d lidar onboard a uav,

    J. N. Hayton, T. Barros, C. Premebida, M. J. Coombes, and U. J. Nunes, “Cnn-based human detection using a 3d lidar onboard a uav,” in IEEE Int. Conf. Auton. Robot. Sys. and Comp. , 2020, pp. 312–318

    work page 2020

  15. [15]

    Learning long-range perception using self-supervision from short-range sensors and odometry,

    M. Nava, J. Guzzi, R. O. Chavez-Garcia, L. M. Gambardella, and A. Giusti, “Learning long-range perception using self-supervision from short-range sensors and odometry,”IEEE Robot. and Autom. Lett., vol. 4, no. 2, pp. 1279–1286, 2019

    work page 2019

  16. [16]

    Uncertainty-aware self-supervised learning of spatial perception tasks,

    M. Nava, A. Paolillo, J. Guzzi, L. M. Gambardella, and A. Giusti, “Uncertainty-aware self-supervised learning of spatial perception tasks,” IEEE Robot. and Autom. Lett. , vol. 6, no. 4, pp. 6693–6700, 2021

    work page 2021

  17. [17]

    Weakly supervised 3d multi-person pose estimation for large-scale scenes based on monocular camera and single lidar,

    P. Cong, Y . Xu, Y . Ren, J. Zhang, L. Xu, J. Wang, J. Yu, and Y . Ma, “Weakly supervised 3d multi-person pose estimation for large-scale scenes based on monocular camera and single lidar,” inAAAI Conference on Artificial Intelligence , vol. 37, no. 1, 2023, pp. 461–469

    work page 2023

  18. [18]

    Fully convolutional networks for semantic segmentation,

    J. Long, E. Shelhamer, and T. Darrell, “Fully convolutional networks for semantic segmentation,” in IEEE/CVF Conf. on Comp. Vision and Pattern Recogn., 2015, pp. 3431–3440

    work page 2015

  19. [19]

    Adam: a method for stochastic optimization,

    D. P. Kingma and J. Ba, “Adam: a method for stochastic optimization,” in Int. Conf. on Learn. Represent. , 2015

    work page 2015

  20. [20]

    Accurate estimation of human body orientation from rgb-d sensors,

    W. Liu, Y . Zhang, S. Tang, J. Tang, R. Hong, and J. Li, “Accurate estimation of human body orientation from rgb-d sensors,” IEEE Trans. on Cybernetics, vol. 43, no. 5, pp. 1442–1452, 2013

    work page 2013

  21. [21]

    3d human motion estimation via motion compression and refinement,

    Z. Luo, S. A. Golestaneh, and K. M. Kitani, “3d human motion estimation via motion compression and refinement,” inAsian Conference on Computer Vision , 2020. 6 TABLE II DESCRIPTION OF SUPPLEMENTARY MATERIALS Supplementary material Description Zenodo Dataset Link Link to the Zenodo page for the publicly available dataset. The excessive size of our dataset ...

    work page arXiv 2020