{"id":"cb3b2f53-52b9-4862-a002-905e4e4ac09b","arxiv_id":"2411.18476","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":4.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":6,"one_line_summary":"A preliminary study reports that extended object tracking with a stereo camera produced visually similar indoor tracking results to a LiDAR sensor costing over ten times more, but without quantitative ground-truth evaluation.","lead":"This paper tests whether a cheap stereo camera can track a mobile robot indoors as well as a LiDAR sensor that costs more than ten times as much. The authors build a simple detector plus extended object tracking and report visually similar results, but the comparison is preliminary and has no ground-truth accuracy check.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The central claim of stereo-camera/LiDAR performance similarity rests on a qualitative trajectory comparison with no ground-truth accuracy metric; the paper's own Section 4 defers such evaluation to future work. This makes the headline result unsupported by the presented evidence.","rationale":"The reader's weakest_assumption identifies exactly the same load-bearing gap: the lack of ground-truth evaluation for the claimed performance similarity. My analysis adds that the mismatch in sampling rates and coordinate frames makes the visual comparison in Figure 5 even less conclusive than a simple 'no ground truth' statement, because the curves are not directly aligned in a common frame. The paper is transparent about this limitation (Section 4), which supports a conditional rather than a reject verdict. The method description is detailed enough to re-implement, and the detection rates (98% vs 91%) are a positive signal that the detector works in both modalities, but tracking accuracy remains unquantified. I therefore agree with the reader that the verdict should remain CONDITIONAL, pending a quantitative validation with ground-truth data. My recommended verdict is unchanged from the reader's, but I emphasize that the conditional should explicitly require the ground-truth accuracy comparison before the cost-saving claim is accepted.","tokens_in":6531,"tokens_out":733,"duration_ms":8070,"concrete_test":"Re-run the two recorded trajectories (or collect new ones) while logging the TurtleBot's onboard odometry or using an external motion-capture/visual-marker system as ground truth. Compute per-timestep centroid error (e.g., RMSE or ATE) for both the LiDAR-based and stereo-camera-based tracks, plus extent (shape) error if ground-truth shape is known. If the stereo error is within a pre-specified tolerance (e.g., 10% of LiDAR error) for both trajectories, the central claim is supported; if the stereo error is substantially larger or the trajectories diverge, the claim should be weakened accordingly.","verdict_should_be":"CONDITIONAL","load_bearing_attack":"The paper's strongest claim is that the stereo camera achieves performance similar to the LiDAR (Abstract, Section 3.3). The only evidence offered is the qualitative visual similarity of estimated trajectory curves in Figure 5, with no ground-truth trajectory, per-frame position error, shape error, or statistical comparison. The text even states: 'In the future, we plan to evaluate the tracking results using ground truth data from the robot's onboard sensors' (Section 4). Without ground truth, 'similar performance' cannot be distinguished from 'similarly wrong estimates.' Moreover, the two sensors have different sampling rates (4.4 Hz vs 30 Hz), different coordinate frames (Figure 5 plots are in each sensor's own frame), and different noise characteristics (Section 3.3), so the visual overlap of curves is not a quantitative measure. The detector's performance (98% vs 91% detection rate) is reported but does not translate to tracking accuracy. Thus the cost-saving conclusion (sub-400 euro stereo vs 4000+ euro LiDAR) is unsupported. This is a load-bearing gap because the paper's entire contribution is the empirical comparison; if accuracy is not measured, the central claim collapses to an anecdotal observation. The concern is not that the comparison is impossible, but that the current manuscript provides no metric that would validate it.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper presents a preliminary comparison of extended object tracking (EOT) on 3D point clouds from a Blickfeld Cube 1 LiDAR and an Intel RealSense D435i stereo camera in an indoor environment. The authors develop a heuristic object detector that removes ground points, clusters candidates with DBSCAN, and selects the target by a feature-cost criterion; the detected points feed a Gaussian-process star-convex EOT framework with an iterated EKF. The paper reports detection rates of 98% for LiDAR and 91% for the stereo camera, shows qualitative trajectory plots in both sensors' local frames, and concludes that the stereo camera achieves tracking performance similar to that of the LiDAR at a fraction of the cost.","tokens_in":6854,"tokens_out":3199,"duration_ms":30310,"significance":"If the central comparison were quantitatively established, this study would be a useful data point for cost-effective indoor EOT, potentially enabling sub-400 euro stereo cameras as substitutes for multi-thousand-euro LiDAR in factory-floor tracking. The paper is transparent about its preliminary nature and explicitly lists its limitations, and it builds on well-established EOT machinery from the cited literature. However, the headline claim of 'similar performance' currently rests on qualitative visual inspection and detection-rate statistics rather than tracking-error metrics, so the significance is conditional on a quantitative evaluation being added.","major_comments":[{"comment":"The central claim that the stereo camera achieves similar performance to LiDAR is supported only by qualitative visual inspection of the estimated centroid trajectories and extents plotted in each sensor's own coordinate frame. The paper itself states in Section 4, 'In the future, we plan to evaluate the tracking results using ground truth data from the robot's onboard sensors,' confirming that no ground-truth accuracy assessment is currently made. Without a quantitative metric such as position RMSE, extent error, or a statistical comparison across trials, the visual overlap cannot distinguish 'similar accuracy' from 'similarly wrong estimates.' This is load-bearing because the entire contribution is the cross-sensor comparison.","section":"Section 3.3, Figure 5, and Section 4"},{"comment":"The two sensors have different sampling rates (4.4 Hz for LiDAR vs 30 Hz for the camera), different local coordinate frames, and different noise characteristics. Figure 5 plots trajectories in each sensor's Cartesian plane, so the apparent similarity is not evaluated on a common reference. The authors should resample and align the estimated trajectories to a common frame and report per-timestep errors with respect to a reference trajectory, or at minimum provide a quantitatively validated sensor alignment that justifies the visual comparison.","section":"Section 3.3"},{"comment":"The reported detection rates (98% for LiDAR and 91% for the camera) are not a substitute for tracking accuracy. A higher detection rate does not imply lower position or extent error, particularly because the detector's thresholds and feature weights (Algorithm 1, parameters 𝛾3, 𝑤𝑓, 𝑓𝑝) are hand-tuned per sensor. The paper should report tracking metrics (e.g., position RMSE, OSPA, or shape-error measures) at matched timestamps and discuss how missed detections and false positives propagate through the EKF, otherwise the cost-saving conclusion is unsupported.","section":"Section 3.3"},{"comment":"The comparison relies on many manually selected per-sensor parameters: ground-plane thresholds 𝛾0 and 𝛾2, RANSAC parameters (𝛼, MAX_ITER, n), clustering parameters 𝑚𝑐, feature prior 𝑓𝑝, feature weights 𝑤𝑓, detection threshold 𝛾3, and the GP/EKF noise parameters. While this is not a logical circularity, it is a robustness gap: the claimed performance similarity may depend critically on these hand-tuned values. A brief sensitivity analysis over the most influential parameters (or at least over the stereo-camera noise-related parameters) would strengthen the conclusion that the comparison is not an artifact of the selected settings.","section":"Sections 3.1 and 3.2"}],"minor_comments":[{"comment":"The mathematical notation has inconsistent spacing and formatting, e.g., '𝑓 (𝑥𝑘+1| 𝑥𝑘)' and the state-vector definition; please proofread the math typography throughout the paper.","section":"Section 2.2"},{"comment":"The units for the process noise parameter are written as '𝜎𝑐 = 0.05√𝑚2𝑠−3', which does not appear to be a standard unit expression; please correct or clarify the intended units.","section":"Section 3.2"},{"comment":"The ground-plane coefficient vectors such as [0, 0, 1, 1]𝑇 for LiDAR and [0, 1, 0, −0.5]𝑇 for the camera imply different coordinate conventions (z-up vs y-up), but the sensor coordinate frames are not defined; please state the conventions explicitly.","section":"Section 3.1"},{"comment":"The text says the robot was tracked 'within a range of 3 meters' and cites the D435i's operational range as 0.3 to 3 meters; please clarify whether the entire trajectory lies inside the validated stereo depth range and how the LiDAR's larger range affects the comparison.","section":"Section 3.3"},{"comment":"The caption refers to 'the light blue curve' but the text mentions 'light blue curve' without consistent color naming; please ensure the color references exactly match the rendered figure.","section":"Figure 5"},{"comment":"The captions (a) 'Camera one', (b) 'Lidar one', (c) 'Camera two', (d) 'Lidar two' are inconsistently styled; please unify the naming (e.g., 'Camera, turning motion' and 'LiDAR, turning motion').","section":"Figure 5"},{"comment":"Several bibliographic entries are incomplete, such as [15] and [19] missing access dates or full URLs; please complete the reference list.","section":"References"}],"recommendation":"major_revision","confidential_remarks":"The reader's concern is well founded: the paper's own statement in Section 4 admits that ground-truth evaluation is future work, and the central claim is not supported by a quantitative accuracy metric. The manuscript is written in a workshop-paper style and is explicitly preliminary, but for a journal-level report the load-bearing gap is the missing evaluation. If the authors add a proper quantitative comparison with aligned trajectories and error metrics, the paper could become acceptable; in its current form, the headline result is an empirical anecdote. I would not reject it outright because the gap is addressable within the manuscript's scope."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Quick take: this is a clearly written preliminary study, but the central claim is not yet supported. The new contribution is applying an existing GP-based extended object tracking pipeline to stereo camera point clouds and comparing it to LiDAR for a single indoor robot. That is a legitimate small extension, and it is the first comparison of this kind that I know of. The detector is described in enough detail (algorithm, all parameters) to re-implement, and the authors are upfront about the hand-tuned nature of the detector and the depth-dependent noise of the camera. The cost argument (€400 vs €4000+) is practically relevant.\n\nThe problem is that the headline claim—stereo achieves performance similar to LiDAR—is based on visual similarity of the estimated trajectories in Figure 5. There is no ground-truth trajectory, no position error, no shape error, no statistical comparison. The paper itself says in Section 4 that ground-truth evaluation is future work. The two sensors run at different rates (4.4 vs 30 Hz) and Figure 5 plots are in each sensor's local frame, so the visual overlap of curves is not a quantitative measure. The reported detection rates (98%/91%) show the detector finds the robot in most frames, but that does not translate into tracking accuracy. So the conclusion 'similar performance' is a hypothesis, not a demonstrated result.\n\nI would not call this a dishonest or sloppy paper. It is clearly labeled preliminary, the limitations section acknowledges the main weaknesses, and the method section is reproducible. For a workshop, this is acceptable as a proof-of-concept. But if the authors want the cost-saving claim to stand, they need to add ground-truth trajectories (from the robot's onboard odometry or motion capture) and report quantitative errors. Without that, the comparison is essentially anecdotal.\n\nThe citation pattern is fine: the GP/EOT framework comes from [4] and [11], and [4] is a co-author's published paper—this is normal for a systems paper building on an established tracker. No circularity.\n\nWho gets value from this: people working on low-cost sensor selection for indoor robot tracking, or edge-deployed perception. A serious referee should engage with it, but the bar for the main claim should be quantitative validation. My recommendation: send to peer review with a request for major revision, specifically adding ground-truth evaluation or softening the claim to 'qualitatively comparable.' As it stands, I would not cite it as evidence of performance equivalence.","headline":"An honest preliminary study whose headline 'similar performance' claim rests on visual inspection, not ground truth.","tokens_in":7395,"tokens_out":3290,"would_cite":false,"duration_ms":29481,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"A stereo camera costing under 400 euros can track a mobile robot indoors about as well as a LiDAR costing more than ten times as much, according to this preliminary study.","keywords":["extended object tracking","stereo camera","LiDAR","point cloud","indoor mobile robot","Gaussian process target model","DBSCAN","smart factory"],"falsifier":"Track the robot along a known path measured by its onboard odometry or an external motion-capture system, compute per-timestep trajectory error for both sensors, and check whether the stereo camera's error is genuinely close to the LiDAR's; if it is not, the claim of similar performance fails.","tokens_in":6365,"feed_emoji":"🤖","tokens_out":6131,"duration_ms":47621,"temperature":0.7,"pith_summary":"The paper reports a preliminary comparison of extended object tracking on 3D point clouds from a LiDAR sensor and a much cheaper stereo camera, both tracking a single mobile robot indoors. It claims that, using the same heuristic detector and Gaussian-process extended object tracker, the stereo camera achieves tracking performance similar to the LiDAR's, despite costing less than one-tenth as much. The tracking results are judged visually from estimated trajectories and extents in two motion scenarios, with detection rates of 98% for LiDAR and 91% for the camera. If the claim holds, surveillance and situational awareness in smart factories could be built around sub-400-euro stereo cameras instead of multi-thousand-euro LiDAR units.","feed_headline":"Stereo camera matches 10x-costlier LiDAR for indoor tracking","feed_subtitle":"Sub-€400 stereo tracking performs like a €4,000+ LiDAR in factory-floor tests.","key_machinery":"The method combines a heuristic detector with an extended object tracker. The detector removes ground points using RANSAC-fitted planes, restricts points to a predefined operation area, clusters remaining points with DBSCAN, computes PCA bounding boxes, and selects the target cluster by comparing geometric features against the robot's known dimensions. The selected points feed an extended object tracker that models the target's extent as a star-convex hypersurface (a shape whose boundary each radial line from the center crosses once) whose radial function is a Gaussian process, updated by an iterated extended Kalman filter with a constant-velocity motion model. This pipeline is identical for both sensors, so the comparison isolates the sensor's point-cloud quality and the fixed hyperparameters.","core_discovery":"The central claim is that a stereo-camera point cloud can drive an extended object tracking pipeline as effectively as a LiDAR point cloud for single-robot tracking in a room-scale indoor setting. The paper asserts that the estimated trajectories and spatial extents produced from the Intel RealSense D435i are similar to those from the Blickfeld Cube 1 in both a turning trajectory and a straight trajectory. This is presented as evidence that the price difference between the sensors, more than tenfold, does not translate into a comparable gap in tracking behavior for this application, and that stereo-based extended object tracking is feasible.","pith_inferences":["Since the paper observes that stereo-camera noise grows with depth, replacing the fixed detector hyperparameters with depth-adaptive thresholds could close the 7-point detection gap and is a natural next step.","The cost-saving conclusion would be stronger with a quantitative benchmark against ground truth, because the current visual comparison alone may not capture small tracking errors that matter for collision avoidance.","The same detector-and-tracker combination could in principle transfer to other low-cost depth sensors with different noise profiles, but the paper does not test that transfer."],"forward_implications":["Room-scale single-robot tracking for smart-factory surveillance could be built on a sub-400-euro stereo camera instead of a multi-thousand-euro LiDAR, cutting sensor cost by more than an order of magnitude.","Because the detector is a fast heuristic rather than a deep network, the same tracking pipeline can run on edge devices.","The reported detection rate is 91% with the camera and 98% with the LiDAR, so the stereo pipeline occasionally loses the target for a frame even when the trajectories look similar.","The camera's short valid depth range (about 3 meters) confines the approach to room-scale indoor tracking rather than the long-range maritime or automotive settings where LiDAR is normally used."],"supporting_citations":[{"why":"It supplies the Gaussian-process extended object tracker and the parameter settings the comparison runs on.","marker":"[4]"},{"why":"It introduces the Gaussian-process target model that parameterizes the star-convex extent.","marker":"[11]"},{"why":"It provides the DBSCAN clustering algorithm used to form target candidates.","marker":"[8]"},{"why":"It defines the Blickfeld Cube 1's range, accuracy, and price, which set the LiDAR baseline.","marker":"[9]"},{"why":"It defines the Intel RealSense D435i's range, depth error, and low cost, which set the camera baseline.","marker":"[10]"},{"why":"It supplies the random hypersurface model for star-convex shapes that the extent parameterization builds on.","marker":"[16]"},{"why":"It provides RANSAC, used to fit the ground plane during detector initialization.","marker":"[17]"},{"why":"It supplies the PCA bounding boxes whose edge lengths and face areas become the geometric detection features.","marker":"[18]"}],"fun_headline_variants":["Stereo camera rivals LiDAR at one-tenth the cost","Cheap stereo tracking matches pricey LiDAR indoors","Budget stereo sensor equals 10x pricier LiDAR","Indoor tracking: stereo camera matches LiDAR for less","Stereo vision tracks as well as LiDAR at 10x lower cost"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The paper assumes that visually similar estimated trajectories and extents are enough to prove similar tracking accuracy, because no ground-truth measurements are used and the paper explicitly defers that evaluation to future work.","fun_headline_variants_meta":{"raw":{"variants":["Stereo camera rivals LiDAR at one-tenth the cost","Cheap stereo tracking matches pricey LiDAR indoors","Budget stereo sensor equals 10x pricier LiDAR","Indoor tracking: stereo camera matches LiDAR for less","Stereo vision tracks as well as LiDAR at 10x lower cost"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000554,"raw_usage":{"total_tokens":2549,"prompt_tokens":765,"completion_tokens":1784,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":381,"completion_tokens_details":{"reasoning_tokens":1698}},"tokens_in":381,"tokens_out":1784,"duration_ms":11107,"temperature":1.0,"reasoning_tokens":1698,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-12T11:08:21.916704+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Track the robot along a known path measured by its onboard odometry or an external motion-capture system, compute per-timestep trajectory error for both sensors, and check whether the stereo camera's error is genuinely close to the LiDAR's; if it is not, the claim of similar performance fails.","supporting_citations":[{"cited_title":"Baerveldt, M","cited_arxiv_id":null,"evidence_quote":"It supplies the Gaussian-process extended object tracker and the parameter settings the comparison runs on."},{"cited_title":"Wahlström, E","cited_arxiv_id":null,"evidence_quote":"It introduces the Gaussian-process target model that parameterizes the star-convex extent."},{"cited_title":"Ester, H.-P","cited_arxiv_id":null,"evidence_quote":"It provides the DBSCAN clustering algorithm used to form target candidates."},{"cited_title":"GmbH, Blickfeld cube 1, 2024","cited_arxiv_id":null,"evidence_quote":"It defines the Blickfeld Cube 1's range, accuracy, and price, which set the LiDAR baseline."},{"cited_title":"Corporation, Intel realsense d435i, 2024","cited_arxiv_id":null,"evidence_quote":"It defines the Intel RealSense D435i's range, depth error, and low cost, which set the camera baseline."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"It provides RANSAC, used to fit the ground plane during detector initialization."},{"cited_title":"Dimitrov, C","cited_arxiv_id":null,"evidence_quote":"It supplies the PCA bounding boxes whose edge lengths and face areas become the geometric detection features."}],"review_version":1}