{"id":"0403275c-3534-4c04-8605-f3f3e157f36a","arxiv_id":"2501.09481","paper_version":3,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":7.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":5,"one_line_summary":"MonoSOWA generates 3D car labels automatically from monocular video and trains a detector that beats prior weakly-supervised methods without using target-domain human annotations.","lead":"This paper trains a 3D car detector using only ordinary dashcam video, without any human-made 3D labels. It shows that auto-generated labels from a depth estimator and a 2D detector can match the accuracy of training on hand-labeled data.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The pipeline's reliance on Metric3Dv2 for metric depth is not independently validated on the target datasets; a systematic depth scale bias would propagate into every pseudo-label and trained detector.","rationale":"The reader identified the same weakest assumption: the off-the-shelf monocular depth estimator's metric accuracy is the linchpin of the auto-labelling pipeline. The paper's own LiDAR-vs-pseudo-LiDAR ablation (Table XII) confirms that depth quality materially affects the final detector, with a ~5 AP point gap on KITTI Easy BEV@0.5. This validates that depth errors are a real limiting factor, not a hypothetical one. However, the paper's empirical results on three datasets show the method works despite this gap, so the concern is about generalization and the strength of the 'scalable' claim, not about the reported numbers being invalid. The reader's CONDITIONAL verdict already accounts for this; no additional adjustment is needed, hence UNCHANGED.","tokens_in":16978,"tokens_out":8716,"duration_ms":84641,"concrete_test":"Compute metric depth error of Metric3Dv2 on a held-out subset of KITTI-360 (or KITTI) against LiDAR: median absolute relative error and scale-invariant error, plus a per-sequence scale factor. Then feed a globally scale-corrected depth map into the auto-labeller and retrain MonoSOWA; if AP changes by more than a few points, the pipeline is sensitive to depth scale bias and the claim of zero-shot scalability is not robust.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The auto-labelling pipeline's correctness rests on Metric3Dv2 producing metric, scale-correct depth in arbitrary driving scenes (Section III-A, Eq. 1). The paper never validates this on the target datasets independently of the final detection AP. The only evidence is Table XII, which compares pseudo-LiDAR to LiDAR on KITTI and shows a ~5 AP BEV Easy gap at 0.5 IoU, indicating non-trivial depth error. If Metric3Dv2 has a systematic scale bias on any target camera/scene, the pseudo-LiDAR points (Eq. 1), the object locations, the LOMM moving/stationary classification, and the yaw estimates all inherit that bias. The detector can then learn a biased depth/position mapping; on a new camera, AP could degrade unpredictably. The paper's 'scalable' claim requires demonstrating that depth scale is accurate across the evaluated datasets, not just that the final AP is high. The absence of an independent depth-scale check is the weakest load-bearing assumption.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper presents MonoSOWA, an automatic pipeline that generates 3D vehicle labels from monocular video without target-domain human annotations. The pipeline lifts per-frame metric depth predictions (Metric3Dv2) into pseudo-LiDAR, segments vehicles with a COCO-pretrained 2D instance segmentation model (MViTv2-H), tracks objects across frames, classifies each instance as stationary or moving using a proposed Local Object Motion Model (LOMM), and aggregates observations to estimate 3D position, dimensions, and yaw. A Saturated Closeness Criterion is introduced for L-shape fitting, and labels are transformed into a Canonical Object Space (COS) to mix data from cameras with different focal lengths. A MonoDETR detector trained on the resulting pseudo-labels is evaluated on KITTI-360, Waymo, and KITTI, where it outperforms prior weakly-supervised monocular methods at 0.5 IoU, serves as an effective pre-training initialization for fine-tuning with small amounts of human labels, and benefits from combining pseudo-labels across datasets. The auto-labelling pipeline runs in roughly 1.3 seconds per frame, about 700x faster than VSRD.","tokens_in":17172,"tokens_out":11580,"duration_ms":98104,"significance":"The significance of the work is substantial if the results hold. It demonstrates a practical, fully automatic alternative to human 3D annotation for monocular detection, with a fast pipeline and the ability to aggregate data from multiple camera setups. The experiments on three public datasets, the cross-dataset combination, and the pre-training results are valuable. The authors also release code and models. The main caveats are that the pipeline relies on off-the-shelf models whose training used human supervision (so the 'without human annotations' claim must be carefully scoped), and the generalization across new cameras hinges on the metric accuracy of the monocular depth estimator, which is not directly validated on the target datasets. These points do not invalidate the headline results, but they need to be addressed for the scalability claim.","major_comments":[{"comment":"The entire auto-labelling pipeline assumes Metric3Dv2 provides metric, scale-correct depth for arbitrary driving scenes. The paper does not evaluate depth scale directly on the three target datasets. The only quantitative comparison is Table XII on KITTI, where replacing pseudo-LiDAR with real LiDAR in the same pipeline improves APBEV@0.5 Easy from 59.76 to 64.50, indicating non-trivial depth errors. Because the claimed scalability to new camera setups and datasets rests on the depth estimator's metric accuracy, please provide a direct depth-scale evaluation (e.g., median absolute relative error, or scale bias against LiDAR/ground-truth depth) on KITTI-360 and Waymo, or explicitly discuss why the end-to-end detection AP is sufficient evidence.","section":"Section III-A, Eq. (1); Supplementary Table XII"},{"comment":"The text states that when identical inputs are used, our method significantly outperforms VSRD in all metrics (referring to Table X). This is contradicted by Table X: with human-annotated masks, the AP3D@0.3 Easy value for the proposed method is 49.03, below VSRD's 50.86. The corresponding BEV value (59.41 vs 58.40) and the Hard values do improve, but 'all metrics' is incorrect. Please revise this sentence and reassess the explanation for the 0.3 IoU gap in Table I.","section":"Section IV-A, Table X"}],"minor_comments":[{"comment":"The title and the phrase 'without human Annotations' overclaim because the pipeline uses a COCO-pretrained 2D detector and a depth estimator that were trained with human-annotated data. The abstract's qualifier 'domain-specific human annotations' is accurate, but the title and the first contribution sentence should be scoped to 'target-domain human annotations' to avoid misleading readers.","section":"Title / Abstract"},{"comment":"The variable ranges are swapped: 'u ∈ [0, h]' should be 'u ∈ [0, w]' and 'v ∈ [0, h]' (image width and height), and the text says 'h and w denotes image height and width'.","section":"Section III-A, Eq. (1)"},{"comment":"The third formula should read zC = z · ωi, not zC = y · ωi.","section":"Section III-D, Eq. (9)"},{"comment":"The definition of σj differs from the usual standard deviation by a factor of 1/√2. Please clarify whether this is intentional and how it affects the z-ratio threshold.","section":"Section III-B, Eq. (4)"},{"comment":"The thresholds Tz and Tm in the LOMM are fixed across datasets, but no ablation is reported for them. Please add a sensitivity analysis or justify the chosen values.","section":"Section IV-B, hyperparameters"},{"comment":"The phrase 'when identical inputs are used, our method significantly outperforms VSRD in all metrics' should be corrected as described in the major comment.","section":"Section IV-A, KITTI-360 paragraph"},{"comment":"The experiments do not report error bars or multiple seeds; given the small gaps in some comparisons (e.g., Table X AP3D@0.3 Easy), please indicate the variance of the reported AP values.","section":"General"},{"comment":"The header/title contains 'MonoSOW A' with an unwanted space; please correct the spacing throughout.","section":"Typos"}],"recommendation":"major_revision","confidential_remarks":"The depth-scale validation is the main risk for the scalability claim; the paper's results on three datasets are strong, but the missing direct depth-scale check on KITTI-360 and Waymo leaves a gap that should be addressed. The title overclaim and the Table X misstatement are fixable. Overall the core method is sound and within scope."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Here's the short version: this paper does what it claims. It trains a monocular 3D detector on driving data without target-domain human labels, and it beats the previous weakly supervised state of the art (VSRD) at 0.5 IoU by a wide margin. The technical moves that matter are LOMM, which separates ego-motion from object motion so moving cars can be used rather than discarded, and the COS transformation, which lets them train one detector on data from cameras with different focal lengths. The ablations support both pieces: without LOMM the pseudo-label quality drops sharply, and without COS the multi-dataset training actually hurts. The cross-dataset result—KITTI+K360 pseudo-labels reaching roughly the same accuracy as KITTI human labels—is a good demonstration.\n\nThe soft spots are moderate. The biggest is the reliance on Metric3D for metric depth. Everything downstream—object locations, moving/stationary classification, yaw—inherits any scale bias in the depth. The paper doesn't validate depth scale on the target datasets directly; it only shows final AP and the LiDAR-vs-pseudo comparison in Table XII. That table shows a real gap: 64.5 vs 59.8 BEV Easy at 0.5 IoU. So the concern is legitimate, but it is not fatal—the method still works, and the gap gives an honest measure of what depth error costs. A sharper test would be reporting depth error on a held-out slice of the target data before training.\n\nOther issues are smaller. There are no error bars anywhere, so we don't know how much of the 38.41 vs 29.07 lead is luck, though the margin is large. The 0.3 IoU comparison in Table I is misleading; when inputs are matched with VSRD (Table X), the method wins at both thresholds, so the claim holds but the presentation is a bit cherry-picked. The title's 'without human annotations' is a wording nit—the 2D detector is COCO-pretrained, which is not target-domain annotation. And Eq. (9) has a z/y typo.\n\nThe central argument holds up. The method is reproducible in principle, the ablations are honest, and the authors state the far-away limitation in the conclusion. This is a solid paper that deserves serious refereeing. My only asks would be error bars and a direct depth-scale check on at least one target dataset. Send it to review.","headline":"A credible auto-labelling pipeline for monocular 3D detection that delivers on its main claim, with one load-bearing assumption about metric depth that deserves a sharper test.","tokens_in":17746,"tokens_out":2778,"would_cite":true,"duration_ms":25288,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"MonoSOWA trains a 3D car detector from plain video with zero human labels, and beats prior weakly supervised methods on KITTI-360.","keywords":["monocular 3D object detection","auto-labelling","pseudo-LiDAR","weak supervision","temporal consistency","Local Object Motion Model","Canonical Object Space","autonomous driving"],"falsifier":"Use KITTI-360's recorded LiDAR only as a validator: for the frames the pipeline auto-labels, compare each vehicle's median pseudo-LiDAR depth against the LiDAR depth for the same instance, and require the per-object depth error to stay below roughly the 0.5-IoU matching tolerance; if the depth error is systematically biased or grows with distance, the label-free accuracy result should not reproduce.","tokens_in":16737,"feed_emoji":"🚗","tokens_out":9320,"duration_ms":83946,"temperature":0.7,"pith_summary":"MonoSOWA is a method for training a monocular 3D object detector without any human annotations on the target data and without LiDAR. The idea is to replace hand-labelled 3D boxes with pseudo-labels automatically generated from video: an off-the-shelf 2D detector finds vehicles, an off-the-shelf metric depth estimator lifts each vehicle's pixels into a point cloud, and temporal aggregation across frames stabilises those noisy point clouds into 7-DOF boxes. A new Local Object Motion Model separates ego-vehicle motion from the object's own motion so that both stationary and moving cars contribute labels, and a canonical object space rescales labels to compensate for different camera focal lengths. On KITTI-360 the detector trained on these pseudo-labels beats VSRD, the previous weakly-supervised baseline, at 0.5 IoU, and the pipeline labels a frame in about 1.3 seconds versus about 15 minutes for VSRD.","feed_headline":"No human labels: video auto-labelling beats weakly supervised detectors","feed_subtitle":"Pipeline is ~700x faster than prior work and pools data from cameras with different focal lengths.","key_machinery":"The load-bearing mechanism is temporal aggregation in metric pseudo-LiDAR, driven by the Local Object Motion Model (LOMM). LOMM computes per-frame median locations of each tracked instance, forms per-frame displacement vectors, and classifies the object as moving when the ratio $z = \\| \\mu_j \\|_2 / \\| \\sigma_j \\|_2$ exceeds a threshold while net distance also exceeds a threshold; this separates the ego-vehicle's apparent motion from the object's actual motion without any learned motion model. For stationary objects the per-frame pseudo-LiDAR clouds are concatenated into one denser cloud before box fitting; for moving objects yaw is read off the trajectory. Box fitting uses the proposed Saturated Closeness Criterion, a saturating distance that suppresses outlier points, replacing the min/max extremes of an earlier L-shape fitter with 10th/90th percentiles. Finally, every pseudo-label is rescaled by $\\omega_i = f^C / f_i$ into Canonical Object Space, so that a single detector can train on and be deployed across cameras with different focal lengths.","core_discovery":"The paper's central claim is that metric, scale-correct pseudo-LiDAR, accumulated over a driving sequence and disentangled into ego-motion and object motion, is enough supervision to train a competitive monocular 3D detector. For each vehicle, the pipeline takes a 2D instance mask from MViTv2, back-projects the pixels through a Metric3Dv2 depth map to get a per-frame point cloud, tracks the vehicle by median position, and classifies it as stationary or moving using the ratio of mean displacement to displacement jitter (LOMM). Stationary vehicles get a fused multi-frame point cloud; moving vehicles get yaw from their trajectory; a Saturated Closeness Criterion then fits an oriented 3D box, and Template Fitting Loss refines position and front/back orientation. The resulting pseudo-labels train a MonoDETR detector, with all boxes rescaled into a Canonical Object Space so that data from cameras with different focal lengths can be pooled. The paper reports that this label-free recipe reaches APBEV/AP3D@0.5 of 38.41/29.98 on KITTI-360 Easy, beating the weakly supervised VSRD (29.07/21.77), and that pre-training with pseudo-labels lets a detector fine-tuned on 25% of KITTI human labels outperform a fully supervised detector trained from scratch on 100%.","pith_inferences":["Beyond the paper, if the off-the-shelf depth estimator's metric accuracy continues to improve, this pipeline should improve without any architecture change; a direct test would be replacing Metric3Dv2 with a newer metric-depth model and checking whether pseudo-label AP rises.","Beyond the paper, the method's components are not vehicle-specific: swapping the 2D detector for another category suggests the same temporal-aggregation recipe could auto-label pedestrians, cyclists, or other rigid objects, though the paper only evaluates cars.","Beyond the paper, training in canonical object space implies a detector produced this way may transfer to an unseen camera whose intrinsics are known; the paper shows cross-dataset transfer between KITTI and KITTI-360 but does not test truly novel camera geometries.","The authors speculate that fusing sparse low-quality LiDAR with dense pseudo-LiDAR could correct depth drift; if that speculation holds, it offers a practical middle path between fully label-free and LiDAR-assisted labelling."],"forward_implications":["Large collections of ordinary driving video, including footage from single-camera vehicles with no LiDAR, become usable as training data for monocular 3D detection.","The roughly 700x labelling speed-up makes auto-labelling practical at dataset scale, where the previous method's 15 minutes per frame is prohibitive.","With only 25% of human labels on KITTI, a detector pre-trained on MonoSOWA pseudo-labels outperforms a fully supervised detector trained on 100% of the labels; on Waymo, 15% human labels plus pseudo-labels nearly matches 100% human labels.","Combining pseudo-labels from KITTI and KITTI-360, two datasets with different cameras, yields accuracy comparable to training on one dataset's human labels, and exceeds human-label training in the Hard category at 0.3 IoU.","Because no human annotations or LiDAR are needed for a new camera setup, the same pipeline can be rerun whenever new cameras appear instead of repeating the annotation process."],"supporting_citations":[{"why":"Metric3Dv2 is the off-the-shelf metric depth estimator whose depth maps are lifted into the pseudo-LiDAR point clouds that seed all labelling.","marker":"[11]"},{"why":"MViTv2-H provides the 2D instance masks and detections that select which pseudo-LiDAR points belong to each vehicle.","marker":"[40]"},{"why":"MonoDETR is the detector architecture trained exclusively on the generated pseudo-labels and evaluated on all three datasets.","marker":"[20]"},{"why":"VSRD is the prior weakly supervised method that defines the baseline to beat and the 15-minutes-per-frame speed comparison.","marker":"[4]"},{"why":"McCraith et al. supplies the template-fitting-into-LiDAR idea and the multi-frame consistency principle for stationary vehicles that the pipeline adapts.","marker":"[29]"},{"why":"Zhang et al. provides the L-shape fitting algorithm whose Closeness Criterion is replaced by the Saturated Closeness Criterion.","marker":"[41]"},{"why":"TCC-Det supplies the Template Fitting Loss used for position refinement and front/back disambiguation.","marker":"[30]"},{"why":"KITTI-360 is the primary dataset for pseudo-label generation, training, and comparison against prior weakly supervised methods.","marker":"[42]"}],"fun_headline_variants":["700x faster pseudo-label pipeline for monocular 3D detection","No labels, no LiDAR: MonoSOWA trains 3D detectors from RGB video","Pseudo-labels from video: 3D detection without human annotation","MonoSOWA: label-free monocular 3D detection, 700x faster","Video auto-labels replace human annotations for 3D detection"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"Everything rests on the off-the-shelf monocular depth estimator producing metric, scale-correct depth for arbitrary driving scenes; if its scale is wrong, every pseudo-LiDAR point, every 3D label, and the trained detector inherit that error.","fun_headline_variants_meta":{"raw":{"variants":["700x faster pseudo-label pipeline for monocular 3D detection","No labels, no LiDAR: MonoSOWA trains 3D detectors from RGB video","Pseudo-labels from video: 3D detection without human annotation","MonoSOWA: label-free monocular 3D detection, 700x faster","Video auto-labels replace human annotations for 3D detection"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000789,"raw_usage":{"total_tokens":3530,"prompt_tokens":1048,"completion_tokens":2482,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":664,"completion_tokens_details":{"reasoning_tokens":2379}},"tokens_in":664,"tokens_out":2482,"duration_ms":19049,"temperature":1.0,"reasoning_tokens":2379,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-10T19:57:34.629850+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Use KITTI-360's recorded LiDAR only as a validator: for the frames the pipeline auto-labels, compare each vehicle's median pseudo-LiDAR depth against the LiDAR depth for the same instance, and require the per-object depth error to stay below roughly the 0.5-IoU matching tolerance; if the depth error is systematically biased or grows with distance, the label-free accuracy result should not reproduce.","supporting_citations":[{"cited_title":"Mvitv2: Improved multiscale vision transformers for classification and detection,","cited_arxiv_id":null,"evidence_quote":"MViTv2-H provides the 2D instance masks and detections that select which pseudo-LiDAR points belong to each vehicle."},{"cited_title":"Monodetr: Depth-guided transformer for monocular 3d object detection,","cited_arxiv_id":null,"evidence_quote":"MonoDETR is the detector architecture trained exclusively on the generated pseudo-labels and evaluated on all three datasets."},{"cited_title":"Vsrd: Instance-aware volumetric silhouette rendering for weakly supervised 3d object detection,","cited_arxiv_id":null,"evidence_quote":"VSRD is the prior weakly supervised method that defines the baseline to beat and the 15-minutes-per-frame speed comparison."},{"cited_title":"Lifting 2D object locations to 3D by discounting LiDAR outliers across objects and views,","cited_arxiv_id":null,"evidence_quote":"McCraith et al. supplies the template-fitting-into-LiDAR idea and the multi-frame consistency principle for stationary vehicles that the pipeline adapts."},{"cited_title":"Efficient l-shape fitting for vehicle detection using laser scanners,","cited_arxiv_id":null,"evidence_quote":"Zhang et al. provides the L-shape fitting algorithm whose Closeness Criterion is replaced by the Saturated Closeness Criterion."},{"cited_title":"TCC-Det: Temporarily consistent cues for weakly-supervised 3D detection,","cited_arxiv_id":null,"evidence_quote":"TCC-Det supplies the Template Fitting Loss used for position refinement and front/back disambiguation."},{"cited_title":"Kitti-360: A novel dataset and bench- marks for urban scene understanding in 2d and 3d,","cited_arxiv_id":null,"evidence_quote":"KITTI-360 is the primary dataset for pseudo-label generation, training, and comparison against prior weakly supervised methods."}],"review_version":1}