{"id":"401ba3d3-e289-4f41-96e0-53c3f761933b","arxiv_id":"2608.01535","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":7.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":4,"one_line_summary":"Using automotive radar Doppler as supervision, STAR-VLM enables a vision-language model to estimate metric radial velocity and motion state of objects from video, outperforming zero-shot task-specific baselines on a nuScenes-based benchmark.","lead":"STAR-VLM trains vision-language models using radar Doppler measurements so they can estimate how fast objects are moving and whether they are moving, directly from camera video. Because radar is already on many cars and needs no human labels, this could make metric motion reasoning for driving VLMs cheaper and more scalable.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Radar Doppler measures velocity along the radar's line of sight, not the camera's; using it as 'camera-relative radial velocity' may systematically mislabel tangential motion and conflate radar-relative with camera-relative metrics.","rationale":"The reader's weakest assumption focused on projection and occlusion filtering of radar points. My concern is different and arguably more load-bearing: even with perfect projection, the physical quantity measured by radar Doppler is not the camera-relative radial velocity that the prompt and evaluation claim. This is a correctness issue with the supervision itself, not just its spatial alignment. It affects both training labels and evaluation ground truth, and it explains the systematic gap between Radar and Anno metrics. The paper's central claim — that STAR-VLM estimates metric camera-relative velocity better than task-specific models — is not supported if the model is instead regressing a radar-frame scalar. However, the claim could be salvaged by either correcting the labels (if possible) or explicitly reframing the task as 'radar-relative radial velocity estimation.' Given this, the reader's CONDITIONAL verdict remains appropriate, though for a stronger reason than the one stated. I therefore recommend no change to the verdict, but the revision must address the Doppler frame mismatch.","tokens_in":9669,"tokens_out":5654,"duration_ms":67514,"concrete_test":"Using nuScenes ground-truth 3D boxes with full velocity, compute for each projected radar point the true camera-relative radial velocity v_cam = v_3D · (P_cam − P_t)/|P_cam − P_t| and compare it to the radar Doppler label v_radar. Report the distribution of |v_cam − v_radar| and the fraction of points where the motion-state labels (moving vs static) disagree. If the median absolute difference exceeds ~0.5 m/s or more than ~5% of points flip labels, the radar supervision is not a valid proxy for camera-relative radial velocity. Also inspect the released code (if any) to confirm whether any radar-to-camera coordinate correction is applied — the paper does not describe one.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The radar Doppler measurement gives the projection of the target's 3D velocity onto the radar–target ray: v_radar = v · (P_t − P_r) / |P_t − P_r|. The prompt and evaluation require v_camera = v · (P_t − P_c) / |P_t − P_c|. In nuScenes, the front radar and front camera are not co-located, so these unit vectors differ. For objects with substantial tangential motion (e.g., parallel traffic), v_radar can be near zero while |v_camera| is large, and the supervision labels a visibly moving object as 'static'. STAR-Bench-anno uses full 3D annotations converted to camera-relative radial velocity, so the two benchmarks (Radar vs Anno) measure different physical quantities. The paper (Sec. III-A, III-B2) describes no coordinate correction or conversion. This systematically biases both training and evaluation: the low MAE on STAR-Bench-radar (1.37) may reflect the model learning the radar-specific scalar, not camera-relative metric velocity, and the higher Anno MAE (1.94) is consistent with this mismatch. Motion classification labels similarly conflate radial motion with full motion, making the 0.94 radar accuracy and the SOTA comparison against SegAnyMo uninterpretable until this is resolved.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"STAR-VLM proposes a radar-supervised training framework for vision-language models (VLMs) to perform spatiotemporal motion classification and metric radial velocity estimation at queried pixels. Using nuScenes front-camera images and front-radar measurements, the authors project radar points into the image, apply a hand-designed occlusion filter, and generate QA pairs from radar-derived Doppler/range labels (STAR-Bench-radar) and from human-annotated 3D boxes (STAR-Bench-anno). They fine-tune Qwen3-VL-4B with supervised fine-tuning, jointly training motion classification, radial velocity estimation, and ego-velocity estimation, using a combination of textual pixel coordinates and visual arrow markers as the pixel reference. They report improvements over off-the-shelf VLMs and over task-specific baselines SegAnyMo (motion classification) and Any4D (velocity estimation), plus results on the public TADBench temporal benchmark.","tokens_in":9981,"tokens_out":5663,"duration_ms":74503,"significance":"If the claims hold, the paper would demonstrate a scalable, annotation-free path to metric velocity reasoning in VLMs using widely deployed automotive radar, extending prior depth-supervision work (DepthLM) from spatial to temporal metric perception. The paper has several strengths: it uses a real-world, publicly available dataset; the internal ablation in Table II shows that joint ego-velocity training improves both motion classification and velocity prediction; and the STAR-Bench-anno evaluation provides an independent label source for the velocity task, mitigating the pure circularity of the radar-based benchmark. However, the headline claim of outperforming task-specific methods is weakened by a coordinate-frame ambiguity in the radar labels, by the self-referential nature of the radar benchmark, and by the lack of controlled comparison against task-specific baselines.","major_comments":[{"comment":"The radar Doppler measurement is the projection of the 3D velocity onto the radar–target ray, not the camera–target ray. The paper's prompt explicitly asks for radial velocity 'relative to the camera,' and STAR-Bench-radar labels are radar Doppler measurements projected onto the image plane, but no coordinate correction or conversion is described anywhere in Sec. III. For objects with substantial tangential motion, the radar-relative and camera-relative radial velocities can differ considerably. Therefore the 'Radar' columns in Table I may reflect learning the radar-specific projection rather than camera-relative metric velocity, undermining the comparison to Any4D and the claim of camera-relative velocity estimation. The authors must either derive and apply the conversion using the known radar-camera extrinsics and ego-motion, or reframe the benchmark and prompts as radar-relative radia","section":"Sec. III-A, Sec. III-B2, Table I"},{"comment":"STAR-Bench-radar is built from the same radar label pipeline used to generate training data; the reported Radar-MAE of 1.37 is therefore partly a measure of self-consistency between training and evaluation label generation, not an independent test of metric velocity estimation. The independent grounding comes only from STAR-Bench-anno, where labels are derived from 3D annotations. The paper should present the annotation-based results as the primary evidence for metric velocity, and treat the radar-based results as a sensor-reproduction check. Additionally, the occlusion filter parameters (N=30 pixels, tau=0.3 m) and the box-occlusion IoU threshold are chosen without sensitivity analysis; if these choices mislabel occluded or non-occluded points, the bias affects both training and evaluation in the same direction.","section":"Sec. III-A, Sec. IV-C, Table I"},{"comment":"The motion-state labels in STAR-Bench-radar are derived from Doppler radial velocity, so an object moving tangentially to the radar ray can have near-zero Doppler and be labeled 'static.' This conflates radial motion with full motion. The comparison to SegAnyMo, which is designed to segment full 3D/2D motion from video, is therefore not measuring the same physical quantity: the high 'Radar' accuracy of 0.94 may reflect that both STAR-VLM and the radar labels use the same radial criterion. The authors should define the threshold and direction convention for 'moving' and 'static,' report motion classification against the annotation-derived full-velocity labels as the primary metric, and discuss the discrepancy between radar- and annotation-derived motion labels.","section":"Sec. III-A, Sec. III-B2, Table I"},{"comment":"The claim of state-of-the-art performance 'outperforming even task-specific methods' is not supported by a controlled comparison. STAR-VLM is fine-tuned on 34K nuScenes images, while SegAnyMo and Any4D are used as pre-trained, off-the-shelf models without nuScenes fine-tuning. The reported gains over these task-specific baselines may be due to the in-domain training data rather than to the radar supervision or the VLM architecture. The authors should either fine-tune or adapt the task-specific baselines on the same training split, or clearly state that the comparison is against zero-shot task-specific models and temper the 'state-of-the-art' wording accordingly.","section":"Sec. IV-B, Sec. IV-D, Table I"}],"minor_comments":[{"comment":"The table header has inconsistent formatting ('VLMS Size' and the layout of the 'Cosmos-Reason 1 7B' entry). Please clean up the table for readability.","section":"Table I"},{"comment":"The sentence 'no regression or regularization loss is needed in pure vision models' is confusing. It should be 'in vision-language models' or similar, and it would help to clarify what loss is used for the velocity text output.","section":"Sec. III-C1"},{"comment":"The caption and text refer to 'depth estimation accuracy' but give no quantitative scale or metric. Please specify the metric (e.g., relative error, RMSE) and the evaluation set used to produce the figure.","section":"Figure 6"},{"comment":"The implementation details state 34K images and ~900K radar points, but not the number of QA pairs used for training. Reporting the number of motion, velocity, and ego-velocity questions would help reproducibility.","section":"Sec. IV-A"},{"comment":"Reference [31] is incomplete; author and venue information should be completed. Also, several arXiv references lack version or DOI information; consider adding them.","section":"References"}],"recommendation":"major_revision","confidential_remarks":"The core idea—using automotive radar Doppler as supervision for VLM metric velocity reasoning—is timely and potentially publishable, but the current manuscript oversells the results. The radar-vs-camera coordinate issue is the most serious technical concern and must be resolved before publication. The paper also needs to be honest about the circularity of the radar-based benchmark and the lack of controlled task-specific baselines. If the authors can supply a coordinate conversion, re-run or reinterpret the experiments, and add sensitivity/calibration analysis for the occlusion filter, the work would be suitable for a major revision. I would not recommend rejection because the underlying approach appears sound and the issues are addressable within the manuscript's scope."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Worth a look. The idea is genuinely new: use automotive radar Doppler as supervision to give a VLM metric temporal grounding. As far as I can tell from the cited literature, no one has done that. The paper is clearly written, the ablations in Table II are informative, and joint training with ego-velocity is a thoughtful addition. The TADBench results being at all competitive is a nice sign that the model learns something general, not just a benchmark artifact.\n\nThe soft spots are real. The biggest one is the sensor frame issue. Radar Doppler measures velocity along the radar-target ray. The prompt asks for radial velocity 'relative to the camera.' The radar and camera in nuScenes are not co-located. The paper shows no coordinate conversion. That means STAR-Bench-radar labels are radar-relative, STAR-Bench-anno labels are camera-relative (from 3D boxes), so the two benchmarks are not measuring the same physical quantity. The fact that the radar MAE (1.37) is much lower than the anno MAE (1.94) is consistent with the model learning the radar-specific scalar rather than camera-relative motion. This is not fatal—for far objects the difference is small—but for nearby traffic it can be several m/s, which is exactly the range the benchmark cares about. The authors need to quantify this discrepancy or explicitly justify the approximation.\n\nRelated, the motion classification labels are derived from radar Doppler. Without ego-motion compensation, a car moving tangentially can have near-zero radial velocity and be labeled 'static.' The paper doesn't describe how motion status is computed from the radar point clouds. This is a real gap.\n\nThe circularity concern the reader flags stands: training and radar evaluation use the same radar label pipeline. Even with a held-out split, that's a self-constructed benchmark. The Anno benchmark is independent of radar labels, but those are object-level and use a different query point, so they aren't a direct validation of pixel-level radial velocity. There are also no error bars anywhere, and the task-specific baselines (SegAnyMo, Any4D) are evaluated zero-shot with no fine-tuning, which weakens the 'outperforms task-specific methods' claim.\n\nWhere do I land? The core mechanism—radar supervision improves VLM motion reasoning—is supported by the ablations and TADBench. The specific SOTA velocity numbers should be taken with salt until the frame issue is addressed. This is worth a serious referee: the idea is significant, the paper is honest about its limitation to relative radial velocity, and the main flaws are fixable with additional analysis rather than requiring a different method. Send it to peer review, but the authors should be asked to handle the coordinate frame and external validation explicitly.","headline":"Radar Doppler as VLM supervision is a genuinely new idea with solid ablations, but the radar-relative vs camera-relative frame mismatch makes the headline velocity numbers less interpretable.","tokens_in":10482,"tokens_out":5019,"would_cite":true,"duration_ms":58258,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"STAR-VLM claims that a vision-language model supervised by projected automotive radar Doppler data can classify object motion and estimate per-pixel metric radial velocity, beating task-specific motion and scene-flow models on the authors'","keywords":["vision-language models","automotive radar","Doppler supervision","radial velocity estimation","motion classification","spatiotemporal reasoning","autonomous driving","metric supervision"],"falsifier":"On held-out nuScenes frames, replace the radar-derived STAR-Bench labels with an independent reference—e.g., velocities from high-rate LiDAR tracking or manual object annotation—and recompute STAR-VLM's motion accuracy and radial velocity MAE on visible, unoccluded objects. If static surfaces are frequently labeled moving, or if the radar labels deviate from the independent reference by more than ~1.4 m/s, the reported gains are an artifact of label noise rather than VLM reasoning.","tokens_in":9571,"feed_emoji":"📡","tokens_out":10652,"duration_ms":91513,"temperature":0.7,"pith_summary":"The paper seeks to establish that a general vision-language model can learn metric spatiotemporal reasoning—deciding whether a queried object is moving and how fast it approaches or recedes—from automotive radar Doppler measurements, without human annotation for its main training signal and without a task-specific architecture. Radar points are projected onto the camera image, filtered for occlusion, and converted into question-answer pairs that ask the model to report the motion state and radial velocity at a marked pixel. On the authors' STAR-Bench evaluation, the 4B model reaches motion classification accuracy 0.80/0.94 (human-annotation/radar labels) and radial velocity MAE 1.94/1.37 m/s, improving on general VLMs and, on these metrics, on task-specific baselines SegAnyMo (0.73/0.77) and Any4D (2.31/3.60). The broader claim is that radar is a scalable, physically grounded supervision source that can give VLMs the temporal counterpart of what LiDAR depth supervision gave spatial perception.","feed_headline":"Radar-trained VLM beats task-specific motion and velocity models","feed_subtitle":"Doppler radar labels, not human annotations, give a VLM a metric grasp of moving-object speed in driving scenes.","key_machinery":"Radar-as-supervisor: automobile-grade radar provides point-wise range and Doppler radial velocity; these returns are projected onto the camera image, filtered by a nearest-neighbor occlusion heuristic (30 pixels, 0.3 m depth gap), and turned into visual-question-answer training pairs with a rendered arrow plus textual pixel coordinates. Cross-task joint training with ego-velocity estimation is the complementary mechanism that lets the model compensate for ego motion and improves both motion classification and radial velocity output.","core_discovery":"Doppler radar can act as a free, physically consistent teacher for VLM spatiotemporal understanding. STAR-VLM projects radar returns with range and Doppler velocity into camera frames, filters occluded points, and builds QA pairs asking the model to output motion state and radial velocity at a marked pixel. Using a 4B Qwen3-VL fine-tuned with standard next-token SFT, no regression loss or architectural change, plus an ego-velocity estimation task that helps separate ego motion from object motion, the model reaches 0.80/0.94 motion accuracy and 1.94/1.37 m/s radial velocity MAE on STAR-Bench, surpassing task-specific methods. The authors take this as evidence that radar is a scalable supervis","pith_inferences":["Because radar Doppler is label-free and already deployed on production vehicles, the same projection-plus-filter supervision recipe could be applied to other multi-modal driving datasets or fleet data, scaling spatiotemporal VLM training without new annotation pipelines.","The occlusion filter is a blunt heuristic; a learned or depth-aware occlusion model would likely recover more valid radar points and could tighten supervision, a testable extension the paper leaves open.","Radial velocity is only one component of full 3D motion; combining this radar-supervised VLM with tangential velocity cues (optical flow or a second radar) is a natural next step toward full-velocity estimation, beyond the limitation the authors state.","The joint pixel-coordinate plus visual-marker prompt design may transfer to other point-level metric queries (depth, object size, distance) beyond velocity, since it solves the pixel-grounding problem generally."],"forward_implications":["A single general VLM can handle both motion classification and metric velocity estimation at a queried pixel, removing the need for task-specific segmentation or scene-flow heads.","Radar-derived supervision can replace human annotation for spatiotemporal training, since Doppler and ego-motion signals are obtained without labeling labor.","Jointly training with ego-velocity estimation improves both motion classification and radial velocity accuracy compared with training on those tasks alone.","The prompting design generalizes beyond marker-based queries: STAR-VLM retains performance on the TADBench format that refers to objects only by natural language.","The same training recipe yields better metric velocity accuracy than task-specific dense scene flow on the evaluation set."],"supporting_citations":[{"why":"Provides the synchronized camera-radar data, ego-pose, and human annotations that STAR-Bench is built from.","marker":"[36]"},{"why":"Supplies the depth-supervision paradigm, visual-marking prompt design, SFT training recipe, and camera focal-length normalization that STAR-VLM adapts.","marker":"[6]"},{"why":"Qwen3-VL is the pretrained base model for STAR-VLM and several of the comparison VLMs.","marker":"[37]"},{"why":"SegAnyMo is the task-specific motion segmentation baseline whose per-pixel outputs are compared on motion classification.","marker":"[30]"},{"why":"Any4D is the task-specific metric scene-flow baseline whose per-pixel radial velocity is compared on velocity estimation.","marker":"[34]"},{"why":"TADBench supplies the public temporal-understanding QA benchmark used to test generalization under a different prompt format.","marker":"[14]"},{"why":"SpatialVLM provides the supervised fine-tuning approach for endowing VLMs with spatial reasoning.","marker":"[3]"}],"fun_headline_variants":["Doppler radar trains VLM to beat speed-specialist models","Radar-labeled VLM nails motion and velocity, outdoes task-specific models","Free radar supervision gives VLM metric motion sense","VLM learns velocity from radar, tops custom models","Radar as teacher: VLM matches task-specific motion experts"],"cache_read_input_tokens":2816,"weakest_assumption_plain":"Radar Doppler returns, after projection to the image and the hand-designed occlusion filter, give correct radial-velocity and motion-state ground truth for the queried pixel; if radar points attach to the wrong image location or the filter systematically removes valid points, the model is trained and scored on false labels.","fun_headline_variants_meta":{"raw":{"variants":["Doppler radar trains VLM to beat speed-specialist models","Radar-labeled VLM nails motion and velocity, outdoes task-specific models","Free radar supervision gives VLM metric motion sense","VLM learns velocity from radar, tops custom models","Radar as teacher: VLM matches task-specific motion experts"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000522,"raw_usage":{"total_tokens":2386,"prompt_tokens":790,"completion_tokens":1596,"prompt_tokens_details":{"cached_tokens":256},"prompt_cache_hit_tokens":256,"prompt_cache_miss_tokens":534,"completion_tokens_details":{"reasoning_tokens":1525}},"tokens_in":534,"tokens_out":1596,"duration_ms":12615,"temperature":1.0,"reasoning_tokens":1525,"cache_read_input_tokens":256,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-06T00:02:22.619482+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"On held-out nuScenes frames, replace the radar-derived STAR-Bench labels with an independent reference—e.g., velocities from high-rate LiDAR tracking or manual object annotation—and recompute STAR-VLM's motion accuracy and radial velocity MAE on visible, unoccluded objects. If static surfaces are frequently labeled moving, or if the radar labels deviate from the independent reference by more than ~1.4 m/s, the reported gains are an artifact of label noise rather than VLM reasoning.","supporting_citations":[{"cited_title":"nuscenes: A multimodal dataset for au- tonomous driving,","cited_arxiv_id":null,"evidence_quote":"Provides the synchronized camera-radar data, ego-pose, and human annotations that STAR-Bench is built from."},{"cited_title":"From Segments to Scenes: Temporal Understanding for Agentic Autonomous Driving via Vision-Language Models","cited_arxiv_id":"2512.05277","evidence_quote":"TADBench supplies the public temporal-understanding QA benchmark used to test generalization under a different prompt format."},{"cited_title":"Spatialvlm: Endowing vision-language mod- els with spatial reasoning capabilities,","cited_arxiv_id":null,"evidence_quote":"SpatialVLM provides the supervised fine-tuning approach for endowing VLMs with spatial reasoning."}],"review_version":1}