{"id":"c5cee6c2-6fff-4f4b-ac07-fd682675e0e5","arxiv_id":"2411.10293","paper_version":3,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":4,"one_line_summary":"RETR adapts DETR to multi-view radar heatmaps with depth-prioritized positional encoding, tri-plane loss, and learnable radar-to-camera transform, achieving 15+ AP and 12+ IoU gains over RFMask.","lead":"The paper presents RETR, a transformer-based detector that fuses horizontal and vertical radar heatmaps to detect and segment people indoors. It reports large gains over baseline detectors on two indoor radar datasets, with code released.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The claimed 15.38 AP / 11.91 IoU margin over 'state-of-the-art' is computed against modified RFMask/DETR baselines on refined labels, not the original published methods, so the headline comparison is unverified as stated.","rationale":"The central claim is not merely that the architecture works, but that it outperforms existing state-of-the-art by a specific margin. That is a comparative empirical claim, and the comparison that substantiates it is not against the existing state of the art. The baselines were modified: RFMask gains an image-plane BBox regression module, DETR gains top-K selection and two-view input, and the HIBER ground truth is refined. These modifications could make the baselines stronger (conservative comparison) or weaker (inflated margin), but the paper does not establish which. The reader's weakest-assumption point about aligned depth axes and rigid radar-to-camera transforms is a legitimate inductive-bias risk, but it is secondary: even if the geometry is perfectly aligned, the headline margin is not established unless the comparison is fair. This is why I identify the baseline and label-refinement issue as the single most load-bearing concern. The fix is inexpensive: run the original RFMask on the original labels and report seed variance. If the margin persists, the central claim stands; if it does not, the abstract's wording should be qualified. Because the concern is addressable and does not by itself invalidate the reported internal gains of RETR, the reader's CONDITIONAL verdict remains appropriate.","tokens_in":21775,"tokens_out":4577,"duration_ms":42929,"concrete_test":"Run the original unmodified RFMask implementation on the original HIBER labels and on MMVR with its original labels, and run RETR under the same conditions; also report three training seeds for RETR and for the modified RFMask baseline. If RETR's AP/IoU margin over the unmodified original RFMask remains at least 15.38 AP on MMVR P2S1 and 11.91 IoU on MMVR, the central claim survives; if the margin drops below these values, the abstract's 'margin' should be revised or qualified.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim is the abstract's assertion that RETR 'outperforms existing state-of-the-art methods' by 15.38+ AP and 11.91+ IoU. The tables do not actually compare against the original published state of the art. Section 5.1 says both baselines were modified: RFMask and DETR were given a unified bi-plane H+I loss, and DETR was augmented with top-K selection and two-view input. Appendix E then states that the HIBER labels were 'refined' into tighter BBoxes and that RFMask cannot predict refined BBoxes, so an image-plane BBox regression module was added to RFMask. Thus the 15.38 AP gain over RFMask in Table 1 is a gain over a re-engineered RFMask, not over the RFMask whose published results define the existing baseline. The 11.91 IoU gain in Table 4 has the same issue. The claimed margin is therefore not a verified margin over existing state-of-the-art unless the modified baseline is at least as strong as the original, which is not demonstrated. Without the original published RFMask numbers or a run of the untouched RFMask on the original labels, the headline margin could reflect baseline modification or label refinement rather than the RETR contributions. No error bars or repeated seeds are reported, so it is also unknown whether the 15.38-point gap is stable across training runs. The central empirical claim rests on a comparison that is not the one advertised in the abstract.","agreement_with_reader":"disagree"},"referee_report":{"model":"deepseek-v4-flash","summary":"This paper presents RETR, a DETR-style transformer for object detection and instance segmentation from two-view indoor radar heatmaps (horizontal azimuth-depth and vertical elevation-depth). RETR replaces hand-crafted proposals and NMS with transformer encoder-decoder associations and contributes a tunable positional encoding (TPE) that changes the dimension split between depth and angle embeddings, a tri-plane set-prediction loss that supervises 3D radar boxes in the horizontal radar, vertical radar, and image planes, and a learnable radar-to-camera rigid transformation parameterized via the SO(3) exponential map. The method is evaluated on the MMVR and HIBER datasets, with ablations on the TPE ratio, top-K size, tri-plane loss, learnable transformation, and training data size, plus instance segmentation results on MMVR.","tokens_in":22103,"tokens_out":8437,"duration_ms":75233,"significance":"The method is a clean adaptation of DETR to multi-view radar heatmaps, and its derivations are technically sound: the TPE factorization in Eq. (12) is an algebraic identity, the SO(3) reparameterization in Eq. (18) is standard, and the complexity analysis in Appendix A is coherent. The code is publicly released. If the reported margins over the actual published baselines hold, RETR would be a strong new state of the art for indoor radar image-plane perception, and the TPE and tri-plane ideas are likely transferable to other multi-view range-sensor settings. The main caveat is that the abstract's 'state-of-the-art' comparison is not demonstrated against the original published methods, because the RFMask and DETR baselines were modified and the HIBER labels were refined. This does not invalidate the internal ablation story, but it materially changes the strength of the headline claim.","major_comments":[{"comment":"The headline claim that RETR 'outperforms existing state-of-the-art methods by a margin of 15.38+ AP for object detection and 11.91+ IoU for instance segmentation' is not supported by the comparison actually reported. Table 1 compares RETR (46.75 AP) with an RFMask variant (31.37 AP) that, according to Section 5.1, was enhanced with a unified bi-plane H+I loss, and Table 4 compares RETR with the same modified RFMask. Appendix E further states that the HIBER BBox labels were refined and that RFMask was modified by adding an image-plane BBox regression module because the original RFMask cannot predict refined BBoxes. The paper does not report the original published RFMask numbers, nor a run of the unmodified RFMask on the original or refined labels. Without those numbers, the 15.38 AP and 11.91 IoU margins could reflect the baseline modifications or label refinement rather than the RETR contributions. Please add the original-baseline comparison (or a careful decomposition) and, ideally, repeated-seed statistics, or rephrase the abstract to state that the comparison is against re-implemented, modified baselines.","section":"Abstract; Section 5.1; Appendix E; Tables 1 and 4"}],"minor_comments":[{"comment":"The text says Table 3a reports AP50 and AP75, but the table lists only AP and AR1; the AP50/AP75 behavior appears only in Figure 13 of Appendix G. Please align the table and text.","section":"Section 5.3, Table 3a"},{"comment":"The text says RETR achieves 77.07 IoU, but Table 4 reports 77.21; the abstract's 11.91 margin also corresponds to 77.21, so the in-text number should be corrected.","section":"Appendix B, Table 4"},{"comment":"The sentence reporting 'the improvement is 15.28 AP from DETR to RETR' does not match Table 6, where the difference is 28.98 - 17.00 = 11.98 AP; please correct this and clarify that the surrounding discussion refers to the MULTI split, not WALK.","section":"Appendix G, before Table 6"},{"comment":"There are typos: 's.t. [ω] = ...' and 'With the above reparameterization, the learnable radar-to-camera coordinate transformation in Eq. 7 reduces' has an ungrammatical capital W; also the 'approx' symbol should be an equality because Eq. (18) is the exact Rodrigues formula for the exponential map on SO(3).","section":"Eq. (18)"}],"recommendation":"major_revision","confidential_remarks":"The paper's comparison issue is compounded by the fact that the MMVR benchmark [26] is from the same research group; the authors should be asked to clarify the relationship and to provide independent baseline numbers. The core method is sound and the code is public, so I view this as fixable within a revision rather than grounds for rejection."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Colleague,\n\nThe thing to know about arXiv:2411.10293 is that it is a solid engineering story with a headline number that is not quite what it claims. The paper extends DETR to multi-view radar heatmaps (horizontal azimuth-depth and vertical elevation-depth) and reports a 15.38 AP gain over the prior SOTA, RFMask. The catch: that comparison is against a modified RFMask — given a bi-plane loss and an image-plane regression module — and on HIBER the labels were tightened. So the \"state-of-the-art\" margin as advertised is not directly supported by the tables. That does not mean the result is wrong; the modifications likely make the baseline stronger rather than weaker. But the authors need to run the original RFMask (or report its published numbers on the original labels) before claiming a SOTA margin.\n\nWhat is genuinely new: applying DETR to this sensor setting, with a top-K selection to keep the token count manageable, a tunable positional encoding that re-weights depth vs. angular similarity (a simple reparameterization of Conditional DETR's concatenated encoding), a tri-plane loss supervising 3D boxes in radar and projected image planes, and a learnable radar-to-camera transform via SO(3) reparameterization. None of these components is individually deep, but the combination is sensible and the ablations show each contributes. The paper ships code, uses two datasets (MMVR and HIBER), and includes useful hyperparameter studies (alpha, K, data size, inference time). The internal comparisons are consistent: adding the vertical view, TPE, and the tri-plane loss each improve AP in the right direction.\n\nSoft spots: the baseline issue is the main one. Also, no error bars or repeated seeds, so we do not know if a 15-point gap is stable. There are a few text/table mismatches (e.g., Table 4 in the appendix says 77.07 IoU while the main text says 77.21; the appendix claims a 15.28 AP gain on HIBER MULTI that does not match the table). Minor but sloppy. On HIBER the gains are smaller (4.32 AP), which the authors attribute to the walking-only actions; that is plausible.\n\nWho this is for: radar perception researchers and anyone applying DETR to non-RGB sensors. It deserves a serious referee. I would send it to review with a request for original-baseline numbers, error bars, and cleanup of the inconsistencies. The core idea is coherent and the implementation is honest — the limitations section even discusses ghost targets and weak reflections.\n\nHope that helps.","headline":"Solid DETR-for-radar engineering, but the advertised 15.38 AP SOTA margin is measured against a modified baseline on refined labels, so the headline claim needs verification.","tokens_in":22647,"tokens_out":3189,"would_cite":true,"duration_ms":28025,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Extending DETR to two radar heatmap views, with depth-prioritized attention, a tri-plane loss, and a learnable radar-to-camera transform, yields the best reported indoor radar detection and segmentation: 15.38+ AP and 11.91+ IoU over…","keywords":["multi-view radar perception","radar detection transformer","DETR extension","indoor radar","instance segmentation","tunable positional encoding","radar-to-camera calibration"],"falsifier":"Evaluate RETR on a two-view radar capture with deliberately shifted range bins between the views or with a non-co-located array pair; a small accuracy drop compared with the aligned setting would show the shared-depth assumption is not load-bearing, while a large drop would confirm it. Separately, measuring AP on an array mount with known non-rigid distortion would test whether the rigid SO(3) transform is essential.","tokens_in":21580,"feed_emoji":"📡","tokens_out":6817,"duration_ms":59833,"temperature":0.7,"pith_summary":"RETR is a transformer-based detector that takes two radar heatmaps, one horizontal (azimuth-depth) and one vertical (elevation-depth), and produces image-plane bounding boxes and instance masks. The paper's claim is that by treating the two views as sharing a depth axis and letting a tunable positional encoding prioritize depth similarity, the DETR-style set-prediction machinery can associate the views without hand-crafted proposals or non-maximum suppression. The authors report that RETR beats prior methods by 15.38+ average precision for detection and 11.91+ IoU for segmentation on two indoor radar datasets. A tri-plane loss supervises the predicted 3D boxes in both radar planes and the image plane, and a learnable SO(3) transform replaces a calibrated radar-to-camera mapping. If true, this would make radar a practical, privacy-preserving sensor for indoor monitoring tasks that normally require cameras.","feed_headline":"Two-view radar transformer tops prior indoor detectors by 15+ AP","feed_subtitle":"It detects and segments people from two radar heatmaps alone, using depth-prioritized attention instead of hand-built proposals.","key_machinery":"The load-bearing object is the tunable positional encoding (TPE), a positional embedding split into a depth part and an angular part whose dimensions are controlled by a single ratio alpha. Because the horizontal and vertical heatmaps share the depth axis, dot products of depth embeddings stay meaningful across views while angular embeddings compare azimuth with elevation. In RETR's self-attention and cross-attention, TPE makes queries and keys at similar depths score higher, and the paper adds a tri-plane loss and a learnable SO(3) transform on top of this mechanism. The encoder pools top-K strongest features from each view and lets self-attention associate them, while the decoder uses object queries with 3D spatial embeddings to attend to both views.","core_discovery":"The central claim is that multi-view radar perception should not be treated as an image problem; the distinctive geometry of two radar views, especially the shared depth axis, should be built into the attention architecture. RETR does this in three moves: a depth-prioritized positional encoding (TPE) that makes cross-view attention prefer tokens at similar depths, a tri-plane set-prediction loss that evaluates the same 3D box in the horizontal radar plane, vertical radar plane, and image plane, and a learnable radar-to-camera transformation parameterized through the exponential map so the rotation stays on SO(3). The paper reports that on the MMVR P2S1 protocol RETR reaches 46.75 AP versus 31.37 for RFMask, a 15.38-point gain, and 77.21 IoU versus 65.30, an 11.91-point gain; on HIBER WALK the detection gain is 4.32 AP.","pith_inferences":["The TPE idea, a shared-axis depth-prioritizing positional encoding, is not radar-specific; any two-view sensing setup with a common range coordinate, such as stereo cameras or LiDAR-camera pairs, could adopt it, so the architectural lesson may transfer.","The reported margins depend on two indoor radar datasets with co-located arrays; an untested but natural next step is evaluating RETR on automotive 4D radar heatmaps, where the shared-depth assumption also holds but clutter and multipath are different.","Because the authors refine the HIBER ground-truth boxes, some of the gap over the baseline may reflect label quality as well as architecture; re-benchmarking with the original labels would isolate the contribution of the model design."],"forward_implications":["End-to-end radar detection and segmentation no longer need proposal networks or non-maximum suppression, matching DETR's pipeline design for images.","Vertical-view supervision is load-bearing: adding the vertical radar plane to the loss improves AP by 4.47 and IoU by 2.06 on the main benchmark.","The learnable radar-to-camera transform can replace a separate calibration step, improving AP from 42.97 to 46.75 on the MMVR P2S1 protocol.","The performance gain over the proposal-based baseline is much larger for diverse activities such as sitting and writing than for walking-only scenes, because fixed-height proposals fail when people are not standing.","RETR runs at about 42 frames per second on an A40 GPU, so the accuracy gain does not come at the cost of real-time operation."],"supporting_citations":[{"why":"contributes the DETR end-to-end detection framework that RETR extends.","marker":"[3]"},{"why":"introduces the concatenated content/positional embedding mechanism that TPE generalizes.","marker":"[21]"},{"why":"provides the RFMask region-proposal baseline and the HIBER dataset used for evaluation.","marker":"[38]"},{"why":"supplies the MMVR multi-view radar dataset and the P2S1 protocol on which the main results are reported.","marker":"[26]"},{"why":"defines the GIoU box loss used in each plane of the tri-plane set-prediction loss.","marker":"[28]"},{"why":"supplies the Hungarian assignment algorithm used for set-prediction matching.","marker":"[12]"},{"why":"gives the Lie-group exponential map used to keep the learned rotation on SO(3).","marker":"[33]"},{"why":"provides the transformer attention formulation used throughout the encoder-decoder architecture.","marker":"[36]"}],"fun_headline_variants":["Radar transformer tops indoor detectors by 15 AP","Depth-first attention lifts radar detection by 15+ AP","Multi-view radar transformer beats prior methods by 15 AP","Indoor radar AI: depth-prioritized transformer gains 15 AP"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The method assumes the horizontal and vertical radar heatmaps share one aligned depth axis and that the radar-to-camera mapping is a single rigid rotation plus translation; if the views are not synchronized or the geometry is non-rigid, the depth-prioritized attention and the projection supervision lose their justification.","fun_headline_variants_meta":{"raw":{"variants":["Radar transformer tops indoor detectors by 15 AP","Depth-first attention lifts radar detection by 15+ AP","Multi-view radar transformer beats prior methods by 15 AP","Indoor radar AI: depth-prioritized transformer gains 15 AP"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000447,"raw_usage":{"total_tokens":2271,"prompt_tokens":974,"completion_tokens":1297,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":590,"completion_tokens_details":{"reasoning_tokens":1227}},"tokens_in":590,"tokens_out":1297,"duration_ms":10416,"temperature":1.0,"reasoning_tokens":1227,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-12T19:46:15.488625+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Evaluate RETR on a two-view radar capture with deliberately shifted range bins between the views or with a non-co-located array pair; a small accuracy drop compared with the aligned setting would show the shared-depth assumption is not load-bearing, while a large drop would confirm it. Separately, measuring AP on an array mount with known non-rigid distortion would test whether the rigid SO(3) transform is essential.","supporting_citations":[{"cited_title":"End-to-end object detection with transformers","cited_arxiv_id":null,"evidence_quote":"contributes the DETR end-to-end detection framework that RETR extends."},{"cited_title":"Mahbubur Rahman, Ryoma Yataka, Sorachi Kato, Pu Wang, Peizhao Li, Adriano Cardace, and Petros Boufounos","cited_arxiv_id":null,"evidence_quote":"supplies the MMVR multi-view radar dataset and the P2S1 protocol on which the main results are reported."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"defines the GIoU box loss used in each plane of the tri-plane set-prediction loss."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"provides the transformer attention formulation used throughout the encoder-decoder architecture."}],"review_version":1}