{"id":"ca09c978-7ebf-4681-8998-8dbc8f3bbdde","arxiv_id":"2608.01495","paper_version":1,"verdict":"CONDITIONAL","confidence":"HIGH","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":4,"one_line_summary":"Frozen DETR-family object embeddings encode enough depth and 3D-location information that trained probes come close to specialized 3D detection models.","lead":"The paper probes the internal object embeddings of five pre-trained 2D detection transformers (DETR and variants) to test what they know about object depth and 3D location. It finds that linear and MLP probes recover surprisingly accurate depth and 3D center estimates from these embeddings, even though the detectors were trained only with 2D boxes and class labels.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Table 3's 3D-location claim lacks a 2D-box-only control; a probe on (cx,cy,w,h) may match DETR's 0.50 m Center MAE, making the location result an artifact of projective geometry rather than 3D structure.","rationale":"The reader's weakest-assumption diagnosis — missing class and class+box controls — is partially correct, but the more load-bearing and more precisely actionable issue is the absence of any 2D-geometry control for the 3D-location experiment. The paper does provide a bbox-only ablation for depth (Table 4), which already goes some way toward addressing 2D cues on that task; the class-prior concern applies mainly to the multi-class NYUv2 depth setting. In contrast, the KITTI 3D-center experiment uses a single object class, so class priors are not the confound there; the confound is the strong projective link between 2D box position/size and 3D center location. Since no bbox-only location probe is reported, the headline '0.2 m from MonoDETR' could be reproduced by a probe that only sees the detector's own 2D output. This does not invalidate the paper, but it means the strongest claim — implicit 3D structure beyond 2D supervision — lacks a critical control. Because the paper already presents itself as a probing study and the reader already assigned CONDITIONAL with the same broad concern, my read does not change the verdict; it refines the required experiment. The proposed test is cheap, uses the paper's existing pipeline, and would settle the interpretation either way.","tokens_in":14239,"tokens_out":4677,"duration_ms":65807,"concrete_test":"Train the same linear and MLP probe architectures on KITTI with inputs (a) the 4-d predicted 2D box, (b) the class-logit vector, (c) box+logits, and (d) the query embedding, using the identical greedy IoU matching, train/test split, and metric. Report per-dimension and Center MAE. If the box+logits probe reaches within 0.1 m of the embedding probe's Center MAE (0.50 m), the 3D-location claim is unsupported. Separately, on NYUv2 depth, run probes using class logits alone and class logits+bbox; if they close most of the gap to the embedding probe (0.51 m MAE), the multi-class depth result is also explained by category priors rather than 3D structure.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim that DETR embeddings encode 3D structure rests on probe accuracy, but the paper's only control for 2D geometric cues — the bounding-box probe in Table 4 — is run exclusively on the monocular depth task. The 3D object-location result in Table 3 has no analogous bbox-only probe, and it is the stronger of the two headline findings. In the KITTI setup, detections are restricted to a single vehicle class, so class priors are largely irrelevant; the confound is instead 2D box geometry. For objects on a ground plane, metric depth is tightly constrained by the bottom edge's vertical position, and x/y location follows from depth plus the box center via the camera projection. A 4-dimensional MLP probe on (cx, cy, w, h) can learn these projective relationships from KITTI data. If that bbox-only probe approaches DETR's 0.50 m Center MAE, then the query embeddings add little 3D-location information beyond what the detector's own 2D output head already encodes, and the 'fractions of a meter from MonoDETR' comparison collapses into a statement about 2D geometry. A secondary gap is the NYUv2 depth result, where multi-class priors could explain part of the embedding probe's advantage over the bbox probe; no class-logits or class-plus-box control is provided. Both omissions are fixable and should gate the 3D-understanding interpretation.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper probes object-level query embeddings from five frozen DETR-family 2D detectors (DETR, Deformable DETR, Conditional DETR, RT-DETR v2, LW-DETR) for two 3D properties: object-center depth and 3D object-center location. Using linear and two-layer MLP probes trained on public COCO-pretrained checkpoints, the authors report that depth can be decoded comparably to or better than a zero-shot Depth Anything 2 baseline, and that 3D location can be decoded to within 0.2 m MAE of MonoDETR, a model trained with 3D supervision. They also include PCA-based compressibility analysis and a bounding-box-probe ablation for depth. The central claim is that 2D DETR embeddings implicitly encode substantial 3D structure despite no 3D supervision.","tokens_in":14636,"tokens_out":5067,"duration_ms":57351,"significance":"If substantiated, the finding would be a significant empirical contribution: it would show that standard 2D detection transformers acquire non-trivial 3D object knowledge as a byproduct of 2D detection training, with implications for transfer learning and 3D-aware architecture design. The study has notable strengths: a standard frozen-representation probing protocol, public and reproducible checkpoints, multiple DETR variants, and a code release. These are positive aspects of the work. However, the interpretation that probe success reflects 3D understanding rather than 2D geometric or categorical shortcuts is currently under-supported, and one numerical inconsistency in the headline 3D location result requires attention.","major_comments":[{"comment":"The 3D location experiment has no non-embedding control. The only bounding-box-only probe is for depth (Table 4). For KITTI, objects are essentially on a ground plane and the target is the bottom center of the 3D box; depth is strongly constrained by the vertical position of the 2D box bottom edge, and x/y follow from the box center and depth via the camera model. A 4-dimensional MLP probe on (cx, cy, w, h) could plausibly match or approach DETR's 0.50 m Center MAE. Without this control, the claim that query embeddings encode 3D location beyond projective 2D geometry is not established. Please add a bbox-only probe (and, ideally, a bbox+class probe) to Table 3.","section":"§4.2, Table 3"},{"comment":"There is an internal inconsistency in the reported 3D location numbers. For DETR, x MAE = 0.33 m, y MAE = 0.12 m, and z MAE = 1.03 m, yet Center MAE is reported as 0.50 m. Since the per-sample Euclidean distance is at least |dz|, the mean center distance cannot be smaller than the mean |dz|. This suggests an error in either the component MAE computation or the Center MAE computation. Because the headline '0.5 m vs 0.3 m' comparison to MonoDETR depends on this number, it must be corrected and re-evaluated.","section":"§4.2, Table 3"},{"comment":"The depth ablation is incomplete for the multi-class NYUv2 setting. The bbox-only probe is a useful control, but it does not control for class distribution or category priors. The authors themselves attribute part of the embedding probe's advantage to 'semantic context, category priors' (§4.3). A probe on class logits alone, and a class+bbox probe, would determine how much of the depth accuracy is due to knowing the object category versus genuine geometric information in the embeddings. This is directly relevant to the paper's central interpretation.","section":"§4.1, Table 4; §4.3"}],"minor_comments":[{"comment":"Training hyperparameters for the 3D location MLP probe (epochs, warmup, learning-rate schedule) are not specified. Please include them for reproducibility.","section":"§3.2 / Appendix C"},{"comment":"Dataset alignment uses DETR as the anchor model and keeps only objects detected by all models. This may affect cross-model comparisons. Reporting unaligned results or a sensitivity check with a different anchor would increase robustness.","section":"§3.3 / §4.1"},{"comment":"Layerwise depth MAE curves are shown without error bars or significance tests. Given the modest test sizes, some of the apparent layer-to-layer differences may be noise.","section":"Figure 2 (Right)"},{"comment":"In the NYUv2 block, MonoDETR's Zero-shot AbsRel is 497.72% and δ1 is 0.00%; this is likely a formatting artifact or an extreme domain-shift result. Consider a footnote to clarify.","section":"Table 2"},{"comment":"Minor language issues: 'out-performed' should be 'outperformed'; 'DETR3D' should be 'DETR 3D' for consistency; some hyphenation of 'pre-trained' is inconsistent.","section":"Throughout"}],"recommendation":"major_revision","confidential_remarks":"The central concern is the missing bbox-only control for the 3D location task. If a 4-D bbox probe matches DETR's reported 0.50 m Center MAE, the paper's strongest claim would be materially weakened. The Center MAE vs z-MAE inconsistency is also a red flag that the headline number may be computed incorrectly. Both issues are fixable, but they should gate acceptance. The paper is otherwise methodologically sound and likely of interest to the CV/interpretability community."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Worth a look. This paper is the first to probe the query embeddings of transformer-based detectors for 3D properties, which is a real gap: prior probing work on detectors stuck to CNNs, and prior 3D-awareness work used generic backbone or region features. The experiments are cleanly designed, the checkpoints are public, code is linked, and the comparison baselines include fine-tuned heads for Depth Anything 2 and MonoDETR, not just zero-shot. The depth result is the more convincing half: Table 4 shows embedding probes beat 2D-box-only probes by a wide margin on both VKITTI2 and NYUv2, so the embeddings are carrying real depth-relevant signal beyond box geometry.\n\nThe soft spot is exactly where the stress-test note lands. Table 3, the 3D location result, has no bbox-only probe. On KITTI, detections are single-class and objects sit on a ground plane, so depth is tightly constrained by the bottom edge's vertical position, and x/y then follow from depth and box center through the camera model. A 4D probe on (cx, cy, w, h) could plausibly get close to DETR's 0.50 m Center MAE. Without that control, the \"fractions of a meter from MonoDETR\" comparison cannot be attributed to 3D structure in the embeddings. This is not a fatal flaw, but it does undercut the paper's headline claim, and it's fixable: run the same probe on the predicted 2D box, and also on class logits plus box. Also missing: any seed variance or confidence intervals on the small aligned datasets (3k/750 for VKITTI2), and the abstract's \"meets or exceeds Depth Anything 2\" only holds against the zero-shot variant—with learned heads, Depth Anything 2 is better on both datasets.\n\nThe central interpretation should be softened until the location control is in. As is, the paper deserves a serious referee, but my recommendation would be major revision: add the missing probes, report variance, and rephrase the claims accordingly.","headline":"A genuinely new probing target with a solid depth result, but the 3D location claim needs a 2D-box-only control before it can support the '3D understanding' interpretation.","tokens_in":15084,"tokens_out":2163,"would_cite":true,"duration_ms":28363,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Pre-trained 2D detection transformers encode object depth and 3D location in their query embeddings, readable with small probes to within fractions of a meter of task-specific 3D models.","keywords":["DETR","object detection","probing","3D understanding","depth estimation","3D object localization","transformer embeddings","monocular depth"],"falsifier":"Train the same MLP probe on the concatenation of class probabilities and the 2D box (center, width, height) instead of the embedding, with identical matching, alignment, and capacity. If that input reaches the same depth MAE and 3D-center error as the embedding probe, the central claim collapses into a 2D-geometry effect.","tokens_in":14173,"feed_emoji":"🎯","tokens_out":12242,"duration_ms":105788,"temperature":0.7,"pith_summary":"The paper sets out to determine whether 2D detection transformers, trained only with class labels and 2D bounding boxes, nonetheless store information about the 3D world inside their object-level embeddings. It probes five DETR variants with linear and two-layer MLP readouts, targeting object depth and camera-centered 3D location. On monocular depth, the best MLP probe on frozen DETR embeddings beats a zero-shot application of a dedicated depth foundation model; on KITTI 3D location, the same probe lands within 0.2 m of a model explicitly trained for monocular 3D detection. If the interpretation holds, standard 2D detection pretraining implicitly organizes object representations around 3D structure, making these embeddings usable as free 3D priors.","feed_headline":"2D detectors hide 3D depth cues that probes can read out","feed_subtitle":"Probing five DETR variants shows depth and 3D location emerge without any 3D supervision","key_machinery":"The load-bearing object is the DETR query embedding $q_i \\in \\mathbb{R}^D$ — the per-object vector output by the transformer decoder before the class and box heads. The paper freezes the detector and trains simple probes (one affine map, or a two-layer ReLU MLP with hidden dimension 256) to map $q_i$ to a target property: depth sampled at the predicted box center, or the KITTI 3D box center in camera coordinates. Greedy IoU matching links each embedding to ground-truth annotations, and a second alignment step ensures that all models are scored on the same detected objects. The mechanism's power is demonstrated by the ablation in Table 4: swapping the embedding for the 4-dimensional predicted","core_discovery":"On the paper's own terms, the central discovery is that the query embeddings produced by pre-trained DETR-family detectors carry recoverable, object-level 3D information despite never being trained on depth or 3D boxes. The evidence is a probing comparison: an MLP probe on DETR embeddings reaches 0.54 m mean absolute error on Virtual KITTI2 and 0.51 m on NYUv2, while zero-shot Depth Anything 2 reaches 1.49 m and 0.61 m; on KITTI, the probe predicts 3D object-center location with 0.50 m mean absolute error versus 0.30 m for MonoDETR, whose architecture and training are specialized for this task. The paper also reports that the two real-time variants (RT-DETR v2 and LW-DETR) carry noticeably l","pith_inferences":["Editorial inference: the reported depth accuracy may be partly attributable to geometric priors like vertical image position, box scale, and class-specific depth distributions; a probe trained jointly on class logits and the 2D box would isolate the 3D content.","Editorial extension: the same protocol could probe yaw, pitch, and 3D extents on KITTI; strong readouts there would show the embeddings encode a full 3D box rather than just a location.","Editorial extension: the real-time variant deficit could be a capacity effect rather than a fundamental limit; probing a scaled-up RT-DETR or LW-DETR would separate those explanations.","Editorial inference: if 2D detection pretraining is sufficient to organize object embeddings around 3D location, then multi-task or self-supervised detection pipelines may already be learning geometric structure that could be exposed by similar probes rather than learned from scratch."],"forward_implications":["Probing frozen DETR embeddings can produce an object-centric depth estimate without retraining the detector, which could feed monocular 3D detection or depth-aware reasoning as a cheap prior.","A 2D detector plus a small readout may serve as a pragmatic substitute for fully 3D-supervised detectors in settings where only 2D labels are abundant and a few meters of location error is tolerable.","The real-time DETR variants' weaker 3D readout implies that latency optimizations can trade away geometric understanding, not just detection accuracy.","PCA compression results indicate the 3D-relevant signal sits in a low-dimensional subspace of the embedding, so compact probes or embedding compression can preserve much of the depth information.","The small absolute gap to MonoDETR (about 0.2 m center MAE at a 25.3 m median object distance) suggests the remaining 3D deficit is modest, motivating architectural changes that add explicit 3D objectives to close it."],"supporting_citations":[{"why":"Introduces probing classifiers, the methodology used to read out properties from frozen embeddings.","marker":"[1]"},{"why":"Supplies the Virtual KITTI2 synthetic outdoor depth dataset used for the depth probe.","marker":"[4]"},{"why":"Defines the original DETR architecture and its query embeddings, the primary object of study.","marker":"[5]"},{"why":"Provides the KITTI benchmark with 3D bounding-box annotations used for the location probe.","marker":"[12]"},{"why":"Defines RT-DETR v2, a real-time variant compared in the probing study.","marker":"[18]"},{"why":"Defines Conditional DETR, a variant compared for its 3D representational capacity.","marker":"[20]"},{"why":"Supplies the NYUv2 real indoor RGB-D depth targets used for the depth probe.","marker":"[27]"},{"why":"Defines Depth Anything 2, the depth foundation model baseline the probes are compared against.","marker":"[32]"},{"why":"Defines MonoDETR, the 3D-supervised baseline for 3D location prediction.","marker":"[34]"},{"why":"Defines Deformable DETR, a variant compared for its 3D representational capacity.","marker":"[36]"}],"fun_headline_variants":["2D-only DETR still encodes 3D depth for probes","Probes pull 3D depth out of 2D DETR query embeddings","Probing 2D detectors reveals hidden 3D depth","No 3D supervision, yet DETR probes predict depth"],"cache_read_input_tokens":2816,"weakest_assumption_plain":"The load-bearing premise is that probe accuracy reflects 3D structure inside the embeddings, not shallow 2D geometry such as vertical image position, box scale, and category priors; the paper's only control is a 4-dimensional bounding-box probe, so a probe that also includes class information could easily overturn the 3D reading.","fun_headline_variants_meta":{"raw":{"variants":["2D-only DETR still encodes 3D depth for probes","Probes pull 3D depth out of 2D DETR query embeddings","Probing 2D detectors reveals hidden 3D depth","No 3D supervision, yet DETR probes predict depth"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.001051,"raw_usage":{"total_tokens":4227,"prompt_tokens":699,"completion_tokens":3528,"prompt_tokens_details":{"cached_tokens":256},"prompt_cache_hit_tokens":256,"prompt_cache_miss_tokens":443,"completion_tokens_details":{"reasoning_tokens":3449}},"tokens_in":443,"tokens_out":3528,"duration_ms":20642,"temperature":1.0,"reasoning_tokens":3449,"cache_read_input_tokens":256,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-06T00:04:45.447610+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Train the same MLP probe on the concatenation of class probabilities and the 2D box (center, width, height) instead of the embedding, with identical matching, alignment, and capacity. If that input reaches the same depth MAE and 3D-center error as the embedding probe, the central claim collapses into a 2D-geometry effect.","supporting_citations":[{"cited_title":"In: European Conference on Computer Vision","cited_arxiv_id":null,"evidence_quote":"Defines the original DETR architecture and its query embeddings, the primary object of study."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Provides the KITTI benchmark with 3D bounding-box annotations used for the location probe."},{"cited_title":"In: Proceedings of the IEEE/CVF In- ternational Conference on Computer Vision","cited_arxiv_id":null,"evidence_quote":"Defines Conditional DETR, a variant compared for its 3D representational capacity."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Defines MonoDETR, the 3D-supervised baseline for 3D location prediction."}],"review_version":1}