{"id":"bb55d8f1-1213-4fb3-b0ae-837f9da2a899","arxiv_id":"2608.00110","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":4,"one_line_summary":"Privileged 3D evidence distilled at training time lets an RGB-only student outperform an RGB-only baseline across four 3D vision-language benchmarks.","lead":"A teacher model that sees depth, object, and relation evidence during training transfers its spatial reasoning to a student model that only receives RGB images at test time. The resulting RGB-only student beats a matched RGB-only baseline on all 11 metrics across four 3D scene benchmarks, showing some 3D understanding can be learned from flat images alone.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The reported student gains are not isolated from generic knowledge-distillation benefits: no strong RGB-only teacher control is provided, so the privileged-evidence attribution (Eq. 18-19) is underdetermined.","rationale":"The reader's verdict is CONDITIONAL, focusing on the unverified corruption operator in evidence-sensitivity weighting (Eqs. 14-15). That is a legitimate concern: if the corruption disturbs the teacher beyond a counterfactual, the sensitivity scores and distillation weights would be biased, affecting the recoverability decomposition. However, I see a more load-bearing gap: the paper never controls for the generic benefit of knowledge distillation from a stronger teacher. The student S receives soft targets from T; any strong teacher, regardless of whether its strength comes from privileged evidence, could improve a student over a task-only baseline. The recoverability decomposition (Eq. 18-19) assumes that S−B is a direct measurement of transferred privileged gain, but without a condition where a strong RGB-only teacher is distilled, this assumption is untested. This is precisely the type of confound that would undermine the abstract's causal wording ('demonstrates that training-time privileged 3D evidence distillation can convert...'). The paper has strong internal support: the matched-baseline protocol, the 'Evidence off' control (Table 4), and the sensitivity-weighting ablation all suggest the effect is real. The proposed control would solidify the attribution. I therefore do not move the verdict, but I add a specific missing condition that should be met before the central claim is accepted as established.","tokens_in":12177,"tokens_out":13566,"duration_ms":150939,"concrete_test":"Run a control where the student is trained with the same architecture, losses, and steps, but with a teacher that has no privileged evidence and is matched in validation performance to T (e.g., an ensemble of 3-5 RGB-only checkpoints, a LoRA rank increase, or additional fine-tuning steps). Use the same logit and structural distillation losses (with uniform weighting or an equivalent sensitivity weighting if applicable). Compare the student's improvement over B on ScanQA CIDEr and Scan2Cap CIDEr@0.5 against the reported gains (10.4 and 19.1). If the control student attains comparable gains, the privileged-evidence attribution is not supported; if it attains significantly smaller gains, the central claim is strengthened.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The paper's central claim is that the RGB-only student S outperforms the matched baseline B because privileged 3D evidence is converted into RGB-only spatial reasoning. The protocol compares S (task loss + logit/structural distillation from the evidence-enabled teacher T) against B (task loss only). This design cannot separate the effect of the teacher's privileged evidence from the general effect of knowledge distillation from a stronger model: soft targets from any better-performing teacher typically regularize and improve a student. The recoverability decomposition (Eq. 18-19) labels Δtrans = S−B as 'transferred privileged gain,' but this attribution requires that the student's improvement is driven specifically by evidence-dependent knowledge. The paper's 'Evidence off' row in Table 4 (97.2 vs B 96.8 on ScanQA CIDEr) shows the evidence, not the extra interface, is what makes T strong; however, no condition exists in which a strong RGB-only teacher (e.g., an ensemble, higher-rank LoRA, or extended fine-tuning without privileged evidence) is distilled into the same student with the same losses. Without such a control, S−B could reflect generic distillation benefits, and the reported recovery ratios (90.4% on ScanQA CIDEr, 52.6% on Scan2Cap) would overstate the transfer of privileged knowledge. This is load-bearing because the abstract's 'demonstrates' claim and the decomposition in Eq. 18-19 both rest on this attribution.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes a privileged-evidence distillation framework for RGB-only 3D vision-language models. A teacher T is trained with depth, object, and relation evidence encoded through a unified evidence interface and injected into selected language layers via sparse residual cross-attention. An RGB-only student S is trained from the frozen teacher using logit distillation, structural (token-interaction) distillation, and evidence-sensitivity weighting that down-weights teacher targets highly dependent on privileged evidence. A matched RGB-only baseline B is trained independently with the same backbone, data, and optimization budget. The paper reports that S outperforms B on all 11 metrics across ScanQA, SQA3D, ScanRefer, and Scan2Cap (e.g., +10.4 ScanQA CIDEr, +19.1 Scan2Cap CIDEr@0.5), and defines a recoverability decomposition Δ_priv = Δ_trans + Δ_res, with transfer ratios R = Δ_trans/Δ_priv.","tokens_in":12629,"tokens_out":5062,"duration_ms":54914,"significance":"The matched baseline is a genuine strength: B and S share initialization, data, trainable-parameter budget, and number of optimization steps, so the comparison S versus B is far cleaner than typical unconstrained KD comparisons. The recoverability decomposition is a transparent and useful accounting identity, and the evidence-sensitivity weighting idea is interesting and plausibly valuable for avoiding imitation of RGB-unsupported teacher behavior. If the attribution to privileged evidence is substantiated, the paper would offer a practical deployment-oriented direction for RGB-only 3D reasoning. However, the central attribution claim is currently underdetermined: no strong RGB-only teacher control isolates privileged-evidence transfer from generic knowledge-distillation benefits, and all results are single-run point estimates without statistical reliability. The corruption operator used for sensitivity weighting and leave-one-out analysis is also asserted, not verified.","major_comments":[{"comment":"The main claim that S−B is a 'transferred privileged gain' requires a control that is absent. The comparison S (task loss + distillation from T) versus B (task loss only) cannot separate the effect of the teacher's privileged 3D evidence from the generic benefit of distilling from a stronger model. Table 4 shows that the teacher with corrupted evidence (97.2 ScanQA CIDEr) is close to B (96.8), which helps rule out the extra interface as the source of teacher strength, but it does not rule out generic KD: a student distilled from a strong RGB-only teacher (e.g., a teacher with more LoRA capacity, an ensemble, or longer training) might achieve a similar S−B gain without any privileged evidence. Please add such an RGB-only-teacher distillation control with the same student objective, or explicitly weaken the attribution language in the abstract, Eq. (18), and the recoverability ratio discus","section":"RGB-Only Student Distillation / Eq. (18)"},{"comment":"All reported results are single-run point estimates with no error bars or significance tests. This matters for the 'all 11 metrics' claim and the recovery ratios: some differences are small in absolute terms, e.g., SQA3D EM S−B = 0.4 and ScanQA BLEU-4 S−B = 0.4 in Table 1; in Table 3 the sensitivity-weighting comparison is 104.8 vs 107.2 CIDEr and 48.9% vs 67.0% recovery. Without multiple seeds or paired bootstrap/statistical tests, the central claim that the student 'consistently outperforms' the baseline could be partly noise. Please report standard deviations over at least 3 runs, or provide paired tests across evaluation examples.","section":"Table 1 / Table 3 / Experimental Setup"},{"comment":"The corruption operator is load-bearing for the evidence-sensitivity weights w(x) and for the leave-one-evidence-out analysis in Table 4. The paper asserts that sample-mismatched depth and shuffled object/relation features 'approximately preserve feature scale and marginal statistics,' but no verification is provided. If the corruption disturbs teacher behavior beyond the intended counterfactual (e.g., by introducing out-of-distribution features), the sensitivity scores would be biased and the distillation weights miscomputed. Please provide diagnostics (e.g., feature mean/variance and teacher-logit statistics under clean vs corrupted evidence), and test sensitivity of the main results to an alternative corruption operator (e.g., moment-matched Gaussian noise or independent permutation variants).","section":"Eqs. (14)-(15) and Table 4"}],"minor_comments":[{"comment":"The notation cM(T) is confusing; c appears to be a metric value, M a metric index, and the superscript notation is nonstandard. Please define the metric M and write M(T), M(S), M(B) or similar.","section":"Eq. (18)"},{"comment":"The row 'Evidence off' is evaluated with the corruption operator applied, so it is not the same as an RGB-only teacher. Clarify this in the caption to avoid implying that the teacher is evaluated without evidence in the ordinary sense.","section":"Table 4"},{"comment":"Recovery percentages in the lower heatmap are reported without denominators, sample sizes, or uncertainty. Cells with 100% recovery are particularly sensitive to near-zero denominators; please provide sample counts and standard errors, or mark cells with small sample sizes.","section":"Figure 3"},{"comment":"No code or data release is mentioned. Given the number of hyperparameters (κ, λ_kd, λ_str, Lv, Nu) and the complexity of the two-phase teacher training, releasing code would substantially improve reproducibility.","section":"General"}],"recommendation":"major_revision","confidential_remarks":"The paper is potentially a solid empirical contribution, but the missing RGB-only-teacher control is a serious attribution gap for the central 'privileged evidence transfer' claim. I would push for that experiment and for statistical reliability before publication. The corruption-operator verification is also needed. The topic and framing fit a vision-language venue, and the matched baseline design is a good starting point, but the current evidence is not yet sufficient to support the abstract's 'demonstrates' claim."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"The paper has a genuinely new setup: privileged 3D evidence (depth, object, relation) is injected into a teacher through a unified evidence-token interface and sparse residual cross-attention, and the student is trained with logit plus structural distillation, down-weighting samples where the teacher depends heavily on evidence. The matched baseline B, teacher T, student S design is a clean analytic device, and the recoverability decomposition is a useful tool for separating transferred gain from residual gap. The empirical results are real: the student beats the baseline on all 11 reported metrics, with large gains on ScanQA and Scan2Cap, and the ablations show that evidence-sensitivity weighting improves recovery and reduces affirmative-answer bias. That is meaningful, reproducible-looking work and I would not be surprised if it became a reference point for RGB-only 3D-VLM deployment.\n\nThe soft spot is exactly what the stress-test note flags: no RGB-only teacher control. The student receives distillation from the privileged teacher, so the gap S−B conflates privileged-evidence transfer with generic knowledge distillation from a stronger model. The \"Evidence off\" row in Table 4 is suggestive—it shows the teacher without evidence is barely above B—but they never distil from that non-privileged teacher with the same losses. Without that control, the decomposition's label of Δtrans as \"transferred privileged gain\" is not fully supported. The abstract's \"demonstrates\" claim overstates what the experiment can distinguish. This is load-bearing for the interpretation, but it is fixable: train an RGB-only teacher (e.g., a stronger LoRA or ensemble without evidence) and distil it into the same student. The current results remain true, but they are about \"distillation from a strong teacher\" rather than specifically \"privileged evidence.\"\n\nOther issues are smaller but real: no code, no error bars or significance tests, single-run point estimates. The corruption operator's scale/marginal-preservation claim is asserted but not verified. These are minor-to-moderate in proportion; they do not undermine the basic student-vs-baseline finding, but they do affect how much weight to put on the fine-grained recoverability numbers.\n\nWho is this for? Researchers working on 3D-VLM distillation, geometry-assisted VLMs, or RGB-only deployment. It deserves a serious referee. My recommendation: send to peer review, but request the RGB-only teacher control as a condition for acceptance, plus error bars and code or a detailed reproducibility appendix.","headline":"Useful distillation framework for RGB-only 3D-VLMs, but the central attribution of student gains to privileged evidence is underdetermined without an RGB-only teacher control.","tokens_in":13028,"tokens_out":2324,"would_cite":true,"duration_ms":27645,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Privileged-evidence distillation can convert training-time 3D cues into RGB-only spatial reasoning, making deployable models stronger.","keywords":["privileged-evidence distillation","RGB-only 3D vision-language models","evidence-sensitivity weighting","spatial reasoning","3D scene understanding","knowledge distillation","recoverability decomposition","hallucination reduction"],"falsifier":"Take a held-out set of scenes with known ground-truth geometry and split it into samples where the answer is recoverable from monocular cues (e.g., relative depth of two visible objects) and samples where it is not (e.g., metric distance to an object fully occluded in the image). Compute the teacher's evidence-sensitivity weights on each subset. The paper's explanation predicts high sensitivity—and hence down-weighting—for the unrecoverable subset and low sensitivity for the recoverable subset. A failure to find this separation, or a re-training without sensitivity weighting that matches the s","tokens_in":1493,"feed_emoji":"📐","tokens_out":5328,"duration_ms":89816,"temperature":0.7,"pith_summary":"The paper tries to establish that a vision-language model that sees only RGB images at deployment can still acquire genuine 3D perception—depth ordering, object relations, spatial layout—if that evidence is used at training time to supervise a teacher, and the teacher's knowledge is selectively distilled into the RGB-only student. It argues that naive imitation of a 3D-enabled teacher is wasteful because some teacher outputs depend on evidence that RGB cannot recover; the correct strategy is to estimate each sample's dependence on 3D evidence and down-weight supervision on highly evidence-dependent samples. The paper reports an RGB-only student that outperforms a carefully matched RGB-only baseline on all 11 benchmark metrics across four 3D scene-understanding tasks, with the largest gains on answer generation and captioning. If true, this means depth sensors and 3D reconstructions are not needed at inference time to get substantially better spatial reasoning—just better training.","feed_headline":"Distilled 3D evidence lifts RGB-only models on all 11 metrics","feed_subtitle":"Depth, object, and relation cues learned during training transfer selectively; inference needs only images and text.","key_machinery":"The load-bearing mechanism is the Unified Evidence Token interface combined with evidence-sensitivity weighting. Depth, object, and relation inputs are encoded into a shared space, combined by a question-conditioned router, compressed into a fixed number of tokens by Q-Former-style cross-attention, and injected as small residual cross-attention updates into selected language layers, so the teacher stays close to the original RGB pathway. To decide what to transfer, the method corrupts evidence by sample-mismatched depth and shuffled object/relation features, measures the Jensen-Shannon divergence between teacher predictions under clean vs corrupted evidence, and down-weights distillation los","core_discovery":"The central claim is that a teacher's privileged 3D gain over a matched RGB-only baseline decomposes into a part RGB can recover and a residual gap, and evidence-sensitivity-guided distillation transfers the recoverable part. The paper's experiments show an RGB-only student beating the baseline on all 11 metrics—10.4 CIDEr on ScanQA and 19.1 CIDEr@0.5 on Scan2Cap—while receiving only RGB frames and a question at inference. Recovery varies by task and skill: 90.4% of the teacher gain on ScanQA but 16.0% on SQA3D, and metric-distance reasoning recovers only about 45%. Transferability is not a fixed property of evidence type; it depends on how much the teacher's target relies on evidence RGB ca","pith_inferences":["A practical triage follows from the recoverability heatmap: invest in additional sensors or 3D annotation chiefly for skills with low recovery (metric distance, some counting), since RGB may never fully internalize them.","The corruption-based sensitivity estimator may transfer to other privileged modalities; if it does, its validity would be confirmed by showing the learned weights are stable across different corruption operators.","One can test the paper's explanation directly with synthetic scenes of known visibility: if sensitivity-weighted distillation down-weights samples whose target object is occluded and the student only improves on visible geometries, the recoverability picture is causal rather than correlational.","The matched-baseline design already controls optimization steps and capacity, but an extra RGB-only control trained with monocular augmentation (e.g., varying viewpoint videos) would say whether the gains are unique to teacher distillation or just to more effective training signal."],"forward_implications":["If the claim holds, 3D-VLMs can be deployed with only RGB images and questions, avoiding depth sensors, point-cloud reconstruction, and 3D annotation at inference time.","Spatial reasoning gains become a training-time cost: any available 3D evidence—sensors, scans, or pseudo-depth—can be injected and selectively distilled without changing the deployable model.","The evidence-sensitivity weighting appears to suppress hallucination and affirmative-answer bias relative to uniform distillation, with 3D-POPE F1 up 8.7 points and Yes% down 24.8 points.","Per-skill recovery results imply that relative-depth and directional-relation skills are mostly RGB-recoverable, while metric-distance reasoning remains strongly dependent on explicit 3D evidence.","The teacher's final-layer injection ablation shows that evidence injection must stay off the final layer; injecting there can push the affirmative-answer rate to 100% on the diagnostic."],"fun_headline_variants":["RGB-only VLMs gain from distilled 3D evidence on all benchmarks","Training-time 3D knowledge transfers to RGB-only student","Teacher 3D, student RGB: gains on all 11 metrics","Distilled 3D cues recover spatial reasoning for RGB-only models","No depth at test time: distillation unlocks 3D reasoning in RGB-only"],"cache_read_input_tokens":14336,"weakest_assumption_plain":"The whole weighting scheme rests on the assumption that corrupting depth by mismatched samples and shuffling object/relation features disrupts only the semantic correspondence of the evidence, while preserving its scale and marginal statistics; if the corruption also disturbs the teacher in unintended ways, the sensitivity scores, the distillation weights, and the reported recovery decomposition would all be biased.","fun_headline_variants_meta":{"raw":{"variants":["RGB-only VLMs gain from distilled 3D evidence on all benchmarks","Training-time 3D knowledge transfers to RGB-only student","Teacher 3D, student RGB: gains on all 11 metrics","Distilled 3D cues recover spatial reasoning for RGB-only models","No depth at test time: distillation unlocks 3D reasoning in RGB-only"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000772,"raw_usage":{"total_tokens":3311,"prompt_tokens":855,"completion_tokens":2456,"prompt_tokens_details":{"cached_tokens":256},"prompt_cache_hit_tokens":256,"prompt_cache_miss_tokens":599,"completion_tokens_details":{"reasoning_tokens":2373}},"tokens_in":599,"tokens_out":2456,"duration_ms":17690,"temperature":1.0,"reasoning_tokens":2373,"cache_read_input_tokens":256,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-04T00:34:34.260929+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Take a held-out set of scenes with known ground-truth geometry and split it into samples where the answer is recoverable from monocular cues (e.g., relative depth of two visible objects) and samples where it is not (e.g., metric distance to an object fully occluded in the image). Compute the teacher's evidence-sensitivity weights on each subset. The paper's explanation predicts high sensitivity—and hence down-weighting—for the unrecoverable subset and low sensitivity for the recoverable subset. A failure to find this separation, or a re-training without sensitivity weighting that matches the s","supporting_citations":[],"review_version":1}