{"id":"615c017d-a339-453c-b010-f046332393a9","arxiv_id":"2607.28590","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":6,"one_line_summary":"Counterfactual present/removed teacher views attribute visually supported corrections and reconstruct student-anchored distillation targets that beat source-mixed multimodal OPD.","lead":"VAD rebuilds distillation targets by keeping only the part of a teacher’s next-token correction that a present-vs-removed visual intervention can explain. On six fine-grained vision benchmarks, 4B and 9B students trained this way beat direct privileged-view and visual-advantage distillation.","discovery_kind":"new_method","skeptic_critique":{"model":"grok-4.5","headline":"The load-bearing risk is that ut may encode crop/degradation artifacts rather than pure visual evidence, so gains may not isolate attribution as claimed.","rationale":"The reader correctly located the weakest assumption: that the fixed marked crop pair yields a ut whose projection isolates visually warranted direction rather than crop/degradation artifacts or residual teacher effects. That assumption is load-bearing for the strongest claim, because every main comparison (Vision-OPD, VA-OPD, Decomposed OPD) and the semantic/offline analyses interpret improvements as evidence for attribution-based reconstruction. The paper’s own limitations and proxy language already flag this; the stress test is whether imperfect or alternative interventions still support the interpretation. No internal contradiction or equation error is apparent; ablations in Table 2 show target construction (not mere scalar shrink) drives gains, which is real support. Multi-seed error bars would raise confidence but are secondary to whether ut is the intended visual direction. Verdict stays CONDITIONAL: accept-shaped if the intervention-robustness check holds, weaker if gains are tied to the specific 0.1× degrade recipe. Agreement with the reader is direct on the same soft spot.","tokens_in":22428,"tokens_out":721,"duration_ms":14788,"concrete_test":"On the 4B stack with fixed seeds/data/budget, replace only the x− construction: (i) same-region Gaussian blur or mild noise without 0.1× down/up; (ii) a wrong-region 2× crop as negative; (iii) random unmarked crops. Recompute Table 1 Avg6 and the Figure 3 r_vis visual/decision share. If gains over Vision-OPD collapse or r_vis enrichment vanishes under (i)/(ii), the proxy is artifact-sensitive and the attribution claim weakens; if both hold, the concern is mitigated.","verdict_should_be":"CONDITIONAL","load_bearing_attack":"The central claim is that projecting the privileged correction rt onto the same-teacher evidence-present vs evidence-removed proxy ut isolates a visually warranted supervision direction, and that supervising with the reconstructed student-anchored target is why VAD beats direct privileged matching and visual-advantage weighting. That claim rests on the intervention in §3.1–3.2 and Setup §4.1: a pre-marked 2× crop as x+ versus the same crop after 0.1× bilinear downsampling and nearest-neighbor upsampling as x−, under fixed teacher and text prefix. The paper correctly calls ut a proxy, not an oracle (§3.2, §5), and notes residual nonvisual effects can remain in r_vis. The soft spot is sharper: because x+ and x− differ in resolution, blur, and crop framing as well as “evidence presence,” ut can systematically pick up degradation/crop artifacts and teacher responses to those artifacts. One-sided projection (Eq. 4) and support/refutation budgeting (Eqs. 6–7) would then reconstruct targets aligned with that mixed direction. Semantic enrichment in Figure 3 and offline answer-token shifts in Figure 4 are consistent with useful signal but do not rule out artifact-aligned components, especially since training uses synthetic marked regions and main tables lack multi-seed uncertainty. If the proxy is substantially artifact-driven, the headline “counterfactual target reconstruction attributes visual evidence” overclaims relative to “a particular crop-vs-blur contrast yields a better OPD target.”","agreement_with_reader":"agree"},"referee_report":{"model":"grok-4.5","summary":"The paper proposes Visual Attribution Distillation (VAD) for multimodal on-policy distillation. It argues that privileged-view teacher corrections are source-mixed and that the key problem is estimating which part of a correction is supported by visual evidence. VAD queries a fixed teacher on evidence-present and evidence-removed views of the same student prefix, forms a signed centered log-probability proxy ut, one-sided-projects the privileged correction rt onto ut, budgets support versus refutation, and reconstructs a student-anchored target q_VAD that supplies primary JSD supervision, with a weak privileged-teacher regularizer. On six fine-grained benchmarks with matched Qwen3.5 4B/9B data and budgets, VAD outperforms Vision-OPD, VA-OPD, V-Zero, Decomposed OPD, and GRPO; ablations, semantic analyses of r_vis versus r_res, offline answer-token effects, held-out checks, and hyperparameter sweeps support the target-reconstruction story.","tokens_in":22844,"tokens_out":1471,"duration_ms":47083,"significance":"If the result holds under the intended interpretation, this is a clear and useful contribution to multimodal OPD: it reframes supervision from where/how strongly to distill toward reconstructing what to distill, with signed support and refutation. The empirical package is comparatively strong for the area—scale-matched post-training, six primary benchmarks, held-out generalization, target ablations (Table 2), offline correct/wrong effects (Fig. 4), semantic separation (Fig. 3), efficiency notes, and released code/models. Even under a weaker reading (a particular crop-versus-degradation contrast yields a better OPD target), the method and evidence would still matter for fine-grained MLLM post-training. The honesty that ut is a proxy, not an oracle, and the explicit residual caveats are strengths.","major_comments":[{"comment":"§3.1–3.2 and §4.1: the central attribution claim rests on ut = ϕ(p+_T) − ϕ(p−_T) from a pre-marked 2× crop (x+) versus the same crop after 0.1× bilinear downsampling and nearest-neighbor upsampling (x−). That pair confounds “evidence present/removed” with resolution, blur, and crop framing. One-sided projection (Eq. 4) and support/refutation budgeting (Eqs. 6–7) will inherit whatever that contrast encodes. Fig. 3 enrichment and Fig. 4 offline shifts are consistent with useful signal but do not separate visual evidence from degradation/crop artifacts. Please add at least one controlled intervention variant (e.g., mask/inpaint, alternate blur without crop change, or full-image vs masked full-image under matched framing) or materially soften claims that VAD “attributes visual evidence” versus “reconstructs targets from this specific contrastive proxy.”","section":"§3.1–3.2, §4.1, Eqs. (3)–(7)"},{"comment":"Table 1 and Table 2 report single-run accuracies without multi-seed means/stds or uncertainty on Avg6, while several margins are 0.2–3 points. Given free parameters (λ, τ+, ζ, c, top-K, degradation) and synthetic marked-region training, the headline ranking versus Vision-OPD/Decomposed OPD needs either repeated seeds for the main 4B/9B comparisons or a clear statement that results are single-run with sensitivity only via the λ/τ+ grid in Fig. 5. Without this, the load-bearing claim that target reconstruction—not run noise or tuning—drives the gains is under-supported.","section":"Table 1, Table 2, Fig. 5"},{"comment":"§4.3 / Fig. 3 and Limitations §5: the paper correctly says r_res is not purely linguistic and r_vis may retain nonvisual teacher effects, yet the narrative still treats proxy alignment as estimating the “visually attributable part” of the correction. The diagnostic in Appendix A also uses the same projection machinery used to define r_vis. Please keep motivation and evaluation distinct: either provide an external grounding check (e.g., human/attribute labels on high-|ut| tokens, or intervention swaps that should flip support/refutation) or revise abstract/intro/conclusion language so the empirical claim is “proxy-aligned reconstruction improves OPD” rather than identifiable visual attribution.","section":"§4.3, Fig. 3, Appendix A, §5"}],"minor_comments":[{"comment":"Figure 1 panels are dense; the 23.2%/22.8% diagnostic numbers are important but hard to parse next to the VA-OPD refutation cartoon. Consider a cleaner layout or moving numeric diagnostics fully to Appendix A with a short callout.","section":"Figure 1"},{"comment":"Eq. (2)–(3): state explicitly whether Vt is always student top-K only or the union with teacher mass, and how the tail bucket enters centered log-probabilities; Appendix D helps but the main text should be self-contained.","section":"§3.1–3.2"},{"comment":"Table 5 efficiency comparison omits Decomposed OPD; either add it or note why timing logs were unavailable so the cost narrative is not selectively incomplete.","section":"Appendix C, Table 5"},{"comment":"Minor notation/consistency: r_VAD versus rvis, q_VAD_T,t versus qone, and occasional PDF artifacts (e.g., “/envel⌢pe”, “Te a c h e r”) should be cleaned for camera-ready.","section":"Throughout"},{"comment":"Related work §2.3: the contrast with Decomposed OPD is useful; a short explicit sentence on when multimodal-vs-text-only information gain and same-teacher evidence-present/removed projection would disagree would help readers.","section":"§2.3"}],"recommendation":"major_revision","confidential_remarks":"Solid empirical ML paper with real engineering value; the main risk is over-claiming causal “visual attribution” from a crop-vs-downsample proxy. I would not reject on novelty grounds relative to Vision-OPD/VA-OPD/Decomposed OPD—the target-reconstruction angle is distinct enough—but I would not accept until the intervention confound and single-run reporting are addressed or the claims are narrowed. Fit for a serious CV/ML journal is reasonable after revision."},"author_rebuttal":null,"desk_editor":{"model":"grok-4.5","letter":"Punchline: this is a clean training-target paper in a crowded multimodal OPD lane. The useful move is not another reweighting of p+_T, but projecting the privileged correction onto a same-teacher evidence-present vs evidence-removed direction and rebuilding a student-anchored support/refutation target.\n\nWhat is actually new is that pipeline end-to-end: ut from centered log-prob shift, one-sided projection to r_vis, budgeted support/refutation, primary JSD on q_VAD plus a weak teacher regularizer. Relative to Vision-OPD (match the crop teacher), VA-OPD/V-Zero (weight or gate while keeping the full teacher target), and Decomposed OPD (text-only vs multimodal info gain), that is a distinct supervision geometry. They run it fairly: same Qwen3.5 4B/9B init, same 6k synthetic marked-region data, matched rollouts/budget, six fine-grained benches, held-out MMVP/CV/MMStar/POPE, target ablations, semantic split of r_vis vs r_res, offline correct/wrong token shifts, λ/τ+/divergence sweeps, and even wall-clock cost. Gains are consistent (roughly +2–3 Avg6 over the best same-stack baselines) and the limitations section matches the method’s real assumptions. Code and models are linked.\n\nSoft spots, in proportion. Main tables look single-run; no multi-seed bars, so margins could shrink. Training still needs pre-marked regions and a fixed 2× crop vs 0.1× down/up degradation, so ut is a proxy that can mix true evidence with blur/crop artifacts—the paper says this, and the stress-test is right that Figure 3/4 do not fully rule it out. That weakens the strongest causal slogan more than it weakens “this target works better under this intervention.” Residual nonvisual teacher effects can sit in r_vis; free knobs (λ, τ+, ζ, clip, top-K) are real but they sweep the important ones. Concurrent 2026 OPD work is dense; citation pattern is appropriate, not thin.\n\nMath is standard projection/JSD, readable, no internal contradiction. Data story is controlled synthetic post-training, not a foundational vision result.\n\nFor people building smaller full-image MLLMs and anyone comparing OPD target construction, this is worth the hour. I would send it to referees; it is important and sharp enough for serious review, with the proxy/artifact and multi-seed points as revision pressure, not desk-reject material. Engage.","headline":"Solid method paper: reconstruct the OPD target from a signed present/removed teacher contrast, with real matched-budget gains; the crop-vs-blur proxy is the caveat, not a collapse of the argument.","tokens_in":23508,"tokens_out":664,"would_cite":true,"duration_ms":22439,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"grok-4.5","headline":"Teacher corrections in multimodal distillation mix visual evidence with language priors; VAD keeps only the part the evidence can explain and rebuilds the learning target around the student.","keywords":["multimodal on-policy distillation","visual attribution","counterfactual target reconstruction","privileged teacher","fine-grained visual perception","support and refutation","knowledge distillation"],"falsifier":"If, under matched data and budgets, reconstructing the target from the proxy-aligned component no longer beats direct matching of the evidence-present teacher and visual-advantage weighting on the six fine-grained benchmarks, or if token analyses no longer show enrichment of visual and correct-answer corrections in the aligned component, the central claim fails.","tokens_in":23283,"feed_emoji":"🔍","tokens_out":928,"duration_ms":18266,"temperature":0.7,"pith_summary":"When a multimodal student is trained by matching a privileged teacher that sees a clearer crop of the image, the teacher’s next-token fixes are mixed: some come from the visual evidence, some from language habits and teacher quirks. This paper argues that the useful move is not only to decide where or how hard to distill, but to estimate which part of each correction the controlled visual change actually supports. Visual Attribution Distillation (VAD) does that by querying the same fixed teacher twice on each student-generated prefix—once with the evidence present and once with it removed—then treating the shift in centered log-probabilities as a signed proxy for the visual evidence direction. It projects the original teacher correction onto that proxy, rebuilds a student-anchored target from the aligned support and refutation, and uses the full privileged teacher only as a weak regularizer. On six fine-grained visual benchmarks at 4B and 9B scales, this reconstructed target beats direct privileged-view matching and visual-advantage weighting, and token analyses show the aligned component is enriched in visual, task-relevant corrections—especially when evidence should refute a wrong answer.","feed_headline":"Keep only the visual part of the teacher’s correction","feed_subtitle":"A present-vs-removed evidence contrast rebuilds distillation targets and beats source-mixed OPD","key_machinery":"Visual Attribution Distillation (VAD): a signed proxy ut from the change in the fixed teacher’s centered log-probabilities when relevant evidence is present versus removed; a one-sided projection of the full teacher correction onto ut that isolates an intervention-aligned component; and budgeted support/refutation reconstruction of a student-anchored target from that component.","core_discovery":"Counterfactual target reconstruction outperforms source-mixed privileged supervision in multimodal on-policy distillation: projecting the teacher-to-student correction onto a same-teacher evidence-present versus evidence-removed centered log-probability proxy, then supervising with the student-anchored reconstructed target, yields stronger fine-grained visual accuracy than direct privileged-view distillation or visual-advantage weighting at both 4B and 9B scales.","pith_inferences":["Any privileged teacher channel—region crops, tools, or future text—may need the same present-versus-absent attribution step before its corrections are trusted as targets.","Richer multi-view or learned directional bases could shrink the residual that still mixes visual and nonvisual teacher effects.","The same counterfactual projection idea could test whether other distillation gains come from true evidence transfer or from teacher-specific language style."],"forward_implications":["Privileged multimodal OPD should reconstruct what to distill from a controlled evidence intervention, not only reweight full teacher matching.","Signed support and refutation matter: evidence that lowers a mistaken token can be as useful as evidence that raises the right one.","A weak full-teacher regularizer can stabilize language and formatting while the primary signal remains the visually attributed target.","Student-anchored targets built this way can improve fine-grained perception without sacrificing held-out generalization relative to the base model."],"fun_headline_variants":["Isolate the visual slice of each teacher correction","Rebuild targets from present-vs-removed evidence contrast","Project mixed corrections onto a visual evidence proxy","Counterfactual reconstruction beats source-mixed OPD","Supervise only the visually attributable teacher signal"],"cache_read_input_tokens":16512,"weakest_assumption_plain":"A single marked-region crop pair—clear evidence versus a heavily downsampled version of the same crop—under unchanged text is enough to mark which teacher corrections the visual evidence truly warrants, rather than crop or degradation artifacts.","fun_headline_variants_meta":{"raw":{"variants":["Isolate the visual slice of each teacher correction","Rebuild targets from present-vs-removed evidence contrast","Project mixed corrections onto a visual evidence proxy","Counterfactual reconstruction beats source-mixed OPD","Supervise only the visually attributable teacher signal"]},"model":"grok-4.5","effort":"low","cost_usd":0.004254,"raw_usage":{"total_tokens":1302,"prompt_tokens":835,"num_sources_used":0,"completion_tokens":70,"cost_in_usd_ticks":42544000,"prompt_tokens_details":{"text_tokens":835,"audio_tokens":0,"image_tokens":0,"cached_tokens":128},"completion_tokens_details":{"audio_tokens":0,"reasoning_tokens":397,"accepted_prediction_tokens":0,"rejected_prediction_tokens":0}},"tokens_in":835,"tokens_out":70,"duration_ms":6864,"temperature":1.0,"reasoning_tokens":397,"cache_read_input_tokens":128,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-07-31T02:54:43.534743+00:00","model_set":{"reader":"grok-4.5"},"falsifier":"If, under matched data and budgets, reconstructing the target from the proxy-aligned component no longer beats direct matching of the evidence-present teacher and visual-advantage weighting on the six fine-grained benchmarks, or if token analyses no longer show enrichment of visual and correct-answer corrections in the aligned component, the central claim fails.","supporting_citations":[],"review_version":1}