OPD-V: Visual On-Policy Self-Distillation with Modality Balance

Aniri; Fei Shen; Jinhe Bi; Peng Liao; Tat-Seng Chua; Volker Tresp; Yunpu Ma; Zengjie Jin

REVIEW 3 major objections 5 minor 44 references

OPD-V: Visual On-Policy Self-Distillation with Modality Balance

T0 review · 3 major / 5 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read OPD-V claims that modality balance—the ratio of visual to textual reliance during generation—can be used as privileged information for on-policy self-distillation, and reports consistent accuracy gains across six benchmarks and four…

desk verdict A new trust-region distillation rule that is likely answer-crop selection rather than genuine modality balance; still worth refereeing because the idea is new and the gains are consistent. read the letter →

arxiv 2608.05131 v1 pith:LHLI34NR submitted 2026-08-05 cs.CV cs.AI

Aniri , Jinhe Bi , Peng Liao , Zengjie Jin , Volker Tresp , Fei Shen , Yunpu Ma , Tat-Seng Chua This is my paper

classification cs.CVcs.AI

keywords on-policyself-distillationmodalityimbalancemultimodallargelanguagemodelsvisualreasoningprivilegedinformationteacher-studentdistillationzoom-inimagemask

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

On-Policy Self-Distillation (OPSD) uses a copy of the student model as a teacher to provide dense supervision at the student's own rollouts, and multimodal OPSD variants condition that teacher on extra visual inputs. The paper argues that these methods overlook Modality Imbalance, the tendency of a multimodal generation to lean on textual priors instead of the image, leaving privileged visual inputs underused. The authors' central proposal is to make modality balance itself the privileged information: two teacher views of the same evidence-centered crop—one zoomed in, one with a random masked region—define a token-level Modality-Balance Logits Margin, and only tokens with a positive margin enter a trust region where the zoomed teacher's distribution is distilled into the student. Across six benchmarks and four backbones the method is reported to improve reasoning accuracy on every evaluation, with a 4B model rising from 64.30 percent to 80.01 percent average accuracy, and to reduce step time by roughly 25–32 percent, so the claimed effect is both a stronger model and a cheaper training run. The insight matters if correct because it suggests the internal visual-textual balance can be measured and steered without additional supervision or external teachers.

What carries the argument

The central object is the Modality-Balance Logits Margin, defined tokenwise as the difference between the log probabilities that two detached teacher copies assign to the student's own generated token under matched context: one teacher sees the Zoom-In Image (the evidence-centered crop), the other sees the Mask Image (a random black rectangle over that crop). The margin selects which on-policy tokens enter the Modality-Balance Trust Region (margin greater than zero), and the selected tokens receive Jensen–Shannon distillation from the Positive Teacher, scaled by the margin itself. The accompanying Modality-Balance Attention Ratio, the ratio of visual to textual attention mass over a response, is used to show that the two teacher conditions actually sit on opposite ends of a visual-to-textual reliance spectrum, so the margin is presented as a directional signal about modality balance rather than mere confidence.

What would settle it

A controlled experiment that replaces the true evidence-centered crop with a misaligned or random crop for the Positive Teacher on the same 6.2K training set and checks whether the reported 15.7-point average gain over the base model survives; if most of the gain disappears, the zoom-in evidence is load-bearing, and if it persists, the margin must be measuring something other than genuine visual relevance.

Watch

Extended reading notes

Core claim

The paper's discovery is that Modality Balance can be converted into token-level supervision for on-policy self-distillation. It constructs a Positive Teacher that scores the student's generated tokens under the Zoom-In Image, the evidence-centered crop magnified from the training data, and a Negative Teacher that scores the same tokens under a Mask Image formed by blacking out a random rectangular region of that crop. The tokenwise Modality-Balance Logits Margin compares the log probabilities the two teachers assign to each on-policy token; positions with a positive margin form the Modality-Balance Trust Region. Within that region the objective applies Jensen–Shannon distillation from the Positive Teacher distribution to the student, weighting each token by its margin. The paper reports that this selection rule improves average accuracy on six benchmarks for all four tested backbones, with Qwen3.5-4B rising from 64.30 to 80.01 percent and beating the strongest matched baseline, Vision-OPD, by 2.91 points, while shortening final responses by 74.5 percent relative to vanilla OPSD and cutting mean step time by 31.8 percent at 4B scale and 24.7 percent at 9B scale.

Load-bearing premise

The method assumes the evidence-centered crop (the bbox_images field in the training data) correctly isolates the task-relevant region for every sample, and that a positive Modality-Balance Logits Margin reflects genuine modality balance rather than leakage of the answer's location; if the crop is misaligned, the Positive Teacher may not be positive and trust-region selection becomes arbitrary.

Editorial extensions

If this is right

If OPD-V is right, a vision-language model can be improved by self-distilling only the tokens where zooming in on the evidence helps more than masking it hurts, which means the training signal is filtered by visual relevance rather than by raw teacher confidence.
The reported efficiency gains follow from the mechanism: shorter rollouts (74.5 percent shorter responses than OPSD by the final steps) and cheaper teacher-input construction (15.3 seconds versus 79.2 seconds of preprocessing on the 9B backbone) more than offset the extra negative-teacher pass.
The gains appear across two different vision-language architectures (Qwen3-VL's interleaved positional injection and Qwen3.5's early fusion hybrid-attention stack), so the mechanism is argued to be architecture-independent rather than a quirk of one perception stack.
A 4B model trained this way is reported to exceed the average accuracy of several much larger single-forward-pass models on the same six-benchmark suite, indicating the method is a compute-efficient route to strong multimodal reasoning.
The ablation results, with the Negative Teacher alone at 71.98 percent and the Positive Teacher alone at 74.62 percent but both at 80.01 percent, show the trust-region selector and the distillation target are complementary rather than redundant.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

A natural extension is to replace the fixed Zoom-In/Mask pair with any input perturbation that shifts visual-textual reliance—resolution, contrast, or different masking patterns—and test whether the positive-margin trust region remains a robust selector; the paper tests only the one pair.
Because the Positive Teacher depends on the bbox_images crop from the training data, OPD-V inherits a hard dependency on evidence localization; a stress test that deliberately misaligns the crop would reveal how much of the gain comes from the zoom mechanism itself rather than from the contrastive pairing.
The margin could serve as a free per-token importance signal beyond distillation, for example to reweight sampling or to filter rollouts for reinforcement learning, a use the paper does not explore.

Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

Short version: the reported gains are large and consistent, and the dual-teacher trust-region idea is genuinely new, but the 'modality balance' story is not actually in the training objective. The margin that selects tokens compares a zoomed-in ground-truth crop against a randomly masked version of that crop, which can easily pick out answer-location tokens rather than visual-reasoning tokens. The paper never measures attention at token level, and the objective in Eq. 9 contains no attention term. That said, the method is still a reasonable answer-crop distillation scheme, and the results justify a serious look.

What's new: OPD-V defines a trust region for on-policy self-distillation using the logit margin between a positive teacher (zoom-in image) and a negative teacher (mask image). This specific token-selection rule is not in Vision-OPD or VA-OPD. The paper also shows consistent improvements across four backbones and six benchmarks, and it reports meaningful training-time savings. The ablations in Fig 6 support the choice of zoom-in and mask as the strongest pair.

Soft spots, in order of severity. First, the central claim: the paper says positive margins identify tokens where visual evidence outweighs textual priors, i.e., 'modality balance.' But delta only compares two image conditions; it says nothing about the relative use of visual and textual information. The attention-ratio measure from Section 2.3 is never used in training, and Figure 1's correlation is response-level, not token-level. So the mechanism is unverified. Second, I_zoom is the ground-truth bounding-box crop from the training data. The positive teacher literally sees the answer region, and the negative teacher has a random hole punched in it. Tokens that pass the delta > 0 test may simply be tokens whose probability rises when the answer is visible. That's a form of answer-location leakage, not necessarily modality balance. Third, there are no error bars or seed variance, the training set is only 6.2K samples, and the gains are very large (e.g., +15.7 points on Qwen3.5-4B). Without code and reproducibility details, I'm inclined to treat the numbers as optimistic. Fourth, the evaluation uses a Yes/No judge for unresolved cases, which can be noisy.

Who should read it: people working on MLLM post-training, self-distillation, or token-level supervision. They'll find the dual-teacher contrast interesting and the efficiency analysis useful.

Recommendation: this deserves peer review, not a desk reject. A referee should ask for code, seed variance, and a token-level analysis that connects delta to attention or another direct measure of visual grounding. If the authors can show the trust region actually tracks visual-evidence tokens, the paper becomes solid. As it stands, the mechanism is plausible but unproven.

Referee Report

3 major / 5 minor

Summary. The paper proposes OPD-V, a visual on-policy self-distillation method for MLLMs that uses two teacher conditions—a Positive Teacher receiving a Zoom-In Image and a Negative Teacher receiving a Mask Image—to define a Modality-Balance Trust Region. Tokens whose log-likelihood under the Positive Teacher exceeds that under the Negative Teacher are selected and weighted by this margin, and Jensen–Shannon distillation from the Positive Teacher is applied on those tokens. Experiments on six benchmarks and four Qwen-family backbones report consistent accuracy gains over matched baselines, including a 15.7-point average improvement on Qwen3.5-4B, with reduced wall-clock step time. The central claimed mechanism is that the positive logit margin identifies tokens whose support increases because the model integrates visual evidence, thereby operationalizing Modality Balance as privileged information.

Significance. If the mechanism were established, this would be a significant and practical contribution: it is a simple, training-efficient token-selection rule that reportedly improves visual reasoning across several MLLM families, and the paper provides a detailed algorithm and hyperparameter table that would make the method easy to reproduce. The response-length reduction and step-time measurements are concrete and falsifiable. However, the paper's central conceptual claim—that the Modality-Balance Logits Margin instantiates Modality Balance—is not currently supported by the evidence in the manuscript. The training objective never uses the Modality-Balance Attention Ratio defined in Eq. (5), and the only connection between the margin and modality balance is a response-level correlational figure. The empirical gains may still be real, but the mechanistic interpretation needs substantially stronger support before the paper can be accepted as a demonstration of modality-balance-based distillation.

major comments (3)

[§3.3–3.4, Eqs. (7)–(9)] The trust region and the final objective are defined entirely through the teacher logit margin δMB_t; the Modality-Balance Attention Ratio ρ from Eq. (5) appears nowhere in the training loss or in token selection. The only evidence linking δMB to modality balance is Figure 1, which is computed at the response level over 5K samples and shows a correlation across margin intervals. This does not establish that an individual token with δMB_t > 0 is a token whose support increases because the model integrates visual evidence. As written, the method is indistinguishable from a gated variant of Vision-OPD that distills tokens favored by the evidence-centered crop. Please provide token-level evidence: for example, compute per-token attention ratios under the Positive and Negative teachers and show that tokens in the trust region have a systematically larger attention-ratio gap, or run an ablation that replaces the δMB gate with a direct attention-ratio-based gate and report whether the gains persist. Without such evidence, the claim that this is a modality-balance intervention is not supported.
[§3.2 and Appendix B.2, Eq. (6)] Appendix B.2 states that I_zoom is taken from the bbox_images field of the Vision-OPD training data, i.e., the ground-truth evidence-centered crop, and I_mask is a random blackout of that same crop. Under these definitions, δMB_t = log q+_t(y_t) − log q−_t(y_t) can be positive simply because the zoomed crop reveals the answer region; no attention measurement is involved. The paper should rule out answer-position leakage by adding controls, such as using a random crop or a crop from a different image region as the Positive Teacher condition, or by comparing margins for tokens that fall inside versus outside the evidence region. If the gains require the ground-truth bounding box rather than a random crop, the mechanism is better described as answer-crop distillation than as modality balance.
[Table 1 and Figure 4] The headline empirical claim—consistent improvement across four backbones and a 15.7-point gain on Qwen3.5-4B—is reported without error bars, number of seeds, or significance tests. Given that the paper emphasizes consistency across many settings, please report mean and standard deviation over at least three seeds for the matched Qwen3.5-4B comparison with the main baselines, and ideally for one additional backbone. This is necessary to assess whether the 2.91-point gap over Vision-OPD and the gains on the other backbones are robust rather than the result of a single favorable run.

minor comments (5)

[§2.3, Eq. (5)] The attention masses αV_t(c) and αT_t(c) are not defined precisely: please specify which layers and heads are used, and how textual attention is aggregated across tokens. This is needed to reproduce the Modality-Balance Attention Ratio in Figure 1.
[Table 1] The parenthetical values in the OPD-V row are described as absolute percentage-point gains but are written with a '%' sign (e.g., +14.1%), which is ambiguous and could be read as a relative improvement. Please use 'pp' or 'points' to avoid confusion.
[Figure 4] The four panels use independently zoomed y-axes. Although the exact scores are annotated, this can make small differences appear large across panels; consider using a common scale or adding a clear visual cue that the axes are not aligned.
[Author block and front matter] The author block has formatting issues, including a missing space in 'Aniri2,3' and the garbled token 'envel⌢pe' after 'Yunpu Ma'; the GitHub link placeholder is also not rendered as a proper URL. These should be cleaned up.
[§4.1.1 and Appendix D] Section 4.1.1 states a 'curated synthetic dataset of 6.2K visual reasoning samples,' while Appendix D gives the exact number 6241; please make these consistent.

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity: the OPD-V objective is not equivalent to its inputs; the modality-balance mechanism is an untested proxy, not a construction-level circular step.

full rationale

The paper's central claim is that Positive/Negative teacher logits margins instantiate Modality Balance. I checked whether Eq. (9) reduces to Eq. (5) or to a fitted quantity. It does not. Eq. (5) defines the Modality-Balance Attention Ratio ρ(c;y) from attention masses, but ρ never appears in the OPD-V loss: Eq. (7) defines δMB_t purely as log q+_t(y_t) − log q−_t(y_t), Eq. (8) selects tokens with δMB_t > 0, and Eq. (9) weights DJS(q+_t, p_θ,t) by δMB_t. The identification of δMB_t with Modality Balance is an empirical proxy supported by Figure 1's response-level correlation, not an equation-level equivalence. This is a measurement-validity gap: the token-level mechanism is not directly verified, but it is not circular. No parameter is fitted to a benchmark subset and then reported as a prediction; the trust-region criterion is a fixed sign condition, not a fitted constant. The I_zoom crop is taken from the external Vision-OPD training data, so reusing it is a data choice, not a self-referential definition. Self-citations (e.g., [8], [35]–[42]) concern prior modality-imbalance and data-selection context and are not load-bearing for the main comparison; the headline results are evaluated on external benchmarks against matched baselines. I therefore find no circular step in the derivation chain.

Assumptions & free parameters 6 free parameters · 4 assumptions · 3 invented entities

The central claim relies on the availability of oracle evidence crops, on a proxy for modality balance, and on hand-chosen hyperparameters. The attention-based measure of modality balance (Eq. 5) is never used in the OPD-V objective (Eq. 9), which is a gap between the stated mechanism and the implemented training signal.

free parameters (6)

Modality-Balance Logits Margin threshold = 0
Tokens with margin > 0 form the trust region; chosen fixed, no sensitivity analysis.
Top-K distillation size = 100
Set to 100 to limit memory; no ablation of K.
Jensen-Shannon beta = 0.5
Standard JSD mixing coefficient; set to 0.5.
EMA update rate = 0.05
Teacher parameter update rate; chosen without analysis.
Rollout correction threshold = 2.0
Clipping threshold for token-level rollout correction.
Random mask region distribution = unspecified
Random rectangular region replacement; distribution over size and location not given, which affects the negative teacher's strength.

assumptions (4)

domain assumption Alpha_V and alpha_T attention masses can be extracted from the MLLM and reliably measure visual versus textual modality use.
Used to define Modality-Balance Attention Ratio in Eq. 5; not validated for hybrid backbones like Qwen3.5 (Gated DeltaNet layers).
ad hoc to paper Zoom-In Image increases modality balance and Mask Image decreases it, so the logit margin reflects modality balance.
Central premise of the method; the connection between margin and attention ratio is empirical (Fig. 1) and only correlational.
domain assumption The evidence-centered crop I_zoom is available and correctly locates the task-relevant region for each training sample.
Relies on Vision-OPD dataset's bbox_images field; if the crop is wrong, the positive teacher is not positive.
ad hoc to paper Tokens with positive margin are the right tokens to distill; negative margin tokens are not useful.
Defines the Modality-Balance Trust Region (Eq. 8); no analysis of what the discarded tokens contain.

invented entities (3)

Positive Teacher with Zoom-In Image
purpose: Provides the distillation target distribution q+_t
A teacher configuration using the evidence-centered crop; its 'positive' property is assumed, not separately verified.
Negative Teacher with Mask Image
purpose: Provides the comparison distribution q-_t to compute the margin
Same model with a random rectangular mask; its 'negative' property depends on the mask being in a task-relevant region, which is random.
Modality-Balance Trust Region
purpose: Selects tokens for self-distillation based on positive margin
Defined in Eq. 8; no external validation that this region corresponds to tokens requiring visual evidence.

how reviews work

0 comments

Cite this review

Pith. "Pith review of OPD-V: Visual On-Policy Self-Distillation with Modality Balance." pith.science (2026). https://pith.science/paper/LHLI34NR

@misc{pith2026260805131,
  author       = {Pith},
  title        = {Pith review of: OPD-V: Visual On-Policy Self-Distillation with Modality Balance},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/LHLI34NR}},
  note         = {Machine review of arXiv:2608.05131}
}

read the original abstract

On-Policy Self-Distillation (OPSD) has become a standard post-training approach for improving visual reasoning in multimodal large language models (MLLMs). Existing methods draw privileged information from diverse input sources to guide self-distillation. Yet these designs overlook Modality Imbalance, a challenge inherent to MLLM reasoning. When textual information dominates generation, the model cannot fully integrate its multimodal input. Consequently, carefully designed privileged information remains underused, limiting the effectiveness of OPSD. To examine this limitation, we construct a Positive Teacher with the Zoom-In Image and a Negative Teacher with the Mask Image, which exhibit different degrees of Modality Imbalance. Changes in their reasoning correctness and token logits reveal that Modality Balance can itself serve as privileged information. Motivated by this finding, we introduce OPD-V, a visual OPSD paradigm that instantiates such information through the Positive Teacher and Negative Teacher. Positive Modality-Balance Logits Margins define a Modality-Balance Trust Region that selects the on-policy tokens used for self-distillation. Experiments across 6 benchmarks, 4 MLLM backbones, and 5 post-training methods show that OPD-V consistently improves reasoning performance while reducing training cost.

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

44 extracted references · 8 canonical work pages

[1]

On-policy distillation of language models: Learning from self-generated mistakes

Rishabh Agarwal, Nino Vieillard, Yongchao Zhou, Piotr Stanczyk, Sabela Ramos Garea, Matthieu Geist, and Olivier Bachem. On-policy distillation of language models: Learning from self-generated mistakes. InInternational Conference on Learning Representations, volume 2024, pages 21246–21263, 2024

work page 2024
[2]

On-policy distillation.Thinking Machines Lab: Connectionism,

Kevin Lu and Thinking Machines Lab. On-policy distillation.Thinking Machines Lab: Connectionism,
[3]

Self- distilled reasoner: On-policy self-distillation for large language models.arXiv preprint arXiv:2601.18734, 2026

Siyan Zhao, Zhihui Xie, Mengchen Liu, Jing Huang, Guan Pang, Feiyu Chen, and Aditya Grover. Self- distilled reasoner: On-policy self-distillation for large language models.arXiv preprint arXiv:2601.18734, 2026

arXiv 2026
[4]

Vision-opd: Learning to see fine details for multimodal llms via on-policy self-distillation.arXiv preprint arXiv:2605.18740, 2026

Qianhao Yuan, Jie Lou, Xing Yu, Hongyu Lin, Le Sun, Xianpei Han, and Yaojie Lu. Vision-opd: Learning to see fine details for multimodal llms via on-policy self-distillation.arXiv preprint arXiv:2605.18740, 2026

arXiv 2026
[5]

Visual-OPSD: Cross-Modal On-Policy Self-Distillation for Efficient Unified Multimodal Reasoning

Pengyu Li, Zhitao Gao, Lingling Zhang, Muye Huang, Yuanming Li, Fangzhi Xu, and Jun Liu. Visual- opsd: Cross-modal on-policy self-distillation for efficient unified multimodal reasoning.arXiv preprint arXiv:2606.18974, 2026

work page Pith review arXiv 2026
[6]

Visual-advantage on-policy distillation for vision-language models

Ruiqi Liu, Xiaolei Lv, Gengsheng Li, Ximo Zhu, Zhiheng Wang, Zhengbo Zhang, Junkai Chen, Zhiheng Li, Bo Li, Jun Gao, and Shu Wu. Visual-advantage on-policy distillation for vision-language models. arXiv preprint arXiv:2605.21924, 2026

arXiv 2026
[7]

Visual Contrastive Self-Distillation

Yijun Liang, Yunjie Tian, Yijiang Li, Yuqi Jia, Furong Huang, Tianyi Zhou, and Di Fu. Visual contrastive self-distillation.arXiv preprint arXiv:2607.21556, 2026

work page Pith review arXiv 2026
[8]

LLaVA steering: Visual instruction tuning with 500x fewer parameters through modality linear representation- steering

Jinhe Bi, Yujun Wang, Haokun Chen, Xun Xiao, Artur Hecker, Volker Tresp, and Yunpu Ma. LLaVA steering: Visual instruction tuning with 500x fewer parameters through modality linear representation- steering. In Wanxiang Che, Joyce Nabende, Ekaterina Shutova, and Mohammad Taher Pilehvar, editors, Proceedings of the 63rd Annual Meeting of the Association for ...

2025

Show all 44 references

[9]

Simon Park, Abhishek Panigrahi, Yun Cheng, Dingli Yu, Anirudh Goyal, and Sanjeev Arora. Generalizing from SIMPLE to HARD visual reasoning: Can we mitigate modality imbalance in VLMs? InProceedings of the 42nd International Conference on Machine Learning, volume 267 ofProceedin...

2025
[10]

Evaluating and steering modality preferences in multimodal large language model.arXiv preprint arXiv:2505.20977, 2025

Yu Zhang, Jinlong Ma, Yongshuai Hou, Xuefeng Bai, Kehai Chen, Yang Xiang, Jun Yu, and Min Zhang. Evaluating and steering modality preferences in multimodal large language model.arXiv preprint arXiv:2505.20977, 2025

2025
[11]

Instruction anchor: Dissecting the mechanistic dynamics of modality arbitration, 2026

Yu Zhang, Mufan Xu, Xuefeng Bai, Kehai Chen, Pengfei Zhang, Yang Xiang, and Min Zhang. Instruction anchor: Dissecting the mechanistic dynamics of modality arbitration, 2026. URLhttps://arxiv.org/ abs/2602.03677

2026 arXiv
[12]

Qwen3.5: Towards native multimodal agents, February 2026

Qwen Team. Qwen3.5: Towards native multimodal agents, February 2026. URLhttps://qwen.ai/ blog?id=qwen3.5

2026
[13]

Qwen3-VL technical report.arXiv preprint arXiv:2511.21631, 2025

Shuai Bai, Yuxuan Cai, Ruizhe Chen, Keqin Chen, Xionghui Chen, Zesen Cheng, et al. Qwen3-VL technical report.arXiv preprint arXiv:2511.21631, 2025

2025 arXiv
[14]

V*: Guided visual search as a core mechanism in multimodal llms

Penghao Wu and Saining Xie. V*: Guided visual search as a core mechanism in multimodal llms. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 13084–13094, 2024

2024
[15]

Zooming without zooming: Region-to-image distillation for fine-grained multimodal perception.arXiv preprint arXiv:2602.11858, 2026

Lai Wei, Liangbo He, Jun Lan, Lingzhong Dong, Yutong Cai, Siyuan Li, Huijia Zhu, Weiqiang Wang, Linghe Kong, Yue Wang, et al. Zooming without zooming: Region-to-image distillation for fine-grained multimodal perception.arXiv preprint arXiv:2602.11858, 2026

2026
[16]

Divide, conquer and combine: A training-free framework for high-resolution image perception in multimodal large language models

Wenbin Wang, Liang Ding, Minyan Zeng, Xiabin Zhou, Li Shen, Yong Luo, Wei Yu, and Dacheng Tao. Divide, conquer and combine: A training-free framework for high-resolution image perception in multimodal large language models. InProceedings of the AAAI Conference on Artificial In...

2025
[17]

Mme-realworld: Could your multimodal llm challenge high-resolution real-world scenarios that are difficult for humans?arXiv preprint arXiv:2408.13257, 2024

Yi-Fan Zhang, Huanyu Zhang, Haochen Tian, Chaoyou Fu, Shuangqing Zhang, Junfei Wu, Feng Li, Kun Wang, Qingsong Wen, Zhang Zhang, et al. Mme-realworld: Could your multimodal llm challenge high-resolution real-world scenarios that are difficult for humans?arXiv preprint arXiv:24...

2024 arXiv
[18]

Gemini 3.1 pro.https://deepmind.google/models/model-cards/gemini-3-1-pro/, 2026

Google. Gemini 3.1 pro.https://deepmind.google/models/model-cards/gemini-3-1-pro/, 2026

2026
[19]

Gemini 3.5 flash

Google. Gemini 3.5 flash. https://deepmind.google/models/model-cards/gemini-3-5-flash/ , 2026

2026
[20]

Introducing gpt-5.4.https://openai.com/index/introducing-gpt-5-4/, 2026

OpenAI. Introducing gpt-5.4.https://openai.com/index/introducing-gpt-5-4/, 2026

2026
[21]

Introducing gpt-5.2.https://openai.com/index/introducing-gpt-5-2/, 2025

OpenAI. Introducing gpt-5.2.https://openai.com/index/introducing-gpt-5-2/, 2025

2025
[22]

Thyme: Think beyond images.arXiv preprint arXiv:2508.11630, 2025

Yi-Fan Zhang, Xingyu Lu, Shukang Yin, Chaoyou Fu, Wei Chen, Xiao Hu, Bin Wen, Kaiyu Jiang, Changyi Liu, Tianke Zhang, et al. Thyme: Think beyond images.arXiv preprint arXiv:2508.11630, 2025

2025 arXiv
[23]

thinking with images

Ziwei Zheng, Michael Yang, Jack Hong, Chenxiao Zhao, Guohai Xu, Le Yang, Chao Shen, and Xing Yu. Deepeyes: Incentivizing" thinking with images" via reinforcement learning.arXiv preprint arXiv:2505.14362, 2025

2025 arXiv
[24]

Deepeyesv2: Toward agentic multimodal model.arXiv preprint arXiv:2511.05271, 2025

Jack Hong, Chenxiao Zhao, ChengLin Zhu, Weiheng Lu, Guohai Xu, and Xing Yu. Deepeyesv2: Toward agentic multimodal model.arXiv preprint arXiv:2511.05271, 2025

2025 arXiv
[25]

Mimo-vl technical report, 2025

LLM-Core-Team Xiaomi. Mimo-vl technical report, 2025. URLhttps://arxiv.org/abs/2506.03569

2025 arXiv
[26]

Sensenova-mars: Empowering multimodal agentic reasoning and search via reinforcement learning.arXiv preprint arXiv:2512.24330, 2025

Yong Xien Chng, Tao Hu, Wenwen Tong, Xueheng Li, Jiandong Chen, Haojia Yu, Jiefan Lu, Hewei Guo, Hanming Deng, Chengjun Xie, et al. Sensenova-mars: Empowering multimodal agentic reasoning and search via reinforcement learning.arXiv preprint arXiv:2512.24330, 2025

2025
[27]

Minicpm-v 4.5: Cooking efficient mllms via architecture, data, and training recipe.arXiv preprint arXiv:2509.18154, 2025

Tianyu Yu, Zefan Wang, Chongyi Wang, Fuwei Huang, Wenshuo Ma, Zhihui He, Tianchi Cai, Weize Chen, Yuxiang Huang, Yuanqian Zhao, et al. Minicpm-v 4.5: Cooking efficient mllms via architecture, data, and training recipe.arXiv preprint arXiv:2509.18154, 2025. 11

2025 arXiv
[28]

Glm-4.5 v and glm-4.1 v-thinking: Towards versatile multimodal reasoning with scalable reinforcement learning.arXiv preprint arXiv:2507.01006, 2025

Wenyi Hong, Wenmeng Yu, Xiaotao Gu, Guo Wang, Guobing Gan, Haomiao Tang, Jiale Cheng, Ji Qi, Junhui Ji, Lihang Pan, et al. Glm-4.5 v and glm-4.1 v-thinking: Towards versatile multimodal reasoning with scalable reinforcement learning.arXiv preprint arXiv:2507.01006, 2025

2025 arXiv
[29]

Kimi k2.6: From code to creation, from one to many.https://www.kimi.com/ai-models/ kimi-k2-6/, 2026

Team, Kimi. Kimi k2.6: From code to creation, from one to many.https://www.kimi.com/ai-models/ kimi-k2-6/, 2026

2026
[30]

Deepseekmath: Pushing the limits of mathematical reasoning in open language models.arXiv preprint arXiv:2402.03300, 2024

Zhihong Shao, Peiyi Wang, Qihao Zhu, Runxin Xu, Junxiao Song, Xiao Bi, Haowei Zhang, Mingchuan Zhang, YK Li, Yang Wu, et al. Deepseekmath: Pushing the limits of mathematical reasoning in open language models.arXiv preprint arXiv:2402.03300, 2024

2024 arXiv
[31]

When more is less: Understanding chain-of-thought length in llms.ArXiv, abs/2502.07266, 2025

Yuyang Wu, Yifei Wang, Tianqi Du, Stefanie Jegelka, and Yisen Wang. When more is less: Understanding chain-of-thought length in llms.ArXiv, abs/2502.07266, 2025. URLhttps://api.semanticscholar. org/CorpusID:276259519

2025 arXiv
[32]

Don’t overthink it

Michael Hassid, Gabriel Synnaeve, Yossi Adi, and Roy Schwartz. Don’t overthink it. preferring shorter thinking chains for improved LLM reasoning, 2026. URLhttps://openreview.net/forum?id= nhUlA8iMkD

2026
[33]

Distilling the knowledge in a neural network.arXiv preprint arXiv:1503.02531, 2015

Geoffrey Hinton, Oriol Vinyals, and Jeff Dean. Distilling the knowledge in a neural network.arXiv preprint arXiv:1503.02531, 2015

2015 arXiv
[34]

Sequence-level knowledge distillation

Yoon Kim and Alexander M Rush. Sequence-level knowledge distillation. InProceedings of the 2016 conference on empirical methods in natural language processing, pages 1317–1327, 2016

2016
[35]

Cot-kinetics: A theoretical modeling assessing lrm reasoning process.ArXiv, abs/2505.13408, 2025

JinheBi, DanqiYan, YifanWang, WenkeHuang, HaokunChen, GuanchengWan, MangYe, XunXiao, Hin rich Schuetze, Volker Tresp, and Yunpu Ma. Cot-kinetics: A theoretical modeling assessing lrm reasoning process.ArXiv, abs/2505.13408, 2025. URLhttps://api.semanticscholar.org/CorpusID:278769227

2025 arXiv
[36]

EchoRL: Reinforcement learning via rollout echoing

Jinhe Bi, Aniri, Minglai Yang, Xingcheng Zhou, Wenke Huang, Sikuan Yan, Yujun Wang, Zixuan Cao, Michael Färber, Xun Xiao, Volker Tresp, and Yunpu Ma. EchoRL: Reinforcement learning via rollout echoing. InForty-third International Conference on Machine Learning, 2026. URL https...

2026
[37]

Seeing before reasoning: Decoupling perception and reasoning for shortcut-resilient multimodal on-policy self-distillation.arXiv preprint arXiv:2606.19120, 2026

Sihan Wang, Xiyao Liu, Lianqing Liu, and Zhi Han. Seeing before reasoning: Decoupling perception and reasoning for shortcut-resilient multimodal on-policy self-distillation.arXiv preprint arXiv:2606.19120, 2026

2026 arXiv
[38]

Prism: Self- pruning intrinsic selection method for training-free multimodal data selection.ArXiv, abs/2502.12119,

Jinhe Bi, Yifan Wang, Danqi Yan, Xun Xiao, Artur Hecker, Volker Tresp, and Yunpu Ma. Prism: Self- pruning intrinsic selection method for training-free multimodal data selection.ArXiv, abs/2502.12119,

arXiv
[39]

Can visual input be compressed? a visual token compression benchmark for large multimodal models, 2025

Tianfan Peng, Yuntao Du, Pengzhou Ji, Shijie Dong, Kailin Jiang, Mingchuan Ma, Yijun Tian, Jinhe Bi, Qian Li, Wei Du, Feng Xiao, and Lizhen Cui. Can visual input be compressed? a visual token compression benchmark for large multimodal models, 2025. URLhttps://arxiv.org/abs/2511.02650

2025
[40]

URLhttps://api.semanticscholar.org/CorpusID:276421326
[41]

Mined: Probing and updating with multimodal time-sensitive knowledge for large multimodal models, 2025

Kailin Jiang, Ning Jiang, Yuntao Du, Yuchen Ren, Yuchen Li, Yifan Gao, Jinhe Bi, Yunpu Ma, Qingqing Liu, Xianhao Wang, Yifan Jia, Hongbo Jiang, Yaocong Hu, Bin Li, and Lei Liu. Mined: Probing and updating with multimodal time-sensitive knowledge for large multimodal models, 20...

2025 arXiv
[42]

Ascd: Attention-steerable contrastive decoding for reducing hallucination in mllm.Proceedings of the AAAI Conference on Artificial Intelligence, 40(12): 10306–10314, Mar

Yujun Wang, Aniri , Jinhe Bi, Soren Pirk, and Yunpu Ma. Ascd: Attention-steerable contrastive decoding for reducing hallucination in mllm.Proceedings of the AAAI Conference on Artificial Intelligence, 40(12): 10306–10314, Mar. 2026. doi: 10.1609/aaai.v40i12.38000. URL https://...

2026 doi
[44]

Kore: Enhancing knowledge injection for large multimodal models via knowledge-oriented augmentations and constraints, 2025

Kailin Jiang, Hongbo Jiang, Ning Jiang, Zhi Gao, Jinhe Bi, Yuchen Ren, Bin Li, Yuntao Du, Lei Liu, and Qing Li. Kore: Enhancing knowledge injection for large multimodal models via knowledge-oriented augmentations and constraints, 2025. URLhttps://arxiv.org/abs/2510.19316. Appe...

2025 arXiv
[2025]

https://thinkingmachines.ai/blog/on-policy-distillation

doi: 10.64434/tml.20251026. https://thinkingmachines.ai/blog/on-policy-distillation

doi

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.

[1] [1]

On-policy distillation of language models: Learning from self-generated mistakes

Rishabh Agarwal, Nino Vieillard, Yongchao Zhou, Piotr Stanczyk, Sabela Ramos Garea, Matthieu Geist, and Olivier Bachem. On-policy distillation of language models: Learning from self-generated mistakes. InInternational Conference on Learning Representations, volume 2024, pages 21246–21263, 2024

work page 2024

[2] [2]

On-policy distillation.Thinking Machines Lab: Connectionism,

Kevin Lu and Thinking Machines Lab. On-policy distillation.Thinking Machines Lab: Connectionism,

[3] [3]

Self- distilled reasoner: On-policy self-distillation for large language models.arXiv preprint arXiv:2601.18734, 2026

Siyan Zhao, Zhihui Xie, Mengchen Liu, Jing Huang, Guan Pang, Feiyu Chen, and Aditya Grover. Self- distilled reasoner: On-policy self-distillation for large language models.arXiv preprint arXiv:2601.18734, 2026

arXiv 2026

[4] [4]

Vision-opd: Learning to see fine details for multimodal llms via on-policy self-distillation.arXiv preprint arXiv:2605.18740, 2026

Qianhao Yuan, Jie Lou, Xing Yu, Hongyu Lin, Le Sun, Xianpei Han, and Yaojie Lu. Vision-opd: Learning to see fine details for multimodal llms via on-policy self-distillation.arXiv preprint arXiv:2605.18740, 2026

arXiv 2026

[5] [5]

Visual-OPSD: Cross-Modal On-Policy Self-Distillation for Efficient Unified Multimodal Reasoning

Pengyu Li, Zhitao Gao, Lingling Zhang, Muye Huang, Yuanming Li, Fangzhi Xu, and Jun Liu. Visual- opsd: Cross-modal on-policy self-distillation for efficient unified multimodal reasoning.arXiv preprint arXiv:2606.18974, 2026

work page Pith review arXiv 2026

[6] [6]

Visual-advantage on-policy distillation for vision-language models

Ruiqi Liu, Xiaolei Lv, Gengsheng Li, Ximo Zhu, Zhiheng Wang, Zhengbo Zhang, Junkai Chen, Zhiheng Li, Bo Li, Jun Gao, and Shu Wu. Visual-advantage on-policy distillation for vision-language models. arXiv preprint arXiv:2605.21924, 2026

arXiv 2026

[7] [7]

Visual Contrastive Self-Distillation

Yijun Liang, Yunjie Tian, Yijiang Li, Yuqi Jia, Furong Huang, Tianyi Zhou, and Di Fu. Visual contrastive self-distillation.arXiv preprint arXiv:2607.21556, 2026

work page Pith review arXiv 2026

[8] [8]

LLaVA steering: Visual instruction tuning with 500x fewer parameters through modality linear representation- steering

Jinhe Bi, Yujun Wang, Haokun Chen, Xun Xiao, Artur Hecker, Volker Tresp, and Yunpu Ma. LLaVA steering: Visual instruction tuning with 500x fewer parameters through modality linear representation- steering. In Wanxiang Che, Joyce Nabende, Ekaterina Shutova, and Mohammad Taher Pilehvar, editors, Proceedings of the 63rd Annual Meeting of the Association for ...

2025