Pith. sign in

REVIEW 5 major objections 6 minor 45 references

Aligning Large Vision-Language Models at Test Time: A Trajectory-Guided Structured Sampling Approach

T0 review · 5 major / 6 minor · reviewed 2026-08-05 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read A training-free framework aligns vision-language models at test time by sampling and refining complete reasoning traces with Metropolis–Hastings updates against a visual-grounding-aware target, matching several reinforcement-trained models

desk verdict Genuine new combination of trajectory priors and block-wise MCMC for test-time LVLM alignment, but the empirical case is undermined by an unexplained weak baseline and missing error bars; worth referee time, expect major revision. read the letter →

arxiv 2608.03204 v1 pith:NPKJA5VN submitted 2026-08-04 cs.CL cs.AI

Tianbao Jiang , Weicong Ni , Gerard de Melo , Linlin Wang This is my paper
classification cs.CLcs.AI
keywords test-timealignmentlargevision-languagemodelsMetropolis-Hastingssamplingvisualgroundingtrajectoryguidancereasoningmemorybankpower-scaledmultimodal
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper sets out to show that a large vision–language model can be aligned to a complex visual reasoning task without any weight update, by spending extra decoding-time compute on sampling and refining complete answer traces. The method is built around a 'reasoning memory bank' of solved questions decomposed into ordered sequences of seven high-level reasoning patterns; at inference, similar questions are retrieved and their patterns are majority-voted into a guidance trajectory that fixes the global step order, while the model itself fills in the content. Within that structure, an iterative Metropolis–Hastings sampler reshapes the model's output distribution toward the joint target $\tilde{P}(Y|V,X)^{\alpha}\cdot R_H(Y)\cdot R_L(Y)$ — a power-sharpened sequence likelihood multiplied by a vision-aware calibration that rewards tokens whose probability is boosted by visual input, an entropy regularizer that favors low-entropy reliable traces, and a linguistic term that penalizes repetition and premature stopping. If correct, the claim matters because the approach closes most of the accuracy gap to reinforcement-learning alignment — which needs expensive offline training and can mismatch the inference distribution — using only test-time compute, and the trajectory-guided variant does so about 1.3–1.5 times more cheaply than full-sequence resampling. Concretely, the paper reports average accuracy rising from 52.0 for the reproduced Qwen2.5-VL-7B baseline to 54.6 (full-sequence multi-objective sampling) and 54.3 (trajectory-guided structured sampling) across MathVista, MathVision, MathVerse, MMMU, and MMStar.

What carries the argument

Two objects carry the argument. (1) The alignment ratio $r_t=(P_{\text{grounded}}(y_t)+\delta)/(P_{\text{ungrounded}}(y_t)+\delta)$ — per-token ratio of next-token probability with visual input to that with visual tokens masked during prefill — converted by softplus-based calibration into a bounded weight $s(r_t)\in[0,1]$ that sharpens visually supported tokens and suppresses the rest. (2) The Metropolis–Hastings acceptance rule $A(Y,Y')=\min(1, p_t(Y')q(Y|Y')/p_t(Y)q(Y'|Y))$ applied to the joint target $p_t(Y)\propto \tilde{P}(Y|V,X)^{\alpha}R_H(Y)R_L(Y)$, under pattern-wise initialization (one new reasoning pattern per iteration), sliding-window proposals over the newest two patterns, and

What would settle it

Regenerate each trajectory under image perturbation: if $r_t$ truly isolates visual grounding, tokens with high $r_t$ should be precisely those whose grounded-branch probability flips when the referenced image region is blurred, cropped, or replaced, while the masked branch stays fixed. A direct control experiment would replace $s(r_t)$ with an uncorrelated random weight and check whether the accuracy gain disappears; if gains persist, the vision-aware calibration is not the mechanism. The claim would also be falsified by a question set where the correct answer is visually presented but lingui

Watch

Extended reading notes

Core claim

The paper claims that a frozen vision–language model can be aligned at decoding time by sampling from a reshaped sequence-level target. The target (Eq. 15) multiplies a power-scaled likelihood $\tilde{P}(Y|V,X)^{\alpha}$ ($\alpha=4$) with a vision-aware weight $s(r_t)\in[0,1]$ built from the alignment ratio $r_t=(P_{\text{grounded}}(y_t)+\delta)/(P_{\text{ungrounded}}(y_t)+\delta)$, an entropy regularizer $R_H(Y)$ favoring low-entropy traces, and a linguistic term $R_L(Y)$ rewarding effective length over repetition. To make MCMC practical, the sampler proposes one reasoning pattern at a time and refines a sliding window over the two most recent patterns, steered by a trajectory majority-vote

Load-bearing premise

The load-bearing premise is that the ratio $r_t$ of next-token probability with visual input to next-token probability with the visual tokens masked is a valid per-token measure of visual grounding; if the masked branch is not a faithful counterfactual, the vision-aware sharpening suppresses useful tokens instead of hallucinations — the paper itself shows the visual term alone lowers MathVista accuracy and only helps once coupled with entropy regularization.

Editorial extensions

If this is right

  • Without any parameter updates, the sampling framework closes most of the accuracy gap to RLVR-trained reasoning models and outperforms several of them on the five-benchmark suite.
  • Trajectory-guided structured sampling preserves most of the accuracy of full-sequence resampling while cutting token consumption from roughly 22–24× to 15–18× of greedy decoding — a 1.3–1.5× saving.
  • Entropy regularization alone is harmful (51.3 vs the 52.0 baseline); it only contributes when the vision-aware term first filters out confidently wrong low-entropy traces.
  • The joint alignment target transfers to other backbones (InternVL2.5-8B, Qwen3-VL-8B) with only the length-saturation parameter $L$ needing per-model adjustment.
  • In this setup stronger alignment shortens responses rather than lengthening them, so the effective-length reward must be calibrated to each dataset's length regime.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The ratio $r_t$ is effectively a per-token attribution of the image's causal contribution to generation; the same quantity could serve as a diagnostic that maps which chain-of-thought steps are visually supported, and it offers a testable route to step-level editing — the case study in Figure 5 already shows the sampler inserting a missing visual-detection step into a trajectory.
  • The annealed schedule tying temperature to the inverse sharpening exponent ($\alpha=\tau^{-1}$) is a generic recipe for MCMC decoding — explore until the reasoning structure is fixed, then sharpen — that could transfer to text-only reasoning models, where the vision-aware term of Eq. 8 would simply be dropped and the entropy and length terms would do all the work.
  • If power-scaled sampling lengthens responses in text-only LLMs but shortens them here (with visual calibration, response length falls as the objective strengthens), the discrepancy points at visual-attention decay over long generations as a likely cause — a hypothesis the paper raises but does not test.
  • Because the memory bank stores step-aligned, editable trajectories with explicit input–output interfaces, it can be treated as a reusable library that other sampling objectives or rule-based filters could operate on, potentially decoupling what structure to reason in from what the vision model sees.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

5 major / 6 minor

Summary. The paper proposes a training-free test-time alignment method for LVLMs. It first constructs a reasoning memory bank using an agentic planner–executor–manager pipeline over seven predefined reasoning patterns, then at inference retrieves similar trajectories, derives a guidance trajectory by majority voting, and performs Metropolis–Hastings refinement within a sliding window over reasoning patterns. The MCMC target (Eq. 15) combines a vision-calibrated power-scaled likelihood, an entropy regularizer, and a linguistic length/repetition control. On five multimodal reasoning benchmarks the authors report average accuracy gains of +2.6 and +2.3 over a reproduced Qwen2.5-VL-7B baseline, with trajectory-guided sampling consuming 1.3–1.5x fewer tokens than full-sequence resampling.

Significance. If substantiated, this is a useful contribution: it shows that decoding-time MCMC with a carefully shaped target can partially substitute for RLVR-style post-training in LVLMs, and the trajectory-memory idea is a sensible way to impose a structural prior without parameter updates. The paper includes a concrete algorithm, a well-motivated objective, ablations isolating each term, and an analysis of token consumption, which are strengths. However, no code or data is released, the reproduced baseline is not transparently described, and the stochastic MCMC results are reported without variance. These gaps currently prevent the empirical claims from being verified.

major comments (5)
  1. [§4.1.3 and Table 1] The central comparative claim is relative to 'Qwen2.5-VL-7B (Reproduced)', whose scores are unexplained and inconsistent with the official numbers in the same table: MMMU 54.0 vs 58.6, MMStar 61.3 vs 63.9, MathVista 70.0 vs 68.2. No decoding details (temperature, prompt, evaluation harness, sampling budget) or error bars are given, although MCMC is stochastic. This makes the reported +2.6/+2.3 average gains unverifiable. The authors should report the exact reproduction protocol, run the official baselines under the same harness, and provide per-seed or bootstrap intervals.
  2. [§B.5 and §4.1.3] Hyperparameters beta=2.0 and lambda_H=1.0 are claimed to be robust, but Figure 9 reports sensitivity only on MMStar. If these values were chosen based on test-set performance on MMStar and then applied to all five datasets, the cross-dataset gains are at risk of selection bias. Please report sensitivity on all benchmarks or use a held-out validation split, and state how the values in §4.1.3 were selected.
  3. [§3.3.1, Eqs. (8)–(10)] The alignment ratio r_t assumes the visually masked branch is a valid counterfactual for per-token grounding. This assumption is load-bearing, and the ablation in Table 3 provides only weak support: adding the visual term alone lowers MathVista accuracy (70.8 vs 72.0), and gains appear only when combined with entropy regularization. The paper would be strengthened by directly validating r_t (e.g., on images where visual evidence is misleading, or by comparing suppressed tokens against ground-truth object annotations) and by reporting the per-dataset behavior of the visual calibration term.
  4. [§4.5, Table 4 vs Table 1] The Qwen3-VL-8B baseline is inconsistent between tables: MMStar is 65.3 in Table 4 but 70.9 in Table 1. If Table 4 uses a different reproduction protocol, this must be stated; otherwise the generalization claim (+7.9 on MMStar) is not credible. The same concern applies to the InternVL2.5-8B results, since no reproduction details are given in §4.5.
  5. [§3.1 and §4.1.1] The memory bank is curated from ThinkLite-70k and retrieval is used to guide inference on MathVista, MathVerse, MathVision, MMMU, and MMStar. If benchmark questions or close variants appear in the seed set, retrieval could leak answer information. The authors should quantify overlap between the memory bank and each evaluation benchmark, and ideally rerun the key comparison with a memory bank built from data disjoint from the test distributions.
minor comments (6)
  1. [§1 and running header] 'three-folds' should be 'threefold'; the running header contains the typo 'Alingment'.
  2. [Tables 2 and 3] Table 3 uses 'Mathvision' while Table 2 uses 'MathVision'; please make capitalization consistent.
  3. [§B.4] The POPE and CHAIR references appear as placeholders ('[?]'); proper citations are needed.
  4. [Algorithm 1 and §3.2] The block proposal distribution q and the exact acceptance probability for windowed updates are not specified; full-sequence MH is attributed to [14], but the TG variant is not reproducible from the text alone.
  5. [Eq. (8)] The definition of r_t is written for 'the token chosen by the grounded branch'; since Eq. (10) calibrates next-token probabilities, it should be defined for every candidate token in the vocabulary.
  6. [§4.7] Table 5 reports token consumption in generated tokens but does not report wall-clock latency or FLOPs; forward-pass counting may not fully capture the actual overhead of the two-branch visual calibration.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the target distribution is a self-referential scoring heuristic, but the claimed accuracy gains are external empirical results; baseline discrepancies are a robustness concern, not circularity.

full rationale

The paper's derivation chain is not circular. The alignment target in Eq. 15 is built from the base model's own likelihood, its predictive entropy, and a grounded/ungrounded probability ratio (Eqs. 8–14), but these quantities are used as an MCMC acceptance score, not as the evidence for the paper's central claim. The central claim—consistent accuracy gains without parameter updates—is supported by external benchmark accuracy numbers. Using a model's own likelihood as a decoding-time reweighting signal is a known heuristic (Karan and Du, external work [14]), and the paper does not present this heuristic as a theorem derived from first principles. No fitted parameter is renamed as a prediction: the hyperparameters (alpha, beta, lambda_H, L) are fixed global constants, and the entropy signal was observed on ThinkLite-70k, not on the evaluation benchmarks. The paper contains no load-bearing self-citations: none of the references is prior work by the present authors, and no uniqueness theorem or ansatz is smuggled in via self-citation. The trajectory patterns and visual-calibration mechanism are explicitly attributed to external prior work. The main concerns are experimental: the reproduced Qwen2.5-VL-7B baseline is lower than official scores on MMMU and MMStar, and decoding details for the baseline are not reported. These are correctness/robustness issues, not circularity. A self-referential target distribution is a legitimate test-time heuristic, and the paper's reported gains remain externally measurable, so the circularity score is 0.

Assumptions & free parameters 9 free parameters · 6 assumptions · 0 invented entities

The method relies on a large set of hand-set hyperparameters, several tuned on an evaluation benchmark (MMStar), plus unspecified key values (search threshold, top-k, MCMC steps). The assumptions are standard MCMC math plus a series of domain heuristics (power-scaling quality, visual counterfactual validity, pattern sufficiency, cross-model transfer, entropy-correctness correlation) that are plausible but not independently verified.

free parameters (9)
  • alpha (power-sampling exponent) = 4
    Set in Section 4.1.3; no sensitivity analysis reported.
  • beta (visual suppression strength) = 2.0
    Selected on MMStar (Figure 9), which is also an evaluation benchmark.
  • lambda_H (entropy weight) = 1.0
    Selected on MMStar (Figure 9); test-set leakage concern.
  • L (length-reward saturation) = 128
    Hand-set; authors state L should be calibrated per dataset (Section 4.4).
  • a, b (length-reward shaping) = a=0.05, b=0.8
    Hand-set; no sensitivity shown.
  • kappa, gamma (annealed schedule threshold and decay) = kappa=4, gamma=0.5
    Hand-set in Section 4.1.3.
  • trajectory-search threshold tau
    Mentioned in Section 3.1 but value not specified.
  • retrieval top-k
    Not specified; only pool size M=100 given in Appendix A.2.
  • MCMC steps N_MCMC for main results
    Scaling analyzed in B.3 but value used in Tables 1-5 not stated.
assumptions (6)
  • standard math Metropolis-Hastings acceptance ratio (Eq. 7) defines a Markov chain with the unnormalized target as stationary distribution.
    Used in Section 3.2 to justify iterative refinement; this is a classical result.
  • domain assumption Power-scaled distribution p_alpha(Y) ∝ P(Y)^alpha concentrates on higher-quality reasoning sequences.
    Borrowed from Karan and Du [14]; no proof given, only empirical evidence in prior work.
  • ad hoc to paper The visually-masked ungrounded branch provides a valid counterfactual for measuring token-level visual grounding.
    Defined in Section 3.3.1 (Eq. 8); the paper gives no validation that the ratio r_t isolates grounding rather than distributional shift from input corruption.
  • domain assumption The seven reasoning patterns in Table 6 are a sufficient decomposition of multimodal reasoning.
    Chosen based on prior works [18,32] in Section 3.1; no evidence that this set covers all reasoning paths or that the pseudo-code abstraction is faithful.
  • domain assumption Trajectories curated with Qwen3-VL-8B transfer to improve Qwen2.5-VL-7B reasoning.
    Used in Section 3.1-3.2; the transfer is only evaluated on Qwen2.5-VL-7B, and Table 4 does not test trajectory guidance on other models.
  • domain assumption Lower predictive entropy is a proxy for answer correctness.
    Motivated by Figure 6 in Appendix B.1; the paper acknowledges this is not infallible (confidently wrong) and couples it with visual calibration.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Aligning Large Vision-Language Models at Test Time: A Trajectory-Guided Structured Sampling Approach." pith.science (2026). https://pith.science/paper/NPKJA5VN

@misc{pith2026260803204,
  author       = {Pith},
  title        = {Pith review of: Aligning Large Vision-Language Models at Test Time: A Trajectory-Guided Structured Sampling Approach},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/NPKJA5VN}},
  note         = {Machine review of arXiv:2608.03204}
}
read the original abstract

Post-training reinforcement learning (RL) algorithms are commonly used to align large vision-language models (LVLMs) with human intent and the requirements of visual reasoning tasks. However, existing RL-based alignment methods are often resource-intensive and encounter mismatches between training objectives and inference-time distributions. To bridge this gap, we propose a novel test-time alignment approach that leverages trajectory-guided structured sampling for dynamic inference-time refinement, achieving better alignment with visual grounding and ensuring logical consistency. Our approach begins with curating a reasoning memory bank via a trajectory learning algorithm, which decomposes complex question solving into ordered sequences of predefined reasoning patterns. It subsequently accomplishes inference-time alignment by first collecting trajectories from reasoning memory bank to establish a global structural reasoning prior, and then using an iterative Markov Chain Monte Carlo (MCMC) algorithm for localized multi-objective refinement of the reasoning trace. Experiments across multiple multimodal reasoning datasets demonstrate that our approach significantly improves accuracy without incurring prohibitive inference overhead. These results establish trajectory-guided test-time sampling as a scalable and effective alternative to traditional post-training alignment, particularly for complex visual reasoning tasks.

Figures

Figures reproduced from arXiv: 2608.03204 by the authors.

Figure 1
Figure 1. Standard autoregressive decoding makes locally [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 2
Figure 2. Overview of our approach. We begin by constructing a reasoning memory bank based on a trajectory learning [PITH_FULL_IMAGE:figures/full_fig_p003_2.png] view at source ↗
Figure 3
Figure 3. Automatic trajectory memory curation. After defining reasoning patterns and preparing the seed dataset, the reasoning [PITH_FULL_IMAGE:figures/full_fig_p004_3.png] view at source ↗
Figures from the paper (7 more)
Figure 4
Figure 4. Figure 4: Likelihood and entropy of Qwen2.5-VL-7B re [PITH_FULL_IMAGE:figures/full_fig_p007_4.png]
Figure 5
Figure 5. Figure 5: Illustrative case study. a 1.3–1.5× reduction relative to multi- objective sampling, while exhibiting longer response length. 4.8 Case Study [PITH_FULL_IMAGE:figures/full_fig_p008_5.png]
Figure 6
Figure 6. Figure 6: Average smoothed entropy over timestep [PITH_FULL_IMAGE:figures/full_fig_p012_6.png]
Figure 7
Figure 7. Figure 7: Effect of MCMC refinement steps on MathVista and [PITH_FULL_IMAGE:figures/full_fig_p012_7.png]
Figure 8
Figure 8. Figure 8: Case study of different sampling variants. [PITH_FULL_IMAGE:figures/full_fig_p013_8.png]
Figure 9
Figure 9. Figure 9: Hyperparameter sensitivity analysis on the MMStar [PITH_FULL_IMAGE:figures/full_fig_p013_9.png]
Figure 10
Figure 10. Figure 10: Visualization of vision-aware calibration weight [PITH_FULL_IMAGE:figures/full_fig_p014_10.png]

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

45 extracted references · 19 canonical work pages

  1. [1]

    Hewett, Mojan Javaheripi, Piero Kauff- mann, James R

    Marah Abdin, Jyoti Aneja, Harkirat Behl, Sébastien Bubeck, Ronen Eldan, Suriya Gunasekar, Michael Harrison, Russell J. Hewett, Mojan Javaheripi, Piero Kauff- mann, James R. Lee, Yin Tat Lee, Yuanzhi Li, Weishung Liu, Caio C. T. Mendes, Anh Nguyen, Eric Price, Gustavo de Rosa, Olli Saarikivi, Adil Salim, Shital Shah, Xin Wang, Rachel Ward, Yue Wu, Dingli Y...

    arXiv 2024

  2. [2]

    Shuai Bai, Yuxuan Cai, Ruizhe Chen, Keqin Chen, Xionghui Chen, Zesen Cheng, Lianghao Deng, Wei Ding, Chang Gao, Chunjiang Ge, Wenbin Ge, Zhifang Guo, Qidong Huang, Jie Huang, Fei Huang, Binyuan Hui, Shutong Jiang, Zhaohai Li, Mingsheng Li, Mei Li, Kaixin Li, Zicheng Lin, Junyang Lin, Xuejing Liu, Jiawei Liu, Chenglong Liu, Yang Liu, Dayiheng Liu, Shixuan ...

    arXiv 2025

  3. [3]

    Shuai Bai, Keqin Chen, Xuejing Liu, Jialin Wang, Wenbin Ge, Sibo Song, Kai Dang, Peng Wang, Shijie Wang, Jun Tang, Humen Zhong, Yuanzhi Zhu, Mingkun Yang, Zhaohai Li, Jianqiang Wan, Pengfei Wang, Wei Ding, Zheren Fu, Yiheng Xu, Jiabo Ye, Xi Zhang, Tianbao Xie, Zesen Cheng, Hang Zhang, Zhibo Yang, Haiyang Xu, and Junyang Lin. 2025. Qwen2.5-VL Technical Rep...

    arXiv 2025

  4. [4]

    Rosenthal

    Austin Brown and Jeffrey S. Rosenthal. 2025. Upper and lower bounds on the subgeometric convergence of adaptive Markov chain Monte Carlo. arXiv:2411.17084 [math.ST] https://arxiv.org/abs/2411.17084

    arXiv 2025

  5. [5]

    Lin Chen, Jinsong Li, Xiaoyi Dong, Pan Zhang, Yuhang Zang, Zehui Chen, Haodong Duan, Jiaqi Wang, Yu Qiao, Dahua Lin, and Feng Zhao. 2024. Are We on the Right Way for Evaluating Large Vision-Language Models?. InThe Thirty-eighth Annual Conference on Neural Information Processing Systems. https: //openreview.net/forum?id=evP9mxNNxJ

    work page 2024

  6. [6]

    Zhe Chen, Weiyun Wang, Yue Cao, Yangzhou Liu, Zhangwei Gao, Erfei Cui, Jinguo Zhu, Shenglong Ye, Hao Tian, Zhaoyang Liu, Lixin Gu, Xuehui Wang, Qingyun Li, Yiming Ren, Zixuan Chen, Jiapeng Luo, Jiahao Wang, Tan Jiang, Bo Wang, Conghui He, Botian Shi, Xingcheng Zhang, Han Lv, Yi Wang, Wenqi Shao, Pei Chu, Zhongying Tu, Tong He, Zhiyong Wu, Huipeng Deng, Ji...

    arXiv 2025

  7. [7]

    Ganqu Cui, Yuchen Zhang, Jiacheng Chen, Lifan Yuan, Zhi Wang, Yuxin Zuo, Haozhan Li, Yuchen Fan, Huayu Chen, Weize Chen, et al . 2025. The Entropy Mechanism of Reinforcement Learning for Reasoning Language Models.arXiv preprint arXiv:2505.22617(2025)

    arXiv 2025

  8. [8]

    Yihe Deng, Hritik Bansal, Fan Yin, Nanyun Peng, Wei Wang, and Kai-Wei Chang

Show all 45 references
  1. [9]

    Gonçalo Faria, Sweta Agrawal, António Farinhas, Ricardo Rei, José G. C. de Souza, and Andre Martins. 2024. QUEST: Quality-Aware Metropolis-Hastings Sampling for Machine Translation. InThe Thirty-eighth Annual Conference on Neural Infor- mation Processing Systems. https://openr...

    2024

  2. [10]

    Nanxing Hu, Xiaoyue Duan, Jinchao Zhang, and Guoliang Kang. 2025. Enhancing Visual Reliance in Text Generation: A Bayesian Perspective on Mitigating Halluci- nation in Large Vision-Language Models. InProceedings of the 33rd ACM Interna- tional Conference on Multimedia(Dublin, ...

    2025

  3. [11]

    Wenxuan Huang, Bohan Jia, Zijie Zhai, Shaosheng Cao, Zheyu Ye, Fei Zhao, Zhe Xu, Yao Hu, and Shaohui Lin. 2025. Vision-R1: Incentivizing Reasoning Capability in Multimodal Large Language Models. arXiv:2503.06749 [cs.CV] https://arxiv.org/abs/2503.06749

    2025 arXiv

  4. [12]

    Fushuo Huo, Wenchao Xu, Zhong Zhang, Haozhao Wang, Zhicheng Chen, and Peilin Zhao. 2025. Self-Introspective Decoding: Alleviating Hallucinations for Large Vision-Language Models. InThe Thirteenth International Conference on Learning Representations. https://openreview.net/foru...

    2025

  5. [13]

    Sekitoshi Kanai, Tsukasa Yoshida, Hiroshi Takahashi, Haru Kuroki, and Kazu- mune Hashimoto. 2025. Test-Time Alignment of LLMs via Sampling-Based Optimal Control in pre-logit space. arXiv:2510.26219 [cs.LG] https://arxiv.org/ abs/2510.26219

    2025 arXiv

  6. [14]

    Aayush Karan and Yilun Du. 2025. Reasoning with Sampling: Your Base Model is Smarter Than You Think. arXiv:2510.14901 [cs.LG] https://arxiv.org/abs/2510. 14901

    2025 arXiv

  7. [15]

    Yoonho Lee, Jonathan Williams, Henrik Marklund, Archit Sharma, Eric Mitchell, Anikait Singh, and Chelsea Finn. 2025. Inference-Time Alignment via Hypothesis Reweighting. InSecond Workshop on Test-Time Adaptation: Putting Updates to the Test! at ICML 2025. https://openreview.ne...

    2025

  8. [17]

    Ming Li, Nan Zhang, Chenrui Fan, Hong Jiao, Yanbin Fu, Sydney Peters, Qingshu Xu, Robert Lissitz, and Tianyi Zhou. 2025. Understanding the Thinking Pro- cess of Reasoning Models: A Perspective from Schoenfeld’s Episode Theory. In Proceedings of the 2025 Conference on Empirical...

    2025 doi

  9. [18]

    Sunzhu Li, Zhiyu Lin, Shuling Yang, Jiale Zhao, and Wei Chen. 2025. ThinkPi- lot: Steering Reasoning Models via Automated Think-prefixes Optimization. arXiv:2510.12063 [cs.AI] https://arxiv.org/abs/2510.12063

    2025

  10. [19]

    Zongxia Li, Wenhao Yu, Chengsong Huang, Rui Liu, Zhenwen Liang, Fux- iao Liu, Jingxi Che, Dian Yu, Jordan Boyd-Graber, Haitao Mi, and Dong Yu

  11. [20]

    Baijiong Lin, Weisen Jiang, Yuancheng Xu, Hao Chen, and Ying-Cong Chen. 2025. PARM: Multi-Objective Test-Time Alignment via Preference-Aware Autoregres- sive Reward Model. InForty-second International Conference on Machine Learning. https://openreview.net/forum?id=zm53HtGiXN

    2025

  12. [21]

    arXiv:2508.19652 [cs.CV] https://arxiv.org/abs/2508.19652

    Self-Rewarding Vision-Language Model via Reasoning Decomposition. arXiv:2508.19652 [cs.CV] https://arxiv.org/abs/2508.19652

    arXiv

  13. [22]

    Pan Lu, Hritik Bansal, Tony Xia, Jiacheng Liu, Chunyuan Li, Hannaneh Hajishirzi, Hao Cheng, Kai-Wei Chang, Michel Galley, and Jianfeng Gao. 2024. MathVista: Evaluating Mathematical Reasoning of Foundation Models in Visual Contexts. InThe Twelfth International Conference on Lea...

    2024

  14. [23]

    Ziyu Liu, Yuhang Zang, Xiaoyi Dong, Pan Zhang, Yuhang Cao, Haodong Duan, Conghui He, Yuanjun Xiong, Dahua Lin, and Jiaqi Wang. 2025. MIA-DPO: Multi- Image Augmented Direct Preference Optimization For Large Vision-Language Models. InThe Thirteenth International Conference on Le...

    2025

  15. [24]

    Fanqing Meng, Lingxiao Du, Zongkai Liu, Zhixiang Zhou, Quanfeng Lu, Daocheng Fu, Tiancheng Han, Botian Shi, Wenhai Wang, Junjun He, Kaipeng Zhang, Ping Luo, Yu Qiao, Qiaosheng Zhang, and Wenqi Shao. 2025. MM-Eureka: Exploring the Frontiers of Multimodal Reasoning with Rule-bas...

    2025 arXiv

  16. [25]

    Pan Lu, Bowen Chen, Sheng Liu, Rahul Thapa, Joseph Boen, and James Zou. 2025. OctoTools: An Agentic Framework with Extensible Tools for Complex Reasoning. arXiv:2502.11271 [cs.LG] https://arxiv.org/abs/2502.11271

    2025 arXiv

  17. [26]

    Xinyu Tian, Shu Zou, Zhaoyuan Yang, Mengqi He, Fabian Waschkowski, Lukas Wesemann, Peter Tu, and Jing Zhang. 2025. More Thought, Less Accuracy? On the Dual Nature of Reasoning in Vision-Language Models. arXiv:2509.25848 [cs.CV] https://arxiv.org/abs/2509.25848

    2025

  18. [27]

    Rosenbluth, Marshall N

    Nicholas Metropolis, Arianna W. Rosenbluth, Marshall N. Rosenbluth, Augusta H. Teller, and Edward Teller. 1953. Equation of State Calculations by Fast Computing Machines.Journal of Chemical Physics21, 6 (1953), 1087–1092

    1953

  19. [28]

    Haozhe Wang, Chao Qu, Zuming Huang, Wei Chu, Fangzhen Lin, and Wenhu Chen. 2025. VL-Rethinker: Incentivizing Self-Reflection of Vision-Language Models with Reinforcement Learning. InThe Thirty-ninth Annual Conference on Neural Information Processing Systems. https://openreview...

    2025

  20. [29]

    Chenxi Wang, Xiang Chen, Ningyu Zhang, Bozhong Tian, Haoming Xu, Shumin Deng, and Huajun Chen. 2025. MLLM can see? Dynamic Correction Decoding for Hallucination Mitigation. InThe Thirteenth International Conference on Learning Representations. https://openreview.net/forum?id=4...

    2025

  21. [30]

    Kuang-Da Wang, Teng-Ruei Chen, Yu Heng Hung, Guo-Xun Ko, Shuoyang Ding, Yueh-Hua Wu, Yu-Chiang Frank Wang, Chao-Han Huck Yang, Wen-Chih Peng, and Ping-Chun Hsieh. 2025. Test-Time Alignment for Large Language Models via Textual Model Predictive Control. arXiv:2502.20795 [cs.CL]...

    2025

  22. [31]

    Ke Wang, Junting Pan, Weikang Shi, Zimu Lu, Houxing Ren, Aojun Zhou, Mingjie Zhan, and Hongsheng Li. 2024. Measuring Multimodal Mathematical Reasoning with MATH-Vision Dataset. InThe Thirty-eight Conference on Neural Information Processing Systems Datasets and Benchmarks Track...

    2024

  23. [32]

    Jinyang Wu, Mingkuan Feng, Guocheng Zhai, Shuai Zhang, Zheng Lian, Fan- grui Lv, Pengpeng Shao, Ruihan Jin, Zhengqi Wen, and Jianhua Tao. 2026. AStar: Boosting Multimodal Reasoning with Automated Structured Thinking. arXiv:2502.02339 [cs.CL] https://arxiv.org/abs/2502.02339

    2026

  24. [33]

    Xiyao Wang, Zhengyuan Yang, Chao Feng, Hongjin Lu, Linjie Li, Chung-Ching Lin, Kevin Lin, Furong Huang, and Lijuan Wang. 2025. SoTA with Less: MCTS- Guided Sample Selection for Data-Efficient Visual Reasoning Self-Improvement. arXiv:2504.07934 [cs.CV] https://arxiv.org/abs/2504.07934

    2025 arXiv

  25. [34]

    Zhongxing Xu, Chengzhi Liu, Qingyue Wei, Juncheng Wu, James Zou, Xin Eric Wang, Yuyin Zhou, and Sheng Liu. 2025. More Thinking, Less Seeing? Assessing Amplified Hallucination in Multimodal Reasoning Models. InThe Thirty-ninth Annual Conference on Neural Information Processing ...

    2025

  26. [35]

    Yuancheng Xu, Udari Madhushani Sehwag, Alec Koppel, Sicheng Zhu, Bang An, Furong Huang, and Sumitra Ganesh. 2025. GenARM: Reward Guided Generation with Autoregressive Reward Model for Test-time Alignment. arXiv:2410.08193 [cs.CL] https://arxiv.org/abs/2410.08193

    2025 arXiv

  27. [36]

    Hao Yin, Gunagzong Si, and Zilei Wang. 2025. ClearSight: Visual Signal Enhance- ment for Object Hallucination Mitigation in Multimodal Large Language Models. 14625–14634. doi:10.1109/CVPR52734.2025.01363

    2025

  28. [37]

    Huanjin Yao, Jiaxing Huang, Wenhao Wu, Jingyi Zhang, Yibo Wang, Shunyu Liu, Yingjie Wang, YuXin Song, Haocheng Feng, Li Shen, and Dacheng Tao

  29. [38]

    InThe Thirty-ninth Annual Conference on Neural Information Processing Systems

    Mulberry: Empowering MLLM with o1-like Reasoning and Reflection via Collective Monte Carlo Tree Search. InThe Thirty-ninth Annual Conference on Neural Information Processing Systems. https://openreview.net/forum?id= lwOV2ACEK9

  30. [39]

    Renrui Zhang, Dongzhi Jiang, Yichi Zhang, Haokun Lin, Ziyu Guo, Pengshuo Qiu, Aojun Zhou, Pan Lu, Kai-Wei Chang, Peng Gao, et al. 2024. MathVerse: Does Your Multi-modal LLM Truly See the Diagrams in Visual Math Problems?arXiv preprint arXiv:2403.14624(2024)

    2024 arXiv

  31. [40]

    Tianyu Yu, Yuan Yao, Haoye Zhang, Taiwen He, Yifeng Han, Ganqu Cui, Jinyi Hu, Zhiyuan Liu, Hai-Tao Zheng, and Maosong Sun. 2024. RLHF-V: Towards Trustworthy MLLMs via Behavior Alignment from Fine-Grained Correctional Human Feedback. In2024 IEEE/CVF Conference on Computer Visio...

    2024

  32. [41]

    Xiang Yue, Yuansheng Ni, Kai Zhang, Tianyu Zheng, Ruoqi Liu, Ge Zhang, Samuel Stevens, Dongfu Jiang, Weiming Ren, Yuxuan Sun, Cong Wei, Botao Yu, Ruibin Yuan, Renliang Sun, Ming Yin, Boyuan Zheng, Zhenzhu Yang, Yibo Liu, Wenhao Huang, Huan Sun, Yu Su, and Wenhu Chen. 2024. MMM...

    2024

  33. [42]

    Jinguo Zhu, Weiyun Wang, Zhe Chen, Zhaoyang Liu, Shenglong Ye, Lixin Gu, Hao Tian, Yuchen Duan, Weijie Su, Jie Shao, Zhangwei Gao, Erfei Cui, Xuehui Wang, Yue Cao, Yangzhou Liu, Xingguang Wei, Hongjie Zhang, Haomin Wang, Weiye Xu, Hao Li, Jiahao Wang, Nianchen Deng, Songze Li,...

    2025 arXiv

  34. [43]

    Ruohong Zhang, Bowen Zhang, Yanghao Li, Haotian Zhang, Zhiqing Sun, Zhe Gan, Yinfei Yang, Ruoming Pang, and Yiming Yang. 2025. Improve Vision Lan- guage Model Chain-of-thought Reasoning. InProceedings of the 63rd Annual Meeting of the Association for Computational Linguistics ...

    2025

  35. [45]

    Stephen Zhao, Rob Brekelmans, Alireza Makhzani, and Roger Grosse. 2024. prob- abilistic inference in language models via twisted sequential Monte Carlo. In Proceedings of the 41st International Conference on Machine Learning(Vienna, Austria)(ICML’24). JMLR.org, Article 2512, 45 pages

    2024

  36. [1662]

    doi:10.18653/v1/2025.acl-long.82

    2025 doi

  37. [2025]

    arXiv:2503.17352 [cs.CV] https://arxiv.org/abs/2503.17352

    OpenVLThinker: Complex Vision-Language Reasoning via Iterative SFT-RL Cycles. arXiv:2503.17352 [cs.CV] https://arxiv.org/abs/2503.17352

    arXiv

Pith tools

Reviewed August 5, 2026 · model on record in the stance chip above.