Pith. sign in

REVIEW 4 major objections 7 minor 6 cited by

Progressive Multimodal Reasoning via Active Retrieval

T0 review · 4 major / 7 minor · reviewed 2026-08-11 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read AR-MCTS shows that retrieving key insights at each reasoning step, rather than beam-search sampling, improves multimodal reasoning.

desk verdict Useful new configuration of per-step retrieval inside MCTS for MLLMs, with a clean GAOKAO split, but the main benchmark results are clouded by cross-contamination in the retrieval corpus. read the letter →

arxiv 2412.14835 v1 pith:ULHYZZW6 submitted 2024-12-19 cs.CL cs.AIcs.CVcs.IR

Guanting Dong , Chenghao Zhang , Mengjie Deng , Yutao Zhu , Zhicheng Dou , Ji-Rong Wen This is my paper · ORCID
classification cs.CLcs.AIcs.CVcs.IR
keywords activeretrievalmultimodalreasoningMonteCarlotreesearchprocessrewardmodelretrieval-augmentedgenerationlargelanguagemodelsverification
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper tries to establish that the weak point of Monte Carlo tree search for multimodal reasoning is not the verifier but the way candidate steps are generated: multimodal models expand reasoning paths from their own internal knowledge, which is often misaligned across image and text. The authors replace beam-search expansion with active retrieval of hybrid-modal insights at every step and use the search itself to produce step-level annotations that train a process reward model. On MATHVISTA, WE-MATH, and the Chinese GAOKAO-MM benchmark, the resulting AR-MCTS framework improves accuracy over self-consistency, self-correction, and outcome reward models across several open and closed multimodal models. The claim matters because it offers a path to reliable stepwise verification in multimodal reasoning without human annotations.

What carries the argument

The carrying object is the AR-MCTS loop built on the factorization in Equation (1), which splits reasoning success into an expansion term (how candidate next steps are generated) and a simulation term (how paths are scored). Expansion uses a unified retrieval module: Contriever for text retrieval and CLIP for cross-modal retrieval, fused through FAISS, followed by a knowledge-concept filter that keeps only retrieved items similar to both the query and its fine-grained concept label. Each MCTS node re-retrieves insights conditioned on the accumulated reasoning steps, and a one-step rollout values each node by whether sampled answers match the ground truth, with back-propagation updating node values. A curriculum process reward model, pre-aligned with step-wise DPO and then fine-tuned with point-wise logistic labels, supplies the step scores used at inference.

What would settle it

Run AR-MCTS with the retrieval corpus restricted to sources with zero n-gram and image overlap with the test sets, or on a newly released multimodal reasoning benchmark created after the corpus was built; if the gains over beam search vanish or shrink to the level of contamination baselines, the central claim fails. A simpler check: for each solved test item, inspect the top retrieved insight and test whether deleting near-duplicate documents removes the improvement.

Watch

Extended reading notes

Core claim

The central claim is that reasoning-path expansion, not just simulation, decides success in multimodal MCTS reasoning: $p(y|x) = \max_i p_\theta(y|r_i,x)\,p_\phi(r_i|x)$. Since MLLMs' internal knowledge is unreliable for multimodal steps, the paper augments each expansion with actively retrieved insights from a hybrid-modal corpus, refreshed step by step, and filters them by knowledge-concept consistency. The search then generates positive and negative step pairs automatically, and a two-stage curriculum, step-wise DPO pre-alignment followed by point-wise fine-tuning, yields a process reward model that scores each step at inference. The paper reports consistent gains across models such as GPT-4o, Qwen2-VL-7B, and InternVL2-8B, with the largest improvements on the multi-step WE-MATH S3 metric and on smaller models.

Load-bearing premise

The load-bearing premise is that the retrieval corpus and the PRM's training data are truly separate from the evaluation benchmarks: the regex filtering, the 13-gram contamination threshold, and the time-based split are assumed to remove any test-influencing material, while cross-including WE-MATH testmini in MATHVISTA's knowledge base and vice versa is assumed not to leak solutions.

Editorial extensions

If this is right

  • If correct, inference-time scaling for multimodal models should spend compute on retrieving per-step knowledge rather than only on sampling more beams.
  • Process reward models for multimodal reasoning can be built automatically from MCTS rollouts, removing the manual step-annotation bottleneck.
  • The framework's largest gains on smaller MLLMs imply active retrieval can compensate for internal knowledge gaps without retraining the base model.
  • Knowledge-concept filtering is a load-bearing component: removing it costs accuracy, indicating that retrieved noise, not just missing knowledge, hurts multimodal reasoning.
  • Generalization to the Chinese GAOKAO-MM benchmark suggests the recipe transfers across languages and disciplines beyond mathematics.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • A natural extension the paper does not pursue: because retrieval, not search depth, appears to carry much of the gain, stronger retrievers or rerankers should yield further improvements at the same sampling budget.
  • The contamination controls (regex filtering, a 13-gram overlap threshold, and the temporal split for GAOKAO-MM) are the main trust boundary; an independent evaluation on a brand-new multimodal reasoning set would settle whether gains are reasoning gains or retrieval of near-duplicate solutions.
  • The per-step re-retrieval mechanism could transfer to other multimodal agent tasks such as science-diagram understanding or medical imaging reasoning, where stepwise knowledge needs are heterogeneous.
  • Since the PRM is trained on a text backbone while scoring multimodal steps, an obvious test is whether jointly aligning the PRM inside the vision-language model itself, which the paper lists as future work, further improves verification.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 7 minor

Summary. The paper proposes AR-MCTS, a framework that augments Monte Carlo Tree Search (MCTS) decoding for multimodal large language models (MLLMs) with active retrieval: at each expansion step, a hybrid-modal retrieval module supplies problem-solving insights that are filtered by knowledge-concept similarity, and a process reward model (PRM) is trained from automatically generated step-level annotations via step-wise DPO and point-wise fine-tuning. The method is evaluated on MATHVISTA and WE-MATH testmini, with an additional cross-domain evaluation on GAOKAO-MM using a temporal split (2010-2021 train, 2022-2023 test), across GPT-4o, GPT-4V, LLaVA-OneVision-72B, InternVL2-8B, Qwen2-VL-7B, and LLaVA-NeXT. The authors report consistent improvements over Self-Consistency, Self-Correction, and ORM baselines, and present ablations, scaling analyses, diversity visualizations, and a contamination analysis of the retrieval corpus.

Significance. If the reported gains are trustworthy, the core idea is valuable: dynamically retrieving step-specific knowledge during MCTS expansion, rather than relying on the model's internal knowledge or static beam search, is a sensible and timely direction for multimodal reasoning, and the automated generation of step-level supervision without human annotation is an important practical contribution. The paper tests the framework on several open and closed MLLMs, which strengthens the generality claim. The GAOKAO temporal split is a genuinely cleaner evaluation design than the two primary benchmarks. However, the significance is conditional: the two benchmark results that carry most of the quantitative weight are compromised by the retrieval-corpus construction, and the DPO objective as written is mathematically vacuous. The framework's promise is therefore not yet demonstrated by the evidence in the manuscript.

major comments (4)
  1. [Supplementary §B.4 and Table 7] The retrieval corpus construction directly undermines the central claim on MATHVISTA and WE-MATH. Supplementary §B.4 states that 'we incorporate the testmini set of WE-MATH into the knowledge base of MATHVISTA and include the testmini set of MATHVISTA into the knowledge base of WE-MATH.' This means that for each benchmark, solved near-duplicate problems from the other benchmark's testmini can be retrieved as 'knowledge' and their answers copied during inference. Table 7 reports a 4.2% 13-gram overlap for MATHVISTA-testmini retrieved by WE-MATH queries, yet the text concludes 'there is no overlap between our retrieval library and the test sets.' The 13-gram exact-overlap test is a weak contamination proxy: paraphrased, translated, or visually re-rendered versions of the same problem evade it, and the regex filtering described in §4.1 is not specified in enough detail to verify that it removes all test-influencing material. Because the reported AR-MCTS gains over the ORM/SC baselines in Table 1 are often only 1-5 points, even a few percent of retrieved near-duplicates could explain the apparent improvements. The evaluation must be redone with the cross-included testmini sets removed (or with a properly held-out corpus), and the contamination analysis should use a stronger method such as embedding-based similarity on both text and images.
  2. [§4.5, LSDPO equation] The step-wise DPO objective as written is a constant. The equation reads log[πθ(y+|Qm)/πθ(y+|Qm)] for the preferred term and log[πref(y−|Qm)/πref(y−|Qm)] for the dispreferred term; in both ratios the same distribution appears in numerator and denominator, so each log term is zero and the entire loss reduces to log σ(0) = log 2, independent of the model. The standard DPO ratio should be πθ(y+|Qm)/πref(y+|Qm) and πθ(y−|Qm)/πref(y−|Qm). This is a load-bearing error because the paper claims that the 'Step-wise DPO Pre-alignment' phase aligns the PRM's reasoning discernment capability; as written, that phase performs no learning. The equation must be corrected and the experiments rerun or clearly reported with the corrected objective.
  3. [§5.1, §4.4, Supplementary §B.4] The PRM training data are not separated from the evaluation benchmarks. In §5.1 the authors state that they use 'four multimodal and two text-only datasets for process annotation, excluding any sources currently under evaluation,' but the four multimodal datasets include MATHVISTA and WE-MATH, the evaluation benchmarks, and Supplementary §B.4 explicitly places the testmini sets of these benchmarks into the retrieval corpus used during the MCTS annotation process. The value function in §4.4 uses ground-truth answers to score rollout paths, so the automatically generated step labels encode the correct answers for problems that are near-duplicates of the test questions. The GAOKAO-MM evaluation with its temporal split is a cleaner design and partially mitigates this concern, but the main quantitative claims rest on MATHVISTA and WE-MATH, where the circularity is present. The authors should either re-run the PRM training with a corpus that strictly excludes the evaluation test sets (and any dataset containing their answers), or present the GAOKAO temporal split as the primary evidence and clearly label the MATHVISTA/WE-MATH results as preliminary.
  4. [Table 1] The paper repeatedly uses the word 'significantly' for improvements that are within the range of sampling noise, and it provides no error bars, confidence intervals, or significance tests. For example, on MATHVISTA (ALL), GPT-4o AR-MCTS is 62.6 versus Self-Consistency 61.8 and ORM 61.9, a 0.7-0.8 point difference on a 1,000-example testmini set; LLaVA-OneVision-72B shows 66.3 versus 66.0 for Self-Consistency. Several WE-MATH differences are similarly around 1 point. Without repeated runs or statistical testing, these small differences cannot support the claim of a 'significant improvement' or the stronger comparative conclusions (§5.2). The authors should report variance across multiple seeds or bootstrap confidence intervals, and temper the language accordingly.
minor comments (7)
  1. [§1, Eq. (1)] Equation (1) is presented as a theoretical model of MCTS expansion and simulation, but no derivation or justification is given for the factorization p(y|x) = max_i pθ(y|r_i,x)·pφ(r_i|x), and the variables i, r_i, and k are not defined precisely enough to make the equation meaningful. The subsequent text does not use this equation in any technical way, so it should be either derived properly or labeled as an intuitive motivation rather than a theoretical model.
  2. [§4.5, L_PFT] The point-wise fine-tuning loss uses π_SDPO(r_i) without defining what this distribution represents. If r_i is a sigmoid score produced by the PRM, the notation should be a scalar probability rather than a model distribution, and the loss should be written as a standard binary cross-entropy over the score. Please clarify.
  3. [§5.4 and Figure 4] The text refers to 'AC-MCTS' in two places in the scaling analysis; this should be 'AR-MCTS'.
  4. [§5.5, Figure 5] The diversity analysis reports cluster counts (38 vs. 46) and a visualization, but no quantitative measure of dispersion or reproducibility. The claim that AR-MCTS 'efficiently covers the problem-solving space' needs a metric such as average pairwise distance or cluster purity, ideally with multiple samples.
  5. [Table 7 and §C.3] The contamination analysis is self-contradictory: it reports a 4.2% overlap for the cross-included testmini set and then concludes that no overlap exists. At minimum, the authors should acknowledge this overlap and explain why they believe it does not affect the evaluation.
  6. [§4.1] The phrase 'filtered out any overlapping portions with our testing benchmark using regular expressions' is too vague to be checked. The authors should provide the actual regexes or a concrete description of what was removed, especially since the testmini sets are later intentionally included in the corpus in §B.4.
  7. [Throughout] There are numerous typos and inconsistent notations (e.g., 'accuarcy' in §4, 'fomulated' in the back-propagation description, 'Seld-Correction' in Table 6, 'MATHVISTA' versus 'MATHVISTA-testmini' in Table 7). A careful proofread is needed.

Circularity Check

2 steps flagged · score 6.0 of 10

Main benchmark gains are partly circular: the retrieval corpus is built from the target benchmarks' own testmini/test-split answers, so the reported improvements on MATHVISTA and WE-MATH are in part answer retrieval rather than reasoning; the GAOKAO temporal split provides independent support.

  1. fitted input called prediction [Supplementary B.1, MATHVISTA dataset description]
    "Additionally, we employ LLaV A-OneVision-70B to generate answers for thetest split, creating an in-domain corpus that can be used for retrieving answers in the testmini set."

    MATHVISTA testmini is one of the two headline evaluation sets (Table 1). The retrieval corpus for it is constructed from MATHVISTA's own test split, with pseudo-answers generated by LLaVA-OneVision-70B, and the paper states the corpus is for retrieving answers in the testmini set. A testmini query can therefore retrieve a solved near-twin from the same benchmark; the AR-MCTS gain over Self-Consistency (e.g., GPT-4o 62.6 vs 61.8; Qwen2-VL 64.1 vs 61.2) is then partly a lookup of the benchmark's own pseudo-solutions. The predicted testmini performance is built into the retrieval input by construction.

  2. fitted input called prediction [Supplementary B.4; contamination analysis in C.3/Table 7]
    "Due to the absence of an appropriate high-quality multimodal reasoning retrieval source or a training set with answer annotation, we incorporate the testmini set of WE-MATH into the knowledge base of MATHVISTA and include the testmini set of MATHVISTA into the knowledge base of WE-MATH."

    For WE-MATH evaluation, the knowledge base explicitly contains MATHVISTA testmini with answers, and vice versa for MATHVISTA. Because both are visual-math benchmarks, a query can retrieve a solved near-duplicate from the other testmini and copy its answer. The paper's own Table 7 reports 4.2% 13-gram overlap between MATHVISTA-testmini and top-50 retrieved items for WE-MATH queries, while AR-MCTS gains over Self-Consistency are of the same order (e.g., Qwen2-VL WE-MATH AVG 28.1 vs 23.6). C.3 then concludes 'there is no overlap,' which is contradicted by the intentional cross-inclusion. The benchmark prediction is thus partly a retrieval of the target test sets, not an independent reasoning measurement.

full rationale

The framework itself is not definitionally circular: AR-MCTS combines active retrieval, MCTS expansion, and a PRM, and the PRM's step-level DPO/SFT objectives are standard losses rather than restatements of the experimental results. There is no load-bearing self-citation chain or imported uniqueness theorem. The circularity is in the evaluation setup. The retrieval corpus for the two headline benchmarks is constructed from the same benchmarks' test data: MATHVISTA test-split pseudo-answers are used to retrieve answers for MATHVISTA testmini, and MATHVISTA testmini and WE-MATH testmini are cross-included in each other's knowledge bases. The paper's contamination analysis uses a 13-gram threshold and nevertheless shows a 4.2% overlap for one of these intentional inclusions, so the claim that the library has no overlap with the test sets is not supported. Reported gains (roughly 2-5 points) are the same magnitude as the measured overlap, so a material share of the improvement on MATHVISTA and WE-MATH may be answer retrieval rather than reasoning. The GAOKAO-MM evaluation uses a genuine temporal split (2010-2021 knowledge base, 2022-2023 test) and provides independent evidence that active retrieval plus PRM verification helps. For that reason the circularity is substantial but not total, corresponding to a score of 6.

Assumptions & free parameters 5 free parameters · 5 assumptions · 0 invented entities

The framework rests on hand-chosen retrieval and training hyperparameters, on the assumption that final-answer labels correctly supervise intermediate steps, and on the assumption that the retrieval corpus does not leak evaluation data. No new physical or conceptual entities are introduced.

free parameters (5)
  • Knowledge concept filtering thresholds T_r and T_kc = not specified in paper
    Used to select retrieved insights; values are hand-chosen and not reported in the main text.
  • MCTS beam size B = not specified for main results
    Controls the number of expansion paths per step; no sensitivity ablation is reported.
  • Sampling temperature during expansion = > 0.6
    Chosen empirically to increase candidate diversity, following prior work.
  • DPO beta and PRM training epochs = beta=0.3; 2 epochs DPO, 3 epochs SFT
    Hyperparameters selected without a sensitivity study.
  • Step value threshold for positive/negative DPO pairs = v > 0.8 positive, v = 0 negative
    Arbitrary cutoff for constructing preference pairs from rollout values.
assumptions (5)
  • domain assumption Correct final-answer rollout is a valid label for every intermediate reasoning step.
    Simulation in Section 4.4 sets V(s_i) to 1 if the rolled-out final answer matches ground truth, assuming step correctness is fully determined by final answer correctness.
  • domain assumption Retrieval filtering removes all evaluation-set overlap from the hybrid corpus.
    Section 4.1 and B.4 claim regex filtering and a 13-gram threshold make the corpus leak-free for MathVista and We-Math.
  • ad hoc to paper Equation (1) is an accurate model of MCTS expansion and simulation for MLLMs.
    Section 1 presents the factorization as a theoretical analysis, but it is a notational restatement and is not derived from any process model.
  • domain assumption PRM trained on text-only and multimodal annotations transfers across MLLMs and to Chinese exam questions.
    Section B.5 uses Qwen2-7B as the PRM backbone for all MLLMs, including GPT-4o, and applies an English-trained PRM to Chinese GAOKAO-MM.
  • standard math UCB selection formula applies to reasoning paths treated as a Markov decision process.
    Section 3 adopts sentence-level MCTS following prior work; standard UCB is used without new justification.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Progressive Multimodal Reasoning via Active Retrieval." pith.science (2026). https://pith.science/paper/ULHYZZW6

@misc{pith2026241214835,
  author       = {Pith},
  title        = {Pith review of: Progressive Multimodal Reasoning via Active Retrieval},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/ULHYZZW6}},
  note         = {Machine review of arXiv:2412.14835}
}
read the original abstract

Multi-step multimodal reasoning tasks pose significant challenges for multimodal large language models (MLLMs), and finding effective ways to enhance their performance in such scenarios remains an unresolved issue. In this paper, we propose AR-MCTS, a universal framework designed to progressively improve the reasoning capabilities of MLLMs through Active Retrieval (AR) and Monte Carlo Tree Search (MCTS). Our approach begins with the development of a unified retrieval module that retrieves key supporting insights for solving complex reasoning problems from a hybrid-modal retrieval corpus. To bridge the gap in automated multimodal reasoning verification, we employ the MCTS algorithm combined with an active retrieval mechanism, which enables the automatic generation of step-wise annotations. This strategy dynamically retrieves key insights for each reasoning step, moving beyond traditional beam search sampling to improve the diversity and reliability of the reasoning space. Additionally, we introduce a process reward model that aligns progressively to support the automatic verification of multimodal reasoning tasks. Experimental results across three complex multimodal reasoning benchmarks confirm the effectiveness of the AR-MCTS framework in enhancing the performance of various multimodal models. Further analysis demonstrates that AR-MCTS can optimize sampling diversity and accuracy, yielding reliable multimodal reasoning.

Figures

Figures reproduced from arXiv: 2412.14835 by the authors.

Figure 1
Figure 1. The statistics of our hybrid-modal retrieval corpus. [PITH_FULL_IMAGE:figures/full_fig_p003_1.png] view at source ↗
Figure 2
Figure 2. The pipeline of our unified multimodal retrieval module. [PITH_FULL_IMAGE:figures/full_fig_p004_2.png] view at source ↗
Figure 3
Figure 3. The overall framework of AR-MCTS: The retrieval module actively retrieves key insights at each step of the MCTS process. Then, [PITH_FULL_IMAGE:figures/full_fig_p005_3.png] view at source ↗
Figures from the paper (3 more)
Figure 5
Figure 5. Figure 5: The visualization of the cadidate reasoning paths. [PITH_FULL_IMAGE:figures/full_fig_p008_5.png]
Figure 4
Figure 4. Figure 4: Scaling analysis on inference samplings. [PITH_FULL_IMAGE:figures/full_fig_p008_4.png]
Figure 7
Figure 7. Figure 7: The composition analysis on retrieval corpus of W [PITH_FULL_IMAGE:figures/full_fig_p022_7.png]

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 6 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. BoostStep: Boosting mathematical capability of Large Language Models via improved single-step reasoning

    cs.CL 2025-01 conditional novelty 7.0 of 10

    Step-aligned in-context learning with a first-try retrieval strategy improves LLM mathematical reasoning over problem-level few-shot prompting on multiple benchmarks.

  2. Tool-Star: Empowering LLM-Brained Multi-Tool Reasoner via Reinforcement Learning

    cs.CL 2025-05 conditional novelty 6.0 of 10

    Tool-Star combines cold-start supervised fine-tuning with a multi-tool self-critic reinforcement learning algorithm and hierarchical rewards to improve LLM tool-use reasoning.

  3. EduFlow: Advancing MLLMs' Problem-Solving Proficiency through Multi-Stage, Multi-Perspective Critique

    cs.AI 2025-07 conditional novelty 5.0 of 10

    A multi-stage framework (data filtering, MCTS-guided trajectory construction, PRM-based reranking) improves Qwen MLLMs' accuracy on K-12 multimodal science benchmarks.

  4. ReARTeR: Retrieval-Augmented Reasoning with Trustworthy Process Rewarding

    cs.CL 2025-01 conditional novelty 5.0 of 10

    ReARTeR improves RAG multi-step reasoning by combining a process reward model, an aligned explanation model, temporal-difference lookahead, and MCTS-based iterative preference training.

  5. Perception, Reason, Think, and Plan: A Survey on Large Multimodal Reasoning Models

    cs.CV 2025-05 conditional novelty 4.0 of 10

    A survey organizes multimodal reasoning research into a staged roadmap and proposes native large multimodal reasoning models that unify perception, generation, and agentic planning.

  6. Empowering Multimodal LLMs with External Tools: A Comprehensive Survey

    cs.CV 2025-08 unverdicted novelty 2.0 of 10

    A survey paper maps how external tools are used to augment multimodal large language models across data, tasks, evaluation, and future directions.

Reference graph

Works this paper leans on

121 extracted references · 29 canonical work pages · cited by 6 Pith papers

  1. [1]

    Menick, Sebastian Borgeaud, Andy Brock, Aida Nematzadeh, Sa- hand Sharifzadeh, Mikolaj Binkowski, Ricardo Barreira, Oriol Vinyals, Andrew Zisserman, and Karén Simonyan

    Jean-Baptiste Alayrac, Jeff Donahue, Pauline Luc, Antoine Miech, Iain Barr, Yana Hasson, Karel Lenc, Arthur Men- sch, Katherine Millican, Malcolm Reynolds, Roman Ring, Eliza Rutherford, Serkan Cabi, Tengda Han, Zhitao Gong, Sina Samangooei, Marianne Monteiro, Jacob L. Menick, Sebastian Borgeaud, Andy Brock, Aida Nematzadeh, Sa- hand Sharifzadeh, Mikolaj B...

    2022

  2. [2]

    Think- ing fast and slow with deep learning and tree search

    Thomas Anthony, Zheng Tian, and David Barber. Think- ing fast and slow with deep learning and tree search. In Advances in Neural Information Processing Systems 30: An- nual Conference on Neural Information Processing Systems 2017, December 4-9, 2017, Long Beach, CA, USA , pages 5360–5370, 2017. 1

    2017

  3. [3]

    Lawrence Zitnick, and Devi Parikh

    Stanislaw Antol, Aishwarya Agrawal, Jiasen Lu, Margaret Mitchell, Dhruv Batra, C. Lawrence Zitnick, and Devi Parikh. VQA: visual question answering. In 2015 IEEE International Conference on Computer Vision, ICCV 2015, Santiago, Chile, December 7-13, 2015, pages 2425–2433. IEEE Computer Society, 2015. 1

    2015

  4. [4]

    Self-rag: Learning to retrieve, generate, and critique through self-reflection

    Akari Asai, Zeqiu Wu, Yizhong Wang, Avirup Sil, and Han- naneh Hajishirzi. Self-rag: Learning to retrieve, generate, and critique through self-reflection. In The Twelfth Interna- tional Conference on Learning Representations, ICLR 2024, Vienna, Austria, May 7-11, 2024. OpenReview.net, 2024. 3

    2024

  5. [5]

    Wikiextractor

    Giusepppe Attardi. Wikiextractor. https://github. com/attardi/wikiextractor, 2015. 5

    2015

  6. [6]

    Qwen-vl: A versatile vision-language model for understanding, localization, text reading, and beyond, 2023

    Jinze Bai, Shuai Bai, Shusheng Yang, Shijie Wang, Sinan Tan, Peng Wang, Junyang Lin, Chang Zhou, and Jingren Zhou. Qwen-vl: A versatile vision-language model for understanding, localization, text reading, and beyond, 2023. 1, 2

    2023

  7. [7]

    Lucas, Peter I

    Cameron Browne, Edward Jack Powley, Daniel Whitehouse, Simon M. Lucas, Peter I. Cowling, Philipp Rohlfshagen, Stephen Tavener, Diego Perez Liebana, Spyridon Samoth- rakis, and Simon Colton. A survey of monte carlo tree search methods. IEEE Trans. Comput. Intell. AI Games, 4(1):1–43,

  8. [8]

    Internlm2 technical report

    Zheng Cai, Maosong Cao, Haojiong Chen, Kai Chen, Keyu Chen, Xin Chen, Xun Chen, Zehui Chen, Zhi Chen, Pei Chu, Xiaoyi Dong, Haodong Duan, Qi Fan, Zhaoye Fei, Yang Gao, Jiaye Ge, Chenya Gu, Yuzhe Gu, Tao Gui, Aijia Guo, Qipeng Guo, Conghui He, Yingfan Hu, Ting Huang, Tao Jiang, Penglong Jiao, Zhenjiang Jin, Zhikai Lei, Jiaxing Li, Jingwen Li, Linyang Li, S...

    arXiv 2024

Show all 121 references
  1. [9]

    Alphamath almost zero: process supervision without process

    Guoxin Chen, Minpeng Liao, Chengxi Li, and Kai Fan. Alphamath almost zero: process supervision without process. CoRR, abs/2405.03553, 2024. 1, 5

    2024 arXiv

  2. [10]

    Step-level value preference optimization for mathematical reasoning

    Guoxin Chen, Minpeng Liao, Chengxi Li, and Kai Fan. Step-level value preference optimization for mathematical reasoning. CoRR, abs/2406.10858, 2024. 5

    2024 arXiv

  3. [11]

    BGE m3-embedding: Multi- lingual, multi-functionality, multi-granularity text em- beddings through self-knowledge distillation

    Jianlv Chen, Shitao Xiao, Peitian Zhang, Kun Luo, Defu Lian, and Zheng Liu. BGE m3-embedding: Multi- lingual, multi-functionality, multi-granularity text em- beddings through self-knowledge distillation. CoRR, abs/2402.03216, 2024. 8

    2024 arXiv

  4. [12]

    Wenhu Chen, Xueguang Ma, Xinyi Wang, and William W. Cohen. Program of thoughts prompting: Disentangling computation from reasoning for numerical reasoning tasks. Trans. Mach. Learn. Res., 2023, 2023. 2

    2023

  5. [13]

    Internvl: Scaling up vision foundation models and aligning for generic visual-linguistic tasks

    Zhe Chen, Jiannan Wu, Wenhai Wang, Weijie Su, Guo Chen, Sen Xing, Muyan Zhong, Qinglong Zhang, Xizhou Zhu, Lewei Lu, Bin Li, Ping Luo, Tong Lu, Yu Qiao, and Jifeng Dai. Internvl: Scaling up vision foundation models and aligning for generic visual-linguistic tasks. arXiv prepri...

    2023 arXiv

  6. [15]

    How far are we to gpt-4v? closing the gap to commercial multimodal models with open- source suites

    Zhe Chen, Weiyun Wang, Hao Tian, Shenglong Ye, Zhang- wei Gao, Erfei Cui, Wenwen Tong, Kongzhi Hu, Jiapeng Luo, Zheng Ma, Ji Ma, Jiaqi Wang, Xiaoyi Dong, Hang Yan, Hewei Guo, Conghui He, Botian Shi, Zhenjiang Jin, Chao Xu, Bin Wang, Xingjian Wei, Wei Li, Wenjian Zhang, Bo Zhan...

    2024 arXiv

  7. [16]

    CORAL: benchmarking multi-turn conversational retrieval-augmentation generation

    Yiruo Cheng, Kelong Mao, Ziliang Zhao, Guanting Dong, Hongjin Qian, Yongkang Wu, Tetsuya Sakai, Ji-Rong Wen, and Zhicheng Dou. CORAL: benchmarking multi-turn conversational retrieval-augmentation generation. CoRR, abs/2410.23090, 2024. 3

    arXiv

  8. [18]

    Training verifiers to solve math word problems

    Karl Cobbe, Vineet Kosaraju, Mohammad Bavarian, Mark Chen, Heewoo Jun, Lukasz Kaiser, Matthias Plappert, Jerry Tworek, Jacob Hilton, Reiichiro Nakano, Christopher Hesse, and John Schulman. Training verifiers to solve math word problems. CoRR, abs/2110.14168, 2021. 3

    arXiv

  9. [19]

    Flashattention-2: Faster attention with better paral- lelism and work partitioning

    Tri Dao. Flashattention-2: Faster attention with better paral- lelism and work partitioning. CoRR, 2023. 4

    2023

  10. [20]

    Bridging the kb-text gap: Lever- aging structured knowledge-aware pre-training for KBQA

    Guanting Dong, Rumei Li, Sirui Wang, Yupeng Zhang, Yun- sen Xian, and Weiran Xu. Bridging the kb-text gap: Lever- aging structured knowledge-aware pre-training for KBQA. In Proceedings of the 32nd ACM International Conference on Information and Knowledge Management, CIKM 2023,...

    2023

  11. [21]

    Self-play with execution feedback: Improving instruction-following capa- bilities of large language models

    Guanting Dong, Keming Lu, Chengpeng Li, Tingyu Xia, Bowen Yu, Chang Zhou, and Jingren Zhou. Self-play with execution feedback: Improving instruction-following capa- bilities of large language models. CoRR, abs/2406.13542,

    arXiv

  12. [22]

    Toward general instruction-following alignment for retrieval-augmented gen- eration

    Guanting Dong, Xiaoshuai Song, Yutao Zhu, Runqi Qiao, Zhicheng Dou, and Ji-Rong Wen. Toward general instruction-following alignment for retrieval-augmented gen- eration. CoRR, abs/2410.09584, 2024. 3

    2024 arXiv

  13. [23]

    How abilities in large language models are affected by supervised fine-tuning data composition

    Guanting Dong, Hongyi Yuan, Keming Lu, Chengpeng Li, Mingfeng Xue, Dayiheng Liu, Wei Wang, Zheng Yuan, Chang Zhou, and Jingren Zhou. How abilities in large language models are affected by supervised fine-tuning data composition. In Proceedings of the 62nd Annual Meeting of the...

    2024

  14. [24]

    Understand what LLM needs: Dual preference alignment for retrieval-augmented generation

    Guanting Dong, Yutao Zhu, Chenghao Zhang, Zechen Wang, Zhicheng Dou, and Ji-Rong Wen. Understand what LLM needs: Dual preference alignment for retrieval-augmented generation. CoRR, abs/2406.18676, 2024. 3

    2024 arXiv

  15. [25]

    Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Ab- hishek Kadian, Ahmad Al-Dahle, Aiesha Letman, Akhil Mathur, Alan Schelten, Amy Yang, Angela Fan, Anirudh Goyal, Anthony Hartshorn, Aobo Yang, Archi Mitra, Archie Sravankumar, Artem Korenev, Arthur Hinsvark, Arun Rao, Aston Z...

    2024 arXiv

  16. [26]

    A density-based algorithm for discovering clusters in large spatial databases with noise

    Martin Ester, Hans-Peter Kriegel, Jörg Sander, and Xiaowei Xu. A density-based algorithm for discovering clusters in large spatial databases with noise. In Proceedings of the Second International Conference on Knowledge Discovery and Data Mining (KDD-96), Portland, Oregon, USA...

    1996

  17. [27]

    Beam search strate- gies for neural machine translation

    Markus Freitag and Yaser Al-Onaizan. Beam search strate- gies for neural machine translation. In Proceedings of the First Workshop on Neural Machine Translation, NMT@ACL 2017, Vancouver, Canada, August 4, 2017 , pages 56–60. Association for Computational Linguistics, 2017. 2

    2017

  18. [28]

    PAL: program-aided language models

    Luyu Gao, Aman Madaan, Shuyan Zhou, Uri Alon, Pengfei Liu, Yiming Yang, Jamie Callan, and Graham Neubig. PAL: program-aided language models. In International Confer- ence on Machine Learning, ICML 2023, 23-29 July 2023, Honolulu, Hawaii, USA, pages 10764–10799. PMLR, 2023. 2

    2023

  19. [29]

    Interpretable contrastive monte carlo tree search reasoning

    Zitian Gao, Boye Niu, Xuzheng He, Haotian Xu, Hongzhang Liu, Aiwei Liu, Xuming Hu, and Lijie Wen. Interpretable contrastive monte carlo tree search reasoning. 2024. 1, 3

    2024

  20. [30]

    Tora: A tool-integrated reasoning agent for mathematical problem solving

    Zhibin Gou, Zhihong Shao, Yeyun Gong, Yelong Shen, Yu- jiu Yang, Minlie Huang, Nan Duan, and Weizhu Chen. Tora: A tool-integrated reasoning agent for mathematical problem solving. In The Twelfth International Conference on Learn- ing Representations, ICLR 2024, Vienna, Austria...

    2024

  21. [31]

    Deepseek-coder: When the large language model meets programming–the rise of code intelligence

    Daya Guo, Qihao Zhu, Dejian Yang, Zhenda Xie, Kai Dong, Wentao Zhang, Guanting Chen, Xiao Bi, Yu Wu, YK Li, et al. Deepseek-coder: When the large language model meets programming–the rise of code intelligence. arXiv preprint arXiv:2401.14196, 2024. 2

    2024 arXiv

  22. [32]

    Self-correction is more than refinement: A learning framework for visual and language reasoning tasks, 2024

    Jiayi He, Hehai Lin, Qingyun Wang, Yi Fung, and Heng Ji. Self-correction is more than refinement: A learning framework for visual and language reasoning tasks, 2024. 6

    2024

  23. [33]

    Measuring mathematical problem solving with the MATH dataset

    Dan Hendrycks, Collin Burns, Saurav Kadavath, Akul Arora, Steven Basart, Eric Tang, Dawn Song, and Jacob Steinhardt. Measuring mathematical problem solving with the MATH dataset. In Proceedings of the Neural Information Process- ing Systems Track on Datasets and Benchmarks 1, ...

    2021

  24. [34]

    Unsupervised dense information retrieval with con- trastive learning

    Gautier Izacard, Mathilde Caron, Lucas Hosseini, Sebas- tian Riedel, Piotr Bojanowski, Armand Joulin, and Edouard Grave. Unsupervised dense information retrieval with con- trastive learning. Trans. Mach. Learn. Res. , 2022, 2022. 4

    2022

  25. [35]

    Flashrag: A modular toolkit for ef- ficient retrieval-augmented generation research

    Jiajie Jin, Yutao Zhu, Xinyu Yang, Chenghao Zhang, and Zhicheng Dou. Flashrag: A modular toolkit for ef- ficient retrieval-augmented generation research. CoRR, abs/2405.13576, 2024. 3

    2024 arXiv

  26. [36]

    Billion- scale similarity search with gpus

    Jeff Johnson, Matthijs Douze, and Hervé Jégou. Billion- scale similarity search with gpus. IEEE Trans. Big Data, 7 (3):535–547, 2021. 4

    2021

  27. [37]

    Deep visual-semantic align- ments for generating image descriptions

    Andrej Karpathy and Li Fei-Fei. Deep visual-semantic align- ments for generating image descriptions. IEEE Trans. Pat- tern Anal. Mach. Intell., 39(4):664–676, 2017. 1

    2017

  28. [38]

    Jina CLIP: your CLIP model is also your text retriever

    Andreas Koukounas, Georgios Mastrapas, Michael Günther, Bo Wang, Scott Martens, Isabelle Mohr, Saba Sturua, Mo- hammad Kalim Akram, Joan Fontanals Martínez, Saahil Ognawala, Susana Guzman, Maximilian Werk, Nan Wang, and Han Xiao. Jina CLIP: your CLIP model is also your text re...

    2024 arXiv

  29. [39]

    Gonzalez, Hao Zhang, and Ion Stoica

    Woosuk Kwon, Zhuohan Li, Siyuan Zhuang, Ying Sheng, Lianmin Zheng, Cody Hao Yu, Joseph E. Gonzalez, Hao Zhang, and Ion Stoica. Efficient memory management for large language model serving with pagedattention. In Pro- ceedings of the ACM SIGOPS 29th Symposium on Operating Syste...

    2023

  30. [40]

    Instructerc: Reforming emotion recognition in conversation with a retrieval multi-task llms framework

    Shanglin Lei, Guanting Dong, Xiaoping Wang, Keheng Wang, and Sirui Wang. Instructerc: Reforming emotion recognition in conversation with a retrieval multi-task llms framework. CoRR, abs/2309.11911, 2023. 3

    2023 arXiv

  31. [41]

    Patrick S. H. Lewis, Ethan Perez, Aleksandra Piktus, Fabio Petroni, Vladimir Karpukhin, Naman Goyal, Heinrich Küt- tler, Mike Lewis, Wen-tau Yih, Tim Rocktäschel, Sebastian Riedel, and Douwe Kiela. Retrieval-augmented generation for knowledge-intensive NLP tasks. In Advances i...

    2020

  32. [43]

    Llava-onevision: Easy visual task transfer

    Bo Li, Yuanhan Zhang, Dong Guo, Renrui Zhang, Feng Li, Hao Zhang, Kaichen Zhang, Yanwei Li, Ziwei Liu, and Chunyuan Li. Llava-onevision: Easy visual task transfer. CoRR, abs/2408.03326, 2024. 6, 4, 5

    2024 arXiv

  33. [44]

    Dotamath: Decomposi- tion of thought with code assistance and self-correction for mathematical reasoning

    Chengpeng Li, Guanting Dong, Mingfeng Xue, Ru Peng, Xiang Wang, and Dayiheng Liu. Dotamath: Decomposi- tion of thought with code assistance and self-correction for mathematical reasoning. CoRR, abs/2407.04078, 2024. 2

    2024 arXiv

  34. [45]

    Llava-med: Training a large language- and-vision assistant for biomedicine in one day

    Chunyuan Li, Cliff Wong, Sheng Zhang, Naoto Usuyama, Haotian Liu, Jianwei Yang, Tristan Naumann, Hoifung Poon, and Jianfeng Gao. Llava-med: Training a large language- and-vision assistant for biomedicine in one day. Advances in Neural Information Processing Systems, 36, 2024. 2

    2024

  35. [46]

    Mugglemath: Assessing the impact of query and response augmentation on math reasoning

    Chengpeng Li, Zheng Yuan, Hongyi Yuan, Guanting Dong, Keming Lu, Jiancan Wu, Chuanqi Tan, Xiang Wang, and Chang Zhou. Mugglemath: Assessing the impact of query and response augmentation on math reasoning. In Proceed- ings of the 62nd Annual Meeting of the Association for Compu...

    2024

  36. [47]

    From matching to generation: A survey on generative information retrieval

    Xiaoxi Li, Jiajie Jin, Yujia Zhou, Yuyao Zhang, Peitian Zhang, Yutao Zhu, and Zhicheng Dou. From matching to generation: A survey on generative information retrieval. CoRR, abs/2404.14851, 2024. 3

    2024 arXiv

  37. [48]

    Making language mod- els better reasoners with step-aware verifier

    Yifei Li, Zeqi Lin, Shizhuo Zhang, Qiang Fu, Bei Chen, Jian-Guang Lou, and Weizhu Chen. Making language mod- els better reasoners with step-aware verifier. In Proceedings of the 61st Annual Meeting of the Association for Compu- tational Linguistics (Volume 1: Long Papers), ACL...

    2023

  38. [49]

    Let’s verify step by step

    Hunter Lightman, Vineet Kosaraju, Yuri Burda, Harrison Edwards, Bowen Baker, Teddy Lee, Jan Leike, John Schul- man, Ilya Sutskever, and Karl Cobbe. Let’s verify step by step. In The Twelfth International Conference on Learning Representations, ICLR 2024, Vienna, Austria, May 7-11,

    2024

  39. [50]

    Retrieval-augmented natural language rea- soning for explainable visual question answering

    Su Hyeon Lim, Minkuk Kim, Hyeon Bae Kim, and Seong Tae Kim. Retrieval-augmented natural language rea- soning for explainable visual question answering. CoRR, abs/2408.17006, 2024. 3

    2024 arXiv

  40. [51]

    OpenReview.net, 2024. 1, 5

    2024

  41. [52]

    Retrieval-augmented multi- modal chain-of-thoughts reasoning for large language mod- els

    Bingshuai Liu, Chenyang Lyu, Zijun Min, Zhanyu Wang, Jinsong Su, and Longyue Wang. Retrieval-augmented multi- modal chain-of-thoughts reasoning for large language mod- els. CoRR, abs/2312.01714, 2023. 2, 3, 5

    2023 arXiv

  42. [53]

    Deductive verifica- tion of chain-of-thought reasoning

    Zhan Ling, Yunhao Fang, Xuanlin Li, Zhiao Huang, Mingu Lee, Roland Memisevic, and Hao Su. Deductive verifica- tion of chain-of-thought reasoning. In Advances in Neural Information Processing Systems 36: Annual Conference on Neural Information Processing Systems 2023, NeurIPS 2...

    2023

  43. [54]

    Visual instruction tuning

    Haotian Liu, Chunyuan Li, Qingyang Wu, and Yong Jae Lee. Visual instruction tuning. In Advances in Neural Information Processing Systems 36: Annual Conference on Neural Information Processing Systems 2023, NeurIPS 2023, New Orleans, LA, USA, December 10 - 16, 2023, 2023. 1, 2

    2023

  44. [55]

    Llava-next: Im- proved reasoning, ocr, and world knowledge

    Haotian Liu, Chunyuan Li, Yuheng Li, Bo Li, Yuanhan Zhang, Sheng Shen, and Yong Jae Lee. Llava-next: Im- proved reasoning, ocr, and world knowledge. 6, 4

  45. [56]

    How much can RAG help the reasoning of llm? CoRR, abs/2410.02338, 2024

    Jingyu Liu, Jiaen Lin, and Yong Liu. How much can RAG help the reasoning of llm? CoRR, abs/2410.02338, 2024. 2, 7

    2024 arXiv

  46. [57]

    Mak- ing PPO even better: Value-guided monte-carlo tree search decoding

    Jiacheng Liu, Andrew Cohen, Ramakanth Pasunuru, Yejin Choi, Hannaneh Hajishirzi, and Asli Celikyilmaz. Mak- ing PPO even better: Value-guided monte-carlo tree search decoding. CoRR, abs/2309.15028, 2023. 1

    2023 arXiv

  47. [58]

    Mathvista: Evaluating mathemat- ical reasoning of foundation models in visual contexts

    Pan Lu, Hritik Bansal, Tony Xia, Jiacheng Liu, Chunyuan Li, Hannaneh Hajishirzi, Hao Cheng, Kai-Wei Chang, Michel Galley, and Jianfeng Gao. Mathvista: Evaluating mathemat- ical reasoning of foundation models in visual contexts. In The Twelfth International Conference on Learni...

    2024

  48. [59]

    #instag: Instruction tagging for analyzing supervised fine-tuning of large language models

    Keming Lu, Hongyi Yuan, Zheng Yuan, Runji Lin, Junyang Lin, Chuanqi Tan, Chang Zhou, and Jingren Zhou. #instag: Instruction tagging for analyzing supervised fine-tuning of large language models. In The Twelfth International Con- ference on Learning Representations, ICLR 2024, ...

    2024

  49. [60]

    Chatkbqa: A generate-then-retrieve framework for knowledge base ques- tion answering with fine-tuned large language models

    Haoran Luo, Haihong E, Zichen Tang, Shiyao Peng, Yikai Guo, Wentai Zhang, Chenghao Ma, Guanting Dong, Meina Song, Wei Lin, Yifan Zhu, and Anh Tuan Luu. Chatkbqa: A generate-then-retrieve framework for knowledge base ques- tion answering with fine-tuned large language models. I...

    2024

  50. [61]

    Wizardmath: Empowering mathematical reasoning for large language models via re- inforced evol-instruct

    Haipeng Luo, Qingfeng Sun, Can Xu, Pu Zhao, Jianguang Lou, Chongyang Tao, Xiubo Geng, Qingwei Lin, Shifeng Chen, and Dongmei Zhang. Wizardmath: Empowering mathematical reasoning for large language models via re- inforced evol-instruct. CoRR, abs/2308.09583, 2023. 1, 2

    2023 arXiv

  51. [64]

    Let’s reward step by step: Step-level reward model as the navigators for reasoning

    Qianli Ma, Haotian Zhou, Tingkai Liu, Jianbo Yuan, Pengfei Liu, Yang You, and Hongxia Yang. Let’s reward step by step: Step-level reward model as the navigators for reasoning. CoRR, abs/2310.10080, 2023. 1, 2

    2023 arXiv

  52. [65]

    Improve mathematical reason- ing in language models by automated process supervision

    Liangchen Luo, Yinxiao Liu, Rosanne Liu, Samrat Phatale, Harsh Lara, Yunxuan Li, Lei Shu, Yun Zhu, Lei Meng, Jiao Sun, and Abhinav Rastogi. Improve mathematical reason- ing in language models by automated process supervision. CoRR, abs/2406.06592, 2024. 7

    2024 arXiv

  53. [66]

    Selfcheck: Using llms to zero-shot check their own step-by-step reason- ing

    Ning Miao, Yee Whye Teh, and Tom Rainforth. Selfcheck: Using llms to zero-shot check their own step-by-step reason- ing. In The Twelfth International Conference on Learning Representations, ICLR 2024, Vienna, Austria, May 7-11,

    2024

  54. [67]

    Self-refine: Iterative refinement with self-feedback

    Aman Madaan, Niket Tandon, Prakhar Gupta, Skyler Hallinan, Luyu Gao, Sarah Wiegreffe, Uri Alon, Nouha Dziri, Shrimai Prabhumoye, Yiming Yang, Shashank Gupta, Bodhisattwa Prasad Majumder, Katherine Hermann, Sean Welleck, Amir Yazdanbakhsh, and Peter Clark. Self-refine: Iterativ...

    2023

  55. [68]

    GPT-4 technical report

    OpenAI. GPT-4 technical report. CoRR, abs/2303.08774,

    arXiv

  56. [69]

    OpenReview.net, 2024. 2

    2024

  57. [71]

    Gpt-4v (ision) system card

    R OpenAI. Gpt-4v (ision) system card. Citekey: gptvision,

  58. [72]

    Hello gpt-4o, 2024

    OpenAI. Hello gpt-4o, 2024. 4

    2024

  59. [73]

    OpenAI, :, Aaron Hurst, Adam Lerer, Adam P. Goucher, Adam Perelman, Aditya Ramesh, Aidan Clark, AJ Ostrow, Akila Welihinda, Alan Hayes, Alec Radford, Aleksander M ˛ adry, Alex Baker-Whitcomb, Alex Beutel, Alex Borzunov, Alex Carney, Alex Chow, Alex Kirillov, Alex Nichol, Alex ...

  60. [74]

    Puterman

    Martin L. Puterman. Chapter 8 markov decision processes. In Stochastic Models, pages 331–434. Elsevier, 1990. 3

    1990

  61. [76]

    Long Ouyang, Jeffrey Wu, Xu Jiang, Diogo Almeida, Car- roll L. Wainwright, Pamela Mishkin, Chong Zhang, Sand- hini Agarwal, Katarina Slama, Alex Ray, John Schulman, Jacob Hilton, Fraser Kelton, Luke Miller, Maddie Simens, Amanda Askell, Peter Welinder, Paul F. Christiano, Jan ...

    2022

  62. [77]

    Learning transferable visual models from natural language supervision

    Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, Gretchen Krueger, and Ilya Sutskever. Learning transferable visual models from natural language supervision. In Proceedings of th...

    2021

  63. [78]

    We-math: Does your large multimodal model achieve human-like mathematical reasoning? CoRR, abs/2407.01284, 2024

    Runqi Qiao, Qiuna Tan, Guanting Dong, Minhui Wu, Chong Sun, Xiaoshuai Song, Zhuoma Gongque, Shanglin Lei, Zhe Wei, Miaoxuan Zhang, Runfeng Qiao, Yifan Zhang, Xiao Zong, Yida Xu, Muxi Diao, Zhimin Bao, Chen Li, and Honggang Zhang. We-math: Does your large multimodal model achie...

    2024 arXiv

  64. [79]

    Making visual sense of oracle bones for you and me

    Runqi Qiao, Lan Yang, Kaiyue Pang, and Honggang Zhang. Making visual sense of oracle bones for you and me. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 12656–12665, 2024. 2

    2024

  65. [80]

    Alignment in multimodal interaction: An integrative frame- work

    Marlou Rasenberg, Asli Özyürek, and Mark Dingemanse. Alignment in multimodal interaction: An integrative frame- work. Cogn. Sci., 44(11), 2020. 2

    2020

  66. [82]

    Manning, Stefano Ermon, and Chelsea Finn

    Rafael Rafailov, Archit Sharma, Eric Mitchell, Christo- pher D. Manning, Stefano Ermon, and Chelsea Finn. Direct preference optimization: Your language model is secretly a reward model. In Advances in Neural Information Process- ing Systems 36: Annual Conference on Neural Info...

    2023

  67. [83]

    Preventing translation quality deterioration caused by beam search decoding in neural machine translation using statistical machine translation

    Emre Satir and Hasan Bulut. Preventing translation quality deterioration caused by beam search decoding in neural machine translation using statistical machine translation. Inf. Sci., 581:791–807, 2021. 2

    2021

  68. [84]

    Deepspeed: System optimizations enable train- ing deep learning models with over 100 billion parameters

    Jeff Rasley, Samyam Rajbhandari, Olatunji Ruwase, and Yuxiong He. Deepspeed: System optimizations enable train- ing deep learning models with over 100 billion parameters

  69. [85]

    Beyond chinchilla-optimal: Accounting for in- ference in language model scaling laws

    Nikhil Sardana, Jacob Portes, Sasha Doubov, and Jonathan Frankle. Beyond chinchilla-optimal: Accounting for in- ference in language model scaling laws. In Forty-first In- ternational Conference on Machine Learning, ICML 2024, Vienna, Austria, July 21-27, 2024. OpenReview.net, 2024. 1

    2024

  70. [86]

    Rewarding progress: Scaling automated process verifiers for llm reasoning

    Amrith Setlur, Chirag Nagpal, Adam Fisch, Xinyang Geng, Jacob Eisenstein, Rishabh Agarwal, Alekh Agarwal, Jonathan Berant, and Aviral Kumar. Rewarding progress: Scaling automated process verifiers for llm reasoning. arXiv preprint arXiv:2410.08146, 2024. 2

    arXiv

  71. [87]

    Proximal policy optimization algo- rithms

    John Schulman, Filip Wolski, Prafulla Dhariwal, Alec Rad- ford, and Oleg Klimov. Proximal policy optimization algo- rithms. CoRR, abs/1707.06347, 2017. 3

    2017 arXiv

  72. [88]

    Rewarding progress: Scaling automated process verifiers for llm reasoning

    Amrith Setlur, Chirag Nagpal, Adam Fisch, Xinyang Geng, Jacob Eisenstein, Rishabh Agarwal, Alekh Agarwal, Jonathan Berant, and Aviral Kumar. Rewarding progress: Scaling automated process verifiers for llm reasoning. 2024. 1

    2024

  73. [89]

    Scaling LLM test-time compute optimally can be more effec- tive than scaling model parameters

    Charlie Snell, Jaehoon Lee, Kelvin Xu, and Aviral Kumar. Scaling LLM test-time compute optimally can be more effec- tive than scaling model parameters. CoRR, abs/2408.03314,

    arXiv

  74. [91]

    Zhihong Shao, Peiyi Wang, Qihao Zhu, Runxin Xu, Junxiao Song, Mingchuan Zhang, Y . K. Li, Y . Wu, and Daya Guo. Deepseekmath: Pushing the limits of mathematical reason- ing in open language models. CoRR, abs/2402.03300, 2024. 1, 2

    2024 arXiv

  75. [92]

    Kakade, and Matthias W

    Niranjan Srinivas, Andreas Krause, Sham M. Kakade, and Matthias W. Seeger. Information-theoretic regret bounds for gaussian process optimization in the bandit setting. IEEE Trans. Inf. Theory, 58(5):3250–3265, 2012. 3

    2012

  76. [93]

    How to bridge the gap between modalities: A comprehensive survey on multimodal large language model

    Shezheng Song, Xiaopeng Li, and Shasha Li. How to bridge the gap between modalities: A comprehensive survey on multimodal large language model. CoRR, abs/2311.07594,

    arXiv

  77. [94]

    Cs-bench: A comprehen- sive benchmark for large language models towards computer science mastery

    Xiaoshuai Song, Muxi Diao, Guanting Dong, Zhengyang Wang, Yujia Fu, Runqi Qiao, Zhexu Wang, Dayuan Fu, Huangxuan Wu, Bin Liang, et al. Cs-bench: A comprehen- sive benchmark for large language models towards computer science mastery. arXiv preprint arXiv:2406.08587, 2024. 2

    2024 arXiv

  78. [95]

    Cheng Tan, Jingxuan Wei, Linzhuang Sun, Zhangyang Gao, Siyuan Li, Bihui Yu, Ruifeng Guo, and Stan Z. Li. Retrieval meets reasoning: Even high-school textbook knowledge ben- efits multimodal reasoning. CoRR, abs/2405.20834, 2024. 2, 3, 4

    2024 arXiv

  79. [96]

    Beats: Optimizing llm mathematical capabilities with backverify and adaptive disambiguate based efficient tree search

    Linzhuang Sun, Hao Liang, Jingxuan Wei, Bihui Yu, Con- ghui He, Zenan Zhou, and Wentao Zhang. Beats: Optimizing llm mathematical capabilities with backverify and adaptive disambiguate based efficient tree search. 2024. 1

    2024

  80. [97]

    Easy-to-hard generalization: Scalable alignment beyond human supervi- sion

    Zhiqing Sun, Longhui Yu, Yikang Shen, Weiyang Liu, Yim- ing Yang, Sean Welleck, and Chuang Gan. Easy-to-hard generalization: Scalable alignment beyond human supervi- sion. CoRR, abs/2403.09472, 2024. 5

    2024 arXiv

  81. [99]

    Htmlrag: Html is bet- ter than plain text for modeling retrieved knowledge in rag systems, 2024

    Jiejun Tan, Zhicheng Dou, Wen Wang, Mang Wang, Weipeng Chen, and Ji-Rong Wen. Htmlrag: Html is bet- ter than plain text for modeling retrieved knowledge in rag systems, 2024. 3

    2024

  82. [100]

    Qwen2.5: A party of foundation models, 2024

    Qwen Team. Qwen2.5: A party of foundation models, 2024. 2

    2024

  83. [101]

    Fran- cis Song, Noah Y

    Jonathan Uesato, Nate Kushman, Ramana Kumar, H. Fran- cis Song, Noah Y . Siegel, Lisa Wang, Antonia Creswell, Geoffrey Irving, and Irina Higgins. Solving math word prob- lems with process- and outcome-based feedback. CoRR, abs/2211.14275, 2022. 1

    2022 arXiv

  84. [102]

    Toward self-improvement of llms via imagination, searching, and criticizing

    Ye Tian, Baolin Peng, Linfeng Song, Lifeng Jin, Dian Yu, Haitao Mi, and Dong Yu. Toward self-improvement of llms via imagination, searching, and criticizing. CoRR, abs/2404.12253, 2024. 3, 6

    2024 arXiv

  85. [103]

    Solving math word problems with process- and outcome-based feedback, 2022

    Jonathan Uesato, Nate Kushman, Ramana Kumar, Francis Song, Noah Siegel, Lisa Wang, Antonia Creswell, Geoffrey Irving, and Irina Higgins. Solving math word problems with process- and outcome-based feedback, 2022. 2

    2022

  86. [105]

    Q*: Improving multi- step reasoning for llms with deliberative planning

    Chaojie Wang, Yanchen Deng, Zhiyi Lv, Zeng Liang, Jujie He, Shuicheng Yan, and Bo An. Q*: Improving multi- step reasoning for llms with deliberative planning. CoRR, abs/2406.14283, 2024. 1, 2

    2024 arXiv

  87. [106]

    Measuring multimodal math- ematical reasoning with math-vision dataset

    Ke Wang, Junting Pan, Weikang Shi, Zimu Lu, Mingjie Zhan, and Hongsheng Li. Measuring multimodal math- ematical reasoning with math-vision dataset. CoRR, abs/2402.14804, 2024. 3, 2

    2024 arXiv

  88. [107]

    Le, Ed H

    Xuezhi Wang, Jason Wei, Dale Schuurmans, Quoc V . Le, Ed H. Chi, Sharan Narang, Aakanksha Chowdhery, and Denny Zhou. Self-consistency improves chain of thought reasoning in language models. In The Eleventh International Conference on Learning Representations, ICLR 2023, Kigali...

    2023

  89. [108]

    Qwen2-vl: Enhancing vision-language model’s perception of the world at any resolution

    Peng Wang, Shuai Bai, Sinan Tan, Shijie Wang, Zhihao Fan, Jinze Bai, Keqin Chen, Xuejing Liu, Jialin Wang, Wen- bin Ge, Yang Fan, Kai Dang, Mengfei Du, Xuancheng Ren, Rui Men, Dayiheng Liu, Chang Zhou, Jingren Zhou, and Junyang Lin. Qwen2-vl: Enhancing vision-language model’s ...

    2024 arXiv

  90. [109]

    Math- shepherd: Verify and reinforce llms step-by-step without human annotations

    Peiyi Wang, Lei Li, Zhihong Shao, Runxin Xu, Damai Dai, Yifei Li, Deli Chen, Yu Wu, and Zhifang Sui. Math- shepherd: Verify and reinforce llms step-by-step without human annotations. In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Vo...

    2024

  91. [110]

    Chi, Quoc V

    Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Brian Ichter, Fei Xia, Ed H. Chi, Quoc V . Le, and Denny Zhou. Chain-of-thought prompting elicits reasoning in large language models. In Advances in Neural Informa- tion Processing Systems 35: Annual Conference on Neural ...

    2022

  92. [111]

    Towards self- improvement of llms via mcts: Leveraging stepwise knowl- edge with curriculum preference learning

    Xiyao Wang, Linfeng Song, Ye Tian, Dian Yu, Baolin Peng, Haitao Mi, Furong Huang, and Dong Yu. Towards self- improvement of llms via mcts: Leveraging stepwise knowl- edge with curriculum preference learning. 2024. 1

    2024

  93. [112]

    General ocr theory: Towards ocr-2.0 via a unified end-to-end model

    Haoran Wei, Chenglong Liu, Jinyue Chen, Jia Wang, Lingyu Kong, Yanming Xu, Zheng Ge, Liang Zhao, Jianjian Sun, Yuang Peng, et al. General ocr theory: Towards ocr-2.0 via a unified end-to-end model. arXiv preprint arXiv:2409.01704,

    arXiv

  94. [113]

    Tree of thoughts: Deliberate problem solving with large language models

    Shunyu Yao, Dian Yu, Jeffrey Zhao, Izhak Shafran, Tom Griffiths, Yuan Cao, and Karthik Narasimhan. Tree of thoughts: Deliberate problem solving with large language models. In Advances in Neural Information Processing Systems 36: Annual Conference on Neural Information Pro- ces...

    2023

  95. [114]

    Evaluating mathematical reasoning beyond accuracy

    Shijie Xia, Xuefeng Li, Yixin Liu, Tongshuang Wu, and Pengfei Liu. Evaluating mathematical reasoning beyond accuracy. CoRR, abs/2404.05692, 2024. 1

    2024 arXiv

  96. [115]

    Qwen2 technical report

    An Yang, Baosong Yang, Binyuan Hui, Bo Zheng, Bowen Yu, Chang Zhou, Chengpeng Li, Chengyuan Li, Dayiheng Liu, Fei Huang, Guanting Dong, Haoran Wei, Huan Lin, Jialong Tang, Jialin Wang, Jian Yang, Jianhong Tu, Jianwei Zhang, Jianxin Ma, Jianxin Yang, Jin Xu, Jingren Zhou, Jinze...

    2024 arXiv

  97. [116]

    Scaling relationship on learning mathematical reasoning with large language models

    Zheng Yuan, Hongyi Yuan, Chengpeng Li, Guanting Dong, Chuanqi Tan, and Chang Zhou. Scaling relationship on learning mathematical reasoning with large language models. CoRR, abs/2308.01825, 2023. 1

    2023 arXiv

  98. [117]

    Advancing LLM reasoning gener- alists with preference trees

    Lifan Yuan, Ganqu Cui, Hanbin Wang, Ning Ding, Xingyao Wang, Jia Deng, Boji Shan, Huimin Chen, Ruobing Xie, Yankai Lin, Zhenghao Liu, Bowen Zhou, Hao Peng, Zhiyuan Liu, and Maosong Sun. Advancing LLM reasoning gener- alists with preference trees. CoRR, abs/2404.02078, 2024. 1

    2024 arXiv

  99. [119]

    Chinese open instruction generalist: A preliminary release

    Ge Zhang, Yemin Shi, Ruibo Liu, Ruibin Yuan, Yizhi Li, Siwei Dong, Yu Shu, Zhaoqun Li, Zekun Wang, Chenghua Lin, Wenhao Huang, and Jie Fu. Chinese open instruction generalist: A preliminary release. CoRR, abs/2304.07987,

    arXiv

  100. [120]

    Accessing GPT-4 level mathematical olympiad solutions via monte carlo tree self-refine with llama-3 8b

    Di Zhang, Xiaoshui Huang, Dongzhan Zhou, Yuqiang Li, and Wanli Ouyang. Accessing GPT-4 level mathematical olympiad solutions via monte carlo tree self-refine with llama-3 8b. CoRR, abs/2406.07394, 2024. 1, 3

    2024 arXiv

  101. [121]

    Rest-mcts*: LLM self-training via process reward guided tree search

    Dan Zhang, Sining Zhoubian, Yisong Yue, Yuxiao Dong, and Jie Tang. Rest-mcts*: LLM self-training via process reward guided tree search. CoRR, abs/2406.03816, 2024. 1, 3, 5

    2024 arXiv

  102. [122]

    Retrieval-augmented generation for ai- generated content: A survey

    Penghao Zhao, Hailin Zhang, Qinhan Yu, Zhengren Wang, Yunteng Geng, Fangcheng Fu, Ling Yang, Wentao Zhang, and Bin Cui. Retrieval-augmented generation for ai- generated content: A survey. CoRR, abs/2402.19473, 2024. 4

    2024 arXiv

  103. [123]

    Mathverse: Does your multi-modal LLM truly see the diagrams in visual math problems? CoRR, abs/2403.14624, 2024

    Renrui Zhang, Dongzhi Jiang, Yichi Zhang, Haokun Lin, Ziyu Guo, Pengshuo Qiu, Aojun Zhou, Pan Lu, Kai-Wei Chang, Peng Gao, and Hongsheng Li. Mathverse: Does your multi-modal LLM truly see the diagrams in visual math problems? CoRR, abs/2403.14624, 2024. 3

    2024 arXiv

  104. [124]

    Multimodal chain-of-thought reasoning in language models

    Zhuosheng Zhang, Aston Zhang, Mu Li, Hai Zhao, George Karypis, and Alex Smola. Multimodal chain-of-thought reasoning in language models. Trans. Mach. Learn. Res., 2024, 2024. 2

    2024

  105. [125]

    Minigpt-4: Enhancing vision-language understanding with advanced large language models

    Deyao Zhu, Jun Chen, Xiaoqian Shen, Xiang Li, and Mo- hamed Elhoseiny. Minigpt-4: Enhancing vision-language understanding with advanced large language models. In The Twelfth International Conference on Learning Repre- sentations, ICLR 2024, Vienna, Austria, May 7-11, 2024 . Op...

    2024

  106. [126]

    VISTA: visualized text embedding for universal multi-modal retrieval

    Junjie Zhou, Zheng Liu, Shitao Xiao, Bo Zhao, and Yong- ping Xiong. VISTA: visualized text embedding for universal multi-modal retrieval. In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), ACL 2024, Bangkok, Thai...

    2024

  107. [127]

    Advanced embedding techniques in multimodal retrieval augmented generation a comprehensive study on cross modal ai applications

    Ren Zhou. Advanced embedding techniques in multimodal retrieval augmented generation a comprehensive study on cross modal ai applications. Journal of Computing and Electronic Information Management, 13(3):16–22, 2024. 3

    2024

  108. [129]

    Math-puma: Progressive upward multimodal alignment to enhance mathematical reasoning

    Wenwen Zhuang, Xin Huang, Xiantao Zhang, and Jin Zeng. Math-puma: Progressive upward multimodal alignment to enhance mathematical reasoning. CoRR, abs/2408.08640,

    arXiv

  109. [130]

    GAOKAO-MM: A chinese human- level benchmark for multimodal models evaluation

    Yi Zong and Xipeng Qiu. GAOKAO-MM: A chinese human- level benchmark for multimodal models evaluation. In Find- ings of the Association for Computational Linguistics, ACL 2024, Bangkok, Thailand and virtual meeting, August 11- 16, 2024, pages 8817–8825. Association for Computat...

    2024

  110. [131]

    Hybrid-Modal Retrieval Corpus Construction 3 4.2

    Methodology 3 4.1. Hybrid-Modal Retrieval Corpus Construction 3 4.2. Unified Multimodal Retrieval Module . . . . 4 4.3. Knowledge Concept Filtering . . . . . . . . 4 4.4. Progressive Multimodal Reasoning Annotation 4 4.5. Curriculum Process Reward Modeling . . . . 5

  111. [132]

    Experimental Setup

    Experiments 6 5.1. Experimental Setup . . . . . . . . . . . . . . 6 5.2. Overall Results . . . . . . . . . . . . . . . . 6 5.3. General Reasoning Domain Verification . . . 7 5.4. Quantitative Analysis . . . . . . . . . . . . . 7 5.5. Does AR-MCTS Improve the Sampling Space? 8

  112. [133]

    Unified Retrieval

    Conclusion 9 A . More Details about AR-MCTS 1 A.1 . The Algorithm Workflow of AR-MCTS . . . 1 B . More Details about Experimental Setup 1 B.1. Benchmarks and Datasets . . . . . . . . . . 1 B.2. Baselines and Backbone Models . . . . . . . 3 B.3. Implementation Details . . . . ....

    2010

  113. [134]

    Curriculum Process Reward Modeling

    We use a global batch size of 128, a weight decay of 0.1, and train for 3 epochs, saving checkpoints every 200 steps. Mixed precision training with bf16 is used, and the maximum context length is set to 8192 tokens. We run all our experiments on 8 NVIDIA A800 GPUs. B.4. Detail...

    2010

Pith tools

Reviewed August 11, 2026 · model on record in the stance chip above.