Pith. sign in

REVIEW 5 major objections 6 minor 1 cited by

Enhancing LLMs' Reasoning-Intensive Multimedia Search Capabilities through Fine-Tuning and Reinforcement Learning

T0 review · 5 major / 6 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read Training a search LLM first by supervised fine-tuning and then by reinforcement learning on search-result quality substantially improves its performance on complex multimedia queries.

desk verdict A plausible two-stage training recipe for LLM search planning that is undermined by an AI-judged benchmark and reward loop; worth refereeing but not as-is. read the letter →

arxiv 2505.18831 v1 pith:O7IERCFD submitted 2025-05-24 cs.IR

Jinzheng Li , Sibo Ju , Yanzhou Su , Hongguang Li , Yiqing Shen This is my paper
classification cs.IR
keywords multimediasearchreasoning-intensiveplanrepresentationsupervisedfine-tuningforsearchingreinforcementlearningfromfeedbackbenchmarklargelanguagemodelsExpertBench-25
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

SearchExpert is a two-stage training recipe that turns an ordinary large language model into a search planner. The paper claims that supervised fine-tuning on compact natural-language search plans, followed by reinforcement learning that pays for the quality of retrieved results, makes LLM-driven search markedly better at complex queries that require multi-step reasoning. On the existing FinSearchBench-24 benchmark the strongest variant reaches 82.33% accuracy, ahead of 76.20% for FinSearch with ChatGPT-4o; on the paper's new SearchExpertBench-25 the same variant reaches 71.50%, versus 51.00% for FinSearch with DeepSeek-v2.5 and 32.50% for Perplexity Pro. The paper also adds a multimedia agent that converts visual input to text and generates visual output, and an automated pipeline for building reasoning-heavy search benchmarks. The core claim, read sympathetically, is that search planning is a trainable skill rather than something prompt engineering alone can elicit.

What carries the argument

The load-bearing object is the text-based DAG representation $\mathrm{Rep}(G)=\{(v_i:k_i)\}\cup\{(v_i\to v_j)\}$, in which search nodes are written as 'keyword plus source' and dependencies as arrows, so the whole plan is plain text rather than executable code. This makes the plan cheap to emit, cheap to train on, and directly executable, which is what lets SFTS treat plans as supervised labels. The second mechanism is the RLSF reward: a weighted geometric mean of an LLM-judged similarity score and an LLM-judged intrinsic-quality score, $\mathit{score}=S_{\mathrm{sim}}^{\alpha}\cdot S_{\mathrm{intrinsic}}^{1-\alpha}$, transformed by $\log(\mathit{score}/(1-\mathit{score}))$; that reward is what shifts training from imitating reference plans to optimizing the outcome of the search.

What would settle it

Take a random sample of about 50 SearchExpertBench-25 questions, have independent human experts verify each correct answer and reasoning chain against the cited sources, then score the top systems blind on that human-verified subset; if SearchExpert no longer leads, the LLM-judged reward and benchmark construction are carrying the result. A second check is to train the same RLSF pipeline with human-written reward labels instead of frozen-LLM scores and see whether the policy's advantage survives.

Watch

Extended reading notes

Core claim

The central discovery is that the difficulty of reasoning-intensive search lies more in the plan than in the engine, and that a plan can be treated as a learnable structured text. SearchExpert rewrites the search plan as a natural-language directed acyclic graph (DAG)—each node is 'keyword plus search source,' each edge is a directional arrow—instead of Python code, cutting token consumption by roughly 42–53 percent. Supervised fine-tuning for searching (SFTS) teaches a base LLM to emit such plans using an automatically built dataset of recent, genuinely search-dependent questions. Reinforcement learning from search feedback (RLSF) executes the planned searches, has a frozen LLM score each response for similarity to a reference answer and for intrinsic quality, and feeds the log-odds of that score through PPO. Experiments show that either stage alone helps, that both stages together give the largest gains, and that the edge widens on questions deliberately designed to require causal inference and multi-hop retrieval.

Load-bearing premise

The load-bearing premise is that the frozen LLM's similarity and quality scores are honest measures of good search results; if those scores reward text that merely looks and reads like an LLM answer rather than information that is actually useful, both the reinforcement signal and the benchmark's ground truth would be biased in the same direction.

Editorial extensions

If this is right

  • Larger base models scale the benefit: after the same two-stage training, accuracy on SearchExpertBench-25 rises from 64.00% with a 7B model to 71.50% with a 32B model, indicating the method transfers across model families.
  • Ablations show SFTS contributes most of the token reduction and RLSF most of the reasoning gain, so the two stages are complementary rather than redundant.
  • Replacing code-based DAG plans with natural-language text cuts token consumption by roughly 42–53% while remaining executable, which makes reasoning-intensive search materially cheaper to run.
  • The automated construction pipeline can generate reasoning-heavy multiple-choice benchmarks without manual annotation, as instantiated in the 200-question SearchExpertBench-25.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Editorial: A testable extension is to relabel a sample of RLSF training queries with human reward judgments; if the human-reward policy does not also improve on human-rated answer quality, the reported RL gains are partly an artifact of the LLM judge.
  • Editorial: The 'plan as text, reward on execution outcome' recipe should transfer to other tool-using agents beyond search, such as code generation or database agents, where execution traces provide natural rewards.
  • Editorial: The benchmark pipeline could be pushed further by having the deep-research report withhold its final conclusion, forcing models to synthesize across raw sources rather than match the prose style of an LLM-generated report.
  • Editorial: The multimedia components are integrated rather than jointly trained; a stronger version would let the planner decide what visual evidence to request, making image captioning part of the search plan rather than a preprocessing step.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

5 major / 6 minor

Summary. The paper introduces SearchExpert, a two-stage training method (SFTS and RLSF) intended to improve LLM-driven multimedia search planning. SFTS fine-tunes an LLM to emit a token-efficient natural-language DAG search plan, using an automated pipeline that crawls recent online text and generates query-plan pairs. RLSF then applies PPO with a reward computed by a frozen LLM (ChatGPT-4o) that scores the executed search response for semantic similarity to a reference answer and for intrinsic quality. The method also includes a multimedia agent that uses BLIP-2 for visual input and DALLE-3 for visual output. The authors construct a new benchmark, SearchExpertBench-25, consisting of 200 multiple-choice questions generated by an LLM pipeline from OpenAI deep-research reports, and report accuracy gains over FinSearch, MindSearch, and Perplexity Pro on both FinSearchBench-24 and SearchExpertBench-25, with human evaluation scores favoring SearchExpert.

Significance. If the reported results hold, SearchExpert would be a practical contribution: it shows that open-weight LLMs can be specialized for complex search planning with modest compute, and it demonstrates token savings from replacing Python-based DAG search plans with natural-language representations. The automated data-construction pipeline and the proposed human-evaluation framework are also useful ingredients for the search-agent community. That said, the significance is currently tempered by the fact that the new benchmark's ground truth and the RLSF reward signal are both generated by the same LLM family (ChatGPT-4o), with no reported human agreement study or independent fact-checking. Without such validation, the headline gains may partly measure agreement with the judge's answer-generation biases rather than genuine search quality. The paper's central claim is therefore defensible only after the evaluation is independently anchored.

major comments (5)
  1. [§2.5, Eq. (10)] SearchExpertBench-25's ground truth is produced entirely by an LLM pipeline: OpenAI deep research generates reports and a frozen LLM generates the questions, correct answers, and chain-of-thought reasoning. The paper reports no human agreement study, inter-annotator statistic, or independent fact-check of the 200 gold answers. At the same time, the RLSF reward (§2.3.2, Eqs. (6)-(9)) uses ChatGPT-4o to judge similarity and intrinsic quality. Because training and evaluation share the same model family, the reported 71.50% accuracy on SearchExpertBench-25 may reflect alignment with the judge's answer-generation preferences rather than improved search planning. Please add a human-validated subset, report agreement metrics, or otherwise demonstrate that the LLM-generated labels are factually correct.
  2. [Abstract and §3.2, Table 2] The two headline relative improvements use inconsistent conventions. On FinSearchBench-24, 36.60% is computed as (82.33 − 60.27)/60.27, i.e., relative to Perplexity Pro's score. On SearchExpertBench-25, 54.54% is computed as (71.50 − 32.50)/71.50, i.e., relative to SearchExpert's own score. These are not comparable, and the abstract gives the impression of a single consistent reporting scheme. Please standardize the improvement calculation (e.g., always relative to the baseline or always relative to the proposed method) and recompute the claims accordingly.
  3. [Table 2 and Table 3] The same configuration, SearchExpert with Qwen2.5-32B on SearchExpertBench-25, is reported with time 18.72 s/answer in Table 2 and 23.22 s/answer in Table 3. This discrepancy undermines the efficiency and token-consumption comparisons, since the two tables are used to support different claims about the same system. Please clarify the measurement protocol and provide a single consistent number, or explain why the times differ.
  4. [§2.2.1 and §2.3.2, Eqs. (2), (3), (8)] The training pipeline depends on thresholds τ (novelty), δ (alignment), and α (reward weight), but none of these is specified in the paper or ablated in the experiments. Without these values, the SFTS and RLSF procedures are not reproducible, and the sensitivity of the results to these hyperparameters is unknown. Please report the chosen values, the range considered, and any stability checks.
  5. [§2.6 and Figure 4] The human evaluation results are only presented as a single radar-style figure with aggregate scores; no information is given about the number of raters, their agreement, the exact prompts/rubrics used, or per-criterion numeric scores. Since the paper uses human evaluation to support the claim of 'superior readability' and 'analytical completeness,' please include the full evaluation protocol and numeric results, including inter-rater reliability if available.
minor comments (6)
  1. [§2.3] The citation '[16?]' appears with a question mark inside the brackets and is incomplete; please fix the reference.
  2. [Figure 1] Figure 1 contains garbled HTML-like text in the crawler and node descriptions (e.g., 'B.Searcher' and stray markup), which makes the figure difficult to interpret; please clean the figure text.
  3. [Table 2] The token usage columns are labeled 'Tokens (bit/answer)' but the values are clearly token counts, not bits; relabel to 'tokens/answer'.
  4. [§3.2] The sentence 'outperforming FinSearch with DeepSeek-v2.5 (51.00%) by 20.50%' refers to a 20.50 percentage-point gap, not a 20.50% relative improvement; please use the correct terminology to avoid confusion.
  5. [Figure 3 caption] The caption describes the earthquake as 'hypothetical,' while the main text and the benchmark example treat it as a real event; clarify whether the case study is based on a real or hypothetical scenario.
  6. [References] Reference [27] cites a paper about Bengali mathematical problem solving as the Qwen2.5-32B source; please cite the official Qwen2.5 technical report instead.

Circularity Check

2 steps flagged · score 6.0 of 10

SearchExpertBench-25 gold answers and the RLSF reward are both produced by the same frozen LLM family (ChatGPT-4o/OpenAI deep research), so the headline gains partly measure agreement with the LLM judge rather than independent search quality.

  1. self definitional [Section 2.5, Eq. (10)]
    "Finally, we generate question-answer pairs from the report with LLM by emphasizing the “What,” “Why,” and “How” dimensions ... (Q,A,CoT)=LLM frozen(SynthesizedInfo,ReasoningPrompt)(10)"

    SearchExpertBench-25's correctness is defined as the output of a frozen LLM over OpenAI deep research reports, with no human agreement study or independent fact-check reported. Accuracy on this benchmark therefore measures agreement with the LLM's own generated answers. This is load-bearing because the paper's headline 71.50% accuracy and 54.54% relative-improvement claim are made on this self-defined benchmark.

  2. other [Section 2.3.2, Eqs. (6)-(9); Section 3.1]
    "S_sim = LLM frozen(x_response, x_answer), (6) ... score = S_sim^alpha * S_intrinsic^(1-alpha), (8) ... reward = log(score(x_response,x_answer)/(1-score(x_response,x_answer))). (9) ... we used ChatGPT-4o as an external reward model through API calls."

    RLSF's reward is an LLM-judged similarity to an LLM-generated reference plus an LLM-judged intrinsic score. The same ChatGPT-4o family is used to generate SearchExpertBench-25 labels (Section 3.1 and Eq. 10). PPO therefore optimizes the model to produce responses that the ChatGPT-4o judge scores as similar to LLM-generated text, and the benchmark evaluates the same agreement with that judge. The claimed reasoning improvement is partially forced by construction.

full rationale

The paper contains a genuine, partially circular validation loop for its new benchmark and its RL reward. SearchExpertBench-25 is constructed entirely by an LLM pipeline (Eq. 10), and the RLSF reward is computed by ChatGPT-4o (Eqs. 6-9), the same model family used to generate the benchmark's questions, answers, and reasoning chains. No human agreement study is reported. Consequently, the headline 71.50% accuracy on SearchExpertBench-25 is not an independent measure of search-planning quality; it reflects agreement between the trained model and the LLM judge/label generator. The FinSearchBench-24 result (82.33% vs. FinSearch's 76.20%) and the human evaluation provide some external evidence, so the circularity is partial rather than total. The abstract's 54.54% and 36.60% relative-improvement numbers also use inconsistent bases (FinSearchBench-24 gain over Perplexity Pro, SearchExpertBench-25 gain relative to SearchExpert's own score), which weakens but does not by itself constitute circularity. No load-bearing self-citation chain was found: FinSearch and FinSphere are cited as baselines/frameworks, not as uniqueness theorems.

Assumptions & free parameters 3 free parameters · 5 assumptions · 0 invented entities

The central claim depends on several unreported thresholds and on AI-generated data and answers. tau, delta, and alpha are free parameters that control data filtering and reward weighting, but no values are given. The domain assumptions that LLM judgments are valid proxies for answer quality and that the generated benchmark keys are correct are load-bearing.

free parameters (3)
  • tau (novelty threshold)
    Filters training queries by semantic similarity between the LLM's answer and the reference answer (Eq. 2); the value is not given in the paper.
  • delta (alignment threshold)
    Determines whether a FinSearch response is aligned before converting its plan (Eq. 3); value not reported.
  • alpha (reward weight)
    Balances semantic similarity and intrinsic quality in the RLSF reward (Eq. 8); value not reported.
assumptions (5)
  • domain assumption LLM-based semantic similarity and intrinsic quality scores are reliable proxies for search result quality.
    Invoked in Eqs. 6-9 for the RLSF reward; no validation against human judgments is provided.
  • domain assumption The automated data construction pipeline produces correct query-answer pairs and aligned responses.
    Section 2.2.1 relies on frozen-LLM generation and the isNovel/isAligned filters, with thresholds tau and delta unreported.
  • domain assumption The benchmark generation pipeline produces correct answer keys for SearchExpertBench-25.
    Section 2.5 and Eq. 10 use LLM-generated Q&A and chains of thought; no human verification is reported.
  • domain assumption FinSearch execution provides an unbiased retrieval backend across all compared systems.
    Section 3.1 states SearchExpert uses FinSearch to execute plans; if FinSearch's retrieval is not neutral, comparisons to FinSearch as a baseline are confounded.
  • domain assumption The natural-language DAG representation preserves the logical structure of the original Python DAG.
    Eq. 1 maps nodes and edges to text; the paper does not measure information loss.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Enhancing LLMs' Reasoning-Intensive Multimedia Search Capabilities through Fine-Tuning and Reinforcement Learning." pith.science (2026). https://pith.science/paper/O7IERCFD

@misc{pith2026250518831,
  author       = {Pith},
  title        = {Pith review of: Enhancing LLMs' Reasoning-Intensive Multimedia Search Capabilities through Fine-Tuning and Reinforcement Learning},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/O7IERCFD}},
  note         = {Machine review of arXiv:2505.18831}
}
read the original abstract

Existing large language models (LLMs) driven search agents typically rely on prompt engineering to decouple the user queries into search plans, limiting their effectiveness in complex scenarios requiring reasoning. Furthermore, they suffer from excessive token consumption due to Python-based search plan representations and inadequate integration of multimedia elements for both input processing and response generation. To address these challenges, we introduce SearchExpert, a training method for LLMs to improve their multimedia search capabilities in response to complex search queries. Firstly, we reformulate the search plan in an efficient natural language representation to reduce token consumption. Then, we propose the supervised fine-tuning for searching (SFTS) to fine-tune LLM to adapt to these representations, together with an automated dataset construction pipeline. Secondly, to improve reasoning-intensive search capabilities, we propose the reinforcement learning from search feedback (RLSF) that takes the search results planned by LLM as the reward signals. Thirdly, we propose a multimedia understanding and generation agent that enables the fine-tuned LLM to process visual input and produce visual output during inference. Finally, we establish an automated benchmark construction pipeline and a human evaluation framework. Our resulting benchmark, SearchExpertBench-25, comprises 200 multiple-choice questions spanning financial and international news scenarios that require reasoning in searching. Experiments demonstrate that SearchExpert outperforms the commercial LLM search method (Perplexity Pro) by 36.60% on the existing FinSearchBench-24 benchmark and 54.54% on our proposed SearchExpertBench-25. Human evaluations further confirm the superior readability.

Figures

Figures reproduced from arXiv: 2505.18831 by the authors.

Figure 1
Figure 1. The two-stage training framework of SearchExpert. (a) Supervised fine-tuning for searching (SFTS): Our automated [PITH_FULL_IMAGE:figures/full_fig_p003_1.png] view at source ↗
Figure 2
Figure 2. Benchmark construction method and example case from SearchExpertBench-25. The left panel illustrates our five-step [PITH_FULL_IMAGE:figures/full_fig_p004_2.png] view at source ↗
Figure 3
Figure 3. Comparison of LLM-driven search methods analyzing a hypothetical April 2025 earthquake in Japan’s manufacturing [PITH_FULL_IMAGE:figures/full_fig_p006_3.png] view at source ↗
Figures from the paper (1 more)
Figure 4
Figure 4. Figure 4: Human evaluation results comparing SearchExpert [PITH_FULL_IMAGE:figures/full_fig_p006_4.png]

Discussion (0). Sign in to comment.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. Reinforcement Fine-Tuning for Reasoning towards Multi-Step Multi-Source Search in Large Language Models

    cs.IR 2025-06 conditional novelty 6.0 of 10

    R-Search trains one LLM to reason, plan a multi-source search graph, and synthesize answers in a single pass, beating several search-augmented baselines.

Reference graph

Works this paper leans on

39 extracted references · 7 canonical work pages · cited by 1 Pith paper

  1. [1]

    Davide Caffagni, Sara Sarto, Marcella Cornia, Lorenzo Baraldi, and Rita Cucchiara

    work page

  2. [2]

    Xuemeng Cai and Lingxiao Jiang. 2025. Adapting Knowledge Prompt Tuning for Enhanced Automated Program Repair. arXiv:arXiv:2504.01523

    work page Pith review arXiv 2025

  3. [3]

    Nico Catalano, Stefano Samele, Paolo Pertino, and Matteo Matteucci. 2025. MARS: a Multimodal Alignment and Ranking System for Few-Shot Segmen- tation. arXiv:arXiv:2504.07942

    work page Pith review arXiv 2025

  4. [4]

    Zehui Chen, Kuikun Liu, Qiuchen Wang, Jiangning Liu, Wenwei Zhang, Kai Chen, and Feng Zhao. 2024. MindSearch: Mimicking Human Minds Elicits Deep AI Searcher. arXiv:arXiv:2407.20183

    arXiv 2024

  5. [5]

    Brown, Miljan Martic, Shane Legg, and Dario Amodei

    Paul Christiano, Jan Leike, Tom B. Brown, Miljan Martic, Shane Legg, and Dario Amodei. 2017. Deep reinforcement learning from human preferences. arXiv:arXiv:1706.03741

    arXiv 2017

  6. [6]

    Neural Spacetimes for DAG Representation Learning

    Haitz Sáez de Ocáriz Borde, Anastasis Kratsios, Marc T. Law, Xiaowen Dong, and Michael Bronstein. 2024. Neural Spacetimes for DAG Representation Learning. arXiv:arXiv:2408.13885

    work page Pith review arXiv 2024

  7. [7]

    DeepSeek-AI, Aixin Liu, Bei Feng, et al. 2024. DeepSeek-V2: A Strong, Economical, and Efficient Mixture-of-Experts Language Model. arXiv:arXiv:2405.04434

    arXiv 2024

  8. [8]

    Shengyuan Ding, Shenxi Wu, Xiangyu Zhao, Yuhang Zang, Haodong Duan, Xiaoyi Dong, Pan Zhang, Yuhang Cao, Dahua Lin, and Jiaqi Wang. 2025. MM- IFEngine: Towards Multimodal Instruction Following. arXiv:arXiv:2504.07957

    arXiv 2025

Show all 39 references
  1. [9]

    Yao Fu, Hao Peng, Litu Ou, Ashish Sabharwal, and Tushar Khot. 2023. Specializing Smaller Language Models towards Multi-Step Reasoning. arXiv:arXiv:2301.12726

    2023 arXiv

  2. [10]

    Minghe Gao, Xuqi Liu, Zhongqi Yue, Yang Wu, Shuang Chen, Juncheng Li, Siliang Tang, Fei Wu, Tat-Seng Chua, and Yueting Zhuang. 2025. Benchmarking Multi- modal CoT Reward Model Stepwise by Visual Program. arXiv:arXiv:2504.06606

    2025 arXiv

  3. [11]

    Yang Gao, Hong Yang, Yizhi Chen, Junxian Wu, Peng Zhang, and Haishuai Wang

  4. [12]

    Beliz Gunel, Jingfei Du, Alexis Conneau, and Ves Stoyanov. 2020. Su- pervised Contrastive Learning for Pre-trained Language Model Fine-tuning. arXiv:arXiv:2011.01403

    2020 arXiv

  5. [13]

    arXiv:arXiv:2502.10459

    LLM4GNAS: A Large Language Model Based Toolkit for Graph Neural Architecture Search. arXiv:arXiv:2502.10459

    arXiv

  6. [14]

    Dongzhi Jiang, Renrui Zhang, Ziyu Guo, et al. 2024. MMSearch: Benchmarking the Potential of Large Models as Multi-modal Search Engines. arXiv:arXiv:2409.12959

    2024 arXiv

  7. [15]

    Shijie Han, Changhai Zhou, Yiqing Shen, Tianning Sun, Yuhua Zhou, Xiaoxia Wang, Zhixiao Yang, Jingshu Zhang, and Hongguang Li. 2025. FinSphere: A Conversational Stock Analysis Agent Equipped with Quantitative Tools based on Real-Time Database. arXiv:arXiv:2501.12399

    2025 arXiv

  8. [16]

    Bowen Jin, Hansi Zeng, Zhenrui Yue, Jinsung Yoon, Sercan Arik, Dong Wang, Hamed Zamani, and Jiawei Han. 2025. Search-R1: Training LLMs to Reason and Leverage Search Engines with Reinforcement Learning. arXiv:arXiv:2503.09516

    2025 arXiv

  9. [17]

    Pengcheng Jiang, Lang Cao, Ruike Zhu, Minhao Jiang, Yunyi Zhang, Jimeng Sun, and Jiawei Han. 2025. RAS: Retrieval-And-Structuring for Knowledge-Intensive LLM Generation. arXiv:arXiv:2502.10996

    2025

  10. [18]

    Harrison Lee, Samrat Phatale, Hassan Mansoor, Thomas Mesnard, Johan Fer- ret, Kellie Lu, Colton Bishop, Ethan Hall, Victor Carbune, Abhinav Rastogi, and Sushant Prakash. 2023. RLAIF vs. RLHF: Scaling Reinforcement Learning from Human Feedback with AI Feedback. Proceedings of t...

    2023 arXiv

  11. [19]

    Mingyu Jin, Qinkai Yu, Dong Shu, Haiyan Zhao, Wenyue Hua, Yanda Meng, Yongfeng Zhang, and Mengnan Du. 2024. The Impact of Reasoning Step Length on Large Language Models. arXiv:arXiv:2401.04925

    2024 arXiv

  12. [20]

    Guohao Li, Hasan Abed Al Kader Hammoud, Hani Itani, Dmitrii Khizbullin, and Bernard Ghanem. 2023. CAMEL: Communicative Agents for "Mind" Explo- ration of Large Language Model Society. InThirty-seventh Conference on Neural Information Processing Systems

    2023

  13. [21]

    Yang Lei, Jiangtong Li, Dawei Cheng, Zhijun Ding, and Changjun Jiang. 2023. CF- Benchmark: Chinese Financial Assistant Benchmark for Large Language Model. arXiv:arXiv:2311.05812

    2023 arXiv

  14. [22]

    Jinzheng Li, Jingshu Zhang, Hongguang Li, and Yiqing Shen. 2024. An Agent Framework for Real-Time Financial Information Searching with Large Language Models. arXiv:arXiv:2502.15684

    2024 arXiv

  15. [23]

    Junnan Li, Dongxu Li, Silvio Savarese, and Steven Hoi. 2023. BLIP-2: Boot- strapping Language-Image Pre-training with Frozen Image Encoders and Large Language Models. arXiv:arXiv:2301.12597

    2023 arXiv

  16. [24]

    Guerreiro, Ricardo Rei, and André F

    José Pombal, Nuno M. Guerreiro, Ricardo Rei, and André F. T. Martins. 2025. Zero-shot Benchmarking: A Framework for Flexible and Scalable Automatic Evaluation of Language Models. arXiv:arXiv:2504.01001

    2025

  17. [25]

    William Peebles and Saining Xie. 2022. Scalable Diffusion Models with Trans- formers. arXiv:arXiv:2212.09748

    2022 arXiv

  18. [26]

    Rothschild, Daniel G

    Sofia Eleni Spatharioti, David M. Rothschild, Daniel G. Goldstein, and Jake M. Hofman. 2023. Comparing Traditional and LLM-based Search for Consumer Choice: A Randomized Experiment. arXiv:arXiv:2307.03744

    2023 arXiv

  19. [27]

    Chen Qian, Wei Liu, Hongzhang Liu, Nuo Chen, Yufan Dang, Jiahao Li, Cheng Yang, Weize Chen, Yusheng Su, Xin Cong, Juyuan Xu, Dahai Li, Zhiyuan Liu, and Maosong Sun. 2023. ChatDev: Communicative Agents for Software Development. arXiv preprint arXiv:2307.07924(2023). https://arx...

    2023 arXiv

  20. [28]

    Gemini Team, Rohan Anil, Sebastian Borgeaud, et al. 2023. Gemini: A Family of Highly Capable Multimodal Models. arXiv:arXiv:2312.11805

    2023 arXiv

  21. [29]

    Saad Tahmid and Sourav Sarker. 2024. Qwen2.5-32B: Leveraging Self-Consistent Tool-Integrated Reasoning for Bengali Mathematical Olympiad Problem Solving. arXiv:arXiv:2411.05934

    2024 arXiv

  22. [30]

    Yu Xia, Jingru Fan, Weize Chen, Siyu Yan, Xin Cong, Zhong Zhang, Yaxi Lu, Yankai Lin, Zhiyuan Liu, and Maosong Sun. 2025. AgentRM: Enhancing Agent Generalization with Reward Modeling. arXiv:arXiv:2502.18407

    2025 arXiv

  23. [31]

    Hugo Touvron, Thibaut Lavril, Gautier Izacard, et al. 2023. LLaMA: Open and Efficient Foundation Language Models. arXiv:arXiv:2302.13971

    2023 arXiv

  24. [32]

    An Yang, Baosong Yang, Binyuan Hui, et al . 2024. Qwen2 Technical Report. arXiv:arXiv:2407.10671

    2024 arXiv

  25. [33]

    Haoyi Xiong, Jiang Bian, Yuchen Li, Xuhong Li, Mengnan Du, Shuaiqiang Wang, Dawei Yin, and Sumi Helal. 2024. When Search Engine Services meet Large Language Models: Visions and Challenges. arXiv:arXiv:2407.00128

    2024 arXiv

  26. [34]

    Jifan Yu, Xiaozhi Wang, Shangqing Tu, et al. 2023. KoLA: Carefully Benchmarking World Knowledge of Large Language Models. arXiv:arXiv:2306.09296

    2023 arXiv

  27. [35]

    Wong, Shu Yang, Junchao Wu, and Lidia S

    Xinyi Yang, Runzhe Zhan, Derek F. Wong, Shu Yang, Junchao Wu, and Lidia S. Chao. 2025. Rethinking Prompt-based Debiasing in Large Language Models. arXiv:arXiv:2503.09219

    2025 arXiv

  28. [36]

    Yang Zhao, Li Du, Xiao Ding, Kai Xiong, Ting Liu, and Bing Qin. 2024. Super- vised Fine-Tuning Achieve Rapid Task Adaption Via Alternating Attention Head Activation Patterns. arXiv:arXiv:2409.15820

    2024 arXiv

  29. [37]

    Ziqiang Yuan, Kaiyuan Wang, Shoutai Zhu, Ye Yuan, Jingya Zhou, Yanlin Zhu, and Wenqi Wei. 2024. FinLLMs: A Framework for Financial Reasoning Dataset Generation with Large Language Models. arXiv:arXiv:2401.10744

    2024 arXiv

  30. [39]

    Yaowei Zheng, Richong Zhang, Junhao Zhang, Yanhan Ye, Zheyan Luo, Zhangchi Feng, and Yongqiang Ma. 2024. LlamaFactory: Unified Efficient Fine-Tuning of 100+ Language Models. arXiv:arXiv:2403.13372

    2024 arXiv

  31. [2025]

    Recurrence-Enhanced Vision-and-Language Transformers for Robust Multimodal Document Retrieval. arXiv:arXiv:2503.01980 Enhancing LLMs’ Reasoning-Intensive Multimedia Search Capabilities through Fine-Tuning and Reinforcement Learning Conference’17, July 2017, Washington, DC, USA

    2017 arXiv

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.