Pith. sign in

REVIEW 3 major objections 6 minor 1 cited by

PBSD turns sparse final-answer rewards into turn-level credit by rewriting answer evidence as a privileged action-likelihood ratio along the observed trajectory.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · grok-4.5

2026-07-12 14:36 UTC pith:QM5VVB33

load-bearing objection Clean Bayes reweighting of GRPO advantages for search agents; real gains, but the signal needs heavy filtering and the RL set is small. the 3 major comments →

arxiv 2606.09348 v2 pith:QM5VVB33 submitted 2026-06-08 cs.LG cs.CL

PBSD: Privileged Bayesian Self-Distillation for Long-Horizon Credit Assignment

Yang Tian , Rui Wang , Xumeng Wen , Junjie Li , Shizhao Sun , Lei Song , Jiang Bian , Bo Zhao This is my paper
classification cs.LG cs.CL
keywords credit assignmentreinforcement learningself-distillationsearch agentsBayesian evidenceprivileged informationlong-horizon RLpolicy optimization
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

Long-horizon search agents get only a final correct/incorrect signal, so successful runs can still contain useless steps and failed runs can still contain useful evidence gathering. This paper claims that trajectory quality can be scored by how much the verified answer becomes more likely after the trajectory is observed, then uses Bayes’ rule to convert that hard posterior-to-prior ratio into a tractable likelihood ratio between an ordinary student policy and the same model conditioned on the ground-truth answer. Decomposing that ratio turn by turn yields a Bayesian evidence score for each intermediate action, which reweights the usual trajectory advantage without adding a separate imitation loss. If the method works as argued, sparse outcome reinforcement learning gains fine-grained credit assignment that stays compatible with standard policy optimization and improves both in-domain accuracy and transfer to harder, longer-context search benchmarks.

Core claim

PBSD establishes that sparse outcome rewards can be turned into Bayes-calibrated turn-level credit by measuring, for each assistant turn, the log-likelihood gap between a privileged answer-conditioned teacher and the ordinary student on the same observed action, then using that gap only to modulate how strongly the trajectory-level advantage is inherited by that turn.

What carries the argument

Bayesian evidence score: the turn-level log-likelihood ratio si,t = log p(action | history, verified answer) − log p(action | history), obtained via Bayes’ rewrite of the posterior-to-prior answer ratio, then clipped and filtered into a multiplicative weight on the trajectory advantage.

Load-bearing premise

The method needs unique, checkable final answers and assumes that likelihood gaps under answer conditioning are reliable evidence about which turns helped or hurt, rather than calibration noise or privileged shortcuts.

What would settle it

Train the same backbone with identical rollouts and data once with PBSD and once with outcome-only GRPO; if PBSD’s turn weights do not raise accuracy on held-out long-horizon search sets (especially hard BrowseComp strata and longer-context evaluation) and ablations that remove non-replay scoring or low-SNR filtering do not collapse those gains, the central claim fails.

Watch this falsifier. Get emailed when new claim-graph text bears on it.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

3 major / 6 minor

Summary. The paper proposes PBSD, a method for turn-level credit assignment in long-horizon RLVR search agents. It measures trajectory quality via the posterior-to-prior ratio of a verified answer, rewrites that ratio by Bayes’ rule as a privileged (answer-conditioned) vs. student action-likelihood ratio, and decomposes the log-ratio into turn-level Bayesian evidence scores si,t. These scores reweight GRPO trajectory advantages through a clipped tanh modulation with low-SNR deadband filtering, without adding a separate distillation loss or exposing privileged answers at inference. On a 30B MoE search agent, PBSD improves over GRPO and several distillation baselines on in-domain validation and stratified BrowseComp, and shows stronger multi-benchmark and short-to-long context transfer than outcome-only RLVR, with ablations on non-replay scoring, δ, and filtering thresholds.

Significance. If the results hold, PBSD is a useful and relatively lightweight contribution to credit assignment for multi-turn tool-using agents: it keeps the terminal verifiable reward as the global signal, uses privileged information only as detached reweighting weights, and is compatible with standard policy optimization. The Bayes rewrite plus autoregressive turn decomposition is clean and correctly applied. Strengths include multi-benchmark evaluation under longer context than training, pass@k analysis, training-dynamics diagnostics, qualitative turn-level case studies, and systematic ablations (Table 3). The work is practically relevant for sparse-reward agentic RL where process rewards and tree search are costly.

major comments (3)
  1. §3.2 and Table 3: The central interpretation—that si,t supplies Bayes-calibrated outcome evidence—depends heavily on post-hoc SNR control. Without low-SNR filtering, validation falls from 40.87 to 34.87; with non-replay evidence scoring disabled on the R3 MoE, it collapses to 27.75 (below GRPO). The paper should more explicitly state that raw likelihood ratios are low-SNR and concentrated near zero, and that the reported gains require the deadband (~30% filtered) and MoE routing hygiene. Clarify what fraction of the lift is attributable to the Bayesian reweighting principle versus these engineering gates, and add a short diagnostic (e.g., correlation of si,t with human/process labels or with future outcome under held-out prefixes) so readers can judge whether the ratios track outcome support rather than length, style, or routing artifacts.
  2. §3.2 (Bayes rewrite and si,t) and §6: The method assumes same-base-model answer-conditioned vs. unconditioned likelihoods are sufficiently calibrated indicators of support for y*. Limitations already flag unique verified answers and model calibration, but the main claims still present the scores as principled Bayesian evidence. Please either (i) provide empirical calibration checks (e.g., whether positive si,t turns are more often on successful paths; sensitivity to answer phrasing/format), or (ii) soften the abstract/intro language from “Bayes-calibrated credit” to “likelihood-ratio reweighting under privileged conditioning,” with calibration as an empirical hypothesis supported by ablations rather than a settled property of the identity.
  3. §4.1–4.2 / Tables 1–2: RL training uses only 575 examples (200 held out for validation). Gains over GRPO are modest on the small validation set (+2.62) and BC(300) (+3.50), while the more striking pattern is that GRPO can hurt BrowseComp relative to SFT whereas PBSD helps under 256K evaluation. Report uncertainty (e.g., multiple seeds or bootstrap intervals on mean@4) for the primary comparisons, and state more carefully that cross-benchmark transfer—not only in-domain accuracy—is the main empirical support for improved long-horizon credit assignment.
minor comments (6)
  1. Abstract: “outcome-base reinforcement learning” → “outcome-based”.
  2. §3.2 vs Algorithm 1: main text uses δ, c, ε+/ε− for modulation and deadband; Algorithm 1 uses η and γ. Unify notation and state the deadband filter in the algorithm (currently only soft clip is shown).
  3. Figure 1 caption and body: “Bayesian evidencescore” / spacing and hyphenation inconsistencies (“Bayesian evidence score”) appear in several places; clean for readability.
  4. Table 2 footnote on BrowseComp-ZH re-annotation is important; ensure the main text states which numbers are on the corrected set so comparisons to prior agents remain interpretable.
  5. §3.1 GRPO write-up: the on-policy gradient approximation omits clipping and KL “for clarity”; a one-sentence note that experiments still use the clipped objective (ε_low=0.2, ε_high=0.28) would avoid confusion.
  6. Case studies (Appendix C) are helpful; consider briefly quantifying how often high-value turns appear in failed vs. successful trajectories in the main text to support the claim that failed trajectories can contain valuable steps.

Circularity Check

0 steps flagged

No significant circularity: Bayes rewrite is an identity; terminal verifier rewards remain the sole global learning signal; privileged likelihoods only detach-reweight advantages.

full rationale

The load-bearing derivation in §3.2 is the identity p(y*|x,τ)/p(y*|x)=p(τ|x,y*)/p(τ|x) followed by the autoregressive split si,t=log p(a_t|x,τ_<t,y*)−log p(a_t|x,τ_<t). That equality is Bayes’ rule applied to the same probability model; it does not define the credit signal in terms of the quantity being predicted, nor does it fit a free parameter to data and re-label the fit as a prediction. Trajectory-level advantages Ai still come from an external verifier reward (group-normalized GRPO); the Bayesian scores enter only as detached multiplicative weights wi,t=1+sign(Ai)clip(tanh(si,t/δ),−c,c) that redistribute magnitude across turns while preserving the sign of Ai. Privileged answers are unavailable at inference and are never used as imitation targets. There is no uniqueness theorem, no self-citation chain that forces the method, and no renaming of a known empirical pattern as a first-principles result. Empirical claims are evaluated on held-out external benchmarks (BrowseComp, BrowseComp-ZH, GAIA, xBench). Concerns about low-SNR scores, routing artifacts, or model calibration (Table 3, §6) are correctness/reliability risks, not circular reductions of outputs to inputs. Therefore the derivation chain is self-contained and non-circular.

Axiom & Free-Parameter Ledger

4 free parameters · 5 axioms · 1 invented entities

The central claim rests on standard probability identities plus domain assumptions of RLVR (verifiable unique answers, on-policy rollouts) and several hand-chosen modulation/filter hyperparameters that control how evidence scores affect advantages. No new physical entities are postulated; the Bayesian evidence score is a derived statistic. The ledger is therefore mostly free parameters and domain assumptions rather than invented ontology.

free parameters (4)
  • tanh scale δ
    Controls sensitivity of advantage modulation to |si,t|; set to 0.1 after ablation; other values (0.5, 1.0, 2.0) degrade results (Table 3).
  • clip threshold c
    Symmetric bound on weight deviation; set to 0.1 so wi,t ∈ [0.9, 1.1]; chosen by authors, not derived.
  • low-SNR thresholds (ε+, ε−)
    Direction-specific deadband for filtering near-zero evidence; default (0.001, 0.003) ~30% filtered; strongly affects validation accuracy in ablation.
  • GRPO group size G and clip (ε_low, ε_high)
    Shared RL hyperparameters (G=8, 0.2/0.28) that shape the base advantage being reweighted.
axioms (5)
  • standard math Bayes’ rule equates the answer posterior-to-prior ratio to the trajectory likelihood ratio under answer-conditioned vs unconditional models.
    Invoked in §3.2 to rewrite Si; mathematically standard under the usual probability model.
  • domain assumption Autoregressive turn log-likelihood ratios measure incremental evidence of each assistant turn for the verified answer.
    Decomposition Si = Σ si,t is exact for likelihoods, but interpreting si,t as causal credit for the outcome is a modeling assumption about what the likelihood gap means in agent trajectories.
  • domain assumption A unique (or sufficiently unique) verified final answer is available at train time for privileged conditioning.
    Required to form the teacher context; stated as a limitation for open-ended tasks in §6.
  • ad hoc to paper Using privileged likelihoods only as detached reweighting weights avoids harmful information leakage into the student policy.
    Claimed in §1 and §3.2; supported by dynamics (teacher–student gap shrinks without KL imitation) but not formally guaranteed.
  • ad hoc to paper For R3-routed MoE models, evidence scores must be recomputed without routing replay so gaps reflect conditioning rather than routing artifacts.
    §3.2 and ablation (a); removing non-replay scoring collapses accuracy, so the method depends on this engineering axiom for MoE.
invented entities (1)
  • Turn-level Bayesian evidence score si,t no independent evidence
    purpose: Scalar measuring how much turn ai,t increases or decreases likelihood under answer-conditioned vs student models; used to modulate advantages.
    Defined in §3.2 as log p(ai,t|x,τ<t,y*) − log p(ai,t|x,τ<t). It is a derived statistic, not an external physical entity; independent_evidence is false because its validity is judged only by downstream RL gains in this paper.

pith-pipeline@v1.1.0-grok45 · 22368 in / 3751 out tokens · 38674 ms · 2026-07-12T14:36:12.592886+00:00 · methodology

0 comments
read the original abstract

Long-horizon agentic tasks pose a fundamental credit assignment challenge for outcome-base reinforcement learning: trajectory-level rewards verify final correctness but provide limited guidance on which intermediate reasoning steps or tool interactions contribute to the outcome. The difficulty is especially pronounced in multi-turn search agents, where successful trajectories may contain misleading actions and failed trajectories may contain valuable evidence-gathering steps. We propose PBSD (Privileged Bayesian Self-Distillation), a Bayes-calibrated self-distillation method for fine-grained credit assignment under sparse final rewards. PBSD measures trajectory quality through the posterior-to-prior probability ratio of the verified answer and applies Bayes' rule to convert this hard-to-estimate answer-side ratio into a tractable likelihood ratio between a standard student model and a privileged answer-conditioned teacher model. Autoregressive decomposition of this Bayesian evidence score yields turn-level signals that identify whether each intermediate turn supports or undermines the verified outcome. Consequently, PBSD provides a principled and elegant reweighting scheme that transforms sparse outcome supervision into Bayes-calibrated turn-level credit signals, while remaining fully compatible with standard policy optimization. Experiments demonstrate that PBSD consistently enhances performance across both in-domain and out-of-domain settings, and effectively transfers knowledge from short-context training to long-context inference, suggesting that its fine-grained credit assignment mechanism facilitates more effective policy learning and yields improved generalization.

Review history (2 revisions) →

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score.

  1. Fetch-then-Explore: Decoupling Selection from Extraction over a Persistent Workspace for Search Agents

    cs.AI 2026-08 conditional novelty 6.0

    Fetch-then-Explore, which stores fetched pages in a per-question workspace and extracts evidence on demand with grep/read, beats visit-and-read and browsing baselines on BrowseComp across three LLM backbones.

Reference graph

Works this paper leans on

40 extracted references · 24 linked inside Pith · cited by 1 Pith paper

  1. [1]

    Deepseek-r1: Incentivizing reasoning capability in llms via reinforcement learning

    Daya Guo, Dejian Yang, Haowei Zhang, Junxiao Song, Peiyi Wang, Qihao Zhu, Runxin Xu, Ruoyu Zhang, Shirong Ma, Xiao Bi, et al. Deepseek-r1: Incentivizing reasoning capability in llms via reinforcement learning. arXiv preprint arXiv:2501.12948, 2025

    Pith/arXiv arXiv 2025

  2. [3]

    Zhihong Shao, Peiyi Wang, Qihao Zhu, Runxin Xu, Junxiao Song, Xiao Bi, Haowei Zhang, Mingchuan Zhang, Y. K. Li, Y. Wu, and Daya Guo. Deepseekmath: Pushing the limits of mathematical reasoning in open language models, 2024. URLhttps://arxiv.org/abs/2402.03300

    Pith/arXiv arXiv 2024

  3. [4]

    Tongyi deepresearch technical report

    Tongyi DeepResearch Team, Baixuan Li, Bo Zhang, Dingchu Zhang, Fei Huang, Guangyu Li, Guoxin Chen, Huifeng Yin, Jialong Wu, Jingren Zhou, et al. Tongyi deepresearch technical report. arXiv preprint arXiv:2510.24701, 2025

    Pith/arXiv arXiv 2025

  4. [5]

    Dr-venus: Towards frontier edge-scale deep research agents with only 10k open data

    Venus Team, Sunhao Dai, Yong Deng, Jinzhen Lin, Yusheng Song, Guoqing Wang, Xiaofeng Wu, Yuqi Zhou, Shuo Yang, Zhenzhe Ying, et al. Dr-venus: Towards frontier edge-scale deep research agents with only 10k open data. arXiv preprint arXiv:2604.19859, 2026

    Pith/arXiv arXiv 2026

  5. [6]

    Opensearch-vl: An open recipe for frontier multimodal search agents

    Shuang Chen, Kaituo Feng, Hangting Chen, Wenxuan Huang, Dasen Dai, Quanxin Shou, Yunlong Lin, Xiangyu Yue, Shenghua Gao, and Tianyu Pang. Opensearch-vl: An open recipe for frontier multimodal search agents. arXiv preprint arXiv:2605.05185, 2026

    Pith/arXiv arXiv 2026

  6. [7]

    Kimi Team, Tongtong Bai, Yifan Bai, Yiping Bao, SH Cai, Yuan Cao, Y Charles, HS Che, Cheng Chen, Guanduo Chen, et al. Kimi k2. 5: Visual agentic intelligence.arXiv preprint arXiv:2602.02276, 2026

    Pith/arXiv arXiv 2026

  7. [8]

    Mind deepresearch technical report.arXiv preprint arXiv:2604.14518, 2026

    MindDR Team and Li Auto Inc. Mind deepresearch technical report.arXiv preprint arXiv:2604.14518, 2026

    Pith/arXiv arXiv 2026

  8. [9]

    Tree search for llm agent reinforcement learning.arXiv preprint arXiv:2509.21240, 2025

    Yuxiang Ji, Ziyu Ma, Yong Wang, Guanhua Chen, Xiangxiang Chu, and Liaoni Wu. Tree search for llm agent reinforcement learning.arXiv preprint arXiv:2509.21240, 2025

    arXiv 2025

  9. [10]

    Treerpo: Tree relative policy optimization

    Zhicheng Yang, Zhijiang Guo, Yinya Huang, Xiaodan Liang, Yiwei Wang, and Jing Tang. Treerpo: Tree relative policy optimization. arXiv preprint arXiv:2506.05183, 2025. 11

    arXiv 2025

  10. [11]

    On-policy distillation

    Kevin Lu and Thinking Machines Lab. On-policy distillation. Thinking Machines Lab: Connectionism, 2025. doi: 10.64434/tml.20251026. https://thinkingmachines.ai/blog/on-policy-distillation

    doi:10.64434/tml.20251026 2025

  11. [12]

    Rethinking on-policy distillation of large language models: Phenomenology, mechanism, and recipe.arXiv preprint arXiv:2604.13016, 2026

    Yaxuan Li, Yuxin Zuo, Bingxiang He, Jinqian Zhang, Chaojun Xiao, Cheng Qian, Tianyu Yu, Huan-ang Gao, Wenkai Yang, Zhiyuan Liu, et al. Rethinking on-policy distillation of large language models: Phenomenology, mechanism, and recipe.arXiv preprint arXiv:2604.13016, 2026

    Pith/arXiv arXiv 2026

  12. [13]

    Self-distilled reasoner: On-policy self-distillation for large language models, 2026

    Siyan Zhao, Zhihui Xie, Mengchen Liu, Jing Huang, Guan Pang, Feiyu Chen, and Aditya Grover. Self-distilled reasoner: On-policy self-distillation for large language models, 2026. URLhttps://arxiv.org/abs/2601.18734

    Pith/arXiv arXiv 2026

  13. [14]

    Self-distilled rlvr.arXiv preprint arXiv:2604.03128, 2026

    Chenxu Yang, Chuanyu Qin, Qingyi Si, Minghui Chen, Naibin Gu, Dingyu Yao, Zheng Lin, Weiping Wang, Jiaqi Wang, and Nan Duan. Self-distilled rlvr.arXiv preprint arXiv:2604.03128, 2026

    Pith/arXiv arXiv 2026

  14. [15]

    Criticsearch: Fine-grained credit assignment for search agents via a retrospective critic.arXiv preprint arXiv:2511.12159, 2025

    Yaocheng Zhang, Haohuan Huang, Zijun Song, Yuanheng Zhu, Qichao Zhang, Zijie Zhao, and Dongbin Zhao. Criticsearch: Fine-grained credit assignment for search agents via a retrospective critic.arXiv preprint arXiv:2511.12159, 2025

    arXiv 2025

  15. [16]

    Rubricem: Meta-rl with rubric-guided policy decomposition beyond verifiable rewards

    Gaotang Li, Bhavana Dalvi Mishra, Zifeng Wang, Jun Yan, Yanfei Chen, Chun-Liang Li, Long T Le, Rujun Han, George Lee, Hanghang Tong, et al. Rubricem: Meta-rl with rubric-guided policy decomposition beyond verifiable rewards. arXiv preprint arXiv:2605.10899, 2026

    Pith/arXiv arXiv 2026

  16. [17]

    Reward hacking in rubric-based reinforcement learning.arXiv preprint arXiv:2605.12474, 2026

    Anas Mahmoud, MohammadHossein Rezaei, Zihao Wang, Anisha Gunjal, Bing Liu, and Yunzhong He. Reward hacking in rubric-based reinforcement learning.arXiv preprint arXiv:2605.12474, 2026

    Pith/arXiv arXiv 2026

  17. [18]

    Stepsearch: Igniting llms search ability via step-wise proximal policy optimization.arXiv preprint arXiv:2505.15107, 2025

    Ziliang Wang, Xuhui Zheng, Kang An, Cijun Ouyang, Jialu Cai, Yuhang Wang, and Yichao Wu. Stepsearch: Igniting llms search ability via step-wise proximal policy optimization.arXiv preprint arXiv:2505.15107, 2025

    Pith/arXiv arXiv 2025

  18. [19]

    Reinforcing multi-turn reasoning in llm agents via turn- level reward design.arXiv preprint arXiv:2505.11821, 2025

    Quan Wei, Siliang Zeng, Chenliang Li, William Brown, Oana Frunza, Wei Deng, Anderson Schneider, Yuriy Nevmyvaka, Yang Katie Zhao, Alfredo Garcia, et al. Reinforcing multi-turn reasoning in llm agents via turn- level reward design.arXiv preprint arXiv:2505.11821, 2025

    arXiv 2025

  19. [20]

    On-policy distillation of language models: Learning from self-generated mistakes

    Rishabh Agarwal, Nino Vieillard, Yongchao Zhou, Piotr Stanczyk, Sabela Ramos Garea, Matthieu Geist, and Olivier Bachem. On-policy distillation of language models: Learning from self-generated mistakes. In The Twelfth International Conference on Learning Representations, 2024. URLhttps://openreview.net/forum? id=3zKtaqxLhW

    2024

  20. [21]

    Revisiting on-policy distillation: Empirical failure modes and simple fixes, 2026

    Yuqian Fu, Haohuan Huang, Kaiwen Jiang, Yuanheng Zhu, and Dongbin Zhao. Revisiting on-policy distillation: Empirical failure modes and simple fixes, 2026. URLhttps://arxiv.org/abs/2603.25562

    Pith/arXiv arXiv 2026

  21. [22]

    Reinforcement learning via self-distillation, 2026

    Jonas Hübotter, Frederike Lübeck, Lejs Behric, Anton Baumann, Marco Bagatella, Daniel Marta, Ido Hakimi, Idan Shenfeld, Thomas Kleine Buening, Carlos Guestrin, and Andreas Krause. Reinforcement learning via self-distillation, 2026. URLhttps://arxiv.org/abs/2601.20802

    Pith/arXiv arXiv 2026

  22. [23]

    On-policy context distillation for language models

    Tianzhu Ye, Li Dong, Xun Wu, Shaohan Huang, and Furu Wei. On-policy context distillation for language models. arXiv preprint arXiv:2602.12275, 2026

    Pith/arXiv arXiv 2026

  23. [24]

    Self-distillation enables continual learning

    Idan Shenfeld, Mehul Damani, Jonas Hübotter, and Pulkit Agrawal. Self-distillation enables continual learning. In ICLR 2026 Workshop on Lifelong Agents: Learning, Aligning, Evolving, 2026. URLhttps://openreview. net/forum?id=HlWA3V6iKF

    2026

  24. [25]

    Privileged information distillation for language models

    Emiliano Penaloza, Dheeraj Vattikonda, Nicolas Gontier, Alexandre Lacoste, Laurent Charlin, and Massimo Caccia. Privileged information distillation for language models. InThe 1st Workshop on Scaling Post-training for LLMs, 2026. URLhttps://openreview.net/forum?id=FbJu6NEBQR

    2026

  25. [26]

    On-policy self-distillation for reasoning compression.arXiv preprint arXiv:2603.05433, 2026

    Hejian Sang, Yuanda Xu, Zhengze Zhou, Ran He, Zhipeng Wang, and Jiachen Sun. On-policy self-distillation for reasoning compression.arXiv preprint arXiv:2603.05433, 2026

    Pith/arXiv arXiv 2026

  26. [27]

    Mirothinker-1.7 & h1: Towards heavy-duty research agents via verification.arXiv preprint arXiv:2603.15726, 2026

    MiroMind Team, S Bai, L Bing, L Lei, R Li, X Li, X Lin, E Min, L Su, B Wang, et al. Mirothinker-1.7 & h1: Towards heavy-duty research agents via verification.arXiv preprint arXiv:2603.15726, 2026

    arXiv 2026

  27. [28]

    Mirothinker: Pushing the performance boundaries of open-source research agents via model, context, and interactive scaling.arXiv preprint arXiv:2511.11793, 2025

    MiroMind Team, Song Bai, Lidong Bing, Carson Chen, Guanzheng Chen, Yuntao Chen, Zhe Chen, Ziyi Chen, Jifeng Dai, Xuan Dong, et al. Mirothinker: Pushing the performance boundaries of open-source research agents via model, context, and interactive scaling.arXiv preprint arXiv:2511.11793, 2025. 12

    Pith/arXiv arXiv 2025

  28. [29]

    Search-r1: Training llms to reason and leverage search engines with reinforcement learning.arXiv preprint arXiv:2503.09516, 2025

    Bowen Jin, Hansi Zeng, Zhenrui Yue, Jinsung Yoon, Sercan Arik, Dong Wang, Hamed Zamani, and Jiawei Han. Search-r1: Training llms to reason and leverage search engines with reinforcement learning.arXiv preprint arXiv:2503.09516, 2025

    Pith/arXiv arXiv 2025

  29. [30]

    Deep- researcher: Scaling deep research via reinforcement learning in real-world environments

    Yuxiang Zheng, Dayuan Fu, Xiangkun Hu, Xiaojie Cai, Lyumanshan Ye, Pengrui Lu, and Pengfei Liu. Deep- researcher: Scaling deep research via reinforcement learning in real-world environments. InProceedings of the 2025 Conference on Empirical Methods in Natural Language Processing, pages 414–431, 2025

    2025

  30. [31]

    Stabilizing moe reinforcement learning by aligning training and inference routers.arXiv e-prints, pages arXiv–2510, 2025

    Wenhan Ma, Hailin Zhang, Liang Zhao, Yifan Song, Yudong Wang, Zhifang Sui, and Fuli Luo. Stabilizing moe reinforcement learning by aligning training and inference routers.arXiv e-prints, pages arXiv–2510, 2025

    2025

  31. [32]

    Openseeker: Democratizing frontier search agents by fully open-sourcing training data.arXiv preprint arXiv:2603.15594, 2026

    Yuwen Du, Rui Ye, Shuo Tang, Xinyu Zhu, Yijun Lu, Yuzhu Cai, and Siheng Chen. Openseeker: Democratizing frontier search agents by fully open-sourcing training data.arXiv preprint arXiv:2603.15594, 2026

    arXiv 2026

  32. [33]

    Browsecomp: A simple yet challenging benchmark for browsing agents.arXiv preprint arXiv:2504.12516, 2025

    Jason Wei, Zhiqing Sun, Spencer Papay, Scott McKinney, Jeffrey Han, Isa Fulford, Hyung Won Chung, Alex Tachard Passos, William Fedus, and Amelia Glaese. Browsecomp: A simple yet challenging benchmark for browsing agents.arXiv preprint arXiv:2504.12516, 2025

    Pith/arXiv arXiv 2025

  33. [34]

    Browsecomp-zh: Benchmarking web browsing ability of large language models in chinese

    Peilin Zhou, Bruce Leon, Xiang Ying, Can Zhang, Yifan Shao, Qichen Ye, Dading Chong, Zhiling Jin, Chenxuan Xie, Meng Cao, et al. Browsecomp-zh: Benchmarking web browsing ability of large language models in chinese. arXiv e-prints, pages arXiv–2504, 2025

    2025

  34. [35]

    Gaia: a benchmark for general ai assistants

    Grégoire Mialon, Clémentine Fourrier, Thomas Wolf, Yann LeCun, and Thomas Scialom. Gaia: a benchmark for general ai assistants. InThe TwelfthInternational Conference on Learning Representations, 2023

    2023

  35. [36]

    xbench: Tracking agents productivity scaling with profession-aligned real- world evaluations.arXiv preprint arXiv:2506.13651, 2025

    Kaiyuan Chen, Yixin Ren, Yang Liu, Xiaobo Hu, Haotong Tian, Tianbao Xie, Fangfu Liu, Haoye Zhang, Hongzhang Liu, Yuan Gong, et al. xbench: Tracking agents productivity scaling with profession-aligned real- world evaluations.arXiv preprint arXiv:2506.13651, 2025

    Pith/arXiv arXiv 2025

  36. [37]

    gpt-oss-120b & gpt-oss-20b model card

    Sandhini Agarwal, Lama Ahmad, Jason Ai, Sam Altman, Andy Applebaum, Edwin Arbus, Rahul K Arora, Yu Bai, Bowen Baker, Haiming Bao, et al. gpt-oss-120b & gpt-oss-20b model card. arXiv preprint arXiv:2508.10925, 2025

    Pith/arXiv arXiv 2025

  37. [38]

    Qwen3 technical report.arXiv preprint arXiv:2505.09388, 2025

    An Yang, Anfeng Li, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chang Gao, Chengen Huang, Chenxu Lv, et al. Qwen3 technical report.arXiv preprint arXiv:2505.09388, 2025

    Pith/arXiv arXiv 2025

  38. [39]

    Llamafactory: Unified efficient fine- tuning of 100+ language models

    Yaowei Zheng, Richong Zhang, Junhao Zhang, Yanhan Ye, and Zheyan Luo. Llamafactory: Unified efficient fine- tuning of 100+ language models. InProceedings of the 62nd annual meeting of the association for computational linguistics (volume 3: system demonstrations), pages 400–410, 2024

    2024

  39. [40]

    Megatron-lm: Training multi-billion parameter language models using model parallelism

    Mohammad Shoeybi, Mostofa Patwary, Raul Puri, Patrick LeGresley, Jared Casper, and Bryan Catanzaro. Megatron-lm: Training multi-billion parameter language models using model parallelism. arXiv preprint arXiv:1909.08053, 2019

    Pith/arXiv arXiv 1909

  40. [41]

    param1":

    Lianmin Zheng, Liangsheng Yin, Zhiqiang Xie, Chuyue Sun, Jeff Huang, Cody H Yu, Shiyi Cao, Christos Kozyrakis, Ion Stoica, Joseph E Gonzalez, et al. Sglang: Efficient execution of structured language model programs. Advancesin neural information processing systems, 37:62557–62583, 2024. 13 A Search Agent System Prompt The search agent is initialized with ...

    2024