Pith. sign in

REVIEW 3 major objections 5 minor 69 references

CoEvoKG: Co-Evolving Knowledge Graphs with Self-Evolving Search Agents

T0 review · 3 major / 5 minor · reviewed 2026-08-04 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read Closed-loop knowledge graph memory lifts search-agent self-play by about 10 macro points on six QA benchmarks.

desk verdict Genuinely new closed-loop KG self-play, but the write-back term that closes the loop is the weakest-supported part (+0.6 points) and the path-support scorer may be self-confirming; referee it, but ask for calibration and variance. read the letter →

arxiv 2608.01904 v1 pith:UHHOMB4Z submitted 2026-08-03 cs.AI

Zhaoyang Li , Zenghuang Fu , Qiuyuan Ai , Ping Jiang , Haoyu Wu , Minghui Wu , Chenxu Zhao , Jie Song
show 1 more author
Guannan He
This is my paper
classification cs.AI
keywords knowledgegraphself-playsearchagentsreinforcementlearningmulti-hopQAevidencememoryLLMtaskgeneration
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

CoEvoKG claims that a search agent trained by self-play improves most when the knowledge graph is kept inside the training loop: it generates multihop questions from entity chains and receives back verified evidence from successful searches, which enriches the graph that generates future tasks and computes rewards. On six QA benchmarks across three backbones, this closed loop raises macro-average accuracy by about 10–12 points over base models and by 2.6–3.7 points over matched-budget self-play and RL baselines. The central idea is that successful search trajectories should not be discarded; they should be written back into the evidence store that drives the next round of training. If correct, self-play for search agents is best modeled as two coupled processes—task distribution and evidence memory—not as RL against a fixed task pool.

What carries the argument

The central mechanism is the evolving chain pool (graph memory): entity chains sampled from a Wikipedia-based knowledge graph, each node an article with its passage and each edge a typed relation label. Three components operate on it in a loop: (1) chain-based task generation, where a proposer turns sampled 2–3 hop chains into questions passing a quality gate; (2) a hybrid solver reward R_s = R_ans(1 + β S_path), where S_path is the geometric mean of a deterministic relation consistency score M(u,v) over adjacent entities in the searched path; and (3) verified evidence write-back, which appends deduplicated retrieved passages from correct, path-supported trajectories back into the originatin

What would settle it

Compute M(u,v) on held-out human-annotated entity-pair support labels; if the score does not track human judgments, the process reward is misaligned and CoEvoKG's reported gains could in part come from training to a heuristic rather than to evidence-supported reasoning.

Watch

Extended reading notes

Core claim

The paper's central claim is that a knowledge graph can serve simultaneously as a source of verifiable multihop training tasks and as a persistent evidence memory for the agent's own successful searches, and that closing the loop between these two roles is what makes self-play stable and effective. In each round a proposer generates questions from sampled entity chains, a solver answers them with multi-turn search, and any correct trajectory whose entity path is supported by graph evidence is verified, deduplicated, and written back into the graph memory. Later rounds sample from enriched chains and score path support against the enriched graph, so the task distribution and the reward signal

Load-bearing premise

The load-bearing premise is that the deterministic relation consistency score M(u,v) is a calibrated proxy for evidence-supported search; the paper states that any calibrated scorer could replace it, but calibration is asserted rather than demonstrated, and the thresholds and weights are deferred to an appendix.

Editorial extensions

If this is right

  • Under matched training budgets, CoEvoKG beats static-data agentic RL (Search-R1) and self-play (SSP) by 2.6–3.7 macro average points, and its validation accuracy keeps rising while SSP degrades after mid-training.
  • The process reward separates evidence-supported correct answers from lucky guesses, so training rewards search behavior rather than answer generation alone.
  • The verified chain pool grows then plateaus as deduplication kicks in, indicating the loop naturally saturates without collapsing onto seed questions.
  • The difficulty-annealed proposer keeps generated questions at the solver's competence frontier, with comparable success rates to seed questions but higher average hop count (2.23 vs ≈1.75), explaining larger gains on multihop benchmarks.
  • The seed-fallback rate declines from ≈0.40 to ≈0.25, showing the proposer increasingly produces admissible questions as training proceeds.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The paper's own ablation isolates write-back as a small (+0.6 point) contribution; extrapolating, the framework's gains may be mostly from chain-grounded task generation and path/difficulty rewards, meaning the memory loop's main role is sustaining a hard task distribution.
  • Because 25–40% of solver slots are human seed questions and this is disclosed but not ablated against SSP, part of the reported gains could come from the fixed seed pool rather than the co-evolution loop; a controlled ablation would settle this.
  • The risk of a self-confirming loop is real: if the graph is written back with the solver's own successful paths, S_path thresholds may become easier to satisfy as training progresses, so the process reward measures internal consistency rather than external truth; a test would be to freeze the graph after some round and continue training to see if gains persist.
  • The framework's design is portable to other structured evidence sources (e.g., citation graphs, molecular interaction networks, legal or biomedical databases) wherever entity chains and verifiable relations exist.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. CoEvoKG couples a task generator and a search agent through a shared knowledge-graph chain pool. The generator samples multihop entity chains and produces questions filtered by a quality gate; the solver is trained with GRPO on a reward that combines answer correctness with a path-support score computed against the current graph; proposer training uses a difficulty-bell reward on solver success rate. Correct, path-supported solver trajectories are written back into the graph, advancing G_t to G_{t+1}, which is intended to provide a persistent evidence memory and richer future tasks. On six QA benchmarks (NQ, TriviaQA, PopQA, HotpotQA, 2WikiMultiHopQA, Bamboogle) and three backbones, the paper reports macro-average gains of +11.2, +10.1, and +11.6 over base models, and +2.6 to +3.7 over Search-R1 and SSP under a matched budget. A build-up ablation attributes +1.5 to KG task generation (p=0.0011), +1.1 to path/difficulty rewards (p=0.0141), and +0.6 to evidence write-back (p=0.0314).

Significance. If the co-evolution loop is real, this is a useful and practical contribution: it shows that self-play search agents can be stabilized by treating the knowledge graph not only as a task source but also as a persistent, evidence-accumulating memory. The paper has several genuine strengths: evaluation on external, human-curated held-out subsets (the SSP release) with greedy decoding; a transparent build-up ablation that reports paired sign tests and openly states that write-back is 'positive but weaker'; a matched training budget across methods; and released code. These design choices make the central claim externally anchored rather than definitionally circular. The main risk is that the verification signal used to gate and reward write-back is computed from the same graph memory that write-back mutates, so the loop's only direct contribution may be partially self-confirming. That concern, together with the unablated seed-fallback mixture, means the manuscript needs additional controls before the co-evolution claim is fully established.

major comments (3)
  1. [§3.3–3.4, Eqs. (7)–(9)] The path-support verification is not independent of the memory it gates. M(u,v) is computed deterministically from records in the current graph G_t, and write-back advances G_t to G_{t+1} using trajectories that passed Spath≥τ_path. Once a correct trajectory adds evidence for an entity pair, later trajectories over the same pair receive a higher M, making the threshold and the Eq. (7) process reward easier to satisfy. This is a self-confirmation loop rather than an external check. The paper concedes in §3.3 that 'any calibrated scorer could replace it' but provides no calibration evidence. This matters because Table 2 attributes only +0.6 macro points (p=0.0314) to write-back, exactly the term most exposed to the bias. Please add an external/oracle verification ablation or calibrate M on a fixed held-out set of trajectory–graph pairs that is frozen before write-back begins.
  2. [§3.2, Fig. 3, Table 2] The seed fallback injects human-written verified questions into 25–40% of solver slots, declining over training. The comparison against SSP in Table 1 therefore conflates KG-chain task generation with a human-question curriculum; no ablation controls for seed fraction. Because C1 (KG task generation) is the largest ablation gain (+1.5, p=0.0011), the reader cannot tell how much of the superiority over SSP comes from KG chains versus the presence of 25–40% verified human questions. Please report an ablation that matches the seed mixture across methods (e.g., SSP augmented with the same seed pool, or CoEvoKG without seed fallback) and report the seed fraction per method per round.
  3. [Table 1, §4.1] The headline matched-budget gains (+2.6 to +3.7 macro points over SSP/Search-R1) are reported without variance, number of seeds, or per-benchmark significance. A single seed and a single checkpoint selection can be noisy; paired sign tests are given only for the Table 2 ablation, not the main results. Please provide at least three seeds (or bootstrapped confidence intervals over the 2,625 held-out examples) for the main table and state the checkpoint-selection variance. This is especially important for Bamboogle, which has only 125 evaluation questions.
minor comments (5)
  1. [§3.5, Eq. (12)] The symbol G_t is used both for the graph memory at round t and for the discounted return in the REINFORCE++ estimator. Rename the return variable (e.g., R_t or G_return_t) to avoid notation collision.
  2. [Eqs. (8)–(9)] Clarify whether the extracted chain êτ always has the same length L as the original chain c. If the solver path length varies, define the relevant length in Eq. (9) explicitly.
  3. [§3.2–3.3, Appendix] The exact values of α, β_text, γ, η, τ_path, the quality-gate retention threshold, the verifier model, and the annealing schedule (p*_s, p*_e, σ) are all deferred to the appendix. Please include these in the supplement or the main text, since they are needed to reproduce the path-support score.
  4. [Fig. 3] The caption does not specify the vertical axes for the pool-size and seed-fallback-rate panels. Add axis labels and units so the 'grows then plateaus' and 'declines' claims can be read quantitatively.
  5. [§4.4] The comparison of mean solver success rates (0.405 vs. 0.410) should state how many rollouts and which checkpoint produced these numbers. A one-line description would make the diagnostic reproducible.

Circularity Check

1 steps flagged · score 4.0 of 10

Path-support gate and write-back are mutually recursive; verified graph memory partially certifies its own outputs, though headline accuracy is externally anchored.

  1. self definitional [Sec. 3.3 'Path support verification' (Eqs. 7–9) and Sec. 3.4 'Verified Graph Memory']
    "We verify ˆcτ against the current evidence graph Gt ... M(u, v) ∈ [0,1] deterministically ... from the current chain evidence: over records mentioning both entities ... by default, evidence write back is applied to trajectories that are both correct and supported by the path score (Rans=1 and Spath ≥ τ path) ... Enriched chains return to the pool, forming a denser graph Gt+1, so later rounds let the proposer condition on richer contexts and the verifier draw on more evidence."

    The write-back gate defines 'supported evidence' via S_path, computed from M(·,·) over records in the current graph G_t (Eq. 9). Section 3.4 then writes exactly those supported trajectories back into G_{t+1}. Hence the evidence used to verify later trajectories is partly the solver's own earlier verified outputs: G_{t+1} = G_t ∪ {τ : R_ans(τ)=1 ∧ S_path(τ; G_t) ≥ τ_path}. The support score is therefore not an independent check; it becomes easier to satisfy as past successes add records for the same entity pairs, so the process reward and the write-back gate are partially self-certifying. The C3 ablation (+0.6 macro points, p=0.0314) is precisely the term most exposed to this self-confirmation. The external accuracy numbers are not circular because final evaluation uses held-out benchmarks,

full rationale

CoEvoKG's headline result is not definitionally circular: macro-average gains are measured on external, human-curated SSP held-out subsets (500 questions per benchmark, 125 for Bamboogle), and the comparison against Search-R1 and SSP is under a matched budget. No fitted parameter is renamed as a prediction; no uniqueness theorem or ansatz is imported from co-author citations. The only overlapping self-citation (Cognitive Scaffold, Ai et al. 2026) is used for related-work contrast, not as load-bearing support. The main circularity concern is the verification loop inside the method: S_path (Eq. 9) is computed deterministically from records in the current graph memory G_t, and Section 3.4 admits trajectories with S_path ≥ τ_path into G_{t+1}. This makes 'verified' evidence self-referential — the graph used to judge support is enriched by the trajectories it judged supported. The paper's own ablation shows write-back is the weakest contribution (+0.6 macro points, p=0.0314), which is consistent with partial self-confirmation rather than independent signal. Seed fallback (≈25–40% of solver slots) is a real confound for the task-generation rows but is disclosed and not a circularity. M's weights/thresholds are deferred to the appendix and 'any calibrated scorer could replace it' is an acknowledged modularity, not a circular step. Overall, central claim retains independent content; one component is partially circular, so score 4.

Assumptions & free parameters 6 free parameters · 7 assumptions · 0 invented entities

The central claim rests on training hyperparameters and domain assumptions rather than derived constants. No new physical or conceptual entities are introduced; the graph memory is a training component whose contribution is measured by ablation (+0.6 macro points). The listed free parameters are hand-set or appendix-deferred, and none is fitted to the evaluation benchmarks, but the paper reports no sensitivity analysis, so the robustness of the headline gains to these settings is unknown.

free parameters (6)
  • beta (solver path-support reward weight) = 0.2
    Eq. 7; scales the process reward term; fixed across all runs.
  • relation score weights alpha, beta_text, gamma, eta = appendix
    Eq. 8 for M(u,v): ordering, mutual textual mention, co-occurrence, bridge decay; values and candidate sets are deferred to the appendix.
  • path support thresholds (local and global tau_path) = appendix
    Sections 3.3-3.4; the global threshold gates both process credit and write-back admission.
  • difficulty bell parameters sigma, p_start, p_end and annealing schedule = appendix
    Eq. 10; target solver success rate annealed over the 314-step budget; same schedule for all backbones.
  • quality gate retention threshold and dimension aggregation = appendix
    Eq. 6; LLM verifier prompt, model, and threshold are appendix-only.
  • solver group size G = 8
    Rollouts per question used for GRPO advantages and proposer difficulty estimation.
assumptions (7)
  • domain assumption Random walks over the KILT Wikipedia knowledge graph produce chains that support unique-answer, genuinely multihop questions.
    Section 3.2: the verifiable task supply assumes chain length and structure imply task difficulty and answerability.
  • domain assumption The E5 dense retriever over Wikipedia returns, within the 8-turn budget, passages sufficient to answer generated questions.
    Sections 3.3 and 4.1: reward computation and final evaluation both rely on retrieval coverage.
  • ad hoc to paper The deterministic relation consistency score M(u,v), with hand-set weights, is a calibrated proxy for trajectory path support.
    Section 3.3: paper states any calibrated scorer could replace it, so calibration is assumed rather than demonstrated; weights are appendix-only.
  • domain assumption The LLM quality gate (Eq. 6) correctly enforces no leakage, chain faithfulness, multihop requirement, single-answer focus, and clarity.
    Section 3.2: a gate failure admits leaky or trivial tasks into the solver training distribution.
  • ad hoc to paper A single annealed difficulty schedule p*(t) is appropriate for all three backbones.
    Section 3.3: same schedule for all backbones; values not present in the main text.
  • standard math GRPO/REINFORCE++ style updates with KL penalty and advantage whitening keep the two-policy self-play loop stable.
    Section 3.5: convergence and non-collapse of the joint proposer-solver objective are assumed.
  • domain assumption The SSP evaluation subsets (500 questions per benchmark) are unbiased external targets and the seed pool excludes them.
    Section 4.1: evaluation fairness rests on this split guarantee.

how reviews work

0 comments
Cite this review

Pith. "Pith review of CoEvoKG: Co-Evolving Knowledge Graphs with Self-Evolving Search Agents." pith.science (2026). https://pith.science/paper/UHHOMB4Z

@misc{pith2026260801904,
  author       = {Pith},
  title        = {Pith review of: CoEvoKG: Co-Evolving Knowledge Graphs with Self-Evolving Search Agents},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/UHHOMB4Z}},
  note         = {Machine review of arXiv:2608.01904}
}
read the original abstract

Large language models can improve with reinforcement learning for search agents, yet existing self play agents repeatedly generate tasks while discarding the knowledge gained during successful searches. We introduce CoEvoKG, a framework that turns a knowledge graph into both a source of verifiable training tasks and a persistent evidence memory for agent evolution. CoEvoKG jointly trains a task generator and a search agent: the generator creates multihop questions from entity chains sampled from the knowledge graph, while the agent learns from rewards for answer correctness and search trajectories whose entity paths are supported by graph evidence. When a search succeeds, CoEvoKG verifies and deduplicates the retrieved evidence, then writes it back to the corresponding graph nodes and edges. Future rounds reuse this enriched graph for task generation and reward computation, closing the loop between model self evolution and knowledge accumulation. Experiments on six QA benchmarks (NQ, TriviaQA, PopQA, HotpotQA, 2WikiMultiHopQA, and Bamboogle) with three backbone models show that CoEvoKG improves macro average accuracy over the corresponding base models by +11.2, +10.1, and +11.6 points on Qwen2.5-3B-Instruct, Qwen2.5-7B-Instruct, and Llama-3.1-8B-Instruct, respectively. Under matched training budgets, CoEvoKG further improves over competitive self play baselines and RL baselines for search agents by +2.6 to +3.7 macro average points across the three backbones. Code is available at https://github.com/lazzy1225/CoEvoKG.

Figures

Figures reproduced from arXiv: 2608.01904 by the authors.

Figure 1
Figure 1. CoEvoKG training loop. Multihop KG chains pro [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. CoEvoKG training loop. The proposer samples multihop entity chains from the current graph memory and generates [PITH_FULL_IMAGE:figures/full_fig_p003_2.png] view at source ↗
Figure 3
Figure 3. Training dynamics on Qwen2.5-7B-Instruct (raw in light, running mean in bold). [PITH_FULL_IMAGE:figures/full_fig_p007_3.png] view at source ↗
Figures from the paper (1 more)
Figure 4
Figure 4. Figure 4: Generated (CoEvoKG) vs. seed questions on [PITH_FULL_IMAGE:figures/full_fig_p007_4.png]

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

69 extracted references · 10 linked inside Pith

  1. [1]

    2025 , month = sep, doi =

    Guo, Daya and others , journal =. 2025 , month = sep, doi =

    2025

  2. [2]

    2024 , eprint =

    DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Models , author =. 2024 , eprint =

    2024

  3. [3]

    International Conference on Learning Representations , year =

    Decoupled Weight Decay Regularization , author =. International Conference on Learning Representations , year =

  4. [4]

    and Wang, Dong and Zamani, Hamed and Han, Jiawei , booktitle =

    Jin, Bowen and Zeng, Hansi and Yue, Zhenrui and Yoon, Jinsung and Arik, Sercan O. and Wang, Dong and Zamani, Hamed and Han, Jiawei , booktitle =. Search-. 2025 , url =

    2025

  5. [5]

    and Zhang, Wen and Chen, Huajun and Yang, Fan and Zhou, Zenan and Chen, Weipeng , booktitle =

    Chen, Mingyang and Sun, Linzhuang and Li, Tianpeng and Sun, Haoze and Zhou, Yijie and Zhu, Chenzheng and Wang, Haofen and Pan, Jeff Z. and Zhang, Wen and Chen, Huajun and Yang, Fan and Zhou, Zenan and Chen, Weipeng , booktitle =. 2025 , url =

    2025

  6. [6]

    Lu, Hongliang and Wen, Yuhang and Cheng, Pengyu and Ding, Ruijin and Guo, Jiaqi and Xu, Haotian and Wang, Chutian and Chen, Haonan and Jiang, Xiaoxi and Jiang, Guanjun , booktitle =. Search. 2026 , url =

    2026

  7. [7]

    2025 , eprint=

    DeepDive: Advancing Deep Search Agents with Knowledge Graphs and Multi-Turn RL , author=. 2025 , eprint=

    2025

  8. [8]

    2025 , eprint =

    DeepResearcher: Scaling Deep Research via Reinforcement Learning in Real-world Environments , author =. 2025 , eprint =

    2025

Show all 69 references
  1. [9]

    The Landscape of Agentic Reinforcement Learning for

    Zhang, Guibin and Geng, Hejia and Yu, Xiaohang and Yin, Zhenfei and Zhang, Zaibin and Tan, Zelin and Zhou, Heng and Li, Zhong-Zhi and Xue, Xiangyuan and Li, Yijiang and Zhou, Yifan and Chen, Yang and Zhang, Chen and Fan, Yutao and Wang, Zihu and Huang, Songtao and Velez, Franc...

    2026

  2. [10]

    Beyond Pass@1: Self-Play with Variational Problem Synthesis Sustains

    Liang, Xiao and Li, Zhong-Zhi and Gong, Yeyun and Shen, Yelong and Wu, Ying Nian and Guo, Zhijiang and Chen, Weizhu , booktitle =. Beyond Pass@1: Self-Play with Variational Problem Synthesis Sustains. 2026 , url =

    2026

  3. [11]

    Retrieval-Augmented Generation for Knowledge-Intensive

    Lewis, Patrick and Perez, Ethan and Piktus, Aleksandra and Petroni, Fabio and Karpukhin, Vladimir and Goyal, Naman and K. Retrieval-Augmented Generation for Knowledge-Intensive. Advances in Neural Information Processing Systems , volume =

  4. [12]

    Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT) , pages =

    Petroni, Fabio and Piktus, Aleksandra and Fan, Angela and Lewis, Patrick and Yazdani, Majid and De Cao, Nicola and Thorne, James and Jernite, Yacine and Karpukhin, Vladimir and Maillard, Jean and Plachouras, Vassilis and Rockt. Proceedings of the 2021 Conference of the North A...

    2021

  5. [13]

    Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing (EMNLP) , pages =

    HotpotQA: A Dataset for Diverse, Explainable Multi-hop Question Answering , author =. Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing (EMNLP) , pages =

    2018

  6. [14]

    Proceedings of the 28th International Conference on Computational Linguistics (COLING) , pages =

    Constructing A Multi-hop QA Dataset for Comprehensive Evaluation of Reasoning Steps , author =. Proceedings of the 28th International Conference on Computational Linguistics (COLING) , pages =

  7. [15]

    International Conference on Learning Representations (ICLR) , year =

    Intrinsic Motivation and Automatic Curricula via Asymmetric Self-Play , author =. International Conference on Learning Representations (ICLR) , year =

  8. [16]

    International Conference on Machine Learning (ICML) , year =

    Self-Play Fine-Tuning Converts Weak Language Models to Strong Language Models , author =. International Conference on Machine Learning (ICML) , year =

  9. [17]

    Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (ACL) , pages =

    Self-Instruct: Aligning Language Models with Self-Generated Instructions , author =. Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (ACL) , pages =

  10. [18]

    Absolute Zero: Reinforced

    Zhao, Andrew and Wu, Yiran and Wu, Tong and Xu, Quentin and Yue, Yang and Lin, Matthieu and Wang, Shenzhi and Wu, Qingyun and Zheng, Zilong and Huang, Gao , booktitle =. Absolute Zero: Reinforced. 2025 , url =

    2025

  11. [19]

    2024 , eprint =

    From Local to Global: A Graph RAG Approach to Query-Focused Summarization , author =. 2024 , eprint =

    2024

  12. [20]

    International Conference on Learning Representations (ICLR) , year =

    Think-on-Graph: Deep and Responsible Reasoning of Large Language Model on Knowledge Graph , author =. International Conference on Learning Representations (ICLR) , year =

  13. [21]

    2025 , eprint =

    EPERM: An Evidence Path Enhanced Reasoning Model for Knowledge Graph Question and Answering , author =. 2025 , eprint =

    2025

  14. [22]

    Advances in Neural Information Processing Systems , volume =

    Reflexion: Language Agents with Verbal Reinforcement Learning , author =. Advances in Neural Information Processing Systems , volume =. 2023 , url =

    2023

  15. [23]

    International Conference on Learning Representations (ICLR) , year =

    Self-RAG: Learning to Retrieve, Generate, and Critique through Self-Reflection , author =. International Conference on Learning Representations (ICLR) , year =

  16. [24]

    2022 , eprint =

    Text Embeddings by Weakly-Supervised Contrastive Pre-training , author =. 2022 , eprint =

    2022

  17. [27]

    Do Not Let Low-Probability Tokens Over-Dominate in

    Yang, Zhihe and Luo, Xufang and Wang, Zilong and Han, Dongqi and He, Zhiyuan and Li, Dongsheng and Xu, Yunjian , booktitle =. Do Not Let Low-Probability Tokens Over-Dominate in. 2026 , url =

    2026

  18. [28]

    Transactions of the Association for Computational Linguistics (TACL) , volume =

    Natural Questions: A Benchmark for Question Answering Research , author =. Transactions of the Association for Computational Linguistics (TACL) , volume =

  19. [29]

    Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (ACL) , pages =

    TriviaQA: A Large Scale Distantly Supervised Challenge Dataset for Reading Comprehension , author =. Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (ACL) , pages =

  20. [30]

    Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (ACL) , pages =

    When Not to Trust Language Models: Investigating Effectiveness of Parametric and Non-Parametric Memories , author =. Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (ACL) , pages =

  21. [31]

    Findings of the Association for Computational Linguistics: EMNLP , pages =

    Measuring and Narrowing the Compositionality Gap in Language Models , author =. Findings of the Association for Computational Linguistics: EMNLP , pages =

  22. [32]

    2024 , eprint =

    The Llama 3 Herd of Models , author =. 2024 , eprint =

    2024

  23. [33]

    2025 , doi =

    Sheng, Guangming and Zhang, Chi and Ye, Zilingfeng and Wu, Xibin and Zhang, Wang and Zhang, Ru and Peng, Yanghua and Lin, Haibin and Wu, Chuan , booktitle =. 2025 , doi =

    2025

  24. [34]

    and Barrett, Clark and Sheng, Ying , booktitle =

    Zheng, Lianmin and Yin, Liangsheng and Xie, Zhiqiang and Sun, Chuyue and Huang, Jeff and Yu, Cody Hao and Cao, Shiyi and Kozyrakis, Christos and Stoica, Ion and Gonzalez, Joseph E. and Barrett, Clark and Sheng, Ying , booktitle =

  25. [35]

    Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (ACL) , pages =

    Cognitive Scaffold: From Fluid Context to Crystallized Memory for Long-Horizon DeepResearch Agents , author =. Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (ACL) , pages =

  26. [36]

    Ai, Q.; Fu, Z.; Li, Z.; Jiang, P.; Wu, H.; Song, J.; and He, G. 2026. Cognitive Scaffold: From Fluid Context to Crystallized Memory for Long-Horizon DeepResearch Agents. In Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (ACL), 25526--25542

    2026

  27. [37]

    Asai, A.; Wu, Z.; Wang, Y.; Sil, A.; and Hajishirzi, H. 2024. Self-RAG: Learning to Retrieve, Generate, and Critique through Self-Reflection. In International Conference on Learning Representations (ICLR)

    2024

  28. [38]

    Z.; Zhang, W.; Chen, H.; Yang, F.; Zhou, Z.; and Chen, W

    Chen, M.; Sun, L.; Li, T.; Sun, H.; Zhou, Y.; Zhu, C.; Wang, H.; Pan, J. Z.; Zhang, W.; Chen, H.; Yang, F.; Zhou, Z.; and Chen, W. 2025. ReSearch : Learning to Reason with Search for LLM s via Reinforcement Learning. In Advances in Neural Information Processing Systems, volume 38

    2025

  29. [39]

    Chen, Z.; Deng, Y.; Yuan, H.; Ji, K.; and Gu, Q. 2024. Self-Play Fine-Tuning Converts Weak Language Models to Strong Language Models. In International Conference on Machine Learning (ICML)

    2024

  30. [40]

    O.; and Larson, J

    Edge, D.; Trinh, H.; Cheng, N.; Bradley, J.; Chao, A.; Mody, A.; Truitt, S.; Metropolitansky, D.; Ness, R. O.; and Larson, J. 2024. From Local to Global: A Graph RAG Approach to Query-Focused Summarization. arXiv:2404.16130

    2024 arXiv

  31. [41]

    Grattafiori, A.; Dubey, A.; Jauhri, A.; et al. 2024. The Llama 3 Herd of Models. arXiv:2407.21783

    2024 arXiv

  32. [42]

    Guo, D.; et al. 2025. DeepSeek-R1 incentivizes reasoning in LLMs through reinforcement learning. Nature, 645(8081): 633--638

    2025

  33. [43]

    Ho, X.; Duong Nguyen, A.-K.; Sugawara, S.; and Aizawa, A. 2020. Constructing A Multi-hop QA Dataset for Comprehensive Evaluation of Reasoning Steps. In Proceedings of the 28th International Conference on Computational Linguistics (COLING), 6609--6625

    2020

  34. [44]

    K.; Xu, H.; and Shen, W

    Hu, J.; Liu, J. K.; Xu, H.; and Shen, W. 2025. REINFORCE++ : Stabilizing Critic-Free Policy Optimization with Global Advantage Normalization. arXiv:2501.03262v9

    2025 arXiv

  35. [45]

    O.; Wang, D.; Zamani, H.; and Han, J

    Jin, B.; Zeng, H.; Yue, Z.; Yoon, J.; Arik, S. O.; Wang, D.; Zamani, H.; and Han, J. 2025. Search- R1 : Training LLM s to Reason and Leverage Search Engines with Reinforcement Learning. In Second Conference on Language Modeling

    2025

  36. [46]

    S.; and Zettlemoyer, L

    Joshi, M.; Choi, E.; Weld, D. S.; and Zettlemoyer, L. 2017. TriviaQA: A Large Scale Distantly Supervised Challenge Dataset for Reading Comprehension. In Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (ACL), 1601--1611

    2017

  37. [47]

    M.; Uszkoreit, J.; Le, Q.; and Petrov, S

    Kwiatkowski, T.; Palomaki, J.; Redfield, O.; Collins, M.; Parikh, A.; Alberti, C.; Epstein, D.; Polosukhin, I.; Devlin, J.; Lee, K.; Toutanova, K.; Jones, L.; Kelcey, M.; Chang, M.-W.; Dai, A. M.; Uszkoreit, J.; Le, Q.; and Petrov, S. 2019. Natural Questions: A Benchmark for Q...

    2019

  38. [48]

    u ttler, H.; Lewis, M.; Yih, W.-t.; Rockt \

    Lewis, P.; Perez, E.; Piktus, A.; Petroni, F.; Karpukhin, V.; Goyal, N.; K \"u ttler, H.; Lewis, M.; Yih, W.-t.; Rockt \"a schel, T.; Riedel, S.; and Kiela, D. 2020. Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks. In Advances in Neural Information Processing ...

    2020

  39. [49]

    N.; Guo, Z.; and Chen, W

    Liang, X.; Li, Z.-Z.; Gong, Y.; Shen, Y.; Wu, Y. N.; Guo, Z.; and Chen, W. 2026. Beyond Pass@1: Self-Play with Variational Problem Synthesis Sustains RLVR . In The Fourteenth International Conference on Learning Representations

    2026

  40. [50]

    Long, X.; Zhuang, L.; Li, A.; Yao, M.; and Wang, S. 2025. EPERM: An Evidence Path Enhanced Reasoning Model for Knowledge Graph Question and Answering. arXiv:2502.16171

    2025 arXiv

  41. [51]

    Lu, H.; Wen, Y.; Cheng, P.; Ding, R.; Guo, J.; Xu, H.; Wang, C.; Chen, H.; Jiang, X.; and Jiang, G. 2026. Search Self-Play : Pushing the Frontier of Agent Capability without Supervision. In The Fourteenth International Conference on Learning Representations

    2026

  42. [52]

    Lu, R.; Hou, Z.; Wang, Z.; Zhang, H.; Liu, X.; Li, Y.; Feng, S.; Tang, J.; and Dong, Y. 2025. DeepDive: Advancing Deep Search Agents with Knowledge Graphs and Multi-Turn RL. arXiv:2509.10446

    2025

  43. [53]

    Mallen, A.; Asai, A.; Zhong, V.; Das, R.; Khashabi, D.; and Hajishirzi, H. 2023. When Not to Trust Language Models: Investigating Effectiveness of Parametric and Non-Parametric Memories. In Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics...

    2023

  44. [54]

    OpenAI . 2024. OpenAI o1 System Card. arXiv:2412.16720

    2024 arXiv

  45. [55]

    Petroni, F.; Piktus, A.; Fan, A.; Lewis, P.; Yazdani, M.; De Cao, N.; Thorne, J.; Jernite, Y.; Karpukhin, V.; Maillard, J.; Plachouras, V.; Rockt \"a schel, T.; and Riedel, S. 2021. KILT : a Benchmark for Knowledge Intensive Language Tasks. In Proceedings of the 2021 Conferenc...

    2021

  46. [56]

    A.; and Lewis, M

    Press, O.; Zhang, M.; Min, S.; Schmidt, L.; Smith, N. A.; and Lewis, M. 2023. Measuring and Narrowing the Compositionality Gap in Language Models. In Findings of the Association for Computational Linguistics: EMNLP, 5687--5711

    2023

  47. [57]

    Qwen ; Yang, A.; Yang, B.; Zhang, B.; Hui, B.; Zheng, B.; Yu, B.; Li, C.; Liu, D.; Huang, F.; Wei, H.; Lin, H.; Yang, J.; Tu, J.; Zhang, J.; Yang, J.; Yang, J.; Zhou, J.; Lin, J.; Dang, K.; Lu, K.; Bao, K.; Yang, K.; Yu, L.; Li, M.; Xue, M.; Zhang, P.; Zhu, Q.; Men, R.; Lin, R...

    2025 arXiv

  48. [58]

    Schulman, J.; Wolski, F.; Dhariwal, P.; Radford, A.; and Klimov, O. 2017. Proximal Policy Optimization Algorithms. arXiv preprint arXiv:1707.06347

    2017 arXiv

  49. [59]

    K.; Wu, Y.; and Guo, D

    Shao, Z.; Wang, P.; Zhu, Q.; Xu, R.; Song, J.; Bi, X.; Zhang, H.; Zhang, M.; Li, Y. K.; Wu, Y.; and Guo, D. 2024. DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Models. arXiv:2402.03300

    2024 arXiv

  50. [60]

    Sheng, G.; Zhang, C.; Ye, Z.; Wu, X.; Zhang, W.; Zhang, R.; Peng, Y.; Lin, H.; and Wu, C. 2025. HybridFlow : A Flexible and Efficient RLHF Framework. In Proceedings of the Twentieth European Conference on Computer Systems, 1279--1297. ACM

    2025

  51. [61]

    Shinn, N.; Cassano, F.; Gopinath, A.; Narasimhan, K.; and Yao, S. 2023. Reflexion: Language Agents with Verbal Reinforcement Learning. In Advances in Neural Information Processing Systems, volume 36

    2023

  52. [62]

    Sukhbaatar, S.; Lin, Z.; Kostrikov, I.; Synnaeve, G.; Szlam, A.; and Fergus, R. 2018. Intrinsic Motivation and Automatic Curricula via Asymmetric Self-Play. In International Conference on Learning Representations (ICLR)

    2018

  53. [63]

    M.; Shum, H.-Y.; and Guo, J

    Sun, J.; Xu, C.; Tang, L.; Wang, S.; Lin, C.; Gong, Y.; Ni, L. M.; Shum, H.-Y.; and Guo, J. 2024. Think-on-Graph: Deep and Responsible Reasoning of Large Language Model on Knowledge Graph. In International Conference on Learning Representations (ICLR)

    2024

  54. [64]

    Wang, L.; Yang, N.; Huang, X.; Jiao, B.; Yang, L.; Jiang, D.; Majumder, R.; and Wei, F. 2022. Text Embeddings by Weakly-Supervised Contrastive Pre-training. arXiv:2212.03533

    2022 arXiv

  55. [65]

    A.; Khashabi, D.; and Hajishirzi, H

    Wang, Y.; Kordi, Y.; Mishra, S.; Liu, A.; Smith, N. A.; Khashabi, D.; and Hajishirzi, H. 2023. Self-Instruct: Aligning Language Models with Self-Generated Instructions. In Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (ACL), 13484--13508

    2023

  56. [66]

    Yang, Z.; Luo, X.; Wang, Z.; Han, D.; He, Z.; Li, D.; and Xu, Y. 2026. Do Not Let Low-Probability Tokens Over-Dominate in RL for LLM s. In The Fourteenth International Conference on Learning Representations

    2026

  57. [67]

    W.; Salakhutdinov, R.; and Manning, C

    Yang, Z.; Qi, P.; Zhang, S.; Bengio, Y.; Cohen, W. W.; Salakhutdinov, R.; and Manning, C. D. 2018. HotpotQA: A Dataset for Diverse, Explainable Multi-hop Question Answering. In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing (EMNLP), 2369--2380

    2018

  58. [68]

    P.; Liao, Y.; Wang, H.; Yang, M.; Ji, H.; Wang, J.; Yan, S.; Torr, P.; and Bai, L

    Zhang, G.; Geng, H.; Yu, X.; Yin, Z.; Zhang, Z.; Tan, Z.; Zhou, H.; Li, Z.-Z.; Xue, X.; Li, Y.; Zhou, Y.; Chen, Y.; Zhang, C.; Fan, Y.; Wang, Z.; Huang, S.; Velez, F. P.; Liao, Y.; Wang, H.; Yang, M.; Ji, H.; Wang, J.; Yan, S.; Torr, P.; and Bai, L. 2026. The Landscape of Agen...

    2026

  59. [69]

    Zhao, A.; Wu, Y.; Wu, T.; Xu, Q.; Yue, Y.; Lin, M.; Wang, S.; Wu, Q.; Zheng, Z.; and Huang, G. 2025. Absolute Zero: Reinforced Self-Play Reasoning with Zero Data. In Advances in Neural Information Processing Systems, volume 38

    2025

  60. [70]

    H.; Cao, S.; Kozyrakis, C.; Stoica, I.; Gonzalez, J

    Zheng, L.; Yin, L.; Xie, Z.; Sun, C.; Huang, J.; Yu, C. H.; Cao, S.; Kozyrakis, C.; Stoica, I.; Gonzalez, J. E.; Barrett, C.; and Sheng, Y. 2024. SGLang : Efficient Execution of Structured Language Model Programs. In Advances in Neural Information Processing Systems, volume 37...

    2024

  61. [71]

    Zheng, Y.; Fu, D.; Hu, X.; Cai, X.; Ye, L.; Lu, P.; and Liu, P. 2025. DeepResearcher: Scaling Deep Research via Reinforcement Learning in Real-world Environments. arXiv:2504.03160

    2025 arXiv

Pith tools

Reviewed August 4, 2026 · model on record in the stance chip above.