Pith. sign in

REVIEW 2 major objections 5 minor 26 references

A small language model can match advanced models at spotting faithfulness hallucinations by training itself against an evolving generator of harder fakes.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · grok-4.5

2026-07-10 14:04 UTC pith:TKPHMAQE

load-bearing objection Clean self-play loop that lifts a 7B detector to GPT-4o-level F1 on RAGTruth; the gating proxies are load-bearing but the ablation and multi-round gains still look real. the 2 major comments →

arxiv 2607.07993 v1 pith:TKPHMAQE submitted 2026-07-08 cs.CL cs.LG

Hallucination Self-Play: Bootstrapping Reinforced Detector via Evolved Generator

Shiping Yang , Shining Liang , Weihao Liu , Wenbiao Ding , Linjun Shou , Lu Cheng , Angel X. Chang This is my paper
classification cs.CL cs.LG
keywords hallucination detectionself-playfaithfulnessRAGRLAIFRLVRsynthetic datareward hacking
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

Faithfulness hallucinations—claims that contradict or go beyond a provided document—are hard to detect at scale because labeled data is scarce and static synthetic generators soon become too easy for the detector. This paper introduces Hallucination Self-Play (HSP): two copies of the same base model take turns as detector and generator. The detector first learns from a little human data, then scores the generator; the generator is trained by reinforcement learning to produce claims that are hard yet still verifiable as hallucinations; those claims are fed back to retrain the detector. Across rounds the pair co-evolves, letting a 7B model reach response-level F1 on the RAGTruth benchmark that matches or exceeds much larger proprietary models, without any external rationales or larger-teacher distillation. A sympathetic reader cares because the method turns the scarcity of annotation into a closed training loop that keeps raising the difficulty of the training signal.

Core claim

By alternating RLAIF evolution of a generator (rewarded by a frozen detector for hard-but-solvable hallucinations) with RLVR optimization of the detector on the resulting synthetic claims, a small base model can progressively bootstrap strong faithfulness detection. On RAGTruth the multi-round process lifts a 7B model from modest SFT baselines to average F1 74.3, matching GPT-4o with chain-of-thought, using only the model’s own generations and simple verifiable labels.

What carries the argument

Hallucination Self-Play (HSP): a closed loop in which the generator is updated by detector-guided rewards that are gated by two proxies (absence of the known ground-truth answer, and introduction of novel named entities) plus a trivial-answer penalty, while the detector is updated by binary prediction-correctness rewards on the balanced synthetic set.

Load-bearing premise

The two simple checks—missing the known answer and adding new named entities—plus a penalty for evasive replies are enough to guarantee that generator outputs labeled as hallucinations are genuinely unfaithful and free of reward-hacking shortcuts.

What would settle it

Human audit of the positive-reward generator outputs after several self-play rounds shows that a large fraction are actually faithful or trivial, or that removing the gating criteria still yields equally strong detectors; either result would break the claim that the loop produces clean, progressively harder training data.

Watch this falsifier. Get emailed when new claim-graph text bears on it.

If this is right

  • Lightweight detectors can keep improving after deployment without new human labels, simply by continuing the self-play loop.
  • Production RAG systems can replace expensive large-model judges with small, continuously self-bootstrapped detectors.
  • The same generator–detector co-evolution pattern can be tried on other hard-to-verify generation faults once suitable proxy gates exist.
  • Cross-task transfer already appears: training only on QA-style hallucinations still lifts summarization and data-to-text detection.
  • Multi-round curricula automatically track detector strength, avoiding the plateau that static synthetic data quickly hits.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • Proxy gates based on answer absence and entity novelty may generalize to non-QA domains if replaced by domain-specific verifiers (e.g., schema constraints for tables).
  • The same self-play skeleton could bootstrap detectors for related unfaithfulness phenomena such as citation fabrication or unsupported numerical claims.
  • Because both roles share one base model, the method suggests a practical path for organizations that possess only a single mid-size checkpoint and no teacher API budget.
  • If reward hacking reappears under longer training, stronger verifiers (small entailment models or retrieval consistency checks) would be the natural next safeguard.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

2 major / 5 minor

Summary. The paper proposes Hallucination Self-Play (HSP), a closed-loop framework in which a detector and a generator, both initialized from the same base LLM, co-evolve without external rationale supervision. The detector is first SFT-initialized (on RAGTruth spans, or via rejection sampling for the CoT variant), then frozen and used as a reward model to evolve the generator via GRPO/RLAIF with a detector-guided reward (Eq. 6) gated by two proxies (absence of HotpotQA ground-truth answer; novel NER entities) plus a trivial-answer penalty (Eq. 7). The evolved generator then synthesizes y=1 claims that, mixed with base-model “faithful” responses, supply binary-verifiable data for RLVR optimization of the detector (Eq. 8). Multi-round self-play is claimed to induce a dynamic curriculum. On RAGTruth, one round of HSP improves SFT detectors (Qwen2.5-7B and Llama-3.1-8B); three rounds of the CoT variant raise a 7B model from RSFT 67.4 / RLVR 70.1 to 74.3 F1, matching GPT-4o w/ CoT (74.5). An ablation (Table 3) shows that removing the gates collapses the hallucination rate to 0 %.

Significance. If the co-evolution claim holds, HSP supplies a practical route for small models to reach frontier-level faithfulness detection without continuous reliance on proprietary LLMs or large human rationale corpora—an important systems contribution for deployable RAG pipelines. The work usefully extends self-play beyond easily verifiable domains (math/code) by introducing explicit anti-hacking gates and a verifiable binary reward, and it demonstrates cross-task transfer from HotpotQA seeds to RAGTruth’s three tasks. Strengths that should be credited include the public code release, the two-model-family results, the multi-round curriculum analysis, and the explicit ablation of reward hacking (Table 3). These elements make the paper a solid, falsifiable contribution even if some proxy assumptions require tighter quantification.

major comments (2)
  1. [§3.2.2, Table 3] §3.2.2 (Eqs. 6–7) and §4.3 / Table 3: The central claim that progressive F1 gains (Table 1: RSFT 67.4 → HSP-R3 74.3) arise from genuine hard hallucinations rests on I_gate correctly certifying y=1 labels. The two proxies (missing HotpotQA GT answer via string/alias check; novel spaCy entities) plus trivial penalty are load-bearing, yet Table 3 reports a 90 % “hallucination rate” on only 10 hand-checked samples. Absence of the answer does not entail contradiction (partial faithful answers omit it), and NER novelty misses attribute/relation errors over existing entities. A larger-scale human audit (or automatic multi-judge agreement) of positively rewarded outputs is required to bound the false-positive rate of the gates; otherwise residual label noise could inflate the reported self-play gains.
  2. [§3.2.3] §3.2.3: Faithful (y=0) examples are produced by prompting the base model and filtering against HotpotQA ground-truth answers; residual hallucinations are declared “acceptable” after filtering but never quantified. Because RLVR uses only binary correctness (Eq. 8), even modest residual noise in the y=0 class can shift decision boundaries. Report the post-filter hallucination rate of the faithful set (e.g., on a held-out sample) and show that the progressive gains survive under controlled noise injection.
minor comments (5)
  1. [Figure 1] Figure 1 caption states that “Components for mitigating reward hacking are omitted for clarity”; a small inset or appendix diagram showing the full reward pipeline (gates + trivial penalty) would help readers reproduce the method.
  2. [Abstract, §1] Abstract and §1 claim “without external supervision,” yet initialization uses human RAGTruth spans and gating uses HotpotQA ground-truth answers. Soften the phrasing to “without external rationale supervision” (as already done in the CoT subsection) for consistency.
  3. [Table 1] Table 1 reports only response-level metrics; span-level F1 (the original RAGTruth protocol) would better substantiate the claim that the detector learns precise localization rather than coarse binary decisions.
  4. [§4.1] §4.1 Implementation Details: list the exact HotpotQA subset size and any decontamination steps against RAGTruth to rule out leakage.
  5. [Throughout] Minor typos: “Insruct” (Llama), “T ask”, “T rainable”, “EV ASIVE”; also “en core web lg” should be “en_core_web_lg”.

Circularity Check

0 steps flagged

No significant circularity: self-play co-evolution uses external human span labels (RAGTruth) for detector init/eval and external HotpotQA GT + NER for generator reward gates; final F1 is an independent empirical measurement, not forced by construction.

full rationale

The paper's central claim is an empirical performance result (7B model reaching ~74.3 response-level F1 on RAGTruth after multi-round HSP, matching GPT-4o w/ CoT) obtained via a closed-loop generator-detector interaction. Walking the derivation: detector cold-start is SFT (or rejection-sampled CoT filtered by span coverage) on human-annotated RAGTruth spans (Eq. 4, §3.1.2); generator reward (Eq. 6-7) is gated by absence of HotpotQA ground-truth answers (string/alias check) or novel NER entities plus trivial-answer penalty (§3.2.2); detector RLVR uses binary correctness against those construction-time labels (Eq. 8); evaluation is on held-out human RAGTruth annotations (Table 1). None of these steps reduces a claimed prediction or first-principles result to its own inputs by definition, fit, or load-bearing self-citation of an unverified uniqueness theorem. Self-play is intentional co-evolution under external verifiable rewards, not tautological. Rejection sampling for CoT rationales is filtered by external spans and is standard distillation, not circular. Ablation (Table 3) shows the gates are necessary to avoid collapse, but that is an empirical safeguard, not a definitional loop. The framework is self-contained against external benchmarks; score 0 is the correct non-finding.

Axiom & Free-Parameter Ledger

3 free parameters · 4 axioms · 2 invented entities

The central claim rests on the closed-loop interaction working under the stated reward design and on the assumption that HotpotQA-derived gates transfer to RAGTruth-style documents. Free parameters are ordinary training hyper-parameters; axioms are standard RL and faithfulness definitions plus the paper-specific gating rules; the only invented constructs are the HSP roles and the composite reward itself, which have no independent external evidence beyond the reported experiments.

free parameters (3)
  • GRPO group size K and temperature
    K=8 (training) / 10 (reward estimation) and temperature 1.0 are chosen by hand; they directly control the variance of the detector-guided reward and therefore the curriculum difficulty.
  • SFT and RL learning rates
    1e-5 (SFT) and 1e-6 (RL) together with batch sizes 32/256 and one epoch are hand-selected; performance is sensitive to these values yet no sweep is reported.
  • Number of synthetic samples per stage
    10k samples per role per round is an arbitrary scale choice that determines how much the generator can evolve before the detector is updated.
axioms (4)
  • domain assumption A claim is hallucinated if it contradicts the document or introduces unsupported facts; absence of the QA ground-truth answer (or its aliases) is a sufficient proxy for contradiction.
    Stated in §3.2.2 and used as the first reward gate; the proxy is never proved complete.
  • domain assumption GRPO with the listed clipping and KL coefficients yields stable policy improvement for both generator and detector.
    Taken from Shao et al. (2024) and applied without further justification in §3.2.1.
  • ad hoc to paper Residual hallucinations remaining after filtering base-model ‘faithful’ generations against ground-truth answers are tolerable for RLVR training.
    Explicitly acknowledged in §3.2.3; no quantitative bound is given.
  • ad hoc to paper Named-entity novelty (spaCy en_core_web_lg) correctly flags unsupported facts.
    Second gating criterion in §3.2.2; NER errors or multi-word facts without new entities can slip through.
invented entities (2)
  • Hallucination Self-Play (HSP) closed loop no independent evidence
    purpose: Co-evolve generator and detector without external supervision after cold-start.
    Defined in §3 and Figure 1; the loop itself is the paper’s central construct and has no prior independent existence.
  • Detector-guided reward with dual gating and trivial-answer penalty no independent evidence
    purpose: Supply a scalar that encourages hard-yet-learnable hallucinations while blocking reward hacking.
    Equations (6)–(7) and the three mitigation components are introduced ad hoc; their necessity is shown only by the internal ablation.

pith-pipeline@v1.1.0-grok45 · 18991 in / 3051 out tokens · 69772 ms · 2026-07-10T14:04:26.445493+00:00 · methodology

0 comments
read the original abstract

Identifying faithfulness hallucinations in LLM-generated outputs remains challenging due to the scarcity of high-quality annotated data. Recent work relies on advanced LLMs to synthesize training data, including rationales, labels, and hallucinated claims. However, these methods treat the generator as a static component, limiting iterative improvement of the detector. To address this limitation, we introduce Hallucination Self-Play (HSP), a novel framework that enables the detector to bootstrap with an evolved generator. HSP involves two roles initialized from the same base model, a detector that assesses the faithfulness of model outputs, and a generator that produces increasingly hard-to-detect hallucinated responses. Specifically, the detector is first fine-tuned on human-labeled data and then employed as a reward model to train the generator via reinforcement learning from AI feedback (RLAIF). In turn, the evolved generator synthesizes hallucination data to further optimize the detector through rule-based reinforcement learning. Experiments on RAGTruth benchmark and two model families demonstrate that the proposed framework can progressively enhance a small LLM to match or even outperform advanced LLMs without external supervision. Our code is available at https://anonymous.4open.science/r/Hallucination-Self-Play-50B5 .

Figures

Figures reproduced from arXiv: 2607.07993 by Angel X. Chang, Linjun Shou, Lu Cheng, Shining Liang, Shiping Yang, Weihao Liu, Wenbiao Ding.

Figure 1
Figure 1. Figure 1: Overview of the HSP framework. The generator is evolved via RLAIF, with the [PITH_FULL_IMAGE:figures/full_fig_p003_1.png] view at source ↗
Figure 2
Figure 2. Figure 2: Prompt template for hallucination generator. [PITH_FULL_IMAGE:figures/full_fig_p013_2.png] view at source ↗
Figure 3
Figure 3. Figure 3: Prompt template for model-based answer alias detection, used as part of the [PITH_FULL_IMAGE:figures/full_fig_p013_3.png] view at source ↗
Figure 4
Figure 4. Figure 4: Prompt template for model-based evasive answer detection, used as part of the [PITH_FULL_IMAGE:figures/full_fig_p014_4.png] view at source ↗
Figure 5
Figure 5. Figure 5: Prompt templates used for the detector w/ CoT across three task types: QA, [PITH_FULL_IMAGE:figures/full_fig_p015_5.png] view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

26 extracted references · 26 canonical work pages · 16 internal anchors

  1. [1]

    Autohall: Automated hallucination dataset generation for large language models.arXiv preprint arXiv:2310.00259,

    Zouying Cao, Yifei Yang, and Hai Zhao. Autohall: Automated hallucination dataset generation for large language models.arXiv preprint arXiv:2310.00259,

    work page arXiv

  2. [2]

    Retaining by Doing: The Role of On-Policy Data in Mitigating Forgetting

    Howard Chen, Noam Razin, Karthik Narasimhan, and Danqi Chen. Retaining by doing: The role of on-policy data in mitigating forgetting.arXiv preprint arXiv:2510.18874, 2025a. Jiaqi Chen, Bang Zhang, Ruotian Ma, Peisong Wang, Xiaodan Liang, Zhaopeng Tu, Xiaolong Li, and Kwan-Yee K Wong. Spc: Evolving self-play critic via adversarial games for llm reasoning.a...

    work page internal anchor Pith review Pith/arXiv arXiv 2024

  3. [3]

    DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learning

    Daya Guo, Dejian Yang, Haowei Zhang, Junxiao Song, Ruoyu Zhang, Runxin Xu, Qihao Zhu, Shirong Ma, Peiyi Wang, Xiao Bi, et al. Deepseek-r1: Incentivizing reasoning capability in llms via reinforcement learning.arXiv preprint arXiv:2501.12948,

    work page internal anchor Pith review Pith/arXiv arXiv

  4. [4]

    GPT-4o System Card

    Aaron Hurst, Adam Lerer, Adam P Goucher, Adam Perelman, Aditya Ramesh, Aidan Clark, AJ Ostrow, Akila Welihinda, Alan Hayes, Alec Radford, et al. Gpt-4o system card.arXiv preprint arXiv:2410.21276,

    work page internal anchor Pith review Pith/arXiv arXiv

  5. [5]

    The FACTS Grounding Leaderboard: Benchmarking LLMs' Ability to Ground Responses to Long-Form Input

    Alon Jacovi, Andrew Wang, Chris Alberti, Connie Tao, Jon Lipovetz, Kate Olszewska, Lukas Haas, Michelle Liu, Nate Keating, Adam Bloniarz, et al. The facts grounding leaderboard: Benchmarking llms’ ability to ground responses to long-form input.arXiv preprint arXiv:2501.03200,

    work page internal anchor Pith review Pith/arXiv arXiv

  6. [6]

    DRP: Distilled Reasoning Pruning with Skill-aware Step Decomposition for Efficient Large Reasoning Models

    URL https: //arxiv.org/abs/2505.13975. Jakub Grudzien Kuba, Mengting Gu, Qi Ma, Yuandong Tian, Vijai Mohan, and Jason Chen. Language self-play for data-free training.arXiv preprint arXiv:2509.07414,

    work page internal anchor Pith review Pith/arXiv arXiv

  7. [7]

    Factcg: Enhancing fact checkers with graph-based multi-hop data

    10 Published as a conference paper at COLM 2026 Deren Lei, Yaxi Li, Siyao Li, Mengya Hu, Rui Xu, Ken Archer, Mingyu Wang, Emily Ching, and Alex Deng. Factcg: Enhancing fact checkers with graph-based multi-hop data. In Proceedings of the 2025 Conference of the Nations of the Americas Chapter of the Association for Computational Linguistics: Human Language ...

    work page 2026

  8. [8]

    From generation to judgment: Opportunities and challenges of llm-as-a-judge

    Dawei Li, Bohan Jiang, Liangjie Huang, Alimohammad Beigi, Chengshuai Zhao, Zhen Tan, Amrita Bhattacharjee, Yuxuan Jiang, Canyu Chen, Tianhao Wu, et al. From generation to judgment: Opportunities and challenges of llm-as-a-judge. InProceedings of the 2025 Conference on Empirical Methods in Natural Language Processing, pp. 2757–2791,

    work page 2025

  9. [9]

    DeepSeek-V3.2: Pushing the Frontier of Open Large Language Models

    Aixin Liu, Aoxue Mei, Bangcai Lin, Bing Xue, Bingxuan Wang, Bingzheng Xu, Bochao Wu, Bowei Zhang, Chaofan Lin, Chen Dong, et al. Deepseek-v3. 2: Pushing the frontier of open large language models.arXiv preprint arXiv:2512.02556,

    work page internal anchor Pith review Pith/arXiv arXiv

  10. [10]

    Boundless Socratic Learning with Language Games

    Tom Schaul. Boundless socratic learning with language games.arXiv preprint arXiv:2411.16905,

    work page internal anchor Pith review Pith/arXiv arXiv

  11. [11]

    Proximal Policy Optimization Algorithms

    John Schulman, Filip Wolski, Prafulla Dhariwal, Alec Radford, and Oleg Klimov. Proximal policy optimization algorithms.arXiv preprint arXiv:1707.06347,

    work page internal anchor Pith review Pith/arXiv arXiv

  12. [12]

    Verifying the verifiers: Unveiling pitfalls and potentials in fact verifiers.arXiv preprint arXiv:2506.13342,

    Wooseok Seo, Seungju Han, Jaehun Jung, Benjamin Newman, Seungwon Lim, Seungbeen Lee, Ximing Lu, Yejin Choi, and Youngjae Yu. Verifying the verifiers: Unveiling pitfalls and potentials in fact verifiers.arXiv preprint arXiv:2506.13342,

    work page arXiv

  13. [13]

    DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Models

    Zhihong Shao, Peiyi Wang, Qihao Zhu, Runxin Xu, Junxiao Song, Xiao Bi, Haowei Zhang, Mingchuan Zhang, YK Li, Yang Wu, et al. Deepseekmath: Pushing the limits of mathe- matical reasoning in open language models.arXiv preprint arXiv:2402.03300,

    work page internal anchor Pith review Pith/arXiv arXiv

  14. [14]

    Mastering Chess and Shogi by Self-Play with a General Reinforcement Learning Algorithm

    David Silver, Thomas Hubert, Julian Schrittwieser, Ioannis Antonoglou, Matthew Lai, Arthur Guez, Marc Lanctot, Laurent Sifre, Dharshan Kumaran, Thore Graepel, et al. Mastering chess and shogi by self-play with a general reinforcement learning algorithm. arXiv preprint arXiv:1712.01815, 2017a. David Silver, Julian Schrittwieser, Karen Simonyan, Ioannis Ant...

    work page internal anchor Pith review Pith/arXiv arXiv 2024

  15. [15]

    Learning to reason for hallucination span detection.arXiv preprint arXiv:2510.02173,

    Hsuan Su, Ting-Yao Hu, Hema Swetha Koppula, Kundan Krishna, Hadi Pouransari, Cheng- Yu Hsieh, Cem Koc, Joseph Yitan Cheng, Oncel Tuzel, and Raviteja Vemulapalli. Learning to reason for hallucination span detection.arXiv preprint arXiv:2510.02173,

    work page arXiv

  16. [16]

    Large language models for data annotation and synthesis: A survey

    11 Published as a conference paper at COLM 2026 Zhen Tan, Dawei Li, Song Wang, Alimohammad Beigi, Bohan Jiang, Amrita Bhattacharjee, Mansooreh Karami, Jundong Li, Lu Cheng, and Huan Liu. Large language models for data annotation and synthesis: A survey. InProceedings of the 2024 Conference on Empirical Methods in Natural Language Processing, pp. 930–957,

    work page 2026

  17. [17]

    Minicheck: Efficient fact-checking of llms on grounding documents

    Liyan Tang, Philippe Laban, and Greg Durrett. Minicheck: Efficient fact-checking of llms on grounding documents. InProceedings of the 2024 Conference on Empirical Methods in Natural Language Processing, pp. 8818–8847,

    work page 2024

  18. [18]

    Qwen3 Technical Report

    An Yang, Anfeng Li, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chang Gao, Chengen Huang, Chenxu Lv, et al. Qwen3 technical report.arXiv preprint arXiv:2505.09388, 2025a. Shiping Yang, Renliang Sun, and Xiaojun Wan. A new benchmark and reverse valida- tion method for passage-level hallucination detection. InFindings of the Association fo...

    work page internal anchor Pith review Pith/arXiv arXiv 2023

  19. [19]

    Quantifying and Improving the Robustness of Retrieval-Augmented Language Models Against Spurious Features in Grounding Data

    Shiping Yang, Jie Wu, Wenbiao Ding, Ning Wu, Shining Liang, Ming Gong, Hengyuan Zhang, and Dongmei Zhang. Quantifying the robustness of retrieval-augmented language models against spurious features in grounding data.arXiv preprint arXiv:2503.05587, 2025b. Zhilin Yang, Peng Qi, Saizheng Zhang, Yoshua Bengio, William Cohen, Ruslan Salakhutdi- nov, and Chris...

    work page internal anchor Pith review Pith/arXiv arXiv 2018

  20. [20]

    DAPO: An Open-Source LLM Reinforcement Learning System at Scale

    Qiying Yu, Zheng Zhang, Ruofei Zhu, Yufeng Yuan, Xiaochen Zuo, Yu Yue, Weinan Dai, Tiantian Fan, Gaohong Liu, Lingjun Liu, et al. Dapo: An open-source llm reinforcement learning system at scale.arXiv preprint arXiv:2503.14476, 2025a. Yiyao Yu, Yuxiang Zhang, Dongdong Zhang, Xiao Liang, Hengyuan Zhang, Xingxing Zhang, Mahmoud Khademi, Hany Hassan Awadalla,...

    work page internal anchor Pith review Pith/arXiv arXiv

  21. [21]

    Find Your Optimal Teacher: Personalized Data Synthesis via Router-Guided Multi-Teacher Distillation

    Hengyuan Zhang, Shiping Yang, Xiao Liang, Chenming Shang, Yuxuan Jiang, Chaofan Tao, Jing Xiong, Hayden Kwok-Hay So, Ruobing Xie, Angel X Chang, et al. Find your optimal teacher: Personalized data synthesis via router-guided multi-teacher distillation.arXiv preprint arXiv:2510.10925,

    work page internal anchor Pith review Pith/arXiv arXiv

  22. [22]

    Absolute Zero: Reinforced Self-play Reasoning with Zero Data

    Andrew Zhao, Yiran Wu, Yang Yue, Tong Wu, Quentin Xu, Matthieu Lin, Shenzhi Wang, Qingyun Wu, Zilong Zheng, and Gao Huang. Absolute zero: Reinforced self-play reasoning with zero data.arXiv preprint arXiv:2505.03335,

    work page internal anchor Pith review Pith/arXiv arXiv

  23. [23]

    LlamaFactory: Unified Efficient Fine-Tuning of 100+ Language Models

    Association for Computational Linguistics. URLhttp://arxiv.org/abs/2403.13372. A Implementation Details All experiments are conducted on 8 NVIDIA A100 GPUs. We implement SFT training usingLlamaFactory(Zheng et al.,

    work page internal anchor Pith review Pith/arXiv arXiv

  24. [24]

    Each training stage for both the detector and generator uses 10k samples

    and RL training using theverlframework (Sheng et al., 2025). Each training stage for both the detector and generator uses 10k samples. The detailed hyperparameters for both training stages are summarized in Table

    work page 2025

  25. [25]

    EVASIVE” if the answer fails to provide the required information. • “NOT EVASIVE

    12 Published as a conference paper at COLM 2026 Hyperparameter SFT RL (GRPO) Learning rate 1e-5 1e-6 LR scheduler cosine cosine Warmup 0.1 (ratio) 5 (steps) Train batch size 32 256 Epochs 2 1 Mini-batch size — 64 Rollout samples (K) — 8 Table 2: Training hyperparameters for SFT and RL stages. B Prompt Templates Below is a question and related passages. Yo...

    work page 2026

  26. [26]

    hallucination list

    Explanation:The answer directly provides the requested information (a year of death), but the year is incorrect. This is a hallucinated answer, not an evasive one. Label: NOT EV ASIVE Question:{question} Answer:{response} Explanation: Label: Figure 4: Prompt template for model-based evasive answer detection, used as part of the trivial answer penalty. 14 ...

    work page 2026