Pith. sign in

REVIEW 4 major objections 6 minor 43 references

Intent Factored Generation: Unleashing the Diversity in Your Language Model

T0 review · 4 major / 6 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read Intent Factored Generation claims that separating LLM sampling into a high-temperature intent stage and a low-temperature response stage yields more semantically diverse outputs without sacrificing quality, improving pass@k on maths and…

desk verdict The reasoning results are solid; the diversity headline depends on a metric the paper never validates. read the letter →

arxiv 2506.09659 v1 pith:4W5AGCPQ submitted 2025-06-11 cs.AI cs.CLcs.LG

Eltayeb Ahmed , Uljad Berdica , Martha Elliott , Danijela Horak , Jakob N. Foerster This is my paper
classification cs.AIcs.CLcs.LG
keywords intentfactoredgenerationsemanticdiversitytemperaturesamplingpass@kevaluationreinforcementlearningfromverifierfeedbackdirectpreferenceoptimizationrelaxedentropydiversecomment
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper claims that the main obstacle to semantically diverse LLM samples is not sampling temperature but the fact that concepts and wording are sampled together in one latent draw. It proposes Intent Factored Generation (IFG): first sample a short intent, such as keywords or a summary, at high temperature, then sample the final response at lower temperature conditioned on the prompt and that intent. Because a small change in intent changes the whole response's meaning, high intent temperature translates into conceptual diversity while low response temperature preserves coherence. Across maths, code, instruction-tuning, and open-ended comment generation, the method reports higher pass@k, better reinforcement-learning-from-verifier results, and a better quality-diversity frontier than direct sampling. For many uses the method needs no training, only a prompt change and two temperatures.

What carries the argument

The load-bearing object is the intent: a short, semantically dense representation such as keywords or a summary, sampled before the response. IFG changes sampling from $r \sim M(p)$ to $i \sim M(p)$ followed by $r \sim M(p,i)$, with separate temperatures $t_i$ and $t_r$, and for reasoning it inserts an intent before each chain-of-thought step. The companion metric RSE computes semantic diversity by asking a prompted LLM whether pairs of generations are similar, building an adjacency matrix, and taking connected components as equivalence classes.

What would settle it

Take a sample of the generated comment sets from the diverse comment generation experiment, have human annotators label which pairs are semantically the same, build clusters from their labels, and recompute entropy for IFG versus direct sampling; if IFG no longer dominates the quality-diversity frontier, the reported diversity gains are an artifact of the LLM judge.

Watch

Extended reading notes

Core claim

IFG's central claim is that factorising the sampling distribution as $i \sim M(p)$ then $r \sim M(p,i)$ makes semantic diversity controllable: the intent $i$ anchors the concepts of the response, so sampling intents hot and responses cold decouples diversity from coherence. The paper reports that this improves pass@k on MATH and LiveCodeBench, improves STaR-based RL from verifier feedback on MATH at 3B, 7B and 14B scales, achieves higher reward at equal diversity after DPO instruction tuning, and shifts the quality-diversity Pareto frontier for generated reader comments. The paper introduces Relaxed Semantic Entropy (RSE), which replaces bidirectional entailment with LLM-judged pairwise similarity and clusters by connected components, as the metric for these diversity comparisons.

Load-bearing premise

The paper's headline diversity results assume that its LLM-judged similarity metric, RSE, measures semantic diversity the way humans would, and that assumption is not validated against human judgments in the paper.

Editorial extensions

If this is right

  • IFG raises pass@k at every tested $k$ on MATH, and on LiveCodeBench it solves more problems in 5 attempts than the baseline solves in 10.
  • IFG+STaR outperforms STaR alone across 3B, 7B and 14B models, with larger gains at larger scale and in early training iterations.
  • DPO+IFG produces a reward-diversity Pareto frontier that dominates vanilla DPO on conversational responses.
  • Finetuned IFG generates reader comments with higher RSE diversity at comparable coherence, and the two-temperature split is necessary: constraining $t_i=t_r$ lowers performance on code, comments, and DPO.
  • IFG is implementable by changing the prompt and alternating temperatures, requiring no extra model or training in the few-shot version.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • If the diversity measure is trusted, IFG provides a drop-in diversity knob for any algorithm that benefits from diverse samples, including tree search, evolutionary methods, and verifier-based RL, since it only wraps the sampling call.
  • The method's ceiling depends on the intent sampler: if the model cannot produce varied intents, high $t_i$ will only shuffle wording, so measuring and steering intent diversity directly is a natural next test.
  • The RSE-based quality-diversity results should be re-checked with human similarity judgments, because an LLM judge that mistakes topic adjacency or wording changes for semantic difference could inflate IFG's apparent advantage.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 6 minor

Summary. The paper introduces Intent Factored Generation (IFG), a two-stage sampling scheme: first sample an intent (keywords, a summary, or a step outline) at a high temperature, then sample the final response conditioned on both the original prompt and that intent at a lower temperature. The authors define Relaxed Semantic Entropy (RSE), an LLM-judged, connected-components variant of semantic entropy, and use it to measure diversity. They report pass@k improvements on MATH and LiveCodeBench, STaR gains on MATH, improved reward-diversity frontiers for DPO instruction tuning, and improved coherence-diversity frontiers on a new Reddit news-comment dataset. IFG is implemented either by few-shot prompting or by finetuning on intent-annotated data, and the code and dataset are open-sourced.

Significance. If the central empirical claims hold, IFG is an attractive and low-cost method for semantic diversity: it changes only prompting and sampling temperatures, can wrap around API models, and integrates naturally with test-time compute and reinforcement learning from verifier feedback. The MATH pass@k and STaR results are measured against an external benchmark with bootstrapped confidence intervals, and the paper is transparent about hyperparameters, prompts, and failure modes, including the MMLU/STaR limitation and the reliance on reliable verifiers. The main risk is that the diversity headline rests on RSE, a new metric introduced and evaluated by the same authors without human validation, so the paper's broadest claim is not yet established. Nonetheless, the method is simple enough that the reasoning-exploration claims can be checked and reused independently of RSE.

major comments (4)
  1. [4.2, Algorithm 2, Figures 4-5] Section 4.2 (Eq. 5, Algorithm 2): RSE is the diversity measure behind the headline claims in Sections 6.3 and 6.4, but it is neither validated against human similarity judgments nor compared with an existing semantic-diversity metric on these tasks. Because the similarity prompt in Appendix I.3 encodes the authors' own notion of 'similar' versus 'different', and because connected components over pairwise LLM judgments can merge long chains of superficially related texts, the improved Pareto frontiers in Figures 4a and 5 may reflect the judge's biases rather than true semantic diversity. The authors should provide a validation study, for example human agreement, sensitivity to prompt wording, and controls with known surface variation, or explicitly restrict the diversity claims to 'RSE diversity'.
  2. [6.2, Table 1] Section 6.2, Table 1: The code-generation claim rests on 50 vs. 45 correct problems at pass@5 and 55 vs. 48 at pass@10 out of 182, with no confidence intervals or significance tests, and pass@1 is lower for IFG (30 vs. 34). These differences are small enough that they could be within sampling noise given the single temperature-tuning procedure described in Section 5.2, so the statement that IFG 'improves pass@k' on LiveCodeBench is not yet supported at the reported precision. Please add uncertainty quantification, such as bootstrap over problems or repeated sampling, or temper the claim.
  3. [5.3, Appendix E.2, Table 6] Section 5.3 and Appendix E.2 (Table 6): The DPO comparison is not fully controlled, because the baseline DPO model was trained on 8 L40S GPUs while the DPO+IFG model was trained on 8 H200s. Even with identical hyperparameters, different hardware introduces nondeterminism, and the appendix does not report seeds or repeated runs. Since Figure 4a is one of the two main pieces of evidence for the diversity-quality frontier, the authors should either retrain under matched conditions or provide a control demonstrating that the hardware difference does not affect the comparison.
  4. [6.4, Figure 5, Appendix C] Section 6.4, Figure 5 and Appendix C: Quality in the comment-generation experiment is measured by 1 - INCOHERENCE from the Perspective API, which is a moderation classifier and not a semantic quality metric. A comment can be irrelevant to the article or subtly nonsensical while still receiving a low INCOHERENCE score, so the Pareto frontier in Figure 5 may overstate the quality of high-RSE IFG comments. The authors' own attempt at LLM-judged quality in Appendix E.4 was unreliable at high temperatures, which underscores the need for human evaluation of relevance and semantic coherence before drawing the diversity-without-quality-loss conclusion.
minor comments (6)
  1. [Appendix D.2] Appendix D.2 contains an unfinished parenthetical 'Note check this' regarding the data collection date range; this should be resolved before publication.
  2. [Appendix E.2 and Figure 6] Appendix E.2 contains typos such as 'tained', 'wheread', and 'hyperparamters', and the caption of Figure 6 reads 'Pareot Plots' instead of 'Pareto Plots'.
  3. [Appendix B, Eq. (10)] Appendix B, Eq. (10) minimizes L_i + L_j, but the loss is defined earlier as L_i and L_r; L_j appears to be a typo for L_r.
  4. [Figure 3 caption] The caption of Figure 3(b) says 'for all values of k' when discussing STaR iterations; this should refer to iterations, not k, and the legend 'IFG Baseline' is ambiguous.
  5. [Section 4.1, Figure 5b reference] Section 4.1 states that IFG intent and response examples are shown in 'Figure 5b', but Figure 5 shows comment-generation results, not the referenced examples; the cross-reference appears to be wrong.
  6. [Appendix I.3] Appendix I.3 contains typos such as 'Differnet' and 'commnet', and the semantic-clustering prompts would benefit from a statement that the judge was not the same model used for generation.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the reasoning gains are measured on external benchmarks, and the RSE-based diversity claims use a new metric that is not fitted to IFG's outputs.

full rationale

The paper's central claims split into two chains. First, IFG improves pass@k and STaR on MATH and LiveCodeBench; these results are measured against external benchmarks and verifier feedback, not against quantities defined in this paper. Temperatures are tuned on separate validation or training subsets before the reported test numbers, so no reported value is a fitted parameter relabeled as a prediction. Second, IFG improves the quality-diversity frontier on instruction-tuning and comment generation. That claim uses Relaxed Semantic Entropy (RSE), introduced in Section 4.2, together with the Perspective API incoherence score and a Bradley-Terry reward model. RSE is an operationalization of semantic diversity rather than a parameter fitted to IFG outputs; the similarity prompts in Appendix I.3 define a judgment criterion and are not optimized against the reported diversity numbers. A concern that RSE may not match human judgments of semantic equivalence, or that Perspective API's INCOHERENCE is a weak quality signal, is a validity and correctness risk, not a circularity defect: the paper does not derive the diversity result from the definition of RSE. The only self-citation is the related-work comparison to Rainbow Teaming, which is not load-bearing. The appended '(Note check this)' in Section D.2 flags an unresolved data-collection detail, but it does not feed into the derivation. Overall, no equation reduces to its own inputs and no fitted value is renamed as a prediction; the reported gains are, at worst, dependent on the validity of the chosen metrics rather than on circular reasoning.

Assumptions & free parameters 4 free parameters · 4 assumptions · 0 invented entities

The central claim rests on an empirical sampling method rather than a formal derivation. It depends on the model's ability to produce and follow intents, on the validity of RSE and Perspective API as proxies, and on tuned generation temperatures. No new physical entities are introduced.

free parameters (4)
  • intent temperature t_i = 0.73 on LiveCodeBench; varied per task and validation split
    Selected by random search on validation data to maximize pass@k or the quality-diversity frontier. The method's benefit depends on t_i being higher than t_r.
  • response temperature t_r = 0.60 on LiveCodeBench; varied per task and validation split
    Selected jointly with t_i; lower than t_i in all reported optimal settings.
  • intent granularity = one intent per chain-of-thought step, one per 1 to 3 lines of code, one per comment
    Hand-chosen design decision that changes how strongly the intent anchors the final generation.
  • keyword length for DPO annotation = no more than 3 words
    Hand-chosen constraint in the annotation prompt used to create DPO intents.
assumptions (4)
  • domain assumption The factorisation p(r|p) = sum_i p(i|p) p(r|p,i) means intent diversity propagates to response diversity.
    Invoked in Section 4.1 and Eq. (2); the paper does not directly measure how strongly the final response is anchored to the sampled intent.
  • domain assumption A prompted LLM's bidirectional similarity judgments are a valid semantic equivalence for clustering outputs in RSE.
    Section 4.2 and Appendix I.3 define RSE through LLM pairwise similarity; no human validation of the similarity judgments is provided.
  • domain assumption Perspective API incoherence and related scores are valid proxies for generated comment quality.
    Used in Section 5.4 and Appendix C as the quality axis for the comment generation Pareto frontier.
  • domain assumption The post-release LiveCodeBench slice is contamination-free.
    Section 5.2 restricts evaluation to problems from competitions after Qwen-2.5-Coder-32B release, relying on the benchmark's contamination controls.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Intent Factored Generation: Unleashing the Diversity in Your Language Model." pith.science (2026). https://pith.science/paper/4W5AGCPQ

@misc{pith2026250609659,
  author       = {Pith},
  title        = {Pith review of: Intent Factored Generation: Unleashing the Diversity in Your Language Model},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/4W5AGCPQ}},
  note         = {Machine review of arXiv:2506.09659}
}
read the original abstract

Obtaining multiple meaningfully diverse, high quality samples from Large Language Models for a fixed prompt remains an open challenge. Current methods for increasing diversity often only operate at the token-level, paraphrasing the same response. This is problematic because it leads to poor exploration on reasoning problems and to unengaging, repetitive conversational agents. To address this we propose Intent Factored Generation (IFG), factorising the sampling process into two stages. First, we sample a semantically dense intent, e.g., a summary or keywords. Second, we sample the final response conditioning on both the original prompt and the intent from the first stage. This allows us to use a higher temperature during the intent step to promote conceptual diversity, and a lower temperature during the final generation to ensure the outputs are coherent and self-consistent. Additionally, we find that prompting the model to explicitly state its intent for each step of the chain-of-thought before generating the step is beneficial for reasoning tasks. We demonstrate our method's effectiveness across a diverse set of tasks. We show this method improves both pass@k and Reinforcement Learning from Verifier Feedback on maths and code tasks. For instruction-tuning, we combine IFG with Direct Preference Optimisation to increase conversational diversity without sacrificing reward. Finally, we achieve higher diversity while maintaining the quality of generations on a general language modelling task, using a new dataset of reader comments and news articles that we collect and open-source. In summary, we present a simple method of increasing the sample diversity of LLMs while maintaining performance. This method can be implemented by changing the prompt and varying the temperature during generation, making it easy to integrate into many algorithms for gains across various applications.

Figures

Figures reproduced from arXiv: 2506.09659 by the authors.

Figure 1
Figure 1. In Intent Factored Generation (IFG), we first sample a semantically dense intent such as [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 2
Figure 2. We characterise LLM responses by intent which represents the semantics, and by phrasing. Typically both of these are latently sampled by the LLM conditioned on the prompt. In IFG we sample the intent explicitly (orange) instead of latently (grey). With IFG sampling, we can sample the intent with a higher temperature than the response to induce semantic diversity while using a lower temperature for the final response… view at source ↗
Figure 3
Figure 3. Here we present results on the MATH dataset. In (a) we plot the pass@k vs [PITH_FULL_IMAGE:figures/full_fig_p007_3.png] view at source ↗
Figures from the paper (14 more)
Figure 4
Figure 4. Figure 4: This figure compares the performance and diversity of DPO+IFG to DPO in an instruction [PITH_FULL_IMAGE:figures/full_fig_p008_4.png]
Figure 5
Figure 5. Figure 5: This plot presents Relaxed Semantic Entropy (RSE) vs Coherence for comments generated [PITH_FULL_IMAGE:figures/full_fig_p009_5.png]
Figure 6
Figure 6. Figure 6: Pareot Plots of results using the 1 - Self-Bleu as a diversity metric. [PITH_FULL_IMAGE:figures/full_fig_p019_6.png]
Figure 7
Figure 7. Figure 7: Pareto Plots using Perplexity as a quality metric. [PITH_FULL_IMAGE:figures/full_fig_p019_7.png]
Figure 8
Figure 8. Figure 8: Relaxed Semantic Entropy results with Diverse Beam Search. The number of beams should [PITH_FULL_IMAGE:figures/full_fig_p020_8.png]
Figure 9
Figure 9. Figure 9: Here we plot coherence vs temperature and profanity vs temperature. For IFG the tempera [PITH_FULL_IMAGE:figures/full_fig_p020_9.png]
Figure 11
Figure 11. Figure 11: Perspective API score correlation on the comments from another set of articles not used to test. This serves as a qualitative sanity check for the self-consistency of the scores [PITH_FULL_IMAGE:figures/full_fig_p021_11.png]
Figure 12
Figure 12. Figure 12: Correlation of Perspective API scores for the comments in test set and for comments in [PITH_FULL_IMAGE:figures/full_fig_p021_12.png]
Figure 13
Figure 13. Figure 13: Score distribution for the test set, direct generation and IFG generations. This shows that [PITH_FULL_IMAGE:figures/full_fig_p022_13.png]
Figure 14
Figure 14. Figure 14: Evolution of each metric’s score as we increase the temperature for the two modes of [PITH_FULL_IMAGE:figures/full_fig_p022_14.png]
Figure 15
Figure 15. Figure 15: An ablation where we do IFG sampling but constrain [PITH_FULL_IMAGE:figures/full_fig_p024_15.png]
Figure 16
Figure 16. Figure 16: This shows the effect of using a higher temperature for the intent sampling as opposed to [PITH_FULL_IMAGE:figures/full_fig_p024_16.png]
Figure 17
Figure 17. Figure 17: Ablation to specifically compare ti = tr and tr > ti to our method of ti > tr. We show across all the diversity values and metrics, our method of sampling higher intent temperature outperforms the other design choices. This is an ablation of the results in [PITH_FULL…
Figure 18
Figure 18. Figure 18: RSE vs temperature for few-shot prompted and finetuned models on the Reddit News [PITH_FULL_IMAGE:figures/full_fig_p025_18.png]

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

43 extracted references · 8 canonical work pages

  1. [1]

    Toolformer: Language models can teach themselves to use tools

    Timo Schick, Jane Dwivedi-Yu, Roberto Dess \` , Roberta Raileanu, Maria Lomeli, Eric Hambro, Luke Zettlemoyer, Nicola Cancedda, and Thomas Scialom. Toolformer: Language models can teach themselves to use tools. Advances in Neural Information Processing Systems, 36: 0 68539--68551, 2023

    2023

  2. [2]

    Witscript 2: A System for Generating Improvised Jokes Without Wordplay

    Joe Toplyn. Witscript 2: A system for generating improvised jokes without wordplay. arXiv preprint arXiv:2302.03036, 2023

    work page Pith review arXiv 2023

  3. [3]

    Coauthor: Designing a human-ai collaborative writing dataset for exploring language model capabilities

    Mina Lee, Percy Liang, and Qian Yang. Coauthor: Designing a human-ai collaborative writing dataset for exploring language model capabilities. In Proceedings of the 2022 CHI conference on human factors in computing systems, pages 1--19, 2022

    work page 2022

  4. [4]

    Is temperature the creativity parameter of large language models? arXiv preprint arXiv:2405.00492, 2024

    Max Peeperkorn, Tom Kouwenhoven, Dan Brown, and Anna Jordanous. Is temperature the creativity parameter of large language models? arXiv preprint arXiv:2405.00492, 2024

    arXiv 2024

  5. [5]

    Semantic uncertainty: Linguistic invariances for uncertainty estimation in natural language generation

    Lorenz Kuhn, Yarin Gal, and Sebastian Farquhar. Semantic uncertainty: Linguistic invariances for uncertainty estimation in natural language generation. arXiv preprint arXiv:2302.09664, 2023

    arXiv 2023

  6. [6]

    Codemonkeys: Scaling test-time compute for software engineering, 2025

    Ryan Ehrlich, Bradley Brown, Jordan Juravsky, Ronald Clark, Christopher Ré, and Azalia Mirhoseini. Codemonkeys: Scaling test-time compute for software engineering, 2025. URL https://arxiv.org/abs/2501.14723

    arXiv 2025

  7. [7]

    Gold-medalist performance in solving olympiad geometry with alphageometry2

    Yuri Chervonyi, Trieu H Trinh, Miroslav Ol s \'a k, Xiaomeng Yang, Hoang Nguyen, Marcelo Menegali, Junehyuk Jung, Vikas Verma, Quoc V Le, and Thang Luong. Gold-medalist performance in solving olympiad geometry with alphageometry2. arXiv preprint arXiv:2502.03544, 2025

    arXiv 2025

  8. [8]

    Teaching large language models to reason with reinforcement learning, 2024

    Alex Havrilla, Yuqing Du, Sharath Chandra Raparthy, Christoforos Nalmpantis, Jane Dwivedi-Yu, Maksym Zhuravinskyi, Eric Hambro, Sainbayar Sukhbaatar, and Roberta Raileanu. Teaching large language models to reason with reinforcement learning, 2024. URL https://arxiv.org/abs/2403.04642

    arXiv 2024

Show all 43 references
  1. [9]

    Diverse beam search: Decoding diverse solutions from neural sequence models

    Ashwin K Vijayakumar, Michael Cogswell, Ramprasath R Selvaraju, Qing Sun, Stefan Lee, David Crandall, and Dhruv Batra. Diverse beam search: Decoding diverse solutions from neural sequence models. arXiv preprint arXiv:1610.02424, 2016

    arXiv

  2. [10]

    Instruction tuning for large language models: A survey

    Shengyu Zhang, Linfeng Dong, Xiaoya Li, Sen Zhang, Xiaofei Sun, Shuhe Wang, Jiwei Li, Runyi Hu, Tianwei Zhang, Fei Wu, et al. Instruction tuning for large language models: A survey. arXiv preprint arXiv:2308.10792, 2023

    2023

  3. [11]

    Chain-of-thought prompting elicits reasoning in large language models, 2023

    Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Brian Ichter, Fei Xia, Ed Chi, Quoc Le, and Denny Zhou. Chain-of-thought prompting elicits reasoning in large language models, 2023. URL https://arxiv.org/abs/2201.11903

    2023 arXiv

  4. [12]

    Measuring mathematical problem solving with the math dataset

    Dan Hendrycks, Collin Burns, Saurav Kadavath, Akul Arora, Steven Basart, Eric Tang, Dawn Song, and Jacob Steinhardt. Measuring mathematical problem solving with the math dataset. arXiv preprint arXiv:2103.03874, 2021 a

    2021 arXiv

  5. [13]

    Livecodebench: Holistic and contamination free evaluation of large language models for code

    Naman Jain, King Han, Alex Gu, Wen-Ding Li, Fanjia Yan, Tianjun Zhang, Sida Wang, Armando Solar-Lezama, Koushik Sen, and Ion Stoica. Livecodebench: Holistic and contamination free evaluation of large language models for code. arXiv preprint arXiv:2403.07974, 2024

    2024 arXiv

  6. [14]

    Reddit. Reddit. https://www.reddit.com

  7. [15]

    A neural probabilistic language model

    Yoshua Bengio, R \'e jean Ducharme, and Pascal Vincent. A neural probabilistic language model. Advances in neural information processing systems, 13, 2000

    2000

  8. [16]

    Language models are unsupervised multitask learners

    Alec Radford, Jeffrey Wu, Rewon Child, David Luan, Dario Amodei, Ilya Sutskever, et al. Language models are unsupervised multitask learners. OpenAI blog, 1 0 (8): 0 9, 2019

    2019

  9. [17]

    Reinforcement learning: An introduction

    Richard S Sutton. Reinforcement learning: An introduction. A Bradford Book, 2018

    2018

  10. [18]

    Deepseek-r1: Incentivizing reasoning capability in llms via reinforcement learning, 2025

    DeepSeek-AI, Daya Guo, Dejian Yang et al. Deepseek-r1: Incentivizing reasoning capability in llms via reinforcement learning, 2025. URL https://arxiv.org/abs/2501.12948

    2025 arXiv

  11. [19]

    Vineppo: Unlocking rl potential for llm reasoning through refined credit assignment, 2024

    Amirhossein Kazemnejad, Milad Aghajohari, Eva Portelance, Alessandro Sordoni, Siva Reddy, Aaron Courville, and Nicolas Le Roux. Vineppo: Unlocking rl potential for llm reasoning through refined credit assignment, 2024. URL https://arxiv.org/abs/2410.01679

    2024 arXiv

  12. [20]

    Eric Zelikman, Yuhuai Wu, Jesse Mu, and Noah D. Goodman. Star: Bootstrapping reasoning with reasoning, 2022. URL https://arxiv.org/abs/2203.14465

    2022 arXiv

  13. [21]

    The primacy bias in deep reinforcement learning, 2022

    Evgenii Nikishin, Max Schwarzer, Pierluca D'Oro, Pierre-Luc Bacon, and Aaron Courville. The primacy bias in deep reinforcement learning, 2022. URL https://arxiv.org/abs/2205.07802

    2022 arXiv

  14. [22]

    Zhao, Kelvin Guu, Adams Wei Yu, Brian Lester, Nan Du, Andrew M

    Jason Wei, Maarten Bosma, Vincent Y. Zhao, Kelvin Guu, Adams Wei Yu, Brian Lester, Nan Du, Andrew M. Dai, and Quoc V. Le. Finetuned language models are zero-shot learners, 2022. URL https://arxiv.org/abs/2109.01652

    2022 arXiv

  15. [23]

    Direct preference optimization: Your language model is secretly a reward model

    Rafael Rafailov, Archit Sharma, Eric Mitchell, Christopher D Manning, Stefano Ermon, and Chelsea Finn. Direct preference optimization: Your language model is secretly a reward model. Advances in Neural Information Processing Systems, 36: 0 53728--53741, 2023

    2023

  16. [24]

    Gonzalez, and Matei Zaharia

    Alan Zhu, Parth Asawa, Jared Quincy Davis, Lingjiao Chen, Boris Hanin, Ion Stoica, Joseph E. Gonzalez, and Matei Zaharia. Bare: Combining base and instruction-tuned language models for better synthetic data generation, 2025. URL https://arxiv.org/abs/2502.01697

    2025 arXiv

  17. [25]

    Rainbow teaming: Open-ended generation of diverse adversarial prompts

    Mikayel Samvelyan, Sharath Chandra Raparthy, Andrei Lupu, Eric Hambro, Aram H Markosyan, Manish Bhatt, Yuning Mao, Minqi Jiang, Jack Parker-Holder, Jakob Foerster, et al. Rainbow teaming: Open-ended generation of diverse adversarial prompts. arXiv preprint arXiv:2402.16822, 2024

    2024 arXiv

  18. [26]

    Illuminating search spaces by mapping elites

    Jean-Baptiste Mouret and Jeff Clune. Illuminating search spaces by mapping elites. arXiv preprint arXiv:1504.04909, 2015

    2015 arXiv

  19. [27]

    CTRL - A Conditional Transformer Language Model for Controllable Generation

    Nitish Shirish Keskar, Bryan McCann, Lav Varshney, Caiming Xiong, and Richard Socher. CTRL - A Conditional Transformer Language Model for Controllable Generation . arXiv preprint arXiv:1909.05858, 2019

    1909 arXiv

  20. [28]

    Guiding language model reasoning with planning tokens

    Xinyi Wang, Lucas Caccia, Oleksiy Ostapenko, Xingdi Yuan, William Yang Wang, and Alessandro Sordoni. Guiding language model reasoning with planning tokens. In First Conference on Language Modeling, 2024. URL https://openreview.net/forum?id=wi9IffRhVM

    2024

  21. [29]

    Xing, and Zhiting Hu

    Bowen Tan, Zichao Yang, Maruan Al - Shedivat, Eric P. Xing, and Zhiting Hu. Progressive generation of long text. CoRR, abs/2006.15720, 2020. URL https://arxiv.org/abs/2006.15720

    2006 arXiv

  22. [30]

    Plan-and-write: Towards better automatic storytelling, 2019

    Lili Yao, Nanyun Peng, Ralph Weischedel, Kevin Knight, Dongyan Zhao, and Rui Yan. Plan-and-write: Towards better automatic storytelling, 2019. URL https://arxiv.org/abs/1811.05701

    2019 arXiv

  23. [31]

    An Yang, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chengyuan Li, Dayiheng Liu, Fei Huang, Haoran Wei, et al. Qwen2. 5 technical report. arXiv preprint arXiv:2412.15115, 2024

    2024 arXiv

  24. [32]

    Qwen2.5-coder technical report, 2024

    Binyuan Hui, Jian Yang, Zeyu Cui et al. Qwen2.5-coder technical report, 2024. URL https://arxiv.org/abs/2409.12186

    2024 arXiv

  25. [33]

    Ulma: Unified language model alignment with human demonstration and point-wise preference, 2024

    Tianchi Cai, Xierui Song, Jiyan Jiang, Fei Teng, Jinjie Gu, and Guannan Zhang. Ulma: Unified language model alignment with human demonstration and point-wise preference, 2024. URL https://arxiv.org/abs/2312.02554

    2024 arXiv

  26. [34]

    Deep reinforcement learning at the edge of the statistical precipice

    Rishabh Agarwal, Max Schwarzer, Pablo Samuel Castro, Aaron C Courville, and Marc Bellemare. Deep reinforcement learning at the edge of the statistical precipice. Advances in neural information processing systems, 34: 0 29304--29320, 2021

    2021

  27. [35]

    Texygen: A benchmarking platform for text generation models

    Yaoming Zhu, Sidi Lu, Lei Zheng, Jiaxian Guo, Weinan Zhang, Jun Wang, and Yong Yu. Texygen: A benchmarking platform for text generation models. In The 41st International ACM SIGIR Conference on Research & Development in Information Retrieval, SIGIR '18, page 1097–1100, New Yor...

    2018

  28. [36]

    Perspective API : Content moderation attributes and languages, 2024

    Perspective API . Perspective API : Content moderation attributes and languages, 2024. URL https://developers.perspectiveapi.com/s/about-the-api-attributes-and-languages. Accessed: January 29, 2024

    2024

  29. [37]

    A new generation of perspective api: Efficient multilingual character-level transformers

    Alyssa Lees, Vinh Q Tran, Yi Tay, Jeffrey Sorensen, Jai Gupta, Donald Metzler, and Lucy Vasserman. A new generation of perspective api: Efficient multilingual character-level transformers. In Proceedings of the 28th ACM SIGKDD conference on knowledge discovery and data mining,...

    2022

  30. [38]

    Measuring massive multitask language understanding, 2021 b

    Dan Hendrycks, Collin Burns, Steven Basart, Andy Zou, Mantas Mazeika, Dawn Song, and Jacob Steinhardt. Measuring massive multitask language understanding, 2021 b . URL https://arxiv.org/abs/2009.03300

    2021 arXiv

  31. [39]

    The pushshift reddit dataset

    Jason Baumgartner, Savvas Zannettou, Brian Keegan, Megan Squire, and Jeremy Blackburn. The pushshift reddit dataset. In Proceedings of the international AAAI conference on web and social media, volume 14, pages 830--839, 2020

    2020

  32. [40]

    Beautiful soup documentation

    Leonard Richardson. Beautiful soup documentation. April, 2007

    2007

  33. [41]

    Dirt cheap web-scale parallel text from the common crawl

    Jason R Smith, Herve Saint-Amand, Magdalena Plamada, Philipp Koehn, Chris Callison-Burch, and Adam Lopez. Dirt cheap web-scale parallel text from the common crawl. Association for Computational Linguistics, 2013

    2013

  34. [42]

    Huggingface's transformers: State-of-the-art natural language processing

    T Wolf. Huggingface's transformers: State-of-the-art natural language processing. arXiv preprint arXiv:1910.03771, 2019

    1910 arXiv

  35. [43]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 gl...

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.