Pith. sign in

REVIEW 3 major objections 3 minor 299 references

ThinkRetrieve: Retrieval-Augmented Reasoning Traces for Test-Time Scaling

T0 review · 3 major / 3 minor · reviewed 2026-08-12 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read Injecting retrieved solved examples into a reasoning model's trace at each step beats sequential self-reflection on every model and benchmark tested.

desk verdict A thorough, well-controlled study of per-step exemplar retrieval for test-time scaling that deserves review, though the missing prompt-only control leaves the central attribution to retrieval content not fully established. read the letter →

arxiv 2608.10928 v1 pith:E3U2RMTX submitted 2026-08-11 cs.AI

Vaibhav Singh , Soumya Suvra Ghosal , Sarvesh Gharat , Soumyabrata Pal , Ramasuri Narayanam , Dinesh Manocha This is my paper · ORCID
classification cs.AI
keywords test-timescalingretrieval-augmentedgenerationlargereasoningmodelschain-of-thoughtin-contextlearningexemplarretrievalmathematicaldrift
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Sequential test-time scaling, which lets a reasoning model 'think longer' by generating more chain-of-thought tokens, runs into diminishing returns and outright collapse: longer traces drift, repeat themselves, and compound errors. ThinkRetrieve replaces some of that self-reflection with retrieval: at each reasoning step, the model states its current intermediate answer, that answer is encoded together with the original question, and the nearest solved problem from an external corpus is injected into the thinking trace as a worked example. The paper's central claim is that this procedural scaffolding makes test-time compute more effective than self-reflection alone, and that the effect is consistent: ThinkRetrieve improves accuracy on every (model, benchmark) cell across five 1.5B–8B reasoning models and four benchmarks, with the largest gains on the hardest set (up to +13.4 absolute points on AIME 2025). It also keeps accuracy stable or increasing as the thinking budget grows, whereas sequential test-time scaling plateaus or degrades. A sympathetic reader would care because it offers a training-free way to make smaller reasoning models substantially more reliable on hard math problems.

What carries the argument

The central object is the interleaved reasoning–retrieval trajectory $\tau_k = (z_1, e_1, z_2, e_2, \ldots, z_k, e_k)$, where $z_t$ are model-generated reasoning steps and $e_t$ is the solved exemplar retrieved after step $t$. At each step the model is prompted to produce an intermediate answer $y_t$, and the retrieval query is formed by jointly encoding the test question with $y_t$: $q_t = \mathrm{Enc}(x_{\mathrm{test}}, y_t)$. The most similar corpus entry, selected by cosine similarity against pre-encoded question–solution pairs, is formatted as an in-context example and injected into the trace together with a fixed continuation prompt that asks the model to extract key takeaways from the example. Exemplar tokens count against the same token budget $B$ as generated tokens, so ThinkRetrieve produces fewer model-generated tokens than sequential test-time scaling at a matched budget; the design isolates retrieval quality, not extra compute, as the cause of the gains. The loop repeats until the budget is exhausted, and the final answer is generated from the fully augmented trajectory.

What would settle it

Construct a test set of 100 multi-step problems whose solution procedures are absent from the example corpus (for example, novel problem classes or deliberately mismatched domains), run ThinkRetrieve and sequential test-time scaling at matched token budgets, and check whether ThinkRetrieve falls to or below the sequential baseline; if it does not, the claim that retrieval quality drives the gains would be falsified. A more direct probe: corrupt the retrieved exemplar's solution steps while keeping its surface form and measure whether the model adopts the corrupted procedure more often than it corrects it.

Watch

Extended reading notes

Core claim

The paper claims that interleaving retrieved solved examples into a reasoning model's thinking trace changes what test-time compute buys: instead of relying on the model's own self-reflection to extend a line of reasoning, ThinkRetrieve periodically closes the thinking block, asks for an intermediate answer, retrieves a structurally similar solved problem from a large example bank, and reopens thinking with that exemplar and a fixed continuation prompt placed in the trace. The retrieved exemplar is a full question–solution pair, so it demonstrates how to reason, not merely which facts to use. Empirically the claim is that this wins on every (model, benchmark) cell across five models (1.5B–8B) on GSM-8K, MATH-500, AIME 2025, and SciQ, with an absolute gain of up to +13.4 points on AIME 2025 and relative gains up to 60%, while maintaining monotonic or stable accuracy as the token budget grows—in contrast to sequential test-time scaling, which plateaus or collapses. The paper further claims the gains come from reduced answer uncertainty: predictive entropy drops by roughly 0.55 nats, per-step confidence decreases monotonically under ThinkRetrieve, and a per-problem help/hurt decomposition shows the method helps far more than it hurts, with help concentrated on hard, procedurally transferable problems.

Load-bearing premise

The method's gains depend on dense cosine similarity, computed with a pre-trained sentence encoder over a query formed by concatenating the test question with the model's intermediate answer, reliably retrieving a solved example whose solution procedure transfers to the test problem; when retrieval selects a misleading exemplar, the injected trace can anchor the model on a confident wrong answer, a failure mode the paper acknowledges in its Limitations.

Editorial extensions

If this is right

  • Sequential test-time scaling is not the only way to spend inference compute: grounding the trace with retrieved worked examples yields stable, sometimes monotonic gains where self-reflection plateaus or collapses, such as DeepSeek-R1-1.5B on GSM-8K staying at 84% while sequential TTS falls from 83% to 52%.
  • Both design choices matter: per-step in-trace injection is necessary (static input-level ICL underperforms) and semantic retrieval relevance is necessary (random per-step retrieval underperforms).
  • At matched total token budget, a single ThinkRetrieve pass beats TTS with self-consistency majority voting for k in {2,4,8} on MATH-500 hard problems, ruling out compute alone as the explanation.
  • The method preferentially helps hard, procedurally rich problems (level 5, Intermediate Algebra) and is neutral on already-saturated subjects, meaning it adds guidance where the model is uncertain without destabilizing competent reasoning.
  • The gains survive an answer-leakage control: excluding corpus entries whose final answer matches the gold answer does not reduce accuracy, so the benefit reflects structural and procedural similarity, not answer copying.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The low-entropy anchoring that drives the gains is a double-edged sword: if a retrieved exemplar is subtly wrong in a load-bearing way, the model may lock in a confident wrong answer. A testable extension would inject deliberately flawed exemplars and measure whether the model adopts or corrects the flaw.
  • The framework's dependence on a matched example corpus suggests that the transfer bottleneck is corpus construction, not model size; building high-coverage procedural corpora for code generation or formal proof could extend the method to non-math domains, a direction the paper flags as open.
  • Because exemplar tokens occupy the same budget as generated tokens, the method implicitly trades generation for retrieval; an adaptive policy that decides when to retrieve versus when to continue thinking could outperform a fixed per-step schedule, but the paper does not explore this.
  • The paper evaluates only open-weight models up to 8B; the mechanism may behave differently at scales where sequential self-reflection is more reliable, and the paper itself notes that even Qwen3-32B shows non-monotonic TTS returns, leaving open whether ThinkRetrieve's advantage persists at larger scale.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 3 minor

Summary. The paper proposes ThinkRetrieve, a test-time scaling framework for large reasoning models. Rather than extending a reasoning trace by sequential self-reflection alone, ThinkRetrieve iteratively closes the think block, prompts the model for an intermediate answer, encodes it together with the original question, retrieves the nearest solved exemplar from an external corpus, and injects that exemplar into the trace together with a fixed continuation prompt. The procedure repeats until a token budget is exhausted. Experiments cover five reasoning models and four benchmarks, comparing against standard thinking, sequential test-time scaling, static input-level ICL, and random per-step retrieval, with additional ablations on retrieval encoder, query formulation, self-consistency at matched compute, and answer-distinct filtering. The paper reports that ThinkRetrieve wins on every (model, benchmark) cell, with gains up to +13.4 absolute points on AIME 2025, and that retrieved exemplars reduce predictive entropy and per-step confidence reversal.

Significance. If the central attribution holds, the contribution is practically significant: it provides a lightweight, model-agnostic intervention that makes additional test-time tokens more effective than self-reflection alone across several open reasoning model families. The paper deserves credit for including multiple controls that go beyond a single baseline: random per-step retrieval, static ICL, compute-matched self-consistency, an answer-distinct retrieval filter, per-problem help/hurt tests, and a top-1 exemplar audit. These controls jointly make a credible case that the gains are not merely answer leakage or extra compute. However, the causal role of the retrieved exemplar content is not yet fully isolated, because no condition applies the continuation prompt alone; the absence of that control is the main load-bearing gap. The manuscript also overstates certainty in small-sample cells without error bars.

major comments (3)
  1. [§3.2.1, Appendix K] The central attribution claim is not established because no condition applies the continuation prompt c without a retrieved exemplar. The prompt template in Appendix K is a strong, domain-specific instruction: 'Wait, let me check again... Here is a similar question and its solution... Now coming back to the original question...' This prompt is used in ThinkRetrieve and in the Rand baseline, but not in sequential TTS or static ICL, which use different continuation instructions. The reported gains over sequential TTS therefore conflate two variables: the content of the retrieved exemplar and the directive to re-read and extract key points. The Rand baseline cannot resolve this, because random exemplars may actively confuse the model and mask a prompt-driven improvement. The authors should add a prompt-only control that inserts c at each step boundary with no exemplar, or with a neutral placeholder, while keeping all other mechanics fixed. If prompt-only already recovers most of the gains on AIME 2025 and MATH-500, the paper's central claim is unsupported; if it does not, the attribution to exemplar content is validated.
  2. [Table 1, Fig. 4] The headline result reports best accuracy across thinking budgets, averaged over three seeds, without standard errors, confidence intervals, or per-budget seed-level values. This is especially consequential for AIME 2025, which has only 30 problems: a difference such as Qwen3-8B's 71.1 versus 68.9 is less than one problem per seed and cannot be distinguished from noise on this evidence. The claim that ThinkRetrieve wins on every (model, benchmark) cell needs uncertainty quantification, such as binomial confidence intervals or paired significance tests, at least for the small-AIME and near-saturated cells. In addition, selecting the best budget post hoc from a grid of budgets can inflate measured gains; the paper should report results at a fixed budget or otherwise correct for this selection, and should show the underlying per-seed numbers.
  3. [§4, corpus decontamination and Table 4] The leakage audit is in part circular. After removing all corpus entries with cosine similarity above 0.90, the maximum retained similarity is necessarily below 0.90, so the statement that 'zero queries exceed the decontamination threshold' is true by construction and cannot by itself validate the threshold. The relevant evidence would be the distribution of similarities to the unfiltered corpus and an assessment of whether pairs just below 0.90, in the 0.88-0.90 band, are genuine near-duplicates or structurally analogous but mathematically distinct problems. Section 5.1 provides a few qualitative examples, but the paper should report unfiltered similarity statistics and either justify the 0.90 threshold independently of the observed maximum or provide a systematic audit of the band just below it.
minor comments (3)
  1. [Table 3, Appendix E] The compute-matched self-consistency comparison is restricted to a stratified subset of MATH-500 levels 4 and 5 and reports single point estimates without confidence intervals; because this comparison is used to rule out 'compute alone' as the explanation, the appendix should state the subset size and report variance or a paired test.
  2. [§4, Tables 1 and 2] The paper says results are averaged over three seeds and reports 'best accuracy across budgets', but it does not specify how many budget values were evaluated per cell or how ties across budgets were broken; this should be stated so that the oracle-selection issue is transparent.
  3. [Appendix H] The answer-distinct retrieval ablation reports that accuracy is 'essentially unchanged' and 'marginally higher' without giving the actual numbers; the per-seed accuracies for the filtered and unfiltered runs should be reported explicitly, since this control is central to ruling out answer leakage.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the reported gains are measured against external benchmarks and are not derived from fitted parameters or load-bearing self-citations.

full rationale

ThinkRetrieve's central claim—that dynamically injecting retrieved solved exemplars into reasoning traces improves test-time scaling—is evaluated on external benchmarks (GSM-8K, MATH-500, AIME 2025, SciQ) against sequential TTS, static input-level ICL, and random per-step retrieval. No parameter of the method is fitted to the evaluation data; the retrieval encoder, corpus, thinking budget, and prompt template are fixed before evaluation. The decontamination threshold is a preprocessing choice, and the post-hoc leakage audit is a sanity check, not a prediction derived from the method's own outputs. The paper's self-citations (Ghosal et al. 2025a and 2025b) motivate diminishing returns and demonstration retrieval but are not load-bearing: the same claims are supported by independent citations (Gema et al., Yue et al., Shojaee et al.) and by the paper's own Qwen3-32B non-monotonic results in Appendix D. The absence of a prompt-only control (Appendix K's continuation prompt without any exemplar) is a potential experimental confound, but it is not circularity: the paper does not define ThinkRetrieve's gains in terms of that prompt, and the Rand baseline, which uses the same prompt, underperforms relevant retrieval. The reported win-on-every-cell result is therefore not equivalent to its inputs by construction.

Assumptions & free parameters 3 free parameters · 5 assumptions · 0 invented entities

The central claim is empirical and depends on retrieval relevance, corpus quality, and model instruction-following rather than on a fitted law. The only numeric lever set by hand is the decontamination threshold and the corpus filters; the rest of the ledger consists of domain assumptions about embeddings, intermediate answers, and corpus coverage.

free parameters (3)
  • Decontamination cosine similarity threshold = 0.90
    Used to remove NuminaMath entries with E5-Large QA-QA similarity above 0.90 to any evaluation query; the post-hoc audit reports maximum retained similarity of 0.898, just below the threshold, indicating the threshold was set close to the observed boundary.
  • Corpus quality filters = solution length >=100 chars, problem length >=50 chars, question_type='math-word-problem'
    Ad hoc filters applied to NuminaMath-1.5 to construct the 309,609-example bank; these choices affect retrieval quality and are not derived from first principles.
  • Top-1 retrieval per step = 1
    The framework retrieves exactly one exemplar per step (Eq. 6); no ablation varies the number of retrieved exemplars, so the choice is a fixed design decision.
assumptions (5)
  • domain assumption E5-Large cosine similarity between QA-pair embeddings is a valid proxy for procedural/structural relevance of solved examples.
    Both corpus construction (Section 4) and retrieval (Eq. 5-6) rely on this; if embeddings capture surface similarity instead of solution transfer, retrieved exemplars can mislead.
  • domain assumption The intermediate answer elicited by appending </think> and 'Final Answer:' reflects the model's current solution state well enough to serve as a retrieval query.
    Eq. 4 and Eq. 5 use y_t as the retrieval query; no direct evaluation of query quality is provided beyond the Q-Q vs QA-QA ablation.
  • domain assumption The decontamination pipeline removes all answer-level leakage while retaining useful structural analogs.
    Section 4 and Appendices G-H provide audits and a control, but the authors acknowledge in Section 7 that latent structural similarities cannot be exhaustively ruled out.
  • domain assumption A domain-matched external corpus of solved problems is available for the target task.
    NuminaMath is used for math and the SciQ training split for SciQ; the paper states that cross-domain robustness without a matched corpus is untested (Section 7).
  • domain assumption Injected exemplars and the continuation prompt are understood by the model as guidance for the original problem.
    The method depends on in-context learning from retrieved exemplars; no training or fine-tuning is used.

how reviews work

0 comments
Cite this review

Pith. "Pith review of ThinkRetrieve: Retrieval-Augmented Reasoning Traces for Test-Time Scaling." pith.science (2026). https://pith.science/paper/E3U2RMTX

@misc{pith2026260810928,
  author       = {Pith},
  title        = {Pith review of: ThinkRetrieve: Retrieval-Augmented Reasoning Traces for Test-Time Scaling},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/E3U2RMTX}},
  note         = {Machine review of arXiv:2608.10928}
}
abstract

Large Reasoning Models (LRMs) improve performance by allocating additional inference-time compute to generate extended chain-of-thought reasoning. However, recent studies reveal that sequential test-time scaling often yields diminishing or even negative returns, as longer traces exhibit increased uncertainty, error compounding, and drift from the original problem. We propose ThinkRetrieve, a test-time scaling framework that augments the reasoning traces of LRMs with dynamically retrieved solved examples at each reasoning step. Given an external corpus of problems paired with step-by-step solutions, ThinkRetrieve retrieves relevant exemplars at each intermediate step and injects them directly into the thinking trace, providing the model with guidance on how to reason rather than merely what facts are relevant. Experiments across five reasoning models (1.5B--8B parameters) on GSM-8K, MATH-500, AIME 2025, and SciQ demonstrate that ThinkRetrieve consistently improves accuracy over standard test-time scaling, with relative gains of up to $60\%$ on AIME 2025.

Figures

Figures reproduced from arXiv: 2608.10928 by the authors.

Figure 1
Figure 1. Comparison of sequential test-time scaling ( [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 2
Figure 2. ThinkRetrieve improves over sequential TTS on every (model, benchmark) cell. Accuracy gain (%) of ThinkRetrieve over sequential test-time scaling for each of the five models, on GSM-8K, MATH-500, and AIME 2025. Every bar is positive, with the largest gains on the hardest benchmark (up to +13.4 on AIME 2025 for Qwen3-1.7B). Absolute numbers for all methods and baselines in [PITH_FULL_IMAGE:figures/full_fig_p006_2.png] view at source ↗
Figure 3
Figure 3. Qualitative comparison on a MATH-500 problem. [PITH_FULL_IMAGE:figures/full_fig_p007_3.png] view at source ↗
Figures from the paper (5 more)
Figure 4
Figure 4. Figure 4: Test-time scaling behavior across models and benchmarks. [PITH_FULL_IMAGE:figures/full_fig_p008_4.png]
Figure 5
Figure 5. Figure 5: Predictive entropy and accuracy on a stratified subset of MATH-500 with Qwen3-1.7B (B=8K). ThinkRe￾trieve lowers mean predictive entropy from 1.51 to 0.96 nats (left) while raising accuracy from 33% to 47% (right), indicating more confident and more correct answers. Vi…
Figure 6
Figure 6. Figure 6: Ablation on retrieval encoder. ThinkRetrieve across three retrieval encoders (BERT, E5-Base, E5-Large) against standard thinking and an input-level RAG baseline on MATH-500 with Qwen3-1.7B. All in-trace variants achieve consistent gains with minimal sensitivity to the …
Figure 7
Figure 7. Figure 7: Ablation on retrieval query formulation. [PITH_FULL_IMAGE:figures/full_fig_p024_7.png]
Figure 8
Figure 8. Figure 8: Qualitative comparison on a geometry problem from MATH-500. [PITH_FULL_IMAGE:figures/full_fig_p026_8.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

299 extracted references · 4 canonical work pages

  1. [1]

    Proceedings of the 3rd Workshop on Noisy User-generated Text , pages =

    Crowdsourcing Multiple Choice Science Questions , author =. Proceedings of the 3rd Workshop on Noisy User-generated Text , pages =. 2017 , url =

    2017

  2. [2]

    Aho and Jeffrey D

    Alfred V. Aho and Jeffrey D. Ullman , title =. 1972

    1972

  3. [3]

    Publications Manual , year = "1983", publisher =

    1983

  4. [4]

    Chandra and Dexter C

    Ashok K. Chandra and Dexter C. Kozen and Larry J. Stockmeyer , year = "1981", title =. doi:10.1145/322234.322243

    arXiv 1981

  5. [5]

    Scalable training of

    Andrew, Galen and Gao, Jianfeng , booktitle=. Scalable training of

  6. [6]

    Dan Gusfield , title =. 1997

    1997

  7. [7]

    Tetreault , title =

    Mohammad Sadegh Rasooli and Joel R. Tetreault , title =. Computing Research Repository , volume =. 2015 , url =

    2015

  8. [8]

    A Framework for Learning Predictive Structures from Multiple Tasks and Unlabeled Data , Volume =

    Ando, Rie Kubota and Zhang, Tong , Issn =. A Framework for Learning Predictive Structures from Multiple Tasks and Unlabeled Data , Volume =. Journal of Machine Learning Research , Month = dec, Numpages =

Show all 299 references
  1. [9]

    Billion-scale similarity search with

    Johnson, Jeff and Douze, Matthijs and J. Billion-scale similarity search with. IEEE Transactions on Big Data , volume=. 2019 , publisher=

    2019

  2. [10]

    arXiv preprint arXiv:2212.03533 , year=

    Text Embeddings by Weakly-Supervised Contrastive Pre-training , author=. arXiv preprint arXiv:2212.03533 , year=

    arXiv

  3. [11]

    Hugging Face repository , howpublished =

    Jia LI and Edward Beeching and Lewis Tunstall and Ben Lipkin and Roman Soletskyi and Shengyi Costa Huang and Kashif Rasul and Longhui Yu and Albert Jiang and Ziju Shen and Zihan Qin and Bin Dong and Li Zhou and Yann Fleureau and Guillaume Lample and Stanislas Polu , title =. H...

    2024

  4. [12]

    2002 , publisher=

    Learning with kernels: support vector machines, regularization, optimization, and beyond , author=. 2002 , publisher=

    2002

  5. [13]

    1999 , publisher=

    Elements of information theory , author=. 1999 , publisher=

    1999

  6. [14]

    2006 , publisher=

    Optimal design of experiments , author=. 2006 , publisher=

    2006

  7. [15]

    The Thirty-ninth Annual Conference on Neural Information Processing Systems , year=

    Chain-of-Retrieval Augmented Generation , author=. The Thirty-ninth Annual Conference on Neural Information Processing Systems , year=

  8. [16]

    Advances in Neural Information Processing Systems 33 , year =

    Language Models are Few-Shot Learners , author =. Advances in Neural Information Processing Systems 33 , year =

  9. [17]

    Proceedings of the 3rd Workshop on Deep Learning for Low-Resource Natural Language Processing (DeepLo) at ACL , year =

    What Makes Good In-Context Examples for GPT-3? , author =. Proceedings of the 3rd Workshop on Deep Learning for Low-Resource Natural Language Processing (DeepLo) at ACL , year =

  10. [18]

    Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing , year =

    Rethinking the Role of Demonstrations: What Makes In-Context Learning Work? , author =. Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing , year =

    2022

  11. [19]

    arXiv preprint arXiv:1704.05426 , year=

    A broad-coverage challenge corpus for sentence understanding through inference , author=. arXiv preprint arXiv:1704.05426 , year=

    arXiv

  12. [20]

    arXiv preprint arXiv:2506.16502 , year=

    Relic: Enhancing Reward Model Generalization for Low-Resource Indic Languages with Few-Shot Examples , author=. arXiv preprint arXiv:2506.16502 , year=

    arXiv

  13. [21]

    PromptRefine: Enhancing Few-Shot Performance on Low-Resource Indic Languages with Example Selection from related Example Banks , author=. Proceedings of the 2025 Conference of the Nations of the Americas Chapter of the Association for Computational Linguistics: Human Language ...

    2025

  14. [22]

    Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics , year =

    Learning to Retrieve Prompts for In-Context Learning , author =. Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics , year =

    2022

  15. [23]

    arXiv preprint arXiv:2503.23513 , year=

    Rare: Retrieval-augmented reasoning modeling , author=. arXiv preprint arXiv:2503.23513 , year=

    arXiv

  16. [24]

    arXiv preprint arXiv:2503.24235 , year=

    A survey on test-time scaling in large language models: What, how, where, and how well? , author=. arXiv preprint arXiv:2503.24235 , year=

    arXiv

  17. [25]

    arXiv preprint arXiv:2312.10997 , volume=

    Retrieval-augmented generation for large language models: A survey , author=. arXiv preprint arXiv:2312.10997 , volume=

    arXiv

  18. [26]

    Advances in neural information processing systems , volume=

    Retrieval-augmented generation for knowledge-intensive nlp tasks , author=. Advances in neural information processing systems , volume=

  19. [27]

    Advances in Neural Information Processing Systems 36 , year =

    Large Language Models are Latent Variable Models: Explaining and Finding Good Demonstrations for In-Context Learning , author =. Advances in Neural Information Processing Systems 36 , year =

  20. [28]

    Proceedings of the 2024 conference on empirical methods in natural language processing , pages=

    A survey on in-context learning , author=. Proceedings of the 2024 conference on empirical methods in natural language processing , pages=

    2024

  21. [29]

    icl: Demonstration-retrieved in-context learning , author=

    Dr. icl: Demonstration-retrieved in-context learning , author=. arXiv preprint arXiv:2305.14128 , year=

    arXiv

  22. [30]

    Proceedings of the 2023 Conference of the Association for Computational Linguistics , year =

    Perplexity-Based Prompt Selection for Large Language Models , author =. Proceedings of the 2023 Conference of the Association for Computational Linguistics , year =

    2023

  23. [31]

    2023 , eprint =

    In-Context Learning Demonstration Selection via Influence Analysis , author =. 2023 , eprint =

    2023

  24. [32]

    Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing , year =

    Representative Demonstration Selection for In-Context Learning with Two-Stage Determinantal Point Process , author =. Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing , year =

    2023

  25. [33]

    Proceedings of the 40th International Conference on Machine Learning , year =

    Compositional Exemplars for In-Context Learning , author =. Proceedings of the 40th International Conference on Machine Learning , year =

  26. [34]

    arXiv preprint arXiv:1102.3975 , year=

    Submodular meets spectral: Greedy algorithms for subset selection, sparse approximation and dictionary selection , author=. arXiv preprint arXiv:1102.3975 , year=

    arXiv

  27. [35]

    Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies , pages=

    Harnessing Multilinguality in Unsupervised Machine Translation for Rare Languages , author=. Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies , pages=

    2021

  28. [36]

    arXiv preprint arXiv:2105.08306 , year=

    Sample efficient linear meta-learning by alternating minimization , author=. arXiv preprint arXiv:2105.08306 , year=

    arXiv

  29. [37]

    arXiv preprint arXiv:2404.16816 , year=

    IndicGenBench: A Multilingual Benchmark to Evaluate Generation Capabilities of LLMs on Indic Languages , author=. arXiv preprint arXiv:2404.16816 , year=

    arXiv

  30. [38]

    arXiv preprint arXiv:2311.07463 , year=

    Megaverse: Benchmarking large language models across languages, modalities, models and tasks , author=. arXiv preprint arXiv:2311.07463 , year=

    arXiv

  31. [39]

    arXiv preprint arXiv:1702.01806 , year=

    Beam search strategies for neural machine translation , author=. arXiv preprint arXiv:1702.01806 , year=

    arXiv

  32. [40]

    arXiv preprint arXiv:1904.09751 , year=

    The curious case of neural text degeneration , author=. arXiv preprint arXiv:1904.09751 , year=

    1904 arXiv

  33. [41]

    arXiv preprint arXiv:1805.04833 , year=

    Hierarchical neural story generation , author=. arXiv preprint arXiv:1805.04833 , year=

    arXiv

  34. [42]

    arXiv preprint arXiv:2112.08633 , year=

    Learning to retrieve prompts for in-context learning , author=. arXiv preprint arXiv:2112.08633 , year=

    arXiv

  35. [43]

    arXiv preprint arXiv:2101.06804 , year=

    What Makes Good In-Context Examples for GPT- 3 ? , author=. arXiv preprint arXiv:2101.06804 , year=

    arXiv

  36. [44]

    arXiv preprint arXiv:2212.10375 , year=

    Self-adaptive in-context learning: An information compression perspective for in-context example selection and ordering , author=. arXiv preprint arXiv:2212.10375 , year=

    arXiv

  37. [45]

    International Conference on Machine Learning , pages=

    Compositional exemplars for in-context learning , author=. International Conference on Machine Learning , pages=. 2023 , organization=

    2023

  38. [46]

    arXiv preprint arXiv:2407.10671 , year=

    Qwen2 technical report , author=. arXiv preprint arXiv:2407.10671 , year=

    arXiv

  39. [47]

    arXiv preprint arXiv:2307.09288 , year=

    Llama 2: Open foundation and fine-tuned chat models , author=. arXiv preprint arXiv:2307.09288 , year=

    arXiv

  40. [48]

    arXiv preprint arXiv:2407.21783 , year=

    The llama 3 herd of models , author=. arXiv preprint arXiv:2407.21783 , year=

    arXiv

  41. [49]

    arXiv preprint arXiv:2404.14219 , year=

    Phi-3 technical report: A highly capable language model locally on your phone , author=. arXiv preprint arXiv:2404.14219 , year=

    arXiv

  42. [50]

    Advances in Applied Probability , volume=

    The coincidence approach to stochastic point processes , author=. Advances in Applied Probability , volume=. 1975 , publisher=

    1975

  43. [51]

    Communications in Mathematical Physics , volume=

    Distributions on Partitions, Point Processes, and the Hypergeometric Kernel , author=. Communications in Mathematical Physics , volume=. 2000 , publisher=

    2000

  44. [52]

    Journal of mathematical physics , volume=

    Detection and``emission''processes of quantum particles in a``chaotic state'' , author=. Journal of mathematical physics , volume=. 1973 , publisher=

    1973

  45. [53]

    Foundations and Trends

    Determinantal point processes for machine learning , author=. Foundations and Trends. 2012 , publisher=

    2012

  46. [54]

    International conference on machine learning , pages=

    Model soups: averaging weights of multiple fine-tuned models improves accuracy without increasing inference time , author=. International conference on machine learning , pages=. 2022 , organization=

    2022

  47. [55]

    Qwen2.5: A Party of Foundation Models , url =

    Qwen Team , month =. Qwen2.5: A Party of Foundation Models , url =

  48. [56]

    2024 , eprint=

    GPT-4 Technical Report , author=. 2024 , eprint=

    2024

  49. [57]

    Proceedings of the tenth workshop on statistical machine translation , pages=

    chrF: character n-gram F-score for automatic MT evaluation , author=. Proceedings of the tenth workshop on statistical machine translation , pages=

  50. [58]

    arXiv preprint arXiv:2205.03983 , year=

    Building machine translation systems for the next thousand languages , author=. arXiv preprint arXiv:2205.03983 , year=

    arXiv

  51. [59]

    2023 , eprint=

    IndicTrans2: Towards High-Quality and Accessible Machine Translation Models for all 22 Scheduled Indian Languages , author=. 2023 , eprint=

    2023

  52. [60]

    2022 , eprint=

    Determinantal Point Process Likelihoods for Sequential Recommendation , author=. 2022 , eprint=

    2022

  53. [61]

    2018 , eprint=

    Fast Greedy MAP Inference for Determinantal Point Process to Improve Recommendation Diversity , author=. 2018 , eprint=

    2018

  54. [62]

    Operations Research , volume=

    An exact algorithm for maximum entropy sampling , author=. Operations Research , volume=. 1995 , publisher=

    1995

  55. [63]

    Foundations and Trends

    The probabilistic relevance framework: BM25 and beyond , author=. Foundations and Trends. 2009 , publisher=

    2009

  56. [64]

    arXiv preprint arXiv:2305.18098 , year=

    Bigtranslate: Augmenting large language models with multilingual translation capability over 100 languages , author=. arXiv preprint arXiv:2305.18098 , year=

    arXiv

  57. [65]

    arXiv preprint arXiv:2308.04948 , year=

    Extrapolating large language models to non-english by aligning languages , author=. arXiv preprint arXiv:2308.04948 , year=

    arXiv

  58. [66]

    arXiv preprint arXiv:2402.07827 , year=

    Aya model: An instruction finetuned open-access multilingual language model , author=. arXiv preprint arXiv:2402.07827 , year=

    arXiv

  59. [67]

    arXiv preprint arXiv:2402.00838 , year=

    Olmo: Accelerating the science of language models , author=. arXiv preprint arXiv:2402.00838 , year=

    arXiv

  60. [68]

    arXiv preprint arXiv:2305.15011 , year=

    Bactrian-x: Multilingual replicable instruction-following models with low-rank adaptation , author=. arXiv preprint arXiv:2305.15011 , year=

    arXiv

  61. [69]

    arXiv preprint arXiv:2305.05940 , year=

    Multilingual llms are better cross-lingual in-context learners with alignment , author=. arXiv preprint arXiv:2305.05940 , year=

    arXiv

  62. [70]

    arXiv preprint arXiv:2406.01771 , year=

    LLMs Beyond English: Scaling the Multilingual Capability of LLMs with Cross-Lingual Feedback , author=. arXiv preprint arXiv:2406.01771 , year=

    arXiv

  63. [71]

    arXiv preprint arXiv:2108.13161 , year=

    Differentiable prompt makes pre-trained language models better few-shot learners , author=. arXiv preprint arXiv:2108.13161 , year=

    arXiv

  64. [72]

    arXiv preprint arXiv:2109.07684 , year=

    Language models are few-shot multilingual learners , author=. arXiv preprint arXiv:2109.07684 , year=

    arXiv

  65. [73]

    arXiv preprint arXiv:2305.07004 , year=

    Not all languages are created equal in llms: Improving multilingual capability by cross-lingual-thought prompting , author=. arXiv preprint arXiv:2305.07004 , year=

    arXiv

  66. [74]

    arXiv preprint arXiv:2308.01223 , year=

    Do Multilingual Language Models Think Better in English? , author=. arXiv preprint arXiv:2308.01223 , year=

    arXiv

  67. [75]

    arXiv preprint arXiv:2206.07682 , year=

    Emergent abilities of large language models , author=. arXiv preprint arXiv:2206.07682 , year=

    arXiv

  68. [76]

    arXiv preprint arXiv:2005.14165 , volume=

    Language models are few-shot learners , author=. arXiv preprint arXiv:2005.14165 , volume=

    2005 arXiv

  69. [77]

    arXiv preprint arXiv:2001.09977 , year=

    Towards a human-like open-domain chatbot , author=. arXiv preprint arXiv:2001.09977 , year=

    2001 arXiv

  70. [78]

    Advances in neural information processing systems , volume=

    Superglue: A stickier benchmark for general-purpose language understanding systems , author=. Advances in neural information processing systems , volume=

  71. [79]

    arXiv preprint arXiv:2010.03648 , year=

    A mathematical exploration of why language models help solve downstream tasks , author=. arXiv preprint arXiv:2010.03648 , year=

    2010 arXiv

  72. [80]

    arXiv preprint arXiv:2111.02080 , year=

    An explanation of in-context learning as implicit bayesian inference , author=. arXiv preprint arXiv:2111.02080 , year=

    arXiv

  73. [81]

    Advances in Neural Information Processing Systems , volume=

    Large language models are latent variable models: Explaining and finding good demonstrations for in-context learning , author=. Advances in Neural Information Processing Systems , volume=

  74. [82]

    arXiv preprint arXiv:2402.10738 , year=

    Let's Learn Step by Step: Enhancing In-Context Learning Ability with Curriculum Learning , author=. arXiv preprint arXiv:2402.10738 , year=

    arXiv

  75. [83]

    arXiv preprint arXiv:2205.01703 , year=

    Improving in-context few-shot learning via self-supervised training , author=. arXiv preprint arXiv:2205.01703 , year=

    arXiv

  76. [84]

    arXiv preprint arXiv:2110.15943 , year=

    Metaicl: Learning to learn in context , author=. arXiv preprint arXiv:2110.15943 , year=

    arXiv

  77. [85]

    arXiv preprint arXiv:2305.08298 , year=

    Symbol tuning improves in-context learning in language models , author=. arXiv preprint arXiv:2305.08298 , year=

    arXiv

  78. [86]

    International conference on machine learning , pages=

    Calibrate before use: Improving few-shot performance of language models , author=. International conference on machine learning , pages=. 2021 , organization=

    2021

  79. [87]

    arXiv preprint arXiv:2204.13509 , year=

    On the effect of pretraining corpora on in-context learning by a large-scale language model , author=. arXiv preprint arXiv:2204.13509 , year=

    arXiv

  80. [88]

    arXiv preprint arXiv:2205.12685 , year=

    Ground-truth labels matter: A deeper look into input-label demonstrations , author=. arXiv preprint arXiv:2205.12685 , year=

    arXiv

  81. [89]

    arXiv preprint arXiv:2303.03846 , year=

    Larger language models do in-context learning differently , author=. arXiv preprint arXiv:2303.03846 , year=

    arXiv

  82. [90]

    arXiv preprint arXiv:2209.11895 , year=

    In-context learning and induction heads , author=. arXiv preprint arXiv:2209.11895 , year=

    arXiv

  83. [91]

    International Conference on Machine Learning , pages=

    Transformers as algorithms: Generalization and stability in in-context learning , author=. International Conference on Machine Learning , pages=. 2023 , organization=

    2023

  84. [92]

    2023 , school=

    What in-context learning “learns” in-context: Disentangling task recognition and task learning , author=. 2023 , school=

    2023

  85. [93]

    arXiv preprint arXiv:2212.10559 , year=

    Why can gpt learn in-context? language models implicitly perform gradient descent as meta-optimizers , author=. arXiv preprint arXiv:2212.10559 , year=

    arXiv

  86. [94]

    arXiv preprint arXiv:2104.08786 , year=

    Fantastically ordered prompts and where to find them: Overcoming few-shot prompt order sensitivity , author=. arXiv preprint arXiv:2104.08786 , year=

    arXiv

  87. [95]

    Advances in neural information processing systems , volume=

    Chain-of-thought prompting elicits reasoning in large language models , author=. Advances in neural information processing systems , volume=

  88. [96]

    arXiv preprint arXiv:2212.10560 , year=

    Self-instruct: Aligning language models with self-generated instructions , author=. arXiv preprint arXiv:2212.10560 , year=

    arXiv

  89. [97]

    arXiv preprint arXiv:2211.01910 , year=

    Large language models are human-level prompt engineers , author=. arXiv preprint arXiv:2211.01910 , year=

    arXiv

  90. [98]

    arXiv preprint arXiv:2210.03493 , year=

    Automatic chain of thought prompting in large language models , author=. arXiv preprint arXiv:2210.03493 , year=

    arXiv

  91. [99]

    arXiv preprint arXiv:2212.06800 , year=

    Diverse demonstrations improve in-context compositional generalization , author=. arXiv preprint arXiv:2212.06800 , year=

    arXiv

  92. [100]

    arXiv preprint arXiv:2302.11042 , year=

    In-context example selection with influences , author=. arXiv preprint arXiv:2302.11042 , year=

    arXiv

  93. [101]

    arXiv preprint arXiv:2302.13539 , year=

    Finding support examples for in-context learning , author=. arXiv preprint arXiv:2302.13539 , year=

    arXiv

  94. [102]

    arXiv preprint arXiv:2305.04320 , year=

    Unified demonstration retriever for in-context learning , author=. arXiv preprint arXiv:2305.04320 , year=

    arXiv

  95. [103]

    arXiv preprint arXiv:2211.04486 , year=

    Active example selection for in-context learning , author=. arXiv preprint arXiv:2211.04486 , year=

    arXiv

  96. [104]

    arXiv preprint arXiv:2305.14502 , year=

    Reticl: Sequential retrieval of in-context examples with reinforcement learning , author=. arXiv preprint arXiv:2305.14502 , year=

    arXiv

  97. [105]

    arXiv preprint arXiv:2310.09881 , year=

    In-context learning with iterative demonstration selection , author=. arXiv preprint arXiv:2310.09881 , year=

    arXiv

  98. [106]

    arXiv preprint arXiv:2407.06057 , year=

    Variational best-of-n alignment , author=. arXiv preprint arXiv:2407.06057 , year=

    arXiv

  99. [107]

    arXiv preprint arXiv:2401.01879 , year=

    Theoretical guarantees on the best-of-n alignment policy , author=. arXiv preprint arXiv:2401.01879 , year=

    arXiv

  100. [108]

    arXiv preprint arXiv:2505.18822 , year=

    AdaCtrl: Towards Adaptive and Controllable Reasoning via Difficulty-Aware Budgeting , author=. arXiv preprint arXiv:2505.18822 , year=

  101. [109]

    lilianweng.github.io , year =

    Why We Think , author =. lilianweng.github.io , year =

  102. [110]

    arXiv preprint arXiv:2505.15400 , year=

    When to Continue Thinking: Adaptive Thinking Mode Switching for Efficient Reasoning , author=. arXiv preprint arXiv:2505.15400 , year=

    arXiv

  103. [111]

    arXiv preprint arXiv:2505.14631 , year=

    Think Only When You Need with Large Hybrid-Reasoning Models , author=. arXiv preprint arXiv:2505.14631 , year=

    arXiv

  104. [112]

    arXiv preprint arXiv:2505.14183 , year=

    ThinkSwitcher: When to Think Hard, When to Think Fast , author=. arXiv preprint arXiv:2505.14183 , year=

    arXiv

  105. [113]

    arXiv preprint arXiv:2505.13417 , year=

    AdaptThink: Reasoning Models Can Learn When to Think , author=. arXiv preprint arXiv:2505.13417 , year=

    arXiv

  106. [114]

    arXiv preprint arXiv:2505.13379 , year=

    Thinkless: LLM Learns When to Think , author=. arXiv preprint arXiv:2505.13379 , year=

    arXiv

  107. [115]

    arXiv preprint arXiv:2504.07128 , year=

    DeepSeek-R1 Thoughtology: Let's< think> about LLM Reasoning , author=. arXiv preprint arXiv:2504.07128 , year=

  108. [116]

    2025 , eprint=

    Towards Thinking-Optimal Scaling of Test-Time Compute for LLM Reasoning , author=. 2025 , eprint=

    2025

  109. [117]

    5: Scaling reinforcement learning with llms , author=

    Kimi k1. 5: Scaling reinforcement learning with llms , author=. arXiv preprint arXiv:2501.12599 , year=

    arXiv

  110. [118]

    arXiv preprint arXiv:2503.09516 , year=

    Search-r1: Training llms to reason and leverage search engines with reinforcement learning , author=. arXiv preprint arXiv:2503.09516 , year=

    arXiv

  111. [119]

    arXiv preprint arXiv:2503.06749 , year=

    Vision-r1: Incentivizing reasoning capability in multimodal large language models , author=. arXiv preprint arXiv:2503.06749 , year=

    arXiv

  112. [120]

    arXiv preprint arXiv:2503.14476 , year=

    Dapo: An open-source llm reinforcement learning system at scale , author=. arXiv preprint arXiv:2503.14476 , year=

    arXiv

  113. [121]

    arXiv preprint arXiv:2504.07615 , year=

    Vlm-r1: A stable and generalizable r1-style large vision-language model , author=. arXiv preprint arXiv:2504.07615 , year=

    arXiv

  114. [122]

    arXiv preprint arXiv:2503.01785 , year=

    Visual-rft: Visual reinforcement fine-tuning , author=. arXiv preprint arXiv:2503.01785 , year=

    arXiv

  115. [123]

    QwQ-32B: Embracing the Power of Reinforcement Learning , url =

    Qwen Team , month =. QwQ-32B: Embracing the Power of Reinforcement Learning , url =

  116. [125]

    2024 , eprint=

    DeepSeek-V3 Technical Report , author=. 2024 , eprint=

    2024

  117. [126]

    arXiv preprint arXiv:2410.01707 , year=

    Interpretable contrastive monte carlo tree search reasoning , author=. arXiv preprint arXiv:2410.01707 , year=

    arXiv

  118. [127]

    arXiv preprint arXiv:2411.16489 , year=

    O1 Replication Journey--Part 2: Surpassing O1-preview through Simple Distillation, Big Progress or Bitter Lesson? , author=. arXiv preprint arXiv:2411.16489 , year=

    arXiv

  119. [128]

    arXiv preprint arXiv:2412.00154 , year=

    o1-coder: an o1 replication for coding , author=. arXiv preprint arXiv:2412.00154 , year=

    arXiv

  120. [129]

    arXiv preprint arXiv:2411.14405 , year=

    Marco-o1: Towards open reasoning models for open-ended solutions , author=. arXiv preprint arXiv:2411.14405 , year=

    arXiv

  121. [130]

    arXiv e-prints , pages=

    Drt-o1: Optimized deep reasoning translation via long chain-of-thought , author=. arXiv e-prints , pages=

  122. [131]

    arXiv preprint arXiv:2410.18982 , year=

    O1 Replication Journey: A Strategic Progress Report--Part 1 , author=. arXiv preprint arXiv:2410.18982 , year=

    arXiv

  123. [132]

    arXiv preprint arXiv:2410.02884 , year=

    Llama-berry: Pairwise optimization for o1-like olympiad-level mathematical reasoning , author=. arXiv preprint arXiv:2410.02884 , year=

    arXiv

  124. [133]

    arXiv preprint arXiv:2410.09671 , year=

    Openr: An open source framework for advanced reasoning with large language models , author=. arXiv preprint arXiv:2410.09671 , year=

    arXiv

  125. [134]

    arXiv preprint arXiv:2501.09686 , year=

    Towards Large Reasoning Models: A Survey of Reinforced Reasoning with Large Language Models , author=. arXiv preprint arXiv:2501.09686 , year=

    arXiv

  126. [135]

    arXiv preprint arXiv:2504.01296 , year=

    Thinkprune: Pruning long chain-of-thought of llms via reinforcement learning , author=. arXiv preprint arXiv:2504.01296 , year=

    arXiv

  127. [136]

    arXiv preprint arXiv:2409.12917 , year=

    Training language models to self-correct via reinforcement learning , author=. arXiv preprint arXiv:2409.12917 , year=

    arXiv

  128. [137]

    arXiv preprint arXiv:2403.09629 , year=

    Quiet-star: Language models can teach themselves to think before speaking , author=. arXiv preprint arXiv:2403.09629 , year=

    arXiv

  129. [138]

    arXiv preprint arXiv:2407.06023 , year=

    Distilling system 2 into system 1 , author=. arXiv preprint arXiv:2407.06023 , year=

    arXiv

  130. [139]

    arXiv preprint arXiv:2311.11829 , year=

    System 2 attention (is something you might need too) , author=. arXiv preprint arXiv:2311.11829 , year=

    arXiv

  131. [140]

    arXiv preprint arXiv:2402.10200 , year=

    Chain-of-thought reasoning without prompting , author=. arXiv preprint arXiv:2402.10200 , year=

    arXiv

  132. [141]

    Yu, Zishun and Tao, Yunzhe and Chen, Liyu and Sun, Tao and Yang, Hongxia , journal=

  133. [142]

    arXiv preprint arXiv:2105.09938 , year=

    Measuring coding challenge competence with apps , author=. arXiv preprint arXiv:2105.09938 , year=

    arXiv

  134. [143]

    5 technical report , author=

    Qwen2. 5 technical report , author=. arXiv preprint arXiv:2412.15115 , year=

    arXiv

  135. [144]

    arXiv preprint arXiv:2402.06332 , year=

    Internlm-math: Open math large language models toward verifiable reasoning , author=. arXiv preprint arXiv:2402.06332 , year=

    arXiv

  136. [145]

    arXiv preprint arXiv:2310.10631 , year=

    Llemma: An open language model for mathematics , author=. arXiv preprint arXiv:2310.10631 , year=

    arXiv

  137. [146]

    Advances in Neural Information Processing Systems , volume=

    Group robust preference optimization in reward-free rlhf , author=. Advances in Neural Information Processing Systems , volume=

  138. [147]

    arXiv preprint arXiv:2312.06585 , year=

    Beyond human data: Scaling self-training for problem-solving with language models , author=. arXiv preprint arXiv:2312.06585 , year=

    arXiv

  139. [148]

    arXiv preprint arXiv:2308.01825 , year=

    Scaling relationship on learning mathematical reasoning with large language models , author=. arXiv preprint arXiv:2308.01825 , year=

    arXiv

  140. [149]

    Advances in Neural Information Processing Systems , volume=

    Star: Bootstrapping reasoning with reasoning , author=. Advances in Neural Information Processing Systems , volume=

  141. [150]

    Learning to reason with llms , author=

  142. [151]

    Advances in Neural Information Processing Systems , volume=

    Self-evaluation guided beam search for reasoning , author=. Advances in Neural Information Processing Systems , volume=

  143. [152]

    arXiv preprint arXiv:2310.09044 , year=

    KCTS: knowledge-constrained tree search decoding with token-level hallucination detection , author=. arXiv preprint arXiv:2310.09044 , year=

    arXiv

  144. [153]

    arXiv preprint arXiv:2310.04406 , year=

    Language agent tree search unifies reasoning acting and planning in language models , author=. arXiv preprint arXiv:2310.04406 , year=

    arXiv

  145. [154]

    arXiv preprint arXiv:2303.05510 , year=

    Planning with large language models for code generation , author=. arXiv preprint arXiv:2303.05510 , year=

    arXiv

  146. [155]

    arXiv preprint arXiv:2309.15028 , year=

    Don't throw away your value model! Generating more preferable text with Value-Guided Monte-Carlo Tree Search decoding , author=. arXiv preprint arXiv:2309.15028 , year=

    arXiv

  147. [156]

    arXiv preprint arXiv:2410.16377 , year=

    A Simple Model of Inference Scaling Laws , author=. arXiv preprint arXiv:2410.16377 , year=

    arXiv

  148. [157]

    arXiv preprint arXiv:2303.06135 , year=

    Rewarding chatbots for real-world engagement with millions of users , author=. arXiv preprint arXiv:2303.06135 , year=

    arXiv

  149. [158]

    arXiv preprint arXiv:2203.11171 , year=

    Self-consistency improves chain of thought reasoning in language models , author=. arXiv preprint arXiv:2203.11171 , year=

    arXiv

  150. [159]

    arXiv preprint arXiv:2402.03300 , year=

    Deepseekmath: Pushing the limits of mathematical reasoning in open language models , author=. arXiv preprint arXiv:2402.03300 , year=

    arXiv

  151. [160]

    Advances in Neural Information Processing Systems , volume=

    Solving quantitative reasoning problems with language models , author=. Advances in Neural Information Processing Systems , volume=

  152. [161]

    arXiv preprint arXiv:2502.14565 , year=

    Revise: Learning to refine at test-time via intrinsic self-verification , author=. arXiv preprint arXiv:2502.14565 , year=

    arXiv

  153. [162]

    Advances in Neural Information Processing Systems , volume=

    Self-refine: Iterative refinement with self-feedback , author=. Advances in Neural Information Processing Systems , volume=

  154. [163]

    arXiv preprint arXiv:2501.11651 , year=

    Advancing Language Model Reasoning through Reinforcement Learning and Inference Scaling , author=. arXiv preprint arXiv:2501.11651 , year=

    arXiv

  155. [164]

    arXiv preprint arXiv:2501.09891 , year=

    Evolving Deeper LLM Thinking , author=. arXiv preprint arXiv:2501.09891 , year=

    arXiv

  156. [165]

    arXiv preprint arXiv:2309.17179 , year=

    Alphazero-like tree-search can guide large language model decoding and training , author=. arXiv preprint arXiv:2309.17179 , year=

    arXiv

  157. [166]

    arXiv preprint arXiv:2405.00451 , year=

    Monte carlo tree search boosts reasoning via iterative preference learning , author=. arXiv preprint arXiv:2405.00451 , year=

    arXiv

  158. [167]

    Advances in Neural Information Processing Systems , volume=

    Transfer q-star: Principled decoding for LLM alignment , author=. Advances in Neural Information Processing Systems , volume=

  159. [168]

    arXiv preprint arXiv:2501.19393 , year=

    s1: Simple test-time scaling , author=. arXiv preprint arXiv:2501.19393 , year=

    arXiv

  160. [169]

    The Twelfth International Conference on Learning Representations , year=

    Let's verify step by step , author=. The Twelfth International Conference on Learning Representations , year=

  161. [170]

    arXiv preprint arXiv:2502.11169 , year=

    Leveraging constrained monte carlo tree search to generate reliable long chain-of-thought for mathematical reasoning , author=. arXiv preprint arXiv:2502.11169 , year=

    arXiv

  162. [171]

    Advances in neural information processing systems , volume=

    Tree of thoughts: Deliberate problem solving with large language models , author=. Advances in neural information processing systems , volume=

  163. [172]

    arXiv preprint arXiv:2103.03874 , year=

    Measuring mathematical problem solving with the math dataset , author=. arXiv preprint arXiv:2103.03874 , year=

    arXiv

  164. [173]

    arXiv preprint arXiv:2402.19255 , year=

    Gsm-plus: A comprehensive benchmark for evaluating the robustness of llms as mathematical problem solvers , author=. arXiv preprint arXiv:2402.19255 , year=

    arXiv

  165. [174]

    arXiv preprint arXiv:2504.00294 , year=

    Inference-time scaling for complex tasks: Where we stand and what lies ahead , author=. arXiv preprint arXiv:2504.00294 , year=

    arXiv

  166. [175]

    American economic review , volume=

    Maps of bounded rationality: Psychology for behavioral economics , author=. American economic review , volume=. 2003 , publisher=

    2003

  167. [176]

    British Journal of Psychology , volume=

    Heuristic and analytic processes in reasoning , author=. British Journal of Psychology , volume=. 1984 , publisher=

    1984

  168. [177]

    arXiv preprint arXiv:2408.03314 , year=

    Scaling llm test-time compute optimally can be more effective than scaling model parameters , author=. arXiv preprint arXiv:2408.03314 , year=

    arXiv

  169. [178]

    arXiv preprint arXiv:2408.00724 , year=

    Inference scaling laws: An empirical analysis of compute-optimal inference for problem-solving with language models , author=. arXiv preprint arXiv:2408.00724 , year=

    arXiv

  170. [179]

    Advances in Neural Information Processing Systems , volume=

    Toward self-improvement of llms via imagination, searching, and criticizing , author=. Advances in Neural Information Processing Systems , volume=

  171. [180]

    arXiv preprint arXiv:2405.03553 , year=

    Alphamath almost zero: process supervision without process , author=. arXiv preprint arXiv:2405.03553 , year=

    arXiv

  172. [181]

    Advances in Neural Information Processing Systems , volume=

    Rl on incorrect synthetic data scales the efficiency of llm math reasoning by eight-fold , author=. Advances in Neural Information Processing Systems , volume=

  173. [182]

    arXiv preprint arXiv:2410.11287 , year=

    Process reward model with q-value rankings , author=. arXiv preprint arXiv:2410.11287 , year=

    arXiv

  174. [183]

    arXiv preprint arXiv:2502.12118 , year=

    Scaling test-time compute without verification or rl is suboptimal , author=. arXiv preprint arXiv:2502.12118 , year=

    arXiv

  175. [184]

    Icml , volume=

    Policy invariance under reward transformations: Theory and application to reward shaping , author=. Icml , volume=

  176. [185]

    arXiv preprint arXiv:2204.05862 , year=

    Training a helpful and harmless assistant with reinforcement learning from human feedback , author=. arXiv preprint arXiv:2204.05862 , year=

    arXiv

  177. [186]

    Advances in neural information processing systems , volume=

    Bellman-consistent pessimism for offline reinforcement learning , author=. Advances in neural information processing systems , volume=

  178. [187]

    Advances in neural information processing systems , volume=

    Towards instance-optimal offline reinforcement learning with pessimism , author=. Advances in neural information processing systems , volume=

  179. [188]

    International Conference on Artificial Intelligence and Statistics , pages=

    Near-optimal provable uniform convergence in offline policy evaluation for reinforcement learning , author=. International Conference on Artificial Intelligence and Statistics , pages=. 2021 , organization=

    2021

  180. [189]

    International Conference on Machine Learning , pages=

    Pythia: A suite for analyzing large language models across training and scaling , author=. International Conference on Machine Learning , pages=. 2023 , organization=

    2023

  181. [190]

    2024 , publisher=

    Gemma: Introducing new state-of-the-art open models , author=. 2024 , publisher=

    2024

  182. [191]

    arXiv preprint arXiv:2309.11235 , year=

    Openchat: Advancing open-source language models with mixed-quality data , author=. arXiv preprint arXiv:2309.11235 , year=

    arXiv

  183. [192]

    2023 , publisher=

    Starling-7b: Improving llm helpfulness and harmlessness with rlaif , author=. 2023 , publisher=

    2023

  184. [193]

    arXiv preprint arXiv:2310.01377 , year=

    Ultrafeedback: Boosting language models with high-quality feedback , author=. arXiv preprint arXiv:2310.01377 , year=

    arXiv

  185. [194]

    arXiv preprint arXiv:2310.16944 , year=

    Zephyr: Direct distillation of lm alignment , author=. arXiv preprint arXiv:2310.16944 , year=

    arXiv

  186. [195]

    arXiv preprint arXiv:2310.06825 , year=

    Mistral 7B , author=. arXiv preprint arXiv:2310.06825 , year=

    arXiv

  187. [196]

    arXiv preprint arXiv:2104.05218 , year=

    FUDGE: Controlled text generation with future discriminators , author=. arXiv preprint arXiv:2104.05218 , year=

    arXiv

  188. [197]

    arXiv preprint arXiv:2402.02992 , year=

    Decoding-time Realignment of Language Models , author=. arXiv preprint arXiv:2402.02992 , year=

    arXiv

  189. [198]

    arXiv preprint arXiv:2402.06147 , year=

    DeAL: Decoding-time Alignment for Large Language Models , author=. arXiv preprint arXiv:2402.06147 , year=

  190. [199]

    arXiv preprint arXiv:2304.06767 , year=

    Raft: Reward ranked finetuning for generative foundation model alignment , author=. arXiv preprint arXiv:2304.06767 , year=

    arXiv

  191. [200]

    arXiv preprint arXiv:2304.05302 , year=

    Rrhf: Rank responses to align language models with human feedback without tears , author=. arXiv preprint arXiv:2304.05302 , year=

    arXiv

  192. [201]

    International conference on machine learning , pages=

    Interactive learning from policy-dependent human feedback , author=. International conference on machine learning , pages=. 2017 , organization=

    2017

  193. [202]

    arXiv preprint arXiv:2302.02676 , year=

    Chain of hindsight aligns language models with feedback , author=. arXiv preprint arXiv:2302.02676 , year=

    arXiv

  194. [203]

    Advances in neural information processing systems , volume=

    Reward learning from human preferences and demonstrations in atari , author=. Advances in neural information processing systems , volume=

  195. [204]

    arXiv preprint arXiv:1806.01946 , year=

    Learning to understand goal specifications by modelling reward , author=. arXiv preprint arXiv:1806.01946 , year=

    arXiv

  196. [205]

    arXiv preprint arXiv:2112.09332 , year=

    Webgpt: Browser-assisted question-answering with human feedback , author=. arXiv preprint arXiv:2112.09332 , year=

    arXiv

  197. [206]

    arXiv preprint arXiv:2206.11871 , year=

    Offline rl for natural language generation with implicit language q learning , author=. arXiv preprint arXiv:2206.11871 , year=

    arXiv

  198. [207]

    arXiv preprint arXiv:2209.14375 , year=

    Improving alignment of dialogue agents via targeted human judgements , author=. arXiv preprint arXiv:2209.14375 , year=

    arXiv

  199. [208]

    arXiv preprint arXiv:2112.00861 , year=

    A general language assistant as a laboratory for alignment , author=. arXiv preprint arXiv:2112.00861 , year=

    arXiv

  200. [209]

    Advances in Neural Information Processing Systems , volume=

    Learning to summarize with human feedback , author=. Advances in Neural Information Processing Systems , volume=

  201. [210]

    Advances in neural information processing systems , volume=

    Deep reinforcement learning from human preferences , author=. Advances in neural information processing systems , volume=

  202. [211]

    Advances in Neural Information Processing Systems , editor=

    A Contrastive Framework for Neural Text Generation , author=. Advances in Neural Information Processing Systems , editor=

  203. [212]

    International Conference on Knowledge Discovery and Information Retrieval , volume=

    Item-user preference mapping with mixture models-data visualization for item preference , author=. International Conference on Knowledge Discovery and Information Retrieval , volume=. 2009 , organization=

    2009

  204. [213]

    arXiv preprint arXiv:2402.01306 , year=

    Kto: Model alignment as prospect theoretic optimization , author=. arXiv preprint arXiv:2402.01306 , year=

    arXiv

  205. [214]

    arXiv preprint arXiv:2404.05868 , year=

    Negative Preference Optimization: From Catastrophic Collapse to Effective Unlearning , author=. arXiv preprint arXiv:2404.05868 , year=

    arXiv

  206. [215]

    arXiv preprint arXiv:1909.08593 , year=

    Fine-tuning language models from human preferences , author=. arXiv preprint arXiv:1909.08593 , year=

    1909 arXiv

  207. [216]

    2022 , eprint=

    Training language models to follow instructions with human feedback , author=. 2022 , eprint=

    2022

  208. [217]

    arXiv preprint arXiv:1707.06347 , year=

    Proximal policy optimization algorithms , author=. arXiv preprint arXiv:1707.06347 , year=

    arXiv

  209. [218]

    arXiv preprint arXiv:2307.15217 , year=

    Open problems and fundamental limitations of reinforcement learning from human feedback , author=. arXiv preprint arXiv:2307.15217 , year=

    arXiv

  210. [219]

    arXiv preprint arXiv:2303.17548 , year=

    Whose opinions do language models reflect? , author=. arXiv preprint arXiv:2303.17548 , year=

    arXiv

  211. [220]

    arXiv preprint arXiv:2310.11564 , year=

    Personalized Soups: Personalized Large Language Model Alignment via Post-hoc Parameter Merging , author=. arXiv preprint arXiv:2310.11564 , year=

    arXiv

  212. [221]

    2018 , eprint=

    Hierarchical Neural Story Generation , author=. 2018 , eprint=

    2018

  213. [222]

    2022 , eprint=

    Learning to summarize from human feedback , author=. 2022 , eprint=

    2022

  214. [223]

    The method of paired comparisons , author=

    Rank analysis of incomplete block designs: I. The method of paired comparisons , author=. Biometrika , volume=. 1952 , publisher=

    1952

  215. [224]

    Journal of the Royal Statistical Society Series C: Applied Statistics , volume=

    The analysis of permutations , author=. Journal of the Royal Statistical Society Series C: Applied Statistics , volume=. 1975 , publisher=

    1975

  216. [225]

    2012 , publisher=

    Individual choice behavior: A theoretical analysis , author=. 2012 , publisher=

    2012

  217. [226]

    arXiv preprint arXiv:2306.04751 , year=

    How Far Can Camels Go? Exploring the State of Instruction Tuning on Open Resources , author=. arXiv preprint arXiv:2306.04751 , year=

    arXiv

  218. [227]

    arXiv preprint arXiv:2304.03277 , year=

    Instruction tuning with gpt-4 , author=. arXiv preprint arXiv:2304.03277 , year=

    arXiv

  219. [228]

    2020 , publisher=

    The alignment problem: Machine learning and human values , author=. 2020 , publisher=

    2020

  220. [229]

    Collective Choice and Social Welfare , title =

    Amartya Sen , publisher =. Collective Choice and Social Welfare , title =. 2017 , lastchecked =. doi:doi:10.4159/9780674974616 , isbn =

    2017 doi

  221. [230]

    arXiv preprint arXiv:2307.12966 , year=

    Aligning large language models with human: A survey , author=. arXiv preprint arXiv:2307.12966 , year=

    arXiv

  222. [231]

    2023 , eprint=

    DICES Dataset: Diversity in Conversational AI Evaluation for Safety , author=. 2023 , eprint=

    2023

  223. [232]

    2023 , eprint=

    The Reasonable Effectiveness of Diverse Evaluation Data , author=. 2023 , eprint=

    2023

  224. [233]

    2021 , eprint=

    Whose Ground Truth? Accounting for Individual and Collective Identities Underlying Dataset Annotation , author=. 2021 , eprint=

    2021

  225. [234]

    AI Magazine , author=

    Truth Is a Lie: Crowd Truth and the Seven Myths of Human Annotation , volume=. AI Magazine , author=. 2015 , month=. doi:10.1609/aimag.v36i1.2564 , abstractNote=

    2015 doi

  226. [235]

    Why Don ' t You Do It Right? Analysing Annotators ' Disagreement in Subjective Tasks

    Sandri, Marta and Leonardelli, Elisa and Tonelli, Sara and Jezek, Elisabetta. Why Don ' t You Do It Right? Analysing Annotators ' Disagreement in Subjective Tasks. Proceedings of the 17th Conference of the European Chapter of the Association for Computational Linguistics. 2023...

    2023 doi

  227. [236]

    Annotators with Attitudes: How Annotator Beliefs And Identities Bias Toxic Language Detection

    Sap, Maarten and Swayamdipta, Swabha and Vianna, Laura and Zhou, Xuhui and Choi, Yejin and Smith, Noah A. Annotators with Attitudes: How Annotator Beliefs And Identities Bias Toxic Language Detection. Proceedings of the 2022 Conference of the North American Chapter of the Asso...

    2022 doi

  228. [237]

    Agreeing to Disagree: Annotating Offensive Language Datasets with Annotators’ Disagreement , url=

    Leonardelli, Elisa and Menini, Stefano and Palmero Aprosio, Alessio and Guerini, Marco and Tonelli, Sara , year=. Agreeing to Disagree: Annotating Offensive Language Datasets with Annotators’ Disagreement , url=. doi:10.18653/v1/2021.emnlp-main.822 , booktitle=

    doi

  229. [238]

    Identifying and Measuring Annotator Bias Based on Annotators ' Demographic Characteristics

    Al Kuwatly, Hala and Wich, Maximilian and Groh, Georg. Identifying and Measuring Annotator Bias Based on Annotators ' Demographic Characteristics. Proceedings of the Fourth Workshop on Online Abuse and Harms. 2020. doi:10.18653/v1/2020.alw-1.21

    2020 doi

  230. [239]

    2023 , eprint=

    Principled Reinforcement Learning with Human Feedback from Pairwise or K -wise Comparisons , author=. 2023 , eprint=

    2023

  231. [240]

    2023 , eprint=

    Direct Preference Optimization: Your Language Model is Secretly a Reward Model , author=. 2023 , eprint=

    2023

  232. [241]

    Blog post, April , volume=

    Koala: A dialogue model for academic research , author=. Blog post, April , volume=

  233. [242]

    arXiv preprint arXiv:2305.14387 , year=

    Alpacafarm: A simulation framework for methods that learn from human feedback , author=. arXiv preprint arXiv:2305.14387 , year=

    arXiv

  234. [243]

    arXiv preprint arXiv:2311.10702 , year=

    Camels in a changing climate: Enhancing lm adaptation with tulu 2 , author=. arXiv preprint arXiv:2311.10702 , year=

    arXiv

  235. [244]

    2020 , eprint=

    Fine-Tuning Language Models from Human Preferences , author=. 2020 , eprint=

    2020

  236. [245]

    2023 , eprint=

    RRHF: Rank Responses to Align Language Models with Human Feedback without tears , author=. 2023 , eprint=

    2023

  237. [246]

    2023 , eprint=

    Aligning Language Models with Preferences through f-divergence Minimization , author=. 2023 , eprint=

    2023

  238. [247]

    2023 , eprint=

    A Survey of Reinforcement Learning from Human Feedback , author=. 2023 , eprint=

    2023

  239. [248]

    2023 , eprint=

    Beyond One-Preference-Fits-All Alignment: Multi-Objective Direct Preference Optimization , author=. 2023 , eprint=

    2023

  240. [249]

    2023 , eprint=

    Safe RLHF: Safe Reinforcement Learning from Human Feedback , author=. 2023 , eprint=

    2023

  241. [250]

    Ethics and Information Technology , year=

    Human-aligned artificial intelligence is a multiobjective problem , author=. Ethics and Information Technology , year=

  242. [251]

    Australasian Conference on Artificial Intelligence , year=

    On the Limitations of Scalarisation for Multi-objective Reinforcement Learning of Pareto Fronts , author=. Australasian Conference on Artificial Intelligence , year=

  243. [252]

    2022 , eprint=

    Fine-tuning language models to find agreement among humans with diverse preferences , author=. 2022 , eprint=

    2022

  244. [253]

    2023 , eprint=

    'Generative CI' through Collective Response Systems , author=. 2023 , eprint=

    2023

  245. [254]

    2022 , eprint=

    Constitutional AI: Harmlessness from AI Feedback , author=. 2022 , eprint=

    2022

  246. [255]

    2023 , eprint=

    Large Language Models as Superpositions of Cultural Perspectives , author=. 2023 , eprint=

    2023

  247. [256]

    2023 , eprint=

    Rewarded soups: towards Pareto-optimal alignment by interpolating weights fine-tuned on diverse rewards , author=. 2023 , eprint=

    2023

  248. [257]

    2023 , eprint=

    BeaverTails: Towards Improved Safety Alignment of LLM via a Human-Preference Dataset , author=. 2023 , eprint=

    2023

  249. [258]

    arXiv preprint arXiv:2401.01335 , year=

    Self-play fine-tuning converts weak language models to strong language models , author=. arXiv preprint arXiv:2401.01335 , year=

    arXiv

  250. [259]

    2019 , url=

    Language Models are Unsupervised Multitask Learners , author=. 2019 , url=

    2019

  251. [260]

    and Daly, Raymond E

    Maas, Andrew L. and Daly, Raymond E. and Pham, Peter T. and Huang, Dan and Ng, Andrew Y. and Potts, Christopher , title =. Proceedings of the 49th Annual Meeting of the Association for Computational Linguistics: Human Language Technologies , month =. 2011 , address =

    2011

  252. [261]

    arXiv preprint arXiv:2310.10556, AAAI , year=

    Sample Complexity of Preference-Based Nonparametric Off-Policy Evaluation with Deep Networks , author=. arXiv preprint arXiv:2310.10556, AAAI , year=

    arXiv

  253. [262]

    The Twelfth International Conference on Learning Representations (ICLR) , year=

    PARL: A Unified Framework for Policy Alignment in Reinforcement Learning , author=. The Twelfth International Conference on Learning Representations (ICLR) , year=

  254. [263]

    arXiv preprint arXiv:2305.18438 , year=

    Reinforcement Learning with Human Feedback: Learning Dynamic Choices via Pessimism , author=. arXiv preprint arXiv:2305.18438 , year=

    arXiv

  255. [264]

    arXiv preprint arXiv:2307.12975 , year=

    Provable Benefits of Policy Learning from Human Preferences in Contextual Bandit Problems , author=. arXiv preprint arXiv:2307.12975 , year=

    arXiv

  256. [265]

    arXiv preprint arXiv:2306.07528 , year=

    Unified Off-Policy Learning to Rank: a Reinforcement Learning Perspective , author=. arXiv preprint arXiv:2306.07528 , year=

    arXiv

  257. [266]

    arXiv preprint arXiv:1811.00511 , year=

    Towards Coherent and Cohesive Long-form Text Generation , author=. arXiv preprint arXiv:1811.00511 , year=

    arXiv

  258. [267]

    Pew Research Center , volume=

    The state of online harassment , author=. Pew Research Center , volume=

  259. [268]

    Robust Solutions of Optimization Problems Affected by Uncertain Probabilities , urldate =

    Aharon Ben-Tal and Dick den Hertog and Anja De Waegenaere and Bertrand Melenberg and Gijs Rennen , journal =. Robust Solutions of Optimization Problems Affected by Uncertain Probabilities , urldate =

  260. [269]

    2018 , eprint=

    Statistics of Robust Optimization: A Generalized Empirical Likelihood Approach , author=. 2018 , eprint=

    2018

  261. [270]

    2020 , eprint=

    Distributionally Robust Neural Networks for Group Shifts: On the Importance of Regularization for Worst-Case Generalization , author=. 2020 , eprint=

    2020

  262. [271]

    2021 , eprint=

    On the Convergence and Sample Efficiency of Variance-Reduced Policy Gradient Method , author=. 2021 , eprint=

    2021

  263. [272]

    2020 , eprint=

    Variational Policy Gradient Method for Reinforcement Learning with General Utilities , author=. 2020 , eprint=

    2020

  264. [273]

    Lindsay , journal =

    Bruce G. Lindsay , journal =. Mixture Models: Theory, Geometry and Applications , urldate =

  265. [274]

    Mixture Models

    Seidel, Wilfried. Mixture Models. International Encyclopedia of Statistical Science. 2011. doi:10.1007/978-3-642-04898-2_368

    2011 doi

  266. [275]

    2016 , isbn =

    Goodfellow, Ian and Bengio, Yoshua and Courville, Aaron , title =. 2016 , isbn =

    2016

  267. [276]

    Bertsekas , title =

    Dimitri P. Bertsekas , title =. 2019 , address =

    2019

  268. [277]

    Sutton and Andrew G

    Richard S. Sutton and Andrew G. Barto , title =. 2018 , edition =

    2018

  269. [278]

    2023 , eprint=

    Nash Learning from Human Feedback , author=. 2023 , eprint=

    2023

  270. [279]

    2024 , eprint=

    A Minimaximalist Approach to Reinforcement Learning from Human Feedback , author=. 2024 , eprint=

    2024

  271. [280]

    2024 , eprint=

    MaxMin-RLHF: Towards Equitable Alignment of Large Language Models with Diverse Human Preferences , author=. 2024 , eprint=

    2024

  272. [281]

    CS Dept., UW Seattle, Seattle, WA, USA, Tech

    Reinforcement learning: Theory and algorithms , author=. CS Dept., UW Seattle, Seattle, WA, USA, Tech. Rep , volume=

  273. [282]

    International Conference on Machine Learning , pages=

    Scaling laws for reward model overoptimization , author=. International Conference on Machine Learning , pages=. 2023 , organization=

    2023

  274. [283]

    2024 , eprint=

    Gemini: A Family of Highly Capable Multimodal Models , author=. 2024 , eprint=

    2024

  275. [284]

    2024 , eprint=

    ARGS: Alignment as Reward-Guided Search , author=. 2024 , eprint=

    2024

  276. [285]

    2024 , eprint=

    Controlled Decoding from Language Models , author=. 2024 , eprint=

    2024

  277. [286]

    2024 , eprint=

    From r to Q^* : Your Language Model is Secretly a Q-Function , author=. 2024 , eprint=

    2024

  278. [287]

    2014 , eprint=

    Sequence to Sequence Learning with Neural Networks , author=. 2014 , eprint=

    2014

  279. [288]

    2016 , eprint=

    Google's Neural Machine Translation System: Bridging the Gap between Human and Machine Translation , author=. 2016 , eprint=

    2016

  280. [289]

    2023 , eprint=

    Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer , author=. 2023 , eprint=

    2023

  281. [290]

    2019 , eprint=

    BERT has a Mouth, and It Must Speak: BERT as a Markov Random Field Language Model , author=. 2019 , eprint=

    2019

  282. [291]

    2020 , eprint=

    The Curious Case of Neural Text Degeneration , author=. 2020 , eprint=

    2020

  283. [292]

    2024 , eprint=

    Large Language Models: A Survey , author=. 2024 , eprint=

    2024

  284. [293]

    2020 , eprint=

    Scaling Laws for Neural Language Models , author=. 2020 , eprint=

    2020

  285. [294]

    2023 , eprint=

    Towards Better Parameter-Efficient Fine-Tuning for Large Language Models: A Position Paper , author=. 2023 , eprint=

    2023

  286. [295]

    2024 , eprint=

    LLMCarbon: Modeling the end-to-end Carbon Footprint of Large Language Models , author=. 2024 , eprint=

    2024

  287. [296]

    2023 , eprint=

    Climate Change from Large Language Models , author=. 2023 , eprint=

    2023

  288. [297]

    2023 , eprint=

    LLaMA: Open and Efficient Foundation Language Models , author=. 2023 , eprint=

    2023

  289. [298]

    2023 , eprint=

    Toxicity in ChatGPT: Analyzing Persona-assigned Language Models , author=. 2023 , eprint=

    2023

  290. [299]

    2024 , eprint=

    AI Alignment: A Comprehensive Survey , author=. 2024 , eprint=

    2024

  291. [300]

    2024 , eprint=

    RewardBench: Evaluating Reward Models for Language Modeling , author=. 2024 , eprint=

    2024

Pith tools

Reviewed August 12, 2026 · model on record in the stance chip above.