Pith. sign in

REVIEW 4 major objections 7 minor 1 cited by

CoT-based Synthesizer: Enhancing LLM Performance through Answer Synthesis

T0 review · 4 major / 7 minor · reviewed 2026-08-10 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read The paper claims that a lightweight synthesizer model, trained on examples where a large model analyzes and combines candidate responses, can produce correct answers even when every candidate is wrong, improving larger policy models on…

desk verdict Useful empirical recipe, but the central claim about cross-candidate synthesis is not proven; the results are consistent with a second-model ensemble. read the letter →

arxiv 2501.01668 v2 pith:MANKCLVK submitted 2025-01-03 cs.CL

Bohan Zhang , Xiaokang Zhang , Jing Zhang , Jifan Yu , Sijia Luo , Jie Tang This is my paper · ORCID
classification cs.CL
keywords inferencescalinganswersynthesischain-of-thoughtself-consistencyBest-of-NmathematicalreasoningtablequestionansweringLLMpost-processing
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper proposes CoT-based Synthesizer, an inference-scaling method that takes a question together with several candidate answers, reasons through the candidates step by step, and writes a new synthesized answer instead of selecting one existing answer. The central claim is that this synthesis process can produce a correct answer even when every candidate is wrong, by extracting and recombining the correct fragments scattered across flawed responses. To make this practical, the paper builds an automated pipeline that samples candidate sets, has a large model produce and filter synthesis examples, and uses them to train a smaller 8-billion-parameter synthesizer. The result, if true, is that a small open model can serve as a post-processing layer that improves much larger and even API-only models, with reported gains of 11.8% for Llama3-8B and 10.3% for GPT-4o on MATH500.

What carries the argument

The central mechanism is the CoT-based Synthesizer: a small language model fine-tuned to map a query with its candidate set $(x, R)$ to a synthesized answer $y$, using chain-of-thought analysis of each candidate and of the relations among candidates. It is trained with a two-stage data pipeline -- synthetic answer generation, in which a large response model analyzes $(x, R)$ and correct outputs are kept by comparing to gold answers, and LLM repair, which prompts the response model to re-synthesize when all candidates are wrong. The same inference prompt is then applied to any policy model's candidates, which is what allows a single 8B model to improve larger and API-based models.

What would settle it

Replace the candidate set at inference time with a deliberately misleading set -- for example, a set of wrong answers that share no correct sub-steps -- and measure whether Synthesizer-8B's output changes or degrades toward the direct answer; if the output is statistically the same as when the candidates are removed or randomized, the model is solving independently rather than synthesizing, which would falsify the central claim. A simpler version is to compare accuracy on identical queries with the genuine candidates versus with candidates drawn from a different question.

Watch

Extended reading notes

Core claim

The paper's discovery is that answer selection is not the only viable form of inference scaling: a model can analyze the relationship between a query and a set of candidate responses and produce a better answer than any candidate contains. Concretely, CoT-based Synthesizer treats the candidate responses as raw material, identifies which reasoning steps are valid, and synthesizes a final answer, so the method remains useful precisely when self-consistency and Best-of-N fail because the correct answer is absent from the candidate set. The experiments claim the trained Synthesizer-8B outperforms these selection baselines across four benchmarks and seven policy models, including generalization to models structurally different from those used to generate the training data.

Load-bearing premise

The load-bearing premise is that the training labels are genuine syntheses of the candidate set: the large response model produces its answer by analyzing and combining $(x, R)$, so the filtered correct outputs teach the smaller model how to synthesize from candidates rather than how to solve the question directly.

Editorial extensions

If this is right

  • Selection-based inference scaling is not the ceiling: post-processing can produce correct answers that do not appear in any candidate, so the candidate set's quality no longer strictly bounds the final accuracy.
  • A small trained model can act as a portable synthesis layer that improves API models it never trained on, transferring across policy models with different architectures and behaviors.
  • Generative synthesis scales with training data differently than scalar verifiers: accuracy keeps rising log-linearly past the point where reward-model training saturates or declines on duplicated instructions.
  • Synthesis removes the exact-match voting requirement, extending inference scaling to open-ended tasks such as free-form table question answering.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Not tested in the paper, the synthesis mechanism plausibly transfers to other post-processing settings where candidates contain partial correctness, such as code repair or document-grounded QA.
  • If the label-generation assumption fails on some queries, the practical gains may partly reflect a two-model ensemble rather than candidate synthesis; an ablation that corrupts or removes candidates would separate those effects.
  • Because the paper groups candidates when the context is long, a long-context synthesizer that reads all candidates at once could improve both accuracy and cost, a direction the paper notes but does not pursue.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 7 minor

Summary. The paper proposes CoT-based Synthesizer, an inference-scaling method in which a trained small LLM takes a question and a set of candidate responses produced by a policy model, reasons about the candidates via chain-of-thought, and synthesizes a final answer. The authors introduce an automated pipeline that uses a sampling LLM to generate candidate sets and a response LLM to produce synthesis labels, filters those labels against gold answers, and then fine-tunes Llama3-8B-Instruct to imitate the synthesis process. Experiments on GSM8k, MATH500, WikiTQ, and FeTaQA across seven policy models report consistent gains over Self-consistency, Best-of-N, and LMCOR, including 11.8% absolute improvement for Llama3-8B and 10.3% for GPT-4o on MATH500. The paper also includes ablations of the data pipeline, scaling experiments over data size and candidate count, and an analysis table showing that the Synthesizer produces correct final answers on some problems where all candidates are incorrect.

Significance. If the central claim is correct, the method is a practical and cost-effective inference-scaling strategy: it trains a single 8B synthesizer once and then improves a variety of policy models, including closed API models, without requiring retraining or a separate verifier. The paper is commendable for releasing code and data, for evaluating across four benchmarks and seven policy models, and for including scaling analyses and ablations of the data-generation pipeline. The reproducibility-oriented reporting (three runs, public repository) is a strength. However, the load-bearing claim that the Synthesizer actually exploits the candidate set R is not established: the training-label construction does not guarantee that labels depend on R, and the ablation evidence is consistent with a simpler explanation in which the trained model is a direct answer generator distilled from the response LLM. Because the paper's novelty and headline results depend on the mechanism of candidate synthesis, this gap is central rather than cosmetic.

major comments (4)
  1. [5.1 / Table 1] The headline claim of "significant" improvement is not supported by any variance or significance reporting. Each result is described as the average of three runs, but no standard deviation, confidence interval, or test (e.g., paired bootstrap or McNemar) is given. Many decisive cells, such as MATH500 for Qwen2.5-14B (78.2 vs. 78.2 for USC) and GSM8k for Llama3.1-70B (94.0 vs. 93.9 for SC), differ by well under the noise floor of a 500- or 1319-sample test set. The paper should report per-cell variance and a significance test for at least the headline comparisons, and should temper the claim of consistent superiority accordingly.
  2. [4.2 / Equation (1)] The training pipeline does not verify that the synthesized label y is actually a function of the candidate set R. In "Synthetic Answer Generation", the response LLM is prompted with x and R and multiple outputs are filtered by equality with the gold answer y_g; this filter removes incorrect answers but does not remove labels that were produced by solving x directly while ignoring R. The same issue applies to "LLM Repair". Because the response LLM is a strong reasoner, a large fraction of filtered labels may be independent of R. Training with Eq. (1) can then fit p_phi(y|x) rather than p_phi(y|x,R), and the gains in Tables 1 and 3 would be explained by ensembling or distillation, not by candidate synthesis. The paper needs a perturbation test that makes the R-dependence observable: for example, shuffle candidate sets across questions at inference (or mask R entirely) and measure the drop in accuracy. Without such a test, the central mechanism behind the paper's title and abstract is unverified.
  3. [5.3 / Table 2] The ablations do not control for the direct-distillation confound. Row "w/o training" shows that an untrained Llama3-8B, given the synthesis prompt, already improves MATH500 accuracy over CoT-prompting by +8.0 for Llama3-8B and +3.6 for Llama3.1-8B; this demonstrates that the prompt alone elicits useful x-only answers and therefore cannot distinguish true synthesis from direct solving. Row "w/o CoT training" even exceeds the full method on MATH500 for Llama3-8B (37.0 vs. 36.0), directly contradicting the claim that the CoT analysis of candidates is beneficial for this setting. The most informative control is missing: train the same 8B model on (x, y) pairs with no R in the input, using the same filtered response-LLM labels, and compare it with the full Synthesizer. If the two perform equally, the paper's stated contribution reduces to a distillation/ensemble effect.
  4. [5.5 / Table 3] The "Correct Count 0" row reports that Synthesizer-8B produces 9 correct final answers when zero of the five candidates are correct, while SC, ArmoRM, and Scalar RM produce 0. Without the denominator, i.e., the number of test instances having zero correct candidates, this figure cannot be interpreted. If, for example, the Synthesizer's own single-pass accuracy on MATH500 is roughly 20-25%, then on a few dozen such instances one would expect about this many correct answers from a model that ignores R entirely. The paper should report the denominator for this row and compare against the x-only control on the same subset, so that the "synthesis from all-wrong candidates" claim can be evaluated.
minor comments (7)
  1. [Abstract / Section 1] The abstract first says "gains of 11.8% for Llama3-8B and 10.3% for GPT-4o on the MATH dataset" but the body reports these on MATH500; align the terminology in the abstract and introduction.
  2. [Table 1] In the FeTaQA block, the entry for USC with GLM-4-Plus appears as "87.187.0", which is likely a missing space between two numbers; please correct.
  3. [Table 2] The table labels the MATH500 results as "MATH", while the text and other tables use MATH500; use one consistent name.
  4. [Appendix C.2 / Table 6] The URL for ArmoRM-Llama3-8B-v0.1 points to a Mixtral-8x7B checkpoint, which appears to be a copy-paste error; the correct model link should be provided.
  5. [5.4 / Figure 4] The caption says the scaling plot is "across the MATH500 and GSM8k datasets," but the figure appears to show Llama3-8B and Qwen2-7B test sets; clarify which dataset/model combination is displayed.
  6. [5.3 / Table 2] For the "w/o training" row, the paper should specify exactly which prompt is given to the untrained Llama3-8B-Instruct (the synthesis prompt from Appendix E or a variant), so the reader can assess whether the +8.0 gain reflects the prompt itself or the candidate content.
  7. [Figure 5] The example is titled "synthesizing a correct answer from all the wrong candidate responses," but several candidate responses contain the correct element -4 (e.g., R1, R3, R4, R5). If "wrong" only means no candidate is fully correct, the caption should say so explicitly; otherwise the example does not demonstrate the stronger claim that no candidate contains the correct answer.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the synthesis model is trained on gold-filtered outputs of a separate response LLM and evaluated on held-out test sets; no claim reduces to its inputs by construction.

full rationale

The paper's central object is a supervised synthesis model p_phi(y|x,R) trained (Eq. 1) on tuples (x,R,y) built by the data-generation pipeline in Section 4.2: the Sampling LLM produces R, the Response LLM Llama3.1-70B-Instruct produces candidate synthesized answers, and those are filtered against the gold answer y_g. Evaluation (Tables 1-3) is on held-out GSM8k, MATH500, WikiTQ, and FeTaQA test sets, with training constructed from the MATH and WikiTQ training splits rather than from the test answers. There is no equation or definition in which a reported result is equivalent to a fitted input: the labels y are not defined as the evaluation targets, the 'ours (Llama3.1-70B)' row is a separate inference-time use of the Response LLM rather than a fitted parameter renamed as a prediction, and the method does not import an 'uniqueness' or ansatz claim from the authors' prior work. The self-citations to TableLLM (Zhang et al., 2024b) and CritiqueLLM (Ke et al., 2023) supply benchmark preprocessing and a data-quality filter, respectively, but neither is load-bearing for the claim that candidate synthesis improves accuracy. The reader's concern that labels may be produced from x alone, making Synthesizer-8B a second-model ensemble rather than a synthesis mechanism, is a legitimate empirical verification gap (the w/o training baseline in Table 2 shows the prompt alone can improve Llama3-8B on MATH500), but it is not circularity: the training objective would still be standard supervised learning on externally filtered labels, and no reported number reduces to a training input by construction. The appended limitations about input-length grouping and inference overhead also raise efficiency concerns, not definitional circularity. Hence the score is 0.

Assumptions & free parameters 5 free parameters · 4 assumptions · 0 invented entities

The ledger captures one fitted hyperparameter set (N, temperature, top-p, LR, filter threshold) and four domain assumptions. No new physical or mathematical entities are introduced. The heaviest assumption is that the training labels are genuine functions of the candidate set; if they are not, the method reduces to a fine-tuned solver.

free parameters (5)
  • Number of candidate responses N = 5
    Chosen for inference and training; not fitted to the target result, but the method's cost and accuracy depend on it.
  • Sampling temperature = 0.9
    Chosen to balance diversity and coherence; not fitted to test data.
  • Top-p sampling threshold = 0.9
    Chosen to avoid meaningless low-probability tokens; standard decoding hyperparameter.
  • Learning rate = 2e-6
    Selected via search over [5e-6, 2e-6, 5e-7] on the training data; a fitted hyperparameter.
  • CritiqueLLM score threshold for TableQA = 8
    Chosen to retain high-quality synthesized answers in the WikiTQ pipeline; affects training data composition.
assumptions (4)
  • domain assumption Candidate responses generated by a single policy model contain complementary information that a synthesizer can combine into a correct answer.
    Section 1 and 4.1 assume this property; if candidates share no complementary information, synthesis cannot help beyond reranking or independent solving.
  • domain assumption The gold answers in the training benchmarks are correct and exact-match filtering selects genuinely correct synthesized answers.
    Section 4.2 uses gold answer yg to filter training labels for MATH; errors in gold or in matching would inject noisy labels.
  • domain assumption Llama3.1-70B-Instruct can produce correct synthesized answers from candidate sets often enough to yield 295k and 87k diverse training examples.
    Section 4.2 and Appendix C.3 rely on this capability; the volume of filtered data depends on it.
  • domain assumption A synthesizer trained on candidates from Llama3-8B-Instruct transfers to structurally distinct policy models (GPT-4o, GLM-4-plus, Qwen2.5-14B).
    Section 5.2 claims generalization; this is tested but not guaranteed by any invariant.

how reviews work

0 comments
Cite this review

Pith. "Pith review of CoT-based Synthesizer: Enhancing LLM Performance through Answer Synthesis." pith.science (2026). https://pith.science/paper/MANKCLVK

@misc{pith2026250101668,
  author       = {Pith},
  title        = {Pith review of: CoT-based Synthesizer: Enhancing LLM Performance through Answer Synthesis},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/MANKCLVK}},
  note         = {Machine review of arXiv:2501.01668}
}
read the original abstract

Current inference scaling methods, such as Self-consistency and Best-of-N, have proven effective in improving the accuracy of LLMs on complex reasoning tasks. However, these methods rely heavily on the quality of candidate responses and are unable to produce correct answers when all candidates are incorrect. In this paper, we propose a novel inference scaling strategy, CoT-based Synthesizer, which leverages CoT reasoning to synthesize superior answers by analyzing complementary information from multiple candidate responses, even when all candidate responses are flawed. To enable a lightweight and cost-effective implementation, we introduce an automated data generation pipeline that creates diverse training data. This allows smaller LLMs trained on this data to improve the inference accuracy of larger models, including API-based LLMs. Experimental results across four benchmark datasets with seven policy models demonstrate that our method significantly enhances performance, with gains of 11.8% for Llama3-8B and 10.3% for GPT-4o on the MATH dataset. The corresponding training data and code are publicly available on https://github.com/RUCKBReasoning/CoT-based-Synthesizer.

Figures

Figures reproduced from arXiv: 2501.01668 by the authors.

Figure 1
Figure 1. An example of our method in mathematical [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. An overview of our method. (1) Synthesizer Inference: The policy model generates diverse candidate responses, which are analyzed and synthesized by the CoT-based synthesizer to produce a high-quality final response. (2) Data Generation Pipeline: The pipeline combines a diverse response generation process using a sampling LLM with query-response relationship analysis to construct high-quality synthetic data. (3) Synt… view at source ↗
Figure 3
Figure 3. Effects of training data size on the perfor [PITH_FULL_IMAGE:figures/full_fig_p007_3.png] view at source ↗
Figures from the paper (3 more)
Figure 4
Figure 4. Figure 4: Effects of inference scaling on the perfor [PITH_FULL_IMAGE:figures/full_fig_p008_4.png]
Figure 5
Figure 5. Figure 5: An example of synthesizing a correct answer from all the wrong candidate responses generated by [PITH_FULL_IMAGE:figures/full_fig_p017_5.png]
Figure 6
Figure 6. Figure 6: An example of synthesizing a correct answer from all the wrong candidate responses generated by [PITH_FULL_IMAGE:figures/full_fig_p018_6.png]

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. Less is Enough: Synthesizing Diverse Data in LLM Feature Space with Sparse Autoencoders

    cs.CL 2026-02 conditional novelty 6.0 of 10

    Coverage of sparse-autoencoder-identified task features predicts post-training performance and can guide synthesis of small, high-impact datasets (2,000 vs. 300,000 samples).

Reference graph

Works this paper leans on

46 extracted references · 3 canonical work pages · cited by 1 Pith paper

  1. [1]

    Josh Achiam, Steven Adler, Sandhini Agarwal, Lama Ahmad, Ilge Akkaya, Florencia Leoni Aleman, Diogo Almeida, Janko Altenschmidt, Sam Altman, Shyamal Anadkat, et al. 2023. Gpt-4 technical report. arXiv preprint arXiv:2303.08774

    arXiv 2023

  2. [2]

    Bradley Brown, Jordan Juravsky, Ryan Ehrlich, Ronald Clark, Quoc V Le, Christopher R \'e , and Azalia Mirhoseini. 2024. Large language monkeys: Scaling inference compute with repeated sampling. arXiv preprint arXiv:2407.21787

    arXiv 2024

  3. [3]

    Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared D Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, Sandhini Agarwal, Ariel Herbert-Voss, Gretchen Krueger, Tom Henighan, Rewon Child, Aditya Ramesh, Daniel Ziegler, Jeffrey Wu, Clemens Winter, Chris Hesse, Mark Chen, Eric Sigler, Mateusz Litwin, Scott Gr...

    2020

  4. [4]

    Xinyun Chen, Renat Aksitov, Uri Alon, Jie Ren, Kefan Xiao, Pengcheng Yin, Sushant Prakash, Charles Sutton, Xuezhi Wang, and Denny Zhou. 2023. Universal self-consistency for large language model generation. arXiv preprint arXiv:2311.17311

    arXiv 2023

  5. [5]

    Aakanksha Chowdhery, Sharan Narang, Jacob Devlin, Maarten Bosma, Gaurav Mishra, Adam Roberts, Paul Barham, Hyung Won Chung, Charles Sutton, Sebastian Gehrmann, Parker Schuh, Kensen Shi, Sasha Tsvyashchenko, Joshua Maynez, Abhishek Rao, Parker Barnes, Yi Tay, Noam Shazeer, Vinodkumar Prabhakaran, Emily Reif, Nan Du, Ben Hutchinson, Reiner Pope, James Bradb...

    2023

  6. [6]

    Paul F Christiano, Jan Leike, Tom Brown, Miljan Martic, Shane Legg, and Dario Amodei. 2017. Deep reinforcement learning from human preferences. Advances in neural information processing systems, 30

    2017

  7. [7]

    Karl Cobbe, Vineet Kosaraju, Mohammad Bavarian, Mark Chen, Heewoo Jun, Lukasz Kaiser, Matthias Plappert, Jerry Tworek, Jacob Hilton, Reiichiro Nakano, et al. 2021. Training verifiers to solve math word problems. arXiv preprint arXiv:2110.14168

    arXiv 2021

  8. [8]

    Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Abhishek Kadian, Ahmad Al-Dahle, Aiesha Letman, Akhil Mathur, Alan Schelten, Amy Yang, Angela Fan, et al. 2024. The llama 3 herd of models. arXiv preprint arXiv:2407.21783

    arXiv 2024

Show all 46 references
  1. [9]

    Ant \'o nio Farinhas, Jos \'e GC de Souza, and Andr \'e FT Martins. 2023. An empirical study of translation hypothesis ensembling with large language models. arXiv preprint arXiv:2310.11430

    2023 arXiv

  2. [10]

    Team GLM, Aohan Zeng, Bin Xu, Bowen Wang, Chenhui Zhang, Da Yin, Dan Zhang, Diego Rojas, Guanyu Feng, Hanlin Zhao, et al. 2024. Chatglm: A family of large language models from glm-130b to glm-4 all tools. arXiv preprint arXiv:2406.12793

    2024 arXiv

  3. [11]

    Chaoqun He, Renjie Luo, Yuzhuo Bai, Shengding Hu, Zhen Leng Thai, Junhao Shen, Jinyi Hu, Xu Han, Yujie Huang, Yuxiang Zhang, et al. 2024. Olympiadbench: A challenging benchmark for promoting agi with olympiad-level bilingual multimodal scientific problems. arXiv preprint arXiv...

    2024 arXiv

  4. [12]

    Dan Hendrycks, Collin Burns, Saurav Kadavath, Akul Arora, Steven Basart, Eric Tang, Dawn Song, and Jacob Steinhardt. 2021. Measuring mathematical problem solving with the math dataset. NeurIPS

    2021

  5. [13]

    Ari Holtzman, Jan Buys, Li Du, Maxwell Forbes, and Yejin Choi. 2020. https://openreview.net/forum?id=rygGQyrFvH The curious case of neural text degeneration . In International Conference on Learning Representations

    2020

  6. [14]

    Dongfu Jiang, Xiang Ren, and Bill Yuchen Lin. 2023. Llm-blender: Ensembling large language models with pairwise ranking and generative fusion. arXiv preprint arXiv:2306.02561

    2023 arXiv

  7. [15]

    Pei Ke, Bosi Wen, Zhuoer Feng, Xiao Liu, Xuanyu Lei, Jiale Cheng, Shengyuan Wang, Aohan Zeng, Yuxiao Dong, Hongning Wang, et al. 2023. Critiquellm: Scaling llm-as-critic for effective and explainable evaluation of large language model generation. arXiv preprint arXiv:2311.18702

    2023 arXiv

  8. [16]

    Woosuk Kwon, Zhuohan Li, Siyuan Zhuang, Ying Sheng, Lianmin Zheng, Cody Hao Yu, Joseph Gonzalez, Hao Zhang, and Ion Stoica. 2023. Efficient memory management for large language model serving with pagedattention. In Proceedings of the 29th Symposium on Operating Systems Princip...

    2023

  9. [17]

    Nathan Lambert, Valentina Pyatkin, Jacob Morrison, LJ Miranda, Bill Yuchen Lin, Khyathi Chandu, Nouha Dziri, Sachin Kumar, Tom Zick, Yejin Choi, et al. 2024. Rewardbench: Evaluating reward models for language modeling. arXiv preprint arXiv:2403.13787

    2024 arXiv

  10. [18]

    Yujia Li, David Choi, Junyoung Chung, Nate Kushman, Julian Schrittwieser, R \'e mi Leblond, Tom Eccles, James Keeling, Felix Gimeno, Agustin Dal Lago, et al. 2022. Competition-level code generation with alphacode. Science, 378(6624):1092--1097

    2022

  11. [19]

    Hunter Lightman, Vineet Kosaraju, Yuri Burda, Harrison Edwards, Bowen Baker, Teddy Lee, Jan Leike, John Schulman, Ilya Sutskever, and Karl Cobbe. 2024. https://openreview.net/forum?id=v8L0pN6EOi Let's verify step by step . In The Twelfth International Conference on Learning Re...

    2024

  12. [20]

    I Loshchilov. 2017. Decoupled weight decay regularization. arXiv preprint arXiv:1711.05101

    2017 arXiv

  13. [21]

    Reiichiro Nakano, Jacob Hilton, Suchir Balaji, Jeff Wu, Long Ouyang, Christina Kim, Christopher Hesse, Shantanu Jain, Vineet Kosaraju, William Saunders, et al. 2021. Webgpt: Browser-assisted question-answering with human feedback. arXiv preprint arXiv:2112.09332

    2021 arXiv

  14. [22]

    Linyong Nan, Chiachun Hsieh, Ziming Mao, Xi Victoria Lin, Neha Verma, Rui Zhang, Wojciech Kry \'s ci \'n ski, Hailey Schoelkopf, Riley Kong, Xiangru Tang, et al. 2022. Fetaqa: Free-form table question answering. Transactions of the Association for Computational Linguistics, 10:35--49

    2022

  15. [23]

    Maxwell Nye, Anders Johan Andreassen, Guy Gur-Ari, Henryk Michalewski, Jacob Austin, David Bieber, David Dohan, Aitor Lewkowycz, Maarten Bosma, David Luan, et al. 2021. Show your work: Scratchpads for intermediate computation with language models. arXiv preprint arXiv:2112.00114

    2021 arXiv

  16. [24]

    Long Ouyang, Jeffrey Wu, Xu Jiang, Diogo Almeida, Carroll Wainwright, Pamela Mishkin, Chong Zhang, Sandhini Agarwal, Katarina Slama, Alex Ray, et al. 2022. Training language models to follow instructions with human feedback. Advances in neural information processing systems, 3...

    2022

  17. [25]

    Panupong Pasupat and Percy Liang. 2015. Compositional semantic parsing on semi-structured tables. arXiv preprint arXiv:1508.00305

    2015 arXiv

  18. [26]

    Jack W Rae, Sebastian Borgeaud, Trevor Cai, Katie Millican, Jordan Hoffmann, Francis Song, John Aslanides, Sarah Henderson, Roman Ring, Susannah Young, et al. 2021. Scaling language models: Methods, analysis & insights from training gopher. arXiv preprint arXiv:2112.11446

    2021 arXiv

  19. [27]

    Freda Shi, Daniel Fried, Marjan Ghazvininejad, Luke Zettlemoyer, and Sida I Wang. 2022. Natural language to code translation with execution. In Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing, pages 3533--3546

    2022

  20. [28]

    Prasann Singhal, Tanya Goyal, Jiacheng Xu, and Greg Durrett. 2023. A long way to go: Investigating length correlations in rlhf. arXiv preprint arXiv:2310.03716

    2023 arXiv

  21. [29]

    Joar Skalse, Nikolaus Howe, Dmitrii Krasheninnikov, and David Krueger. 2022. Defining and characterizing reward gaming. Advances in Neural Information Processing Systems, 35:9460--9471

    2022

  22. [30]

    Nisan Stiennon, Long Ouyang, Jeffrey Wu, Daniel Ziegler, Ryan Lowe, Chelsea Voss, Alec Radford, Dario Amodei, and Paul F Christiano. 2020. Learning to summarize with human feedback. Advances in Neural Information Processing Systems, 33:3008--3021

    2020

  23. [31]

    Yuxuan Tong, Xiwen Zhang, Rui Wang, Ruidong Wu, and Junxian He. 2024. Dart-math: Difficulty-aware rejection tuning for mathematical problem-solving. arXiv preprint arXiv:2407.13690

    2024 arXiv

  24. [32]

    Hugo Touvron, Louis Martin, Kevin Stone, Peter Albert, Amjad Almahairi, Yasmine Babaei, Nikolay Bashlykov, Soumya Batra, Prajjwal Bhargava, Shruti Bhosale, et al. 2023. Llama 2: Open foundation and fine-tuned chat models. arXiv preprint arXiv:2307.09288

    2023 arXiv

  25. [33]

    Jonathan Uesato, Nate Kushman, Ramana Kumar, Francis Song, Noah Siegel, Lisa Wang, Antonia Creswell, Geoffrey Irving, and Irina Higgins. 2022. Solving math word problems with process-and outcome-based feedback. arXiv preprint arXiv:2211.14275

    2022 arXiv

  26. [34]

    Giorgos Vernikos, Arthur Bra z inskas, Jakub Adamek, Jonathan Mallinson, Aliaksei Severyn, and Eric Malmi. 2023. Small language models improve giants by rewriting their outputs. arXiv preprint arXiv:2305.13514

    2023 arXiv

  27. [35]

    Giorgos Vernikos and Andrei Popescu-Belis. 2024. Don't rank, combine! combining machine translation hypotheses using quality estimation. arXiv preprint arXiv:2401.06688

    2024 arXiv

  28. [36]

    Haoxiang Wang, Wei Xiong, Tengyang Xie, Han Zhao, and Tong Zhang. 2024. Interpretable preferences via multi-objective reward modeling and mixture-of-experts. arXiv preprint arXiv:2406.12845

    2024 arXiv

  29. [37]

    Xuezhi Wang, Jason Wei, Dale Schuurmans, Quoc Le, Ed Chi, Sharan Narang, Aakanksha Chowdhery, and Denny Zhou. 2022. Self-consistency improves chain of thought reasoning in language models. arXiv preprint arXiv:2203.11171

    2022 arXiv

  30. [38]

    Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Fei Xia, Ed Chi, Quoc V Le, Denny Zhou, et al. 2022. Chain-of-thought prompting elicits reasoning in large language models. Advances in neural information processing systems, 35:24824--24837

    2022

  31. [39]

    Yangzhen Wu, Zhiqing Sun, Shanda Li, Sean Welleck, and Yiming Yang. 2024. An empirical analysis of compute-optimal inference for problem-solving with language models. arXiv preprint arXiv:2408.00724

    2024 arXiv

  32. [40]

    An Yang, Baosong Yang, Binyuan Hui, Bo Zheng, Bowen Yu, Chang Zhou, Chengpeng Li, Chengyuan Li, Dayiheng Liu, Fei Huang, et al. 2024. Qwen2 technical report. arXiv preprint arXiv:2407.10671

    2024 arXiv

  33. [41]

    Fei Yu, Anningzhe Gao, and Benyou Wang. 2023. Outcome-supervised verifiers for planning in mathematical reasoning. arXiv preprint arXiv:2311.09724

    2023 arXiv

  34. [42]

    Yue Yu, Zhengxing Chen, Aston Zhang, Liang Tan, Chenguang Zhu, Richard Yuanzhe Pang, Yundi Qian, Xuewei Wang, Suchin Gururangan, Chao Zhang, et al. 2024. Self-generated critiques boost reward modeling for language models. arXiv preprint arXiv:2411.16646

    2024 arXiv

  35. [43]

    Lunjun Zhang, Arian Hosseini, Hritik Bansal, Mehran Kazemi, Aviral Kumar, and Rishabh Agarwal. 2024 a . Generative verifiers: Reward modeling as next-token prediction. arXiv preprint arXiv:2408.15240

    2024 arXiv

  36. [44]

    Xiaokang Zhang, Jing Zhang, Zeyao Ma, Yang Li, Bohan Zhang, Guanlin Li, Zijun Yao, Kangli Xu, Jinchang Zhou, Daniel Zhang-Li, et al. 2024 b . Tablellm: Enabling tabular data manipulation by llms in real office usage scenarios. arXiv preprint arXiv:2403.19318

    2024 arXiv

  37. [45]

    online" 'onlinestring :=

    ENTRY address archivePrefix author booktitle chapter edition editor eid eprint eprinttype howpublished institution journal key month note number organization pages publisher school series title type volume year doi pubmed url lastchecked label extra.label sort.label short.list...

  38. [46]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 gl...

Pith tools

Reviewed August 10, 2026 · model on record in the stance chip above.