Pith. sign in

REVIEW 3 major objections 6 minor 68 references

APT: Improving Specialist LLM Performance with Weakness Case Acquisition and Iterative Preference Training

T0 review · 3 major / 6 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read APT claims that a specialist LLM can be improved by iteratively preference-training it on only its own low-scored answers plus a matching set of retrieved similar cases, without eroding general ability.

desk verdict A practical, well-ablated recipe for specializing LLMs on weak cases; the central claim holds up, but the load-bearing scorer and test-set-tuned hyperparameters need scrutiny in review. read the letter →

arxiv 2506.03483 v1 pith:2PPDFFXJ submitted 2025-06-04 cs.CL

Jun Rao , Zepeng Lin , Xuebo Liu , Xiaopeng Ke , Lian Lian , Dong Jin , Shengjun Cheng , Jun Yu
show 1 more author
Min Zhang
This is my paper
classification cs.CL
keywords preferenceoptimizationDPOiterativetrainingbadcasegenerationsimilarretrievaldomain-specificLLMcatastrophicforgettingspecialistmodels
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper tries to show that a specialist LLM can be sharpened on a target domain without dulling its general abilities by training only on the model's own mistakes. The proposed APT method has the model answer its training questions, uses an external scorer to flag the weakest answers, retrieves similar correct examples from a general instruction pool, and then runs iterative preference training on those pairs. On Llama-2 and Mistral-7B, this raises math, coding, and instruction-following scores by up to about six points over mixed-data training while leaving the six-benchmark general average flat or slightly higher. If true, it gives a cheap, task-agnostic recipe for upgrading a model where it is weak instead of re-training it on more positive data.

What carries the argument

The engine is a three-part loop. First, bad-case generation: the current model predicts answers for the domain SFT set, and an assessment model (Prometheus 2-7B) scores each prediction against the reference answer; predictions scoring below 4 become the dispreferred half of a preference pair, with the reference as the preferred half. Second, similar-case retrieval: each bad case is tagged (InsTag), the bad cases are averaged per tag into an embedding, and the most cosine-similar pool samples, one per bad case, are pulled in to form $\mathcal{D}_{\text{retrieval}}$. Third, iterative preference training: the model is tuned with the DPO objective plus an SFT-loss constraint, $\mathcal{L} = \mathcal{L}_{\text{DPO}} + \alpha \mathcal{L}_{\text{SFT}}$, and the new model re-enters the loop.

What would settle it

Check the scorer against verifiable ground truth: on GSM8K, take predictions the scorer rates below 4 and compare them to exact-match correctness. If a substantial share of those 'bad cases' are actually correct answers, the filter is not selecting true errors; then rerunning APT with only verified-wrong cases should change the gains, and rerunning it with random labels should erase them.

Watch

Extended reading notes

Core claim

On the paper's own terms, the central discovery is that negative self-generated examples are a sufficient training signal: the model's own low-scored outputs, used as dispreferred responses in DPO-style preference optimization, teach it to fix exactly the cases it gets wrong, and retrieving one similar correct example per bad case prevents overfitting and general-capacity loss. The method deliberately discards all high-scoring self-generated data and all unselected training data, training only on $\mathcal{D}_{\text{error}} \cup \mathcal{D}_{\text{retrieval}}$. Iterating the generate-score-retrieve-train loop further reduces the number of low-scored outputs and improves downstream scores, with the largest gain appearing in the first iteration.

Load-bearing premise

The method's load-bearing premise is that the external scorer's 1-to-5 score reliably tells genuine mistakes from acceptable answers; if the scorer is noisy or biased, the 'bad cases' and the preference signal built from them are wrong, and the promised domain gains would not transfer.

Editorial extensions

If this is right

  • Domain gains up to +5.9 on Llama-2-7B and +6.0 on Mistral-7B-V0.3 can be obtained by training only on error-derived preference data, beating continued SFT and data-mixing baselines.
  • General capability, measured as the average over MMLU, BBH, ARC, BoolQ, OpenBookQA, and WinoGrande, stays flat or rises slightly, so the selective training does not induce measurable forgetting.
  • Harder errors matter more: selecting only score-1 or score-below-4 predictions trains better and more efficiently than using all self-generated data.
  • Retrieving an equal number of similar cases (1x) is optimal; retrieving 2x or 3x reduces performance.
  • Iterating the loop improves the target domain further, and the method also lifts stronger models (13B and math-specialized checkpoints), suggesting the mechanism is not limited to weak models.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Because the bad-case filter is the only thing separating 'teach' from 'skip,' APT's ceiling is the scorer's judgment; a calibrated or verifiable scorer (e.g., exact-match on math) should raise the ceiling further, while a biased one would silently convert scoring errors into training errors.
  • The tag-based retrieval is a form of fine-grained hard-negative mining for preference data; this suggests that any cheap grouping signal, domain labels, clustering, or learned retrieval, could replace the tagger, and the paper's own ablation predicts finer grouping helps.
  • A direct stress test would use random selection of the same data volume: if random 'bad cases' perform nearly as well, the reported gains come from extra training data rather than from weakness targeting; the paper's comparison to non-selected 'Pred.' data already suggests selection matters, but random selection is the cleaner control.
  • APT should combine naturally with verifiable-answer domains such as math and code execution, where a pass/fail signal can replace the learned scorer and remove the weakest assumption.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 6 minor

Summary. The paper proposes APT, a framework for improving domain-specific LLM performance by iteratively (1) using a frozen assessment model (Prometheus 2-7B) to score the model's own predictions against ground truth, keeping only low-scoring 'bad cases' (score < 4) as dis-preferred data, (2) retrieving a matched number of similar cases from a large instruction pool via tag-based embedding similarity, and (3) optimizing a DPO objective regularized by an SFT loss. Experiments on Llama-2-7B and Mistral-7B-V0.3 report gains over mixed training and DMT baselines on GSM8K, HumanEval, and AlpacaEval while keeping the average of six general benchmarks roughly flat. Ablations examine the score threshold, retrieval method and scale, number of iterations, and the optimization objective, and scaling experiments on larger and math-specialized models are included.

Significance. If the reported gains are reproducible, APT is a useful, relatively simple addition to the specialist-finetuning toolbox: it turns a small amount of self-generated negative data plus retrieved neighbors into consistent domain improvements without degrading general benchmarks. The paper's ablations are reasonably complete: it isolates the contribution of bad-case filtering, similar-case retrieval, the tag-based retrieval granularity, the SFT regularizer, and the iterative loop, and it evaluates on two base models plus several stronger backbones. The use of independent domain benchmarks for the headline results (GSM8K exact match, HumanEval pass@10, AlpacaEval) means the central improvement claim is not vacuous. However, the paper's reliance on a single learned scorer for all data selection, and its selection of two key hyperparameters (the score threshold and retrieval scale) on the same test benchmarks used for the headline numbers, leave the central claim needing additional validation before it can be taken as established.

major comments (3)
  1. [§3.2, Limitations] The score<4 criterion from Prometheus 2-7B is the sole gate for constructing D_error, and the paper's own Limitations admit that 'the performance of our model is primarily constrained by the quality of the scoring information provided.' Table 2 only shows that Prometheus outperforms self-scoring on the final domain benchmarks; it does not establish that the score<4 labels correspond to genuinely erroneous predictions. Because the selected training data and the iterative reward signal both depend on this scorer, a systematic scorer bias (e.g., favoring longer or better-formatted answers) could produce the observed gains without actually fixing reasoning errors. I would like to see a validation of the scorer against a small human-annotated set or against exact match for GSM8K and unit tests for HumanEval, or a version of APT run with a task-specific correctness signal (exact match, unit tests) to confirm the mechanism is not scorer-artifact-driven.
  2. [§4.3, §4.4, §A.5] The bad-case threshold (score<4), the retrieval scale (1x), and the regularization weight alpha=0.5 are all selected by comparing variants on the same test benchmarks that appear in the headline Table 1 (Figure 3, Table 4, and Figure 8 in Appendix A.5). No held-out validation split is used for these choices, and no error bars or significance tests are reported. This selection-on-the-evaluation-data pattern makes the reported gains (e.g., +5.9 on Dolly for Llama-2) potentially optimistic. Please report multiple seeds or bootstrap confidence intervals, and either fix the hyperparameters a priori or tune them on a validation split disjoint from the test sets.
  3. [§4.6, Figure 6] Figure 6 measures the reduction in 'erroneous samples' using the same Prometheus <4 criterion that defined the bad-case training set. This makes the robustness analysis partly self-referential: the model may be learning to please the scorer rather than to be correct. The reduction in scorer-detected errors should be corroborated with the independent metrics already used elsewhere in the paper (exact match for GSM8K, pass@10 for HumanEval, and AlpacaEval win rate), and the two sources of evidence should be reported side by side.
minor comments (6)
  1. [Abstract and throughout] The model name 'LLama-2' should be 'Llama-2' consistently.
  2. [Table 7] In several rows of the 'Ours' entries, numbers are run together (e.g., '70.959.4'); please fix the spacing in the Avg column.
  3. [§4.4, Table 4 caption] The caption says the retrieval size is 'justified in the ablation study (Figure 3)', but Figure 3 concerns the score threshold, not retrieval scale; the justification is in Table 4 itself. Please correct the cross-reference.
  4. [Eq. (7)] The function ℓ in Eq. (7) is never defined; please state explicitly that it is the logistic loss, since the DPO derivation in Rafailov et al. uses the logistic function.
  5. [§3.5] The claim that TAG processing requires 'only an additional 5% of the total time' is vague: please specify the denominator (e.g., 5% of the bad-case-generation plus training time) and provide timing measurements rather than a single percentage.
  6. [Ethics Statement] The statement that code cannot be publicly linked weakens reproducibility; beyond contacting the authors, please consider releasing code and data in a non-company repository or providing a more detailed implementation appendix.

Circularity Check

1 steps flagged · score 2.0 of 10

APT's headline gains are measured on independent external benchmarks, so the central claim is not circular; however, the Figure 6 'error reduction' evidence uses the same Prometheus score<4 predicate that selects the training bad cases, making that robustness analysis self-referential.

  1. self definitional [Section 4.6 (Method Robustness) / Figure 6; cf. Section 3.2 and Figure 2]
    "By selecting data instance (xi, yi, ai) with low score si, we identify cases where the model struggles... We show the number of erroneous samples, in which assessment model judgment is less than 4 scores, before (Baseline) and after training (Ours) across three domains in Figure 6."

    The Figure 6 robustness metric counts 'erroneous samples' using exactly the same predicate (Prometheus score < 4) that Section 3.2 and Figure 2 use to select the bad cases that constitute D_error for training. Therefore the observed reduction in error cases is measured by the same judge that defines and selects the weakness cases; it cannot independently confirm that the model produces fewer genuine errors. This does not affect the main Table 1 gains, which come from external benchmarks (GSM8K exact match, HumanEval pass@10, AlpacaEval with GPT-3.5, and six general benchmarks), but it makes the Section 4.6 robustness evidence self-referential rather than independent.

full rationale

The paper's central claim is a training procedure whose headline results are evaluated on held-out external test sets (GSM8K exact match, HumanEval, AlpacaEval, MMLU/BBH/ARC/BoolQ/OpenBookQA/WinoGrande), none of which are defined by the Prometheus scorer. The bad-case selection in Section 3.2 gates training data on f_theta0(x,y,a) < 4, but the reported domain gains are not the same quantity as that gate, so the main result is not forced by construction. The Prometheus 2-7B scorer is an external model (Kim et al., 2024), and the paper's Limitations explicitly concede dependence on scoring quality; that is a correctness risk, not a definitional identity. The paper's self-citations (Rao et al. 2022, 2023a, 2024; Deng et al. 2023, 2024) are routine related-work references and are not load-bearing. Threshold (<4) and retrieval-scale (1x) are selected through ablations on the same benchmarks, which is a test-set selection risk rather than a circular derivation and is best filed under robustness/overfitting concerns. The only genuine self-referential element is the Figure 6 'number of error cases' evidence, which re-uses the same score<4 criterion that generated the training data; because the central benchmark comparison is independent, this warrants a score of 2 rather than higher.

Assumptions & free parameters 3 free parameters · 4 assumptions · 0 invented entities

The central claim rests on empirical assumptions about scorer reliability, retrieval transfer, the DPO+SFT objective, and the adequacy of the general benchmark suite, plus several empirically chosen hyperparameters. No new physical or mathematical entities are introduced.

free parameters (3)
  • Bad-case score threshold = Score < 4 on Prometheus 2-7B 1-5 scale
    Section 3.2 and Figure 3: selecting only low-scoring cases outperforms using all responses; this threshold is chosen empirically, not derived.
  • Retrieval data scale = 1x (equal number of retrieved cases as bad cases)
    Section 4.4, Table 4: 1x chosen as best; 2x and 3x degrade performance.
  • SFT/DPO regularization coefficient = 0.5
    Appendix A.5 and Figure 8: regularization parameter in Eq. 8 set after hyperparameter exploration; text is ambiguous whether it weights DPO or SFT.
assumptions (4)
  • domain assumption Prometheus 2-7B provides valid scalar quality scores aligned with human judgment for identifying model errors.
    Section 3.2 uses these scores to select bad cases; Limitations section concedes dependence on scoring model quality.
  • domain assumption Tag-based retrieval (InsTag plus MiniLM embeddings) surfaces similar cases whose preferred-dispreferred pairs transfer to error correction.
    Section 3.3 and ablations in Table 3; no theoretical guarantee that retrieval by tag/embedding similarity improves learning.
  • domain assumption DPO loss plus SFT loss on preference pairs improves domain behavior without catastrophic forgetting.
    Section 3.4, Eq. 8; optimization behavior is empirical, and the paper relies on prior DPO analyses for the loss form.
  • domain assumption General capability is adequately measured by the average of six selected benchmarks: MMLU, BBH, ARC, BoolQ, OpenBookQA, WinoGrande.
    Section A.4/A.7; this is a convenience sample, not a comprehensive general-ability measure.

how reviews work

0 comments
Cite this review

Pith. "Pith review of APT: Improving Specialist LLM Performance with Weakness Case Acquisition and Iterative Preference Training." pith.science (2026). https://pith.science/paper/2PPDFFXJ

@misc{pith2026250603483,
  author       = {Pith},
  title        = {Pith review of: APT: Improving Specialist LLM Performance with Weakness Case Acquisition and Iterative Preference Training},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/2PPDFFXJ}},
  note         = {Machine review of arXiv:2506.03483}
}
read the original abstract

Large Language Models (LLMs) often require domain-specific fine-tuning to address targeted tasks, which risks degrading their general capabilities. Maintaining a balance between domain-specific enhancements and general model utility is a key challenge. This paper proposes a novel approach named APT (Weakness Case Acquisition and Iterative Preference Training) to enhance domain-specific performance with self-generated dis-preferred weakness data (bad cases and similar cases). APT uniquely focuses on training the model using only those samples where errors occur, alongside a small, similar set of samples retrieved for this purpose. This targeted training minimizes interference with the model's existing knowledge base, effectively retaining generic capabilities. Experimental results on the LLama-2 and Mistral-V0.3 models across various benchmarks demonstrate that APT ensures no reduction in generic capacity and achieves superior performance on downstream tasks compared to various existing methods. This validates our method as an effective strategy for enhancing domain-specific capabilities without sacrificing the model's broader applicability.

Figures

Figures reproduced from arXiv: 2506.03483 by the authors.

Figure 1
Figure 1. Similar to how people improve in weaker areas [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. An overview of our approach APT. The main components of our method include: 1) Bad Case Generation: [PITH_FULL_IMAGE:figures/full_fig_p003_2.png] view at source ↗
Figure 3
Figure 3. The impact of different response scores and [PITH_FULL_IMAGE:figures/full_fig_p006_3.png] view at source ↗
Figures from the paper (5 more)
Figure 4
Figure 4. Figure 4: Domain evaluation of iteration. The iteration [PITH_FULL_IMAGE:figures/full_fig_p007_4.png]
Figure 7
Figure 7. Figure 7: Scalability on Stronger Models. (a) Applying [PITH_FULL_IMAGE:figures/full_fig_p008_7.png]
Figure 5
Figure 5. Figure 5: Comparing multiple different preference opti [PITH_FULL_IMAGE:figures/full_fig_p008_5.png]
Figure 6
Figure 6. Figure 6: Comparison of the Number of Error Cases across Methods. After our method’s training, the model’s error samples were significantly reduced com￾pared to the baseline and DMT. domain effect in three domains. We show that models trained with our objective learned to reflec…
Figure 8
Figure 8. Figure 8: The exploration of hyperparameters. preference loss is set to 0.5. A.6 Retrieval Details Our retrieval method builds on the error-preference dataset generated during the bad case generation step. Using the tag-based similarity method, we select an equal amount of retri…

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

68 extracted references · 26 canonical work pages

  1. [1]

    Jinze Bai, Shuai Bai, Yunfei Chu, Zeyu Cui, Kai Dang, Xiaodong Deng, Yang Fan, Wenbin Ge, Yu Han, Fei Huang, et al. 2023. https://arxiv.org/abs/2309.16609 Qwen technical report . arXiv preprint arXiv:2309.16609

    arXiv 2023

  2. [2]

    Sahil Chaudhary. 2023. Code alpaca: An instruction-following llama model for code generation. https://github.com/sahil280114/codealpaca

    2023

  3. [3]

    Mark Chen, Jerry Tworek, Heewoo Jun, Qiming Yuan, Henrique Pond \' e de Oliveira Pinto, Jared Kaplan, Harrison Edwards, Yuri Burda, Nicholas Joseph, Greg Brockman, et al. 2021. https://arxiv.org/abs/2107.03374 Evaluating large language models trained on code . arXiv preprint arXiv:2107.03374

    arXiv 2021

  4. [4]

    Zixiang Chen, Yihe Deng, Huizhuo Yuan, Kaixuan Ji, and Quanquan Gu. 2024. https://openreview.net/forum?id=O4cHTxW9BS Self-play fine-tuning converts weak language models to strong language models . In Forty-first International Conference on Machine Learning, ICML 2024, Vienna, Austria, July 21-27, 2024 . OpenReview.net

    work page 2024

  5. [5]

    Jiale Cheng, Yida Lu, Xiaotao Gu, Pei Ke, Xiao Liu, Yuxiao Dong, Hongning Wang, Jie Tang, and Minlie Huang. 2024. https://doi.org/10.18653/v1/2024.findings-emnlp.397 A uto D etect: Towards a unified framework for automated weakness detection in large language models . In Findings of the Association for Computational Linguistics: EMNLP 2024, pages 6786--68...

    work page doi:10.18653/v1/2024.findings-emnlp.397 2024

  6. [6]

    Hyung Won Chung, Le Hou, Shayne Longpre, Barret Zoph, Yi Tay, William Fedus, Yunxuan Li, Xuezhi Wang, Mostafa Dehghani, Siddhartha Brahma, et al. 2024. https://jmlr.org/papers/v25/23-0870.html Scaling instruction-finetuned language models . J. Mach. Learn. Res., 25:70:1--70:53

    work page 2024

  7. [7]

    Christopher Clark, Kenton Lee, Ming-Wei Chang, Tom Kwiatkowski, Michael Collins, and Kristina Toutanova. 2019. https://doi.org/10.18653/v1/N19-1300 B ool Q : Exploring the surprising difficulty of natural yes/no questions . In Proceedings of the 2019 Conference of the North A merican Chapter of the Association for Computational Linguistics: Human Language...

    doi:10.18653/v1/n19-1300 2019

  8. [8]

    Peter Clark, Isaac Cowhey, Oren Etzioni, Tushar Khot, Ashish Sabharwal, Carissa Schoenick, and Oyvind Tafjord. 2018. https://arxiv.org/abs/1803.05457 Think you have solved question answering? try arc, the ai2 reasoning challenge . arXiv preprint arXiv:1803.05457

    arXiv 2018

Show all 68 references
  1. [9]

    Karl Cobbe, Vineet Kosaraju, Mohammad Bavarian, Mark Chen, Heewoo Jun, Lukasz Kaiser, Matthias Plappert, Jerry Tworek, Jacob Hilton, Reiichiro Nakano, et al. 2021. https://arxiv.org/abs/2110.14168 Training verifiers to solve math word problems . arXiv preprint arXiv:2110.14168

    2021 arXiv

  2. [10]

    Mike Conover, Matt Hayes, Ankit Mathur, Jianwei Xie, Jun Wan, Sam Shah, Ali Ghodsi, Patrick Wendell, Matei Zaharia, and Reynold Xin. 2023. https://www.databricks.com/blog/2023/04/12/dolly-first-open-commercially-viable-instruction-tuned-llm Free dolly: Introducing the world's ...

    2023

  3. [11]

    Cover and P

    T. Cover and P. Hart. 1967. https://doi.org/10.1109/TIT.1967.1053964 Nearest neighbor pattern classification . IEEE Transactions on Information Theory, 13(1):21--27

    1967

  4. [12]

    Hexuan Deng, Liang Ding, Xuebo Liu, Meishan Zhang, Dacheng Tao, and Min Zhang. 2023. https://doi.org/10.1609/AAAI.V37I11.26497 Improving simultaneous machine translation with monolingual data . In Thirty-Seventh AAAI Conference on Artificial Intelligence, AAAI 2023, Thirty-Fif...

    2023 doi

  5. [13]

    Hexuan Deng, Wenxiang Jiao, Xuebo Liu, Min Zhang, and Zhaopeng Tu. 2024. https://proceedings.neurips.cc/paper_files/paper/2024/file/3eec719ab86712d32b065c5977f94ad0-Paper-Datasets_and_Benchmarks_Track.pdf Newterm: Benchmarking real-time new terms for large language models with...

    2024

  6. [14]

    Guanting Dong, Hongyi Yuan, Keming Lu, Chengpeng Li, Mingfeng Xue, Dayiheng Liu, Wei Wang, Zheng Yuan, Chang Zhou, and Jingren Zhou. 2024. https://doi.org/10.18653/v1/2024.acl-long.12 How abilities in large language models are affected by supervised fine-tuning data compositio...

    2024 doi

  7. [15]

    Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Abhishek Kadian, Ahmad Al-Dahle, Aiesha Letman, Akhil Mathur, Alan Schelten, Amy Yang, Angela Fan, et al. 2024. https://arxiv.org/abs/2407.21783 The llama 3 herd of models . arXiv preprint arXiv:2407.21783

    2024 arXiv

  8. [16]

    Hashimoto

    Yann Dubois, Chen Xuechen Li, Rohan Taori, Tianyi Zhang, Ishaan Gulrajani, Jimmy Ba, Carlos Guestrin, Percy Liang, and Tatsunori B. Hashimoto. 2023. https://papers.nips.cc/paper_files/paper/2023/hash/5fc47800ee5b30b8777fdd30abcaaf3b-Abstract-Conference.html Alpacafarm: A simul...

    2023

  9. [17]

    Leo Gao, Jonathan Tow, Baber Abbasi, Stella Biderman, Sid Black, Anthony DiPofi, Charles Foster, Laurence Golding, Jeffrey Hsu, Alain Le Noac'h, et al. 2024. https://doi.org/10.5281/zenodo.12608602 A framework for few-shot language model evaluation

    2024 doi

  10. [18]

    Dan Hendrycks, Collin Burns, Steven Basart, Andy Zou, Mantas Mazeika, Dawn Song, and Jacob Steinhardt. 2021. https://openreview.net/forum?id=d7KBjmI3GmQ Measuring massive multitask language understanding . In 9th International Conference on Learning Representations, ICLR 2021,...

    2021

  11. [19]

    Jiwoo Hong, Noah Lee, and James Thorne. 2024. https://doi.org/10.18653/v1/2024.emnlp-main.626 ORPO : Monolithic preference optimization without reference model . In Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing, pages 11170--11189, Miam...

    2024 doi

  12. [20]

    Or Honovich, Thomas Scialom, Omer Levy, and Timo Schick. 2023. https://doi.org/10.18653/v1/2023.acl-long.806 Unnatural instructions: Tuning language models with (almost) no human labor . In Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics...

    2023 doi

  13. [21]

    Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen - Zhu, Yuanzhi Li, Shean Wang, Lu Wang, and Weizhu Chen

    Edward J. Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen - Zhu, Yuanzhi Li, Shean Wang, Lu Wang, and Weizhu Chen. 2022. https://openreview.net/forum?id=nZeVKeeFYf9 Lora: Low-rank adaptation of large language models . In The Tenth International Conference on Learning Representat...

    2022

  14. [22]

    Hamish Ivison, Yizhong Wang, Jiacheng Liu, Zeqiu Wu, Valentina Pyatkin, Nathan Lambert, Noah A Smith, Yejin Choi, and Hannaneh Hajishirzi. 2024. https://openreview.net/forum?id=JMBWTlazjW Unpacking DPO and PPO : Disentangling best practices for learning from preference feedbac...

    2024

  15. [23]

    Ashvini Jindal. 2023. https://huggingface.co/akjindal53244/Arithmo-Mistral-7B Arithmo-mistral-7b: Mathematical reasoning model . Hugging Face

    2023

  16. [24]

    Vladimir Karpukhin, Barlas Oguz, Sewon Min, Patrick Lewis, Ledell Wu, Sergey Edunov, Danqi Chen, and Wen-tau Yih. 2020. https://doi.org/10.18653/v1/2020.emnlp-main.550 Dense passage retrieval for open-domain question answering . In Proceedings of the 2020 Conference on Empiric...

    2020 doi

  17. [25]

    Seungone Kim, Juyoung Suk, Shayne Longpre, Bill Yuchen Lin, Jamin Shin, Sean Welleck, Graham Neubig, Moontae Lee, Kyungjae Lee, and Minjoon Seo. 2024. https://doi.org/10.18653/v1/2024.emnlp-main.248 Prometheus 2: An open source language model specialized in evaluating other la...

    2024 doi

  18. [26]

    Xin Lai, Zhuotao Tian, Yukang Chen, Senqiao Yang, Xiangru Peng, and Jiaya Jia. 2024. https://arxiv.org/abs/2406.18629 Step-dpo: Step-wise preference optimization for long-chain reasoning of llms . arXiv preprint arXiv:2406.18629

    2024 arXiv

  19. [27]

    Hashimoto

    Xuechen Li, Tianyi Zhang, Yann Dubois, Rohan Taori, Ishaan Gulrajani, Carlos Guestrin, Percy Liang, and Tatsunori B. Hashimoto. 2023. Alpacaeval: An automatic evaluator of instruction-following models. https://github.com/tatsu-lab/alpaca\_eval

    2023

  20. [28]

    Wing Lian, Bleys Goodson, Eugene Pentland, Austin Cook, Chanvichet Vong, and "Teknium". 2023. Openorca: An open dataset of gpt augmented flan reasoning traces. https://https://huggingface.co/Open-Orca/OpenOrca

    2023

  21. [29]

    Le, Barret Zoph, Jason Wei, et al

    Shayne Longpre, Le Hou, Tu Vu, Albert Webson, Hyung Won Chung, Yi Tay, Denny Zhou, Quoc V. Le, Barret Zoph, Jason Wei, et al. 2023. https://proceedings.mlr.press/v202/longpre23a.html The flan collection: Designing data and methods for effective instruction tuning . In Internat...

    2023

  22. [30]

    Keming Lu, Hongyi Yuan, Zheng Yuan, Runji Lin, Junyang Lin, Chuanqi Tan, Chang Zhou, and Jingren Zhou. 2024. https://openreview.net/forum?id=pszewhybU9 \#instag: Instruction tagging for analyzing supervised fine-tuning of large language models . In The Twelfth International Co...

    2024

  23. [31]

    Todor Mihaylov, Peter Clark, Tushar Khot, and Ashish Sabharwal. 2018. https://doi.org/10.18653/v1/D18-1260 Can a suit of armor conduct electricity? a new dataset for open book question answering . In Proceedings of the 2018 Conference on Empirical Methods in Natural Language P...

    2018 doi

  24. [32]

    OpenAI. 2023. https://arxiv.org/abs/2303.08774 GPT-4 technical report . arXiv preprint arXiv:2303.08774

    2023 arXiv

  25. [33]

    Arka Pal, Deep Karkhanis, Samuel Dooley, Manley Roberts, Siddartha Naidu, and Colin White. 2024. https://arxiv.org/abs/2402.13228 Smaug: Fixing failure modes of preference optimisation with dpo-positive . arXiv preprint arXiv:2402.13228

    2024 arXiv

  26. [34]

    Jupinder Parmar, Shrimai Prabhumoye, Joseph Jennings, Mostofa Patwary, Sandeep Subramanian, Dan Su, Chen Zhu, Deepak Narayanan, Aastha Jhunjhunwala, Ayush Dattagupta, et al. 2024. https://arxiv.org/abs/2402.16819 Nemotron-4 15b technical report . arXiv preprint arXiv:2402.16819

    2024 arXiv

  27. [35]

    Baolin Peng, Chunyuan Li, Pengcheng He, Michel Galley, and Jianfeng Gao. 2023. https://arxiv.org/abs/2304.03277 Instruction tuning with GPT-4 . arXiv preprint arXiv:2304.03277

    2023 arXiv

  28. [36]

    Manning, Stefano Ermon, and Chelsea Finn

    Rafael Rafailov, Archit Sharma, Eric Mitchell, Christopher D. Manning, Stefano Ermon, and Chelsea Finn. 2023. https://papers.nips.cc/paper_files/paper/2023/hash/a85b405ed65c6477a4fe8302b5e06ce7-Abstract-Conference.html Direct preference optimization: Your language model is sec...

    2023

  29. [37]

    Jun Rao, Liang Ding, Shuhan Qi, Meng Fang, Yang Liu, Li Shen, and Dacheng Tao. 2023 a . https://doi.org/10.1109/TMM.2023.3236837 Dynamic contrastive distillation for image-text retrieval . IEEE Transactions on Multimedia, pages 1--13

    2023

  30. [38]

    Jun Rao, Xuebo Liu, Lian Lian, Shengjun Cheng, Yunjie Liao, and Min Zhang. 2024. https://doi.org/10.18653/v1/2024.emnlp-main.561 C ommon IT : Commonality-aware instruction tuning for large language models via data partitions . In EMNLP, pages 10064--10083, Miami, Florida, USA....

    2024 doi

  31. [39]

    Jun Rao, Xv Meng, Liang Ding, Shuhan Qi, Xuebo Liu, Min Zhang, and Dacheng Tao. 2023 b . https://doi.org/10.1109/TMM.2023.3321480 Parameter-efficient and student-friendly knowledge distillation . IEEE Trans. Multim. , pages 1--12

    2023

  32. [40]

    Jun Rao, Fei Wang, Liang Ding, Shuhan Qi, Yibing Zhan, Weifeng Liu, and Dacheng Tao. 2022. https://arxiv.org/pdf/2203.03853 Where does the performance improvement come from - a reproducibility concern about image-text retrieval . In SIGIR

    2022 arXiv

  33. [41]

    Jeff Rasley, Samyam Rajbhandari, Olatunji Ruwase, and Yuxiong He. 2020. https://dl.acm.org/doi/10.1145/3394486.3406703 Deepspeed: System optimizations enable training deep learning models with over 100 billion parameters . In KDD '20: The 26th ACM SIGKDD Conference on Knowledg...

    2020

  34. [42]

    Nils Reimers and Iryna Gurevych. 2019. https://doi.org/10.18653/v1/D19-1410 Sentence- BERT : Sentence embeddings using S iamese BERT -networks . In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference...

    2019 doi

  35. [43]

    Keisuke Sakaguchi, Ronan Le Bras, Chandra Bhagavatula, and Yejin Choi. 2020. https://doi.org/10.1609/AAAI.V34I05.6399 Winogrande: An adversarial winograd schema challenge at scale . In The Thirty-Fourth AAAI Conference on Artificial Intelligence, AAAI 2020, The Thirty-Second I...

    2020 doi

  36. [44]

    William Saunders, Catherine Yeh, Jeff Wu, Steven Bills, Long Ouyang, Jonathan Ward, and Jan Leike. 2022. https://arxiv.org/abs/2206.05802 Self-critiquing models for assisting human evaluators . arXiv preprint arXiv:2206.05802

    2022 arXiv

  37. [45]

    Sheng Shen, Le Hou, Yanqi Zhou, Nan Du, Shayne Longpre, Jason Wei, Hyung Won Chung, Barret Zoph, William Fedus, Xinyun Chen, et al. 2023. https://arxiv.org/abs/2305.14705 Flan-moe: Scaling instruction-finetuned language models with sparse mixture of experts . arXiv preprint ar...

    2023 arXiv

  38. [46]

    Chufan Shi, Yixuan Su, Cheng Yang, Yujiu Yang, and Deng Cai. 2023. https://doi.org/10.18653/v1/2023.emnlp-main.947 Specialist or generalist? instruction tuning for specific NLP tasks . In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, p...

    2023 doi

  39. [47]

    Mirac Suzgun, Nathan Scales, Nathanael Sch \"a rli, Sebastian Gehrmann, Yi Tay, Hyung Won Chung, Aakanksha Chowdhery, Quoc Le, Ed Chi, Denny Zhou, et al. 2023. https://doi.org/10.18653/v1/2023.findings-acl.824 Challenging BIG -bench tasks and whether chain-of-thought can solve...

    2023 doi

  40. [48]

    Hashimoto

    Rohan Taori, Ishaan Gulrajani, Tianyi Zhang, Yann Dubois, Xuechen Li, Carlos Guestrin, Percy Liang, and Tatsunori B. Hashimoto. 2023. Stanford alpaca: An instruction-following llama model. https://github.com/tatsu-lab/stanford\_alpaca

    2023

  41. [49]

    Smith, Iz Beltagy, et al

    Yizhong Wang, Hamish Ivison, Pradeep Dasigi, Jack Hessel, Tushar Khot, Khyathi Raghavi Chandu, David Wadden, Kelsey MacMillan, Noah A. Smith, Iz Beltagy, et al. 2023 a . https://papers.nips.cc/paper_files/paper/2023/hash/ec6413875e4ab08d7bc4d8e225263398-Abstract-Datasets_and_B...

    2023

  42. [50]

    Smith, Daniel Khashabi, and Hannaneh Hajishirzi

    Yizhong Wang, Yeganeh Kordi, Swaroop Mishra, Alisa Liu, Noah A. Smith, Daniel Khashabi, and Hannaneh Hajishirzi. 2023 b . https://doi.org/10.18653/v1/2023.acl-long.754 Self-instruct: Aligning language models with self-generated instructions . In Proceedings of the 61st Annual ...

    2023 doi

  43. [51]

    Yizhong Wang, Swaroop Mishra, Pegah Alipoormolabashi, Yeganeh Kordi, Amirreza Mirzaei, Atharva Naik, Arjun Ashok, Arut Selvan Dhanasekaran, Anjana Arunkumar, David Stap, et al. 2022. https://doi.org/10.18653/v1/2022.emnlp-main.340 Super- N atural I nstructions: Generalization ...

    2022 doi

  44. [52]

    Chi, Quoc V

    Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Brian Ichter, Fei Xia, Ed H. Chi, Quoc V. Le, and Denny Zhou. 2022. http://papers.nips.cc/paper\\\_files/paper/2022/hash/9d5609613524ecf4f15af0f7b31abca4-Abstract-Conference.html Chain-of-thought prompting elicits reasoni...

    2022

  45. [53]

    Zhiyong Wu, Yaoxiang Wang, Jiacheng Ye, and Lingpeng Kong. 2023. https://doi.org/10.18653/v1/2023.acl-long.79 Self-adaptive in-context learning: An information compression perspective for in-context example selection and ordering . In Proceedings of the 61st Annual Meeting of ...

    2023 doi

  46. [54]

    Can Xu, Qingfeng Sun, Kai Zheng, Xiubo Geng, Pu Zhao, Jiazhan Feng, Chongyang Tao, Qingwei Lin, and Daxin Jiang. 2024 a . https://openreview.net/forum?id=CfXh93NDgH Wizardlm: Empowering large pre-trained language models to follow complex instructions . In The Twelfth Internati...

    2024

  47. [55]

    Canwen Xu, Daya Guo, Nan Duan, and Julian McAuley. 2023. https://doi.org/10.18653/v1/2023.emnlp-main.385 Baize: An open-source chat model with parameter-efficient tuning on self-chat data . In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processi...

    2023 doi

  48. [56]

    Yifan Xu, Xiao Liu, Xinghan Liu, Zhenyu Hou, Yueyan Li, Xiaohan Zhang, Zihan Wang, Aohan Zeng, Zhengxiao Du, Zhao Wenyi, et al. 2024 b . https://doi.org/10.18653/v1/2024.findings-emnlp.569 C hat GLM -math: Improving math problem-solving in large language models with a self-cri...

    2024 doi

  49. [57]

    Zhaorui Yang, Tianyu Pang, Haozhe Feng, Han Wang, Wei Chen, Minfeng Zhu, and Qian Liu. 2024. https://doi.org/10.18653/v1/2024.acl-long.58 Self-distillation bridges distribution gap in language model fine-tuning . In Proceedings of the 62nd Annual Meeting of the Association for...

    2024 doi

  50. [58]

    Kwok, Zhenguo Li, Adrian Weller, and Weiyang Liu

    Longhui Yu, Weisen Jiang, Han Shi, Jincheng Yu, Zhengying Liu, Yu Zhang, James T. Kwok, Zhenguo Li, Adrian Weller, and Weiyang Liu. 2024. https://openreview.net/forum?id=N8N0hgNDRt Metamath: Bootstrap your own mathematical questions for large language models . In The Twelfth I...

    2024

  51. [59]

    Hansi Zeng, Hamed Zamani, and Vishwa Vinay. 2022. https://api.semanticscholar.org/CorpusID:248426770 Curriculum learning for dense retrieval distillation . Proceedings of the 45th International ACM SIGIR Conference on Research and Development in Information Retrieval

    2022

  52. [60]

    Weihao Zeng, Can Xu, Yingxiu Zhao, Jian-Guang Lou, and Weizhu Chen. 2024. https://doi.org/10.18653/v1/2024.emnlp-main.397 Automatic instruction evolving for large language models . In Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing, pages...

    2024 doi

  53. [61]

    Haokun Zhao, Haixia Han, Jie Shi, Chengyu Du, Jiaqing Liang, and Yanghua Xiao. 2024 a . https://arxiv.org/abs/2404.08707 Large language model can continue evolving from mistakes . arXiv preprint arXiv:2404.08707

    2024 arXiv

  54. [62]

    Siyan Zhao, John Dang, and Aditya Grover. 2024 b . https://openreview.net/forum?id=DpFeMH4l8Q Group preference optimization: Few-shot alignment of large language models . In The Twelfth International Conference on Learning Representations, ICLR 2024, Vienna, Austria, May 7-11,...

    2024

  55. [63]

    Xinping Zhao, Dongfang Li, Yan Zhong, Boren Hu, Yibin Chen, Baotian Hu, and Min Zhang. 2024 c . https://doi.org/10.18653/v1/2024.emnlp-main.178 SEER : Self-aligned evidence extraction for retrieval-augmented generation . In Proceedings of the 2024 Conference on Empirical Metho...

    2024 doi

  56. [64]

    Xinping Zhao, Yan Zhong, Zetian Sun, Xinshuo Hu, Zhenyu Liu, Dongfang Li, Baotian Hu, and Min Zhang. 2025. https://aclanthology.org/2025.findings-naacl.165/ F unnel RAG : A coarse-to-fine progressive retrieval paradigm for RAG . In Findings of the Association for Computational...

    2025

  57. [65]

    Chunting Zhou, Pengfei Liu, Puxin Xu, Srinivasan Iyer, Jiao Sun, Yuning Mao, Xuezhe Ma, Avia Efrat, Ping Yu, Lili Yu, et al. 2023. https://papers.nips.cc/paper_files/paper/2023/hash/ac662d74829e4407ce1d126477f4a03a-Abstract-Conference.html LIMA: less is more for alignment . In...

    2023

  58. [66]

    Qihao Zhu, Daya Guo, Zhihong Shao, Dejian Yang, Peiyi Wang, Runxin Xu, Y Wu, Yukun Li, Huazuo Gao, Shirong Ma, et al. 2024. https://arxiv.org/abs/2406.11931 Deepseek-coder-v2: Breaking the barrier of closed-source models in code intelligence . arXiv preprint arXiv:2406.11931

    2024 arXiv

  59. [67]

    URL: " 'urlintro :=

    ENTRY address author booktitle chapter edition editor howpublished institution journal key month note number organization pages publisher school series title type volume year eprint doi pubmed url lastchecked label extra.label sort.label short.list INTEGERS output.state before...

  60. [68]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 gl...

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.