Pith. sign in

REVIEW 2 major objections 5 minor 45 references

SCoder: Iterative Self-Distillation for Bootstrapping Small-Scale Data Synthesizers to Empower Code LLMs

T0 review · 2 major / 5 minor · reviewed 2026-08-15 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read This paper claims that small open-source code LLMs, bootstrapped by iterative self-distillation from just 10K proprietary samples, can generate instruction data that lets a 6.7B code model match or beat models trained on far larger…

desk verdict A real step forward in cheap code instruction data synthesis, but the LiveCodeBench headline is fitted, not tested. read the letter →

arxiv 2509.07858 v1 pith:MF76GHFS submitted 2025-09-09 cs.AI

Xinyu Zhang , Changzhi Zhou , Linmei Hu , Luhao Zhang , Xiancai Chen , Haomin Fu , Yang Yang , Mengdi Zhang This is my paper · ORCID
classification cs.AI
keywords codegenerationinstructiontuningdatasynthesisself-distillationsmalllanguagemodelsinfluenceestimationselectionsynthetictraining
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper tries to show that small open-source code LLMs (7B, 8B, 14B) can be turned into powerful generators of code instruction data with only a small seed of proprietary samples, eliminating the need to distill large volumes of data from costly APIs. The authors propose an iterative self-distillation loop in which the synthesizer is repeatedly trained on its own most influential self-generated samples, selected by multi-checkpoint sampling, multi-aspect scoring, and gradient-based influence estimation. Using 10K GPT-4o samples as a seed and two bootstrap iterations, the resulting synthesizers generate 60K instruction data on which a DeepSeek-Coder-6.7B-Base model is fine-tuned. The resulting SCoder models match or beat open-source baselines that were trained on more and larger proprietary-distilled datasets on HumanEval, MBPP, LiveCodeBench, and BigCodeBench. If this holds, high-quality code instruction data no longer requires expensive proprietary distillation.

What carries the argument

The load-bearing mechanism is the iterative self-distillation loop that alternates data generation with synthesizer training. Each iteration starts with M×N candidate problem-solution pairs sampled from M checkpoints of the current synthesizer per code snippet; a multi-aspect scorer evaluates each candidate across ten aspects and aggregates them into a weighted score, where the aspect weights w are fit by ridge regression over K=20 fine-tuning experiments evaluated on an out-of-distribution test set. The selected candidates are then filtered by gradient-based influence estimation: a LoRA reference model trained on the 10K proprietary samples is used to compute projected gradients for each self-distilled sample (via a Rademacher projection), and only samples whose gradients align with the average proprietary-sample gradient are kept. These influential samples train the synthesizer itself for the next iteration, and the final bootstrapped synthesizer generates the 60K instruction data used to fine-tune the target code model. The machinery's job is to keep the self-distillation loop from drifting toward low-quality or repetitive data while minimizing the number of proprietary samples needed.

What would settle it

Hold out a newer or disjoint set of code-generation problems (e.g., later LiveCodeBench months or another contest benchmark), refit the scoring weights without looking at it, regenerate the synthesizer, and rerun the main comparison; if the LiveCodeBench and BigCodeBench advantages vanish while HumanEval/MBPP gains persist, the headline numbers partly measure test-set tuning.

Watch

Extended reading notes

Core claim

The central claim is that instruction-data synthesis is not a capability reserved for large proprietary models: a 7B-14B open-source model, trained once on 10K proprietary synthesis samples, can bootstrap itself through iterative self-distillation into a synthesizer whose output trains a 6.7B target model to state-of-the-art or matching performance. In the paper's experiments, two bootstrap iterations with 20K and then 40K self-distilled samples raise the quality of the synthesizer's data on all four benchmarks; the best model, SCoder-Q14-DS-6.7B, surpasses the best open-source baseline by 5.9% on LiveCodeBench and 9.7% on BigCodeBench on average. The paper further claims that all three selection components are necessary: removing multi-checkpoint sampling, multi-aspect scoring, or gradient-based influence estimation degrades target-model performance by 4.8-8.1 points on the reported benchmarks. It also reports that synthesized data scores higher than the 110K evol-codealpaca-v1 dataset on all ten quality aspects judged by a GPT-4o-based evaluation.

Load-bearing premise

The scoring weights that decide which self-distilled samples are kept are tuned on LiveCodeBench, and the final models are then compared on LiveCodeBench, so some of the reported gain on that benchmark may be tuning to the test set rather than a general improvement in data quality.

Editorial extensions

If this is right

  • With only 10K proprietary samples as seed, two bootstrap iterations let a Qwen2.5-Coder-14B synthesizer lift a DeepSeek-Coder-6.7B target from 65.3 to 70.1 HumanEval and 73.7 to 76.5 MBPP.
  • The final SCoder models, trained on 60K synthesizer-generated data plus the common 110K evol-codealpaca-v1 warm-up, match or beat open-source baselines that use larger proprietary-distilled datasets (75K-110K+).
  • Ablations show that each selection component contributes: removing multi-checkpoint sampling, multi-aspect scoring, or gradient-based influence estimation costs at least 4.8 points on HumanEval and 8.1 points on LiveCodeBench.
  • Data scaling holds: increasing synthesized data size improves the target model across benchmarks, surpassing DeepSeek-Coder-6.7B-Instruct on most benchmarks.
  • The bootstrap stabilizes after two iterations; a third iteration yields no further gains, suggesting a natural stopping point.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The method's convergence argument (Banach fixed point, Nash equilibrium framing) is domain-agnostic; the same loop should transfer to math, general instruction, or other structured generation tasks if a seed set of high-quality samples and an appropriate snippet pool exist.
  • The gradient-influence filter's anchor is the proprietary seed set, not the test set; if the anchor were replaced by any small high-quality reference set, the pipeline could become fully open-source without relying on proprietary models at all.
  • The cost analysis implies that the one-time 10K API seed is the dominant proprietary cost; after training, the synthesizer generates instruction data at a marginal cost far below API distillation, so the method's advantage grows with scale.
  • Because the multi-aspect scoring weights are fitted on LiveCodeBench, the method's reported advantage on that benchmark may be optimistic; on benchmarks not used in weight fitting (HumanEval and MBPP), the gains are real but smaller.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

2 major / 5 minor

Summary. The paper proposes SCoder, a method for bootstrapping small open-source LLMs (7B-14B) into effective synthesizers of code instruction data, thereby reducing reliance on proprietary-LLM distillation. The synthesizers are first trained on 10K GPT-4o distilled samples and then improved through two iterations of self-distillation. At each iteration, candidate instruction samples are generated by multi-checkpoint sampling, scored by a multi-aspect scorer with ridge-regression weights w, and filtered by gradient-based influence estimation against the proprietary seed data. The resulting instruction data are used to fine-tune DeepSeek-Coder-6.7B-Base (after a shared 110K evol-codealpaca-v1 stage). The main results (Table 3) show that SCoder-Q14-DS-6.7B outperforms or matches open-source baselines on HumanEval, MBPP, LiveCodeBench, and BigCodeBench; ablations (Table 4) show that each proposed component contributes; data-scaling experiments (Figure 3) and transfer to other base models (Table 8) support the generality of the approach.

Significance. If the results hold, this is a valuable empirical contribution: it demonstrates that small open-source models can produce instruction data competitive with proprietary-LLM-distilled data, at substantially lower API cost. The paper is careful to control for the shared 110K evol-codealpaca-v1 base across baselines, and it provides ablations for all three proposed components, a data-scaling study, a cost analysis, and transfer experiments on additional base models. These are concrete strengths. The main caveat is that the multi-aspect scoring weights are fitted on LiveCodeBench, which is also a headline evaluation benchmark, so the reported LiveCodeBench advantage is not a clean held-out result; the other benchmark gains are less affected.

major comments (2)
  1. [§3.3, Eq. (2); Appendix C; Table 3] The multi-aspect scoring weights w in Eq. (2) are estimated by ridge regression on K=20 experiments evaluated on LiveCodeBench (202410-202501), as stated in Appendix C. LiveCodeBench then appears as a headline benchmark in Table 3 (LCB Full and its sub-splits). Consequently, the reported LiveCodeBench advantage of SCoder-Q14 over the best open-source baseline (22.2 vs 21.0, claimed as a 5.9% gain in §4.4) is not a clean held-out comparison: part of the gap may reflect tuning w to that benchmark. The ablation in Table 4 shows that removing multi-aspect scoring drops LiveCodeBench from 21.4 to 19.9, so this component is a major contributor to the reported gain. I request that the authors either refit w on a benchmark not used in Table 3 and demonstrate that the LiveCodeBench gains survive, or report LiveCodeBench only as a development-set result, or show that a fixed a-priori weight vector yields similar LiveCodeBench performance. The HumanEval, MBPP, and BigCodeBench comparisons are not affected by this coupling and should be emphasized accordingly.
  2. [Appendix E, Eqs. (7)-(9)] The convergence analysis rests on the contraction condition L_T L_G < 1 stated as Assumption (A3), but no evidence is provided that this condition holds for the actual training operator T and data generation map G used in the experiments; the Lipschitz constants are not estimated. As written, the result is a conditional statement about an abstract operator, not a proof that the iterative self-distillation procedure used in the paper converges. Please either remove or substantially soften the theoretical claim, or provide an empirical check (e.g., estimate the relevant Lipschitz constants on the actual models) and state clearly that the contraction condition is an unverified assumption.
minor comments (5)
  1. [Abstract and §1] The abstract and introduction state that SCoder achieves 'state-of-the-art code generation capabilities' without qualifying that the comparison is against open-source models; Table 3 shows that GPT-o1-Preview substantially outperforms SCoder on several benchmarks. Please qualify the claim.
  2. [§4.4] The sentence 'surpasses the best open-source baselines by 5.9% and 9.7% on average' should specify whether the percentages are relative or absolute, and it should identify which sub-benchmarks are being averaged, since Table 3 reports multiple LiveCodeBench and BigCodeBench columns.
  3. [§3.3] Calling LiveCodeBench an 'out-of-distribution (OOD) test set' is confusing because the same benchmark family is used for the main evaluation in Table 3; unless the exact problem split used for weight fitting is disjoint from the evaluated split, a term such as 'development set' would be more accurate.
  4. [Figure 4] The legend '60K null; 40K 40K; 20K 20K; 40K 20K' is not self-explanatory; please clarify what the two numbers denote (e.g., amount of self-distilled data used in iteration 1 and iteration 2).
  5. [Appendix G and Table 7] The reference-model sensitivity study reports only HumanEval and LiveCodeBench; since LiveCodeBench is the benchmark coupled to weight fitting, please also report MBPP and BigCodeBench for these sensitivity runs.

Circularity Check

1 steps flagged · score 6.0 of 10

LiveCodeBench headline result is tuned: weight vector w in Eq. (2) is fitted on LiveCodeBench, which is also Table 3's key benchmark.

  1. fitted input called prediction [Section 3.3 (Eq. 2), Appendix C; headline claim in Section 4.4 / Table 3]
    "To derive the weight vector w, we conduct K=20 experiments and evaluate the results on LiveCodeBench (202410-202501). ... The fine-tuned model is then evaluated on an out-of-distribution (OOD) test set to obtain the corresponding performance score y_k. ... SCoder-Q14-DS-6.7B surpasses the best open-source baselines by 5.9% and 9.7% on average in the challenging LiveCodeBench and BigCodeBench, respectively."

    The weight vector w in Eq. (2) is estimated by ridge regression to maximize target-model performance on LiveCodeBench, as stated in Appendix C. The same LiveCodeBench benchmark then appears in Table 3 as one of the headline evaluation sets, and the paper specifically highlights the LiveCodeBench advantage (5.9% over the best baseline). Because w directly controls which self-distilled samples are retained in the data-selection pipeline, the reported LiveCodeBench result is not a fresh held-out prediction; it is the outcome of a model-selection procedure tuned on that benchmark. The gains on HumanEval, MBPP, and BigCodeBench were not used to fit w and remain independent, but the LiveCodeBench-specific superiority is partially forced by the fitting.

full rationale

The main circularity is the coupling between the fitted multi-aspect scoring weights and the headline LiveCodeBench evaluation. Appendix C explicitly states that the K=20 experiments used to solve Eq. (2) are evaluated on LiveCodeBench (202410-202501), and Table 3 reports LiveCodeBench (Full) as a central result. Thus the LiveCodeBench numbers are not a clean held-out test of the method; they reflect tuning of the data-selection weights on that benchmark. This matches the 'fitted input called prediction' pattern and makes the LiveCodeBench-specific claim partially circular. However, the paper's broader claim of matching or outperforming baselines also rests on HumanEval, MBPP, and BigCodeBench, none of which were used to fit w, so the central result does not collapse. I found no load-bearing self-citation: the cited prior work by overlapping authors (e.g., RefineCoder) is only mentioned in related work and is not used to justify any core premise. The theoretical analysis in Appendix E is a conditional contraction-mapping argument with explicitly stated assumptions, not a circular derivation. The gradient-based influence estimation is transparently anchored to the 10K proprietary seed samples, which is a design choice rather than a circular step. Overall, the circularity is partial and confined to the LiveCodeBench headline advantage.

Assumptions & free parameters 2 free parameters · 3 assumptions · 0 invented entities

The central result rests on two fitted/tuned elements: the scorer weights (fitted to a benchmark) and the schedule/hyperparameters. The theoretical convergence argument depends on an unverified contraction assumption. No new physical or conceptual entities are introduced.

free parameters (2)
  • scoring aspect weights w = not reported
    Fit by ridge regression in Eq. (2) to maximize fine-tuned model performance on LiveCodeBench across K=20 experiments.
  • hyperparameters M, N, K, lambda, data schedule = M=5, N=3, K=20, 20K/40K schedule
    Chosen by the authors; the sensitivity analysis in Figure 4 shows the schedule matters, so these are not innocuous.
assumptions (3)
  • domain assumption Training on a few proprietary-LLM distillation samples enhances the synthesis capability of small LLMs.
    Supported only by the specific experiments in Table 1; may not generalize across model families or domains.
  • domain assumption Gradient similarity between a self-distilled sample and the average gradient of proprietary samples is a valid proxy for sample influence on target-model performance.
    Borrowed from LESS/TRAK; not verified for this pipeline beyond the final benchmark numbers.
  • ad hoc to paper The contraction condition L_T*L_G < 1 in Appendix E holds for the actual training and data-generation processes.
    No evidence is provided that these Lipschitz constants exist or are bounded below 1 for neural networks; the convergence proof is conditional on an unverified assumption.

how reviews work

0 comments
Cite this review

Pith. "Pith review of SCoder: Iterative Self-Distillation for Bootstrapping Small-Scale Data Synthesizers to Empower Code LLMs." pith.science (2026). https://pith.science/paper/MF76GHFS

@misc{pith2026250907858,
  author       = {Pith},
  title        = {Pith review of: SCoder: Iterative Self-Distillation for Bootstrapping Small-Scale Data Synthesizers to Empower Code LLMs},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/MF76GHFS}},
  note         = {Machine review of arXiv:2509.07858}
}
read the original abstract

Existing code large language models (LLMs) often rely on large-scale instruction data distilled from proprietary LLMs for fine-tuning, which typically incurs high costs. In this paper, we explore the potential of small-scale open-source LLMs (e.g., 7B) as synthesizers for high-quality code instruction data construction. We first observe that the data synthesis capability of small-scale LLMs can be enhanced by training on a few superior data synthesis samples from proprietary LLMs. Building on this, we propose a novel iterative self-distillation approach to bootstrap small-scale LLMs, transforming them into powerful synthesizers that reduce reliance on proprietary LLMs and minimize costs. Concretely, in each iteration, to obtain diverse and high-quality self-distilled data, we design multi-checkpoint sampling and multi-aspect scoring strategies for initial data selection. Furthermore, to identify the most influential samples, we introduce a gradient-based influence estimation method for final data filtering. Based on the code instruction datasets from the small-scale synthesizers, we develop SCoder, a family of code generation models fine-tuned from DeepSeek-Coder. SCoder models achieve state-of-the-art code generation capabilities, demonstrating the effectiveness of our method.

Figures

Figures reproduced from arXiv: 2509.07858 by the authors.

Figure 1
Figure 1. Left: The performance of code generation models on HumanEval using data provided by different synthesizers (Qwen2.5-Coder-7B or -14B). Right: The performance of our SCoder and the baseline. SCoder uses 60K instruction data generated by a small-scale syn￾thesizer, and the baseline uses 75K instruction data gen￾erated by proprietary LLMs. All code generation mod￾els are fine-tuned from DeepSeek-Coder-6.7B-Base. with u… view at source ↗
Figure 2
Figure 2. Overview of our iterative self-distillation bootstrap method. In each iteration, we sample outputs from multiple checkpoints and evaluate them with a multi-aspect scorer for diversity and reliability. We then use a gradient-based influence estimation method to select the most influential samples, which is done by evaluating the gradient similarity between the self-distilled and proprietary LLM-distilled code instruc… view at source ↗
Figure 3
Figure 3. Impact of data scaling. The dashed lines [PITH_FULL_IMAGE:figures/full_fig_p008_3.png] view at source ↗
Figures from the paper (3 more)
Figure 4
Figure 4. Figure 4: Comparison of different selection methods [PITH_FULL_IMAGE:figures/full_fig_p008_4.png]
Figure 6
Figure 6. Figure 6: Data Synthesis Prompt [PITH_FULL_IMAGE:figures/full_fig_p016_6.png]
Figure 7
Figure 7. Figure 7: Multi-Aspect Scoring Prompt [PITH_FULL_IMAGE:figures/full_fig_p017_7.png]

Review history (2 revisions) →

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

45 extracted references · 11 canonical work pages

  1. [1]

    Dai, Anja Hauth, Katie Millican, and et al

    Rohan Anil, Sebastian Borgeaud, Yonghui Wu, Jean - Baptiste Alayrac, Jiahui Yu, Radu Soricut, Johan Schalkwyk, Andrew M. Dai, Anja Hauth, Katie Millican, and et al. 2023. https://doi.org/10.48550/ARXIV.2312.11805 Gemini: A family of highly capable multimodal models . CoRR, abs/2312.11805

    arXiv doi:10.48550/arxiv.2312.11805 2023

  2. [2]

    Nye, Maarten Bosma, Henryk Michalewski, David Dohan, Ellen Jiang, Carrie J

    Jacob Austin, Augustus Odena, Maxwell I. Nye, Maarten Bosma, Henryk Michalewski, David Dohan, Ellen Jiang, Carrie J. Cai, Michael Terry, Quoc V. Le, and Charles Sutton. 2021. https://arxiv.org/abs/2108.07732 Program synthesis with large language models . CoRR, abs/2108.07732

    arXiv 2021

  3. [3]

    Amos Azaria, Rina Azoulay, and Shulamit Reches. 2024. https://doi.org/10.1162/dint\_a\_00235 Chatgpt is a remarkable tool—for experts . Data Intelligence, 6(1):240--296

    doi:10.1162/dint 2024

  4. [4]

    Sahil Chaudhary. 2023. Code alpaca: An instruction-following llama model for code generation. https://github.com/sahil280114/codealpaca

    2023

  5. [5]

    Huajun Chen. 2024. https://doi.org/10.3724/2096-7004.di.2024.0001 Large knowledge model: Perspectives and challenges . Data Intelligence, 6(3):587--620

    arXiv 2024

  6. [6]

    Mark Chen, Jerry Tworek, Heewoo Jun, Qiming Yuan, Henrique Pond \' e de Oliveira Pinto, Jared Kaplan, Harri Edwards, Yuri Burda, Nicholas Joseph, Greg Brockman, Alex Ray, Raul Puri, and et al. 2021. https://arxiv.org/abs/2107.03374 Evaluating large language models trained on code . CoRR, abs/2107.03374

    arXiv 2021

  7. [7]

    Aakanksha Chowdhery, Sharan Narang, Jacob Devlin, Maarten Bosma, Gaurav Mishra, Adam Roberts, Paul Barham, Hyung Won Chung, Charles Sutton, Sebastian Gehrmann, Parker Schuh, Kensen Shi, and et al. 2023. https://jmlr.org/papers/v24/22-1144.html Palm: Scaling language modeling with pathways . J. Mach. Learn. Res., 24:240:1--240:113

    work page 2023

  8. [8]

    Bosheng Ding, Chengwei Qin, Ruochen Zhao, Tianze Luo, Xinze Li, Guizhen Chen, Wenhan Xia, Junjie Hu, Anh Tuan Luu, and Shafiq Joty. 2024. https://doi.org/10.18653/V1/2024.FINDINGS-ACL.97 Data augmentation using llms: Data perspectives, learning paradigms and challenges . In Findings of the Association for Computational Linguistics, ACL 2024, Bangkok, Thai...

    doi:10.18653/v1/2024.findings-acl.97 2024

Show all 45 references
  1. [9]

    Daya Guo, Qihao Zhu, Dejian Yang, Zhenda Xie, Kai Dong, Wentao Zhang, Guanting Chen, Xiao Bi, Y. Wu, Y. K. Li, Fuli Luo, Yingfei Xiong, and Wenfeng Liang. 2024. https://doi.org/10.48550/ARXIV.2401.14196 Deepseek-coder: When the large language model meets programming - the rise...

    2024 doi arXiv

  2. [10]

    Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen - Zhu, Yuanzhi Li, Shean Wang, Lu Wang, and Weizhu Chen

    Edward J. Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen - Zhu, Yuanzhi Li, Shean Wang, Lu Wang, and Weizhu Chen. 2022. https://openreview.net/forum?id=nZeVKeeFYf9 Lora: Low-rank adaptation of large language models . In The Tenth International Conference on Learning Representat...

    2022

  3. [11]

    Binyuan Hui, Jian Yang, Zeyu Cui, Jiaxi Yang, Dayiheng Liu, Lei Zhang, Tianyu Liu, Jiajun Zhang, Bowen Yu, Kai Dang, and et al. 2024. https://doi.org/10.48550/ARXIV.2409.12186 Qwen2.5-coder technical report . CoRR, abs/2409.12186

    2024 doi arXiv

  4. [12]

    Naman Jain, King Han, Alex Gu, Wen - Ding Li, Fanjia Yan, Tianjun Zhang, Sida Wang, Armando Solar - Lezama, Koushik Sen, and Ion Stoica. 2024. https://doi.org/10.48550/ARXIV.2403.07974 Livecodebench: Holistic and contamination free evaluation of large language models for code ...

    2024 doi arXiv

  5. [13]

    William B Johnson, Joram Lindenstrauss, et al. 1984. Extensions of lipschitz mappings into a hilbert space. Contemporary mathematics, 26(189-206):1

    1984

  6. [14]

    Raymond Li, Loubna Ben Allal, Yangtian Zi, Niklas Muennighoff, Denis Kocetkov, Chenghao Mou, Marc Marone, Christopher Akiki, Jia Li, Jenny Chim, Qian Liu, and et al. 2023. https://openreview.net/forum?id=KoFOg41haE Starcoder: may the source be with you! Trans. Mach. Learn. Res., 2023

    2023

  7. [15]

    Choi, Junyoung Chung, Nate Kushman, Julian Schrittwieser, R \' e mi Leblond, Tom Eccles, James Keeling, Felix Gimeno, Agustin Dal Lago, and et al

    Yujia Li, David H. Choi, Junyoung Chung, Nate Kushman, Julian Schrittwieser, R \' e mi Leblond, Tom Eccles, James Keeling, Felix Gimeno, Agustin Dal Lago, and et al. 2022. https://doi.org/10.48550/ARXIV.2203.07814 Competition-level code generation with alphacode . CoRR, abs/2203.07814

    2022 doi arXiv

  8. [16]

    Jiawei Liu, Chunqiu Steven Xia, Yuyao Wang, and Lingming Zhang. 2023. http://papers.nips.cc/paper\_files/paper/2023/hash/43e9d647ccd3e4b7b5baab53f0368686-Abstract-Conference.html Is your code generated by chatgpt really correct? rigorous evaluation of large language models for...

    2023

  9. [17]

    Anton Lozhkov, Raymond Li, Loubna Ben Allal, Federico Cassano, Joel Lamy - Poirier, Nouamane Tazi, Ao Tang, Dmytro Pykhtar, Jiawei Liu, Yuxiang Wei, and et al. 2024. https://doi.org/10.48550/ARXIV.2402.19173 Starcoder 2 and the stack v2: The next generation . CoRR, abs/2402.19173

    2024 doi arXiv

  10. [18]

    Ziyang Luo, Can Xu, Pu Zhao, Qingfeng Sun, Xiubo Geng, Wenxiang Hu, Chongyang Tao, Jing Ma, Qingwei Lin, and Daxin Jiang. 2024. https://openreview.net/forum?id=UnUwSIgK5W Wizardcoder: Empowering code large language models with evol-instruct . In The Twelfth International Confe...

    2024

  11. [19]

    Niklas Muennighoff, Qian Liu, Armel Randy Zebaze, Qinkai Zheng, Binyuan Hui, Terry Yue Zhuo, Swayam Singh, Xiangru Tang, Leandro von Werra, and Shayne Longpre. 2024. https://openreview.net/forum?id=mw1PWNSWZP Octopack: Instruction tuning code large language models . In The Twe...

    2024

  12. [20]

    Erik Nijkamp, Bo Pang, Hiroaki Hayashi, Lifu Tu, Huan Wang, Yingbo Zhou, Silvio Savarese, and Caiming Xiong. 2023. https://openreview.net/forum?id=iaYcJKpY2B\_ Codegen: An open large language model for code with multi-turn program synthesis . In The Eleventh International Conf...

    2023

  13. [21]

    OpenAI. 2023. https://doi.org/10.48550/ARXIV.2303.08774 GPT-4 technical report . CoRR, abs/2303.08774

    2023 doi arXiv

  14. [22]

    OpenAI. 2024 a . Gpt-4. https://openai.com/index/gpt-4-research/

    2024

  15. [23]

    OpenAI. 2024 b . Learning to reason with llms. https://openai.com/index/learning-to-reason-with-llms/

    2024

  16. [24]

    Sung Min Park, Kristian Georgiev, Andrew Ilyas, Guillaume Leclerc, and Aleksander Madry. 2023. https://proceedings.mlr.press/v202/park23c.html TRAK: attributing model behavior at scale . In International Conference on Machine Learning, ICML 2023, 23-29 July 2023, Honolulu, Haw...

    2023

  17. [25]

    Garima Pruthi, Frederick Liu, Satyen Kale, and Mukund Sundararajan. 2020. https://proceedings.neurips.cc/paper/2020/hash/e6385d39ec9394f2f3a354d9d2b88eec-Abstract.html Estimating training data influence by tracing gradient descent . In Advances in Neural Information Processing...

    2020

  18. [26]

    Baptiste Rozi \` e re, Jonas Gehring, Fabian Gloeckle, Sten Sootla, Itai Gat, Xiaoqing Ellen Tan, Yossi Adi, Jingyu Liu, Tal Remez, and et al. 2023. https://doi.org/10.48550/ARXIV.2308.12950 Code llama: Open foundation models for code . CoRR, abs/2308.12950

    2023 doi arXiv

  19. [27]

    Gardner, Yiming Yang, Milad Hashemi, Graham Neubig, Parthasarathy Ranganathan, Osbert Bastani, and Amir Yazdanbakhsh

    Alexander Shypula, Aman Madaan, Yimeng Zeng, Uri Alon, Jacob R. Gardner, Yiming Yang, Milad Hashemi, Graham Neubig, Parthasarathy Ranganathan, Osbert Bastani, and Amir Yazdanbakhsh. 2024. https://openreview.net/forum?id=ix7rLVHXyY Learning performance-improving code edits . In...

    2024

  20. [28]

    Zifan Song, Yudong Wang, Wenwei Zhang, Kuikun Liu, Chengqi Lyu, Demin Song, Qipeng Guo, Hang Yan, Dahua Lin, Kai Chen, and Cairong Zhao. 2024. https://arxiv.org/abs/2405.19265 Alchemistcoder: Harmonizing and eliciting code capability by hindsight tuning on multi-source data . ...

    2024 arXiv

  21. [29]

    Ziegler, Ryan Lowe, Chelsea Voss, Alec Radford, Dario Amodei, and Paul Christiano

    Nisan Stiennon, Long Ouyang, Jeff Wu, Daniel M. Ziegler, Ryan Lowe, Chelsea Voss, Alec Radford, Dario Amodei, and Paul Christiano. 2022. https://arxiv.org/abs/2009.01325 Learning to summarize from human feedback . Preprint, arXiv:2009.01325

    2022 arXiv

  22. [30]

    Smith, Luke Zettlemoyer, and Tao Yu

    Hongjin Su, Weijia Shi, Jungo Kasai, Yizhong Wang, Yushi Hu, Mari Ostendorf, Wen - tau Yih, Noah A. Smith, Luke Zettlemoyer, and Tao Yu. 2023. https://doi.org/10.18653/V1/2023.FINDINGS-ACL.71 One embedder, any task: Instruction-finetuned text embeddings . In Findings of the As...

    2023 doi

  23. [31]

    Yaoxiang Wang, Haoling Li, Xin Zhang, Jie Wu, Xiao Liu, Wenxiang Hu, Zhongxin Guo, Yangyu Huang, Ying Xin, Yujiu Yang, et al. 2025. Epicoder: Encompassing diversity and complexity in code generation. arXiv preprint arXiv:2501.04694

    2025

  24. [32]

    Smith, Daniel Khashabi, and Hannaneh Hajishirzi

    Yizhong Wang, Yeganeh Kordi, Swaroop Mishra, Alisa Liu, Noah A. Smith, Daniel Khashabi, and Hannaneh Hajishirzi. 2023. https://doi.org/10.18653/V1/2023.ACL-LONG.754 Self-instruct: Aligning language models with self-generated instructions . In Proceedings of the 61st Annual Mee...

    2023 doi

  25. [33]

    Yuxiang Wei, Zhe Wang, Jiawei Liu, Yifeng Ding, and Lingming Zhang. 2024. https://openreview.net/forum?id=XUeoOBid3x Magicoder: Empowering code generation with oss-instruct . In Forty-first International Conference on Machine Learning, ICML 2024, Vienna, Austria, July 21-27, 2...

    2024

  26. [34]

    Yutong Wu, Di Huang, Wenxuan Shi, Wei Wang, Lingzhe Gao, Shihao Liu, Ziyuan Nan, Kaizhao Yuan, Rui Zhang, Xishan Zhang, Zidong Du, Qi Guo, Yewen Pu, Dawei Yin, Xing Hu, and Yunji Chen. 2024. https://doi.org/10.48550/ARXIV.2407.05700 Inversecoder: Unleashing the power of instru...

    2024 doi arXiv

  27. [35]

    Mengzhou Xia, Sadhika Malladi, Suchin Gururangan, Sanjeev Arora, and Danqi Chen. 2024. https://openreview.net/forum?id=PG5fV50maR LESS: selecting influential data for targeted instruction tuning . In Forty-first International Conference on Machine Learning, ICML 2024, Vienna, ...

    2024

  28. [36]

    Can Xu, Qingfeng Sun, Kai Zheng, Xiubo Geng, Pu Zhao, Jiazhan Feng, Chongyang Tao, Qingwei Lin, and Daxin Jiang. 2024. https://openreview.net/forum?id=CfXh93NDgH Wizardlm: Empowering large pre-trained language models to follow complex instructions . In The Twelfth Internationa...

    2024

  29. [37]

    An Yang, Beichen Zhang, Binyuan Hui, Bofei Gao, Bowen Yu, Chengpeng Li, Dayiheng Liu, Jianhong Tu, Jingren Zhou, Junyang Lin, et al. 2024. Qwen2. 5-math technical report: Toward mathematical expert model via self-improvement. arXiv preprint arXiv:2409.12122

    2024 arXiv

  30. [38]

    Zhaojian Yu, Xin Zhang, Ning Shang, Yangyu Huang, Can Xu, Yishujie Zhao, Wenxiang Hu, and Qiufeng Yin. 2023. https://doi.org/10.48550/ARXIV.2312.14187 Wavecoder: Widespread and versatile enhanced instruction tuning with refined data generation . CoRR, abs/2312.14187

    2023 doi arXiv

  31. [39]

    Xilin Zhang, Zhixin Mao, Ziwen Chen, and Shen Gao. 2024. https://doi.org/10.3724/2096-7004.di.2024.0013 Effective tool augmented multi-agent framework for data analysis . Data Intelligence, 6(4):923--945

    2024

  32. [40]

    Qinkai Zheng, Xiao Xia, Xu Zou, Yuxiao Dong, Shan Wang, Yufei Xue, Lei Shen, Zihan Wang, Andi Wang, Yang Li, Teng Su, Zhilin Yang, and Jie Tang. 2023. https://doi.org/10.1145/3580305.3599790 Codegeex: A pre-trained model for code generation with multilingual benchmarking on hu...

    2023

  33. [41]

    Tianyu Zheng, Ge Zhang, Tianhao Shen, Xueling Liu, Bill Yuchen Lin, Jie Fu, Wenhu Chen, and Xiang Yue. 2024. https://doi.org/10.18653/V1/2024.FINDINGS-ACL.762 Opencodeinterpreter: Integrating code generation with execution and refinement . In Findings of the Association for Co...

    2024 doi

  34. [42]

    Changzhi Zhou, Xinyu Zhang, Dandan Song, Xiancai Chen, Wanli Gu, Huipeng Ma, Yuhang Tian, Mengdi Zhang, and Linmei Hu. 2025. Refinecoder: Iterative improving of large language models via adaptive critique refinement for code generation. arXiv preprint arXiv:2502.09183

    2025 arXiv

  35. [43]

    Terry Yue Zhuo, Minh Chien Vu, Jenny Chim, Han Hu, Wenhao Yu, Ratnadira Widyasari, Imam Nur Bani Yusuf, Haolan Zhan, Junda He, Indraneil Paul, Simon Brunner, Chen Gong, and et al. 2024. https://doi.org/10.48550/ARXIV.2406.15877 Bigcodebench: Benchmarking code generation with d...

    2024 doi arXiv

  36. [44]

    online" 'onlinestring :=

    ENTRY address archivePrefix author booktitle chapter edition editor eid eprint eprinttype howpublished institution journal key month note number organization pages publisher school series title type volume year doi pubmed url lastchecked label extra.label sort.label short.list...

  37. [45]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 gl...

Pith tools

Reviewed August 15, 2026 · model on record in the stance chip above.