Pith. sign in

REVIEW 6 major objections 4 minor 39 references

AKD : Adversarial Knowledge Distillation For Large Language Models Alignment on Coding tasks

T0 review · 6 major / 4 minor · reviewed 2026-08-16 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read A training loop that turns a large language model into an adversary can align small code models on roughly 1,400 synthetic exercises.

desk verdict A plausible DPO-curriculum idea that the experiments do not actually support; the paper's own Table 5 shows parity with SFT, so the central claim is unproven. read the letter →

arxiv 2505.06267 v1 pith:EVZHQYSG submitted 2025-05-05 cs.SE cs.AIcs.LG

Ilyas Oulkadda , Julien Perez This is my paper · ORCID
classification cs.SEcs.AIcs.LG
keywords adversarialknowledgedistillationdirectpreferenceoptimizationsyntheticcurriculacodegenerationlanguagemodelalignmentcurriculumlearning
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper proposes Adversarial Knowledge Distillation (AKD), a training loop in which a large teacher model writes coding exercises and solutions, a small student model attempts them, and the student's failures determine which exercises appear next. The authors claim this adversarially ordered curriculum, trained with a direct preference optimization loss, yields meaningful accuracy gains on code-generation benchmarks using only about 1,400 synthetic samples, without a large human-annotated dataset. If correct, this matters because data scarcity and compute cost, not just model size, are the main bottlenecks for improving code language models: a smaller model could be aligned cheaply from a larger one, and the curriculum can be regenerated as the student improves. The reported gains are uneven across model pairs and benchmarks, and the speculative-decoding experiments showed no improvement, but the central claim stands on the HumanEval results and the comparison against a non-adversarial DPO baseline.

What carries the argument

The engine of the method is the margin reward $M = R(c) - R(r)$, the gap between the teacher's reward for a chosen solution and the student's reward for a rejected one, which is converted into a sampling probability $P(p) = \exp(-M(p)) / \sum_{p' \in B} \exp(-M(p'))$ over the batch of prompts. A small margin means the student cannot distinguish the better solution, so that exercise is given higher weight in the next adversarial dataset. This margin-ranked softmax is what turns open-ended synthetic data generation into a curriculum that continually targets the student's current weak points.

What would settle it

Run the AKD loop exactly as specified, then execute every teacher-generated 'chosen' solution against the exercise's own test cases or compile it; if a material fraction fail, the preference signal is corrupted. Separately, replace the margin-based softmax sampler with a random or fixed-order sampler over the same synthetic exercises: if the random curriculum reaches the same HumanEval accuracy, the adversarial ordering is not what drives the gains.

Watch

Extended reading notes

Core claim

The central claim is that a teacher-student pair engaged in an adversarial game can align a small code model to the teacher's distribution using only synthetic data. The teacher generates coding exercises and solutions that are labeled 'chosen,' while the student's attempted solutions are labeled 'rejected'; a direct preference optimization loss then pushes the student toward the teacher's answers. The adversarial part comes from the margin reward, the difference between the rewards assigned to chosen and rejected answers, and a softmax over negative margins that makes low-margin exercises more likely to be sampled in the next training round. Three prompting strategies, incremental, opposite, and deceptive, generate new exercises that target the student's weaknesses. The authors report that this pipeline improves student pass rates over standard fine-tuning and over unordered DPO, and that it matches a handcrafted 5,000-sample benchmark with a 1,600-sample synthetic dataset.

Load-bearing premise

The load-bearing premise is that the teacher model's generated solutions, labeled 'chosen' for preference training, are actually correct and aligned with expert human standards, since the paper applies no compiler or test-execution check to verify them.

Editorial extensions

If this is right

  • Small models could be aligned to larger teachers with datasets of roughly 1.4k exercise-solution pairs, sharply reducing the need for human annotations.
  • Adversarial curriculum sampling can beat presenting the same preference data in random order, so the ordering of exercises is itself a training signal.
  • The teacher-student margin can serve as an automatic indicator of where a student is weak, guiding both data generation and data selection without human intervention.
  • On the HumanEval benchmark, AKD matched the performance of fine-tuning on a handcrafted 5,000-sample dataset while using about 1,600 samples, suggesting synthetic data can substitute for curated benchmarks.
  • The same framework did not improve speculative decoding for same-family teacher-student pairs, so the benefits of AKD are real but not automatic across all uses of teacher-student setups.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • If the margin signal is computable from any reward proxy, the same loop could be ported to domains with automatic correctness signals, such as math proofs, formal specifications, or database queries, where a compiler or verifier would replace the teacher's subjective labels.
  • The gains concentrate on HumanEval-style function-completion exercises rather than MBPP-style natural-language problems, so a testable extension is to generate exercises in the natural-language format and check whether the adversarial-vs-DPO gap persists there.
  • Since the paper's own conclusion names compiler feedback as future work, adding execution-based filtering of 'chosen' solutions would either strengthen AKD by cleaning the preference labels or change its behavior when the teacher is wrong.
  • The absence of speculative-decoding gains within same-family teacher-student pairs suggests a natural next test: cross-family pairs that share a tokenizer, which could turn AKD into a practical factory for building assistant models for faster inference.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

6 major / 4 minor

Summary. The paper proposes Adversarial Knowledge Distillation (AKD), a fine-tuning framework for code LLMs in which a teacher model generates exercises and reference solutions, a student model generates attempt solutions, and a DPO-style loss is optimized over adversarially selected curricula. The selection is driven by margin rewards between teacher and student solutions, and three prompting strategies (incremental, opposite, deceptive) generate new exercises. Experiments compare AKD with self-supervised fine-tuning and a DPO baseline on HumanEval and MBPP for several teacher-student pairs, and also report a speculative decoding study. The central claim is that AKD yields meaningful performance improvements with limited data and outperforms standard fine-tuning and DPO.

Significance. If the empirical claims were reliable, AKD would be a useful contribution to automated curriculum generation for code alignment, combining self-play-style adversarial sampling with preference optimization and knowledge distillation in a resource-limited setting. The paper's strengths include concrete prompt templates in the appendix, experiments across multiple model families (1B-8B), and an honest report of a null result for speculative decoding. However, the significance is substantially limited by the absence of controlled, repeated experiments, by internal numerical contradictions, and by the fact that the headline outperformance claim is contradicted by the paper's own Table 5.

major comments (6)
  1. [Section 5 and Table 5] The conclusion that AKD "outperform[s] traditional methods such as standard fine-tuning and Direct Preference Optimization" is directly contradicted by Table 5, where AKD, SFT on the AKD dataset, and SFT on APPS all report 38% accuracy on HumanEval. This is parity, not outperformance. Moreover, Table 2 reports deltas relative to the initial student model rather than against a controlled SFT baseline, so no evidence for superiority over SFT is actually presented in the paper.
  2. [Table 6 and Section 4.3] The only evidence for an advantage over DPO is Table 6, which reports a single 3-point difference (while the text says "4 percentage point improvement") between a DPO baseline and AKD. The table gives no number of seeds, no standard errors, and no description of how the DPO baseline was constructed, so the claimed advantage over DPO cannot be assessed and may easily be noise.
  3. [Tables 2 and 5] The same teacher-student pair, Qwen2.5-Coder-7B / Llama-3.2-1B, is reported with 34.7% on HumanEval in Table 2 and with 38% in Table 5, and the dataset sizes are given as 0.7M tokens and 1.6k samples, respectively. No explanation is provided for these inconsistent numbers, which erodes confidence in the reliability of the reported evaluations.
  4. [Section 3] The loss function defined before the "Adversarial Step" is -log(exp(R(c,x))/(exp(R(c,x))+exp(R(r,x)))), which is a binary classification loss over reward estimates, not the DPO objective of Rafailov et al. that optimizes a policy ratio relative to a reference policy. The method is therefore not DPO as named, and the comparison against a "DPO baseline" in Table 6 is not a controlled DPO ablation.
  5. [Sections 3 and 4] The teacher's solutions are labeled "chosen" without any test-execution or compiler validation, and the exercises produced by the three adversarial strategies are also not verified for correctness. In addition, the abstract's claims about improving "security" and "reliability" are never measured: no security benchmark, adversarial robustness test, or correctness-by-execution metric appears in the experiments.
  6. [Section 3 and Appendix 6] Section 3 states that the appendix provides "the exact prompts used for dataset generation, the parameters for each prompting strategy, and the iterative training schedule," but Appendix 6 contains only prompt templates. The values of Delta D and theta from the Incremental and Opposite approaches are not specified, and the iterative scheduling beyond "Repetitions 5" in Table 3 is not detailed, so the proposed method cannot be reproduced from the paper as written.
minor comments (4)
  1. [Table 2 caption] The caption's statement that "constant improvements are seen on HumanEval" is too strong given the negative HumanEval delta of -2.5 in the Llama3.1-8B / Llama-3.2-1B row, and the broader claim of constant improvement is further weakened by negative MBPP or MBPP+ deltas in three of five rows.
  2. [Section 4.4] The subsection on speculative decoding is titled "Speculative Decoding with Teacher-Student Framework" and promises an evaluation of speed and accuracy, but it reports no quantitative results and only discusses why the experiment did not show improvements.
  3. [Conclusion] The conclusion says the synthetic datasets contain "approximately 1400 samples," while Table 5 reports a dataset size of 1.6k samples; these numbers should be reconciled.
  4. [Throughout] There are numerous typos and grammatical errors, including "exercices", "datasaets", "we have introduce", "This approach can improves", and "As a consequence to the expensive cost". These should be corrected in any revision.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: AKD is evaluated on external benchmarks, and the margin-based sampling is a curriculum rule rather than a self-fulfilling prediction.

full rationale

The central claim—that AKD improves coding performance and reliability—is tested against external benchmarks (HumanEval, HumanEval+, MBPP, MBPP+ via EvalPlus), which are not generated by the method, so the headline comparison is not self-referential. The adversarial component selects prompts using P(p) = exp(-M(p)) / sum exp(-M(p')) with M = R(c) - R(r); although this uses the student's own reward margins, it is an active-learning/curriculum rule rather than a definition that makes the target result true by construction. The paper fits no parameter to the evaluation benchmarks and invokes no uniqueness theorem or load-bearing self-citation. The strongest circularity-adjacent concern is internal inconsistency: Table 5 reports AKD at 38% HumanEval, identical to both SFT baselines, while the Conclusion claims AKD 'outperform[s] traditional methods,' and Table 6's single 3-point DPO gain is unreplicated. That is an evidentiary or correctness problem, not a circular derivation, because the outcome metric is external and the training objective does not encode the benchmark score. No step in the derivation chain reduces by construction to its own input, so the honest finding is no significant circularity.

Assumptions & free parameters 5 free parameters · 4 assumptions · 0 invented entities

The method relies on the teacher's correctness, the validity of margin-based difficulty signals, and an unstated relationship between the simplified loss and standard DPO. The prompt-evolution strategies introduce unnamed parameters (Delta D, theta) that are not quantified. No new theoretical entities are postulated.

free parameters (5)
  • Difficulty increment (Delta D) in Incremental Approach
    Used in Section 3 to define new exercise difficulty, but no value or range is reported in the paper or appendix.
  • Similarity threshold (theta) in Opposite Approach
    Used in Section 3 to constrain how different new exercises are, but never quantified.
  • DPO beta = 0.01
    Chosen hyperparameter in Table 3; the paper does not report a sweep.
  • Number of adversarial repetitions = 5
    Set in Table 3; the text notes more repetitions degrade performance.
  • Number of subtopics and exercises per topic = 10 and 10
    Dataset generation parameters from Table 3.
assumptions (4)
  • domain assumption Teacher-generated solutions are correct and aligned with expert standards.
    Section 3 states the teacher's solutions are "assumed to align with expert human standards" and are labeled chosen. No unit tests or compiler checks are applied.
  • domain assumption Margin rewards computed from the student's own loss are a reliable measure of exercise difficulty.
    Section 3 uses M = R(c) - R(r) to sample the hardest prompts; the paper provides no validation that low margin correlates with learning value.
  • ad hoc to paper The stated loss function is a valid DPO-style objective.
    The loss in Section 3 omits the reference model and beta from the cited DPO paper; the paper does not clarify how the reference model is handled, so the equivalence to DPO is an assumption.
  • domain assumption The three prompt generation strategies produce valid Python exercises of appropriate difficulty.
    The appendix provides templates but no verification pipeline (compiler, tests) for the generated exercises or solutions.

how reviews work

0 comments
Cite this review

Pith. "Pith review of AKD : Adversarial Knowledge Distillation For Large Language Models Alignment on Coding tasks." pith.science (2026). https://pith.science/paper/EVZHQYSG

@misc{pith2026250506267,
  author       = {Pith},
  title        = {Pith review of: AKD : Adversarial Knowledge Distillation For Large Language Models Alignment on Coding tasks},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/EVZHQYSG}},
  note         = {Machine review of arXiv:2505.06267}
}
read the original abstract

The widespread adoption of Large Language Models (LLMs) for code generation, exemplified by GitHub Copilot\footnote{A coding extension powered by a Code-LLM to assist in code completion tasks} surpassing a million users, highlights the transformative potential of these tools in improving developer productivity. However, this rapid growth also underscores critical concerns regarding the quality, safety, and reliability of the code they generate. As Code-LLMs evolve, they face significant challenges, including the diminishing returns of model scaling and the scarcity of new, high-quality training data. To address these issues, this paper introduces Adversarial Knowledge Distillation (AKD), a novel approach that leverages adversarially generated synthetic datasets to distill the capabilities of larger models into smaller, more efficient ones. By systematically stress-testing and refining the reasoning capabilities of Code-LLMs, AKD provides a framework for enhancing model robustness, reliability, and security while improving their parameter-efficiency. We believe this work represents a critical step toward ensuring dependable automated code generation within the constraints of existing data and the cost-efficiency of model execution.

Figures

Figures reproduced from arXiv: 2505.06267 by the authors.

Figure 1
Figure 1. Adversarial Knowledge Distillation framework comprises three components, differentiated by color: models (green), training processes (red), and synthetic data generation elements (blue). In the initial step, topics and profession seeds are used to generate an initial dataset. Profession is acting as a persona to help diversify the dataset. For subsequent iterations, exercises are sampled based on margin rewards calc… view at source ↗
Figure 2
Figure 2. Margin Rewards during training: At the 200-step mark, an adversarial step generates a new dataset, continuing training with the same optimizer states. A small performance dip is ob￾served in some experiments due to increased exercise difficulty. mance, the hyperparameters are detailed in [PITH_FULL_IMAGE:figures/full_fig_p006_2.png] view at source ↗

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

39 extracted references · 13 canonical work pages

  1. [1]

    Llama 3 model card

    AI@Meta. Llama 3 model card. 2024. URL https://github.com/meta-llama/llama3/blob/main/MODEL_CARD.md

    2024

  2. [2]

    Program synthesis with large language models, 2021

    Austin, J., Odena, A., Nye, M., Bosma, M., Michalewski, H., Dohan, D., Jiang, E., Cai, C., Terry, M., Le, Q., and Sutton, C. Program synthesis with large language models, 2021. URL https://arxiv.org/abs/2108.07732

    arXiv 2021

  3. [3]

    A framework for the evaluation of code generation models

    Ben Allal, L., Muennighoff, N., Kumar Umapathi, L., Lipkin, B., and von Werra, L. A framework for the evaluation of code generation models. https://github.com/bigcode-project/bigcode-evaluation-harness, 2022

    2022

  4. [4]

    Cosmopedia, 2024

    Ben Allal, L., Lozhkov, A., Penedo, G., Wolf, T., and von Werra, L. Cosmopedia, 2024. URL https://huggingface.co/datasets/HuggingFaceTB/cosmopedia

    2024

  5. [5]

    Bowman, S. R., Hyun, J., Perez, E., Chen, E., Pettit, C., Heiner, S., Lukošiūtė, K., Askell, A., Jones, A., Chen, A., Goldie, A., Mirhoseini, A., McKinnon, C., Olah, C., Amodei, D., Amodei, D., Drain, D., Li, D., Tran-Johnson, E., Kernion, J., Kerr, J., Mueller, J., Ladish, J., Landau, J., Ndousse, K., Lovitt, L., Elhage, N., Schiefer, N., Joseph, N., Mer...

    2022

  6. [6]

    Mixtext: Linguistically-informed interpolation of hidden space for semi-supervised text classification

    Chen, J., Yang, Z., and Yang, D. Mixtext: Linguistically-informed interpolation of hidden space for semi-supervised text classification. arXiv preprint arXiv:2004.12239, 2020

    arXiv 2004

  7. [7]

    Chen, M., Tworek, J., Jun, H., Yuan, Q., de Oliveira Pinto, H. P., Kaplan, J., Edwards, H., Burda, Y., Joseph, N., Brockman, G., Ray, A., Puri, R., Krueger, G., Petrov, M., Khlaaf, H., Sastry, G., Mishkin, P., Chan, B., Gray, S., Ryder, N., Pavlov, M., Power, A., Kaiser, L., Bavarian, M., Winter, C., Tillet, P., Such, F. P., Cummings, D., Plappert, M., Ch...

    2021

  8. [8]

    DeepSeek-AI, Guo, D., Yang, D., Zhang, H., Song, J., Zhang, R., Xu, R., Zhu, Q., Ma, S., Wang, P., Bi, X., Zhang, X., Yu, X., Wu, Y., Wu, Z. F., Gou, Z., Shao, Z., Li, Z., Gao, Z., Liu, A., Xue, B., Wang, B., Wu, B., Feng, B., Lu, C., Zhao, C., Deng, C., Zhang, C., Ruan, C., Dai, D., Chen, D., Ji, D., Li, E., Lin, F., Dai, F., Luo, F., Hao, G., Chen, G., ...

    arXiv 2025

Show all 39 references
  1. [9]

    and Li, Y

    Eldan, R. and Li, Y. Tinystories: How small can language models be and still speak coherent english?, 2023

    2023

  2. [10]

    Grattafiori, A., Dubey, A., Jauhri, A., Pandey, A., Kadian, A., Al-Dahle, A., Letman, A., Mathur, A., Schelten, A., Vaughan, A., Yang, A., Fan, A., Goyal, A., Hartshorn, A., Yang, A., Mitra, A., Sravankumar, A., Korenev, A., Hinsvark, A., Rao, A., Zhang, A., Rodriguez, A., Gre...

    2024 arXiv

  3. [11]

    Gunasekar, S., Zhang, Y., Aneja, J., Mendes, C. C. T., Giorno, A. D., Gopi, S., Javaheripi, M., Kauffmann, P., de Rosa, G., Saarikivi, O., Salim, A., Shah, S., Behl, H. S., Wang, X., Bubeck, S., Eldan, R., Kalai, A. T., Lee, Y. T., and Li, Y. Textbooks are all you need, 2023 a

    2023

  4. [12]

    Gunasekar, S., Zhang, Y., Aneja, J., Mendes, C. C. T., Giorno, A. D., Gopi, S., Javaheripi, M., Kauffmann, P., de Rosa, G., Saarikivi, O., Salim, A., Shah, S., Behl, H. S., Wang, X., Bubeck, S., Eldan, R., Kalai, A. T., Lee, Y. T., and Li, Y.-F. Textbooks are all you need. ArX...

    2023 arXiv

  5. [13]

    K., Luo, F., Xiong, Y., and Liang, W

    Guo, D., Zhu, Q., Yang, D., Xie, Z., Dong, K., Zhang, W., Chen, G., Bi, X., Wu, Y., Li, Y. K., Luo, F., Xiong, Y., and Liang, W. Deepseek-coder: When the large language model meets programming -- the rise of code intelligence, 2024. URL https://arxiv.org/abs/2401.14196

    2024 arXiv

  6. [14]

    Distilling the knowledge in a neural network, 2015

    Hinton, G., Vinyals, O., and Dean, J. Distilling the knowledge in a neural network, 2015. URL https://arxiv.org/abs/1503.02531

    2015 arXiv

  7. [15]

    Parameter-efficient transfer learning for nlp, 2019

    Houlsby, N., Giurgiu, A., Jastrzebski, S., Morrone, B., de Laroussilhe, Q., Gesmundo, A., Attariyan, M., and Gelly, S. Parameter-efficient transfer learning for nlp, 2019

    2019

  8. [16]

    J., Shen, Y., Wallis, P., Allen-Zhu, Z., Li, Y., Wang, S., Wang, L., and Chen, W

    Hu, E. J., Shen, Y., Wallis, P., Allen-Zhu, Z., Li, Y., Wang, S., Wang, L., and Chen, W. Lora: Low-rank adaptation of large language models, 2021

    2021

  9. [17]

    Qwen2.5-coder technical report, 2024

    Hui, B., Yang, J., Cui, Z., Yang, J., Liu, D., Zhang, L., Liu, T., Zhang, J., Yu, B., Lu, K., Dang, K., Fan, Y., Zhang, Y., Yang, A., Men, R., Huang, F., Zheng, B., Miao, Y., Quan, S., Feng, Y., Ren, X., Ren, X., Zhou, J., and Lin, J. Qwen2.5-coder technical report, 2024. URL ...

    2024 arXiv

  10. [18]

    Q., Sablayrolles, A., Roux, A., Mensch, A., Savary, B., Bamford, C., Chaplot, D

    Jiang, A. Q., Sablayrolles, A., Roux, A., Mensch, A., Savary, B., Bamford, C., Chaplot, D. S., de Las Casas, D., Hanna, E. B., Bressand, F., Lengyel, G., Bour, G., Lample, G., Lavaud, L. R., Saulnier, L., Lachaux, M.-A., Stock, P., Subramanian, S., Yang, S., Antoniak, S., Scao...

    2024 arXiv

  11. [19]

    Q., Sablayrolles, A., Roux, A., Mensch, A., Savary, B., Bamford, C., Chaplot, D

    Jiang, A. Q., Sablayrolles, A., Roux, A., Mensch, A., Savary, B., Bamford, C., Chaplot, D. S., de las Casas, D., Hanna, E. B., Bressand, F., Lengyel, G., Bour, G., Lample, G., Lavaud, L. R., Saulnier, L., Lachaux, M.-A., Stock, P., Subramanian, S., Yang, S., Antoniak, S., Scao...

    2024

  12. [20]

    D., Savarese, S., and Hoi, S

    Le, H., Wang, Y., Gotmare, A. D., Savarese, S., and Hoi, S. C. H. Coderl: Mastering code generation through pretrained models and deep reinforcement learning, 2022. Advances in Neural Information Processing Systems 35

    2022

  13. [21]

    Fast inference from transformers via speculative decoding, 2023

    Leviathan, Y., Kalman, M., and Matias, Y. Fast inference from transformers via speculative decoding, 2023. URL https://arxiv.org/abs/2211.17192

    2023 arXiv

  14. [22]

    B., Zi, Y., Muennighoff, N., Kocetkov, D., Mou, C., Marone, M., Akiki, C., Li, J., Chim, J., Liu, Q., Zheltonozhskii, E., Zhuo, T

    Li, R., Allal, L. B., Zi, Y., Muennighoff, N., Kocetkov, D., Mou, C., Marone, M., Akiki, C., Li, J., Chim, J., Liu, Q., Zheltonozhskii, E., Zhuo, T. Y., Wang, T., Dehaene, O., Davaadorj, M., Lamy-Poirier, J., Monteiro, J., Shliazhko, O., Gontier, N., Meade, N., Zebaze, A., Yee...

    2023

  15. [23]

    S., Wang, Y., and Zhang, L

    Liu, J., Xia, C. S., Wang, Y., and Zhang, L. Is your code generated by chat GPT really correct? rigorous evaluation of large language models for code generation. In Thirty-seventh Conference on Neural Information Processing Systems, 2023. URL https://openreview.net/forum?id=1qvx610Cu7

    2023

  16. [24]

    Evaluating language models for efficient code generation

    Liu, J., Xie, S., Wang, J., Wei, Y., Ding, Y., and Zhang, L. Evaluating language models for efficient code generation. In First Conference on Language Modeling, 2024. URL https://openreview.net/forum?id=IBCBMeAhmC

    2024

  17. [25]

    On llms-driven synthetic data generation, curation, and evaluation: A survey

    Long, L., Wang, R., Xiao, R., Zhao, J., Ding, X., Chen, G., and Wang, H. On llms-driven synthetic data generation, curation, and evaluation: A survey. In Annual Meeting of the Association for Computational Linguistics, 2024. URL https://api.semanticscholar.org/CorpusID:270688337

    2024

  18. [26]

    OpenAI, O., Plappert, M., Sampedro, R., Xu, T., Akkaya, I., Kosaraju, V., Welinder, P., D'Sa, R., Petron, A., d. O. Pinto, H. P., Paino, A., Noh, H., Weng, L., Yuan, Q., Chu, C., and Zaremba, W. Asymmetric self-play for automatic goal discovery in robotic manipulation, 2021

    2021

  19. [27]

    Ouyang, L., Wu, J., Jiang, X., Almeida, D., Wainwright, C. L., Mishkin, P., Zhang, C., Agarwal, S., Slama, K., Ray, A., Schulman, J., Hilton, J., Kelton, F., Miller, L., Simens, M., Askell, A., Welinder, P., Christiano, P., Leike, J., and Lowe, R. Training language models to f...

    2022

  20. [28]

    D., and Finn, C

    Rafailov, R., Sharma, A., Mitchell, E., Ermon, S., Manning, C. D., and Finn, C. Direct preference optimization: Your language model is secretly a reward model, 2023

    2023

  21. [29]

    E., Adi, Y., Liu, J., Sauvestre, R., Remez, T., Rapin, J., Kozhevnikov, A., Evtimov, I., Bitton, J., Bhatt, M., Ferrer, C

    Rozière, B., Gehring, J., Gloeckle, F., Sootla, S., Gat, I., Tan, X. E., Adi, Y., Liu, J., Sauvestre, R., Remez, T., Rapin, J., Kozhevnikov, A., Evtimov, I., Bitton, J., Bhatt, M., Ferrer, C. C., Grattafiori, A., Xiong, W., Défossez, A., Copet, J., Azhar, F., Touvron, H., Mart...

    2024

  22. [30]

    and Plank, B

    Ruder, S. and Plank, B. Semi-supervised learning for natural language processing. arXiv preprint arXiv:1804.09530, 2018

    2018 arXiv

  23. [31]

    Distilbert, a distilled version of bert: smaller, faster, cheaper and lighter, 2020

    Sanh, V., Debut, L., Chaumond, J., and Wolf, T. Distilbert, a distilled version of bert: smaller, faster, cheaper and lighter, 2020. URL https://arxiv.org/abs/1910.01108

    2020 arXiv

  24. [32]

    I., and Abbeel, P

    Schulman, J., Levine, S., Moritz, P., Jordan, M. I., and Abbeel, P. Trust region policy optimization, 2017 a . URL https://arxiv.org/abs/1502.05477

    2017 arXiv

  25. [33]

    Proximal policy optimization algorithms, 2017 b

    Schulman, J., Wolski, F., Dhariwal, P., Radford, A., and Klimov, O. Proximal policy optimization algorithms, 2017 b

    2017

  26. [34]

    Intrinsic motivation and automatic curricula via asymmetric self-play, 2018

    Sukhbaatar, S., Lin, Z., Kostrikov, I., Synnaeve, G., Szlam, A., and Fergus, R. Intrinsic motivation and automatic curricula via asymmetric self-play, 2018. ICLR 2018

    2018

  27. [35]

    Qwen2.5: A party of foundation models, September 2024

    Team, Q. Qwen2.5: A party of foundation models, September 2024. URL https://qwenlm.github.io/blog/qwen2.5/

    2024

  28. [36]

    Deceptprompt: Exploiting llm-driven code generation via adversarial natural language instructions, 2023

    Wu, F., Liu, X., and Xiao, C. Deceptprompt: Exploiting llm-driven code generation via adversarial natural language instructions, 2023

    2023

  29. [37]

    Xie, Q., Dai, Z., Hovy, E., Luong, M.-T., and Le, Q. V. Unsupervised data augmentation for consistency training. arXiv preprint arXiv:1904.12848, 2020

    1904 arXiv

  30. [38]

    A comprehensive survey on transfer learning, 2020

    Zhuang, F., Qi, Z., Duan, K., Xi, D., Zhu, Y., Zhu, H., Xiong, H., and He, Q. A comprehensive survey on transfer learning, 2020. URL https://arxiv.org/abs/1911.02685

    2020 arXiv

  31. [39]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 gl...

Pith tools

Reviewed August 16, 2026 · model on record in the stance chip above.