Pith. sign in

REVIEW 4 major objections 6 minor 1 cited by

Multi-Agent Collaboration for Multilingual Code Instruction Tuning

T0 review · 4 major / 6 minor · reviewed 2026-08-08 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read A multi-agent collaboration framework that lets language-specialized agents discuss and synthesize new instructions produces a multilingual dataset (X-INSTRUCT) that, when used to fine-tune a code LLM, enables knowledge transfer between…

desk verdict The multi-agent data-synthesis idea is legitimate, but the paper's own tables contradict its headline claim and the ablation that carries the main argument is mismatched with the main results. read the letter →

arxiv 2502.07487 v1 pith:EPLBC2SL submitted 2025-02-11 cs.CL

Jian Yang , Wei Zhang , Jiaxi Yang , Yibo Miao , Shanghaoran Quan , Zhenhe Wu , Qiyao Peng , Liqun Yang
show 4 more authors
Tianyu Liu Zeyu Cui Binyuan Hui Junyang Lin
This is my paper · ORCID
classification cs.CL
keywords multilingualcodegenerationinstructiontuningmulti-agentcollaborationcross-lingualknowledgetransfersyntheticdataLLMfine-tuningDPOpreferencelearningMultiPL-Ebenchmark
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper tries to establish that a multi-agent collaboration framework can generate a multilingual instruction dataset that, when used to fine-tune a code LLM, enables knowledge transfer between programming languages and improves performance on multilingual code generation benchmarks. The authors build language-specialized agents from seed instructions, let them discuss and synthesize new instructions and solutions, and use the resulting dataset X-INSTRUCT to fine-tune Qwen2.5-Coder, yielding Qwen2.5-xCoder. The claimed payoff is a model that outperforms previous open baselines across Python, Java, C++, C#, TypeScript, JavaScript, PHP, and Bash, and that reduces the cross-lingual gap. A sympathetic reader would care because it suggests a data-centric route to improving code models for many languages without writing new code for each language.

What carries the argument

The central machinery is the multilingual multi-agent framework, a set of language-specific agents (one per programming language) that each carry a profile, a memory of previously generated instruction-answer pairs, and a reflection summarizing their merits and faults. The agents communicate either through a central agent or in parallel, randomly selecting memory samples to synthesize new instructions and solutions, optionally targeting a new or existing language. Memory is a priority queue with a similarity filter to avoid duplicates, and is updated with each new generation. The paper assigns the improvement specifically to the data produced by this discussion process (Ds3), rather than to the seed data alone.

What would settle it

Re-run the 7B SFT training exactly as described, evaluate on the same benchmarks, and check whether the Python pass@1 score matches Table 1's 86.0 rather than Table 2's 90.9; then also evaluate the base Qwen2.5-Coder without any X-INSTRUCT training. If the Table 2 row ① cannot reproduce Table 1's number, or if removing Ds3 changes little when the control is properly set, the central attribution of gains to the multi-agent generated data is not supported.

Watch

Extended reading notes

Core claim

The central claim is that cross-lingual code knowledge can be transferred by having multiple language-specific agents collaborate during instruction-data synthesis. Each agent is initialized with instruction data derived from code snippets in its own language, maintains a memory of its generated samples and reflections on their merits and faults, and exchanges messages with other agents to produce new instruction-answer pairs, sometimes in another language, sometimes in a new one. When the resulting agent-generated subset (Ds3) is combined with the seed data and used for supervised fine-tuning, it is the component responsible for most of the performance gain (Table 2: removing Ds3 drops the average from 79.8 to 72.8). The fine-tuned model Qwen2.5-xCoder, in SFT and DPO variants, consistently outperforms the compared open-source baselines on HumanEval, MBPP, and MultiPL-E, and the paper interprets this as evidence of effective knowledge transfer that alleviates negative language interference.

Load-bearing premise

The claim that the multi-agent data Ds3 is what drives the improvement assumes that the configuration labeled '①' in Table 2 is the same 7B model reported in Table 1 as Qwen2.5-xCoder (SFT), and that the base Qwen2.5-Coder without X-INSTRUCT would serve as the control; if those numbers do not refer to the same training run, the ablation cannot attribute the observed change to Ds3.

Editorial extensions

If this is right

  • If the claim holds, fine-tuning a code LLM on X-INSTRUCT yields a model that outperforms prior open-source instruction-tuned code models on HumanEval, MBPP, and MultiPL-E across eight languages.
  • The agent-generated subset Ds3 is the principal driver of the gain, so the mechanism is not just larger synthetic data but the collaborative cross-lingual synthesis itself.
  • The framework reduces the cross-lingual gap, implying that languages with comparatively little training data benefit from knowledge transferred from better-resourced languages.
  • Adding DPO data generated by this framework further improves the SFT model, extending the benefit to preference alignment.
  • Because the framework works with multiple base models (Code Llama and DeepSeek-Coder are mentioned for multilingual understanding), the data-generation method is model-agnostic.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Editorial inference: If the mechanism is real, the same discussion-and-memory loop could be applied to other multilingual structured tasks (e.g., code repair, code translation, or even multilingual summarization), wherever cross-lingual transfer of structured knowledge is desired.
  • Editorial inference: An ablation that removes the memory/reflection component while keeping discussion would isolate whether the gain comes from explicit cross-lingual instruction synthesis or from the diversity of generated samples; the paper does not separate these.
  • Editorial inference: A testable extension is to measure per-language gains against the amount of seed data per language; the paper does not report this, so it is unknown whether low-resource languages are the main beneficiaries or whether well-represented languages improve too.
  • Editorial inference: One could also evaluate the generated X-INSTRUCT directly, e.g., by asking human raters to score cross-lingual consistency, to verify that the 'knowledge transfer' is semantic rather than a form of translation memorization.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 6 minor

Summary. The paper proposes a multi-agent collaboration framework for multilingual code instruction tuning. Language-specific agents generate instruction data from code snippets, discuss and collaborate to produce new cross-lingual instruction-solution pairs, and maintain a memory of past generations to guide future synthesis. The resulting dataset X-INSTRUCT is used to fine-tune Qwen2.5-Coder, yielding a model family called Qwen2.5-xCoder, which is evaluated on HumanEval, MBPP, and MultiPL-E. The central claim is that X-INSTRUCT enables knowledge transfer across programming languages, that Qwen2.5-xCoder consistently outperforms previous baselines, and that the multi-agent generated subset Ds3 is the main driver of the improvements.

Significance. The idea of language-specific agents with memory and reflection for cross-lingual instruction data generation is a plausible and potentially useful contribution to multilingual code LLM training. If the empirical claims were supported, the framework could provide a scalable way to transfer knowledge among programming languages and reduce cross-lingual performance gaps. However, the current manuscript does not establish these claims: the main results table shows that Qwen2.5-xCoder underperforms the standard Qwen2.5-Coder-Instruct models at most model sizes, and the ablation study is internally inconsistent, with a row labeled as 7B actually matching the 32B DPO numbers. The paper also suffers from duplicate sections and unclear presentation. The central contribution is therefore not supported by the evidence as presented.

major comments (4)
  1. [Section 4.1, Table 1] The abstract and Section 4.1 claim that Qwen2.5-xCoder 'consistently outperforms' previous baselines, but Table 1 contradicts this. For example, Qwen2.5-xCoder (SFT) 7B has an average Pass@1 of 67.6 versus 76.5 for Qwen2.5-Coder-7B-Instruct; at 32B the DPO model averages 74.5 versus 79.4 for Qwen2.5-Coder-32B-Instruct. The proposed model underperforms the standard Qwen2.5-Coder-Instruct at 3B, 7B, 14B, and 32B, so the central claim of consistent superiority is not supported by the paper's own main table.
  2. [Section 5, Table 2] The ablation study's row ① is described as 'Qwen2.5-xCoder based on the Qwen2.5-Coder-7B,' but the reported values (Python 90.9, Java 76.6, C++ 72.7, C# 79.1) exactly match the Qwen2.5-xCoder (DPO) 32B row in Table 1, not any 7B configuration. Row ② matches the 32B SFT row. Therefore Table 2 does not evaluate the model it claims to analyze, and the attribution of improvements to Ds3 is unsupported.
  3. [Section 3.3, Section 5, Figure 5] The ablation is confounded with training data size. Removing Ds3 removes roughly 67K of the 97.3K SFT samples, and Figure 5 shows that performance improves monotonically with the amount of tuning data. The observed drop when Ds3 is removed could therefore be due simply to reduced data volume. No size-matched control is reported, and the base Qwen2.5-Coder model without X-INSTRUCT is never evaluated, so the causal role of the multi-agent generated data cannot be isolated.
  4. [Table 1] Several entries in Table 1 are implausible and suggest evaluation or labeling errors. For instance, Qwen2.5-xCoder (DPO) 3B reports Java at 11.4 and Qwen2.5-xCoder (DPO) 7B reports Java at 23.4, while the same models score in the 70-80 range for C++, C#, TypeScript, and JavaScript. Such extreme anomalies in a single language undermine confidence in the reliability of the entire table.
minor comments (6)
  1. [Section 3.6] Section 3.6, 'Baselines,' appears twice verbatim; one copy should be removed.
  2. [Section 1, Contributions] The third contribution states the models are 'based on Code Llama, and Deepseek-Coder,' but the experiments in Section 3.1 use Qwen2.5-Coder as the foundation; this inconsistency should be corrected.
  3. [Section 4.1] The text refers to 'the experimental results in Figure 1' when discussing multilingual results, but Figure 1 is an illustrative example; the reference should be to Table 1.
  4. [Section 2.1, Figure 1] The term 'Universal Code' appears in Figure 1 and the model overview but is never formally defined or evaluated; the paper should clarify what it means and whether it is a claimed emergent representation.
  5. [Section 2.6] Section 2.6 states that the DPO data is synthesized using the multi-agent collaboration framework, but the following sentences describe sampling 128 responses and code-execution verification; the role of the agents in DPO data generation should be made explicit.
  6. [Table 2] Table 2 uses the label '③' for two different rows ('②-Ds3' and '②-Ds2 (Ds1)'), which is confusing and should be fixed.

Circularity Check

0 steps flagged · score 2.0 of 10

No significant circularity: the central claim is tested on external benchmarks and the base model citation is not load-bearing.

full rationale

The manuscript derives X-INSTRUCT through LLM-based multi-agent generation and then fine-tunes Qwen2.5-Coder, evaluating on HumanEval, MBPP, and MultiPL-E. These benchmarks are external and independent of the generated data; no benchmark score is defined in terms of the fitted data, and no fitted parameter is renamed as a prediction. The self-citations to Qwen2.5-Coder and to the authors' prior benchmarks identify the base model and related work but do not carry the load-bearing claim that Ds3 improves multilingual transfer; that claim is an empirical comparison. The Table 2 ablation inconsistency (row ① matching the 32B DPO row rather than the stated 7B SFT configuration) is a correctness/reproducibility concern, not circularity, because it does not make the outcome equivalent to the input by construction. Accordingly, the paper shows no circular step of the enumerated kinds; a score of 2 reflects only the presence of minor self-citations.

Assumptions & free parameters 4 free parameters · 4 assumptions · 1 invented entities

The central claim rests on several unreported design choices (scorer threshold, memory capacity, similarity threshold), on the reliability of LLM-generated test cases and scorers, and on the assumption that the ablation configuration matches the main evaluation. None of these are validated with independent evidence.

free parameters (4)
  • LLM scorer quality threshold = unreported
    Used to filter low-quality generated instruction samples in Section 2.2; the cutoff score is chosen by the authors and never specified, yet it determines the composition of X-INSTRUCT.
  • Memory capacity T = unreported
    The priority queue holding each agent's generation history (Section 2.3) has a capacity T that is not reported, although it controls how much history the agents use in discussion.
  • Similarity threshold for deduplication = unreported
    New samples are added to memory only if their similarity to existing samples is below a threshold (Section 2.3); this threshold is not reported.
  • Number of seed code snippets = approximately 100K
    The paper selects about 100K code snippets from GitHub (Section 3.1); the exact selection criteria and language distribution are not described.
assumptions (4)
  • standard math SFT loss in Eq. (1) and DPO loss in Eq. (2) are the correct training objectives for instruction tuning.
    The paper relies on standard supervised fine-tuning and direct preference optimization losses without derivation or adaptation.
  • domain assumption LLM-generated test cases used to verify DPO responses are correct.
    Section 2.6 says test cases are fed from the LLM into code snippets to verify correctness, but there is no independent validation of those test cases.
  • domain assumption The LLM scorer's judgments correspond to data quality.
    Both the seed data filtering (Section 2.2) and the memory reflection (Section 2.3) depend on the scorer in Figure 4, yet its reliability is not evaluated.
  • ad hoc to paper Fine-tuning on X-INSTRUCT does not degrade existing code capabilities.
    The paper does not include a controlled comparison against the base model, so any gain or loss from the dataset is not isolated.
invented entities (1)
  • Universal Code
    purpose: A purported language-agnostic pseudocode representation that Qwen2.5-xCoder translates into target languages (Figure 1 caption).
    Universal Code appears only in the Figure 1 caption and is not defined, formalized, or evaluated anywhere in the paper.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Multi-Agent Collaboration for Multilingual Code Instruction Tuning." pith.science (2026). https://pith.science/paper/EPLBC2SL

@misc{pith2026250207487,
  author       = {Pith},
  title        = {Pith review of: Multi-Agent Collaboration for Multilingual Code Instruction Tuning},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/EPLBC2SL}},
  note         = {Machine review of arXiv:2502.07487}
}
read the original abstract

Recent advancement in code understanding and generation demonstrates that code LLMs fine-tuned on a high-quality instruction dataset can gain powerful capabilities to address wide-ranging code-related tasks. However, most previous existing methods mainly view each programming language in isolation and ignore the knowledge transfer among different programming languages. To bridge the gap among different programming languages, we introduce a novel multi-agent collaboration framework to enhance multilingual instruction tuning for code LLMs, where multiple language-specific intelligent agent components with generation memory work together to transfer knowledge from one language to another efficiently and effectively. Specifically, we first generate the language-specific instruction data from the code snippets and then provide the generated data as the seed data for language-specific agents. Multiple language-specific agents discuss and collaborate to formulate a new instruction and its corresponding solution (A new programming language or existing programming language), To further encourage the cross-lingual transfer, each agent stores its generation history as memory and then summarizes its merits and faults. Finally, the high-quality multilingual instruction data is used to encourage knowledge transfer among different programming languages to train Qwen2.5-xCoder. Experimental results on multilingual programming benchmarks demonstrate the superior performance of Qwen2.5-xCoder in sharing common knowledge, highlighting its potential to reduce the cross-lingual gap.

Figures

Figures reproduced from arXiv: 2502.07487 by the authors.

Figure 1
Figure 1. An example of Qwen2.5-xCoder. The Code LLM solves the code generation question by “translat￾ing” the pseudocode description (Universal Code) into executable code of the target programming language. (LLMs) based on the Transformer architecture. The emergence of code LLMs equipped with instruc￾tion tuning has advanced a revolutionary step in many code downstream tasks, where LLMs are first trained on massive codebases… view at source ↗
Figure 2
Figure 2. Overview of multilingual multi-agent data generation framework. we first construct the multilingual [PITH_FULL_IMAGE:figures/full_fig_p003_2.png] view at source ↗
Figure 3
Figure 3. Prompt of the multilingual multi-agent frame [PITH_FULL_IMAGE:figures/full_fig_p004_3.png] view at source ↗
Figures from the paper (2 more)
Figure 5
Figure 5. Figure 5: Evaluation results (average scores of 8 pro [PITH_FULL_IMAGE:figures/full_fig_p008_5.png]
Figure 6
Figure 6. Figure 6: Length distribution of problem and solution [PITH_FULL_IMAGE:figures/full_fig_p008_6.png]

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. P2P: Automated Paper-to-Poster Generation and Fine-Grained Benchmark

    cs.CL 2025-05 conditional novelty 6.0 of 10

    P2P is a multi-agent framework that automatically generates HTML-rendered academic posters from papers, backed by a 30k instruction dataset and a 121-pair evaluation benchmark.

Reference graph

Works this paper leans on

37 extracted references · 1 canonical work pages · cited by 1 Pith paper

  1. [1]

    Nye, Maarten Bosma, Henryk Michalewski, David Dohan, Ellen Jiang, Carrie J

    Jacob Austin, Augustus Odena, Maxwell I. Nye, Maarten Bosma, Henryk Michalewski, David Dohan, Ellen Jiang, Carrie J. Cai, Michael Terry, Quoc V. Le, and Charles Sutton. 2021. http://arxiv.org/abs/2108.07732 Program synthesis with large language models . CoRR, abs/2108.07732

    arXiv 2021

  2. [2]

    Federico Cassano, John Gouwar, Daniel Nguyen, Sydney Nguyen, Luna Phipps-Costin, Donald Pinckney, Ming-Ho Yee, Yangtian Zi, Carolyn Jane Anderson, Molly Q Feldman, et al. 2023. Multipl-e: a scalable and polyglot approach to benchmarking neural code generation. IEEE Transactions on Software Engineering

    work page 2023

  3. [3]

    Linzheng Chai, Shukai Liu, Jian Yang, Yuwei Yin, Ke Jin, Jiaheng Liu, Tao Sun, Ge Zhang, Changyu Ren, Hongcheng Guo, et al. 2024. Mceval: Massively multilingual code evaluation. arXiv preprint arXiv:2406.07436

    arXiv 2024

  4. [4]

    Sahil Chaudhary. 2023. Code alpaca: An instruction-following llama model for code generation. https://github.com/sahil280114/codealpaca

    2023

  5. [5]

    Mark Chen, Jerry Tworek, Heewoo Jun, Qiming Yuan, Henrique Pond \' e de Oliveira Pinto, Jared Kaplan, Harrison Edwards, Yuri Burda, Nicholas Joseph, Greg Brockman, Alex Ray, Raul Puri, Gretchen Krueger, Michael Petrov, Heidy Khlaaf, Girish Sastry, Pamela Mishkin, Brooke Chan, Scott Gray, Nick Ryder, Mikhail Pavlov, Alethea Power, Lukasz Kaiser, Mohammad B...

    arXiv 2021

  6. [6]

    Zhangyin Feng, Daya Guo, Duyu Tang, Nan Duan, Xiaocheng Feng, Ming Gong, Linjun Shou, Bing Qin, Ting Liu, Daxin Jiang, and Ming Zhou. 2020. https://doi.org/10.18653/V1/2020.FINDINGS-EMNLP.139 Codebert: A pre-trained model for programming and natural languages . In Findings of the Association for Computational Linguistics: EMNLP 2020, Online Event, 16-20 N...

    doi:10.18653/v1/2020.findings-emnlp.139 2020

  7. [7]

    Suriya Gunasekar, Yi Zhang, Jyoti Aneja, Caio C \' e sar Teodoro Mendes, Allie Del Giorno, Sivakanth Gopi, Mojan Javaheripi, Piero Kauffmann, Gustavo de Rosa, Olli Saarikivi, Adil Salim, Shital Shah, Harkirat Singh Behl, Xin Wang, S \' e bastien Bubeck, Ronen Eldan, Adam Tauman Kalai, Yin Tat Lee, and Yuanzhi Li. 2023. https://doi.org/10.48550/ARXIV.2306....

    arXiv doi:10.48550/arxiv.2306.11644 2023

  8. [9]

    Daya Guo, Qihao Zhu, Dejian Yang, Zhenda Xie, Kai Dong, Wentao Zhang, Guanting Chen, Xiao Bi, Y Wu, YK Li, et al. 2024 b . Deepseek-coder: When the large language model meets programming--the rise of code intelligence. arXiv preprint arXiv:2401.14196

    arXiv 2024

Show all 37 references
  1. [10]

    Chawla, Olaf Wiest, and Xiangliang Zhang

    Taicheng Guo, Xiuying Chen, Yaqi Wang, Ruidi Chang, Shichao Pei, Nitesh V. Chawla, Olaf Wiest, and Xiangliang Zhang. 2024 c . https://doi.org/10.48550/ARXIV.2402.01680 Large language model based multi-agents: A survey of progress and challenges . CoRR, abs/2402.01680

    2024 doi arXiv

  2. [11]

    Binyuan Hui, Jian Yang, Zeyu Cui, Jiaxi Yang, Dayiheng Liu, Lei Zhang, Tianyu Liu, Jiajun Zhang, Bowen Yu, Kai Dang, et al. 2024. Qwen2. 5-coder technical report. arXiv preprint arXiv:2409.12186

    2024 arXiv

  3. [12]

    Raymond Li, Loubna Ben Allal, Yangtian Zi, Niklas Muennighoff, Denis Kocetkov, Chenghao Mou, Marc Marone, Christopher Akiki, Jia Li, Jenny Chim, Qian Liu, Evgenii Zheltonozhskii, Terry Yue Zhuo, Thomas Wang, Olivier Dehaene, Mishig Davaadorj, Joel Lamy - Poirier, Jo \ a o Mont...

    2023 doi arXiv

  4. [13]

    Yuanzhi Li, S \' e bastien Bubeck, Ronen Eldan, Allie Del Giorno, Suriya Gunasekar, and Yin Tat Lee. 2023 b . https://doi.org/10.48550/ARXIV.2309.05463 Textbooks are all you need II: phi-1.5 technical report . CoRR, abs/2309.05463

    2023 doi arXiv

  5. [14]

    Shukai Liu, Linzheng Chai, Jian Yang, Jiajun Shi, He Zhu, Liran Wang, Ke Jin, Wei Zhang, Hualei Zhu, Shuyue Guo, et al. 2024 a . Mdeval: Massively multilingual code debugging. arXiv preprint arXiv:2411.02310

    2024 arXiv

  6. [15]

    Siyao Liu, He Zhu, Jerry Liu, Shulin Xin, Aoyan Li, Rui Long, Li Chen, Jack Yang, Jinxiang Xia, ZY Peng, et al. 2024 b . Fullstack bench: Evaluating llms as full stack coder. arXiv preprint arXiv:2412.00535

    2024 arXiv

  7. [16]

    Yue Liu, Thanh Le - Cong, Ratnadira Widyasari, Chakkrit Tantithamthavorn, Li Li, Xuan - Bach Dinh Le, and David Lo. 2023. https://doi.org/10.48550/ARXIV.2307.12596 Refining chatgpt-generated code: Characterizing and mitigating code quality issues . CoRR, abs/2307.12596

    2023 doi arXiv

  8. [17]

    Anton Lozhkov, Raymond Li, Loubna Ben Allal, Federico Cassano, Joel Lamy-Poirier, Nouamane Tazi, Ao Tang, Dmytro Pykhtar, Jiawei Liu, Yuxiang Wei, et al. 2024. Starcoder 2 and the stack v2: The next generation. arXiv preprint arXiv:2402.19173

    2024 arXiv

  9. [18]

    Shuai Lu, Daya Guo, Shuo Ren, Junjie Huang, Alexey Svyatkovskiy, Ambrosio Blanco, Colin B. Clement, Dawn Drain, Daxin Jiang, Duyu Tang, Ge Li, Lidong Zhou, Linjun Shou, Long Zhou, Michele Tufano, Ming Gong, Ming Zhou, Nan Duan, Neel Sundaresan, Shao Kun Deng, Shengyu Fu, and S...

    2021

  10. [19]

    Ziyang Luo, Can Xu, Pu Zhao, Qingfeng Sun, Xiubo Geng, Wenxiang Hu, Chongyang Tao, Jing Ma, Qingwei Lin, and Daxin Jiang. 2023. https://doi.org/10.48550/ARXIV.2306.08568 Wizardcoder: Empowering code large language models with evol-instruct . CoRR, abs/2306.08568

    2023 doi arXiv

  11. [20]

    Erik Nijkamp, Bo Pang, Hiroaki Hayashi, Lifu Tu, Huan Wang, Yingbo Zhou, Silvio Savarese, and Caiming Xiong. 2023. https://openreview.net/pdf?id=iaYcJKpY2B\_ Codegen: An open large language model for code with multi-turn program synthesis . In The Eleventh International Confer...

    2023

  12. [21]

    OpenAI. 2023. https://doi.org/10.48550/ARXIV.2303.08774 GPT-4 technical report . CoRR, abs/2303.08774

    2023 doi arXiv

  13. [22]

    Long Ouyang, Jeffrey Wu, Xu Jiang, Diogo Almeida, Carroll L. Wainwright, Pamela Mishkin, Chong Zhang, Sandhini Agarwal, Katarina Slama, Alex Ray, John Schulman, Jacob Hilton, Fraser Kelton, Luke Miller, Maddie Simens, Amanda Askell, Peter Welinder, Paul F. Christiano, Jan Leik...

    2022

  14. [23]

    Alec Radford, Karthik Narasimhan, Tim Salimans, Ilya Sutskever, et al. 2018. Improving language understanding by generative pre-training

    2018

  15. [24]

    Baptiste Rozi \` e re, Jonas Gehring, Fabian Gloeckle, Sten Sootla, Itai Gat, Xiaoqing Ellen Tan, Yossi Adi, Jingyu Liu, Tal Remez, J \' e r \' e my Rapin, Artyom Kozhevnikov, Ivan Evtimov, Joanna Bitton, Manish Bhatt, Cristian Canton - Ferrer, Aaron Grattafiori, Wenhan Xiong,...

    2023 doi arXiv

  16. [25]

    Lei Wang, Chen Ma, Xueyang Feng, Zeyu Zhang, Hao Yang, Jingsen Zhang, Zhiyuan Chen, Jiakai Tang, Xu Chen, Yankai Lin, Wayne Xin Zhao, Zhewei Wei, and Ji - Rong Wen. 2023 a . https://doi.org/10.48550/ARXIV.2308.11432 A survey on large language model based autonomous agents . Co...

    2023 doi arXiv

  17. [26]

    Smith, Daniel Khashabi, and Hannaneh Hajishirzi

    Yizhong Wang, Yeganeh Kordi, Swaroop Mishra, Alisa Liu, Noah A. Smith, Daniel Khashabi, and Hannaneh Hajishirzi. 2023 b . https://doi.org/10.18653/V1/2023.ACL-LONG.754 Self-instruct: Aligning language models with self-generated instructions . In Proceedings of the 61st Annual ...

    2023 doi

  18. [27]

    Joty, and Steven C

    Yue Wang, Weishi Wang, Shafiq R. Joty, and Steven C. H. Hoi. 2021. https://doi.org/10.18653/V1/2021.EMNLP-MAIN.685 Codet5: Identifier-aware unified pre-trained encoder-decoder models for code understanding and generation . In Proceedings of the 2021 Conference on Empirical Met...

    2021 doi

  19. [28]

    Yuxiang Wei, Zhe Wang, Jiawei Liu, Yifeng Ding, and Lingming Zhang. 2023. https://doi.org/10.48550/ARXIV.2312.02120 Magicoder: Source code is all you need . CoRR, abs/2312.02120

    2023 doi arXiv

  20. [29]

    Zhuokui Xie, Yinghao Chen, Chen Zhi, Shuiguang Deng, and Jianwei Yin. 2023. https://doi.org/10.48550/ARXIV.2305.04764 Chatunitest: a chatgpt-based automated unit test generation tool . CoRR, abs/2305.04764

    2023 doi arXiv

  21. [30]

    Weixiang Yan, Yuchen Tian, Yunzhe Li, Qian Chen, and Wen Wang. 2023. https://aclanthology.org/2023.findings-emnlp.337 Codetransocean: A comprehensive multilingual benchmark for code translation . In Findings of the Association for Computational Linguistics: EMNLP 2023, Singapo...

    2023

  22. [31]

    Jian Yang, Jiaxi Yang, Ke Jin, Yibo Miao, Lei Zhang, Liqun Yang, Zeyu Cui, Yichang Zhang, Binyuan Hui, and Junyang Lin. 2024 a . Evaluating and aligning codellms on human preference. arXiv preprint arXiv:2412.05210

    2024 arXiv

  23. [32]

    Jian Yang, Jiajun Zhang, Jiaxi Yang, Ke Jin, Lei Zhang, Qiyao Peng, Ken Deng, Yibo Miao, Tianyu Liu, Zeyu Cui, et al. 2024 b . Execrepobench: Multi-level executable code completion evaluation. arXiv preprint arXiv:2412.11990

    2024 arXiv

  24. [33]

    Zhaojian Yu, Xin Zhang, Ning Shang, Yangyu Huang, Can Xu, Yishujie Zhao, Wenxiang Hu, and Qiufeng Yin. 2023. https://doi.org/10.48550/ARXIV.2312.14187 Wavecoder: Widespread and versatile enhanced instruction tuning with refined data generation . CoRR, abs/2312.14187

    2023 doi arXiv

  25. [34]

    Renrui Zhang, Jiaming Han, Aojun Zhou, Xiangfei Hu, Shilin Yan, Pan Lu, Hongsheng Li, Peng Gao, and Yu Qiao. 2023. https://doi.org/10.48550/ARXIV.2303.16199 Llama-adapter: Efficient fine-tuning of language models with zero-init attention . CoRR, abs/2303.16199

    2023 doi arXiv

  26. [35]

    Qinkai Zheng, Xiao Xia, Xu Zou, Yuxiao Dong, Shan Wang, Yufei Xue, Zihan Wang, Lei Shen, Andi Wang, Yang Li, Teng Su, Zhilin Yang, and Jie Tang. 2023. https://doi.org/10.48550/ARXIV.2303.17568 Codegeex: A pre-trained model for code generation with multilingual evaluations on h...

    2023 doi arXiv

  27. [36]

    Terry Yue Zhuo, Minh Chien Vu, Jenny Chim, Han Hu, Wenhao Yu, Ratnadira Widyasari, Imam Nur Bani Yusuf, Haolan Zhan, Junda He, Indraneil Paul, et al. 2024. Bigcodebench: Benchmarking code generation with diverse function calls and complex instructions. arXiv preprint arXiv:2406.15877

    2024 arXiv

  28. [37]

    URL: " 'urlintro :=

    ENTRY address author booktitle chapter edition editor howpublished institution journal key month note number organization pages publisher school series title type volume year eprint doi pubmed url lastchecked label extra.label sort.label short.list INTEGERS output.state before...

  29. [38]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 gl...

Pith tools

Reviewed August 8, 2026 · model on record in the stance chip above.