Pith. sign in

REVIEW 4 cited by

PanGu-Coder: Program Synthesis with Function-Level Language Modeling

Not yet reviewed by Pith; the record is open.

Re-run · record.json Download PDF Read on arXiv ↗

This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.

SPECIMEN: schema-true, not a live event

T0 review · schema-true

One-sentence machine reading of the paper's core claim.

pith:XXXXXXXX · record.json · timestamp

arxiv 2207.11280 v1 pith:IA6BSBOZ submitted 2022-07-22 cs.LG cs.AIcs.CLcs.PLcs.SE

PanGu-Coder: Program Synthesis with Function-Level Language Modeling

Fenia Christopoulou , Gerasimos Lampouras , Milan Gritta , Guchun Zhang , Yinpeng Guo , Zhongqi Li , Qi Zhang , Meng Xiao
show 14 more authors
Bo Shen Lin Li Hao Yu Li Yan Pingyi Zhou Xin Wang Yuchi Ma Ignacio Iacobacci Yasheng Wang Guangtai Liang Jiansheng Wei Xin Jiang Qianxiang Wang Qun Liu
This is my paper
classification cs.LG cs.AIcs.CLcs.PLcs.SE
keywords languagepangu-codermodellingprogrammingcausalcodecombinationdata
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved
0 comments
Share X Bluesky LinkedIn Reddit HN
read the original abstract

We present PanGu-Coder, a pretrained decoder-only language model adopting the PanGu-Alpha architecture for text-to-code generation, i.e. the synthesis of programming language solutions given a natural language problem description. We train PanGu-Coder using a two-stage strategy: the first stage employs Causal Language Modelling (CLM) to pre-train on raw programming language data, while the second stage uses a combination of Causal Language Modelling and Masked Language Modelling (MLM) training objectives that focus on the downstream task of text-to-code generation and train on loosely curated pairs of natural language program definitions and code functions. Finally, we discuss PanGu-Coder-FT, which is fine-tuned on a combination of competitive programming problems and code with continuous integration tests. We evaluate PanGu-Coder with a focus on whether it generates functionally correct programs and demonstrate that it achieves equivalent or better performance than similarly sized models, such as CodeX, while attending a smaller context window and training on less data.

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Forward citations

Cited by 4 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score.

  1. Contextualized Code Pretraining for Code Generation

    cs.SE 2026-05 unverdicted novelty 6.0

    Introduces contextualized code pretraining with caller-callee pairs from static analysis to train CallerGen models that outperform baselines on the new CallerEval benchmark.

  2. GRACE: Graph-Guided Repository-Aware Code Completion through Hierarchical Code Fusion

    cs.SE 2025-09 conditional novelty 5.0

    GRACE combines a multi-level code graph, hybrid text-structure retrieval, and graph fusion to improve repository-level code completion over vanilla and graph-based RAG baselines.

  3. StarCoder: may the source be with you!

    cs.CL 2023-05 accept novelty 5.0

    StarCoderBase matches or beats OpenAI's code-cushman-001 on multi-language code benchmarks; the Python-fine-tuned StarCoder reaches 40% pass@1 on HumanEval while retaining other-language performance.

  4. A Survey on Large Language Models for Code Generation

    cs.CL 2024-06 unverdicted novelty 3.0

    A systematic literature review that organizes recent work on LLMs for code generation into a taxonomy covering data curation, model advances, evaluations, ethics, environmental impact, and applications, with benchmark...