Pith. sign in

REVIEW 7 cited by

PanGu-Coder2: Boosting Large Language Models for Code with Ranking Feedback

Not yet reviewed by Pith; the record is open.

Re-run · record.json Download PDF Read on arXiv ↗

This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.

SPECIMEN: schema-true, not a live event

T0 review · schema-true

One-sentence machine reading of the paper's core claim.

pith:XXXXXXXX · record.json · timestamp

arxiv 2307.14936 v1 pith:HIREMSAF submitted 2023-07-27 cs.CL cs.AIcs.LGcs.PLcs.SE

Bo Shen , Jiaxin Zhang , Taihong Chen , Daoguang Zan , Bing Geng , An Fu , Muhan Zeng , Ailun Yu
show 4 more authors
Jichuan Ji Jingyang Zhao Yuenan Guo Qianxiang Wang
This is my paper · ORCID
classification cs.CLcs.AIcs.LGcs.PLcs.SE
keywords codemodelsgenerationlanguagelargepangu-coder2boostfeedback
verification ladder T0 review T1 audit T2 compute T3 formal
0 comments
read the original abstract

Large Language Models for Code (Code LLM) are flourishing. New and powerful models are released on a weekly basis, demonstrating remarkable performance on the code generation task. Various approaches have been proposed to boost the code generation performance of pre-trained Code LLMs, such as supervised fine-tuning, instruction tuning, reinforcement learning, etc. In this paper, we propose a novel RRTF (Rank Responses to align Test&Teacher Feedback) framework, which can effectively and efficiently boost pre-trained large language models for code generation. Under this framework, we present PanGu-Coder2, which achieves 62.20% pass@1 on the OpenAI HumanEval benchmark. Furthermore, through an extensive evaluation on CoderEval and LeetCode benchmarks, we show that PanGu-Coder2 consistently outperforms all previous Code LLMs.

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 7 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. Large Language Models for Code Generation from Multilingual Prompts: A Curated Benchmark and a Study on Code Quality

    cs.SE 2026-07 conditional novelty 6.0 of 10

    Prompt language affects LLM code generation, but English is not consistently best: Chinese prompts improve Python correctness on CoderEval, while quality and lexicon effects vary by model and programming language.

  2. LessLeak-Bench: A First Investigation of Data Leakage in LLMs Across 83 Software Engineering Benchmarks

    cs.SE 2025-02 conditional novelty 6.0 of 10

    Across 83 SE benchmarks, average leakage into StarCoder's pretraining data is 4.8% (Python), 2.8% (Java), and 0.7% (C/C++), but QuixBugs and BigCloneBench are 100% and 55.7% leaked.

  3. ViUniT: Visual Unit Tests for More Robust Visual Programming

    cs.CV 2024-12 conditional novelty 6.0 of 10

    ViUniT automatically generates synthetic image-answer unit tests to evaluate visual programs, improving selection, re-prompting, refusal, and reinforcement learning across VQA and image-text matching.

  4. Unseen Horizons: Unveiling the Real Capability of LLM Code Generation Beyond the Familiar

    cs.SE 2024-12 conditional novelty 6.0 of 10

    A code-obfuscation benchmark shows LLM code generation pass rates fall sharply when descriptions, code, and dependencies are rewritten to remove training-data familiarity.

  5. Distilling Desired Comments for Enhanced Code Review with Large Language Models

    cs.SE 2024-12 conditional novelty 5.0 of 10

    Desiview identifies desired review comments from code review datasets using the perplexity difference of the actual fix with and without each comment, and the distilled data improves LLaMA-based code review models.

  6. The Current Challenges of Software Engineering in the Era of Large Language Models

    cs.SE 2024-12 conditional novelty 4.0 of 10

    The paper reports 26 challenges in LLM-based software engineering, grouped into seven aspects, derived from a structured discussion among 24 academics and practitioners.

  7. CoopetitiveV: Leveraging LLM-powered Coopetitive Multi-Agent Prompting for High-quality Verilog Generation

    cs.LG 2024-12 reject novelty 4.0 of 10

    A coopetitive multi-agent LLM framework, with a researcher, a prosecutor critic, and two revisers, pushes Verilog code generation to near-perfect pass rates on standard benchmarks.

Pith tools