Pith. sign in

REVIEW 11 cited by

PanGu-Coder2: Boosting Large Language Models for Code with Ranking Feedback

Not yet reviewed by Pith; the record is open.

Re-run · record.json Download PDF Read on arXiv ↗

This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.

SPECIMEN: schema-true, not a live event

T0 review · schema-true

One-sentence machine reading of the paper's core claim.

pith:XXXXXXXX · record.json · timestamp

arxiv 2307.14936 v1 pith:HIREMSAF submitted 2023-07-27 cs.CL cs.AIcs.LGcs.PLcs.SE

Bo Shen , Jiaxin Zhang , Taihong Chen , Daoguang Zan , Bing Geng , An Fu , Muhan Zeng , Ailun Yu
show 4 more authors
Jichuan Ji Jingyang Zhao Yuenan Guo Qianxiang Wang
This is my paper · ORCID
classification cs.CLcs.AIcs.LGcs.PLcs.SE
keywords codemodelsgenerationlanguagelargepangu-coder2boostfeedback
verification ladder T0 review T1 audit T2 compute T3 formal

Signed reviews

No signed human review yet.

0 comments
read the original abstract

Large Language Models for Code (Code LLM) are flourishing. New and powerful models are released on a weekly basis, demonstrating remarkable performance on the code generation task. Various approaches have been proposed to boost the code generation performance of pre-trained Code LLMs, such as supervised fine-tuning, instruction tuning, reinforcement learning, etc. In this paper, we propose a novel RRTF (Rank Responses to align Test&Teacher Feedback) framework, which can effectively and efficiently boost pre-trained large language models for code generation. Under this framework, we present PanGu-Coder2, which achieves 62.20% pass@1 on the OpenAI HumanEval benchmark. Furthermore, through an extensive evaluation on CoderEval and LeetCode benchmarks, we show that PanGu-Coder2 consistently outperforms all previous Code LLMs.

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 11 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. Towards Mitigating API Hallucination in Code Generated by LLMs with Hierarchical Dependency Aware

    cs.SE 2025-05 conditional novelty 7.0 of 10

    MARIN reduces API hallucinations in LLM-generated code by supplying project dependency context and constraining decoding to the project's valid APIs, as shown on a new 416-sample Java benchmark and Huawei internal projects.

  2. Large Language Models for Code Generation from Multilingual Prompts: A Curated Benchmark and a Study on Code Quality

    cs.SE 2026-07 conditional novelty 6.0 of 10

    Prompt language affects LLM code generation, but English is not consistently best: Chinese prompts improve Python correctness on CoderEval, while quality and lexicon effects vary by model and programming language.

  3. MemoCoder: Automated Function Synthesis using LLM-Supported Agents

    cs.SE 2025-07 conditional novelty 6.0 of 10

    A multi-agent LLM framework with a memory of past code fixes (MemoCoder) improves Pass@10 and Pass@50 on MBPP, HumanEval, and LiveCodeBench over zero-shot and self-repair baselines by 3 to 14 percentage points.

  4. LessLeak-Bench: A First Investigation of Data Leakage in LLMs Across 83 Software Engineering Benchmarks

    cs.SE 2025-02 conditional novelty 6.0 of 10

    Across 83 SE benchmarks, average leakage into StarCoder's pretraining data is 4.8% (Python), 2.8% (Java), and 0.7% (C/C++), but QuixBugs and BigCloneBench are 100% and 55.7% leaked.

  5. ViUniT: Visual Unit Tests for More Robust Visual Programming

    cs.CV 2024-12 conditional novelty 6.0 of 10

    ViUniT automatically generates synthetic image-answer unit tests to evaluate visual programs, improving selection, re-prompting, refusal, and reinforcement learning across VQA and image-text matching.

  6. Unseen Horizons: Unveiling the Real Capability of LLM Code Generation Beyond the Familiar

    cs.SE 2024-12 conditional novelty 6.0 of 10

    A code-obfuscation benchmark shows LLM code generation pass rates fall sharply when descriptions, code, and dependencies are rewritten to remove training-data familiarity.

  7. The Art of Repair: Optimizing Iterative Program Repair with Instruction-Tuned Models

    cs.SE 2025-05 conditional novelty 5.0 of 10

    Using only 1,000 fine-tuning examples with full fine-tuning improves the number of plausible repairs by up to 78%, and iterative generation strategies help base models most on complex benchmarks like Defects4J.

  8. Distilling Desired Comments for Enhanced Code Review with Large Language Models

    cs.SE 2024-12 conditional novelty 5.0 of 10

    Desiview identifies desired review comments from code review datasets using the perplexity difference of the actual fix with and without each comment, and the distilled data improves LLaMA-based code review models.

  9. The Current Challenges of Software Engineering in the Era of Large Language Models

    cs.SE 2024-12 conditional novelty 4.0 of 10

    The paper reports 26 challenges in LLM-based software engineering, grouped into seven aspects, derived from a structured discussion among 24 academics and practitioners.

  10. CoopetitiveV: Leveraging LLM-powered Coopetitive Multi-Agent Prompting for High-quality Verilog Generation

    cs.LG 2024-12 reject novelty 4.0 of 10

    A coopetitive multi-agent LLM framework, with a researcher, a prosecutor critic, and two revisers, pushes Verilog code generation to near-perfect pass rates on standard benchmarks.

  11. Generalist Virtual Agents: A Survey on Autonomous Agents Across Digital Platforms

    cs.MA 2024-11 conditional novelty 3.0 of 10

    A survey that proposes the Generalist Virtual Agent concept and taxonomies for agent environments, tasks, perceptions, actions, models, and evaluation, concluding that real-world-like environments favor human-like int...

Pith tools