Pith. sign in

REVIEW 5 cited by

Large Language Models Meet NL2Code: A Survey

Not yet reviewed by Pith; the record is open.

Re-run · record.json Download PDF Read on arXiv ↗

This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.

SPECIMEN: schema-true, not a live event

T0 review · schema-true

One-sentence machine reading of the paper's core claim.

pith:XXXXXXXX · record.json · timestamp

arxiv 2212.09420 v2 pith:ZAKR6WKB submitted 2022-12-19 cs.SE cs.AIcs.CLcs.PL

Daoguang Zan , Bei Chen , Fengji Zhang , Dianjie Lu , Bingchao Wu , Bei Guan , Yongji Wang , Jian-Guang Lou This is my paper
classification cs.SEcs.AIcs.CLcs.PL
keywords modelsnl2codelanguagelargecodesurveydevelopmentexisting
verification ladder T0 review T1 audit T2 compute T3 formal
0 comments
read the original abstract

The task of generating code from a natural language description, or NL2Code, is considered a pressing and significant challenge in code intelligence. Thanks to the rapid development of pre-training techniques, surging large language models are being proposed for code, sparking the advances in NL2Code. To facilitate further research and applications in this field, in this paper, we present a comprehensive survey of 27 existing large language models for NL2Code, and also review benchmarks and metrics. We provide an intuitive comparison of all existing models on the HumanEval benchmark. Through in-depth observation and analysis, we provide some insights and conclude that the key factors contributing to the success of large language models for NL2Code are "Large Size, Premium Data, Expert Tuning". In addition, we discuss challenges and opportunities regarding the gap between models and humans. We also create a website https://nl2code.github.io to track the latest progress through crowd-sourcing. To the best of our knowledge, this is the first survey of large language models for NL2Code, and we believe it will contribute to the ongoing development of the field.

Discussion (0). Sign in to comment.

Forward citations

Cited by 5 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. Better Call Grep: Evaluating and Improving Grep-Like Lexical Retrieval for Repository-Level Code Completion

    cs.SE 2026-01 conditional novelty 6.0 of 10

    LLM-generated ripgrep queries plus BM25 re-ranking and line-interval de-duplication outperform graph- and RL-based retrievers for repository-level code completion on CrossCodeEval and RepoEval-Updated.

  2. IaC Generation with LLMs: An Error Taxonomy and A Study on Configuration Knowledge Injection

    cs.AI 2025-12 conditional novelty 6.0 of 10

    Structured configuration knowledge injected as a knowledge graph significantly improves technical correctness of LLM-generated Terraform code but not alignment with user intent.

  3. EnvX: Agentize Everything with Agentic AI

    cs.AI 2025-09 conditional novelty 5.0 of 10

    EnvX converts GitHub repositories into chat-controllable agents and reports a 74% execution completion rate and 52% task pass rate on GitTaskBench.

  4. GRACE: Graph-Guided Repository-Aware Code Completion through Hierarchical Code Fusion

    cs.SE 2025-09 conditional novelty 5.0 of 10

    GRACE combines a multi-level code graph, hybrid text-structure retrieval, and graph fusion to improve repository-level code completion over vanilla and graph-based RAG baselines.

  5. Evaluation of Large Language Model-Driven AutoML in Data and Model Management from Human-Centered Perspective

    cs.HC 2025-07 reject novelty 4.0 of 10

    A 15-participant within-subjects study claims LLM-driven AutoML outperforms traditional AutoML on speed, accuracy, and usability, though supporting data are not provided and several numbers conflict.

Pith tools