Pith Number

pith:FAL4JNNF

pith:2022:FAL4JNNFGADJ252EXPWX3ECZKQ

not attested not anchored not stored refs resolved

CodeT: Code Generation with Generated Tests

Anh Nguyen, Bei Chen, Daoguang Zan, Fengji Zhang, Jian-Guang Lou, Weizhu Chen, Zeqi Lin

CodeT generates test cases with the same model to select correct code samples via dual execution agreement.

arxiv:2207.10397 v2 · 2022-07-21 · cs.CL · cs.AI · cs.PL · cs.SE

Open paper page JSON Open Graph Bundle Merged state Verified badge What is a Pith Number?

Add to your LaTeX paper

\usepackage{pith}
\pithnumber{FAL4JNNFGADJ252EXPWX3ECZKQ}

Prints a linked badge after your title and injects PDF metadata. Compiles on arXiv. Learn more · Embed verified badge

Record completeness

1 Bitcoin timestamp

2 Internet Archive

3 Author claim open · sign in to claim

4 Citations open

5 Replications open

✓ Portable graph bundle live · download bundle · merged state

The bundle contains the canonical record plus signed events. A mirror can host it anywhere and recompute the same current state with the deterministic merge algorithm.

Claims

C1strongest claim

CodeT improves the pass@1 metric on HumanEval to 65.8%, an absolute improvement of 18.8% over the code-davinci-002 model and more than 20% over previous state-of-the-art results.

C2weakest assumption

That agreement between independently generated code samples on independently generated tests reliably indicates functional correctness rather than shared bugs or test weaknesses.

C3one line summary

CodeT improves code generation accuracy by using the same model to create test cases and then selecting solutions via output agreement on those tests, raising HumanEval pass@1 from 47% to 65.8%.

References

17 extracted · 17 resolved · 6 Pith anchors

[1] Program Synthesis with Large Language Models · arXiv:2108.07732

[2] Language models are few-shot learners 1901

[3] Evaluating Large Language Models Trained on Code · arXiv:2107.03374

[4] Training Verifiers to Solve Math Word Problems · arXiv:2110.14168

[5] InCoder: A Generative Model for Code Infilling and Synthesis · arXiv:2204.05999

Cited by

26 papers in Pith

CodePori: Large-Scale System for Autonomous Software Development Using Multi-Agent Technology

RubricRefine: Improving Tool-Use Agent Reliability with Training-Free Pre-Execution Refinement

Task Abstention for Large Language Models in Code Generation

Probing Privacy Leaks in LLM-based Code Generation via Test Generation

MOSAIC: Multi-agent Orchestration for Task-Intelligent Scientific Coding

Receipt and verification

First computed	2026-05-17T23:38:49.699265Z
Builder	pith-number-builder-2026-05-17-v1
Signature	Pith Ed25519 (`pith-v1-2026-05`) · public key
Schema	pith-number/v1.0

Canonical hash

2817c4b5a530069d7744bbed7d905954374e394adc098726acb9656241cd5447

Aliases

arxiv: 2207.10397 · arxiv_version: 2207.10397v2 · doi: 10.48550/arxiv.2207.10397 · pith_short_12: FAL4JNNFGADJ · pith_short_16: FAL4JNNFGADJ252E · pith_short_8: FAL4JNNF

Agent API

Resolver JSON Graph JSON Events JSON Schema Signing key

Verify this Pith Number yourself

curl -sH 'Accept: application/ld+json' https://pith.science/pith/FAL4JNNFGADJ252EXPWX3ECZKQ \
  | jq -c '.canonical_record' \
  | python3 -c "import sys,json,hashlib; b=json.dumps(json.loads(sys.stdin.read()), sort_keys=True, separators=(',',':'), ensure_ascii=False).encode(); print(hashlib.sha256(b).hexdigest())"
# expect: 2817c4b5a530069d7744bbed7d905954374e394adc098726acb9656241cd5447

Canonical record JSON

{
  "metadata": {
    "abstract_canon_sha256": "18964b1cece2710bd0d5489f437f4f7246a382cc6eab0a23b0b624827c6804ca",
    "cross_cats_sorted": [
      "cs.AI",
      "cs.PL",
      "cs.SE"
    ],
    "license": "http://arxiv.org/licenses/nonexclusive-distrib/1.0/",
    "primary_cat": "cs.CL",
    "submitted_at": "2022-07-21T10:18:37Z",
    "title_canon_sha256": "9f454b5fd5666f3fd6b9b1f77c86081aef821f4aef5eae5e56439466fc5afe5f"
  },
  "schema_version": "1.0",
  "source": {
    "id": "2207.10397",
    "kind": "arxiv",
    "version": 2
  }
}