Pith. sign in

REVIEW 2 cited by

AixBench: A Code Generation Benchmark Dataset

Not yet reviewed by Pith; the record is open.

Re-run · record.json Download PDF Read on arXiv ↗

This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.

SPECIMEN: schema-true, not a live event

T0 review · schema-true

One-sentence machine reading of the paper's core claim.

pith:XXXXXXXX · record.json · timestamp

arxiv 2206.13179 v2 pith:RFXSIEZ2 submitted 2022-06-27 cs.SE

AixBench: A Code Generation Benchmark Dataset

Yiyang Hao (1) , Ge Li (2) , Yongqiang Liu (1) , Xiaowei Miao (1) , He Zong (1) , Siyuan Jiang (1) , Yang Liu (1) , He Wei (1) ((1) aiXcoder
show 1 more author
(2) Peking University)
This is my paper
classification cs.SE
keywords codedatasetbenchmarkevaluatingevaluationgeneratedgenerationsamples
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved
0 comments
read the original abstract

We present a benchmark dataset for evaluating method-level code generation task. The benchmark contains a dataset of 175 samples for automated evaluation and a dataset of 161 samples for manual evaluation. We also present a new metric for automatically evaluating the correctness of the generated code, and a set of criteria to manually evaluating the overall quality of the generated code.

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Forward citations

Cited by 2 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score.

  1. Improving LLM Code Generation via Requirement-Aware Curriculum Reinforcement Learning

    cs.SE 2026-05 unverdicted novelty 6.0

    REC RL improves LLM code generation by automatically assessing and optimizing requirement difficulty with adaptive curriculum sampling, yielding 1.23-5.62% Pass@1 gains over baselines.

  2. A Study of LLMs' Preferences for Libraries and Programming Languages

    cs.SE 2025-03 unverdicted novelty 6.0

    Empirical study of eight LLMs finds overuse of popular libraries like NumPy in up to 45% of unnecessary cases and strong default preference for Python even when suboptimal.