Pith. sign in

REVIEW 12 cited by

KodCode: A Diverse, Challenging, and Verifiable Synthetic Dataset for Coding

Not yet reviewed by Pith; the record is open.

Re-run · record.json Download PDF Read on arXiv ↗

This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.

SPECIMEN: schema-true, not a live event

T0 review · schema-true

One-sentence machine reading of the paper's core claim.

pith:XXXXXXXX · record.json · timestamp

arxiv 2503.02951 v2 pith:ZK5KF34H submitted 2025-03-04 cs.LG cs.AIcs.CL

Zhangchen Xu , Yang Liu , Yueqin Yin , Mingyuan Zhou , Radha Poovendran This is my paper · ORCID
classification cs.LGcs.AIcs.CL
keywords codingdiversekodcodedatasetmodelsverifiablechallengingdata
verification ladder T0 review T1 audit T2 compute T3 formal
0 comments
read the original abstract

We introduce KodCode, a synthetic dataset that addresses the persistent challenge of acquiring high-quality, verifiable training data across diverse difficulties and domains for training Large Language Models for coding. Existing code-focused resources typically fail to ensure either the breadth of coverage (e.g., spanning simple coding tasks to advanced algorithmic problems) or verifiable correctness (e.g., unit tests). In contrast, KodCode comprises question-solution-test triplets that are systematically validated via a self-verification procedure. Our pipeline begins by synthesizing a broad range of coding questions, then generates solutions and test cases with additional attempts allocated to challenging problems. Finally, post-training data synthesis is done by rewriting questions into diverse formats and generating responses under a test-based reject sampling procedure from a reasoning model (DeepSeek R1). This pipeline yields a large-scale, robust and diverse coding dataset. KodCode is suitable for supervised fine-tuning and the paired unit tests also provide great potential for RL tuning. Fine-tuning experiments on coding benchmarks (HumanEval(+), MBPP(+), BigCodeBench, and LiveCodeBench) demonstrate that KodCode-tuned models achieve state-of-the-art performance, surpassing models like Qwen2.5-Coder-32B-Instruct and DeepSeek-R1-Distill-Llama-70B.

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 12 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. Cross-Tokenizer On-Policy Distillation via Byte-Prefix Marginalization

    cs.LG 2026-07 conditional novelty 7.0 of 10

    Byte-Prefix Marginalization maps a teacher's next-token distribution onto the student's vocabulary through shared byte prefixes plus an explicit residual, giving a mass-preserving target for on-policy distillation acr...

  2. Enigmata: Scaling Logical Reasoning in Large Language Models with Synthetic Verifiable Puzzles

    cs.CL 2025-05 conditional novelty 7.0 of 10

    Enigmata's synthetic puzzles with verifiable rewards lift a 32B model to 32.8% on ARC-AGI, above o3-mini-high and o1, and give small apparent gains on math and STEM when added to Seed1.5-Thinking.

  3. Non-vacuous Generalization Bounds for Reinforcement Learning with Verifiable Rewards

    cs.LG 2026-07 reject novelty 6.0 of 10

    Non-vacuous PAC-Bayes generalization bounds for billion-parameter RLVR models, obtained by a Gumbel-max reparameterization and aggressive TinyLoRA distillation/quantization, are claimed for four tasks.

  4. Selective Left-Shift: Turning Test-Time Compute and Difficulty-based Curation into Training Data for Low-Resource Code Generation

    cs.LG 2026-07 conditional novelty 6.0 of 10

    Left-shifting iterative compiler/test refinement into verified SFT data, then GRPO on difficulty-curated IO rewards, lifts Qwen3-8B Julia pass@1 past prior SOTA at 1/3 data and 1/6 cost, and bootstraps Ballerina.

  5. HardTests: Synthesizing High-Quality Test Cases for LLM Coding

    cs.CL 2025-05 conditional novelty 6.0 of 10

    HardTestGen generates higher-precision, higher-recall test suites for 47,136 competitive programming problems, improving test precision by 11.3 points and recall by 17.5 points over TACO and CodeContests when judging ...

  6. InfoSynth: Information-Guided Benchmark Synthesis for LLMs

    cs.CL 2026-01 conditional novelty 5.0 of 10

    Using KL divergence and entropy on embeddings, InfoSynth scores benchmark novelty/diversity and guides a genetic pipeline that generates new, code-verified Python problems from seeds.

  7. Dream-Coder 7B: An Open Diffusion Language Model for Code

    cs.CL 2025-09 conditional novelty 5.0 of 10

    An open 7B diffusion language model for code, adapted from Qwen2.5-Coder, reaches 21.4% pass@1 on LiveCodeBench and shows adaptive, non-left-to-right generation patterns.

  8. OpenCodeReasoning-II: A Simple Test Time Scaling Approach via Self-Critique

    cs.CL 2025-07 conditional novelty 5.0 of 10

    A 2.5M-example code reasoning dataset with critique traces enables Qwen2.5-based models to surpass prior open-weight distilled models on LiveCodeBench via test-time self-critique selection.

  9. Reinforcement Learning Optimization for Large-Scale Learning: An Efficient and User-Friendly Scaling Library

    cs.LG 2025-06 conditional novelty 5.0 of 10

    ROLL is a distributed RL training library for LLMs with sample-level rollout scheduling and flexible device mapping, demonstrated on RLVR and agentic tasks but without baseline or scaling measurements.

  10. Self-Correcting Code Generation Using Small Language Models

    cs.CL 2025-05 conditional novelty 5.0 of 10

    CoCoS, an online RL method with accumulated and fine-grained unit-test rewards, enables 1B-parameter language models to self-correct generated code and outperforms existing self-correction baselines on MBPP, HumanEval...

  11. CODE-DITING: A Reasoning-Based Metric for Functional Alignment in Code Evaluation

    cs.SE 2025-05 conditional novelty 5.0 of 10

    CODE-DITING distills reasoning from a large reasoning LLM into 1.5B and 7B judges that evaluate generated code without reference solutions or test cases, beating larger judges under majority voting.

  12. Large Language Models for Fault Localization: An Empirical Study

    cs.SE 2025-10 conditional novelty 4.0 of 10

    An empirical study of four LLMs for statement-level Java fault localization finds Gemini-2.5-Flash most accurate, bug-report context strongly helpful, and few-shot/CoT gains inconsistent.

Pith tools