Pith. sign in

REVIEW 17 cited by

KodCode: A Diverse, Challenging, and Verifiable Synthetic Dataset for Coding

Not yet reviewed by Pith; the record is open.

Re-run · record.json Download PDF Read on arXiv ↗

This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.

SPECIMEN: schema-true, not a live event

T0 review · schema-true

One-sentence machine reading of the paper's core claim.

pith:XXXXXXXX · record.json · timestamp

arxiv 2503.02951 v2 pith:ZK5KF34H submitted 2025-03-04 cs.LG cs.AIcs.CL

Zhangchen Xu , Yang Liu , Yueqin Yin , Mingyuan Zhou , Radha Poovendran This is my paper · ORCID
classification cs.LGcs.AIcs.CL
keywords codingdiversekodcodedatasetmodelsverifiablechallengingdata
verification ladder T0 review T1 audit T2 compute T3 formal

Signed reviews

No signed human review yet.

0 comments
read the original abstract

We introduce KodCode, a synthetic dataset that addresses the persistent challenge of acquiring high-quality, verifiable training data across diverse difficulties and domains for training Large Language Models for coding. Existing code-focused resources typically fail to ensure either the breadth of coverage (e.g., spanning simple coding tasks to advanced algorithmic problems) or verifiable correctness (e.g., unit tests). In contrast, KodCode comprises question-solution-test triplets that are systematically validated via a self-verification procedure. Our pipeline begins by synthesizing a broad range of coding questions, then generates solutions and test cases with additional attempts allocated to challenging problems. Finally, post-training data synthesis is done by rewriting questions into diverse formats and generating responses under a test-based reject sampling procedure from a reasoning model (DeepSeek R1). This pipeline yields a large-scale, robust and diverse coding dataset. KodCode is suitable for supervised fine-tuning and the paired unit tests also provide great potential for RL tuning. Fine-tuning experiments on coding benchmarks (HumanEval(+), MBPP(+), BigCodeBench, and LiveCodeBench) demonstrate that KodCode-tuned models achieve state-of-the-art performance, surpassing models like Qwen2.5-Coder-32B-Instruct and DeepSeek-R1-Distill-Llama-70B.

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 17 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. Cross-Tokenizer On-Policy Distillation via Byte-Prefix Marginalization

    cs.LG 2026-07 conditional novelty 7.0 of 10

    Byte-Prefix Marginalization maps a teacher's next-token distribution onto the student's vocabulary through shared byte prefixes plus an explicit residual, giving a mass-preserving target for on-policy distillation acr...

  2. Enigmata: Scaling Logical Reasoning in Large Language Models with Synthetic Verifiable Puzzles

    cs.CL 2025-05 conditional novelty 7.0 of 10

    Enigmata's synthetic puzzles with verifiable rewards lift a 32B model to 32.8% on ARC-AGI, above o3-mini-high and o1, and give small apparent gains on math and STEM when added to Seed1.5-Thinking.

  3. d1: Scaling Reasoning in Diffusion Large Language Models via Reinforcement Learning

    cs.CL 2025-04 conditional novelty 7.0 of 10

    A diffusion language model can be trained to reason via supervised fine-tuning plus a new policy-gradient RL method, diffu-GRPO, improving benchmark accuracy over the base model.

  4. Non-vacuous Generalization Bounds for Reinforcement Learning with Verifiable Rewards

    cs.LG 2026-07 reject novelty 6.0 of 10

    Non-vacuous PAC-Bayes generalization bounds for billion-parameter RLVR models, obtained by a Gumbel-max reparameterization and aggressive TinyLoRA distillation/quantization, are claimed for four tasks.

  5. Selective Left-Shift: Turning Test-Time Compute and Difficulty-based Curation into Training Data for Low-Resource Code Generation

    cs.LG 2026-07 conditional novelty 6.0 of 10

    Left-shifting iterative compiler/test refinement into verified SFT data, then GRPO on difficulty-curated IO rewards, lifts Qwen3-8B Julia pass@1 past prior SOTA at 1/3 data and 1/6 cost, and bootstraps Ballerina.

  6. HardTests: Synthesizing High-Quality Test Cases for LLM Coding

    cs.CL 2025-05 conditional novelty 6.0 of 10

    HardTestGen generates higher-precision, higher-recall test suites for 47,136 competitive programming problems, improving test precision by 11.3 points and recall by 17.5 points over TACO and CodeContests when judging ...

  7. InfoSynth: Information-Guided Benchmark Synthesis for LLMs

    cs.CL 2026-01 conditional novelty 5.0 of 10

    Using KL divergence and entropy on embeddings, InfoSynth scores benchmark novelty/diversity and guides a genetic pipeline that generates new, code-verified Python problems from seeds.

  8. Dream-Coder 7B: An Open Diffusion Language Model for Code

    cs.CL 2025-09 conditional novelty 5.0 of 10

    An open 7B diffusion language model for code, adapted from Qwen2.5-Coder, reaches 21.4% pass@1 on LiveCodeBench and shows adaptive, non-left-to-right generation patterns.

  9. OpenCodeReasoning-II: A Simple Test Time Scaling Approach via Self-Critique

    cs.CL 2025-07 conditional novelty 5.0 of 10

    A 2.5M-example code reasoning dataset with critique traces enables Qwen2.5-based models to surpass prior open-weight distilled models on LiveCodeBench via test-time self-critique selection.

  10. Reinforcement Learning Optimization for Large-Scale Learning: An Efficient and User-Friendly Scaling Library

    cs.LG 2025-06 conditional novelty 5.0 of 10

    ROLL is a distributed RL training library for LLMs with sample-level rollout scheduling and flexible device mapping, demonstrated on RLVR and agentic tasks but without baseline or scaling measurements.

  11. Self-Correcting Code Generation Using Small Language Models

    cs.CL 2025-05 conditional novelty 5.0 of 10

    CoCoS, an online RL method with accumulated and fine-grained unit-test rewards, enables 1B-parameter language models to self-correct generated code and outperforms existing self-correction baselines on MBPP, HumanEval...

  12. CODE-DITING: A Reasoning-Based Metric for Functional Alignment in Code Evaluation

    cs.SE 2025-05 conditional novelty 5.0 of 10

    CODE-DITING distills reasoning from a large reasoning LLM into 1.5B and 7B judges that evaluate generated code without reference solutions or test cases, beating larger judges under majority voting.

  13. DeepDistill: Enhancing LLM Reasoning Capabilities via Large-Scale Difficulty-Graded Data Training

    cs.CL 2025-04 conditional novelty 5.0 of 10

    A two-stage SFT recipe using pass-rate and coefficient-of-variation based data selection from a 40M-response distilled dataset lifts Qwen2.5-72B to 79.2% on AIME2024, nearly matching RL-trained reasoning models.

  14. ReasoningV: Efficient Verilog Code Generation with Adaptive Hybrid Reasoning Model

    cs.AR 2025-04 conditional novelty 5.0 of 10

    A 7B Verilog generation model with a verified 5K dataset, two-stage training, and an adaptive reasoning router reaches 57.8% pass@1 on VerilogEval-human, 10.4 points over the prior best open-source Verilog-specific model.

  15. Large Language Models for Fault Localization: An Empirical Study

    cs.SE 2025-10 conditional novelty 4.0 of 10

    An empirical study of four LLMs for statement-level Java fault localization finds Gemini-2.5-Flash most accurate, bug-report context strongly helpful, and few-shot/CoT gains inconsistent.

  16. Exploring the Potential of Offline RL for Reasoning in LLMs: A Preliminary Study

    cs.CL 2025-05 conditional novelty 3.0 of 10

    Applying LD-DPO to the DeepDistill-32B model improves average benchmark scores by 3.3 points, but the gains are uneven, the tables are inconsistent, and the evidence is preliminary.

  17. 100 Days After DeepSeek-R1: A Survey on Replication Studies and More Directions for Reasoning Language Models

    cs.CL 2025-05 conditional novelty 3.0 of 10

    A survey of DeepSeek-R1 replication studies covering SFT, RLVR, algorithm variants, reward design, and future directions.

Pith tools