REVIEW 4 major objections 4 minor 2 cited by
VERIRL: Boosting the LLM-based Verilog Code Generation via Reinforcement Learning
T0 review · 4 major / 4 minor · reviewed 2026-08-05 · deepseek-v4-flash
Pith's one-line read This paper claims that an iterative reinforcement-learning pipeline with trace-back rescoring and sample-balanced weighting outperforms prior Verilog code-generation methods, achieving state-of-the-art test pass rates and functional correct
desk verdict Potentially useful RL-for-Verilog recipe, but the abstract alone can't support the SOTA claim; the testbench overlap question is the first thing to check. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The key machinery is the iterative RL pipeline that co-evolves the policy and reward models. The Trace-back based Rescore mechanism uses reasoning paths and iterative refinement to generate more reliable reward labels, addressing sparse and noisy signals. The sample-balanced weighting strategy adaptively reweights training examples based on reward-probability distributions, mitigating overfitting and catastrophic forgetting. The Veribench-53K dataset, curated from over 700K Verilog problems with structured prompts, complexity labels, and diverse testbenches, supplies the training ground.
What would settle it
Run VERIRL and the best baselines on a separate, independently written set of Verilog prompts with hidden testbenches. If the reported pass-rate and functional-correctness gains shrink or disappear on this out-of-sample benchmark, the central claim would be falsified.
Extended reading notes
Core claim
VERIRL's central claim is that Verilog code generation can be substantially improved by an iterative RL loop that co-trains a policy model and a reward model, rather than by distilling outputs from a large proprietary model. The paper reports state-of-the-art performance on Verilog generation tasks, with gains in test pass rate, functional correctness, and compilation robustness over prior baselines such as CraftRTL and DeepSeek-style RL approaches. The authors attribute these gains to their trace-back based rescoring of sparse reward signals and to their sample-balanced weighting that stabilizes training.
Load-bearing premise
The Veribench-53K dataset, curated from over 700K Verilog problems, is high-quality: its prompts, complexity labels, and testbenches are correct and representative of real Verilog tasks.
Editorial extensions
If this is right
- Verilog code generation can be improved with RL without relying on distillation from closed-source models, making the approach more reproducible and cost-effective.
- The trace-back rescoring mechanism may generalize to other structured code domains with sparse or delayed feedback, such as VHDL, SystemVerilog, or hardware verification.
- The sample-balanced weighting strategy could reduce catastrophic forgetting in RL fine-tuning of LLMs in other application areas.
- The public release of VERIRL and the Veribench-53K dataset enables direct comparison and further research on RL for hardware description languages.
Reading between the lines
- If the reported gains hold, the dominant cost of LLM-based Verilog generation may shift from model scale to dataset curation and reward design, suggesting that other hardware languages could see similar gains from targeted RL.
- The trace-back rescore could be applied to settings where correctness is only observed at the end of a long synthesis or simulation trajectory, such as logic synthesis or physical design.
- A direct head-to-head evaluation against CraftRTL on an identical, independently written testbench suite would clarify how much of the improvement is due to the RL algorithm versus the dataset itself.
- The dependence on dataset quality implies that audit of Veribench-53K's testbenches and labels is a high-leverage check; any systematic bias there would transfer directly into the reward model and the policy.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The manuscript, as submitted, consists of an abstract for VERIRL, a reinforcement-learning framework for Verilog code generation. The abstract describes four contributions: (i) Veribench-53K, a curated dataset of over 700K Verilog problems with prompts, complexity labels, and testbenches; (ii) a trace-back based rescore mechanism for reward modeling; (iii) a sample-balanced weighting strategy for RL fine-tuning; and (iv) an iterative RL pipeline that co-evolves the policy and reward models. The authors claim state-of-the-art performance on Verilog generation, with improvements in test pass rate, functional correctness, and compilation robustness, and state that the method outperforms CraftRTL and DeepSeek-style approaches while using a smaller high-quality dataset. The full text was not provided to the referee; this assessment is based solely on the abstract.
Significance. If the claimed results hold, the work would be a useful contribution to code generation for hardware description languages, a domain that has indeed received less attention than software code generation. The proposed approach of iterative co-evolution of policy and reward models with a smaller curated dataset is a plausible alternative to large-scale closed-source distillation, and the promise of public release of the code and dataset is commendable. However, the significance cannot be properly evaluated from the abstract alone: no quantitative results, baselines, or dataset statistics are given, and the central claims are currently unverified. The methodological ideas are not new in general—RL for code generation and reward-model resoring have been explored—so the contribution rests on the specific instantiation and the empirical evidence, neither of which is available in the submission.
major comments (4)
- [Abstract] The central claim of state-of-the-art performance with 'substantial gains in test pass rate, functional correctness, and compilation robustness' is not accompanied by a single number, baseline comparison, or description of the evaluation setup. No error bars, dataset splits, or ablations are reported. As it stands, the claim is unfalsifiable from the manuscript, and an independent reader cannot verify the reported superiority over CraftRTL or DeepSeek-style approaches.
- [Abstract — Veribench-53K and evaluation leakage] The abstract states that Veribench-53K contains 'diverse testbenches' that 'support reward model training,' while the experiments report test pass rates. It is not stated whether the testbenches used to train the reward model are disjoint from those used as the evaluation oracle for the final pass-rate metrics. If the same testbenches are reused, the reported gains could be inflated by reward overfitting rather than reflecting genuine generalization. The authors must specify the train/validation/test split at both the prompt and testbench level.
- [Abstract — Trace-back based Rescore and reward model training] The reward model is trained on reasoning paths produced by the policy, and this same reward model is then used to rescore and co-evolve the policy. This creates a potential self-referential feedback loop: the reward model may become overfitted to the policy's own distribution, and the RL update may exploit reward-model errors. The abstract does not describe any safeguards, such as holdout sets of policy-generated traces, reward-model validation against ground-truth testbenches, or a mechanism to prevent reward hacking. Without such safeguards, the reported functional-correctness gains are difficult to interpret.
- [Abstract — Veribench-53K dataset quality] The dataset is self-constructed by curating over 700K Verilog problems, but the abstract provides no information about the filtering heuristics, quality-control measures, or validation of the complexity labels and testbenches. Since the reward model and RL policy are both trained on this dataset, systematic noise in the labels or testbenches would be learned by the model and could corrupt the reported improvements. At minimum, the authors should report dataset statistics, inter-annotator agreement, and a manual verification sample.
minor comments (4)
- [Abstract] The method is named VERIRL, but the public repository is listed as 'github.com/omniAI-Lab/VeriRL'. Please unify the naming to avoid ambiguity.
- [Abstract] The abstract references CraftRTL and DeepSeek-style approaches without citations. Full references should be provided in the manuscript.
- [Abstract] The terms 'Trace-back based Rescore' and 'sample-balanced weighting strategy' are introduced at a high level but not defined. Equations or algorithmic pseudocode are needed for the full paper.
- [Abstract] The phrase 'smaller but high-quality dataset combined with RL optimization' is vague. A quantitative comparison of dataset sizes relative to prior work would make the claim more precise.
Circularity Check
No circular reasoning identified; abstract reports a self-contained RL pipeline with no derivation reducing to its inputs.
full rationale
The abstract describes a pipeline: construct a dataset, add a trace-back rescore mechanism for reward signals, add a sample-balanced weighting strategy, and iteratively co-evolve policy and reward models. No equations or derivations are given, so there is no self-definitional reduction to exhibit. The only potential concern is whether the testbenches used for reward-model training overlap with those used for final test-pass-rate evaluation, but the abstract does not claim a train/eval split nor does it present any equation or fitted parameter that is then renamed as a prediction. Without specific textual evidence of such overlap or of any quantity being defined in terms of the result it is supposed to predict, this remains a data-leakage risk rather than a demonstrated circular step. No load-bearing self-citation appears. Hence the circularity score is 0; the paper's stated claims are not shown to be circular from the available abstract.
Assumptions & free parameters
assumptions (3)
- domain assumption Veribench-53K is high-quality: prompts, complexity labels, and testbenches are correct and representative of real Verilog tasks.
- domain assumption Testbenches, and the test pass rate they define, faithfully measure functional correctness and compilation robustness.
- domain assumption Reinforcement learning with iterative policy and reward model co-evolution converges to an improved policy without catastrophic forgetting.
Cite this review
Pith. "Pith review of VERIRL: Boosting the LLM-based Verilog Code Generation via Reinforcement Learning." pith.science (2026). https://pith.science/paper/GEMED6VI
@misc{pith2026250818462,
author = {Pith},
title = {Pith review of: VERIRL: Boosting the LLM-based Verilog Code Generation via Reinforcement Learning},
year = {2026},
howpublished = {\url{https://pith.science/paper/GEMED6VI}},
note = {Machine review of arXiv:2508.18462}
}
read the original abstract
Recent advancements in code generation have shown remarkable success across software domains, yet hardware description languages (HDLs) such as Verilog remain underexplored due to their concurrency semantics, syntactic rigidity, and simulation complexity. In this work, we address these challenges by introducing a reinforcement learning (RL) framework tailored for Verilog code generation. We first construct Veribench-53K, a high-quality dataset curated from over 700K Verilog problems, enriched with structured prompts, complexity labels, and diverse testbenches. To tackle the problem of sparse and noisy reward signals, we propose a Trace-back based Rescore mechanism that leverages reasoning paths and iterative refinement to enhance feedback reliability and support reward model training. Furthermore, to mitigate catastrophic forgetting and overfitting during RL fine-tuning, we introduce a sample-balanced weighting strategy that adaptively balances learning dynamics based on reward-probability distributions. These innovations are integrated into an iterative RL pipeline that co-evolves the policy and reward models. In contrast to recent work such as CraftRTL, which relies on large-scale closed-source model distillation, and DeepSeek-style approaches that struggle with sparse feedback, our method demonstrates superior performance using a smaller but high-quality dataset combined with RL optimization. Experiments on Verilog generation tasks demonstrate state-of-the-art performance, with substantial gains in test pass rate, functional correctness, and compilation robustness. Our findings highlight the potential of RL-driven approaches for structured code generation in hardware-centric domains. VERIRL is publicly available at https://github.com/omniAI-Lab/VeriRL.
Forward citations
Cited by 2 Pith papers
-
A Progressive Approach to Synthesizable RTL Design Generation Using LLMs
VeriRefine boosts LLM-generated RTL correctness to 94.0% on RTLLM v2.0 and 98.1% on VerilogEval-Human v2 by refining and auditing a per-signal intermediate representation before code generation.
-
TestDecision: Sequential Test Suite Generation via Greedy Optimization and Reinforcement Learning
By proving test suite coverage is monotone submodular and training LLMs with RL to maximize marginal gains, TestDecision improves branch coverage 38-52% and bug detection up to 95% over base models on ULT and LiveCodeBench.
Reference graph
Works this paper leans on
-
[1]
11em plus .33em minus .07em 4000 4000 100 4000 4000 500 `\.=1000 = #1 \@IEEEnotcompsoconly \@IEEEcompsoconly #1 * [1] 0pt [0pt][0pt] #1 * [1] 0pt [0pt][0pt] #1 * \| ** #1 \@IEEEauthorblockNstyle \@IEEEcompsocnotconfonly \@IEEEauthorblockAstyle \@IEEEcompsocnotconfonly \@IEEEcompsocconfonly \@IEEEauthordefaulttextstyle \@IEEEcompsocnotconfonly \@IEEEauthor...
- [2]
- [3]
-
[6]
B. Hui, J. Yang, Z. Cui, J. Yang, D. Liu, L. Zhang, T. Liu, J. Zhang, B. Yu, K. Lu et al., ``Qwen2. 5-coder technical report,'' arXiv preprint arXiv:2409.12186, 2024
arXiv 2024
-
[7]
D. Guo, Q. Zhu, D. Yang, Z. Xie, K. Dong, W. Zhang, G. Chen, X. Bi, Y. Wu, Y. Li et al., ``Deepseek-coder: When the large language model meets programming--the rise of code intelligence,'' arXiv preprint arXiv:2401.14196, 2024
arXiv 2024
-
[8]
M. Chen, J. Tworek, H. Jun, Q. Yuan, and et al., ``Evaluating large language models trained on code,'' 2021. [Online]. Available: https://arxiv.org/abs/2107.03374
arXiv 2021
-
[9]
B. Steenhoek, M. Tufano, N. Sundaresan, and A. Svyatkovskiy, ``Reinforcement learning from automatic feedback for high-quality unit test generation,'' arXiv preprint arXiv:2310.02368, 2023
arXiv 2023
- [10]
Show all 37 references
-
[11]
Y. Tsai, M. Liu, and H. Ren, ``Rtlfixer: Automatically fixing rtl syntax errors with large language model,'' New York, NY, USA, 2024. [Online]. Available: https://doi.org/10.1145/3649329.3657353
2024
-
[12]
F. Cui, C. Yin, K. Zhou, Y. Xiao, G. Sun, Q. Xu, Q. Guo, D. Song, D. Lin, X. Zhang et al., ``Origen: Enhancing rtl code generation with code-to-code augmentation and self-reflection,'' International Conference on Computer-Aided Design, 2024. [Online]. Available: https://api.se...
2024
-
[13]
S. Liu, W. Fang, Y. Lu, Q. Zhang, H. Zhang, and Z. Xie, ``Rtlcoder: Outperforming GPT-3.5 in design RTL generation with our open-source dataset and lightweight solution,'' in IEEE International Workshop on LLM-Aided Design (LAD), 2024. [Online]. Available: https://api.semantic...
2024
-
[14]
M. Gao, J. Zhao, Z. Lin, W. Ding, X. Hou, Y. Feng, C. Li, and M. Guo, ``Autovcoder: A systematic framework for automated verilog code generation using llms,'' in Proceedings of the International Conference on Computer Design (ICCD), 2024. [Online]. Available: https://api.seman...
2024
-
[15]
Gehring, K
J. Gehring, K. Zheng, J. Copet, V. Mella, Q. Carbonneaux, T. Cohen, and G. Synnaeve, ``Rlef: Grounding code llms in execution feedback with reinforcement learning,'' arXiv preprint arXiv:2410.02089, 2024
2024 arXiv
-
[16]
Hu, ``Reinforce++: A simple and efficient approach for aligning large language models,'' arXiv preprint arXiv:2501.03262, 2025
J. Hu, ``Reinforce++: A simple and efficient approach for aligning large language models,'' arXiv preprint arXiv:2501.03262, 2025
2025 arXiv
-
[17]
Nadimi, G
B. Nadimi, G. O. Boutaib, and H. Zheng, ``Pyranet: A multi-layered hierarchical dataset for verilog,'' arXiv preprint arXiv:2412.06947, 2024
2024
-
[18]
M. Liu, N. Pinckney, B. Khailany, and H. Ren, ``Verilogeval: Evaluating large language models for verilog code generation,'' in 2023 IEEE/ACM International Conference on Computer-Aided Design (ICCAD), 2023. [Online]. Available: https://api.semanticscholar.org/CorpusID:261822682
2023
-
[19]
Liu, Y.-D
M. Liu, Y.-D. Tsai, W. Zhou, and H. Ren, ``Craftrtl: High-quality synthetic data generation for verilog code models with correct-by-construction non-textual representations and targeted code repair,'' arXiv preprint arXiv:2409.12993, 2024
2024 arXiv
-
[20]
Y. Yang, F. Teng, P. Liu, M. Qi, C. Lv, J. Li, X. Zhang, and Z. He, ``Haven: Hallucination-mitigated llm for verilog code generation aligned with hdl engineers,'' arXiv preprint arXiv:2501.04908, 2025
2025 arXiv
-
[21]
T. Chu, Y. Zhai, J. Yang, S. Tong, S. Xie, S. Levine, and Y. Ma, `` SFT memorizes, RL generalizes: A comparative study of foundation model post-training,'' in The Second Conference on Parsimony and Learning (Recent Spotlight Track), 2025. [Online]. Available: https://openrevie...
2025
-
[22]
C. Yang, H. J. Kang, J. Shi, and D. Lo, ``Acecode: A reinforcement learning framework for aligning code efficiency and correctness in code language models,'' arXiv preprint arXiv:2412.17264, 2024
2024 arXiv
-
[23]
Zhang, S
Y. Zhang, S. Wu, Y. Yang, J. Shu, J. Xiao, C. Kong, and J. Sang, ``o1-coder: an o1 replication for coding,'' arXiv preprint arXiv:2412.00154, 2024
2024 arXiv
-
[24]
D. Guo, D. Yang, H. Zhang, J. Song, R. Zhang, R. Xu, Q. Zhu, S. Ma, P. Wang, X. Bi et al., ``Deepseek-r1: Incentivizing reasoning capability in llms via reinforcement learning,'' arXiv preprint arXiv:2501.12948, 2025
2025 arXiv
-
[25]
J. Wei, X. Wang, D. Schuurmans, M. Bosma, F. Xia, E. Chi, Q. V. Le, D. Zhou et al., ``Chain-of-thought prompting elicits reasoning in large language models,'' Advances in neural information processing systems, vol. 35, pp. 24\,824--24\,837, 2022. [Online]. Available: https://a...
2022
-
[26]
Z. Shao, P. Wang, Q. Zhu, R. Xu, J. Song, X. Bi, H. Zhang, M. Zhang, Y. Li, Y. Wu et al., ``Deepseekmath: Pushing the limits of mathematical reasoning in open language models,'' arXiv preprint arXiv:2402.03300, 2024
2024 arXiv
-
[27]
T. Liu, Y. Zhao, R. Joshi, M. Khalman, M. Saleh, P. J. Liu, and J. Liu, ``Statistical rejection sampling improves preference optimization,'' in The Twelfth International Conference on Learning Representations, 2024. [Online]. Available: https://openreview.net/forum?id=xbjSwwrQOe
2024
-
[28]
W. R. Gilks and P. Wild, ``Adaptive rejection sampling for gibbs sampling,'' Journal of the Royal Statistical Society: Series C (Applied Statistics), vol. 41, no. 2, pp. 337--348, 1992
1992
- [29]
-
[30]
R. A. Bradley and M. E. Terry, ``Rank analysis of incomplete block designs: I. the method of paired comparisons,'' Biometrika, vol. 39, no. 3/4, pp. 324--345, 1952
1952
-
[31]
Zhang, Y
H. Zhang, Y. Lei, L. Gui, M. Yang, Y. He, H. Wang, and R. Xu, ``Cppo: Continual learning for reinforcement learning with human feedback,'' in The Twelfth International Conference on Learning Representations, 2024
2024
-
[32]
Zheng, R
Y. Zheng, R. Zhang, J. Zhang, Y. Ye, Z. Luo, Z. Feng, and Y. Ma, ``Llamafactory: Unified efficient fine-tuning of 100+ language models,'' arXiv preprint arXiv:2403.13372, 2024
2024 arXiv
-
[33]
J. Hu, X. Wu, Z. Zhu, W. Wang, D. Zhang, Y. Cao et al., ``Openrlhf: An easy-to-use, scalable and high-performance rlhf framework,'' arXiv preprint arXiv:2405.11143, 2024
2024 arXiv
-
[34]
Li et al., ``Starcoder: may the source be with you!'' arXiv preprint, 2023
R. Li et al., ``Starcoder: may the source be with you!'' arXiv preprint, 2023. [Online]. Available: https://arxiv.org/abs/2305.06161
2023 arXiv
-
[35]
Roziere, J
B. Roziere, J. Gehring, F. Gloeckle, S. Sootla, I. Gat, X. E. Tan, Y. Adi, J. Liu, R. Sauvestre, T. Remez et al., ``Code llama: Open foundation models for code,'' arXiv preprint arXiv:2308.12950, 2023
2023 arXiv
-
[36]
J. Bai, S. Bai, Y. Chu, Z. Cui, K. Dang, X. Deng, Y. Fan, W. Ge, Y. Han, F. Huang et al., ``Qwen technical report,'' arXiv preprint arXiv:2309.16609, 2023
2023 arXiv
-
[37]
Liu et al., ``Chipnemo: Domain-adapted llms for chip design,'' arXiv preprint arXiv:2311.00176, 2023
M. Liu et al., ``Chipnemo: Domain-adapted llms for chip design,'' arXiv preprint arXiv:2311.00176, 2023. [Online]. Available: https://arxiv.org/abs/2311.00176
2023 arXiv
-
[38]
Thakur, B
S. Thakur, B. Ahmad, Z. Fan, H. Pearce, B. Tan, R. Karri, B. Dolan-Gavitt, and S. Garg, ``Benchmarking large language models for automated verilog rtl code generation,'' in 2023 Design, Automation & Test in Europe Conference & Exhibition (DATE). 1em plus 0.5em minus 0.4em IEEE...
2023
-
[39]
Z. Pei, H. Zhen, M. Yuan, Y. Huang, and B. Yu, ``Betterv: Controlled verilog generation with discriminative guidance,'' in International Conference on Machine Learning (ICML), Vienna, July 2024, pp. 21--27. [Online]. Available: https://api.semanticscholar.org/CorpusID:267500201
2024
Reviewed August 5, 2026 · model on record in the stance chip above.
Discussion (0). Sign in to comment.