Pith. sign in

REVIEW 5 major objections 6 minor 2 cited by

DeepRTL2: A Versatile Model for RTL-Related Tasks

T0 review · 5 major / 6 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read The paper claims DeepRTL2, one family of fine-tuned language models, can write RTL code, explain it, search it, check functional equivalence, and predict area and delay from the same embeddings.

desk verdict Useful RTL embedding model and new benchmarks, but the abstract overclaims SOTA and the embedding evals share a pipeline with training data. read the letter →

arxiv 2506.15697 v1 pith:SD2LBFJZ submitted 2025-05-28 cs.AR cs.CLcs.LG

Yi Liu , Hongji Zhang , Yunhao Zhou , Zhengyuan Shi , Changran Xu , Qiang Xu This is my paper
classification cs.ARcs.CLcs.LG
keywords RTLcodegenerationVerilogunderstandingsemanticsearchfunctionalityequivalencecheckingperformancepredictioncontrastivelearninghardnegativeslanguagemodelembeddings
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper tries to establish that a single family of language models, DeepRTL2, can cover the whole RTL design loop: writing Verilog from natural-language specifications, explaining existing Verilog, retrieving relevant modules from a codebase by semantic query, judging whether two designs are functionally equivalent, and predicting area and delay from code alone. Earlier RTL work concentrated on generation and understanding, leaving the embedding side, code search, equivalence screening, and performance prediction, without a dedicated model; DeepRTL2 is put forward as the first comprehensive solution, fine-tuned so one checkpoint produces both text and fixed-length vectors. If the claim holds, hardware designers could replace several separate tools with one model and get a common vector representation of RTL designs for reuse, early verification, and optimization. The paper reports state-of-the-art results on all five evaluated tasks, with the largest gains over general-purpose baselines on the embedding-based tasks.

What carries the argument

The central object is DeepRTL2, a family of decoder-only language models that both generate RTL text and emit fixed-length embeddings through position-weighted mean pooling of the last hidden states. The training recipe has two heads sharing one backbone: a next-token loss for generation and understanding, plus a contrastive embedding loss with hard negatives for representation learning. That embedding space is the workhorse: search and equivalence checking reduce to cosine similarities with a threshold, and area/delay prediction is a separate tree-based regressor fitted on the same vectors. The other load-bearing piece is the data pipeline: annotated modules, automated code rewrites validated by logic equivalence checking, and synthesis-derived area and delay targets give the embedding space its semantic content.

What would settle it

Take Verilog modules written after the model's training cutoff by independent designers, build search and equivalence pairs without using the paper's annotation or rewriting prompts, and rerun the evaluation. If equivalence average precision falls to about 0.5, the chance level, and code-search F1 falls to the level of general-purpose embedding baselines, the central claim of generalizable RTL embeddings is refuted.

Watch

Extended reading notes

Core claim

DeepRTL2's central claim is that one model can serve as both an RTL code generator and an RTL encoder. The paper fine-tunes two open decoder-only instruction models in two stages: first a curriculum over curated Verilog modules with line comments, specifications, and high-level functional descriptions assembled from public Verilog code and three open RTL datasets; then a contrastive stage that adds rewritten-code pairs, some functionally equivalent and some not, so the model learns to put matching code and descriptions close together in vector space while pushing hard negatives apart. At inference, cosine similarity between code and description embeddings performs natural-language search; cosine similarity between two code embeddings performs equivalence checking against a threshold; and a gradient-boosted regressor on the embeddings predicts area and delay from synthesis-derived targets. The reported result is state-of-the-art performance on all five evaluated tasks, with the largest gains on the embedding tasks, where general-purpose embedding baselines score near chance.

Load-bearing premise

The results stand or fall on whether the embedding benchmarks measure genuine generalization: the same automated annotation and code-rewriting process produced both the training data and the evaluation benchmarks, and the paper does not describe a held-out split.

Editorial extensions

If this is right

  • RTL search becomes a standard retrieval step: a designer's natural-language query can be matched against a large Verilog codebase with cosine similarity, not keyword search.
  • Equivalence checking gains a cheap pre-filter: thresholded embedding similarity can flag likely-identical modules before expensive formal verification, which the paper explicitly frames as a supplement rather than a replacement.
  • PPA estimates can be obtained at the RTL stage from the same embeddings, giving early area/delay feedback before synthesis and physical design.
  • A single checkpoint can be used for generation and representation, so downstream EDA tools could share one encoder instead of maintaining separate generation and embedding models.
  • Open-source models at the 6-8 billion parameter scale can match or beat far larger commercial general-purpose models on RTL-specific tasks, including understanding despite teacher-generated annotations.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The paper does not describe a held-out split for the embedding benchmarks, and the same automated annotation and code-rewriting process produced both training data and evaluation pairs; a benchmark built from independently written designs would clarify how much of the margin is generalization.
  • If the embeddings genuinely capture functional similarity, the same vector space could support tasks the paper does not evaluate, such as clustering modules by function, detecting near-duplicate designs in a codebase, or localizing the changed logic between two revisions.
  • The ablation pattern, where hard negatives slightly lower code-search F1 but raise equivalence and PPA scores, suggests the embedding space is tunable per task; a search-specialized variant might recover the lost F1 while keeping the other gains.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

5 major / 6 minor

Summary. The manuscript introduces DeepRTL2, two fine-tuned decoder-only LLMs (Llama-3.1-8B-Instruct and DeepSeek-Coder-6.7B-Instruct) trained with curriculum learning followed by GRIT-style generative-representational instruction tuning. The models are evaluated on five tasks: RTL code generation, RTL code understanding, natural language code search, functionality equivalence checking, and area/delay prediction. The authors construct new datasets and benchmarks for the embedding tasks and compare against general-purpose embedding models and existing RTL models, reporting strong performance. The central claims are that DeepRTL2 is the first model to provide a comprehensive solution to diverse EDA challenges and that it achieves state-of-the-art performance across all evaluated tasks.

Significance. If the SOTA claims were valid, DeepRTL2 would be a useful contribution: embedding-based RTL tasks are underexplored, the use of Yosys and ABC for ground truth in the equivalence and PPA tasks is a strength, and the ablations around curriculum learning and hard negatives are informative. However, the paper's own results contradict the headline claim for code generation, and the embedding benchmarks are not shown to be independent of the training distribution. The technical contribution is therefore not established at the level claimed, although the datasets and training pipeline could become valuable if re-evaluated under controlled held-out conditions.

major comments (5)
  1. [Abstract; §5.1, Table 1] The abstract and conclusion state that DeepRTL2 achieves 'state-of-the-art performance across all evaluated tasks,' but Table 1 shows that o1-preview achieves functional pass@1 = 50.00%, pass@5 = 60.86%, and pass@10 = 62.52%, while the best DeepRTL2 model (DeepSeek) achieves 38.50%, 52.62%, and 55.99% (Llama: 33.70%, 49.57%, 52.90%). The text in §5.1 concedes that o1-preview outperforms all other models. The SOTA claim must therefore be restricted to open-source models, or the abstract and conclusion must be revised.
  2. [§3.2.1, Table 6, Appendix F] The natural language code search benchmark is not held out. Section 3.2.1 states that the benchmark is built by GPT-4o rephrasing of the functional descriptions from the understanding dataset, which was created by the same GPT-4o annotation pipeline used for training data (§3.1.1). Table 6 lists 59,700 code search samples as training data, and no split is described. Appendix F further uses this same rephrased set and its GPT-4o rewrites to construct contrastive training samples. Consequently, the F1 gap over general-purpose embeddings (Table 2) is consistent with the model matching the benchmark-generation distribution rather than with generalization to new queries. The authors need to provide a disjoint, ideally human-authored, test set and demonstrate that training data were excluded.
  3. [§3.2.2, Appendix C, Appendix F] The functionality equivalence benchmark shares both the source designs and the rewrite procedure with training. The benchmark is built by applying the feedback-driven GPT-4o chain-of-thought rewrite to 50 RTLLM v2.0 designs (§3.2.2), and the same rewrite procedure is used in Appendix F to create contrastive training samples from the code search training set. In addition, RTLLM v2.0 designs can plausibly appear in training through RTLCoder, MG-Verilog, and DeepCircuitX augmentation (§3.1.1). No overlap filtering or held-out split is reported. The average precision results in Table 5 therefore do not support the claim of superior generalization, and the benchmark must be reconstructed from designs and rewrites that are provably absent from training.
  4. [§4.2, Table 8] The equivalence-checking evaluation selects thresholds on the test set. Section 4.2 states that for each model the authors 'first identify the optimal accuracy threshold' and then 'determine the best F1 threshold' on the benchmark before reporting accuracy and F1. This is test-set threshold fitting; it inflates all threshold-dependent metrics. The authors should either fix a single threshold a priori or choose thresholds on a validation split, and report average precision independently of any threshold tuning.
  5. [Tables 2, 4, 5, 7, 8] No error bars, standard deviations, or significance tests are reported for any embedding or generation result. Given the small benchmark sizes (50 problems for generation, 400 pairs for equivalence), single-run differences, especially the modest gaps in Tables 5 and 8, may not be statistically meaningful. The authors should report multiple sampling seeds or confidence intervals.
minor comments (6)
  1. [Equation (1); §4.2] In Equation (1), 'trails' should be 'trials', and the notation n = 20 and c should be defined more carefully. Additionally, reporting the best pass@k across temperatures (0.2, 0.5, 0.8) needs justification or per-temperature reporting, because this selection inflates the reported values.
  2. [Table 4] The text-embedding-3-large row reports identical r2_score values (0.699) for area and delay; please verify whether this is a typo.
  3. [Tables 3 and 8] There are formatting errors, such as 'DeepRTL21st(Llama)' without a space, '13.8437.97' without separator, and '0.4781.000' in Table 8. These should be cleaned.
  4. [§3.1.2] The 500-module understanding benchmark is derived from the same code generation dataset; please clarify whether any of these modules or their annotations appear in the training data and how overlap was prevented.
  5. [Introduction; §2.2] The claims 'first model to provide a comprehensive solution' and 'first model explicitly designed for RTL embedding' are stronger than the evidence presented; please temper them or provide a systematic comparison with prior embedding-based RTL models.
  6. [Appendix A] The human evaluation validates annotation accuracy but not the quality of the embedding benchmarks; this should be stated explicitly so readers are not misled about benchmark validation.

Circularity Check

3 steps flagged · score 6.0 of 10

Embedding-task results reduce to training/benchmark construction overlap: the code-search and equivalence benchmarks are generated by the same GPT-4o pipelines as the training data, with no held-out split described; the equivalence accuracy/F1 thresholds are tuned on the test benchmark.

  1. fitted input called prediction [Sec. 3.2.1 (Natural Language Code Search), Sec. 4.2, Appendix D Table 6]
    "For this task, we reuse the dataset and benchmark from the RTL code understanding task. However, since the functional descriptions in the understanding dataset often contain specific identifiers, introducing the risk of data leakage, and are too complex for direct use in practical code search, we employ GPT-4o to rephrase the descriptions into a user query format, as shown in Figure 2. ... After this rephrasing process, we obtain the natural language code search dataset and benchmark in the format {(user_query_i,RTL_code_i)}. ..."

    The code-search evaluation set is produced by the same GPT-4o rephrasing operation and from the same source understanding data as the training pairs. Appendix D lists 59,700 natural language code search samples as training data and states that every non-PPA dataset in Table 6 is used for training, while no held-out split is described. The reported F1 can therefore be obtained by matching GPT-4o's rephrasing style and identifier-removal behavior; it is not an independent test of query-to-code retrieval generalization.

  2. fitted input called prediction [Sec. 3.2.2 (Functionality Equivalence Checking), Appendix F, Sec. 4.2]
    "To construct a dataset for this task, we develop a feedback-driven CoT prompting strategy using GPT-4o, as shown in Figure 3. ... Additionally, we adapt RTLLM v2.0 ... to construct a new benchmark for functionality equivalence checking. Applying the same feedback-driven CoT strategy to its 50 verified Verilog designs, we generate multiple alternative implementations, expanding our benchmark to 400 code pairs. ... To construct a dataset for contrastive learning, we first prompt GPT-4o to rewrite Verilog code snippets from the natural language code search training set."

    The benchmark pairs for equivalence checking are generated with the same feedback-driven GPT-4o rewrite-then-Yosys procedure that Appendix F uses to create contrastive training samples for the embedding head. The model is therefore trained to separate exactly the kind of GPT-4o-rewritten pairs on which it is then evaluated, and no separate held-out rewrite source or held-out split is described. The average precision comparison mainly measures fit to the rewrite generator's distribution rather than general functional-equivalence ability.

1 more flagged steps
  1. fitted input called prediction [Sec. 4.2, functionality equivalence evaluation protocol]
    "For each model, we first identify the optimal accuracy threshold and compute the accuracy score. We then determine the best F1 threshold and report the F1, precision, and recall scores. Finally, we calculate the average precision score based on the similarity scores of the code pairs and their corresponding ground-truth labels."

    The reported accuracy and F1 scores are obtained by selecting thresholds on the same benchmark used for evaluation, so those numbers are test-set-optimized statistics rather than out-of-sample predictions. This inflates the apparent advantage over baselines whose thresholds are also tuned on the same set; it is a fitted quantity presented as an evaluation outcome, even though the primary average-precision metric is threshold-free.

full rationale

The derivation chain for the two novel embedding benchmarks is not self-contained. For natural language code search, Section 3.2.1 creates the dataset and benchmark by rephrasing the same understanding data with GPT-4o, and Appendix D states that the 59,700 code-search samples are used for training, with no held-out split described; therefore the reported F1 is at least partly a measure of how well the model imitates the GPT-4o rephrasing distribution. For functionality equivalence checking, the benchmark pairs are generated with the same feedback-driven GPT-4o rewrite-plus-Yosys procedure used in Appendix F to create contrastive training examples, so the model is evaluated on the same generative distribution it was trained to separate; additionally Section 4.2 selects accuracy and F1 thresholds on the test benchmark, making those reported numbers test-set-optimized. These issues do not infect the whole paper: code generation is evaluated on the external RTLLM v2.0 benchmark, code understanding uses a human-annotated 500-module benchmark, and performance prediction has external Yosys/ABC labels with an 80:20 split. Hence the overall circularity is partial and concentrated in the embedding-based state-of-the-art claims, warranting a 6 rather than a higher score.

Assumptions & free parameters 3 free parameters · 4 assumptions · 0 invented entities

The model itself is a fine-tuned version of existing base models, so no new physical entities are introduced. The free parameters and axioms listed above are the main external inputs the central claims rest on.

free parameters (3)
  • equivalence threshold = per-model optimal threshold
    In Section 4.2, binary labels for equivalence are assigned by comparing cosine similarity to a predefined threshold; the optimal threshold is identified on the test set for each model, which is fitting an evaluation hyperparameter to the test data.
  • generation temperature = 0.2, 0.5, 0.8 (best selected)
    Following RTLCoder, pass@k is evaluated at multiple temperatures and the best result is reported; this is a hyperparameter selection made on the benchmark, not a parameter-free measurement.
  • embedding temperature (tau) = 0.02
    The contrastive loss temperature in Appendix H is chosen by hand as a training hyperparameter; it is not fitted to the target tasks but affects the learned embedding geometry.
assumptions (4)
  • domain assumption GRIT multi-task training enables a single decoder-only LLM to both generate text and produce useful embeddings.
    The method directly adapts GRIT (Muennighoff et al., 2025) and assumes the multi-task objective transfers to RTL code and descriptions.
  • domain assumption GPT-4o-generated annotations are accurate enough for training and benchmark construction.
    Section 3.1.1 and Appendix A report about 90% accuracy in human evaluation for high-level descriptions, which means roughly 10% of the training and benchmark labels contain errors.
  • domain assumption Yosys logic equivalence checking correctly classifies equivalent and inequivalent RTL pairs.
    Used in Section 3.2.2 to validate the equivalence checking dataset and benchmark; assumes the tool is a reliable oracle for functional equivalence.
  • domain assumption The GitHub-sourced Verilog modules are representative of real RTL designs.
    The training and benchmark modules come from public GitHub repositories; the paper assumes this corpus reflects the distribution of industrial RTL code.

how reviews work

0 comments
Cite this review

Pith. "Pith review of DeepRTL2: A Versatile Model for RTL-Related Tasks." pith.science (2026). https://pith.science/paper/SD2LBFJZ

@misc{pith2026250615697,
  author       = {Pith},
  title        = {Pith review of: DeepRTL2: A Versatile Model for RTL-Related Tasks},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/SD2LBFJZ}},
  note         = {Machine review of arXiv:2506.15697}
}
read the original abstract

The integration of large language models (LLMs) into electronic design automation (EDA) has significantly advanced the field, offering transformative benefits, particularly in register transfer level (RTL) code generation and understanding. While previous studies have demonstrated the efficacy of fine-tuning LLMs for these generation-based tasks, embedding-based tasks, which are equally critical to EDA workflows, have been largely overlooked. These tasks, including natural language code search, RTL code functionality equivalence checking, and performance prediction, are essential for accelerating and optimizing the hardware design process. To address this gap, we present DeepRTL2, a family of versatile LLMs that unifies both generation- and embedding-based tasks related to RTL. By simultaneously tackling a broad range of tasks, DeepRTL2 represents the first model to provide a comprehensive solution to the diverse challenges in EDA. Through extensive experiments, we show that DeepRTL2 achieves state-of-the-art performance across all evaluated tasks.

Figures

Figures reproduced from arXiv: 2506.15697 by the authors.

Figure 1
Figure 1. The overview of DeepRTL2. It can handle both generation- and embedding-based tasks at the RTL stage. [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 2
Figure 2. The annotation process for the RTL code generation/understanding dataset. After obtaining the high-level functional description, we prompt GPT-4o to rephrase it into a user query format, which is then used to construct the natural language code search dataset. ules containing syntax errors, ensuring that only syntactically valid Verilog code is retained. For annotation, we adopt the chain-of-thought (CoT) prompting … view at source ↗
Figure 3
Figure 3. The feedback-driven code rewrite process. [PITH_FULL_IMAGE:figures/full_fig_p005_3.png] view at source ↗
Figures from the paper (2 more)
Figure 4
Figure 4. Figure 4: The instruction for rephrasing the code de [PITH_FULL_IMAGE:figures/full_fig_p012_4.png]
Figure 5
Figure 5. Figure 5: The code rewrite instructions used to construct the functionality equivalence checking dataset. [PITH_FULL_IMAGE:figures/full_fig_p013_5.png]

Discussion (0). Sign in to comment.

Forward citations

Cited by 2 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. TrojanGYM: A Detector-in-the-Loop LLM for Adaptive RTL Hardware Trojan Insertion

    cs.CR 2026-01 conditional novelty 6.0 of 10

    TrojanGYM couples LLM-driven RTL Trojan insertion with GNN detector feedback, producing evasive Trojans (up to 83.33% evasion under best-LLM oracle selection).

  2. Beyond Tokens: Enhancing RTL Quality Estimation via Structural Graph Learning

    cs.LG 2025-08 conditional novelty 6.0 of 10

    StructRTL uses self-supervised learning on control data flow graphs, plus knowledge distillation from post-mapping netlists, to beat prior LLM-based methods for predicting circuit area and delay from RTL code on a 13,...

Reference graph

Works this paper leans on

45 extracted references · 20 canonical work pages · cited by 2 Pith papers

  1. [1]

    Satanjeev Banerjee and Alon Lavie. 2005. Meteor: An automatic metric for mt evaluation with improved correlation with human judgments. In Proceedings of the acl workshop on intrinsic and extrinsic evaluation measures for machine translation and/or summarization, pages 65--72

    2005

  2. [2]

    Parishad BehnamGhader, Vaibhav Adlakha, Marius Mosbach, Dzmitry Bahdanau, Nicolas Chapados, and Siva Reddy. 2024. Llm2vec: Large language models are secretly powerful text encoders. arXiv preprint arXiv:2404.05961

    arXiv 2024

  3. [3]

    Jason Blocklove, Siddharth Garg, Ramesh Karri, and Hammond Pearce. 2023. Chip-chat: Challenges and opportunities in conversational hardware design. In 2023 ACM/IEEE 5th Workshop on Machine Learning for CAD (MLCAD), pages 1--6. IEEE

    work page 2023

  4. [4]

    Andres M Bran and Philippe Schwaller. 2024. Transformers and large language models for chemistry and drug discovery. In Drug Development Supported by Informatics, pages 143--163. Springer

    work page 2024

  5. [5]

    Robert Brayton and Alan Mishchenko. 2010. Abc: An academic industrial-strength verification tool. In Computer Aided Verification: 22nd International Conference, CAV 2010, Edinburgh, UK, July 15-19, 2010. Proceedings 22, pages 24--40. Springer

    work page 2010

  6. [6]

    Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared D Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, et al. 2020. Language models are few-shot learners. Advances in neural information processing systems, 33:1877--1901

    2020

  7. [7]

    Kaiyan Chang, Kun Wang, Nan Yang, Ying Wang, Dantong Jin, Wenlong Zhu, Zhirong Chen, Cangyuan Li, Hao Yan, Yunhao Zhou, et al. 2024. Data is all you need: Finetuning llms for chip design via an automated design-data augmentation framework. In Proceedings of the 61st ACM/IEEE Design Automation Conference, pages 1--6

    work page 2024

  8. [8]

    Lei Chen, Yiqi Chen, Zhufei Chu, Wenji Fang, Tsung-Yi Ho, Ru Huang, Yu Huang, Sadaf Khan, Min Li, Xingquan Li, et al. 2024. Large circuit models: opportunities and challenges. Science China Information Sciences, 67(10):1--42

    work page 2024

Show all 45 references
  1. [9]

    Tianqi Chen and Carlos Guestrin. 2016. Xgboost: A scalable tree boosting system. In Proceedings of the 22nd acm sigkdd international conference on knowledge discovery and data mining, pages 785--794

    2016

  2. [10]

    Xiangli Chen, Yuehua Meng, and Gang Chen. 2023. Incremental verilog parser. In 2023 International Symposium of Electronics Design Automation (ISEDA), pages 236--240. IEEE

    2023

  3. [11]

    Fan Cui, Chenyang Yin, Kexing Zhou, Youwei Xiao, Guangyu Sun, Qiang Xu, Qipeng Guo, Demin Song, Dahua Lin, Xingcheng Zhang, et al. 2024. Origen: Enhancing rtl code generation with code-to-code augmentation and self-reflection. arXiv preprint arXiv:2407.16237

    2024 arXiv

  4. [12]

    Jacob Devlin. 2018. Bert: Pre-training of deep bidirectional transformers for language understanding. arXiv preprint arXiv:1810.04805

    2018 arXiv

  5. [13]

    Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Abhishek Kadian, Ahmad Al-Dahle, Aiesha Letman, Akhil Mathur, Alan Schelten, Amy Yang, Angela Fan, et al. 2024. The llama 3 herd of models. arXiv preprint arXiv:2407.21783

    2024 arXiv

  6. [14]

    Wenji Fang, Yao Lu, Shang Liu, Qijun Zhang, Ceyu Xu, Lisa Wu Wills, Hongce Zhang, and Zhiyao Xie. 2023. Masterrtl: A pre-synthesis ppa estimation framework for any rtl design. In 2023 IEEE/ACM International Conference on Computer Aided Design (ICCAD), pages 1--9. IEEE

    2023

  7. [15]

    Google. 2021. https://github.com/google/skywater-pdk Skywater pdk . Accessed: 2025-02-11

    2021

  8. [16]

    Daya Guo, Qihao Zhu, Dejian Yang, Zhenda Xie, Kai Dong, Wentao Zhang, Guanting Chen, Xiao Bi, Yu Wu, YK Li, et al. 2024. Deepseek-coder: When the large language model meets programming--the rise of code intelligence. arXiv preprint arXiv:2401.14196

    2024 arXiv

  9. [17]

    Aaron Hurst, Adam Lerer, Adam P Goucher, Adam Perelman, Aditya Ramesh, Aidan Clark, AJ Ostrow, Akila Welihinda, Alan Hayes, Alec Radford, et al. 2024. Gpt-4o system card. arXiv preprint arXiv:2410.21276

    2024 arXiv

  10. [18]

    Ting Jiang, Shaohan Huang, Zhongzhi Luan, Deqing Wang, and Fuzhen Zhuang. 2023. Scaling sentence embeddings with large language models. arXiv preprint arXiv:2307.16645

    2023 arXiv

  11. [19]

    Chankyu Lee, Rajarshi Roy, Mengyao Xu, Jonathan Raiman, Mohammad Shoeybi, Bryan Catanzaro, and Wei Ping. 2024. Nv-embed: Improved techniques for training llms as generalist embedding models. arXiv preprint arXiv:2405.17428

    2024 arXiv

  12. [20]

    Yibin Lei, Di Wu, Tianyi Zhou, Tao Shen, Yu Cao, Chongyang Tao, and Andrew Yates. 2024. Meta-task prompting elicits embedding from large language models. arXiv preprint arXiv:2402.18458

    2024 arXiv

  13. [21]

    Zeju Li, Changran Xu, Zhengyuan Shi, Zedong Peng, Yi Liu, Yunhao Zhou, Lingfeng Zhou, Chengyu Ma, Jianyuan Zhong, Xi Wang, et al. 2025. Deepcircuitx: A comprehensive repository-level dataset for rtl code understanding, generation, and ppa analysis. arXiv preprint arXiv:2502.18297

    2025 arXiv

  14. [22]

    Chin-Yew Lin. 2004. Rouge: A package for automatic evaluation of summaries. In Text summarization branches out, pages 74--81

    2004

  15. [23]

    Mingjie Liu, Nathaniel Pinckney, Brucek Khailany, and Haoxing Ren. 2023. Verilogeval: Evaluating large language models for verilog code generation. In 2023 IEEE/ACM International Conference on Computer Aided Design (ICCAD), pages 1--8. IEEE

    2023

  16. [24]

    Shang Liu, Wenji Fang, Yao Lu, Jing Wang, Qijun Zhang, Hongce Zhang, and Zhiyao Xie. 2024. Rtlcoder: Fully open-source and efficient llm-assisted rtl code generation technique. IEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems

    2024

  17. [25]

    Yi Liu, Changran XU, Yunhao Zhou, Zeju Li, and Qiang Xu. 2025. https://openreview.net/forum?id=2hcfoCHKoB Deep RTL : Bridging verilog understanding and generation with a unified representation model . In The Thirteenth International Conference on Learning Representations

    2025

  18. [26]

    Yao Lu, Shang Liu, Qijun Zhang, and Zhiyao Xie. 2024. Rtllm: An open-source benchmark for design rtl generation with large language model. In 2024 29th Asia and South Pacific Design Automation Conference (ASP-DAC), pages 722--727. IEEE

    2024

  19. [27]

    Niklas Muennighoff. 2022. Sgpt: Gpt sentence embeddings for semantic search. arXiv preprint arXiv:2202.08904

    2022 arXiv

  20. [28]

    Niklas Muennighoff, Hongjin SU, Liang Wang, Nan Yang, Furu Wei, Tao Yu, Amanpreet Singh, and Douwe Kiela. 2025. https://openreview.net/forum?id=BC4lIvfSzv Generative representational instruction tuning . In The Thirteenth International Conference on Learning Representations

    2025

  21. [29]

    Niklas Muennighoff, Nouamane Tazi, Lo \" c Magne, and Nils Reimers. 2022. Mteb: Massive text embedding benchmark. arXiv preprint arXiv:2210.07316

    2022 arXiv

  22. [30]

    Arvind Neelakantan, Tao Xu, Raul Puri, Alec Radford, Jesse Michael Han, Jerry Tworek, Qiming Yuan, Nikolas Tezak, Jong Wook Kim, Chris Hallacy, et al. 2022. Text and code embeddings by contrastive pre-training. arXiv preprint arXiv:2201.10005

    2022 arXiv

  23. [31]

    Kishore Papineni, Salim Roukos, Todd Ward, and Wei-Jing Zhu. 2002. Bleu: a method for automatic evaluation of machine translation. In Proceedings of the 40th annual meeting of the Association for Computational Linguistics, pages 311--318

    2002

  24. [32]

    Colin Raffel, Noam Shazeer, Adam Roberts, Katherine Lee, Sharan Narang, Michael Matena, Yanqi Zhou, Wei Li, and Peter J Liu. 2020. Exploring the limits of transfer learning with a unified text-to-text transformer. Journal of machine learning research, 21(140):1--67

    2020

  25. [33]

    Karan Singhal, Shekoofeh Azizi, Tao Tu, S Sara Mahdavi, Jason Wei, Hyung Won Chung, Nathan Scales, Ajay Tanwani, Heather Cole-Lewis, Stephen Pfohl, et al. 2023. Large language models encode clinical knowledge. Nature, 620(7972):172--180

    2023

  26. [34]

    Jacob Mitchell Springer, Suhas Kotha, Daniel Fried, Graham Neubig, and Aditi Raghunathan. 2024. Repetition improves language model embeddings. arXiv preprint arXiv:2402.15449

    2024 arXiv

  27. [35]

    Shailja Thakur, Baleegh Ahmad, Zhenxing Fan, Hammond Pearce, Benjamin Tan, Ramesh Karri, Brendan Dolan-Gavitt, and Siddharth Garg. 2023. Benchmarking large language models for automated verilog rtl code generation. In 2023 Design, Automation & Test in Europe Conference & Exhib...

    2023

  28. [36]

    Shailja Thakur, Baleegh Ahmad, Hammond Pearce, Benjamin Tan, Brendan Dolan-Gavitt, Ramesh Karri, and Siddharth Garg. 2024. Verigen: A large language model for verilog code generation. ACM Transactions on Design Automation of Electronic Systems, 29(3):1--31

    2024

  29. [37]

    Shobha Vasudevan, Wenjie Joe Jiang, David Bieber, Rishabh Singh, C Richard Ho, Charles Sutton, et al. 2021. Learning semantic representations to verify hardware designs. Advances in Neural Information Processing Systems, 34:23491--23504

    2021

  30. [38]

    Liang Wang, Nan Yang, Xiaolong Huang, Linjun Yang, Rangan Majumder, and Furu Wei. 2023. Improving text embeddings with large language models. arXiv preprint arXiv:2401.00368

    2023 arXiv

  31. [39]

    Clifford Wolf, Johann Glaser, and Johannes Kepler. 2013. Yosys - a free Verilog synthesis suite. In Proceedings of the 21st Austrian Workshop on Microelectronics (Austrochip), volume 97

    2013

  32. [40]

    Ziqi Yan, Jiqiang Liu, Gang Li, Zhen Han, and Shuo Qiu. 2017. Privmin: Differentially private minhash for jaccard similarity computation. arXiv preprint arXiv:1705.07258

    2017 arXiv

  33. [41]

    PEI Zehua, Huiling Zhen, Mingxuan Yuan, Yu Huang, and Bei Yu. 2024. Betterv: Controlled verilog generation with discriminative guidance. In Forty-first International Conference on Machine Learning

    2024

  34. [42]

    Yongan Zhang, Zhongzhi Yu, Yonggan Fu, Cheng Wan, and Yingyan Celine Lin. 2024. Mg-verilog: Multi-grained dataset towards enhanced llm-assisted verilog generation. In 2024 IEEE LLM Aided Design Workshop (LAD), pages 1--5. IEEE

    2024

  35. [43]

    Yang Zhao, Di Huang, Chongxiao Li, Pengwei Jin, Ziyuan Nan, Tianyun Ma, Lei Qi, Yansong Pan, Zhenxing Zhang, Rui Zhang, et al. 2024. Codev: Empowering llms for verilog generation through multi-level summarization. arXiv preprint arXiv:2407.10424

    2024 arXiv

  36. [44]

    online" 'onlinestring :=

    ENTRY address archivePrefix author booktitle chapter edition editor eid eprint eprinttype howpublished institution journal key month note number organization pages publisher school series title type volume year doi pubmed url lastchecked label extra.label sort.label short.list...

  37. [45]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 gl...

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.