{"id":"c4c335f7-c6c9-4447-a2d3-01ef11ba352c","arxiv_id":"2508.03558","paper_version":1,"verdict":"UNVERDICTED","confidence":"LOW","novelty_score":6.0,"correctness_risk":"unknown","formal_verification":"none","parameter_count":0,"one_line_summary":"A fine-tuned 7B LLM with AST-guided prompting generates HLS code that is near 100% synthesizable and 75% functionally correct, according to the abstract.","lead":"This paper proposes SAGE-HLS, a fine-tuned large language model that generates high-level synthesis (HLS) code for hardware designs. The authors claim near-perfect synthesizability and 75% functional correctness, but the submitted full text is a different paper, leaving these claims unverified.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Submission's full text is an unrelated PyLate paper; the SAGE-HLS claims are unverifiable as submitted, and the Verilog-to-C porting assumption cannot be assessed.","rationale":"The reader's UNVERDICTED verdict is correct given the manuscript mismatch. I partially agree with the reader's stated weakest assumption: the Verilog-to-C porting process is indeed a critical assumption for the central claim. However, the more immediate load-bearing concern is that the submitted full text is a completely different paper (PyLate), so no evidence is available to evaluate the porting process, the fine-tuning strategy, or the evaluation framework. This is a manuscript-integrity issue that blocks any substantive assessment; it is not a critique of the technical argument itself. The proposed test—fetching the real arXiv paper—is the minimal step that would settle whether the claimed paper exists and whether its evaluation supports the abstract's numbers. If the real paper is retrieved and passes the dataset/evaluation checks, the concern would be resolved and the verdict could move toward a more substantive review. Since the reader already reached UNVERDICTED for reasons that overlap with this concern, my read does not change the verdict.","tokens_in":7514,"tokens_out":2680,"duration_ms":30990,"concrete_test":"Download the actual arXiv record for 2508.03558 and verify whether its full text matches the SAGE-HLS abstract rather than the PyLate paper. If it matches, inspect the evaluation and dataset sections to confirm that (a) the ported C dataset is validated for functional equivalence against the original Verilog (e.g., through simulation on the original testbenches), and (b) the 75% functional-correctness figure is computed on held-out reference Verilog testbenches with generated C simulated and compared to expected outputs. If the retrieved full text is PyLate, the submission is not the claimed paper and the empirical claims remain entirely unsupported.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central empirical claim—near 100% synthesizability and 75% functional correctness for SAGE-HLS—cannot be checked against the submitted manuscript because the full text is the PyLate paper (cs.IR), not SAGE-HLS. The submission therefore contains no methodology, dataset construction details, fine-tuning procedure, or evaluation protocol. The weakest link in the stated argument is the dataset construction step: 'Verilog-to-C/C++ porting, converting verified and synthesizable Verilog codes into corresponding C.' For the reported accuracy to transfer to real HLS tasks, the ported C code must be (i) functionally equivalent to the original Verilog, (ii) written in HLS-synthesizable C/C++ idioms, and (iii) representative of realistic HLS designs. If the porting tool silently changes behavior (e.g., bit-width truncation, loop unrolling, memory access patterns), the fine-tuned model could memorize artifacts rather than learn correct HLS coding. Likewise, the evaluation framework based on VerilogEval is described only in the abstract; without seeing how functional correctness is computed—whether it simulates generated C against reference Verilog testbenches, whether testbenches are held out, and whether the 75% figure is a single-run point estimate—the headline number is not interpretable. Because the supplied document omits all of this, the correct stance is UNVERDICTED rather than acceptance or rejection of the technical claims.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The submission presents an abstract claiming a new fine-tuned LLM, SAGE-HLS, for high-level synthesis (HLS) code generation, with a reported near-100% synthesizability rate and 75% functional correctness on VerilogEval. However, the supplied full text is the PyLate paper (arXiv:2508.03555, cs.IR), which describes a library for late-interaction retrieval models and contains no mention of SAGE-HLS, HLS, Verilog, or any of the claimed experiments. The manuscript as submitted therefore consists of an abstract and a body that are about entirely different papers. No methodology, dataset construction details, fine-tuning procedure, or evaluation protocol for SAGE-HLS is present, making the central claims unverifiable.","tokens_in":7788,"tokens_out":2977,"duration_ms":29896,"significance":"If the abstract's claims were fully supported, SAGE-HLS would be a significant contribution to EDA and HLS automation, potentially opening a new direction for fine-tuning LLMs on HLS-specific code. The idea of creating an HLS dataset via Verilog-to-C/C++ porting is interesting, and the reported performance numbers would be impressive. However, the submitted manuscript does not contain the technical content needed to validate any of these claims. The paper cannot be assessed for soundness, and its potential significance is therefore entirely conditional on a body of work that is absent from the submission.","major_comments":[{"comment":"The full text is the PyLate paper, which is about late-interaction models for neural retrieval and is unrelated to the abstract's topic. Sections 1 through 6 of the full text discuss ColBERT, MaxSim, training libraries, and BEIR benchmarks; they never mention SAGE-HLS, high-level synthesis, Verilog, or the claimed experiments. Consequently, the central claim of the abstract—that SAGE-HLS achieves near-100% synthesizability and 75% functional correctness—is entirely unsupported by any methodology, results, or analysis in the manuscript. This is a load-bearing failure that cannot be remedied by local revision.","section":"Full Text (all sections)"},{"comment":"The abstract states that the dataset is created by 'Verilog-to-C/C++ porting, converting verified and synthesizable Verilog codes into corresponding C,' yielding 16.7K HLS codes. The full text provides no details of this porting process: there is no description of the source Verilog corpus, the porting tool or rules, handling of bit-widths, control flow, loops, or memory access patterns, or insertion of HLS directives. Without these details it is impossible to assess whether the resulting C code is functionally equivalent to the Verilog, whether it follows HLS-synthesizable idioms, or whether it is representative of realistic HLS designs. The correctness of the fine-tuning data is the foundational assumption of the entire approach and is left unstated.","section":"Abstract, dataset construction"},{"comment":"The abstract mentions a 'semi-automated evaluation framework using VerilogEval to assess the functionality of the generated HLS code,' but the full text contains no description of this framework. In particular, it is unknown whether functional correctness is measured by simulation against reference Verilog testbenches, whether the testbenches are held out from the training data, how synthesizability is checked, or whether the reported 75% figure is a single-run point estimate or an average over multiple runs. This makes the headline numbers uninterpretable and impossible to reproduce.","section":"Abstract, evaluation framework"},{"comment":"The abstract claims SAGE-HLS is 'the first-of-its-kind fine-tuned LLM specifically for HLS code generation.' The full text provides no comparison to prior work, such as existing HLS-oriented LLM approaches, prompt-engineering methods, or RAG-based systems. Without a literature context and comparative experiments, this novelty claim cannot be evaluated.","section":"Abstract, novelty claim"}],"minor_comments":[{"comment":"There is a typo: 'fined-tuned' should be 'fine-tuned'.","section":"Abstract"},{"comment":"The abstract refers to 'QwenCoder (2.5) 7B' without a citation or version identifier; providing the exact model name and a reference would help reproducibility.","section":"Abstract"}],"recommendation":"reject","confidential_remarks":"This submission appears to have the wrong full text: the abstract describes SAGE-HLS (cs.PL/EDA) while the body is the PyLate paper (cs.IR). This is not a technical weakness that can be addressed in a normal revision; it is a fundamental mismatch that invalidates the manuscript as submitted. I recommend that the editor desk-reject or return the submission to the authors with instructions to upload the correct full text. If a proper SAGE-HLS manuscript is subsequently submitted, it should be reviewed as a new submission."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"The one thing you should know: the submitted full text is PyLate, an information retrieval library paper, not SAGE-HLS. There is no methodology, no dataset construction, no evaluation protocol, no ablation — just an abstract claiming a fine-tuned LLM for HLS code generation with 100% synthesizability and 75% functional correctness. As a submission, it is not reviewable.\n\nWhat is genuinely interesting, based only on the abstract, is the proposal itself: a first fine-tuned LLM for HLS, built from a 16.7K Verilog-to-C ported dataset, with AST-guided instruction tuning and a VerilogEval-based evaluation harness. If the dataset is real and the porting preserves functional equivalence, that would be a useful contribution to the EDA community. The idea of moving beyond prompt engineering and RAG for HLS is sound.\n\nNow the soft spots. The obvious one is the manuscript mismatch — that alone is a desk-reject reason. Beyond that, the abstract's numbers are not interpretable. Near-100% synthesizability and 75% correctness are reported without error bars, test splits, or baseline comparisons. The weakest link is the Verilog-to-C porting step. For the fine-tuning to transfer to real HLS tasks, the ported C must be functionally equivalent to the original Verilog, written in synthesizable HLS idioms, and representative of actual designs. If the porting script introduces subtle bit-width or loop-structure changes, the model could memorize artifacts rather than learn HLS. The AST guidance could also be a thin wrapper. None of this can be checked because the document doesn't contain the details.\n\nI'm not manufacturing flaws here. The central idea is worth a serious look, and the abstract is coherent. But a reviewer cannot evaluate a paper that isn't there. The right move is to desk reject this submission due to the wrong full text and invite the authors to resubmit with the correct manuscript. If that corrected version ships, it deserves full peer review.\n\nFor your own work: don't cite SAGE-HLS based on this abstract alone, and don't put this in a reading group until the real paper appears.","headline":"As submitted, the paper cannot be reviewed: the full text is an unrelated PyLate IR paper, so every SAGE-HLS claim sits on an abstract alone.","tokens_in":8302,"tokens_out":1611,"would_cite":false,"duration_ms":17766,"reading_group":"no","serious_thinker":"unclear","would_accept_peer_review":false},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"SAGE-HLS claims that a fine-tuned 7B model, trained on 16.7K Verilog-to-C ported designs with AST-guided prompts, generates HLS code that is nearly always synthesizable and functionally correct 75% of the time.","keywords":["high-level synthesis","LLM code generation","Verilog-to-C porting","abstract syntax tree","instruction fine-tuning","synthesizability","functional correctness","QwenCoder"],"falsifier":"Run SAGE-HLS on a held-out suite of HLS designs that were not created by the paper's own Verilog-to-C/C++ porting process, synthesize each output with a standard HLS tool, and compare functional behavior against the original Verilog by simulation or formal equivalence; if the functional correctness rate drops well below 75% on such out-of-distribution designs, the central claim fails.","tokens_in":7354,"feed_emoji":"⚙️","tokens_out":7222,"duration_ms":76401,"temperature":0.7,"pith_summary":"High-level synthesis turns C/C++ into hardware, and this paper tries to show that a fine-tuned large language model can do that code-generation step on its own. The central claim is that SAGE-HLS, a 7B-parameter model fine-tuned on 16.7K HLS examples created by porting verified Verilog designs into C/C++, produces code that is almost always synthesizable and functionally correct about three-quarters of the time. To make that work, the authors build a Verilog-to-C/C++ porting pipeline, use instruction prompts guided by abstract syntax trees, and check outputs with a semi-automated VerilogEval-based framework. If true, this would move HLS code generation beyond prompt engineering and retrieval-augmented generation toward a dedicated fine-tuned model.","feed_headline":"LLM writes HLS code that is 100% synthesizable, 75% correct","feed_subtitle":"A 7B model fine-tuned on 16.7K ported Verilog designs compiles almost always and passes functional checks most of the time.","key_machinery":"The central mechanism is the pairing of a Verilog-to-C/C++ porting pipeline with AST-guided instruction fine-tuning. The porting pipeline converts verified, synthesizable Verilog designs into corresponding C/C++ code, producing a 16.7K-example training set. The fine-tuning strategy frames code generation as an instruction-prompting task in which the model must produce C/C++ code from a prompt while an abstract syntax tree guides the structure, which is how a 7B base model reaches high synthesizability. A semi-automated VerilogEval-based evaluation framework then checks whether the generated HLS code actually behaves as intended.","core_discovery":"The paper's core claim is that a relatively small fine-tuned model, QwenCoder 2.5 7B, can produce HLS code with a near-100% synthesizability rate and a 75% functional correctness rate, provided it is trained on a purpose-built HLS dataset and prompted with AST-guided instructions. The authors present this as the first fine-tuned LLM specifically trained for HLS code generation. The dataset is assembled by porting verified, synthesizable Verilog designs into equivalent C/C++, giving the model a large set of known-good input-output pairs rather than relying on general code corpora. The authors read the near-perfect synthesizability as evidence that the model has internalized HLS-specific syntax and structure, while the 75% correctness figure marks the remaining gap as semantic fidelity to the intended behavior.","pith_inferences":["Because the reported numbers come from a dataset built by the authors' own porting pipeline, the crucial transfer test is whether the model holds up on hand-written HLS code outside that pipeline; until such a test is done, the 75% figure should be read as a within-pipeline result.","The near-100% synthesizability with only 75% correctness suggests the model has mastered HLS syntax but often misses behavioral intent; a natural extension is training with simulation feedback or formal equivalence checking in the loop.","Extending the Verilog-to-C/C++ porting pipeline to include HLS pragmas and directives could turn the same method into a design-space-exploration tool that generates alternative implementations with different area and latency trade-offs.","The supplied full text of the manuscript is an unrelated paper, so this summary is grounded in the abstract; the detailed experimental evidence needed to audit the dataset and evaluation is not present in the provided text."],"forward_implications":["If the reported numbers hold, HLS designers could use the model to generate candidate C/C++ implementations that are very likely to compile for synthesis, saving iterations on syntax and interface errors.","The 16.7K Verilog-to-C/C++ ported dataset would become a reusable resource for training and benchmarking future HLS-focused language models.","AST-guided instruction prompting appears to be a transferable recipe for structured code-generation tasks beyond HLS.","The VerilogEval-based semi-automated evaluation framework gives the community a concrete way to measure the functionality, not just the syntax, of generated HLS code.","The wide gap between synthesizability and functional correctness implies that the next bottleneck is semantic accuracy, so future fine-tuning should target behavioral equivalence rather than compilability."],"supporting_citations":[],"fun_headline_variants":["Fine-tuned LLM writes HLS code: near-100% synthesizable, 75% correct","AST-guided LLM for HLS: near-100% synthesizable, 75% correct","7B LLM fine-tuned for HLS hits near-100% synth, 75% correctness","First HLS-tuned LLM: near-100% synthesizable, 75% functional","SAGE-HLS: Syntax-aware LLM for HLS codegen with 75% accuracy"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that the Verilog-to-C/C++ porting produces C code that is correct, representative of real HLS programs, and suitable for fine-tuning, because the reported 100% and 75% success rates inherit whatever errors, omissions, or unusual coding patterns that porting process introduces.","fun_headline_variants_meta":{"raw":{"variants":["Fine-tuned LLM writes HLS code: near-100% synthesizable, 75% correct","AST-guided LLM for HLS: near-100% synthesizable, 75% correct","7B LLM fine-tuned for HLS hits near-100% synth, 75% correctness","First HLS-tuned LLM: near-100% synthesizable, 75% functional","SAGE-HLS: Syntax-aware LLM for HLS codegen with 75% accuracy"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000602,"raw_usage":{"total_tokens":2849,"prompt_tokens":1023,"completion_tokens":1826,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":639,"completion_tokens_details":{"reasoning_tokens":1701}},"tokens_in":639,"tokens_out":1826,"duration_ms":17281,"temperature":1.0,"reasoning_tokens":1701,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-06T04:20:30.624342+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run SAGE-HLS on a held-out suite of HLS designs that were not created by the paper's own Verilog-to-C/C++ porting process, synthesize each output with a standard HLS tool, and compare functional behavior against the original Verilog by simulation or formal equivalence; if the functional correctness rate drops well below 75% on such out-of-distribution designs, the central claim fails.","supporting_citations":[],"review_version":1}