{"id":"14230039-7245-4c9b-9911-6cf029272aca","arxiv_id":"2411.09422","paper_version":2,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":3,"one_line_summary":"OpenLS-DGF generates and packages a 966k-circuit multi-task logic synthesis dataset from 46 designs, demonstrated on four ML tasks with high reported accuracy.","lead":"OpenLS-DGF is a pipeline that turns 46 combinational circuit designs into more than 966,000 Boolean circuits in seven representations, with optional technology mapping and timing data. The released OpenLS-D-v1 dataset is offered as a common resource for circuit classification, circuit ranking, QoR prediction, and probability prediction in logic synthesis.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Equivalence of logic-blasted circuits is asserted rather than proven; Theorem 1 validates dependency structure only, and the reported checks may not compare all 966k intermediate networks to the source functions.","rationale":"Reader's weakest assumption is the same one I would stress: Boolean-equivalence preservation through all seven logic types is the invariant that makes the dataset a single reusable artifact. I agree with the CONDITIONAL verdict. The paper has real independent support: the pipeline is concrete, the artifacts are open-sourced, and Step 7's claimed equivalence checks, if actually run on every file, would be strong evidence. But the manuscript does not report those checks or make their object unambiguous, and Theorem 1 cannot substitute for them. My concern is not that the conversions are known to be wrong; it is that the central correctness property is asserted, and the special MIG conversion is exactly the case where a hidden assumption is plausible. A stratified equivalence audit would settle the question. If the audit passes, the dataset claim should stand; if it fails, the framework needs a corrected conversion or explicit filtering of non-equivalent circuits. For now, CONDITIONAL remains the right verdict.","tokens_in":20879,"tokens_out":6192,"duration_ms":60922,"concrete_test":"Independently verify functional equivalence on a stratified sample from OpenLS-D-v1: draw, say, 100 circuits per logic type (including at least 100 MIGs) across designs and recipe indices; parse the stored GraphML/Verilog; build a miter against the corresponding original design or abc.aig.pt; and run ABC 'cec' or SAT, or exhaustive simulation for small designs. Report the mismatch rate separately by logic type. If any sampled circuit fails, the dataset labels are corrupted; if all sampled circuits pass, the concern is resolved for that sample and the framework's equivalence invariant is empirically supported.","verdict_should_be":"UNCHANGED","load_bearing_attack":"OpenLS-D-v1's core utility is that every circuit carrying a design label is Boolean-equivalent to that design: circuit classification, ranking, QoR, and probability labels all depend on it. The only proof offered for the logic-blasting step is Theorem 1, which claims preservation of 'dependency relationships' and argues from node matching plus the fact that MAJ3 can implement AND. That is not a functional-equivalence proof, and it is weakest exactly where the authors introduce an ad hoc method: the MIG 'topological node-wise conversion' in Step 4, chosen because {NOT, MAJ3} cannot be covered by a standard supergate library. No algorithm or invariant for this conversion is given. Step 7 says files 'undergo verification using combinational equivalence-checking tools,' but the only concrete statement is that LSILS files are checked against their corresponding gate-level netlists; if those netlists are mapped from the same possibly-wrong Boolean networks, the check does not establish equivalence to the source design, and no pass rates are reported. If even a small fraction of the 966k circuits are functionally wrong, the classification labels and downstream conclusions are not trustworthy.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper introduces OpenLS-DGF, a seven-step dataset generation framework built on Yosys, ABC, LSILS, iEDA, and LogicFactory, which converts combinational designs into six Boolean network representations (AIG, OIG, XAG, MIG, PRIMARY, GTG) plus ASIC and FPGA netlists with area and timing QoR, packaged into PyTorch-ready files through a custom Circuit class. The released OpenLS-D-v1 dataset contains 46 designs and more than 966,000 Boolean circuits generated from 1,000 synthesis recipes per design. The authors demonstrate the dataset's versatility through four downstream tasks: circuit classification, circuit ranking, QoR prediction, and probability prediction, and argue that OpenLS-D-v1 supports adaptive sub-dataset extraction and incremental extension.","tokens_in":21097,"tokens_out":4699,"duration_ms":46024,"significance":"If the equivalence and packaging claims hold, OpenLS-D-v1 is a valuable community resource: it is larger and more multi-task oriented than OpenABC-D, stores both Verilog and GraphML representations, includes an open-source circuit engine, and is publicly released. The four tasks span the main stages of logic synthesis and the dataset provides a single artifact from which multiple task-specific sub-datasets can be derived. These strengths are substantial and make the work a credible contribution to ML-for-EDA. However, the validation is currently weaker than the central claims require: the equivalence proof for logic blasting is not a functional-equivalence proof, the verification chain in Step 7 is under-specified, and the downstream experiments are single-run demonstrations without error bars or adequate baselines. These are fixable concerns rather than reasons to reject the paper.","major_comments":[{"comment":"The proof of Theorem 1 asserts that logic blasting preserves dependency relationships because nodes can be precisely matched and each MAJ3 gate can implement AND. This is not a proof of Boolean functional equivalence: a node-wise structural mapping plus a functional-completeness argument does not imply that every multi-input subcircuit is replaced by a functionally equivalent one, especially for the 'topological node-wise conversion' used for MIG, for which no algorithm or invariant is described. Since circuit classification, ranking, QoR, and probability labels all assume that every one of the 966k circuits is Boolean-equivalent to its source design, this is a load-bearing gap. Please either provide a formal equivalence invariant for each conversion (including MIG), or report per-type combinational equivalence checking of every converted network against the original design, with pass rates.","section":"Section III-B, Step 4 and Theorem 1"},{"comment":"The verification statement says that LSILS-generated files are checked against their corresponding gate-level netlists in abc.aig.pt. This checks the mapped netlist against the possibly already-incorrect Boolean network from the same pipeline, not against the original source design; it therefore cannot certify that the stored circuits realize the intended functions. The paper also does not report how many files passed, which tool configurations were used, or whether any files failed and were discarded. Please specify the verification chain from the source design through each intermediate representation and give the pass/fail counts for all logic types.","section":"Section III-B, Step 7"},{"comment":"The QoR prediction setup is under-specified. The model outputs a softmax over 'the overall distribution', but the binning scheme (number of bins, boundaries, normalization, and how the QoR value is mapped to a bin index) is not defined, and the MAPE numbers in Table VI are reported without error bars, repeated trials, or comparison to a simple baseline such as predicting the median or mean of the training distribution. Without this information the reported area MAPE of 0.69% and timing MAPE of 6.49% cannot be reproduced or assessed. Please provide the binning details, the split sizes, and variance estimates.","section":"Section V-E, Table VI"},{"comment":"The probability prediction comparison is unclear: the 'Comparison' column in Table VII mixes PE improvements and time speedups with ambiguous notation, and the DeepGate2 and GraphSAGE baselines are not described in terms of architecture, features, training setup, or whether DeepGate2 is the published model or a reimplementation. No error bars or trial counts are given. As written, the table does not support the claim that the proposed method outperforms the baselines. Please specify the baselines and report means and standard deviations over multiple runs.","section":"Section V-F, Table VII"},{"comment":"Across all four downstream tasks, the experimental evidence is single-run and largely lacks baselines: classification is supported by a t-SNE plot and one accuracy number at epoch 10, ranking reports accuracy for one split without variance, and the QoR and probability tasks likewise report point estimates. While these experiments are presented as demonstrations of dataset applicability rather than as state-of-the-art benchmarks, the paper's claim that the dataset achieves 'prominent diversity and applicability' would be materially stronger if at least one task included a standard baseline, error bars, or multiple random seeds.","section":"Sections V-C to V-F"}],"minor_comments":[{"comment":"The text writes 'OoR' where 'QoR' is intended; please fix the typo.","section":"Section V-D, Problem Formulation"},{"comment":"The caption of Fig. 14 says 'all three models achieve high accuracy', but the figure and surrounding text present a single model; Table V lists three models but no per-model figure is shown.","section":"Section V-D, Fig. 14"},{"comment":"The label 'PRIMAYR' should read 'PRIMARY'.","section":"Fig. 2"},{"comment":"The cosine similarity matrix is normalized to [0.3, 0.6], which makes the reported average similarity of 0.44 an artifact of the normalization rather than a raw similarity value; please report raw values or justify the normalization.","section":"Section IV-A-2"},{"comment":"The per-design counts in the table sum to 21,000 only if the 7,000 ASIC and 7,000 FPGA netlists are counted as part of the total; the grouping would be clearer if Boolean network counts and netlist counts were listed separately.","section":"Table IV"}],"recommendation":"major_revision","confidential_remarks":"The manuscript is within scope for a circuits/EDA venue and the open-source release is a genuine asset. My main concern is verification depth and experimental rigor rather than novelty. I would be willing to re-review a revised version that addresses the equivalence-checking evidence, the verification chain in Step 7, and the under-specified experimental protocols."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Short version: this is a real dataset contribution worth engaging with, but the equivalence-preservation claim for the logic-blasting step is asserted, not proven, and the paper's own verification description doesn't fully close that gap. If you use the dataset, spot-check the MIG/PRIMARY conversions yourself.\n\nWhat's actually new: OpenLS-D-v1 is the first multi-task logic synthesis dataset I know of that combines seven logic types, 1000 recipes per design, and four tasks from one artifact. The framework is described in enough detail to be reproducible in principle: seven steps, explicit tools (Yosys, ABC, LSILS, iEDA), and both Verilog and GraphML preservation. The circuit-ranking task is new, and the adaptive extraction layer is sensible. The scale (966k circuits, 46 designs) is real and the storage/compression numbers are credible.\n\nCredit where due: the dataset generation pipeline is the contribution, and the packaging design is thoughtful. The diversity analysis (Fig. 6) is a reasonable sanity check. The four downstream tasks are demonstrations, not benchmarks, and the paper mostly treats them that way, which is appropriate for a dataset paper.\n\nSoft spots, in proportion: Theorem 1 is not a functional-equivalence proof. It says the mapping step matches nodes and preserves 'dependency relationships,' then notes MAJ3 can implement AND. That's structural correspondence plus a nod to functional completeness; it doesn't prove the MIG conversion is Boolean-equivalent. The stress-test note is right that the MIG 'topological node-wise conversion' is the weakest link because it's ad hoc and no algorithm is given. The paper does say Step 7 runs combinational equivalence checking, but the concrete description only compares LSILS netlists against ABC netlists, and no pass rates are reported. If the ABC AIGs are themselves assumed correct, that's a reasonable chain, but it's an assumption the paper doesn't state explicitly. So the correctness of 966k circuits is not as airtight as the abstract implies.\n\nAlso minor: the task experiments have no error bars or baselines except DeepGate2/GraphSAGE in Table VII, and that table's formatting is confusing. Reproducibility depends on a GitHub readme without a commit hash; for a dataset paper that's a real weakness.\n\nWho this is for: anyone working on ML for logic synthesis who needs a common dataset to compare methods. It deserves a serious referee; the dataset itself is valuable and the framework is mostly credible, but the equivalence claim needs to be either proven properly or backed by systematic equivalence-checking results with pass rates. I'd recommend accepting after minor-to-major revision, depending on whether they supply the missing verification evidence.","headline":"A genuinely useful multi-task dataset for EDA-ML, with the central equivalence claim for logic-blasting asserted rather than proven — worth reviewing, but verify before trusting.","tokens_in":21653,"tokens_out":2334,"would_cite":true,"duration_ms":20812,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"OpenLS-DGF is an open-source flow that builds one adaptive logic-synthesis dataset spanning Boolean representation, logic optimization, and technology mapping; its OpenLS-D-v1 (966k circuits) supports four ML tasks.","keywords":["logic synthesis","machine learning","dataset generation","Boolean networks","GraphML","technology mapping","QoR prediction","circuit ranking"],"falsifier":"Run an independent combinational equivalence check on a random sample, or the full set, of the generated OIG, XAG, MIG, PRIMARY, and GTG circuits against their source AIGs, and separately compare node-dependency graphs before and after the conversion; a single functionally inequivalent sampled circuit, or any mismatch in dependency structure, would falsify the framework's label-correctness claim.","tokens_in":20702,"feed_emoji":"⚙️","tokens_out":16877,"duration_ms":135610,"temperature":0.7,"pith_summary":"OpenLS-DGF aims to give the logic-synthesis machine-learning community a single, open, adaptive dataset-generation flow instead of the task-specific datasets used before. The paper describes a seven-step pipeline that starts from 46 combinational designs, applies 1,000 logic-optimization recipes per design, blasts each optimized And-Inverter Graph (AIG) into seven logic representations, maps each representation to ASIC and FPGA netlists, measures timing, and packages the result as over 966,000 Boolean circuits with quality-of-results metadata. The central claim is that one such dataset can support several distinct ML tasks at once—the paper demonstrates circuit classification, circuit ranking, QoR prediction, and probability prediction—so models for different synthesis stages can be trained and compared on identical circuits. If this is right, it would replace bespoke task-specific datasets with a standard resource and make ML work in logic synthesis measurable on a common ground.","feed_headline":"One dataset flow yields 966k circuits for ML","feed_subtitle":"OpenLS-D-v1 packages seven logic types with QoR data so one artifact can train classifiers, rankers, and predictors.","key_machinery":"The carrying mechanism is the closed loop between the conversion pipeline and the paper's Circuit class. On the synthesis side, the logic-blasting step turns one optimized AIG per recipe into six further Boolean-network types; Theorem 1 asserts that the mapping step preserves node dependencies by matching nodes before and after blasting, and that each MAJ3 gate can stand in for an AND gate. On the data side, the Circuit class is a generic graph object that stores each node's type, name, fanins, and a 64-bit truth table, keeps the original Boolean-circuit index alongside an internal index, and converts GraphML into a graph-learning tensor format with identical node indices. That index-preserving bridge is what lets a single dataset item be loaded, simulated, relabeled, and repacked into task-specific sub-datasets without losing the connection to the original circuit.","core_discovery":"OpenLS-DGF is presented as an adaptive, open-source flow that covers the three fundamental stages of logic synthesis—Boolean representation, logic optimization, and technology mapping—in seven steps. Starting from 46 combinational benchmark designs, it applies 1,000 random optimization recipes per design, then logic-blasts each optimized AIG into a family of seven Boolean-network types (AIG, OIG, XAG, MIG, PRIMARY, and GTG), maps each to ASIC and FPGA netlists, runs static timing analysis, and packages everything into per-design tensor files. The resulting OpenLS-D-v1 contains over 966,000 Boolean circuits, each stored in Verilog and GraphML with quality-of-results metadata; the paper claims its Theorem 1 guarantees the logic-blasting conversion preserves node dependencies and that all generated files are verified by combinational equivalence checking. Four downstream tasks—circuit classification, circuit ranking, QoR prediction, and probability prediction—are implemented on sub-datasets extracted from OpenLS-D-v1, with reported classification accuracy near 99.8%, ranking accuracy near 99.5%, area prediction errors between 0.69% and 1.17%, and probability prediction errors as low as 0.0008.","pith_inferences":["Editorial extension: the per-recipe indexing shared across all packaged files makes OpenLS-D-v1 ready-made for transfer-learning benchmarks—train a classifier or QoR predictor on one task and test whether the learned circuit embeddings transfer to ranking or probability prediction.","Editorial extension: the paper's observation that QoR intervals stabilize after a few hundred recipes suggests a cheap generation-time stopping rule—monitor the QoR distribution's spread and stop adding recipes once it stops changing, which would cut the reported 76-hour generation cost.","Editorial extension: because the Circuit class stores a 64-bit truth table per node, the dataset could support node-level functional-equivalence tasks, such as cut enumeration or resubstitution candidate generation, without any new conversion work.","Editorial extension: if the AIG-to-MIG conversion were ever formally verified as functional equivalence rather than topological matching, the dataset's guarantees would become stronger; a direct test would be to implement a supergate-based MIG mapper and compare its outputs with the topological conversion's outputs."],"forward_implications":["One OpenLS-D-v1 item can feed at least four tasks—classification, ranking, QoR prediction, and probability prediction—so ML models for different synthesis stages can be trained and compared on identical circuits from a single download.","Circuit ranking is introduced as a new task: if a model can rank Boolean representations before technology mapping, designers can skip the expensive mapping and timing runs for inferior variants.","QoR prediction on unseen recipes, unseen designs, and unseen recipe-design combinations is reported to reach mean absolute percentage errors between 0.69% and 1.17% for area and between 6.49% and 7.87% for timing, suggesting the dataset carries enough structure for generalization.","The observation that different Boolean representations of the same design can have non-overlapping QoR regions means representation choice is a real design decision, not a cosmetic one.","Because intermediate Verilog files and TCL scripts are preserved, researchers can insert additional synthesis steps at any point and regenerate only the affected dataset files."],"supporting_citations":[{"why":"Supplies combinational benchmark designs used as source inputs.","marker":"[16]"},{"why":"Supplies a second set of combinational benchmark designs, broadening design diversity.","marker":"[17]"},{"why":"Supplies IP-core designs that extend the source-design pool.","marker":"[18]"},{"why":"Provides the prior dataset that this work compares against and positions itself relative to.","marker":"[19]"},{"why":"Serves as the frontend parser for translating source designs into generic technology circuits and for checking raw files.","marker":"[26]"},{"why":"Provides the logic-optimization commands and recipe sequences that produce the 1,000 AIG variants per design.","marker":"[27]"},{"why":"Performs the logic blasting into multiple Boolean-network types and the ASIC/FPGA technology mapping.","marker":"[28]"},{"why":"Supplies the standard-cell library used for ASIC technology mapping.","marker":"[31]"},{"why":"Provides static timing analysis used to compute the quality-of-results metadata.","marker":"[32]"},{"why":"Defines the graph tensor format that the dataset is packaged into for machine-learning tasks.","marker":"[33]"}],"fun_headline_variants":["966k circuits from 46 designs via OpenLS-DGF","OpenLS-DGF: one flow, seven logic types, 966k circuits","OpenLS-DGF: 7 logic types, 966k circuits, 4 ML tasks"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that every logic-blasting conversion, including the topological AIG-to-MIG conversion, preserves both Boolean equivalence and node dependencies for all 966,000 generated circuits.","fun_headline_variants_meta":{"raw":{"variants":["966k circuits from 46 designs via OpenLS-DGF","OpenLS-DGF: one flow, seven logic types, 966k circuits","OpenLS-DGF: 7 logic types, 966k circuits, 4 ML tasks"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.0009,"raw_usage":{"total_tokens":3934,"prompt_tokens":1064,"completion_tokens":2870,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":680,"completion_tokens_details":{"reasoning_tokens":2803}},"tokens_in":680,"tokens_out":2870,"duration_ms":18676,"temperature":1.0,"reasoning_tokens":2803,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-12T20:39:27.347593+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run an independent combinational equivalence check on a random sample, or the full set, of the generated OIG, XAG, MIG, PRIMARY, and GTG circuits against their source AIGs, and separately compare node-dependency graphs before and after the conversion; a single functionally inequivalent sampled circuit, or any mismatch in dependency structure, would falsify the framework's label-correctness claim.","supporting_citations":[{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Supplies IP-core designs that extend the source-design pool."},{"cited_title":"IWLS 2005 benchmarks,","cited_arxiv_id":null,"evidence_quote":"Supplies combinational benchmark designs used as source inputs."},{"cited_title":"The EPFL combina- tional benchmark suite,","cited_arxiv_id":null,"evidence_quote":"Supplies a second set of combinational benchmark designs, broadening design diversity."},{"cited_title":"ABC: An Academic Industrial-Strength Verification Tool,","cited_arxiv_id":null,"evidence_quote":"Provides the logic-optimization commands and recipe sequences that produce the 1,000 AIG variants per design."},{"cited_title":"SkyWater SKY130 PDK,","cited_arxiv_id":null,"evidence_quote":"Supplies the standard-cell library used for ASIC technology mapping."}],"review_version":1}