{"id":"6461425b-5ab0-4bbd-b4bf-c0aedac8f40d","arxiv_id":"2501.09954","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":4,"one_line_summary":"Contrastive learning and a unified ordinal output let a transformer predict optimal accelerator hardware configurations for DNN workloads at constant time, with 91% accuracy on a MAESTRO-based dataset.","lead":"A machine learning system trained on simulated chip-design examples predicts good accelerator configurations for neural network workloads in one pass, instead of running a slow search. It reports 15% better agreement with simulator-chosen optima than earlier learning methods and lower simulated latency on unseen models including LLMs.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The accuracy and deployment claims are benchmarked against ConfuciuX-generated labels, but the paper never checks whether ConfuciuX returns true optima or compares directly with ConfuciuX, so the central 'replace iterative search' claim is not yet tested.","rationale":"I read the paper as an ML engineering contribution: the architecture, contrastive stage, UOV, ablations, and open-sourced code/dataset are real positives. The strongest claim, however, is about identifying optimal hardware, and that claim rests on a single empirical foundation: the ConfuciuX-generated labels. The paper provides no convergence analysis, no comparison against exhaustive search over the small output space, and no deployment run of ConfuciuX. The reader's weakest_assumption names exactly this, and I agree. A secondary issue, undefined accuracy metric, is real but less load-bearing because the deployment latency comparison could in principle compensate; it does not, since it omits the key search baseline. My proposed check would settle the concern: if labels are verified as optimal and AIRCHITECT v2 beats ConfuciuX on latency, the paper's central claim stands; otherwise it should be reframed as 'learned imitation of ConfuciuX with faster inference.' No verdict change beyond conditional.","tokens_in":11528,"tokens_out":5902,"duration_ms":61535,"concrete_test":"On a random subset of 100 held-out inputs, enumerate all 768 (PE, buffer-size) configurations with MAESTRO to obtain the true minimum-latency label; compare (a) ConfuciuX labels, (b) AIRCHITECT v2 predictions, and (c) Table III accuracy recomputed against exhaustive labels. Separately, run ConfuciuX on the unseen workloads in §IV-C and compare achieved MAESTRO latency and wall-clock time against AIRCHITECT v2. If ConfuciuX labels disagree with exhaustive optima on a substantial fraction of samples, or ConfuciuX matches/beats AIRCHITECT v2 on deployment latency, the central claims collapse to 'imitates ConfuciuX'. If they agree, the concern is resolved.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The dataset (§III-A) is created by running ConfuciuX [12] with latency as reward, and Table III's 91.17% accuracy is agreement with those labels on a held-out split. If ConfuciuX returns locally optimal or heuristic configurations, every accuracy number measures fidelity to ConfuciuX, not distance to the true optimum; the model's ceiling is the search quality of the label generator. The deployment evaluation (§IV-C) strengthens this concern: it compares against AIRCHITECT v1, GANDSE, and VAESA+BO, but omits ConfuciuX itself, the search method the method is meant to replace. The 1.7x latency figure is therefore not evidence that one-shot inference beats iterative search; it only shows the model beats the other learned baselines on this benchmark. A direct comparison to ConfuciuX, or to exhaustive enumeration over the 768-configuration output space, is required before the central claim can be accepted.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes AIRCHITECT v2, an encoder-decoder transformer for one-shot hardware accelerator design space exploration. The method trains an encoder with a contrastive loss plus a latency-prediction loss, then trains a decoder with UOV heads that combine classification (bucket selection) and regression (within-bucket prediction). The evaluation uses a 100K-sample MAESTRO-based dataset whose labels are generated by running the ConfuciuX RL search, and reports 91.17% layer-level prediction accuracy, outperforming AIRCHITECT v1 (77.60%) and GANDSE (84.39%), plus a model-level deployment study claiming 1.7x inference-latency improvement on unseen DNN/LLM workloads. The paper also releases code, data, and trained models.","tokens_in":11693,"tokens_out":5378,"duration_ms":54301,"significance":"If the empirical claims hold, the paper offers a useful step toward constant-time DSE for accelerator resource allocation, and the public release of the dataset and code is a concrete contribution. The ablations for the contrastive objective, the UOV-versus-classification comparison, and the embedding-space search comparison against VAE are informative. However, the current evaluation does not yet establish the central claim that one-shot inference can replace iterative search: the accuracy numbers are agreement with ConfuciuX-generated labels, and the deployment comparison omits ConfuciuX itself. The reported gains therefore need additional evidence before the paper can be accepted as a demonstration of replacing search-based DSE.","major_comments":[{"comment":"The reported 91.17% accuracy is agreement with labels produced by executing ConfuciuX with latency as the reward, but the paper does not define the accuracy metric (exact match on both PE and buffer size, or per-dimension agreement) and does not establish that ConfuciuX returns globally optimal configurations. Because the labels are generated by a search method, every accuracy number is bounded above by the label generator's search quality; without an oracle or an exhaustive check over the 768-configuration output space, the claim that the model identifies \"optimal design points\" is not supported. Please define the metric and report label-quality statistics, for example the distribution of latency gaps between ConfuciuX labels and the best configuration found by exhaustive enumeration.","section":"Section III-A, Table III"},{"comment":"The model-level deployment evaluation compares AIRCHITECT v2 with AIRCHITECT v1, GANDSE, and VAESA+BO, but omits ConfuciuX itself, the RL search method whose labels define the task and whose role the paper claims to replace. The 1.7x latency improvement is therefore relative to other learned baselines only; a direct head-to-head with ConfuciuX (with matched number of MAESTRO evaluations or wall-clock time), or with exhaustive enumeration, is required to support the central claim that one-shot inference replaces iterative search.","section":"Section IV-C, Figure 7"},{"comment":"Hyperparameter K (the number of UOV buckets) is chosen using accuracy and model-size curves in Figure 8(b), but the paper describes only an 80K/20K train/test split with no validation set. If the accuracy curve is computed on the same test split used for Table III, selecting K=16 on the test set leaks test information into the reported 91.17%; please report a validation-based selection procedure and state clearly which split each ablation uses.","section":"Section IV-A, Section IV-D, Figure 8(b)"},{"comment":"All accuracy and latency results are single point estimates with no error bars, confidence intervals, or significance tests. Given that training involves stochastic contrastive sampling and 500/100 epochs, the claim of a 15% improvement over baselines needs repeated-seed results or a paired significance test to establish that the gap is not within run-to-run variance.","section":"Tables II-III, Figure 7"}],"minor_comments":[{"comment":"Reference [18], cited for UOV, is an indoor depth-completion paper (NTrans-Net) and appears unrelated to unified ordinal vectors; please cite the actual source or add a derivation/definition to make the provenance clear.","section":"Section III-D, reference [18]"},{"comment":"The infoNCE expression in Equation (1) repeats the same exponential terms in the numerator and denominator; please use explicit positive and negative index sets to avoid the apparent redundancy.","section":"Equation (1)"},{"comment":"The first row of Table II is described in the text as training without both objectives and with an L2 loss term, but the table has no row label for that configuration; please make the ablation conditions explicit in the table.","section":"Table II"},{"comment":"Figure 7 is normalized to AIRCHITECT v2 with no absolute latency values or explicit workload names in the caption; report actual numbers so readers can assess the claimed 1.7x improvement.","section":"Figure 7"},{"comment":"The abstract and conclusion use \"foundation models\" while the body focuses on LLMs; align the terminology, and fix the typo \"MAESRO\" in Section III-E.","section":"Abstract and Section VI"}],"recommendation":"major_revision","confidential_remarks":"The main gate for this paper is whether the evaluation can be strengthened within its scope: adding a ConfuciuX or exhaustive baseline to Section IV-C, defining the accuracy metric, and clarifying the validation procedure for K. These are fixable with additional experiments rather than new conceptual machinery. The UOV citation issue is worth checking editorially, as the cited paper appears unrelated to the proposed representation."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"The headline result is real but narrower than advertised: AIRCHITECT v2 learns to reproduce ConfuciuX's design choices with 91% agreement on a held-out split, and it beats the other learned baselines (v1, GANDSE, VAESA+BO) by a solid margin. What's not shown is that one-shot inference beats iterative search. ConfuciuX, the thing this method is meant to replace, never appears in the deployment comparison. So the 1.7x latency claim is against other learned methods, not against the search-based status quo.\n\nWhat the paper does well: the dataset and code are public, the ablations are clear, and UOV is shown to reduce model size while improving accuracy across two architectures. The contrastive embedding also helps a downstream BO search converge faster than a VAE embedding. That is a useful engineering result.\n\nSoft spots, in order of importance. First, the labels are ConfuciuX's outputs, and the paper never asks how close ConfuciuX is to the true optimum. Accuracy numbers therefore measure fidelity to ConfuciuX. A direct comparison to ConfuciuX, or exhaustive enumeration over the 768-output configs, would settle this. Second, the accuracy metric is never defined. Is a prediction correct only on exact match, or does a near-miss within the same bucket count? Third, there are no error bars or significance tests; we only see point estimates from what appears to be a single run. Fourth, K=16 is selected using the test-set accuracy curve in Figure 8(b), which is a form of test-set leakage unless there is a validation split. The paper is not specific about this.\n\nThese are fixable in revision. The architecture and training pipeline are reasonable, and the released artifacts make this independently checkable. As it stands, the paper deserves a serious referee, but the central claim needs to be tested against the search method it claims to replace.","headline":"A solid ML-for-DSE engineering contribution that learns to mimic ConfuciuX, but the paper never compares against ConfuciuX itself, so the 'replace iterative search' claim is not yet tested.","tokens_in":12267,"tokens_out":2550,"would_cite":false,"duration_ms":24706,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"AIrchitect v2 claims that optimal accelerator configurations can be predicted in one pass by a contrastively trained transformer with unified ordinal outputs, reaching 91.17% accuracy on a 100K-sample MAESTRO dataset.","keywords":["design space exploration","DNN accelerator","transformer","contrastive learning","unified ordinal vectors","one-shot prediction","MAESTRO","hardware resource allocation"],"falsifier":"Sample a set of GEMM workloads, run AIrchitect v2, then run an exhaustive or large Monte Carlo search over the full MAESTRO resource configuration space for the same workloads; if an alternative configuration reliably achieves lower simulated latency than the predicted one, the claim that the model identifies optimal design points is not supported.","tokens_in":11308,"feed_emoji":"⚙️","tokens_out":6177,"duration_ms":55669,"temperature":0.7,"pith_summary":"AIrchitect v2 aims to turn hardware design space exploration (DSE) for DNN accelerators from an iterative, search-based process into a single learned inference. The paper claims that a transformer encoder-decoder, trained with contrastive learning and a new output representation called Unified Ordinal Vectors, can take a workload description and directly predict the optimal number of processing elements and buffer size. On a 100,000-sample dataset generated for a MAESTRO-modeled accelerator, the model reaches 91.17% accuracy in matching the optimal configuration, about 15% better than prior learning-based methods. The same model, applied to DNN and LLM workloads not seen in training, yields hardware choices with roughly 1.7x lower simulated inference latency than the baselines. A sympathetic reader would care because this suggests the expensive search phase of accelerator design can be replaced by a constant-time predictor.","feed_headline":"One-shot model picks optimal chip designs with 91% accuracy","feed_subtitle":"Transformer plus contrastive learning replaces iterative search and cuts latency 1.7x on unseen LLM workloads.","key_machinery":"The load-bearing machinery is the Unified Ordinal Vector (UOV), an output encoding that represents a hardware configuration as a vector of bucket probabilities: the design space is discretized into K ordered buckets, and a value in bucket r_n is encoded with monotonically increasing entries up to that bucket and zeros after it, so the representation carries both which bucket the design falls in (classification) and where inside the bucket it lies (regression). The encoder is trained with a contrastive loss plus a latency-prediction loss to produce a uniform, smooth embedding space resistant to the dataset's long-tailed label distribution, and the decoder is trained with a unification loss to turn embedding points into UOV predictions. This combination is what lets the model make fine-grained one-shot predictions over a design space of roughly $10^{9}$ configurations.","core_discovery":"The paper's central claim is that the hardware accelerator design space can be learned end-to-end well enough that optimal resource allocation becomes a one-shot prediction problem. For each layer of a workload, the model receives GEMM tensor dimensions (M, N, K) and a dataflow choice, and outputs the recommended number of processing elements and L2 buffer size. The authors report that this approach achieves 91.17% prediction accuracy on the MAESTRO-based dataset, outperforming AIRCHITECT v1 (77.60%) and GANDSE (84.39%), and that the recommended hardware gives a 1.7x improvement in simulated inference latency on unseen DNN and LLM workloads relative to the baseline techniques. The contribution is presented as a combination of three components: contrastive pretraining that smooths the non-uniform and long-tailed performance landscape, an encoder-decoder transformer that maps inputs into a uniform embedding space, and Unified Ordinal Vectors that blend classification and regression so the model can predict fine-grained configurations without an unmanageable output space.","pith_inferences":["The paper's evaluation measures accuracy against ConfuciuX-generated labels, so the 91.17% number is best read as fidelity to that search procedure; whether it equals true optimality is not established by the reported experiments.","If contrastive embeddings generalize the way the paper suggests, the same encoder-decoder recipe could apply to other discrete-continuous co-design problems, such as mapping selection, memory hierarchy sizing, or compiler autotuning, though the paper only demonstrates the MAESTRO resource-allocation task.","The 1.7x latency gain on unseen LLMs is computed with the same simulator used to generate training labels; an independent cost model or real silicon measurement would be a stronger test of practical value.","The deployment pipeline selects one per-layer recommended configuration for a whole model; a natural extension would be to learn a joint model-level recommendation instead of aggregating per-layer choices, which the paper leaves open."],"forward_implications":["Hardware resource allocation for a given workload and mapping can be done in a single forward pass, removing the need for iterative sampling during design space exploration.","Contrastive pretraining makes the learned embedding robust to non-uniform, non-convex performance landscapes and long-tailed data, which are the properties that hurt earlier supervised DSE models.","Because UOV buckets scale sublinearly relative to classification labels, the approach can extend to larger accelerator design spaces without proportional growth in model size.","The recommended configurations transfer to unseen models, including LLMs, with about 1.7x lower simulated latency than the compared baselines.","UOV is not tied to AIrchitect v2; the paper shows it also improves accuracy and reduces model size when applied to AIrchitect v1."],"supporting_citations":[{"why":"Generates the dataset labels by running ConfuciuX reinforcement-learning search with latency as the reward, so it defines the ground-truth optimal configurations the model is trained and evaluated against.","marker":"[12]"},{"why":"Models the accelerator and estimates latency, providing the design space and the evaluation metric used throughout the paper.","marker":"[19]"},{"why":"AIrchitect v1 is the prior classification-based one-shot DSE method that this work extends and the main accuracy baseline.","marker":"[5]"},{"why":"GANDSE is the generative-adversarial baseline that AIrchitect v2 is compared against.","marker":"[16]"},{"why":"VAESA plus Bayesian optimization is the latent-space baseline and provides the embedding-space comparison for the contrastive learning ablation.","marker":"[11]"},{"why":"Supplies the Unified Ordinal Vector representation that the decoder heads are built on.","marker":"[18]"},{"why":"Provides the generalized focal loss formulation adapted as the unification loss for training the UOV heads.","marker":"[31]"},{"why":"Supplies the encoder-decoder transformer architecture that backs the model.","marker":"[28]"},{"why":"Provides the infoNCE-style contrastive loss used for stage 1 embedding training.","marker":"[21]"}],"fun_headline_variants":["One-shot chip design: transformer beats search by 15%","AIrchitect v2: learn hardware design space, skip iterative search","Contrastive transformer for hardware design: 91% accurate, 1.7x latency","Unified representations make chip design a single forward pass","From search to prediction: AIrchitect v2's 15% edge"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The dataset labels are generated by ConfuciuX, a reinforcement-learning search, and are treated as the true optimal hardware configurations; if that search often returns locally optimal designs, the reported accuracy measures agreement with those labels rather than genuine design quality.","fun_headline_variants_meta":{"raw":{"variants":["One-shot chip design: transformer beats search by 15%","AIrchitect v2: learn hardware design space, skip iterative search","Contrastive transformer for hardware design: 91% accurate, 1.7x latency","Unified representations make chip design a single forward pass","From search to prediction: AIrchitect v2's 15% edge"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000261,"raw_usage":{"total_tokens":1660,"prompt_tokens":1077,"completion_tokens":583,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":693,"completion_tokens_details":{"reasoning_tokens":485}},"tokens_in":693,"tokens_out":583,"duration_ms":6128,"temperature":1.0,"reasoning_tokens":485,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-10T19:29:33.038371+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Sample a set of GEMM workloads, run AIrchitect v2, then run an exhaustive or large Monte Carlo search over the full MAESTRO resource configuration space for the same workloads; if an alternative configuration reliably achieves lower simulated latency than the predicted one, the claim that the model identifies optimal design points is not supported.","supporting_citations":[{"cited_title":"MAESTRO: A data-centric approach to understand reuse, performance, and hardware cost of DNN mappings,","cited_arxiv_id":null,"evidence_quote":"Models the accelerator and estimates latency, providing the design space and the evaluation metric used throughout the paper."},{"cited_title":"Gandse: Generative adversarial network-based design space exploration for neural network accelerator design,","cited_arxiv_id":null,"evidence_quote":"GANDSE is the generative-adversarial baseline that AIrchitect v2 is compared against."},{"cited_title":"Learn- ing a continuous and reconstructible latent space for hardware accelerator design,","cited_arxiv_id":null,"evidence_quote":"VAESA plus Bayesian optimization is the latent-space baseline and provides the embedding-space comparison for the contrastive learning ablation."},{"cited_title":"Ntrans- net: A multi-scale neutrosophic-uncertainty guided transformer network for indoor depth completion,","cited_arxiv_id":null,"evidence_quote":"Supplies the Unified Ordinal Vector representation that the decoder heads are built on."},{"cited_title":"Contrastive quant: quantization makes stronger contrastive learning,","cited_arxiv_id":null,"evidence_quote":"Provides the infoNCE-style contrastive loss used for stage 1 embedding training."}],"review_version":1}