HiFi-LLP: High-Fidelity, Low-Cost Latency Predictors with Confidence for Robust HW-NAS

Alexander Frickenstein; Behzad Shomali; Judeson Anthony Fernando; Lukas Frickenstein; Manoj Rohit Vemparala; Moritz Thoma; Nael Fasfous; Pierpaolo Mori; Shambhavi Balamuthu Sampath; Walter Stechele

REVIEW 2 major objections 4 minor 12 references

HiFi-LLP: High-Fidelity, Low-Cost Latency Predictors with Confidence for Robust HW-NAS

T0 review · 2 major / 4 minor · reviewed 2026-07-14 · grok-4.5

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read A latency predictor trained on only 100 samples, with built-in confidence, speeds hardware-aware NAS by up to 8.6 imes while keeping competitive Pareto fronts.

desk verdict Solid engineering paper: GATv2+GP latency predictor with real confidence, 100-sample gains on LatBench, and a hybrid NAS loop that actually works. read the letter →

arxiv 2607.11746 v1 pith:Q4SH7PS7 submitted 2026-07-13 cs.LG cs.AR

Shambhavi Balamuthu Sampath , Behzad Shomali , Nael Fasfous , Moritz Thoma , Judeson Anthony Fernando , Lukas Frickenstein , Pierpaolo Mori , Manoj Rohit Vemparala

show 2 more authors

Alexander Frickenstein Walter Stechele

This is my paper · ORCID

classification cs.LGcs.AR

keywords hardware-awareNASlatencypredictiongraphattentionnetworksGaussianprocessconfidence-awaresearchedgedevicessample-efficientpredictors

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Hardware-aware neural architecture search needs real latency numbers from the target chip, but measuring every candidate on-device is slow and sequential. Prior predictors either demand tens of thousands of measurements or give answers that can mislead the search. This paper shows that a graph-attention network topped with a Gaussian-process head can be trained from scratch on just 100 measured architectures and still produce both a latency estimate and a confidence score. The confidence score lets a hybrid search keep high-confidence predictions and route only the uncertain ones back to real hardware. On six edge and desktop platforms the resulting predictor improves the fraction of predictions within 10 % of true latency by as much as nine percentage points and reaches Spearman rank correlations up to 0.996; the hybrid loop finishes up to 8.6 times faster than pure hardware-in-the-loop search while recovering a Pareto front nearly identical to the oracle.

What carries the argument

HiFi-LLP: an 11-layer GATv2 network whose graph embeddings feed two Gaussian-process heads (one for absolute latency, one for pairwise ranking). The GP variance becomes the confidence score that decides whether a candidate must be measured on hardware.

What would settle it

Retrain HiFi-LLP on 100 fresh random samples from any LatBench device, then check whether the 10 % accuracy bound still exceeds the prior GraphSAGE baseline by several points and whether the hybrid NAS with the fixed 90th-percentile threshold still recovers a Pareto front within a few accuracy points of the pure hardware-in-the-loop front.

Watch

Extended reading notes

Core claim

A platform-specific latency predictor built from graph attention layers and a Gaussian-process regression head can be trained from scratch on only 100 measured architectures, supply both a mean latency and a usable uncertainty estimate, and thereby enable a hybrid NAS loop that replaces most hardware measurements with confident predictions while preserving search quality.

Load-bearing premise

The device-specific latency scalers found by a one-time grid search, and a single uncertainty threshold taken from one desktop CPU, remain valid across all six evaluated platforms without further retuning that itself needs extra measurements.

Editorial extensions

If this is right

Hardware-aware NAS can be run with roughly an order-of-magnitude fewer real device measurements while still returning architectures that meet latency constraints.
New or updated compilers and chips need only a few hundred fresh measurements rather than tens of thousands to obtain a usable platform-specific predictor.
Bayesian-optimization sampling guided by the same confidence scores further reduces the data needed when a predictor must be adapted to a previously unseen device.
Any search algorithm that can query a black-box latency oracle can replace most of those queries with HiFi-LLP predictions and fall back only on low-confidence cases.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

The same confidence-gated hybrid loop could be applied to other expensive hardware metrics (energy, memory bandwidth) once analogous low-data predictors exist.
If the uncertainty estimates remain calibrated under distribution shift, the method offers a practical way to keep NAS continuous as hardware and tool-chains evolve.
The pairwise ranking head may be reusable as a cheap surrogate inside multi-objective evolutionary search even when absolute latency numbers are not required.

Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

The useful part of this paper is a platform-specific latency predictor that is both sample-efficient and uncertainty-aware, plus a hybrid NAS loop that routes low-confidence predictions back to hardware. That combination is practical and the numbers back it up.

What is new is the concrete architecture (GATv2 backbone, residual FCs, dual GP heads for latency and pairwise rank) trained from scratch on 100 samples, the multi-seed gains over HiFi-SAGE and BRP-NAS on LatBench (up to ~9 p.p. on the 10% bound, Spearman up to 0.996), and the explicit confidence gate that yields up to 8.6× speedup while staying close to the HIL Pareto front. The ablations cleanly isolate the scaler, the GP head, and the switch from GraphSAGE to GATv2. The BO sampler comparison in the low-data regime is also clean and favors their approach. Tables report means and standard deviations; variance is markedly lower than the baselines. No circular math—everything is measured against held-out LatBench latencies.

Soft spots are real but engineering-scale. Device-specific latency scalers are grid-searched; the hybrid threshold is the 90th-percentile STD on DCPU and is not re-tuned per device. Early-stopping patience hurts MGPU under the fixed protocol they report. Code is not released. None of these break the central claim: the hybrid still falls back to HIL when uncertain, so the free parameters cannot silently invent a good architecture. The literature positioning is fair; they correctly treat HELP/MultiPredict as quasi-generalized and focus on the platform-specific low-data niche.

This is for people who actually retarget models to edge silicon and need a predictor they can retrain after a compiler or board change. It is not a theory paper. I would send it to peer review; the experiments are careful enough to deserve referee time. Worth reading if you work on HW-NAS or latency modeling; I would cite the hybrid loop and the 100-sample numbers.

Referee Report

2 major / 4 minor

Summary. The paper introduces HiFi-LLP, a platform-specific DNN latency predictor that models architectures as graphs, uses a GATv2 backbone with residual connections and LCM readout, and attaches Gaussian-process regression heads for both absolute latency and pairwise ranking. Trained from scratch on only 100 (or 500) LatBench samples, it reports up to ~9 p.p. gains on the 10 % accuracy bound and Spearman ρ up to 0.996 across six devices, with markedly lower seed-to-seed variance than HiFi-SAGE or BRP-NAS. A confidence-gated hybrid NAS routes predictions whose GP standard deviation exceeds a fixed threshold (1.1778) to real hardware-in-the-loop measurements, yielding up to 8.6 imes wall-clock speedup while preserving a competitive Pareto front. An ablation isolates the contributions of device-specific latency scaling, the GP head and the GATv2 backbone; a separate experiment shows Bayesian-optimization sampling improves low-data adaptation over random and several SOTA samplers.

Significance. If the reported numbers hold, the work supplies a practical, low-data alternative to both exhaustive HIL loops and large-sample or meta-trained predictors. The explicit confidence metric and the hybrid routing scheme are genuine engineering advances that address the reliability gap left by earlier GNN predictors. Multi-seed tables, an ablation of design choices, Oracle-NAS and hybrid-NAS experiments, and a BO-sampler comparison constitute a solid empirical package for a systems-oriented ML venue. The free parameters (device scalers, uncertainty threshold) are openly reported and the hybrid still falls back to HIL, so residual risks are mainly engineering rather than foundational.

major comments (2)

Section IV-B and the hybrid-NAS experiments (Tables V–VI) rely on a single, DCPU-derived 90th-percentile STD threshold (1.1778) that is never re-tuned or validated on the other five LatBench devices. Because the hybrid claim of “up to 8.6 imes speedup while preserving a competitive Pareto front” is demonstrated only under this fixed threshold, the manuscript should either (a) report the same hybrid protocol with device-specific or cross-validated thresholds, or (b) quantify how sensitive the speedup/Pareto results are to modest changes in τ. Without that evidence the transferability of the confidence gate remains an untested free parameter.
Table III shows that on MGPU with 100 samples HiFi-LLP under-performs HiFi-SAGE (48.95 % vs 61.77 % on the 10 % bound). The authors attribute this to early-stopping patience and note that raising patience recovers 68 %, yet they retain the original hyper-parameters “for consistency.” Given that the abstract and introduction advertise consistent out-performance “across six devices,” either the MGPU result should be re-run under a fair early-stopping regime or the claim should be qualified to the five devices where the advantage is unambiguous.

minor comments (4)

Figure 2 caption and surrounding text do not state whether the ablation curves are means over seeds or single runs; adding error bars or a short note would strengthen the design-decision narrative.
Table II lists “AE evolution cycles 1000 / population 256” but the hybrid-NAS experiments in Table VI report far fewer evaluated models; a sentence clarifying the search budget actually used would avoid confusion.
The device-specific scalers (DCPU:400, …) are given only in prose; a small table or appendix entry would make them easier to reproduce.
A few typographical inconsistencies appear (e.g., “HiFi-SAGE improves on BRP-NAS” vs later “HiFi-SAGE [7]”; “Ourscertain” missing a space). A light copy-edit pass would clean them up.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: all accuracy, fidelity, and hybrid-NAS claims are evaluated against held-out LatBench HIL measurements; GP variance and device scalers are free parameters, not quantities defined by the target metrics.

full rationale

HiFi-LLP is a GATv2 + Gaussian-process regression model trained on measured latencies. Its reported 10 % accuracy bounds, Spearman ρ values, Oracle-NAS missed-accuracy/FP/FN counts, and hybrid-NAS speed-ups are all computed by comparing model outputs to independent hardware-in-the-loop ground truth on LatBench. The GP head produces a mean and variance by standard Bayesian regression; the variance is used only as a filter threshold (90th-percentile STD = 1.1778 on DCPU) that routes uncertain predictions back to real HIL measurements. Device-specific latency scalers are chosen by a cheap grid search and openly reported; they do not redefine the evaluation metrics. No equation equates a claimed prediction to a fitted constant by construction, no uniqueness theorem is imported from prior author work, and the hybrid framework remains falsifiable because low-confidence cases fall back to actual hardware. The derivation chain is therefore self-contained against external benchmarks.

Assumptions & free parameters 4 free parameters · 3 assumptions · 2 invented entities

The work is empirical; its claims rest on standard GNN and GP machinery plus a handful of device-specific free parameters chosen by grid search or percentile heuristics. No new physical entities or unproved mathematical axioms are introduced.

free parameters (4)

device-specific latency scaler = DCPU:400, DGPU:250, MDSP:800, EGPU:100, ETPU:750, MGPU:100
Chosen by exhaustive search over [50,2000] for each of the six devices so that the GP head trains stably; values (DCPU:400, DGPU:250, …) are reported but remain free parameters of the method.
uncertainty threshold τ (STD > 1.1778) = 1.1778
Set to the 90th percentile of predicted standard deviations on the DCPU training set and then reused for hybrid NAS; directly controls how many candidates fall back to HIL.
number of GP inducing points = 500
Fixed at 500 (learned) for all experiments; affects both accuracy and compute of the probabilistic head.
GATv2 depth / hidden size / LCM readout dim = 11 / 96 / 128
11 layers, hidden 96, aggregation 128—selected after ablation starting from HiFi-SAGE; treated as fixed architectural hyper-parameters.

assumptions (3)

domain assumption Graph attention (GATv2) message-passing plus LCM readout yields a sufficiently expressive embedding of DNN computational graphs for latency regression.
Invoked throughout Section IV; justified only by empirical ablation against GraphSAGE, not by a theoretical guarantee.
domain assumption Matérn-5/2 GP with learned inducing points supplies a well-calibrated uncertainty estimate usable as a routing signal.
Section III-B and IV; standard GP practice, but calibration is never formally verified beyond the hybrid-NAS experiments.
domain assumption LatBench latency measurements are noise-free ground truth for the six target platforms.
All accuracy and rank metrics treat LatBench numbers as exact; measurement variance is never modeled.

invented entities (2)

HiFi-LLP architecture (GATv2 backbone + dual GP heads)
purpose: End-to-end latency and pairwise-rank prediction with explicit confidence.
New combination of existing components; no independent theoretical existence outside the paper’s experiments.
confidence-gated hybrid HW-NAS loop
purpose: Route only low-confidence predictions to real HIL measurements.
Engineering construct defined by the threshold τ; its value is demonstrated only inside the paper’s own NAS runs.

how reviews work

0 comments

Cite this review

Pith. "Pith review of HiFi-LLP: High-Fidelity, Low-Cost Latency Predictors with Confidence for Robust HW-NAS." pith.science (2026). https://pith.science/paper/Q4SH7PS7

@misc{pith2026260711746,
  author       = {Pith},
  title        = {Pith review of: HiFi-LLP: High-Fidelity, Low-Cost Latency Predictors with Confidence for Robust HW-NAS},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/Q4SH7PS7}},
  note         = {Machine review of arXiv:2607.11746}
}

abstract

With deep neural networks (DNNs) increasingly deployed on edge devices, hardware (HW)-aware optimization techniques--such as HW-aware compression and HW-aware neural architecture search (HW-NAS)--have become essential. These methods rely on real feedback from the target hardware to tailor DNN architectures for efficient deployment. While the search can be parallelized, latency measurements via hardware-in-the-loop (HIL) remain a bottleneck due to their sequential nature. Recent approaches use latency predictors to replace costly HIL feedback, but challenges persist: (1) platform-specific predictors often require tens of thousands of samples, and (2) inaccurate predictions can mislead the NAS process. To address this, we introduce HiFi-LLP, a high-fidelity, low-cost latency predictor based on graph attention networks, augmented with a confidence metric. HiFi-LLP outperforms prior platform-specific predictors by up to 9 percentage points (p.p.) in the 10% accuracy bound and achieves a Spearman's rank correlation of up to 0.996 across six devices in the LatBench dataset. We further propose a hybrid NAS framework that routes low-confidence predictions to HIL, achieving up to 8.6$\times$ speedup compared to typical NAS while maintaining a competitive Pareto front.

Figures

Figures reproduced from arXiv: 2607.11746 by the authors.

**Figure 1.** Architecture of HiFi-LLP. A. Evaluation Metrics a) Accuracy Bounds: They represent the proportion of predictions with errors within thresholds (e.g., 10%) of actual values. For n samples, let At be the actual value for observation t, dt the absolute difference between actual and predicted values, and 1[·] an indicator function returning 1 if the condition is true, 0 otherwise. The error bound for an α% threshold is:… view at source ↗

**Figure 2.** Ablation study of design decisions [PITH_FULL_IMAGE:figures/full_fig_p003_2.png] view at source ↗

**Figure 3.** Our proposed hybrid NAS pipeline. Hyperparameter Value Hyperparameter Value Hyperparameter Value LR 1e−3 GNN type GATv2 GP kernel Matérn (ν=5/2) LR schedule Reduce on plateau GNN aggregator LCM GP inducing points 500 (learned) Optimizer Adam Aggregation size 128 AE evolution cycles 1000 Batch size 8 GNN layers 11 AE population size 256 Training epochs 300 GNN hidden size 96 AE samples size 128 Early stopping ✓(patie… view at source ↗

Figures from the paper (3 more)

**Figure 4.** Figure 4: Latency distribution for devices in LatBench. [PITH_FULL_IMAGE:figures/full_fig_p004_4.png]

**Figure 6.** Figure 6: Pareto front of Ours, BRP-NAS and HIL experiment. [PITH_FULL_IMAGE:figures/full_fig_p005_6.png]

**Figure 7.** Figure 7: Qualitative comparison of various samplers. [PITH_FULL_IMAGE:figures/full_fig_p006_7.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

12 extracted references · 4 linked inside Pith

[1]

On latency predictors for neural architecture search,

Y . Akhauri and M. Abdelfattah, “On latency predictors for neural architecture search,”Proceedings of Machine Learning and Systems, vol. 6, pp. 512–523, 2024

2024
[2]

Transfer nas with meta- learned bayesian surrogates,

G. Shala, T. Elsken, F. Hutter, and J. Grabocka, “Transfer nas with meta- learned bayesian surrogates,” inThe Eleventh International Conference on Learning Representations, 2023

2023
[3]

Multi-predict: few shot predictors for efficient neural architecture search,

Y . Akhauri and M. S. Abdelfattah, “Multi-predict: few shot predictors for efficient neural architecture search,”arXiv preprint arXiv:2306.02459, 2023

arXiv 2023
[4]

Help: Hardware-adaptive efficient latency prediction for nas via meta-learning,

H. Lee, S. Lee, S. Chong, and S. J. Hwang, “Help: Hardware-adaptive efficient latency prediction for nas via meta-learning,”arXiv preprint arXiv:2106.08630, 2021

arXiv 2021
[5]

Brp-nas: Prediction-based nas using gcns,

L. Dudziak, T. Chau, M. Abdelfattah, R. Lee, H. Kim, and N. Lane, “Brp-nas: Prediction-based nas using gcns,”Advances in neural infor- mation processing systems, vol. 33, pp. 10 480–10 490, 2020

2020
[6]

Nn-meter: Towards accurate latency prediction of deep-learning model inference on diverse edge devices,

L. L. Zhang, S. Han, J. Wei, N. Zheng, T. Cao, Y . Yang, and Y . Liu, “Nn-meter: Towards accurate latency prediction of deep-learning model inference on diverse edge devices,” inProceedings of the 19th Annual International Conference on Mobile Systems, Applications, and Services, 2021, pp. 81–93

2021
[7]

Hifi-sage: High fidelity graphsage- based latency estimators for dnn optimization,

S. Balamuthu Sampath, L. Hecht, M. Thoma, L. Frickenstein, P. Mori, N. Fasfous, M. Rohit, A. Frickenstein, W. Stechele, D. Mueller- Gritschneder, and C. Passerone, “Hifi-sage: High fidelity graphsage- based latency estimators for dnn optimization,” 03 2025, pp. 1–7

2025
[8]

How attentive are graph attention networks?

S. Brody, U. Alon, and E. Yahav, “How attentive are graph attention networks?”arXiv preprint arXiv:2105.14491, 2021

arXiv 2021

Show all 12 references

[9]

Learnable commutative monoids for graph neural networks,

E. Ong and P. Veli ˇckovi´c, “Learnable commutative monoids for graph neural networks,” inLearning on Graphs Conference. PMLR, 2022, pp. 43–1

2022
[10]

Nas-bench-201: Extending the scope of repro- ducible neural architecture search,

X. Dong and Y . Yang, “Nas-bench-201: Extending the scope of repro- ducible neural architecture search,”arXiv preprint arXiv:2001.00326, 2020

2001 arXiv
[11]

Regularized evolution for image classifier architecture search,

E. Real, A. Aggarwal, Y . Huang, and Q. V . Le, “Regularized evolution for image classifier architecture search,” inProceedings of the aaai conference on artificial intelligence, vol. 33, no. 01, 2019, pp. 4780– 4789

2019
[12]

Visualizing data using t-sne

L. Van der Maaten and G. Hinton, “Visualizing data using t-sne.”Journal of machine learning research, vol. 9, no. 11, 2008

2008

Pith tools

Reviewed July 14, 2026 · model on record in the stance chip above.

[1] [1]

On latency predictors for neural architecture search,

Y . Akhauri and M. Abdelfattah, “On latency predictors for neural architecture search,”Proceedings of Machine Learning and Systems, vol. 6, pp. 512–523, 2024

2024

[2] [2]

Transfer nas with meta- learned bayesian surrogates,

G. Shala, T. Elsken, F. Hutter, and J. Grabocka, “Transfer nas with meta- learned bayesian surrogates,” inThe Eleventh International Conference on Learning Representations, 2023

2023

[3] [3]

Multi-predict: few shot predictors for efficient neural architecture search,

Y . Akhauri and M. S. Abdelfattah, “Multi-predict: few shot predictors for efficient neural architecture search,”arXiv preprint arXiv:2306.02459, 2023

arXiv 2023

[4] [4]

Help: Hardware-adaptive efficient latency prediction for nas via meta-learning,

H. Lee, S. Lee, S. Chong, and S. J. Hwang, “Help: Hardware-adaptive efficient latency prediction for nas via meta-learning,”arXiv preprint arXiv:2106.08630, 2021

arXiv 2021

[5] [5]

Brp-nas: Prediction-based nas using gcns,

L. Dudziak, T. Chau, M. Abdelfattah, R. Lee, H. Kim, and N. Lane, “Brp-nas: Prediction-based nas using gcns,”Advances in neural infor- mation processing systems, vol. 33, pp. 10 480–10 490, 2020

2020

[6] [6]

Nn-meter: Towards accurate latency prediction of deep-learning model inference on diverse edge devices,

L. L. Zhang, S. Han, J. Wei, N. Zheng, T. Cao, Y . Yang, and Y . Liu, “Nn-meter: Towards accurate latency prediction of deep-learning model inference on diverse edge devices,” inProceedings of the 19th Annual International Conference on Mobile Systems, Applications, and Services, 2021, pp. 81–93

2021

[7] [7]

Hifi-sage: High fidelity graphsage- based latency estimators for dnn optimization,

S. Balamuthu Sampath, L. Hecht, M. Thoma, L. Frickenstein, P. Mori, N. Fasfous, M. Rohit, A. Frickenstein, W. Stechele, D. Mueller- Gritschneder, and C. Passerone, “Hifi-sage: High fidelity graphsage- based latency estimators for dnn optimization,” 03 2025, pp. 1–7

2025

[8] [8]

How attentive are graph attention networks?

S. Brody, U. Alon, and E. Yahav, “How attentive are graph attention networks?”arXiv preprint arXiv:2105.14491, 2021

arXiv 2021