REVIEW 4 major objections 5 minor 28 references
A Data-Driven Approach to Dataflow-Aware Online Scheduling for Graph Neural Network Inference
T0 review · 4 major / 5 minor · reviewed 2026-08-12 · deepseek-v4-flash
Pith's one-line read Learned latency models pick the fastest GNN dataflow 91% of the time
desk verdict A well-integrated simulation study of learned dataflow selection for GNN accelerators, but the missing hardware validation makes the headline numbers proof-of-concept rather than findings. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The central object is a latency predictor mapping a graph, an intra-phase dataflow, and an inter-phase dataflow to an execution latency estimate, implemented as a set of 24 LightGBM gradient-boosted regression models, one per configuration, trained on cycles simulated by STONNE-$\Omega$. The input features combine ordinary graph statistics (node count, edge count, density, clustering coefficient, degree quantiles) with custom cycle-estimate features for the aggregation and combination phases, and the target latency is log-transformed before training. The scheduling machinery is a shortest-job-first dispatcher that uses the predicted latency as the job-length estimate, deciding both which accelerator in a heterogeneous multi-accelerator system receives a graph and which tiling configuration that accelerator should use.
What would settle it
Run the trained 24 regressors against latency measurements on a real GNN accelerator (FPGA or ASIC) for graphs drawn from the same test sets; if the top-1 dataflow selection accuracy drops far below the reported 91% (or below random choice for some datasets) or the MAPE climbs far above 3.78%, the simulator-to-hardware transferability premise fails. A cheaper version: compare STONNE-$\Omega$'s cycle counts to cycle-accurate RTL simulation on a handful of representative graphs.
Extended reading notes
Core claim
On the paper's own terms: GNN inference latency for a given graph and dataflow is a learnable function of cheap graph statistics. Training one gradient-boosted tree per dataflow configuration on latency labels produced by the STONNE-$\Omega$ simulator over synthetic graphs from Graphlaxy yields regressors whose relative error stays under 5 percent for medium and large in-distribution graphs. Ranking the 24 configurations by predicted latency selects the true optimal dataflow with 91.28 percent top-1 accuracy on the in-distribution medium dataset, and using the same predictions inside a shortest-job-first online scheduler yields up to a 3.17x speedup in mean completion time and 6.26x speedup in mean execution time versus the strongest feasible baseline, closely matching an oracle that knows true job lengths.
Load-bearing premise
The central premise is that STONNE-$\Omega$'s simulated cycle counts accurately represent real GNN accelerator latency; the paper validates predictions only against the simulator, not against silicon, so if the simulator diverges from real hardware, the reported accuracies and speedups describe a modeled system rather than physical accelerators.
Editorial extensions
If this is right
- A single flexible accelerator can choose its dataflow per input graph instead of using a fixed configuration, achieving up to 93.63% latency improvement over a random choice and within 0.56% of the oracle on the medium in-distribution dataset.
- The learned dataflow selection transfers to out-of-distribution real-world graphs with limited degradation, staying under 15% from optimal across all tested datasets and under 5% for half of them, suggesting the mapping is not overfit to synthetic graphs.
- The same predictors enable an online scheduling policy that approximates the performance of an oracle shortest-job-first scheduler with true job lengths, and does so at negligible runtime cost: roughly 2,312 accelerator cycles per prediction versus an average job waiting time of 18,756 cycles.
- Because the predictor's inference cost is constant regardless of graph size, the relative overhead of this approach shrinks as graphs grow, making it progressively more attractive for large GNN workloads.
Reading between the lines
- Our inference: if the simulator-to-hardware gap is small, this result suggests dataflow selection for GNNs could become a per-graph runtime decision in production systems, shifting accelerator design toward more flexible dataflow support instead of fixed, design-time configuration choices.
- Our inference: the latency predictors could be reused for other resource-management decisions beyond scheduling, such as choosing among heterogeneous accelerators in a data center, partitioning graphs across devices, or guiding compiler-style mapping searches for GNN layers.
- Our inference: a testable extension is to replace or augment the degree-quantile features with spectral graph features (for example, eigenvalues of the normalized Laplacian) and measure whether top-1 accuracy improves on the hardest out-of-distribution datasets, Pattern and Enzymes, where the paper reports the lowest selection accuracy.
- Our inference: the reported 1000x turnaround-time speedup is tied to the heavy-load, high-utilization setting built into the experiments, so on lighter workloads the advantage of SJF-Predicted over simpler baselines would shrink; stress-testing across utilization levels would map where the method retains its edge.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper proposes a data-driven framework for predicting GNN inference latency across 24 dataflow configurations on spatial accelerators, using lightGBM regressors trained on STONNE-Omega simulations of synthetic Graphlaxy graphs. It reports top-1 dataflow-selection accuracy up to 91.28% and a MAPE of 3.78% on in-distribution graphs, with substantially worse performance on real-world out-of-distribution datasets. The predictors are then embedded in an SJF-based online scheduler for a heterogeneous multi-accelerator setup, with claimed speedups up to 3.17x in mean completion time and 6.26x in mean execution time over the best feasible baseline.
Significance. If STONNE-Omega's cycle counts faithfully represent hardware behavior, the method is a practical, low-overhead way to choose dataflows and to guide online scheduling for GNN inference. The paper has real strengths: it builds a large synthetic training set, includes an ablation study in Table 4, evaluates on held-out real-world graph datasets, and measures predictor cost with a compiled LightGBM runtime. The main limitation is that all labels, all evaluation numbers, and the scheduling results are internal to a simulator that is not validated against silicon, FPGA, or an independently cycle-accurate model in this manuscript. As a result, the headline accuracy and speedup figures are currently claims about a modeled system. The scheduling comparison also uses favorable baseline settings that conflate dataflow selection with scheduling quality. These issues are fixable and do not undermine the value of the proposed framework as a simulation-based design tool, but the hardware-level conclusions need either additional validation or more careful framing.
major comments (4)
- [Section 3.1] The entire training and evaluation pipeline uses STONNE-Omega cycle counts as ground truth, but the manuscript provides no validation of the simulator against real hardware or an independently validated cycle-accurate model. Because the task is to rank 24 dataflows, even unbiased per-configuration cycle errors can reorder configurations that are close in latency, and the scheduling results inherit the same issue. Please add a validation study on at least a subset of the 24 configurations (e.g., against FPGA/RTL measurements or a published cycle-accurate model) and report per-configuration error distributions and rank-flip rates, or substantially temper the claims so that they are explicitly about the simulator's model.
- [Table 1 and Table 4] Features S1--S6 are hand-derived closed-form cycle estimates from the same loop-level analytical model family that underlies STONNE-Omega, and the ablation in Table 4 shows that adding them reduces Graphlaxy.Medium MAPE from 35.77% to 10.76% (and further to 3.78% with log transformation). This means the learned regressor is to a large extent fitting the simulator's own analytical expressions, which weakens the 'data-driven' interpretation and limits expectations for hardware whose behavior deviates from those expressions. Please discuss this explicitly and, if feasible, report the scheduling results with a model trained without the S1--S6 features to separate the contribution of the analytical priors from the learned part.
- [Section 4.2, Figure 6] In the first scheduling scenario, all feasible baselines (Random, FCFS, LCFS, SJF-Nodes, SJF-Edges) use random tiling selection, while the proposed SJF-Predicted uses predicted tiling. The reported speedups 'against the best feasible baseline' therefore conflate dataflow-selection quality with job-ordering quality. A fairer feasible baseline would combine SJF-Nodes or SJF-Edges with the best fixed tiling per dataset (which can be chosen offline from Table 3). Without such a baseline, the 3.17x and 6.26x figures may overstate the scheduling-only benefit.
- [Abstract and Section 4.2] The abstract and conclusion claim 'more than 1000× speedup in turnaround time', but Section 4.2 reports no raw numeric value for this metric; Figures 6 and 7 are normalized separately by the largest value, so the reader cannot verify the claim. Please provide the absolute values or the computed speedup, or remove the claim.
minor comments (5)
- [Table 1] The formula for S5 uses T_F without a definition; it should be T_Fa (or defined explicitly). The expression for S6, S3 + S5/T_Va, is also dimensionally unclear because S5 already appears to sum over all nodes; please clarify.
- [Section 4.2] The paper says scheduling experiments are run 'over five runs with a fixed seed for all algorithms'; if the same seed is used for all runs, the runs are deterministic repetitions. Please clarify whether multiple seeds were used and report variance across seeds.
- [Figures 6 and 7] Normalizing each metric separately by its largest value makes it difficult to compare absolute differences across metrics and algorithms; consider reporting a table of raw mean values alongside the normalized figures.
- [Section 4.2] The Pareto inter-arrival distribution is not fully specified; reproducibility would benefit from reporting the shape/scale parameters or the exact procedure used to obtain a mean utilization of 85%.
- [Section 2.2] Reference [25] appears twice in the citation list '[19, 25, 25]'; this looks like a typo and should be corrected.
Circularity Check
No significant circularity: the learned predictors are evaluated against held-out STONNE-Omega outputs, and the hand-crafted cycle-count features are inputs, not fitted labels.
full rationale
I walked the paper's derivation chain: Graphlaxy synthetic graphs provide topological variability, STONNE-Omega supplies latency labels for 24 dataflows, per-configuration LightGBM regressors learn the graph/config-to-latency map, and the learned latencies are used both to select the argmax dataflow and to drive SJF-Predicted scheduling. At no point is the target latency defined as a function of the model's own parameters or of the engineered features; the regression objective is explicit (minimize squared error between pi_theta and l_hat), and the ablation in Section 3.2 / Table 4 shows the base model has large error (35.77% MAPE on Graphlaxy.Medium) that only falls to 3.78% when composite features and log-transform are added, which is evidence of genuine learning rather than a tautology. The S1-S6 analytical cycle estimates in Table 1 are domain-knowledge input features, not fitted outputs or renamed labels, and the paper does not claim that STONNE-Omega's cycle model is identical to those formulas. The citations to Garg et al. [5] (STONNE-Omega and the dataflow taxonomy) and Wassington/Abadal [24] (Graphlaxy) do include a current co-author, but no load-bearing conclusion is derived from those citations: the simulator and generator are used as tools, and all reported prediction, ranking, and scheduling metrics are measured on held-out graphs within the same simulation environment. No uniqueness theorem is imported, no ansatz is smuggled in via citation, and no known empirical pattern is merely renamed. The absence of a comparison of STONNE-Omega to real silicon is a correctness/generalization risk, not a circularity step under the specified criteria.
Assumptions & free parameters
free parameters (4)
- lightGBM regressor parameters (tree splits and leaf weights for 24 models) =
not reported
- Pareto inter-arrival distribution parameters =
set to yield 85% mean utilization
- Hand-designed composite features S1-S6 =
formulas in Table 1
- Tiling scheme set (a-h) =
8 schemes from Table 2
assumptions (4)
- domain assumption STONNE-Omega simulation accurately models GNN accelerator latency
- domain assumption Graphlaxy synthetic graphs span the feature space of real graphs
- domain assumption Shortest-job-first minimizes mean completion time in heterogeneous multi-accelerator settings with release times
- domain assumption GCN is representative of GNN inference workloads
Cite this review
Pith. "Pith review of A Data-Driven Approach to Dataflow-Aware Online Scheduling for Graph Neural Network Inference." pith.science (2026). https://pith.science/paper/RP4CM5KT
@misc{pith2026241116342,
author = {Pith},
title = {Pith review of: A Data-Driven Approach to Dataflow-Aware Online Scheduling for Graph Neural Network Inference},
year = {2026},
howpublished = {\url{https://pith.science/paper/RP4CM5KT}},
note = {Machine review of arXiv:2411.16342}
}
abstract
Graph Neural Networks (GNNs) have shown significant promise in various domains, such as recommendation systems, bioinformatics, and network analysis. However, the irregularity of graph data poses unique challenges for efficient computation, leading to the development of specialized GNN accelerator architectures that surpass traditional CPU and GPU performance. Despite this, the structural diversity of input graphs results in varying performance across different GNN accelerators, depending on their dataflows. This variability in performance due to differing dataflows and graph properties remains largely unexplored, limiting the adaptability of GNN accelerators. To address this, we propose a data-driven framework for dataflow-aware latency prediction in GNN inference. Our approach involves training regressors to predict the latency of executing specific graphs on particular dataflows, using simulations on synthetic graphs. Experimental results indicate that our regressors can predict the optimal dataflow for a given graph with up to 91.28% accuracy and a Mean Absolute Percentage Error (MAPE) of 3.78%. Additionally, we introduce an online scheduling algorithm that uses these regressors to enhance scheduling decisions. Our experiments demonstrate that this algorithm achieves up to $3.17\times$ speedup in mean completion time and $6.26\times$ speedup in mean execution time compared to the best feasible baseline across all datasets.
Figures
Figures from the paper (3 more)
Reference graph
Works this paper leans on
-
[5]
Raveesh Garg, Eric Qin, Francisco Muñoz-Matrínez, Robert Guirado, Akshay Jain, Sergi Abadal, José L Abellán, Manuel E Acacio, Eduard Alarcón, Sivasankaran Rajamanickam, et al . 2022. Understanding the design-space of sparse/dense multiphase GNN dataflows on spatial accelerators. In 2022 IEEE International Parallel and Distributed Processing Symposium (IPD...
work page 2022
-
[1]
Sergi Abadal, Akshay Jain, Robert Guirado, Jorge López-Alonso, and Eduard Alarcón. 2021. Computing Graph Neural Networks: A Survey from Algorithms to Accelerators. ACM Comput. Surv. 54, 9, Article 191 (oct 2021), 38 pages. https://doi.org/10.1145/3477141
doi:10.1145/3477141 2021
-
[2]
Francesco G Blanco, Enrico Russo, Maurizio Palesi, Davide Patti, Giuseppe Ascia, and Vincenzo Catania. 2024. Deep Reinforcement Learning based Online Sched- uling Policy for Deep Neural Network Multi-Tenant Multi-Accelerator Systems. In 2024 61th ACM/IEEE Design Automation Conference (DAC) . IEEE
work page 2024
-
[3]
Simon Boehm. [n. d.]. lleaves. https://github.com/siboehm/lleaves
-
[4]
Joshi, Anh Tuan Luu, Thomas Laurent, Yoshua Bengio, and Xavier Bresson
Vijay Prakash Dwivedi, Chaitanya K. Joshi, Anh Tuan Luu, Thomas Laurent, Yoshua Bengio, and Xavier Bresson. 2022. Benchmarking Graph Neural Networks. arXiv:2003.00982 [cs.LG] https://arxiv.org/abs/2003.00982
arXiv 2022
-
[6]
Tong Geng, Ang Li, Runbin Shi, Chunshu Wu, Tianqi Wang, Yanfei Li, Pouya Haghi, Antonino Tumeo, Shuai Che, Steve Reinhardt, et al. 2020. AWB-GCN: A graph convolutional network accelerator with runtime workload rebalancing. In 2020 53rd Annual IEEE/ACM International Symposium on Microarchitecture (MICRO). IEEE, 922–936
2020
-
[7]
Will Hamilton, Zhitao Ying, and Jure Leskovec. 2017. Inductive representation learning on large graphs. Advances in neural information processing systems 30 (2017)
2017
-
[8]
Victor JB Jung, Arne Symons, Linyan Mei, Marian Verhelst, and Luca Benini
Show all 28 references
-
[9]
Guolin Ke, Qi Meng, Thomas Finley, Taifeng Wang, Wei Chen, Weidong Ma, Qiwei Ye, and Tie-Yan Liu. 2017. Lightgbm: A highly efficient gradient boosting decision tree. Advances in neural information processing systems 30 (2017), 3146– 3154
2017
-
[10]
Seah Kim, Hasan Genc, Vadim Vadimovich Nikiforov, Krste Asanović, Borivoje Nikolić, and Yakun Sophia Shao. 2023. MoCA: Memory-centric, adaptive execu- tion for multi-tenant deep neural networks. In2023 IEEE International Symposium on High-Performance Computer Architecture (HPC...
2023
-
[11]
Kipf and Max Welling
Thomas N. Kipf and Max Welling. 2016. Semi-Supervised Classification with Graph Convolutional Networks. CoRR abs/1609.02907 (2016). arXiv:1609.02907 http://arxiv.org/abs/1609.02907
2016 arXiv
-
[12]
Kolodziej, Mohsen Aznaveh, Matthew Bullock, Jarrett David, Timothy A
Scott P. Kolodziej, Mohsen Aznaveh, Matthew Bullock, Jarrett David, Timothy A. Davis, Matthew Henderson, Yifan Hu, and Read Sandstrom. 2019. The SuiteSparse Matrix Collection Website Interface. Journal of Open Source Software 4, 35 (2019),
2019
-
[13]
Jan Karel Lenstra, AHG Rinnooy Kan, and Peter Brucker. 1977. Complexity of machine scheduling problems. In Annals of discrete mathematics. Vol. 1. Elsevier, 343–362
1977
-
[14]
Alexander Lindermayr, Nicole Megow, and Martin Rapp. 2023. Speed-oblivious online scheduling: knowing (precise) speeds is not necessary. In International Conference on Machine Learning . PMLR, 21312–21334
2023
-
[15]
Linyan Mei, Pouya Houshmand, Vikram Jain, Sebastian Giraldo, and Marian Verhelst. 2021. ZigZag: Enlarging joint architecture-mapping design space explo- ration for DNN accelerators. IEEE Trans. Comput. 70, 8 (2021), 1160–1174
2021
-
[16]
Kriege, Franka Bause, Kristian Kersting, Petra Mutzel, and Marion Neumann
Christopher Morris, Nils M. Kriege, Franka Bause, Kristian Kersting, Petra Mutzel, and Marion Neumann. 2020. TUDataset: A collection of benchmark datasets for learning with graphs. In ICML 2020 Workshop on Graph Representation Learning and Beyond (GRL+ 2020) . arXiv:2007.08663...
2020 arXiv
-
[17]
Eneko Osaba and Esther Villar. 2023. QOPTLib: a Quantum Computing Oriented Benchmark for Combinatorial Optimization Problems. https://doi.org/10.17632/ h32z9kcz3s.1
2023
-
[18]
Cynthia Phillips, Clifford Stein, and Joel Wein. 1998. Minimizing average com- pletion time in the presence of release dates. Mathematical Programming 82, 1 (1998), 199–223
1998
-
[19]
Rishov Sarkar, Stefan Abi-Karam, Yuqi He, Lakshmi Sathidevi, and Cong Hao
-
[20]
Linus Schrage. 1968. Letter to the Editor—A Proof of the Optimality of the Shortest Remaining Processing Time Discipline. Operations Research 16, 3 (1968), 687–690. https://doi.org/10.1287/opre.16.3.687
1968 doi
-
[21]
Andreas S Schulz and Martin Skutella. 2002. Scheduling unrelated machines by randomized rounding. SIAM Journal on Discrete Mathematics 15, 4 (2002), 450–469
2002
-
[22]
In 2023 IEEE International Symposium on High- Performance Computer Architecture (HPCA)
FlowGNN: A dataflow architecture for real-time workload-agnostic graph neural network inference. In 2023 IEEE International Symposium on High- Performance Computer Architecture (HPCA). IEEE, 1099–1112
2023
-
[23]
Petar Veličković, Guillem Cucurull, Arantxa Casanova, Adriana Romero, Pietro Liò, and Yoshua Bengio. 2018. Graph Attention Networks. InInternational Confer- ence on Learning Representations . https://openreview.net/forum?id=rJXMpikCZ
2018
-
[24]
Axel Wassington and Sergi Abadal. 2022. Bias reduction via cooperative bar- gaining in synthetic graph dataset generation. arXiv preprint arXiv:2205.13901 (2022)
2022 arXiv
-
[25]
Vivienne Sze, Yu-Hsin Chen, Tien-Ju Yang, and Joel S Emer. 2017. Efficient processing of deep neural networks: A tutorial and survey. Proc. IEEE 105, 12 (2017), 2295–2329
2017
-
[28]
Mingyu Yan, Lei Deng, Xing Hu, Ling Liang, Yujing Feng, Xiaochun Ye, Zhimin Zhang, Dongrui Fan, and Yuan Xie. 2020. Hygcn: A gcn accelerator with hybrid architecture. In 2020 IEEE International Symposium on High Performance Computer Architecture (HPCA). IEEE, 15–29. 7
2020
-
[1244]
https://doi.org/10.21105/joss.01244
-
[2023]
In 2023 IEEE 5th International Conference on Artificial Intelligence Circuits and Systems (AICAS)
SALSA: Simulated Annealing based Loop-Ordering Scheduler for DNN Accelerators. In 2023 IEEE 5th International Conference on Artificial Intelligence Circuits and Systems (AICAS). IEEE, 1–5
2023
Reviewed August 12, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.