Pith. sign in

REVIEW 3 major objections 7 minor 1 cited by

Edge ML deadline governors fail when they ignore the memory clock and only model CPU and GPU.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · grok-4.5

2026-07-12 13:53 UTC pith:E3RFW7UZ

load-bearing objection Measured on-hardware result: EMC-blind governors miss 25–28% at tight deadlines while a two-cell EMC-aware refit stays under 2% QoS, with energy and two-SKU evidence; the static-EMC repair assumption is the softest part, not the gap itself. the 3 major comments →

arxiv 2606.16106 v3 pith:E3RFW7UZ submitted 2026-06-15 cs.PF cs.ARcs.DC

Edge-Inference Governors Need Memory-Clock State

Jaehoon Kang This is my paper
classification cs.PF cs.ARcs.DC
keywords edge inferenceDVFS governorsmemory clockEMClatency estimationdeadline QoSfrequency-aware modelingJetson Orin
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

On integrated edge chips the memory controller has its own frequency domain, separate from the CPU and GPU. Standard frequency-aware latency models used by deadline governors never see that clock, so they under-predict latency and pick infeasible operating points. A live governor measured on Jetson Orin boards shows the cost: an EMC-blind GPU-only fit misses 25–28 percent of cycles at tight deadlines, while a two-cell refit that conditions on the actual memory frequency holds misses to at most 1.3 percent under a 2 percent QoS budget and selects the lowest energy-feasible clock for periodic vision. The same failure appears for MobileNetV2, a vision transformer, and LLM token decode, replicates on two Orin SKUs, and survives a fused TensorRT engine. The paper therefore argues that the memory clock is a required deployment state, not an optional tuning knob, and that a simple per-lockable-point table already restores feasibility where reactive mean-level calibration cannot.

Core claim

Frequency-aware latency estimators that model only the CPU×GPU plane leave the memory controller clock (EMC) as an unobserved deployment state. On Jetson Orin that omission drives a deadline governor to infeasible clocks: an EMC-blind fit profiled at one memory point and deployed at another misses 25–28 percent of cycles at tight deadlines, whereas an EMC-aware two-cell refit at the deployment EMC holds misses to ≤1.3 percent under a 2 percent QoS budget by selecting the lowest budget-feasible (and, for periodic vision, energy-minimal) GPU clock.

What carries the argument

The per-lockable-point EMC table (equivalently a two-cell refit of T = k/f_gpu + b at the deployment EMC). It supplies the missing state so the governor can locate a budget-feasible operating point; parametric 1/f memory terms and online constant-offset drift calibration cannot repair the memory-clock-dependent slope error.

Load-bearing premise

The memory clock is treated as a fixed pin set by the power profile that can differ between profiling and deployment, so a static table over the four lockable points is a sufficient repair.

What would settle it

Run the identical blind and aware governors on a platform where the stock power profile keeps the memory clock the same in profiling and deployment, or where the memory clock is scaled dynamically under load; if the measured miss-rate gap collapses, the claim that a static missing EMC state is decisive does not hold for that platform.

Watch this falsifier — get emailed when new claim-graph text bears on it.

If this is right

  • Deadline-aware DVFS governors on integrated SoCs must condition latency models on each lockable memory-clock point rather than only the CPU×GPU grid.
  • Aggregate QoS percentages hide weakly-hard risk because deadline misses arrive in long consecutive bursts; continuation probability or ⟨m,k⟩ statistics must be reported alongside miss rate.
  • Energy-optimal feasible points are duty-dependent: lowest feasible clock for periodic vision, highest feasible clock for saturated token decode.
  • Frequency-search procedures that assume latency is non-increasing in clock are unsafe once non-monotonic inversions are possible.
  • Actuation lag (roughly 1/5/8 ms on CPU/GPU/EMC) bounds how quickly any online governor can react; the deployed fix works by choosing the operating point ahead of the run.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • Any integrated SoC whose memory fabric sits on an independent DVFS domain will exhibit an analogous missing-state failure for CPU×GPU-only governors, even outside the NVIDIA Orin family.
  • Reactive online calibrators that only correct a constant bias will keep failing on memory-bound workloads whose latency slope itself moves with the memory clock.
  • Admission policies that also probe decode horizon (KV-cache growth) and GPU co-tenancy are a natural next layer once the EMC state is already required for feasibility.
  • Knife-edge distributions near the miss cliff mean that probe variance and headroom become first-class parts of any measured guarantee, not afterthoughts.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

3 major / 7 minor

Summary. The paper argues that on integrated edge SoCs whose memory fabric has an independent DVFS domain (EMC on Jetson Orin), frequency-aware latency estimators used by deadline-aware DVFS governors are incomplete if they condition only on CPU imes GPU clocks. The central evidence is a deployed governor on Orin NX: an EMC-blind GPU-only fit profiled at EMC 3199 MHz and deployed at 2133 MHz misses 25–28% of cycles at tight deadlines (e.g., 28.4±3.4% at D=4.6 ms over three 1,000-cycle repeats), while a two-cell EMC-aware refit holds misses to ≤1.3% under a 2% QoS budget by selecting the lowest budget-feasible GPU clock—the energy-minimal feasible point for periodic vision (calibrated 1 ms-sampled VDD_IN). The failure is shown across MobileNetV2, ViT-Small, and Qwen2.5 decode; EMC median penalties of ~+12–45% and a low-clock collapse replicate on Nano and NX; a scoped non-monotonic inversion, miss clustering under contention, and 1/5/8 ms actuation lags are characterized; TensorRT roughly halves but does not remove the EMC effect. The authors release the harness and scope claims to BPMP-class Orin parts.

Significance. If the result holds, it is a concrete state-abstraction correction for edge-inference governors: CPU×GPU estimators (e.g., FLAME-style T=k/f+b) fold a deployment-deciding variable into a constant b, and that omission produces infeasible operating points under realistic power-profile pins. The work is valuable for systems and real-time edge ML because it closes the loop with on-hardware deployment, three-repeat miss rates, calibrated module-rail energy, six roofline-spanning workloads, two SKUs sharing lockable EMC points, TensorRT robustness, and an explicit pitfall catalog (P1–P6). Releasing the harness and raw cycle data is a clear strength. The paper does not claim the prior art is wrong within its CPU×GPU scope; it shows that deployed deadline control on integrated SoCs needs the memory-clock state. That framing is proportionate and useful.

major comments (3)
  1. §IV–§V-B and Table VI: The measured blind-vs-aware miss gap is solid, but the claim that a static per-lockable-point (or two-cell) table is a sufficient repair rests on EMC being a fixed deployment constant set by nvpmodel (zero transitions observed with CPU/GPU pinned). If production stacks co-scale EMC with demand, or if arbitration differs, a memory-clock-dependent slope error can reappear and online constant-offset drift calibration cannot fix it (Table V). Please either (a) measure under unpinned/dynamic EMC or a second power-management path, or (b) explicitly bound the sufficiency claim to static profile pins and state what re-validation is required when EMC moves online. This does not overturn the central feasibility result but is load-bearing for the “table is enough” prescription.
  2. §IV, Tables II–IV vs. abstract framing: Feasibility is defined as measured miss rate below τ=2% (soft QoS), not hard zero-miss. That is stated, but the paper repeatedly invokes weakly-hard / (m,k) language in §VII and the introduction. Please make the contract with the real-time reader sharper in §IV: report empirical consecutive-miss / continuation statistics for the same deployed blind and aware cells (not only aggregate miss %), so the weakly-hard risk of the blind policy is quantified on the same runs that establish the 25–28% gap.
  3. §XII and §VII: Burst and stall claims are measured against a single IsolBench-style streaming-write adversary. The paper acknowledges this, but the shared-stall signature and continuation probabilities (up to ~0.74) are then used to motivate reporting practice for governors. Either add one alternate contention shape (e.g., GPU co-runner or second inference stream) for the worst MobileNetV2 cell, or move the burst-process prescriptions more clearly into “under this adversary class” so they are not read as platform-universal.
minor comments (7)
  1. Fig. 1 caption and §IV: Clarify that “aware pick need not be the lowest-energy feasible dot” because selection is T+M≤D, not energy minimization; a short sentence in the caption would prevent misreading the bold dots as energy optima.
  2. Table V: State explicitly how many iterations the FLAME-style drift calibrator was allowed to converge and whether the held-out EMC 2133 MHz evaluation includes the convergence window (the text mentions ~100–300 iterations spent on missed deadlines).
  3. §V-A / Fig. 3: The non-replication of the L2-GEMM inversion on NX is important; consider a one-row summary table (SKU × runtime × GPU pin → sign of 2133↔3199) so the scoping is scannable.
  4. §IX: The pre-specified 100 µs stall demotion criterion is good practice; note in the text that raw p95 is reported alongside the probe noise floor so readers do not subtract non-quantile quantities.
  5. Terminology consistency: abstract/body alternate “≤1.3%” (MobileNet) with other workloads’ ≤0.9%/≤0.2%; keep a single primary MobileNet number in the abstract and point to Tables III–IV for the others.
  6. References [1] and [4] are cited as 2025/2026 arXiv preprints; if camera-ready, update status and ensure the FLAME comparison remains accurate against the cited version’s stated CPU×GPU scope.
  7. Reproducibility: Zenodo DOI and GitHub are given; add the exact L4T/jetpack and nvpmodel profile names used for the 15 W / 25 W pins in a one-line “reproduce Table II” recipe.

Circularity Check

0 steps flagged

No significant circularity: central blind-vs-aware miss gaps and energy orderings are measured on-device against external clocks and rails, not forced by the estimator form or self-citation.

full rationale

This is an empirical systems paper whose load-bearing results are hardware measurements, not a derivation that reduces to its inputs. The T(f)=k/f+b form is imported from FLAME as a standard baseline estimator; the paper fits coefficients at one EMC point (or refits two cells at the deployment EMC), uses the resulting (T+M) rule only to select a GPU clock, then deploys that clock on the live device and counts deadline misses and integrates calibrated VDD_IN energy over independent 1 000-cycle (or 300-token) runs (Tables II–IV, Fig. 1–2). The measured miss rates (25–28 % blind vs ≤1.3 % aware) and energy deltas are therefore external observations, not identities of the fitted k,b. Held-out estimator-error tables (V, VII) likewise evaluate predictions against separate latency sweeps; the two-cell refit is presented as a practical repair whose sufficiency is then validated by the same on-wire deployment, not claimed as a first-principles prediction. Online drift calibration is shown not to close the slope gap (Table V), which is an independent negative result rather than a circular rescue. There are no uniqueness theorems, self-citation chains that carry the central claim, ansatzes smuggled via the authors’ prior work, or renamings of known patterns. The paper is self-contained against external hardware benchmarks (lockable EMC points, INA3221 rails, tegrastats traces, Nsight timelines). Score 0 is therefore the correct, proportionate finding.

Axiom & Free-Parameter Ledger

5 free parameters · 6 axioms · 2 invented entities

Load-bearing content is almost entirely empirical measurement on two Orin boards plus standard latency-model form T=k/f+b from prior estimators. Free parameters are policy thresholds and fit coefficients, not hidden physics constants. Domain assumptions are Jetson BPMP/EMC lock behavior and stock nvpmodel pinning. No new physical entities are postulated; 'missing deployment state' and 'per-lockable-point table' are framing and method, not invented mediators.

free parameters (5)
  • QoS miss budget τ
    Deployment feasibility threshold set to 2%; authors note separation is insensitive to exact τ but a stricter budget would demand larger margins on the tightest cells.
  • Latency model coefficients (k, b) per EMC point / workload
    Fitted from GPU-frequency sweeps or two-cell refits; central to blind vs aware policy selection though the miss gap is measured, not assumed from the fit alone.
  • Margin model M = T(r^{-1})
    Hand-chosen safety margin rule in the (point model, margin model) governor pair used for clock selection.
  • GPD exceedance threshold (p99) and profiling window split
    Used for tail-margin heuristic; validated out-of-sample but threshold/window choices are analyst parameters.
  • Deadline grid near workload floor
    Tight deadlines where blind/aware diverge are selected as the regime of interest; looser deadlines hide the gap in the tail.
axioms (6)
  • domain assumption Baseline frequency-aware latency form T(f)=k/f+b with b absorbing 'frequency-independent' overheads including memory transfers (FLAME-style).
    Invoked throughout §I–§II and §IV–§V as the state-of-the-art estimator structure being stress-tested.
  • domain assumption On these boards the stock power profile pins EMC at a fixed lockable rate with no dynamic transitions under the measured workloads (Table VI).
    Justifies treating EMC as a static deployment state repairable by per-point tables rather than continuous co-scaling.
  • domain assumption Only four EMC rates lock in practice on both Orin SKUs: {204, 665.6, 2133, 3199} MHz.
    Bounds profiling cost of the proposed repair; stated from BPMP behavior in §III-A.
  • ad hoc to paper Feasibility means measured miss rate below deployment budget τ (soft QoS), not hard real-time zero-miss.
    Explicit definition in §IV; shapes all 'feasible frontier' claims.
  • domain assumption Module-input VDD_IN rail integration at 1 ms is an adequate energy metric for policy comparison (with compute-rail cross-check).
    §III-D and §XII; orderings reproduce on compute rail except inversion penalty localization.
  • standard math Standard math/statistics for medians, GPD tail fits as quantile heuristics, and geometric independence baselines for burst analysis.
    Used in §VII and Appendix B; authors correctly weaken iid pWCET claims because exceedances cluster.
invented entities (2)
  • EMC as 'missing deployment state' (state-abstraction framing) independent evidence
    purpose: Names the unobserved variable that decides deadline feasibility and energy under CPU×GPU estimators.
    Framing device, not a new physical object; operationalized as the lockable EMC frequency the estimator does not take as input. Independent evidence is the measured latency/energy shifts themselves.
  • Per-lockable-point EMC latency table T=k(f_emc)/f_gpu + b(f_emc, w) independent evidence
    purpose: Minimal repair that restores deployed feasibility without a new control algorithm.
    Methodological construct; sufficiency is empirical on these boards and fails to fully repair the bandwidth-bound proxy without a full sweep.

pith-pipeline@v1.1.0-grok45 · 36508 in / 4357 out tokens · 50387 ms · 2026-07-12T13:53:37.432250+00:00 · methodology

0 comments
read the original abstract

On integrated edge SoCs whose memory fabric is governed independently of the compute clocks, frequency-aware latency estimators let deadline-aware DVFS governors schedule ML inference by modeling latency over CPU and GPU clocks -- but they do not condition on the memory clock (EMC), a deployment state that decides whether a governor meets its deadlines and at what energy. We show this with a deployed, measured governor on Jetson Orin: an EMC-blind GPU-only fit misses 25-28% of cycles at tight deadlines, whereas an EMC-aware two-cell refit holds misses to <=0.9% under a 2% QoS budget -- selecting a budget-feasible operating point proactively, where latency-only reactive calibration cannot repair memory-clock-dependent slope error. Across six models on two Orin SKUs, the core MobileNetV2 and ViT-Small results replicate on both boards; detection and LLM deployments reproduce the failure on the NX. Sustained deployment requires two further state layers: decode horizon, where KV-cache growth erodes tight-deadline feasibility over long responses, and GPU co-tenancy, whose occupancy opens queueing tails. A contract-admission policy composes the three layers from bounded probes and deploys live, including a joint cell decoding 2,000 tokens against an active GPU co-tenant where tenancy-blind admission misses 32% of tokens. Under the reference-probe maximum-guard accounting, every accepted contract is decisively measured-feasible. Fresh-probe stress tests then expose a probe-variance failure mode at knife-edge admissions; a dispersion-banded guard, validated on held-out trials, eliminates the observed probe-induced low-clock selections (held-out aggregate 1.19%); and one bin of headroom yielded observed per-launch compliance across eight fresh launches. A governor's guarantee is thus a measured conservatism ladder -- state repair, probe-variance banding, and headroom for launch realization.

Figures

Figures reproduced from arXiv: 2606.16106 by Jaehoon Kang.

Figure 1
Figure 1. Figure 1: The paper in one figure: a deployed governor’s feasible-energy frontier across three workload classes (NX, calibrated 1 ms-sampled module-rail energy; point colour = measured deadline-miss %, warmer = higher miss; points above the 2% deployment miss budget are outlined in red and infeasible). At a tight deadline the EMC-blind GPU-only fit selects an infeasible clock in every case; the EMC-aware fit selects… view at source ↗
Figure 2
Figure 2. Figure 2: Deployed EMC-aware vs. EMC-blind governor on the NX (Mo [PITH_FULL_IMAGE:figures/full_fig_p006_2.png] view at source ↗
Figure 3
Figure 3. Figure 3: Cross-SKU EMC curves (median latency, normalized to 3199 MHz): [PITH_FULL_IMAGE:figures/full_fig_p008_3.png] view at source ↗
Figure 4
Figure 4. Figure 4: Median latency vs. locked EMC frequency (normalized to 3199 MHz; [PITH_FULL_IMAGE:figures/full_fig_p008_4.png] view at source ↗
Figure 5
Figure 5. Figure 5: Workload-dependence of the EMC effect in the realistic upper range [PITH_FULL_IMAGE:figures/full_fig_p009_5.png] view at source ↗
Figure 6
Figure 6. Figure 6: The L2-resident GEMM on the latency–energy plane (calibrated [PITH_FULL_IMAGE:figures/full_fig_p010_6.png] view at source ↗
Figure 7
Figure 7. Figure 7: Post-hoc deadline-miss curves (100k cycles/cell). Locked-clock [PITH_FULL_IMAGE:figures/full_fig_p011_7.png] view at source ↗
Figure 8
Figure 8. Figure 8: Miss-burst lengths at a p99.9-tight deadline vs. the geometric [PITH_FULL_IMAGE:figures/full_fig_p011_8.png] view at source ↗
Figure 10
Figure 10. Figure 10: Per-kernel GEMM execution times (Nsight Systems, 960 kernels, [PITH_FULL_IMAGE:figures/full_fig_p012_10.png] view at source ↗
Figure 11
Figure 11. Figure 11: Per-domain transition costs. Workload-observed stalls stay below [PITH_FULL_IMAGE:figures/full_fig_p014_11.png] view at source ↗
Figure 12
Figure 12. Figure 12: In-sample illustration of the heaviest cell’s tail: the Gaussian survival [PITH_FULL_IMAGE:figures/full_fig_p019_12.png] view at source ↗
Figure 13
Figure 13. Figure 13: Trace-driven governor simulation (complementary to the measured deployment of §IV, [PITH_FULL_IMAGE:figures/full_fig_p019_13.png] view at source ↗

Review history (2 revisions) →

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score.

  1. StreamingQEC: Streaming Quantum Error Correction in Tightly Integrated Quantum-Classical Systems via Certified Recurrence

    quant-ph 2026-07 conditional novelty 6.0

    A new simulator shows that quantum error correction's classical pipeline can be compressed exactly with a certified recurrence and approximated with a staged-fluid model, and that transfer and decoder bottlenecks shif...

Reference graph

Works this paper leans on

33 extracted references · 9 linked inside Pith · cited by 1 Pith paper

  1. [1]

    Taming asynchronous CPU-GPU coupling for frequency-aware latency estimation on mobile edge,

    J. Chen, J. You, Z. Liu, and Z. Li, “Taming asynchronous CPU-GPU coupling for frequency-aware latency estimation on mobile edge,” 2026, accessed 2026-06-11. [Online]. Available: https://arxiv.org/abs/2604.15357

    Pith/arXiv arXiv 2026

  2. [2]

    DVFS-aware DNN inference on GPUs: Latency modeling and performance analysis,

    Y . Han, Z. Nan, S. Zhou, and Z. Niu, “DVFS-aware DNN inference on GPUs: Latency modeling and performance analysis,” 2025, accessed 2026-06-11. [Online]. Available: https://arxiv.org/abs/2502.06295

    Pith/arXiv arXiv 2025

  3. [3]

    NVIDIA Jetson Linux Developer Guide (Release 36.5): Platform Power and Performance — Jetson Orin Nano Series, Jetson Orin NX Series and Jetson AGX Orin Series,

    NVIDIA Corporation, “NVIDIA Jetson Linux Developer Guide (Release 36.5): Platform Power and Performance — Jetson Orin Nano Series, Jetson Orin NX Series and Jetson AGX Orin Series,” https://docs.nvidia.com/jetson/ archives/r36.5/DeveloperGuide/SD/PlatformPowerAndPerformance/ JetsonOrinNanoSeriesJetsonOrinNxSeriesAndJetsonAgxOrinSeries. 5.4 5.5 5.6 5.7 5.8...

    2026

  4. [4]

    Joint memory frequency and computing frequency scaling for energy-efficient DNN inference,

    Y . Han, Z. Nan, S. Zhou, and Z. Niu, “Joint memory frequency and computing frequency scaling for energy-efficient DNN inference,” 2025, accessed 2026-06-11. [Online]. Available: https://arxiv.org/abs/ 2509.17970

    arXiv 2025

  5. [5]

    zTT: Learning- based DVFS with zero thermal throttling for mobile devices,

    S. Kim, K. Bin, S. Ha, K. Lee, and S. Chong, “zTT: Learning- based DVFS with zero thermal throttling for mobile devices,” inProceedings of the 19th Annual International Conference on Mobile Systems, Applications, and Services (MobiSys ’21). Virtual Event, Wisconsin: ACM, 2021, pp. 41–53. [Online]. Available: https://doi.org/10.1145/3458864.3468161

    doi:10.1145/3458864.3468161 2021

  6. [6]

    Methodology for GPU frequency switching latency measurement,

    D. Velicka, O. Vysocky, and L. Riha, “Methodology for GPU frequency switching latency measurement,” 2025. [Online]. Available: https://arxiv.org/abs/2502.20075

    Pith/arXiv arXiv 2025

  7. [7]

    Evaluation of CPU frequency transition latency,

    A. Mazouz, A. Laurent, B. Pradelle, and W. Jalby, “Evaluation of CPU frequency transition latency,”Computer Science – Research and Development, vol. 29, no. 3–4, pp. 187–195, 2014. [Online]. Available: https://doi.org/10.1007/s00450-013-0240-x

    doi:10.1007/s00450-013-0240-x 2014

  8. [8]

    Evaluating the energy impact of device parameters for DNN inference on edge,

    A. Dutt, S. P. Rachuri, A. Lobo, N. Shaik, A. Gandhi, and Z. Liu, “Evaluating the energy impact of device parameters for DNN inference on edge,” inProceedings of the 14th International Green and Sustainable Computing Conference (IGSC ’23). ACM, 2023, pp. 52–55, accessed 2026-06-11. [Online]. Available: https: //doi.org/10.1145/3634769.3634809

    doi:10.1145/3634769.3634809 2023

  9. [9]

    DeepRT: A soft real time scheduler for computer vision applications on the edge,

    Z. Yang, K. Nahrstedt, H. Guo, and Q. Zhou, “DeepRT: A soft real time scheduler for computer vision applications on the edge,” inProceedings of the 6th ACM/IEEE Symposium on Edge Computing (SEC), 2021, pp. 271–284. [Online]. Available: https://arxiv.org/abs/2105.01803

    Pith/arXiv arXiv 2021

  10. [10]

    DARIS: An oversubscribed spatio- temporal scheduler for real-time DNN inference on GPUs,

    A. F. Babaei and T. Chantem, “DARIS: An oversubscribed spatio- temporal scheduler for real-time DNN inference on GPUs,” in Proceedings of the 62nd ACM/IEEE Design Automation Conference (DAC). IEEE, 2025, pp. 1–7, accessed 2026-06-11. [Online]. Available: https://arxiv.org/abs/2504.08795

    Pith/arXiv arXiv 2025

  11. [11]

    RTGPU: Real-time GPU scheduling of hard deadline parallel tasks with fine-grain utilization,

    A. Zou, J. Li, C. D. Gill, and X. Zhang, “RTGPU: Real-time GPU scheduling of hard deadline parallel tasks with fine-grain utilization,”IEEE Transactions on Parallel and Distributed Systems, vol. 34, no. 5, pp. 1450–1465, May 2023. [Online]. Available: https://ieeexplore.ieee.org/document/10012550

    arXiv 2023

  12. [12]

    Taming non-blocking caches to improve isolation in multicore real-time systems,

    P. K. Valsan, H. Yun, and F. Farshchi, “Taming non-blocking caches to improve isolation in multicore real-time systems,” in2016 IEEE Real- Time and Embedded Technology and Applications Symposium (RTAS), 2016, pp. 161–172, accessed 2026-06-11. Origin of the IsolBench benchmark suite, including the Bandwidth memory-bandwidth adversary benchmark; code at htt...

    arXiv 2016

  13. [13]

    Protecting real-time GPU kernels on integrated CPU-GPU SoC platforms,

    W. Ali and H. Yun, “Protecting real-time GPU kernels on integrated CPU-GPU SoC platforms,” in30th Euromicro Conference on Real-Time Systems (ECRTS 2018), ser. Leibniz International Proceedings in Informatics (LIPIcs), vol. 106. Schloss Dagstuhl– Leibniz-Zentrum f ¨ur Informatik, 2018, pp. 19:1–19:22, accessed 2026- 06-11. [Online]. Available: https://drop...

    doi:10.4230/lipics.ecrts.2018.19 2018

  14. [14]

    Profiling concurrent vision inference workloads on NVIDIA Jetson – extended,

    A. Chakraborty, W. Tavernier, A. Kourtis, M. Pickavet, A. Oikonomakis, and D. Colle, “Profiling concurrent vision inference workloads on NVIDIA Jetson – extended,” 2025, accessed 2026-06-11. [Online]. Available: https://arxiv.org/abs/2508.08430

    Pith/arXiv arXiv 2025

  15. [15]

    MemGuard: Memory bandwidth reservation system for efficient performance isola- tion in multi-core platforms,

    H. Yun, G. Yao, R. Pellizzoni, M. Caccamo, and L. Sha, “MemGuard: Memory bandwidth reservation system for efficient performance isola- tion in multi-core platforms,” inProceedings of the 19th IEEE Real-Time and Embedded Technology and Applications Symposium (RTAS 2013). IEEE, 2013, pp. 55–64

    2013

  16. [16]

    A dynamic priority assignment technique for streams with (m,k)-firm deadlines,

    M. Hamdaoui and P. Ramanathan, “A dynamic priority assignment technique for streams with (m,k)-firm deadlines,”IEEE Transactions on Computers, vol. 44, no. 12, pp. 1443–1451, 1995

    1995

  17. [17]

    Weakly hard real-time systems,

    G. Bernat, A. Burns, and A. Llamos ´ı, “Weakly hard real-time systems,” IEEE Transactions on Computers, vol. 50, no. 4, pp. 308–321, 2001

    2001

  18. [18]

    Control- system stability under consecutive deadline misses constraints,

    M. Maggio, A. Hamann, E. Mayer-John, and D. Ziegenbein, “Control- system stability under consecutive deadline misses constraints,” in Proceedings of the 32nd Euromicro Conference on Real-Time Systems (ECRTS 2020), ser. LIPIcs, vol. 165. Schloss Dagstuhl – Leibniz- Zentrum f ¨ur Informatik, 2020, pp. 21:1–21:24

    2020

  19. [19]

    Statistical analysis of WCET for scheduling,

    S. Edgar and A. Burns, “Statistical analysis of WCET for scheduling,” inProceedings of the 22nd IEEE Real-Time Systems Symposium (RTSS 2001). IEEE Computer Society, 2001, pp. 215–224

    2001

  20. [20]

    Measurement-based probabilistic timing analysis for multi-path pro- grams,

    L. Cucu-Grosjean, L. Santinelli, M. Houston, C. Lo, T. Vardanega, L. Kosmidis, J. Abella, E. Mezzetti, E. Qui ˜nones, and F. J. Cazorla, “Measurement-based probabilistic timing analysis for multi-path pro- grams,” inProceedings of the 24th Euromicro Conference on Real-Time Systems (ECRTS 2012). IEEE Computer Society, 2012, pp. 91–101

    2012

  21. [21]

    A survey of probabilistic timing analysis techniques for real-time systems,

    R. I. Davis and L. Cucu-Grosjean, “A survey of probabilistic timing analysis techniques for real-time systems,”Leibniz Transactions on Embedded Systems (LITES), vol. 6, no. 1, pp. 03:1–03:60, 2019

    2019

  22. [22]

    The tail at scale,

    J. Dean and L. A. Barroso, “The tail at scale,”Communications of the ACM, vol. 56, no. 2, pp. 74–80, Feb. 2013. [Online]. Available: https://doi.org/10.1145/2408776.2408794

    doi:10.1145/2408776.2408794 2013

  23. [23]

    MLPerf inference benchmark,

    V . J. Reddi, C. Cheng, D. Kanter, P. Mattson, G. Schmuelling, C.-J. Wu, B. Anderson, M. Breughe, M. Charlebois, W. Chou, R. Chukka, C. Coleman, S. Davis, P. Deng, G. Diamos, J. Duke, D. Fick, J. S. Gardner, I. Hubara, S. Idgunji, T. B. Jablin, J. Jiao, T. St. John, P. Kanwar, D. Lee, J. Liao, A. Lokhmotov, F. Massa, P. Meng, P. Micikevicius, C. Osborne, ...

    Pith/arXiv arXiv 2020

  24. [24]

    Efficiently scaling transformer inference,

    R. Pope, S. Douglas, A. Chowdhery, J. Devlin, J. Bradbury, J. Heek, K. Xiao, S. Agrawal, and J. Dean, “Efficiently scaling transformer inference,” inProceedings of Machine Learning and Systems, D. Song, M. Carbin, and T. Chen, Eds., vol. 5. Curan, 2023, pp. 606–624. [On- line]. Available: https://proceedings.mlsys.org/paper files/paper/2023/ hash/c4be71ab...

    2023

  25. [25]

    Full stack optimization of transformer inference: a survey,

    S. Kim, C. Hooper, T. Wattanawong, M. Kang, R. Yan, H. Genc, G. Dinh, Q. Huang, K. Keutzer, M. W. Mahoney, Y . S. Shao, and A. Gholami, “Full stack optimization of transformer inference: a survey,” 2023, accessed 2026-06-11. [Online]. Available: https: //arxiv.org/abs/2302.14017

    Pith/arXiv arXiv 2023

  26. [26]

    ONNX runtime,

    Microsoft, “ONNX runtime,” https://github.com/microsoft/onnxruntime, release v1.23.0, CUDA execution provider. Accessed 2026-06-12

    2026

  27. [27]

    MobileNetV2: Inverted residuals and linear bottlenecks,

    M. Sandler, A. G. Howard, M. Zhu, A. Zhmoginov, and L. Chen, “MobileNetV2: Inverted residuals and linear bottlenecks,” in2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). IEEE, 2018, pp. 4510–4520

    2018

  28. [28]

    An image is worth 16x16 words: Transformers for image recognition at scale,

    A. Dosovitskiy, L. Beyer, A. Kolesnikov, D. Weissenborn, X. Zhai, T. Unterthiner, M. Dehghani, M. Minderer, G. Heigold, S. Gelly, J. Uszkoreit, and N. Houlsby, “An image is worth 16x16 words: Transformers for image recognition at scale,” in9th International Conference on Learning Representations (ICLR 2021), 2021. [Online]. Available: https://openreview.n...

    2021

  29. [29]

    Qwen2.5 technical report,

    Qwen Team, “Qwen2.5 technical report,” 2024. [Online]. Available: https://arxiv.org/abs/2412.15115

    Pith/arXiv arXiv 2024

  30. [30]

    llama.cpp: LLM inference in C/C++,

    G. Gerganov and contributors, “llama.cpp: LLM inference in C/C++,” https://github.com/ggml-org/llama.cpp, commit ac4cdde, CUDA back- end, Q4 K M GGUF. Accessed 2026-06-12

    2026

  31. [31]

    Roofline: An insightful visual performance model for multicore architectures,

    S. Williams, A. Waterman, and D. A. Patterson, “Roofline: An insightful visual performance model for multicore architectures,”Communications of the ACM, vol. 52, no. 4, pp. 65–76, 2009

    2009

  32. [32]

    TensorRT developer guide,

    NVIDIA, “TensorRT developer guide,” https://docs.nvidia.com/ deeplearning/tensorrt/, accessed 2026-06-12

    2026

  33. [33]

    Real-time group scheduling,

    The Linux Kernel documentation, “Real-time group scheduling,” documentation/scheduler/sched-rt-group.rst. Accessed 2026-06-12. [On- line]. Available: https://docs.kernel.org/scheduler/sched-rt-group.html

    2026