Pith. sign in

REVIEW 5 major objections 6 minor 59 references

QPART: Adaptive Model Quantization and Dynamic Workload Balancing for Accuracy-aware Edge Inference

T0 review · 5 major / 6 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read QPART claims that for each edge inference request the server should jointly optimize layer-wise quantization bit widths and a network partition point, solving a closed-form problem that keeps accuracy degradation below 1 percent while…

desk verdict Worthwhile problem, broken closed-form solution: the derived bit-widths don't actually depend on the accuracy budget, so the paper's core guarantee is unsupported. read the letter →

arxiv 2506.23934 v1 pith:YCJA3GXK submitted 2025-06-30 cs.DC cs.AIcs.LGcs.PF

Xiangchen Li , Saeid Ghafouri , Bo Ji , Hans Vandierendonck , Deepu John , Dimitrios S. Nikolopoulos This is my paper
classification cs.DCcs.AIcs.LGcs.PF
keywords adaptiveinferenceservingmodelquantizationpartitioningedgecomputingaccuracydegradationlayer-wisebit-widthoffloadingclosed-formoptimization
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

QPART claims that an inference-serving platform should adapt its response to each edge request rather than ship a fixed pre-trained model. For a request carrying a model type, an accuracy requirement, and the device's computation and channel capacity, the server chooses a layer-wise quantization bit width for every layer in the first model segment and a partition point that splits the network between device and server, minimizing weighted time, energy, and server cost while holding accuracy degradation within the requested bound. The paper derives a closed-form expression for the optimal bit widths using KKT conditions on a layer-wise accuracy-degradation metric based on quantization noise and per-layer robustness. In simulations on MNIST, SVHN, CIFAR-10/100, and ImageNet models, the transmitted model segment shrinks to roughly 12-18 percent of its original size with measured accuracy loss between 0.08 and 0.66 percent. If correct, one pre-trained model can serve a heterogeneous fleet of devices without retraining or storing model variants.

What carries the argument

The load-bearing object is the layer-wise accuracy-degradation metric $\psi_l^w = \|\sigma_l^w\|_2^2 / \rho_l(a)$, where the numerator is quantization noise from quantizing layer $l$'s weights (modeled as $s_l e^{-\ln 4\, b_l}$) and $\rho_l(a)$ is a per-layer robustness parameter measured by injecting adversarial noise and recording the noise threshold that changes the classification. The communication payload is modeled as $Z = b_p z_p^x + \sum_{l=1}^p b_l z_l^w$, combining quantized activation size and weight sizes. The argument is carried by solving a Lagrange problem whose Hessian the paper shows to be positive semidefinite, producing Eq. 27, which equates normalized noise terms across layers, and Eq. 40, which gives the partition-point layer's bit width directly. A stack of two algorithms realizes this: an offline algorithm enumerating partition points and five accuracy levels to precompute bit-width patterns, and an online algorithm that picks the stored pattern minimizing the weighted objective.

What would settle it

Quantize a trained model with the bit widths QPART's offline algorithm selects for a requested degradation $a$, then measure end-to-end accuracy on a disjoint test set and compare it with the predicted sum of per-layer terms; if the actual degradation exceeds $a$ for a network where quantization noise correlates across layers, such as a deep residual network, the additive accuracy model cannot guarantee the claimed sub-1 percent loss.

Watch

Extended reading notes

Core claim

The central claim is that joint layer-wise quantization and model partitioning can be solved exactly under a linear, additive model of quantization-induced accuracy loss. The objective is $\min_{b,p} \omega(T_{\mathrm{local}}+T_{\mathrm{tran}}+T_{\mathrm{server}}) + \tau(E_{\mathrm{local}}+E_{\mathrm{tran}}) + \eta C$, subject to the accuracy-degradation constraint that the layer-wise quantization-noise terms sum to no more than a bound $\Delta$ tied to the request's accuracy demand. With quantization noise modeled as $\|\sigma_l^w\|_2^2 = s_l e^{-\ln 4\, b_l}$, the KKT stationarity conditions collapse into a chain of equalities across layers (Eq. 27) and a direct closed form for the bit width at the partition point (Eq. 40). The paper reports that this solution, precomputed offline for candidate partition points and five accuracy levels and looked up online, reduces the communication payload to 11.88-18.12 percent of the original model size while keeping accuracy degradation between 0.08 and 0.66 percent.

Load-bearing premise

The guarantee rests on the assumption that a network's total accuracy loss is the sum of independent per-layer quantization-noise terms, with each layer's robustness measured once on the served model and dataset.

Editorial extensions

If this is right

  • A serving platform can answer each request with a model segment tailored to that device's clock rate, memory, channel capacity, and accuracy demand, using one stored pre-trained model.
  • Because the bit-width solution is closed-form and the noise and robustness measurements are done offline, runtime serving is a lookup over precomputed patterns, so the optimization does not add latency to the request path.
  • Communication payload, the part of offloading that dominates latency in mobile-edge settings, drops to about 12-18 percent of the original model size, with reported accuracy degradation between 0.08 and 0.66 percent.
  • The weights in the objective give a tunable trade-off: a device with ample battery can set the energy weight low and squeeze latency, while a cost-sensitive device can set the server-cost weight high to minimize server usage.
  • In the paper's comparative experiments, QPART reports the lowest time and energy consumption across all partition points when measured against autoencoder-based and model-pruning-based offloading schemes.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The authors leave implicit that the same closed-form solution could be rounded to hardware-supported precisions such as 4 or 8 bits and re-checked against the $\Delta$ constraint, turning the continuous analytic solution into a deployable integer-quantization scheme.
  • A natural extension is online calibration: re-measuring the per-layer robustness $\rho_l$ and noise scale $s_l$ on the device's own data distribution would let QPART track input drift, at the cost of additional measurement passes.
  • Because the objective weights are request-level parameters, the framework could be embedded in a scheduler that negotiates service-level agreements, letting the server choose the operating point that best meets the device's latency budget and battery status.
  • The additive degradation model should be stress-tested on architectures with skip connections, where quantization noise from one layer can be re-amplified downstream; the reported experiments do not isolate that interaction effect.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

5 major / 6 minor

Summary. The paper proposes QPART, an edge inference serving system that jointly selects layer-wise quantization bit-widths and a model partition point for each inference request, minimizing weighted time, energy, and server cost subject to an accuracy-degradation constraint. The claimed contribution is a closed-form KKT solution (Eqs. 27 and 40) together with offline and online algorithms, and simulations report a communication payload reduction to roughly 12–18% of the original size with accuracy degradation below 1%.

Significance. The system concept is timely and practically motivated: it addresses per-request adaptation of a pre-trained model without retraining, and the evaluation covers several datasets and architectures (MNIST, SVHN, CIFAR-10/100, ImageNet with ResNet variants). The explicit modeling of time, energy, server cost, channel rate, and accuracy is a useful framework. However, the central closed-form derivation in Section IV contains load-bearing mathematical errors: the objective is linear in the bit-widths so the printed Hessian is not the objective's Hessian, Eq. 40 removes the accuracy budget from the partition-point bit-width, and the p-derivative is taken from a constraint that does not depend on p. As written, the paper does not establish that Eq. 27 / Eq. 40 solves the constrained problem in Eq. 23, and the reported accuracy results are calibrated rather than independently predicted. The idea is salvageable, but the theoretical core and the accuracy evaluation need substantial rework.

major comments (5)
  1. [§IV, Eqs. 23, 28, 34, 37] The optimization problem is stated inconsistently. Eq. 23 constrains only the activation noise at p and the weight-quantization noise for layers from p onward, while Eq. 28, which is used in the proof, constrains the full-network sum ∑_{l=1}^L s_l e^{-ln4 b_l}/ρ_l ≤ Δ. Eq. 34 then defines g(b,p) with no dependence on p, yet Eq. 37 claims ∂g/∂p = -s_p e^{-ln4 b_p}/ρ_p; the derivative should be zero. Consequently, the p-stationarity equation in Eq. 38 is not a valid KKT condition of the stated problem. In addition, p is a discrete partition index, so the continuous derivative with respect to p requires explicit justification.
  2. [§IV, Eqs. 28–29] The convexity claim is unsupported. The objective in Eq. 28 is linear in each b_l, because the only b-dependent term is ε ∑ b_l z_l; hence ∂²f/∂b_l² = 0 and all cross-partials are zero. The matrix printed in Eq. 29 is therefore not a Hessian of the stated objective: it is nonsymmetric, with a nonzero (∂²f/∂p∂b_p) entry while the corresponding (∂²f/∂b_p∂p) entry is zero, and it is not positive semidefinite. KKT sufficiency, and with it the claimed global optimality of the closed-form solution, does not follow from the presented argument.
  3. [§IV, Eqs. 39–40] Eq. 40 is the most serious defect in the derivation. Equating the first and last expressions in Eq. 39 cancels ρ_p, s_p, and e^{-ln4 b_p} and yields b_p = (ξo(p) - δo(p) - z_p/ln4)/(ε z_p). This expression contains no accuracy budget Δ, no requested accuracy a, and no robustness values; the bit-width at the partition point is independent of the accuracy constraint. A correct KKT solution would determine the multiplier λ from the inequality constraint, and the bit-widths would shift when Δ changes. As written, Eqs. 27 and 40 cannot enforce Eq. 23, so the claimed accuracy guarantee is unsupported.
  4. [§IV, Algorithm 1; §V, Table IV] The empirical accuracy results cannot be credited to the optimization as stated. Algorithm 1 step 8 records the noise threshold at which accuracy degradation equals the requested a, and ρ_l and s_l are measured on the served model and dataset (Eq. 22 and Algorithm 1 steps 7–10). The below-1% degradation values in Table IV are therefore calibrated, not predicted, and the accuracy constraint is enforced by construction. If the calibration data overlaps the evaluation data, the reported degradation is not an independent validation. The text should specify a held-out calibration protocol and should separate calibration measurements from evaluation results.
  5. [§III, Eq. 22; §IV, Algorithm 1 step 9] Eq. 22 does not define a genuine per-layer robustness parameter. It is written as a ratio whose numerator is the mean quantization noise at layer l, which depends on b_l through Eqs. 18–19, and whose denominator is the adversarial noise. If ρ_l depends on b_l, then Eq. 27 cannot be solved for b_l without circularity. Additionally, Algorithm 1 step 9 says to calculate ρ_l by Eq. 20, but Eq. 20 defines ψ, not ρ; the intended formula should be Eq. 22 with carefully defined inputs independent of the unknown bit-widths.
minor comments (6)
  1. [§III–§IV, notation] The index conventions are inconsistent: Eq. 14 uses b_p z^x_p + ∑_{l=1}^p b_l z^w_l, Eq. 23 refers to ∑_{l=p}^N, Eq. 27 defines z = [z^w_p ... z^w_N z^x_p], and Eq. 28 uses ∑_{l=p}^{L+1}. The roles of N, L, and L+1 should be harmonized throughout.
  2. [§V, Table IV] The text says ResNet18, ResNet34, and ResNet50 were implemented on ImageNet, but Table IV lists only ResNet18 and ResNet34; either add the ResNet50 row or correct the text.
  3. [§IV, Algorithm 1] Line 3, 'for l = p, 1 do', is ambiguous; it should be written as a descending loop, e.g., 'for l = p, p-1, ..., 1 do'.
  4. [§V, opening paragraph] The sentence 'there are two components: executing module, communication module and performance module' lists three components; please rephrase.
  5. [§II-B, related work] 'NFaaS' appears once and should be 'INFaaS'.
  6. [§III, Eq. 22 and surrounding text] The notation ρ_l(a) is used in Eq. 22, while the text uses ρ_l(a_t^n); please unify the notation and define all indices.

Circularity Check

2 steps flagged · score 6.0 of 10

The below-1% accuracy result is calibrated in Algorithm 1 rather than predicted by Eq. 27/40; the closed-form solution drops every accuracy term (Δ, ρ, s), so the accuracy claim reduces to the empirical noise-threshold fit.

  1. fitted input called prediction [Section IV, Algorithm 1 (Offline Model Quantization Algorithm), steps 7-11; Eq. 18-27]
    "Incrementally introduce noise into the parameter of layer l, observe the accuracy degradation, and record the noise threshold σl at which the accuracy degradation equals a. Calculate robustness parameter ρl by Eq.20; Calculate sl using Eq. 18, by fixing the bl; Calculate the quantization bit width bl for layer l, by Eq. 27"

    The layer-wise bit widths that QPART presents as the solution of the constrained optimization Eq. 23 are actually obtained by measuring, for each layer, the noise level at which the observed accuracy degradation equals the target a, then feeding that measured threshold into ρ_l and s_l and computing b_l from Eq. 27. The accuracy constraint is thus met by construction: the reported 'accuracy degradation below 1%' is the calibration target built into Algorithm 1, not a prediction produced by the KKT solution. The claimed closed-form solution does not rescue this: Eq. 40 cancels ρ_p, s_p, and e^{-ln4 b_p}, so b_p contains no Δ, no a, no ρ, and no s; the only place the accuracy requirement enters is the empirical thresholding step.

  2. other [Section IV, proof of Eq. 27, Eq. 39-40]
    "By further combining the first term and the last term in Eq.39 for resolution, we have: bp = (ξo(p) − δo(p) − zp/ln4)/(ϵzp) (40), which means we can directly get quantization bit-width for layer p with given partition point p."

    Eq. 40 is obtained by equating the first and last expressions of Eq. 39. Both sides contain the same factor ρ_p/(s_p e^{-ln4 b_p}); after cancellation, every accuracy-dependent quantity disappears. A correct KKT solution of Eq. 23 would have a Lagrange multiplier set by the active accuracy constraint, so the bit widths should shift when Δ changes. Eq. 40 cannot do that. This shows the paper's own formula for the 'optimal' bit width is independent of the accuracy constraint it is supposed to enforce, and the remaining dependence on the accuracy target is entirely in the empirical fit of Algorithm 1, not in the derived solution.

full rationale

The accuracy-related part of QPART's central claim is circular: the bit-widths are fixed by empirically recording the noise threshold at which per-layer accuracy degradation equals the target, and the headline 'accuracy degradation below 1%' is the same target appearing as a measured result. Eq. 40 corroborates this by cancelling the robustness and noise terms, leaving a bit width formula with no dependence on Δ or ρ. I do not treat the Section IV derivation errors (linear objective, incorrect Hessian, p-derivative of a p-free constraint) as circularity; they are internal correctness defects, as is the uncited reliance on the linear/additive accuracy metric from [33]. The time, energy, communication-payload, and server-cost components are genuine simulated measurements and are not circular; that is why the score is 6 (partial circularity) rather than 8 or 10.

Assumptions & free parameters 5 free parameters · 5 assumptions · 0 invented entities

The paper pulls the accuracy-degradation metric (Eq. 18-22), the energy model (Eq. 6), and the channel model (Eq. 11-13) from prior literature, so the genuinely new content is the closed-form chain rule for bit widths and the partition enumeration. The main fitted quantities are the per-layer robustness rho_l and noise scale s_l, measured on the served model and dataset; the accuracy budget Delta and objective weights are hand-set. No new physical or scientific entities are introduced.

free parameters (5)
  • rho_l (per-layer robustness) = not tabulated; computed from dataset D via Eq. 22 and Algorithm 1 step 8
    Measured on the exact model and dataset being served; it converts measured noise thresholds into bit-width decisions, so the accuracy outcome depends on this fitted quantity.
  • s_l (per-layer quantization noise scale) = not tabulated; set via Eq. 18 with fixed b_l (Algorithm 1 step 10)
    Fitted per layer from the same model and dataset; it appears directly in the bit-width chain of Eq. 27.
  • Delta (accuracy degradation budget) = one of {a1..a5}; not numerically specified
    Chosen by hand as one of five accuracy levels; the below-1% result follows from choosing a level near 1%.
  • omega, tau, eta (objective weights) = 1, 1, 1 in Table II
    User-selected weights over time, energy, and server cost; all reported optimum trade-offs depend on these hand-set values.
  • gamma_local, gamma_server, kappa = 5, 1.25, 3e-27
    Hand-set simulator hardware parameters taken from prior literature (Table II); not validated against real hardware in this paper.
assumptions (5)
  • domain assumption Quantization noise decays as sigma = s e^{-ln4 b}, and the accuracy degradation metric is linear, additive, and independent across layers
    Imported from reference [33]; invoked in Section IV (Eq. 18-22) with the claim that 'the linearity and additivity of this metric have already been established'. The entire closed-form solution depends on it.
  • domain assumption The minimum adversarial noise sigma* can be computed on dataset D and defines per-layer robustness rho_l
    Eq. 22 requires a dataset-level computation of the noise that flips classifications; the algorithm to compute it is not specified, and it is measured on the same dataset family as the evaluation.
  • domain assumption Energy per cycle is kappa f^2 and clock frequency is proportional to supply voltage in the low-voltage regime
    Used for Elocal in Eq. 6, citing [52] and [53]; this is a hardware model rather than a derived law for the target devices.
  • standard math Channel capacity follows Shannon's formula with small-scale fading and additive noise
    Eq. 11-13; standard communication model, though the simulation fixes the rate at 200 Mbps rather than drawing fading states.
  • ad hoc to paper The discrete partition index p can be treated as continuous for the KKT optimum
    Eq. 27-40 derive stationarity in p; the printed Hessian in Eq. 29 is not the Hessian of the stated objective, and the algorithm enumerates p. The closed form for p is therefore not established.

how reviews work

0 comments
Cite this review

Pith. "Pith review of QPART: Adaptive Model Quantization and Dynamic Workload Balancing for Accuracy-aware Edge Inference." pith.science (2026). https://pith.science/paper/YCJA3GXK

@misc{pith2026250623934,
  author       = {Pith},
  title        = {Pith review of: QPART: Adaptive Model Quantization and Dynamic Workload Balancing for Accuracy-aware Edge Inference},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/YCJA3GXK}},
  note         = {Machine review of arXiv:2506.23934}
}
read the original abstract

As machine learning inferences increasingly move to edge devices, adapting to diverse computational capabilities, hardware, and memory constraints becomes more critical. Instead of relying on a pre-trained model fixed for all future inference queries across diverse edge devices, we argue that planning an inference pattern with a request-specific model tailored to the device's computational capacity, accuracy requirements, and time constraints is more cost-efficient and robust to diverse scenarios. To this end, we propose an accuracy-aware and workload-balanced inference system that integrates joint model quantization and inference partitioning. In this approach, the server dynamically responds to inference queries by sending a quantized model and adaptively sharing the inference workload with the device. Meanwhile, the device's computational power, channel capacity, and accuracy requirements are considered when deciding. Furthermore, we introduce a new optimization framework for the inference system, incorporating joint model quantization and partitioning. Our approach optimizes layer-wise quantization bit width and partition points to minimize time consumption and cost while accounting for varying accuracy requirements of tasks through an accuracy degradation metric in our optimization model. To our knowledge, this work represents the first exploration of optimizing quantization layer-wise bit-width in the inference serving system, by introducing theoretical measurement of accuracy degradation. Simulation results demonstrate a substantial reduction in overall time and power consumption, with computation payloads decreasing by over 80% and accuracy degradation kept below 1%.

Figures

Figures reproduced from arXiv: 2506.23934 by the authors.

Figure 1
Figure 1. Layer-wise Parameter Size Reduction This figure explains how the [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 2
Figure 2. Data Flow between Edge Devices and QPART System multiple distinct inference requests will be initiated from edge devices, each characterized by varying neural network structures. We denote an inference query r from a mobile device in the time slot t by a tuple r = (θ, a), where θ indicates the specific neural network model and a is the accuracy demand, which is defined as the maximal acceptable accuracy degradation … view at source ↗
Figure 3
Figure 3. Layer-wise Parameter Size Reduction. From the figure, a substantial [PITH_FULL_IMAGE:figures/full_fig_p005_3.png] view at source ↗
Figures from the paper (5 more)
Figure 5
Figure 5. Figure 5: Layer-wise Performance Comparison accuracy degradation thresholds and testing the optimized parameter size. This relationship is discerned by aggregating the sizes of the parameters across all layers. It is evident from the figure that the overall model parameter size …
Figure 6
Figure 6. Figure 6: Optimized Model Size v.s. Accuracy Fig.7, Fig.8, and Fig.9 compare the proposed algorithm [PITH_FULL_IMAGE:figures/full_fig_p009_6.png]
Figure 8
Figure 8. Figure 8: Layer-wise Energy Consumption Comparison [PITH_FULL_IMAGE:figures/full_fig_p010_8.png]
Figure 9
Figure 9. Figure 9: Layer-wise Time consumption Comparison mented ResNet architectures, including ResNet18, ResNet34, and ResNet50, on the ImageNet dataset to further benchmark our framework. Table IV illustrates the efficacy of our pro￾posed QPART when applied across these datasets and m…
Figure 10
Figure 10. Figure 10: Layer-wise Communication Payload Comparison [PITH_FULL_IMAGE:figures/full_fig_p011_10.png]

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

59 extracted references · 44 canonical work pages

  1. [1]

    Imagenet classification with deep convolutional neural networks,

    A. Krizhevsky, I. Sutskever, and G. E. Hinton, “Imagenet classification with deep convolutional neural networks,” Advances in neural informa- tion processing systems , vol. 25, 2012

    2012

  2. [2]

    Very deep convolutional networks for large-scale image recognition,

    K. Simonyan and A. Zisserman, “Very deep convolutional networks for large-scale image recognition,” arXiv preprint arXiv:1409.1556 , 2014

    arXiv 2014

  3. [3]

    Distributed representations of words and phrases and their composi- tionality,

    T. Mikolov, I. Sutskever, K. Chen, G. S. Corrado, and J. Dean, “Distributed representations of words and phrases and their composi- tionality,” Advances in neural information processing systems , vol. 26, 2013

    work page 2013

  4. [4]

    Attention is all you need,

    A. Vaswani, N. Shazeer, N. Parmar, J. Uszkoreit, L. Jones, A. N. Gomez, Ł. Kaiser, and I. Polosukhin, “Attention is all you need,” Advances in neural information processing systems , vol. 30, 2017

    2017

  5. [5]

    End to end learning for self-driving cars,

    M. Bojarski, D. Del Testa, D. Dworakowski, B. Firner, B. Flepp, P. Goyal, L. D. Jackel, M. Monfort, U. Muller, J. Zhang et al. , “End to end learning for self-driving cars,” arXiv preprint arXiv:1604.07316 , 2016

    arXiv 2016

  6. [6]

    Efficient processing of deep neural networks: A tutorial and survey,

    V . Sze, Y .-H. Chen, T.-J. Yang, and J. S. Emer, “Efficient processing of deep neural networks: A tutorial and survey,” Proceedings of the IEEE , vol. 105, no. 12, pp. 2295–2329, 2017

    2017

  7. [7]

    Edge intelligence: Paving the last mile of artificial intelligence with edge computing,

    Z. Zhou, X. Chen, E. Li, L. Zeng, K. Luo, and J. Zhang, “Edge intelligence: Paving the last mile of artificial intelligence with edge computing,” Proceedings of the IEEE , vol. 107, no. 8, pp. 1738–1762, 2019

    2019

  8. [8]

    Edge computing: Vision and challenges,

    W. Shi, J. Cao, Q. Zhang, Y . Li, and L. Xu, “Edge computing: Vision and challenges,” IEEE internet of things journal , vol. 3, no. 5, pp. 637–646, 2016

    work page 2016

Show all 59 references
  1. [9]

    Exploring edge tpu for network intrusion detection in iot,

    S. Hosseininoorbin, S. Layeghy, M. Sarhan, R. Jurdak, and M. Portmann, “Exploring edge tpu for network intrusion detection in iot,” Journal of Parallel and Distributed Computing , vol. 179, p. 104712, 2023

    2023

  2. [10]

    A survey on optimized implementation of deep learning models on the nvidia jetson platform,

    S. Mittal, “A survey on optimized implementation of deep learning models on the nvidia jetson platform,” Journal of Systems Architecture , vol. 97, pp. 428–442, 2019

    2019

  3. [11]

    Mobilenetv2: Inverted residuals and linear bottlenecks,

    M. Sandler, A. Howard, M. Zhu, A. Zhmoginov, and L.-C. Chen, “Mobilenetv2: Inverted residuals and linear bottlenecks,” in Proceedings of the IEEE conference on computer vision and pattern recognition , 2018, pp. 4510–4520

    2018

  4. [12]

    Efficientnet: Rethinking model scaling for convolutional neural networks,

    M. Tan, “Efficientnet: Rethinking model scaling for convolutional neural networks,” arXiv preprint arXiv:1905.11946 , 2019

    1905 arXiv

  5. [13]

    Early stopping-but when?

    L. Prechelt, “Early stopping-but when?” in Neural Networks: Tricks of the trade. Springer, 2002, pp. 55–69

    2002

  6. [14]

    Single-layer vision trans- formers for more accurate early exits with less overhead,

    A. Bakhtiarnia, Q. Zhang, and A. Iosifidis, “Single-layer vision trans- formers for more accurate early exits with less overhead,” Neural Networks, vol. 153, pp. 461–473, 2022

    2022

  7. [15]

    1xn pattern for pruning convolutional neural networks,

    M. Lin, Y . Zhang, Y . Li, B. Chen, F. Chao, M. Wang, S. Li, Y . Tian, and R. Ji, “1xn pattern for pruning convolutional neural networks,” IEEE Transactions on Pattern Analysis and Machine Intelligence , vol. 45, no. 4, pp. 3999–4008, 2022

    2022

  8. [16]

    Rethinking the value of network pruning,

    Z. Liu, M. Sun, T. Zhou, G. Huang, and T. Darrell, “Rethinking the value of network pruning,” arXiv preprint arXiv:1810.05270 , 2018

    2018 arXiv

  9. [17]

    Fast and accurate streaming cnn infer- ence via communication compression on the edge,

    D. Hu and B. Krishnamachari, “Fast and accurate streaming cnn infer- ence via communication compression on the edge,” in 2020 IEEE/ACM Fifth International Conference on Internet-of-Things Design and Imple- mentation (IoTDI). IEEE, 2020, pp. 157–163

    2020

  10. [18]

    Knowledge distillation for mobile edge computation offloading,

    H. Chen, L. Zeng, S. Yu, and X. Chen, “Knowledge distillation for mobile edge computation offloading,” arXiv preprint arXiv:2004.04366, 2020

    2004 arXiv

  11. [19]

    Neurosurgeon: Collaborative intelligence between the cloud and mobile edge,

    Y . Kang, J. Hauswald, C. Gao, A. Rovinski, T. Mudge, J. Mars, and L. Tang, “Neurosurgeon: Collaborative intelligence between the cloud and mobile edge,” ACM SIGARCH Computer Architecture News, vol. 45, no. 1, pp. 615–629, 2017

    2017

  12. [20]

    Edge assisted real-time object detec- JOURNAL OF LATEX CLASS FILES, VOL. 14, NO. 8, AUGUST 2021 12 tion for mobile augmented reality,

    L. Liu, H. Li, and M. Gruteser, “Edge assisted real-time object detec- JOURNAL OF LATEX CLASS FILES, VOL. 14, NO. 8, AUGUST 2021 12 tion for mobile augmented reality,” in The 25th annual international conference on mobile computing and networking , 2019, pp. 1–16

    2021

  13. [21]

    Machine learning at face- book: Understanding inference at the edge,

    C.-J. Wu, D. Brooks, K. Chen, D. Chen, S. Choudhury, M. Dukhan, K. Hazelwood, E. Isaac, Y . Jia, B. Jia et al., “Machine learning at face- book: Understanding inference at the edge,” in 2019 IEEE international symposium on high performance computer architecture (HPCA). IEEE, 2...

    2019

  14. [22]

    From smart to deep: Robust activ- ity recognition on smartwatches using deep learning,

    S. Bhattacharya and N. D. Lane, “From smart to deep: Robust activ- ity recognition on smartwatches using deep learning,” in 2016 IEEE International conference on pervasive computing and communication workshops (PerCom Workshops). IEEE, 2016, pp. 1–6

    2016

  15. [23]

    On- device learning systems for edge intelligence: A software and hardware synergy perspective,

    Q. Zhou, Z. Qu, S. Guo, B. Luo, J. Guo, Z. Xu, and R. Akerkar, “On- device learning systems for edge intelligence: A software and hardware synergy perspective,” IEEE Internet of Things Journal , vol. 8, no. 15, pp. 11 916–11 934, 2021

    2021

  16. [24]

    Federated learning over wireless fading channels,

    M. M. Amiri and D. G ¨und¨uz, “Federated learning over wireless fading channels,” IEEE Transactions on Wireless Communications , vol. 19, no. 5, pp. 3546–3557, 2020

    2020

  17. [25]

    A mathematical theory of communication,

    C. E. Shannon, “A mathematical theory of communication,” The Bell system technical journal , vol. 27, no. 3, pp. 379–423, 1948

    1948

  18. [26]

    Deep learning for healthcare: review, opportunities and challenges,

    R. Miotto, F. Wang, S. Wang, X. Jiang, and J. T. Dudley, “Deep learning for healthcare: review, opportunities and challenges,” Briefings in bioinformatics, vol. 19, no. 6, pp. 1236–1246, 2018

    2018

  19. [27]

    Application of deep learning on iot-enabled smart grid monitoring,

    I. Al-Omari, S. Hadayeghparast, and H. Karimipour, “Application of deep learning on iot-enabled smart grid monitoring,” AI-Enabled Threat Detection and Security Analysis for Industrial IoT , pp. 77–103, 2021

    2021

  20. [28]

    Recommended for you: The netflix prize and the production of algorithmic culture,

    B. Hallinan and T. Striphas, “Recommended for you: The netflix prize and the production of algorithmic culture,”New media & society, vol. 18, no. 1, pp. 117–137, 2016

    2016

  21. [29]

    Sentiment analysis with machine learning methods on social media,

    M. S. Basarslan, F. Kayaalp et al. , “Sentiment analysis with machine learning methods on social media,” Advances in Distributed Computing and Artificial Intelligence Journal , 2020

    2020

  22. [30]

    Deep compression: Compressing deep neural networks with pruning, trained quantization and huffman coding,

    S. Han, H. Mao, and W. J. Dally, “Deep compression: Compressing deep neural networks with pruning, trained quantization and huffman coding,” arXiv preprint arXiv:1510.00149 , 2015

    2015 arXiv

  23. [31]

    Energy-efficient neural networks using approximate computation reuse,

    X. Jiao, V . Akhlaghi, Y . Jiang, and R. K. Gupta, “Energy-efficient neural networks using approximate computation reuse,” in 2018 Design, Automation & Test in Europe Conference & Exhibition (DATE) . IEEE, 2018, pp. 1223–1228

    2018

  24. [32]

    Edge ai: On-demand accelerating deep neural network inference via edge computing,

    E. Li, L. Zeng, Z. Zhou, and X. Chen, “Edge ai: On-demand accelerating deep neural network inference via edge computing,” IEEE Transactions on Wireless Communications, vol. 19, no. 1, pp. 447–457, 2019

    2019

  25. [33]

    Adaptive quantization for deep neural network,

    Y . Zhou, S.-M. Moosavi-Dezfooli, N.-M. Cheung, and P. Frossard, “Adaptive quantization for deep neural network,” in Proceedings of the AAAI Conference on Artificial Intelligence , vol. 32, no. 1, 2018

    2018

  26. [34]

    Energy-aware inference offloading for dnn-driven applications in mobile edge clouds,

    Z. Xu, L. Zhao, W. Liang, O. F. Rana, P. Zhou, Q. Xia, W. Xu, and G. Wu, “Energy-aware inference offloading for dnn-driven applications in mobile edge clouds,” IEEE Transactions on Parallel and Distributed Systems, vol. 32, no. 4, pp. 799–814, 2020

    2020

  27. [35]

    Deep compressive offloading: Speeding up neural network inference by trading edge computation for network latency,

    S. Yao, J. Li, D. Liu, T. Wang, S. Liu, H. Shao, and T. Abdelzaher, “Deep compressive offloading: Speeding up neural network inference by trading edge computation for network latency,” in Proceedings of the 18th conference on embedded networked sensor systems , 2020, pp. 476–488

    2020

  28. [36]

    Dis- tributed inference acceleration with adaptive dnn partitioning and of- floading,

    T. Mohammed, C. Joe-Wong, R. Babbar, and M. Di Francesco, “Dis- tributed inference acceleration with adaptive dnn partitioning and of- floading,” in IEEE INFOCOM 2020-IEEE Conference on Computer Communications. IEEE, 2020, pp. 854–863

    2020

  29. [37]

    Computation offloading for fast cnn inference in edge computing,

    Q. Yang, X. Luo, P. Li, T. Miyazaki, and X. Wang, “Computation offloading for fast cnn inference in edge computing,” in Proceedings of the Conference on Research in Adaptive and Convergent Systems , 2019, pp. 101–106

    2019

  30. [38]

    Optimiza- tion of offloading policies for accuracy-delay tradeoffs in hierarchical inference,

    H. B. Beytur, A. G. Aydin, G. de Veciana, and H. Vikalo, “Optimiza- tion of offloading policies for accuracy-delay tradeoffs in hierarchical inference,” in IEEE INFOCOM 2024-IEEE Conference on Computer Communications. IEEE, 2024, pp. 1989–1998

    2024

  31. [39]

    Multi-agent deep reinforcement learning-based inference task schedul- ing and offloading for maximum inference accuracy under time and energy constraints,

    A. Ben Sada, A. Khelloufi, A. Naouri, H. Ning, N. Aung, and S. Dhelim, “Multi-agent deep reinforcement learning-based inference task schedul- ing and offloading for maximum inference accuracy under time and energy constraints,” Electronics, vol. 13, no. 13, p. 2580, 2024

    2024

  32. [40]

    {INFaaS}: Automated model-less inference serving,

    F. Romero, Q. Li, N. J. Yadwadkar, and C. Kozyrakis, “ {INFaaS}: Automated model-less inference serving,” in 2021 USENIX Annual Technical Conference (USENIX ATC 21) , 2021, pp. 397–411

    2021

  33. [41]

    Fann-on-mcu: An open-source toolkit for energy-efficient neural network inference at the edge of the internet of things,

    X. Wang, M. Magno, L. Cavigelli, and L. Benini, “Fann-on-mcu: An open-source toolkit for energy-efficient neural network inference at the edge of the internet of things,” IEEE Internet of Things Journal , vol. 7, no. 5, pp. 4403–4417, 2020

    2020

  34. [42]

    Construct- ing energy-efficient mixed-precision neural networks through principal component analysis for edge intelligence,

    I. Chakraborty, D. Roy, I. Garg, A. Ankit, and K. Roy, “Construct- ing energy-efficient mixed-precision neural networks through principal component analysis for edge intelligence,” Nature Machine Intelligence, vol. 2, no. 1, pp. 43–55, 2020

    2020

  35. [43]

    Decision early-exit: An efficient approach to hasten offloading in branchynets,

    M. S. Barbosa, R. G. Pacheco, R. S. Couto, D. S. Medeiros, and M. E. M. Campista, “Decision early-exit: An efficient approach to hasten offloading in branchynets,” in 2022 IEEE Latin-American Conference on Communications (LATINCOM). IEEE, 2022, pp. 1–6

    2022

  36. [44]

    A deep reinforcement learning based research for optimal offloading decision,

    J. Ren, D. Yang, Y . Yuan, H. Wei, and Z. Wang, “A deep reinforcement learning based research for optimal offloading decision,” AIP Advances, vol. 13, no. 8, 2023

    2023

  37. [45]

    Improving device-edge cooperative inference of deep learning via 2-step pruning,

    W. Shi, Y . Hou, S. Zhou, Z. Niu, Y . Zhang, and L. Geng, “Improving device-edge cooperative inference of deep learning via 2-step pruning,” in IEEE INFOCOM 2019-IEEE Conference on Computer Communica- tions Workshops (INFOCOM WKSHPS) . IEEE, 2019, pp. 1–6

    2019

  38. [46]

    Graph reinforcement learning-based cnn inference offloading in dynamic edge computing,

    N. Li, A. Iosifidis, and Q. Zhang, “Graph reinforcement learning-based cnn inference offloading in dynamic edge computing,” 2022

    2022

  39. [47]

    Adaee: Adaptive early-exit dnn inference through multi-armed bandits,

    R. G. Pacheco, M. Shifrin, R. S. Couto, D. S. Menasch ´e, M. K. Hanawal, and M. E. M. Campista, “Adaee: Adaptive early-exit dnn inference through multi-armed bandits,” in ICC 2023-IEEE International Conference on Communications . IEEE, 2023, pp. 3726–3731

    2023

  40. [48]

    Optimizing job offloading schedule for collabora- tive dnn inference,

    Y . Duan and J. Wu, “Optimizing job offloading schedule for collabora- tive dnn inference,” IEEE Transactions on Mobile Computing , 2023

    2023

  41. [49]

    Improving the accuracy-latency trade-off of edge-cloud com- putation offloading for deep learning services,

    X. Zhao, M. Hosseinzadeh, N. Hudson, H. Khamfroush, and D. E. Lucani, “Improving the accuracy-latency trade-off of edge-cloud com- putation offloading for deep learning services,” in 2020 IEEE Globecom Workshops (GC Wkshps. IEEE, 2020, pp. 1–6

    2020

  42. [50]

    Selective task offloading for maximum inference accuracy and energy efficient real-time iot sensing systems,

    A. B. Sada, A. Khelloufi, A. Naouri, H. Ning, and S. Dhelim, “Selective task offloading for maximum inference accuracy and energy efficient real-time iot sensing systems,” arXiv preprint arXiv:2402.16904 , 2024

    2024 arXiv

  43. [51]

    Energy consump- tion of neural networks on nvidia edge boards: an empirical model,

    S. Lahmer, A. Khoshsirat, M. Rossi, and A. Zanella, “Energy consump- tion of neural networks on nvidia edge boards: an empirical model,” in 2022 20th international symposium on modeling and optimization in mobile, ad hoc, and wireless networks (WiOpt) . IEEE, 2022, pp. 365–371

    2022

  44. [52]

    The en- ergy/frequency convexity rule: Modeling and experimental validation on mobile devices,

    K. De V ogeleer, G. Memmi, P. Jouvelot, and F. Coelho, “The en- ergy/frequency convexity rule: Modeling and experimental validation on mobile devices,” in Parallel Processing and Applied Mathematics: 10th International Conference, PPAM 2013, Warsaw, Poland, September 8-11, 201...

    2013

  45. [53]

    Processor design for portable systems,

    T. D. Burd and R. W. Brodersen, “Processor design for portable systems,” Journal of VLSI signal processing systems for signal, image and video technology , vol. 13, no. 2-3, pp. 203–221, 1996

    1996

  46. [54]

    Going deeper with embedded fpga platform for convolutional neural network,

    J. Qiu, J. Wang, S. Yao, K. Guo, B. Li, E. Zhou, J. Yu, T. Tang, N. Xu, S. Song et al., “Going deeper with embedded fpga platform for convolutional neural network,” in Proceedings of the 2016 ACM/SIGDA international symposium on field-programmable gate arrays , 2016, pp. 26–35

    2016

  47. [55]

    A survey of quantization methods for efficient neural network infer- ence,

    A. Gholami, S. Kim, Z. Dong, Z. Yao, M. W. Mahoney, and K. Keutzer, “A survey of quantization methods for efficient neural network infer- ence,” arXiv preprint arXiv:2103.13630 , 2021

    2021 arXiv

  48. [56]

    Quantized neural networks: Training neural networks with low pre- cision weights and activations,

    I. Hubara, M. Courbariaux, D. Soudry, R. El-Yaniv, and Y . Bengio, “Quantized neural networks: Training neural networks with low pre- cision weights and activations,” The Journal of Machine Learning Research, vol. 18, no. 1, pp. 6869–6898, 2017

    2017

  49. [57]

    Dorefa-net: Training low bitwidth convolutional neural networks with low bitwidth gradients,

    S. Zhou, Y . Wu, Z. Ni, X. Zhou, H. Wen, and Y . Zou, “Dorefa-net: Training low bitwidth convolutional neural networks with low bitwidth gradients,” arXiv preprint arXiv:1606.06160 , 2016

    2016 arXiv

  50. [58]

    Deep reinforcement learning-based anti-jamming algorithm using dual action network,

    X. Li, J. Chen, X. Ling, and T. Wu, “Deep reinforcement learning-based anti-jamming algorithm using dual action network,” IEEE Transactions on Wireless Communications, 2022

    2022

  51. [59]

    Gradient-based learning applied to document recognition,

    Y . LeCun, L. Bottou, Y . Bengio, and P. Haffner, “Gradient-based learning applied to document recognition,” Proceedings of the IEEE , vol. 86, no. 11, pp. 2278–2324, 1998

    1998

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.