Pith. sign in

REVIEW 5 major objections 6 minor 42 references

P$^2$U: Progressive Precision Update For Efficient Model Distribution

T0 review · 5 major / 6 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read Transmitting a low-precision model first and then a compact residual update beats direct 16-bit quantization in accuracy while using less bandwidth and startup time.

desk verdict P2U is successive refinement in a new coat: the evaluation is broad but the central update-size numbers are implausibly small and contradict the exactness claim. read the letter →

arxiv 2506.22871 v1 pith:J3UIE4UT submitted 2025-06-28 cs.LG cs.MM

Homayun Afrabandpey , Hamed Rezazadegan Tavakoli This is my paper · ORCID
classification cs.LGcs.MM
keywords modeldistributionquantizationresidualupdatebandwidthefficiencystartuplatencycompressionedgecomputingfederatedlearning
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper proposes Progressive Precision Update (P2U), a transmission scheme that sends a low-bit quantized model first, lets the receiver start inference immediately, and then sends a model update equal to the difference between the original high-precision model and the low-precision version. The receiver adds the update to the low-precision weights to reconstruct a high-precision proxy, so the accuracy loss from aggressive quantization is largely recovered. The central empirical claim is that P2U improves the accuracy-bandwidth-latency tradeoff: with a 4-bit base model plus a 32-bit update it surpasses direct 16-bit quantization in final top-1 accuracy on all three tested datasets, while transmitting less data and starting inference sooner. The paper argues this matters because model distribution in federated learning, edge computing, and IoT is dominated by bandwidth and startup latency, and P2U is complementary to existing compression techniques such as sparsification and pruning.

What carries the argument

The central object is the precision update $\Delta W = W^h - W^l$, an integer residual that carries exactly the information lost when the model is quantized to a low bit-width. P2U entropy-codes the low-precision base and the residual separately, transmits the base first for immediate inference, then transmits the residual; the receiver reconstructs the proxy $W' = W^l + \Delta W$. The argument that this works is a first-order Taylor expansion of the network output around $W^l$, which shows the proxy output error is governed by residual terms of order $O(\delta^2)$, so for small quantization gaps the proxy is nearly identical in output to the high-precision model.

What would settle it

Take a model whose quantized residual is deliberately made high-entropy, for instance a network with unusually large weight magnitudes and little quantization redundancy, and compare the entropy-coded size of $\Delta W$ with the direct 16-bit bitstream; if the residual size is not a small fraction of the low-precision model, P2U's total bandwidth no longer beats direct quantization.

Watch

Extended reading notes

Core claim

P2U decomposes a model for transfer into two pieces: a low-precision integer version $W^l$ (e.g., 8-bit or 4-bit) and an update $\Delta W = W^h - W^l$ computed against the original high-precision model $W^h$ (32-bit integer in the experiments). The receiver reconstructs $W' = W^l + \Delta W$ and uses it for inference. The update is not a retrained model; it is an exact arithmetic residual, and because the low-precision model already captures most of the weight values, the residual bitstream is very small, for example 0.17 MB for VGG16 at 8-bit versus 112.83 MB for the base model. A first-order Taylor expansion of the network output around $W^l$ shows that the output difference between the proxy and the high-precision model is bounded by higher-order terms $|R' - R_h|$, of order $O(\delta^2)$ when the weight difference $\delta = \|W^h - W^l\|$ is small. Across MobileNet-v2, ResNet18, EfficientNet-b4, and VGG16 and on chest X-ray, PASCAL-VOC, and CIFAR-100, P2U reaches final accuracies at or above direct 16-bit quantization with lower total bitstream sizes and startup times.

Load-bearing premise

The residual update must remain much smaller than the low-precision model itself, so that base-plus-update transmission costs less than sending a higher-precision model directly; if the residual is not highly compressible or cannot be stored exactly at low bit-width, the bandwidth and latency gains shrink.

Editorial extensions

If this is right

  • Aggressive quantization (4-bit) becomes usable as a deployment strategy: a receiver can run a coarse model almost immediately and later refine it, keeping final accuracy close to the 16-bit baseline.
  • P2U can be stacked on top of any compression method; combining the residual update with sparsification, pruning, or other encoding should cut bitstreams further.
  • In a sequential channel, the bandwidth requirement for P2U is capped by the larger of the base and update bitstreams rather than their sum, so the reported worst-case totals are conservative.
  • Choosing the base precision becomes a tunable application knob: 8-bit gives the most balanced gains, while 4-bit is best when bandwidth or startup time dominates.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The same residual mechanism applies beyond quantization: any small-magnitude weight difference, such as a fine-tuning update or a federated round delta, could be transmitted as a compact residual after a low-precision base, though the paper only tests quantization residuals.
  • The Taylor bound suggests a testable design rule: models whose loss landscape has small second-order terms should show even closer proxy accuracy, so combining P2U with flat-minima training could improve reconstruction fidelity.
  • Because the update can be encoded adaptively at lower bit-widths, the 32-bit residual in the reported experiments is not a fundamental overhead; a production system could pick the update precision from the residual's value range and shrink total size further.
  • The paper leaves LLM distribution untested; if LLM quantization residuals are similarly low-entropy, P2U could reduce download sizes for large models, but the sensitivity of LLM outputs to small weight perturbations makes that an open empirical question.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

5 major / 6 minor

Summary. This paper proposes Progressive Precision Update (P2U), a two-stage model distribution scheme in which the server first sends a low-precision quantized model to enable fast client-side inference, and then sends an update defined as the difference between the original high-precision model and the low-precision model. The receiver adds the update to form a 'proxy' of the high-precision model, with the aim of improving the accuracy-bandwidth-latency tradeoff relative to direct quantization. The authors present a Taylor-expansion argument as a theoretical guarantee, and report experiments on MobileNet-v2, ResNet18, EfficientNet-b4, and VGG16 over three datasets, claiming that P2U consistently beats direct 16-bit quantization in accuracy while using less bandwidth and lower startup time.

Significance. If the reported tradeoffs are correct, P2U would indeed be a simple and broadly applicable drop-in method for efficient model distribution, complementary to existing compression techniques and requiring no retraining. The experimental coverage (four architectures, three datasets) is reasonable, and the writing is mostly clear. However, the significance is currently undercut by three issues: the theoretical section is tautological rather than a real bound; the reported update bitstream sizes are implausibly small for exact 32-bit residuals; and the comparison against 16-bit quantization is made with a post-hoc-selected 4-bit configuration. These issues must be resolved before the central claim can be accepted.

major comments (5)
  1. [Section 3.1, Eqs. (1)-(5)] The 'theoretical guarantee' is a tautology. Because W' is defined as W^l + (W^h - W^l), one immediately obtains W' = W^h, so Eq. (5) reduces to |f(W^h,x) - f(W^h,x)| ≤ |R' - R_h|, which is vacuous (or identically zero if the Taylor expansions are exact). If the transmitted update is actually the quantized ΔW_Q, then ΔW_Q ≠ W^h - W^l, and the substitution used in Eq. (4) is invalid. The authors should either remove this section or replace it with a proper analysis that explicitly treats update quantization error, e.g., a bound involving ||ΔW - ΔW_Q|| and a Lipschitz constant of the network output.
  2. [Section 4.2.2 and Tables 3-5] The reported bitstream sizes for the update are implausibly small for a 32-bit exact residual. For VGG16 (138.4M parameters), an update of 0.17 MB (Table 3) corresponds to roughly 0.01 bits per parameter; no entropy coder can losslessly compress an exact 32-bit residual to this size unless the residual is almost entirely zero or additional sparsification is applied, which is not described. For ResNet18, 0.02 MB in Table 3 is similarly far below the per-parameter entropy of a nontrivial residual. Moreover, if the update truly preserved the exact difference (as claimed in Section 4.2.2), the proxy accuracy would be independent of the base precision for a fixed high-precision model; instead, tables 4 and 5 show proxy accuracies varying with base precision (e.g., 75.49, 75.6, and 75.18 for VGG16). The paper must disclose the actual encoding of the update (e.g., sparsity, lossy quantization, or an additional transform), report the codec configuration, and include a direct 32-bit baseline so these sizes can be sanity-checked.
  3. [Section 4.2.1, Table 2 and footnote 5] The headline comparison against direct 16-bit quantization uses the 'best-performing' low-precision configuration (4-bit), selected post hoc as stated in footnote 5. Table 4 shows that for VGG16 on PASCAL-VOC the 8-bit base achieves proxy accuracy 75.6, which is higher than the 4-bit base's 75.18; Table 5 shows that the 4-bit base is not uniquely best for ResNet18 either. Therefore, the claim that P2U 'consistently surpasses' 16-bit direct quantization is not robust across precision choices. Report all P2U precision configurations and derive any summary claim from the full set of results.
  4. [Section 4.2.2, worst-case estimates paragraph] The definition of bandwidth usage is internally inconsistent. Tables 2-6 use 'Total' as the sum of the low-precision and update sizes, which is the actual number of bytes transferred when the update is sent after the low-precision model. However, the text states that the bandwidth requirement is 'capped by the larger of the bitstream sizes' when sequential transmission is used. This conflates peak transmission rate with total data volume and, if used to justify the reported savings, understates the bandwidth cost. The paper should specify whether the reported 'Size' is total bytes transferred or peak transfer size, and use the same definition throughout.
  5. [Tables 2-6] No direct 32-bit transmission baseline is reported. Since the P2U proxy is intended to recover the high-precision model, the natural reference point is the size, time, and accuracy of transmitting the 32-bit model with the same codec. Without this baseline, the proxy accuracy improvements over 16-bit direct quantization cannot be interpreted, and the reported update sizes cannot be validated. Adding a 32-bit baseline is necessary to support the central tradeoff claim.
minor comments (6)
  1. [Footnote 5] The footnote references 'Tables II-IV', but the manuscript's tables are numbered 2, 3, 4, and 5; the cross-reference should be corrected.
  2. [Abstract and body] The notation for the proposed method is inconsistent (e.g., 'P$^2$U', 'P 2U', and 'P2U'); a single rendering should be used throughout.
  3. [Section 4.2] Although the paper states that results are averaged over 10 runs, no standard deviations or confidence intervals are reported; these should be included for the accuracy numbers.
  4. [Section 4.2.2] There is a typo, 'Thirds', in the paragraph beginning 'Thirds, For larger...'.
  5. [Eq. (5)] The norms on R' and R_h are not defined; clarify whether these are function norms or another metric.
  6. [Table 1] The model sizes in Table 1 appear inconsistent with the parameter counts and a float32 assumption (e.g., VGG16: 138.4M parameters at 4 bytes/param gives 553.6 MB, not 489 MB). Please clarify the unit (decimal vs. binary MB) and the exact precision used for these sizes.

Circularity Check

1 steps flagged · score 6.0 of 10

The theoretical guarantee in Section 3.1 is definitional: the proxy is constructed to equal the high-precision model, so Eq. (5) restates the identity that defines the update. The empirical comparisons are independent experiments, but the accuracy-dominance claim partly inherits this triviality.

  1. self definitional [Section 3.1, Eqs. (1)-(5) and surrounding text]
    "The update is defined as: ∆W = W h − W l, (1) ... At the receiver, we construct a high-precision proxy for W h as W ′ = W l + ∆W. ... |f (W ′, x) − f (W h, x)| ≤ |R′ − Rh|. (5)"

    Substituting Eq. (1) into W′ = W_l + ΔW gives W′ = W_h exactly (for the non-quantized updates used in the analysis). Hence f(W′, x) = f(W_h, x) and R′ = R_h, so Eq. (5) is the identity 0 ≤ 0. The Taylor expansion and the two assumptions are unnecessary; the claimed 'minimal accuracy loss' is exactly zero by construction, not an approximation result. The paper presents this as a proof that the proxy approximates the high-precision model with high fidelity, but the proof merely restates the definition of the update. Consequently, the theoretical guarantee has no independent content, and the empirical finding that the proxy beats direct 16-bit quantization is partly the trivial statement that a 32-bit model (or a near-exact reconstruction) can outperform quantized models.

full rationale

The only clear circularity is the theoretical analysis in Section 3.1. Because W′ is defined as W_l + ΔW and ΔW is defined as W_h − W_l, the proxy is identical to the high-precision model before quantization effects are considered; the Taylor-expansion bound is therefore a tautology. The paper's central empirical claim—better accuracy with lower total size and latency than direct quantization—is based on experiments against external baselines, so it is not circular in the statistical sense. However, the accuracy comparison is weakened by the definitional identity: if the update were exactly preserved, the proxy accuracy would equal the 32-bit model accuracy, making 'P2U surpasses 16-bit' a restatement of 'full precision beats 16-bit quantization.' The separate, non-circular concern about implausibly small update bitstreams (e.g., 0.17 MB for VGG16, about 0.01 bits/parameter) and the internal inconsistency between 'preserving the exact difference' and proxy accuracy varying with base precision are correctness/verification issues, not additional circularity. Self-citations to NNCodec and the NNR standard are used as implementation tools, not as load-bearing justification for the method, so they do not raise the score. Overall, partial circularity: the formal guarantee reduces to the construction, while the empirical tradeoff claims retain independent experimental content that should be verified with exact bitstream accounting.

Assumptions & free parameters 4 free parameters · 5 assumptions · 0 invented entities

The central claim rests on the NNCodec quantization and coding stack and on the empirical fact that residual update bitstreams are small; neither is independently verified here. No new theoretical entities are introduced.

free parameters (4)
  • Per-tensor quantization scale and zero-point (NNCodec uniform quantization) = not reported
    Controls the actual bitstream sizes and dequantized accuracies; central to the reported sizes, but the values are internal to NNCodec and not given.
  • Update representation precision = 32-bit integer
    Design choice guaranteeing exact residual, but it determines update bitstream size; an adaptive 16-bit variant is proposed but not used.
  • Low-precision base bit-width = 4-bit, 8-bit, or 16-bit per experiment
    Hand-selected per scenario; the paper's conclusions about aggressive quantization depend on which level is chosen.
  • Training hyperparameters = learning rate 1e-3, 50 epochs, batch size 64
    Hand-chosen to give satisfactory baselines; affects reported accuracies, though not the P2U mechanism itself.
assumptions (5)
  • domain assumption Uniform quantization with NNCodec and DeepCABAC entropy coding is applied to models and updates.
    Section 4.1.3; the reported sizes and accuracies depend on this coding stack and its configuration.
  • domain assumption The server has access to the model at arbitrary integer precisions or can quantize it on the fly.
    Section 3, step 3 assumes the repository can prepare the requested precision; not always true in practice.
  • domain assumption The receiver can run inference on the low-precision model and later apply the update without retraining.
    Steps 4 and 8 in Figure 1; needed for the progressive deployment claim.
  • domain assumption The update entropy coding is lossless, so the reconstructed proxy equals the high-precision model.
    The paper states 32-bit integers preserve the exact difference; if the coding is lossy, the proxy differs from the high-precision model.
  • standard math First-order Taylor expansion and smoothness of f are valid at the relevant operating point.
    Section 3.1 Eq. (3); used only for the theoretical bound, not load-bearing because the exact update makes W' = W_h by construction.

how reviews work

0 comments
Cite this review

Pith. "Pith review of P$^2$U: Progressive Precision Update For Efficient Model Distribution." pith.science (2026). https://pith.science/paper/J3UIE4UT

@misc{pith2026250622871,
  author       = {Pith},
  title        = {Pith review of: P$^2$U: Progressive Precision Update For Efficient Model Distribution},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/J3UIE4UT}},
  note         = {Machine review of arXiv:2506.22871}
}
abstract

Efficient model distribution is becoming increasingly critical in bandwidth-constrained environments. In this paper, we propose a simple yet effective approach called Progressive Precision Update (P$^2$U) to address this problem. Instead of transmitting the original high-precision model, P$^2$U transmits a lower-bit precision model, coupled with a model update representing the difference between the original high-precision model and the transmitted low precision version. With extensive experiments on various model architectures, ranging from small models ($1 - 6$ million parameters) to a large model (more than $100$ million parameters) and using three different data sets, e.g., chest X-Ray, PASCAL-VOC, and CIFAR-100, we demonstrate that P$^2$U consistently achieves better tradeoff between accuracy, bandwidth usage and latency. Moreover, we show that when bandwidth or startup time is the priority, aggressive quantization (e.g., 4-bit) can be used without severely compromising performance. These results establish P$^2$U as an effective and practical solution for scalable and efficient model distribution in low-resource settings, including federated learning, edge computing, and IoT deployments. Given that P$^2$U complements existing compression techniques and can be implemented alongside any compression method, e.g., sparsification, quantization, pruning, etc., the potential for improvement is even greater.

Figures

Figures reproduced from arXiv: 2506.22871 by the authors.

Figure 1
Figure 1. Workflow of P2U. After receiving the request for a ML model with a specific low-precision level from the edge device, the server first delivers the model with the requested precision level and then sends an update as the difference between the original (high-precision) model and the delivered low-precision model. 6. the server computes the model update as the difference between the original high-precision and transf… view at source ↗

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

42 extracted references · 38 canonical work pages

  1. [1]

    Very deep convolutional networks for large-scale image recognition,

    K. Simonyan and A. Zisserman, “Very deep convolutional networks for large-scale image recognition,” in 3rd International Conference on Learning Representations (ICLR) , 2015

    work page 2015

  2. [2]

    Sparse: Sparse architecture search for cnns on resource-constrained microcontrollers,

    I. Fedorov, R. P. Adams, M. Mattina, and P. Whatmough, “Sparse: Sparse architecture search for cnns on resource-constrained microcontrollers,” Advances in Neural Information Processing Systems (NeurIPS), vol. 32, 2019

    work page 2019

  3. [3]

    Sparse binary compression: Towards distributed deep learning with minimal communication,

    F. Sattler, S. Wiedemann, K.-R. M¨ uller, and W. Samek, “Sparse binary compression: Towards distributed deep learning with minimal communication,” IEEE International Joint Conference on Neural Networks (IJCNN) , pp. 1–8, 2019

    work page 2019

  4. [4]

    Gradient sparsification for communication-efficient distributed optimization,

    J. Wangni, J. Wang, J. Liu, and T. Zhang, “Gradient sparsification for communication-efficient distributed optimization,” Advances in Neural Information Processing Systems (NeurIPS) , vol. 31, 2018

    work page 2018

  5. [5]

    Compressing cnns using multilevel filter pruning for the edge nodes of multimedia internet of things,

    X. Liu, L. Wu, C. Dai, and H.-C. Chao, “Compressing cnns using multilevel filter pruning for the edge nodes of multimedia internet of things,” IEEE Internet of Things Journal , vol. 8, no. 14, pp. 11 041–11 051, 2021

    work page 2021

  6. [6]

    An efficient pruning scheme of deep neural networks for internet of things applications,

    C. Qi, S. Shen, R. Li, Z. Zhao, Q. Liu, J. Liang, and H. Zhang, “An efficient pruning scheme of deep neural networks for internet of things applications,” EURASIP Journal on Advances in Signal Processing, vol. 2021, no. 1, p. 31, 2021

    work page 2021

  7. [7]

    Stochastic binary-ternary quantization for communication efficient federated computation,

    R. Goutham, H. Afrabandpey, F. Cricri, H. Zhang, E. Aksu, M. Hannuksela, and H. R. Tavakoli, “Stochastic binary-ternary quantization for communication efficient federated computation,” IEEE International Conference on Image Processing (ICIP) , pp. 2097–2101, 2022

    work page 2022

  8. [8]

    Fixed-sign binary neural network: An efficient design of neural network for internet-of-things devices,

    Y. Li, Y. Bao, and W. Chen, “Fixed-sign binary neural network: An efficient design of neural network for internet-of-things devices,” IEEE Access, vol. 8, pp. 164 858–164 863, 2020

    work page 2020

Show all 42 references
  1. [9]

    Communication-efficient federated learning with binary neural networks,

    Y. Yang, Z. Zhang, and Q. Yang, “Communication-efficient federated learning with binary neural networks,” IEEE Journal on Selected Areas in Communications , vol. 39, no. 12, pp. 3836–3850, 2021

    2021

  2. [10]

    Deep compression: Compressing deep neural network with pruning, trained quantization and huffman coding,

    S. Han, H. Mao, and W. J. Dally, “Deep compression: Compressing deep neural network with pruning, trained quantization and huffman coding,” in Proceedings of the 4th International Conference on Learning Representations (ICLR), 2016

    2016

  3. [11]

    Deepcabac: A universal compression algorithm for deep neural networks,

    S. Wiedemann, H. Kirchhoffer, S. Matlage, P. Haase, A. Marban, T. Marinˇ c, D. Neumann, T. Nguyen, H. Schwarz, T. Wiegand, D. Marpe, and W. Samek, “Deepcabac: A universal compression algorithm for deep neural networks,” IEEE Journal of Selected Topics in Signal Processing , vo...

    2020

  4. [12]

    A white paper on neural network quantization,

    M. Nagel, M. Fournarakis, R. A. Amjad, Y. Bondarenko, M. Van Baalen, and T. Blankevoort, “A white paper on neural network quantization,” arXiv preprint arXiv:2106.08295 , 2021

    2021 arXiv

  5. [13]

    The quantization model of neural scaling,

    E. J. Michaud, Z. Liu, U. Girit, and M. Tegmark, “The quantization model of neural scaling,” Thirty-seventh Conference on Neural Information Processing Systems (NeurIPS) , 2023

    2023

  6. [14]

    Adabits: Neural network quantization with adaptive bit-widths,

    Q. Jin, L. Yang, and Z. Liao, “Adabits: Neural network quantization with adaptive bit-widths,” Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) , pp. 2146–2156, 2020

    2020

  7. [15]

    Towards accurate post-training network quantization via bit-split and stitching,

    P. Wang, Q. Chen, X. He, and J. Cheng, “Towards accurate post-training network quantization via bit-split and stitching,” International Conference on Machine Learning (ICML) , pp. 9847–9856, 2020

    2020

  8. [16]

    Progressive weight pruning of deep neural networks using admm,

    S. Ye, T. Zhang, K. Zhang, J. Li, K. Xu, Y. Yang, F. Yu, J. Tang, M. Fardad, S. Liuet al., “Progressive weight pruning of deep neural networks using admm,” arXiv preprint arXiv:1810.07378 , 2018

    2018 arXiv

  9. [17]

    Dynamic network surgery for efficient dnns,

    Y. Guo, A. Yao, and Y. Chen, “Dynamic network surgery for efficient dnns,” Advances in Neural Information Processing Systems (NeurIPS) , vol. 29, 2016

    2016

  10. [18]

    Woodfisher: Efficient second-order approximation for neural network compression,

    S. P. Singh and D. Alistarh, “Woodfisher: Efficient second-order approximation for neural network compression,” Advances in Neural Information Processing Systems (NeurIPS) , vol. 33, pp. 18 098– 18 109, 2020

    2020

  11. [19]

    Pruning neural networks without any data by iteratively conserving synaptic flow,

    H. Tanaka, D. Kunin, D. L. Yamins, and S. Ganguli, “Pruning neural networks without any data by iteratively conserving synaptic flow,” Advances in Neural Information Processing Systems (NeurIPS) , vol. 33, pp. 6377–6389, 2020

    2020

  12. [20]

    Hybrid pruning and sparsification,

    H. R. Tavakoli, J. Wabnig, F. Cricri, H. Zhang, E. Aksu, and I. Saniee, “Hybrid pruning and sparsification,” IEEE International Conference on Image Processing (ICIP) , pp. 3542–3546, 2021

    2021

  13. [21]

    Deep neural network compression by in-parallel pruning-quantization,

    F. Tung and G. Mori, “Deep neural network compression by in-parallel pruning-quantization,” IEEE Transactions on Pattern Analysis and Machine Intelligence , vol. 42, no. 3, pp. 568–579, 2018

    2018

  14. [22]

    Pruning by explaining: A novel criterion for deep neural network pruning,

    S.-K. Yeom, P. Seegerer, S. Lapuschkin, A. Binder, S. Wiedemann, K.-R. M¨ uller, and W. Samek, “Pruning by explaining: A novel criterion for deep neural network pruning,” Pattern Recognition, vol. 115, p. 107899, 2021

    2021

  15. [23]

    Joint matrix decomposition for deep convolutional neural networks compression,

    S. Chen, J. Zhou, W. Sun, and L. Huang, “Joint matrix decomposition for deep convolutional neural networks compression,” Neurocomputing, vol. 516, pp. 11–26, 2023

    2023

  16. [24]

    Deep convolutional neural network compression method: Tensor ring decomposition with variational bayesian approach,

    W. Liu, M. Zhang, C. Shi, N. Zhang, and J. Liu, “Deep convolutional neural network compression method: Tensor ring decomposition with variational bayesian approach,” Neural Processing Letters, vol. 56, no. 2, pp. 1–17, 2024

    2024

  17. [25]

    Sparse low rank factorization for deep neural network compression,

    S. Swaminathan, D. Garg, R. Kannan, and F. Andres, “Sparse low rank factorization for deep neural network compression,” Neurocomputing, vol. 398, pp. 185–196, 2020

    2020

  18. [26]

    Overview of the neural network compression and representation (nnr) standard,

    H. Kirchhoffer, P. Haase, W. Samek, K. M¨ uller, H. Rezazadegan-Tavakoli, F. Cricri, E. B. Aksu, M. M. Hannuksela, W. Jiang, W. Wang, S. Liu, S. Jain, S. Hamidi-Rad, F. Racap´ e, and W. Bailer, “Overview of the neural network compression and representation (nnr) standard,” IEE...

    2022

  19. [27]

    Model compression via distillation and quantization,

    A. Polino, R. Pascanu, and D. Alistarh, “Model compression via distillation and quantization,” in International Conference on Learning Representations (ICLR) , 2018

    2018

  20. [28]

    DistilBERT, a distilled version of bert: smaller, faster, cheaper and lighter

    V. Sanh, “DistilBERT, a distilled version of bert: smaller, faster, cheaper and lighter.” in Proceedings of Thirty-third Conference on Neural Information Processing Systems (NeurIPS) , 2019

    2019

  21. [29]

    Communication-efficient learning of deep networks from decentralized data,

    B. McMahan, E. Moore, D. Ramage, S. Hampson, and B. A. y Arcas, “Communication-efficient learning of deep networks from decentralized data,” Artificial intelligence and statistics(AISTATS) , pp. 1273–1282, 2017

    2017

  22. [30]

    Wireless federated distillation for distributed edge learning with heterogeneous data,

    J.-H. Ahn, O. Simeone, and J. Kang, “Wireless federated distillation for distributed edge learning with heterogeneous data,” IEEE 30th Annual International Symposium on Personal, Indoor and Mobile Radio Communications (PIMRC) , pp. 1–6, 2019. 12

    2019

  23. [31]

    Mix2fld: Downlink federated learning after uplink federated distillation with two-way mixup,

    S. Oh, J. Park, E. Jeong, H. Kim, M. Bennis, and S.-L. Kim, “Mix2fld: Downlink federated learning after uplink federated distillation with two-way mixup,” IEEE Communications Letters , vol. 24, no. 10, pp. 2211–2215, 2020

    2020

  24. [32]

    On the importance of temporal dependencies of weight updates in communication efficient federated learning,

    H. Afrabandpey, G. Rangu, H. Zhang, F. Criri, E. Aksu, and H. R. Tavakoli, “On the importance of temporal dependencies of weight updates in communication efficient federated learning,” IEEE International Conference on Visual Communications and Image Processing (VCIP) , pp. 1–5, 2022

    2022

  25. [33]

    Neural network coding of difference updates for efficient distributed learning communication,

    D. Becking, K. M¨ uller, P. Haase, H. Kirchhoffer, G. Tech, W. Samek, H. Schwarz, D. Marpe, and T. Wiegand, “Neural network coding of difference updates for efficient distributed learning communication,” IEEE Transactions on Multimedia , pp. 1–16, 2024

    2024

  26. [34]

    Quantized compressed sensing for communication-efficient federated learning,

    Y. Oh, N. Lee, and Y.-S. Jeon, “Quantized compressed sensing for communication-efficient federated learning,” in IEEE Globecom Workshops (GC Wkshps) , 2021, pp. 1–6

    2021

  27. [35]

    Distributed learning of deep neural network over multiple agents,

    O. Gupta and R. Raskar, “Distributed learning of deep neural network over multiple agents,” Journal of Network and Computer Applications , vol. 116, pp. 1–8, 2018

    2018

  28. [36]

    Traffic flow prediction with big data: A deep learning approach,

    Y. Lv, Y. Duan, W. Kang, Z. Li, and F.-Y. Wang, “Traffic flow prediction with big data: A deep learning approach,” IEEE Transactions on Intelligent Transportation Systems , vol. 16, no. 2, pp. 865–873, 2014

    2014

  29. [37]

    Large-scale transportation network congestion evolution prediction using deep learning theory,

    X. Ma, H. Yu, Y. Wang, and Y. Wang, “Large-scale transportation network congestion evolution prediction using deep learning theory,” PloS one , vol. 10, no. 3, p. e0119044, 2015

    2015

  30. [38]

    Large dataset of labeled optical coherence tomography (OCT) and chest x-ray images,

    D. Kermany, K. Zhang, and M. Goldbaum, “Large dataset of labeled optical coherence tomography (OCT) and chest x-ray images,” Mendeley Data , vol. 3, pp. 10–17 632, 2018

    2018

  31. [39]

    The PASCAL Visual Object Classes Challenge 2012 (VOC2012) Results,

    M. Everingham, L. Van Gool, C. K. I. Williams, J. Winn, and A. Zisserman, “The PASCAL Visual Object Classes Challenge 2012 (VOC2012) Results,” http://www.pascal- network.org/challenges/VOC/voc2012/workshop/index.html

    2012

  32. [40]

    Learning multiple layers of features from tiny images,

    A. Krizhevsky and G. Hinton, “Learning multiple layers of features from tiny images,” 2009

    2009

  33. [41]

    Nncodec: An open source software implementation of the neural network coding iso/iec standard,

    D. Becking, P. Haase, H. Kirchhoffer, K. M¨ uller, W. Samek, and D. Marpe, “Nncodec: An open source software implementation of the neural network coding iso/iec standard,” in ICML 2023 Workshop Neural Compression: From Information Theory to Applications , 2023

    2023

  34. [42]

    Overview of the neural network compression and representation (nnr) standard,

    H. Kirchhoffer, P. Haase, W. Samek, K. M¨ uller, H. Rezazadegan-Tavakoli, F. Cricri, E. B. Aksu, M. M. Hannuksela, W. Jiang, W. Wang et al. , “Overview of the neural network compression and representation (nnr) standard,” IEEE Transactions on Circuits and Systems for Video Tec...

    2021

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.