Pith. sign in

REVIEW 1 major objections 1 minor 118 cited by

A White Paper on Neural Network Quantization

T0 review · 1 major / 1 minor · reviewed 2026-05-17 · grok-4.3

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read Post-training quantization suffices for 8-bit neural networks with near floating-point accuracy in most cases.

desk verdict Practical consolidation of PTQ and QAT pipelines for common models, but the 8-bit accuracy claims still hinge on having suitable calibration data. read the letter →

arxiv 2106.08295 v1 pith:2PGW4AHI submitted 2021-06-15 cs.LG cs.AIcs.CV

Markus Nagel , Marios Fournarakis , Rana Ali Amjad , Yelysei Bondarenko , Mart van Baalen , Tijmen Blankevoort This is my paper · ORCID
classification cs.LGcs.AIcs.CV
keywords neuralnetworkquantizationpost-trainingquantization-awaretraininglow-bitinferencemodelcompressionedgeAIdeployment
checked against Cost.FunctionalEquationFoundation.DAlembert.Inevitability
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper establishes practical methods for quantizing neural network weights and activations to low bit widths while minimizing accuracy loss. It distinguishes post-training quantization, a lightweight approach that requires no retraining and works well for 8 bits, from quantization-aware training, which needs data and fine-tuning but supports lower precisions. Tested pipelines are provided that achieve strong results on common models. This matters because it reduces the power and latency of inference, making advanced networks viable on edge devices with limited resources. If the claims hold, quantization becomes a standard, low-effort optimization rather than a risky trade-off.

What carries the argument

Post-Training Quantization (PTQ) and Quantization-Aware Training (QAT) as two classes of algorithms that control the noise from reduced bit-width representations of weights and activations.

What would settle it

Applying the PTQ pipeline to a new model on a standard benchmark and observing accuracy significantly below the floating-point baseline.

Watch

Extended reading notes

Core claim

State-of-the-art algorithms for post-training quantization mitigate quantization noise sufficiently to reach close to floating-point accuracy at 8 bits without retraining or labeled data, while quantization-aware training enables competitive performance at lower bit widths when fine-tuning is applied.

Load-bearing premise

The described pipelines and algorithms will deliver the claimed accuracy on arbitrary models and tasks without model-specific adjustments or access to representative data.

Editorial extensions

If this is right

  • 8-bit quantization can be applied to most networks as a push-button step with minimal accuracy impact.
  • Lower bit-widths become accessible for tasks where some fine-tuning data is available.
  • Power consumption and inference latency decrease substantially for edge deployment.
  • Quantization noise can be managed without full model retraining in many scenarios.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • These methods might combine with other efficiency techniques like pruning to achieve even greater savings.
  • Hardware designers could optimize specifically for the bit widths these pipelines target.
  • Future models might be designed with quantization in mind from the start to maximize benefits.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Request a human review

A listed scientist reviews the paper for a fee and the review publishes here regardless of verdict. See the reviewers or get listed.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

1 major / 1 minor

Summary. The manuscript is a white paper on neural network quantization for reducing inference costs on edge devices. It provides a hardware-motivated introduction followed by coverage of two main approaches: Post-Training Quantization (PTQ), described as a lightweight push-button method requiring no re-training or labeled data that achieves close to floating-point accuracy for 8-bit quantization in most cases on common models, and Quantization-Aware Training (QAT), which requires fine-tuning with labeled data but supports lower bit-widths with competitive accuracy. The authors supply tested pipelines drawn from existing literature together with their own extensive experimentation.

Significance. If the experimental outcomes hold, the work offers clear practical value by consolidating state-of-the-art PTQ and QAT pipelines into accessible guidance for practitioners. The emphasis on both methods, combined with reported results on standard deep-learning models and tasks, positions the paper as a useful reference for deploying quantized networks under power and latency constraints.

major comments (1)
  1. [Abstract] Abstract: the claim that PTQ is sufficient for 8-bit quantization with close to floating-point accuracy 'in most cases' is central to the paper's practical message yet rests on unexamined details of calibration data selection and model-specific behavior; without explicit bounds or failure-mode characterization (e.g., for architectures exhibiting large activation outliers), the generality of the statement remains difficult to evaluate from the provided summary of results.
minor comments (1)
  1. [Abstract] Abstract: the statement that PTQ requires 'no re-training or labelled data' is technically correct for the core algorithm but could usefully note that a small unlabeled calibration set drawn from the target distribution is still needed to determine scales and zero-points in standard pipelines.

Simulated Author's Rebuttal

1 responses · 0 unresolved

Thank you for the constructive feedback on our white paper. We address the referee's major comment below and will revise the manuscript to improve precision.

read point-by-point responses

  1. Referee: [Abstract] Abstract: the claim that PTQ is sufficient for 8-bit quantization with close to floating-point accuracy 'in most cases' is central to the paper's practical message yet rests on unexamined details of calibration data selection and model-specific behavior; without explicit bounds or failure-mode characterization (e.g., for architectures exhibiting large activation outliers), the generality of the statement remains difficult to evaluate from the provided summary of results.

    Authors: We appreciate this observation regarding the scope of the abstract claim. The statement is grounded in the extensive experiments and tested pipelines detailed throughout the manuscript, which cover standard models (e.g., ResNets, MobileNets, BERT) on image and language tasks. In these cases, 8-bit PTQ with representative unlabeled calibration sets (typically 1000-5000 samples) yields accuracy within ~1% of floating-point baselines, as reported in the results sections. The PTQ section explicitly discusses calibration data selection and hardware-motivated choices such as per-channel quantization to handle activation ranges. We acknowledge that the abstract could more clearly bound the claim and note potential edge cases. In the revised version, we will update the abstract to qualify 'in most cases' as applying to the common models and tasks evaluated, and we will add a concise paragraph in the PTQ discussion characterizing observed failure modes (including sensitivity to large activation outliers in certain architectures) along with how the recommended pipelines address them. This strengthens the practical guidance without altering the core experimental findings. revision: yes

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity detected in derivation or claim chain

full rationale

The document is a white paper summarizing PTQ and QAT pipelines drawn from prior literature, with claims about 8-bit accuracy supported by referenced experimentation rather than any original mathematical derivation. No equations, fitted parameters, or self-referential definitions appear in the abstract or described structure; the central statements about PTQ sufficiency rest on external citations and reported results instead of reducing to inputs defined within the paper itself. Self-citations, if present, are not load-bearing for any uniqueness theorem or ansatz. The work is therefore self-contained against external benchmarks with no reduction by construction.

Assumptions & free parameters 0 free parameters · 1 assumptions · 0 invented entities

The central guidance rests on standard assumptions about quantization noise behavior in deep networks and the availability of representative calibration or training data; no new free parameters, axioms, or invented entities are introduced.

assumptions (1)
  • domain assumption Quantization noise in trained networks can be mitigated by the described PTQ and QAT procedures without violating typical convergence or generalization properties.
    Invoked when asserting that the pipelines achieve near floating-point accuracy at 8 bits for common models.

how reviews work

0 comments
Cite this review

Pith. "Pith review of A White Paper on Neural Network Quantization." pith.science (2026). https://pith.science/paper/2PGW4AHI

@misc{pith2026210608295,
  author       = {Pith},
  title        = {Pith review of: A White Paper on Neural Network Quantization},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/2PGW4AHI}},
  note         = {Machine review of arXiv:2106.08295}
}
read the original abstract

While neural networks have advanced the frontiers in many applications, they often come at a high computational cost. Reducing the power and latency of neural network inference is key if we want to integrate modern networks into edge devices with strict power and compute requirements. Neural network quantization is one of the most effective ways of achieving these savings but the additional noise it induces can lead to accuracy degradation. In this white paper, we introduce state-of-the-art algorithms for mitigating the impact of quantization noise on the network's performance while maintaining low-bit weights and activations. We start with a hardware motivated introduction to quantization and then consider two main classes of algorithms: Post-Training Quantization (PTQ) and Quantization-Aware-Training (QAT). PTQ requires no re-training or labelled data and is thus a lightweight push-button approach to quantization. In most cases, PTQ is sufficient for achieving 8-bit quantization with close to floating-point accuracy. QAT requires fine-tuning and access to labeled training data but enables lower bit quantization with competitive results. For both solutions, we provide tested pipelines based on existing literature and extensive experimentation that lead to state-of-the-art performance for common deep learning models and tasks.

Discussion (0). Continue with ORCID to comment.

Lean theorems connected to this paper

Citations machine-checked in the Pith Canon. Every link opens the source theorem in the public Lean library.

  • Cost.FunctionalEquation washburn_uniqueness_aczel unclear
    ?
    unclear

    Relation between the paper passage and the cited Recognition theorem.

    PTQ requires no re-training or labelled data and is thus a lightweight push-button approach to quantization. In most cases, PTQ is sufficient for achieving 8-bit quantization with close to floating-point accuracy.

What do these tags mean?
matches
The paper's claim is directly supported by a theorem in the formal canon.
supports
The theorem supports part of the paper's argument, but the paper may add assumptions or extra steps.
extends
The paper goes beyond the formal theorem; the theorem is a base layer rather than the whole result.
uses
The paper appears to rely on the theorem as machinery.
contradicts
The paper's claim conflicts with a theorem or certificate in the canon.
unclear
Pith found a possible connection, but the passage is too broad, indirect, or ambiguous to say the theorem truly supports the claim.

Forward citations

Showing 60 of 118 Pith papers that cite this

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. See all 118 Pith citations

  1. Neural Network Quantization by Learning Low-Loss Subspaces

    cs.CV 2026-06 unverdicted novelty 7.0 of 10

    Learning quantization-aware linear paths in weight space yields a midpoint whose direct quantization matches quantization-aware training performance without using straight-through estimators.

  2. JOMP: Jointly-Optimized Mixed-Precision Quantization Across Neural Video Coding Frameworks and Buffering Strategies

    eess.IV 2026-06 unverdicted novelty 7.0 of 10

    JOMP jointly optimizes mixed-precision quantization parameters and bit widths across neural video coding frameworks, achieving rate-distortion performance comparable to DCVC-FM while cutting bit operations by 87.6%.

  3. Q-ARVD: Quantizing Autoregressive Video Diffusion Models

    cs.CV 2026-05 unverdicted novelty 7.0 of 10

    Q-ARVD introduces final-quality-aware frame weighting and outlier-aware adaptive dual-scale quantization to enable accurate low-bit inference for autoregressive video diffusion models.

  4. Characterizing Learning in Deep Neural Networks using Tractable Algorithmic Complexity Analysis

    cs.LG 2026-05 unverdicted novelty 7.0 of 10

    QuBD extends algorithmic complexity estimation to quantized DNN weights, revealing that complexity decreases during learning, increases with overfitting, follows grokking patterns, and correlates with generalization.

  5. DPQuant: Efficient and Differentially-Private Model Training via Dynamic Quantization Scheduling

    cs.LG 2025-09 unverdicted novelty 7.0 of 10

    DPQuant uses epoch-wise probabilistic layer rotation and DP loss sensitivity to quantize only a changing subset of layers, reducing accuracy degradation from quantization noise in DP-SGD and delivering up to 2.21x thr...

  6. Quantization Meets OOD: Generalizable Quantization-aware Training from a Flatness Perspective

    cs.CV 2025-08 conditional novelty 7.0 of 10

    Quantization-aware training degrades out-of-distribution accuracy, and a flatness-aware method with gradient-disorder freezing, FQAT, partially recovers it.

  7. FPTQuant: Function-Preserving Transforms for LLM Quantization

    cs.LG 2025-06 conditional novelty 7.0 of 10

    FPTQuant introduces function-preserving transforms that make transformer activations amenable to static 4-bit quantization with minimal inference overhead.

  8. When Do Neural Networks Learn World Models?

    cs.LG 2025-02 conditional novelty 7.0 of 10

    With Boolean variables, a low-degree bias, and a task distribution weighted toward simple functions of the latents, multi-task training provably recovers the latent world model up to permutations and negations.

  9. Behavior Backdoor for Deep Learning Models

    cs.LG 2024-12 conditional novelty 7.0 of 10

    A new backdoor attack keeps a model normal until it is quantized, after which it flips to a chosen target prediction for all inputs.

  10. Reclaiming Residual Knowledge: A Novel Paradigm to Low-Bit Quantization

    cs.CV 2024-08 unverdicted novelty 7.0 of 10

    CoRa reclaims quantization residuals in pre-trained ConvNets by searching low-rank adapter architectures instead of weights, matching SOTA accuracy on ImageNet in 3-4 bit settings with under 250 iterations on 1600 images.

  11. A Theoretical Framework for Stochastic Activity Prediction in Tensor Accelerator Wallace-Tree Multipliers

    cs.AR 2026-07 reject novelty 6.5 of 10

    SAP predicts Wallace-tree switching from a one-bit Bernoulli proxy of operand Hamming weight and freezes inputs under a deterministic safety controller, with Lipschitz, information-retention, and optimality theorems.

  12. FORGE: Forward-Only Test-Time Adaptation for Integer-Only Vision Models on Microcontrollers

    cs.CV 2026-09 accept novelty 6.0 of 10

    FORGE restores adaptation on folded integer-only convolutional networks via per-channel recalibration to clean training statistics, recovering most of gradient-based TENT's accuracy gain.

  13. A Closed-Loop Evaluation of Capability Loss and Recovery in Compressed Driving Policies

    cs.AI 2026-09 conditional novelty 6.0 of 10

    A stage-by-stage driving test shows pruning first destroys an AI driver's skill, distillation recovers it only with balanced rehearsal data, and INT8 quantization then re-breaks stop-and-go behavior that FP16 preserves.

  14. A Layer Importance Metric for Quantization Accounting for the Speed-Quality Trade-off in Autoregressive Models

    cs.LG 2026-08 conditional novelty 6.0 of 10

    A new metric combining SQNR-based quality and roofline-based speed scores allows analytical prioritization of layers for quantization in small autoregressive language models.

  15. FAMPWQ: Fisher Information-based Adaptive Mixed Precision Weight Quantization for Effective LLM Inference

    cs.LG 2026-08 conditional novelty 6.0 of 10

    FAMPWQ uses quantization-perturbed Fisher information and a PPO-based allocator to produce mixed-precision weight quantization that outperforms uniform and existing mixed-precision baselines at 3-4 average bits.

  16. KV Cache Compression Through the Lens of Transform Coding

    cs.LG 2026-08 conditional novelty 6.0 of 10

    An attention-aware transform-coding bit allocation keeps KV cache compression within evaluation noise of FP16 at about 5.8x on Llama-3.1-8B and Qwen-2.5-7B.

  17. Post-training Quantization for Hybrid Iterative Generative Models

    cs.LG 2026-08 conditional novelty 6.0 of 10

    HyGenQ quantizes hybrid autoregressive-diffusion image generators to W8A8 without retraining, avoiding model collapse by decoupling outlier channels and rescaling amplified anomalies.

  18. Calibration Bets on the Past: Post-Training Quantization for Financial Time-Series Forecasting

    cs.LG 2026-08 conditional novelty 6.0 of 10

    Static 4-bit activation quantization can remove 11 to 62 percent of a volatility model's ranking skill under default abs-max calibration, and the damage depends strongly on how activation ranges are calibrated.

  19. Achieving Near-Zero-Overhead Multi-Model Hierarchical Classification in Real-Time Detection Pipelines

    cs.CV 2026-08 conditional novelty 6.0 of 10

    A documented methodology lets custom INT8 classifiers run entirely on Jetson DLA cores in parallel with GPU detection, adding only about six percent pipeline overhead.

  20. SQuaT: Self-Supervised Knowledge Distillation via Student-Aware Quantized Teacher Features

    cs.LG 2026-08 conditional novelty 6.0 of 10

    SQuaT projects teacher features onto the student's quantization lattice to eliminate the unattainable-target lower bound in label-free QAT distillation.

  21. Hadamard-Domain Model Quantization for Learned Image Coding

    eess.IV 2026-08 conditional novelty 6.0 of 10

    Uniform INT8 quantization of learned image codecs becomes more accurate when tensors are first re-expressed in a layer-adaptive Hadamard basis.

  22. 3D Lane Detection with Odometry for High-Speed Vehicle Racing

    cs.CV 2026-07 conditional novelty 6.0 of 10

    Fusing multi-camera lane predictions with odometry pre-integration improves 3D lane detection on a new racing dataset, reaching F1 > 0.9 at nearly 300 Hz.

  23. Moving Like a Human: Ego-Motion-Normalized Temporal Signatures for Real-Time Aerial Person Tracking on Milliwatt-Class Hardware

    cs.CV 2026-07 conditional novelty 6.0 of 10

    A 22k-parameter, 7.6-MFLOP stateless detector fed ego-motion-normalized residual-motion channels tracks small aerial persons at 31.85 FPS on a Pi Zero 2W, beating YOLOv8n by 2.7x AP25 with 16x lower latency.

  24. Voltron: Enabling Elastic Multi-Device Execution of LLM Inference for Empowered Edge Intelligence

    cs.DC 2026-07 conditional novelty 6.0 of 10

    A framework called Voltron elastically distributes LLM inference across heterogeneous edge devices using layer-wise hybrid parallelism and mixed precision, achieving up to 16.5% higher accuracy than single-device exec...

  25. Lyapunov-Guided Training for Hardware-Safe Neural Networks Under Fixed-Point Arithmetic

    cs.LG 2026-07 conditional novelty 6.0 of 10

    Monotone Lyapunov projection of layerwise hidden-state energy suppresses two's-complement overflow under wrapping fixed-point QAT/PTQ, recovering 86.55% MNIST accuracy where unconstrained models collapse to chance.

  26. QuantGuard: Learnable Rounding for Repairing Quantization-Conditioned Backdoors in LLMs

    cs.CR 2026-06 unverdicted novelty 6.0 of 10

    QuantGuard uses differentiable rounding control to break quantization-boundary backdoors in LLMs, reducing post-quantization attack success to clean-model levels across six models and INT8/FP4/NF4.

  27. What Survives When You Compress a Recursive Reasoner for the Edge?

    cs.LG 2026-06 unverdicted novelty 6.0 of 10

    Aggressive compression of recursive reasoners keeps local predictions intact but destroys global reasoning accuracy, recoverable with calibrated INT4 and detectable via carry-trajectory fidelity.

  28. ScalePredictor: Instance-aware Scale Learning for Accurate Quantization of Vision Transformers

    cs.CV 2026-06 unverdicted novelty 6.0 of 10

    ScalePredictor is a dynamic post-training quantization framework for Vision Transformers that predicts per-instance scales from shallow-layer activation ranges using a Taylor-motivated polynomial projection module.

  29. Smoothness-Based Derandomization of PAC-Bayes Bounds

    cs.LG 2026-06 unverdicted novelty 6.0 of 10

    Derives smoothness-based PAC-Bayes derandomization bounds for deterministic predictors using Rademacher complexity of the Jensen gap class, yielding Jacobian/Hessian flatness terms and a practical regularizer tested o...

  30. AI Level of Detail: Distance-Aware ML Model Precision Selection for Real-Time Human Motion Prediction in Games

    cs.GR 2026-06 unverdicted novelty 6.0 of 10

    AI LOD uses distance to select quantized precision levels of ML motion models for efficient NPC animation in games.

  31. Beyond Output Matching: Preserving Internal Geometry in NVFP4 LLM Distillation

    cs.AI 2026-06 unverdicted novelty 6.0 of 10

    CKA-guided alignment of layerwise Gram matrices during NVFP4 QAD preserves internal representational geometry and improves downstream accuracy beyond KL-only output matching.

  32. OScaR: The Occam's Razor for Extreme KV Cache Quantization in LLMs and Beyond

    cs.LG 2026-05 unverdicted novelty 6.0 of 10

    OScaR mitigates token norm imbalance via canalized rotation and omni-token scaling to enable near-lossless INT2 KV cache quantization with up to 3x decoding speedup and 5.3x memory reduction.

  33. You Had One Job: Per-Task Quantization Using LLMs' Hidden Representations

    cs.CL 2025-11 reject novelty 6.0 of 10

    TAQ estimates per-layer importance from hidden representations and output sensitivity on task calibration data to allocate mixed precision in a training-free PTQ setting, outperforming task-agnostic baselines on accur...

  34. SQAP-VLA: A Synergistic Quantization-Aware Pruning Framework for High-Performance Vision-Language-Action Models

    cs.CV 2025-09 conditional novelty 6.0 of 10

    A training-free framework jointly quantizes a VLA model to 4 bits and prunes visual tokens, recovering or exceeding full-precision success rates at 1.93x speedup.

  35. Cache Management for Mixture-of-Experts LLMs -- extended version

    cs.LG 2025-09 conditional novelty 6.0 of 10

    The paper formalizes expert caching in MoE LLMs as layered paging, proves near-tight competitive lower bounds, and shows a layer-aware LRU variant beats standard LRU on real traces.

  36. HAP: Hybrid Adaptive Parallelism for Efficient Mixture-of-Experts Inference

    cs.DC 2025-08 conditional novelty 6.0 of 10

    HAP uses ILP over module-specific hybrid parallel choices to speed up MoE inference, reporting up to 1.77x versus tensor parallelism on tested GPUs.

  37. Systematic Characterization of LLM Quantization: A Performance, Energy, and Quality Perspective

    cs.PF 2025-08 conditional novelty 6.0 of 10

    No single LLM quantization method dominates performance, energy, and quality; the best choice depends on task, request length, load, parallelism, and GPU type.

  38. eMamba: Efficient Acceleration Framework for Mamba Models in Edge Computing

    cs.LG 2025-08 conditional novelty 6.0 of 10

    An end-to-end Mamba edge accelerator using hardware-friendly approximations, INT8 quantization, and NAS achieves 4.95x-5.62x lower latency and 1.63x-19.9x smaller models than ViT/CNN baselines.

  39. Reliable Evaluation Protocol for Low-Precision Retrieval

    cs.IR 2025-08 unverdicted novelty 6.0 of 10

    Proposes High-Precision Scoring (HPS) and Tie-aware Retrieval Metrics (TRM) to reduce tie-induced instability in low-precision retrieval evaluation.

  40. Task-Specific Zero-shot Quantization-Aware Training for Object Detection

    cs.CV 2025-07 conditional novelty 6.0 of 10

    A zero-shot quantization-aware training method for object detectors that synthesizes task-specific images with bounding-box labels via adaptive label sampling, then distills task-specific knowledge into the quantized network.

  41. DMQ: Dissecting Outliers of Diffusion Models for Post-Training Quantization

    cs.CV 2025-07 conditional novelty 6.0 of 10

    A post-training quantization method that combines learned channel scaling and power-of-two scaling keeps diffusion image quality high at 4-bit weight, 6-bit activation precision.

  42. Post-Training Quantization of Generative and Discriminative LSTM Text Classifiers: A Study of Calibration, Class Balance, and Robustness

    cs.LG 2025-07 conditional novelty 6.0 of 10

    Generative LSTM classifiers under post-training quantization are far more sensitive than discriminative ones to calibration data class balance and input noise, especially at 3- to 5-bit widths.

  43. Can Quantized Audio Language Models Perform Zero-Shot Spoofing Detection?

    cs.SD 2025-06 conditional novelty 6.0 of 10

    Zero-shot audio-language models are not reliable spoof detectors because they over-predict 'spoof', and FP16 quantization keeps this bias while INT8 worsens it.

  44. Assigning Distinct Roles to Quantized and Low-Rank Matrices Toward Optimal Weight Decomposition

    cs.LG 2025-06 conditional novelty 6.0 of 10

    ODLRI initializes the low-rank component using activation-outlier channels, improving low-bit compression of large language models over the CALDERA baseline.

  45. Chameleon: A Multiplier-Free Temporal Convolutional Network Accelerator for End-to-End Few-Shot and Continual Learning from Sequential Data

    cs.AR 2025-05 conditional novelty 6.0 of 10

    Chameleon is a 40-nm CMOS accelerator that performs end-to-end few-shot and continual learning on-chip using prototypical networks and TCN embeddings, and runs keyword spotting at 3.1 uW.

  46. Automatic mixed precision for optimizing gained time with constrained loss mean-squared-error based on model partition to sequential sub-graphs

    cs.LG 2025-05 conditional novelty 6.0 of 10

    The paper derives an additive loss-MSE sensitivity metric and a hardware-aware time-gain model, then uses integer programming to assign per-layer FP8 or BF16 formats for LLM inference.

  47. A probabilistic framework for dynamic quantization

    cs.LG 2025-05 conditional novelty 6.0 of 10

    An input-adaptive 8-bit quantization scheme estimates activation ranges via a lightweight probabilistic surrogate, achieving near-dynamic accuracy with static-like memory overhead.

  48. LightNobel: Improving Sequence Length Limitation in Protein Structure Prediction Model via Adaptive Activation Quantization

    cs.AR 2025-05 conditional novelty 6.0 of 10

    LightNobel combines token-wise adaptive activation quantization with a custom accelerator to run protein structure prediction on sequences up to about 10,000 amino acids within 80GB memory.

  49. Sampling-Aware Quantization for Diffusion Models

    cs.CV 2025-05 unverdicted novelty 6.0 of 10

    A quantization technique for diffusion models that aligns sampling trajectories to preserve high-order sampler performance under quantization noise.

  50. FGMP: Fine-Grained Mixed-Precision Weight and Activation Quantization for Hardware-Accelerated LLM Inference

    cs.AR 2025-04 conditional novelty 6.0 of 10

    A Fisher-weighted, block-level mixed-precision quantization method plus a custom mixed-precision datapath runs 70% of Llama-2-7B weight and activation blocks in FP4 with under 1% perplexity loss versus all-FP8.

  51. Membership Inference Risks in Quantized Models: A Theoretical and Empirical Study

    stat.ML 2025-02 conditional novelty 6.0 of 10

    Quantizers can be ranked by privacy using r_Q, a rate constant built from the loss gap and variance of low-loss quantized checkpoints along the training trajectory.

  52. TQ-DiT: Efficient Time-Aware Quantization for Diffusion Transformers

    cs.LG 2025-02 conditional novelty 6.0 of 10

    TQ-DiT quantizes diffusion transformers to 8-bit and 6-bit precision by grouping timesteps and splitting activations into multiple regions, reporting FID 4.91 at W8A8 and 8.58 at W6A6 on ImageNet 256x256 DiT-XL-2.

  53. Quark: Implementing Convolutional Neural Networks Entirely on Programmable Data Plane

    cs.NI 2025-01 conditional novelty 6.0 of 10

    A pruned, 7-bit quantized CNN was deployed entirely on an Intel Tofino programmable switch, achieving 97.3% anomaly detection accuracy, 42.66us latency, and near line-rate throughput.

  54. Coded Deep Learning: Framework and Algorithm

    cs.LG 2025-01 conditional novelty 6.0 of 10

    A from-scratch quantization-aware training framework that jointly minimizes loss and the entropy of probabilistically quantized weights/activations, achieving 2-3 bits per weight with accuracy close to full precision.

  55. FlexQuant: Elastic Quantization Framework for Locally Hosted LLM on Edge Devices

    cs.AI 2025-01 conditional novelty 6.0 of 10

    FlexQuant generates a family of shared-parameter quantized LLMs by gradually replacing modules with lower-bit versions, cutting storage and improving memory granularity.

  56. Dissecting Bit-Level Scaling Laws in Quantizing Vision Generative Models

    cs.CV 2025-01 conditional novelty 6.0 of 10

    Token-based language-style vision models (VAR, LlamaGen) tolerate quantization better than diffusion models, and a custom TopKLD distillation loss pushes their low-bit scaling roughly one precision level higher.

  57. PTQ4VM: Post-Training Quantization for Visual Mamba

    cs.CV 2024-12 conditional novelty 6.0 of 10

    PTQ4VM quantizes Visual Mamba models to 8-bit and 6-bit with small accuracy loss by adding per-token static quantization and joint learning of smoothing scales and step sizes.

  58. 1.58-bit FLUX

    cs.CV 2024-12 reject novelty 6.0 of 10

    A post-training method reduces 99.5% of FLUX.1-dev's transformer weights to ternary values and reports roughly comparable text-to-image quality with large storage and memory savings.

  59. PTSBench: A Comprehensive Post-Training Sparsity Benchmark Towards Algorithms and Models

    cs.LG 2024-12 conditional novelty 6.0 of 10

    PTSBench benchmarks post-training sparsity techniques and model families, finding learning-based allocation and block-wise reconstruction most effective, and attention-based models most sparsity-friendly.

  60. GAQAT: gradient-adaptive quantization-aware training for domain generalization

    cs.CV 2024-12 conditional novelty 6.0 of 10

    A gradient-disorder trigger that selectively freezes task gradients of quantizer scale factors improves quantized domain-generalization accuracy, including near-lossless 4-bit results on DomainNet.

See all 118 Pith citations

Pith tools

Reviewed May 17, 2026 · model on record in the stance chip above.