Pith. sign in

REVIEW 4 major objections 7 minor 48 references

RWKVQuant: Quantizing the RWKV Family with Proxy Guided Hybrid of Scalar and Vector Quantization

T0 review · 4 major / 7 minor · reviewed 2026-08-16 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read RWKVQuant proposes a hybrid scalar/vector post-training quantization scheme that puts RWKV-6-14B at 3.275 bits per weight with less than 1% zero-shot accuracy loss and 2.14x faster generation.

desk verdict A useful PTQ recipe for RWKV that mostly delivers on its claims, with a real but fixable calibration-protocol weakness around the headline bitrate. read the letter →

arxiv 2505.03803 v1 pith:XTV63DAC submitted 2025-05-02 cs.LG cs.AI

Chen Xu , Yuxuan Yue , Zukang Xu , Xing Hu , Jiangyong Yu , Zhixuan Chen , Sifan Zhou , Zhihang Yuan
show 1 more author
Dawei Yang
This is my paper · ORCID
classification cs.LGcs.AI
keywords post-trainingquantizationRWKVscalarvectorhybridcodebookoptimizationLLMcompressioninformationentropy
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

RWKVQuant claims that RWKV models fail under standard Transformer post-training quantization for two structural reasons: their non-linear operators block the parameter fusion used by smooth- and rotation-based scalar quantizers, and their unusually uniform weight distributions defeat cluster-based vector quantizers. The paper proposes a per-layer hybrid that routes each weight tensor to scalar quantization (GPTQ-style compensation) or vector quantization (GPTVQ-style codebooks), using a coarse information-entropy proxy for overall uniformity and a fine proxy built from weighted higher-order central moments for local outliers. For RWKV's element-wise multiplication layers, it adds an activation-weighted codebook optimization that directly minimizes the output error. If the method is right, RWKV-6-14B can run at 3.275 bits per weight with under 1% zero-shot accuracy loss and a 2.14x generation speedup, which is the concrete promise for deploying RWKV on memory-limited devices.

What carries the argument

The load-bearing object is the coarse-to-fine proxy: weights are flattened, sorted, and converted to adjacent gaps $G'$; the coarse proxy is the information-entropy gap $P_c = H(\hat G') - H(G')$ between uniform gaps and observed gaps, and the fine proxy is $P_f = \sum_{k=2}^K v_k |M_k|$, a weighted sum of higher-order central moments of $G'$ that magnifies local outliers. A layer gets scalar quantization only if $P_c < \tau_c$ and $P_f < \tau_f$; otherwise it gets vector quantization. The second mechanism is the element-wise codebook optimizer, a weighted K-means where the squared calibration activations $X^2$ weight each element, with percentile clipping before averaging over batches, so codebooks minimize $\|X \odot \mu - X \odot \mathrm{Deq}(Q(\mu))\|_F^2$ rather than unweighted weight error.

What would settle it

Use the thresholds the paper reports for RWKV-7 ($\tau_c=1.54$, $\tau_f=30$) on RWKV-6-3B and check both the layer split and LAMBADA perplexity; if the split drifts from roughly 9:1 or perplexity rises by more than the paper's reported margin, the proxy has not isolated a transferable property of RWKV weights.

Watch

Extended reading notes

Core claim

The central discovery is that no single existing quantizer fits RWKV, but the two families compensate each other. Smooth- and rotation-based scalar methods (GPTQ, AWQ, QuaRot) fail because RWKV's token-shift, sigmoid, and exponential operators sit in the fusion path, so the extra scaling or rotation parameters cannot be folded away and instead add computation. Cluster-based vector methods (K-Means, GPTVQ, VPTQ) fail because RWKV weights have a larger fraction of uniformly distributed values, which K-means represents poorly. RWKVQuant's answer is to decide per layer: when both a coarse entropy-based uniformity test and a fine weighted-moment outlier test come out low, scalar quantization is safe; otherwise vector quantization. On top of that, the codebook is fit with squared calibration activations as weights and percentile-clipped batch averaging, so the element-wise multiplication layers' actual output error is minimized. The paper reports that this hybrid beats both pure scalar and pure vector baselines across seven RWKV-6/7 language models and six VRWKV vision models, including the headline 3.275-bpw result on RWKV-6-14B.

Load-bearing premise

The central assumption is that the two cutoff values that pick scalar versus vector quantization for each layer carry over from one RWKV model to another, even though the paper tunes them per model to force a fixed 9:1 split and its own appendix warns that this split may be arbitrary.

Editorial extensions

If this is right

  • A near-lossless 3.275-bit weight-only PTQ becomes possible across the RWKV family, not just on one model size: the same recipe holds for RWKV-7 0.1B-1.47B, RWKV-6 1B-14B, and VRWKV vision models.
  • The reported uniformity gap (about 60% of RWKV layers prefer scalar quantization versus about 10% for LLaMA) gives a measurable architectural signature that explains why Transformer PTQ transfers poorly to RWKV and where to look in other linear-RNN hybrids.
  • Activation-weighted codebook fitting improves LAMBADA perplexity and zero-shot accuracy on every RWKV model tested, so RWKV-style element-wise multiplication layers should not be quantized with unweighted K-means codebooks.
  • For an edge-deployment engineer, the concrete operating point is RWKV-6-14B at 3.275 bpw: memory drops from 26.07 GB to 9.21 GB and generation speed rises from 16.02 to 34.32 tokens/s on an A6000.
  • Because the proxy itself only inspects the weights, the per-layer SQ/VQ decision can be recomputed in O(M) time for each layer without running the model, which keeps the method cheap enough for many model sizes.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The fixed 9:1 split is a knob, not a law: an automatic search over the threshold values would likely find a different allocation that trades bitrate against accuracy, so the 3.275 bpw number is better read as one point on a curve than as the method's natural operating point.
  • Applying the same proxy at channel or block granularity instead of whole layers, which the paper lists as future work, should recover either higher accuracy or a lower bitrate on the same models.
  • If the uniformity difference between RWKV and LLaMA is causal rather than incidental, then other architectures that rely on element-wise weighting or linear recurrences may show the same pattern, and the hybrid recipe would be a natural first thing to try on them.
  • A sensitivity test on calibration set size for the percentile-clipped weighted codebook would be the quickest way to see whether the reported accuracy depends on the exact 128-sample calibration choice.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 7 minor

Summary. This manuscript introduces RWKVQuant, a post-training quantization framework for RWKV-family models. The method combines scalar quantization (GPTQ-style) and vector quantization (GPTVQ-style) at the layer level, guided by a two-stage proxy: a coarse entropy-based uniformity measure and a fine-grained central-moment measure for detecting local outliers. A second contribution is a codebook-optimization scheme for element-wise multiplication layers, using activation-weighted KMeans with percentile clipping during batch integration. Experiments on RWKV-6, RWKV-7, and VRWKV models claim 3.275 bpw weight-only quantization with less than 1% average zero-shot accuracy loss and a 2.14x speedup on RWKV-6-14B, with consistent gains over both scalar-only and vector-only baselines.

Significance. If the empirical claims held, this would be a useful contribution to PTQ outside the Transformer family. The paper identifies a genuine structural obstacle—nonlinear operators in RWKV block fusion-based SQ—and a distributional obstacle—uniform weights degrade cluster-based VQ—and the proposed hybrid proxy is simple, interpretable, and has O(M) cost. The Taylor-expansion derivation of the fine-grained proxy is mathematically sound as a heuristic, and the element-wise codebook optimization is a principled adaptation that the ablations support. However, the current empirical evidence is compromised by test-set calibration, threshold selection on evaluation metrics, and multiple inconsistencies in the main results tables. The significance of the paper is therefore conditional on a properly controlled evaluation that separates calibration and threshold selection from test metrics.

major comments (4)
  1. [Section 4.1, Appendix A.5, Table 12] Section 4.1 states that 128 samples from the corresponding test datasets are used for calibration and that tau_c and tau_f are dynamically set per model to force a 9:1 split between 3.25-bpw SQ layers and 3.5-bpw VQ layers. Appendix A.5 describes these thresholds as empirically set, and Table 12 sweeps tau_c and tau_f while reporting the same zero-shot and LAMBADA metrics used in Table 2. This means the headline 3.275 bpw is fixed by construction rather than determined by the proxy, and the reported accuracy is obtained under a protocol in which the evaluation benchmarks influence both calibration and hyperparameter selection. The claimed <1% accuracy loss and family-wide gains are not established under a held-out protocol; the authors should use a disjoint calibration/validation split and specify the threshold-selection rule before any test-set evaluation.
  2. [Table 2 vs. Tables 9 and 10] The main results table contains entries that do not match the appendix tables. For example, for RWKV7-0.5B at 3.25 bpw, Table 2 lists GPTQ as (0-shot9=41.16, LAMBADA PPL=23.29), but Table 9 lists GPTQ 3.25 as (avg=43.69, Wiki2=15.97) and AWQ 3.25 as (avg=41.16, Wiki2=23.29); the Table 2 AWQ entry for the same model, (68.92, 5.92), appears in neither Table 9 nor Table 10. Because Table 2 is the primary evidence for the hybrid method's superiority, these inconsistencies must be resolved and the table regenerated from the same run logs used for the appendix tables.
  3. [Table 5 vs. Table 2] The ablation table reports a GPTVQ zero-shot accuracy of 48.29 for RWKV6-7B, whereas Table 2 reports 58.57 at 3.25 bpw and 59.70 at 3.5 bpw for the same model and method; similar mismatches appear for RWKV7-0.5B GPTQ (45.36 in Table 5 vs. 45.73 in Table 2). Since the ablation is intended to show that the hybrid beats each single method at a matched bitrate, these numbers must come from the same evaluation pipeline. As printed, the conclusion that hybrid quantization is responsible for the observed gain is unsupported.
  4. [Section 4.2] The text states that on 'larger models such as RWKV6-7B and RWKV7-14B' the method results in almost no increase in perplexity and less than one point accuracy drop, but RWKV7-14B does not appear in Table 2 and no results for it are reported anywhere in the paper. Either provide the RWKV7-14B experiments or correct the sentence to refer to the models actually evaluated.
minor comments (7)
  1. [Section 1] There are several typos, including 'Scaler Quanzization' and 'a an essential technique'; these should be corrected.
  2. [Table 6 heading] The heading uses 'RWK7' instead of 'RWKV7'.
  3. [Figure 3 caption] The caption refers to thresholds as mu_c and mu_f, while the text and equations use tau_c and tau_f; please unify the notation.
  4. [Table 8 heading] The header appears corrupted: it contains duplicated 'RWKV6-T' columns and an incomplete model list, and the corresponding model names in the body do not line up with the columns.
  5. [Appendix A.5] The sentence 'ensuring that the 3.25 bpw SQ proportion is approximately one-tenth and nine-tenths for 3.5 bpw VQ' is ambiguous and appears inverted relative to the 9:1 SQ:VQ ratio stated in Section 4.1.
  6. [Section 4.3] The sentence 'For fairness, the weights of all multiplication operations are quantized using the RTN method' is unclear in context, since the ablation compares GPTQ, GPTVQ, and the hybrid; please clarify what is held fixed across the compared configurations.
  7. [Tables 9 and 10] The appendix row for RWKV7-0.5B at 3.5 bpw RTN appears corrupted, with values (18.13, 19.36, 43.47, ...) that look like a shifted copy from another row; please regenerate the affected rows.

Circularity Check

1 steps flagged · score 4.0 of 10

The reported 3.275-bpw bitrate is enforced by per-model threshold tuning, so the headline bitrate is an input constraint; the accuracy and ablation claims retain independent content.

  1. self definitional [Section 4.1 (Experimental Settings); Appendix A.5 (Limitations and Future Work); Abstract headline claim]
    "In our method, we dynamically setτc andτf according to different models, ensuring that SQ with a bpw of 3.25 is used in nine-tenths of the layers, while VQ with a bpw of 3.5 is used in one-tenth."

    The reported 3.275 bpw is the weighted average of the two forced bit-widths: 0.9 × 3.25 + 0.1 × 3.5 = 3.275. Because τc and τf are 'dynamically set' per model to produce exactly this 9:1 SQ/VQ split, the proxy cannot determine the allocation or bitrate from the weight statistics; the target bitrate determines the thresholds. The paper then presents this enforced number as the achieved 'about 3-bit' result (Abstract; Table 2), and Appendix A.5 concedes the values 'were empirically set' and 'might not reflect the most balanced or effective proportion.' The headline bitrate is therefore an input constraint on the experiment rather than an output of the coarse-to-fine proxy; only the accuracy numbers remain genuinely empirical.

full rationale

The central accuracy comparisons, the hybrid ablation (Table 5), the proxy ablation (Table 6), and the codebook-optimization ablation (Tables 7 and 11) are empirical and are not forced by the threshold construction: the reported gains over SQ-only, VQ-only, and ablated variants could have gone either way and are therefore real evidence for the method's accuracy claims. However, the abstract's headline 'about 3-bit' result is partially circular: the 3.275 bpw figure is fixed by construction because τc and τf are selected per model to enforce a preset 9:1 mixture of 3.25-bpw SQ and 3.5-bpw VQ layers. The paper's own limitation statement (Appendix A.5) admits the allocation was empirically set and may not be balanced or effective. I do not count the separately disclosed protocol issue that 128 test-set samples are used for calibration as a circularity under the hard rules, since it is benchmark contamination rather than a derivation equivalent to its inputs by definition; it is nonetheless a validity risk. No load-bearing self-citation was identified: the only overlapping-author citation (Yang et al., 2024, in Section 2.2) is used as a general reference for scalar quantization and does not support the central claims. Score 4 reflects one partially circular construction while the accuracy and speedup claims remain independently testable.

Assumptions & free parameters 5 free parameters · 6 assumptions · 0 invented entities

No new physical or architectural entities are postulated; the proxy is a decision rule rather than an invented explanatory entity. The load-bearing free parameters are the two proxy thresholds, which are tuned per model to enforce the reported bitrate, plus the unstated K and clipping percentile.

free parameters (5)
  • tau_c (coarse proxy threshold) = 1.54 (RWKV7 example); varied per model
    Layers with Pc < tau_c and Pf < tau_f use SQ; thresholds are tuned per model to realize a fixed 9:1 SQ/VQ layer split, directly setting the reported 3.275 bpw.
  • tau_f (fine proxy threshold) = 30 (RWKV7 example); varied per model
    Controls outlier detection; Appendix A.5 admits data-dependent optimal values.
  • K (central moment order) = not reported
    K appears in Eq. 17 as a hyper-parameter but its value is never stated in the experiments.
  • clipping percentile = not reported
    Section 3.2 uses a percentile-based clipping of activations before averaging but does not state the percentile.
  • SQ/VQ layer allocation ratio = 90% SQ at 3.25 bpw, 10% VQ at 3.5 bpw
    The 3.275 bpw headline number is the weighted average of this enforced allocation, not an emergent result.
assumptions (6)
  • standard math Information entropy H(G') is maximized only when the sorted-weight gaps G' are uniform.
    Used to define the coarse proxy Pc in Eq. 9; standard property of discrete entropy.
  • standard math The Taylor expansion of Pc around the uniform distribution can be truncated at order K with negligible remainder, and cross terms vanish because the entropy function is separable.
    Underlies Eq. 12-14 and the fine proxy Pf; separability is true for H(G') = -sum G'_i log G'_i, but truncation to K terms is a heuristic.
  • domain assumption Minimizing per-layer quantization MSE on 128 calibration samples is a good proxy for final zero-shot task accuracy.
    Standard PTQ assumption, but calibration samples are drawn from the same test datasets used for evaluation, creating mild selection bias.
  • domain assumption RWKV activations are approximately normally distributed, so clipping a small percentile before averaging yields a representative activation scale.
    Justifies the codebook optimization in Section 3.2; Figure 4 shows one illustrative layer only.
  • domain assumption RWKV's nonlinear operators (token-shift, sigmoid, exponential) prevent fusion of smooth- and rotation-based quantization parameters, so SQ methods add runtime overhead.
    Motivates the hybrid; supported by a single unquantified statement that QuaRot increases FLOP by more than 99% for RWKV-7.
  • domain assumption The per-layer binary choice between GPTQ-style SQ and GPTVQ-style VQ is sufficient to capture the accuracy/compression trade-off.
    The paper leaves finer-grained (channel-level) selection to future work, implying that layer-level granularity is a modeling choice.

how reviews work

0 comments
Cite this review

Pith. "Pith review of RWKVQuant: Quantizing the RWKV Family with Proxy Guided Hybrid of Scalar and Vector Quantization." pith.science (2026). https://pith.science/paper/XTV63DAC

@misc{pith2026250503803,
  author       = {Pith},
  title        = {Pith review of: RWKVQuant: Quantizing the RWKV Family with Proxy Guided Hybrid of Scalar and Vector Quantization},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/XTV63DAC}},
  note         = {Machine review of arXiv:2505.03803}
}
read the original abstract

RWKV is a modern RNN architecture with comparable performance to Transformer, but still faces challenges when deployed to resource-constrained devices. Post Training Quantization (PTQ), which is a an essential technique to reduce model size and inference latency, has been widely used in Transformer models. However, it suffers significant degradation of performance when applied to RWKV. This paper investigates and identifies two key constraints inherent in the properties of RWKV: (1) Non-linear operators hinder the parameter-fusion of both smooth- and rotation-based quantization, introducing extra computation overhead. (2) The larger amount of uniformly distributed weights poses challenges for cluster-based quantization, leading to reduced accuracy. To this end, we propose RWKVQuant, a PTQ framework tailored for RWKV models, consisting of two novel techniques: (1) a coarse-to-fine proxy capable of adaptively selecting different quantization approaches by assessing the uniformity and identifying outliers in the weights, and (2) a codebook optimization algorithm that enhances the performance of cluster-based quantization methods for element-wise multiplication in RWKV. Experiments show that RWKVQuant can quantize RWKV-6-14B into about 3-bit with less than 1% accuracy loss and 2.14x speed up.

Figures

Figures reproduced from arXiv: 2505.03803 by the authors.

Figure 1
Figure 1. Accuracy-model size curve. Results of zero-shot [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. Model Structure of RWKV-7. It contains several [PITH_FULL_IMAGE:figures/full_fig_p003_2.png] view at source ↗
Figure 3
Figure 3. Zero-shot accuracy when applying different quantization methods to specific weights. For the weight in each [PITH_FULL_IMAGE:figures/full_fig_p005_3.png] view at source ↗
Figures from the paper (6 more)
Figure 5
Figure 5. Figure 5: Comparison of SQ proportion between RWKV [PITH_FULL_IMAGE:figures/full_fig_p008_5.png]
Figure 4
Figure 4. Figure 4: Effectiveness of clipping for batch integration. [PITH_FULL_IMAGE:figures/full_fig_p008_4.png]
Figure 6
Figure 6. Figure 6: Unifrom weights without outliers in RWKV7-0.1B different layers. [PITH_FULL_IMAGE:figures/full_fig_p011_6.png]
Figure 7
Figure 7. Figure 7: Non-uniform weights in RWKV7-0.1B different layers. [PITH_FULL_IMAGE:figures/full_fig_p011_7.png]
Figure 8
Figure 8. Figure 8: Unifrom weights with outliers in RWKV7-0.1B different layers. [PITH_FULL_IMAGE:figures/full_fig_p012_8.png]
Figure 9
Figure 9. Figure 9: Compute-to-memory-ratio for different models. [PITH_FULL_IMAGE:figures/full_fig_p012_9.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

48 extracted references · 16 canonical work pages

  1. [1]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 global.max substring 't := if while FUNCTION format.date year duplicate empty "emp...

  2. [2]

    OpenReview, 2024

    Ostquant: Refining large language model quantization with orthogonal and scaling transformations for better distribution fitting. OpenReview, 2024. URL https://openreview.net/forum?id=rAcgDBdKnP

    work page 2024

  3. [3]

    Coefficient of variation

    Abdi, H. Coefficient of variation. Encyclopedia of research design, 1 0 (5): 0 169--171, 2010

    2010

  4. [4]

    L., Li, B., Jaggi, M., Alistarh, D., Hoefler, T., and Hensman, J

    Ashkboos, S., Mohtashami, A., Croci, M. L., Li, B., Jaggi, M., Alistarh, D., Hoefler, T., and Hensman, J. Quarot: Outlier-free 4-bit inference in rotated llms. arXiv preprint arXiv:2404.00456, 2024

    arXiv 2024

  5. [5]

    Piqa: Reasoning about physical commonsense in natural language

    Bisk, Y., Zellers, R., Gao, J., Choi, Y., et al. Piqa: Reasoning about physical commonsense in natural language. In Proceedings of the AAAI conference on artificial intelligence, volume 34, pp.\ 7432--7439, 2020

    2020

  6. [6]

    Blinkdl/rwkv-lm: 0.01, August 2021

    Bo, P. Blinkdl/rwkv-lm: 0.01, August 2021. URL https://doi.org/10.5281/zenodo.5196577

    doi:10.5281/zenodo.5196577 2021

  7. [7]

    A systematic classification of knowledge, reasoning, and context within the arc dataset

    Boratko, M., Padigela, H., Mikkilineni, D., Yuvraj, P., Das, R., McCallum, A., Chang, M., Fokoue-Nkoutche, A., Kapanipathi, P., Mattei, N., et al. A systematic classification of knowledge, reasoning, and context within the arc dataset. arXiv preprint arXiv:1806.00358, 2018

    arXiv 2018

  8. [8]

    Onlysportslm: Optimizing sports-domain language models with sota performance under billion parameters

    Chen, Z., Li, C., Xie, X., and Dube, P. Onlysportslm: Optimizing sports-domain language models with sota performance under billion parameters. arXiv preprint arXiv:2409.00286, 2024

    arXiv 2024

Show all 48 references
  1. [9]

    Cryscan. eloise. Github, 2023. URL https://github.com/cryscan/eloise

    2023

  2. [10]

    Imagenet: A large-scale hierarchical image database

    Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., and Fei-Fei, L. Imagenet: A large-scale hierarchical image database. In 2009 IEEE conference on computer vision and pattern recognition, pp.\ 248--255. Ieee, 2009

    2009

  3. [11]

    Vision-rwkv: Efficient and scalable visual perception with rwkv-like architectures

    Duan, Y., Wang, W., Chen, Z., Zhu, X., Lu, L., Lu, T., Qiao, Y., Li, H., Dai, J., and Wang, W. Vision-rwkv: Efficient and scalable visual perception with rwkv-like architectures. arXiv preprint arXiv:2403.02308, 2024

    2024 arXiv

  4. [12]

    Extreme compression of large language models via additive quantization

    Egiazarian, V., Panferov, A., Kuznedelev, D., Frantar, E., Babenko, A., and Alistarh, D. Extreme compression of large language models via additive quantization. arXiv preprint arXiv:2401.06118, 2024

    2024 arXiv

  5. [13]

    Elman, J. L. Finding structure in time. Cognitive science, 14 0 (2): 0 179--211, 1990

    1990

  6. [14]

    Gptq: Accurate post-training quantization for generative pre-trained transformers

    Frantar, E., Ashkboos, S., Hoefler, T., and Alistarh, D. Gptq: Accurate post-training quantization for generative pre-trained transformers. arXiv preprint arXiv:2210.17323, 2022

    2022 arXiv

  7. [15]

    Asymptotically optimal block quantization

    Gersho, A. Asymptotically optimal block quantization. IEEE Transactions on information theory, 25 0 (4): 0 373--380, 1979

    1979

  8. [16]

    Gumbel, E. J. The Distribution of the Range . The Annals of Mathematical Statistics, 18 0 (3): 0 384 -- 412, 1947. doi:10.1214/aoms/1177730387. URL https://doi.org/10.1214/aoms/1177730387

    1947

  9. [17]

    Quantization and training of neural networks for efficient integer-arithmetic-only inference

    Jacob, B., Kligys, S., Chen, B., Zhu, M., Tang, M., Howard, A., Adam, H., and Kalenichenko, D. Quantization and training of neural networks for efficient integer-arithmetic-only inference. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR),...

    2018

  10. [18]

    and Koshizuka, T

    Konno, H. and Koshizuka, T. Mean-absolute deviation model. Iie Transactions, 37 0 (10): 0 893--900, 2005

    2005

  11. [19]

    Crafting papers on machine learning

    Langley, P. Crafting papers on machine learning. In Langley, P. (ed.), Proceedings of the 17th International Conference on Machine Learning (ICML 2000), pp.\ 1207--1216, Stanford, CA, 2000. Morgan Kaufmann

    2000

  12. [20]

    Infofusion

    LeoLin4258. Infofusion. Github, 2024. URL https://github.com/LeoLin4258/Infofusion

    2024

  13. [21]

    A survey of rwkv

    Li, Z., Xia, T., Chang, Y., and Wu, Y. A survey of rwkv. arXiv preprint arXiv:2412.14847, 2024

    2024 arXiv

  14. [22]

    Awq: Activation-aware weight quantization for llm compression and acceleration

    Lin, J., Tang, J., Tang, H., Yang, S., Dang, X., and Han, S. Awq: Activation-aware weight quantization for llm compression and acceleration. arXiv preprint arXiv:2306.00978, 2023

    2023 arXiv

  15. [23]

    Lin, T.-Y., Maire, M., Belongie, S., Hays, J., Perona, P., Ramanan, D., Doll \'a r, P., and Zitnick, C. L. Microsoft coco: Common objects in context. In Computer Vision--ECCV 2014: 13th European Conference, Zurich, Switzerland, September 6-12, 2014, Proceedings, Part V 13, pp....

    2014

  16. [24]

    L., Cao, T., Li, C., and Yang, M

    Liu, Y., Wen, J., Wang, Y., Ye, S., Zhang, L. L., Cao, T., Li, C., and Yang, M. Vptq: Extreme low-bit vector post-training quantization for large language models. arXiv preprint arXiv:2409.17066, 2024 a

    2024 arXiv

  17. [25]

    Spinquant--llm quantization with learned rotations

    Liu, Z., Zhao, C., Fedorov, I., Soran, B., Choudhary, D., Krishnamoorthi, R., Chandra, V., Tian, Y., and Blankevoort, T. Spinquant--llm quantization with learned rotations. arXiv preprint arXiv:2405.16406, 2024 b

    2024 arXiv

  18. [26]

    Lloyd, S. P. Least squares quantization in pcm. IEEE Transactions on Information Theory, 28 0 (2): 0 129--137, 1982

    1982

  19. [27]

    Can a suit of armor conduct electricity? a new dataset for open book question answering

    Mihaylov, T., Clark, P., Khot, T., and Sabharwal, A. Can a suit of armor conduct electricity? a new dataset for open book question answering. arXiv preprint arXiv:1809.02789, 2018

    2018 arXiv

  20. [28]

    Moon, T. K. The expectation-maximization algorithm. IEEE Signal processing magazine, 13 0 (6): 0 47--60, 1996

    1996

  21. [29]

    Chatrwkv-in-wechat-version-1

    MrTom34. Chatrwkv-in-wechat-version-1. Github, 2023. URL https://github.com/MrTom34/ChatRWKV-in-wechat-Version-1

    2023

  22. [30]

    Sciq: an invitation and recommendations to combine science and inuit qaujimajatuqangit for meaningful engagement of inuit communities in research

    Pedersen, C., Otokiak, M., Koonoo, I., Milton, J., Maktar, E., Anaviapik, A., Milton, M., Porter, G., Scott, A., Newman, C., et al. Sciq: an invitation and recommendations to combine science and inuit qaujimajatuqangit for meaningful engagement of inuit communities in research...

    2020

  23. [31]

    Rwkv: Reinventing rnns for the transformer era

    Peng, B., Alcaide, E., Anthony, Q., Albalak, A., Arcadinho, S., Biderman, S., Cao, H., Cheng, X., Chung, M., Grella, M., et al. Rwkv: Reinventing rnns for the transformer era. arXiv preprint arXiv:2305.13048, 2023

    2023 arXiv

  24. [32]

    Eagle and finch: Rwkv with matrix-valued states and dynamic recurrence

    Peng, B., Goldstein, D., Anthony, Q., Albalak, A., Alcaide, E., Biderman, S., Cheah, E., Du, X., Ferdinan, T., Hou, H., et al. Eagle and finch: Rwkv with matrix-valued states and dynamic recurrence. arXiv preprint arXiv:2404.05892, 2024

    2024 arXiv

  25. [33]

    Language models are unsupervised multitask learners

    Radford, A., Wu, J., Child, R., Luan, D., Amodei, D., Sutskever, I., et al. Language models are unsupervised multitask learners. OpenAI blog, 1 0 (8): 0 9, 2019

    2019

  26. [34]

    Qa dataset explosion: A taxonomy of nlp resources for question answering and reading comprehension

    Rogers, A., Gardner, M., and Augenstein, I. Qa dataset explosion: A taxonomy of nlp resources for question answering and reading comprehension. ACM Computing Surveys, 55 0 (10): 0 1--45, 2023

    2023

  27. [35]

    L., Bhagavatula, C., and Choi, Y

    Sakaguchi, K., Bras, R. L., Bhagavatula, C., and Choi, Y. Winogrande: An adversarial winograd schema challenge at scale. Communications of the ACM, 64 0 (9): 0 99--106, 2021

    2021

  28. [36]

    Shannon, C. E. A mathematical theory of communication. The Bell system technical journal, 27 0 (3): 0 379--423, 1948

    1948

  29. [37]

    Omniquant: Omnidirectionally calibrated quantization for large language models

    Shao, W., Chen, M., Zhang, Z., Xu, P., Zhao, L., Li, Z., Zhang, K., Gao, P., Qiao, Y., and Luo, P. Omniquant: Omnidirectionally calibrated quantization for large language models. CoRR, abs/2308.13137, 2023

    2023 arXiv

  30. [38]

    spion. notgpt. Github, 2023. URL https://github.com/spion/notgpt

    2023

  31. [39]

    Methodus incrementorum directa & inversa

    Taylor, B. Methodus incrementorum directa & inversa. Inny, 1717

  32. [40]

    Llama 2: Open foundation and fine-tuned chat models

    Touvron, H., Martin, L., Stone, K., Albert, P., Almahairi, A., Babaei, Y., Bashlykov, N., Batra, S., Bhargava, P., Bhosale, S., et al. Llama 2: Open foundation and fine-tuned chat models. arXiv preprint arXiv:2307.09288, 2023

    2023 arXiv

  33. [41]

    Gptvq: The blessing of dimensionality for llm quantization

    van Baalen, M., Kuzmin, A., Nagel, M., Couperus, P., Bastoul, C., Mahurin, E., Blankevoort, T., and Whatmough, P. Gptvq: The blessing of dimensionality for llm quantization. arXiv preprint arXiv:2402.15319, 2024

    2024 arXiv

  34. [42]

    Attention is all you need

    Vaswani, A. Attention is all you need. Advances in Neural Information Processing Systems, 2017

    2017

  35. [43]

    Smoothquant: Accurate and efficient post-training quantization for large language models

    Xiao, G., Lin, J., Seznec, M., Demouth, J., and Han, S. Smoothquant: Accurate and efficient post-training quantization for large language models. arXiv preprint arXiv:2211.10438, 2022

    2022 arXiv

  36. [44]

    Post-training quantization for re-parameterization via coarse & fine weight splitting

    Yang, D., He, N., Hu, X., Yuan, Z., Yu, J., Xu, C., and Jiang, Z. Post-training quantization for re-parameterization via coarse & fine weight splitting. Journal of Systems Architecture, 147: 0 103065, 2024

    2024

  37. [45]

    J., Yan, Y., Chen, B., Sun, G., and Keutzer, K

    Yuan, Z., Shang, Y., Zhou, Y., Dong, Z., Xue, C., Wu, B., Li, Z., Gu, Q., Lee, Y. J., Yan, Y., Chen, B., Sun, G., and Keutzer, K. Llm inference unveiled: Survey and roofline model insights, 2024

    2024

  38. [46]

    Hellaswag: Can a machine really finish your sentence? arXiv preprint arXiv:1905.07830, 2019

    Zellers, R., Holtzman, A., Bisk, Y., Farhadi, A., and Choi, Y. Hellaswag: Can a machine really finish your sentence? arXiv preprint arXiv:1905.07830, 2019

    1905 arXiv

  39. [47]

    Semantic understanding of scenes through the ade20k dataset

    Zhou, B., Zhao, H., Puig, X., Xiao, T., Fidler, S., Barriuso, A., and Torralba, A. Semantic understanding of scenes through the ade20k dataset. International Journal of Computer Vision, 127: 0 302--321, 2019

    2019

  40. [48]

    and Chen, T

    Zhou, X. and Chen, T. Bsbp-rwkv: Background suppression with boundary preservation for efficient medical image segmentation. In Proceedings of the 32nd ACM International Conference on Multimedia, pp.\ 4938--4946, 2024

    2024

Pith tools

Reviewed August 16, 2026 · model on record in the stance chip above.