Pith. sign in

REVIEW 4 major objections 5 minor 78 references

Reduced Matrix Multiplication: Input-Adaptive Matrix-Product Reduction for LLM Inference

T0 review · 4 major / 5 minor · reviewed 2026-08-14 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read Replacing each Transformer matrix product with a slice along the input's highest-energy columns preserves model behavior under moderate retention, with attention-side products far more reducible than MLPs.

desk verdict A solid, clearly-written paper on an attention-side reduction trick that is over-scoped to all Transformer matrix products, with a correct but narrow theoretical guarantee and a few comparison/latency issues that revision can fix. read the letter →

arxiv 2608.13426 v1 pith:KGRULLYA submitted 2026-08-13 cs.LG cs.AIcs.CL

Zixuan Lan , Yanhong Li , Jiawei Zhou This is my paper · ORCID
classification cs.LGcs.AIcs.CL
keywords reducedmatrixmultiplicationinput-adaptiveinferenceactivation-awaredimensionselectionTransformerattentionredundancyMLPsensitivityretentionratiotraining-freeacceleration
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Reduced Matrix Multiplication (RMM) is a training-free scheme for cutting Transformer inference cost: for every matrix product $Y=AB$ in attention and MLP layers, it keeps only the $\lceil\rho d\rceil$ columns of $A$ with the largest L2 norms and computes $A_{:,I}B_{I,:}$. The paper claims that under moderate retention ratios ($\rho=0.7$ to $0.9$) this preserves model behavior across discriminative QA, summarization, long-context, and vision-language tasks in models spanning 1B to 70B parameters, with degradation that grows smoothly as $\rho$ falls. A central empirical finding is the structural asymmetry inside Transformers: attention-side products such as $QK^\top$ and $PV$ tolerate aggressive reduction, while MLP projections---especially the Up projection---degrade sharply. If the claim holds, RMM gives practitioners a predictable accuracy--efficiency dial for inference without retraining or weight modification.

What carries the argument

The object that carries the argument is the reduced product $\mathrm{RMM}_\rho(A,B)=A_{:,I}B_{I,:}$ with $I=\mathrm{TopK}(\{\|A_{:,j}\|_2\}_{j=1}^d,\lceil\rho d\rceil)$. The analysis decomposes $AB=\sum_{j=1}^d A_{:,j}B_{j,:}$, so discarding dimension $j$ costs $\|A_{:,j}\|_2\|B_{j,:}\|_2$. Because $B$ is unobserved at selection time, Theorem 1 treats selection as a minimax game in which an adversary may place all of $B$'s row energy on the discarded columns; TopK minimizes the worst-case residual $\|A_{:,\bar I}\|_F$. The resulting bound, relative error $\le\sqrt{\epsilon_A(\rho)\epsilon_B(\rho)}$, is what lets the paper link activation-energy concentration to output fidelity and justify the method without seeing the weights or keys/values in advance.

What would settle it

Measure, for every targeted matrix product at $\rho=0.7$ in a small model, the product of discarded-energy fractions $\sqrt{\epsilon_A(\rho)\epsilon_B(\rho)}$ and compare it with the actual output perturbation or downstream accuracy drop; if a component with high retained activation energy (for example, the MLP Up projection, which retains 82.24\% of activation energy) still shows a much larger drop than the bound predicts, the alignment assumption is false for that component and the general claim 'moderate reduction preserves behavior' fails there.

Watch

Extended reading notes

Core claim

The central discovery is that the contraction axis of a Transformer matrix product carries input-dependent redundancy, and that a deterministic, activation-aware slice exploits it. Formally, for $Y=AB$, RMM selects $I=\mathrm{TopK}(\{\|A_{:,j}\|_2\}_{j=1}^d,\lceil\rho d\rceil)$ and returns $A_{:,I}B_{I,:}$. The paper proves (Theorem 1) that among all selection rules that see only $A$, this choice is minimax optimal against a worst-case $B$ whose row norms concentrate on the discarded columns, and bounds the relative error by $\sqrt{\epsilon_A(\rho)\epsilon_B(\rho)}$, the product of the discarded-energy fractions of $A$ and $B$. Empirically, at $\rho=0.8$ the reduced model stays close to the unmodified model across the evaluated benchmarks, and attention-side components remain stable even at $\rho=0.5$; MLP components, and the Up projection in particular, do not. The paper interprets this as evidence that redundancy is not uniform across Transformer components and that component-aware retention policies are needed.

Load-bearing premise

The practical claim rests on the assumption that the information in the unobserved operand $B$ lines up with the high-norm columns of the input $A$, so that the energy discarded with the low-norm columns is small; the paper proves only that picking the largest columns is the best defence against a worst-case $B$, not that real $B$ behaves this way.

Editorial extensions

If this is right

  • At $\rho=0.8$, RMM stays within a few points of the full model on commonsense QA, MMLU, GSM8K, and HumanEval across the evaluated 7B to 70B models, and larger models generally tolerate more aggressive reduction.
  • Attention-side matrix products ($QK^\top$, $PV$, and the Q projection) degrade only mildly even at $\rho=0.5$, while MLP reduction---especially the Up projection---causes sharp accuracy drops, so safe deployment should apply component-specific retention ratios.
  • In autoregressive generation and long-context RULER tasks, degradation is smooth rather than abrupt, with no systematic increase in error up to 30K tokens.
  • The same principle transfers to vision-language models: at $\rho=0.8$ RMM matches the dense baseline on POPE and BLINK subsets, and at $\rho=0.5$ it clearly outperforms static and random pruning.
  • Wall-clock gains grow with sequence length: around $1.05\times$ at 1K tokens, $1.27\times$ at 2K, and $1.40\times$ at 4K on the 8B model in single-GPU latency tests, and RMM avoids out-of-memory at 4K on the 70B model.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The authors leave implicit that the practical success of RMM should be predictable from activation-energy concentration alone, so measuring $\epsilon_A(\rho)$ at each layer could serve as a cheap, label-free proxy for how aggressively that layer can be reduced.
  • The method is a special case of a more general principle---contract only the dimensions where the input's energy is concentrated---so the same selection rule could be applied to KV-cache compression or to other bilinear operations, with per-component retention ratios rather than one global $\rho$.
  • The minimax theorem concerns a worst-case adversary; real models are not adversarial. A testable extension is to measure the actual discarded row-energy fraction $\epsilon_B(\rho)$ of weights, keys, and values and check whether it stays small; if it does, a tighter data-dependent error bound than the minimax one would hold.
  • Because RMM recomputes indices every decoding step, it effectively performs a local, input-dependent projection. This suggests that training objectives that encourage energy concentration in activations could make inference-time reduction substantially more aggressive than post-hoc selection alone.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. The paper proposes Reduced Matrix Multiplication (RMM), a training-free, input-adaptive method that reduces the contraction dimension of matrix products in Transformer inference. For a product Y = AB, RMM selects a subset I of dimension indices by the L2 column norms of A and computes A_{:,I} B_{I,:}, with the retention ratio rho controlling how many indices are kept. The authors apply RMM to attention-internal products (QK^T, PV) and to MLP/linear projections, and evaluate it on models from 1B to 70B across discriminative QA, summarization, long-context, and vision-language benchmarks. They report that attention-side computations are substantially more reducible than MLP components, that larger models tolerate more aggressive reduction, and that custom Triton kernels yield wall-clock speedups. A minimax optimality theorem (Appendix E.1) and an approximation error bound (Proposition 1, Corollary 1) are provided as theoretical support.

Significance. If the central claims held in their full generality, RMM would be a useful, simple addition to the inference-time pruning toolbox: it is training-free, requires no weight modification, has a single user-controlled retention ratio, and is evaluated across a broad sweep of models and tasks. The paper has notable strengths: it ships a large empirical matrix (Tables 3, 9, 10, 16), includes controlled ablations that separate dynamic vs. static and activation-aware vs. random selection, reports compute-normalized component analysis (Table 12), and explicitly acknowledges several limitations, including the need for heterogeneous retention policies and the fact that RMM does not reduce stored weights. The theoretical statements are transparently derived and are correct for the surrogate objective they address. However, the scope of the empirical support is narrower than the abstract claims, and the theoretical justification rests on an information-asymmetry premise that does not hold in the method's own applications. These issues are load-bearing for the paper's headline claim that RMM reduces 'Transformer matrix products' generally.

major comments (4)
  1. [Abstract and Section 4] The abstract and Section 1 state that RMM reduces 'the high-dimensional matrix products in attention and feed-forward layers' and 'Transformer matrix products' generally, but Section 4 says the main-paper results apply reduction to attention-side matrix multiplications only. This is not a minor scope remark: Appendix Table 16 shows that applying RMM to the whole MLP block at RR=0.7 and RR=0.5 drops average QA accuracy from 69.79 to 55.93 and 40.28 respectively, far worse than the attention-side reductions reported in the main text. The general claim is therefore unsupported, and the paper should either restrict its headline claims to attention-side reductions or provide component-specific retention policies and report their aggregate behavior.
  2. [Section 3.2 and Appendix E.1] Theorem 1 justifies TopK selection by column norm as minimax optimal under an 'information asymmetry' in which B is not observed at selection time. But in every application of RMM in this paper, B is available before the GEMM is executed: in QK^T the key matrix K is already computed, in PV the attention matrix P is available, and in MLP projections the weight matrix W is fixed in memory. Thus the one-sided premise does not hold for RMM's own applications, and a B-aware selection rule could in principle do better. The practical claim therefore rests on the unmeasured assumption that the discarded rows of B carry little energy. Equation (17) bounds the relative error by sqrt(epsilon_A epsilon_B), but epsilon_B is never reported anywhere in the paper. Appendix B.4 (Table 12) shows the problem: reducing MLP-Up at RR=0.7 retains 82.24% of activation energy yet loses 16.32 accuracy points, while attention-side reduction retains 89.69% and loses only 3.52 points, so the A-side energy diagnostic alone cannot predict whether RMM preserves model behavior. I ask the authors to report epsilon_B for the tested components or otherwise provide empirical evidence about B-side energy concentration, and to compare RMM against a B-aware selection baseline to make the practical claim credible.
  3. [Section 5.1, Tables 1 and 2] The comparison against SparseGPT, Wanda, SliceGPT, and magnitude pruning under a 'fixed retention ratio' of RR=0.5 does not match computation or memory budgets across methods. The static baselines remove weights permanently at a given sparsity, while RMM removes activation columns dynamically and retains all weights; the same nominal '0.5' therefore corresponds to different FLOPs, memory footprints, and architectural structures. The reported advantages over baselines may reflect the type of pruning (activation-adaptive vs. weight-static) rather than RMM as such, but the current exposition does not state what is matched. The authors should specify the matched quantity (e.g., measured FLOPs, measured latency, or parameter count) and, if possible, report runtime for each method under identical backends.
  4. [Section 6.3 and Table 8] The end-to-end wall-clock comparison in Table 8 uses a HuggingFace SDPA dense baseline against a custom Triton RMM implementation, so the reported speedup conflates algorithmic reduction with kernel engineering. The kernel-level GEMM numbers in Table 7 are more controlled, but the end-to-end claim should use the same implementation backend for both the dense and reduced variants, or at least include a dense Triton baseline. Otherwise the 1.05x speedup at sequence length 1024 and the larger numbers at longer sequences are not attributable solely to RMM's contraction-axis reduction.
minor comments (5)
  1. [Table 2 and Table 10] The H2O rows list identical ROUGE and BERTScore values at RR=0.8 and RR=0.5; the paper explains this by H2O's fixed token budget, but the footnote should be placed directly in the table caption so readers do not misinterpret the duplicated numbers as a copy-paste error.
  2. [Table 10, Qwen3-1 7B H2O row] In the Qwen3-1 7B block of Table 10, the H2O row reports ROUGE-2 = 9.10 with ROUGE-1 = 4.20, which is odd because ROUGE-2 should generally not exceed ROUGE-1; please verify these numbers and correct or annotate the table.
  3. [Table 12] The 'Retained energy' column reports only A-side activation energy; since the text in Section B.4 uses this diagnostic to compare MLP-Up and MLP-Down, the column label should clarify that it is the retained activation energy of the input operand, not the energy of the discarded B rows.
  4. [Appendix B.4] The phrase 'MLP reduction should be applied selectively across the Up, Gate, and Down projections' is a useful recommendation, but the main-text conclusion in Section 6.2 should state the same nuance rather than saying flatly that 'attention-side computations are substantially more reducible than MLP components', since MLP-Down at RR=0.7 loses only 3.51 points.
  5. [Appendix A.2] The speculative theoretical perspective is clearly labeled as heuristic, which is good; however, the term 'local tangent space' is used without a precise definition and could be simplified or removed, as it does not connect to any formal statement used elsewhere in the paper.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: RMM is an input-adaptive selection rule whose theoretical support is a self-contained surrogate bound, and whose behavioral claims are evaluated against external benchmarks rather than derived from the method's own construction.

full rationale

The paper's derivation chain is not circular. RMM is defined as A_{:,I}B_{I,:} with I chosen by top-k column norms of A under a user-specified retention ratio rho. No parameter is fitted to the evaluation data, and the retention ratio is not inferred from the data. The approximation error bound (Proposition 1 and Corollary 1) follows directly from the triangle inequality and Cauchy-Schwarz; it is a standard, parameter-free bound and does not assume the empirical conclusion. Theorem 1 in Appendix E.1 is a minimax statement about a clearly stated surrogate: given that B's row norms are unknown and chosen adversarially with unit total energy, top-k by ||A_{:,j}||_2 minimizes the resulting upper bound. The proof is self-contained and the result is exactly the minimization of the discarded A-side energy, so it is a valid mathematical lemma rather than a restatement of the empirical results. The one-sided information asymmetry is an explicit modeling assumption, and Remark 3 concedes that a jointly optimal rule could do better if B were observable; this is a scope limitation, not circularity. The central behavioral claim -- that RMM preserves model behavior across Transformer matrix products -- is not derived from Theorem 1; it is established empirically against external benchmarks (SparseGPT, Wanda, H2O, TEAL, etc.) with no reliance on self-citations. The paper even reports non-successful cases: Appendix B.4 shows that reducing the MLP-Up projection at RR=0.7 retains 82.24% of activation energy but drops accuracy by 16.32 points, and the text explicitly states that retained energy alone does not fully determine downstream performance. Such self-reported limitations confirm that the empirical findings are not forced by the construction. There are no load-bearing self-citations and no fitted input renamed as a prediction. Honest non-finding is therefore appropriate: no significant circularity.

Assumptions & free parameters 1 free parameters · 4 assumptions · 0 invented entities

No new physical or architectural entities are introduced. RMM selects existing contraction-axis indices using activation column norms. The only hand-chosen quantity is the retention ratio, which is a deployment control rather than a fitted parameter. The theoretical justification rests on standard inequalities plus a domain assumption about worst-case alignment between A and B.

free parameters (1)
  • retention ratio rho = Swept from 0.5 to 0.9, with 0.7 recommended for attention-side reduction.
    User-specified fraction of the contraction dimension retained in each reduced matrix product. The paper sweeps retention ratios across experiments and recommends RR=0.7 via a label-free consistency sweep (Appendix B.8), so it is chosen by hand or deployment constraint rather than fitted to task labels.
assumptions (4)
  • standard math Cauchy-Schwarz inequality and Frobenius norm properties.
    Used in the proofs of Proposition 1 and Theorem 1 in Appendix E to bound the approximation error and to solve the inner maximization.
  • domain assumption The selection problem can be modeled as a minimax game where the unobserved operand B is an arbitrary matrix with bounded row norm, independent of A.
    This is the load-bearing modeling assumption behind Theorem 1. The paper states in Appendix E.1: 'Since we observe A but not B, we formulate the selection problem as a minimax game over independent per-dimension adversaries.' The practical validity of this worst-case model for real Transformer weights is not proven.
  • domain assumption Transformer hidden-state activation energy is concentrated in a small number of dimensions, so the discarded energy ratio epsilon_A(rho) is small at moderate retention.
    Used in Appendix E.4 to interpret the error bound: 'When the activation energy of A is concentrated in a small number of dimensions... the discarded energy ratio is small.' This is presented as a widely observed empirical regularity, not as a theorem.
  • domain assumption Reduction errors from individual matrix products do not accumulate catastrophically across layers, except where the ablations show they do.
    The component-wise analysis in Appendix C shows that errors do accumulate when the whole MLP block is reduced, and the paper recommends heterogeneous retention policies. The general robustness claim therefore implicitly assumes that error accumulation is manageable for the targeted components.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Reduced Matrix Multiplication: Input-Adaptive Matrix-Product Reduction for LLM Inference." pith.science (2026). https://pith.science/paper/KGRULLYA

@misc{pith2026260813426,
  author       = {Pith},
  title        = {Pith review of: Reduced Matrix Multiplication: Input-Adaptive Matrix-Product Reduction for LLM Inference},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/KGRULLYA}},
  note         = {Machine review of arXiv:2608.13426}
}
read the original abstract

Transformer-based language models achieve strong performance but incur substantial inference cost due to repeated high-dimensional matrix multiplications. We propose Reduced Matrix Multiplication (RMM), a training-free, input-adaptive inference method that reduces Transformer matrix products by selecting informative slices along their contraction dimensions, without modifying model weights. Under a simple retention-ratio control, RMM provides a smooth and predictable accuracy-efficiency trade-off. Across language models ranging from 1B to 70B parameters, we find that reduction tolerance depends on the model family, task, component, and retention ratio, although it often improves with model scale. Under moderate reduction, RMM remains robust across the evaluated discriminative, autoregressive generation, and long-context settings. We further show that the same principle extends to multimodal vision-language inference. Mechanistic ablations reveal a structural asymmetry within Transformers: attention-side computations are substantially more reducible than MLP components. Finally, wall-clock benchmarks with custom kernels on an NVIDIA A100 show that these computational savings can translate into practical runtime gains, especially at longer sequence lengths. Together, these results position RMM as a scalable direction for input-adaptive inference-time optimization.

Figures

Figures reproduced from arXiv: 2608.13426 by the authors.

Figure 1
Figure 1. Application of RMM in major computations of Transformer language models. [PITH_FULL_IMAGE:figures/full_fig_p004_1.png] view at source ↗
Figure 2
Figure 2. First-output-token attention maps on Qwen 2.5-VL-7B at RR = 0.5. RMM preserves dense￾like attention to relevant objects and correct answers, while static and random pruning produce less aligned patterns and incorrect answers. task ablation, we draw evidence from comparisons across multiple tasks. The importance of dynamic selection is reflected in the consistent gap between RMM and its static variant, which fixes th… view at source ↗
Figure 3
Figure 3. Llama-3.2-3B and Llama-3.2-1B of Different Tasks and RR. Red Line is Llama-3.2-3B [PITH_FULL_IMAGE:figures/full_fig_p014_3.png] view at source ↗

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

78 extracted references · 16 canonical work pages

  1. [1]

    Advances in neural information processing systems , volume=

    Attention is all you need , author=. Advances in neural information processing systems , volume=

  2. [2]

    2021 , howpublished=

    A Mathematical Framework for Transformer Circuits , author=. 2021 , howpublished=

    2021

  3. [3]

    2020 , eprint=

    Scaling Laws for Neural Language Models , author=. 2020 , eprint=

    2020

  4. [4]

    2023 , eprint=

    SparseGPT: Massive Language Models Can Be Accurately Pruned in One-Shot , author=. 2023 , eprint=

    work page 2023

  5. [5]

    2015 , eprint=

    Learning both Weights and Connections for Efficient Neural Networks , author=. 2015 , eprint=

    work page 2015

  6. [7]

    2023 , eprint=

    Structured Pruning of Self-Supervised Pre-trained Models for Speech Recognition and Understanding , author=. 2023 , eprint=

    work page 2023

  7. [9]

    2023 , eprint=

    LLM-Pruner: On the Structural Pruning of Large Language Models , author=. 2023 , eprint=

    work page 2023

  8. [10]

    2024 , eprint=

    SliceGPT: Compress Large Language Models by Deleting Rows and Columns , author=. 2024 , eprint=

    2024

Show all 78 references
  1. [11]

    2024 , eprint=

    DISP-LLM: Dimension-Independent Structural Pruning for Large Language Models , author=. 2024 , eprint=

    2024

  2. [12]

    2024 , eprint=

    A Simple and Effective Pruning Approach for Large Language Models , author=. 2024 , eprint=

    2024

  3. [14]

    2024 , eprint=

    LLMLingua-2: Data Distillation for Efficient and Faithful Task-Agnostic Prompt Compression , author=. 2024 , eprint=

    2024

  4. [15]

    2025 , eprint=

    Deep Think with Confidence , author=. 2025 , eprint=

    2025

  5. [16]

    Advances in Neural Information Processing Systems , volume=

    H2o: Heavy-hitter oracle for efficient generative inference of large language models , author=. Advances in Neural Information Processing Systems , volume=

  6. [17]

    The Twelfth International Conference on Learning Representations , year=

    Efficient Streaming Language Models with Attention Sinks , author=. The Twelfth International Conference on Learning Representations , year=

  7. [18]

    2025 , eprint=

    TokenSkip: Controllable Chain-of-Thought Compression in LLMs , author=. 2025 , eprint=

    2025

  8. [19]

    2024 , eprint=

    FlashAttention-3: Fast and Accurate Attention with Asynchrony and Low-precision , author=. 2024 , eprint=

    2024

  9. [20]

    2025 , eprint=

    Native Sparse Attention: Hardware-Aligned and Natively Trainable Sparse Attention , author=. 2025 , eprint=

    2025

  10. [21]

    2024 , eprint=

    What Matters in Transformers? Not All Attention is Needed , author=. 2024 , eprint=

    2024

  11. [22]

    2020 , eprint=

    Analyzing Redundancy in Pretrained Transformer Models , author=. 2020 , eprint=

    2020

  12. [23]

    and Kannan, R

    Drineas, P. and Kannan, R. , booktitle=. Fast Monte-Carlo algorithms for approximate matrix multiplication , year=

  13. [24]

    SIAM Journal on computing , volume=

    Fast Monte Carlo algorithms for matrices II: Computing a low-rank approximation to a matrix , author=. SIAM Journal on computing , volume=. 2006 , publisher=

    2006

  14. [25]

    2011 , eprint=

    Randomized algorithms for matrices and data , author=. 2011 , eprint=

    2011

  15. [26]

    2024 , eprint=

    The Llama 3 Herd of Models , author=. 2024 , eprint=

    2024

  16. [27]

    2025 , eprint=

    Qwen3 Technical Report , author=. 2025 , eprint=

    2025

  17. [28]

    2025 , eprint=

    Qwen2.5-VL Technical Report , author=. 2025 , eprint=

    2025

  18. [29]

    S em E val-2012 Task 7: Choice of Plausible Alternatives: An Evaluation of Commonsense Causal Reasoning

    Gordon, Andrew and Kozareva, Zornitsa and Roemmele, Melissa. S em E val-2012 Task 7: Choice of Plausible Alternatives: An Evaluation of Commonsense Causal Reasoning. * SEM 2012: The First Joint Conference on Lexical and Computational Semantics -- Volume 1: Proceedings of the m...

    2012

  19. [30]

    Proceedings of the AAAI conference on artificial intelligence , volume=

    Piqa: Reasoning about physical commonsense in natural language , author=. Proceedings of the AAAI conference on artificial intelligence , volume=

  20. [31]

    2018 , eprint=

    Think you have Solved Question Answering? Try ARC, the AI2 Reasoning Challenge , author=. 2018 , eprint=

    2018

  21. [32]

    2019 , eprint=

    CommonsenseQA: A Question Answering Challenge Targeting Commonsense Knowledge , author=. 2019 , eprint=

    2019

  22. [33]

    2021 , eprint=

    Measuring Massive Multitask Language Understanding , author=. 2021 , eprint=

    2021

  23. [34]

    2016 , eprint=

    Pointer Sentinel Mixture Models , author=. 2016 , eprint=

    2016

  24. [35]

    Proceedings of the IEEE international conference on computer vision , pages=

    Aligning books and movies: Towards story-like visual explanations by watching movies and reading books , author=. Proceedings of the IEEE international conference on computer vision , pages=

  25. [36]

    2021 , eprint=

    Training Verifiers to Solve Math Word Problems , author=. 2021 , eprint=

    2021

  26. [37]

    2021 , eprint=

    Evaluating Large Language Models Trained on Code , author=. 2021 , eprint=

    2021

  27. [38]

    2024 , eprint=

    RULER: What's the Real Context Size of Your Long-Context Language Models? , author=. 2024 , eprint=

    2024

  28. [40]

    2018 , eprint=

    ROUGE 2.0: Updated and Improved Measures for Evaluation of Summarization Tasks , author=. 2018 , eprint=

    2018

  29. [41]

    2020 , eprint=

    BERTScore: Evaluating Text Generation with BERT , author=. 2020 , eprint=

    2020

  30. [42]

    2023 , eprint=

    Evaluating Object Hallucination in Large Vision-Language Models , author=. 2023 , eprint=

    2023

  31. [43]

    2024 , eprint=

    BLINK: Multimodal Large Language Models Can See but Not Perceive , author=. 2024 , eprint=

    2024

  32. [44]

    2024 , eprint=

    CATS: Contextually-Aware Thresholding for Sparsity in Large Language Models , author=. 2024 , eprint=

    2024

  33. [45]

    2025 , eprint=

    Training-Free Activation Sparsity in Large Language Models , author=. 2025 , eprint=

    2025

  34. [46]

    Croci, Marcelo Gennari do Nascimento, Torsten Hoefler, and James Hensman

    Saleh Ashkboos, Maximilian L. Croci, Marcelo Gennari do Nascimento, Torsten Hoefler, and James Hensman. 2024. https://arxiv.org/abs/2401.15024 Slicegpt: Compress large language models by deleting rows and columns . Preprint, arXiv:2401.15024

    2024 arXiv

  35. [47]

    Shuai Bai, Keqin Chen, Xuejing Liu, Jialin Wang, Wenbin Ge, Sibo Song, Kai Dang, Peng Wang, Shijie Wang, Jun Tang, Humen Zhong, Yuanzhi Zhu, Mingkun Yang, Zhaohai Li, Jianqiang Wan, Pengfei Wang, Wei Ding, Zheren Fu, Yiheng Xu, and 8 others. 2025. https://arxiv.org/abs/2502.13...

    2025 arXiv

  36. [48]

    Yonatan Bisk, Rowan Zellers, Jianfeng Gao, Yejin Choi, and 1 others. 2020. Piqa: Reasoning about physical commonsense in natural language. In Proceedings of the AAAI conference on artificial intelligence, volume 34, pages 7432--7439

    2020

  37. [49]

    Mark Chen, Jerry Tworek, Heewoo Jun, Qiming Yuan, Henrique Ponde de Oliveira Pinto, Jared Kaplan, Harri Edwards, Yuri Burda, Nicholas Joseph, Greg Brockman, Alex Ray, Raul Puri, Gretchen Krueger, Michael Petrov, Heidy Khlaaf, Girish Sastry, Pamela Mishkin, Brooke Chan, Scott G...

    2021 arXiv

  38. [50]

    Peter Clark, Isaac Cowhey, Oren Etzioni, Tushar Khot, Ashish Sabharwal, Carissa Schoenick, and Oyvind Tafjord. 2018. https://arxiv.org/abs/1803.05457 Think you have solved question answering? try arc, the ai2 reasoning challenge . Preprint, arXiv:1803.05457

    2018 arXiv

  39. [51]

    Karl Cobbe, Vineet Kosaraju, Mohammad Bavarian, Mark Chen, Heewoo Jun, Lukasz Kaiser, Matthias Plappert, Jerry Tworek, Jacob Hilton, Reiichiro Nakano, Christopher Hesse, and John Schulman. 2021. https://arxiv.org/abs/2110.14168 Training verifiers to solve math word problems . ...

    2021 arXiv

  40. [52]

    Petros Drineas, Ravi Kannan, and Michael W Mahoney. 2006. Fast monte carlo algorithms for matrices ii: Computing a low-rank approximation to a matrix. SIAM Journal on computing, 36(1):158--183

    2006

  41. [53]

    Elias Frantar and Dan Alistarh. 2023. https://arxiv.org/abs/2301.00774 Sparsegpt: Massive language models can be accurately pruned in one-shot . Preprint, arXiv:2301.00774

    2023 arXiv

  42. [54]

    Smith, Wei-Chiu Ma, and Ranjay Krishna

    Xingyu Fu, Yushi Hu, Bangzheng Li, Yu Feng, Haoyu Wang, Xudong Lin, Dan Roth, Noah A. Smith, Wei-Chiu Ma, and Ranjay Krishna. 2024. https://arxiv.org/abs/2404.12390 Blink: Multimodal large language models can see but not perceive . Preprint, arXiv:2404.12390

    2024 arXiv

  43. [55]

    Yichao Fu, Xuewei Wang, Yuandong Tian, and Jiawei Zhao. 2025. https://arxiv.org/abs/2508.15260 Deep think with confidence . Preprint, arXiv:2508.15260

    2025 arXiv

  44. [56]

    Shangqian Gao, Chi-Heng Lin, Ting Hua, Tang Zheng, Yilin Shen, Hongxia Jin, and Yen-Chang Hsu. 2024. https://arxiv.org/abs/2410.11988 Disp-llm: Dimension-independent structural pruning for large language models . Preprint, arXiv:2410.11988

    2024 arXiv

  45. [57]

    Andrew Gordon, Zornitsa Kozareva, and Melissa Roemmele. 2012. https://aclanthology.org/S12-1052/ S em E val-2012 task 7: Choice of plausible alternatives: An evaluation of commonsense causal reasoning . In * SEM 2012: The First Joint Conference on Lexical and Computational Sem...

    2012

  46. [58]

    Aaron Grattafiori, Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Abhishek Kadian, Ahmad Al-Dahle, Aiesha Letman, Akhil Mathur, Alan Schelten, Alex Vaughan, Amy Yang, Angela Fan, Anirudh Goyal, Anthony Hartshorn, Aobo Yang, Archi Mitra, Archie Sravankumar, Artem Korenev, Art...

    2024 arXiv

  47. [59]

    Song Han, Jeff Pool, John Tran, and William J. Dally. 2015. https://arxiv.org/abs/1506.02626 Learning both weights and connections for efficient neural networks . Preprint, arXiv:1506.02626

    2015 arXiv

  48. [60]

    Dan Hendrycks, Collin Burns, Steven Basart, Andy Zou, Mantas Mazeika, Dawn Song, and Jacob Steinhardt. 2021. https://arxiv.org/abs/2009.03300 Measuring massive multitask language understanding . Preprint, arXiv:2009.03300

    2021 arXiv

  49. [61]

    Cheng-Ping Hsieh, Simeng Sun, Samuel Kriman, Shantanu Acharya, Dima Rekesh, Fei Jia, Yang Zhang, and Boris Ginsburg. 2024. https://arxiv.org/abs/2404.06654 Ruler: What's the real context size of your long-context language models? Preprint, arXiv:2404.06654

    2024 arXiv

  50. [62]

    Brown, Benjamin Chess, Rewon Child, Scott Gray, Alec Radford, Jeffrey Wu, and Dario Amodei

    Jared Kaplan, Sam McCandlish, Tom Henighan, Tom B. Brown, Benjamin Chess, Rewon Child, Scott Gray, Alec Radford, Jeffrey Wu, and Dario Amodei. 2020. https://arxiv.org/abs/2001.08361 Scaling laws for neural language models . Preprint, arXiv:2001.08361

    2020 arXiv

  51. [63]

    Donghyun Lee, Je-Yong Lee, Genghan Zhang, Mo Tiwari, and Azalia Mirhoseini. 2024. https://arxiv.org/abs/2404.08763 Cats: Contextually-aware thresholding for sparsity in large language models . Preprint, arXiv:2404.08763

    2024 arXiv

  52. [64]

    Yifan Li, Yifan Du, Kun Zhou, Jinpeng Wang, Wayne Xin Zhao, and Ji-Rong Wen. 2023 a . https://arxiv.org/abs/2305.10355 Evaluating object hallucination in large vision-language models . Preprint, arXiv:2305.10355

    2023 arXiv

  53. [65]

    Yucheng Li, Bo Dong, Chenghua Lin, and Frank Guerin. 2023 b . Compressing context to enhance inference efficiency of large language models. arXiv preprint arXiv:2310.06201

    2023 arXiv

  54. [66]

    Liu, Shang-Wen Li, and Hung-yi Lee

    Andy T. Liu, Shang-Wen Li, and Hung-yi Lee. 2021. https://doi.org/10.1109/taslp.2021.3095662 Tera: Self-supervised learning of transformer encoder representation for speech . IEEE/ACM Transactions on Audio, Speech, and Language Processing, 29:2351–2366

    2021

  55. [67]

    James Liu, Pragaash Ponnusamy, Tianle Cai, Han Guo, Yoon Kim, and Ben Athiwaratkun. 2025. https://arxiv.org/abs/2408.14690 Training-free activation sparsity in large language models . Preprint, arXiv:2408.14690

    2025 arXiv

  56. [68]

    Xinyin Ma, Gongfan Fang, and Xinchao Wang. 2023. https://arxiv.org/abs/2305.11627 Llm-pruner: On the structural pruning of large language models . Preprint, arXiv:2305.11627

    2023 arXiv

  57. [69]

    Stephen Merity, Caiming Xiong, James Bradbury, and Richard Socher. 2016. https://arxiv.org/abs/1609.07843 Pointer sentinel mixture models . Preprint, arXiv:1609.07843

    2016 arXiv

  58. [70]

    Ramesh Nallapati, Bowen Zhou, Cicero dos Santos, C a g lar Gu l c ehre, and Bing Xiang. 2016. https://doi.org/10.18653/v1/K16-1028 Abstractive text summarization using sequence-to-sequence RNN s and beyond . In Proceedings of the 20th SIGNLL Conference on Computational Natural...

    2016 doi

  59. [71]

    Vicky Zhao, Lili Qiu, and Dongmei Zhang

    Zhuoshi Pan, Qianhui Wu, Huiqiang Jiang, Menglin Xia, Xufang Luo, Jue Zhang, Qingwei Lin, Victor Rühle, Yuqing Yang, Chin-Yew Lin, H. Vicky Zhao, Lili Qiu, and Dongmei Zhang. 2024. https://arxiv.org/abs/2403.12968 Llmlingua-2: Data distillation for efficient and faithful task-...

    2024 arXiv

  60. [72]

    Yifan Peng, Kwangyoun Kim, Felix Wu, Prashant Sridhar, and Shinji Watanabe. 2023. https://arxiv.org/abs/2302.14132 Structured pruning of self-supervised pre-trained models for speech recognition and understanding . Preprint, arXiv:2302.14132

    2023 arXiv

  61. [73]

    Hassan Sajjad, Fahim Dalvi, Nadir Durrani, and Preslav Nakov. 2023. https://doi.org/10.1016/j.csl.2022.101429 On the effect of dropping layers of pre-trained transformer models . Computer Speech & Language, 77:101429

    2023

  62. [74]

    Jay Shah, Ganesh Bikshandi, Ying Zhang, Vijay Thakkar, Pradeep Ramani, and Tri Dao. 2024. https://arxiv.org/abs/2407.08608 Flashattention-3: Fast and accurate attention with asynchrony and low-precision . Preprint, arXiv:2407.08608

    2024 arXiv

  63. [75]

    Zico Kolter

    Mingjie Sun, Zhuang Liu, Anna Bair, and J. Zico Kolter. 2024. https://arxiv.org/abs/2306.11695 A simple and effective pruning approach for large language models . Preprint, arXiv:2306.11695

    2024 arXiv

  64. [76]

    Alon Talmor, Jonathan Herzig, Nicholas Lourie, and Jonathan Berant. 2019. https://arxiv.org/abs/1811.00937 Commonsenseqa: A question answering challenge targeting commonsense knowledge . Preprint, arXiv:1811.00937

    2019 arXiv

  65. [77]

    Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, ukasz Kaiser, and Illia Polosukhin. 2017. Attention is all you need. Advances in neural information processing systems, 30

    2017

  66. [78]

    Guangxuan Xiao, Yuandong Tian, Beidi Chen, Song Han, and Mike Lewis. 2024. Efficient streaming language models with attention sinks. In The Twelfth International Conference on Learning Representations

    2024

  67. [79]

    An Yang, Anfeng Li, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chang Gao, Chengen Huang, Chenxu Lv, Chujie Zheng, Dayiheng Liu, Fan Zhou, Fei Huang, Feng Hu, Hao Ge, Haoran Wei, Huan Lin, Jialong Tang, and 41 others. 2025. https://arxiv.org/abs/2505.09388 Qw...

    2025 arXiv

  68. [80]

    Jingyang Yuan, Huazuo Gao, Damai Dai, Junyu Luo, Liang Zhao, Zhengyan Zhang, Zhenda Xie, Y. X. Wei, Lean Wang, Zhiping Xiao, Yuqing Wang, Chong Ruan, Ming Zhang, Wenfeng Liang, and Wangding Zeng. 2025. https://arxiv.org/abs/2502.11089 Native sparse attention: Hardware-aligned ...

    2025 arXiv

  69. [81]

    Zhenyu Zhang, Ying Sheng, Tianyi Zhou, Tianlong Chen, Lianmin Zheng, Ruisi Cai, Zhao Song, Yuandong Tian, Christopher R \'e , Clark Barrett, and 1 others. 2023. H2o: Heavy-hitter oracle for efficient generative inference of large language models. Advances in Neural Information...

    2023

  70. [82]

    Yukun Zhu, Ryan Kiros, Rich Zemel, Ruslan Salakhutdinov, Raquel Urtasun, Antonio Torralba, and Sanja Fidler. 2015. Aligning books and movies: Towards story-like visual explanations by watching movies and reading books. In Proceedings of the IEEE international conference on com...

    2015

Pith tools

Reviewed August 14, 2026 · model on record in the stance chip above.