Pith. sign in

REVIEW 3 major objections 5 minor 300 references

A Fisher-weighted quantization objective aligns channel scaling with task loss, improving low-bit MLLM accuracy without auxiliary modules.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · deepseek-v4-flash

2026-08-01 08:30 UTC pith:4DQALQLD

load-bearing objection Fisher-weighted channel scaling is a useful, cheap empirical tweak for MLLM PTQ, but the loss-perturbation derivation overclaims; the method works better than its theory. the 3 major comments →

arxiv 2607.21076 v1 pith:4DQALQLD submitted 2026-07-23 cs.CV

C-PTQ: Fisher-weighted Channel-wise Sensitivity for Post-training Quantization of MLLMs

Jiameng Li , Han Zhou , Matthew B. Blaschko This is my paper
classification cs.CV
keywords post-training quantizationmultimodal large language modelsFisher informationchannel-wise scalingHessian approximationlow-bit quantizationtask sensitivityactivation-aware weight quantization
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

This paper claims that the standard practice of minimizing plain mean-squared quantization error when choosing channel-wise scaling factors is misaligned with what actually matters for a multimodal language model's task performance, because channels with small quantization error can still be the ones whose disruption most hurts the supervised fine-tuning loss. To fix this, C-PTQ replaces the MSE objective with a Fisher-weighted version: each channel's quantization residual is penalized by the average squared gradient of the SFT loss, computed on a 128-image calibration set. This is motivated as a tractable diagonal approximation of the Hessian of the task loss. The authors show that this simple change—only the objective in the scaling grid search changes—consistently improves accuracy over prior scaling-based methods at 3-bit weights and at 4-bit weights with 8- or 6-bit activations, across three MLLM families and eight benchmarks, without needing LoRA or other compensating modules. A sympathetic reader would care because it suggests a cheap, plug-in upgrade to an established compression technique, and because it reframes quantization sensitivity as a task-loss curvature question rather than a reconstruction-error question.

Core claim

On its own terms, the paper establishes that channel-wise scaling for post-training quantization can be guided by task sensitivity instead of reconstruction error. The discovery is that the loss perturbation caused by quantization, expanded to second order, is controlled by a quadratic form ΔYᵀHΔY; approximating the Hessian with the diagonal empirical Fisher matrix (the mean squared output gradient per channel) turns this into a simple weighted MSE in which each channel's error is weighted by its average squared gradient. Using this Fisher-weighted objective in the scaling-factor grid search yields state-of-the-art quantized accuracy on Qwen2.5VL, InternVL2, and LLaVA-OV at W3A16, W4A8, and

What carries the argument

The central object is the Fisher-weighted MSE objective (Eqs. 11 and 14): the quantization residual Δy_i per token is weighted element-wise by √f, where f_j is the diagonal empirical Fisher entry for channel j, computed as the mean over tokens of the squared gradient of the SFT loss with respect to the output activation. This converts the intractable Hessian quadratic form into a decoupled per-channel weighted error, so the scaling-factor search directly penalizes channels whose perturbation most affects the task loss. It is a one-time gradient pass plus a grid search, so the method inherits the efficiency of activation-aware scaling while adding task-loss curvature information.

Load-bearing premise

The load-bearing premise is that the empirical Fisher matrix, computed from squared output gradients on a 128-image calibration set, faithfully captures the curvature of the task loss; if these gradients do not reflect benchmark-relevant loss sensitivity, the Fisher-weighted objective is just a heuristic reweighting of the MSE.

What would settle it

Compute the exact diagonal Hessian of the SFT loss with respect to output activations on a small MLLM (or a single layer) and compare the ranking of channel weights to the empirical Fisher weights; if the rankings diverge substantially on channels where the two methods' predictions differ, then the Hessian-approximation story is wrong. Alternatively, measure per-channel loss change when quantizing one channel at a time and check whether Fisher weights predict that change better than plain MSE or activation magnitude.

Watch this falsifier. Get emailed when new claim-graph text bears on it.

If this is right

  • If the claim holds, any existing channel-wise scaling method can adopt the Fisher-weighted objective with a single extra gradient pass, potentially improving quantized MLLM accuracy without architectural changes.
  • The method's alignment of sensitivity and scaling at the channel level suggests that coarser modality- or token-level sensitivity measures lose information; channel-level curvature is sufficient.
  • Because calibration with text-only data works nearly as well, the method can be applied when multimodal calibration data is unavailable, unlike modality-aware methods.
  • The success of the diagonal Fisher approximation implies the dense Hessian's off-diagonal structure is not needed for scaling search, keeping the method practical for very large models.
  • Since the approach also works on plain LLMs, the benefit is not specific to multimodal models but generalizes to decoder-only quantization.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • This method is essentially a variance-weighted reconstruction error; one could test whether using the full covariance or a block-diagonal version helps in settings with correlated channels, beyond what the paper's dense-Fisher ablation shows.
  • The reliance on calibration gradients suggests a potential failure mode when the calibration distribution is far from deployment distribution; a stress-test would be calibrating on pure text vs. images with different task mixes and measuring channel-weight stability.
  • The paper leaves the vision encoder and projector unquantized; an extension would apply the same Fisher-weighted idea to those components, where the loss landscape may have different curvature.
  • The claim that Fisher approximates Hessian could be directly probed by comparing Fisher weights to exact Hessian diagonals on a small model; if they diverge, the gains might instead come from variance weighting acting as a regularizer.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. The paper proposes C-PTQ, a post-training quantization method for multimodal large language models. The method modifies the standard channel-wise scaling (CWS) search used by AWQ: instead of minimizing the plain MSE of the layer output residual, it minimizes a Fisher-weighted MSE in which each output channel's residual is penalized by the average squared gradient of the SFT loss with respect to that channel (Eqs. 11, 14). The authors motivate this objective by a Taylor expansion of the task loss perturbation, replacing the Hessian by the empirical Fisher matrix and then diagonalizing it. Experiments are conducted on Qwen2.5VL-7B/32B, InternVL2-8B, and LLaVA-OV-7B under W3A16, W4A8, and W4A6 settings across eight standard multimodal benchmarks, with additional LLM experiments in the appendix. The paper reports consistent average improvements over AWQ, MBQ, QIG, SQ, and MASQuant, with quantization runtime comparable to MBQ and much lower than QIG. The code is released.

Significance. If the empirical gains hold, C-PTQ is a useful and cheap contribution: it is a plug-in replacement for the scoring objective in AWQ-style scaling, requiring only one additional gradient pass over a 128-image calibration set, and it avoids LoRA-style auxiliary parameters. The paper's strengths include a broad evaluation across three MLLM families, multiple bit-width settings, and several ablations that directly address the main design choices: calibration-set robustness (Tab. 4), dense versus diagonal Fisher (Tab. 6), and the effect of the first-order term (Tab. 6 and App. A.3). The methodological narrative, however, overstates the theoretical grounding. The derivation from a second-order loss perturbation to the Fisher-weighted MSE relies on two approximations—dropping the first-order term and replacing the Hessian with the empirical Fisher—neither of which is valid in the way presented. The empirical results can support the method as a sensitivity heuristic, but the current text claims a principled bridge from task loss to quantization objective that the presented evidence does not establish.

major comments (3)
  1. [§3.2, Eq. (7)] The vanishing-first-order argument is not transferable from weights to activations. OBD assumes a local optimum in weight space, where ∂L/∂W = 0. Here the expansion is in output activations Y, and ∂L/∂Y is generally not zero for a converged network. The manuscript's own choice f_j = E[g^2_ij] (Eq. 10) also contradicts the statement that the expected gradient is close to zero. The ablation in Tab. 6 suggests that including the first-order term changes the average by less than 0.3 points, which is useful empirical evidence, but it does not justify Eq. (7). Please either replace the derivation with an explicit sensitivity-heuristic framing or provide evidence that the first-order term is small for activation perturbations on the calibration distribution.
  2. [§3.2, Eqs. (8)–(11)] The identification H ≈ F is not generally valid for the SFT cross-entropy loss on layer outputs. For final logits with softmax probabilities p and one-hot target t, the exact Hessian diagonal is diag(p) − pp^T, whereas the diagonal of the empirical Fisher is E[(p_j − t_j)^2]; these are different objects. For intermediate layers there is no given argument that the empirical Fisher of output gradients is a faithful Hessian proxy. Tab. 6 compares dense and diagonal Fisher variants, and Tab. 5 compares against GPTQ's activation-based Hessian; neither validates Fisher against the exact Hessian. Consequently, the statements that C-PTQ is 'strictly guided by task sensitivity' and 'bridges the optimization gap' (Sec. 3.2) are not established by the derivation. A direct comparison of the Fisher diagonal with the true Hessian diagonal on a small layer, or a clear reframing as a gradient-variance w
  3. [§3.3, Eq. (12)–(14)] The objective is written as proportional to a Fisher-weighted MSE, but the implementation normalizes the Fisher factors by their mean (App. A.1) and searches the scaling exponent α per layer. The interaction between normalization, α, and the scale of ΔY is not discussed; as written, Eq. (14) is not strictly proportional to the loss perturbation of Eq. (11). This is a smaller point, but it contributes to the gap between the derivation and the deployed objective.
minor comments (5)
  1. [Table 5 caption] 'even worse then the naive' should be 'even worse than the naive'.
  2. [Eq. (3)] The norm notation ‖∆y_i‖ is missing the subscript 2 used in Eq. (4). Also, the definitions of N_v and N_t in the sums should be made explicit (vision tokens vs. text tokens).
  3. [Figures 1 and 5] The y-axis labels 'Avg error' and 'Avg grad' should specify whether absolute or squared values are plotted; currently the reader must infer this from the text.
  4. [Algorithm 1] The 40-step grid search mentioned in Sec. 4.1 is not reflected in the algorithm pseudocode; adding the search range and number of steps would make the method fully reproducible.
  5. [App. A.3] Eqs. (15)–(17) define the first-order-augmented objective and an upper bound used in Tab. 6, but the exact implementation of the upper bound (e.g., how the absolute values are pooled over tokens and channels) is not specified. Please clarify.

Circularity Check

0 steps flagged

No circularity: Fisher-weighted objective is a calibration-set sensitivity heuristic, not a fit to benchmark results.

full rationale

I find no circular step. The derivation (Eqs. 5–14) starts from a Taylor expansion of the SFT loss, approximates the Hessian by the empirical Fisher matrix, and uses the diagonal Fisher weights f_j = E[g^2_ij] as a per-channel importance measure. These weights are computed from calibration gradients, and the scaling vector is then selected by minimizing the weighted reconstruction residual. The reported benchmark accuracies are external to this optimization: no benchmark label or final accuracy is used to define f_j or s, so the gains are not forced by construction. The Hessian/Fisher approximation and the dropped first-order term are debatable modeling choices—backpropagated gradients at hidden layers are not zero, and H≈F is not exact for cross-entropy—but these are validity concerns, not circularity. The only self-citation (Li, Tiulpin, and Blaschko 2026, MI-Pruner) appears in related work and is not load-bearing. The central claim is therefore self-contained with respect to its inputs.

Axiom & Free-Parameter Ledger

2 free parameters · 5 axioms · 0 invented entities

The method introduces no new physical or architectural entities. It relies on four approximations: Taylor truncation, vanishing first-order term, empirical Fisher as Hessian, and diagonal Fisher. These are standard in compression but are load-bearing assumptions here.

free parameters (2)
  • Per-channel Fisher weights f_j = estimated from calibration gradients; normalized by mean
    Computed as E_{:,j}[g^2_{ij}] on the calibration set and used to weight the reconstruction error in Eq. (14). The values are data-dependent and central to the method.
  • Scaling exponent alpha (per layer) = grid-searched over 40 steps per layer
    Determines the channel-wise scale s = x^alpha as in AWQ; selected by minimizing the Fisher-weighted error. Not provided as a fixed value.
axioms (5)
  • standard math Taylor expansion of the task loss around the full-precision output, with third-order terms dropped (Eq. 5-6)
    Foundation for the loss perturbation; validity requires the quantization residual to be small, which is not guaranteed at 3-4 bits.
  • domain assumption First-order gradient term vanishes because the model is at a local optimum (OBD assumption)
    Used to drop g^T DeltaY in Eq. (7); empirically tested in App. A.3 but not guaranteed on a 128-sample calibration set.
  • domain assumption Empirical Fisher information approximates the Hessian of the loss w.r.t. output activations (Eq. 8)
    Holds for softmax cross-entropy at the true model distribution, but here gradients are computed on a finite calibration set with actual labels; not proven.
  • domain assumption Diagonal approximation of the Fisher matrix is sufficient to capture channel importance (Eq. 10)
    Ablated against dense Fisher (Table 6) with comparable results, but the diagonal still ignores cross-channel correlations.
  • domain assumption Channel-wise scaling via s = x^alpha with grid search is an effective parameterization (borrowed from AWQ)
    Assumed without derivation; standard in the field.

pith-pipeline@v1.3.0-alltime-deepseek · 17301 in / 12987 out tokens · 121382 ms · 2026-08-01T08:30:38.820833+00:00 · methodology

0 comments
read the original abstract

Multimodal large language models (MLLMs) require huge memory and computational costs, which limits their practical deployment. Post-training quantization (PTQ) techniques offer an efficient solution for model compression and inference acceleration. Yet, the quantized model faces performance degradation due to outlier channels, which are highly sensitive to quantization and substantially impair activation fidelity and task accuracy. To protect these salient channels during quantization, existing PTQ methods leverage modality- or token-level metrics to guide channel-wise scaling (CWS) of LLM decoders. However, these orthogonal measurements fail to capture channel-wise impacts on task-specific loss, and the misalignment between importance and scaling factors ultimately leads to suboptimal performance. To address this issue, we propose C-PTQ, a unified channel-wise PTQ method that harmonizes task-specific loss perturbation and quantization error. Motivated by second-order derivatives, we design a Fisher-weighted objective as a tractable Hessian approximation, seamlessly injecting task sensitivity into the scaling process. Notably, we achieve state-of-the-art performance without auxiliary modules like LoRA, thereby maintaining high efficiency. Experiments on Qwen2.5VL, InternVL2 and LLaVA-OV across 8 benchmarks demonstrate our effectiveness in both weight-only and weight-activation settings.

Figures

Figures reproduced from arXiv: 2607.21076 by Han Zhou, Jiameng Li, Matthew B. Blaschko.

Figure 1
Figure 1. Figure 1: Visualization of channel-wise quantization er￾ror and gradients (Qwen2.5VL-7B). The round-to-nearest squared quantization error and squared gradients show a mis￾alignment (red arrow). , which compromises the channel￾wise scaling search. As shown in [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. Figure 2: Overview. (a) Channel-wise scaling (Lin et al. 2024) aims to smooth outlier channels with large magnitudes to reduce quantization error. However, channels with minimal errors can still exhibit substantial SFT gradients with degraded performance, which calls for fine-grained sensitivity measures. (b) The block layouts show the interest of dimension on sensitivity metrics, including modality- (Li et al. 2025… view at source ↗
Figure 3
Figure 3. Figure 3: Our Fisher-weighted quantization objective. The diagonal Fisher F˜ works as channel-wise importance in quantization objective. For simplification, we assume to￾kens Nk=5 and output channels Nout c =4 here. From Hessian to Fisher Information. Since computing the exact Hessian matrix is less feasible for MLLMs, we ap￾proximate H using the empirical Fisher Information Matrix F computed on the calibration set.… view at source ↗
Figure 4
Figure 4. Figure 4: Ablation study on channel-wise importance (LLaVA-OV-7B). We compare our diagonal Fisher against the mean absolute activation and gradient-based channel￾wise importance metrics to highlight our significance. values (i.e., mean:,j (|yij |)) and the mean absolute gradient values (i.e., mean:,j (|gij |)). The results in [PITH_FULL_IMAGE:figures/full_fig_p006_4.png] view at source ↗
Figure 5
Figure 5. Figure 5: Visualization of channel-wise quantization error and gradients (Qwen2.5VL-7B, InternVL2-7B). We recommend viewing the layer-wise subfigure vertically. The red arrows indicate misaligned channels, where small errors correspond to disproportionately large gradients. Under such misalignment, optimizing towards minimizing quantization error can induce excessively large gradients and poor downstream performance… view at source ↗
Figure 6
Figure 6. Figure 6: Ablation study on more model scales (W3A16). Our method maintains SOTA performance across scales. perturbation ∆L in Eqn. (11) is formulated as: ∆L ≈ g ⊤∆Y + 1 2 ∆Y⊤diag(f) ∆Y (15) = X Nk i=1 (g ⊤∆yi + 1 2 ∥ √ f ⊙ ∆yi∥ 2 ). (16) Leveraging the triangle inequality, we obtain the following upper bound on the loss perturbation: ∆L ≤ X Nk i=1 (|g| ⊤|∆yi | + 1 2 ∥ √ f ⊙ ∆yi∥ 2 ). (17) The experimental results a… view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

300 extracted references · 63 linked inside Pith

  1. [1]

    Scaling Learning Algorithms Towards

    Bengio, Yoshua and LeCun, Yann , booktitle =. Scaling Learning Algorithms Towards

  2. [2]

    arXiv preprint arXiv:2601.07107 , year=

    MEDVISTAGYM: A Scalable Training Environment for Thinking with Medical Images via Tool-Integrated Reinforcement Learning , author=. arXiv preprint arXiv:2601.07107 , year=

    arXiv

  3. [3]

    arXiv preprint arXiv:2509.07864 , year=

    D-LEAF: Localizing and Correcting Hallucinations in Multimodal LLMs via Layer-to-head Attention Diagnostics , author=. arXiv preprint arXiv:2509.07864 , year=

    arXiv

  4. [4]

    ICLR , year=

    Efficient streaming language models with attention sinks , author=. ICLR , year=

  5. [5]

    CVPR , year=

    IBISAgent: Reinforcing Pixel-Level Visual Reasoning in MLLMs for Universal Biomedical Object Referring and Segmentation , author=. CVPR , year=

  6. [6]

    IEEE Transactions on Medical Imaging , year =

    MedLVR: Latent Visual Reasoning for Reliable Medical Visual Question Answering , author =. IEEE Transactions on Medical Imaging , year =

  7. [7]

    ICML , year=

    Incentivizing Tool-augmented Thinking with Images for Medical Image Analysis , author=. ICML , year=

  8. [8]

    arXiv preprint arXiv:2508.03351 , year=

    Vlmq: Efficient post-training quantization for large vision-language models via hessian augmentation , author=. arXiv preprint arXiv:2508.03351 , year=

    arXiv

  9. [9]

    CVPR , year=

    Mbq: Modality-balanced quantization for large vision-language models , author=. CVPR , year=

  10. [10]

    ICML , year=

    VEQ: Modality-Adaptive Quantization for MoE Vision-Language Models , author=. ICML , year=

  11. [11]

    Elias Frantar and Saleh Ashkboos and Torsten Hoefler and Dan Alistarh , booktitle=

  12. [12]

    MLSys , year=

    AWQ: Activation-aware Weight Quantization for LLM Compression and Acceleration , author=. MLSys , year=

  13. [13]

    Radiology Advances , year=

    How to implement a radiologist led whole-body MRI screening program , author=. Radiology Advances , year=

  14. [14]

    ICLR , year=

    Medagent-pro: Towards evidence-based multi-modal medical diagnosis via reasoning agentic workflow , author=. ICLR , year=

  15. [15]

    TPAMI , year=

    Moviechat+: Question-aware sparse memory for long video question answering , author=. TPAMI , year=

  16. [16]

    CVPRW , year=

    On speculative decoding for multimodal large language models , author=. CVPRW , year=

  17. [17]

    EMNLP , year=

    Specvlm: Enhancing speculative decoding of video llms via verifier-guided token pruning , author=. EMNLP , year=

  18. [18]

    arXiv preprint arXiv:2604.16175 , year=

    MARCH: Multi-Agent Radiology Clinical Hierarchy for CT Report Generation , author=. arXiv preprint arXiv:2604.16175 , year=

    Pith/arXiv arXiv

  19. [19]

    ACL , year=

    MARVEL: Unlocking the Multi-Modal Capability of Dense Retrieval via Visual Module Plugin , author=. ACL , year=

  20. [20]

    arXiv preprint arXiv:2603.16876 , year=

    Multi-Modal Multi-Agent Reinforcement Learning for Radiology Report Generation: Radiologist-Like Workflow with Clinically Verifiable Rewards , author=. arXiv preprint arXiv:2603.16876 , year=

    Pith/arXiv arXiv

  21. [21]

    arXiv preprint arXiv:2603.13956 , year=

    EviAgent: Evidence-Driven Agent for Radiology Report Generation , author=. arXiv preprint arXiv:2603.13956 , year=

    arXiv

  22. [22]

    Eyes"," Hands

    EchoAgent: Towards Reliable Echocardiography Interpretation with" Eyes"," Hands" and" Minds" , author=. arXiv preprint arXiv:2604.05541 , year=

    Pith/arXiv arXiv

  23. [23]

    NAACL , year=

    Fact-Aware Multimodal Retrieval Augmentation for Accurate Medical Radiology Report Generation , author=. NAACL , year=

  24. [24]

    EMNLP , year=

    Video Compression Commander: Plug-and-Play Inference Acceleration for Video Large Language Models , author=. EMNLP , year=

  25. [25]

    Abdi and Dongsheng Li and Chin-Yew Lin and Yuqing Yang and Lili Qiu , title =

    Huiqiang Jiang and Yucheng Li and Chengruidong Zhang and Qianhui Wu and Xufang Luo and Surin Ahn and Zhenhua Han and Amir H. Abdi and Dongsheng Li and Chin-Yew Lin and Yuqing Yang and Lili Qiu , title =. NeurIPS , year =

  26. [26]

    NeurIPS D&B Track , year=

    Radgraph: Extracting clinical entities and relations from radiology reports , author=. NeurIPS D&B Track , year=

  27. [27]

    Li, Yucheng and Jiang, Huiqiang and Zhang, Chengruidong and Wu, Qianhui and Luo, Xufang and Ahn, Surin and Abdi, Amir H and Li, Dongsheng and Gao, Jianfeng and Yang, Yuqing and Qiu, Lili , booktitle=

  28. [28]

    arXiv preprint arXiv:2510.10991 , year=

    A Survey on Agentic Multimodal Large Language Models , author=. arXiv preprint arXiv:2510.10991 , year=

    arXiv

  29. [29]

    arXiv preprint arXiv:2601.12538 , year=

    Agentic reasoning for large language models , author=. arXiv preprint arXiv:2601.12538 , year=

    Pith/arXiv arXiv

  30. [30]

    ICLR , year=

    Flow matching for generative modeling , author=. ICLR , year=

  31. [31]

    ICLR , year=

    - MoD: Exploring Mixture-of-Depth Adaptation for Multimodal Large Language Models , author=. ICLR , year=

  32. [32]

    EMNLP , pages=

    Pointwise mutual information based metric and decoding strategy for faithful generation in document grounded dialogs , author=. EMNLP , pages=

  33. [33]

    arXiv preprint arXiv:2210.13210 , year=

    Mutual information alleviates hallucinations in abstractive summarization , author=. arXiv preprint arXiv:2210.13210 , year=

    Pith/arXiv arXiv

  34. [34]

    ICML , year=

    Sparsevlm: Visual token sparsification for efficient vision-language model inference , author=. ICML , year=

  35. [35]

    CVPR , year=

    Accelerating Multimodal Large Language Models by Searching Optimal Vision Token Reduction , author=. CVPR , year=

  36. [36]

    CVPR , year=

    Quant Experts: Token-aware Adaptive Error Reconstruction with Mixture of Experts for Large Vision-Language Models Quantization , author=. CVPR , year=

  37. [37]

    NeurIPS , year=

    Q-vlm: Post-training quantization for large vision-language models , author=. NeurIPS , year=

  38. [38]

    Chen, Junying and Gui, Chi and Gao, Anningzhe and Ji, Ke and Wang, Xidong and Wan, Xiang and Wang, Benyou , booktitle=

  39. [39]

    arXiv preprint arXiv:2604.12144 , year=

    VERITAS: Verifiable Epistemic Reasoning for Image-Derived Hypothesis Testing via Agentic Systems , author=. arXiv preprint arXiv:2604.12144 , year=

    Pith/arXiv arXiv

  40. [40]

    arXiv preprint arXiv:2603.09909 , year=

    Medmaslab: A unified orchestration framework for benchmarking multimodal medical multi-agent systems , author=. arXiv preprint arXiv:2603.09909 , year=

    arXiv

  41. [41]

    MIDL , year=

    CARE: Confidence-aware Ratio Estimation for Medical Biomarkers , author=. MIDL , year=

  42. [42]

    ACL Findings , year=

    MedAgents: Large Language Models as Collaborators for Zero-shot Medical Reasoning , author=. ACL Findings , year=

  43. [43]

    CVPR , year=

    AdaptVision: Efficient Vision-Language Models via Adaptive Visual Acquisition , author=. CVPR , year=

  44. [44]

    CVPR , year=

    Unified Spatiotemporal Token Compression for Video-LLMs at Ultra-Low Retention , author=. CVPR , year=

  45. [45]

    arXiv preprint arXiv:2604.00886 , year=

    PixelPrune: Pixel-Level Adaptive Visual Token Reduction via Predictive Coding , author=. arXiv preprint arXiv:2604.00886 , year=

    arXiv

  46. [46]

    , journal=

    Li, Jiameng and Tiulpin, Aleksei and Blaschko, Matthew B. , journal=

  47. [47]

    AAAI , year=

    Mmg-vid: Maximizing marginal gains at segment-level and token-level for efficient video llms , author=. AAAI , year=

  48. [48]

    arXiv preprint arXiv:2603.05663 , year=

    Keeping the Evidence Chain: Semantic Evidence Allocation for Training-Free Token Pruning in Video Temporal Grounding , author=. arXiv preprint arXiv:2603.05663 , year=

    Pith/arXiv arXiv

  49. [49]

    arXiv preprint arXiv:2602.13315 , year=

    IDPruner: Harmonizing Importance and Diversity in Visual Token Pruning for MLLMs , author=. arXiv preprint arXiv:2602.13315 , year=

    arXiv

  50. [50]

    arXiv preprint arXiv:2604.00757 , year=

    IWP: Token Pruning as Implicit Weight Pruning in Large Vision Language Models , author=. arXiv preprint arXiv:2604.00757 , year=

    Pith/arXiv arXiv

  51. [51]

    AAAI , year=

    PosPrune: Visual Token Pruning with Positional Bias Correction for Efficient Large Vision-Language Models , author=. AAAI , year=

  52. [52]

    ICCV , year=

    Beyond text-visual attention: Exploiting visual cues for effective token pruning in vlms , author=. ICCV , year=

  53. [53]

    CVPR , year=

    UniComp: Rethinking Video Compression Through Informational Uniqueness , author=. CVPR , year=

  54. [54]

    NeurIPS , year=

    Holitom: Holistic token merging for fast video large language models , author=. NeurIPS , year=

  55. [55]

    ICLR , year=

    FlashVID: Efficient Video Large Language Models via Training-free Tree-based Spatiotemporal Token Merging , author=. ICLR , year=

  56. [56]

    arXiv preprint arXiv:2603.28610 , year=

    ResAdapt: Adaptive Resolution for Efficient Multimodal Reasoning , author=. arXiv preprint arXiv:2603.28610 , year=

    arXiv

  57. [57]

    ICLR , year=

    PPE: Positional Preservation Embedding for Token Compression in Multimodal Large Language Models , author=. ICLR , year=

  58. [58]

    arXiv preprint arXiv:2501.13826 , year=

    Video-mmmu: Evaluating knowledge acquisition from multi-discipline professional videos , author=. arXiv preprint arXiv:2501.13826 , year=

    Pith/arXiv arXiv

  59. [59]

    ICCV , year=

    Lvbench: An extreme long video understanding benchmark , author=. ICCV , year=

  60. [60]

    CVPR , year=

    Visionzip: Longer is better but not necessary in vision language models , author=. CVPR , year=

  61. [61]

    CVPR , year=

    ApET: Approximation-Error Guided Token Compression for Efficient VLMs , author=. CVPR , year=

  62. [62]

    ECCV , pages=

    An image is worth 1/2 tokens after layer 2: Plug-and-play inference acceleration for large vision-language models , author=. ECCV , pages=

  63. [63]

    ECCV , year=

    Microsoft coco: Common objects in context , author=. ECCV , year=

  64. [64]

    , author=

    OpenImages: A public dataset for large-scale multi-label and multi-class image classification. , author=. Dataset available from https://github.com/openimages , year=

  65. [65]

    arXiv preprint arXiv:2503.11549 , year=

    Similarity-aware token pruning: Your vlm but faster , author=. arXiv preprint arXiv:2503.11549 , year=

    Pith/arXiv arXiv

  66. [66]

    Algorithmic Learning Theory , pages=

    Submodular combinatorial information measures with applications in machine learning , author=. Algorithmic Learning Theory , pages=. 2021 , organization=

    2021

  67. [67]

    CVPR , year=

    Lifting the Veil on Visual Information Flow in MLLMs: Unlocking Pathways to Faster Inference , author=. CVPR , year=

  68. [68]

    arXiv preprint arXiv:2604.03414 , year=

    KiToke: Kernel-based Interval-aware Token Compression for Video Large Language Models , author=. arXiv preprint arXiv:2604.03414 , year=

    Pith/arXiv arXiv

  69. [69]

    arXiv preprint arXiv:2603.16932 , year=

    Look Where It Matters: High-Resolution Crops Retrieval for Efficient VLMs , author=. arXiv preprint arXiv:2603.16932 , year=

    arXiv

  70. [70]

    NeurIPS , year=

    Visionthink: Smart and efficient vision language model via reinforcement learning , author=. NeurIPS , year=

  71. [71]

    NeurIPS , year=

    Videomae: Masked autoencoders are data-efficient learners for self-supervised video pre-training , author=. NeurIPS , year=

  72. [72]

    Proceedings of the 49th annual meeting of the association for computational linguistics: human language technologies , year=

    A class of submodular functions for document summarization , author=. Proceedings of the 49th annual meeting of the association for computational linguistics: human language technologies , year=

  73. [73]

    Proceedings of the 21st annual international ACM SIGIR conference on Research and development in information retrieval , year=

    The use of MMR, diversity-based reranking for reordering documents and producing summaries , author=. Proceedings of the 21st annual international ACM SIGIR conference on Research and development in information retrieval , year=

  74. [74]

    arXiv preprint physics/0004057 , year=

    The information bottleneck method , author=. arXiv preprint physics/0004057 , year=

    Pith/arXiv arXiv

  75. [75]

    CVPR , year=

    VLM-PTQ: Efficient Post-Training Quantization for Large Vision-Language Models , author=. CVPR , year=

  76. [76]

    CVPR Findings , year=

    ForestPrune: High-ratio Visual Token Compression for Video Multimodal Large Language Models via Spatial-Temporal Forest Modeling , author=. CVPR Findings , year=

  77. [77]

    NeurIPS , year=

    Vision Transformers with Self-Distilled Registers , author=. NeurIPS , year=

  78. [78]

    NeurIPS , year=

    What matters when building vision-language models? , author=. NeurIPS , year=

  79. [79]

    and Ermon, Stefano and Rudra, Atri and R

    Dao, Tri and Fu, Daniel Y. and Ermon, Stefano and Rudra, Atri and R. Flash. NeurIPS , year=

  80. [80]

    Dao, Tri , booktitle=. Flash

Showing first 80 references.