Pith. sign in

REVIEW 3 major objections 6 minor 40 references

qZACH-ViT: Quantization-Aware Intrinsic Explanations with Recursive Attribution-Stabilized Optimization

T0 review · 3 major / 6 minor · reviewed 2026-08-01 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read This paper claims that a compact zero-token vision transformer whose class logit is exactly the spatial average of patch-level evidence can be quantized to executable INT8 graphs without losing decisions or intrinsic explanations—and that t

desk verdict The deployment-fidelity results are the real contribution and they hold up; the all-seven-datasets improvement claim is real but tied to a self-made baseline that needs stronger provenance. read the letter →

arxiv 2607.15421 v1 pith:PN2B2GOP submitted 2026-07-16 cs.LG cs.CV

Athanasios Angelakis This is my paper · ORCID
classification cs.LGcs.CV
keywords quantization-awaretrainingintrinsicexplainabilityvisiontransformersINT8deploymentgradientsurgerymedicalimageclassificationzero-tokenarchitectureexactlogitdecomposition
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper sets out to prove that interpretability and quantization do not have to be traded off: a compact vision transformer can be built so that its class logit is literally the spatial average of patch-level evidence, then trained with quantization-aware W8A8 and converted to executable INT8 graphs that retain both decisions and evidence. Across 964,920 source-to-INT8 comparisons, converted models agree with their full-precision sources 99.9751% of the time, shift the primary metric by only 0.000133 on average, and keep intrinsic maps nearly identical (cosine 0.999955, rank correlation 0.9944, top-10% overlap 0.9692). The deployed INT8 models actually improve on the FP32 baseline mean on all seven datasets (+0.0313 with plain training, +0.0368 with the new optimizer), while producing artifacts 70% smaller and 1.4–2.4× faster on CPU. A second contribution, RASO, norm-matches and conflict-projects the attribution gradient so explanation objectives cannot oppose the classification gradient; it significantly lowers sufficiency error and improves noise stability, but does not dominate every faithfulness metric.

What carries the argument

The load-bearing identity is exact raw-evidence completeness: z = ρz^(1) + (1−ρ)z^(2), with z^(r) = (1/N)Σ_i e_i^(r), so each stage logit is the spatial average of patch-level class-evidence head outputs and the final logit is a fixed weighted combination (ρ = 0.25). This makes the explanation part of the predictor rather than an auxiliary head trained to imitate a post-hoc map. The second mechanism is RASO: it computes separate classification and attribution gradients, rescales the attribution gradient to the norm of the classification gradient, and if the inner product is negative, removes the component parallel to the classification gradient before adding λ_a = 0.1 times the result to the

What would settle it

Rerun the seven-dataset protocol with the original published ZACH-ViT implementation as the FP32 baseline instead of the parameter-matched reimplementation; if the deployed INT8 qZACH-ViT models no longer exceed that baseline on all seven datasets, the central predictive-gain claim collapses.

Watch

Extended reading notes

Core claim

The central claim is that qZACH-ViT's intrinsic evidence is not a post-hoc saliency map but the accounting mechanism of the prediction itself: the final logit is a weighted average of two stage logits, each of which is the spatial mean of per-patch evidence vectors (Eq. 3), so the raw evidence exactly reconstructs the logit by construction. The paper then shows this property survives real quantization: all 16 learned projections become signed INT8 matrix multiplications with INT32 accumulation in executable graphs, and the converted models retain predictions at 99.9751% exact agreement, primary metrics within a maximum absolute change of 0.004386, and intrinsic maps with near-unit similarity

Load-bearing premise

The headline 'improves on all seven datasets' is measured against the author's own parameter-matched reimplementation of the published baseline rather than the original implementation, so if that reimplementation is weaker than the original, the gains could reflect yardstick drift instead of the value of qZACH-ViT or RASO.

Editorial extensions

If this is right

  • Intrinsic explanations can be part of a deployable integer-quantized model rather than a post-training add-on, so evidence stability can be audited at deployment time.
  • A compact zero-token architecture with recursive evidence heads achieves exact logit decomposition with negligible parameter overhead (129–1,419 extra parameters).
  • RASO's asymmetric gradient surgery improves sufficiency error and input-noise stability for the attribution objective without sacrificing classification, but task-dependent best-condition results mean no single optimizer dominates.
  • The deployment audits—16 signed INT8 projections, 99.9751% prediction agreement, 0.000133 mean metric change—provide a template for verifying that converted models execute the intended integer operations.
  • Post-hoc explainers such as gradient-based attention rollout can still beat intrinsic maps on some faithfulness metrics, so intrinsic evidence is a fast, deployment-stable complement rather than a universal replacement.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • If the exact-completeness identity is treated as a contract, the same framework could provide per-patch contribution certificates for regulated settings; the paper itself stops short of any clinical claim.
  • The consistent gains over the FP32 baseline may come partly from the architecture change and quantization-aware regularization rather than from explanation optimization—qZACH-ViT + Adam already improves 53/70 pairs—so a natural ablation would separate the evidence heads from the QAT procedure.
  • RASO is a specific instance of gradient surgery; comparing it with established multi-objective gradient methods on the same controlled protocol would clarify whether its value comes from the asymmetric projection or from the recursive attribution objective itself.
  • Because normalized maps are not themselves additive logit decompositions, the zero-reconstruction-error audit certifies bookkeeping, not clinical meaning; localization-grounded evaluation would be the real test of whether the preserved evidence is medically useful.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 6 minor

Summary. The paper introduces qZACH-ViT, a quantization-aware extension of the zero-token, position-free ZACH-ViT backbone, and RASO, an asymmetric gradient-combination procedure that norm-matches the attribution gradient to the classification gradient and projects away conflicting components. The method adds two patch-level evidence heads whose raw weighted average reconstructs the class logit, plus a W8A8 quantization-aware training path. The authors evaluate four controlled conditions (FP32 ZACH-ViT + Adam; qZACH-ViT + Adam; qZACH-ViT + Adam + attribution loss; qZACH-ViT + RASO) on seven MedMNIST datasets with 50 training images per class and ten seeds, completing 280 runs. All 210 qZACH-ViT checkpoints are converted to actual mixed-precision ONNX INT8 graphs with 16 signed MatMulInteger projections. The main empirical claims are that deployed INT8 qZACH-ViT improves the FP32 ZACH-ViT baseline mean on all seven datasets, that source-to-INT8 prediction agreement is 99.9751% across 964,920 comparisons, that intrinsic maps retain high similarity after conversion, and that RASO significantly improves sufficiency error and input-noise stability over the attribution-loss control without dominating all XAI metrics. The paper is careful to report both positive and negative results and includes extensive audits: graph inspection, runtime profiling, parameter-randomization checks, and reproducibility archives.

Significance. If the empirical claims hold, this is a useful contribution to compact, intrinsically explainable vision transformers and their deployment. The paper ships a substantial controlled study: 280 training runs, shared initial states among the three qZACH conditions, actual ONNX INT8 conversion and execution audits, multiplicity-corrected paired tests, and 964,920 source-to-INT8 prediction comparisons. The exact completeness property in Eq. (3) is correctly presented as an algebraic identity of the evidence-head construction, not as a semantic guarantee. The deployment validation is unusually thorough: every converted graph is checked for the presence and execution of integer matrix multiplications, and the limitations section is candid about mixed-precision operation, runtime-stack confounding, and the absence of localization ground truth. The main risk to the headline claims is external-validity: the FP32 ZACH-ViT baseline is a self-made PyTorch reimplementation explicitly not claimed to numerically replicate the original TensorFlow implementation, and the 'improves on all seven datasets' statement depends on that baseline. The paper also uses an unconventional thresholded AUC@0.5 pri

major comments (3)
  1. [§4.1; Limitations 9; Abstract; §5.1] The load-bearing claim that deployed INT8 qZACH-ViT 'improves the FP32 ZACH-ViT baseline mean on all seven datasets' is measured against a PyTorch reimplementation that the authors state is 'not ... an assertion of numerical replication of the earlier TensorFlow implementation.' If the reimplementation is unintentionally weaker than the original ZACH-ViT—through missing training details such as ShuffleStrides augmentation, initialization, or framework-specific numerics—the reported mean gains (+0.0313 and +0.0368) would partly reflect a yardstick artifact. Internal controls across the four conditions do not address this external-validity gap. Please validate the baseline against the original implementation or published ZACH-ViT results on the same protocol; if that is not possible, revise the abstract, §5.1, and Conclusion to restrict the claim to 'our parameter-matched PyTorch baseline'
  2. [§4.4; Appendix B.1, Table 8] The primary binary metric is AUC@0.5, the ROC-AUC of predictions thresholded at 0.5, rather than conventional probability AUROC. The 'all seven datasets' claim depends on this inherited metric: on PneumoniaMNIST, qZACH-ViT + Adam shows a gain in AUC@0.5 (0.746 vs 0.738) but a slight decrease in conventional probability AUROC (0.836 vs 0.837). Please make this metric dependence explicit in the abstract or temper the all-seven claim, and discuss whether thresholded AUC is the appropriate primary measure for these binary tasks. The secondary table is a good start but the main text should not let the custom metric carry the headline.
  3. [§3.2, §3.5, §5.1] The predictive comparison between qZACH-ViT + Adam and the ZACH-ViT baseline conflates the architectural extension (evidence heads, token-wise MLP, recursive fusion, float-shadow path) with quantization-aware training. The paper does not include a full-precision qZACH-ViT + Adam condition, so it cannot isolate whether the observed predictive gains come from QAT, from the added evidence head parameters, or from the changed training objective. This does not invalidate the qZACH-ViT contribution, but it should be stated more carefully; currently the abstract and §5.1 attribute the gains to the deployed qZACH-ViT as a whole, which is defensible, but a reader should be told which components are not separately ablated.
minor comments (6)
  1. [Abstract / §4.1] Please add a phrase such as 'relative to our PyTorch reimplementation of ZACH-ViT' wherever the baseline is referenced in the abstract, to prevent readers from inferring a comparison with the original TensorFlow implementation.
  2. [§4.3 / Table 3] The latency comparison is end-to-end between PyTorch eager fake-QAT and ONNX Runtime INT8. This is acknowledged in Limitations, but the abstract and Figure 3 should also state that the speedup includes runtime-stack differences, not only quantization.
  3. [§5.2 / Table 2] The row labels 'Map cosine', 'Rank ρ', and 'Top-10% overlap' in Table 2 should specify that these are means over 3,600 matched intrinsic-map pairs; the caption says this but the table itself is terse.
  4. [Appendix B.1] Table 8 reports probability AUROC, AUC@0.5, and accuracy for binary tasks. Please add a sentence in the main text explaining why AUC@0.5 differs from probability AUROC and why it was inherited from the predecessor protocol.
  5. [§4.5 / Table 5] For the same-model XAI comparison, random attribution is reported as a negative control; its higher insertion AUC than the intrinsic method illustrates metric non-robustness. This is good, but the authors could add one explicit sentence interpreting why insertion AUC can favor random maps.
  6. [§5.6 / Table 11] The parameter-randomization result for the intrinsic map (cosine 0.963, rank ρ −0.163, top-10% overlap 0.109) is an important negative-control finding. It may deserve a slightly more prominent place in the main text, since it shows that cosine alone is insufficient.

Circularity Check

1 steps flagged · score 2.0 of 10

Main measured claims are self-contained; only the 'exact completeness' property is a definitional identity, and the baseline reimplementation creates a comparison-validity caveat, not circularity.

  1. self definitional [Section 3.2, Eq. (3); Related Work; Section 6]
    "Stage logits and the final logit are z(r)=1/N ∑ e(r)_i, z=ρ z(1)+(1−ρ)z(2), with ρ=0.25. Equation 3 is an exact decomposition: the implementation audit obtains zero reconstruction error at floating-point precision for all seven output dimensions tested. ... The raw evidence is complete by construction, whereas the normalized map is used for visualization and regularization."

    Eq. (3) defines the final logit as the weighted average of stage logits, and each stage logit is defined as the spatial average of e(r). Hence the assertion that the raw evidence exactly reconstructs the logit, and the audit's zero reconstruction error, are true by construction: the logit is literally the average of the evidence. The later claim that 'this exact completeness is stronger than claiming that an attention matrix explains a prediction' presents a definitional identity as a conceptual finding. This does not affect the empirical INT8 retention or paired test-performance measurements, so it is a minor self-definitional item rather than a central circularity.

full rationale

The paper's headline quantities are empirical measurements against external artifacts: official MedMNIST test splits, generated ONNX graphs, ONNX Runtime execution profiles, and fixed random seeds. The 99.9751% source-to-INT8 agreement, 0.000133 mean primary-metric change, map cosine/rank/overlap values, latency, and paired metric gains are computed, not fitted and renamed as predictions. The only construction-level identity is Eq. (3): the logit is defined as the weighted average of the evidence heads, so 'exact completeness' and zero reconstruction error hold by definition. That is transparent and does not bias the measured deployment results. Self-citations to the ZACH-ViT lineage are used for context and architecture provenance, not as a uniqueness theorem or as a substitute for an experiment; the baseline is explicitly a parameter-matched PyTorch reimplementation that is not claimed to numerically replicate the earlier TensorFlow implementation. This makes the 'improves on all seven datasets' headline vulnerable to yardstick drift, but that is a comparison-validity or correctness risk, not a circular reduction: no fitted parameter is relabeled as an outcome. Score 2 reflects the minor by-construction 'completeness' contribution and self-referential baseline context, not central circularity.

Assumptions & free parameters 4 free parameters · 4 assumptions · 2 invented entities

The central claims load on: two hand-set design constants (ρ, τ), one hand-set optimizer weight (λ_a), a fixed W8A8 training configuration with no ablations, three domain assumptions about data/metadata and metric validity, and two new architectural entities. The two entities carry measurable handles (logit reconstruction; INT8 retention; paired XAI tests), so they are not hat-drawn. The heavier external load is the self-cited ZACH-ViT lineage and the use of the author's own reimplementation as the baseline yardstick.

free parameters (4)
  • ρ (recursive fusion weight) = 0.25
    Eqs. 3 and 5; chosen by hand to blend the two evidence depths; no sensitivity analysis reported.
  • τ (attribution softmax temperature) = 1
    Eq. 4; chosen by hand for normalized maps; no sweep reported.
  • λ_a (attribution gradient weight) = 0.1
    Eqs. 11-12 and the loss-only control (L_cls + 0.1 L_attr); chosen by hand; no grid search reported.
  • W8A8 QAT configuration = signed [−127,127]; activation EMA momentum 0.95; observers frozen after 3 epochs; LR 1e-4; 23 epochs; batch 16
    Sections 3.4 and 4.1; fixed hyperparameters with no ablation. The 3-epoch observer freeze is load-bearing for the conversion fidelity measured in Table 2.
assumptions (4)
  • domain assumption MedMNIST official splits and labels are treated as clean ground truth; no localization targets exist
    Section 5.6: 'official MedMNIST NPZ archives were inspected directly... no masks'; the absence of masks justifies omitting localization metrics but also limits faithfulness evidence to perturbation/stability proxies.
  • domain assumption Fake-quantized training (Eq. 6, STE, frozen observers) transfers to the deployed MatMulInteger graph (Eq. 7)
    Sections 3.4-3.5. The retention numbers in Table 2 validate this post hoc, but during training it is an unverified transfer assumption; the Reproducibility Statement's 'corrected audit' note suggests summaries changed during reconciliation.
  • domain assumption Deletion/insertion/sufficiency-style perturbation metrics are meaningful faithfulness proxies despite out-of-distribution insertion biases
    Sections 4.6 and Limitations item 10; the paper cites Gomez et al. 2022 on deletion-style metric limitations, then proceeds with them, protected by multiple criteria.
  • standard math Eq. 3 logit-completeness is an algebraic identity (z := ρz^(1) + (1−ρ)z^(2)); the zero-reconstruction-error audit is definitional bookkeeping
    Section 3.2. The audit verifies the implementation computes the identity; it does not establish semantic meaning of the maps — a distinction the paper itself draws in Section 3.3.
invented entities (2)
  • Intrinsic patch-level evidence heads e^(1), e^(2) and recursive map A^rec independent evidence
    purpose: Produce patch-wise class evidence whose spatial mean reconstructs the class logit (Eq. 3), enabling intrinsic explanations during training and inference.
    Falsifiable handles exist outside the training loop: zero logit-reconstruction error at FP precision, and measured INT8 retention (cosine 0.999955, rank ρ 0.9944 over 3,600 map pairs).
  • Float-shadow quantized-consistency objective (second JS term of Eq. 8) independent evidence
    purpose: Regularize attribution retention under the training-time quantization model by matching quantized and float-shadow maps.
    Its effect is falsifiable: paired RASO-vs-loss-only tests on 70 dataset-seed units (Table 10), with three metrics significant after Holm correction.

how reviews work

0 comments
Cite this review

Pith. "Pith review of qZACH-ViT: Quantization-Aware Intrinsic Explanations with Recursive Attribution-Stabilized Optimization." pith.science (2026). https://pith.science/paper/PN2B2GOP

@misc{pith2026260715421,
  author       = {Pith},
  title        = {Pith review of: qZACH-ViT: Quantization-Aware Intrinsic Explanations with Recursive Attribution-Stabilized Optimization},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/PN2B2GOP}},
  note         = {Machine review of arXiv:2607.15421}
}
abstract

Compact medical-image classifiers need efficiency and interpretable evidence, yet these goals are often addressed separately. We introduce qZACH-ViT, a quantization-aware extension of the zero-token (CLS-token-free), position-free ZACH-ViT backbone with recursive intrinsic patch-level class evidence. We also introduce Recursive Attribution-Stabilized Optimization (RASO), which norm-matches classification and attribution gradients and removes attribution components that conflict with classification. We evaluate four controlled conditions on seven MedMNIST datasets using 50 training images per class and ten fixed seeds, completing 280 runs. All 210 qZACH-ViT checkpoints are converted to executable mixed-precision ONNX INT8 graphs containing 16 signed INT8 MatMulInteger projections with INT32 accumulation. Deployed mixed-precision INT8 qZACH-ViT with Adam improves the FP32 ZACH-ViT baseline mean on all seven datasets, with a mean paired gain of 0.0313 in the dataset-specific primary metric; qZACH-ViT with RASO yields a mean gain of 0.0368. Across 964,920 source-to-INT8 test comparisons, prediction agreement is 99.9751\%, with a mean absolute primary-metric change of 0.000133 and a maximum of 0.004386. Across 3,600 matched intrinsic maps, mean cosine similarity is 0.999955, mean rank correlation is 0.9944, and mean top-10\% overlap is 0.9692. ONNX artifacts are 70.0\% smaller than source checkpoints and provide $1.41\times$ and $2.39\times$ end-to-end CPU speedups with one and four threads. RASO significantly reduces sufficiency error and improves input-noise stability over Adam with the same attribution loss, but does not dominate every predictive or explainable artificial intelligence (XAI) metric. These results establish qZACH-ViT as a deployable compact intrinsically explainable model and RASO as a targeted stability-oriented optimization procedure.

Figures

Figures reproduced from arXiv: 2607.15421 by the authors.

Figure 1
Figure 1. Mean paired change of the actual ONNX INT8 qZACH-ViT models relative to the FP32 ZACH-ViT baseline. Error bars are 95% confidence intervals across the ten seed-wise differences within each dataset [PITH_FULL_IMAGE:figures/full_fig_p010_1.png] view at source ↗
Figure 2
Figure 2. Changed predictions per 10,000 source-to-INT8 comparisons. Values use all official test images for every dataset, condition, and seed [PITH_FULL_IMAGE:figures/full_fig_p011_2.png] view at source ↗
Figure 3
Figure 3. Serialized size and CPU latency normalized independently to the corresponding source value. The actual INT8 ONNX artifact is 70.0% smaller, and end-to-end inference is 1.41 times and 2.39 times faster with one and four threads, respectively [PITH_FULL_IMAGE:figures/full_fig_p012_3.png] view at source ↗
Figures from the paper (11 more)
Figure 4
Figure 4. Figure 4: Selected source-model intrinsic-XAI metrics normalized within each metric for visualization. Raw values and statistical tests are reported in [PITH_FULL_IMAGE:figures/full_fig_p013_4.png]
Figure 5
Figure 5. Figure 5: Mean fraction of RASO training batches with a negative classification-attribution gradient inner product. Error bars show variation across epochs and seeds. nonnegative maps. Gradient Attention Rollout-ZT and RISE have near-unit cosine and extremely small JS after rand…
Figure 6
Figure 6. Figure 6: SaCo versus explanation runtime for the same source qZACH-ViT + RASO predictions. The intrinsic explanation lies on the fast end of the tradeoff but is not the most faithful by SaCo. because several non-projection operations remain FP32, and the latency comparison incl…
Figure 7
Figure 7. Figure 7: Matched XAI methods for one OCTMNIST test image and one source qZACH-ViT + RASO checkpoint. The original and overlay are shown for each method. The model predicts class 3 while the true class is 2 [PITH_FULL_IMAGE:figures/full_fig_p016_7.png]
Figure 8
Figure 8. Figure 8: Intrinsic maps from the three source qZACH-ViT conditions for the same OCTMNIST image. The visual difference is descriptive; no anatomical localization target is available. 16 [PITH_FULL_IMAGE:figures/full_fig_p016_8.png]
Figure 9
Figure 9. Figure 9: BloodMNIST matched XAI methods. 23 [PITH_FULL_IMAGE:figures/full_fig_p023_9.png]
Figure 10
Figure 10. Figure 10: PathMNIST matched XAI methods. 24 [PITH_FULL_IMAGE:figures/full_fig_p024_10.png]
Figure 11
Figure 11. Figure 11: BreastMNIST matched XAI methods. 25 [PITH_FULL_IMAGE:figures/full_fig_p025_11.png]
Figure 12
Figure 12. Figure 12: PneumoniaMNIST matched XAI methods. 26 [PITH_FULL_IMAGE:figures/full_fig_p026_12.png]
Figure 13
Figure 13. Figure 13: DermaMNIST matched XAI methods. 27 [PITH_FULL_IMAGE:figures/full_fig_p027_13.png]
Figure 14
Figure 14. Figure 14: OrganAMNIST matched XAI methods. 28 [PITH_FULL_IMAGE:figures/full_fig_p028_14.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

40 extracted references · 6 linked inside Pith

  1. [1]

    Angelakis, A

    A. Angelakis, A. Mousa, M. L. A. Heldeweg, L. A. Biesheuvel, M. A. Haaksma, J. M. Smit, P. R. Tuinman, and P. W. G. Elbers. ZACH-ViT : A Zero-Token Vision Transformer with ShuffleStrides Data Augmentation for Robust Lung Ultrasound Classification. arXiv preprint arXiv:2510.17650, 2025

    arXiv 2025

  2. [2]

    Angelakis

    A. Angelakis. ZACH-ViT : Regime-Dependent Inductive Bias in Compact Vision Transformers for Medical Imaging. arXiv preprint arXiv:2602.17929, 2026

    arXiv 2026

  3. [3]

    Angelakis and M

    A. Angelakis and M. Gomez-Barrero. Extending ZACH-ViT to Robust Medical Imaging: Corruption and Adversarial Stress Testing in Low-Data Regimes. arXiv preprint arXiv:2604.06099, 2026

    arXiv 2026

  4. [4]

    J. Yang, R. Shi, D. Wei, Z. Liu, L. Zhao, B. Ke, H. Pfister, and B. Ni. MedMNIST v2 : A Large-Scale Lightweight Benchmark for 2D and 3D Biomedical Image Classification. Scientific Data, 10:41, 2023

    2023

  5. [5]

    Dosovitskiy, L

    A. Dosovitskiy, L. Beyer, A. Kolesnikov, D. Weissenborn, X. Zhai, T. Unterthiner, M. Dehghani, M. Minderer, G. Heigold, S. Gelly, J. Uszkoreit, and N. Houlsby. An Image Is Worth 16x16 Words: Transformers for Image Recognition at Scale. In International Conference on Learning Representations, 2021

    2021

  6. [6]

    Touvron, M

    H. Touvron, M. Cord, M. Douze, F. Massa, A. Sablayrolles, and H. J\'egou. Training Data-Efficient Image Transformers and Distillation through Attention. In Proceedings of the 38th International Conference on Machine Learning, pages 10347--10357, 2021

    2021

  7. [7]

    Jacob, S

    B. Jacob, S. Kligys, B. Chen, M. Zhu, M. Tang, A. Howard, H. Adam, and D. Kalenichenko. Quantization and Training of Neural Networks for Efficient Integer-Arithmetic-Only Inference. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 2704--2713, 2018

    2018

  8. [8]

    Gholami, S

    A. Gholami, S. Kim, Z. Dong, Z. Yao, M. W. Mahoney, and K. Keutzer. A Survey of Quantization Methods for Efficient Neural Network Inference. In Low-Power Computer Vision: Improve the Efficiency of Artificial Intelligence, pages 291--326. Chapman and Hall/CRC, 2022

    2022

Show all 40 references
  1. [9]

    Z. Li, T. Yang, P. Wang, and J. Cheng. Q-ViT : Fully Differentiable Quantization for Vision Transformer. arXiv preprint arXiv:2201.07703, 2022a

    arXiv

  2. [10]

    Z. Yuan, C. Xue, Y. Chen, Q. Wu, and G. Sun. PTQ4ViT : Post-Training Quantization for Vision Transformers with Twin Uniform Quantization. In European Conference on Computer Vision, pages 191--207, 2022

    2022

  3. [11]

    Y. Lin, T. Zhang, P. Sun, Z. Li, and S. Zhou. FQ-ViT : Post-Training Quantization for Fully Quantized Vision Transformer. In Proceedings of the Thirty-First International Joint Conference on Artificial Intelligence, pages 1173--1179, 2022

    2022

  4. [12]

    Li and Q

    Z. Li and Q. Gu. I-ViT : Integer-Only Quantization for Efficient Vision Transformer Inference. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 17065--17075, 2023

    2023

  5. [13]

    Y. Li, S. Xu, B. Zhang, X. Cao, P. Gao, and G. Guo. Q-ViT : Accurate and Fully Quantized Low-Bit Vision Transformer. In Advances in Neural Information Processing Systems, volume 35, pages 34451--34463, 2022b

  6. [14]

    Z. Li, J. Xiao, L. Yang, and Q. Gu. RepQ-ViT : Scale Reparameterization for Post-Training Quantization of Vision Transformers. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 17227--17236, 2023

    2023

  7. [15]

    Sundararajan, A

    M. Sundararajan, A. Taly, and Q. Yan. Axiomatic Attribution for Deep Networks. In Proceedings of the 34th International Conference on Machine Learning, pages 3319--3328, 2017

    2017

  8. [16]

    R. R. Selvaraju, M. Cogswell, A. Das, R. Vedantam, D. Parikh, and D. Batra. Grad-CAM : Visual Explanations from Deep Networks via Gradient-Based Localization. In Proceedings of the IEEE International Conference on Computer Vision, pages 618--626, 2017

    2017

  9. [17]

    Abnar and W

    S. Abnar and W. Zuidema. Quantifying Attention Flow in Transformers. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 4190--4197, 2020

    2020

  10. [18]

    Chefer, S

    H. Chefer, S. Gur, and L. Wolf. Transformer Interpretability Beyond Attention Visualization. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 782--791, 2021

    2021

  11. [19]

    Petsiuk, A

    V. Petsiuk, A. Das, and K. Saenko. RISE : Randomized Input Sampling for Explanation of Black-Box Models. In Proceedings of the British Machine Vision Conference, 2018

    2018

  12. [20]

    Jain and B

    S. Jain and B. C. Wallace. Attention Is Not Explanation. In Proceedings of NAACL-HLT, pages 3543--3556, 2019

    2019

  13. [21]

    utt, S. D\

    P.-J. Kindermans, S. Hooker, J. Adebayo, M. Alber, K. T. Sch\"utt, S. D\"ahne, D. Erhan, and B. Kim. The (Un)reliability of Saliency Methods. In Explainable AI: Interpreting, Explaining and Visualizing Deep Learning, pages 267--280. Springer, 2019

    2019

  14. [22]

    Adebayo, J

    J. Adebayo, J. Gilmer, M. Muelly, I. Goodfellow, M. Hardt, and B. Kim. Sanity Checks for Saliency Maps. In Advances in Neural Information Processing Systems, volume 31, 2018

    2018

  15. [23]

    Alvarez-Melis and T

    D. Alvarez-Melis and T. S. Jaakkola. Towards Robust Interpretability with Self-Explaining Neural Networks. In Advances in Neural Information Processing Systems, volume 31, 2018

    2018

  16. [24]

    C. Chen, O. Li, C. Tao, A. J. Barnett, J. Su, and C. Rudin. This Looks Like That: Deep Learning for Interpretable Image Recognition. In Advances in Neural Information Processing Systems, volume 32, 2019

    2019

  17. [25]

    P. W. Koh, T. Nguyen, Y. S. Tang, S. Mussmann, E. Pierson, B. Kim, and P. Liang. Concept Bottleneck Models. In Proceedings of the 37th International Conference on Machine Learning, pages 5338--5348, 2020

    2020

  18. [26]

    B\"ohle, N

    M. B\"ohle, N. Singh, M. Fritz, and B. Schiele. B-cos Alignment for Inherently Interpretable CNNs and Vision Transformers. arXiv preprint arXiv:2306.10898, 2023a

    arXiv

  19. [27]

    B\"ohle, M

    M. B\"ohle, M. Fritz, and B. Schiele. Holistically Explainable Vision Transformers. arXiv preprint arXiv:2301.08669, 2023b

    arXiv

  20. [28]

    Yeh, C.-Y

    C.-K. Yeh, C.-Y. Hsieh, A. Suggala, D. I. Inouye, and P. K. Ravikumar. On the (In)fidelity and Sensitivity of Explanations. In Advances in Neural Information Processing Systems, volume 32, 2019

    2019

  21. [29]

    Hooker, D

    S. Hooker, D. Erhan, P.-J. Kindermans, and B. Kim. A Benchmark for Interpretability Methods in Deep Neural Networks. In Advances in Neural Information Processing Systems, volume 32, 2019

    2019

  22. [30]

    om, L. Weber, D. Bareeva, D. Krakowczyk, F. Motzkus, W. Samek, S. Lapuschkin, and M. M.-C. H\

    A. Hedstr\"om, L. Weber, D. Bareeva, D. Krakowczyk, F. Motzkus, W. Samek, S. Lapuschkin, and M. M.-C. H\"ohne. Quantus: An Explainable AI Toolkit for Responsible Evaluation of Neural Network Explanations and Beyond. Journal of Machine Learning Research, 24(34):1--11, 2023

    2023

  23. [31]

    Ghorbani, A

    A. Ghorbani, A. Abid, and J. Zou. Interpretation of Neural Networks Is Fragile. In Proceedings of the AAAI Conference on Artificial Intelligence, 33:3681--3688, 2019

    2019

  24. [32]

    Dombrowski, M

    A.-K. Dombrowski, M. Alber, C. Anders, M. Ackermann, K.-R. M\"uller, and P. Kessel. Explanations Can Be Manipulated and Geometry Is to Blame. In Advances in Neural Information Processing Systems, volume 32, 2019

    2019

  25. [33]

    Gomez, T

    T. Gomez, T. Fr\'eour, and H. Mouch\`ere. Metrics for Saliency Map Evaluation of Deep Learning Explanation Methods. arXiv preprint arXiv:2201.13291, 2022

    2022 arXiv

  26. [34]

    J. Wu, W. Kang, H. Tang, Y. Hong, and Y. Yan. On the Faithfulness of Vision Transformer Explanations. arXiv preprint arXiv:2404.01415, 2024

    2024 arXiv

  27. [35]

    Sener and V

    O. Sener and V. Koltun. Multi-Task Learning as Multi-Objective Optimization. In Advances in Neural Information Processing Systems, volume 31, 2018

    2018

  28. [36]

    Z. Chen, V. Badrinarayanan, C.-Y. Lee, and A. Rabinovich. GradNorm : Gradient Normalization for Adaptive Loss Balancing in Deep Multitask Networks. In Proceedings of the 35th International Conference on Machine Learning, pages 794--803, 2018

    2018

  29. [37]

    T. Yu, S. Kumar, A. Gupta, S. Levine, K. Hausman, and C. Finn. Gradient Surgery for Multi-Task Learning. In Advances in Neural Information Processing Systems, volume 33, pages 5824--5836, 2020

    2020

  30. [38]

    Z. Wang, Y. Tsvetkov, S. Ruder, and G. Neubig. Gradient Vaccine: Investigating and Improving Multi-Task Optimization in Massively Multilingual Models. In International Conference on Learning Representations, 2021

    2021

  31. [39]

    B. Liu, X. Liu, X. Jin, P. Stone, and Q. Liu. Conflict-Averse Gradient Descent for Multi-Task Learning. In Advances in Neural Information Processing Systems, volume 34, pages 18878--18890, 2021

    2021

  32. [40]

    Navon, A

    A. Navon, A. Shamsian, I. Achituve, H. Maron, K. Kawaguchi, G. Chechik, and E. Fetaya. Multi-Task Learning as a Bargaining Game. In Proceedings of the 39th International Conference on Machine Learning, pages 16428--16446, 2022

    2022

Pith tools

Reviewed August 1, 2026 · model on record in the stance chip above.