REVIEW 3 major objections 5 minor 28 references
Enhancing Vision Transformer Explainability Using Artificial Astrocytes
T0 review · 3 major / 5 minor · reviewed 2026-08-07 · deepseek-v4-flash
Pith's one-line read The paper claims that adding artificial astrocytes to the first self-attention block of a pretrained Vision Transformer, with no training, makes Grad-CAM and Grad-CAM++ heatmaps align significantly better with human relevance maps from…
desk verdict Plausible training-free ViT explainability trick, but the evaluation selects hyperparameters on the test set so the claimed significance doesn't hold. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The machinery is the astrocytic linear layer, which replaces the linear layer at the end of the first self-attention block. For each neuron, a diagonal modulation matrix $M(t)$ accumulates over $k$ iterations, updating as $M_{ii}(t) = M_{ii}(t-1) \cdot m_i(t)$, where $m_i(t)$ is $\alpha \geq 1$ when the neuron has been active for at least $\tau$ iterations, $0 < \beta < 1$ when it has been inactive for at least $\tau$ iterations, and $1$ otherwise. Neuron activity is tracked by a bounded counter $A_i$ that increments when the output $y_i$ meets the activation threshold $\phi$ and decrements otherwise. The output after $k$ iterations is normalized by the ratio of the mean per-token norms with and without modulation, so the scale matches a standard linear layer. The CLS token's output determines each neuron's activation level, and the modulation applies to all tokens collectively, propagating through the residual stream to the final attention block where the explanation is extracted.
What would settle it
Record the top-1 class assigned by ViT and by ViTA for each of the 2,982 ClickMe images; if a substantial fraction of predictions differ, recompute the Spearman, DSC, and SSIM alignment only on images where the two models agree, and check whether ViTA's advantage persists.
Extended reading notes
Core claim
The central claim is that incorporating artificial astrocytes into the first self-attention block of a pretrained Vision Transformer improves the alignment of gradient-based CAM explanations with human ground truth. On 2,982 ClickMe images, the astrocyte-modified model (ViTA) produces heatmaps that score higher than a standard ViT on Spearman, DSC, and SSIM for both Grad-CAM and Grad-CAM++, with all six technique-metric combinations reported as statistically significant; for example, Grad-CAM SSIM rises from a mean of 0.262 to 0.436. The authors interpret this as the astrocytic modulation focusing explanations on the object of interest while suppressing background content, and they emphasize that the approach is training-free, method-agnostic, and requires only optimization of astrocytic hyperparameters.
Load-bearing premise
The evaluation assumes that the astrocytic modulation leaves the model's predicted class unchanged on the ClickMe images, because the paper states the astrocytic layer should be used only for explainability and not classification, but never verifies that ViTA and ViT produce the same top-1 prediction.
Editorial extensions
If this is right
- With Grad-CAM, ViTA raises mean SSIM alignment from 0.262 to 0.436 on ClickMe, with a reported p-value of 3.9e-290.
- With Grad-CAM++, ViTA raises mean SSIM from 0.271 to 0.334 and improves Spearman and DSC as well; all six technique-metric combinations show statistically significant gains.
- The best astrocytic configurations favor high sensitivity and excitatory tendency (low $\phi$, $\alpha$ of 1.25 or 1.5, $\beta$ of 0.005 or 0.05), suggesting the mechanism works by amplifying active neurons and suppressing inactive ones.
- Because no training is involved, the gain is obtained by hyperparameter search only, making the approach applicable to any pretrained ViT with the same first-block linear layer structure.
- The resulting heatmaps concentrate on the object and reduce background, as the authors observe in the activation maps.
Reading between the lines
- A testable extension is to apply the same astrocytic layer to other transformer backbones, such as DINOv2 or segmentation-oriented encoders, and check whether the human-alignment gain reproduces outside the exact ViT configuration tested.
- Because the paper selects astrocytic hyperparameters by grid search on the same ClickMe images used for evaluation, a held-out split would reveal how much of the reported gain survives configuration transfer to new images.
- If the mechanism truly sharpens strong activations and suppresses weak ones, it should also affect attention-based explanations, not only CAM heatmaps; testing ViTA with attention-rollout or relevance-propagation methods would extend the claim beyond the gradient-based family.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper proposes ViTA, a training-free modification of a pretrained Vision Transformer in which an artificial astrocyte-like modulation is applied iteratively to the linear layer of the first self-attention block. The authors compare Grad-CAM and Grad-CAM++ heatmaps from ViT and ViTA against human relevance heatmaps from the ClickMe dataset using Spearman correlation, Dice similarity, and SSIM. They grid-search five astrocytic hyperparameters, select the best configuration per XAI method and metric, and report one-tailed Wilcoxon rank-sum p-values claiming significant human-alignment improvements.
Significance. If the reported improvements are genuine, ViTA would offer a simple, training-free, and method-agnostic way to increase the human alignment of ViT explanations, with potential applications in medical and other high-stakes domains. The work is also of interest for connecting glial-inspired mechanisms to explainability rather than purely to classification performance. However, the current evaluation does not provide valid statistical evidence for these claims because the same ClickMe images are used both for hyperparameter selection and for significance testing, and the authors do not verify that the modulation preserves the class being explained. The paper also does not release code or detailed reproducibility artifacts, so the exact grid search and experimental pipeline cannot be independently checked.
major comments (3)
- [Section 4, Tables 1 and 2] The grid search is performed on the same 2,982 ClickMe images that are later used to compute the one-tailed Wilcoxon rank-sum p-values in Table 2. Because the best configuration is selected by maximizing overlap with the human ground truth on exactly those images, the reported gains and p-values are inflated by selection: even under the null hypothesis that ViTA has no true effect, the maximum over the 405 declared configurations (and more, given the unlisted α=1.25) is expected to beat the baseline on the same data. A valid test requires a held-out split (or nested selection) so that the configurations in Table 1 are chosen on a training subset and the p-values in Table 2 are computed on a disjoint test subset. This is the central inferential claim, and it is currently unsupported.
- [Section 3.1.2] The paper states that the astrocytic linear layer 'should only be used for explainability and not classification,' but it never verifies that ViTA's predicted top-1 class matches ViT's on the ClickMe images. Grad-CAM and Grad-CAM++ produce class-specific heatmaps; if ViTA's class prediction differs from ViT's, the ViTA heatmaps explain a different decision than the ViT heatmaps, and the reported alignment gain could be an artifact of class change rather than of astrocytic modulation. The authors should report class agreement between ViT and ViTA, and either restrict the comparison to images with matching class predictions or explain the same class in both models.
- [Section 4 and Table 1] Table 1 lists α=1.25 for three of the six best configurations, but the declared grid for α in Section 4 is [1.05, 1.2, 1.5]. Either the grid was larger than declared or the table contains an error. This discrepancy matters because the size of the search space is part of the selection-bias calculation, and because reproducibility requires the exact grid. The authors should clarify the full search space and correct the table or the grid.
minor comments (5)
- [Section 3.1] The text says 'placing the artificial astrocytes in the first decoder block will maximize their influence throughout the network,' but the description and Figure 2 refer to the first encoder (self-attention) block; 'decoder' should be 'encoder.'
- [Equation (9)] The notation ∥yi(0)∥2 and ∥yi(k)∥2 is ambiguous: it should read ∥yi(0)∥2 (the L2 norm of the output vector for token i), and the mean should be explicitly taken over tokens. Please clarify.
- [Section 3.2] The description that 'Grad-CAM++ uses second-order gradients' is imprecise; Grad-CAM++ computes first-order gradients and weights them using terms that involve second-order derivatives. Consider rephrasing for accuracy.
- [Figure 3] The caption says that SSIM scores appear above the images, but the values are not legible in the provided figure. Please ensure the numbers are readable or provide them in a separate table.
- [Table 2] The table reports only mean, median, and standard deviation; adding standard errors or confidence intervals for the mean differences would better convey the size of the effect, particularly given the extremely small p-values.
Circularity Check
The claimed statistically significant improvements are partially forced by selecting astrocytic hyperparameters on the same ClickMe images that later produce the reported p-values.
-
fitted input called prediction
[Section 4, Results (grid search; Tables 1 and 2)]
"First, we conducted a grid search to identify the optimal astrocytic configuration. Parameter values that maximized overlap of the heatmaps (activation maps) generated employing the different XAI techniques with the ground truth were chosen. ... The best configuration for each XAI technique and metric is provided in Tab. 1. ... Finally, a one-tailed Wilcoxon rank-sum test was employed to evaluate statistical significance. ... Tab."
The same 2,982 ClickMe images are used both to select the five astrocytic hyperparameters (k, tau, phi, alpha, beta) by maximizing Spearman, DSC, and SSIM overlap with human ground truth, and then to compute the means, medians, and Wilcoxon p-values in Table 2. With 405 declared grid combinations, and Table 1 even listing alpha=1.25 outside the declared alpha grid, the best configuration's improvement over ViT on that same selection set is expected under the null hypothesis purely from selection.
full rationale
The method itself is not defined in terms of the evaluation metrics: the astrocytic modulation equations (Eqs. 2-8) specify a fixed, training-free transformation of the first self-attention linear layer, and the evaluation compares ViTA heatmaps with ClickMe ground truth using standard metrics. Self-citations to prior astrocyte work [8, 12] are used only for design choices such as the 1:1 neuron-to-astrocyte ratio, not to justify the explainability improvement, so they are not load-bearing. The one substantive circularity is in the evaluation protocol: hyperparameters are optimized on the full ClickMe subset to maximize the very overlaps that are then reported as ViTA's gains, and the Wilcoxon tests are run on the same images. This makes the statistical significance claim partially a consequence of selection rather than of the astrocytic mechanism. The class-preservation concern raised by the statement that the astrocytic layer 'should only be used for explainability and not classification' is a validity threat, not a circularity, and is therefore not scored here. A held-out split or a nested selection correction would remove the fitted-input aspect; without it, the central quantitative claim is inflated by construction.
Assumptions & free parameters
free parameters (5)
- k (number of iterations) =
Values 4, 6, or 8 depending on metric in Table 1
- tau (response speed) =
1 or 3 in Table 1
- phi (activation threshold) =
-0.5 or 0.2 in Table 1
- alpha (excitatory factor) =
1.25 or 1.5 in Table 1, but declared grid is [1.05, 1.2, 1.5]
- beta (inhibitory factor) =
0.005, 0.05, or 0.25 in Table 1
assumptions (5)
- domain assumption Grad-CAM and Grad-CAM++ as implemented in pytorch-grad-cam produce valid explanations for ViT.
- domain assumption ClickMe human heatmaps are a valid human-aligned ground truth for ViT explainability.
- ad hoc to paper The astrocytic modulation does not change the class being explained.
- ad hoc to paper Reported p-values from the one-tailed Wilcoxon rank-sum test are meaningful despite tuning hyperparameters on the same data.
- domain assumption The normalization in Eq. (9) preserves the relative token structure needed for CAM.
invented entities (1)
-
artificial astrocyte modulation unit
Cite this review
Pith. "Pith review of Enhancing Vision Transformer Explainability Using Artificial Astrocytes." pith.science (2026). https://pith.science/paper/C5A7E5G6
@misc{pith2026250521513,
author = {Pith},
title = {Pith review of: Enhancing Vision Transformer Explainability Using Artificial Astrocytes},
year = {2026},
howpublished = {\url{https://pith.science/paper/C5A7E5G6}},
note = {Machine review of arXiv:2505.21513}
}
read the original abstract
Machine learning models achieve high precision, but their decision-making processes often lack explainability. Furthermore, as model complexity increases, explainability typically decreases. Existing efforts to improve explainability primarily involve developing new eXplainable artificial intelligence (XAI) techniques or incorporating explainability constraints during training. While these approaches yield specific improvements, their applicability remains limited. In this work, we propose the Vision Transformer with artificial Astrocytes (ViTA). This training-free approach is inspired by neuroscience and enhances the reasoning of a pretrained deep neural network to generate more human-aligned explanations. We evaluated our approach employing two well-known XAI techniques, Grad-CAM and Grad-CAM++, and compared it to a standard Vision Transformer (ViT). Using the ClickMe dataset, we quantified the similarity between the heatmaps produced by the XAI techniques and a (human-aligned) ground truth. Our results consistently demonstrate that incorporating artificial astrocytes enhances the alignment of model explanations with human perception, leading to statistically significant improvements across all XAI techniques and metrics utilized.
Figures
Reference graph
Works this paper leans on
-
[1]
Learning deep features for discrimina- tive localization
Bolei Zhou, Aditya Khosla, Agata Lapedriza, Aude Oliva, and Antonio Torralba. Learning deep features for discrimina- tive localization. InProceedings of the IEEE conference on computer vision and pattern recognition, pages 2921–2929,
-
[2]
The explainability of transformers: Current status and directions.Computers, 13(4):92, 2024
Paolo Fantozzi and Maurizio Naldi. The explainability of transformers: Current status and directions.Computers, 13(4):92, 2024. 1, 4
work page 2024
-
[3]
Wojciech Samek, Thomas Wiegand, and Klaus-Robert M¨uller. Explainable artificial intelligence: Understanding, visualizing and interpreting deep learning models.arXiv preprint arXiv:1708.08296, 2017. 1
arXiv 2017
-
[4]
Towards a rigorous science of interpretable machine learning.arXiv preprint arXiv:1702.08608, 2017
Finale Doshi-Velez and Been Kim. Towards a rigorous science of interpretable machine learning.arXiv preprint arXiv:1702.08608, 2017. 1
arXiv 2017
-
[5]
Learning what and where to attend
Drew Linsley, Dan Shiebler, Sven Eberhardt, and Thomas Serre. Learning what and where to attend. InInternational Conference on Learning Representations, 2019. 2, 4
work page 2019
-
[6]
Multi- layer perceptron with chaos glial network
Chihiro Ikuta, Yoko Uwate, and Yoshifumi Nishio. Multi- layer perceptron with chaos glial network. InIEEE Work- shop on Nonlinear Circuit, Networks, pages 11–13. Citeseer,
-
[7]
Artificial astrocytes improve neu- ral network performance.PloS one, 6(4):e19109, 2011
Ana B Porto-Pazos, Noha Veiguela, Pablo Mesejo, Marta Navarrete, Alberto Alvarellos, Oscar Ib ´a˜nez, Alejandro Pa- zos, and Alfonso Araque. Artificial astrocytes improve neu- ral network performance.PloS one, 6(4):e19109, 2011. 2
work page 2011
-
[8]
Training-free approach of convolutional neu- ral networks with astrocyte-inspired architectures
Ana Ribas-Rodriguez, Vanessa Aguiar-Pulido, and Fran- cisco Cedron. Training-free approach of convolutional neu- ral networks with astrocyte-inspired architectures. InLatinx in AI@ NeurIPS 2024, 2024. 2
work page 2024
Show all 28 references
-
[9]
Modeling work- ing memory in a spiking neuron network accompanied by astrocytes.Frontiers in Cellular Neuroscience, 15:631485,
Susanna Yu Gordleeva, Yuliya A Tsybina, Mikhail I Krivonosov, Mikhail V Ivanchenko, Alexey A Zaikin, Vic- tor B Kazantsev, and Alexander N Gorban. Modeling work- ing memory in a spiking neuron network accompanied by astrocytes.Frontiers in Cellular Neuroscience, 15:631485,
-
[10]
Astronet: When astrocyte meets artificial neural network
Mengqiao Han, Liyuan Pan, and Xiabi Liu. Astronet: When astrocyte meets artificial neural network. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 20258–20268, 2023. 2
2023
-
[11]
Ma-net: Re- thinking neural unit in the light of astrocytes
Mengqiao Han, Liyuan Pan, and Xiabi Liu. Ma-net: Re- thinking neural unit in the light of astrocytes. InProceedings of the AAAI Conference on Artificial Intelligence, volume 38, pages 2040–2048, 2024. 2
2024
-
[12]
Building transformers from neurons and astro- cytes.Proceedings of the National Academy of Sciences, 120(34):e2219150120, 2023
Leo Kozachkov, Ksenia V Kastanenka, and Dmitry Kro- tov. Building transformers from neurons and astro- cytes.Proceedings of the National Academy of Sciences, 120(34):e2219150120, 2023. 2, 3
2023
-
[13]
Transformer inter- pretability beyond attention visualization
Hila Chefer, Shir Gur, and Lior Wolf. Transformer inter- pretability beyond attention visualization. InProceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 782–791, 2021. 2
2021
-
[14]
Class-discriminative attention maps for vision transformers
Lennart Brocki, Jakub Binda, and Neo Christopher Chung. Class-discriminative attention maps for vision transformers. arXiv preprint arXiv:2312.02364, 2023. 2
2023 arXiv
-
[15]
Harmonizing the object recognition strategies of deep neu- ral networks with humans.Advances in neural information processing systems, 35:9432, 2022
Thomas Fel, Ivan Felipe, Drew Linsley, and Thomas Serre. Harmonizing the object recognition strategies of deep neu- ral networks with humans.Advances in neural information processing systems, 35:9432, 2022. 2, 4
2022
-
[16]
Improving vit interpretability with patch-level mask prediction.Pattern Recognition Letters, 187:73–79, 2025
Junyong Kang, Byeongho Heo, and Junsuk Choe. Improving vit interpretability with patch-level mask prediction.Pattern Recognition Letters, 187:73–79, 2025. 2
2025
-
[17]
Sanzgiri, and Philip G
Alfonso Araque, Vladimir Parpura, Rita P. Sanzgiri, and Philip G. Haydon. Tripartite synapses: glia, the unacknowl- edged partner.Trends in Neurosciences, 22(5):208–215,
-
[18]
Astrocyte–synapse interactions and cell adhesion molecules.The FEBS journal, 290(14):3512–3526, 2023
Margaux Saint-Martin and Yukiko Goda. Astrocyte–synapse interactions and cell adhesion molecules.The FEBS journal, 290(14):3512–3526, 2023. 2
2023
-
[19]
Human as- trocytes: structure and functions in the healthy brain.Brain Structure and Function, 222(5):2017–2029, 2017
Flora Vasile, Elena Dossi, and Nathalie Rouach. Human as- trocytes: structure and functions in the healthy brain.Brain Structure and Function, 222(5):2017–2029, 2017. 2
2017
-
[20]
Loose excitation–secretion coupling in astrocytes.Glia, 64(5):655– 667, 2016
Nina Vardjan, Vladimir Parpura, and Robert Zorec. Loose excitation–secretion coupling in astrocytes.Glia, 64(5):655– 667, 2016. 2
2016
-
[21]
Pytorch library for cam methods.https://github.com/jacobgil/ pytorch-grad-cam, 2021
Jacob Gildenblat and contributors. Pytorch library for cam methods.https://github.com/jacobgil/ pytorch-grad-cam, 2021. 4
2021
-
[22]
Grad-cam: Why did you say that?arXiv preprint arXiv:1611.07450, 2016
Ramprasaath R Selvaraju, Abhishek Das, Ramakrishna Vedantam, Michael Cogswell, Devi Parikh, and Dhruv Ba- tra. Grad-cam: Why did you say that?arXiv preprint arXiv:1611.07450, 2016. 4
2016 arXiv
-
[23]
Grad-cam++: General- ized gradient-based visual explanations for deep convolu- tional networks
Aditya Chattopadhay, Anirban Sarkar, Prantik Howlader, and Vineeth N Balasubramanian. Grad-cam++: General- ized gradient-based visual explanations for deep convolu- tional networks. In2018 IEEE winter conference on appli- cations of computer vision (WACV), pages 839–847. IEEE,
-
[24]
Pytorch image models.https : / / github
Ross Wightman. Pytorch image models.https : / / github . com / rwightman / pytorch - image - models, 2019. 4
2019
-
[25]
Towards explain- able deep visual saliency models.Computer Vision and Im- age Understanding, 235:103782, 2023
Sai Phani Kumar Malladi, Jayanta Mukherjee, Mohamed- Chaker Larabi, and Santanu Chaudhury. Towards explain- able deep visual saliency models.Computer Vision and Im- age Understanding, 235:103782, 2023. 4
2023
-
[26]
The proof and measurement of association between two things.The American Journal of Psychology, 15(1):72–101, 1904
C Spearman. The proof and measurement of association between two things.The American Journal of Psychology, 15(1):72–101, 1904. 4
1904
-
[27]
Measures of the amount of ecologic association between species.Ecology, 26(3):297–302, 1945
Lee R Dice. Measures of the amount of ecologic association between species.Ecology, 26(3):297–302, 1945. 4
1945
-
[28]
Image quality assessment: from error visibility to structural similarity.IEEE transactions on image processing, 13(4):600–612, 2004
Zhou Wang, Alan C Bovik, Hamid R Sheikh, and Eero P Si- moncelli. Image quality assessment: from error visibility to structural similarity.IEEE transactions on image processing, 13(4):600–612, 2004. 4
2004
Reviewed August 7, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.