Pith. sign in

REVIEW 3 major objections 6 minor 25 references

Hybrid Vision Transformer-Mamba Framework for Autism Diagnosis via Eye-Tracking Analysis

T0 review · 3 major / 6 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read A hybrid ViT-Mamba network is claimed to detect autism from eye-tracking data at 0.96 accuracy on the Saliency4ASD dataset.

desk verdict The reported 0.96 accuracy is unsupported because the test-set arithmetic implies same-subject leakage in a 28-participant dataset. read the letter →

arxiv 2506.06886 v1 pith:7QAEGZKL submitted 2025-06-07 cs.CV

Wafaa Kasri , Yassine Himeur , Abigail Copiaco , Wathiq Mansoor , Ammar Albanna , Valsamma Eapen This is my paper
classification cs.CV
keywords autismspectrumdisordereye-trackingVisionTransformerMambaSaliency4ASDmultimodalfusionexplainableAIdeeplearning
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper tries to establish that a hybrid deep learning model combining a Vision Transformer and a Vision Mamba state-space network can diagnose autism spectrum disorder from eye-tracking data more accurately than traditional machine learning and single deep models. Tested on the Saliency4ASD dataset, the model reports 0.96 accuracy, 0.95 F1-score, 0.97 sensitivity, and 0.94 specificity, beating SVC, Random Forest, XGBoost, CNN-LSTM, and a standalone ViT. The value, if true, is a more scalable and interpretable screening tool for settings with limited access to expert diagnosis. The claim rests on the model capturing both spatial fixation patterns and long-range temporal gaze dynamics through attention-based fusion.

What carries the argument

The load-bearing mechanism is the two-branch hybrid architecture. A Vision Transformer embeds gaze fixation patches as tokens and applies self-attention to encode spatial layout, while a Vision Mamba module applies a state-space recurrence $h_t = A h_{t-1} + B x_t$, $y_t = C h_t + D x_t$ to model temporal gaze shifts. An attention-based fusion layer then combines the temporal Mamba output with speech and facial feature streams by computing modality weights as a softmax over tanh projections, and a sigmoid classifier produces the ASD probability. This machinery carries the argument because the reported performance depends on jointly capturing spatial and temporal gaze cues rather than using handcrafted features alone.

What would settle it

Run a leave-one-subject-out evaluation on Saliency4ASD, training on 27 of the 28 children and testing on the remaining child; if accuracy falls substantially below 0.96, the reported number reflects the model recognizing specific children rather than detecting ASD.

Watch

Extended reading notes

Core claim

The central claim is that combining a Vision Transformer for spatial gaze encoding with a Vision Mamba state-space model for temporal dynamics, then fusing the resulting representations with speech and facial features through learned attention weights, classifies ASD versus non-ASD gaze behavior on Saliency4ASD at 0.96 accuracy, 0.95 F1-score, 0.97 sensitivity, and 0.94 specificity, outperforming the compared baselines. The paper attributes this gain to the hybrid's ability to model both where a child looks and how gaze shifts over time, with the attention fusion selecting the most informative modalities for each prediction.

Load-bearing premise

The paper's accuracy depends on the assumption that the 70/15/15 split of gaze records keeps each child's data entirely on one side of the split, so the model never sees the same child in both training and testing.

Editorial extensions

If this is right

  • If the reported results hold, a ViT-Mamba gaze model could serve as a low-cost initial ASD screening tool in telehealth settings.
  • The attention-based fusion of spatial, temporal, speech, and facial features would mark a step beyond handcrafted feature classifiers for gaze-based diagnosis.
  • Because the framework includes explainability layers, clinicians could inspect which gaze regions and temporal patterns drive a positive ASD prediction.
  • Remote and resource-constrained clinics without access to specialist assessment could deploy such a model on recorded eye-tracking sessions.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • An open extension is to test the same architecture with leave-one-subject-out cross-validation over the 28 children, which would reveal whether the accuracy reflects generalization to new people rather than recognition of particular individuals.
  • Because Saliency4ASD contains only 14 children per group and 300 images, a natural stress test is cross-dataset evaluation on an independent eye-tracking ASD cohort to see whether the reported 0.96 accuracy persists across ages, recording hardware, and cultural settings.
  • If the speech and facial feature streams are not present in the released version of Saliency4ASD, the multimodal fusion step could be evaluated with and without those inputs to locate whether the reported gain comes from gaze data alone.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 6 minor

Summary. The manuscript proposes a hybrid Vision Transformer-Mamba (ViT-Mamba) architecture for autism spectrum disorder (ASD) diagnosis from eye-tracking data, with an attention-based fusion mechanism intended to integrate visual, speech, and facial cues. It reports results on the Saliency4ASD dataset: 0.96 accuracy, 0.95 F1-score, 0.97 sensitivity, and 0.94 specificity, claiming to outperform SVC, Random Forest, XGBoost, CNN-LSTM, and standalone ViT. The paper also describes data preprocessing, feature engineering, an ablation study, and an explainability objective. The central evaluation, however, relies on a train/test split that appears to leak participants between training and test sets, and several components of the proposed pipeline, notably the multimodal features, are not supported by the dataset or described experiments.

Significance. If the reported results were valid, a robust eye-tracking-based ASD screening tool with 0.96 accuracy would be clinically valuable. The paper addresses a timely problem and proposes a plausible combination of modern vision architectures, and it includes an ablation study that, if properly validated, could be informative. However, the manuscript provides no code, no subject-level validation, and no external dataset; the reported metrics rest on a split that is arithmetically inconsistent with a subject-disjoint evaluation. The claimed multimodal integration is also not operationalized because the multimodal features are not part of the Saliency4ASD dataset. As presented, the central claims are not established.

major comments (3)
  1. [Section III.F and Section IV, Fig. 4] Section III.F states that the data are split 70/15/15 'while also avoiding any data leakage,' but the test set reported in Section IV and Fig. 4 contains exactly 300 samples (150 ASD, 150 non-ASD). The Saliency4ASD dataset has 28 participants (14 per group) and 300 stimuli. A subject-disjoint 15% test split would leave roughly 4 participants, which cannot produce a balanced test set of 300 samples under any natural aggregation: it would yield either about 4 participant-level feature vectors or about 1,200 image-level records if all 300 stimuli per test participant are used. A record-level 15% split of the 8,400 total stimulus-participant records would yield 1,260 records, also not 300. The only straightforward way to obtain a balanced 300-sample test set is to draw 150 gaze records from each group across all participants, which places the same child in both training and test. This within-subject leakage means the reported accuracy and F1-score do not establish diagnostic generalization, and the claim that the proposed model outperforms the baselines is unsupported.
  2. [Section III.C, Eq. (4), and Algorithm 1] The manuscript introduces speech prosody features f_s, facial action units f_v, and physiological signals f_p in Section III.C, and Eq. (4) and Algorithm 1 incorporate them in a multimodal attention fusion. However, the Saliency4ASD dataset contains only eye-tracking data (fixations, saccades, saliency maps), and no source, preprocessing details, or experimental results for the additional modalities are provided. The comparison in Table II is described as using eye-tracking inputs, so it is unclear whether the multimodal features were used at all. This is load-bearing because the abstract and the proposed fusion contribution depend on multimodal integration; without these data, Eq. (4) is not operationalized and the claimed fusion advantage is not demonstrated.
  3. [Section IV and Table II] The evaluation reports single-point metrics for all models with no error bars, confidence intervals, or significance tests, and the model architecture and fusion strategy were selected on the same Saliency4ASD benchmark. Given the very small number of participants and the lack of subject-level cross-validation, the reported gains in Table II may reflect selection bias or memorization rather than diagnostic accuracy. The manuscript should provide subject-disjoint splits, participant-level metrics, and code or split scripts to make the evaluation reproducible and to support the comparative claims.
minor comments (6)
  1. [Figure 4] Figure 4 is captioned as an ROC curve but the displayed content is a confusion matrix; the surrounding text also refers to it as an ROC curve, which is misleading.
  2. [Throughout] The manuscript contains numerous typos and formatting errors, including 'Prprocessing', 'overfiting', 'augmentaion', 'temp0ral', 'compair', 'imballanced', 'diff.', 'respectivly', and 'diagnosiss' in Sections III and IV; these should be corrected in a thorough editing pass.
  3. [References] References [13], [14], [15], and [20] are incomplete, with entries marked 'TBD, details to be updated'; final manuscripts cannot contain such placeholders.
  4. [Section III.B and III.F] The preprocessing section mentions creating synthetic heatmaps, and training mentions domain adaptation via adversarial loss, but neither is described with enough detail to be reproduced; the text should specify the augmentation magnitudes, heatmap generation procedure, and adversarial loss formulation.
  5. [Section IV] The manuscript does not report training hyperparameters such as learning rate, batch size, number of epochs, or hardware, which are necessary for reproducibility.
  6. [Figure 3] The ROC curve in Figure 3 is described only qualitatively; the manuscript should provide the numerical AUC value and ideally confidence intervals, especially given the small sample size.

Circularity Check

1 steps flagged · score 6.0 of 10

The reported 0.96 accuracy on Saliency4ASD is undermined by a test-set size that is arithmetically incompatible with a subject-disjoint 70/15/15 split; the headline 'prediction' therefore reduces to within-subject memorization rather than diagnostic generalization.

  1. fitted input called prediction [Section III.F (Model Training and Optimization) and Section IV.C / Fig. 4 (Evaluation Metrics and Confusion Matrix)]
    "The dataset is carefully divided into training (70%), validation (15%), and test (15%) splits to keep a fair balance between ASD and control participants, while also avoiding any data leakage issues during training and evaluation. ... From a total of 150 actual ASD instances, the model correctly identified 145, with just 5 mislabeled. Likewise, it accurately classified 143 out of 150 non-ASD samples."

    Saliency4ASD contains only 14 ASD and 14 TD children (the paper's own ref [7] and Table I). A subject-disjoint 15% test split leaves about 4 subjects (2 per group), which cannot produce the reported 150 ASD plus 150 non-ASD test samples. The only arithmetic route to a balanced 300-sample test set is to reuse records from the same children who supplied training data. The model is then fitted to subject-specific gaze idiosyncrasies and 'predicts' labels on other records of the same subjects; the high accuracy is statistically forced by subject identity leakage, not by a learned ASD/TD diagnostic rule. The paper's claim to have 'avoided any data leakage' is contradicted by this test-set arithmetic unless a subject-disjoint split script is supplied, and none is.

full rationale

The paper contains no self-citation chain, no imported uniqueness theorem, and no ansatz smuggled in by citation; the ViT-Mamba architecture and attention fusion are generic and not derived from the target metrics. The single load-bearing circularity is in the evaluation protocol: the reported balanced test set of 300 samples (150 per class) cannot be obtained from Saliency4ASD's 28 participants under the stated 70/15/15 subject-aware split. The numerical result therefore behaves as a fitted-input-called-prediction: the model is evaluated on records from the same individuals used for training, so the headline accuracy is inflated by memorization of person-specific gaze patterns. Because the central claim ('outperformed existing methods, achieving 0.96 accuracy...') rests on this split, the circularity score is set at 6 rather than lower. If the authors later provide a subject-exclusive split script or per-subject partition, this step would be resolved and the score should drop to 0-1; as written, the manuscript's own numbers force the leaky reading.

Assumptions & free parameters 6 free parameters · 5 assumptions · 0 invented entities

The central claim rests on the Saliency4ASD dataset being usable at the record level, on the availability of multimodal features that the dataset does not document, and on several unreported architectural and training choices. No new physical or conceptual entities are introduced.

free parameters (6)
  • Attention fusion weights w and W = not reported
    Learned parameters in Eq. (4) that assign modality weights; fitted to the training data, values not reported.
  • ViT patch size and embedding dimension = not reported
    Chosen by hand; not reported, despite directly controlling spatial encoding.
  • Mamba state dimension and number of layers = not reported
    Chosen by hand; not reported, despite controlling temporal modeling capacity.
  • Learning rate, optimizer schedule, dropout, weight decay = not reported
    Training hyperparameters mentioned as Adam and SGD but values and schedules are omitted.
  • Augmentation magnitude for jittering and synthetic heatmaps = not reported
    Augmentation is described qualitatively; strength and generation procedure are not specified, affecting generalization claims.
  • Domain adaptation adversarial loss weight = not reported
    Mentioned in Section III.F but no value or implementation detail is given.
assumptions (5)
  • domain assumption Gaze patterns in Saliency4ASD carry diagnostic signal that is learnable from fixation maps.
    Section III.A assumes eye-tracking differences can train diagnostic models.
  • domain assumption The eye-tracking sequence can be represented as patch tokens for a Vision Transformer.
    Section III.D defines X in R^{T x d} and z0 = X_patch + E_pos without justifying the patchification.
  • ad hoc to paper Speech prosody and facial action unit features are available for Saliency4ASD participants.
    Section III.C and III.E use fs and fv, but the cited dataset provides only eye-tracking data; no source for these modalities is given.
  • ad hoc to paper A 70/15/15 split of records with balanced classes prevents data leakage.
    Section III.F claims no leakage but does not describe a subject-wise split; with 28 subjects, record-level splitting is likely leaky.
  • standard math State-space model equations (Eq. 3) are a valid representation of gaze dynamics.
    The Mamba state-space formulation is standard, but its parameters are fitted.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Hybrid Vision Transformer-Mamba Framework for Autism Diagnosis via Eye-Tracking Analysis." pith.science (2026). https://pith.science/paper/7QAEGZKL

@misc{pith2026250606886,
  author       = {Pith},
  title        = {Pith review of: Hybrid Vision Transformer-Mamba Framework for Autism Diagnosis via Eye-Tracking Analysis},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/7QAEGZKL}},
  note         = {Machine review of arXiv:2506.06886}
}
read the original abstract

Accurate Autism Spectrum Disorder (ASD) diagnosis is vital for early intervention. This study presents a hybrid deep learning framework combining Vision Transformers (ViT) and Vision Mamba to detect ASD using eye-tracking data. The model uses attention-based fusion to integrate visual, speech, and facial cues, capturing both spatial and temporal dynamics. Unlike traditional handcrafted methods, it applies state-of-the-art deep learning and explainable AI techniques to enhance diagnostic accuracy and transparency. Tested on the Saliency4ASD dataset, the proposed ViT-Mamba model outperformed existing methods, achieving 0.96 accuracy, 0.95 F1-score, 0.97 sensitivity, and 0.94 specificity. These findings show the model's promise for scalable, interpretable ASD screening, especially in resource-constrained or remote clinical settings where access to expert diagnosis is limited.

Figures

Figures reproduced from arXiv: 2506.06886 by the authors.

Figure 1
Figure 1. Hybrid ViT/Mamba on Saliency4ASD dataset [PITH_FULL_IMAGE:figures/full_fig_p003_1.png] view at source ↗
Figure 1
Figure 1. Three sample images from each of the seven classes iild fh f [PITH_FULL_IMAGE:figures/full_fig_p004_1.png] view at source ↗
Figure 3
Figure 3. ROC Curve of the Hybrid ViT-Mamba Model for ASD Diagnosis. [PITH_FULL_IMAGE:figures/full_fig_p005_3.png] view at source ↗
Figures from the paper (1 more)
Figure 4
Figure 4. Figure 4: ROC Curve of the Hybrid ViT-Mamba Model for ASD Diagnosis. [PITH_FULL_IMAGE:figures/full_fig_p005_4.png]

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

25 extracted references · 24 canonical work pages

  1. [1]

    Deep learning to interpret autism spectrum disorder behind the camera,

    S. Chen, M. Jiang, and Q. Zhao, “Deep learning to interpret autism spectrum disorder behind the camera,”IEEE Transactions on Cognitive and Developmental Systems, 2024

    work page 2024

  2. [2]

    Visual question answering driven eye tracking paradigm for identifying children with autism spectrum disorder,

    J. Qi, Y . Huang, Y . Zhang, S. Zhang, M. Tian, Y . Tian, F. Meng, L. Guan, and T. Chang, “Visual question answering driven eye tracking paradigm for identifying children with autism spectrum disorder,” inProceedings of the 32nd ACM International Conference on Multimedia, 2024, pp. 5847–5855

    work page 2024

  3. [3]

    Asdnet: A robust involution-based architecture for diagnosis of autism spectrum disorder utilising eye-tracking technol- ogy,

    N. Mumenin, M. A. Yousuf, M. A. Nashiry, A. Azad, S. A. Alyami, P. Lio’, and M. A. Moni, “Asdnet: A robust involution-based architecture for diagnosis of autism spectrum disorder utilising eye-tracking technol- ogy,”IET Computer Vision, 2024

    work page 2024

  4. [4]

    Early identification of autism spectrum disorder based on machine learning with eye-tracking data,

    Q. Wei, W. Dong, D. Yu, K. Wang, T. Yang, Y . Xiao, D. Long, H. Xiong, J. Chen, X. Xuet al., “Early identification of autism spectrum disorder based on machine learning with eye-tracking data,”Journal of Affective Disorders, vol. 358, pp. 326–334, 2024

    work page 2024

  5. [6]

    Enhanced multilevel autism classifi- cation for children using eye-tracking and hybrid cnn-rnn deep learning models,

    S. Cheekaty and G. Muneeswari, “Enhanced multilevel autism classifi- cation for children using eye-tracking and hybrid cnn-rnn deep learning models,”Neural Computing and Applications, pp. 1–24, 2024

    work page 2024

  6. [7]

    A dataset of eye movements for the children with autism spectrum disorder,

    H. Duan, G. Zhai, X. Min, Z. Che, Y . Fang, X. Yang, J. Guti ´errez, and P. L. Callet, “A dataset of eye movements for the children with autism spectrum disorder,” inProceedings of the ACM Multimedia Systems Conference (MMSys’19). ACM, June 2019

    work page 2019

  7. [8]

    Learning to predict autism spectrum disorder based on the visual patterns of eye-tracking scanpaths,

    R. Carette, M. Elbattah, and F. Cilia, “Learning to predict autism spectrum disorder based on the visual patterns of eye-tracking scanpaths,” inProceedings of the 12th International Joint Conference on Biomedical Engineering Systems and Technologies (BIOSTEC 2019), 2019

    work page 2019

  8. [9]

    Advances in neuroimaging and deep learning for emotion detection: A systematic review of cognitive neuroscience and algorithmic innova- tions,

    C. Halkiopoulos, E. Gkintoni, A. Aroutzidis, and H. Antonopoulou, “Advances in neuroimaging and deep learning for emotion detection: A systematic review of cognitive neuroscience and algorithmic innova- tions,”Diagnostics, vol. 15, no. 4, p. 456, 2025

    work page 2025

Show all 25 references
  1. [10]

    A convolu- tional deep neural network approach to predict autism spectrum disorder based on eye-tracking scan paths,

    M. Alsaidi, N. Obeid, N. Al-Madi, H. Hiary, and I. Aljarah, “A convolu- tional deep neural network approach to predict autism spectrum disorder based on eye-tracking scan paths,”Information, vol. 15, no. 3, p. 133, 2024

    2024

  2. [11]

    A novel multi- modal model to assist the diagnosis of autism spectrum disorder using eye-tracking data,

    B. Benabderrahmane, M. Gharzouli, and A. Benlecheb, “A novel multi- modal model to assist the diagnosis of autism spectrum disorder using eye-tracking data,” vol. 12, p. Article 40, 2024

    2024

  3. [12]

    Identification of autism spectrum disorder via an eye-tracking based representation learning model,

    C. Xia, K. Chen, K. Li, and H. Li, “Identification of autism spectrum disorder via an eye-tracking based representation learning model,” in Proceedings of the 7th International Conference on Bioinformatics Research and Applications (ICBRA ’20). ACM, 2020, pp. 59–65

    2020

  4. [13]

    Eye tracking biomarkers for autism spectrum disorder detection using machine learning and deep learning techniques: Review,

    R. A. Jeyarani and R. Senthilkumar, “Eye tracking biomarkers for autism spectrum disorder detection using machine learning and deep learning techniques: Review,”TBD, 2024, details to be updated based on publication information

    2024

  5. [14]

    Eye tracking-based diagnosis and early detection of autism spectrum disorder using machine learning and deep learning techniques,

    I. A. Ahmed, E. M. Senan, T. H. Rassem, M. A. H. Ali, H. S. A. Shatnawi, S. M. Alwazer, and M. Alshahrani, “Eye tracking-based diagnosis and early detection of autism spectrum disorder using machine learning and deep learning techniques,”TBD, 2024, details to be updated based ...

    2024

  6. [15]

    Applying eye tracking with deep learning techniques for early-stage detection of autism spectrum disorders,

    Z. A. T. Ahmed, E. Albalawi, T. H. H. Aldhyani, M. E. Jadhav, P. Janrao, and M. R. M. Obeidat, “Applying eye tracking with deep learning techniques for early-stage detection of autism spectrum disorders,”TBD, 2024, details to be updated based on publication information

    2024

  7. [16]

    Vitasd: Robust vision transformer baselines for autism spectrum disorder facial diagnosis,

    X. Cao, W. Ye, E. Sizikova, X. Bai, M. Coffee, H. Zeng, and J. Cao, “Vitasd: Robust vision transformer baselines for autism spectrum disorder facial diagnosis,”ICASSP 2023 - 2023 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pp. 1–5, 2023

    2023

  8. [17]

    Vision mamba: A comprehensive survey and taxonomy,

    X. Liu, C. Zhang, and L. Zhang, “Vision mamba: A comprehensive survey and taxonomy,”arXiv preprint arXiv:2405.04404, 2024. [Online]. Available: https://arxiv.org/abs/2405.04404

    2024 arXiv

  9. [18]

    Machine learning-based early diagnosis of autism according to eye movements of real and artificial faces scanning,

    M. Salhoferet al., “Machine learning-based early diagnosis of autism according to eye movements of real and artificial faces scanning,” Frontiers in Neuroscience, vol. 17, p. 1170951, 2023

    2023

  10. [19]

    Using machine learning to diagnose autism based on eye tracking data,

    W. Hameedet al., “Using machine learning to diagnose autism based on eye tracking data,”Diagnostics, vol. 15, no. 1, p. 66, 2023

    2023

  11. [20]

    Eye tracking biomarkers for autism spectrum disorder detection using machine learning techniques,

    “Eye tracking biomarkers for autism spectrum disorder detection using machine learning techniques,”International Journal of Medical Infor- matics, 2023

    2023

  12. [21]

    Early identification of autism spectrum disorder based on machine learning with eye-tracking data,

    A. Ahmedet al., “Early identification of autism spectrum disorder based on machine learning with eye-tracking data,”Journal of Affective Disorders, 2024

    2024

  13. [22]

    Utilizing deep learning models in an intelligent eye- tracking system for autism spectrum disorder diagnosis,

    R. Qasemet al., “Utilizing deep learning models in an intelligent eye- tracking system for autism spectrum disorder diagnosis,”Frontiers in Medicine, vol. 11, p. 1436646, 2024

    2024

  14. [23]

    Machine learning based on eye-tracking data to identify autism spectrum disorder: A systematic review,

    Z. Luet al., “Machine learning based on eye-tracking data to identify autism spectrum disorder: A systematic review,”Journal of Biomedical Informatics, vol. 136, p. 104261, 2022

    2022

  15. [24]

    A novel multi-modal model to assist the diagnosis of autism spectrum disorder using eye-tracking data,

    Y . Zhouet al., “A novel multi-modal model to assist the diagnosis of autism spectrum disorder using eye-tracking data,”Health Information Science and Systems, 2024

    2024

  16. [25]

    Enhanced multilevel autism classification for children using eye tracking data,

    F. Aleneziet al., “Enhanced multilevel autism classification for children using eye tracking data,”Neural Computing and Applications, 2024

    2024

  17. [26]

    Autism spectrum disorder,

    C. Lord, M. Elsabbagh, G. Baird, and J. Veenstra-Vanderweele, “Autism spectrum disorder,”The lancet, vol. 392, no. 10146, pp. 508–520, 2018

    2018

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.