Pith. sign in

REVIEW 3 major objections 5 minor 15 references

This paper claims that sparse winner-takes-all token selection plus a what/where pathway split makes video transformers more efficient, more robust, and more aligned with human EEG than similar-scale single-stream models.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · deepseek-v4-flash

2026-08-01 17:27 UTC pith:DQG7Q6ZM

load-bearing objection Genuinely useful what/where split with a thorough EEG study, but the SOTA claims rest on an uncontrolled baseline comparison and the efficiency numbers are overstated. the 3 major comments →

arxiv 2607.17625 v1 pith:DQG7Q6ZM submitted 2026-07-20 cs.CV cs.LG

Brain-Aligned Multi-Stream Video Transformers with Sparse Self-Selection

Amir Hosein Fadaei , Mahyar Maleki , Mohammad-Reza A. Dehaqani This is my paper
classification cs.CV cs.LG
keywords video transformerssparse attentionwinner-takes-all selectionwhat/where pathway splitbrain-model alignmentEEG representational similarityaction recognitiondifferentiable discrete selection
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

This paper tries to establish that two biologically inspired inductive biases—explicit separation of video processing into a fine-grained 'what' stream and a fast 'where' stream, and replacement of dense self-attention with a sparse winner-takes-all token selection—improve both engineering outcomes and neural alignment of video transformers. On Kinetics-400 and Something-Something V2, the authors' best variant (SWW-Former with attention fusion) reports 82.55% and 73.18% top-1 accuracy (the fraction of clips whose single best predicted class is correct), which they claim outperforms comparable-scale state-of-the-art models by more than 0.75 and 1 percentage points while sitting on the accuracy-versus-inference-time Pareto frontier. The same model shows smaller accuracy drops under spatial perturbations than a monolithic transformer baseline, and a peak EEG rank-correlation of 0.18, about 78% of the estimated noise ceiling—the maximum correlation a model could achieve given inter-subject reliability. If these claims hold, they matter because they suggest sparsity and pathway specialization are not just efficiency tricks but useful constraints for building video models whose internal representations resemble human visual processing. The main load-bearing premise is that comparison baselines were trained under the identical pipeline; if that premise fails, the comparative claims weaken.

Core claim

The authors claim that a neuro-inspired split-and-fuse video transformer, in which a high-resolution low-frame-rate 'what' stream and a low-resolution high-frame-rate 'where' stream are processed separately and then fused, combined with a sparse winner-takes-all selection module that replaces dense self-attention, achieves the best accuracy-efficiency trade-off among models of comparable scale and pretraining on Kinetics-400 and Something-Something V2, reaches 82.55% and 73.18% top-1 accuracy respectively, and produces representations that correlate more strongly with time-resolved human EEG than standard video transformer baselines (peak rank correlation 0.18, about 78% of the noise ceiling

What carries the argument

The central mechanism is a 'self-selection' block that replaces the softmax-weighted sum over all value tokens in self-attention with a per-query winner-takes-all gate: for each query token, the key with the highest similarity score is selected and only that key's value vector is gathered as the output. During training the hard argmax is relaxed through a temperature-annealed, noise-injected discrete sampling surrogate so that gradients can flow; at inference it reduces to a deterministic argmax-and-gather, removing the dense post-attention matrix multiplication and cutting the attention map's memory footprint from quadratic in the number of tokens to linear. The second piece is the split-an

Load-bearing premise

The load-bearing premise is that the baseline models were retrained under the paper's exact same protocol; if the published baseline numbers come from different training regimes, the claimed accuracy and Pareto advantages are not established.

What would settle it

Retrain every baseline under the paper's declared protocol (ImageNet initialization, 30-epoch Kinetics-400, 20-epoch SSv2, batch size 64) and measure top-1 accuracy and inference time on the same hardware; if any published baseline matches or beats SWW-Former at the same latency, the Pareto-frontier claim fails. A complementary decisive check is a preregistered replication of the EEG rank-correlation at the ~195 ms peak with a new participant pool and stimulus set: if the SWW-Former correlation does not exceed the strongest monolithic transformer baseline, the brain-alignment claim fails.

Watch this falsifier. Get emailed when new claim-graph text bears on it.

If this is right

  • If the central claims are correct, dense self-attention is not necessary for accurate video understanding: a single selected value token per query can match or beat dense attention once the model is split into complementary streams.
  • The accuracy-latency Pareto results imply that the sparse routing module can buy efficiency under a fixed compute budget without sacrificing recognition accuracy, which is directly relevant for resource-constrained deployment.
  • The EEG results suggest that brain-model alignment is a measurable architectural target: models with what/where separation and sparse competition can be ranked by how closely they track human neural responses, and improved by adding biologically inspired constraints.
  • The complementary temporal-versus-parietal alignment of the two streams indicates that the ventral/dorsal distinction can be operationalized in artificial vision systems, providing a testbed for hypotheses about the organization of human visual cortex.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • A natural extension the paper does not test is whether the self-selection module transfers to other transformer families; dropping the module into a hierarchical windowed video transformer and measuring both accuracy and EEG alignment under the same protocol would test that generality.
  • The paper's fixed-temperature experiments show accuracy stays within about two percentage points across a wide temperature range, which suggests the winner-takes-all routing is insensitive to the exploration schedule and could be dropped into pipelines with limited hyperparameter tuning.
  • Because the EEG experiment also included scrambled videos and static images, a concrete follow-up would use those conditions to test whether the model's brain alignment depends on genuine motion information rather than static appearance alone.
  • If the efficiency and alignment gains replicate at larger scale, the design principle could be applied to large multimodal video models, where dense attention is a major cost and brain-alignment constraints could serve as a regularizer.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. This paper introduces a two-stream video transformer with a 'what' stream (high spatial, low temporal resolution) and a 'where' stream (low spatial, high temporal resolution), fused by concatenation or bidirectional cross-attention, and a 'self-selection' block that replaces dense self-attention with a winner-takes-all gather over values, trained with a hard Gumbel-softmax relaxation. The authors evaluate the model on Kinetics-400 and Something-Something V2, reporting top-1 accuracies of 82.55% and 73.18% for SWW-Former with attention fusion, and claim Pareto-optimal accuracy-latency trade-offs, improved robustness under spatial perturbations, and higher EEG RSA correspondence (peak 0.18, about 78% of the noise ceiling) relative to several video transformer baselines, using new 128-channel EEG recordings for HVU clips.

Significance. If the results hold, the paper provides a concrete demonstration that pathway specialization and sparse competitive routing can be useful inductive biases for video transformers, and it is one of relatively few studies to evaluate video transformers against time-resolved EEG with stream-wise and ROI analyses. Strengths include the released code, ablation of fusion strategies and Gumbel temperature, a documented EEG protocol with time-shift and scrambled-video controls, and a generally careful discussion of limitations. However, the headline engineering and brain-alignment claims currently rest on uncontrolled baseline comparisons, an overstated complexity argument, and a biased noise-ceiling estimator; until these are addressed, the empirical conclusions remain conditional.

major comments (3)
  1. [Table 2; §4.1] The controlled-comparison claim is not supported. §4.1 states that 'identical data pipelines and augmentation strategies ... across all models and baselines' ensure fair comparison, yet every baseline accuracy in Table 2 exactly matches the original publications (e.g., TimeSFormer 77.9/59.1, VideoSwin-B 82.7/69.6, MViT-B 80.3/72.1, MTV-B 81.8/67.6, VideoMAE-V2 81.5/71.2). Exact reproduction under the paper's ImageNet-init, 30-epoch, batch-size-64 protocol is implausible because the original papers used different epochs, batch sizes, resolutions, and pretraining strategies. No retraining logs, config files, seeds, or variance estimates are provided. If the baselines are literature numbers, the claims of 'outperforming similar-scale SOTA by more than 0.75/1 percentage points' and the Pareto-frontier analyses in Figs. 2 and 3 are comparisons against a mismatched training distribution and ar
  2. [§3.2, Eq. (6)–(9); Table 3] The complexity argument is overstated. The paper says the single-winner approach lowers compute from O(BIJD) to O(BID) 'in the gather step,' but the score computation S=QK^T in Eq. (6) remains O(BIJD), so the asymptotic cost of the block is still O(BIJD); only the post-softmax multiplication AV and the softmax are removed. This is consistent with Table 3, where replacing dense attention with self-selection reduces GFLOPs by only about 3.0–3.1% (1763.90→1710.33 for attention fusion; 1763.86→1709.55 for concat fusion). The abstract and Section 1 frame sparse selection as substantially lowering compute and memory; the measured savings do not support that framing. Please report the leading-order cost including QK^T, state the precise savings from removing AV, and adjust the efficiency claims accordingly.
  3. [§4.6, noise ceiling] The noise ceiling is computed as the mean Spearman correlation between each participant's EEG RDM and the group-average RDM 'including that participant.' This inflates the ceiling because each participant contributes to the reference RDM against which they are correlated. The standard approach is split-half or leave-one-out averaging (e.g., correlate each participant's RDM with the average of the other participants). Since the headline '78% of the noise ceiling' is normalized by this quantity, the reported fraction may be overstated. Please recompute the ceiling with a leave-one-out or split-half estimator and report how the model fraction changes.
minor comments (5)
  1. [§4.1, §4.4] The Gumbel-softmax temperature schedule is described only as 'moderate value' and 'annealed exponentially'; provide the exact initialization, final value, and schedule. The fixed-temperature ablation in §4.4 does not replace this information.
  2. [§3.1; Table 1] The 'what' and 'where' streams are described as high/low resolution, but both streams are listed with 224×224 spatial input. Clarify that the difference is patch size (16 vs. 32) and frame count, not the spatial resolution of the decoded frames.
  3. [§4.6] Specify which model layer or embedding (final CLS token, pooled tokens, or fused representation) enters the RSA computation. This is needed to interpret the layer-wise appendix and to compare across models.
  4. [Table 2] Minor typo: 'V-JEPA-G (V2)]' has an extra bracket.
  5. [§3.2, Eq. (11)–(12)] The statement that hard Gumbel-softmax ensures the forward output 'matches exactly the discrete selection map M' is not strictly correct during training, because the selected index is argmax of G = S + gumbel noise, whereas inference uses argmax of S. The text should clarify that the match is only in the zero-temperature, no-noise limit.

Circularity Check

0 steps flagged

No construction-level circularity; central claims are measured against external benchmarks and EEG data. Minor self-citation is ancillary.

full rationale

The derivation chain is not circular at the level of equations or fitted inputs. The split-and-fuse architecture and sparse winner-takes-all block are defined by Eqs. (1)-(13) as architectural choices; no parameter is fitted to the EEG RSA results or to the accuracy gains. Video recognition (Table 2) reports independent training on Kinetics-400 and SSv2; although the claim of an 'identical data pipeline' for all baselines is not supported by per-baseline retraining logs and the baseline numbers coincide with literature values, this is a comparison-control weakness rather than a circularity, because the model's own accuracies are not derived from those baselines. Robustness drops (Table 4) are fresh measurements; the citation to Fadaei and Dehaqani (2024) supplies only the perturbation protocol and is not load-bearing for the outcome. The EEG RSA analysis computes model RDMs and EEG RDMs separately and correlates them; the noise ceiling is derived from EEG inter-subject reliability alone, and the peak-time selection is a post-hoc reporting choice, not a parameter fit. The ventral/dorsal interpretation uses operational stream definitions (high-resolution/low-frame-rate vs low-resolution/high-frame-rate) and then finds complementary scalp-region correlations; this is an empirical outcome, not imposed by the definitions. The manuscript itself flags EEG and backbone limitations, further supporting the non-circular verdict. The only issue touching the circularity rubric is one minor self-citation, not load-bearing; hence score 2 rather than 0.

Axiom & Free-Parameter Ledger

3 free parameters · 4 axioms · 0 invented entities

The paper's core empirical claims rest on a small set of architectural hyperparameters and standard NeuroAI assumptions. The most consequential ledger entry is the noise-ceiling computation: by including each participant in the group average, the reported '78% of noise ceiling' becomes harder to interpret. No new physical or theoretical entities are introduced.

free parameters (3)
  • Gumbel-softmax temperature schedule = annealed from around 1.0 down toward 0.1 in main runs; fixed values 0.1-10 tested in ablation
    Temperature controls sharpness of winner-takes-all selection. The paper shows accuracy is stable across the range, so it is not a tightly fitted parameter, but it is a hand-chosen hyperparameter of the method.
  • Stream input configurations = what: patch 16, 8 frames, 224x224; where: patch 32, 32 frames, 224x224
    The spatial/temporal resolution split is central to the what/where design and is chosen by hand without a data-driven selection procedure.
  • Number of transformer layers per stream = 8
    An 8-layer TimeSFormer-derived encoder is used for each stream; this depth is a design choice not derived from the analysis.
axioms (4)
  • standard math Gumbel-softmax straight-through estimator provides usable gradients for the hard argmax selection.
    The method relies on the standard Gumbel-softmax reparameterization and straight-through gradient approximation, as stated in Section 3.2.
  • domain assumption Scalp EEG RDMs at the 195 ms peak reflect category-selective visual representations comparable to model embeddings.
    The RSA analysis assumes that the time-resolved EEG representational geometry at the decoding peak is a valid target for comparing video model embeddings; this is a standard but nontrivial NeuroAI assumption.
  • ad hoc to paper The noise ceiling computed by including each participant in the group-average RDM is a valid upper bound.
    The paper explicitly computes the ceiling as the mean correlation between each participant's RDM and the group-average RDM 'including that participant'. This is non-standard and inflates the ceiling relative to a leave-one-participant-out estimate, directly affecting the 78% claim.
  • domain assumption TimeSFormer-derived 8-layer encoders are representative backbones, and self-selection preserves representational capacity for the subsequent layers.
    The evaluation is conducted only on a TimeSFormer-derived backbone; whether the findings generalize to other transformer families is an acknowledged open question in Section 5.

pith-pipeline@v1.3.0-alltime-deepseek · 22536 in / 11144 out tokens · 111109 ms · 2026-08-01T17:27:05.266804+00:00 · methodology

0 comments
read the original abstract

Modern video transformers typically ignore principles from primate vision and are rarely evaluated against neural data, limiting their biological interpretability. We introduce a sparse winner-takes-all token selection module that replaces dense self-attention to improve efficiency and approximate competitive routing observed in biological visual circuits. We further propose a neuro-inspired split-and-fuse video transformer which uses two complementary pathways: a high-resolution, low-frame-rate "what" stream and a low-resolution, high-frame-rate "where" stream, fused before classification. On Kinetics-400 and Something-Something V2, our best variant operates on the Pareto frontier of accuracy versus inference time among models of comparable scale and pretraining, and showing improved robustness to spatial perturbations. Using representational similarity analysis between model embeddings and time-resolved EEG recordings for the same video stimuli, our model attains a peak brain-model correlation of 0.18 (about 78% of the noise ceiling) and consistently outperforms strong video transformer baselines, suggesting that pathway specialization and sparse competition are useful inductive biases for efficient, brain-aligned video understanding.

Figures

Figures reproduced from arXiv: 2607.17625 by Amir Hosein Fadaei, Mahyar Maleki, Mohammad-Reza A. Dehaqani.

Figure 1
Figure 1. Figure 1: Overview of the neuro-aligned split-and-fuse architecture. The top panel shows the model split into a "what" stream that processes high-resolution frames at a lower frame rate and a "where" stream that processes lower-resolution frames at a higher frame rate to capture motion. The two streams are fused before the classifier via either simple feature concatenation or a cross-attention-based fusion block. Th… view at source ↗
Figure 2
Figure 2. Figure 2: Trade-off between top-1 accuracy and inference time (ms per batch) on Kinetics-400 (Carreira and Zisserman, 2017) for our models and others of similar size and pretraining. Models in the top-left region are both more accurate and faster. Bubble size indicates the number of trainable parameters. Stiefelhagen and Gool, 2020). Signals were high-pass filtered (1 Hz), notch-filtered (49-51 Hz), re-referenced to… view at source ↗
Figure 3
Figure 3. Figure 3: Trade-off between top-1 accuracy and inference time (ms per batch) on SSv2 (Goyal et al., 2017) for our models and others of similar size and pretraining. Models in the top-left region are both more accurate and faster. Bubble size indicates the number of trainable parameters. were constructed for each participant at each time point from scalp-recorded responses, and flattened analogously. Spearman correla… view at source ↗
Figure 4
Figure 4. Figure 4: Effect of fixed Gumbel-softmax temperature 𝜏 on top-1 validation accuracy of SWW-Former (Attention), with error bars indicating one standard deviation across 5 runs. Accuracy varies by less than about two percentage points across the tested range and peaks around 𝜏 = 0.3 [PITH_FULL_IMAGE:figures/full_fig_p014_4.png] view at source ↗
Figure 5
Figure 5. Figure 5: Percentage drop in top-1 accuracy under spatial augmentations on (a) Kinetics-400 (Carreira and Zisserman, 2017) and (b) Something-Something V2 (Goyal et al., 2017). The error bars report the standard deviation of 5 trials and p-values test differences from TimeSformer and are calculated by bootstrap test (N=100,000). correlation by SWW-Former attention (P-value >0.33; two-sided paired Wilcoxon signed-rank… view at source ↗
Figure 6
Figure 6. Figure 6: Model-EEG alignment. (a) Time-resolved multiclass LDA decoding accuracy for 21 visual subcategories. (b) Time-resolved mean Spearman RSA correlations between model RDMs and EEG RDMs with noise ceiling. (c) Peak mean RSA correlations (195 ms) across models (The error bars indicate SEM), with noise ceiling. (d) Pairwise Wilcoxon signed-rank tests on peak RSA scores. baselines. The SWW-Former recorded average… view at source ↗
Figure 7
Figure 7. Figure 7: Channel-wise and regional RSA between video models and EEG at peak latency (195 ms). The top two rows show scalp topographies of Spearman correlations. The middle two rows show significance maps from paired t-tests against best model per lobe where darker green indicates lower p-values and significant difference from SWW-Former with concatenation fusion. The plot shows mean RSA by ROI including frontal, pa… view at source ↗
Figure 8
Figure 8. Figure 8: Layer-wise S What stream-EEG RSA analysis. Layer-wise RSA in the S What stream peaks at around 190 ms, with higher layers exhibiting the strongest correlations. in early layers to more focal, high-amplitude patterns in occipital, temporal, and parietal regions in deeper layers. In the S Where stream, the strongest correlations emerge over parietal and lateral occipital sensors in higher layers, echoing the… view at source ↗
Figure 9
Figure 9. Figure 9: Layer-wise S Where stream-EEG RSA analysis. Layer-wise RSA reveals that deeper S Where layers show progressively stronger alignment with EEG around 195 ms. B.2. Consent form To document the ethical safeguards of our EEG experiment, we include a translated version of the participant informed consent form used in the study. This form outlines the purpose of the research, the non-invasive EEG pro￾cedures, pot… view at source ↗
Figure 10
Figure 10. Figure 10: Layer-wise ROI S What stream-EEG RSA analysis at peak timepoint. The later layers show ventral-biased occipital-temporal topographies, consistent with object and appearance-focused ventral processing [PITH_FULL_IMAGE:figures/full_fig_p021_10.png] view at source ↗
Figure 11
Figure 11. Figure 11: Layer-wise ROI S Where stream-EEG RSA analysis at peak timepoint. Later layers show increasingly focal parietal-occipital topographies, highlighting motion and spatially tuned dorsal-like representations. viewing stimuli. The session includes several segments, with breaks and refreshments provided in between. An EEG cap with 128 gel-based electrodes will be placed on my head. I will be asked to attentivel… view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

15 extracted references · 6 linked inside Pith

  1. [6]

    Large scale holistic video understanding, in: Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part V 16, pp. 593–610. Dosovitskiy, A., Beyer, L., Kolesnikov, A., Weissenborn, D., Zhai, X., Unterthiner, T., Dehghani, M., Minderer, M., Heigold, G., Gelly, S., et al., . An image is worth 16x16 words: Transfo...

    2020

  2. [8]

    Kriegeskorte,N.,Mur,M.,Bandettini,P.A.,2008

    Reformer: The efficient transformer, in: International Conference on Learning Representations. Kriegeskorte,N.,Mur,M.,Bandettini,P.A.,2008. Representationalsimilarityanalysis-connectingthebranchesofsystemsneuroscience. Frontiers in systems neuroscience 2,

    2008

  3. [9]

    7083–7093

    Tsm: Temporal shift module for efficient video understanding, in: Proceedings of the IEEE/CVF international conference on computer vision, pp. 7083–7093. Liu, Z., Hu, H., Lin, Y., Yao, Z., Xie, Z., Wei, Y., Ning, J., Cao, Y., Zhang, Z., Dong, L., et al., 2022a. Swin transformer v2: Scaling up capacity and resolution, in: Proceedings of the IEEE/CVF confer...

    2021

  4. [12]

    Simonyan,K.,Zisserman,A.,2014

    Outrageously large neural networks: The sparsely-gated mixture-of-experts layer, in: International Conference on Learning Representations. Simonyan,K.,Zisserman,A.,2014. Two-streamconvolutionalnetworksforactionrecognitioninvideos. Advancesinneuralinformationprocessing systems

    2014

  5. [13]

    bioRxiv , 2024–08

    Conclusions about neural network to brain alignment are profoundly impacted by the similarity measure. bioRxiv , 2024–08. Tran, D., Wang, H., Torresani, L., Feiszli, M.,

    2024

  6. [14]

    arXiv preprint arXiv:2006.04768

    Linformer: Self-attention with linear complexity. arXiv preprint arXiv:2006.04768 . Wang, Y., Li, K., Li, X., Yu, J., He, Y., Chen, G., Pei, B., Zheng, R., Wang, Z., Shi, Y., et al.,

    Pith/arXiv arXiv 2006

  7. [15]

    Videoprism: A foundational visual encoder for video understanding, in: International Conference on Machine Learning, PMLR. pp. 60785–60811. Zhou,H.,Friedman,H.S.,VonDerHeydt,R.,2000. Codingofborderownershipinmonkeyvisualcortex. JournalofNeuroscience20,6594–6611. A.H. Fadaei et al.:Preprint submitted to ElsevierPage 23 of 23

    2000

  8. [2009]

    Imagenet: A large-scale hierarchical image database, in: 2009 IEEE conference on computer vision and pattern recognition, Ieee. pp. 248–255. Desimone, R., Duncan, J., et al.,

    2009

  9. [2017]

    Kietzmann,T.C.,Spoerer,C.J.,Sörensen,L.K.,Cichy,R.M.,Hauk,O.,Kriegeskorte,N.,2019

    Categorical reparameterization with gumbel-softmax, in: International Conference on Learning Representations. Kietzmann,T.C.,Spoerer,C.J.,Sörensen,L.K.,Cichy,R.M.,Hauk,O.,Kriegeskorte,N.,2019. Recurrenceisrequiredtocapturetherepresentational dynamics of the human visual system. Proceedings of the National Academy of Sciences 116, 21854–21863. Kitaev, N., ...

    2019

  10. [2019]

    arXiv preprint arXiv:1904.10509

    Generating long sequences with sparse transformers. arXiv preprint arXiv:1904.10509 . Choromanski, K.M., Likhosherstov, V., Dohan, D., Song, X., Gane, A., Sarlos, T., Hawkins, P., Davis, J.Q., Mohiuddin, A., Kaiser, L., et al., . Rethinking attention with performers, in: International Conference on Learning Representations. Cichy, R.M., Kaiser, D.,

    Pith/arXiv arXiv 1904

  11. [2020]

    arXiv preprint arXiv:2004.05150

    Longformer: The long-document transformer. arXiv preprint arXiv:2004.05150 . Bertasius,G.,Wang,H.,Torresani,L.,2021. Isspace-timeattentionallyouneedforvideounderstanding?,in:InternationalConferenceonMachine Learning, p

    Pith/arXiv arXiv 2004

  12. [2021]

    Tokenlearner: What can 8 learned tokens do for images and videos? arXiv preprint arXiv:2106.11297 . A.H. Fadaei et al.:Preprint submitted to ElsevierPage 22 of 23 Brain-Aligned Multi-Stream Video Transformers with Sparse Self-Selection Schirrmeister,R.T.,Springenberg,J.T.,Fiederer,L.D.J.,Glasstetter,M.,Eggensperger,K.,Tangermann,M.,Hutter,F.,Burgard,W.,Ba...

    Pith/arXiv arXiv 2017

  13. [2023]

    biorxiv 2022.07

    Aligning model and macaque inferior temporal cortex representations improves model-to-human behavioral alignment and adversarial robustness.. biorxiv 2022.07. 01.498495, in: International Conference on Learning Representations. Deng, J., Dong, W., Socher, R., Li, L.J., Li, K., Fei-Fei, L.,

    2022

  14. [2024]

    arXiv preprint arXiv:2403.16128

    Enhancing video transformers for action understanding with vlm-aided training. arXiv preprint arXiv:2403.16128 . Rao,Y.,Zhao,W.,Liu,B.,Lu,J.,Zhou,J.,Hsieh,C.J.,2021. Dynamicvit:Efficientvisiontransformerswithdynamictokensparsification. Advances in neural information processing systems 34, 13937–13949. Ryoo, M.S., Piergiovanni, A., Arnab, A., Dehghani, M.,...

    Pith/arXiv arXiv 2021

  15. [2025]

    arXiv preprint arXiv:2506.09985

    V-jepa 2: Self-supervised video models enable understanding, prediction and planning. arXiv preprint arXiv:2506.09985 . Beltagy, I., Peters, M.E., Cohan, A.,

    Pith/arXiv arXiv