Multimodal Alignment with Cross-Attentive GRUs for Fine-Grained Video Understanding

Junhwa Kim; Namho Kim

REVIEW 4 major objections 5 minor 31 references

Multimodal Alignment with Cross-Attentive GRUs for Fine-Grained Video Understanding

T0 review · 4 major / 5 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read A three-stream GRU cross-attention model that fuses video motion, static frames, and generated captions reports average F1 above 0.94 on DVD violence detection and CCC around 0.89 on Aff-Wild2 valence-arousal estimation.

desk verdict The architecture is a reasonable recipe, but the experiments contain zero comparisons, so the paper's central performance claims are unfalsifiable. read the letter →

arxiv 2507.03531 v1 pith:2JOK26E4 submitted 2025-07-04 cs.CV cs.AI

Namho Kim , Junhwa Kim This is my paper

classification cs.CVcs.AI

keywords multimodalfusioncross-attentionGRUvideounderstandingviolencedetectionvalence-arousalestimationCLIPPrism

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper tries to establish that a single video can be understood more reliably when three of its views—motion, static appearance, and machine-generated captions—are encoded separately and then fused by a two-stage cross-attention mechanism built on lightweight GRU sequence encoders. The authors report that on the DVD violence-detection benchmark the fused model reaches an average F1 above 0.94 across five folds, and on Aff-Wild2 valence-arousal regression it maintains a concordance correlation coefficient around 0.89. Their claim is that this fusion strategy significantly outperforms unimodal and naive fusion baselines, with cross-attention and feature-level augmentation as the main drivers of the improvement. A sympathetic reader would care because the architecture uses frozen pretrained encoders, so the learned part is small enough for efficient deployment on fine-grained video tasks.

What carries the argument

The central object is the cross-attentive GRU fusion block. Each modality stream produces a sequence of features—VideoPrism output for 16 sampled frames, CLIP image embeddings for 16 static frames, and CLIP text embeddings for 4 MiniGPT-4 captions—which a single-layer GRU compresses to $h^{(v)}$, $h^{(i)}$, and $h^{(t)}$. The image state $h^{(i)}$ is then used as the query in two attention stages, $z^{(i\leftarrow v)} = \mathrm{Attention}(Q=h^{(i)}, K=h^{(v)}, V=h^{(v)})$ and $z^{(i\leftarrow t)} = \mathrm{Attention}(Q=h^{(i)}, K=h^{(t)}, V=h^{(t)})$, and the concatenation $h^{(\mathrm{fused})} = [z^{(i\leftarrow v)}; z^{(i\leftarrow t)}] \in \mathbb{R}^{2d_h}$ is passed to a shallow MLP head for classification or regression. This image-as-query design is what carries the fusion claim: it creates the paper's two-stage cross-modal exchange in which static appearance mediates between motion and language.

What would settle it

Run the proposed architecture on the official DVD and Aff-Wild2 validation protocols with the same five folds and report per-fold F1 and CCC; the central claim fails if those numbers cannot be reproduced, or if a unimodal video-only stream or a plain concatenation baseline matches or beats the fused model on either benchmark.

Watch

Extended reading notes

Core claim

The central discovery, as the authors state it, is that letting the image-stream representation act as the query in two successive attention steps—first over the video-stream representation and then over the text-stream representation—yields a fused embedding that captures motion, spatial detail, and semantic content better than any single stream. Concretely, per-fold validation F1 on DVD ranges from 0.9380 to 0.9450 and CCC on Aff-Wild2 ranges from 0.8869 to 0.8948, and the paper attributes the consistency of these numbers to the cross-attention fusion and to feature-level augmentation and autoencoding used as regularizers during training.

Load-bearing premise

The evaluation premise is that the reported numbers come from valid five-fold splits of the official DVD and Aff-Wild2 protocols and are computed with the same preprocessing and metric definitions used by prior work on those benchmarks.

Editorial extensions

If this is right

If the reported five-fold numbers reproduce under official protocols, the framework is stable enough across splits to serve as a drop-in fusion head for video classification and affect regression.
Because the video, image, and text encoders are frozen, adapting the framework to a new fine-grained video task requires training only the GRUs, the attention stages, and the prediction head—a much smaller optimization problem than full transformer fusion.
The design carries a modularity claim: replace MiniGPT-4 captions with any other semantic stream, such as audio transcripts, and the same cross-attention fusion should remain effective; the authors list audio as future work.
The paper's attribution of the gains to cross-attention and feature-level augmentation is a testable commitment: disabling either component should lower DVD F1 or Aff-Wild2 CCC, as the claimed ablations would show.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

Beyond the paper: the image-as-query choice effectively anchors the fused representation on static scene semantics, and a direct extension would test whether video-as-query or text-as-query changes performance on tasks where motion or language is the dominant cue.
Beyond the paper: if the reported performance holds, the same three-stream recipe may transfer to neighboring fine-grained video problems such as action quality assessment, deepfake detection, or video captioning, where motion, appearance, and language cues are similarly complementary.
Beyond the paper: the comparison to prior work is only meaningful under the official challenge protocols, so a worthwhile reproducibility check is to rerun the exact five-fold splits and confirm the given per-fold numbers, since the submitted text does not specify preprocessing or metric details.
Beyond the paper: the abstract's mention of feature-level augmentation and autoencoding as regularizers suggests a broader design principle—regularizing multimodal fusion through reconstruction—that could be studied independently of the cross-attention mechanism.

Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

Read it. The architecture is sensible but the paper as submitted does not support its own claims. The specific recipe — frozen VideoPrism video features, CLIP image and text features, GRU temporal encoders, and a two-stage image-as-query cross-attention — is a reasonable combination, and the writing is clear about the design choices. That combination is new as a recipe, though none of the components are.

What the paper does well: it picks a plausible set of frozen encoders, uses Focal Loss for the imbalanced DVD task and MSE for regression, and keeps the trainable machinery light. If replication is attempted, the sampling procedure (64-frame windows, 16 frames for video/image, 4 frames for captions) is concrete. There's also no sign of citation inflation or self-promotion; the reference list is standard.

The soft spots are large. Table 1 reports only the multimodal model's per-fold validation numbers for DVD and Aff-Wild2. There is no unimodal baseline, no naive concatenation baseline, no comparison to any published method, no error bars, and no ablation. The abstract says the fusion 'significantly outperforms unimodal baselines' and credits cross-attention and feature-level augmentation, but the methods section never describes the augmentation or the autoencoding regularization. Those are load-bearing claims with zero supporting experiments. The stress-test note is on target. The Aff-Wild2 CCC around 0.89 is also surprising for a model without audio, but the lack of any comparison is the bigger problem.

Who should read this? Someone who wants a concrete lightweight fusion recipe to try, maybe as a starting point for their own work. The reported numbers are not evidence of anything except that the model runs and produces stable-looking validation values.

Recommendation: reject as-is. I'd be open to seeing a revised version with actual baselines, ablations, error bars, and a description of the augmentation/autoencoding. That would be a solid workshop paper, possibly more. If the editor asks me whether to send it out, I'd say send it to a knowledgeable referee just to get a second opinion on whether the missing experiments are as fatal as they look — but a desk reject is also defensible.

Referee Report

4 major / 5 minor

Summary. The paper proposes a multimodal framework for fine-grained video understanding that fuses video, image, and text representations using GRU-based sequence encoders and bidirectional cross-attention. The model is evaluated on the DVD violence-detection dataset and the Aff-Wild2 valence-arousal dataset, and the authors report average five-fold validation scores above 0.94 F1 for DVD and approximately 0.89 CCC for Aff-Wild2. The abstract and conclusion claim that the proposed fusion strategy significantly outperforms unimodal baselines and that cross-attention and feature-level augmentation contribute notably to performance, but the submitted text contains no such comparisons or ablations.

Significance. If validated, the proposed architecture would be a lightweight and modular alternative to transformer-based multimodal fusion, which is a useful practical contribution for edge deployment. The use of frozen pretrained encoders and GRU sequence modeling is sound in principle. However, the significance is currently not established because the central comparative claims are unsupported: no baselines, ablations, error bars, or external comparisons are reported. The manuscript also lacks a reproducible experimental protocol. The architecture idea is interesting, but the evidence presented is insufficient to judge whether the method actually improves over simpler alternatives.

major comments (4)

[Abstract; Section 4.2, Table 1] The central claim that the proposed fusion strategy 'significantly outperforms unimodal baselines' is unsupported. Table 1 reports only the proposed model's five-fold validation numbers for the multimodal configuration, with no unimodal baselines, no naive-fusion baseline, no state-of-the-art comparisons, no ablation, and no variance estimates. Without any comparison condition, the word 'outperforms' is unfalsifiable, and the attributed contributions of cross-attention and feature augmentation cannot be tested.
[Abstract; Section 3; Section 5] The abstract and conclusion state that the model is regularized through feature-level augmentation and autoencoding techniques, but neither technique is described in the methodology (Section 3) nor evaluated in the experiments (Section 4). The reader is left to infer what these techniques are and how they affect results. Either provide exact definitions, equations, and ablation results, or remove the claim from the abstract and conclusion.
[Section 4.1] The experimental setup is under-specified to the point of non-reproducibility. The text only states that five folds follow 'official validation protocols' and gives optimizer, learning rate, batch size, and early stopping. It omits preprocessing, metric definitions, the number of clips or frames per fold, label distributions, GRU hidden dimension, and the sampling details used for the folds. The very flat five-fold numbers in Table 1 cannot be interpreted without knowing whether folds are video-level or clip-level and how metrics are aggregated.
[Section 4.2] The reported Aff-Wild2 CCC of about 0.89 is surprising because the model uses only video, image, and text, whereas most competitive ABAW systems also use audio. This is not an internal contradiction, but it underscores the need for a concrete comparison against prior results on the same official protocol. Without such a comparison, the 'robustness and generalizability' conclusion drawn in Section 4.2 is not supported.

minor comments (5)

[Section 4.2] The sentence 'This demonstrates the robustness and generalizability of the proposed cross-attention-based fusion architecture' is not a logical consequence of Table 1, which contains no comparison; it should be rephrased as a factual report of observed scores.
[Keywords] The keyword 'Valance-Arousal' is a typo; it should be 'Valence-Arousal'.
[References [2] and [3]] References [2] and [3] appear to cite the same ABAW 8th workshop work with different formatting; merge them into a single entry to avoid duplication.
[Section 3] Figure 1 is referenced but not visible in the submitted text, so the architecture diagram cannot be verified; a rendered figure is necessary.
[Notation in Eqs. (1)-(6)] The notation uses superscripts for modalities and subscripts for sequence positions, but the relation between the sampled frame counts (16 for video/image, 4 for text) and the '64 segments' mentioned at the start of Section 3 is not made explicit.

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity found: the paper's validation numbers are produced by its own model on external benchmarks, with no definitional equivalence, fitted-input renaming, or load-bearing self-citation.

full rationale

The paper's derivation chain, Equations (1)-(9), defines a forward architecture mapping frozen encoder outputs to a fused embedding, followed by a task-specific head trained with standard losses. Table 1 reports five-fold validation scores on the DVD and Aff-Wild2 benchmarks, whose labels are external ground truth. There is no parameter that is fitted to a subset and then renamed as a prediction of a closely related quantity, no uniqueness theorem imported from the authors' prior work, and no self-citation chain that carries the argument. The abstract and introduction claim that the fusion strategy 'significantly outperforms unimodal baselines' and that ablation studies confirm the importance of cross-attention and feature augmentation, but the submitted text contains no baseline results and no ablation table. That is a serious evidentiary gap regarding the comparative claims, not a circularity: the absence of a comparison condition makes the claim unfalsifiable, but it does not make the reported numbers equivalent to the model's inputs by construction. Similarly, the use of early stopping and architecture selection on validation folds is a standard model-selection risk (potential overfitting to the validation split), not a definitional reduction of the predicted scores to the training objective. Because no specific circular step can be quoted and exhibited from the paper's equations or citations, the appropriate finding is no significant circularity with score 0.

Assumptions & free parameters 4 free parameters · 4 assumptions · 0 invented entities

The paper contributes a network architecture whose behavior is entirely determined by trained parameters and unstated hyperparameters. It introduces no new theory, no new entities, and no independent evidence for its premises; the central results depend on the validity of the frozen encoders, the benchmark protocols, and the temporal sampling scheme.

free parameters (4)

GRU hidden dimension d_h
Chosen by hand and not reported; the fused embedding dimension 2*d_h depends on it.
Number of sampled frames and window size = window 64, samples 16, 16, 4
Architecture choices set without justification; the central results depend on them.
Trainable parameters of GRU, attention, and MLP head
Fitted on the training folds; their learned values are the actual mechanism producing reported numbers, but no details are given.
Learning rate and batch size = 3e-4, batch 8
Chosen hyperparameters; early stopping criterion and Focal Loss parameters are unspecified.

assumptions (4)

domain assumption Frozen encoders VideoPrism, CLIP image, CLIP text, and MiniGPT provide task-relevant and complementary representations.
Sections 3.1.1 to 3.1.3 use these encoders without evidence that their embeddings capture violence or affect cues in this setting.
domain assumption Official DVD and Aff-Wild2 validation protocols and labels are used correctly.
Section 4.1 says official validation protocols are used but gives no split details, preprocessing, or metric definitions; all reported scores depend on this.
standard math The attention mechanism is the standard scaled dot-product attention and GRU updates follow Cho et al.
Equations 7 and 8 invoke Attention without defining it; standard definitions are assumed.
domain assumption The sequence order of sampled frames and captions is temporally meaningful after uniform sampling.
Uniform sampling over a 64-frame window in Sections 3.1.1 to 3.1.3 is assumed to preserve enough temporal structure for GRU modeling, but no analysis supports this.

how reviews work

0 comments

Cite this review

Pith. "Pith review of Multimodal Alignment with Cross-Attentive GRUs for Fine-Grained Video Understanding." pith.science (2026). https://pith.science/paper/2JOK26E4

@misc{pith2026250703531,
  author       = {Pith},
  title        = {Pith review of: Multimodal Alignment with Cross-Attentive GRUs for Fine-Grained Video Understanding},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/2JOK26E4}},
  note         = {Machine review of arXiv:2507.03531}
}

read the original abstract

Fine-grained video classification requires understanding complex spatio-temporal and semantic cues that often exceed the capacity of a single modality. In this paper, we propose a multimodal framework that fuses video, image, and text representations using GRU-based sequence encoders and cross-modal attention mechanisms. The model is trained using a combination of classification or regression loss, depending on the task, and is further regularized through feature-level augmentation and autoencoding techniques. To evaluate the generality of our framework, we conduct experiments on two challenging benchmarks: the DVD dataset for real-world violence detection and the Aff-Wild2 dataset for valence-arousal estimation. Our results demonstrate that the proposed fusion strategy significantly outperforms unimodal baselines, with cross-attention and feature augmentation contributing notably to robustness and performance.

Figures

Figures reproduced from arXiv: 2507.03531 by the authors.

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

31 extracted references · 24 canonical work pages

[1]

Learning transferable visual models from natural language supervision

Alec Radford, Jong Wook Kim, Christine Hallacy, et al. Learning transferable visual models from natural language supervision. In Proceedings of the International Conference on Machine Learning (ICML), pages 8748–8763. PMLR, 2021

work page 2021
[2]

Cowen, Stefanos Zafeiriou, Irene Kotsia, Eric Granger, Marco Pedersoli, Simon L

Dimitrios Kollias, Panagiotis Tzirakis, Alan S. Cowen, Stefanos Zafeiriou, Irene Kotsia, Eric Granger, Marco Pedersoli, Simon L. Bacon, Alice Baird, Chris Gagne, Chunchang Shao, Guanyu Hu, Soufiane Belharbi, and Muhammad Haseeb Aslam. Advancements in affective and behavior analysis: The 8th abaw workshop and competition. 2025

work page 2025
[3]

Advancements in affective and behavior analysis: The 8th abaw workshop and competition

Dimitrios Kollias, Panagiotis Tzirakis, Alan Cowen, Irene Kotsia, UK Cogitat, Eric Granger, Marco Pedersoli, Simon Bacon, Alice Baird, Chunchang Shao, et al. Advancements in affective and behavior analysis: The 8th abaw workshop and competition

work page
[4]

7th ABAW Competition: Multi-Task Learning and Compound Expression Recognition

Dimitrios Kollias, Stefanos Zafeiriou, Irene Kotsia, Abhinav Dhall, Shreya Ghosh, Chunchang Shao, and Guanyu Hu. 7th abaw competition: Multi-task learning and compound expression recognition. arXiv preprint arXiv:2407.03835, 2024

work page Pith review arXiv 2024
[5]

The 6th affective behavior analysis in-the-wild (abaw) competition

Dimitrios Kollias, Panagiotis Tzirakis, Alan Cowen, Stefanos Zafeiriou, Irene Kotsia, Alice Baird, Chris Gagne, Chunchang Shao, and Guanyu Hu. The 6th affective behavior analysis in-the-wild (abaw) competition. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 4587–4598, 2024

work page 2024
[6]

Distribution matching for multi-task learning of classification tasks: A large-scale study on faces & beyond

Dimitrios Kollias, Viktoriia Sharmanska, and Stefanos Zafeiriou. Distribution matching for multi-task learning of classification tasks: A large-scale study on faces & beyond. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 38, pages 2813–2821, 2024

work page 2024
[7]

Abaw: Valence-arousal estimation, expression recognition, action unit detection & emotional reaction intensity estimation challenges

Dimitrios Kollias, Panagiotis Tzirakis, Alice Baird, Alan Cowen, and Stefanos Zafeiriou. Abaw: Valence-arousal estimation, expression recognition, action unit detection & emotional reaction intensity estimation challenges. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 5888–5897, 2023

work page 2023
[8]

Multi-label compound expression recognition: C-expr database & network

Dimitrios Kollias. Multi-label compound expression recognition: C-expr database & network. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 5589–5598, 2023

work page 2023

Show all 31 references

[9]

Abaw: Valence-arousal estimation, expression recognition, action unit detection & emotional reaction intensity estimation challenges

Dimitrios Kollias, Panagiotis Tzirakis, Alice Baird, Alan Cowen, and Stefanos Zafeiriou. Abaw: Valence-arousal estimation, expression recognition, action unit detection & emotional reaction intensity estimation challenges. In Proceedings of the IEEE/CVF Conference on Computer ...

2023
[10]

Abaw: Valence-arousal estimation, expression recognition, action unit detection & multi-task learning challenges

Dimitrios Kollias. Abaw: Valence-arousal estimation, expression recognition, action unit detection & multi-task learning challenges. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 2328–2336, 2022

2022
[11]

Analysing affective behavior in the second abaw2 competition

Dimitrios Kollias and Stefanos Zafeiriou. Analysing affective behavior in the second abaw2 competition. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 3652–3660, 2021

2021
[12]

Analysing affective behavior in the first abaw 2020 competition

D Kollias, A Schulc, E Hajiyev, and S Zafeiriou. Analysing affective behavior in the first abaw 2020 competition. In 2020 15th IEEE International Conference on Automatic Face and Gesture Recognition (FG 2020) , pages 794–800, 2020

2020
[13]

Distribution matching for heterogeneous multi-task learning: A large-scale face study

Dimitrios Kollias, Viktoriia Sharmanska, and Stefanos Zafeiriou. Distribution matching for heterogeneous multi-task learning: A large-scale face study. arXiv preprint arXiv:2105.03790, 2021

2021 arXiv
[14]

Affect analysis in-the-wild: Valence-arousal, expressions, action units and a unified framework

Dimitrios Kollias and Stefanos Zafeiriou. Affect analysis in-the-wild: Valence-arousal, expressions, action units and a unified framework. arXiv preprint arXiv:2103.15792, 2021

2021 arXiv
[15]

Expression, affect, action unit recognition: Aff-wild2, multi-task learning and arcface

Dimitrios Kollias and Stefanos Zafeiriou. Expression, affect, action unit recognition: Aff-wild2, multi-task learning and arcface. In arXiv preprint arXiv:1910.04855. arXiv, 2019

1910 arXiv
[16]

Face behavior a la carte: Expressions, affect and action units in a single network

Dimitrios Kollias, Viktoriia Sharmanska, and Stefanos Zafeiriou. Face behavior a la carte: Expressions, affect and action units in a single network. arXiv preprint arXiv:1910.11111, 2019

1910 arXiv
[17]

Deep affect prediction in-the-wild: Aff-wild database and challenge, deep architectures, and beyond

Dimitrios Kollias, Panagiotis Tzirakis, Mihalis A Nicolaou, Athanasios Papaioannou, Guoying Zhao, Björn Schuller, Irene Kotsia, and Stefanos Zafeiriou. Deep affect prediction in-the-wild: Aff-wild database and challenge, deep architectures, and beyond. International Journal of...

2019
[18]

Dvd: A comprehensive dataset for advancing violence detection in real-world scenarios

Dimitrios Kollias, Damith Senadeera, Jianian Zheng, Kaushal Yadav, Greg Slabaugh, Muhammad Awais, and Xiaoyun Yang. Dvd: A comprehensive dataset for advancing violence detection in real-world scenarios. https: //www.researchgate.net/publication/392397877, 2025. DOI: 10.13140/R...

2025
[19]

Deep residual learning for image recognition

Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pages 770–778. IEEE, 2016

2016
[20]

Efficientnetv2: Smaller models and faster training

Mingxing Tan and Quoc Le. Efficientnetv2: Smaller models and faster training. InProceedings of the International Conference on Machine Learning (ICML), pages 10096–10106. PMLR, 2021

2021
[21]

Masked autoencoders are scalable vision learners

Kaiming He, Xinlei Chen, Saining Xie, et al. Masked autoencoders are scalable vision learners. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 16000–16009. IEEE, 2022

2022
[22]

Cnn architectures for large-scale audio classification

Shawn Hershey, Sourish Chaudhuri, Daniel Ellis, et al. Cnn architectures for large-scale audio classification. In Proceedings of the IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pages 131–135. IEEE, 2017

2017
[23]

wav2vec 2.0: A framework for self- supervised learning of speech representations

Alexei Baevski, Yuhao Zhou, Abdelrahman Mohamed, and Michael Auli. wav2vec 2.0: A framework for self- supervised learning of speech representations. In Advances in Neural Information Processing Systems (NeurIPS), pages 12449–12460. NeurIPS, 2020

2020
[24]

Bert: Pre-training of deep bidirectional transformers for language understanding

Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. Bert: Pre-training of deep bidirectional transformers for language understanding. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics (NAACL), pag...

2019
[25]

Attention is all you need

Ashish Vaswani, Noam Shazeer, Niki Parmar, et al. Attention is all you need. In Advances in Neural Information Processing Systems (NeurIPS), pages 5998–6008. NeurIPS, 2017

2017
[26]

Learning phrase representations using rnn encoder-decoder for statistical machine translation

Kyunghyun Cho, Bart van Merriënboer, Caglar Gulcehre, et al. Learning phrase representations using rnn encoder-decoder for statistical machine translation. In Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 1724–1734. ACL, 2014

2014
[27]

An empirical evaluation of generic convolutional and recurrent networks for sequence modeling

Shaojie Bai, J Zico Kolter, and Vladlen Koltun. An empirical evaluation of generic convolutional and recurrent networks for sequence modeling. In arXiv preprint arXiv:1803.01271. arXiv, 2018

2018 arXiv
[28]

Videoprism: Sparse video token mixer for temporal video understanding

Mihai Georgescu, Min-Hung Liu, Nikolaos Sarafianos, Xiyang Dai, et al. Videoprism: Sparse video token mixer for temporal video understanding. arXiv preprint arXiv:2305.15001, 2023

2023 arXiv
[29]

Minigpt-4: Enhancing vision-language understanding with advanced large language models

Dakuo Zhu, Jun Zhang, Yusheng Wang, Zhiyi Li, et al. Minigpt-4: Enhancing vision-language understanding with advanced large language models. arXiv preprint arXiv:2304.10592, 2023

2023 arXiv
[30]

Focal loss for dense object detection

Tsung-Yi Lin, Priya Goyal, Ross Girshick, Kaiming He, and Piotr Dollár. Focal loss for dense object detection. In Proceedings of the IEEE international conference on computer vision, pages 2980–2988, 2017

2017
[31]

Decoupled weight decay regularization

Ilya Loshchilov and Frank Hutter. Decoupled weight decay regularization. In International Conference on Learning Representations (ICLR), 2019. 6

2019

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.

[1] [1]

Learning transferable visual models from natural language supervision

Alec Radford, Jong Wook Kim, Christine Hallacy, et al. Learning transferable visual models from natural language supervision. In Proceedings of the International Conference on Machine Learning (ICML), pages 8748–8763. PMLR, 2021

work page 2021

[2] [2]

Cowen, Stefanos Zafeiriou, Irene Kotsia, Eric Granger, Marco Pedersoli, Simon L

Dimitrios Kollias, Panagiotis Tzirakis, Alan S. Cowen, Stefanos Zafeiriou, Irene Kotsia, Eric Granger, Marco Pedersoli, Simon L. Bacon, Alice Baird, Chris Gagne, Chunchang Shao, Guanyu Hu, Soufiane Belharbi, and Muhammad Haseeb Aslam. Advancements in affective and behavior analysis: The 8th abaw workshop and competition. 2025

work page 2025

[3] [3]

Advancements in affective and behavior analysis: The 8th abaw workshop and competition

Dimitrios Kollias, Panagiotis Tzirakis, Alan Cowen, Irene Kotsia, UK Cogitat, Eric Granger, Marco Pedersoli, Simon Bacon, Alice Baird, Chunchang Shao, et al. Advancements in affective and behavior analysis: The 8th abaw workshop and competition

work page

[4] [4]

7th ABAW Competition: Multi-Task Learning and Compound Expression Recognition

Dimitrios Kollias, Stefanos Zafeiriou, Irene Kotsia, Abhinav Dhall, Shreya Ghosh, Chunchang Shao, and Guanyu Hu. 7th abaw competition: Multi-task learning and compound expression recognition. arXiv preprint arXiv:2407.03835, 2024

work page Pith review arXiv 2024

[5] [5]

The 6th affective behavior analysis in-the-wild (abaw) competition

Dimitrios Kollias, Panagiotis Tzirakis, Alan Cowen, Stefanos Zafeiriou, Irene Kotsia, Alice Baird, Chris Gagne, Chunchang Shao, and Guanyu Hu. The 6th affective behavior analysis in-the-wild (abaw) competition. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 4587–4598, 2024

work page 2024

[6] [6]

Distribution matching for multi-task learning of classification tasks: A large-scale study on faces & beyond

Dimitrios Kollias, Viktoriia Sharmanska, and Stefanos Zafeiriou. Distribution matching for multi-task learning of classification tasks: A large-scale study on faces & beyond. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 38, pages 2813–2821, 2024

work page 2024

[7] [7]

Abaw: Valence-arousal estimation, expression recognition, action unit detection & emotional reaction intensity estimation challenges

Dimitrios Kollias, Panagiotis Tzirakis, Alice Baird, Alan Cowen, and Stefanos Zafeiriou. Abaw: Valence-arousal estimation, expression recognition, action unit detection & emotional reaction intensity estimation challenges. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 5888–5897, 2023

work page 2023

[8] [8]

Multi-label compound expression recognition: C-expr database & network

Dimitrios Kollias. Multi-label compound expression recognition: C-expr database & network. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 5589–5598, 2023

work page 2023