Pith. sign in

REVIEW 2 major objections 5 minor 50 references

EPIC-Fusion: Audio-Visual Temporal Binding for Egocentric Action Recognition

T0 review · 2 major / 5 minor · reviewed 2026-08-14 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read This paper claims that fusing RGB, optical flow, and audio at mid-level features within a temporal binding window, before any temporal aggregation, outperforms late fusion and sets state-of-the-art results on the EPIC-Kitchens egocentric…

desk verdict Solid multimodal fusion paper, but the claimed advantage of temporal binding over late fusion is confounded by joint training; needs a clean control. read the letter →

arxiv 1908.08498 v1 pith:FQPBDGKH submitted 2019-08-22 cs.CV

Evangelos Kazakos , Arsha Nagrani , Andrew Zisserman , Dima Damen This is my paper · ORCID
classification cs.CV
keywords egocentricactionrecognitiontemporalbindingwindowmulti-modalfusionaudio-visuallearningopticalflowaudioEPIC-Kitchens
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Egocentric action recognition typically treats each input stream separately, pooling RGB and optical flow over time and only averaging their scores at the end. This paper argues that fusing the streams earlier, at mid-level network features, and allowing each modality to be sampled at a slightly different moment within a temporal binding window, lets the network exploit the fact that sound, motion, and appearance of an action are not synchronous. On the EPIC-Kitchens benchmark, the proposed Temporal Binding Network, trained end-to-end on RGB, Flow, and Audio, beats every single modality and every late-fusion baseline, and the audio stream turns out to be nearly as informative as appearance for recognizing verbs.

What carries the argument

The load-bearing object is the Temporal Binding Window (TBW): a range of temporal offsets around each anchor sample within which the other modalities are selected. The network's innovation is to fuse before aggregation—mid-level features from the three streams are concatenated and mapped to a shared multimodal representation per window, and the same modality, fusion, and classification weights are reused across all temporal segments, so the model learns a single binding function rather than per-segment ones. At training the window is as wide as the whole action; at test the best width is around 120 to 180 milliseconds. The comparison to late fusion and to gating variants is what carries the argument that binding, not extra parameters, produces the gain.

What would settle it

Train the same architecture twice: once with near-synchronous sampling and once with the full-action window, then evaluate both at the optimal tight window; if the full-window model does not beat the near-synchronous one by the margin it reports over late fusion, the binding mechanism is not the cause.

Watch

Extended reading notes

Core claim

The paper's central claim is that a Temporal Binding Network—three modality-specific convnets whose mid-level features are concatenated and passed through a shared fusion layer before any temporal pooling—outperforms both individually trained streams and score-averaged late fusion on egocentric action recognition. The network samples one anchor RGB frame per segment and draws Flow and Audio samples uniformly from a window of width plus-or-minus b around it; weights are shared across temporal segments, and the segment-level predictions are averaged. With b set to the full action length during training and tested at much narrower widths, the single model reaches 64.75 top-1 verb accuracy on EPIC-Kitchens seen kitchens versus 55.49 for a late-fusion TSN with audio, and adding audio to RGB+Flow raises top-5 action accuracy by about five points on the seen split and four on the unseen split. The paper also claims that the resulting ensemble was state of the art on both seen and unseen test splits at the time of submission.

Load-bearing premise

During training, the non-visual samples are picked anywhere inside the whole action, while the test-time best window is only about a thirtieth of the action, so the method's success depends on loose random pairing transferring to tight pairing—and the paper does not train a model on the tight window to check this.

Editorial extensions

If this is right

  • Audio should be treated as a first-class modality for egocentric action recognition: on seen kitchens its top-1 verb accuracy alone is close to RGB, and it lifts the fused model's top-5 action accuracy by about five points on the seen split and four on the unseen split.
  • Simple concatenation fusion outperforms context gating and gated multimodal units inside the same network, suggesting the binding window itself, rather than the fusion operator, is the main source of improvement.
  • The gains are concentrated in rare classes: mean class accuracy for tail verbs improves by 63 percent under fusion versus 34 percent for the top 10 percent of classes, so the approach partially counteracts the dataset's heavy class imbalance.
  • A single binding-window width is not required at test time: averaging predictions across several overlapping windows makes the model robust to window width, so it can be deployed without precise action-synchronization annotations.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The wide training window is also a data-augmentation scheme; the paper does not isolate how much of the gain is temporal binding versus simply seeing many random temporal pairings, so a controlled comparison with a synchronously trained fusion model using the same random samples would settle it.
  • Because the test-time optimum is an absolute time interval of roughly 120 to 180 milliseconds, the model may effectively learn a fixed perceptual integration window rather than an action-relative one; grouping test actions by duration and recomputing the optimal window would discriminate between these explanations.
  • Audio opens a path to recognizing actions that occur outside the camera's field of view, as the paper's 'eat' example suggests, but the paper does not measure how often off-screen audio alone carries the classification; a per-class audio-only error analysis would quantify that.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

2 major / 5 minor

Summary. The paper introduces a Temporal Binding Network (TBN) for egocentric action recognition from RGB, optical flow, and audio on EPIC-Kitchens. The key idea is to fuse modalities at a mid-level representation before temporally aggregating, within a temporal binding window (TBW): the video is split into K segments, RGB anchor frames are sampled, and Flow and Audio samples are drawn uniformly from a window of width ±b around each anchor; the three streams' BN-Inception features are concatenated and fed through shared fully connected layers to produce per-window verb and noun predictions, which are averaged. The network is trained end-to-end. The paper reports large gains over individual modalities and over a TSN-based late-fusion baseline with audio, analyzes fusion strategies (concatenation, context gating, gating fusion), studies TBW width effects, shows audio's complementary value per class, and reports state-of-the-art results on the EPIC-Kitchens public leaderboard at submission.

Significance. If the temporal-binding attribution is accepted, this is a strong contribution: it targets an under-explored temporal-asynchrony aspect of multimodal egocentric recognition, provides one of the first audio-visual egocentric action-recognition systems, uses an externally defined benchmark, includes per-class and tail-class analyses, tests robustness to irrelevant audio, and releases code and models. The empirical evidence for the absolute performance of TBN is solid; the ablations of fusion mechanisms and single-modality strengths are informative. However, the central explanatory claim—that mid-level fusion within a TBW, rather than joint end-to-end training or random-sampling augmentation, is responsible for the gains over late fusion—is not yet isolated by the experiments as written.

major comments (2)
  1. [Section 4.2, Table 4] The comparison between TBN and 'Ours (TSN w. Audio)' changes two variables at once: fusion level (mid-level within segments vs late score averaging) and training protocol (jointly trained streams vs independently trained streams). The S1 top-1 action accuracy gap (34.80 vs 23.95) could therefore be due mainly to joint optimization rather than temporal binding. Please add a late-fusion control in which all three streams are trained jointly (e.g., TBN architecture with per-modality score aggregation, or TSN streams trained with a shared loss and end-to-end backpropagation) and a mid-level synchronous-fusion control (b≈0 during both training and testing) to separate the effects.
  2. [Section 4.2, 'The effect of TBW width'] The authors train with b ∈ {T/6, T/3, T} and report only 'little difference' without numbers, while the narrow windows that perform best at test time (b≈T/30 to T/20, Fig. 7) are never used during training. Since b=T means the auxiliary modalities are sampled from the entire action segment, the training procedure is a form of random cross-modal data augmentation; this alone, together with joint training, could explain the gains over late fusion without invoking temporally localized binding. Please report numeric results for all training widths and, ideally, train with b in the range that is optimal at inference (or with a width schedule) to demonstrate that the binding window matters during training and that training/test widths are consistent.
minor comments (5)
  1. [Section 4.1] The audio sampling rate is stated as '24000kHz'; this should be 24 kHz.
  2. [Section 4.2, Table 4] The ensemble of five TBNs is said to use different TBW widths, but the training-width ablation reports little difference among widths; specify which widths are used in the ensemble and how predictions are combined.
  3. [Section 3.2 and Figure 2] The test-time sampling procedure is described only briefly as '25 evenly spaced samples for each modality'; clarify how these samples are mapped to TBWs and how many TBWs are aggregated when making a video-level prediction.
  4. [Figure 7] The caption and text should state explicitly how the 'Sync' baseline is constructed (b≈0) and clarify the x-axis scale for the width parameter.
  5. [Appendix B] The leaderboard snapshots are useful, but the text should state the exact snapshot date and clarify the relationship between 'at the time of submission' and the final challenge ranking.

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity: the architecture is evaluated against an external public benchmark, and self-citations provide only the dataset and baselines, not the derivation.

full rationale

The paper's central claim is empirical: the TBN architecture, which fuses RGB, Flow, and Audio within a temporal binding window before temporal aggregation, outperforms individual modalities and a late-fusion TSN baseline on the EPIC-Kitchens benchmark. The evaluation is self-contained against an externally hosted public leaderboard, and the comparisons include independently published baselines such as Attention Clusters and the EPIC-Kitchens baseline. The self-citation to [8] supplies the dataset and one of the baseline results; it does not serve as the proof of the method's advantage and would not by itself force any of the reported numbers. No equation in Section 3 reuses an output as an input, no fitted parameter is renamed as a prediction, and no uniqueness theorem or prior-work ansatz is invoked to make the design choice compulsory. The TBW-width study is perhaps under-powered, and the TBN-versus-late-fusion comparison changes joint training and fusion time simultaneously, but these are experimental-design concerns about attribution, not circularity: the reported accuracies are not constructed from the claim they are meant to support. The derivation is therefore not circular.

Assumptions & free parameters 3 free parameters · 3 assumptions · 0 invented entities

The paper is an empirical architecture proposal. It introduces a model component, the temporal binding window, borrowed from neuroscience rather than a new physical or mathematical entity. The main free parameters are hyperparameters: window width, segment count, and audio window length. The validity of the EPIC-Kitchens benchmark and the transfer of the TBW concept are the key untested assumptions.

free parameters (3)
  • TBW width b = T in training; T/30 to T/20 optimal in testing
    Central hyperparameter; chosen relative to action length. Training uses the full action length while test sweeps show the best performance around 120-180ms.
  • Number of segments K = 3
    Follows the TSN sparse sampling recipe; not ablated in the paper, so the sensitivity of the result to this choice is unknown.
  • Audio window length = 1.28 seconds
    Fixed for all actions, including very short ones, and allowed to extend beyond action boundaries. This may not be optimal for all classes.
assumptions (3)
  • domain assumption The EPIC-Kitchens public leaderboard is a fair and unbiased comparison for egocentric action recognition.
    The central claim of state-of-the-art performance depends on the leaderboard reflecting accurate and comparable scores. The dataset paper is self-cited, which is standard but worth noting.
  • domain assumption The neuroscience construct of a multisensory temporal binding window transfers to machine learning fusion of RGB, Flow, and Audio.
    The method is motivated by the human TBW in Section 1. If random pairing within a window does not mimic this binding behavior, the benefit may reduce to data augmentation rather than true temporal binding.
  • domain assumption Using the RGB sample as the temporal anchor and randomly sampling Flow and Audio within a window around it is a valid asymmetric design.
    The architecture treats RGB as the reference modality. This asymmetry is untested; other modalities could serve as anchors, and the choice may affect the results.

how reviews work

0 comments
Cite this review

Pith. "Pith review of EPIC-Fusion: Audio-Visual Temporal Binding for Egocentric Action Recognition." pith.science (2026). https://pith.science/paper/FQPBDGKH

@misc{pith2026190808498,
  author       = {Pith},
  title        = {Pith review of: EPIC-Fusion: Audio-Visual Temporal Binding for Egocentric Action Recognition},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/FQPBDGKH}},
  note         = {Machine review of arXiv:1908.08498}
}
read the original abstract

We focus on multi-modal fusion for egocentric action recognition, and propose a novel architecture for multi-modal temporal-binding, i.e. the combination of modalities within a range of temporal offsets. We train the architecture with three modalities -- RGB, Flow and Audio -- and combine them with mid-level fusion alongside sparse temporal sampling of fused representations. In contrast with previous works, modalities are fused before temporal aggregation, with shared modality and fusion weights over time. Our proposed architecture is trained end-to-end, outperforming individual modalities as well as late-fusion of modalities. We demonstrate the importance of audio in egocentric vision, on per-class basis, for identifying actions as well as interacting objects. Our method achieves state of the art results on both the seen and unseen test sets of the largest egocentric dataset: EPIC-Kitchens, on all metrics using the public leaderboard.

Figures

Figures reproduced from arXiv: 1908.08498 by the authors.

Figure 1
Figure 1. As the width of the temporal binding window in [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. Left: our proposed Temporal Binding Network (TBN). Modalities are sampled within a TBW, and modality￾specific weights (same colour) are shared amongst different inputs. Modalities are fused with mid-level fusion and trained jointly. Predictions from multiple TBWs, possibly overlapping, are averaged. Right: TSN [42] with an additional audio stream performing late fusion. Modalities are trained independently. Note tha… view at source ↗
Figure 3
Figure 3. A single TBN block showing architectural de [PITH_FULL_IMAGE:figures/full_fig_p004_3.png] view at source ↗
Figures from the paper (6 more)
Figure 4
Figure 4. Figure 4: Verb (left) and noun (right) classes’ performances [PITH_FULL_IMAGE:figures/full_fig_p005_4.png]
Figure 5
Figure 5. Figure 5: Confusion matrix for the largest-15 verb classes, [PITH_FULL_IMAGE:figures/full_fig_p006_5.png]
Figure 6
Figure 6. Figure 6: Per-class accuracies for the S1 test set for verbs (top) and nouns (bottom) for fusion and single modalities. We select verb classes with more than 10 samples, and noun classes with more than 30 samples. The classes are presented in the order of number of samples per c…
Figure 7
Figure 7. Figure 7: Effect of TBW width for verbs (left) and nouns [PITH_FULL_IMAGE:figures/full_fig_p007_7.png]
Figure 8
Figure 8. Figure 8: Our submission on the action recognition chal [PITH_FULL_IMAGE:figures/full_fig_p011_8.png]
Figure 9
Figure 9. Figure 9: Confusion matrices for largest-15 verb classes (top) and the largest-15 noun classes (bottom), without (left) and [PITH_FULL_IMAGE:figures/full_fig_p012_9.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

50 extracted references · 45 canonical work pages

  1. [1]

    Marks, Dhruv Batra, and Devi Parikh

    Huda Alamri, Chiori Hori, Tim K. Marks, Dhruv Batra, and Devi Parikh. Audio visual scene-aware dialog (avsd) track for natural language generation in dstc7. In DSTC7 at AAAI2019 Workshop, 2018. 2

    work page 2018

  2. [2]

    Look, listen and learn

    Relja Arandjelovic and Andrew Zisserman. Look, listen and learn. In ICCV, 2017. 1, 2

    work page 2017

  3. [3]

    Objects that sound

    Relja Arandjelovic and Andrew Zisserman. Objects that sound. In ECCV, September 2018. 1, 2

    work page 2018

  4. [4]

    John Arevalo, Thamar Solorio, Manuel Montes-y Gmez, and Fabio A. Gonzlez. Gated multimodal units for information fusion. In ICLRW, 2017. 7

    work page 2017

  5. [5]

    Sound- net: Learning sound representations from unlabeled video

    Yusuf Aytar, Carl V ondrick, and Antonio Torralba. Sound- net: Learning sound representations from unlabeled video. In NIPS, 2016. 1, 2

    work page 2016

  6. [6]

    See, hear, and read: Deep aligned representations

    Yusuf Aytar, Carl V ondrick, and Antonio Torralba. See, hear, and read: Deep aligned representations. CoRR, abs/1706.00932, 2017. 1, 2

    arXiv 2017

  7. [7]

    Quo vadis, action recognition? a new model and the kinetics dataset

    Joao Carreira and Andrew Zisserman. Quo vadis, action recognition? a new model and the kinetics dataset. In CVPR,

  8. [8]

    Scaling egocentric vision: The epic-kitchens dataset

    Dima Damen, Hazel Doughty, Giovanni Maria Farinella, Sanja Fidler, Antonino Furnari, Evangelos Kazakos, Davide Moltisanti, Jonathan Munro, Toby Perrett, Will Price, and Michael Wray. Scaling egocentric vision: The epic-kitchens dataset. In ECCV, 2018. 1, 2, 4, 5, 8

    work page 2018

Show all 50 references
  1. [9]

    You-do, i-learn: Dis- covering task relevant objects and their modes of interaction from multi-user egocentric video

    Dima Damen, Teesid Leelasawassuk, Osian Haines, Andrew Calway, and Walterio Mayol-Cuevas. You-do, i-learn: Dis- covering task relevant objects and their modes of interaction from multi-user egocentric video. In BMVC, 2014. 1

    2014

  2. [10]

    Convolutional two-stream network fusion for video action recognition

    Christoph Feichtenhofer, Axel Pinz, and Andrew Zisserman. Convolutional two-stream network fusion for video action recognition. In CVPR, 2016. 1, 2, 3

    2016

  3. [11]

    2.5D visual sound

    Ruohan Gao and Kristen Grauman. 2.5D visual sound. In CVPR, 2019. 1, 2

    2019

  4. [12]

    A better baseline for ava

    Rohit Girdhar, Joo Carreira, Carl Doersch, and Andrew Zis- serman. A better baseline for ava. In ActivityNet Workshop at CVPR, 2018. 2

    2018

  5. [13]

    Actionvlad: Learning spatio-temporal aggregation for action classification

    Rohit Girdhar, Deva Ramanan, Abhinav Gupta, Josef Sivic, and Bryan Russell. Actionvlad: Learning spatio-temporal aggregation for action classification. In CVPR, 2017. 2

    2017

  6. [14]

    Batch normalization: Accelerating deep network training by reducing internal co- variate shift

    Sergey Ioffe and Christian Szegedy. Batch normalization: Accelerating deep network training by reducing internal co- variate shift. In ICML, 2015. 5

    2015

  7. [15]

    Aggregating local descriptors into a compact image representation

    Herve Jegou, Matthijs Douze, Cordelia Schmid, and Patrick Perez. Aggregating local descriptors into a compact image representation. In CVPR, 2010. 3

    2010

  8. [16]

    Coopera- tive learning of audio and video models from self-supervised synchronization

    Bruno Korbar, Du Tran, and Lorenzo Torresani. Coopera- tive learning of audio and video models from self-supervised synchronization. In NIPS, pages 7763–7774. 2018. 2

    2018

  9. [17]

    Lee, Joydeep Ghosh, and Kristen Grauman

    Yong J. Lee, Joydeep Ghosh, and Kristen Grauman. Dis- covering important people and objects for egocentric video summarization. In CVPR, 2012. 1

    2012

  10. [18]

    Action recognition with coarse-to-fine deep feature integration and asynchronous fusion

    Weiyao Lin, Yang Mi, Jianxin Wu, Ke Lu, and Hongkai Xiong. Action recognition with coarse-to-fine deep feature integration and asynchronous fusion. AAAI, 2018. 2, 4

    2018

  11. [19]

    Attention clusters: Purely attention based local feature integration for video classification

    Xiang Long, Chuang Gan, Gerard de Melo, Jiajun Wu, Xiao Liu, and Shilei Wen. Attention clusters: Purely attention based local feature integration for video classification. In CVPR, June 2018. 2, 8

    2018

  12. [20]

    Multimodal keyless attention fu- sion for video classification

    Xiang Long, Chuang Gan, Gerard Melo, Xiao Liu, Yandong Li, Fu Li, and Shilei Wen. Multimodal keyless attention fu- sion for video classification. In AAAI Conference on Artifi- cial Intelligence, 2018. 2

    2018

  13. [21]

    Minghuang Ma, Haoqi Fan, and Kris M. Kitani. Going deeper into first-person activity recognition. In CVPR, 2016. 1, 2

    2016

  14. [22]

    Learnable pooling with context gating for video classification

    Antoine Miech, Ivan Laptev, and Josef Sivic. Learnable pooling with context gating for video classification. CoRR, abs/1706.06905, 2017. 1, 7

    2017 arXiv

  15. [23]

    Learning a Text-Video Embedding from Imcomplete and Heteroge- neous Data

    Antoine Miech, Ivan Laptev, and Josef Sivic. Learning a Text-Video Embedding from Imcomplete and Heteroge- neous Data. In arXiv, 2018. 1, 2

    2018

  16. [24]

    Trespassing the boundaries: Labeling temporal bounds for object interactions in egocentric video

    Davide Moltisanti, Michael Wray, Walterio Mayol-Cuevas, and Dima Damen. Trespassing the boundaries: Labeling temporal bounds for object interactions in egocentric video. In ICCV, 2017. 2

    2017

  17. [25]

    Pierre Mgevand, Sophie Molholm, Ashabari Nayak, and John J. Foxe. Recalibration of the multisensory temporal window of integration results from changing task demands. PLOS ONE, 8, 2013. 2

    2013

  18. [26]

    Learnable PINs: Cross-modal embeddings for person iden- tity

    Arsha Nagrani, Samuel Albanie, and Andrew Zisserman. Learnable PINs: Cross-modal embeddings for person iden- tity. ECCV, 2018. 2

    2018

  19. [27]

    Seeing voices and hearing faces: Cross-modal biometric matching

    Arsha Nagrani, Samuel Albanie, and Andrew Zisserman. Seeing voices and hearing faces: Cross-modal biometric matching. In CVPR, 2018. 1, 2

    2018

  20. [28]

    Andrew Owens and Alexei A. Efros. Audio-visual scene analysis with self-supervised multisensory features. In ECCV, 2018. 1, 2

    2018

  21. [29]

    McDermott, William T

    Andrew Owens, Jiajun Wu, Josh H. McDermott, William T. Freeman, and Antonio Torralba. Ambient sound provides supervision for visual learning. In ECCV, 2016. 1, 2

    2016

  22. [30]

    Parise, Charles Spence, and Marc O

    CesareaV . Parise, Charles Spence, and Marc O. Ernst. When correlation implies causation in multisensory integration. Current Biology, 22(1):46 – 49, 2012. 2

    2012

  23. [31]

    Automatic differentiation in pytorch

    Adam Paszke, Sam Gross, Soumith Chintala, Gregory Chanan, Edward Yang, Zachary DeVito, Zeming Lin, Al- ban Desmaison, Luca Antiga, and Adam Lerer. Automatic differentiation in pytorch. In NIPS-W, 2017. 5

    2017

  24. [32]

    Detecting activities of daily living in first-person camera views

    Hamed Pirsiavash and Deva Ramanan. Detecting activities of daily living in first-person camera views. In CVPR, 2012. 1

    2012

  25. [33]

    On the momentum term in gradient descent learning algorithms

    Ning Qian. On the momentum term in gradient descent learning algorithms. Neural Networks, 12(1):145–151, 1999. 5

    1999

  26. [34]

    Learning to localize sound source in visual scenes

    Arda Senocak, Tae-Hyun Oh, Junsik Kim, Ming-Hsuan Yang, and In So Kweon. Learning to localize sound source in visual scenes. In CVPR, 2018. 1, 2

    2018

  27. [35]

    Sigurdsson, Abhinav Gupta, Cordelia Schmid, Ali Farhadi, and Karteek Alahari

    Gunnar A. Sigurdsson, Abhinav Gupta, Cordelia Schmid, Ali Farhadi, and Karteek Alahari. Actor and observer: Joint modeling of first and third-person videos. In CVPR, 2018. 1

    2018

  28. [36]

    Two-stream con- volutional networks for action recognition in videos

    Karen Simonyan and Andrew Zisserman. Two-stream con- volutional networks for action recognition in videos. In NIPS. 2014. 2

    2014

  29. [37]

    Suriya Singh, Chetan Arora, and C. V . Jawahar. First person action recognition using deep learned descriptors. In CVPR,

  30. [38]

    Multimodal multi-stream deep learning for egocentric activity recognition

    Sibo Song, Vijay Chandrasekhar, Bappaditya Mandal, Liyuan Li, Joo-Hwee Lim, Giduthuri Sateesh Babu, Phyo Phyo San, and Ngai-Man Cheung. Multimodal multi-stream deep learning for egocentric activity recognition. InCVPRW,

  31. [39]

    Stevenson, Magdalena M

    Ryan A. Stevenson, Magdalena M. Wilson, Albert R. Pow- ers, and Mark T. Wallace. The effects of visual training on multisensory temporal processing. Experimental Brain Re- search, 225(4):479–489, 2013. 2

    2013

  32. [40]

    Attention is all we need: Nailing down object-centric attention for egocen- tric activity recognition

    Swathikiran Sudhakaran and Oswald Lanz. Attention is all we need: Nailing down object-centric attention for egocen- tric activity recognition. In BMVC, 2018. 2

    2018

  33. [41]

    Wallace and Ryan A

    Mark T. Wallace and Ryan A. Stevenson. The construct of the multisensory temporal binding window and its dysregula- tion in developmental disabilities.Neuropsychologia, 64:105 – 123, 2014. 2

    2014

  34. [42]

    Temporal segment networks: Towards good practices for deep action recogni- tion

    Limin Wang, Yuanjun Xiong, Zhe Wang, Yu Qiao, Dahua Lin, Xiaoou Tang, and Luc Van Gool. Temporal segment networks: Towards good practices for deep action recogni- tion. In ECCV, 2016. 1, 2, 3, 4, 5, 8

    2016

  35. [43]

    Multi-stream multi-class fusion of deep net- works for video classification

    Zuxuan Wu, Yu-Gang Jiang, Xi Wang, Hao Ye, and Xi- angyang Xue. Multi-stream multi-class fusion of deep net- works for video classification. In ACM International Con- ference on Multimedia, 2016. 2

    2016

  36. [44]

    Kitani, and Yoichi Sato

    Ryo Yonetani, Kris M. Kitani, and Yoichi Sato. Recognizing micro-actions and reactions from paired egocentric videos. In CVPR, 2016. 1

    2016

  37. [45]

    Temporal relational reasoning in videos

    Bolei Zhou, Alex Andonian, Aude Oliva, and Antonio Tor- ralba. Temporal relational reasoning in videos. In ECCV,

  38. [46]

    Temporal perception and prediction in ego-centric video

    Yipin Zhou and Tamara L Berg. Temporal perception and prediction in ego-centric video. In ICCV, 2015. 1 Appendices This additional material includes a description of the qualitative examples in the supplementary video in App. A. This is followed by the leaderboard results in A...

    2015

  39. [47]

    ‘open’, ‘fridge’), as well as when all modalities are under-performing (e.g

    Fusion can improve results when all modalities are individually performing well for both verb and noun classes (e.g. ‘open’, ‘fridge’), as well as when all modalities are under-performing (e.g. ‘scoop’, ‘salad’)

  40. [48]

    ‘divide’, ‘fish’)

    Fusion can though be difficult at times, particularly when two of the three modalities are uninformative (e.g. ‘divide’, ‘fish’)

  41. [49]

    ‘switch’, ‘paper’)

    All nouns for which audio is outperforming other modalities have distinct sounds (e.g. ‘switch’, ‘paper’)

  42. [50]

    ‘chicken’, ‘salt’)

    Similarly, audio is least distinctive when the noun does not have a sound per se or its sound depends on the action (e.g. ‘chicken’, ‘salt’). D. Code and Models Python code of our TBN model, and pre-trained model on EPIC-Kitchens is available at http://github.com/ekazakos/temp...

Pith tools

Reviewed August 14, 2026 · model on record in the stance chip above.