REVIEW 18 cited by
Video Swin Transformer
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
Video Swin Transformer
read the original abstract
The vision community is witnessing a modeling shift from CNNs to Transformers, where pure Transformer architectures have attained top accuracy on the major video recognition benchmarks. These video models are all built on Transformer layers that globally connect patches across the spatial and temporal dimensions. In this paper, we instead advocate an inductive bias of locality in video Transformers, which leads to a better speed-accuracy trade-off compared to previous approaches which compute self-attention globally even with spatial-temporal factorization. The locality of the proposed video architecture is realized by adapting the Swin Transformer designed for the image domain, while continuing to leverage the power of pre-trained image models. Our approach achieves state-of-the-art accuracy on a broad range of video recognition benchmarks, including on action recognition (84.9 top-1 accuracy on Kinetics-400 and 86.1 top-1 accuracy on Kinetics-600 with ~20x less pre-training data and ~3x smaller model size) and temporal modeling (69.6 top-1 accuracy on Something-Something v2). The code and models will be made publicly available at https://github.com/SwinTransformer/Video-Swin-Transformer.
Forward citations
Cited by 18 Pith papers
-
RobustSora: De-Watermarked Benchmark for Robust AI-Generated Video Detection
RobustSora benchmark demonstrates that current AI video detectors rely heavily on visible watermarks, with average accuracy drops of 6.6 percentage points when watermarks are erased and increased false alarms when wat...
-
From Articulated Kinematics to Routed Visual Control for Action-Conditioned Surgical Video Generation
A kinematic-to-visual lifting paradigm combined with hierarchically routed control generates action-conditioned surgical videos with better faithfulness, fidelity, and efficiency.
-
SignMAE: Segmentation-Driven Self-Supervised Learning for Sign Language Recognition
SignMAE uses segmentation-driven masking in a mask-and-reconstruct self-supervised task to learn fine-grained sign representations, achieving state-of-the-art accuracy on WLASL, NMFs-CSL, and Slovo with fewer frames a...
-
VTI-CoT: Visual-Textual Interleaved Chain of Thought for Video Reasoning
VTI-CoT proposes a visual-textual interleaved chain-of-thought method for video reasoning, built via automated annotation and OCR compression, claiming SOTA performance and better training efficiency on same-scale models.
-
Tensor Memory: Fixed-Size Recurrent State for Long-Horizon Transformers
Tensor Memory augments Transformers with a constant-size 3D voxel grid using differentiable soft writes at predicted locations, local interaction, and gated recurrent dynamics to decouple memory capacity from sequence length.
-
CAM-VFD: Cross-Attention Multimodal Video Forgery Detection
CAM-VFD detects video forgeries by using cross-attention to identify contradictions between CLIP appearance, VideoMAE motion, and MiDaS depth features.
-
MegaScale-Omni: A Hyper-Scale, Workload-Resilient System for MultiModal LLM Training in Production
MegaScale-Omni delivers 1.27x-7.57x higher throughput for dynamic multimodal LLM training by decoupling encoder and LLM parallelism, using unified colocation, and applying adaptive workload balancing.
-
DVAR: Adversarial Multi-Agent Debate for Video Authenticity Detection
DVAR turns video authenticity detection into an iterative debate between a generative hypothesis agent and a natural mechanism agent, resolved via minimum description length and a knowledge base for better generalizat...
-
Florence: A New Foundation Model for Computer Vision
Florence is a new vision foundation model that learns universal visual-language representations from web-scale data and reports state-of-the-art results on 44 benchmarks including 83.74% zero-shot ImageNet top-1 accuracy.
-
GMoT: Gated Motion-Aware Tokenization for Fine-Grained Micro-Gesture Video Reasoning with Multimodal LLMs
GMoT's gated motion tokens improve multimodal LLM micro-gesture recognition on iMiGUE and SMG, with limited support for reasoning-grounding claims.
-
Spatio-Temporal Fusion Model for Standard View Classification of Echocardiographic Videos
The authors release the EV9V dataset of 5138 videos and introduce an uncertainty-aware CNN-LSTM model that achieves competitive performance on standard echocardiographic view classification.
-
ConvFormer3D-TAP: Phase/Uncertainty-Aware Front-End Fusion for Cine CMR View Classification Pipelines
ConvFormer3D-TAP classifies six cine CMR views at 96% accuracy using 3D conv tokenization, multiscale attention, and uncertainty-aware multi-clip fusion on 150k sequences.
-
Spatio-Temporal Wildfire Spread Prediction in Canada using a Video Swin-Hybrid-U-Net and Satellite Imagery
Hybrid Video Swin-U-Net forecasts next-day fire incidence maps from spatio-temporal satellite and meteorological sequences for major Canadian wildfires 2014-2023.
-
MLT-Dedup: Efficient Large-Scale Online Video Deduplication via Multi-Level Representations and Spatial-Temporal Matching
MLT-Dedup achieves 91% reduction in online video repetition rates at 90% precision and 5x indexing capacity using multi-level representations and differential feature-enhanced similarity on a real-world platform.
-
Multimodal Anomaly Detection for Human-Robot Interaction
MADRI detects anomalies in human-robot pick-and-place tasks by reconstructing multimodal feature vectors from video, internal sensors, and scene graphs, with multimodal versions outperforming vision-only on a custom dataset.
-
Structured Spectral Graph Representation Learning for Multi-label Abnormality Analysis from 3D CT Scans
A graph-of-slice-triplets encoder with spectral convolution outperforms 3D CNN/Transformer baselines on multi-label chest CT abnormality classification and transfers to report generation and abdominal CT.
-
Every Subtlety Counts: Fine-grained Person Independence Micro-Action Recognition via Distributionally Robust Optimization
A Person Independence Universal Micro-action Recognition Framework combines Distributionally Robust Optimization with temporal-frequency alignment at the feature level and group-invariant regularization at the loss le...
-
Video-Language Understanding: A Survey from Model Architecture, Model Training, and Data Perspectives
Survey summarizing video-language understanding tasks, challenges, and methods from architecture, training, and data perspectives, including performance comparisons and future directions.
discussion (0)
Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.