Pith. sign in

REVIEW 4 major objections 6 minor 7 cited by

Scaling 4D Representations

T0 review · 4 major / 6 minor · reviewed 2026-08-11 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read The paper claims that purely self-supervised masked auto-encoding from video scales: larger video transformers, up to 22B parameters, consistently improve four geometry-and-motion tasks—camera pose, point tracking, object tracking, and…

desk verdict The 20M-to-4B scaling story is solid and useful, but the 'all the way to 22B' claim is undercut by a confounded recipe change, so the headline conclusion should be read with more caution than the paper admits. read the letter →

arxiv 2412.15212 v2 pith:CE3YOYL2 submitted 2024-12-19 cs.CV cs.AIcs.LG

João Carreira , Dilara Gokay , Michael King , Chuhan Zhang , Ignacio Rocco , Aravindh Mahendran , Thomas Albert Keck , Joseph Heyward
show 27 more authors
Skanda Koppula Etienne Pot Goker Erdogan Yana Hasson Yi Yang Klaus Greff Guillaume Le Moing Sjoerd van Steenkiste Daniel Zoran Drew A. Hudson Pedro Vélez Luisa Polanía Luke Friedman Chris Duvarney Ross Goroshin Kelsey Allen Jacob Walker Rishabh Kabra Eric Aboussouan Jennifer Sun Thomas Kipf Carl Doersch Viorica Pătrăucean Dima Damen Pauline Luc Mehdi S. M. Sajjadi Andrew Zisserman
This is my paper · ORCID
classification cs.CVcs.AIcs.LG
keywords self-supervisedvideorepresentationlearningmaskedauto-encodingmodelscaling4Dsceneunderstandingtransformerscameraposeestimationpointtrackingdepth
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Self-supervised video pretraining has been seen as saturating around 1B parameters, but that judgement comes mostly from semantic benchmarks such as action classification. This paper re-runs the comparison on non-semantic 4D tasks—camera pose estimation, point and object tracking, and monocular depth, which combine three spatial dimensions with time—and claims that a deliberately simplified masked autoencoder (SimpleMAE) improves consistently as the video transformer grows from 20M to 22B parameters, the largest self-supervised video model reported to date. Under a shared frozen-feature protocol with identical attention readouts, the 22B model outperforms every baseline, including language-supervised video models and the previous ~1B self-supervised video models, on all four 4D tasks. The same trend holds for short, medium, and long finetuning, and a 90M student distilled from the 4B teacher beats same-size models. If the scaling claim is right, it shifts attention from language-supervised, classification-centric pretraining toward pure video self-supervision and geometric-temporal evaluation.

What carries the argument

The load-bearing object is SimpleMAE, a deliberately bare-bones video masked autoencoder: it masks 95% of 2x16x16 space-time patches at random, feeds the rest through a Vision Transformer, appends a learned grid of latent tokens in the last few self-attention blocks, and decodes each latent patch with a single linear layer—no target normalization, no tube masking, no separate decoder. For evaluation, features from the encoder (read at 95% depth for the 4D tasks) are fed to task-specific cross-attention readouts, which are trained on top of either frozen or finetuned backbones. This shared protocol is what lets the paper attribute differences in performance to the pretrained representation rather than to the readout design.

What would settle it

Train or re-evaluate the 22B model under the same recipe as the smaller models—224x224 input, four latent decoding layers, 2x16x16 reconstruction patches, and 488k training updates—and compare frozen-feature results on RE10k, Perception Test, ScanNet, and Waymo against 4DS-e. If the 22B model does not beat the 4B model, or if a static-image model matches it on ScanNet depth (where monocular cues dominate short clips), the scaling conclusion is not established.

Watch

Extended reading notes

Core claim

On the paper's own terms, the discovery is that masked auto-encoding with video transformers scales when the targets are 4D: from a 20M-parameter model to a 22B-parameter model (4DS-j), frozen-feature performance improves monotonically on RealEstate10k camera pose (mean endpoint error 0.73 to 0.24), Perception Test point tracking (average Jaccard 75.9 to 83.4), ScanNet depth (absolute relative error 2.05 to 0.84), and Waymo Open object tracking (IoU 69.6 to 78.3). The same upward trend holds with finetuning, and the largest model is best on all four tasks. The paper argues this contradicts the common belief that MAE has mediocre scaling properties, and that prior work missed the trend because it evaluated on semantic tasks such as Something-Something v2 and Kinetics, where gains saturate earlier and language supervision helps more.

Load-bearing premise

The claim stands on the premise that the four tasks, evaluated with one shared frozen-feature protocol, genuinely measure spatial-temporal (4D) representation quality, and that the improvements are caused by model scale rather than by protocol choices or appearance-based shortcuts such as monocular depth cues.

Editorial extensions

If this is right

  • The same frozen-feature protocol, applied to any new video backbone, now has a reference scaling curve on four 4D tasks instead of only classification benchmarks.
  • If the trend continues beyond 22B, larger self-supervised video models should keep improving camera pose, tracking, and depth without any language supervision.
  • Language-supervised pretraining, while strong on classification, is not necessary—and may be harmful—for geometric-temporal representation quality.
  • Distillation from the large 4DS models offers a practical route to small, strong 4D backbones, as shown by the 90M distilled model beating its same-size counterpart.
  • Semantic and 4D tasks rank models differently, so conclusions about scaling from action-recognition benchmarks should not be transferred to geometry-and-motion tasks.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • One testable extension is to fit the seven 4DS points with a power law; the paper does not claim a scaling law, but its data suggest a smooth, still-improving curve that would make such a law testable with one or two larger runs.
  • The 95%-depth readout finding suggests that precise geometric-temporal information is concentrated in late layers; probing intermediate layers on a fixed model could reveal where 4D structure emerges during pretraining.
  • Because the 22B checkpoint uses a different input resolution, decoding grid, and update count, a controlled rerun matching the smaller models' recipe would separate scale from recipe; the paper does not provide this control.
  • If the scaling transfers, tasks like optical flow, video object segmentation, and physical prediction—all 4D in nature—are natural next beneficiaries, even though the paper does not evaluate them.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 6 minor

Summary. The paper argues that pure self-supervised masked auto-encoding on video does scale, contrary to the common belief that MAE saturates, provided evaluation targets non-semantic 4D tasks (camera pose, point tracking, object tracking, depth) rather than classification. It introduces SimpleMAE, a stripped-down video MAE without target normalization, tube masking, or a separate decoder, and trains a family of 7 ViT models (4DS) from 20M to 22B parameters on 170M web videos. The authors evaluate frozen and finetuned attention-based readouts on RE10k pose, Perception Test point tracking, Waymo object tracking, ScanNet depth, and SSv2/Kinetics classification, comparing against DINOv2, SigLIP, VideoPrism, V-JEPA, and VideoMAE baselines. The empirical core is a large set of internally consistent tables showing that 4DS models improve with size on the 4D tasks, with the 22B model best, plus supporting studies: data scaling, readout-layer sweep, decoding hyperparameter ablation, and distillation from 4B to 90M.

Significance. If the central claim is accepted, the paper is a substantial empirical contribution: it provides the first demonstration that video-only self-supervised models can be scaled beyond 1B parameters with consistent downstream gains on geometric and temporal tasks, it offers a large family of released checkpoints, and it challenges the semantic-centric view of scaling by showing that classification benchmarks may have obscured MAE's scaling behavior. The evaluation protocol is unusually careful: all models share identical readout architectures, training example budgets, and optimization schedules, and the paper includes controlled ablation studies (data scaling, layer depth, decoding choices, distillation). The main fragility is the 22B data point, which changes several training variables at once, together with the absence of uncertainty estimates on the headline monotonic trends. The paper explicitly acknowledges that it does not produce proper scaling laws, which is an honest limitation rather than a defect.

major comments (4)
  1. [Supp. Tab. 5, Supp. Tab. 8; Sec. 3.2, 4.1] The 4B-to-22B interval is not a controlled scale comparison. Supp. Tab. 5 changes five protocol variables simultaneously for the 22B model (input resolution 224 to 256, decoding layers 4 to 2, output patch size 2x16x16 to 4x32x32, training updates 488k to 1M, pretraining clips 1B to 2B, plus a different learning rate). Supp. Tab. 8, which the paper itself reports, shows that the 22B-style decoding configuration is worse than the default for a 4B model on RE10k (0.32 vs 0.29), ScanNet (1.15 vs 0.95), and SSv2 (60.6 vs 65.8). Because these changes pull in opposite directions, the observed 22B improvements in Tables 2, 3, and 9 cannot be cleanly attributed to parameter count; this is precisely the interval that the abstract's 'all the way to 22B' assertion relies on. Please add a controlled 4B run using the 22B recipe, or explicitly restate the headline as a well-controlled 20M-to-4B trend plus a separately-confounded 22B data point.
  2. [Tables 2, 3, 9; Fig. 7] The paper claims 'consistent' improvement without reporting any uncertainty. In the frozen evaluation, 4DS-G (1.8B) and 4DS-e (3.8B) tie on RE10k (0.27) and 4DS-G is better than 4DS-e on Perception Test (82.7 vs 82.4); in short finetuning RE10k is 0.29 for 4DS-G versus 0.28 for 4DS-e. The top-end increments are often tiny, for example frozen Waymo 78.0 (4DS-e) to 78.3 (4DS-j). Without seeds or confidence intervals, the strict monotonic reading of the results is not supported. Please report multiple seeds with confidence intervals, or soften the monotonicity claim to a general trend with stated noise.
  3. [Sec. 4.1, Table 2, Sec. A.2.6] The ScanNet depth task may not measure 4D geometric-temporal representation as cleanly as the other three tasks. The paper itself notes that image models are strong on ScanNet 'likely because monocular cues dominate in short clips', and indeed DinoV2-g, a pure image model, achieves 0.91 AbsRel frozen versus 4DS-e's 0.95, and remains competitive under long finetuning (0.66 vs 0.65). The scaling improvement on ScanNet may therefore partly reflect monocular appearance or shape cues rather than temporal 3D understanding. Please quantify the temporal contribution (for example, a frame-order-perturbation test) or explicitly relegate depth to a secondary role when claiming gains in 4D representation quality.
  4. [Sec. 4.2, Fig. 6] The readout layer at 95% depth (75% for semantic tasks) was selected after inspecting a sweep on the same downstream tasks. Because the same choice is applied to all models, the across-size comparison is probably not severely biased, but the reported absolute numbers may be optimistic. This is a moderate concern rather than a fatal one; reporting results for a second readout layer (e.g., the average over layers) would strengthen the robustness of the scaling conclusion.
minor comments (6)
  1. [Sec. 1, Contributions] The third contribution is written as a fragment: 'which produced c) three new MAE-VIT models...' Please rewrite for grammatical completeness.
  2. [Fig. 4 (caption)] Fig. 4b and 4c include the 22B model, but the caption of Fig. 4a only states that the 22B model is omitted from the reconstruction panel; please clarify that the omission applies only to the MAE reconstruction subfigure.
  3. [Supp. Sec. B] There is an empty placeholder in the point tracking qualitative description: 'provided in .' The figure reference is missing.
  4. [Supp. Sec. A.2.6] The sentence 'Neither evaluates the method on ScanNet videos and they all evaluate zero-shot' is internally contradictory given the immediately following sentence that DUSt3R zero-shot 'gets 0.088' on ScanNet; please rephrase.
  5. [Table 6] For the Perception Test readout, the table header says 'CrossAttention(qkv size=1024, num heads=8)' but the text in Sec. A.2.4 says 'cross-attention with 1024 parameters'; please use consistent terminology (channels vs parameters).
  6. [Supp. Tab. 5] The table does not list the learning rate chosen for the 22B model despite the text mentioning different learning rates; adding this row would make the protocol fully reproducible.

Circularity Check

0 steps flagged · score 1.0 of 10

No significant circularity: the scaling evidence is measured on external 4D benchmarks, not derived from the training objective.

full rationale

The paper's central claim—that masked auto-encoding with video transformers scales on 4D tasks—rests on evaluations using external benchmarks (RealEstate10k with SfM ground truth, Perception Test with human point-track annotations, ScanNet with RGB-D depth, Waymo Open with human boxes, and SSv2/Kinetics action labels). These targets are not computed from the SimpleMAE pretraining loss, so the scaling trend is not self-defined. The RE10k pose task is newly introduced but is anchored to external structure-from-motion poses and is compared against a zero-shot DUSt3R baseline, so it does not reduce to a fitted quantity. The selection of the 95%-depth readout layer after inspecting a sweep (Fig. 6) is a hyperparameter choice on validation, not a fitted prediction of the reported scaling curve; the same protocol is applied across model sizes, and the trends are consistent under frozen and finetuned evaluation. Self-citations to Perception Test, TAP-Vid, TAPIR, MooG, and BootsTAP are references to published benchmarks and evaluation protocols, not load-bearing premises that make the scaling result true by construction. The main internal-validity caveat—the 22B model changes resolution, decoding layers, updates, and data volume simultaneously (Table 5), and the paper's own decoding ablation (Table 8) suggests the 22B-style decoder is worse on some tasks—is a confound in attributing the 4B-to-22B gain purely to parameter count. This is a correctness or experimental-design concern, not a circularity: the reported numbers are still measured against external ground truth rather than being equivalent to the training inputs. No equation, fitted parameter, or cited theorem makes the scaling conclusion equivalent to its own premises.

Assumptions & free parameters 6 free parameters · 6 assumptions · 0 invented entities

This is an empirical scaling paper, so the ledger contains no fitted physical constants and no invented entities. The central claim rests on hand-chosen protocol values (masking ratio, decoding configuration, readout layer depth) and on domain assumptions about the video corpus and the downstream tasks. The most consequential hand-chosen values are the decoding hyperparameters, which are ablated and show the 22B model uses the worse setting for memory reasons, and the 95% readout layer, which was selected after inspecting the reported tasks. The paper's own limitation section acknowledges that no scaling laws were produced.

free parameters (6)
  • MAE masking ratio = 0.95
    Hand-set and standard for MAE-family image and video models. Defines the difficulty of the reconstruction pretext task and is not tuned in this paper.
  • Number of latent decoding layers = 4 (models up to 4B), 2 (22B)
    Chosen by memory constraints for 22B. Supplementary Table 8 ablates this and shows 4 layers help (RE10k EPE 0.29 vs 0.33, SSv2 65.8 vs 61.9), so the 22B model runs a suboptimal setting that could understate its performance.
  • Decoding output patch size = (2,16,16) up to 4B, (4,32,32) for 22B
    Ablated in supplementary Table 8; the finer patch setting is better on all tested tasks, and the 22B model uses the coarser setting due to memory.
  • Readout feature layer depth = 95% of blocks for 4D tasks, 75% for semantic tasks
    Selected after inspecting a sweep of six candidate layers per model (Fig. 6) on the same tasks whose results are reported. This post hoc choice affects every number in Tables 2 and 3.
  • Learning rate and warmup schedule = 1e-4 with 10k warmup (up to 4B), 2e-4 with 100k warmup (22B)
    Chosen by short runs or necessity; the 22B values differ from the smaller models, making that data point configuration-mismatched.
  • Training updates and batch size = 488,282 steps at batch 2048 (up to 4B); 1,000,000 steps at batch 2048 or 8192 (22B)
    Larger models receive more total compute. Footnote 2 reports the faster 22B run (batch 8192) gave worse results, so these choices partly determine the reported trend and the paper does not separate compute from parameter count.
assumptions (6)
  • domain assumption The 170M-video web corpus supplies sufficient and representative 4D signal for pretraining.
    Invoked in Section 3.2. If the corpus is unrepresentative, the measured scaling will not transfer to other video distributions.
  • domain assumption The four downstream tasks (RE10k pose, Perception Test points, Waymo boxes, ScanNet depth) require genuine spatiotemporal geometric reasoning.
    Section 4.1 and Fig. 3. The paper itself notes ScanNet depth may be dominated by monocular cues in short clips, so a portion of the trend could reflect appearance rather than 4D understanding.
  • domain assumption Frozen-feature evaluation with a shared attention readout is a fair, comparable measure of representation quality across heterogeneous backbones.
    Section 3.3. Baselines are evaluated with limited training and single-clip inference, producing lower numbers than published; the fairness of this protocol for image vs video and text-supervised vs pure models is assumed.
  • domain assumption The standard ViT sizes S through e plus the new j configuration are comparable instantiations of one architecture family.
    Table 1. Width, depth, and heads follow established ViT scaling; the j configuration (4096 width, 64 depth) is new and could behave differently at this scale.
  • standard math Procrustes projection onto SO(3) plus the virtual-cube EPE metric is an adequate measure of relative pose accuracy.
    Supplementary A.2.3. Standard technique after Bregier 2021; the choice of 8 auxiliary cube points shapes all RE10k numbers.
  • ad hoc to paper A single readout layer at 95% depth is a reasonable compromise for all 4D tasks.
    Fig. 6 shows 95% is a compromise across tasks, but the choice is made from the same validation results being reported, introducing mild selection into the evaluation protocol.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Scaling 4D Representations." pith.science (2026). https://pith.science/paper/CE3YOYL2

@misc{pith2026241215212,
  author       = {Pith},
  title        = {Pith review of: Scaling 4D Representations},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/CE3YOYL2}},
  note         = {Machine review of arXiv:2412.15212}
}
abstract

Scaling has not yet been convincingly demonstrated for pure self-supervised learning from video. However, prior work has focused evaluations on semantic-related tasks $\unicode{x2013}$ action classification, ImageNet classification, etc. In this paper we focus on evaluating self-supervised learning on non-semantic vision tasks that are more spatial (3D) and temporal (+1D = 4D), such as camera pose estimation, point and object tracking, and depth estimation. We show that by learning from very large video datasets, masked auto-encoding (MAE) with transformer video models actually scales, consistently improving performance on these 4D tasks, as model size increases from 20M all the way to the largest by far reported self-supervised video model $\unicode{x2013}$ 22B parameters. Rigorous apples-to-apples comparison with many recent image and video models demonstrates the benefits of scaling 4D representations. Pretrained models are available at https://github.com/google-deepmind/representations4d .

Figures

Figures reproduced from arXiv: 2412.15212 by the authors.

Figure 1
Figure 1. Performance keeps improving on all downstream video [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 2
Figure 2. SimpleMAE framework. Videos are encoded into space-time tokens via one non-overlapping convolution typical of ViTs, then 95% are randomly dropped. A transformer model then processes the remaining tokens with self-attention layers; first alone, then jointly with a learned grid of latent tokens on the last few self-attention layers (last two for the 22B model). Each of these learned tokens is decoded back to a patch u… view at source ↗
Figure 3
Figure 3. Evaluation downstream tasks. Individual frames and annotations from the five tasks considered in this paper. The majority of previous work has focused on classification, whereas in this paper we do a deep dive into the other four tasks that are less semantic and more about spatial and temporal (4D) scene perception. cropping down to 224x224 (models up to 4B parameters) or 256x256 resolution (22B parameter models) af… view at source ↗
Figures from the paper (11 more)
Figure 4
Figure 4. Figure 4: Qualitative results of 4DS models (20M-22B). We first illustrate MAE training: masked and reconstructed frames on Epic [PITH_FULL_IMAGE:figures/full_fig_p007_4.png]
Figure 5
Figure 5. Figure 5: Pose estimation qualitative results on RealEstate10k. [PITH_FULL_IMAGE:figures/full_fig_p007_5.png]
Figure 6
Figure 6. Figure 6: The performance of the frozen 4DS-j model when attaching attention-based readouts at different layers of the model (as a [PITH_FULL_IMAGE:figures/full_fig_p008_6.png]
Figure 7
Figure 7. Figure 7: Model performance with increasing model sizes ranging from 20M parameters to 22B parameters, on all five tasks, with frozen [PITH_FULL_IMAGE:figures/full_fig_p009_7.png]
Figure 8
Figure 8. Figure 8: Top-1 accuracy of the 20M parameter 4DS-S (blue) vs the 4B parameter 4DS-e model (orange) on the SSv2 (above) and [PITH_FULL_IMAGE:figures/full_fig_p019_8.png]
Figure 9
Figure 9. Figure 9: Comparison of depth map predictions from MAE models with varying parameter sizes (20M to 22B) on the ScanNet dataset. [PITH_FULL_IMAGE:figures/full_fig_p020_9.png]
Figure 10
Figure 10. Figure 10: Visualization of predicted and ground truth bounding boxes for object tracking on the Waymo Open Dataset. Results are shown [PITH_FULL_IMAGE:figures/full_fig_p021_10.png]
Figure 11
Figure 11. Figure 11: Point tracking error visualization for 20M (left) and 22B parameter (right) 4DS models on the Perception Test dataset. We show [PITH_FULL_IMAGE:figures/full_fig_p022_11.png]
Figure 12
Figure 12. Figure 12: Reconstructed frames on the Epic Kitchens [ [PITH_FULL_IMAGE:figures/full_fig_p022_12.png]
Figure 13
Figure 13. Figure 13: Radar plot comparing the frozen evaluation performance on all tasks of the image model DinoV2-g, video model VideoPrism-g, [PITH_FULL_IMAGE:figures/full_fig_p023_13.png]
Figure 14
Figure 14. Figure 14: Pearson’s correlation coefficients for all frozen attention-based evaluation tasks across all models. This visualization is based on [PITH_FULL_IMAGE:figures/full_fig_p023_14.png]

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 7 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. Unique Lives, Shared World: Learning from Single-Life Videos

    cs.CV 2025-12 conditional novelty 7.0 of 10

    Vision models trained independently on single egocentric lives converge to aligned geometric representations, and about 30 hours of one life matches 30 hours of diverse video for depth-estimation pretraining.

  2. Self-Supervised Learning of Structured Dynamics from Videos

    cs.CV 2026-07 conditional novelty 6.0 of 10

    A two-token 'primary/residual' future-feature predictor separates camera from object motion, outpacing frozen-feature baselines and matching larger supervised models on several probes.

  3. SeeSE3: Emergence of 3D Space in Vision Features

    cs.CV 2026-07 conditional novelty 6.0 of 10

    Self-supervised vision features, especially DINOv2, contain a subspace that a small trained adapter can map to 3D camera motion, enabling pose estimation and latent-space navigation without explicit 3D reconstruction.

  4. Gen4U: Unifying Video Generation and Understanding via Diffusion

    cs.CV 2026-07 conditional novelty 6.0 of 10

    Frozen video diffusion models, probed at optimal depth and noise levels, produce representations competitive with discriminative encoders across semantic and geometric video tasks in a single forward pass.

  5. SciVid: Cross-Domain Evaluation of Video Models in Scientific Applications

    cs.CV 2025-07 conditional novelty 6.0 of 10

    General-purpose video foundation models, adapted with lightweight readout heads, reach state-of-the-art performance on three of five scientific video benchmarks.

  6. MoSiC: Optimal-Transport Motion Trajectory for Dense Self-Supervised Learning

    cs.CV 2025-06 conditional novelty 6.0 of 10

    MoSiC clusters dense point tracks in videos and propagates the cluster assignments along the tracks, improving DINOv2's dense representations by 1 to 6 percent on segmentation and in-context benchmarks.

  7. From Image to Video: An Empirical Study of Diffusion Representations

    cs.CV 2025-02 conditional novelty 6.0 of 10

    Video-pretrained diffusion features beat matched image-pretrained features on most perception tasks, with the largest gains on motion and geometry tasks, but still trail contrastive models on semantics.

Reference graph

Works this paper leans on

83 extracted references · 71 canonical work pages · cited by 7 Pith papers

  1. [1]

    Learning to see by moving

    Pulkit Agrawal, Joao Carreira, and Jitendra Malik. Learning to see by moving. In ICCV, 2015. 2

    2015

  2. [2]

    Flamingo: a visual language model for few-shot learning.NeurIPS, 2022

    Jean-Baptiste Alayrac, Jeff Donahue, Pauline Luc, Antoine Miech, Iain Barr, Yana Hasson, Karel Lenc, Arthur Mensch, Katherine Millican, Malcolm Reynolds, et al. Flamingo: a visual language model for few-shot learning.NeurIPS, 2022. 2

    2022

  3. [3]

    Self-supervised learning by cross-modal audio-video clustering

    Humam Alwassel, Dhruv Mahajan, Lorenzo Torresani, Bernard Ghanem, and Du Tran. Self-supervised learning by cross-modal audio-video clustering. In NeurIPS, 2020. 2

    2020

  4. [4]

    Look, listen and learn

    Relja Arandjelovi ´c and Andrew Zisserman. Look, listen and learn. In ICCV, 2017

    work page 2017

  5. [5]

    Objects that sound

    Relja Arandjelovi ´c and Andrew Zisserman. Objects that sound. In ECCV, 2018. 2

    work page 2018

  6. [6]

    Vivit: A video vi- sion transformer

    Anurag Arnab, Mostafa Dehghani, Georg Heigold, Chen Sun, Mario Lu ˇci´c, and Cordelia Schmid. Vivit: A video vi- sion transformer. In 2021 IEEE/CVF International Confer- ence on Computer Vision (ICCV) , pages 6816–6826, 2021. 5

    work page 2021

  7. [7]

    Yuille, Trevor Darrell, Jitendra Malik, and Alexei A

    Yutong Bai, Xinyang Geng, Karttikeya Mangalam, Amir Bar, Alan L. Yuille, Trevor Darrell, Jitendra Malik, and Alexei A. Efros. Sequential modeling enables scalable learn- ing for large vision models. In CVPR, 2024. 2

    work page 2024

  8. [8]

    Revisiting feature prediction for learning visual rep- resentations from video

    Adrien Bardes, Quentin Garrido, Jean Ponce, Xinlei Chen, Michael Rabbat, Yann LeCun, Mido Assran, and Nicolas Ballas. Revisiting feature prediction for learning visual rep- resentations from video. Transactions on Machine Learning Research, 2024. Featured Certification. 1, 2, 4, 5, 6, 12

    work page 2024

Show all 83 references
  1. [9]

    Physion: Evaluating physical prediction from vision in humans and machines

    Daniel Bear, Elias Wang, Damian Mrowca, Felix Binder, Hsiao-Yu Tung, Pramod RT, Cameron Holdaway, Sirui Tao, Kevin Smith, Fan-Yun Sun, Fei-Fei Li, Nancy Kanwisher, Josh Tenenbaum, Dan Yamins, and Judith Fan. Physion: Evaluating physical prediction from vision in humans and mac...

    2021

  2. [10]

    Zoedepth: Zero-shot transfer by com- bining relative and metric depth

    Shariq Farooq Bhat, Reiner Birkl, Diana Wofk, Peter Wonka, and Matthias M ¨uller. Zoedepth: Zero-shot transfer by com- bining relative and metric depth. arXiv preprint, 2023. 5

    2023

  3. [11]

    Deep regression on manifolds: a 3d rotation case study

    Romain Br ´egier. Deep regression on manifolds: a 3d rotation case study. In 3DV, 2021. 4, 2

    2021

  4. [12]

    Activitynet: A large-scale video benchmark for human activity understanding

    Fabian Caba Heilbron, Victor Escorcia, Bernard Ghanem, and Juan Carlos Niebles. Activitynet: A large-scale video benchmark for human activity understanding. In CVPR,

  5. [13]

    Generative pre- training from pixels

    Mark Chen, Alec Radford, Rewon Child, Jeffrey Wu, Hee- woo Jun, David Luan, and Ilya Sutskever. Generative pre- training from pixels. In ICML, 2020. 2

    2020

  6. [14]

    A simple framework for contrastive learning of visual representations

    Ting Chen, Simon Kornblith, Mohammad Norouzi, and Ge- offrey Hinton. A simple framework for contrastive learning of visual representations. In ICML, 2020. 5

    2020

  7. [15]

    PaLI-3 vision language models: Smaller, faster, stronger

    Xi Chen, Xiao Wang, Lucas Beyer, Alexander Kolesnikov, Jialin Wu, Paul V oigtlaender, Basil Mustafa, Sebastian Goodman, Ibrahim Alabdulmohsin, Piotr Padlewski, et al. PaLI-3 vision language models: Smaller, faster, stronger. arXiv preprint, 2023. 1, 5, 6, 12

    2023

  8. [16]

    A unified architecture for natural language processing: Deep neural networks with multitask learning

    Ronan Collobert and Jason Weston. A unified architecture for natural language processing: Deep neural networks with multitask learning. In ICML, 2008. 2

    2008

  9. [17]

    Scannet: Richly-annotated 3d reconstructions of indoor scenes

    Angela Dai, Angel X Chang, Manolis Savva, Maciej Hal- ber, Thomas Funkhouser, and Matthias Nießner. Scannet: Richly-annotated 3d reconstructions of indoor scenes. In CVPR, 2017. 2, 4, 5 9

    2017

  10. [18]

    Scaling egocentric vision: The epic-kitchens dataset

    Dima Damen, Hazel Doughty, Giovanni Maria Farinella, Sanja Fidler, Antonino Furnari, Evangelos Kazakos, Davide Moltisanti, Jonathan Munro, Toby Perrett, Will Price, and Michael Wray. Scaling egocentric vision: The epic-kitchens dataset. In ECCV, 2018. 7, 10

    2018

  11. [19]

    Scaling vision transformers to 22 billion pa- rameters

    Mostafa Dehghani, Josip Djolonga, Basil Mustafa, Piotr Padlewski, Jonathan Heek, Justin Gilmer, Andreas Peter Steiner, Mathilde Caron, Robert Geirhos, Ibrahim Alabdul- mohsin, et al. Scaling vision transformers to 22 billion pa- rameters. In ICML, 2023. 2, 3

    2023

  12. [20]

    Unsuper- vised visual representation learning by context prediction

    Carl Doersch, Abhinav Gupta, and Alexei A Efros. Unsuper- vised visual representation learning by context prediction. In ICCV, 2015. 2

    2015

  13. [21]

    TAP-vid: A bench- mark for tracking any point in a video

    Carl Doersch, Ankush Gupta, Larisa Markeeva, Adria Re- casens Continente, Lucas Smaira, Yusuf Aytar, Joao Car- reira, Andrew Zisserman, and Yi Yang. TAP-vid: A bench- mark for tracking any point in a video. In NeurIPS, 2022. 3, 4

    2022

  14. [22]

    TAPIR: Tracking any point with per-frame initialization and temporal refinement

    Carl Doersch, Yi Yang, Mel Vecerik, Dilara Gokay, Ankush Gupta, Yusuf Aytar, Joao Carreira, and Andrew Zisserman. TAPIR: Tracking any point with per-frame initialization and temporal refinement. In ICCV, 2023. 4

    2023

  15. [23]

    Boot- sTAP: Bootstrapped training for tracking any point

    Carl Doersch, Pauline Luc, Yi Yang, Dilara Gokay, Skanda Koppula, Ankush Gupta, Joseph Heyward, Ignacio Rocco, Ross Goroshin, Jo˜ao Carreira, and Andrew Zisserman. Boot- sTAP: Bootstrapped training for tracking any point. arXiv preprint, 2024. 4

    2024

  16. [24]

    An image is worth 16x16 words: Transformers for image recognition at scale

    Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xiaohua Zhai, Thomas Unterthiner, Mostafa Dehghani, Matthias Minderer, Georg Heigold, Syl- vain Gelly, Jakob Uszkoreit, and Neil Houlsby. An image is worth 16x16 words: Transformers for image recognition ...

    2021

  17. [25]

    Prob- ing the 3d awareness of visual foundation models

    Mohamed El Banani, Amit Raj, Kevis-Kokitsi Maninis, Ab- hishek Kar, Yuanzhen Li, Michael Rubinstein, Deqing Sun, Leonidas Guibas, Justin Johnson, and Varun Jampani. Prob- ing the 3d awareness of visual foundation models. In CVPR,

  18. [26]

    Scalable pre- training of large autoregressive image models

    Alaaeldin El-Nouby, Michal Klein, Shuangfei Zhai, Miguel Angel Bautista, Alexander Toshev, Vaishaal Shankar, Joshua M Susskind, and Armand Joulin. Scalable pre- training of large autoregressive image models. ICML, 2024. 2

    2024

  19. [27]

    SA Vi++: Towards end-to-end object-centric learning from real-world videos

    Gamaleldin Fathy Elsayed, Aravindh Mahendran, Sjoerd van Steenkiste, Klaus Greff, Michael Curtis Mozer, and Thomas Kipf. SA Vi++: Towards end-to-end object-centric learning from real-world videos. In NeurIPS, 2022. 4, 5

    2022

  20. [28]

    Spatiotemporal residual networks for video action recogni- tion

    Christoph Feichtenhofer, Axel Pinz, and Richard Wildes. Spatiotemporal residual networks for video action recogni- tion. In NeurIPS, 2016. 2

    2016

  21. [29]

    Slowfast networks for video recognition

    Christoph Feichtenhofer, Haoqi Fan, Jitendra Malik, and Kaiming He. Slowfast networks for video recognition. In ICCV, 2019. 2

    2019

  22. [30]

    A large-scale study on unsupervised spatiotemporal representation learning

    Christoph Feichtenhofer, Haoqi Fan, Bo Xiong, Ross Gir- shick, and Kaiming He. A large-scale study on unsupervised spatiotemporal representation learning. In CVPR, 2021. 2

    2021

  23. [31]

    Distributed hier- archical processing in the primate cerebral cortex

    Daniel J Felleman and David C Van Essen. Distributed hier- archical processing in the primate cerebral cortex. Cerebral cortex (New York, NY: 1991), 1991. 2

    1991

  24. [32]

    Learning invariance from transformation se- quences

    Peter F ¨oldi´ak. Learning invariance from transformation se- quences. Neural computation, 1991. 2

    1991

  25. [33]

    M. A. Goodale and A. D. Milner. Separate visual pathways for perception and action. Trends in Neurosciences, 1992. 2

    1992

  26. [34]

    Learn- ing to linearize under uncertainty

    Ross Goroshin, Michael F Mathieu, and Yann LeCun. Learn- ing to linearize under uncertainty. NeurIPS, 2015. 2

    2015

  27. [35]

    The” something something” video database for learning and evaluating visual common sense

    Raghav Goyal, Samira Ebrahimi Kahou, Vincent Michal- ski, Joanna Materzynska, Susanne Westphal, Heuna Kim, Valentin Haenel, Ingo Fruend, Peter Yianilos, Moritz Mueller-Freitag, et al. The” something something” video database for learning and evaluating visual common sense. In ...

    2017

  28. [36]

    Klaus Greff, Francois Belletti, Lucas Beyer, Carl Doersch, Yilun Du, Daniel Duckworth, David J. Fleet, Dan Gnanapra- gasam, Florian Golemo, Charles Herrmann, Thomas Kipf, Abhijit Kundu, Dmitry Lagun, Issam Laradji, Hsueh- Ti (Derek) Liu, Henning Meyer, Yishu Miao, Derek Nowrou...

    2022

  29. [37]

    Memory- augmented dense predictive coding for video representation learning

    Tengda Han, Weidi Xie, and Andrew Zisserman. Memory- augmented dense predictive coding for video representation learning. In ECCV, 2020. 2

    2020

  30. [38]

    Self- supervised co-training for video representation learning

    Tengda Han, Weidi Xie, and Andrew Zisserman. Self- supervised co-training for video representation learning. NeurIPS, 2020. 2

    2020

  31. [39]

    Masked autoencoders are scalable vision learners

    Kaiming He, Xinlei Chen, Saining Xie, Yanghao Li, Piotr Doll´ar, and Ross Girshick. Masked autoencoders are scalable vision learners. In CVPR, 2022. 2, 3

    2022

  32. [40]

    Data-efficient image recognition with contrastive predictive coding

    Olivier J H ´enaff, Aravind Srinivas, Jeffrey De Fauw, Ali Razavi, Carl Doersch, SM Ali Eslami, and Aaron Van Den Oord. Data-efficient image recognition with contrastive predictive coding. In ICML, 2020. 2

    2020

  33. [41]

    Representation learn- ing with video deep infomax

    R Devon Hjelm and Philip Bachman. Representation learn- ing with video deep infomax. arXiv preprint, 2020. 2

    2020

  34. [42]

    The kinetics human action video dataset, 2017

    Will Kay, Joao Carreira, Karen Simonyan, Brian Zhang, Chloe Hillier, Sudheendra Vijayanarasimhan, Fabio Viola, Tim Green, Trevor Back, Paul Natsev, Mustafa Suleyman, and Andrew Zisserman. The kinetics human action video dataset, 2017. 1, 2, 4, 6

    2017

  35. [43]

    Condi- tional object-centric learning from video

    Thomas Kipf, Gamaleldin Fathy Elsayed, Aravindh Mahen- dran, Austin Stone, Sara Sabour, Georg Heigold, Rico Jon- schkowski, Alexey Dosovitskiy, and Klaus Greff. Condi- tional object-centric learning from video. In ICLR, 2022. 4

    2022

  36. [44]

    Coopera- tive learning of audio and video models from self-supervised synchronization

    Bruno Korbar, Du Tran, and Lorenzo Torresani. Coopera- tive learning of audio and video models from self-supervised synchronization. In NeurIPS, 2018. 2

    2018

  37. [45]

    Decoupled weight decay regularization

    Ilya Loshchilov and Frank Hutter. Decoupled weight decay regularization. ICLR, 2019. 4

    2019

  38. [46]

    Object vision and spatial vision: two cortical path- ways

    Mortimer Mishkin, Leslie G Ungerleider, and Kathleen A Macko. Object vision and spatial vision: two cortical path- ways. Trends in neurosciences, 1983. 2 10

    1983

  39. [47]

    Deep learning from temporal coherence in video

    Hossein Mobahi, Ronan Collobert, and Jason Weston. Deep learning from temporal coherence in video. In ICML, 2009. 2

    2009

  40. [48]

    Audio- visual instance discrimination with cross-modal agreement

    Pedro Morgado, Nuno Vasconcelos, and Ishan Misra. Audio- visual instance discrimination with cross-modal agreement. CVPR, 2021. 2

    2021

  41. [49]

    Atlas: End- to-end 3d scene reconstruction from posed images

    Zak Murez, Tarrence van As, James Bartolozzi, Ayan Sinha, Vijay Badrinarayanan, and Andrew Rabinovich. Atlas: End- to-end 3d scene reconstruction from posed images. InECCV,

  42. [50]

    Maxime Oquab, Timoth ´ee Darcet, Th´eo Moutakanni, Huy V . V o, Marc Szafraniec, Vasil Khalidov, Pierre Fernandez, Daniel HAZIZA, Francisco Massa, Alaaeldin El-Nouby, Mido Assran, Nicolas Ballas, Wojciech Galuba, Russell Howes, Po-Yao Huang, Shang-Wen Li, Ishan Misra, Michael ...

    2024

  43. [51]

    Fully sharded data parallel: faster ai training with fewer gpus

    Myle Ott, Sam Shleifer, Min Xu, Priya Goyal, Quentin Du- val, and Vittorio Caggiano. Fully sharded data parallel: faster ai training with fewer gpus. https://engineering. fb.com/2021/07/15/open-source/fsdp/ , 2021. Accessed: November 9, 2024. 4

    2021

  44. [52]

    Audio-visual scene analysis with self-supervised multisensory features

    Andrew Owens and Alexei A Efros. Audio-visual scene analysis with self-supervised multisensory features. In ECCV, 2018. 2

    2018

  45. [53]

    Context encoders: Feature learning by inpainting

    Deepak Pathak, Philipp Krahenbuhl, Jeff Donahue, Trevor Darrell, and Alexei A Efros. Context encoders: Feature learning by inpainting. In Proceedings of the IEEE con- ference on computer vision and pattern recognition , pages 2536–2544, 2016. 2, 3

    2016

  46. [54]

    Learning features by watching ob- jects move

    Deepak Pathak, Ross Girshick, Piotr Doll ´ar, Trevor Darrell, and Bharath Hariharan. Learning features by watching ob- jects move. In CVPR, 2017. 2

    2017

  47. [55]

    Perception test: A diagnostic benchmark for mul- timodal video models

    Viorica Patraucean, Lucas Smaira, Ankush Gupta, Adria Re- casens Continente, Larisa Markeeva, Dylan Sunil Banarse, Skanda Koppula, Joseph Heyward, Mateusz Malinowski, Yi Yang, Carl Doersch, Tatiana Matejovicova, Yury Sulsky, An- toine Miech, Alexandre Fr´echette, Hanna Klimcza...

    2023

  48. [56]

    Asano, Ruth Fong, Jo ˜ao F

    Mandela Patrick, Yuki M. Asano, Ruth Fong, Jo ˜ao F. Henriques, Geoffrey Zweig, and Andrea Vedaldi. Multi- modal self-supervision from generalized data transforma- tions. arXiv preprint, 2020. 2

    2020

  49. [57]

    Spatiotempo- ral contrastive video representation learning

    Rui Qian, Tianjian Meng, Boqing Gong, Ming-Hsuan Yang, Huisheng Wang, Serge Belongie, and Yin Cui. Spatiotempo- ral contrastive video representation learning. InCVPR, 2021. 2

    2021

  50. [58]

    Improving language understanding by generative pre-training

    Alec Radford, Karthik Narasimhan, Tim Salimans, and Ilya Sutskever. Improving language understanding by generative pre-training. Preprint, 2018. 2

    2018

  51. [59]

    Learning transferable visual models from natural language supervision, 2021

    Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, Gretchen Krueger, and Ilya Sutskever. Learning transferable visual models from natural language supervision, 2021. 2

    2021

  52. [60]

    Vi- sion transformers for dense prediction

    Ren ´e Ranftl, Alexey Bochkovskiy, and Vladlen Koltun. Vi- sion transformers for dense prediction. In ICCV, 2021. 5

    2021

  53. [61]

    Video (lan- guage) modeling: a baseline for generative models of natural videos

    MarcAurelio Ranzato, Arthur Szlam, Joan Bruna, Michael Mathieu, Ronan Collobert, and Sumit Chopra. Video (lan- guage) modeling: a baseline for generative models of natural videos. arXiv preprint, 2014. 2

    2014

  54. [62]

    Broaden your views for self-supervised video learning

    Adri `a Recasens, Pauline Luc, Jean-Baptiste Alayrac, Luyu Wang, Florian Strub, Corentin Tallec, Mateusz Malinowski, Viorica P ˘atr˘aucean, Florent Altch ´e, Michal Valko, Jean- Bastien Grill, A ¨aron van den Oord, and Andrew Zisserman. Broaden your views for self-supervised v...

    2021

  55. [63]

    Learning to localize sound source in visual scenes

    Arda Senocak, Tae-Hyun Oh, Junsik Kim, Ming-Hsuan Yang, and In So Kweon. Learning to localize sound source in visual scenes. In CVPR, 2018. 2

    2018

  56. [64]

    Two-stream con- volutional networks for action recognition in videos

    Karen Simonyan and Andrew Zisserman. Two-stream con- volutional networks for action recognition in videos. In ICLR, 2014. 2

    2014

  57. [65]

    A short note on the kinetics-700-2020 human action dataset

    Lucas Smaira, Jo ˜ao Carreira, Eric Noland, Ellen Clancy, Amy Wu, and Andrew Zisserman. A short note on the kinetics-700-2020 human action dataset. arXiv preprint arXiv:2010.10864, 2020. 4

    2020 arXiv

  58. [66]

    Scalability in perception for autonomous driving: Waymo open dataset

    Pei Sun, Henrik Kretzschmar, Xerxes Dotiwalla, Aur ´elien Chouard, Vijaysai Patnaik, Paul Tsui, James Guo, Yin Zhou, Yuning Chai, Benjamin Caine, Vijay Vasudevan, Wei Han, Jiquan Ngiam, Hang Zhao, Aleksei Timofeev, Scott Et- tinger, Maxim Krivokon, Amy Gao, Aditya Joshi, Yu Zh...

    2020

  59. [67]

    Videomae: Masked autoencoders are data-efficient learners for self-supervised video pre-training

    Zhan Tong, Yibing Song, Jue Wang, and Limin Wang. Videomae: Masked autoencoders are data-efficient learners for self-supervised video pre-training. NeurIPS, 2022. 1, 2, 3, 4, 5, 6, 12

    2022

  60. [68]

    RealEstate10K

    Richard Tucker and Noah Snavely. RealEstate10K. https: //google.github.io/realestate10k/index. html, 2023. Accessed: November, 2024. 2, 4

    2023

  61. [69]

    Hud- son, Thomas Albert Keck, Joao Carreira, Alexey Dosovit- skiy, Mehdi S

    Sjoerd van Steenkiste, Daniel Zoran, Yi Yang, Yulia Rubanova, Rishabh Kabra, Carl Doersch, Dilara Gokay, Joseph Heyward, Etienne Pot, Klaus Greff, Drew A. Hud- son, Thomas Albert Keck, Joao Carreira, Alexey Dosovit- skiy, Mehdi S. M. Sajjadi, and Thomas Kipf. Moving off- the-g...

  62. [70]

    An- ticipating the future by watching unlabeled video

    Carl V ondrick, Hamed Pirsiavash, and Antonio Torralba. An- ticipating the future by watching unlabeled video. arXiv preprint, 2015. 2

    2015

  63. [71]

    Videomae v2: Scaling video masked autoencoders with dual masking

    Limin Wang, Bingkun Huang, Zhiyu Zhao, Zhan Tong, Yi- nan He, Yi Wang, Yali Wang, and Yu Qiao. Videomae v2: Scaling video masked autoencoders with dual masking. In CVPR, 2023. 1, 2, 5, 6, 12

    2023

  64. [72]

    Masked video distillation: Rethinking masked feature mod- eling for self-supervised video representation learning

    Rui Wang, Dongdong Chen, Zuxuan Wu, Yinpeng Chen, Xiyang Dai, Mengchen Liu, Lu Yuan, and Yu-Gang Jiang. Masked video distillation: Rethinking masked feature mod- eling for self-supervised video representation learning. In 11 Proceedings of the IEEE/CVF conference on computer v...

    2023

  65. [73]

    Dust3r: Geometric 3d vi- sion made easy

    Shuzhe Wang, Vincent Leroy, Yohann Cabon, Boris Chidlovskii, and Jerome Revaud. Dust3r: Geometric 3d vi- sion made easy. In CVPR, 2024. 2, 5

    2024

  66. [74]

    Unsupervised learning of visual representations using videos

    Xiaolong Wang and Abhinav Gupta. Unsupervised learning of visual representations using videos. In ICCV, 2015. 2

    2015

  67. [75]

    Internvideo: General video foundation models via generative and discriminative learning

    Yi Wang, Kunchang Li, Yizhuo Li, Yinan He, Bingkun Huang, Zhiyu Zhao, Hongjie Zhang, Jilan Xu, Yi Liu, Zun Wang, et al. Internvideo: General video foundation models via generative and discriminative learning. arXiv preprint,

  68. [76]

    Less is more: Consistent video depth estimation with masked frames modeling

    Yiran Wang, Zhiyu Pan, Xingyi Li, Zhiguo Cao, Ke Xian, and Jianming Zhang. Less is more: Consistent video depth estimation with masked frames modeling. In Proceedings of the 30th ACM International Conference on Multimedia , pages 6347–6358, 2022. 5

    2022

  69. [77]

    Controlling space and time with dif- fusion models

    Daniel Watson, Saurabh Saxena, Lala Li, Andrea Tagliasac- chi, and David J Fleet. Controlling space and time with dif- fusion models. arXiv preprint, 2024. 2

    2024

  70. [78]

    Slow feature analysis: Unsupervised learning of invariances

    Laurenz Wiskott and Terrence J Sejnowski. Slow feature analysis: Unsupervised learning of invariances. Neural com- putation, 2002. 2

    2002

  71. [79]

    Depth anything: Unleashing the power of large-scale unlabeled data

    Lihe Yang, Bingyi Kang, Zilong Huang, Xiaogang Xu, Jiashi Feng, and Hengshuang Zhao. Depth anything: Unleashing the power of large-scale unlabeled data. In CVPR, 2024. 5

    2024

  72. [80]

    Sigmoid loss for language image pre-training

    Xiaohua Zhai, Basil Mustafa, Alexander Kolesnikov, and Lucas Beyer. Sigmoid loss for language image pre-training. In ICCV, 2023. 5

    2023

  73. [81]

    A general protocol to probe large vision models for 3d physical understanding

    Guanqi Zhan, Chuanxia Zheng, Weidi Xie, and Andrew Zis- serman. A general protocol to probe large vision models for 3d physical understanding. arXiv preprint, 2024. 3

    2024

  74. [82]

    Videoprism: A foundational visual encoder for video understanding

    Long Zhao, Nitesh B Gundavarapu, Liangzhe Yuan, Hao Zhou, Shen Yan, Jennifer J Sun, Luke Friedman, Rui Qian, Tobias Weyand, Yue Zhao, et al. Videoprism: A foundational visual encoder for video understanding. ICML, 2024. 1, 2, 4, 5, 6, 12

    2024

  75. [83]

    Taking something from somewhere

    Tinghui Zhou, Richard Tucker, John Flynn, Graham Fyffe, and Noah Snavely. Stereo magnification: Learning view syn- thesis using multiplane images. TOG, 2018. 4, 2 12 Scaling 4D Representations Supplementary Material This supplementary material provides additional details on pr...

    2018

Pith tools

Reviewed August 11, 2026 · model on record in the stance chip above.