Pith. sign in

REVIEW 3 major objections 6 minor 1 cited by

A vision model trained only on one person's first-person video learns geometric representations that align with models trained on other lives and match diverse web data at the same scale.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · deepseek-v4-flash

2026-08-03 18:39 UTC pith:7NI7LJ5K

load-bearing objection A genuinely new empirical claim about single-life video pretraining, but the alignment metric is unvalidated and the headline parity rests on private data. the 3 major comments →

arxiv 2512.04085 v2 pith:7NI7LJ5K submitted 2025-12-03 cs.CV

Unique Lives, Shared World: Learning from Single-Life Videos

Tengda Han , Sayna Ebrahimi , Dilara Gokay , Li Yang Ku , Maks Ovsjanikov , Iva Babukova , Daniel Zoran , Viorica Patraucean
show 3 more authors
Joao Carreira Andrew Zisserman Dima Damen
This is my paper
classification cs.CV
keywords egocentric videoself-supervised learningrepresentation alignmentsingle-life learningcross-view completiondepth estimationcorrespondence alignment scoreshared world hypothesis
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

This paper tries to establish that a single person's visual experience — the egocentric video one individual records over a week — is enough, on its own, to learn a generalizable geometric understanding of the world. It trains separate models on 20 different 'lives' and finds that models trained independently on different lives converge to a similar patch-level geometry, as measured by a new Correspondence Alignment Score (CAS). It also finds these single-life models transfer to unseen indoor depth-estimation benchmarks, and that at 30 hours of data they perform comparably to a model trained on the same amount of diverse web video, though the diverse baseline keeps an edge on one benchmark. If right, this supports a 'Shared World Hypothesis': the shared physics of the world leaves a measurable imprint in independently trained vision models, making dense personal video a viable substitute for diverse data in learning geometry.

Core claim

The paper's central claim is that the egocentric video stream of one life is a sufficient source of supervision for learning transferable 3D geometry. Concretely: models trained from scratch on one person's videos — indoor kitchens, city walking tours, or week-long mixed lives of up to 38 hours — learn representations that (i) produce similar cross-attention maps when probed on the same test pairs, quantified by the new CAS score; (ii) transfer to monocular depth estimation in unseen environments via lightweight probing; and (iii) match, within error bars, a size-matched model trained on diverse web video at the 30-hour mark on two of three downstream tasks. The alignment emerges after rough

What carries the argument

The load-bearing machinery is the Correspondence Alignment Score (CAS), which measures how similarly two independently trained models link patches between a source and a target image. For each patch in the source, each model's decoder cross-attention maps pick its top-k most-attended patches in the target; CAS is the average fraction of overlap between the two models' top-k sets. The paper also relies on the cross-view completion training objective — predicting masked patches of one frame using another frame of the same scene — as the source of geometric supervision, and on two pairing strategies (temporal proximity and spatial overlap) for forming training pairs.

Load-bearing premise

The load-bearing premise is that the Correspondence Alignment Score (CAS) measures shared geometric understanding rather than shared appearance statistics: two models agreeing on which patches attend to which other patches is taken as evidence of a shared world, not a shared dataset or a shared training objective; if CAS is inflated by dataset identity or by the architecture itself, the central alignment claim loses its footing.

What would settle it

Calibrate CAS against ground-truth correspondences on a standard correspondence benchmark: for each pair of independently trained models, compute CAS and also measure actual matching error on pixel-accurate pairs; if high-CAS pairs are not the pairs with low matching error, CAS is measuring something other than shared geometry.

Watch this falsifier — get emailed when new claim-graph text bears on it.

If this is right

  • If single-life training at 30 hours is genuinely comparable to size-matched diverse data, then long egocentric recordings from one person are a practical data source for learning geometric priors, without needing to aggregate data across many individuals.
  • The observed scaling trend implies that larger single-life datasets — longer than a week — should continue to improve alignment and downstream transfer, potentially closing the gap with much larger diverse datasets.
  • Because alignment emerges only after roughly 30–60 minutes of data, there is a minimum experience threshold below which a single life does not yet imprint the shared structure of the world.
  • The pairing-strategy result implies that the way frames are paired matters more than raw duration: combining spatial and temporal co-visibility signals gives the best transfer, and simple temporal pairing already outperforms synthetic augmentation.
  • The finding that a different self-supervised objective also learns usable geometry from single lives suggests the paradigm is not tied to one architecture.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • The block-diagonal structure the paper reports in its CAS matrix — models from the same dataset aligning more with each other — suggests a plausible confound: part of the measured 'alignment' may come from shared dataset statistics (same rooms, same camera, same activity types) rather than from the shared physical world. The paper does not fully ablate this, so the strong claim about shared world
  • If the alignment claim holds, a natural testable extension is to predict downstream transfer performance from CAS values: models that align more with a reference should transfer better, giving a cheap proxy for data quality without running task-specific heads.
  • The single-life framing invites a transfer to robotics and embodied AI: a robot's own continuous sensory stream could be treated as a life, and alignment with other agents' streams could be used to certify that the robot has learned the same geometry of the world.
  • One could push the paradigm toward semantics: the paper focuses on geometry, but the dense, redundant stream of one life may also support object permanence and semantic categories; whether single-life data can learn semantics as efficiently as geometry is left open.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

3 major / 6 minor

Summary. The paper introduces a 'single-life' learning paradigm in which a separate vision model is trained from scratch on the egocentric video stream of a single individual, using the CroCo cross-view completion objective. Using three sources of egocentric data (HD-Epic kitchens, Walking Tours city walks, and a private week-long Anonymous Lives dataset, totaling 20 'lives'), the authors report three findings: (1) models trained independently on different lives converge to a highly aligned geometric representation, measured by a new Correspondence Alignment Score (CAS) based on top-k overlap of decoder cross-attention maps; (2) these single-life models transfer to unseen depth estimation and correspondence tasks; and (3) at the 30-hour scale, single-life training is competitive with training on a size-matched subset of Kinetics-400. The paper also provides non-life control videos, a DINOv2 extension, and an analysis of temporal versus spatial pairing strategies.

Significance. If the central alignment claim can be substantiated with a validated metric, the paper is significant. It proposes a novel data regime for geometric representation learning, demonstrates that dense single-stream egocentric data can rival diverse web data on some downstream tasks at matched data budgets, and offers a new patch-level, training-free similarity metric. The strengths include evaluation on external benchmarks (NYU-Depth-v2, ScanNet, HPatches) untouched by pretraining, error bars across lives, a controlled size-matched K400 baseline, and an architectural extension to DINOv2. The main weakness is that the alignment claim rests on a metric that is not calibrated against ground-truth correspondence, leaving an appearance/domain-statistics confound unresolved.

major comments (3)
  1. [§3.4, Eq. (3); §5.2, Q1; Figs. 4–6] The central claim that independently trained single-life models converge to a shared geometric understanding rests on CAS. CAS measures agreement between top-k decoder cross-attention maps on held-out pairs, not agreement with ground-truth correspondence. Because these cross-attention maps are the same QK conduit optimized by L_CroCo (§3.2), two models may agree because they share the architecture/objective or low-level image statistics rather than 3D structure. The block-diagonal structure in Fig. 5 is the signature of a dataset/appearance confound, and the text acknowledges it only qualitatively. A direct calibration is available since the same models are scored zero-shot on HPatches (App. C.1): compute CAS on HPatches pairs and compare it with per-patch ground-truth flow, or measure agreement between each model's attention and the ground-truth correspondence. I also recommend reportin
  2. [§5.2, Fig. 7; Abstract] The abstract's claim of 'comparable performance' to 30 hours of diverse web data is only partially supported. On NYU-Depth-v2 the ALD lives reach 0.570–0.580 δ1 versus 0.590 for K400-30h, which is within error but not a clear match; on ScanNet K400 has a distinct advantage at all durations; on HPatches the curves cross near 30h. The text's statement that single-life models 'nearly match or even surpass' the size-matched baseline should be qualified task-by-task, and the paper should state explicitly which comparisons are statistically equivalent (e.g., overlapping confidence intervals) and which are not. This does not undermine the 'comparable' claim, but the current wording overstates the evidence on ScanNet.
  3. [§3.4; Algorithm 1] CAS uses a fixed top-k parameter k=5 with no sensitivity analysis. Since the metric's absolute values and the block structure in Fig. 5 may depend on k, a small sweep (e.g., k ∈ {1,3,5,10,20}) would establish that the alignment conclusions are not an artifact of the chosen neighborhood size. This is particularly relevant because the expected overlap under a random model changes with k.
minor comments (6)
  1. [§5.2 (DINOv2 paragraph)] Typo: 'achiev' should be 'achieves'.
  2. [Fig. 3] The 'Depth' and 'Flow Magnitude' rows list 'O2' twice in the dataset ordering; the duplicate appears to be a typo.
  3. [§5.2, Fig. 6 discussion] The citation '[19]' appears mid-sentence ('...as the center between these.[19]') and is not attached to a meaningful claim; please move or remove it.
  4. [App. C.5 / Fig. 15] The appendix contains the marker 'Confidential — Google DeepMind' above the video-segmentation figure. This label should be removed from a public arXiv submission; it is inconsistent with the paper's public status.
  5. [App. C.4 / Table 6] The DINOv2 results show that the Minecraft control (O4) aligns strongly with natural-life models (CAS ≈ 0.51 with CroCo), which nuances the blanket statement that 'non-life videos fail to align.' The authors should acknowledge this exception explicitly in the main text or appendix discussion.
  6. [§4] The Anonymous Lives Dataset (ALD) is private and access-restricted. While understandable, this limits reproducibility of the headline 30-hour result; consider releasing anonymized model checkpoints or features, or at least addressing the privacy/access tradeoff explicitly.

Circularity Check

0 steps flagged

No significant circularity: generalization is tested on external benchmarks and the alignment metric is a proxy, not a conclusion defined by construction.

full rationale

The paper's central claims are not circular. Generalization is measured on held-out external benchmarks (NYU-Depth-v2, ScanNet, HPatches) that are untouched by single-life pretraining, and the size-matched Kinetics-400 baseline provides an independent yardstick; no fitted parameter is renamed as a prediction, and no equation reduces the claimed result to its own inputs. The Correspondence Alignment Score (CAS, Eq. 3) is introduced as an explicit operationalization of functional similarity between independently trained models, and while it is an unvalidated surrogate—arguably reflecting appearance or domain statistics as visible in the block-diagonal structure of Fig. 5 and in the DINOv2 appendix where Minecraft videos yield high CAS—this is a validity/correctness concern, not circularity: the paper also evaluates the same cross-attention conduit zero-shot on HPatches against ground-truth flow, providing external evidence that the conduit carries geometric information. Self-citations (CroCo architecture [42], dataset papers, evaluation protocol [7]) are data/implementation references or external prior work and do not carry the load-bearing argument. Therefore the derivation is self-contained against external benchmarks and no circular step can be exhibited by reduction to the paper's own equations or definitions.

Axiom & Free-Parameter Ledger

5 free parameters · 5 axioms · 2 invented entities

Central claims rest on a philosophical premise (Shared World Hypothesis), borrowed architectural assumptions (CroCo cross-view completion encodes geometry), a data assumption (temporal proximity implies overlap; egocentricity is essential), and an unvalidated new metric (CAS). Free parameters include CAS k=5, the 95% masking ratio, hand-set epoch schedules, and per-life Jaccard/masking choices selected on the downstream evaluation tasks themselves.

free parameters (5)
  • CAS top-k parameter k = 5
    Mutual-top-k neighborhood size in the new alignment metric (App B.5). No sensitivity analysis shown; alignment results in Figs 4-6 depend on this choice.
  • Temporal-pair masking ratio = 0.95
    95% masking for all temporal-pairing pretraining (App B.1); interacts with how much reconstruction signal each pair provides and was not swept for the main temporal results.
  • Per-life Jaccard co-visibility threshold = 0.5 / 0.7 / 0.9 per life
    Grid-searched per HD-Epic participant (App B.4, Table 3) with performance evaluated via probing on ScanNet and NYU-Depth-v2 - the same tasks used for the final pairing-strategy figures (Fig 8).
  • Per-life spatial masking ratio = 0.5 / 0.7 / 0.9 per life
    Selected in the same per-life sweep as the Jaccard threshold (App B.4, Table 3); reported spatial/union gains therefore partly incorporate selection on the evaluation tasks.
  • Training epochs per dataset = 100 (HD-Epic) / 30 (WT) / 10 (ALD)
    Chosen so all models reach roughly 200-250K iterations (App B.1, B.3); a hand-set schedule that equalizes compute across lives and life sizes but is not derived from the data.
axioms (5)
  • domain assumption Shared World Hypothesis: all lives inherit universal 3D Euclidean geometry and object permanence from the shared physical world.
    Stated in §1 as the paper's thesis and used to predict alignment in §5.2 Q1. It is a philosophical premise, not derived from the experiments.
  • domain assumption CroCo cross-view completion with heavy masking encodes rich geometric information when trained on single-life pairs.
    Adopted from [2, 42] (§3.2); the whole pipeline inherits the validity of this objective as a geometric pretraining signal.
  • domain assumption Temporal proximity of frames implies non-trivial viewpoint overlap.
    §3.3: 'most temporally close frames have a non-trivial viewpoint overlap.' Load-bearing for WT/ALD results where no camera poses exist and only temporal pairing is used.
  • ad hoc to paper CAS agreement on held-out pairs is a valid proxy for shared geometric (not merely statistical) understanding.
    §3.4, Eq. 3; the alignment claim rests entirely on this new unvalidated metric. Fig 5's block-diagonal structure shows strong dataset-identity sensitivity, indicating a domain-statistics component.
  • domain assumption A 'life' is adequately represented by the collected videos (up to 38h) and the fixed 250K-iteration budget gives a fair comparison across life sizes.
    App B.1/B.3; the 30min-1h emergence threshold and the parity claims are conditional on the fixed iteration budget and on the representativeness of each person's collected week.
invented entities (2)
  • Correspondence Alignment Score (CAS) no independent evidence
    purpose: Measures functional alignment of independently trained models at patch level (§3.4).
    New instrument introduced by the authors; never calibrated against ground-truth correspondence even though the same models are scored on HPatches. Internal controls (non-life videos near zero) give discrimination but not geometric ground truth.
  • 'Single-life learning' paradigm (one model per individual stream) independent evidence
    purpose: Defines the data unit and training protocol of the study (Eq. 1).
    Its predictions are tested on public downstream benchmarks (NYU, ScanNet, HPatches) unused during pretraining, providing external falsifiable handles.

pith-pipeline@v1.3.0-alltime-deepseek · 24115 in / 25103 out tokens · 218809 ms · 2026-08-03T18:39:08.682336+00:00 · methodology

0 comments
read the original abstract

We introduce the "single-life" learning paradigm, where we train a distinct vision model exclusively on egocentric videos captured by one individual. We leverage the multiple viewpoints naturally captured within a single life to learn a visual encoder in a self-supervised manner. Our experiments demonstrate three key findings. First, models trained independently on different lives develop a highly aligned geometric understanding. We demonstrate this by training visual encoders on distinct datasets each capturing a different life, both indoors and outdoors, as well as introducing a novel cross-attention-based metric to quantify the functional alignment of the internal representations developed by different models. Second, we show that single-life models learn generalizable geometric representations that effectively transfer to downstream tasks, such as depth estimation, in unseen environments. Third, we demonstrate that training on up to 30 hours from one week of the same person's life leads to comparable performance to training on 30 hours of diverse web data, highlighting the strength of single-life representation learning. Overall, our results establish that the shared structure of the world, both leads to consistency in models trained on individual lives, and provides a powerful signal for visual representation learning.

Figures

Figures reproduced from arXiv: 2512.04085 by Andrew Zisserman, Daniel Zoran, Dilara Gokay, Dima Damen, Iva Babukova, Joao Carreira, Li Yang Ku, Maks Ovsjanikov, Sayna Ebrahimi, Tengda Han, Viorica Patraucean.

Figure 1
Figure 1. Figure 1: Single-Life Learning Paradigm. We train a distinct model on egocentric video from one person’s experience, and study the alignment and generalization of these models. distinct, they all arise from and are conditioned by the same underlying physical world. Structural properties such as 3D Euclidean geometry and object permanence are universals that leave a consistent imprint on all visual data. In this pape… view at source ↗
Figure 2
Figure 2. Figure 2: Correspondence Alignment Score (CAS). Given a test pair of images, we extract cross-attention maps, Ai and Aj , from independently trained models θ ∗ i and θ ∗ j . For each query patch in the source image, we identify the top-K most attended-to patches in the target by both models, and compute their intersection. more realistic and complex setting. We use two signals to generate pairs for learning. Tempora… view at source ↗
Figure 3
Figure 3. Figure 3: Properties of every ‘life’ used for training (colored by dataset). Left: distribution of continuous variables per life, shown in box [PITH_FULL_IMAGE:figures/full_fig_p005_3.png] view at source ↗
Figure 4
Figure 4. Figure 4: Effect of ‘life size’ measured by CAS with CroCo. For [PITH_FULL_IMAGE:figures/full_fig_p006_4.png] view at source ↗
Figure 5
Figure 5. Figure 5: CAS score matrix comparing models trained on individ [PITH_FULL_IMAGE:figures/full_fig_p006_5.png] view at source ↗
Figure 6
Figure 6. Figure 6: A 2D MDS visualization of the models, using the CAS [PITH_FULL_IMAGE:figures/full_fig_p007_6.png] view at source ↗
Figure 7
Figure 7. Figure 7: Generalization performance scales robustly with single-life data duration. We evaluate models trained on single lives of increasing [PITH_FULL_IMAGE:figures/full_fig_p008_7.png] view at source ↗
Figure 8
Figure 8. Figure 8: Relative performance gains of single life learning on [PITH_FULL_IMAGE:figures/full_fig_p008_8.png] view at source ↗
Figure 9
Figure 9. Figure 9: Relative performance gains of single life learning on [PITH_FULL_IMAGE:figures/full_fig_p008_9.png] view at source ↗
Figure 10
Figure 10. Figure 10: Visualization of pairing strategies. Given a (a) source image, we generate pairs using different strategies. (b) A temporal pair is a future frame from the video, capturing natural motion. (c) A spatial pair is a non-consecutive frame with high geometric overlap, found using camera poses. (d) An augmented pair is a simple 2D transformation of the source image itself. (e) A random pair is randomly selected… view at source ↗
Figure 11
Figure 11. Figure 11: Relative performance gains on zero-shot HPatches correspondence. The plots show the percentage gain in performance (AEPE, lower is better) for different pairing strategies relative to the Augmented Pairing baseline (the 0% line) for our three main datasets: (a) HD-Epic, (b) Walking Tours, and (c) Anonymous Lives. source frame P01 P02 WT1 WT2 ALD1 ALD2 [PITH_FULL_IMAGE:figures/full_fig_p015_11.png] view at source ↗
Figure 12
Figure 12. Figure 12: HPatches qualitative results from six single-life models. We query a subset of source pixels (shown as “dot”), and visualize the [PITH_FULL_IMAGE:figures/full_fig_p015_12.png] view at source ↗
Figure 13
Figure 13. Figure 13: DINOv2 CAS metric across models. We do not observe [PITH_FULL_IMAGE:figures/full_fig_p016_13.png] view at source ↗
Figure 14
Figure 14. Figure 14: A 2D MDS visualization of the single-life DINOv2 [PITH_FULL_IMAGE:figures/full_fig_p016_14.png] view at source ↗
Figure 15
Figure 15. Figure 15: Video segmentation tracking results on DAVIS 2017 dataset from three single-life models. From top: HD-Epic P01,P02,P04, [PITH_FULL_IMAGE:figures/full_fig_p017_15.png] view at source ↗
Figure 16
Figure 16. Figure 16: A collection of sample frames for each of the datasets. [PITH_FULL_IMAGE:figures/full_fig_p018_16.png] view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score.

  1. SeeSE3: Emergence of 3D Space in Vision Features

    cs.CV 2026-07 conditional novelty 6.0

    Self-supervised vision features, especially DINOv2, contain a subspace that a small trained adapter can map to 3D camera motion, enabling pose estimation and latent-space navigation without explicit 3D reconstruction.

Reference graph

Works this paper leans on

49 extracted references · 10 linked inside Pith · cited by 1 Pith paper

  1. [1]

    Gpt-4 technical report.arXiv preprint arXiv:2303.08774,

    Josh Achiam, Steven Adler, Sandhini Agarwal, Lama Ah- mad, Ilge Akkaya, Florencia Leoni Aleman, Diogo Almeida, Janko Altenschmidt, Sam Altman, Shyamal Anadkat, et al. Gpt-4 technical report.arXiv preprint arXiv:2303.08774,

    Pith/arXiv arXiv

  2. [2]

    Cross-view completion models are zero-shot correspondence estimators

    Honggyu An, Jin Hyeon Kim, Seonghoon Park, Jaewoo Jung, Jisang Han, Sunghwan Hong, and Seungryong Kim. Cross-view completion models are zero-shot correspondence estimators. InCVPR, pages 1103–1115, 2025. 2, 3, 4, 5, 6, 13, 15

    2025

  3. [3]

    Anonymous lives dataset - a private dataset

    Anonymous. Anonymous lives dataset - a private dataset. Unpublished private dataset, 2025. Access restricted. 1, 5

    2025

  4. [4]

    Hpatches: A benchmark and evaluation of handcrafted and learned local descriptors

    Vassileios Balntas, Karel Lenc, Andrea Vedaldi, and Krys- tian Mikolajczyk. Hpatches: A benchmark and evaluation of handcrafted and learned local descriptors. InCVPR, pages 5173–5182, 2017. 6

    2017

  5. [5]

    Revis- iting model stitching to compare neural representations

    Yamini Bansal, Preetum Nakkiran, and Boaz Barak. Revis- iting model stitching to compare neural representations. In NeurIPS, pages 225–236, 2021. 2

    2021

  6. [6]

    JAX: composable transformations of Python+NumPy programs, 2018

    James Bradbury, Roy Frostig, Peter Hawkins, Matthew James Johnson, Chris Leary, Dougal Maclau- rin, George Necula, Adam Paszke, Jake VanderPlas, Skye Wanderman-Milne, and Qiao Zhang. JAX: composable transformations of Python+NumPy programs, 2018. 12

    2018

  7. [7]

    Scal- ing 4d representations.arXiv preprint arXiv:2412.15212,

    Jo ˜ao Carreira, Dilara Gokay, Michael King, Chuhan Zhang, Ignacio Rocco, Aravindh Mahendran, Thomas Albert Keck, Joseph Heyward, Skanda Koppula, Etienne Pot, et al. Scal- ing 4d representations.arXiv preprint arXiv:2412.15212,

    Pith/arXiv arXiv

  8. [8]

    Learning from one continuous video stream

    Jo ˜ao Carreira, Michael King, Viorica Patraucean, Dilara Gokay, Catalin Ionescu, Yi Yang, Daniel Zoran, Joseph Hey- ward, Carl Doersch, Yusuf Aytar, et al. Learning from one continuous video stream. InCVPR, 2024. 1, 2

    2024

  9. [9]

    Multidimensional scal- ing.Measurement, judgment and decision making, pages 179–250, 1998

    J Douglas Carroll and Phipps Arabie. Multidimensional scal- ing.Measurement, judgment and decision making, pages 179–250, 1998. 6

    1998

  10. [10]

    Internvl: Scaling up vision foundation mod- els and aligning for generic visual-linguistic tasks

    Zhe Chen, Jiannan Wu, Wenhai Wang, Weijie Su, Guo Chen, Sen Xing, Muyan Zhong, Qinglong Zhang, Xizhou Zhu, Lewei Lu, et al. Internvl: Scaling up vision foundation mod- els and aligning for generic visual-linguistic tasks. InCVPR, pages 24185–24198, 2024. 1

    2024

  11. [11]

    Gemini 2.5: Pushing the frontier with advanced reasoning, multimodality, long context, and next generation agentic capabilities.arXiv preprint arXiv:2507.06261, 2025

    Gheorghe Comanici, Eric Bieber, Mike Schaekermann, Ice Pasupat, Noveen Sachdeva, Inderjit Dhillon, Marcel Blis- tein, Ori Ram, Dan Zhang, Evan Rosen, et al. Gemini 2.5: Pushing the frontier with advanced reasoning, multimodality, long context, and next generation agentic capabilities.arXiv preprint arXiv:2507.06261, 2025. 1

    Pith/arXiv arXiv 2025

  12. [12]

    Scannet: Richly-annotated 3d reconstructions of indoor scenes

    Angela Dai, Angel X Chang, Manolis Savva, Maciej Hal- ber, Thomas Funkhouser, and Matthias Nießner. Scannet: Richly-annotated 3d reconstructions of indoor scenes. In CVPR, 2017. 5

    2017

  13. [13]

    Rescaling egocentric vision: Collection, pipeline and challenges for epic-kitchens-100.IJCV, 130: 33–55, 2022

    Dima Damen, Hazel Doughty, Giovanni Maria Farinella, Antonino Furnari, Jian Ma, Evangelos Kazakos, Davide Moltisanti, Jonathan Munro, Toby Perrett, Will Price, and Michael Wray. Rescaling egocentric vision: Collection, pipeline and challenges for epic-kitchens-100.IJCV, 130: 33–55, 2022. 2, 5

    2022

  14. [14]

    Depthcues: Evaluating monoc- ular depth perception in large vision models

    Duolikun Danier, Mehmet Ayg ¨un, Changjian Li, Hakan Bilen, and Oisin Mac Aodha. Depthcues: Evaluating monoc- ular depth perception in large vision models. InCVPR, pages 20049–20059, 2025. 2

    2025

  15. [15]

    Project aria: A new tool for egocentric multi- modal ai research, 2023

    Jakob Engel, Kiran Somasundaram, Michael Goesele, Al- bert Sun, Alexander Gamino, Andrew Turner, Arjang Talat- tof, Arnie Yuan, Bilal Souti, Brighid Meredith, Cheng Peng, Chris Sweeney, Cole Wilson, Dan Barnes, Daniel DeTone, David Caruso, Derek Valleroy, Dinesh Ginjupalli, Duncan Frost, et al. Project aria: A new tool for egocentric multi- modal ai resea...

    2023

  16. [16]

    Ego4d: Around the World in 3,000 Hours of Egocentric Video

    Kristen Grauman, Andrew Westbury, Eugene Byrne, Zachary Chavis, Antonino Furnari, Rohit Girdhar, Jackson Hamburger, Hao Jiang, Miao Liu, Xingyu Liu, Miguel Mar- tin, Tushar Nagarajan, Ilija Radosavovic, Santhosh Kumar Ramakrishnan, Fiona Ryan, Jayant Sharma, Michael Wray, Mengmeng Xu, et al. Ego4d: Around the World in 3,000 Hours of Egocentric Video. InCV...

    2022

  17. [17]

    Siamese masked autoencoders

    Agrim Gupta, Jiajun Wu, Jia Deng, and Fei-Fei Li. Siamese masked autoencoders. InNeurIPS, 2023. 1, 2, 4

    2023

  18. [18]

    Learning from streaming video with orthogonal gradients

    Tengda Han, Dilara Gokay, Joseph Heyward, Chuhan Zhang, Daniel Zoran, Viorica P ˘atr˘aucean, Jo ˜ao Carreira, Dima Damen, and Andrew Zisserman. Learning from streaming video with orthogonal gradients. InCVPR, 2025. 2

    2025

  19. [19]

    Masked autoencoders are scalable vision learners

    Kaiming He, Xinlei Chen, Saining Xie, Yanghao Li, Piotr Doll´ar, and Ross Girshick. Masked autoencoders are scalable vision learners. InCVPR, 2022. 3, 7

    2022

  20. [20]

    Masked autoencoders are scalable vision learners

    Kaiming He, Xinlei Chen, Saining Xie, Yanghao Li, Piotr Doll´ar, and Ross Girshick. Masked autoencoders are scalable vision learners. InCVPR, pages 16000–16009, 2022. 2

    2022

  21. [21]

    The platonic representation hypothesis.arXiv preprint arXiv:2405.07987, 2024

    Minyoung Huh, Brian Cheung, Tongzhou Wang, and Phillip Isola. The platonic representation hypothesis.arXiv preprint arXiv:2405.07987, 2024. 2, 4

    Pith/arXiv arXiv 2024

  22. [22]

    Space-time correspondence as a contrastive random walk

    Allan Jabri, Andrew Owens, and Alexei Efros. Space-time correspondence as a contrastive random walk. InNeurIPS, pages 19545–19560, 2020. 16

    2020

  23. [23]

    Harnessing the universal geometry of embeddings.arXiv preprint arXiv:2505.12540, 2025

    Rishi Jha, Collin Zhang, Vitaly Shmatikov, and John X Mor- ris. Harnessing the universal geometry of embeddings.arXiv preprint arXiv:2505.12540, 2025. 2

    arXiv 2025

  24. [24]

    The kinetics hu- man action video dataset.arXiv preprint arXiv:1705.06950,

    Will Kay, Joao Carreira, Karen Simonyan, Brian Zhang, Chloe Hillier, Sudheendra Vijayanarasimhan, Fabio Viola, Tim Green, Trevor Back, Paul Natsev, et al. The kinetics hu- man action video dataset.arXiv preprint arXiv:1705.06950,

    Pith/arXiv arXiv

  25. [25]

    Similarity of neural network representa- tions revisited

    Simon Kornblith, Mohammad Norouzi, Honglak Lee, and Geoffrey Hinton. Similarity of neural network representa- tions revisited. InICML, pages 3519–3529, 2019. 2, 4

    2019

  26. [26]

    Decoupled weight decay regularization.arXiv preprint arXiv:1711.05101, 2017

    Ilya Loshchilov and Frank Hutter. Decoupled weight decay regularization.arXiv preprint arXiv:1711.05101, 2017. 12 9

    Pith/arXiv arXiv 2017

  27. [27]

    Egoloc: Revisiting 3d object local- ization from egocentric videos with visual queries

    Jinjie Mai, Abdullah Hamdi, Silvio Giancola, Chen Zhao, and Bernard Ghanem. Egoloc: Revisiting 3d object local- ization from egocentric videos with visual queries. InICCV, pages 45–57, 2023. 2

    2023

  28. [28]

    Do vision and language encoders represent the world similarly? InCVPR, pages 14334–14343, 2024

    Mayug Maniparambil, Raiymbek Akshulakov, Yasser Ab- delaziz Dahou Djilali, Mohamed El Amine Seddik, Sanath Narayan, Karttikeya Mangalam, and Noel E O’Connor. Do vision and language encoders represent the world similarly? InCVPR, pages 14334–14343, 2024. 2

    2024

  29. [29]

    Dinov2: Learning robust visual features without supervision

    Maxime Oquab, Timoth ´ee Darcet, Th ´eo Moutakanni, Huy V o, Marc Szafraniec, Vasil Khalidov, Pierre Fernandez, Daniel Haziza, Francisco Massa, Alaaeldin El-Nouby, et al. Dinov2: Learning robust visual features without supervision. arXiv preprint arXiv:2304.07193, 2023. 1, 3, 8, 15

    Pith/arXiv arXiv 2023

  30. [30]

    Hd-epic: A highly-detailed egocentric video dataset

    Toby Perrett, Ahmad Darkhalil, Saptarshi Sinha, Omar Emara, Sam Pollard, Kranti Parida, Kaiting Liu, Prajwal Gatti, Siddhant Bansal, Kevin Flanagan, Jacob Chalk, Zhi- fan Zhu, Rhodri Guerrier, Fahd Abdelazim, Bin Zhu, Da- vide Moltisanti, Michael Wray, Hazel Doughty, and Dima Damen. Hd-epic: A highly-detailed egocentric video dataset. InCVPR, 2025. 1, 2, 4, 11

    2025

  31. [31]

    The 2017 davis challenge on video object segmentation.arXiv preprint arXiv:1704.00675, 2017

    Jordi Pont-Tuset, Federico Perazzi, Sergi Caelles, Pablo Ar- bel´aez, Alex Sorkine-Hornung, and Luc Van Gool. The 2017 davis challenge on video object segmentation.arXiv preprint arXiv:1704.00675, 2017. 16

    Pith/arXiv arXiv 2017

  32. [32]

    Vi- sion transformers for dense prediction

    Ren ´e Ranftl, Alexey Bochkovskiy, and Vladlen Koltun. Vi- sion transformers for dense prediction. InICCV, pages 12179–12188, 2021. 5, 14

    2021

  33. [33]

    It’s a (blind) match! towards vision-language correspon- dence without parallel data

    Dominik Schnaus, Nikita Araslanov, and Daniel Cremers. It’s a (blind) match! towards vision-language correspon- dence without parallel data. InCVPR, pages 24983–24992,

  34. [34]

    Indoor segmentation and support inference from rgbd images

    Nathan Silberman, Derek Hoiem, Pushmeet Kohli, and Rob Fergus. Indoor segmentation and support inference from rgbd images. InECCV, pages 746–760. Springer, 2012. 5

    2012

  35. [35]

    Krishna Kumar Singh, Kayvon Fatahalian, and Alexei A. Efros. Krishnacam: Using a longitudinal, single-person, egocentric dataset for scene understanding tasks. InIEEE Winter Conference on Applications of Computer Vision (WACV), 2016. 2

    2016

  36. [36]

    Raft: Recurrent all-pairs field transforms for optical flow, 2020

    Zachary Teed and Jia Deng. Raft: Recurrent all-pairs field transforms for optical flow, 2020. 12

    2020

  37. [37]

    Understanding the emergence of multimodal representation alignment.arXiv preprint arXiv:2502.16282,

    Megan Tjandrasuwita, Chanakya Ekbote, Liu Ziyin, and Paul Pu Liang. Understanding the emergence of multimodal representation alignment.arXiv preprint arXiv:2502.16282,

    Pith/arXiv arXiv

  38. [38]

    VideoMAE: Masked autoencoders are data-efficient learners for self-supervised video pre-training

    Zhan Tong, Yibing Song, Jue Wang, and Limin Wang. VideoMAE: Masked autoencoders are data-efficient learners for self-supervised video pre-training. InNeurIPS, 2022. 3

    2022

  39. [39]

    EPIC Fields: Marrying 3D Geometry and Video Understanding

    Vadim Tschernezki, Ahmad Darkhalil, Zhifan Zhu, David Fouhey, Iro Larina, Diane Larlus, Dima Damen, and Andrea Vedaldi. EPIC Fields: Marrying 3D Geometry and Video Understanding. InNeurIPS, 2023. 2

    2023

  40. [40]

    Is ImageNet worth 1 video? learning strong image encoders from 1 long unlabelled video

    Shashanka Venkataramanan, Mamshad Nayeem Rizve, Jo ˜ao Carreira, Yuki M Asano, and Yannis Avrithis. Is ImageNet worth 1 video? learning strong image encoders from 1 long unlabelled video. InICLR, 2024. 1, 2, 4, 5, 11

    2024

  41. [41]

    Dust3r: Geometric 3d vi- sion made easy

    Shuzhe Wang, Vincent Leroy, Yohann Cabon, Boris Chidlovskii, and Jerome Revaud. Dust3r: Geometric 3d vi- sion made easy. InCVPR, pages 20697–20709, 2024. 2, 11

    2024

  42. [42]

    Croco: Self-supervised pre-training for 3d vision tasks by cross-view completion

    Philippe Weinzaepfel, Vincent Leroy, Thomas Lucas, Ro- main Br´egier, Yohann Cabon, Vaibhav Arora, Leonid Ants- feld, Boris Chidlovskii, Gabriela Csurka, and J ´erˆome Re- vaud. Croco: Self-supervised pre-training for 3d vision tasks by cross-view completion. InNeurIPS, pages 3502–3516,

  43. [43]

    Croco v2: Improved cross-view completion pre- training for stereo matching and optical flow

    Philippe Weinzaepfel, Thomas Lucas, Vincent Leroy, Yohann Cabon, Vaibhav Arora, Romain Br ´egier, Gabriela Csurka, Leonid Antsfeld, Boris Chidlovskii, and J ´erˆome Revaud. Croco v2: Improved cross-view completion pre- training for stereo matching and optical flow. InICCV, pages 17969–17980, 2023. 2

    2023

  44. [44]

    Qwen3-omni technical report.arXiv preprint arXiv:2509.17765, 2025

    Jin Xu, Zhifang Guo, Hangrui Hu, Yunfei Chu, Xiong Wang, Jinzheng He, Yuxuan Wang, Xian Shi, Ting He, Xinfa Zhu, et al. Qwen3-omni technical report.arXiv preprint arXiv:2509.17765, 2025. 1

    Pith/arXiv arXiv 2025

  45. [45]

    Learn- ing streaming video representation via multitask training

    Yibin Yan, Jilan Xu, Shangzhe Di, Yikun Liu, Yudi Shi, Qirui Chen, Zeqian Li, Yifei Huang, and Weidi Xie. Learn- ing streaming video representation via multitask training. In ICCV, 2025. 2

    2025

  46. [46]

    Depth any- thing v2, 2024

    Lihe Yang, Bingyi Kang, Zilong Huang, Zhen Zhao, Xiao- gang Xu, Jiashi Feng, and Hengshuang Zhao. Depth any- thing v2, 2024. 11

    2024

  47. [47]

    Assess- ing and learning alignment of unimodal vision and language models

    Le Zhang, Qian Yang, and Aishwarya Agrawal. Assess- ing and learning alignment of unimodal vision and language models. InCVPR, pages 14604–14614, 2025. 2

    2025

  48. [48]

    non-life

    Tinghui Zhou, Matthew Brown, Noah Snavely, and David G Lowe. Unsupervised learning of depth and ego-motion from video. InCVPR, pages 1851–1858, 2017. 2 10 Unique Lives, Shared World: Learning from Single-Life Videos Appendix Table of Contents A . Dataset Details 11 A.1 . Overview of single-life datasets. . . . . . . . 11 A.2 . How we obtain dataset proper...

    arXiv 2017

  49. [2022]

    1, 2, 3, 5, 12, 14, 15