Pith. sign in

REVIEW 4 major objections 5 minor 67 references

ReViV: Reconstructing the Viewer and the View in 4D from Monocular Egocentric Video

T0 review · 4 major / 5 minor · reviewed 2026-08-15 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read One first-person video yields the wearer's motion and a metric 4D scene in a single pass.

desk verdict Strong unified body/hand/gaze model, but the metric 4D claim is unsupported because all depth metrics are scale-invariant and the alignment step is never validated. read the letter →

arxiv 2607.17790 v1 pith:K7BPDK6W submitted 2026-07-20 cs.CV cs.AI

Xiaozhong Lyu , Gen Li , Zhiyin Qian , Xucong Zhang , Marc Pollefeys , Siyu Tang This is my paper · ORCID
classification cs.CVcs.AI
keywords egocentricvision4Dreconstructionmaskedgenerativemodelinghumanmotionestimationhandposegazecameratrajectorydepth
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper tries to show that a single monocular video from a head-mounted camera contains enough information to reconstruct, at the same time, the wearer's full-body motion, hand poses, gaze direction, the camera's trajectory, and the scene depth. It argues that these signals should not be predicted separately, because they constrain one another: where the wearer looks and reaches is tied to the scene geometry, and the camera motion is tied to the body. ReViV casts the whole reconstruction as one masked generative modeling problem over a shared discrete token space, and reports that a single feed-forward transformer, trained on a seven-billion-token multimodal corpus, reconstructs all modalities at 0.7 seconds per two-second clip. If the claim holds, wearable devices could build a metric 4D model of the user-and-world interaction directly from raw video, without SLAM or external hand trackers.

What carries the argument

The Masked Generative Egocentric Transformer (MGET), a T5-style encoder-decoder trained on a sequence of discrete tokens produced by modality-specific vector-quantized autoencoders (VQ-VAEs). Each continuous signal, be it body joints, hand joints, gaze, camera trajectory, RGB, or depth, is quantized into a codebook vocabulary; during training, random masks over the unified token sequence force the model to predict any subset of modalities from any other subset, thereby approximating the joint distribution $p(X,Y)$. At inference, only the RGB tokens and a continuous vision-transformer branch are visible, and the missing tokens are decoded iteratively with classifier-free guidance. The continuous branch preserves high-frequency visual detail that the discrete quantization would otherwise discard.

What would settle it

Take ReViV's predicted depth on the Aria Digital Twin sequences that have ground-truth metric depth, skip the usual clip-wise scale-and-shift alignment, and compare absolute scale; if the scale error is large precisely on sequences where the floor is visible, the floor-fitting step, not the kinematic predictions, is what carries the metric claim.

Watch

Extended reading notes

Core claim

The central claim is that body, hand, gaze, camera trajectory, and depth can be treated as one joint distribution over discrete tokens, and that a transformer trained to predict randomly masked tokens from this distribution learns the cross-modal couplings that make each individual prediction more accurate and temporally consistent. On the unseen Aria Digital Twin benchmark, ReViV reports lower aligned pose error and higher motion similarity for body reconstruction than diffusion baselines that are given ground-truth or estimated camera trajectories, and it reports lower hand pose errors than optimization-based hand trackers on HoloAssist, HOT3D, ARCTIC, and TACO. The camera trajectory and gaze predictions also improve on the egocentric pretraining baseline, while depth remains competitive with a specialized monocular depth model but does not surpass it. The metric 4D alignment is obtained by a lightweight post-hoc scale step: fitting a floor plane from the predicted body and scene, or anchoring to an off-the-shelf metric depth estimate when no floor is visible.

Load-bearing premise

The load-bearing premise is that the automatically generated depth labels and the later floor-fitting or anchor-based scaling are accurate enough to define the metric 4D frame; if these are systematically off, the metric reconstruction claim weakens even if the body and hand pose results stand.

Editorial extensions

If this is right

  • Egocentric body and hand reconstruction no longer needs precomputed camera trajectories, point clouds, or dedicated hand trackers; raw RGB suffices, which widens the range of casual wearable footage the method can process.
  • Because all modalities share one token sequence, predictions are temporally consistent by construction, so missing or occluded observations, such as hands out of view, can be filled from learned kinematic and scene priors.
  • Inference at 0.7 seconds per two-second clip puts the reconstruction in a range where real-time or near-real-time wearable assistants are conceivable.
  • Scaling helps: the ablations show accuracy rises with model size and with the size of the pretraining corpus, and that joint training on data without body annotations still improves body reconstruction.
  • A metric-aligned 4D coordinate system can be recovered by a simple floor-fitting or anchor step after relative depth prediction, without dense bundle adjustment.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • If the joint-distribution objective really is the source of the gains, the same tokenization could be pretrained on even larger unlabeled egocentric video, treating the depth pseudo-labels as one noisy modality among several rather than as ground truth.
  • A natural stress test would be measuring metric scale error on long recordings with known floor heights; the floor-percentile heuristic is a place where the 4D claim could silently degrade.
  • The quantization bottleneck that limits depth accuracy suggests that a hybrid discrete-continuous decoder, for example a diffusion head for depth, could close the gap to specialized depth models without sacrificing the cross-modal reasoning.
  • If these results transfer to in-the-wild footage beyond the benchmarks, downstream augmented-reality and robotics systems could use one model for egomotion, intent, and scene geometry, removing the need to fuse separate specialist outputs.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. ReViV proposes a unified masked generative transformer (MGET) that takes a single monocular egocentric RGB video and jointly infers body motion, hand motion, gaze, camera trajectory, and affine-invariant depth, with a post-hoc alignment step to obtain a metric 4D reconstruction. The system tokenizes each modality with a VQ-VAE, trains on a 7B-token multimodal corpus assembled from existing datasets plus Video Depth Anything pseudo-labels, and decodes with iterative masked sampling. The paper reports held-out ADT evaluations for body, camera, gaze, and depth, plus hand evaluations on HoloAssist, HOT3D, ARCTIC, and TACO, and claims state-of-the-art accuracy and efficiency for holistic egocentric reconstruction, with code and models open-sourced.

Significance. If the results hold, the paper would be a notable step toward a single feed-forward, hardware-free egocentric 4D reconstruction system: body, hand, gaze, camera, and depth at 0.7 s per 2-second clip, with strong generalization to the held-out ADT benchmark. The main quantitative strengths are the body and hand results, which use standard metrics and beat recent diffusion-based and optimization-based baselines by large margins, and the explicit exclusion of ADT from training. The paper also contains systematic ablations (masking strategy, model capacity, data scaling, ViT branch) and ships open-source code, which increases reproducibility. However, the 'metric 4D' and 'state-of-the-art camera tracking' parts of the central claim are not supported by the reported experiments, and the body comparison protocol is not fully matched.

major comments (4)
  1. [Sec. 4.1, Sec. 5.5, Supplement D] The paper's title and abstract claim a 'metric 4D reconstruction', but the depth evaluation in Table 4 cannot support this. All depth metrics are computed after a clip-wise scale-and-translation alignment to ground truth (Sec. 5.5), which removes global scale and offset errors; the metric alignment step described in Sec. 4.1 and Supplement D (floor percentiles, or VIPE anchors) is never evaluated against the metric ground truth that ADT provides. Since ADT has metric depth and body annotations, the authors should report an absolute-scale error (e.g., metric Abs Rel before alignment, scale-error distribution, or a comparison of the aligned reconstruction against the metric GT) to substantiate the 'metric 4D' claim. As written, the tables remain identical whether the Sec. D alignment is correct or systematically biased.
  2. [Sec. 5.3, Table 4, Abstract] The abstract claims 'state-of-the-art ... camera tracking', but Table 4 reports ATE of 0.015 for ReViV versus 0.005 for VIPE, so VIPE is better on the primary absolute-trajectory metric. The paper acknowledges this gap in Sec. 5.3 but the abstract and contributions overstate the result. The claim should either be revised to 'competitive' or accompanied by evidence that the feedforward setting is the intended point of comparison rather than raw accuracy; the current wording is contradicted by the paper's own table.
  3. [Sec. 5.1, Table 2] The body-motion SOTA claim is weakened by an uneven comparison protocol. EgoAllo and UniEgoMotion are not retrained on the same data; the evaluation protocol states that this is impossible for the authors because their pretraining corpus lacks SMPL-X annotations. However, the reported comparison therefore mixes model architecture, training data, and input modalities (ReViV uses full-image semantics and multi-modal pretraining, while baselines use camera trajectories). To support the 'SOTA from monocular video' claim, the paper should either provide a same-data retrained baseline (e.g., using a smaller annotated subset), report results on an additional unseen dataset, or explicitly frame the comparison as 'out-of-the-box baselines' and quantify the effect of the input-camera information via an ablation that feeds estimated trajectories to ReViV.
  4. [Sec. 3.1, Sec. 5.5] The depth pseudo-labels from Video Depth Anything are used for large-scale geometric pretraining (Sec. 3.1) but their quality is never validated against the ADT ground truth before training. If VDA has biased scale or if the floor-fitting/VIPE anchor fails on sequences without a visible floor, the 'shared metric 4D coordinate system' in Fig. 2 will be inaccurate even though all reported tables are unchanged because of the clip-wise alignment in Sec. 5.5. The authors should add a pseudo-label-quality analysis (e.g., scale-error distribution of VDA on ADT) or explicitly state that the metric 4D claim is conditional on the alignment step's validity, which is currently unverified.
minor comments (5)
  1. [Table 1] The table heading reads 'T able 1' due to a spacing error; please fix the typo.
  2. [Sec. 5.2] For the HoloAssist hand comparison, Dyn-HaMR is evaluated on a random 3,000-clip subset while ReViV and HaMeR are evaluated on the full 27,910-clip validation set; the paper should report the standard error or repeated random splits so that the comparison is not confounded by subset selection.
  3. [Sec. 5.5, Table 4] The depth comparison would benefit from also reporting scale-invariant metrics (e.g., sc-inv or delta thresholds at multiple tolerances) to separate geometric quality from the alignment choice; the current pair of Abs Rel and delta_1.25 is rather coarse.
  4. [Supplement C] The inference details state that classifier-free guidance uses w=2 and top-p=0.8, but no sensitivity analysis is provided for these decoding hyperparameters; adding a small ablation would strengthen reproducibility.
  5. [Eq. (3)] The objective in Eq. (3) excludes the continuous ViT tokens from the prediction loss, but the main text says the model predicts 'all' masked tokens; please clarify the role of the xViT tokens in the masked prediction formulation.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: ReViV's central predictions are validated on held-out ADT data, and the EgoM2P self-citations are transparent prior-work reuse rather than a self-fulfilling argument.

full rationale

The paper's derivation chain is not circular. The depth branch is trained on pseudo-labels from Video Depth Anything (Sec. 3.1) together with public dataset annotations, while the main benchmark, Aria Digital Twin, is explicitly excluded from training: "we conduct all quantitative evaluations on the Aria Digital Twin dataset, which was strictly excluded from our training data." The reported body, hand, gaze, camera, and depth numbers are therefore external validations against held-out ground truth, not algebraic consequences of the training objective. The reuse of EgoM2P from the same research group is visible in the dataset foundation, the gaze/camera tokenizer architecture, and the masked-decoding recipe, but EgoM2P is a prior published system, the gaze and camera tokenizers are retrained from scratch on the new 7B-token corpus, and ReViV is compared directly against EgoM2P on ADT rather than assuming its outputs. No uniqueness theorem, fitted parameter, or defined quantity is imported to force the conclusion. The only substantive concern is that the 'metric 4D' claim is not fully substantiated by the scale-invariant depth metrics and the unvalidated floor/VIPE alignment step (Sec. 4.1 and Supplement D); however, that is a validation gap or correctness risk, not circularity, because the alignment is a transparent post-hoc scale restoration applied after prediction and is not renamed as a predicted output. Score 0 reflects the absence of any step in which the paper's output reduces by construction to its input.

Assumptions & free parameters 5 free parameters · 5 assumptions · 0 invented entities

No new physical entities are introduced. The model rests on standard deep learning components, pseudo-labeled training data, and several hand-chosen hyperparameters. The most consequential assumptions are the validity of the pseudo-labels and the fairness of the baseline comparisons; both are acknowledged only partially.

free parameters (5)
  • VQ-VAE codebook sizes = body: 2048; hand: 1024; gaze: 512; camera: 256
    Chosen by validation loss; the body codebook ablation in Sec. F.3 of the supplement shows 2048 is best while 4096 degrades accuracy.
  • Spatial-temporal compression factors for body/hand tokenizers = temporal: 2; spatial: 3
    Ablation in Sec. F.3 shows the [2,3] configuration outperforms [2,7] and [4,3]; selected by reconstruction error.
  • Inference decoding hyperparameters = steps=3; classifier-free guidance weight=2; top-p=0.8
    Adopted from EgoM2P and chosen by hand; no sensitivity analysis is reported.
  • Floor-fitting alignment thresholds = lowest 50% of feet z; lowest 5% of scene points
    Hand-chosen percentiles in Sec. D of the supplement for metric scale alignment; no robustness study is provided.
  • Mask sampling Dirichlet concentration values = sampled from {0.01, 0.1, 1, 10}
    Algorithm 1 in the supplement draws alpha from this set; the distribution is a design choice with limited ablation.
assumptions (5)
  • ad hoc to paper Masked token prediction over all mask configurations approximates the full joint distribution p(X,Y).
    Sec. 4.3 and Eq. 3 treat the masked cross-entropy objective as a proxy for learning the joint distribution; this is a modeling assumption, not a proven equivalence.
  • domain assumption Video Depth Anything pseudo-labels are accurate and temporally consistent enough to supervise depth training.
    Sec. 3.1 uses VDA-generated depth as ground truth for large-scale pretraining; the paper does not validate pseudo-label quality against held-out metric depth before training.
  • domain assumption The gravity-aligned global body frame built from the first-frame camera up-vector is a valid shared world frame across datasets.
    Sec. 3.2 defines this representation and assumes the first-frame up-vector is ground-parallel and that body and hand annotations from different datasets can be harmonized this way.
  • domain assumption Evaluation on ADT is a fair proxy for generalization and baseline comparisons are meaningful without identical training supervision.
    Sec. 5.1 states that baselines cannot be retrained under identical supervision, so the SOTA comparison assumes the pretraining scale advantage does not make the comparison unfair.
  • domain assumption The pretrained TMR motion encoder provides a valid latent space for computing FID and similarity metrics.
    Sec. 5.1 uses TMR embeddings to measure motion realism and correspondence; this assumes the external encoder is appropriate for egocentric body motions.

how reviews work

0 comments
Cite this review

Pith. "Pith review of ReViV: Reconstructing the Viewer and the View in 4D from Monocular Egocentric Video." pith.science (2026). https://pith.science/paper/K7BPDK6W

@misc{pith2026260717790,
  author       = {Pith},
  title        = {Pith review of: ReViV: Reconstructing the Viewer and the View in 4D from Monocular Egocentric Video},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/K7BPDK6W}},
  note         = {Machine review of arXiv:2607.17790}
}
read the original abstract

Egocentric devices, such as wearable front-facing cameras, provide a unique perspective for capturing the continuous interaction between a human viewer and the surrounding environment. A holistic and efficient multimodal model capable of reconstructing this 4D representation is therefore highly desirable. However, existing approaches often rely on auxiliary inputs such as pre-computed camera trajectories, treat scene perception and human ego-motion modeling as separate problems despite their strong interdependency, and suffer from slow inference time. To address these limitations, we present ReViV, the first unified framework for holistic egocentric 4D reconstruction that extracts both viewer and view dynamics from a single monocular RGB video. We formulate the task as learning the full joint probability distribution over multimodal signals, including RGB video, camera trajectory, gaze direction, full-body motion, hand motion, and depth. Powered by a Masked Generative Egocentric Transformer, ReViV operates within a single feed-forward architecture to simultaneously reconstruct the temporally consistent 4D reconstruction across the viewer and the view with fast inference speed. Extensive experiments on diverse benchmarks, including HoloAssist, HOT3D, ARCTIC, Aria Digital Twin, and TACO, demonstrate that ReViV achieves state-of-the-art accuracy and efficiency across holistic ego-body, hand, and gaze reconstruction, camera tracking, while maintaining highly competitive egocentric depth estimation without relying on heavy task-specific priors. Code and models are fully open-sourced: https://reviv4d.github.io/.

Figures

Figures reproduced from arXiv: 2607.17790 by the authors.

Figure 1
Figure 1. We present ReViV, a unified framework for holistic egocentric 4D reconstruc￾tion. Given a monocular egocentric RGB video, our method jointly estimates human￾centric modalities, including body, hand pose, and gaze, alongside scene-aware cam￾era trajectory and depth estimation. These predictions are integrated into a tem￾porally consistent viewer–view reconstruction from a monocular egocentric video. video has become … view at source ↗
Figure 2
Figure 2. ReViV Architecture. (Left) Modality-specific VQ-VAEs discretize heteroge￾neous viewer (gaze, hand, body) and view (RGB, depth, camera) continuous modalities into a unified token sequence. (Middle) A Masked Generative Egocentric Transformer (MGET) learns the joint probability distribution by predicting randomly masked to￾kens (grey blocks), capturing both intra- and cross-modal dynamics. (Right) During inference, MGE… view at source ↗
Figure 3
Figure 3. Qualitative Results of Egocentric Hand Reconstruction. [PITH_FULL_IMAGE:figures/full_fig_p012_3.png] view at source ↗
Figures from the paper (3 more)
Figure 1
Figure 1. Figure 1: Qualitative visualization of egocentric camera tracking results. [PITH_FULL_IMAGE:figures/full_fig_p023_1.png]
Figure 2
Figure 2. Figure 2: Comparison of camera trajec￾tories projected on 2D plane [PITH_FULL_IMAGE:figures/full_fig_p024_2.png]
Figure 4
Figure 4. Figure 4: Qualitative visualization of egocentric body motion reconstruction. [PITH_FULL_IMAGE:figures/full_fig_p026_4.png]

Review history (2 revisions) →

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

67 extracted references · 55 canonical work pages

  1. [1]

    arXiv preprint arXiv:2501.03575 (2025)

    Agarwal, N., Ali, A., Bala, M., Balaji, Y., Barker, E., Cai, T., Chattopadhyay, P., Chen, Y., Cui, Y., Ding, Y., et al.: Cosmos world foundation model platform for physical ai. arXiv preprint arXiv:2501.03575 (2025)

    arXiv 2025

  2. [2]

    Advances in Neural Information Processing Systems37, 61872–61911 (2024)

    Bachmann, R., Kar, O.F., Mizrahi, D., Garjani, A., Gao, M., Griffiths, D., Hu, J., Dehghan, A., Zamir, A.: 4m-21: An any-to-any vision model for tens of tasks and modalities. Advances in Neural Information Processing Systems37, 61872–61911 (2024)

    2024

  3. [3]

    CVPR (2025)

    Banerjee, P., Shkodrani, S., Moulon, P., Hampali, S., Han, S., Zhang, F., Zhang, L., Fountain, J., Miller, E., Basol, S., Newcombe, R., Wang, R., Engel, J.J., Hodan, T.: HOT3D: Hand and object tracking in 3D from egocentric multi-view videos. CVPR (2025)

    work page 2025

  4. [4]

    In: The IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (June 2022)

    Chang, H., Zhang, H., Jiang, L., Liu, C., Freeman, W.T.: Maskgit: Masked genera- tive image transformer. In: The IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (June 2022)

    work page 2022

  5. [5]

    In: CVPR (2025)

    Chen, S., Guo, H., Zhu, S., Zhang, F., Huang, Z., Feng, J., Kang, B.: Video depth anything: Consistent depth estimation for super-long videos. In: CVPR (2025)

    work page 2025

  6. [6]

    arXiv preprint arXiv:2510.06219 (2025)

    Chen, Y., Chen, X., Xue, Y., Chen, A., Xiu, Y., Pons-Moll, G.: Human3r: Everyone everywhere all at once. arXiv preprint arXiv:2510.06219 (2025)

    arXiv 2025

  7. [7]

    NeurIPS37, 96181–96206 (2024)

    Chu, W.H., Ke, L., Fragkiadaki, K.: Dreamscene4d: Dynamic multi-object scene generation from monocular videos. NeurIPS37, 96181–96206 (2024)

    work page 2024

  8. [8]

    IEEE TPAMI43(11), 4125–4141 (2020)

    Damen, D., Doughty, H., Farinella, G.M., Fidler, S., Furnari, A., Kazakos, E., Molti- santi, D., Munro, J., Perrett, T., Price, W., et al.: The epic-kitchens dataset: Col- lection, challenges and baselines. IEEE TPAMI43(11), 4125–4141 (2020)

    work page 2020

Show all 67 references
  1. [9]

    In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition

    Deshmukh, M., Akada, H., Rhodin, H., Theobalt, C., Golyanik, V.: E-3dpsm: A state machine for event-based egocentric 3d human pose estimation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. pp. 14017–14026 (2026)

    2026

  2. [10]

    Duan, Y., Wei, F., Dai, Q., He, Y., Chen, W., Chen, B.: 4d-rotor gaussian splatting: towardsefficientnovelviewsynthesisfordynamicscenes.In:ACMSIGGRAPH2024 Conference Papers. pp. 1–11 (2024)

    2024

  3. [11]

    In: Pro- ceedings IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (2023)

    Fan, Z., Taheri, O., Tzionas, D., Kocabas, M., Kaufmann, M., Black, M.J., Hilliges, O.: ARCTIC: A dataset for dexterous bimanual hand-object manipulation. In: Pro- ceedings IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (2023)

    2023

  4. [12]

    In: Pro- ceedings of the IEEE/CVF International Conference on Computer Vision

    Feng, H., Zhang, J., Wang, Q., Ye, Y., Yu, P., Black, M.J., Darrell, T., Kanazawa, A.: St4rtrack: Simultaneous 4d reconstruction and tracking in the world. In: Pro- ceedings of the IEEE/CVF International Conference on Computer Vision. pp. 8503– 8513 (2025)

    2025

  5. [13]

    In: CVPR

    Grauman, K., Westbury, A., Byrne, E., Chavis, Z., Furnari, A., Girdhar, R., Ham- burger, J., Jiang, H., Liu, M., Liu, X., et al.: Ego4d: Around the world in 3,000 hours of egocentric video. In: CVPR. pp. 18995–19012 (2022)

    2022

  6. [14]

    In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)

    Grauman, K., Westbury, A., Torresani, L., Kitani, K., Malik, J., Afouras, e.a.: Ego-exo4d: Understanding skilled human activity from first- and third-person per- spectives. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). pp. 19383–...

    2024

  7. [15]

    In: 2025 International Conference on 3D Vision (3DV)

    Guzov, V., Jiang, Y., Hong, F., Pons-Moll, G., Newcombe, R., Liu, C.K., Ye, Y., Ma, L.: Hmd 2: Environment-aware motion generation from single egocentric head-mounted device. In: 2025 International Conference on 3D Vision (3DV). pp. 1394–1405. IEEE (2025) ReViV: Reconstructing...

    2025

  8. [16]

    International Journal of Computer Vision134(2), 62 (2026)

    He, Y., Huang, Y., Chen, G., Lu, L., Pei, B., Xu, J., Lu, T., Sato, Y.: Bridg- ing perspectives: A survey on cross-view collaborative intelligence with egocentric- exocentric vision. International Journal of Computer Vision134(2), 62 (2026)

    2026

  9. [18]

    In: CVPR

    Jiang, H., Grauman, K.: Seeing invisible poses: Estimating 3d body pose from egocentric video. In: CVPR. pp. 3501–3509. IEEE (2017)

    2017

  10. [19]

    In: ICCV

    Jiang, H., Ithapu, V.K.: Egocentric pose estimation from human vision span. In: ICCV. pp. 10986–10994. IEEE (2021)

    2021

  11. [20]

    In: Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV)

    Kwon, T., Tekin, B., Stühmer, J., Bogo, F., Pollefeys, M.: H2o: Two hands ma- nipulating objects for first person interaction recognition. In: Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV). pp. 10138–10148 (October 2021)

    2021

  12. [21]

    In: CVPR

    Lei, J., Weng, Y., Harley, A.W., Guibas, L., Daniilidis, K.: Mosca: Dynamic gaus- sian fusion from casual videos via 4d motion scaffolds. In: CVPR. pp. 6165–6177 (2025)

    2025

  13. [23]

    In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)

    Li, G., Zhao, K., Zhang, S., Lyu, X., Dusmanu, M., Zhang, Y., Pollefeys, M., Tang, S.: Egogen: An egocentric synthetic data generator. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). pp. 14497–14509 (June 2024)

    2024

  14. [24]

    In: CVPR

    Li, J., Liu, K., Wu, J.: Ego-body pose estimation via ego-head pose estimation. In: CVPR. pp. 17142–17151 (2023)

    2023

  15. [25]

    In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition

    Li, Z., Dwivedi, S.K., Maric, F., Chacon, C., Bertsch, N., Arcadu, F., Hodan, T., Ramamonjisoa, M., Wonka, P., Zhao, A., et al.: Egoposeformer v2: Accurate egocentric human motion estimation for ar/vr. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern R...

    2026

  16. [26]

    In: The Thirteenth International Conference on Learning Representations (2025)

    LIU, Q., Liu, Y., Wang, J., Lyu, X., Wang, P., Wang, W., Hou, J.: MoDGS: Dynamic gaussian splatting from casually-captured monocular videos with depth priors. In: The Thirteenth International Conference on Learning Representations (2025)

    2025

  17. [27]

    arXiv preprint arXiv:2401.08399 (2024)

    Liu, Y., Yang, H., Si, X., Liu, L., Li, Z., Zhang, Y., Liu, Y., Yi, L.: Taco: Bench- marking generalizable bimanual tool-action-object understanding. arXiv preprint arXiv:2401.08399 (2024)

    2024 arXiv

  18. [28]

    ACM Trans

    Loper, M., Mahmood, N., Romero, J., Pons-Moll, G., Black, M.J.: SMPL: A skinned multi-person linear model. ACM Trans. Graphics (Proc. SIGGRAPH Asia) 34(6), 248:1–248:16 (Oct 2015)

    2015

  19. [29]

    In: CVPR

    Lu, J., Huang, T., Li, P., Dou, Z., Lin, C., Cui, Z., Dong, Z., Yeung, S.K., Wang, W., Liu, Y.: Align3r: Aligned monocular depth estimation for dynamic videos. In: CVPR. pp. 22820–22830 (2025)

    2025

  20. [30]

    In: 2024 International Conference on 3D Vision (3DV)

    Luiten, J., Kopanas, G., Leibe, B., Ramanan, D.: Dynamic 3d gaussians: Track- ing by persistent dynamic view synthesis. In: 2024 International Conference on 3D Vision (3DV). pp. 800–809. IEEE (2024)

    2024

  21. [31]

    NeurIPS34, 25019–25032 (2021) 18 X

    Luo, Z., Hachiuma, R., Yuan, Y., Kitani, K.: Dynamics-regulated kinematic policy for egocentric pose estimation. NeurIPS34, 25019–25032 (2021) 18 X. Lyu et al

    2021

  22. [32]

    In: ECCV

    Ma, L., Ye, Y., Hong, F., Guzov, V., Jiang, Y., Postyeni, R., Pesqueira, L., Gamino, A., Baiyya, V., Kim, H.J., et al.: Nymeria: A massive collection of multimodal egocentric daily motion in the wild. In: ECCV. pp. 445–465. Springer (2024)

    2024

  23. [33]

    In: CVPR

    Ng,E.,Xiang,D.,Joo,H.,Grauman,K.:You2me:Inferringbodyposeinegocentric video via first and second person interactions. In: CVPR. pp. 9890–9900 (2020)

    2020

  24. [34]

    In: Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV)

    Pan, X., Charron, N., Yang, Y., Peters, S., Whelan, T., Kong, C., Parkhi, O., Newcombe,R.,Ren,Y.C.:Ariadigitaltwin:Anewbenchmarkdatasetforegocentric 3d machine perception. In: Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV). pp. 20133–20143 (Octo...

    2023

  25. [35]

    In: NeurIPS (2025)

    Pani, A., Yang, Y.: Gaze-vlm: Bridging gaze and vlms through attention regular- ization for egocentric understanding. In: NeurIPS (2025)

    2025

  26. [37]

    In: Proceedings IEEE Conf

    Pavlakos, G., Choutas, V., Ghorbani, N., Bolkart, T., Osman, A.A.A., Tzionas, D., Black, M.J.: Expressive body capture: 3D hands, face, and body from a single image. In: Proceedings IEEE Conf. on Computer Vision and Pattern Recognition (CVPR). pp. 10975–10985 (2019)

    2019

  27. [38]

    In: CVPR (2024)

    Pavlakos, G., Shan, D., Radosavovic, I., Kanazawa, A., Fouhey, D., Malik, J.: Reconstructing hands in 3D with transformers. In: CVPR (2024)

    2024

  28. [39]

    Petrovich, M., Black, M.J., Varol, G.: Tmr: Text-to-motion retrieval using con- trastive3dhumanmotionsynthesis.In:ProceedingsoftheIEEE/CVFInternational Conference on Computer Vision. pp. 9488–9497 (2023)

    2023

  29. [40]

    In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) (2024)

    Piccinelli, L., Yang, Y.H., Sakaridis, C., Segu, M., Li, S., Van Gool, L., Yu, F.: UniDepth: Universal monocular metric depth estimation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) (2024)

    2024

  30. [41]

    Journal of Machine Learning Research21(140), 1–67 (2020)

    Raffel, C., Shazeer, N., Roberts, A., Lee, K., Narang, S., Matena, M., Zhou, Y., Li, W., Liu, P.J.: Exploring the limits of transfer learning with a unified text-to-text transformer. Journal of Machine Learning Research21(140), 1–67 (2020)

    2020

  31. [42]

    Computer Vision and Image Understand- ing211, 103252 (2021)

    Rodin, I., Furnari, A., Mavroeidis, D., Farinella, G.M.: Predicting the future from first person (egocentric) vision: A survey. Computer Vision and Image Understand- ing211, 103252 (2021)

    2021

  32. [43]

    In: SIGGRAPH Asia

    Stearns, C., Harley, A., Uy, M., Dubost, F., Tombari, F., Wetzstein, G., Guibas, L.: Dynamic gaussian marbles for novel view synthesis of casual monocular videos. In: SIGGRAPH Asia. pp. 1–11 (2024)

    2024

  33. [44]

    IEEE Transactions on Pattern Analysis and Machine Intelligence45(6), 6794–6806 (2020)

    Tome, D., Alldieck, T., Peluse, P., Pons-Moll, G., Agapito, L., Badino, H., De la Torre, F.: Selfpose: 3d egocentric pose estimation from a headset mounted camera. IEEE Transactions on Pattern Analysis and Machine Intelligence45(6), 6794–6806 (2020)

    2020

  34. [45]

    In: ICCV

    Tome, D., Peluse, P., Agapito, L., Badino, H.: xr-egopose: Egocentric 3d human pose from an hmd camera. In: ICCV. pp. 7728–7738 (2019)

    2019

  35. [46]

    In: ICCV

    Wang, J., Liu, L., Xu, W., Sarkar, K., Theobalt, C.: Estimating egocentric 3d human pose in global space. In: ICCV. pp. 11500–11509 (2021)

    2021

  36. [47]

    In: ICCV

    Wang, Q., Ye, V., Gao, H., Zeng, W., Austin, J., Li, Z., Kanazawa, A.: Shape of motion: 4d reconstruction from a single video. In: ICCV. pp. 9660–9672 (2025)

    2025

  37. [48]

    In: CVPR

    Wang, Q., Zhang, Y., Holynski, A., Efros, A.A., Kanazawa, A.: Continuous 3d perception model with persistent state. In: CVPR. pp. 10510–10522 (2025) ReViV: Reconstructing the Viewer and the View in 4D from Ego Video 19

    2025

  38. [49]

    Wang, S., Yang, X., Shen, Q., Jiang, Z., Wang, X.: Gflow: recovering 4d world from monocular video. In: Proceedings of the Thirty-Ninth AAAI Conference on Artificial Intelligence and Thirty-Seventh Conference on Innovative Applications of Artificial Intelligence and Fifteenth ...

    2025

  39. [50]

    In: Pro- ceedings of the IEEE/CVF International Conference on Computer Vision (ICCV)

    Wang, X., Kwon, T., Rad, M., Pan, B., Chakraborty, I., Andrist, S., Bohus, D., Fe- niello, A., Tekin, B., Frujeri, F.V., Joshi, N., Pollefeys, M.: Holoassist: an egocentric human interaction dataset for interactive ai assistants in the real world. In: Pro- ceedings of the IEEE...

    2023

  40. [51]

    In: CVPR

    Wu, G., Yi, T., Fang, J., Xie, L., Zhang, X., Wei, W., Liu, W., Tian, Q., Wang, X.: 4d gaussian splatting for real-time dynamic scene rendering. In: CVPR. pp. 20310–20320 (2024)

    2024

  41. [52]

    arXiv preprint arXiv:2507.12462 (2025)

    Xiao, Y., Wang, J., Xue, N., Karaev, N., Makarov, Y., Kang, B., Zhu, X., Bao, H., Shen, Y., Zhou, X.: Spatialtrackerv2: 3d point tracking made easy. arXiv preprint arXiv:2507.12462 (2025)

    2025 arXiv

  42. [53]

    IEEE Robotics and Automation Letters8(10), 6635–6642 (2023)

    Xin, J., Wang, L., Xu, K., Yang, C., Yin, B.: Learning interaction regions and motion trajectories simultaneously from egocentric demonstration videos. IEEE Robotics and Automation Letters8(10), 6635–6642 (2023)

    2023

  43. [54]

    IEEE Transactions on Visualization and Computer Graphics25(5), 2093– 2101 (2019)

    Xu, W., Chatterjee, A., Zollhoefer, M., Rhodin, H., Fua, P., Seidel, H.P., Theobalt, C.: Mo 2 cap 2: Real-time mobile 3d motion capture with a cap-mounted fisheye camera. IEEE Transactions on Visualization and Computer Graphics25(5), 2093– 2101 (2019)

    2019

  44. [55]

    arXiv preprint arXiv:2506.08015 (2025)

    Xu, Z., Li, Z., Dong, Z., Zhou, X., Newcombe, R., Lv, Z.: 4dgt: Learning a 4d gaussian transformer using real-world monocular videos. arXiv preprint arXiv:2506.08015 (2025)

    2025

  45. [56]

    In: CVPR

    Yang, Z., Gao, X., Zhou, W., Jiao, S., Zhang, Y., Jin, X.: Deformable 3d gaussians for high-fidelity monocular dynamic scene reconstruction. In: CVPR. pp. 20331– 20341 (2024)

    2024

  46. [58]

    In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (2023)

    Yu, L., Cheng, Y., Sohn, K., Lezama, J., Zhang, H., Chang, H., Hauptmann, A.G., Yang, M.H., Hao, Y., Essa, I., Jiang, L.: MAGVIT: Masked generative video trans- former. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (2023)

    2023

  47. [59]

    In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) (June 2025)

    Yu, Z., Zafeiriou, S., Birdal, T.: Dyn-hamr: Recovering 4d interacting hand motion from a dynamic camera. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) (June 2025)

    2025

  48. [61]

    In: ICCV

    Yuan, Y., Kitani, K.: Ego-pose estimation and forecasting as real-time pd control. In: ICCV. pp. 10082–10092 (2019)

    2019

  49. [62]

    In: NeurIPS (2025)

    Yun,H.,Na,J.,Kim,J.,Murdock,C.,Kim,G.:Gazebeyondtheframe:Forecasting egocentric 3d visual span. In: NeurIPS (2025)

    2025

  50. [63]

    rgb” denotes discrete video tokens, whereas “ViT

    Zhang, D., Li, G., Li, J., Bressieux, M., Hilliges, O., Pollefeys, M., Van Gool, L., Wang, X.: Egogaussian: Dynamic scene understanding from egocentric video with 3d gaussian splatting. In: 2025 International Conference on 3D Vision (3DV). pp. 1091–1102. IEEE (2025) ReViV: Rec...

    2025

  51. [64]

    w/ VIPE cam Frame 05 Frame 30 Frame 55 Input GT Ours EgoAllo [5] w/ GT cam UniEgoMotion

  52. [65]

    4: Qualitative visualization of egocentric body motion reconstruction

    w/ GT cam Frame 05 Frame 30 Frame 55 Fig. 4: Qualitative visualization of egocentric body motion reconstruction. ReViV achieves state-of-the-art reconstruction accuracy and semantic alignment using purely monocular egocentric video input. Unlike recent baselines that produce u...

  53. [66]

    In: NVIDIA Research Whitepapers arXiv:2508.10934 (2025)

    Huang, J., Zhou, Q., Rabeti, H., Korovko, A., Ling, H., Ren, X., Shen, T., Gao, J., Slepichev, D., Lin, C.H., Ren, J., Xie, K., Biswas, J., Leal-Taixe, L., Fidler, S.: Vipe: Video pose engine for 3d geometric perception. In: NVIDIA Research Whitepapers arXiv:2508.10934 (2025)

    2025 arXiv

  54. [67]

    In: Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV)

    Li, G., Chen, Y., Wu, Y., Zhao, K., Pollefeys, M., Tang, S.: Egom2p: Egocentric multimodal multitask pretraining. In: Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV). pp. 10830–10843 (October 2025)

    2025

  55. [68]

    In: International Conference on Learning Representations (2019)

    Loshchilov, I., Hutter, F.: Decoupled weight decay regularization. In: International Conference on Learning Representations (2019)

    2019

  56. [69]

    In: Proceedings of the IEEE/CVF International Conference on Computer Vision

    Patel, C., Nakamura, H., Kyuragi, Y., Kozuka, K., Niebles, J.C., Adeli, E.: Uniego- motion: A unified model for egocentric motion reconstruction, forecasting, and gen- eration. In: Proceedings of the IEEE/CVF International Conference on Computer Vision. pp. 10318–10329 (2025)

    2025

  57. [70]

    In: Pro- ceedings of the Computer Vision and Pattern Recognition Conference

    Yi, B., Ye, V., Zheng, M., Li, Y., Müller, L., Pavlakos, G., Ma, Y., Malik, J., Kanazawa, A.: Estimating body and hand motion in an ego-sensed world. In: Pro- ceedings of the Computer Vision and Pattern Recognition Conference. pp. 7072– 7084 (2025)

    2025

  58. [71]

    In: Interna- tional Conference on Learning Representations (2022)

    Yu, J., Li, X., Koh, J.Y., Zhang, H., Pang, R., Qin, J., Ku, A., Xu, Y., Baldridge, J., Wu, Y.: Vector-quantized image modeling with improved VQGAN. In: Interna- tional Conference on Learning Representations (2022)

    2022

  59. [72]

    arXiv preprint arXiv:2411.09145 (2024)

    Yuan, C., Chen, G., Yi, L., Gao, Y.: Self-supervised monocular 4d scene recon- struction for egocentric videos. arXiv preprint arXiv:2411.09145 (2024)

    2024 arXiv

Pith tools

Reviewed August 15, 2026 · model on record in the stance chip above.