Pith.

Computer Vision and Pattern Recognition · arXiv:2611.08477 v3 · submitted 5 Nov 2026

EgoGeneralist: Unifying Egocentric Action-Video Tasks through Video-Native Modeling

Yanlin Jin, Bowen Yang, Cheng Zhang, Yifan Yang, Zhe Li, Jianfei Cai, Kai Zhu

The paper says

“Modern video models possess strong visual priors, yet do not natively represent human action in metric 3D space. We present EgoGeneralist, which adapts pretrained video generators to unify action-conditioned video generation, pose estimation, and joint action-video generation for egocentric manipulation.”

more of the abstract

Our role-adaptive action renderer provides video-native representations for multidirectional action-video learning, making metric motion usable as both a spatial control signal and a recoverable generation target. Cross-stream action-video coupling establishes close correspondence between metric motion and visual dynamics through lightweight adaptation while preserving the pretrained visual prior. Together, these designs establish a generalist framework spanning six action-video tasks. With only narrow-domain data, EgoGeneralist generalizes to diverse unseen scenes and interactions with effective action control and visual quality, bridging pretrained visual knowledge and metric 3D human motion.

from the abstract

Pith says

Machine review · deepseek-v4-flash · 6 Nov 2026 · of version 1 of the paper; version 3 is newer · what was read was not recorded · not checked by a person

It can be wrong. Challenge this review or reply to it.

Holds up, with conditions. A genuinely unified egocentric action-video architecture with an honest ablation story; the headline pose numbers should be read as near-ceiling in-domain readout, not a 4x win over specialists. This line is wrong

3 major objections · 7 minor

How often objections from reviews like this one are right: not yet measured

Are you an author? What you can do.

Not checked: figures, code, what the references say.

The paper says, at §4.4, Table 3; Eq. (5); Supp. A.1

The review did not quote this passage.

Pith says: Objection 1 of 3 · major

The headline in-domain pose result — hand Abs-MPJPE 27.73 mm on EgoDex versus 111.98 mm for HaWoR — is not a protocol-symmetric comparison, and the paper's own text and numbers make the asymmetry visible. The paper states only that 'skeletal definitions differ' and restricts body evaluation to shared upper-limb joints; how the MANO-based hand outputs of HaWoR, HaMeR, MINT, and …

more of objection 1

… ReViV were mapped into the EgoDex hand-joint space that defines the reference is never specified, whereas EgoGeneralist's Posegrid decoder (Eq. 5, Supp. A.1) is trained on EgoDex ground-truth Posegrid renderings and outputs joints and camera natively in the GT convention, making it structurally immune to re-targeting error. The same asymmetry affects absolute and world metrics: EgoGeneralist's camera is supervised directly in the GT coordinate frame while HaWoR's is SLAM-derived, so Abs- and W-MPJPE conflate camera-estimation quality with hand-pose quality. The authors appropriately acknowledge in the Limitations that EgoDex multi-camera fusion introduces projection mismatches; the unresolved point is that the in-domain margin is best explained as readout of the training annotation pipeline, which the paper's own numbers support: the GT-readout ceiling in Table 5 is 21.67 mm Abs on EgoDex, within about 6 mm of the full-pipeline 27.73 mm, and on Xperience-10M the Abs advantage shrinks to 19.4 mm while PA-MPJPE reverses (12.42 vs. 8.38 in HaWoR's favor). Please specify the hand re-targeting and matched-joint protocol (or evaluate all methods on a shared joint set), separate local-pose metrics from camera-dependent metrics, and revise the corresponding claims in §4.4 and Contribution 3 accordingly.

The paper says, at §4.5, Table 4; Supp. B.1

The review did not quote this passage.

Pith says: Objection 2 of 3 · major

The HOT3D world-action comparison requires a precise statement of what each method receives as its initial state. Supp. B.1 says the missing HOT3D body state is completed using EgoDex pose and bone-length statistics, described as 'an approximate initialization that need not be accurate,' and Table 4 notes only that VITRA and EgoGeneralist 'also receive initial action states.'

more of objection 2

As reported, EgoGeneralist conditions on the completed body state together with the initial observation, while the content and provenance of VITRA's and Being-H0's initial states are not specified, so the information available to each method is not established to be comparable. Please specify the exact initial-state dimensions and source for each baseline, and provide a sensitivity analysis of the headline Reach@2 gap (58.59% vs. 31.31%) with respect to the accuracy of the completed body state, so that the result can be attributed to the model rather than to the initialization protocol.

The paper says, at §4.5, Fig. 6; Conclusion

The review did not quote this passage.

Pith says: Objection 3 of 3 · major

Text-only joint action-video generation is one of the five claimed tasks and the basis for the statement that the framework 'offer[s] a new means of action-video data generation,' yet its support is qualitative only; the authors disclose this ('we qualitatively examine'), but the abstract and contribution list present it as an established capability.

more of objection 3

Please add at least one quantitative protocol for this task — for example, decoding the generated Posegrid and measuring action-video alignment or motion plausibility with an existing metric such as EgoVideo-S — or explicitly scope the claim in the abstract and conclusions to a qualitative demonstration.

7 minor objections

The paper says, at Table 3 (caption)

Pith says: minor objection 1 of 7

The 'Fail' legend in the Table 3 caption ('no failure / some fail / highest failure rate') is garbled; please state precisely what the failure statistic counts (frames, clips, or predictions), over what denominator, and which direction is better.

The paper says, at §4.2

Pith says: minor objection 2 of 7

The sentence '111 clips randomly sampled from the EgoDex test split, one per activity category' describes stratified selection rather than simple random sampling; please rephrase and state the selection procedure for reproducibility.

show 5 more

The paper says, at Fig. 2

Pith says: minor objection 3 of 7

The target-gating notation ('Condition OFF / Target ON') in Fig. 2 is difficult to parse; an explicit legend linking the gating symbols to the stream roles in Eqs. (1)-(3) would clarify the mechanism.

The paper says, at §3.2, Eq. (4)

Pith says: minor objection 4 of 7

The anchor-cage depths (0.25 m and 0.70 m) are fixed design choices; a sentence on how they were chosen, or a brief sensitivity note, would help readers adapt the method to other camera geometries.

The paper says, at Ref. [13]

Pith says: minor objection 5 of 7

The EgoDex reference lists 'Mouli sivapurapu'; the name should be capitalized.

The paper says, at Supp. A

Pith says: minor objection 6 of 7

Please report the total number of episodes in the full one-epoch training run, so that the '0.65% of the full training episodes' figure used to describe the ablations in Supp. C can be placed in context.

The paper says, at §4.3, Table 1

Pith says: minor objection 7 of 7

FVD is reported on benchmarks of 111 and 50 clips with no uncertainty estimate; since all methods share the identical clip set, a bootstrap confidence interval or a brief note on variance would strengthen the comparison.

What would settle it

Recommend one recomputation: re-run Table 3's hand metrics on both datasets with an explicitly stated, symmetric protocol. (i) Define a common joint set containing only joints that exist natively in both the EgoDex hand skeleton and each baseline output space (e.g., MCP/PIP joints, excluding fingertips and wrist where definitions differ), apply the identical re-targeting transform to all methods, and state it; alternatively transform the EgoDex GT into each baseline's native space and evaluate there. (ii) Report errors both including and excluding failure frames, since HaWoR's 9.65% EgoDex failure rate is excluded from its error by default. (iii) Report the out-of-distribution readout ceiling: render GT Xperience-10M poses as Posegrid, pass through the frozen VAE and decoder D, and compare that Abs-MPJPE with the 91.01 mm obtained from video.

from the skeptic's reading

How the review went

4 readings by deepseek-v4-flash · 6 Nov 2026

  1. Reader holds up, with conditions confidence moderate
  2. Skeptic partly agrees 2 min 38 s · 23,647 tokens read, 23,672 written
  3. Editor wrote the verdict above 34 s · 25,487 tokens read, 1,849 written
  4. Referee major revision: 3 major objections, 7 minor 2 min 19 s · 24,130 tokens read, 22,382 written

Not run on this paper: the simulated authors' defence.

Pith did not check this review's objections against the paper.

The review in full

The review in brief, machine-written

Short version: this is a real architecture paper. The new piece is role-adaptive rendering - an ego-view skeleton plus a fixed 3D anchor cage for conditioning, and a multiview Posegrid for metric readout - paired with cross-stream attention and target-gated LoRA inside a frozen video DiT. That combination is not in the cited work, and it buys six inference modes over five tasks without per-task denoisers. The related work is honest about hand-only unified models, the ablations run on matched budgets, and the evaluation clips are fixed and shared across methods. That is real effort and it shows.

The main soft spot is the pose comparison, and the stress-test note lands. The 27.73 vs 111.98 mm EgoDex hand Abs-MPJPE is not protocol-symmetric. EgoGeneralist's Posegrid decoder is trained on EgoDex GT renderings and outputs joints and camera natively in that coordinate convention, while the MANO-based baselines must be re-targeted into EgoDex hand-joint space with the mapping left unspecified, and their cameras come from SLAM. The paper's own OOD numbers confirm the asymmetry: on Xperience-10M the Abs gap shrinks to 19.4 mm and PA-MPJPE reverses (12.42 vs 8.38). The supported claim is near-ceiling in-domain readout plus competitive OOD behavior, not a 4x superiority over specialists. That needs a stated re-targeting protocol, matched camera handling, and ideally error bars - currently no table has them, so several headline differences could be within noise.

The other soft spots are ordinary. No code or model release, so the two strongest generalization results (OOD generation and HOT3D reaching) cannot be independently checked. The Xperience-10M benchmark filtering is described but not quantified. The HOT3D body initialization is approximate, and the authors say so. None of these sink the paper. The authors also acknowledge EgoDex projection mismatches in the Limitations, which is genuine but is a different issue from the pose protocol asymmetry.

The citation pattern is fine. The self-citations to cross-domain LoRA designs are relevant, and the related work does not hide the existence of hand-only unified systems.

Bottom line: this deserves a serious referee. The main revision should be a pose-evaluation protocol appendix and a variance-aware comparison, not a rewrite of the method.

The strongest objection

Table 3's pose results — the paper's most striking instance, hand Abs-MPJPE 27.73 mm vs 111.98 mm for HaWoR on EgoDex — rest on a comparison that is not protocol-symmetric, and the asymmetry is visible in the paper's own text and numbers. First, the paper states only that 'skeletal definitions differ' and that body evaluation uses shared upper-limb joints; how hand outputs from MANO-based methods (HaWoR, HaMeR, MINT, ReViV) were mapped into the EgoDex hand-joint space that defines the reference is never specified. EgoGeneralist's Posegrid decoder (Eq. 5, Supp. A.1) is trained on EgoDex GT Posegrid renderings and outputs joints and camera natively in the GT convention, so it is structurally immune to re-targeting error that every baseline must pay. The same asymmetry applies to absolute and world metrics: EgoGeneralist's camera is supervised in the GT coordinate convention while HaWoR's is SLAM-derived, so Abs-/W-MPJPE comparisons mix camera-estimation quality with hand-pose quality. Second, the in-domain margin is best explained as readout of the training annotation pipeline rather than general pose-estimation superiority, and the paper's own OOD rows confirm this: on Xperience-10M the Abs-MPJPE advantage shrinks to 19.4 mm and PA-MPJPE reverses (12.42 mm vs HaWoR 8.38 mm), with W-A near parity. Third, the GT-readout ceiling (21.67 mm Abs at a reduced decoder budget, Tab. 5) sits close to the full-pipeline 27.73 mm in-domain, while the corresponding OOD ceiling is not reported, leaving open how much of the 91.01 mm OOD Abs error is representation loss. The authors acknowledge EgoDex projection mismatches (Limitations), but that is a different issue from the unaddressed hand-joint and camera-coordinate protocol asymmetry. The claim that is actually supported is 'competitive OOD and near-ceiling in-domain readout,' not a 4x superiority over specialists.

The central claim has three load-bearing conditions: (1) action-conditioned generation reflects real control, not just base-model quality; (2) metric readout genuinely recovers action from video rather than reproducing the training annotation convention; (3) the OOD policy result is robust to benchmark construction. Condition (2) is the least secure, and it is where the paper's most striking instance lives. The reader's weakest_assumption (annotation-rendering accuracy) is related but author-acknowledged (Limitations, b194-b195); my concern is the unacknowledged protocol asymmetry of Table 3. The paper says only that upper-limb evaluation uses shared joints, saying nothing about hand-joint mapping; since EgoGeneralist's decoder is trained on the very EgoDex GT convention used as the reference, the baseline-vs-proposed comparison is structurally favorable. The paper's own Xperience-10M rows are the strongest evidence: the Abs advantage drops from 84 mm to 19 mm and PA-MPJPE reverses (12.42 vs 8.38, with MINT at 7.90), consistent with in-domain annotation-pipeline readout rather than general superiority. The reader's stated reasons (missing error bars, no artifacts, benchmark construction) are real but secondary; a protocol artifact would be a correctness issue, not just a verifiability issue, and the matched-joint recomputation I propose settles it directly, without asking authors to re-prove any cited result. I credit the paper's independent support: the anchor ablation (Xperience PSNR 19.67 to 15.78 without the cage), the cross-stream-coupling ablations (Tab. 6), the transparent disclosure of separate I2V/T2V checkpoints (Supp. A), and the internally consistent near-ceiling in-domain pipeline. None of this changes the disposition: the unification architecture, the generation results, and the HOT3D policy result remain, so the verdict stays CONDITIONAL. What changes is the specific load-bearing condition: Table 3's hand numbers must be re-verified under a symmetric joint/coordinate protocol, and the pose contribution should be phrased as competitive-OOD rather than surpassing specialists by a large factor, unless the recomputation preserves the gap.

The referee's report

This is a substantial and well-constructed framework paper. The central idea — representing metric 3D action as video so a frozen pretrained generator can consume it as a condition and produce it as a target — is coherent, novel, and convincingly ablated: the authors report matched-budget ablations for the rendering and coupling designs (Supp. Tab. 6), an inference-time anchor removal that degrades Xperience PSNR from 19.67 to 15.78 (Tab. 1), an ablation trained without the anchor (Supp. C.1), and a Posegrid readout ceiling (Tab. 5) that lets the reader bound the readout contribution to the pose results. The out-of-distribution action-conditioned generation results are the strongest part of the evidence and support the generalization claim. The main weaknesses are the pose-estimation comparison in Table 3, which is not protocol-symmetric as presented, and the qualitative-only support for text-only joint generation. Both are repairable within the manuscript's scope; if repaired, the paper would be a strong contribution to egocentric world modeling.

Pith's reading of the paper (a machine's summary, not the authors')

EgoGeneralist aims to show that a pretrained video generator can become a generalist for egocentric action and video: given a prescribed metric 3D action it generates the matching first-person video, given a video it recovers the metric 3D action (upper-body joints, hands, and camera pose), and given only text it generates both in sync. The central move is to represent action as video instead of as a separate numeric code, so the action flows through the same frozen video encoder as the RGB frames. A role-adaptive renderer draws the action as an ego-view map with a fixed 3D anchor cage when it is a condition, and as a four-panel orthographic Posegrid when it is a target to be generated; shared attention plus task-gated low-rank adapters couple the two latent streams while leaving the pretrained backbone untouched. Trained only on controlled laboratory recordings, the model reportedly generalizes to unseen scenes and beats task-specific baselines on several benchmarks, e.g. 27.73 mm hand error versus 111.98 mm for a specialist hand tracker on the same test set, and 58.59% versus 31.31% target-reaching on an unseen policy benchmark. If these results hold, one pretrained model could replace separate systems for action-conditioned video synthesis, egocentric pose estimation, and paired action-video data generation.

What carries the argument. The load-bearing object is the role-adaptive action renderer paired with cross-stream action–video coupling. Given a metric 3D action sequence $A=\{a_t\}_{t=1}^{F}$ (joint positions and camera pose in the initial camera frame), the renderer produces one of two video representations: an ego-view action map, which projects the skeleton into the image plane of the RGB video and appends a fixed 3D anchor cage reprojected at each timestep so that camera translation and rotation become visible as anchor motion, or a multiview Posegrid, a 2x2 canvas of front, side, and top orthographic skeleton views plus the egocentric view that keeps 3D geometry explicit. These renderings are encoded by the frozen video VAE as a second latent stream beside the RGB stream, and a shared diffusion transformer lets each stream attend to the other's keys and values while task-gated LoRA adapters, active only for target streams, learn modality-specific denoising. A frozen action decoder, trained on ground-truth Posegrid latents, maps generated latents back to metric 3D joints and camera pose. This arrangement is what lets one pipeline act as video generator, pose estimator, and joint sampler without task-specific networks.

Core claim. The paper's central claim is that metric 3D human action can be made a native input and output of a pretrained video diffusion model without retraining the backbone or adding conditioning encoders. The model renders the action differently according to its role: an ego-view skeleton plus a fixed 3D anchor cage when the action conditions generation, so the generator sees the motion in the same image plane as the video and the anchor's changing projection disambiguates camera motion; and a multiview Posegrid when the action is a target, because only the complementary orthographic views preserve the depth and articulation needed for metric readout. A shared frozen video VAE turns both the rendering and the RGB video into latent streams that a shared diffusion transformer denoises with joint self-attention across streams and low-rank adapters activated only for the stream being generated, trained by a single flow-matching objective across all three directions: action to video, video to action, and text to synchronized action plus video. The same setup, the authors report, produces the best or near-best scores on five evaluation settings, including out-of-distribution scenes never seen in training.

The premise it rests on. The whole method rests on the 3D action annotations attached to each training video frame being geometrically aligned with what the video actually shows, since both the condition that drives generation and the target that teaches pose recovery are rendered straight from those annotations — and the paper itself acknowledges that EgoDex's multi-camera fusion can misalign them.

If the paper is right (Pith's inferences, not the authors')

Action-conditioned video generation and metric pose estimation stop being separate models: the same checkpoint generates video from motion and recovers motion from video, with no specialized encoders or per-task denoisers.

The pretrained visual prior transfers beyond the training domain: adapted only on laboratory EgoDex clips, the model keeps control and visual quality on in-the-wild scenes, and the 3D anchor cage demonstrably carries the camera control (removing it at inference drops out-of-distribution PSNR from 19.67 to 15.78).

Pose recovery is competitive with specialists while being joint: on EgoDex the reported hand Abs-MPJPE of 27.73 mm and the lowest upper-limb errors among compared baselines come from the same model that also predicts hand, body, and camera motion, with zero failed pose-prediction frames.

Text-only joint generation opens a route to synthetic paired action–video data: from a prompt alone the model emits synchronized video and metric motion in scenes far outside its training distribution.

Interaction knowledge needed for target-reaching emerges without robotics training data: on the unseen HOT3D benchmark the model's Reach@2 of 58.59% more than doubles the 31.31% of the VITRA policy baseline while jointly generating video.

A stress test the paper does not run: take a known synthetic action, render it into a Posegrid, and require the frozen decoder to return it exactly; comparing that error to the 27.73 mm reported on real video would isolate how much of the pose error comes from the rendering-decoder loop rather than from image-to-latent generation.

If reprojecting fixed 3D anchors is what makes camera motion visible to the generator, the same idea could extend to static object or scene anchors, adding object-level viewpoint control and occlusion handling to a single ego-view condition; the paper does not explore that direction.

Because the authors concede that EgoDex's multi-camera fusion can misalign 3D annotations with pixels, retraining on a synthetic ego dataset with perfect annotation–image alignment would quantify how much of the reported performance is currently capped by annotation noise.

The text-only joint generation results are qualitative; decoding the generated action and measuring its agreement with the motion visible in the generated video would turn the data-synthesis promise into a measurable claim about paired-data quality.

Check this review

Pith stored no record of this review's passes and took no fingerprint of it. Reviews made by code from 13 Nov 2026 on carry both.

How often reviews like this one are right

Reviews like this one: written by deepseek-v4-flash, where what was read was not recorded.

How often objections from reviews like this one are right: not yet measured

Every value above, in the review's public record (JSON)

Reviews by people

This review was written by a machine. No person has reviewed this paper on Pith yet. A scientist listed on Pith can review it for a fee, and their review is published here whatever it concludes.

Request a human review

How Pith checks papers · The full audit of this paper

The paper

Abstract

Modern video models possess strong visual priors, yet do not natively represent human action in metric 3D space. We present EgoGeneralist, which adapts pretrained video generators to unify action-conditioned video generation, pose estimation, and joint action-video generation for egocentric manipulation. Our role-adaptive action renderer provides video-native representations for multidirectional action-video learning, making metric motion usable as both a spatial control signal and a recoverable generation target. Cross-stream action-video coupling establishes close correspondence between metric motion and visual dynamics through lightweight adaptation while preserving the pretrained visual prior. Together, these designs establish a generalist framework spanning six action-video tasks. With only narrow-domain data, EgoGeneralist generalizes to diverse unseen scenes and interactions with effective action control and visual quality, bridging pretrained visual knowledge and metric 3D human motion.

The paper's own sentences, arranged, kept beside the review and separate from it. Each label is drawn from a closed vocabulary; the number beside it is the probability the classifier gave that label. No sentence in this record was written by a model.

abstract compiled · 2026-11-06

The authors' main claim, as the classifier picked it out 0.99

“Despite adaptation only on controlled laboratory recordings, EgoGeneralist generalizes to diverse unseen scenes and interactions with effective action control and visual quality, bridging pretrained visual knowledge and metric 3D human motion.”

How the main claim is stated: asserted 1.00

What the abstract itself states, as probabilities: claims first 0.07 · states limitation 0.87 · human participants 0.28 · controlled comparison 0.06 · randomized assignment 0.01 · states quantitative result 0.05 · compares with prior results 0.13 · mentions code or data availability 0.03

The abstract, sentence by sentence

This table labels the abstract only (arXiv listing abstract v1). The number is the probability the classifier gave the label.

SentenceRoleThe authors' own workStates the main conclusionStrength
Modern video models possess strong visual priors, yet do not natively represent human action in metric 3D space. gap 0.59 0.10 0.04 asserted 1.00
We present EgoGeneralist, which adapts pretrained video generators to unify action-conditioned egocentric video generation, pose estimation, and joint action-video generation. contribution 1.00 0.93 0.12 asserted 1.00
Our role-adaptive action renderer provides video-native representations for multidirectional action-video learning, making metric motion usable as both a spatial control signal and a recoverable generation target. method 0.91 0.90 0.10 asserted 1.00
Cross-stream action-video coupling establishes close correspondence between metric motion and visual dynamics through lightweight adaptation while preserving the pretrained visual prior. method 0.93 0.85 0.06 asserted 1.00
Together, these designs establish a generalist framework spanning five action-video tasks. contribution 0.86 0.83 0.17 asserted 1.00
Despite adaptation only on controlled laboratory recordings, EgoGeneralist generalizes to diverse unseen scenes and interactions with effective action control and visual quality, bridging pretrained visual knowledge and metric 3D human motion. result 0.96 0.93 0.75 asserted 1.00

What kind of paper the classifier reports this is: method or tool 1.00 · evidence from observation or experiment 0.92

What was examined, and what was not

Sources read

  • arXiv listing abstract v1

Not examined

  • full text
  • figures and tables
  • references and citing sentences
  • relations to other papers

Levels run

  • P abstract

Not run

  • S sections
  • T body sentences
  • C citations
  • R cross-paper relations

How the labels were made

Compiled by classifier-1.13.0 via TypeSafe API, question bank abstract-p1-20260923, 2026-11-06: 1 request, 4,926 input tokens.

Every quoted sentence on this record is the authors'. Every label is from Pith's closed vocabulary, with the probability shown beside it. This record as JSON

Figures (6)
Figure 1 of the paper
Figure 1. Five capabilities of EgoGeneralist. Our framework unifies action-conditioned ego video generation, egocentric 3D pose estimation, and joint action–video generation, defining six inference modes across the five tasks illustrated above. Reusing the pretrained video prior enables generalization to unseen scenes with physically coherent interactions. Abstract Modern video models possess strong visual priors, yet do not natively represent human action in metric 3D space. We present EgoGeneralist, which adapts pretrained video gen￾erators to unify action-conditioned egocentric video…
Figure 2 of the paper
Figure 2. Overview of EgoGeneralist. Role-adaptive rendering (Sec. 3.2) tailors the visual representation of metric 3D action to its task role: an ego-view map with a fixed 3D anchor for conditioning, or a multiview Posegrid for generation and metric readout. A shared frozen video VAE encodes rendered action and video into two latent streams. Cross-stream coupling (Sec. 3.3) uses joint self-attention and attention LoRAs activated only for generation targets, retaining the backbone weights and per-stream input structure. The bottom panel illustrates condition–target assignments for the three…
Figure 3 of the paper
Figure 3. Action-conditioned image-to-video generation. Comparisons on EgoDex (left) and Xperience-10M (right, out-of-distribution). Adapted only on laboratory recordings, EgoGeneralist generalizes to in-the-wild scenes. The generated videos follow the prescribed motion with physically coherent interactions, correctly removing the purple block and tilting the bucket
Figure 5 of the paper
Figure 5. World-action policy on HOT3D. Top: EgoGeneralist’s generated video and 3D action in ego and side views. Bottom: action predictions from all three methods in a shared top view. GT object meshes and scene points are used only for visualization. Additional examples are provided in supplementary Fig. S7. ing VITRA’s target-reaching criterion. We report minimum fingertip-to-target surface distance over the predicted trajec￾tory (MinDist), reaching rates within 2/5 cm (Reach@2/5), and percentage distance reduction from the initial state (Progress)
Figure 6 of the paper
Figure 6. Text-only joint action–video generation. Each example shows jointly generated video and 3D action, with the action projected onto video frames. Insets show the generated 3D upper-body skeleton and camera pose. Additional examples appear in Figs. S8 and S9

Showing 5 of the paper's 6 figures. Figures are the authors', reproduced from the paper.

Challenges and discussion

Disagree with the review, or know something it missed? Leave a comment, or challenge this review formally: a challenge becomes a permanent entry in this paper's record, and its decision is published here.

No comments yet.

Sign in to comment