Computer Vision and Pattern Recognition · arXiv:2610.28594 v1 · submitted 20 Oct 2026 · 87 references
Learning the 4D World through Point Trajectory Forecasting
The paper says
“Reliable multi-view forecasting requires videos from different cameras to describe the same evolving 3D scene.”
more of the abstract
To this end, we introduce Video-Point Model (VPM), a world model that conditions on multi-view video histories and a language instruction to jointly forecast future videos from arbitrary viewpoints and the 3D trajectories of persistent scene points. We extend a pretrained world model with camera-conditioned multi-view processing that integrates complementary observations across cameras, and add a trajectory head that predicts how physical points move through time. Each trajectory tracks a single point across all views, and supervising these trajectories alongside video generation therefore anchors appearance changes in every camera to a shared description of 3D motion. To support training and evaluation, we build VPM-Data, a simulation corpus pairing synchronized multi-view videos with persistent point trajectories, and VP-WorldBench, a forecasting benchmark spanning simulated and real environments. We further extend VPM to robot control by adding robot actions as a token modality in the generator. Action tokens share the point-supervised backbone and are denoised jointly with future videos, thereby grounding the policy in the same 3D-aware representation. On VP-WorldBench and existing 4D generation benchmarks, point supervision improves both video fidelity and 3D motion accuracy over a video-only counterpart, and in simulated and real-robot experiments, it also raises the manipulation success of the action-enabled model.
from the abstract
Pith says
Machine review · deepseek-v4-flash · 21 Oct 2026 · what was read was not recorded · not checked by a person
It can be wrong. Challenge this review or reply to it.
Holds up, with conditions. This paper makes a credible empirical case that persistent 3D point-trajectory supervision improves multi-view video forecasting and robot manipulation, with the main caveat that its 3D-motion metric is pipeline-limited and its headline tables lack error bars. This line is wrong
3 major objections · 3 minor
How often objections from reviews like this one are right: not yet measured
Not checked: figures, code, what the 87 references say.
The paper says, at A.2, Eq. (11); Tables 2, 16, 19
The review did not quote this passage.
A classifier read this sentence of the paper as acknowledging the concern (probability 0.66):
“Neither metric reaches its ideal value even for a perfect generator.”
Pith says: Objection 1 of 3 · major
The 3D-motion-accuracy leg of the central claim is not yet cleanly separable from the video-fidelity leg. δ3Davg is computed by running MVTracker on generated videos and MoGe-2 for depth, and the paper itself discloses that the pipeline does not reach its ideal value even for a perfect generator and reports ground-truth upper bounds of 73.27, 61.87, 80.40, and 57.98 on the four splits.
more of objection 1
The problem is not that VPM lies below these ceilings; it is that the +6.4 to +13.9 δ3Davg gains coincide with FVD/PSNR improvements on the same splits, so a plausible mechanism is improved trackability and depth estimation on visually cleaner generations. The direct trajectory-head rows in Table 19 (e.g., real-world Robot, 51.78 to 67.22) mitigate the external-tracker concern but do not eliminate it, because the head's depth is jointly supervised and its features are read from the same improved video decoder. I ask for a decomposition: report per-threshold tracking-success rates and position error conditioned on successful tracking, separate 2D localization error from depth error, and add an FVD-matched control (for example, a video-only model trained with a longer schedule or different guidance) to show that the δ3Davg gain is not simply a restatement of higher video quality.
The paper says, at 3.1 vs 6.1; Eq. (4)
The review did not quote this passage.
Pith says: Objection 2 of 3 · major
There is a protocol inconsistency that affects how the 3D-trajectory results should be read. Section 3.1 states that the model receives the camera parameters of the requested future views, while Section 6.1 and Appendix A.1 state that in the main evaluation no method, including VPM, receives camera parameters for future frames.
more of objection 2
Equation (4) unprojects the predicted per-view 2D positions and depths using future camera-to-reference transforms T_vτ, so it is unclear how the trajectory head's 3D outputs are obtained without those transforms. Please clarify whether the default-protocol trajectory-head results and the δ3Davg values in Tables 2 and 19 use ground-truth future extrinsics for 3D lifting; if they do, say so explicitly as a limitation of the evaluation, and if they do not, describe how T_vτ is inferred from the history.
The paper says, at 6.4, Table 5, A.3
The review did not quote this passage.
Pith says: Objection 3 of 3 · major
The real-robot success results are based on 24 trials per task with partial-credit scoring, and the claimed gains appear to be within plausible binomial sampling variability (for example, the banana task reported as 79.2 vs 58.3 corresponds to roughly 19 vs 14 successful trials per 24).
more of objection 3
The pattern across all three tasks is encouraging, but without per-task trial counts, score distributions, or confidence intervals, the strength of the real-robot leg of the claim is hard to assess. Please report confidence intervals or a simple significance test, or at minimum the raw per-task scores, so that the reader can separate a robust effect from sampling noise.
3 minor objections
The paper says, at Title and headings
Pith says: minor objection 1 of 3
The title and several headings are missing spaces in the extracted text (for example, 'THE4D WORLD THROUGH POINTTRAJECTORYFORECASTING' and 'RoboCasaKitchen'); please correct these in the final version.
The paper says, at A.2, Eq. (12)
Pith says: minor objection 2 of 3
The derivation of the cross-view MEt3R floor in Eq. (12) is terse; a short explanation of why the 1/sqrt(2) factor compensates for the half-length averages would help reproducibility.
show 1 more
The paper says, at E.4, Table 23
Pith says: minor objection 3 of 3
In the PointMotionBench table, the sentence in E.4 says VPM improves PWT on DAVIS over MolmoMotion-AR with three-frame input; this is correct only because PWT is a higher-is-better metric, so please state the direction of PWT explicitly to avoid confusion.
What would settle it
Recompute δ3Davg (Eq. 11) for VPM(V) and VPM(V+T) with two pipeline modifications: (i) restrict the evaluation to the set of query-point, view, and timestep tuples for which MVTracker yields a usable estimate in both models' videos, so the two models are compared on a trackability-matched support; and (ii) on the simulation splits, substitute the simulator's ground-truth depth for MoGe-2 depth, removing the depth-estimator component. If the V→V+T gap (currently +6.4 to +13.9 points) persists on this common support and with oracle depth, the 3D-coherence interpretation survives. If the gap shrinks materially, the reported 3D-motion-accuracy improvement largely measured pipeline trackability and video quality rather than the coherence of generated 3D motion.
from the skeptic's reading
How the review went
4 readings by deepseek-v4-flash · 21 Oct 2026
- Reader holds up, with conditions
- Skeptic agrees
- Editor wrote the verdict above
- Referee major revision: 3 major objections, 3 minor
Not run on this paper: the simulated authors' defence.
Pith did not check this review's objections against the paper.
The review in full
The review in brief, machine-written
What you should know first: this is a strong, conditionally acceptable empirical paper, and the central claim survives reading. Persistent metric-3D trajectories that keep point identity across views and future time, used as joint supervision for multi-view video forecasting, is genuinely new relative to Track4Gen, MVTrack4Gen, and MolmoMotion. The architecture is sensible: Plücker camera embeddings, a fourth RoPE axis for view identity, and a trajectory head with cross-view correspondence loss. The dataset and benchmark are large, carefully validated, and the paper discloses its own limitations in unusual detail.
What the paper does well: the evidence for direction is consistent. VPM(V+T) beats VPM(V) on every split of VP-WorldBench, on ReCamMaster, CMU Panoptic, and RoboCasa Kitchen, and the scaling result — VPM(V+T) at 100K samples beating VPM(V) at 600K — is the strongest practical argument. The per-frame-vs-trajectory ablation (Table 22) is well designed: same backbone, data, and schedule, so it isolates temporal identity. The robot results also favor point supervision across LIBERO, RoboCasa, and real pick-and-place, though the real-world trials are only 24 per task.
Soft spots, in proportion: the main one is the δ3Davg metric. Section A.2 states plainly that the metric does not reach its ideal value even for a perfect generator, with ground-truth pipeline upper bounds as low as 57.98 on Real-world General. The VPM(V)-to-VPM(V+T) gain there is 26.54 to 33.92, entirely below that ceiling. The authors acknowledge the floor, but the sharper issue remains: cleaner videos are easier to track, so part of the δ3Davg gain may be a byproduct of the FVD gains. That does not kill the paper — cross-view MEt3R and the direct trajectory-head results (Table 19) agree in direction — but it means the magnitude of the distinct 3D-motion-accuracy contribution is provisional. Headline tables carry no error bars, and no code or data are linked, so numerical verification has to wait. These are addressable, not fatal.
Who this is for: anyone working on geometric supervision for video world models or world-action models will get real value. It deserves a serious referee; I would send it to review with a request for artifacts, error bars, and a trackability-matched analysis of the 3D-motion gains.
The strongest objection
The central claim is a conjunction: point supervision improves video fidelity, 3D motion accuracy, and manipulation success over a video-only counterpart at equal budget. The fidelity and manipulation legs are credible: FVD gains replicate across all four splits and external benchmarks, and VPM(V+T+A) beats VPM(V+A) on every simulated and real task. The 3D-motion-accuracy leg is the soft spot. Its headline evidence is δ3Davg (Eq. 11), which does not use the model's trajectory head: MVTracker tracks points in the generated videos, MoGe-2 supplies depth, and extracted tracks are compared with ground truth at depth-adaptive thresholds. Section A.2 states that the metric does not reach its ideal value even for a perfect generator (sentence b338) and reports pipeline upper bounds of 73.27, 61.87, 80.40, and 57.98 on the four splits, while VPM(V) scores 43.62, 28.71, 50.83, 26.54 and VPM(V+T) scores 50.05, 36.80, 64.75, 33.92. Sitting below a pipeline ceiling is expected for any real model; the unresolved issue is sharper. Tracking success and depth error plausibly improve when generated videos become visually cleaner and smoother, and trajectory supervision improves FVD by 6–42 points on the same splits. If part of the +6.4-to-+13.9 δ3Davg gap is a trackability byproduct of higher video quality, the distinct 3D-motion-accuracy contribution is partly a restatement of the fidelity gain, and its magnitude is not established. Table 19 compares the supervised trajectory head with MVTracker but is not trackability-matched, and for VPM(V), trained without L_track, the head's own outputs are not an independently meaningful tracker. Table 22's controlled per-frame-versus-trajectory ablation supports the temporal-identity mechanism but inherits the same pipeline-mediated metric. Cross-view MEt3R (Eq. 10) agrees in direction, which supports the qualitative conclusion; the concern here is magnitude, not direction.
The paper's claim is that persistent 3D point-trajectory supervision is an effective, transferable training signal: it improves video fidelity, 3D motion accuracy, and manipulation success over video-only training at equal budget. I checked each leg. Fidelity: FVD/PSNR gains of VPM(V+T) over VPM(V) are consistent on all four VP-WorldBench splits (Table 2) and replicate out-of-domain on ReCamMaster, CMU Panoptic, and RoboCasa Kitchen (Tables 3-4). The per-frame-geometry ablation (Table 22) is genuinely controlled, since depth, pointmap, and trajectory heads supervise the same 3D positions, and it shows that temporal identity drives most of the gain. This leg is solid. Manipulation: VPM(V+T+A) beats VPM(V+A) on LIBERO, LIBERO-Plus, RoboCasa PnP, and on all three real tasks (Table 5); the direction is uniform. The real-world margins are 20.9, 6.2, and 10.4 points on 24 scored trials, with the smallest below one binomial standard error at these rates, so the magnitude is provisional even though the consistent sign is believable. Motion accuracy: the reader's weakest assumption is correct. δ3Davg (Eq. 11) passes generated videos through MVTracker and MoGe-2, and the pipeline's own ceiling, disclosed in Section A.2, ranges from 57.98 (Real General) to 80.40 (Real Robot). The ceiling itself is not damning, since any imperfect generator must score below it, but it quantifies how much of the metric is pipeline behavior. The sharper problem is the confound between two quantities the paper wants to separate: generated-motion coherence and generated-video quality. Trajectory supervision improves FVD by 6-42 points on the same splits, and a tracker operating on cleaner, smoother videos is likely to succeed on more points and at tighter thresholds. Part of the V→V+T δ3Davg gap could therefore be a mechanical trackability effect, making the 3D-motion-accuracy improvement partially redundant with the fidelity improvement. Table 19 does not close this: it reports the supervised head's own predictions, which are not trackability-matched, and the VPM(V) rows are hard to interpret because VPM(V) is trained without L_track. Table 22 inherits the same pipeline metric. MEt3R (Eq. 10) agrees in direction, which supports the qualitative conclusion; I am not claiming the effect is an artifact, only that its distinct magnitude is unestablished until the pipeline confound is removed. The proposed test, common-support recomputation with oracle depth on simulation splits, isolates exactly this. The zero-shot trajectory-to-IK rollouts (Appendix A.4) are a further falsifiable demonstration that predicted trajectories carry actionable 3D information, independent of the metric debate. Secondary, non-load-bearing observations: the simulation benchmark shares its generation pipeline with training data (disclosed in Section C.1 and mitigated by external benchmarks), real-robot success rates carry no intervals, and no artifacts are linked.
The referee's report
The paper's contribution is substantial if the claim holds: persistent 3D point-trajectory supervision would be a transferable and data-efficient training signal for multi-view world models and world-action models. The study is unusually thorough: the equal-budget comparison VPM(V) vs VPM(V+T) is consistent across every split and metric; the geometric evaluation uses an external tracker (MVTracker) and, separately, the model's own trajectory head; the per-frame geometry ablation in Table 22 supports the temporal-identity interpretation; and the paper explicitly discloses the δ3Davg pipeline upper bounds and states that the metric does not reach its ideal value even for a perfect generator (sentence b338). The dataset and benchmark are described in sufficient detail to be reusable. The main unresolved issue is whether the 3D-motion-accuracy gains are empirically separable from the video-fidelity gains, and some protocol details around future camera parameters and real-robot trial statistics need to be sharpened.
Pith's reading of the paper (a machine's summary, not the authors')
This paper claims that teaching a multi-view video forecaster where specific physical points will move in 3D makes its forecasts better on two fronts: the generated videos look closer to the true future, and the motion they depict is more geometrically coherent. The authors build Video-Point Model (VPM) on a pretrained world-model backbone, adding camera-conditioned multi-view processing and a trajectory head that predicts how points queried from the observed frames move through time in every view. Under an equal training budget, point supervision beats video-only training on every split of a new benchmark spanning simulated and real scenes, and an action-enabled variant achieves higher robot manipulation success in simulation and on a real arm. The paper also releases VPM-Data, a simulation corpus pairing synchronized multi-view videos with persistent 3D point trajectories, and VP-WorldBench, the evaluation suite.
What carries the argument. The load-bearing object is the persistent 3D point trajectory: a single point identity sampled from observed frames, expressed in a fixed reference frame, and predicted across views and future time. It enters training through two objectives — a trajectory head that iteratively refines per-view 2D positions, depths, and confidences (loss $L_\text{track}$), and a cross-view correspondence loss $L_\text{corr}$ that supervises the generator's attention features directly by classifying which spatial cell each point maps to in other views and times. Camera-conditioned multi-view processing (Plücker ray embeddings plus a learnable view axis added to the rotary positional encoding) lets the generator mix tokens from different cameras in one sequence while keeping their geometry distinct. A three-stage schedule first adapts the generator to multi-view data, warms up the trajectory head alone, then jointly optimizes video, trajectory, and correspondence objectives; the action-enabled extension shares this backbone by adding action tokens denoised jointly with the future videos.
Core claim. The central claim is that a persistent 3D point trajectory is a shared description of motion across cameras. Because each trajectory follows the same physical point, supervising trajectories alongside video generation anchors appearance changes in every view to one 3D motion, and the paper provides controlled evidence that this anchor improves both the generated video and the accuracy of the motion it depicts. VPM (video plus trajectories) beats its video-only counterpart on every split and metric of VP-WorldBench, including the real-world splits, and the gain transfers out of domain to ReCamMaster, CMU Panoptic, and RoboCasa Kitchen. The same supervision also helps action learning: the variant trained with videos, trajectories, and actions outperforms the video-plus-action variant on every manipulation benchmark and every real-robot task. An ablation against per-frame depth or pointmap supervision isolates the active ingredient as temporal identity, meaning the supervision must track the same point across time, not just know the geometry at each frame.
The premise it rests on. The 3D-motion-accuracy claim is measured by running an external tracker (MVTracker) and monocular depth estimator (MoGe-2) on generated videos and comparing their extracted tracks with ground truth, and the paper acknowledges that this pipeline does not reach its ideal value even for a perfect generator (ground-truth upper bound as low as 57.98 on Real-world General); if part of the reported gain reflects the external pipeline rather than the coherence of generated motion, the magnitude of the 3D-accuracy claim is not established.
If the paper is right (Pith's inferences, not the authors')
Point-trajectory supervision is an effective auxiliary objective for multi-view world models: VPM (V+T) improves both video fidelity and 3D motion accuracy over VPM (V) on every split, at under one second of extra inference time per clip.
The supervision transfers zero-shot to out-of-domain 4D generation benchmarks (ReCamMaster, CMU Panoptic, RoboCasa Kitchen), so its benefit is not confined to the simulation domain used for training.
Action-enabled world models inherit the benefit: the video-plus-trajectory-plus-action variant outperforms the video-plus-action variant on every simulated benchmark and every real-robot task.
Trajectory supervision is more data-efficient than scaling video data: VPM (V+T) trained on 100K samples beats VPM (V) trained on the full 600K samples on every split.
Predicted trajectories alone carry actionable information: converting them into commands by inverse kinematics completes unseen real-robot tasks zero-shot, without action labels, although the paper reports this only qualitatively.
The per-frame-versus-trajectory ablation points to temporal identity, not per-frame geometry, as the active ingredient; a natural extension to test is whether cheaper correspondence objectives (optical flow, keypoint persistence, scene flow) confer a similar anchor when dense metric 3D supervision is unavailable.
The paper's stated next step is denser, non-rigid supervision (cloth, hair, deformables); if the anchoring mechanism is general, gains should be largest exactly where rigid-surface supervision is weakest, which is testable on the 48-camera real split.
Because the paper reports that the $\delta^{\text{3D}}_{\text{avg}}$ pipeline does not reach its ideal value even on ground-truth videos (upper bound 57.98 on Real-world General), an independent re-evaluation with a different tracker and depth estimator would separate generated-motion coherence from tracker-side effects.
Check this review
Pith stored no record of this review's passes and took no fingerprint of it. Reviews made by code from 13 Nov 2026 on carry both.
How often reviews like this one are right
Reviews like this one: written by deepseek-v4-flash, where what was read was not recorded.
How often objections from reviews like this one are right: not yet measured
Reviews by people
This review was written by a machine. No person has reviewed this paper on Pith yet. A scientist listed on Pith can review it for a fee, and their review is published here whatever it concludes.
The paper
Abstract
Reliable multi-view forecasting requires videos from different cameras to describe the same evolving 3D scene. To this end, we introduce Video-Point Model (VPM), a world model that conditions on multi-view video histories and a language instruction to jointly forecast future videos from arbitrary viewpoints and the 3D trajectories of persistent scene points. We extend a pretrained world model with camera-conditioned multi-view processing that integrates complementary observations across cameras, and add a trajectory head that predicts how physical points move through time. Each trajectory tracks a single point across all views, and supervising these trajectories alongside video generation therefore anchors appearance changes in every camera to a shared description of 3D motion. To support training and evaluation, we build VPM-Data, a simulation corpus pairing synchronized multi-view videos with persistent point trajectories, and VP-WorldBench, a forecasting benchmark spanning simulated and real environments. We further extend VPM to robot control by adding robot actions as a token modality in the generator. Action tokens share the point-supervised backbone and are denoised jointly with future videos, thereby grounding the policy in the same 3D-aware representation. On VP-WorldBench and existing 4D generation benchmarks, point supervision improves both video fidelity and 3D motion accuracy over a video-only counterpart, and in simulated and real-robot experiments, it also raises the manipulation success of the action-enabled model.
The paper's own sentences, arranged, kept beside the review and separate from it. Each label is drawn from a closed vocabulary; the number beside it is the probability the classifier gave that label. No sentence in this record was written by a model.
abstract compiled · 2026-10-21
The authors' main claim, as the classifier picked it out 1.00
“On VP-WorldBench and existing 4D generation benchmarks, point supervision improves both video fidelity and 3D motion accuracy over a video-only counterpart, and in simulated and real-robot experiments, it also raises the manipulation success of the action-enabled model.”
How the main claim is stated: asserted 1.00
What the abstract itself states, as probabilities: claims first 0.07 · states limitation 0.09 · human participants 0.05 · controlled comparison 0.93 · randomized assignment 0.02 · states quantitative result 0.05 · compares with prior results 0.91 · mentions code or data availability 0.12
The abstract, sentence by sentence
This table labels the abstract only (arXiv listing abstract v1). The number is the probability the classifier gave the label.
| Sentence | Role | The authors' own work | States the main conclusion | Strength |
|---|---|---|---|---|
| Reliable multi-view forecasting requires videos from different cameras to describe the same evolving 3D scene. | background 0.99 | 0.08 | 0.03 | asserted 1.00 |
| To this end, we introduce Video-Point Model (VPM), a world model that conditions on multi-view video histories and a language instruction to jointly forecast future videos from arbitrary viewpoints and the 3D trajectories of persistent scene points. | contribution 0.98 | 0.91 | 0.07 | asserted 0.98 |
| We extend a pretrained world model with camera-conditioned multi-view processing that integrates complementary observations across cameras, and add a trajectory head that predicts how physical points move through time. | method 0.96 | 0.91 | 0.04 | asserted 0.98 |
| Each trajectory tracks a single point across all views, and supervising these trajectories alongside video generation therefore anchors appearance changes in every camera to a shared description of 3D motion. | method 0.71 | 0.64 | 0.07 | asserted 1.00 |
| To support training and evaluation, we build VPM-Data, a simulation corpus pairing synchronized multi-view videos with persistent point trajectories, and VP-WorldBench, a forecasting benchmark spanning simulated and real environments. | contribution 0.73 | 0.96 | 0.04 | asserted 0.99 |
| We further extend VPM to robot control by adding robot actions as a token modality in the generator. | contribution 0.66 | 0.95 | 0.02 | asserted 0.99 |
| Action tokens share the point-supervised backbone and are denoised jointly with future videos, thereby grounding the policy in the same 3D-aware representation. | method 0.98 | 0.88 | 0.06 | asserted 0.99 |
| On VP-WorldBench and existing 4D generation benchmarks, point supervision improves both video fidelity and 3D motion accuracy over a video-only counterpart, and in simulated and real-robot experiments, it also raises the manipulation success of the action-enabled model. | result 1.00 | 0.95 | 0.87 | asserted 1.00 |
What kind of paper the classifier reports this is: method or tool 1.00 · evidence from observation or experiment 0.88
What was examined, and what was not
Sources read
- arXiv listing abstract v1
Not examined
- full text
- figures and tables
- references and citing sentences
- relations to other papers
Levels run
- P abstract
Not run
- S sections
- T body sentences
- C citations
- R cross-paper relations
How the labels were made
Compiled by classifier-1.13.0 via TypeSafe API, question bank abstract-p1-20260923, 2026-10-21: 1 request, 6,392 input tokens.
Every quoted sentence on this record is the authors'. Every label is from Pith's closed vocabulary, with the probability shown beside it. This record as JSON
Figures (14)
Figures are the authors', reproduced from the paper.
References (87)
- [1] Cosmos 3: Omnimodal world models for physical ai , author=. arXiv preprint arXiv:2606.02800 , year= on PithPith's reviewarXiv
- [2] Geometric Action Model for Robot Policy Learning , author=. arXiv preprint arXiv:2606.17046 , year= on PithPith's reviewarXiv
- [3] No Pose, No Problem in 4D: Feed-Forward Dynamic Gaussians from Unposed Multi-View Videos , author=. arXiv preprint arXiv:2605.22190 , year= on PithPith's reviewarXiv
- [4] 3d scene prompting for scene-consistent camera-controllable video generation , author=. International Conference on Learning Representations , volume= on Pith
- [5] Stable video diffusion: Scaling latent video diffusion models to large datasets , author=. arXiv preprint arXiv:2311.15127 , year= on PithPith's reviewarXiv
- [6] Learning universal policies via text-guided video generation , author=. Advances in neural information processing systems , volume= on Pith
- [7] Spatial-Aware Multi-View Video Diffusion as a Data-Efficient Robot Policy , author=. arXiv preprint arXiv:2604.03181 , year= on PithPith's reviewarXiv
- [8] Sv4d: Dynamic 3d content generation with multi-frame and multi-view consistency , author=. International Conference on Learning Representations , volume= on Pith
- [9] Sam 2: Segment anything in images and videos , author=. International Conference on Learning Representations , volume= on Pith
- [10] Geometry-aware 4d video generation for robot manipulation , author=. International Conference on Learning Representations , volume= on Pith
- [11] Fantasyworld: Geometry-consistent world modeling via unified video and 3d prediction , author=. International Conference on Learning Representations , volume= on Pith
- [12] Tapip3d: Tracking any point in persistent 3d geometry , author=. Advances in Neural Information Processing Systems , volume= on Pith
- [13] Hunyuanvideo 1.5 technical report , author=. arXiv preprint arXiv:2511.18870 , year= on PithPith's reviewarXiv
- [14] Unified 4d world action modeling from video priors with asynchronous denoising , author=. arXiv preprint arXiv:2604.26694 , year= on PithPith's reviewarXiv
- [15] MotionForesight: Re-purposing Video Models for Future 3D Scene-Flow Prediction , author=. arXiv preprint arXiv:2607.16192 , year= on PithPith's reviewarXiv
- [16] Learning 4D Geometric Priors for Inference-Efficient World Action Models , author=. arXiv preprint arXiv:2607.05468 , year= on PithPith's reviewarXiv
- [17] 4D-WAM: Infusing Spatiotemporal Awareness into World Action Models through Trajectory Fields , author=. arXiv preprint arXiv:2608.08023 , year= on PithPith's reviewarXiv
- [18] Gem-4d: Geometry-enhanced video world models for robot manipulation , author=. arXiv preprint arXiv:2605.22882 , year= on PithPith's reviewarXiv
- [19] Videovla: Video generators can be generalizable robot manipulators , author=. Advances in neural information processing systems , volume= on Pith
- [20] Wan: Open and advanced large-scale video generative models , author=. arXiv preprint arXiv:2503.20314 , volume= on PithPith's reviewarXiv
show 67 more
- [21] CoTracker3: Simpler and Better Point Tracking by Pseudo-Labeling Real Videos , author=. 2025 IEEE/CVF International Conference on Computer Vision (ICCV) , pages=. 2025 , organization= on Pith
- [22] TAPVid-MV: A Benchmark for Tracking Any Point in 3D Across Multiple Views , author=. arXiv preprint arXiv:2609.01899 , year= on PithPith's reviewarXiv
- [23] BEDLAM2. 0: Synthetic humans and cameras in motion , author=. Advances in Neural Information Processing Systems , volume= on Pith
- [24] Intelligence, Physical and Black, Kevin and Brown, Noah and Darpinian, James and Dhabalia, Karan and Driess, Danny and Esmail, Adnan and Equi, Michael and Finn, Chelsea and Fusai, Niccolo and others , journal=. _ on Pith
- [25] NVIDIA and Johan Bjorck and Fernando Castañeda and Nikita Cherniadev and Xingye Da and Runyu Ding and Linxi "Jim" Fan and Yu Fang and Dieter Fox and Fengyuan Hu and Spencer Huang and Joel Jang and Zhenyu Jiang and Jan Kautz and Kaushil Kundalia and Lawrence Lao and Zhiqi Li an... on Pith
- [26] Geometry forcing: Marrying video diffusion and 3d representation for consistent world modeling , author=. International Conference on Learning Representations , volume= on Pith
- [27] Videorepa: Learning physics for video generation through relational alignment with foundation models (2025) , author=. arXiv preprint arXiv:2505.23656 , year= on PithPith's reviewarXiv
- [28] Videojam: Joint appearance-motion representations for enhanced motion generation in video models , author=. arXiv preprint arXiv:2502.02492 , year= on PithPith's reviewarXiv
- [29] Track4gen: Teaching video diffusion models to track points improves video generation , author=. 2025 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) , pages=. 2025 , organization= on Pith
- [30] Classifier-free diffusion guidance , author=. arXiv preprint arXiv:2207.12598 , year= on PithPith's reviewarXiv
- [31] Panoptic studio: A massively multiview system for social motion capture , author=. Proceedings of the IEEE international conference on computer vision , pages= on Pith
- [32] Flow straight and fast: Learning to generate and transfer data with rectified flow , author=. arXiv preprint arXiv:2209.03003 , year= on PithPith's reviewarXiv
- [33] Recammaster: Camera-controlled generative rendering from a single video , author=. 2025 IEEE/CVF International Conference on Computer Vision (ICCV) , pages=. 2025 , organization= on Pith
- [34] Flow matching for generative modeling , author=. arXiv preprint arXiv:2210.02747 , year= on PithPith's reviewarXiv
- [35] Light field networks: Neural scene representations with single-evaluation rendering , author=. Advances in neural information processing systems , volume= on Pith
- [36] Cameractrl: Enabling camera control for text-to-video generation , author=. arXiv preprint arXiv:2404.02101 , year= on PithPith's reviewarXiv
- [37] Redirector: Creating any-length video retakes with rotary camera encoding , author=. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages= on Pith
- [38] MVTrack4Gen: Multi-View Point Tracking as Geometric Supervision for 4D Video Generation , author=. arXiv preprint arXiv:2606.26087 , year= on PithPith's reviewarXiv
- [39] Cotracker: It is better to track together , author=. European conference on computer vision , pages=. 2024 , organization= on Pith
- [40] Mv-tap: Tracking any point in multi-view videos , author=. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages= on Pith
- [41] OmniRobotHome: A Multi-Camera Platform for Real-Time Multiadic Human-Robot Interaction , author=. arXiv preprint arXiv:2604.28197 , year= on PithPith's reviewarXiv
- [42] World action models are zero-shot policies , author=. arXiv preprint arXiv:2602.15922 , year= on PithPith's reviewarXiv
- [43] Depth anything 3: Recovering the visual space from any views , author=. arXiv preprint arXiv:2511.10647 , year= on PithPith's reviewarXiv
- [44] Ego-exo4d: Understanding skilled human activity from first-and third-person perspectives , author=. Proceedings of the IEEE/CVF conference on computer vision and pattern recognition , pages= on Pith
- [45] Droid: A large-scale in-the-wild robot manipulation dataset , author=. arXiv preprint arXiv:2403.12945 , year= on PithPith's reviewarXiv
- [46] Robocasa365: A large-scale simulation framework for training and benchmarking generalist robots , author=. International Conference on Learning Representations , volume= on Pith
- [47] MolmoMotion: Forecasting Point Trajectories in 3D with Language Instruction , author=. arXiv preprint arXiv:2606.18558 , year= on PithPith's reviewarXiv
- [48] Mixture-of-transformers: A sparse and scalable architecture for multi-modal foundation models , author=. arXiv preprint arXiv:2411.04996 , year= on PithPith's reviewarXiv
- [49] Multi-view 3d point tracking , author=. Proceedings of the IEEE/CVF International Conference on Computer Vision , pages= on Pith
- [50] On the continuity of rotation representations in neural networks , author=. 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) , pages=. 2019 , organization= on Pith
- [51] Zhang, Zhengyou , title =. IEEE Transactions on Pattern Analysis and Machine Intelligence , volume =. 2000 , month = on Pith
- [52] Sch\". Structure-from-Motion Revisited , booktitle= on Pith
- [53] Revisiting Photometric Ambiguity for Accurate Gaussian-Splatting Surface Reconstruction , author=. International Conference on Machine Learning , year= on Pith
- [54] Quo vadis, action recognition? a new model and the kinetics dataset , author=. 2017 IEEE conference on computer vision and pattern recognition (CVPR) , pages=. 2017 , organization= on Pith
- [55] Towards accurate generative models of video: A new metric & challenges , author=. arXiv preprint arXiv:1812.01717 , year= on PithPith's reviewarXiv
- [56] Moge-2: Accurate monocular geometry with metric scale and sharp details , author=. Advances in Neural Information Processing Systems , volume= on Pith
- [57] Libero: Benchmarking knowledge transfer for lifelong robot learning , author=. Advances in Neural Information Processing Systems , volume= on Pith
- [58] Han, Jisang and An, Honggyu and Jung, Jaewoo and Narihira, Takuya and Seo, Junyoung and Fukuda, Kazumi and Kim, Chaehyun and Hong, Sunghwan and Mitsufuji, Yuki and Kim, Seungryong , journal=. D\^ on Pith
- [59] Cats: Cost aggregation transformers for visual correspondence , author=. Advances in Neural Information Processing Systems , volume= on Pith
- [60] Neural matching fields: Implicit representation of matching fields for visual correspondence , author=. Advances in Neural Information Processing Systems , volume= on Pith
- [61] Deep matching prior: Test-time optimization for dense correspondence , author=. Proceedings of the IEEE/CVF international conference on computer vision , pages= on Pith
- [62] Unifying feature and cost aggregation with transformers for semantic and visual correspondence , author=. arXiv preprint arXiv:2403.11120 , year= on PithPith's reviewarXiv
- [63] Cats++: Boosting cost aggregation with convolutions and transformers , author=. IEEE Transactions on Pattern Analysis and Machine Intelligence , volume=. 2022 , publisher= on Pith
- [64] Cross-view completion models are zero-shot correspondence estimators , author=. Proceedings of the Computer Vision and Pattern Recognition Conference , pages= on Pith
- [65] Unifying correspondence pose and nerf for generalized pose-free novel view synthesis , author=. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages= on Pith
- [66] Pf3plat: Pose-free feed-forward 3d gaussian splatting , author=. arXiv preprint arXiv:2410.22128 , year= on PithPith's reviewarXiv
- [67] TORA: Topological Representation Alignment for 3D Shape Assembly , author=. arXiv preprint arXiv:2604.04050 , year= on PithPith's reviewarXiv
- [68] Litept: Lighter yet stronger point transformer , author=. arXiv preprint arXiv:2512.13689 , year= on PithPith's reviewarXiv
- [69] Emergent Outlier View Rejection in Visual Geometry Grounded Transformers , author=. arXiv preprint arXiv:2512.04012 , year= on PitharXiv
- [70] PROSE: Training-Free Egocentric Scene Registration with Vision-Language Models , author=. arXiv preprint arXiv:2606.16569 , year= on PithPith's reviewarXiv
- [71] Libero-plus: In-depth robustness analysis of vision-language-action models , author=. arXiv preprint arXiv:2510.13626 , year= on PithPith's reviewarXiv
- [72] HRDexDB: A Paired Human-Robot Dataset for Cross-Embodiment Dexterous Grasping , author=. arXiv preprint arXiv:2604.14944 , year= on PithPith's reviewarXiv
- [73] Introducing hot3d: An egocentric dataset for 3d hand and object tracking , author=. arXiv preprint arXiv:2406.09598 , year= on PithPith's reviewarXiv
- [74] The 2017 davis challenge on video object segmentation , author=. arXiv preprint arXiv:1704.00675 , year= on PithPith's reviewarXiv
- [75] St4rtrack: Simultaneous 4d reconstruction and tracking in the world , author=. 2025 IEEE/CVF International Conference on Computer Vision (ICCV) , pages=. 2025 , organization= on Pith
- [76] Pai-bench: A comprehensive benchmark for physical ai , author=. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages= on Pith
- [77] Rethinking video generation model for the embodied world , author=. Forty-third International Conference on Machine Learning , year= on Pith
- [78] Videogpt: Video generation using vq-vae and transformers , author=. arXiv preprint arXiv:2104.10157 , year= on PithPith's reviewarXiv
- [79] Videopoet: A large language model for zero-shot video generation , author=. arXiv preprint arXiv:2312.14125 , year= on PithPith's reviewarXiv
- [80] Magi-1: Autoregressive video generation at scale , author=. arXiv preprint arXiv:2505.13211 , year= on PithPith's reviewarXiv
- [81] Dust3r: Geometric 3d vision made easy , author=. 2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) , pages=. 2024 , organization= on Pith
- [82] Met3r: Measuring multi-view consistency in generated images , author=. 2025 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) , pages=. 2025 , organization= on Pith
- [83] Scalable diffusion models with transformers , author=. 2023 IEEE/CVF International Conference on Computer Vision (ICCV) , pages=. 2023 , organization= on Pith
- [84] Dinov2: Learning robust visual features without supervision , author=. arXiv preprint arXiv:2304.07193 , year= on PithPith's reviewarXiv
- [85] Grounding image matching in 3d with mast3r , author=. European conference on computer vision , pages=. 2024 , organization= on Pith
- [86] Featup: A model-agnostic framework for features at any resolution , author=. International Conference on Learning Representations , volume= on Pith
- [87] Spatialboost: Enhancing visual representation through language-guided reasoning , author=. European Conference on Computer Vision , pages=. 2026 , organization= on Pith
Challenges and discussion
Disagree with the review, or know something it missed? Leave a comment, or challenge this review formally: a challenge becomes a permanent entry in this paper's record, and its decision is published here.
Come back by email
A short list of the week's newly reviewed papers. Nothing is sent until you confirm from your inbox, and every email has a link that stops it. No account.
No comments yet.
Sign in to comment