Pith. sign in

REVIEW 3 major objections 4 minor 31 references

Warping positional encodings to projected target-view locations turns them into a geometric routing signal, letting video diffusion transformers follow camera paths and edit geometry while preserving the pretrained video prior.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · deepseek-v4-flash

2026-08-01 22:37 UTC pith:SIIADSW6

load-bearing objection Useful PE-Field-to-video extension with two genuine fixes, but the missing projected-PE ablation leaves the quantitative case for the core mechanism unproven. the 3 major comments →

arxiv 2607.15667 v1 pith:SIIADSW6 submitted 2026-07-17 cs.CV

PE-Field 4D: Video Generation Models as Canvas

Yunpeng Bai , Haoxiang Li , Qixing Huang This is my paper
classification cs.CV
keywords video generationdiffusion transformerspositional encodingcamera controlnovel view synthesisgeometry-aware editingcross-attention4D editing
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

This paper argues that in video diffusion transformers, positional encoding does more than index tokens—it actively decides which parts of a reference video a generated token will copy. The authors' claim is that if reference tokens are assigned positional codes equal to their projected locations in the target camera view, the denoising network will retrieve content from position-aligned regions of the input. They build a lightweight cross-attention branch that carries these projected encodings, with a depth offset to separate overlapping layers and a frame-repetition trick to fix temporal compression ambiguity. On camera re-trajectory, novel-view video synthesis, and viewpoint-dependent editing, the resulting model improves geometric consistency and camera accuracy while keeping the pre-trained video prior intact. The wider claim is that positional encoding can serve as a general, cheap interface for injecting 3D geometry into transformer-based generative video models.

Core claim

The paper's central discovery is that warping the positional encodings of reference tokens to their projected target-view coordinates turns cross-attention into a geometry-aware retrieval process. Concretely, each reference frame token is re-encoded as (t + Δ(d), h', w'), where (h', w') is where that 3D point lands in the target camera plane and Δ(d) is a small normalized depth offset added to the temporal axis so tokens that project to the same 2D spot remain distinguishable by depth. The target video latent keeps its original spatiotemporal positional encoding, so the pretrained video prior is preserved. The authors show that a pretrained video diffusion transformer already biases attentio

What carries the argument

The load-bearing object is the projected positional encoding assigned to reference context tokens. For each reference token, its source depth and camera pose are used to reproject it into the target view, yielding (h', w'); a normalized depth offset Δ(d) in [0, 0.1] is added to the temporal coordinate t so that tokens overlapping in 2D are ordered front-to-back. This encoded context is consumed by an attention branch where target-latent queries attend jointly to a global self-attention memory and a grouped, per-frame local context memory. A frame-repetition step ensures each compressed latent context frame corresponds to exactly one original frame (with fractional temporal indices like 1, 1.

Load-bearing premise

The entire geometry channel is only as good as the monocular depth and camera-pose estimates used to build the projected positional encodings—and the same estimates are used to measure camera accuracy, so errors in them could push both the method and its evaluation in the same direction.

What would settle it

Take a synthetic or scanned scene with ground-truth depth and camera poses, build the projected encodings from those ground-truth values, and compare against the same model fed with the monocular estimates. If the gap in camera error and geometric-consistency metrics between the two settings is small, the method's gain is robust; if the ground-truth-conditioned version is dramatically better, the paper's reported improvement could be an artifact of sharing the estimator between conditioning and evaluation. A second concrete check: randomly permute the depth offsets while keeping the 2D project

Watch this falsifier — get emailed when new claim-graph text bears on it.

If this is right

  • If the claim holds, camera re-trajectory and novel-view video synthesis can be controlled by changing only the positional codes of reference tokens, without retraining the video model from scratch.
  • The same cross-attention branch should generalize to any task that needs spatial grounding—animating a single image, pose-guided editing, or injecting layout constraints—since it does not rely on task-specific supervision.
  • Because the target latent's native spatiotemporal positional encoding is untouched, the pre-trained video prior remains intact; only lightweight adapters and the new attention branch need training.
  • The ablation results imply that both the depth-aware temporal offset and the frame-repetition strategy are necessary; removing either degrades geometric consistency and camera accuracy.
  • Positional encoding may become a standard interface for injecting 3D geometric structure into transformer-based generative models, complementing explicit 3D reconstruction approaches.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • A natural extension not explored in the paper is to apply the same projected-positional-encoding cross-attention to other coordinate-parameterized generation tasks, such as audio source separation or point-cloud generation, where source and target coordinate frames can be related by a known transform.
  • The paper's evaluation shares the same monocular estimator for both constructing the conditioning geometry and measuring camera accuracy; a cleaner test would inject ground-truth depth and poses on synthetic scenes to isolate the attention-routing effect from estimator bias.
  • The attention-bias explanation suggests a direct, quantitative test: measuring whether attention mass shifts exactly to the projected locations when the positional codes are warped, and whether the shift magnitude predicts downstream camera error.
  • The frame-repetition token overhead could be avoided by a learned, per-frame anchor encoding that compresses context; the paper mentions this only as future work, but it is a concrete next step for longer videos.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

3 major / 4 minor

Summary. The paper extends the PE-Field idea to video diffusion transformers. It inserts a cross-attention branch into a pretrained video DiT, where context tokens from reference frames are assigned positional encodings based on their projected target-view coordinates (with a depth-dependent temporal offset) instead of their source-view coordinates. The target latent tokens retain their original spatiotemporal positional encoding. The method is trained with LoRA on MultiCam data and evaluated on camera re-trajectory, novel-view video synthesis, and geometry-aware editing against ReCamMaster, GEN3C, TrajectoryCrafter, and ReDirector. The main claim is that warping the positional encodings of context tokens to target-view geometry creates a position-based retrieval mechanism that improves spatial controllability while preserving the generative prior.

Significance. If the mechanism is validated, the paper offers a lightweight and general way to inject geometric structure into pretrained video diffusion transformers without modifying the target video's latent encoding. This is a conceptually interesting extension of PE-Field, and the proposed depth-aware temporal offset and frame-repetition strategy for temporal compression are plausible engineering contributions. The code is promised open-sourced, and the paper includes ablations for depth disambiguation and temporal compression ambiguity. However, as presented, the evidence does not isolate the effect of the projected positional encoding from the contribution of the newly added cross-attention branch, which is load-bearing for the central claim. The evaluation also has a partial circularity and lacks statistical significance measures.

major comments (3)
  1. [Sec. 4.4, Table 1] The central claim is that projecting context-token positional encodings to target-view coordinates is what creates geometry-aware retrieval. The ablations remove depth ('w/o Depth') and temporal disambiguation ('w/o Temporal Dis.'), but both variants retain the projected target-view PE. There is no control that uses the same cross-attention + LoRA setup with un-projected (source-view) positional encodings or with no positional reassignment for context tokens. Thus the reported gains over ReCamMaster/GEN3C/TrajectoryCrafter/ReDirector could stem from the added cross-attention memory and LoRA fine-tuning rather than from the PE warping, which is the paper's stated novelty. Please add an ablation that replaces the projected PE with the original source-view PE (or a fixed/random PE) under otherwise identical conditions; this control directly tests the proposed mechanism.
  2. [Sec. 4.2, Table 1] TransErr and RotErr are computed from relative pose estimates obtained with ViPE (Sec. 4.2), and ViPE is also used to estimate the depth and camera poses that produce the projected positional encodings (Sec. 4.1). If ViPE has systematic bias, the method's conditioning and the camera-accuracy evaluation are distorted in the same direction. This circularity does not invalidate the other independent metrics (VBench, MEt3R, Dyn-MEt3R), but it weakens the headline camera-accuracy numbers. Please report camera-accuracy errors using an independent pose estimator (e.g., a different SLAM/SfM method) or use re-projection-based errors that do not depend on the same ViPE estimates.
  3. [Table 1] All numbers are reported from a single run with no standard deviations or significance tests. The margins over the best baseline are often small (e.g., Dyn-MEt3R 0.8235 vs. 0.8041; TransErr 0.0142 vs. 0.0189; RotErr 1.887 vs. 2.107). Without multiple seeds (or at least error bars on the evaluation set), it is not possible to assess whether the improvements are meaningful. Please provide variance estimates or a paired significance test for the main comparisons.
minor comments (4)
  1. [Sec. 3.3, Eq. (10)] The depth offset Δ(d) is said to be normalized to [0, 0.1], but it is not specified how raw depth is converted to this range (e.g., per-scene min/max normalization, inverse depth, or log depth). Please state the exact normalization and its ordering convention (front-to-back or back-to-front).
  2. [Sec. 3.4] The fractional temporal coordinates 1, 1.25, 1.5, 1.75 are given as an example, but it is not clear how these are derived for the general case (e.g., different compression factors, varying numbers of repeated frames). Please provide the general formula and clarify whether the generated latent's own temporal coordinate is aligned to the integer part of these fractions.
  3. [Fig. 2 and Fig. 7] The 'position-aligned attention' figures are qualitative illustrations from a pretrained model. The text in Sec. 3.1 says this 'provides direct evidence', but without quantitative attention maps or a controlled experiment, the evidence is only suggestive. Consider adding a quantitative analysis of attention alignment.
  4. [Sec. 3.2, Eq. (4)] The cross-attention concatenates Kx and Ky along the sequence dimension, and the queries are computed only from x. It may be useful to state explicitly whether the LoRA adapters also affect the newly inserted cross-attention parameters (Wq, Wk, Wv) and whether those are initialized from the original self-attention weights or randomly.

Circularity Check

1 steps flagged

Moderate circularity on one evaluation axis: TransErr/RotErr share ViPE with the conditioning; the central PE-warping claim itself is not reduced to its inputs.

specific steps
  1. other [Sec. 4.1 (Implementation Details) and Sec. 4.2 (Generation Results/metrics)]
    "Since the data set does not provide ground-truth depth maps, we estimate the depth and corresponding camera parameters using ViPE [Huang et al. 2025b], and use them as input of geometric conditions to construct the projected positional encodings and geometry-aware context tokens. ... In addition, we report translation and rotation errors, denoted as TransErr and RotErr, which quantify how well the generated video follows the target camera motion based on relative pose estimation from ViPE [Huang et al. 2025b]."

    The same estimator, ViPE, is used to produce the geometric input to the method (depth and camera parameters that define the projected positional encodings) and to estimate the relative poses used to compute the TransErr/RotErr camera-accuracy metrics. Thus any systematic bias or shared failure mode in ViPE feeds both the conditioning and the measurement: the reported camera-accuracy numbers are partly measuring consistency with ViPE's own geometric estimates rather than an independent ground truth. This does not reduce the whole method to its inputs (MEt3R/Dyn-MEt3R and VBench are independent), but it makes the camera-accuracy axis partially self-referential.

full rationale

The paper's central derivation is not circular in the strong sense: the projected positional encoding is a design choice, not a fitted parameter; no equation makes the generated video equal to the conditioning; and the ablations vary depth and temporal disambiguation while leaving the core mechanism in place. The main evidence for the central claim—that projected position codes encourage retrieval from position-aligned regions—is an empirical observation on a pretrained video DiT (Sec. 3.1, Fig. 2), and the prior PE-Field result is self-cited but supplemented by an in-paper qualitative validation. The main substantive circularity concern is the dual use of ViPE: it builds the projected positional encodings and also supplies the relative pose estimates behind TransErr/RotErr, so one evaluation axis is partly self-consistent with the conditioning rather than an independent check. The missing ablation that removes the projected-PE component is an evidential gap about attribution of gains, but it is not a circular-reasoning step because the predictions are not identical to the inputs by construction. Overall, the bulk of the evaluation and the central architectural claim remain independently grounded, warranting a moderate score rather than a high one.

Axiom & Free-Parameter Ledger

3 free parameters · 4 axioms · 0 invented entities

No new physical entities are introduced; the method reuses depth, pose, and a modified positional encoding. The free parameters are the depth-offset range, the fractional timestamp schedule, and LoRA training choices. The main load-bearing assumptions are ViPE's reliability and the video model's latent spatial bias.

free parameters (3)
  • Depth offset scale Δ(d) = normalized to [0, 0.1]
    Introduced ad hoc in Sec. 3.3 to disambiguate tokens with similar 2D reprojections; the 0.1 range is chosen to stay small relative to the frame spacing.
  • Fractional temporal coordinates for repeated source frames = 1, 1.25, 1.5, 1.75 for four source frames per latent step
    Chosen in Sec. 3.4 to map repeated frames into one compressed latent interval; any monotone fractional schedule would work, but this exact grid is hand-set.
  • LoRA training hyperparameters = rank=32, learning rate=1e-4
    Hand-set in Sec. 4.1; not derived and not justified by a search or analysis.
axioms (4)
  • domain assumption ViPE estimates of depth and camera poses are accurate enough to build projected positional encodings
    Sec. 4.1 states depth and camera parameters are estimated by ViPE; if biased, both conditioning and evaluation degrade.
  • domain assumption Pretrained Wan2.1 video DiT has a usable positional bias so that projected PE shifts attention
    Sec. 3.1 and Fig. 2 provide a qualitative validation; the method depends on this bias surviving LoRA fine-tuning.
  • domain assumption Repeating source frames before VAE encoding yields latent frames that represent one source frame
    Sec. 3.4 claims this resolves temporal compression ambiguity, but the paper does not verify that the VAE still avoids mixing content.
  • standard math RoPE can accommodate fractional modified temporal coordinates t + Δ(d)
    Sec. 3.3 uses Eq. (9)-(10); this is a property of rotary embeddings, but the paper does not validate it for this specific model.

pith-pipeline@v1.3.0-alltime-deepseek · 11743 in / 10686 out tokens · 103197 ms · 2026-08-01T22:37:31.283526+00:00 · methodology

0 comments
read the original abstract

Diffusion Transformers have recently achieved strong performance in video generation, yet controlling scene geometry under viewpoint changes and camera motion remains challenging. In this work, we revisit the role of positional encoding in video diffusion transformers and show that it provides a useful spatial bias for geometry-aware control. Specifically, if reference tokens are encoded according to their projected locations in the target view, the denoising model is encouraged to retrieve content from position aligned regions of the input video. Building on this observation, we introduce a geometry-aware cross-attention mechanism that enables target video latent tokens to attend to structured context tokens derived from reference images or frames. To establish correspondence between the reference content and the target camera trajectory, we equip the context tokens with a projected positional encoding scheme that combines target-view 2D reprojection with depth-aware disambiguation. At the same time, we preserve the original spatiotemporal positional encoding of the generated video latent, allowing geometric guidance to be injected while maintaining consistency with the video model's native latent structure. The resulting framework provides a simple and effective approach for controllable video generation. It improves spatial controllability in viewpoint-dependent editing tasks, including camera re-trajectory, novel-view video synthesis, and geometry-aware video editing, while preserving the generative prior of the underlying video diffusion model. The code is available at: https://github.com/MTLab/PE-Field.

Figures

Figures reproduced from arXiv: 2607.15667 by Haoxiang Li, Qixing Huang, Yunpeng Bai.

Figure 1
Figure 1. Figure 1: Overview of our geometry-aware video generation framework. Given an input image/video, we reconstruct its scene geometry and project reference [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. Figure 2: Visualization of position-aligned attention in a pretrained video [PITH_FULL_IMAGE:figures/full_fig_p003_2.png] view at source ↗
Figure 3
Figure 3. Figure 3: Overview of the proposed geometry-aware cross-attention mechanism for video diffusion transformers. Given the noisy target video latent tokens [PITH_FULL_IMAGE:figures/full_fig_p004_3.png] view at source ↗
Figure 4
Figure 4. Figure 4: Qualitative comparison on the bmx-trees video from DAVIS [Pont-Tuset et al. 2017]. Under the transformed camera trajectory, our method preserves more accurate scene geometry and fine structures, such as the bicycle frame, wheels, and thin foreground/background elements. Compared with TrajectoryCrafter, ReCamMaster, and GEN3C, our results show better geometric consistency and fewer distortions in detailed r… view at source ↗
Figure 6
Figure 6. Figure 6: Ablation study on depth and temporal disambiguation. Compared [PITH_FULL_IMAGE:figures/full_fig_p007_6.png] view at source ↗
Figure 7
Figure 7. Figure 7: Additional visualization of position-aligned attention in a pretrained video diffusion transformer. After assigning projected positional codes to context [PITH_FULL_IMAGE:figures/full_fig_p010_7.png] view at source ↗
Figure 8
Figure 8. Figure 8: Additional qualitative comparisons. Our method better preserves object structure under large camera changes and produces more position-aligned [PITH_FULL_IMAGE:figures/full_fig_p010_8.png] view at source ↗
Figure 9
Figure 9. Figure 9: Image-to-video generation results of our method on the DL3DV dataset. The leftmost image is the input image. [PITH_FULL_IMAGE:figures/full_fig_p011_9.png] view at source ↗
Figure 10
Figure 10. Figure 10: This example shows a case where the reconstructed geometry is inaccurate. Our method can still generate plausible video results even when the [PITH_FULL_IMAGE:figures/full_fig_p011_10.png] view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

31 extracted references · 1 canonical work pages

  1. [2]

    InProceedings of the SIGGRAPH Asia 2025 Conference Papers

    Uni3c: Unifying precisely 3d-enhanced camera and human motion controls for video generation. InProceedings of the SIGGRAPH Asia 2025 Conference Papers. 1–12. Boyuan Chen, Diego Martí Monsó, Yilun Du, Max Simchowitz, Russ Tedrake, and Vin- cent Sitzmann

    2025

  2. [4]

    Oasis: A universe in a transformer.URL: https://oasis-model. github. io(2024). Wanquan Feng, Jiawei Liu, Pengqi Tu, Tianhao Qi, Mingzhen Sun, Tianxiang Ma, Songtao Zhao, Siyu Zhou, and Qian He

    2024

  3. [5]

    Ao Gao, Jingyu Gong, Xin Tan, Zhizhong Zhang, and Yuan Xie

    I2vcontrol-camera: Precise video camera control with adjustable motion strength.arXiv preprint arXiv:2411.06525 (2024). Ao Gao, Jingyu Gong, Xin Tan, Zhizhong Zhang, and Yuan Xie

    Pith/arXiv arXiv 2024

  4. [7]

    Yuchao Gu, Weijia Mao, and Mike Zheng Shou

    Seedance 1.0: Exploring the Boundaries of Video Generation Models.arXiv preprint arXiv:2506.09113(2025). Yuchao Gu, Weijia Mao, and Mike Zheng Shou. 2025a. Long-context autoregressive video modeling with next-frame prediction.arXiv preprint arXiv:2503.19325(2025). Zekai Gu, Rui Yan, Jiahao Lu, Peng Li, Zhiyang Dou, Chenyang Si, Zhen Dong, Qifeng Liu, Chen...

    Pith/arXiv arXiv 2025

  5. [8]

    Hao He, Ceyuan Yang, Shanchuan Lin, Yinghao Xu, Meng Wei, Liangke Gui, Qi Zhao, Gordon Wetzstein, Lu Jiang, and Hongsheng Li

    Cameractrl: Enabling camera control for text-to-video generation.arXiv preprint arXiv:2404.02101(2024). Hao He, Ceyuan Yang, Shanchuan Lin, Yinghao Xu, Meng Wei, Liangke Gui, Qi Zhao, Gordon Wetzstein, Lu Jiang, and Hongsheng Li

    Pith/arXiv arXiv 2024

  6. [9]

    Jiahui Huang, Qunjie Zhou, Hesam Rabeti, Aleksandr Korovko, Huan Ling, Xuanchi Ren, Tianchang Shen, Jun Gao, Dmitry Slepichev, Chen-Hsuan Lin, et al

    Cameractrl ii: Dynamic scene exploration via camera-controlled video diffusion models.arXiv preprint arXiv:2503.10592(2025). Jiahui Huang, Qunjie Zhou, Hesam Rabeti, Aleksandr Korovko, Huan Ling, Xuanchi Ren, Tianchang Shen, Jun Gao, Dmitry Slepichev, Chen-Hsuan Lin, et al . 2025b. Vipe: Video pose engine for 3d geometric perception.arXiv preprint arXiv:2...

    Pith/arXiv arXiv 2025

  7. [10]

    Weijie Kong, Qi Tian, Zijian Zhang, Rox Min, Zuozhuo Dai, Jin Zhou, Jiangfeng Xiong, Xin Li, Bo Wu, Jianwei Zhang, et al

    World and human action models towards gameplay ideation.Nature 638, 8051 (2025), 656–663. Weijie Kong, Qi Tian, Zijian Zhang, Rox Min, Zuozhuo Dai, Jin Zhou, Jiangfeng Xiong, Xin Li, Bo Wu, Jianwei Zhang, et al

    2025

  8. [11]

    Yao-Chih Lee, Zhoutong Zhang, Jiahui Huang, Jui-Hsien Wang, Joon-Young Lee, Jia-Bin Huang, Eli Shechtman, and Zhengqi Li

    Hunyuanvideo: A systematic framework for large video generative models.arXiv preprint arXiv:2412.03603(2024). Yao-Chih Lee, Zhoutong Zhang, Jiahui Huang, Jui-Hsien Wang, Joon-Young Lee, Jia-Bin Huang, Eli Shechtman, and Zhengqi Li

    Pith/arXiv arXiv 2024

  9. [12]

    Ruilong Li, Brent Yi, Junchen Liu, Hang Gao, Yi Ma, and Angjoo Kanazawa

    Generative Video Motion Editing with 3D Point Tracks.arXiv preprint arXiv:2512.02015(2025). Ruilong Li, Brent Yi, Junchen Liu, Hang Gao, Yi Ma, and Angjoo Kanazawa. 2025a. Cameras as Relative Positional Encoding.Advances in Neural Information Processing Systems(2025). Teng Li, Guangcong Zheng, Rui Jiang, Shuigen Zhan, Tao Wu, Yehao Lu, Yining Lin, Chuanyu...

    arXiv 2025

  10. [13]

    InProceedings of the Computer Vision and Pattern Recognition Conference

    You see it, you got it: Learning 3d creation on pose-free videos at scale. InProceedings of the Computer Vision and Pattern Recognition Conference. 2016–2029. Byeongjun Park, Hyojun Go, Hyelin Nam, Byung-Hoon Kim, Hyungjin Chung, and Changick Kim. 2025a. Steerx: Creating any camera-free 3d and 4d scenes with geo- metric steering. InProceedings of the IEEE...

    arXiv 2016

  11. [15]

    Jianlin Su, Murtadha Ahmed, Yu Lu, Shengfeng Pan, Wen Bo, and Yunfeng Liu

    History-guided video diffusion.arXiv preprint arXiv:2502.06764 (2025). Jianlin Su, Murtadha Ahmed, Yu Lu, Shengfeng Pan, Wen Bo, and Yunfeng Liu

    Pith/arXiv arXiv 2025

  12. [16]

    Team Wan, Ang Wang, Baole Ai, Bin Wen, Chaojie Mao, Chen-Wei Xie, Di Chen, Feiwu Yu, Haiming Zhao, Jianxiao Yang, et al

    Roformer: Enhanced transformer with rotary position embedding.Neurocomputing 568 (2024), 127063. Team Wan, Ang Wang, Baole Ai, Bin Wen, Chaojie Mao, Chen-Wei Xie, Di Chen, Feiwu Yu, Haiming Zhao, Jianxiao Yang, et al

    2024

  13. [17]

    Zhouxia Wang, Ziyang Yuan, Xintao Wang, Yaowei Li, Tianshui Chen, Menghan Xia, Ping Luo, and Ying Shan

    Wan: Open and advanced large-scale video generative models.arXiv preprint arXiv:2503.20314(2025). Zhouxia Wang, Ziyang Yuan, Xintao Wang, Yaowei Li, Tianshui Chen, Menghan Xia, Ping Luo, and Ying Shan

    Pith/arXiv arXiv 2025

  14. [18]

    InACM SIGGRAPH 2024 Conference Papers

    Motionctrl: A unified and flexible motion controller for video generation. InACM SIGGRAPH 2024 Conference Papers. 1–11. Tong Wu, Shuai Yang, Ryan Po, Yinghao Xu, Ziwei Liu, Dahua Lin, and Gordon Wet- zstein

    2024

  15. [19]

    Yu Wu, Minsik Jeon, Jen-Hao Rick Chang, Oncel Tuzel, and Shubham Tulsiani

    Video World Models with Long-term Spatial Memory.arXiv preprint arXiv:2506.05284(2025). Yu Wu, Minsik Jeon, Jen-Hao Rick Chang, Oncel Tuzel, and Shubham Tulsiani

    Pith/arXiv arXiv 2025

  16. [20]

    Tianxing Xu, Zixuan Wang, Guangyuan Wang, Li Hu, Zhongyi Zhang, Peng Zhang, Bang Zhang, and Song-Hai Zhang

    RayRoPE: Projective Ray Positional Encoding for Multi-view Attention.arXiv preprint arXiv:2601.15275(2026). Tianxing Xu, Zixuan Wang, Guangyuan Wang, Li Hu, Zhongyi Zhang, Peng Zhang, Bang Zhang, and Song-Hai Zhang

    Pith/arXiv arXiv 2026

  17. [21]

    UCM: Unifying Camera Control and Memory with Time-aware Positional Encoding Warping for World Models. arXiv:2602.22960 [cs.CV] https://arxiv.org/abs/2602.22960 Zhuoyi Yang, Jiayan Teng, Wendi Zheng, Ming Ding, Shiyu Huang, Jiazheng Xu, Yuan- ming Yang, Wenyi Hong, Xiaohan Zhang, Guanyu Feng, et al

    Pith/arXiv arXiv

  18. [22]

    Meng You, Zhiyu Zhu, Hui Liu, and Junhui Hou

    Cogvideox: Text- to-video diffusion models with an expert transformer.arXiv preprint arXiv:2408.06072 (2024). Meng You, Zhiyu Zhu, Hui Liu, and Junhui Hou

    Pith/arXiv arXiv 2024

  19. [23]

    Mark YU, Wenbo Hu, Jinbo Xing, and Ying Shan

    Nvs-solver: Video diffusion model as zero-shot novel view synthesizer.arXiv preprint arXiv:2405.15364(2024). Mark YU, Wenbo Hu, Jinbo Xing, and Ying Shan

    Pith/arXiv arXiv 2024

  20. [24]

    Wei Yu, Runjia Qian, Yumeng Li, Liquan Wang, Songheng Yin, Sri Siddarth Chakar- avarthy P, Dennis Anthony, Yang Ye, Yidi Li, Weiwei Wan, and Animesh Garg

    Trajectorycrafter: Redirect- ing camera trajectory for monocular videos via diffusion models.arXiv preprint arXiv:2503.05638(2025). Wei Yu, Runjia Qian, Yumeng Li, Liquan Wang, Songheng Yin, Sri Siddarth Chakar- avarthy P, Dennis Anthony, Yang Ye, Yidi Li, Weiwei Wan, and Animesh Garg

    Pith/arXiv arXiv 2025

  21. [25]

    arXiv:2603.17117 doi:10.48550/ARXIV.2603.17117 Wangbo Yu, Jinbo Xing, Li Yuan, Wenbo Hu, Xiaoyu Li, Zhipeng Huang, Xiangjun Gao, Tien-Tsin Wong, Ying Shan, and Yonghong Tian

    MosaicMem: Hybrid Spatial Memory for Controllable Video World Models.CoRR abs/2603.17117 (2026). arXiv:2603.17117 doi:10.48550/ARXIV.2603.17117 Wangbo Yu, Jinbo Xing, Li Yuan, Wenbo Hu, Xiaoyu Li, Zhipeng Huang, Xiangjun Gao, Tien-Tsin Wong, Ying Shan, and Yonghong Tian

    doi:10.48550/arxiv.2603.17117 2026

  22. [26]

    Andy Zeng, Shuran Song, Matthias Nießner, Matthew Fisher, Jianxiong Xiao, and Thomas A

    Viewcrafter: Tam- ing video diffusion models for high-fidelity novel view synthesis.arXiv preprint arXiv:2409.02048(2024). Andy Zeng, Shuran Song, Matthias Nießner, Matthew Fisher, Jianxiong Xiao, and Thomas A. Funkhouser

    Pith/arXiv arXiv 2024

  23. [27]

    In2017 IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2017, Honolulu, HI, USA, July 21-26,

    3DMatch: Learning Local Geometric Descriptors from RGB-D Reconstructions. In2017 IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2017, Honolulu, HI, USA, July 21-26,

    2017

  24. [28]

    doi:10.1109/CVPR.2017.29 Shangjin Zhai, Zhichao Ye, Jialin Liu, Weijian Xie, Jiaqi Hu, Zhen Peng, Hua Xue, Danpeng Chen, Xiaomeng Wang, Lei Yang, et al

    IEEE Computer Society, 199–208. doi:10.1109/CVPR.2017.29 Shangjin Zhai, Zhichao Ye, Jialin Liu, Weijian Xie, Jiaqi Hu, Zhen Peng, Hua Xue, Danpeng Chen, Xiaomeng Wang, Lei Yang, et al

    work page doi:10.1109/cvpr.2017.29 2017

  25. [29]

    Lvmin Zhang and Maneesh Agrawala

    Unified Camera Positional Encoding for Controlled Video Generation.arXiv preprint arXiv:2512.07237(2025). Lvmin Zhang and Maneesh Agrawala

    arXiv 2025

  26. [30]

    PE-Field 4D: Video Generation Models as Canvas•9 Zheng Zhu, Xiaofeng Wang, Wangbo Zhao, Chen Min, Bohan Li, Nianchen Deng, Min Dou, Yuqi Wang, Botian Shi, Kai Wang, et al

    Packing input frame context in next-frame prediction models for video generation.arXiv preprint arXiv:2504.12626(2025). PE-Field 4D: Video Generation Models as Canvas•9 Zheng Zhu, Xiaofeng Wang, Wangbo Zhao, Chen Min, Bohan Li, Nianchen Deng, Min Dou, Yuqi Wang, Botian Shi, Kai Wang, et al

    arXiv 2025

  27. [31]

    10•Bai, Li, and Huang WarpingGeneration Fig

    Is sora a world simulator? a comprehensive survey on general world models and beyond.arXiv preprint arXiv:2405.03520(2024). 10•Bai, Li, and Huang WarpingGeneration Fig

    arXiv 2024

  28. [2017]

    arXiv preprint arXiv:1704.00675(2017)

    The 2017 davis challenge on video object segmentation. arXiv preprint arXiv:1704.00675(2017). Xuanchi Ren, Tianchang Shen, Jiahui Huang, Huan Ling, Yifan Lu, Merlin Nimier- David, Thomas Müller, Alexander Keller, Sanja Fidler, and Jun Gao

    Pith/arXiv arXiv 2017

  29. [2024]

    Etched Decart, Quinn McIntyre, Spruce Campbell, Xinlei Chen, and Robert Wachen

    Diffusion forcing: Next-token prediction meets full-sequence diffusion.Advances in Neural Information Processing Systems37 (2024), 24081–24125. Etched Decart, Quinn McIntyre, Spruce Campbell, Xinlei Chen, and Robert Wachen

    2024

  30. [2025]

    In Proceedings of the Computer Vision and Pattern Recognition Conference

    Ac3d: Analyzing and improving 3d camera control in video diffusion transformers. In Proceedings of the Computer Vision and Pattern Recognition Conference. 22875–22889. Jianhong Bai, Menghan Xia, Xiao Fu, Xintao Wang, Lianrui Mu, Jinwen Cao, Zuozhu Liu, Haoji Hu, Xiang Bai, Pengfei Wan, et al. 2025b. Recammaster: Camera-controlled generative rendering from...

    Pith/arXiv arXiv 2025

  31. [2026]

    arXiv:2604.20155 [cs.CV] https://arxiv.org/abs/2604.20155 Yu Gao, Haoyuan Guo, Tuyen Hoang, Weilin Huang, Lu Jiang, Fangyuan Kong, Huixia Li, Jiashi Li, Liang Li, Xiaojie Li, et al

    GSCompleter: A Distillation-Free Plugin for Metric-Aware 3D Gaussian Splatting Completion in Seconds. arXiv:2604.20155 [cs.CV] https://arxiv.org/abs/2604.20155 Yu Gao, Haoyuan Guo, Tuyen Hoang, Weilin Huang, Lu Jiang, Fangyuan Kong, Huixia Li, Jiashi Li, Liang Li, Xiaojie Li, et al

    Pith/arXiv arXiv