Pith. sign in

REVIEW 5 major objections 5 minor 42 references

Controllable Pedestrian Video Editing for Multi-View Driving Scenarios via Motion Sequence

T0 review · 5 major / 5 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read A pose-guided video inpainting pipeline can insert, replace, or remove pedestrians across synchronized driving cameras, and training with the synthesized footage improves BEVFormer pedestrian detection mAP.

desk verdict Sensible multi-view pedestrian editing pipeline; the claimed detection improvement needs stronger statistical footing before it lands. read the letter →

arxiv 2508.00299 v1 pith:44LEIHRN submitted 2025-08-01 cs.CV cs.AIcs.RO

Danzhen Fu , Jiagao Hu , Daiguo Zhou , Fei Wang , Zepeng Wang , Wenhua Liao This is my paper
classification cs.CVcs.AIcs.RO
keywords pedestrianvideoeditingmulti-viewdrivingscenariosinpaintingposesequencecontroldiffusiongenerationautonomousdataaugmentationBEVdetectionnuScenes
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper tries to establish that a diffusion-based video editing framework can manipulate pedestrians in multi-view driving footage under user control while keeping scenes realistic and consistent across cameras. The motivation is that dangerous pedestrian behaviors are rare in real driving data, so synthesizing them could strengthen perception models. The framework crops pedestrians from each camera, stitches the crops into a common canvas, masks them out, and regenerates them from skeleton pose sequences and text prompts. The authors report that training BEVFormer with the synthesized sequences raises pedestrian mAP from 0.4427 to 0.4577, which they take as evidence that the generated data improves detection.

What carries the argument

The load-bearing mechanism is a dynamic crop-and-resize step that converts per-view pedestrian boxes into a standardized composite canvas (for six views, 960 by 720 pixels) so generation happens at a consistent scale. Around each box the mask is expanded to 1.6 times the bounding-box size, creating a visible blending buffer, and the unmasked background is encoded into control latents for inpainting. Pose guidance comes from DWPose skeleton sequences encoded through a pose encoder and concatenated with the inpainting latents; CogVideoX v1.5 serves as the diffusion backbone, and CLIP text embeddings carry clothing-color prompts into the cross-attention layers.

What would settle it

Retrain the same detector with and without synthetic data under identical epochs, data-mixing ratio, and compute, across several seeds; if the mAP difference between the two conditions overlaps zero across seeds, the paper's central claim is not supported.

Watch

Extended reading notes

Core claim

The central discovery is that pedestrian editing in multi-view driving video can be cast as a mask-based video inpainting problem on a unified multi-view canvas, with the edited content controlled by pose sequences. Because all camera crops are resized to one scale and arranged in a fixed layout, the model sees cross-view pedestrian appearance jointly, and zero-filled placeholders keep missing or occluded views aligned. This design supports insertion, replacement, removal, and clothing-color changes while preserving background and temporal continuity. The paper's quantitative claim is that a BEVFormer pedestrian detector trained with the generated data improves at every BEV distance threshold, with mAP rising from 0.4427 to 0.4577.

Load-bearing premise

The claim that the 0.4577 mAP beats 0.4427 because of the synthetic data rests on a single unrepeated training run with no error bars or fixed protocol details, so the gain could be training noise.

Editorial extensions

If this is right

  • Training a BEVFormer detector with synthetic pedestrian sequences improves mAP at all four BEV distance thresholds (0.5, 1, 2, and 4 meters).
  • The same editing pipeline can insert, replace, and remove pedestrians in synchronized multi-camera streams while preserving background and temporal coherence.
  • Text prompts give attribute-level control over edited pedestrians, such as changing clothing colors across frames and views.
  • Because motion is supplied as a 3D sequence projected to each view, the same action can be rendered from several camera perspectives without re-annotation.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • If the mAP gain is genuine, scaling this pipeline could replace parts of real-world data collection for pedestrian safety cases, since rare poses and placements can be authored explicitly.
  • The zero-filled placeholder strategy implies the model may learn to tolerate missing views; a stress test with occluded cameras would show whether cross-view consistency degrades gracefully or silently.
  • The same crop-stitch-inpaint recipe would likely transfer to other road users such as cyclists, scooters, and workers, and to attributes beyond clothing such as carrying objects or occluding pedestrians.
  • A testable extension is varying the 1.6 times mask expansion and measuring both visual blend quality and downstream detection mAP to find whether the buffer size is a sensitive hyperparameter.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

5 major / 5 minor

Summary. The paper presents a multi-view pedestrian video editing framework for driving scenarios. The method crops pedestrian regions in each camera view, resizes and tiles them into a unified canvas, masks out the target regions, and generates pedestrian content using a CogVideoX-based diffusion model conditioned on DWPose keypoint sequences and text prompts. The framework supports pedestrian insertion, replacement, removal, and clothing-color changes. The authors evaluate the approach qualitatively with examples and quantitatively by training BEVFormer on nuScenes data augmented with their generated pedestrians, reporting an mAP improvement from 0.4427 to 0.4577.

Significance. If the reported gains are real, the framework would provide a practical data-augmentation tool for addressing long-tail pedestrian scenarios in autonomous driving. The idea of local cropping and tiling to achieve multi-view consistency is straightforward and potentially useful, and the external downstream evaluation on BEVFormer is an appropriate way to measure utility. However, the current evidence is not yet at the standard required to support these claims: the quantitative evaluation is a single-run comparison with no variance or statistical significance, and the qualitative assessment lacks quantitative video-quality metrics or user studies. The paper also does not ablate the key design choices. The strengths are the clear problem formulation and the sensible use of standard components; the reproducibility is hampered by under-specified training and inference details.

major comments (5)
  1. [§4.3, Table 1] The central quantitative claim that synthetic data improves BEV pedestrian detection is supported only by a single training/evaluation run for one BEVFormer variant, with no error bars, multiple seeds, or statistical test. The reported mAP increase of 0.015 could plausibly arise from random seed variation. Furthermore, the training protocol is underspecified: the paper does not state the mixing ratio of synthetic to real samples, the number of epochs, the learning rate, or the compute budget. Without these details, the reader cannot attribute the improvement to the proposed editing method rather than to other differences in training.
  2. [§4.3] The experimental setup confounds the effect of the generation method with changes in the training label distribution: replacing real pedestrians with synthetic ones alters the number and difficulty of pedestrian instances. A control experiment using a simpler augmentation (e.g., copy-paste of real pedestrian crops, or random repositioning) is needed to establish that the multi-view editing pipeline, rather than mere dataset rebalancing or relabeling, is responsible for the improvement.
  3. [§3.2 and §3.3] The masking and cropping protocol is specified inconsistently: Section 3.2 states that bounding boxes are expanded by a 'fixed margin ratio' and resized to 'a uniform resolution (e.g., 480×240 pixels)', while Section 3.3 states that 'Each bounding box undergoes independent isotropic expansion along width and height dimensions, followed by region cropping that results in 1.6× the original bounding box dimensions.' These two descriptions imply different expansion mechanisms, and the paper never defines the fixed margin ratio or how the 1.6 factor is applied. This ambiguity prevents reproduction.
  4. [§4.1 and §4.4] There is no quantitative evaluation of generation quality or editing fidelity. The paper does not report standard video generation metrics (e.g., FVD, LPIPS), identity or pose consistency scores, or a human perceptual study. The qualitative figures in Figures 3–6 are small and not accompanied by any measured criteria, so claims of 'visual realism', 'spatiotemporal coherence', and 'cross-view consistency' are not substantiated. In addition, there are no ablations for the dynamic cropping, the multi-view tiling layout, the mask buffer zone, or the sequence length, so the contribution of each component is unknown.
  5. [§4.2] The inference-time description of motion control is underspecified. The paper states that 'we first specify a 3D motion sequence which is projected to multi-view perspectives using calibrated camera parameters', but it does not define the representation of the 3D motion sequence (e.g., SMPL parameters, joint positions, or a skeleton), the projection method, or how occlusions across views are handled. Without these details, the claimed controllability via user-specified motion is not reproducible.
minor comments (5)
  1. [§2.2] There is a typographical error: 'V AEs' should read 'VAEs'.
  2. [Throughout] The notation for the pose estimator alternates between 'DWpose' and 'DWPose'; please use a consistent spelling.
  3. [§4.1] The paper states that the model was trained on '85-frame sequences'; as CogVideoX v1.5 typically operates on 49-frame clips, please clarify whether this is the raw clip length, the length after temporal upsampling, or a custom configuration.
  4. [Figures 3–6] The dashed boxes and captions are helpful, but the figures would benefit from indicating the image resolution and frame indices so the reader can better judge temporal coherence.
  5. [§4.4] The phrase 'visually indistinguishable from real pedestrians' in the Pedestrian Insertion paragraph is an overclaim; without a perceptual study or quantitative metric, such a statement cannot be supported.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the central claim rests on an external downstream detection benchmark, not on fitted inputs or self-citations.

full rationale

The paper's derivation chain is not circular. The generation model is trained on nuScenes with a standard masked-latent MSE objective and pose conditioning; no parameter is fitted to the target mAP, and no equation defines the reported mAP improvement in terms of the diffusion training loss. The load-bearing quantitative claim (Table 1) is an external downstream evaluation: synthetic training sequences are produced from the nuScenes training split, used to train BEVFormer, and evaluated on the held-out nuScenes validation set. This benchmark is independent of the generation method's own objective, and it could in principle falsify the method. The qualitative claims are supported by directly inspected output images. The lack of multiple seeds, error bars, and training-protocol details is a statistical-evidence concern, not circularity, and the paper does not rely on self-citations or author-imported uniqueness results.

Assumptions & free parameters 5 free parameters · 5 assumptions · 0 invented entities

The framework leans on pretrained diffusion and pose-estimation models; the only novel machinery is the multi-view crop-and-stitch pipeline and masking protocol, which introduce several hand-picked hyperparameters but no new physical or mathematical entities.

free parameters (5)
  • bbox expansion ratio = 1.6x original box (Sec 3.3)
    Isotropic expansion of pedestrian boxes before masking and cropping; chosen without ablation, affects how much context the model sees.
  • uniform crop resolution = 480x240 pixels (Sec 4.1)
    All cropped regions resized to a common scale for generation; fixed design choice with no reported sensitivity analysis.
  • multi-view tiling layout = 2 rows x 3 columns (Sec 4.1)
    Arrangement of six camera views into a composite canvas; a fixed hyperparameter.
  • training clip length = 85 frames (Sec 4.1)
    Sequence length for video training; fixed without justification.
  • mask buffer zone = 1.6x bounding box (Sec 3.3)
    The expanded mask area is meant to ease blending; the same ratio as the expansion, no ablation.
assumptions (5)
  • domain assumption Diffusion-based video generation with ControlNet-style conditioning learns to inpaint masked pedestrian regions and synthesize controllable motion.
    The method rests on the transferability of pretrained CogVideoX v1.5 and end-to-end training; Secs 3.3-3.4.
  • domain assumption DWPose provides accurate, temporally coherent whole-body keypoints for small distant pedestrians in driving scenes.
    Used to generate motion sequences; Sec 3.4.
  • domain assumption View projection of 3D pedestrian annotations via calibrated camera intrinsics and extrinsics is correct, yielding aligned 2D boxes across views.
    Necessary for the multi-view consistency framework; Sec 3.2.
  • domain assumption The pretrained video understanding model extracts reliable clothing attributes from reference pedestrians.
    Text prompts for color control are generated from this model; Sec 4.1.
  • standard math MSE-based noise prediction is a sufficient training objective for the inpainting and pose control task.
    Follows CogVideoX v1.5 training; Sec 3.4.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Controllable Pedestrian Video Editing for Multi-View Driving Scenarios via Motion Sequence." pith.science (2026). https://pith.science/paper/44LEIHRN

@misc{pith2026250800299,
  author       = {Pith},
  title        = {Pith review of: Controllable Pedestrian Video Editing for Multi-View Driving Scenarios via Motion Sequence},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/44LEIHRN}},
  note         = {Machine review of arXiv:2508.00299}
}
read the original abstract

Pedestrian detection models in autonomous driving systems often lack robustness due to insufficient representation of dangerous pedestrian scenarios in training datasets. To address this limitation, we present a novel framework for controllable pedestrian video editing in multi-view driving scenarios by integrating video inpainting and human motion control techniques. Our approach begins by identifying pedestrian regions of interest across multiple camera views, expanding detection bounding boxes with a fixed ratio, and resizing and stitching these regions into a unified canvas while preserving cross-view spatial relationships. A binary mask is then applied to designate the editable area, within which pedestrian editing is guided by pose sequence control conditions. This enables flexible editing functionalities, including pedestrian insertion, replacement, and removal. Extensive experiments demonstrate that our framework achieves high-quality pedestrian editing with strong visual realism, spatiotemporal coherence, and cross-view consistency. These results establish the proposed method as a robust and versatile solution for multi-view pedestrian video generation, with broad potential for applications in data augmentation and scenario simulation in autonomous driving.

Figures

Figures reproduced from arXiv: 2508.00299 by the authors.

Figure 1
Figure 1. Architecture of the proposed pedestrian editing pipeline with motion sequence control. [PITH_FULL_IMAGE:figures/full_fig_p003_1.png] view at source ↗
Figure 2
Figure 2. Dynamic-scale cropping with subsequent resizing to uni [PITH_FULL_IMAGE:figures/full_fig_p004_2.png] view at source ↗
Figure 3
Figure 3. Qualitative results of pedestrian replacement in a multi-view scenario. Gray dashed boxes indicate the original pedestrian, while [PITH_FULL_IMAGE:figures/full_fig_p005_3.png] view at source ↗
Figures from the paper (3 more)
Figure 4
Figure 4. Figure 4: Qualitative results of pedestrian insertion in a multi-view scenario. Inserted pedestrians are highlighted by pink dashed boxes. [PITH_FULL_IMAGE:figures/full_fig_p006_4.png]
Figure 5
Figure 5. Figure 5: Qualitative results of pedestrian removal in a multi-view scenario. Pedestrians targeted for removal are indicated by green dashed [PITH_FULL_IMAGE:figures/full_fig_p007_5.png]
Figure 6
Figure 6. Figure 6: Qualitative results of clothing color manipulation in a multi-view scenario. Edited pedestrians are highlighted with cyan dashed [PITH_FULL_IMAGE:figures/full_fig_p008_6.png]

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

42 extracted references · 20 canonical work pages

  1. [1]

    nuscenes: A multi- modal dataset for autonomous driving

    Holger Caesar, Varun Bankiti, Alex H Lang, Sourabh V ora, Venice Erin Liong, Qiang Xu, Anush Krishnan, Yu Pan, Gi- ancarlo Baldan, and Oscar Beijbom. nuscenes: A multi- modal dataset for autonomous driving. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 11621–11631, 2020. 4

    work page 2020

  2. [2]

    Realtime multi-person 2d pose estimation using part affinity fields

    Zhe Cao, Tomas Simon, Shih-En Wei, and Yaser Sheikh. Realtime multi-person 2d pose estimation using part affinity fields. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 7291–7299, 2017. 2

    work page 2017

  3. [3]

    Magicpose: Realistic human poses and facial expressions retargeting with identity-aware diffusion

    Di Chang, Yichun Shi, Quankai Gao, Jessica Fu, Hongyi Xu, Guoxian Song, Qing Yan, Yizhe Zhu, Xiao Yang, and Mo- hammad Soleymani. Magicpose: Realistic human poses and facial expressions retargeting with identity-aware diffusion. arXiv preprint arXiv:2311.12052, 2023. 1, 2

    arXiv 2023

  4. [4]

    Structure and content-guided video synthesis with diffusion models

    Patrick Esser, Johnathan Chiu, Parmida Atighehchian, Jonathan Granskog, and Anastasis Germanidis. Structure and content-guided video synthesis with diffusion models. In Proceedings of the IEEE/CVF international conference on computer vision, pages 7346–7356, 2023. 2

    work page 2023

  5. [5]

    Magicdrive3d: Controllable 3d genera- tion for any-view rendering in street scenes

    Ruiyuan Gao, Kai Chen, Zhihao Li, Lanqing Hong, Zhenguo Li, and Qiang Xu. Magicdrive3d: Controllable 3d genera- tion for any-view rendering in street scenes. arXiv preprint arXiv:2405.14475, 2024. 1

    arXiv 2024

  6. [6]

    Vista: A generalizable driving world model with high fidelity and versatile controllability

    Shenyuan Gao, Jiazhi Yang, Li Chen, Kashyap Chitta, Yihang Qiu, Andreas Geiger, Jun Zhang, and Hongyang Li. Vista: A generalizable driving world model with high fidelity and versatile controllability. arXiv preprint arXiv:2405.17398, 2024. 1

    arXiv 2024

  7. [7]

    8 Densepose: Dense human pose estimation in the wild

    Rıza Alp G ¨uler, Natalia Neverova, and Iasonas Kokkinos. 8 Densepose: Dense human pose estimation in the wild. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 7297–7306, 2018. 3

    work page 2018

  8. [8]

    Animatediff: Animate your personalized text- to-image diffusion models without specific tuning

    Yuwei Guo, Ceyuan Yang, Anyi Rao, Zhengyang Liang, Yaohui Wang, Yu Qiao, Maneesh Agrawala, Dahua Lin, and Bo Dai. Animatediff: Animate your personalized text- to-image diffusion models without specific tuning. arXiv preprint arXiv:2307.04725, 2023. 2

    arXiv 2023

Show all 42 references
  1. [9]

    Animate anyone: Consistent and controllable image- to-video synthesis for character animation

    Li Hu. Animate anyone: Consistent and controllable image- to-video synthesis for character animation. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 8153–8163, 2024. 1, 2, 4

    2024

  2. [10]

    Composer: creative and controllable im- age synthesis with composable conditions

    Lianghua Huang, Di Chen, Yu Liu, Yujun Shen, Deli Zhao, and Jingren Zhou. Composer: creative and controllable im- age synthesis with composable conditions. In Proceedings of the 40th International Conference on Machine Learning , pages 13753–13773, 2023. 2

    2023

  3. [11]

    Text2performer: Text- driven human video generation

    Yuming Jiang, Shuai Yang, Tong Liang Koh, Wayne Wu, Chen Change Loy, and Ziwei Liu. Text2performer: Text- driven human video generation. In Proceedings of the IEEE/CVF International Conference on Computer Vision , pages 22747–22757, 2023. 2

    2023

  4. [12]

    Dreampose: Fashion image-to-video synthesis via stable diffusion

    Johanna Karras, Aleksander Holynski, Ting-Chun Wang, and Ira Kemelmacher-Shlizerman. Dreampose: Fashion image-to-video synthesis via stable diffusion. In 2023 IEEE/CVF International Conference on Computer Vision (ICCV), pages 22623–22633. IEEE, 2023. 1, 2

    2023

  5. [13]

    3d gaussian splatting for real-time radiance field rendering

    Bernhard Kerbl, Georgios Kopanas, Thomas Leimk ¨uhler, and George Drettakis. 3d gaussian splatting for real-time radiance field rendering. ACM Trans. Graph., 42(4):139–1,

  6. [14]

    Text2video-zero: Text- to-image diffusion models are zero-shot video generators

    Levon Khachatryan, Andranik Movsisyan, Vahram Tade- vosyan, Roberto Henschel, Zhangyang Wang, Shant Navasardyan, and Humphrey Shi. Text2video-zero: Text- to-image diffusion models are zero-shot video generators. In Proceedings of the IEEE/CVF International Conference on Comput...

    2023

  7. [15]

    Hunyuanvideo: A systematic framework for large video generative models

    Weijie Kong, Qi Tian, Zijian Zhang, Rox Min, Zuozhuo Dai, Jin Zhou, Jiangfeng Xiong, Xin Li, Bo Wu, Jianwei Zhang, et al. Hunyuanvideo: A systematic framework for large video generative models. arXiv preprint arXiv:2412.03603, 2024. 2

    2024 arXiv

  8. [16]

    Smpl: a skinned multi- person linear model

    Matthew Loper, Naureen Mahmood, Javier Romero, Gerard Pons-Moll, and Michael J Black. Smpl: a skinned multi- person linear model. ACM Transactions on Graphics (TOG), 34(6):1–16, 2015. 2

    2015

  9. [17]

    Follow your pose: Pose- guided text-to-video generation using pose-free videos

    Yue Ma, Yingqing He, Xiaodong Cun, Xintao Wang, Siran Chen, Xiu Li, and Qifeng Chen. Follow your pose: Pose- guided text-to-video generation using pose-free videos. In Proceedings of the AAAI Conference on Artificial Intelli- gence, pages 4117–4125, 2024. 2

    2024

  10. [18]

    T2i-adapter: Learning adapters to dig out more controllable ability for text-to-image diffusion models

    Chong Mou, Xintao Wang, Liangbin Xie, Yanze Wu, Jian Zhang, Zhongang Qi, and Ying Shan. T2i-adapter: Learning adapters to dig out more controllable ability for text-to-image diffusion models. In Proceedings of the AAAI conference on artificial intelligence, pages 4296–4304, 2024. 2

    2024

  11. [19]

    Recondreamer: Crafting world models for driving scene reconstruction via online restora- tion

    Chaojun Ni, Guosheng Zhao, Xiaofeng Wang, Zheng Zhu, Wenkang Qin, Guan Huang, Chen Liu, Yuyin Chen, Yida Wang, Xueyang Zhang, et al. Recondreamer: Crafting world models for driving scene reconstruction via online restora- tion. In Proceedings of the Computer Vision and Pattern...

    2025

  12. [20]

    Glide: Towards photorealistic image generation and editing with text-guided diffusion models.arXiv preprint arXiv:2112.10741, 2021

    Alex Nichol, Prafulla Dhariwal, Aditya Ramesh, Pranav Shyam, Pamela Mishkin, Bob McGrew, Ilya Sutskever, and Mark Chen. Glide: Towards photorealistic image generation and editing with text-guided diffusion models.arXiv preprint arXiv:2112.10741, 2021. 2

    2021 arXiv

  13. [21]

    Fatezero: Fus- ing attentions for zero-shot text-based video editing

    Chenyang Qi, Xiaodong Cun, Yong Zhang, Chenyang Lei, Xintao Wang, Ying Shan, and Qifeng Chen. Fatezero: Fus- ing attentions for zero-shot text-based video editing. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 15932–15942, 2023. 2

    2023

  14. [22]

    Learning transferable visual models from natural language supervi- sion

    Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, et al. Learning transferable visual models from natural language supervi- sion. In International conference on machine learning, ...

    2021

  15. [23]

    High-resolution image synthesis with latent diffusion models

    Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Bj ¨orn Ommer. High-resolution image synthesis with latent diffusion models. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 10684–10695, 2022. 2

    2022

  16. [24]

    Photorealistic text-to-image diffusion models with deep language understanding

    Chitwan Saharia, William Chan, Saurabh Saxena, Lala Li, Jay Whang, Emily L Denton, Kamyar Ghasemipour, Raphael Gontijo Lopes, Burcu Karagol Ayan, Tim Salimans, et al. Photorealistic text-to-image diffusion models with deep language understanding. Advances in neural information...

    2022

  17. [25]

    Make-a-video: Text-to-video generation without text-video data

    Uriel Singer, Adam Polyak, Thomas Hayes, Xi Yin, Jie An, Songyang Zhang, Qiyuan Hu, Harry Yang, Oron Ashual, Oran Gafni, et al. Make-a-video: Text-to-video generation without text-video data. arXiv preprint arXiv:2209.14792 ,

    arXiv

  18. [26]

    Object- stitch: Object compositing with diffusion model

    Yizhi Song, Zhifei Zhang, Zhe Lin, Scott Cohen, Brian Price, Jianming Zhang, Soo Ye Kim, and Daniel Aliaga. Object- stitch: Object compositing with diffusion model. In Pro- ceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 18310–18319, 2023. 2

    2023

  19. [27]

    Wan: Open and advanced large-scale video generative models

    Team Wan, Ang Wang, Baole Ai, Bin Wen, Chaojie Mao, Chen-Wei Xie, Di Chen, Feiwu Yu, Haiming Zhao, Jianx- iao Yang, et al. Wan: Open and advanced large-scale video generative models. arXiv preprint arXiv:2503.20314, 2025. 2

    2025 arXiv

  20. [28]

    Disco: Disentangled control for realistic human dance generation

    Tan Wang, Linjie Li, Kevin Lin, Yuanhao Zhai, Chung- Ching Lin, Zhengyuan Yang, Hanwang Zhang, Zicheng Liu, and Lijuan Wang. Disco: Disentangled control for realistic human dance generation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition ,...

    2024

  21. [29]

    Unianimate: Taming unified video diffusion mod- els for consistent human image animation

    Xiang Wang, Shiwei Zhang, Changxin Gao, Jiayu Wang, Xiaoqiang Zhou, Yingya Zhang, Luxin Yan, and Nong Sang. Unianimate: Taming unified video diffusion mod- els for consistent human image animation. arXiv preprint arXiv:2406.01188, 2024. 1, 2 9

    2024 arXiv

  22. [30]

    Drivedreamer: Towards real-world- drive world models for autonomous driving

    Xiaofeng Wang, Zheng Zhu, Guan Huang, Xinze Chen, Jia- gang Zhu, and Jiwen Lu. Drivedreamer: Towards real-world- drive world models for autonomous driving. In European Conference on Computer Vision , pages 55–72. Springer,

  23. [31]

    Panacea: Panoramic and controllable video generation for autonomous driving

    Yuqing Wen, Yucheng Zhao, Yingfei Liu, Fan Jia, Yanhui Wang, Chong Luo, Chi Zhang, Tiancai Wang, Xiaoyan Sun, and Xiangyu Zhang. Panacea: Panoramic and controllable video generation for autonomous driving. In Proceedings of the IEEE/CVF Conference on Computer Vision and Patter...

    2024

  24. [32]

    Tune-a-video: One-shot tuning of image diffusion models for text-to-video generation

    Jay Zhangjie Wu, Yixiao Ge, Xintao Wang, Stan Weixian Lei, Yuchao Gu, Yufei Shi, Wynne Hsu, Ying Shan, Xiaohu Qie, and Mike Zheng Shou. Tune-a-video: One-shot tuning of image diffusion models for text-to-video generation. In Proceedings of the IEEE/CVF International Conference...

    2023

  25. [33]

    Magicanimate: Temporally consistent human im- age animation using diffusion model

    Zhongcong Xu, Jianfeng Zhang, Jun Hao Liew, Hanshu Yan, Jia-Wei Liu, Chenxu Zhang, Jiashi Feng, and Mike Zheng Shou. Magicanimate: Temporally consistent human im- age animation using diffusion model. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Reco...

    2024

  26. [34]

    Effec- tive whole-body pose estimation with two-stages distillation

    Zhendong Yang, Ailing Zeng, Chun Yuan, and Yu Li. Effec- tive whole-body pose estimation with two-stages distillation. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 4210–4220, 2023. 2, 4

    2023

  27. [35]

    Cogvideox: Text-to-video diffusion models with an expert transformer

    Zhuoyi Yang, Jiayan Teng, Wendi Zheng, Ming Ding, Shiyu Huang, Jiazheng Xu, Yuanming Yang, Wenyi Hong, Xiao- han Zhang, Guanyu Feng, et al. Cogvideox: Text-to-video diffusion models with an expert transformer. arXiv preprint arXiv:2408.06072, 2024. 2, 4, 5

    2024 arXiv

  28. [36]

    Ip- adapter: Text compatible image prompt adapter for text-to- image diffusion models

    Hu Ye, Jun Zhang, Sibo Liu, Xiao Han, and Wei Yang. Ip- adapter: Text compatible image prompt adapter for text-to- image diffusion models. arXiv preprint arXiv:2308.06721,

    arXiv

  29. [37]

    Adding conditional control to text-to-image diffusion models

    Lvmin Zhang, Anyi Rao, and Maneesh Agrawala. Adding conditional control to text-to-image diffusion models. In Proceedings of the IEEE/CVF international conference on computer vision, pages 3836–3847, 2023. 2

    2023

  30. [38]

    Mim- icmotion: High-quality human motion video generation with confidence-aware pose guidance

    Yuang Zhang, Jiaxi Gu, Li-Wen Wang, Han Wang, Junqi Cheng, Yuefeng Zhu, and Fangyuan Zou. Mim- icmotion: High-quality human motion video generation with confidence-aware pose guidance. arXiv preprint arXiv:2406.19680, 2024. 1, 2

    2024 arXiv

  31. [39]

    Drivedreamer4d: World models are effective data machines for 4d driving scene rep- resentation

    Guosheng Zhao, Chaojun Ni, Xiaofeng Wang, Zheng Zhu, Xueyang Zhang, Yida Wang, Guan Huang, Xinze Chen, Boyuan Wang, Youyi Zhang, et al. Drivedreamer4d: World models are effective data machines for 4d driving scene rep- resentation. In Proceedings of the Computer Vision and Pat...

    2025

  32. [40]

    Drivedreamer-2: Llm-enhanced world models for diverse driving video generation

    Guosheng Zhao, Xiaofeng Wang, Zheng Zhu, Xinze Chen, Guan Huang, Xiaoyi Bao, and Xingang Wang. Drivedreamer-2: Llm-enhanced world models for diverse driving video generation. In Proceedings of the AAAI Con- ference on Artificial Intelligence, pages 10412–10420, 2025. 1

    2025

  33. [41]

    Open-sora: Democratizing efficient video production for all

    Zangwei Zheng, Xiangyu Peng, Tianji Yang, Chenhui Shen, Shenggui Li, Hongxin Liu, Yukun Zhou, Tianyi Li, and Yang You. Open-sora: Democratizing efficient video production for all. arXiv preprint arXiv:2412.20404, 2024. 2

    2024 arXiv

  34. [42]

    Champ: Controllable and consistent human image an- imation with 3d parametric guidance

    Shenhao Zhu, Junming Leo Chen, Zuozhuo Dai, Zilong Dong, Yinghui Xu, Xun Cao, Yao Yao, Hao Zhu, and Siyu Zhu. Champ: Controllable and consistent human image an- imation with 3d parametric guidance. In European Confer- ence on Computer Vision, pages 145–162. Springer, 2024. 1, 2, 4 10

    2024

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.