Pith. sign in

REVIEW 4 major objections 5 minor 2 cited by

Low-Cost Test-Time Adaptation for Robust Video Editing

T0 review · 4 major / 5 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read The paper claims that a lightweight test-time adaptation loop, built from motion-guided masked reconstruction and prompt perturbation, can be added to existing video diffusion editors to reduce temporal artifacts and prompt overfitting…

desk verdict The first TTA framework for video editing has a sensible high-level design, but the paper as written is not implementable: the auxiliary losses have no gradient path to the UNet, and the experiments are purely qualitative. read the letter →

arxiv 2507.21858 v1 pith:WGDJK5WP submitted 2025-07-29 cs.CV

Jianhui Wang , Yinda Chen , Yangfan He , Xinyuan Song , Yi Xin , Dapeng Zhang , Zhongwei Wan , Bin Li
show 1 more author
Rongchao Zhang
This is my paper
classification cs.CV
keywords test-timeadaptationvideoeditingself-supervisedlearningmaskedautoencodermotion-awaremaskingpromptoverfittingtemporalconsistencydiffusionmodels
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Vid-TTA is a plug-in test-time adaptation loop for existing diffusion-based video editors. The paper claims that by fine-tuning a video editor's UNet on each input clip during inference, using self-supervised tasks rather than labeled training data, both temporal consistency and prompt robustness improve without costly retraining. The two auxiliary tasks are motion-aware masked reconstruction of video latents, which forces the model to rebuild high-motion patches, and prompt perturbation with masked reconstruction, which is meant to stop the model overfitting to simple prompts. A meta-learned weighting module adjusts the contribution of the noise, video, and prompt losses per video. If the claim holds, any current video editing model can be upgraded by adding this lightweight loop.

What carries the argument

The load-bearing object is the per-video adaptation objective $L_{\text{total}} = w_1 L_{\text{noise}} + w_2 \lambda_{\text{video}} L_{\text{video}} + w_3 \lambda_{\text{text}} L_{\text{prompt}}$. $L_{\text{video}}$ is a masked autoencoding loss computed on VAE latent frames: object detection and optical flow identify high-motion patches, those patches are masked with ratio 0.75 inside the detected bounding box and 0.2 in the background, and the model must reconstruct the masked latents. $L_{\text{prompt}}$ is a masked language reconstruction loss: augmented prompts, generated by back-translation, synonym substitution, and syntactic reformation, are token-masked at 30 percent and a linear head predicts the original token distribution from the text encoder's hidden states. The meta-learning part is a small MLP that concatenates a spatially averaged video feature with the text encoder's mean hidden state and outputs three softmax weights, so the loss balance adapts to each clip. The claim is that these three losses jointly push the UNet toward motion-consistent and prompt-robust behavior at test time.

What would settle it

Trace the gradient of $L_{\text{prompt}}$ with respect to the UNet parameters during the adaptation step. If it is zero or blocked, because the loss depends only on the text encoder and the linear reconstruction head, then the prompt-robustness mechanism cannot be operating through the UNet update and the claimed mitigation of prompt overfitting is unexplained. A second check is to ablate $L_{\text{prompt}}$ on a fixed benchmark and measure prompt-fidelity metrics; if results do not change, the prompt component is inert.

Watch

Extended reading notes

Core claim

The paper's central claim is that test-time adaptation, previously applied to image classification and video understanding, can be transplanted onto video editing as a per-clip fine-tuning step. Vid-TTA takes an existing diffusion video editor and, during inference on a given clip, updates the UNet weights against a total loss made of the original noise-prediction loss, a motion-guided masked autoencoding loss on VAE latents, and a masked prompt-reconstruction loss on text-encoder hidden states. Motion selection is explicit: object detection localizes moving entities, optical flow ranks patches inside the detected box by motion intensity, and the highest-motion patches plus random background patches are masked for reconstruction. Prompt robustness is pursued by generating semantically equivalent augmented prompts and asking the model to reconstruct masked tokens from the text encoder's hidden representation. The three losses are combined with per-video weights emitted by a small MLP, and the paper claims this mechanism yields better temporal consistency and prompt fidelity than the unadapted editor, at low extra cost.

Load-bearing premise

The claim rests on the assumption that gradients from the prompt-reconstruction loss $L_{\text{prompt}}$, which is defined entirely on text-encoder outputs, can reach and usefully update the UNet parameters; the paper does not specify that gradient path.

Editorial extensions

If this is right

  • Any existing UNet-based video editing model could accept Vid-TTA as a plug-in and gain improved temporal coherence on clips with fast or complex motion, without retraining on annotated data.
  • Edits specified by unusual or detailed prompts should be applied more faithfully, with fewer spurious changes to unrelated objects or colors.
  • Because the loss weights are chosen per video, the same adaptation loop should scale across different editing models and content types, spending more optimization effort where motion or prompt difficulty is highest.
  • The per-instance fine-tuning cost is meant to stay low enough, running on a single GPU for a short adaptation, to be practical in content-creation workflows.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The paper's evidence is visual; a quantitative benchmark measuring temporal consistency and prompt fidelity would be needed to confirm the claimed gains and to make the method comparable to existing editing baselines.
  • The prompt-reconstruction loss as written operates on text-encoder hidden states, so unless a gradient path to the UNet exists, the prompt-robustness benefit likely collapses into the video-reconstruction loss; an ablation that zeroes $L_{\text{prompt}}$ would test this.
  • The motion-masking recipe, object detection plus optical-flow-ranked patch masking, could transfer to other test-time adaptation settings, such as image editing or domain adaptation, wherever salient regions need to be preserved during self-supervised reconstruction.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. The manuscript proposes Vid-TTA, a test-time adaptation framework for video editing. The method combines a motion-aware masked autoencoding loss on video latents, a prompt perturbation plus reconstruction loss on text-encoder hidden states, and a meta-learning-style dynamic loss weighting module, with the aim of fine-tuning the UNet backbone of existing video editing models on each test video. The paper claims that Vid-TTA is the first test-time adaptation method for video editing, that it improves temporal consistency and mitigates prompt overfitting, and that it does so with low computational overhead. The experimental section, however, contains only qualitative visualizations and no quantitative results, and the conclusion's reference to human studies is not backed by any described study.

Significance. If the framework were correctly specified and its gains quantitatively established, test-time adaptation for video editing would be a timely and potentially useful contribution, particularly the idea of using self-supervised reconstruction tasks during inference. However, as written, the central mechanism is not implementable: the two auxiliary losses in Eqs. (5) and (6) are not functions of the UNet parameters that the method claims to update. Moreover, the empirical evidence is restricted to qualitative figures, with no metrics, tables, or statistical tests, and the claimed human studies are never described. Because the core optimization procedure is underspecified and the central empirical claim is unsupported, the significance of the contribution cannot be assessed in its current form. The paper does provide a clear motivation and a reasonable high-level decomposition of the problem, but the missing formal definitions and missing experiments are load-bearing gaps.

major comments (4)
  1. [Section 3.2, Eq. (5)] The video reconstruction loss is defined as L_video = (1/N) Σ ||z_t(i) − \tilde z_t(i)||², where \tilde z_t = M_t ⊙ z_t and z_t is the VAE-encoded input latent. Since z_t does not depend on the UNet parameters θ, this loss is independent of θ; ∂L_video/∂θ = 0 by construction. Minimizing this loss drives the masked entries of z_t toward zero rather than adapting the editing model. The paper's central claim, made in Section 3.1 and Figure 2(c), is that the UNet is updated using auxiliary reconstruction tasks, but Eq. (5) provides no gradient path to θ. The loss needs to be redefined in terms of a UNet forward pass, e.g., by reconstructing masked latents after they are processed through the editing model, or by comparing UNet outputs on masked versus unmasked inputs.
  2. [Section 3.3, Eq. (6) and Section 3.4, Eq. (11)] The prompt reconstruction loss L_prompt = CE(ϕ(T_m), T_a) is computed entirely on text-encoder hidden states through a linear head ϕ, and the text encoder is described as frozen in Section 3.3. Consequently, this loss has no dependence on the UNet parameters θ, so ∂L_prompt/∂θ = 0. The only remaining candidate term in Eq. (11), L_noise, is never defined anywhere in the manuscript. Thus the total loss in Eq. (11) is not a well-specified objective for adapting θ. The authors need to either specify how gradients from L_prompt reach the UNet (e.g., by perturbing the text-conditioning input to the UNet and backpropagating through it) or remove the claim that this loss contributes to UNet adaptation.
  3. [Section 4 and Section 5] The experiments section contains only 'Experimental Settings' (Section 4.1) and qualitative visualizations; there are no quantitative results, no metrics (e.g., CLIP score, temporal consistency, Fréchet Video Distance), no tables, and no statistical tests. The abstract and conclusion claim that 'extensive experiments' demonstrate significant improvements and that gains are 'confirmed by human studies,' but no human study is described and no quantitative finding appears anywhere in the manuscript. Section 5.1 says visual results 'corroborate our quantitative findings,' yet no such findings are present. This omission undermines the central empirical claims of the paper.
  4. [Section 3.4, Eq. (10)] The dynamic loss balancing is described as 'meta-learning driven,' but Eq. (10) simply computes softmax(ψ([v_p; t])), where ψ is an MLP and g is a linear mapping. No meta-training procedure is specified: there is no outer-loop objective, no meta-training dataset, and no description of how g and ψ are learned. As presented, this is an ordinary learned or hand-initialized weighting network, not a meta-learning mechanism. The authors need to either provide the meta-learning formulation (e.g., bi-level optimization over a distribution of videos) or temper the claim.
minor comments (5)
  1. [Section 3.2, Eq. (5)] The index t is used inconsistently: L_video is written as a sum over N terms without specifying whether the sum is over frames or spatial positions, and the relation between z_t and z_t(i) should be clarified.
  2. [Section 4.1] The hyperparameters n, r_f, r_b, and the token masking ratio are introduced in Section 3 but their chosen values are only partially reported in Section 4.1; the number of augmented prompts n is stated as '3 additional prompts' but the variable n is never explicitly defined.
  3. [Conclusion] The conclusion states that gains are 'confirmed by human studies,' but no human evaluation is described in Section 4 or 5; either a detailed description of the user study should be added or the sentence should be removed.
  4. [Throughout] There are numerous typos and formatting issues, including 'adesert' and 'correctlyrendered' in the Figure 1 caption, inconsistent capitalization of 'arXiv' in the references, duplicated references (e.g., entries [20] and [21], [32] and [33], [35] and [36] appear to be duplicates), and inconsistent use of 'V AE' spacing.
  5. [Section 5.1, Figure 5] The caption for Figure 5 is cut off mid-sentence ('In scenes like ...'), and the figure itself is not described with enough detail to determine what is being compared in the latent-MAE versus pixel-MAE ablation.

Circularity Check

0 steps flagged · score 2.0 of 10

No circular derivation: the claimed editing gains rest on qualitative comparisons, and the formal gaps in the TTA losses are implementability problems rather than input-output circularity.

full rationale

Vid-TTA does not fit parameters to the quantities it claims to predict; no target metric is optimized to produce the reported improvements, and the experimental section contains only qualitative before/after visuals. The two auxiliary losses in Eqs. (5) and (6) do not appear to be functions of the UNet parameters being adapted, and L_noise in Eq. (11) is never defined, so the test-time gradient path is unspecified. This is a serious completeness/validity issue, but it is not circularity: there is no equation in which a claimed prediction is identical by construction to the fitted input. The related-work section cites several papers sharing co-authors (e.g., Refs. [10], [31], [34], [46], [57], [58]), but these citations are contextual and none supplies a load-bearing premise such as a uniqueness theorem or a fitted parameter that forces the central result. The conclusion's assertion that gains are 'confirmed by human studies' is also unsupported by the reported experiments. These are soundness and reporting concerns, not circularity; thus the paper receives a low circularity score despite its methodological gaps.

Assumptions & free parameters 6 free parameters · 5 assumptions · 0 invented entities

The method introduces several hand-set hyperparameters, relies on pretrained detectors and flow models, and assumes a gradient path from prompt reconstruction to UNet that is not established. The MLP weight predictor is invoked as meta-learned but its training is unspecified.

free parameters (6)
  • r_f (object mask ratio) = 0.75
    Chosen by hand in Section 4.1 to determine how many object patches to mask in Eq. (4).
  • r_b (background mask ratio) = 0.2
    Chosen by hand in Section 4.1 for background masking in Eq. (4).
  • n (number of augmented prompts) = 3
    Section 4.1 states each prompt is augmented to 3 variants for the prompt reconstruction task.
  • prompt token masking ratio = 0.3
    Section 4.1 sets 30% of tokens in augmented prompts be masked for L_prompt.
  • lambda_video = 0.1
    Scaling factor for L_video in Eq. (11), set by hand in Section 4.1.
  • lambda_text = 0.1
    Scaling factor for L_prompt in Eq. (11), set by hand in Section 4.1.
assumptions (5)
  • domain assumption YOLOv8 object detection and GMFlow optical flow accurately localize moving objects and high-motion patches.
    The masking in Eq. (4) depends on these pretrained models; failures would misguide reconstruction and hurt adaptation.
  • ad hoc to paper The text reconstruction loss L_prompt can produce gradients that improve the UNet's prompt handling.
    L_prompt is defined on text-encoder outputs, yet the paper states UNet parameters are updated using this loss; no gradient path is shown.
  • ad hoc to paper The linear mapping g and MLP psi are meta-learned and provide effective per-video loss weights.
    Section 3.4 invokes meta-learning but gives no training procedure, dataset, or loss for these modules.
  • domain assumption The VAE latent space supports meaningful masked autoencoding for video.
    The reconstruction loss in Eq. (5) operates in latent space, assumed to preserve the information needed for temporal consistency.
  • domain assumption Test-time fine-tuning of the UNet with auxiliary losses does not degrade the original editing capabilities.
    The paper provides no evaluation of catastrophic forgetting or side effects on unedited regions.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Low-Cost Test-Time Adaptation for Robust Video Editing." pith.science (2026). https://pith.science/paper/WGDJK5WP

@misc{pith2026250721858,
  author       = {Pith},
  title        = {Pith review of: Low-Cost Test-Time Adaptation for Robust Video Editing},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/WGDJK5WP}},
  note         = {Machine review of arXiv:2507.21858}
}
read the original abstract

Video editing is a critical component of content creation that transforms raw footage into coherent works aligned with specific visual and narrative objectives. Existing approaches face two major challenges: temporal inconsistencies due to failure in capturing complex motion patterns, and overfitting to simple prompts arising from limitations in UNet backbone architectures. While learning-based methods can enhance editing quality, they typically demand substantial computational resources and are constrained by the scarcity of high-quality annotated data. In this paper, we present Vid-TTA, a lightweight test-time adaptation framework that personalizes optimization for each test video during inference through self-supervised auxiliary tasks. Our approach incorporates a motion-aware frame reconstruction mechanism that identifies and preserves crucial movement regions, alongside a prompt perturbation and reconstruction strategy that strengthens model robustness to diverse textual descriptions. These innovations are orchestrated by a meta-learning driven dynamic loss balancing mechanism that adaptively adjusts the optimization process based on video characteristics. Extensive experiments demonstrate that Vid-TTA significantly improves video temporal consistency and mitigates prompt overfitting while maintaining low computational overhead, offering a plug-and-play performance boost for existing video editing models.

Figures

Figures reproduced from arXiv: 2507.21858 by the authors.

Figure 1
Figure 1. (Top) The input source video and prompt. (Middle) An attempt to switch the scene to a desert, which clearly exposes issues in the paddle motion, including discontinuities, flickering, and occasional disappearance. (Bottom) An attempt at more de￾tailed editing. While the “at sunset” and “golden sky” elements are correctly rendered, the intended change to “vivid crimson pad￾dles” was not applied. Instead, unintended m… view at source ↗
Figure 2
Figure 2. Overview of Vid-TTA. (a) We first detect moving objects with YOLO and apply optical flow estimation inside the bounding [PITH_FULL_IMAGE:figures/full_fig_p004_2.png] view at source ↗
Figure 3
Figure 3. In the top row (box), YOLOv8 detects and draws bounding boxes around the moving subject in each video frame; the second [PITH_FULL_IMAGE:figures/full_fig_p005_3.png] view at source ↗
Figures from the paper (3 more)
Figure 4
Figure 4. Figure 4: Qualitative comparison of six video editing models before and after applying Vid-TTA. From top to bottom in each sub-figure: [PITH_FULL_IMAGE:figures/full_fig_p006_4.png]
Figure 5
Figure 5. Figure 5: Comparison of Latent MAE and Pixel MAE effects on video quality: In scenes like ’Iron Man surfing in the desert’ and ’Mickey [PITH_FULL_IMAGE:figures/full_fig_p007_5.png]
Figure 6
Figure 6. Figure 6: Comparison of Latent MAE with and without flow mask [PITH_FULL_IMAGE:figures/full_fig_p007_6.png]

Discussion (0). Sign in to comment.

Forward citations

Cited by 2 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score.

  1. ElasticTTT: Prior-Preserving Test-Time Tuning for Video Editing

    cs.CV 2026-07 conditional novelty 6.0 of 10

    Test-time tuning of video diffusion models collapses generation toward the source video; ElasticTTT counters this with noisy targets, contrastive source-prompt guidance, and asynchronous region-wise noise scheduling, ...

  2. ElasticTTT: Prior-Preserving Test-Time Tuning for Video Editing

    cs.CV 2026-07 conditional novelty 6.0 of 10

    A test-time tuning framework with three regularization techniques that preserves the generative prior of a video diffusion model during one-shot editing, achieving state-of-the-art results on the authors' benchmark.

Reference graph

Works this paper leans on

58 extracted references · 30 canonical work pages · cited by 1 Pith paper

  1. [1]

    Semantic self- adaptation: Enhancing generalization with a single sample,

    Sherwin Bahmani, Oliver Hahn, Eduard Zamfir, Nikita Araslanov, Daniel Cremers, and Stefan Roth. Semantic self- adaptation: Enhancing generalization with a single sample,

    work page

  2. [2]

    Pix2video: Video editing using image diffusion

    Duygu Ceylan, Chun-Hao P Huang, and Niloy J Mitra. Pix2video: Video editing using image diffusion. InICCV, pages 23206–23217, 2023. 1, 2

    work page 2023

  3. [3]

    FLATTEN: optical FLow-guided ATTENtion for consistent text-to-video edit- ing

    Yuren Cong, Mengmeng Xu, christian simon, Shoufa Chen, Jiawei Ren, Yanping Xie, Juan-Manuel Perez-Rua, Bodo Rosenhahn, Tao Xiang, and Sen He. FLATTEN: optical FLow-guided ATTENtion for consistent text-to-video edit- ing. InThe Twelfth International Conference on Learning Representations, 2024. 1, 2

    work page 2024

  4. [4]

    NextLevelBERT: Masked language mod- eling with higher-level representations for long documents

    Tamara Czinczoll, Christoph H ¨ones, Maximilian Schall, and Gerard De Melo. NextLevelBERT: Masked language mod- eling with higher-level representations for long documents. InProceedings of the 62nd Annual Meeting of the Associa- tion for Computational Linguistics (Volume 1: Long Papers), pages 4656–4666, Bangkok, Thailand, 2024. Association for Computatio...

    work page 2024

  5. [5]

    Bert: Pre-training of deep bidirectional trans- formers for language understanding

    Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. Bert: Pre-training of deep bidirectional trans- formers for language understanding. InNorth American Chapter of the Association for Computational Linguistics,

    work page

  6. [6]

    Test-time training with masked autoencoders.Advances in Neural Information Processing Systems, 35:29374–29385,

    Yossi Gandelsman, Yu Sun, Xinlei Chen, and Alexei Efros. Test-time training with masked autoencoders.Advances in Neural Information Processing Systems, 35:29374–29385,

  7. [7]

    Tokenflow: Consistent diffusion features for consistent video editing.arXiv preprint arXiv:2307.10373, 2023

    Michal Geyer, Omer Bar-Tal, Shai Bagon, and Tali Dekel. Tokenflow: Consistent diffusion features for consistent video editing.arXiv preprint arXiv:2307.10373, 2023. 1, 2

    arXiv 2023

  8. [8]

    Masked autoencoders are scalable vision learners

    Kaiming He, Xinlei Chen, Saining Xie, Yanghao Li, Piotr Doll´ar, and Ross Girshick. Masked autoencoders are scalable vision learners. InProceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 16000– 16009, 2022. 2

    2022

Show all 58 references
  1. [9]

    Ddpm-moco: Advancing industrial surface defect generation and detec- 7 tion with generative and contrastive learning

    Yangfan He, Xinyan Wang, and Tianyu Shi. Ddpm-moco: Advancing industrial surface defect generation and detec- 7 tion with generative and contrastive learning. InInterna- tional Joint Conference on Artificial Intelligence, pages 34–

  2. [10]

    Enhancing low-cost video editing with lightweight adaptors and temporal-aware inversion.arXiv preprint arXiv:2501.04606, 2025

    Yangfan He, Sida Li, Jianhui Wang, Kun Li, Xinyuan Song, Xinhang Yuan, Keqin Li, Kuan Lu, Menghao Huo, Jingqun Tang, et al. Enhancing low-cost video editing with lightweight adaptors and temporal-aware inversion.arXiv preprint arXiv:2501.04606, 2025. 2

    2025 arXiv

  3. [11]

    Diffusionbert: Improving generative masked language models with diffusion models

    Zhengfu He, Tianxiang Sun, Kuan Wang, Xuanjing Huang, and Xipeng Qiu. Diffusionbert: Improving generative masked language models with diffusion models. InAnnual Meeting of the Association for Computational Linguistics,

  4. [12]

    Imagen video: High definition video generation with diffusion mod- els.arXiv preprint arXiv:2210.02303, 2022

    Jonathan Ho, William Chan, Chitwan Saharia, Jay Whang, Ruiqi Gao, Alexey Gritsenko, Diederik P Kingma, Ben Poole, Mohammad Norouzi, David J Fleet, et al. Imagen video: High definition video generation with diffusion mod- els.arXiv preprint arXiv:2210.02303, 2022. 1

    arXiv

  5. [13]

    Cogvideo: Large-scale pretraining for text-to-video generation via transformers.arXiv preprint arXiv:2205.15868, 2022

    Wenyi Hong, Ming Ding, Wendi Zheng, Xinghan Liu, and Jie Tang. Cogvideo: Large-scale pretraining for text-to-video generation via transformers.arXiv preprint arXiv:2205.15868, 2022. 1

    2022 arXiv

  6. [14]

    Ground-a-video: Zero- shot grounded video editing using text-to-image diffusion models

    Hyeonho Jeong and Jong Chul Ye. Ground-a-video: Zero- shot grounded video editing using text-to-image diffusion models. InThe Twelfth International Conference on Learn- ing Representations, 2024. 2

    2024

  7. [15]

    Text2video-zero: Text-to- image diffusion models are zero-shot video generators.arXiv preprint arXiv:2303.13439, 2023

    Levon Khachatryan, Andranik Movsisyan, Vahram Tade- vosyan, Roberto Henschel, Zhangyang Wang, Shant Navasardyan, and Humphrey Shi. Text2video-zero: Text-to- image diffusion models are zero-shot video generators.arXiv preprint arXiv:2303.13439, 2023. 1

    2023 arXiv

  8. [16]

    Masked vision and language modeling for multi-modal representation learn- ing

    Gukyeong Kwon, Zhaowei Cai, Avinash Ravichandran, Er- han Bas, Rahul Bhotika, and Stefano Soatto. Masked vision and language modeling for multi-modal representation learn- ing. InThe Eleventh International Conference on Learning Representations, 2023. 3

    2023

  9. [17]

    Vidtome: Video token merging for zero-shot video editing

    Xirui Li, Chao Ma, Xiaokang Yang, and Ming-Hsuan Yang. Vidtome: Video token merging for zero-shot video editing. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2024. 1, 2

    2024

  10. [18]

    Video test-time adaptation for action recognition

    Wei Lin, Muhammad Jehanzeb Mirza, Mateusz Kozinski, Horst Possegger, Hilde Kuehne, and Horst Bischof. Video test-time adaptation for action recognition. InCVPR, 2023. 3

    2023

  11. [19]

    Sora: A review on background, technology, limitations, and opportunities of large vision models.arXiv preprint arXiv:2402.17177, 2024

    Yixin Liu, Kai Zhang, Yuan Li, Zhiling Yan, Chujie Gao, Ruoxi Chen, Zhengqing Yuan, Yue Huang, Hanchi Sun, Jian- feng Gao, et al. Sora: A review on background, technology, limitations, and opportunities of large vision models.arXiv preprint arXiv:2402.17177, 2024. 1

    2024 arXiv

  12. [20]

    Fatezero: Fus- ing attentions for zero-shot text-based video editing

    Chenyang Qi, Xiaodong Cun, Yong Zhang, Chenyang Lei, Xintao Wang, Ying Shan, and Qifeng Chen. Fatezero: Fus- ing attentions for zero-shot text-based video editing. In 2023 IEEE/CVF International Conference on Computer Vi- sion (ICCV), pages 15886–15896, 2023. 2

    2023

  13. [21]

    Fatezero: Fusing attentions for zero-shot text-based video editing

    Chenyang Qi, Xiaodong Cun, Yong Zhang, Chenyang Lei, Xintao Wang, Ying Shan, and Qifeng Chen. Fatezero: Fusing attentions for zero-shot text-based video editing. arXiv:2303.09535, 2023. 1

    2023 arXiv

  14. [22]

    Learning transferable visual models from natural language supervi- sion

    Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, et al. Learning transferable visual models from natural language supervi- sion. InInternational conference on machine learning, p...

    2021

  15. [23]

    You only look once: Unified, real-time object de- tection, 2016

    Joseph Redmon, Santosh Divvala, Ross Girshick, and Ali Farhadi. You only look once: Unified, real-time object de- tection, 2016. 3

    2016

  16. [24]

    U-net: Convolutional networks for biomedical image segmentation

    Olaf Ronneberger, Philipp Fischer, and Thomas Brox. U-net: Convolutional networks for biomedical image segmentation. InMedical Image Computing and Computer-Assisted Inter- vention, pages 234–241. Springer, 2015. 2

    2015

  17. [25]

    Cloudfixer: Test-time adaptation for 3d point clouds via diffusion-guided geometric transformation

    Hajin Shim, Changhun Kim, and Eunho Yang. Cloudfixer: Test-time adaptation for 3d point clouds via diffusion-guided geometric transformation. InEuropean Conference on Com- puter Vision, pages 454–471. Springer, 2024. 3

    2024

  18. [26]

    Mm-tta: Multi-modal test-time adaptation for 3d se- mantic segmentation

    Inkyu Shin, Masayoshi Tomizuka, Soonmin Lim, and Wei Zhan. Mm-tta: Multi-modal test-time adaptation for 3d se- mantic segmentation. InInternational Conference on 3D Vi- sion, pages 1229–1238, 2022. 3

    2022

  19. [27]

    Make-a-video: Text-to-video generation without text-video data.arXiv preprint arXiv:2209.14792,

    Uriel Singer, Adam Polyak, Thomas Hayes, Xi Yin, Jie An, Songyang Zhang, Qiyuan Hu, Harry Yang, Oron Ashual, Oran Gafni, et al. Make-a-video: Text-to-video generation without text-video data.arXiv preprint arXiv:2209.14792,

    arXiv

  20. [28]

    Efros, and Moritz Hardt

    Yu Sun, Xiaolong Wang, Zhuang Liu, John Miller, Alexei A. Efros, and Moritz Hardt. Test-time training with self- supervision for generalization under distribution shifts. In International Conference on Machine Learning, 2020. 2

    2020

  21. [29]

    VideoMAE: Masked autoencoders are data-efficient learners for self-supervised video pre-training

    Zhan Tong, Yibing Song, Jue Wang, and Limin Wang. VideoMAE: Masked autoencoders are data-efficient learners for self-supervised video pre-training. InAdvances in Neural Information Processing Systems, 2022. 2, 3

    2022

  22. [30]

    Evaluation and analysis of hal- lucination in large vision-language models.arXiv preprint arXiv:2308.15126, 2023

    Junyang Wang, Yiyang Zhou, Guohai Xu, Pengcheng Shi, Chenlin Zhao, Haiyang Xu, Qinghao Ye, Ming Yan, Ji Zhang, Jihua Zhu, et al. Evaluation and analysis of hal- lucination in large vision-language models.arXiv preprint arXiv:2308.15126, 2023. 2

    2023 arXiv

  23. [31]

    Enhancing code llms with reinforce- ment learning in code generation: A survey.arXiv preprint arXiv:2412.20367, 2024

    Junqiao Wang, Zeng Zhang, Yangfan He, Zihao Zhang, Yuyang Song, Tianyu Shi, Yuchen Li, Hengyuan Xu, Kunyu Wu, Xin Yi, et al. Enhancing code llms with reinforce- ment learning in code generation: A survey.arXiv preprint arXiv:2412.20367, 2024. 2

    2024 arXiv

  24. [32]

    Videomae v2: Scaling video masked autoencoders with dual masking

    Limin Wang, Bingkun Huang, Zhiyu Zhao, Zhan Tong, Yi- nan He, Yi Wang, Yali Wang, and Yu Qiao. Videomae v2: Scaling video masked autoencoders with dual masking. In Proceedings of the IEEE/CVF conference on computer vi- sion and pattern recognition, pages 14549–14560, 2023. 2

    2023

  25. [33]

    Videomae v2: Scaling video masked autoencoders with dual masking

    Limin Wang, Bingkun Huang, Zhiyu Zhao, Zhan Tong, Yi- nan He, Yi Wang, Yali Wang, and Yu Qiao. Videomae v2: Scaling video masked autoencoders with dual masking. In Proceedings of the IEEE/CVF Conference on Computer Vi- sion and Pattern Recognition (CVPR), pages 14549–14560,

  26. [34]

    Unitmge: Uniform text-motion generation and 8 editing model via diffusion

    Ruoyu Wang, Yangfan He, Tengjiao Sun, Xiang Li, and Tianyu Shi. Unitmge: Uniform text-motion generation and 8 editing model via diffusion. In2025 IEEE/CVF Winter Con- ference on Applications of Computer Vision (WACV), pages 6104–6114. IEEE, 2025. 2

    2025

  27. [36]

    Zero-shot video editing using off-the-shelf image diffusion models.arXiv preprint arXiv:2303.17599, 2023

    Wen Wang, kangyang Xie, Zide Liu, Hao Chen, Yue Cao, Xinlong Wang, and Chunhua Shen. Zero-shot video editing using off-the-shelf image diffusion models.arXiv preprint arXiv:2303.17599, 2023. 1

    2023 arXiv

  28. [37]

    Backpropagation- free test-time adaptation for 3d point cloud segmentation

    Yun Wang, Chang Xiao, Jaesung Ahn, Zhipeng Cao, Chun Shen, Junghyun Lee, and Antoni B Chan. Backpropagation- free test-time adaptation for 3d point cloud segmentation. In Proceedings of the IEEE/CVF Conference on Computer Vi- sion and Pattern Recognition, pages 1229–1238, 2024. 3

    2024

  29. [38]

    Tune-a-video: One-shot tuning of image diffusion models for text-to-video generation

    Jay Zhangjie Wu, Yixiao Ge, Xintao Wang, Stan Weixian Lei, Yuchao Gu, Yufei Shi, Wynne Hsu, Ying Shan, Xiaohu Qie, and Mike Zheng Shou. Tune-a-video: One-shot tuning of image diffusion models for text-to-video generation. In Proceedings of the IEEE/CVF International Conference...

    2023

  30. [39]

    Zehao Xiao and Cees G. M. Snoek. Beyond model adapta- tion at test time: A survey, 2024. 2

    2024

  31. [40]

    Beyond distribution shift: A comprehensive survey of test-time adaptation in computer vision.arXiv preprint arXiv:2402.07347, 2024

    Zhuoran Xiao, Jian Wang, Han Zhou, Hongxin Liu, Yingjun Hao, Ke Lu, Jie Ou, and Jun Yang. Beyond distribution shift: A comprehensive survey of test-time adaptation in computer vision.arXiv preprint arXiv:2402.07347, 2024. 2

    2024 arXiv

  32. [41]

    Lumina-mgpt 2.0: Stand- alone autoregressive image modeling.arXiv preprint arXiv:2507.17801, 2025

    Yi Xin, Juncheng Yan, Qi Qin, Zhen Li, Dongyang Liu, Shicheng Li, Victor Shea-Jay Huang, Yupeng Zhou, Ren- rui Zhang, Le Zhuo, et al. Lumina-mgpt 2.0: Stand- alone autoregressive image modeling.arXiv preprint arXiv:2507.17801, 2025. 2

    2025 arXiv

  33. [42]

    Resurrect mask autoregressive modeling for efficient and scalable image generation.arXiv preprint arXiv:2507.13032, 2025

    Yi Xin, Le Zhuo, Qi Qin, Siqi Luo, Yuewen Cao, Bin Fu, Yangfan He, Hongsheng Li, Guangtao Zhai, Xiaohong Liu, et al. Resurrect mask autoregressive modeling for efficient and scalable image generation.arXiv preprint arXiv:2507.13032, 2025. 2

    2025 arXiv

  34. [43]

    Modality-collaborative test- time adaptation for action recognition

    Baochen Xiong, Xiaoshan Yang, Yaguang Song, Yaowei Wang, and Chang sheng Xu. Modality-collaborative test- time adaptation for action recognition. In2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 26722–26731, 2024. 3

    2024

  35. [44]

    Gmflow: Learning optical flow via global matching

    Haofei Xu, Jing Zhang, Jianfei Cai, Hamid Rezatofighi, and Dacheng Tao. Gmflow: Learning optical flow via global matching. InProceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 8121–8130,

  36. [45]

    Videogpt: Video generation using vq-vae and trans- formers.arXiv preprint arXiv:2104.10157, 2021

    Wilson Yan, Yunzhi Zhang, Pieter Abbeel, and Aravind Srinivas. Videogpt: Video generation using vq-vae and trans- formers.arXiv preprint arXiv:2104.10157, 2021. 1

    2021 arXiv

  37. [46]

    Wcdt: World-centric diffusion transformer for traffic scene generation.arXiv preprint arXiv:2404.02082, 2024

    Chen Yang, Yangfan He, Aaron Xuxiang Tian, Dong Chen, Jianhui Wang, Tianyu Shi, Arsalan Heydarian, and Pei Liu. Wcdt: World-centric diffusion transformer for traffic scene generation.arXiv preprint arXiv:2404.02082, 2024. 2

    2024 arXiv

  38. [47]

    Rerender a video: Zero-shot text-guided video-to-video translation

    Shuai Yang, Yifan Zhou, Ziwei Liu, and Chen Change Loy. Rerender a video: Zero-shot text-guided video-to-video translation. 2023. 1, 2

    2023

  39. [48]

    Videograin: Modulating space-time attention for multi- grained video editing.arXiv preprint arXiv:2502.17258,

    Xiangpeng Yang, Linchao Zhu, Hehe Fan, and Yi Yang. Videograin: Modulating space-time attention for multi- grained video editing.arXiv preprint arXiv:2502.17258,

    arXiv

  40. [49]

    Cogvideox: Text-to-video diffusion models with an expert transformer.arXiv preprint arXiv:2408.06072, 2024

    Zhuoyi Yang, Jiayan Teng, Wendi Zheng, Ming Ding, Shiyu Huang, Jiazheng Xu, Yuanming Yang, Wenyi Hong, Xiao- han Zhang, Guanyu Feng, et al. Cogvideox: Text-to-video diffusion models with an expert transformer.arXiv preprint arXiv:2408.06072, 2024. 1

    2024 arXiv

  41. [50]

    Space-time diffusion features for zero-shot text-driven motion transfer.arXiv preprint arxiv:2311.17009, 2023

    Danah Yatim, Rafail Fridman, Omer Bar-Tal, Yoni Kas- ten, and Tali Dekel. Space-time diffusion features for zero-shot text-driven motion transfer.arXiv preprint arxiv:2311.17009, 2023. 2

    2023 arXiv

  42. [51]

    mplug-owl: Modularization empowers large language models with multimodality.arXiv preprint arXiv:2304.14178, 2023

    Qinghao Ye, Haiyang Xu, Guohai Xu, Jiabo Ye, Ming Yan, Yiyang Zhou, Junyang Wang, Anwen Hu, Pengcheng Shi, Yaya Shi, et al. mplug-owl: Modularization empowers large language models with multimodality.arXiv preprint arXiv:2304.14178, 2023. 2

    2023 arXiv

  43. [52]

    Rapid network adaptation: Learning to adapt neural networks using test-time feedback, 2023

    Teresa Yeo, O ˘guzhan Fatih Kar, Zahra Sodagar, and Amir Zamir. Rapid network adaptation: Learning to adapt neural networks using test-time feedback, 2023. 2

    2023

  44. [53]

    Temporal coherent test time opti- mization for robust video classification

    Chenyu Yi, SIYUAN Y ANG, Yufei Wang, Haoliang Li, Yap peng Tan, and Alex Kot. Temporal coherent test time opti- mization for robust video classification. InThe Eleventh In- ternational Conference on Learning Representations, 2023. 3

    2023

  45. [54]

    Controlvideo: Training-free controllable text-to-video generation

    Yabo Zhang, Yuxiang Wei, Dongsheng Jiang, XIAOPENG ZHANG, Wangmeng Zuo, and Qi Tian. Controlvideo: Training-free controllable text-to-video generation. InThe Twelfth International Conference on Learning Representa- tions, 2024. 2

    2024

  46. [55]

    Analyzing and mitigating object hallucination in large vision-language models.arXiv preprint arXiv:2310.00754,

    Yiyang Zhou, Chenhang Cui, Jaehong Yoon, Linjun Zhang, Zhun Deng, Chelsea Finn, Mohit Bansal, and Huaxiu Yao. Analyzing and mitigating object hallucination in large vision-language models.arXiv preprint arXiv:2310.00754,

    arXiv

  47. [56]

    Calibrated self-rewarding vision language models.Advances in Neural Information Process- ing Systems, 37:51503–51531, 2024

    Yiyang Zhou, Zhiyuan Fan, Dongjie Cheng, Sihan Yang, Zhaorun Chen, Chenhang Cui, Xiyao Wang, Yun Li, Linjun Zhang, and Huaxiu Yao. Calibrated self-rewarding vision language models.Advances in Neural Information Process- ing Systems, 37:51503–51531, 2024. 2

    2024

  48. [57]

    Reagent-v: A reward-driven multi-agent framework for video understanding.arXiv preprint arXiv:2506.01300,

    Yiyang Zhou, Yangfan He, Yaofeng Su, Siwei Han, Joel Jang, Gedas Bertasius, Mohit Bansal, and Huaxiu Yao. Reagent-v: A reward-driven multi-agent framework for video understanding.arXiv preprint arXiv:2506.01300,

    arXiv

  49. [58]

    Glimpse: Do large vision-language models truly think with videos or just glimpse at them?arXiv preprint arXiv:2507.09491, 2025

    Yiyang Zhou, Linjie Li, Shi Qiu, Zhengyuan Yang, Yuyang Zhao, Siwei Han, Yangfan He, Kangqi Li, Haonian Ji, Zihao Zhao, et al. Glimpse: Do large vision-language models truly think with videos or just glimpse at them?arXiv preprint arXiv:2507.09491, 2025. 2

    2025 arXiv

  50. [59]

    Anyprefer: An agentic framework for preference data synthesis.arXiv preprint arXiv:2504.19276, 2025

    Yiyang Zhou, Zhaoyang Wang, Tianle Wang, Shangyu Xing, Peng Xia, Bo Li, Kaiyuan Zheng, Zijian Zhang, Zhaorun Chen, Wenhao Zheng, et al. Anyprefer: An agentic framework for preference data synthesis.arXiv preprint arXiv:2504.19276, 2025. 2 9

    2025 arXiv

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.