Pith. sign in

REVIEW 5 major objections 5 minor 83 references

PRIMEdit: Probability Redistribution for Instance-aware Multi-object Video Editing with Benchmark Dataset

T0 review · 5 major / 5 minor · reviewed 2026-08-11 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read PRIMEdit claims a training-free way to edit several objects in one video at once, keeping each edit inside its target mask.

desk verdict PRIMEdit is a solid zero-shot multi-instance video editing method with a useful benchmark, but the evaluation is inbred enough that 'significantly outperforms' is not yet established. read the letter →

arxiv 2412.12877 v2 pith:25TTT54D submitted 2024-12-17 cs.CV

Samuel Teodoro , Agus Gunawan , Soo Ye Kim , Jihyong Oh , Munchurl Kim This is my paper · ORCID
classification cs.CV
keywords multi-instancevideoeditingzero-shotdiffusionmodelscross-attentionmodulationattentionleakageinstancemasksbenchmarkCLIP-basedevaluation
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper tries to establish that a zero-shot, training-free framework can edit multiple distinct objects in a single video simultaneously, with each edit staying inside its own mask and not bleeding into other objects. It argues that existing global or single-object editors fail exactly when several instances need different prompts at once, either because a global caption cannot specify who gets edited or because bounding-box conditions leak across overlapping regions. PRIMEdit's answer is to reshape cross-attention probabilities inside each mask and to sample each instance separately before fusing, which the paper reports as more faithful and leakage-free than recent methods on a new 200-video benchmark. A reader should care because this is the capability behind practical text-driven video fix-ups: change one person's outfit, another's prop, and the background's color in a single pass without retraining.

What carries the argument

The load-bearing object is the cross-attention map between image features and text tokens in a diffusion U-Net. IPR treats each row of that map as probability mass and redistributes it among the start-of-sequence, text, and end tokens, with the mask deciding which side of the boundary receives the redistribution; a dynamic strength decays over sampling steps and avoids padding-token manipulation. DMS is the second mechanism: Series Noise Sampling edits instances sequentially against inverted background latents, latent fusion combines the per-instance results, re-inversion harmonizes them, and Parallel Noise Sampling keeps the captions separate in the remaining steps. The Cross-Instance Accuracy score is the evaluation machinery: a per-row winner-take-all over vision-language similarities between cropped instances and instance captions, averaged down the diagonal.

What would settle it

Run PRIMEdit with the paper's fixed hyperparameters on a held-out set of videos that were not used for any tuning, compute the Cross-Instance Accuracy and per-instance textual faithfulness, and compare against the strongest mask-based multi-instance baseline; if PRIMEdit does not lead on leakage prevention or faithfulness on that held-out set, the central claim fails.

Watch

Extended reading notes

Core claim

On its own terms, the paper claims that multi-instance video editing can be decomposed into per-instance probability redistribution plus disentangled sampling. Instance-centric Probability Redistribution (IPR) works only in cross-attention layers: outside the mask, attention to text and end tokens is moved to the start-of-sequence token so no edit appears; inside the mask, probability is moved from the start token to text and end tokens so the described object actually appears there. Disentangled Multi-instance Sampling (DMS) then edits each instance independently with its own caption while the background is pinned to the inverted input, fuses the instance latents, re-inverts briefly, and finishes in a parallel mode that keeps captions separate. With those two mechanisms, the paper claims edits are faithful, temporally consistent, and free of attention leakage; it also contributes the MIVE dataset of 200 annotated videos and the Cross-Instance Accuracy (CIA) score, which asks whether each cropped instance matches its own caption better than any other caption.

Load-bearing premise

The method's reported edge over other editors presumes that the MIVE benchmark is a fair, representative test and that tuning the framework's hyperparameters on those same videos does not inflate its scores.

Editorial extensions

If this is right

  • Users can give each object its own prompt and get edits confined to that object's mask, without writing one long global caption or retraining the model.
  • Overlapping or nearby objects no longer need bounding-box separation, since mask-based probability redistribution replaces box conditions.
  • The MIVE dataset gives the field a common testbed with many instances per object class, small-to-large masks, and instance-level captions, plus the CIA metric for leakage.
  • Because IPR only touches cross-attention and DMS avoids spatio-temporal attention modulation, the method runs faster and uses less memory than the mask-based baseline it is compared against.
  • The same framework handles a single instance and partial-instance edits, not only dense multi-object scenes.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • If the benchmark's hyperparameters were tuned on the same 200 videos used in the headline comparison, the size of the advantage over untuned baselines may shrink on arbitrary user videos; an out-of-benchmark test with fixed settings would settle that.
  • The winner-take-all binarization in CIA discards how decisively one caption wins; a soft variant that rewards large margins could be more sensitive to partial leakage.
  • The mask-only interface suggests a natural extension to video object tracking: given masks for one frame, propagate masks and edit consistently through the clip without user re-annotation.
  • Reflective surfaces are a known gap; combining the approach with reflection-aware masking could extend it to scenes with mirrors and polished floors.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

5 major / 5 minor

Summary. The paper proposes PRIMEdit, a zero-shot framework for multi-instance video editing that combines two modules: Instance-centric Probability Redistribution (IPR), which manipulates cross-attention probabilities to localize edits within instance masks, and Disentangled Multi-instance Sampling (DMS), which separates per-instance denoising to reduce attention leakage. The authors also introduce the MIVE dataset (200 videos with per-instance masks and captions) and the Cross-Instance Accuracy (CIA) score. Quantitative comparisons against seven baselines, extensive ablations, and a user study are reported, with the claim that PRIMEdit significantly outperforms recent state-of-the-art methods in faithfulness, accuracy, and leakage prevention.

Significance. The task of multi-instance video editing is timely and the proposed IPR and DMS designs are conceptually simple and potentially useful. If the empirical claims are robust, PRIMEdit would be a strong zero-shot method and the MIVE dataset and CIA score could provide a useful evaluation resource. The ablation study is thorough, covering the IPR and DMS components and several hyperparameters. However, the evaluation as presented does not yet establish the headline claim of significant superiority: the method is tuned on the same videos used in the headline comparison, results lack uncertainty quantification, the CIA metric needs further scrutiny, and the code and dataset are not released.

major comments (5)
  1. [Sec. 5 Implementation details; Supp. D.2–D.3, Tables 9–10] The headline comparison in Table 2 is performed on the same 200-video MIVE set that was used to select all key hyperparameters: the IPR step fraction (10%), λ and λ_r (both 0.5), the SNS step fraction (40%), and the re-inversion steps l (2) are chosen by ablations on the full MIVE set (Supp. D.2–D.3). Baselines are run with their default settings, so PRIMEdit is effectively optimized on the test set while the baselines are not. This asymmetry can inflate the observed margins (e.g., CIA 0.6705 vs 0.5868) and does not support the abstract's claim of "significantly outperforms" for general videos. I request a held-out validation split or a separate test set for hyperparameter selection, and a report of the robustness of the results across different splits.
  2. [Eq. (2), Sec. 3.2] The formula for λ_S is degenerate: min(mean(A_{I,j=S}), min(A_{I,j=S})) simplifies to min(A_{I,j=S}), and the warm-up value W is introduced without a precise definition of its role in the expression. As written, the equation is not a well-defined dynamic schedule, and the IPR method cannot be reproduced from the text. Please provide a corrected formulation, a clear definition of W, and pseudo-code for the IPR update.
  3. [Sec. 4.2, Eq. (10)] The CIA score assigns a binary 1 to the maximum similarity in each row of the n×n similarity matrix. If two instance target captions are semantically similar in CLIP space, a perfectly faithful edit may not receive the largest similarity for its own crop, producing a 0 and penalizing correct behavior. Thus, the metric conflates attention leakage with caption ambiguity, and it may differentially favor methods that produce overly discriminative or artifact-laden crops. Please provide an analysis of how CIA behaves under caption similarity, or consider a soft version based on normalized ranks or thresholds.
  4. [Table 2, Table 5, Sec. 5.1] No error bars, multiple random seeds, or statistical significance tests are reported. The improvements over the strongest baselines are small in several local metrics (e.g., LTF 0.2048 vs 0.2026 for VideoGrain), and the user-study temporal-consistency win rate of PRIMEdit (33.65%) is close to FLATTEN's (32.60%). Without uncertainty quantification, the claim of "significantly outperforms" is not statistically established. Please report the mean and standard deviation over at least three random seeds per method and add paired significance tests (e.g., bootstrap or Wilcoxon) for the quantitative and user-study comparisons.
  5. [Sec. 1, Data availability] The paper introduces a new dataset (MIVE) and a new metric (CIA), but neither the dataset nor the code or evaluation scripts are released. The project page (https://kaist-viclab.github.io/primedit-site/) shows demos but not the dataset or implementation. Without releasing these resources, the central empirical claims cannot be reproduced or independently verified, and the proposed benchmark cannot serve the community as intended. Please make the dataset, code, and metric implementation publicly available.
minor comments (5)
  1. [Throughout] There are several typos and grammatical issues, including "addtition" (Sec. 3.2), "ommitted" (Supp. B.1), and "we ourperform" (Supp. C.3). The manuscript would benefit from a careful proofread.
  2. [Sec. 5.1, User study] The user study's temporal-consistency result for PRIMEdit is close to that of FLATTEN; please add a statistical significance test or temper the wording related to temporal consistency.
  3. [Supp. Table 5] In the full quantitative results, PRIMEdit ranks fourth in Global Temporal Consistency and Global Textual Faithfulness; the main text should acknowledge this more explicitly rather than focusing only on local scores.
  4. [Supp. G, Limitations] The limitations section only discusses reflection consistency; it should also mention the evaluation limitations raised here (test-set tuning, lack of error bars) and how they are addressed.
  5. [Sec. B.1, Supp.] The caption-generation pipeline in Fig. 8 suggests a level of automation that contradicts the described "Manual Fix and Annotation" step; please clarify the human effort involved in producing the MIVE captions and masks.

Circularity Check

1 steps flagged · score 4.0 of 10

Hyperparameters fitted on the same MIVE videos used for the headline comparison make the 'significantly outperforms' claim statistically forced on that benchmark; no definitional circularity.

  1. fitted input called prediction [Sec. 5 Implementation details; Supp. D.2 (IPR Ablations) and D.3 (DMS Ablations); headline results in Table 2]
    "We evaluate our PRIMEdit framework on our MIVE dataset (Sec. 4.1), editing 12-32 frames per video... We perform SNS for the first 40% of the denoising steps and perform PNS for the remaining steps. Our re-inversion steps l is set to l = 2. ... Based on these observations, we choose to apply IPR to 10% of the sampling steps. ... we use λ = 0.5 as our default. ... we select λr = 0.5 to achieve a balance between overall editing faithfulness and the minimization of artifacts. ... setting SNS steps to the first 40% of the sampling process achieves a balanced trade-off..."

    The five central hyperparameters (IPR step fraction, λ, λr, SNS fraction, and re-inversion steps) are selected using quantitative ablations over the same MIVE dataset that produces the headline comparison in Table 2, while the baselines are run 'using its default settings' (Sec. 5.1). PRIMEdit is therefore effectively fitted to the evaluation set, and the reported margins (e.g., CIA 0.6705 vs 0.5868) are not a prediction on held-out data but the result of optimizing scores on those same videos. The claim that PRIMEdit 'significantly outperforms' SOTA on MIVE is thus statistically forced by construction of the comparison, even though the method itself is not definitionally circular.

full rationale

The paper's core contribution is an inference-time algorithm, not a derivation, and there is no load-bearing self-citation: the method rests on standard diffusion components (Stable Diffusion, AnimateDiff, ControlNet, DDIM inversion) and external baselines are run and compared by the authors. No uniqueness theorem or author-invented prior result is invoked to force a choice. The CIA metric, though author-defined, is a transparent argmax-over-rows computation and is not equivalent to the method's cross-attention manipulation despite both using CLIP embeddings. The main circularity concern is empirical rather than definitional: the authors build the MIVE benchmark, define the primary leakage metric (CIA), and tune all key hyperparameters (IPR step fraction, λ, λr, SNS fraction, re-inversion steps) on the same 200 videos used for the headline quantitative table, while baselines are evaluated with their default settings. This test-set-tuning asymmetry can inflate the reported advantage and makes the headline 'significantly outperforms' claim not independently established for general videos. The user study provides a mitigating human-preference signal, but it evaluates the same tuned model on 30 videos drawn from the same dataset, so it does not break the inbreeding. Overall, the algorithmic derivation is self-contained, but the central empirical claim is partly forced by fitting to the evaluation set, warranting a score of 4.

Assumptions & free parameters 6 free parameters · 5 assumptions · 2 invented entities

The method is a heuristic engineering contribution, not a derivation. It relies on standard DDIM inversion, on the Stable Diffusion/AnimateDiff/ControlNet backbone, on accurate VIPSeg masks, and on the validity of CLIP for both guiding and measuring edits. The main ad hoc element is the IPR attention redistribution, justified by empirical observations. The MIVE dataset and CIA metric are introduced by the authors and have no external validation in the manuscript.

free parameters (6)
  • IPR redistribution strength lambda = 0.5
    Tuned empirically on MIVE as a trade-off between faithfulness and artifacts (Sec. 3.2, Supp. D.2).
  • IPR token split ratio lambda_r = 0.5
    Controls redistribution between text tokens and end-of-sequence token; chosen by ablations on MIVE (Sec. 3.2, Supp. D.2).
  • IPR application fraction = first 10% of DDIM steps
    Chosen by ablation on MIVE; more steps improve faithfulness but can hurt structure (Supp. D.2, Fig. 18).
  • SNS step fraction = first 40% of DDIM steps
    Chosen by ablation balancing background preservation, artifacts, and temporal consistency (Supp. D.3).
  • Re-inversion steps l = 2
    Chosen by ablation to balance temporal consistency and blurring artifacts (Supp. D.3).
  • Warm-up schedule W = decays from lambda=0.5 to 0 during first 10% of steps
    Part of the lambda_S schedule, meant to increase early-step editing fidelity (Sec. 3.2).
assumptions (5)
  • standard math DDIM inversion and denoising preserve the input video's structure.
    Used throughout the inversion-based editing pipeline (Sec. 3.1 and Supp. Eq. 11-12).
  • domain assumption Stable Diffusion v1.5 with AnimateDiff motion modules and ControlNet depth provides a suitable generative prior for video editing.
    Chosen as the backbone; no comparison against other backbone choices is provided (Sec. 5 Implementation details).
  • domain assumption Instance masks from VIPSeg are accurate and temporally consistent.
    Masks are the core conditioning signal; the dataset construction explicitly selects only videos where all instances remain visible (Sec. 4.1, Supp. B.1).
  • ad hoc to paper Manipulating cross-attention probabilities inside and outside masks improves localization without introducing artifacts.
    This is the core heuristic of IPR, justified by empirical observations rather than by derivation (Sec. 3.2, Supp. D.1).
  • domain assumption CLIP embeddings are a valid proxy for caption-instance alignment.
    Used both to guide IPR and to compute LTF and CIA metrics; the paper itself notes CLIP's limited compositional reasoning (Sec. 4.2, Supp. C.2).
invented entities (2)
  • MIVE Dataset
    purpose: Benchmark of 200 videos with instance masks and per-instance captions for multi-instance video editing.
    Introduced in this paper; no download link or external validation is provided in the manuscript.
  • Cross-Instance Accuracy (CIA) Score
    purpose: Metric to quantify editing leakage across instances in multi-instance video editing.
    Introduced and used only by the authors; no external analysis yet. It relies on CLIP embeddings, which also guide the method.

how reviews work

0 comments
Cite this review

Pith. "Pith review of PRIMEdit: Probability Redistribution for Instance-aware Multi-object Video Editing with Benchmark Dataset." pith.science (2026). https://pith.science/paper/25TTT54D

@misc{pith2026241212877,
  author       = {Pith},
  title        = {Pith review of: PRIMEdit: Probability Redistribution for Instance-aware Multi-object Video Editing with Benchmark Dataset},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/25TTT54D}},
  note         = {Machine review of arXiv:2412.12877}
}
abstract

Recent AI-based video editing has enabled users to edit videos through simple text prompts, significantly simplifying the editing process. However, recent zero-shot video editing techniques primarily focus on global or single-object edits, which can lead to unintended changes in other parts of the video. When multiple objects require localized edits, existing methods face challenges, such as unfaithful editing, editing leakage, and lack of suitable evaluation datasets and metrics. To overcome these limitations, we propose $\textbf{P}$robability $\textbf{R}$edistribution for $\textbf{I}$nstance-aware $\textbf{M}$ulti-object Video $\textbf{Edit}$ing ($\textbf{PRIMEdit}$). PRIMEdit is a zero-shot framework that introduces two key modules: (i) Instance-centric Probability Redistribution (IPR) to ensure precise localization and faithful editing and (ii) Disentangled Multi-instance Sampling (DMS) to prevent editing leakage. Additionally, we present our new MIVE Dataset for video editing featuring diverse video scenarios, and introduce the Cross-Instance Accuracy (CIA) Score to evaluate editing leakage in multi-instance video editing tasks. Our extensive qualitative, quantitative, and user study evaluations demonstrate that PRIMEdit significantly outperforms recent state-of-the-art methods in terms of editing faithfulness, accuracy, and leakage prevention, setting a new benchmark for multi-instance video editing.

Figures

Figures reproduced from arXiv: 2412.12877 by the authors.

Figure 1
Figure 1. Given a video, instance masks, and target instance captions, our PRIMEdit framework enables faithful and disentangled edits [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. Limitations of previous SOTA methods. (a) Con [PITH_FULL_IMAGE:figures/full_fig_p002_2.png] view at source ↗
Figure 3
Figure 3. The overall framework of our PRIMEdit, given [PITH_FULL_IMAGE:figures/full_fig_p004_3.png] view at source ↗
Figures from the paper (21 more)
Figure 4
Figure 4. Figure 4: A comparative illustration of our IPR versus others (top) [PITH_FULL_IMAGE:figures/full_fig_p004_4.png]
Figure 5
Figure 5. Figure 5: Qualitative comparison for three videos (with increasing difficulty from left to right) in our MIVE dataset. (a) shows the color [PITH_FULL_IMAGE:figures/full_fig_p007_5.png]
Figure 7
Figure 7. Figure 7: Ablation study on DMS (Sec. 3.3). model tends to reconstruct the input frames when modula￾tion is omitted. The reconstruction thereby preserves the high LTC of the input but suffers from unfaithful editing (low LTF and IA) and leakage (low CIA). Introducing cross￾atten…
Figure 6
Figure 6. Figure 6: Ablation study on IPR (Sec. 3.2). achieves the best scores in key multi-instance VE metrics (LTC, LTF, IA, and CIA) demonstrating its superior abil￾ity to maintain temporal consistency, fine-grained textual alignment, and instance-aware modifications. Note also that ou…
Figure 8
Figure 8. Figure 8: MIVE Dataset caption generation pipeline for each video. Yellow box: The process starts by prompting LLaVA [ [PITH_FULL_IMAGE:figures/full_fig_p013_8.png]
Figure 9
Figure 9. Figure 9: Sample frames and captions from our MIVE Dataset (Part 1). The colored texts are the instance target captions. For each video, [PITH_FULL_IMAGE:figures/full_fig_p014_9.png]
Figure 10
Figure 10. Figure 10: Sample frames and captions from our MIVE Dataset (Part 2). The colored texts are the instance target captions. For each video, [PITH_FULL_IMAGE:figures/full_fig_p020_10.png]
Figure 11
Figure 11. Figure 11: Visualization of the Cross-Instance Accuracy (CIA) Score computation. We calculate the Local Textual Faithfulness (LTF) [PITH_FULL_IMAGE:figures/full_fig_p021_11.png]
Figure 12
Figure 12. Figure 12: Qualitative comparison for three videos (with increasing difficulty from left to right) in our MIVE dataset. (a) shows the [PITH_FULL_IMAGE:figures/full_fig_p022_12.png]
Figure 13
Figure 13. Figure 13: Our user study interface and questionnaire form. Par [PITH_FULL_IMAGE:figures/full_fig_p023_13.png]
Figure 15
Figure 15. Figure 15: Video-P2P [36] results on recursive multi-instance edit￾ing. The artifacts that accumulate when Video-P2P is used re￾peatedly for multi-instance editing is shown in red arrow. Our PRIMEdit prevents this error accumulation since we do not edit the frames recursively. 2…
Figure 14
Figure 14. Figure 14: Attention weight visualization. Input Video-P2P [36] PRIMEdit (Ours) In an office, a astronaut interacts with an electronic equipment on a table, observed by a man and a woman, against a white wall with a pillar. One screen is closer to the demo, and another screen is…
Figure 16
Figure 16. Figure 16: IPR analysis: Effect of altering the attention probability values of padding token [PITH_FULL_IMAGE:figures/full_fig_p024_16.png]
Figure 17
Figure 17. Figure 17: IPR analysis: Various scenarios of redistributing the attention probability values of tokens [PITH_FULL_IMAGE:figures/full_fig_p025_17.png]
Figure 18
Figure 18. Figure 18: Ablation study on IPR: (a) percentage of sampling steps where we apply our IPR. Increasing the IPR step percentage generally [PITH_FULL_IMAGE:figures/full_fig_p026_18.png]
Figure 19
Figure 19. Figure 19: Ablation study on IPR: (b) λ. Increasing λ steers the editing more towards the edit caption, but increasing it too much to λ = 0.7 may cause oversaturation, e.g., the alien’s face and clothes. The best trade-off is λ = 0.5. 27 [PITH_FULL_IMAGE:figures/full_fig_p027_19.png]
Figure 20
Figure 20. Figure 20: Ablation study on IPR: (c) [PITH_FULL_IMAGE:figures/full_fig_p028_20.png]
Figure 21
Figure 21. Figure 21: Ablation study on DMS: (a) Ablation on the number of SNS steps. Increasing the number of SNS steps reduces artifacts in [PITH_FULL_IMAGE:figures/full_fig_p029_21.png]
Figure 22
Figure 22. Figure 22: Ablation study on DMS: (b) Ablation on the number of re-inversion steps [PITH_FULL_IMAGE:figures/full_fig_p030_22.png]
Figure 23
Figure 23. Figure 23: Qualitative results on four DAVIS Dataset videos. Our PRIMEdit maintains faithful and temporally consistent edits, particularly [PITH_FULL_IMAGE:figures/full_fig_p031_23.png]
Figure 24
Figure 24. Figure 24: Limitation. Our method struggles in scenes with reflec [PITH_FULL_IMAGE:figures/full_fig_p032_24.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

83 extracted references · 63 canonical work pages

  1. [1]

    Spatext: Spatio-textual representation for con- trollable image generation

    Omri Avrahami, Thomas Hayes, Oran Gafni, Sonal Gupta, Yaniv Taigman, Devi Parikh, Dani Lischinski, Ohad Fried, and Xi Yin. Spatext: Spatio-textual representation for con- trollable image generation. In CVPR. IEEE, 2023. 3, 6, 13

    2023

  2. [2]

    Text2live: Text-driven layered image and video editing

    Omer Bar-Tal, Dolev Ofri-Amar, Rafail Fridman, Yoni Kas- ten, and Tali Dekel. Text2live: Text-driven layered image and video editing. In ECCV, pages 707–723. Springer, 2022. 2

    2022

  3. [3]

    MultiDiffusion: Fusing diffusion paths for controlled image generation

    Omer Bar-Tal, Lior Yariv, Yaron Lipman, and Tali Dekel. MultiDiffusion: Fusing diffusion paths for controlled image generation. In ICML, pages 1737–1752, 2023. 3

    2023

  4. [4]

    Align your latents: High-resolution video synthesis with la- tent diffusion models

    Andreas Blattmann, Robin Rombach, Huan Ling, Tim Dock- horn, Seung Wook Kim, Sanja Fidler, and Karsten Kreis. Align your latents: High-resolution video synthesis with la- tent diffusion models. In CVPR, pages 22563–22575, 2023. 2, 3

    work page 2023

  5. [5]

    Video generation models as world simulators, 2024

    Tim Brooks, Bill Peebles, Connor Holmes, Will DePue, Yufei Guo, Li Jing, David Schnurr, Joe Taylor, Troy Luh- man, Eric Luhman, et al. Video generation models as world simulators, 2024. 3

    2024

  6. [6]

    https://huggingface.co/cerspense /zeroscope v2 576w, 2023

    cerspense. https://huggingface.co/cerspense /zeroscope v2 576w, 2023. 3

    work page 2023

  7. [7]

    Pix2video: Video editing using image diffusion

    Duygu Ceylan, Chun-Hao P Huang, and Niloy J Mitra. Pix2video: Video editing using image diffusion. In ICCV, pages 23206–23217, 2023. 2

    work page 2023

  8. [8]

    Stable- video: Text-driven consistency-aware diffusion video edit- ing

    Wenhao Chai, Xun Guo, Gaoang Wang, and Yan Lu. Stable- video: Text-driven consistency-aware diffusion video edit- ing. In ICCV, pages 23040–23050, 2023. 2

    work page 2023

Show all 83 references
  1. [9]

    Attend-and-excite: Attention-based se- mantic guidance for text-to-image diffusion models

    Hila Chefer, Yuval Alaluf, Yael Vinker, Lior Wolf, and Daniel Cohen-Or. Attend-and-excite: Attention-based se- mantic guidance for text-to-image diffusion models. ACM Trans. Graph., 42(4):1–10, 2023. 3

    2023

  2. [10]

    Training- free layout control with cross-attention guidance

    Minghao Chen, Iro Laina, and Andrea Vedaldi. Training- free layout control with cross-attention guidance. In WACV, pages 5343–5353, 2024. 3

    2024

  3. [11]

    Gentron: Diffusion transformers for image and video generation

    Shoufa Chen, Mengmeng Xu, Jiawei Ren, Yuren Cong, Sen He, Yanping Xie, Animesh Sinha, Ping Luo, Tao Xiang, and Juan-Manuel Perez-Rua. Gentron: Diffusion transformers for image and video generation. In CVPR, 2024. 3

    2024

  4. [12]

    Consistent video- to-video transfer using synthetic dataset

    Jiaxin Cheng, Tianjun Xiao, and Tong He. Consistent video- to-video transfer using synthetic dataset. In ICLR, 2024. 2

    2024

  5. [13]

    Slicedit: Zero- shot video editing with text-to-image diffusion models using spatio-temporal slices

    Nathaniel Cohen, Vladimir Kulikov, Matan Kleiner, Inbar Huberman-Spiegelglas, and Tomer Michaeli. Slicedit: Zero- shot video editing with text-to-image diffusion models using spatio-temporal slices. In ICML, 2024. 6

    2024

  6. [14]

    FLATTEN: optical FLow-guided ATTENtion for consistent text-to-video edit- ing

    Yuren Cong, Mengmeng Xu, christian simon, Shoufa Chen, Jiawei Ren, Yanping Xie, Juan-Manuel Perez-Rua, Bodo Rosenhahn, Tao Xiang, and Sen He. FLATTEN: optical FLow-guided ATTENtion for consistent text-to-video edit- ing. In ICLR, 2024. 2, 3, 6, 7, 15, 16, 17

    2024

  7. [15]

    Diffusion mod- els beat gans on image synthesis

    Prafulla Dhariwal and Alexander Nichol. Diffusion mod- els beat gans on image synthesis. NeurIPS, 34:8780–8794,

  8. [16]

    Reflecting reality: Enabling diffusion models to produce faithful mirror reflections

    Ankit Dhiman, Manan Shah, Rishubh Parihar, Yash Bhalgat, Lokesh R Boregowda, and R Venkatesh Babu. Reflecting reality: Enabling diffusion models to produce faithful mirror reflections. arXiv preprint arXiv:2409.14677, 2024. 19

    2024 arXiv

  9. [17]

    The llama 3 herd of models

    Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Ab- hishek Kadian, Ahmad Al-Dahle, Aiesha Letman, Akhil Mathur, Alan Schelten, Amy Yang, Angela Fan, et al. The llama 3 herd of models. arXiv preprint arXiv:2407.21783 ,

    arXiv

  10. [18]

    Structure and content-guided video synthesis with diffusion models

    Patrick Esser, Johnathan Chiu, Parmida Atighehchian, Jonathan Granskog, and Anastasis Germanidis. Structure and content-guided video synthesis with diffusion models. In ICCV, pages 7346–7356, 2023. 3

    2023

  11. [19]

    Preserve your own correlation: A noise prior for video diffusion models

    Songwei Ge, Seungjun Nah, Guilin Liu, Tyler Poon, Andrew Tao, Bryan Catanzaro, David Jacobs, Jia-Bin Huang, Ming- Yu Liu, and Yogesh Balaji. Preserve your own correlation: A noise prior for video diffusion models. In ICCV, 2023. 3

    2023

  12. [20]

    Tokenflow: Consistent diffusion features for consistent video editing

    Michal Geyer, Omer Bar-Tal, Shai Bagon, and Tali Dekel. Tokenflow: Consistent diffusion features for consistent video editing. In ICLR, 2024. 2, 3, 6, 14, 15, 16, 17

    2024

  13. [21]

    Animatediff: Animate your personalized text-to- image diffusion models without specific tuning

    Yuwei Guo, Ceyuan Yang, Anyi Rao, Zhengyang Liang, Yaohui Wang, Yu Qiao, Maneesh Agrawala, Dahua Lin, and Bo Dai. Animatediff: Animate your personalized text-to- image diffusion models without specific tuning. ICLR, 2024. 3, 6

    2024

  14. [22]

    Latent video diffusion models for high-fidelity long video generation

    Yingqing He, Tianyu Yang, Yong Zhang, Ying Shan, and Qifeng Chen. Latent video diffusion models for high-fidelity long video generation. arXiv preprint arXiv:2211.13221 ,

    arXiv

  15. [23]

    Prompt-to-prompt im- age editing with cross attention control

    Amir Hertz, Ron Mokady, Jay Tenenbaum, Kfir Aberman, Yael Pritch, and Daniel Cohen-Or. Prompt-to-prompt im- age editing with cross attention control. arXiv preprint arXiv:2208.01626, 2022. 3

    2022 arXiv

  16. [24]

    Classifier-free diffusion guidance

    Jonathan Ho and Tim Salimans. Classifier-free diffusion guidance. arXiv preprint arXiv:2207.12598, 2022. 6, 12

    2022 arXiv

  17. [25]

    Denoising dif- fusion probabilistic models

    Jonathan Ho, Ajay Jain, and Pieter Abbeel. Denoising dif- fusion probabilistic models. NeurIPS, 33:6840–6851, 2020. 3

    2020

  18. [26]

    Imagen video: High definition video generation with diffusion mod- els

    Jonathan Ho, William Chan, Chitwan Saharia, Jay Whang, Ruiqi Gao, Alexey Gritsenko, Diederik P Kingma, Ben Poole, Mohammad Norouzi, David J Fleet, et al. Imagen video: High definition video generation with diffusion mod- els. arXiv preprint arXiv:2210.02303, 2022. 3

    arXiv

  19. [27]

    Tifa: Accurate and interpretable text-to-image faithfulness evaluation with question answering

    Yushi Hu, Benlin Liu, Jungo Kasai, Yizhong Wang, Mari Os- tendorf, Ranjay Krishna, and Noah A Smith. Tifa: Accurate and interpretable text-to-image faithfulness evaluation with question answering. In ICCV, pages 20406–20417, 2023. 15, 16

    2023

  20. [28]

    Ground-a-video: Zero- shot grounded video editing using text-to-image diffusion models

    Hyeonho Jeong and Jong Chul Ye. Ground-a-video: Zero- shot grounded video editing using text-to-image diffusion models. In ICLR, 2024. 2, 3, 5, 6, 7, 8, 15, 16, 17 9

    2024

  21. [29]

    Dreammotion: Space-time self-similar score distillation for zero-shot video editing

    Hyeonho Jeong, Jinho Chang, Geon Yeong Park, and Jong Chul Ye. Dreammotion: Space-time self-similar score distillation for zero-shot video editing. In ECCV, 2024. 6

    2024

  22. [30]

    Rave: Randomized noise shuf- fling for fast and consistent video editing with diffusion mod- els

    Ozgur Kara, Bariscan Kurtkaya, Hidir Yesiltepe, James M Rehg, and Pinar Yanardag. Rave: Randomized noise shuf- fling for fast and consistent video editing with diffusion mod- els. In CVPR, pages 6507–6516, 2024. 2, 3, 6, 7, 15, 16, 17

    2024

  23. [31]

    Unlocking creator-ai synergy: Challenges, requirements, and design opportunities in ai- powered short-form video production

    Jini Kim and Hajun Kim. Unlocking creator-ai synergy: Challenges, requirements, and design opportunities in ai- powered short-form video production. In Proceedings of the CHI Conference on Human Factors in Computing Systems , pages 1–23, 2024. 1

    2024

  24. [32]

    Dense text-to-image generation with attention modulation

    Yunji Kim, Jiyoung Lee, Jin-Hwa Kim, Jung-Woo Ha, and Jun-Yan Zhu. Dense text-to-image generation with attention modulation. In ICCV, pages 7701–7711, 2023. 2, 3, 4, 8, 16, 17

    2023

  25. [33]

    Gligen: Open-set grounded text-to-image generation

    Yuheng Li, Haotian Liu, Qingyang Wu, Fangzhou Mu, Jian- wei Yang, Jianfeng Gao, Chunyuan Li, and Yong Jae Lee. Gligen: Open-set grounded text-to-image generation. In CVPR, pages 22511–22521, 2023. 2, 3

    2023

  26. [34]

    Microsoft coco: Common objects in context

    Tsung-Yi Lin, Michael Maire, Serge Belongie, James Hays, Pietro Perona, Deva Ramanan, Piotr Doll´ar, and C Lawrence Zitnick. Microsoft coco: Common objects in context. In ECCV, pages 740–755. Springer, 2014. 15, 16

    2014

  27. [35]

    Improved baselines with visual instruction tuning

    Haotian Liu, Chunyuan Li, Yuheng Li, and Yong Jae Lee. Improved baselines with visual instruction tuning. In CVPR, pages 26296–26306, 2024. 5, 12, 13

    2024

  28. [36]

    Video-p2p: Video editing with cross-attention control

    Shaoteng Liu, Yuechen Zhang, Wenbo Li, Zhe Lin, and Jiaya Jia. Video-p2p: Video editing with cross-attention control. In CVPR, pages 8599–8608, 2024. 2, 3, 6, 16, 23

    2024

  29. [37]

    Short-form videos for public library marketing: performance analytics of douyin in china

    Ying Liu, Dickson KW Chiu, and Kevin KW Ho. Short-form videos for public library marketing: performance analytics of douyin in china. Applied Sciences, 13(6):3386, 2023. 1

    2023

  30. [38]

    Crepe: Can vision-language foundation models reason compositionally? In CVPR, pages 10910–10921, 2023

    Zixian Ma, Jerry Hong, Mustafa Omer Gul, Mona Gandhi, Irena Gao, and Ranjay Krishna. Crepe: Can vision-language foundation models reason compositionally? In CVPR, pages 10910–10921, 2023. 15, 16

    2023

  31. [39]

    Large-scale video panoptic segmen- tation in the wild: A benchmark

    Jiaxu Miao, Xiaohan Wang, Yu Wu, Wei Li, Xu Zhang, Yun- chao Wei, and Yi Yang. Large-scale video panoptic segmen- tation in the wild: A benchmark. In CVPR, pages 21033– 21043, 2022. 5, 12

    2022

  32. [40]

    GLIDE: Towards photorealis- tic image generation and editing with text-guided diffusion models

    Alexander Quinn Nichol, Prafulla Dhariwal, Aditya Ramesh, Pranav Shyam, Pamela Mishkin, Bob Mcgrew, Ilya Sutskever, and Mark Chen. GLIDE: Towards photorealis- tic image generation and editing with text-guided diffusion models. In ICML, pages 16784–16804, 2022. 3

    2022

  33. [41]

    The 2017 davis challenge on video object segmentation.arXiv preprint arXiv:1704.00675, 2017

    Jordi Pont-Tuset, Federico Perazzi, Sergi Caelles, Pablo Ar- bel´aez, Alex Sorkine-Hornung, and Luc Van Gool. The 2017 davis challenge on video object segmentation.arXiv preprint arXiv:1704.00675, 2017. 5

    2017 arXiv

  34. [42]

    Fatezero: Fus- ing attentions for zero-shot text-based video editing

    Chenyang Qi, Xiaodong Cun, Yong Zhang, Chenyang Lei, Xintao Wang, Ying Shan, and Qifeng Chen. Fatezero: Fus- ing attentions for zero-shot text-based video editing. In ICCV, pages 15932–15942, 2023. 2, 3

    2023

  35. [43]

    Learn- ing transferable visual models from natural language super- vision

    Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, et al. Learn- ing transferable visual models from natural language super- vision. In ICML, pages 8748–8763. PMLR, 2021. 2, 4, 6, 13

    2021

  36. [44]

    Hierarchical text-conditional image gener- ation with clip latents

    Aditya Ramesh, Prafulla Dhariwal, Alex Nichol, Casey Chu, and Mark Chen. Hierarchical text-conditional image gener- ation with clip latents. arXiv preprint arXiv:2204.06125, 1 (2):3, 2022. 3

    2022 arXiv

  37. [45]

    Towards robust monocular depth estimation: Mixing datasets for zero-shot cross-dataset transfer

    Ren ´e Ranftl, Katrin Lasinger, David Hafner, Konrad Schindler, and Vladlen Koltun. Towards robust monocular depth estimation: Mixing datasets for zero-shot cross-dataset transfer. IEEE TPAMI, 44(3):1623–1637, 2020. 4, 6

    2020

  38. [46]

    High-resolution image syn- thesis with latent diffusion models

    Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Bj¨orn Ommer. High-resolution image syn- thesis with latent diffusion models. In CVPR, pages 10684– 10695, 2022. 2, 3, 4, 6, 8, 12, 15, 17

    2022

  39. [47]

    Photorealistic text-to-image diffusion models with deep language understanding

    Chitwan Saharia, William Chan, Saurabh Saxena, Lala Li, Jay Whang, Emily L Denton, Kamyar Ghasemipour, Raphael Gontijo Lopes, Burcu Karagol Ayan, Tim Salimans, et al. Photorealistic text-to-image diffusion models with deep language understanding. NeurIPS, 35:36479–36494, 2022. 3

    2022

  40. [48]

    Noisecollage: A layout-aware text-to-image diffusion model based on noise cropping and merging

    Takahiro Shirakawa and Seiichi Uchida. Noisecollage: A layout-aware text-to-image diffusion model based on noise cropping and merging. In CVPR, pages 8921–8930, 2024. 2, 3

    2024

  41. [49]

    Video edit- ing via factorized diffusion distillation

    Uriel Singer, Amit Zohar, Yuval Kirstain, Shelly Sheynin, Adam Polyak, Devi Parikh, and Yaniv Taigman. Video edit- ing via factorized diffusion distillation. InECCV, pages 450–

  42. [50]

    Automation in video editing: Assisted work- flows in video editing

    Than Htut Soe. Automation in video editing: Assisted work- flows in video editing. In AutomationXP@ CHI, 2021. 1

    2021

  43. [51]

    Denois- ing diffusion implicit models

    Jiaming Song, Chenlin Meng, and Stefano Ermon. Denois- ing diffusion implicit models. In ICLR, 2021. 3, 5, 12

    2021

  44. [52]

    Score-based generative modeling through stochastic differential equa- tions

    Yang Song, Jascha Sohl-Dickstein, Diederik P Kingma, Ab- hishek Kumar, Stefano Ermon, and Ben Poole. Score-based generative modeling through stochastic differential equa- tions. In ICLR, 2021. 3

    2021

  45. [53]

    The rise of mobile and social short-form video: an in-depth measure- ment study of vine

    Baptist Vandersmissen, Fr ´ederic Godin, Abhineshwar Tomar, Wesley De Neve, and Rik Van de Walle. The rise of mobile and social short-form video: an in-depth measure- ment study of vine. In Workshop on Social Multimedia and Storytelling (SoMuS 2014), pages 1–10, 2014. 1

    2014

  46. [54]

    Modelscope text-to-video technical report

    Jiuniu Wang, Hangjie Yuan, Dayou Chen, Yingya Zhang, Xiang Wang, and Shiwei Zhang. Modelscope text-to-video technical report. arXiv preprint arXiv:2308.06571, 2023. 3

    2023 arXiv

  47. [55]

    Instancediffusion: Instance-level control for image generation

    Xudong Wang, Trevor Darrell, Sai Saketh Rambhatla, Rohit Girdhar, and Ishan Misra. Instancediffusion: Instance-level control for image generation. In CVPR, pages 6232–6242,

  48. [56]

    Videocomposer: Compositional video synthesis with motion controllability

    Xiang Wang, Hangjie Yuan, Shiwei Zhang, Dayou Chen, Ji- uniu Wang, Yingya Zhang, Yujun Shen, Deli Zhao, and Jin- gren Zhou. Videocomposer: Compositional video synthesis with motion controllability. NeurIPS, 36, 2024. 3, 8

    2024

  49. [57]

    Lavie: High-quality video gener- ation with cascaded latent diffusion models

    Yaohui Wang, Xinyuan Chen, Xin Ma, Shangchen Zhou, Ziqi Huang, Yi Wang, Ceyuan Yang, Yinan He, Jiashuo Yu, Peiqing Yang, et al. Lavie: High-quality video gener- ation with cascaded latent diffusion models. arXiv preprint arXiv:2309.15103, 2023. 3 10

    2023 arXiv

  50. [58]

    Bovik, H.R

    Zhou Wang, A.C. Bovik, H.R. Sheikh, and E.P. Simoncelli. Image quality assessment: from error visibility to structural similarity. IEEE TIP, 13(4):600–612, 2004. 6

    2004

  51. [59]

    Tune-a-video: One-shot tuning of image diffusion models for text-to-video generation

    Jay Zhangjie Wu, Yixiao Ge, Xintao Wang, Stan Weixian Lei, Yuchao Gu, Yufei Shi, Wynne Hsu, Ying Shan, Xiaohu Qie, and Mike Zheng Shou. Tune-a-video: One-shot tuning of image diffusion models for text-to-video generation. In ICCV, pages 7623–7633, 2023. 2, 3

    2023

  52. [60]

    Cvpr 2023 text guided video edit- ing competition

    Jay Zhangjie Wu, Xiuyu Li, Difei Gao, Zhen Dong, Jin- bin Bai, Aishani Singh, Xiaoyu Xiang, Youzeng Li, Zuwei Huang, Yuanxi Sun, et al. Cvpr 2023 text guided video edit- ing competition. arXiv preprint arXiv:2310.16003, 2023. 5, 6

    2023 arXiv

  53. [61]

    Boxdiff: Text-to-image synthesis with training-free box-constrained diffusion

    Jinheng Xie, Yuexiang Li, Yawen Huang, Haozhe Liu, Wen- tian Zhang, Yefeng Zheng, and Mike Zheng Shou. Boxdiff: Text-to-image synthesis with training-free box-constrained diffusion. In ICCV, pages 7452–7461, 2023. 3

    2023

  54. [62]

    Dynamic prompt learning: Addressing cross- attention leakage for text-based image editing

    Fei Yang, Shiqi Yang, Muhammad Atif Butt, Joost van de Weijer, et al. Dynamic prompt learning: Addressing cross- attention leakage for text-based image editing. NeurIPS, 36: 26291–26303, 2023. 2

    2023

  55. [63]

    Fresco: Spatial-temporal correspondence for zero-shot video translation

    Shuai Yang, Yifan Zhou, Ziwei Liu, and Chen Change Loy. Fresco: Spatial-temporal correspondence for zero-shot video translation. In CVPR, pages 8703–8712, 2024. 2, 3, 6, 14, 15, 16, 17

    2024

  56. [64]

    Eva: Zero-shot accurate attributes and multi-object video editing

    Xiangpeng Yang, Linchao Zhu, Hehe Fan, and Yi Yang. Eva: Zero-shot accurate attributes and multi-object video editing. arXiv preprint arXiv:2403.16111, 2024. 6

    2024 arXiv

  57. [65]

    Videograin: Modulating space-time attention for multi- grained video editing

    Xiangpeng Yang, Linchao Zhu, Hehe Fan, and Yi Yang. Videograin: Modulating space-time attention for multi- grained video editing. In ICLR, 2025. 2, 3, 4, 5, 6, 7, 15, 16, 17, 19

    2025

  58. [66]

    Reco: Region-controlled text-to-image genera- tion

    Zhengyuan Yang, Jianfeng Wang, Zhe Gan, Linjie Li, Kevin Lin, Chenfei Wu, Nan Duan, Zicheng Liu, Ce Liu, Michael Zeng, et al. Reco: Region-controlled text-to-image genera- tion. In CVPR, pages 14246–14255, 2023. 3

    2023

  59. [67]

    Space-time diffusion features for zero-shot text-driven motion transfer

    Danah Yatim, Rafail Fridman, Omer Bar-Tal, Yoni Kasten, and Tali Dekel. Space-time diffusion features for zero-shot text-driven motion transfer. In CVPR, 2024. 3

    2024

  60. [68]

    Magvit: Masked generative video transformer

    Lijun Yu, Yong Cheng, Kihyuk Sohn, Jos ´e Lezama, Han Zhang, Huiwen Chang, Alexander G Hauptmann, Ming- Hsuan Yang, Yuan Hao, Irfan Essa, et al. Magvit: Masked generative video transformer. InCVPR, pages 10459–10469,

  61. [69]

    Video probabilistic diffusion models in projected latent space

    Sihyun Yu, Kihyuk Sohn, Subin Kim, and Jinwoo Shin. Video probabilistic diffusion models in projected latent space. In CVPR, pages 18456–18466, 2023. 3

    2023

  62. [70]

    Deepcens: A deep learning-based system for real-time image and video censorship

    Asim Sinan Yuksel and Fatma Gulsah Tan. Deepcens: A deep learning-based system for real-time image and video censorship. Expert Systems, 40(10):e13436, 2023. 2

    2023

  63. [71]

    Adding conditional control to text-to-image diffusion models

    Lvmin Zhang, Anyi Rao, and Maneesh Agrawala. Adding conditional control to text-to-image diffusion models. In ICCV, pages 3836–3847, 2023. 3, 4, 6, 12

    2023

  64. [72]

    Controlvideo: Training-free controllable text-to-video generation

    Yabo Zhang, Yuxiang Wei, Dongsheng Jiang, Xiaopeng Zhang, Wangmeng Zuo, and Qi Tian. Controlvideo: Training-free controllable text-to-video generation. In ICLR,

  65. [73]

    Avid: Any-length video inpainting with diffusion model

    Zhixing Zhang, Bichen Wu, Xiaoyan Wang, Yaqiao Luo, Luxin Zhang, Yinan Zhao, Peter Vajda, Dimitris Metaxas, and Licheng Yu. Avid: Any-length video inpainting with diffusion model. In CVPR, pages 7162–7172, 2024. 2, 3, 5, 12, 13 11 PRIMEdit: Probability Redistribution for Insta...

    2024

  66. [76]

    (12) ControlNet [71] condition e can be added as additional guidance for the sampling and can be obtained from any structured information (e.g., depth maps)

    to perform editing: ˆzi t−1 = √αt−1 ˆzi t − √1 − αtϵθ(It)√αt + √1 − αt−1ϵθ(It). (12) ControlNet [71] condition e can be added as additional guidance for the sampling and can be obtained from any structured information (e.g., depth maps). The edited frame ˆf i = D(ˆzi

  67. [77]

    thing” instances for our task, we still generate captions for the “stuff

    is obtained using the decoder D of the LDM. A classifier-free guidance [24] scale of scf g= 1 and a larger scale scf g ≫ 1 are used during inversion and denoising, respectively. B. Dataset and Metrics Additional Details B.1. MIVE Dataset Construction To create our MIVE Dataset...

  68. [78]

    Image Captioning 2. Text Summarization Input Frames “A <21247>cat</21247> sits on a <4618360>white carpeted floor</4618360>, engaging with its <65474>toys</65474>, against a backdrop of <60415>white curtain</60415>, a <12779264>wooden table</12779264>, and a <7895160>white wal...

  69. [79]

    Can you describe the scene in this image if the image must contain cat, curtain, toy, floor, wall, and table or desk?

    Manual Fix and Annotation “Can you describe the scene in this image if the image must contain cat, curtain, toy, floor, wall, and table or desk?” *choose the caption that includes most of the instances *randomly select instance target caption from the five variations “You are ...

  70. [80]

    Manually including instances Llama 3 missed in summarization

  71. [81]

    Manually adding starting and ending tags to instances

  72. [82]

    Fixing grammar with the help of Llama 3 “A <21247>white cat with brown spots</21247> sits on a <4618360>white carpeted floor</4618360>, engaging with its <65474>toys</65474>, against a backdrop of <60415>white curtain</60415>, a <12779264>white wooden bench</12779264>, and a <...

  73. [83]

    A young boy in a green shirt stands on a porch in front of a door with a carpet and nearby a wall, smiling for a photo taken by a person with a mobile phone

    Target Caption Generation “For all of the following prompts with the given video ID, can you retexture or swap each of the listed objects with others of similar shape? For each object, there is a 50:50 chance of retexturing or swapping. Please generate five variations for each...

    1960

  74. [466]

    Springer, 2024. 5, 6

    2024

  75. [2024]

    2, 3, 6, 7, 15, 16, 17

Pith tools

Reviewed August 11, 2026 · model on record in the stance chip above.