Pith. sign in

REVIEW 4 major objections 5 minor 63 references

Transforming Static Images Using Generative Models for Video Salient Object Detection

T0 review · 4 major / 5 minor · reviewed 2026-08-12 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read Training on image-flow pairs synthesized by a video diffusion model plus a flow estimator achieves state-of-the-art video salient object detection on four benchmarks.

desk verdict A simple, reproducible recipe for VSOD training data, with a real but fixable missing-baseline gap in the central claim. read the letter →

arxiv 2411.13975 v1 pith:TUNT5C3K submitted 2024-11-21 cs.CV

Suhwan Cho , Minhyeok Lee , Jungho Lee , Sangyoun Lee This is my paper · ORCID
classification cs.CV
keywords videosalientobjectdetectionimage-to-videogenerationopticalflowdiffusionmodelssyntheticdataaugmentationtwo-streamnetworkStableRAFT
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper aims to show that image-to-video diffusion models can replace hand-crafted spatial warping as a way to synthesize training data for video salient object detection (VSOD), the task of locating and segmenting the most attention-grabbing object in every frame. The recipe starts from a static image, lets a diffusion model generate several plausible successor frames, and uses a pre-trained optical flow estimator to produce motion maps between the source and each generated frame; these synthetic image-flow pairs are then mixed with real video clips to train a simple two-stream network. On four standard VSOD benchmarks the trained model reaches the highest reported structure-measure scores. The wider promise is that realistic, object-aware motion cues can be produced from abundant static images, reducing the field's dependence on costly video annotations.

What carries the argument

The load-bearing machinery is a data-creation pipeline: a source image is fed to Stable Video Diffusion, which denoises latents conditioned on the source to produce 14 generated frames; a pre-trained RAFT model then estimates optical flow from the source to each generated frame, yielding paired samples with spatially aligned masks and flows. Because the diffusion model generates whole frames rather than warping pixels, the flows can separate foreground and background motion. The segmentation network is a two-stream encoder with MiT-b2 backbones for appearance and motion, lightweight attention fusion, and a progressive decoder; training mixes simulated data with real video data at a 2:1:1 ratio. This setup turns 10,553 DUTS training images plus 5,019 test images into 15,572 synthetic image-flow pairs.

What would settle it

Train the same network on the same number of simulated pairs built with traditional spatial warps, such as affine or spline transformations, instead of diffusion-generated frames; if that model matches or beats the diffusion-based one on the four benchmarks, the claim that diffusion-based generation is the source of the improvement is falsified.

Watch

Extended reading notes

Core claim

The central claim is that the bottleneck for flow-guided VSOD is not the segmentation architecture but the quality and quantity of motion cues in training data. Conventional image-to-video simulation via affine warps or splines produces optical flows that do not reflect each object's independent movement, because the warps move all pixels according to one global deformation. The paper claims that Stable Video Diffusion, an image-to-video diffusion model, transforms a static source image into several novel frames while preserving object-level context, so that optical flow maps estimated by RAFT between the source and these generated frames carry plausible, object-specific motion. Paired with DUTS images under a 2:1:1 mixture of simulated data, DAVIS 2016 training, and DAVSOD training, this data supplies enough realistic motion cues that a simple MiT-b2 two-stream network reaches new state-of-the-art S-measure results: 94.5% on DAVIS 2016, 92.6% on FBMS, 80.3% on DAVSOD, and 96.2% on ViSal.

Load-bearing premise

The entire transfer depends on the unstated assumption that RAFT optical flows computed between a real static image and a diffusion-generated frame are statistically close enough to real video flows that a network trained on them generalizes to real videos.

Editorial extensions

If this is right

  • Simulated image-flow pairs alone outperform training on real video data on FBMS, DAVSOD, and ViSal, and the mixed training set outperforms both on every benchmark.
  • Because each source image yields multiple paired samples, the synthetic dataset scales linearly with the number of static images, so large image collections can be converted into motion-annotated training data.
  • The gains are obtained with a simple two-stream backbone, suggesting that data quality can substitute for architectural complexity in flow-guided saliency.
  • The same recipe should transfer to other two-stream video tasks that rely on optical flow as input, since it produces paired RGB-flow samples without manual video annotation.
  • The paper's own limitation analysis implies that improving the generator's temporal consistency would further improve the simulated flows and downstream accuracy.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Extension: the paper leaves implicit a test that would isolate whether the gain comes from diffusion-generated realism or simply from having 15,572 extra training pairs; ablating with the same volume of warped-image pairs would settle it.
  • Extension: the simulated flows could be evaluated directly—for example by comparing their statistics or warping error against real video flows—to give a quantitative proxy for transfer before training a network.
  • Extension: a natural follow-up is to chain generated frames, using frame t as the source for frame t+1, to produce longer-range motion and reduce the current reliance on a single static anchor frame.
  • Extension: the reported checkerboard artifacts in simulated flows suggest a filtering or augmentation strategy that treats these artifacts as hard negatives, which the paper does not explore.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. The paper proposes a data-simulation strategy for video salient object detection (VSOD): starting from static images in the DUTS dataset, it uses Stable Video Diffusion to generate a short video from each image, estimates optical flows between the source image and each generated target frame with RAFT, and trains a two-stream encoder-decoder network (RGB plus flow) on a mixture of these synthetic image-flow pairs and real video data from DAVIS2016 and DAVSOD. The method, named RealFlow, is reported to achieve the highest S-measure on DAVIS2016 (94.5%), FBMS (92.6%), DAVSOD (80.3%), and ViSal (96.2%), and top F-measure on three of the four benchmarks. The paper also includes ablations on training protocol, backbone size, and input resolution, along with qualitative comparisons of simulated and real flows.

Significance. If the central claim is correct, the paper would demonstrate a practical way to leverage large-scale image datasets for motion-guided video tasks without costly video annotations, using off-the-shelf image-to-video diffusion and optical flow estimation. The proposed pipeline is simple and reproducible, and the authors commit to releasing code and models. The main experiments use standard benchmarks and metrics, and the ablation in Table 2 isolates the effect of simulated versus real training data to some degree. However, the causal interpretation that diffusion-based flow realism, rather than sheer training-data volume and diversity, drives the improvement is not yet supported, because the paper omits the obvious controlled baseline of training on flows from traditional warping of the same static images. The reported gains over prior state-of-the-art methods are small in several cases, and no variance estimates are provided, so the strength of the empirical claim is uncertain despite the strong headline numbers.

major comments (4)
  1. [Section 3.2, Table 2, and Section 5] The central claim of the paper, stated in the Abstract and Introduction, is that diffusion-based image-to-video generation produces 'realistic' or 'plausible' optical flows that improve VSOD training, unlike affine or spline warping. The ablation in Table 2 only compares training on real video, simulated diffusion data, and a mixture of the two. It never includes a 'Warped' condition in which the same DUTS source images are deformed with thin-plate spline or affine transformations, flows are estimated with the same RAFT model, and the same two-stream network is trained. Without this control, the advantage of Simulated over Real is confounded by training-set size and diversity: the DUTS-based synthetic set yields many more training pairs (about 15,572 source images, each producing multiple flow pairs) than the roughly 91 real training videos in DAVIS2016 and DAVSOD. I request that the authors add this warped-data baseline and report the same metrics. If the warped baseline matches or exceeds the diffusion-based result, the contribution reduces to generic data augmentation rather than diffusion-specific motion realism, and the state-of-the-art numbers lose their current causal interpretation.
  2. [Table 1 and Section 4.3] All results are reported as single-run numbers without error bars or multiple seeds. Differences with prior work are often small: on DAVIS2016 the S-measure is 94.5 vs. 93.5 for OAST, on DAVSOD it is 80.3 vs. 79.8 for TGFormer, and on ViSal it is 96.2 vs. 95.2 for TGFormer. These margins may be within run-to-run variation for a randomly initialized network, especially with a training process that involves stochastic data shuffling and diffusion sampling. The authors should report mean and standard deviation over at least three independent training runs for the main comparison (Table 1) and for the key ablation in Table 2, or otherwise demonstrate that the observed differences are reproducible and significant.
  3. [Section 3.3, 'Dataset construction' in Section 3.2] The mixture ratio of simulated, DAVIS2016, and DAVSOD data is set to 2:1:1 with no sensitivity analysis and no justification. This ratio is load-bearing because it determines how much synthetic data is used relative to real data, and the comparison in Table 2 between 'Simulated' and 'Real' does not control for total sample count. I recommend sweeping the mixture ratio (e.g., 1:1:1, 3:1:1, and the current 2:1:1) and reporting the resulting scores. Additionally, the text in Section 3.2 is internally inconsistent about the total dataset size: it first says 'from N source images, we can obtain N*T paired data samples' and then states that a total of 15,572 data pairs is obtained from 15,572 DUTS images, which would be the case only if T=1. Please clarify the exact number of training pairs used.
  4. [Section 5 (Limitation) and Section 4.5] The paper's own limitation statement concedes that generated frames 'often lack continuous motion' and that flow maps 'frequently exhibit checkerboard artifacts.' This directly weakens the premise, stated in the Introduction and Figure 1, that the simulated flows are realistic and capture independent object motion. The qualitative evidence in Figures 5 and 6 is not sufficient to establish that the simulated flow distribution is closer to real video flows than flows from warped images. The authors should either provide quantitative evidence that the simulated flows resemble real flows (for example, statistics of flow magnitude, distribution of flow boundaries, or a learned discriminator's accuracy) or demonstrate empirically that the checkerboard artifacts and non-continuous motion do not harm training, for instance by ablating post-processing of the flows. Without this, the mechanism behind the performance gain remains unclear.
minor comments (5)
  1. [Section 3.2, Eq. (2) and Eq. (3)] The notation 'P T 1' and 'Q T 1' is typeset awkwardly; it should be P_1^T, and the subscripts/superscripts need to be cleaned up.
  2. [Throughout, figure captions] The figures contain the watermark '© 2023 Adobe. All Rights Reserved. Adobe Confidential.' This is not part of a normal academic manuscript and should be removed; its presence suggests the figures were taken from an internal Adobe document or the manuscript template was not properly cleaned.
  3. [Abstract and Section 4.3] The Abstract claims 'state-of-the-art performance across all public benchmark datasets' without qualification, but Section 4.3 correctly notes that the F-measure is top on three of the four benchmarks; on FBMS, RealFlow's F-measure (90.6) is below OAST and TGFormer (91.9). Please align the Abstract's wording with the more precise statement in Section 4.3.
  4. [Section 3.1, Eq. (1)] The dimensionality notation 'x ∈ RC×T HW' is malformed; it should be R^{C×T×H×W}, and 'repeat(zs, T)' should specify the repetition over the temporal dimension.
  5. [References, [47]] The text cites 'deterministic DDIM sampler [47]' but reference [47] is Song and Ermon, 'Improved techniques for training score-based generative models.' The DDIM sampler is from Song et al., 'Denoising Diffusion Implicit Models' (2020). The citation should be corrected.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the synthetic training pipeline relies on external pretrained models and standard benchmarks, with no fitted quantity fed back from the evaluation sets.

full rationale

The derivation chain is: DUTS still images and masks are used as sources; Stable Video Diffusion generates target frames; RAFT estimates source-to-target optical flows; the resulting (image, flow, mask) pairs train a two-stream VSOD network; and the trained network is evaluated on DAVIS2016, FBMS, DAVSOD, and ViSal. Every load-bearing input is external: SVD and RAFT are pretrained by other groups, DUTS masks are human annotations, and benchmark splits are standard. No parameter is fitted to the evaluation benchmarks and then renamed a prediction. The paper's self-citations (e.g., TMO) appear as comparative baselines or related work, not as the justification for the central method. Using RAFT to produce both synthetic training flows and the test-time flow inputs is a consistency choice, not a circular reduction: the supervised target remains human saliency masks, not the flow itself. The absence of a warped-data control and the acknowledged limitations (checkerboard flow artifacts, non-continuous motion) weaken the causal claim that diffusion-based motion realism is uniquely responsible for the gains, but these are experimental-design concerns, not cases where an output is equivalent to an input by construction. Therefore the circularity score is 0.

Assumptions & free parameters 4 free parameters · 4 assumptions · 0 invented entities

The method's contribution is empirical: it reuses external pretrained models (SVD, RAFT) and a standard saliency dataset (DUTS). No new physical entities or theoretical constructs are introduced. The main load-bearing choices are the generation settings and the training mixture ratio, which are specified and reproducible but not exhaustively tuned; the central result does not depend on fitting model parameters to the VSOD benchmarks.

free parameters (4)
  • classifier-free guidance scale for SVD sampling = 3.0 (first frame), 1.0 (last frame)
    Chosen to balance generation quality and fidelity; stated in Section 3.1. Not tuned in this paper, but a hand-set number that influences generated frame content and hence flow quality.
  • DDIM sampling steps = 25
    Deterministic DDIM with 25 steps, stated in Section 3.1. Affects generation quality and diversity.
  • decoding chunk size = 8
    Set to trade generation quality against memory, Section 3.1. A hand-chosen implementation detail.
  • mixture ratio (simulated : DAVIS16 : DAVSOD) = 2:1:1
    Proportion of simulated vs real training data in Section 3.3. Chosen without reported sensitivity analysis; the central claim that mixing helps depends on this ratio being effective.
assumptions (4)
  • domain assumption Optical flow is a useful and sufficient motion representation for video salient object detection.
    The entire two-stream design assumes flow adds signal beyond appearance. Section 2 reviews many flow-guided VSOD methods, and the paper builds on that consensus.
  • domain assumption RAFT accurately estimates optical flow on synthetic image pairs consisting of a real image and a diffusion-generated frame.
    Section 3.2 uses a pretrained RAFT without fine-tuning on generated data. If RAFT fails on synthetic frames, the training signal is noisy; no quantitative validation of flow accuracy on generated pairs is provided.
  • domain assumption Stable Video Diffusion generates video frames that preserve the source image semantics while moving objects independently.
    Central premise of the augmentation method, stated in Section 3.1. Only supported qualitatively (Figure 5) and by downstream benchmark improvement, not by a direct measurement of semantic preservation or flow realism.
  • domain assumption The source image's ground-truth saliency mask remains a valid training target when paired with flow computed between the source and a generated frame.
    Section 3.2 creates pairs (I_s, F_{s->t}) with the mask of I_s. If the generated frame implies a different saliency configuration, supervision is misaligned. The authors acknowledge in Section 5 that generated first frames can differ from the source, yet still pair them.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Transforming Static Images Using Generative Models for Video Salient Object Detection." pith.science (2026). https://pith.science/paper/TUNT5C3K

@misc{pith2026241113975,
  author       = {Pith},
  title        = {Pith review of: Transforming Static Images Using Generative Models for Video Salient Object Detection},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/TUNT5C3K}},
  note         = {Machine review of arXiv:2411.13975}
}
read the original abstract

In many video processing tasks, leveraging large-scale image datasets is a common strategy, as image data is more abundant and facilitates comprehensive knowledge transfer. A typical approach for simulating video from static images involves applying spatial transformations, such as affine transformations and spline warping, to create sequences that mimic temporal progression. However, in tasks like video salient object detection, where both appearance and motion cues are critical, these basic image-to-video techniques fail to produce realistic optical flows that capture the independent motion properties of each object. In this study, we show that image-to-video diffusion models can generate realistic transformations of static images while understanding the contextual relationships between image components. This ability allows the model to generate plausible optical flows, preserving semantic integrity while reflecting the independent motion of scene elements. By augmenting individual images in this way, we create large-scale image-flow pairs that significantly enhance model training. Our approach achieves state-of-the-art performance across all public benchmark datasets, outperforming existing approaches.

Figures

Figures reproduced from arXiv: 2411.13975 by the authors.

Figure 1
Figure 1. Visualization of the optical flow generation process from [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. Overview of our training sample simulation process. Starting with a static source image, target images are generated using an [PITH_FULL_IMAGE:figures/full_fig_p003_2.png] view at source ↗
Figure 3
Figure 3. Visualization of our two-stream network architecture, [PITH_FULL_IMAGE:figures/full_fig_p004_3.png] view at source ↗
Figures from the paper (5 more)
Figure 4
Figure 4. Figure 4: Qualitative comparison between state-of-the-art methods and the proposed RealFlow. [PITH_FULL_IMAGE:figures/full_fig_p006_4.png]
Figure 5
Figure 5. Figure 5: Aligned qualitative comparison between real video data and our simulated data. [PITH_FULL_IMAGE:figures/full_fig_p007_5.png]
Figure 6
Figure 6. Figure 6: Direct qualitative comparison of actual and simulated [PITH_FULL_IMAGE:figures/full_fig_p007_6.png]
Figure 7
Figure 7. Figure 7: Visualization of the flow map generated between the [PITH_FULL_IMAGE:figures/full_fig_p008_7.png]
Figure 8
Figure 8. Figure 8: Visualization of the limitations of our simulated flows. [PITH_FULL_IMAGE:figures/full_fig_p008_8.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

63 extracted references · 21 canonical work pages

  1. [1]

    Frequency-tuned salient region de- tection

    Radhakrishna Achanta, Sheila Hemami, Francisco Estrada, and Sabine Susstrunk. Frequency-tuned salient region de- tection. In 2009 IEEE conference on computer vision and pattern recognition, pages 1597–1604. IEEE, 2009. v

    2009

  2. [2]

    Stem-seg: Spatio-temporal em- beddings for instance segmentation in videos

    Ali Athar, Sabarinath Mahadevan, Aljosa Osep, Laura Leal- Taix´e, and Bastian Leibe. Stem-seg: Spatio-temporal em- beddings for instance segmentation in videos. In Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XI 16 , pages 158–177. Springer, 2020. ii

    2020

  3. [3]

    Stable video diffusion: Scaling latent video diffusion models to large datasets

    Andreas Blattmann, Tim Dockhorn, Sumith Kulal, Daniel Mendelevitch, Maciej Kilian, Dominik Lorenz, Yam Levi, Zion English, Vikram V oleti, Adam Letts, et al. Stable video diffusion: Scaling latent video diffusion models to large datasets. arXiv preprint arXiv:2311.15127, 2023. iii

    arXiv 2023

  4. [4]

    A thin-plate spline and the decomposition of deformations

    Fred L Bookstein and WDK Green. A thin-plate spline and the decomposition of deformations. Mathematical Methods in Medical Imaging, 2(14-28):3, 1993. ii

    1993

  5. [5]

    Video generation models as world simulators

    Tim Brooks, Bill Peebles, Connor Holmes, Will DePue, Yufei Guo, Li Jing, David Schnurr, Joe Taylor, Troy Luh- man, Eric Luhman, et al. Video generation models as world simulators. 2024. URL https://openai. com/research/video- generation-models-as-world-simulators, 3, 2024. iii

    work page 2024

  6. [6]

    Video salient object detection via contrastive features and attention modules

    Yi-Wen Chen, Xiaojie Jin, Xiaohui Shen, and Ming-Hsuan Yang. Video salient object detection via contrastive features and attention modules. In Proceedings of the IEEE/CVF winter conference on applications of computer vision, pages 1320–1329, 2022. v

    2022

  7. [7]

    Xmem: Long- term video object segmentation with an atkinson-shiffrin memory model

    Ho Kei Cheng and Alexander G Schwing. Xmem: Long- term video object segmentation with an atkinson-shiffrin memory model. In European Conference on Computer Vi- sion, pages 640–658. Springer, 2022. ii

    2022

  8. [8]

    Rethink- ing space-time networks with improved memory coverage for efficient video object segmentation

    Ho Kei Cheng, Yu-Wing Tai, and Chi-Keung Tang. Rethink- ing space-time networks with improved memory coverage for efficient video object segmentation. Advances in Neural Information Processing Systems, 34:11781–11794, 2021. ii

    2021

Show all 63 references
  1. [9]

    Global contrast based salient region detection

    Ming-Ming Cheng, Niloy J Mitra, Xiaolei Huang, Philip HS Torr, and Shi-Min Hu. Global contrast based salient region detection. IEEE transactions on pattern analysis and ma- chine intelligence, 37(3):569–582, 2014. ii

    2014

  2. [10]

    Tack- ling background distraction in video object segmentation

    Suhwan Cho, Heansung Lee, Minhyeok Lee, Chaewon Park, Sungjun Jang, Minjung Kim, and Sangyoun Lee. Tack- ling background distraction in video object segmentation. In European Conference on Computer Vision, pages 446–462. Springer, 2022. ii

    2022

  3. [11]

    Treating mo- tion as option to reduce motion dependency in unsupervised video object segmentation

    Suhwan Cho, Minhyeok Lee, Seunghoon Lee, Chaewon Park, Donghyeong Kim, and Sangyoun Lee. Treating mo- tion as option to reduce motion dependency in unsupervised video object segmentation. In Proceedings of the IEEE/CVF winter conference on applications of computer vision, page...

    2023

  4. [12]

    Dual pro- totype attention for unsupervised video object segmentation

    Suhwan Cho, Minhyeok Lee, Seunghoon Lee, Dogyoon Lee, Heeseung Choi, Ig-Jae Kim, and Sangyoun Lee. Dual pro- totype attention for unsupervised video object segmentation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 19238–19247, 2024. iii

    2024

  5. [13]

    3d u-net: learn- ing dense volumetric segmentation from sparse annota- tion

    ¨Ozg¨un C ¸ ic ¸ek, Ahmed Abdulkadir, Soeren S Lienkamp, Thomas Brox, and Olaf Ronneberger. 3d u-net: learn- ing dense volumetric segmentation from sparse annota- tion. In Medical Image Computing and Computer-Assisted Intervention–MICCAI 2016: 19th International Conference, At...

    2016

  6. [14]

    The pascal visual object classes (voc) challenge

    Mark Everingham, Luc Van Gool, Christopher KI Williams, John Winn, and Andrew Zisserman. The pascal visual object classes (voc) challenge. International journal of computer vision, 88:303–338, 2010. ii

    2010

  7. [15]

    Structure-measure: A new way to evaluate foreground maps

    Deng-Ping Fan, Ming-Ming Cheng, Yun Liu, Tao Li, and Ali Borji. Structure-measure: A new way to evaluate foreground maps. In Proceedings of the IEEE international conference on computer vision, pages 4548–4557, 2017. ii, v, vi

    2017

  8. [16]

    Shifting more attention to video salient object detection

    Deng-Ping Fan, Wenguan Wang, Ming-Ming Cheng, and Jianbing Shen. Shifting more attention to video salient object detection. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 8554–8564,

  9. [17]

    Bidirectionally learning dense spatio-temporal feature prop- agation network for unsupervised video object segmentation

    Jiaqing Fan, Tiankang Su, Kaihua Zhang, and Qingshan Liu. Bidirectionally learning dense spatio-temporal feature prop- agation network for unsupervised video object segmentation. In Proceedings of the 30th ACM International Conference on Multimedia, pages 3646–3655, 2022. v

    2022

  10. [18]

    Temporally efficient gabor transformer for unsu- pervised video object segmentation

    Jiaqing Fan, Tiankang Su, Kaihua Zhang, Bo Liu, and Qing- shan Liu. Temporally efficient gabor transformer for unsu- pervised video object segmentation. In Proceedings of the 31st ACM International Conference on Multimedia , pages 3394–3402, 2023. v

    2023

  11. [19]

    Pyramid constrained self- attention network for fast video salient object detection

    Yuchao Gu, Lijuan Wang, Ziqin Wang, Yun Liu, Ming- Ming Cheng, and Shao-Ping Lu. Pyramid constrained self- attention network for fast video salient object detection. In Proceedings of the AAAI conference on artificial intelli- gence, pages 10869–10876, 2020. v

    2020

  12. [20]

    Sparsectrl: Adding sparse controls to text-to-video diffusion models

    Yuwei Guo, Ceyuan Yang, Anyi Rao, Maneesh Agrawala, Dahua Lin, and Bo Dai. Sparsectrl: Adding sparse controls to text-to-video diffusion models. In European Conference on Computer Vision, pages 330–348. Springer, 2025. iii

    2025

  13. [21]

    Semantic contours from inverse detectors

    Bharath Hariharan, Pablo Arbel ´aez, Lubomir Bourdev, Subhransu Maji, and Jitendra Malik. Semantic contours from inverse detectors. In 2011 international conference on com- puter vision, pages 991–998. IEEE, 2011. ii

    2011

  14. [22]

    Classifier-free diffusion guidance

    Jonathan Ho and Tim Salimans. Classifier-free diffusion guidance. arXiv preprint arXiv:2207.12598, 2022. iv

    2022 arXiv

  15. [23]

    Denoising dif- fusion probabilistic models

    Jonathan Ho, Ajay Jain, and Pieter Abbeel. Denoising dif- fusion probabilistic models. Advances in neural information processing systems, 33:6840–6851, 2020. iii

    2020

  16. [24]

    Full-duplex strategy for video object segmentation

    Ge-Peng Ji, Keren Fu, Zhe Wu, Deng-Ping Fan, Jianbing Shen, and Ling Shao. Full-duplex strategy for video object segmentation. In Proceedings of the IEEE/CVF international conference on computer vision, pages 4922–4933, 2021. iii, v, vi

    2021

  17. [25]

    Casnet: A cross-attention siamese net- work for video salient object detection

    Yuzhu Ji, Haijun Zhang, Zequn Jie, Lin Ma, and QM Jonathan Wu. Casnet: A cross-attention siamese net- work for video salient object detection. IEEE transactions on neural networks and learning systems, 32(6):2676–2690,

  18. [26]

    Auto-encoding variational bayes

    Diederik P Kingma. Auto-encoding variational bayes. arXiv preprint arXiv:1312.6114, 2013. iii

    2013 arXiv

  19. [27]

    Adam: A method for stochastic optimization

    Diederik P Kingma and Jimmy Ba. Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 ,

    arXiv

  20. [28]

    Pika 1.0, 2023

    Pika Labs. Pika 1.0, 2023. https://pika.art/. iii

    2023

  21. [29]

    Unsupervised video object seg- mentation via prototype memory network

    Minhyeok Lee, Suhwan Cho, Seunghoon Lee, Chaewon Park, and Sangyoun Lee. Unsupervised video object seg- mentation via prototype memory network. In Proceedings of the IEEE/CVF winter conference on applications of com- puter vision, pages 5924–5934, 2023. iii

    2023

  22. [30]

    Guided slot attention for unsupervised video object segmentation

    Minhyeok Lee, Suhwan Cho, Dogyoon Lee, Chaewon Park, Jungho Lee, and Sangyoun Lee. Guided slot attention for unsupervised video object segmentation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 3807–3816, 2024. iii

    2024

  23. [31]

    Mo- tion guided attention for video salient object detection

    Haofeng Li, Guanqi Chen, Guanbin Li, and Yizhou Yu. Mo- tion guided attention for video salient object detection. In Proceedings of the IEEE/CVF international conference on computer vision, pages 7274–7283, 2019. iii

    2019

  24. [32]

    Movideo: Motion-aware video generation with diffusion model

    Jingyun Liang, Yuchen Fan, Kai Zhang, Radu Timofte, Luc Van Gool, and Rakesh Ranjan. Movideo: Motion-aware video generation with diffusion model. In European Con- ference on Computer Vision , pages 56–74. Springer, 2025. iii

    2025

  25. [33]

    Making a case for 3d convolutions for object segmentation in videos

    Sabarinath Mahadevan, Ali Athar, Aljo ˇsa O ˇsep, Sebastian Hennen, Laura Leal-Taix´e, and Bastian Leibe. Making a case for 3d convolutions for object segmentation in videos. arXiv preprint arXiv:2008.11516, 2020. ii, v

    2008 arXiv

  26. [34]

    Segmentation of moving objects by long term video analysis

    Peter Ochs, Jitendra Malik, and Thomas Brox. Segmentation of moving objects by long term video analysis. IEEE trans- actions on pattern analysis and machine intelligence, 36(6): 1187–1200, 2013. ii, v, vi, vii

    2013

  27. [35]

    Fast video object segmentation by reference- guided mask propagation

    Seoung Wug Oh, Joon-Young Lee, Kalyan Sunkavalli, and Seon Joo Kim. Fast video object segmentation by reference- guided mask propagation. In Proceedings of the IEEE con- ference on computer vision and pattern recognition , pages 7376–7385, 2018. ii

    2018

  28. [36]

    Video object segmentation using space-time memory networks

    Seoung Wug Oh, Joon-Young Lee, Ning Xu, and Seon Joo Kim. Video object segmentation using space-time memory networks. In Proceedings of the IEEE/CVF international conference on computer vision, pages 9226–9235, 2019. ii

    2019

  29. [37]

    Multi-scale interactive network for salient object detection

    Youwei Pang, Xiaoqi Zhao, Lihe Zhang, and Huchuan Lu. Multi-scale interactive network for salient object detection. In Proceedings of the IEEE/CVF conference on computer vi- sion and pattern recognition, pages 9413–9422, 2020. v

    2020

  30. [38]

    Hierarchical feature align- ment network for unsupervised video object segmentation

    Gensheng Pei, Fumin Shen, Yazhou Yao, Guo-Sen Xie, Zhenmin Tang, and Jinhui Tang. Hierarchical feature align- ment network for unsupervised video object segmentation. In European Conference on Computer Vision , pages 596–

  31. [39]

    A benchmark dataset and evaluation methodology for video object segmentation

    Federico Perazzi, Jordi Pont-Tuset, Brian McWilliams, Luc Van Gool, Markus Gross, and Alexander Sorkine-Hornung. A benchmark dataset and evaluation methodology for video object segmentation. In Proceedings of the IEEE conference on computer vision and pattern recognition, page...

  32. [40]

    Learning video object segmentation from static images

    Federico Perazzi, Anna Khoreva, Rodrigo Benenson, Bernt Schiele, and Alexander Sorkine-Hornung. Learning video object segmentation from static images. In Proceedings of the IEEE conference on computer vision and pattern recog- nition, pages 2663–2672, 2017. ii

    2017

  33. [41]

    High-resolution image synthesis with latent diffusion models

    Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Bj ¨orn Ommer. High-resolution image synthesis with latent diffusion models. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 10684–10695, 2022. iii

    2022

  34. [42]

    Gen-2, 2023

    runway. Gen-2, 2023. https://runwayml.com/ research/gen-2. iii

    2023

  35. [43]

    Kernelized memory network for video object segmentation

    Hongje Seong, Junhyuk Hyun, and Euntai Kim. Kernelized memory network for video object segmentation. In Com- puter Vision–ECCV 2020: 16th European Conference, Glas- gow, UK, August 23–28, 2020, Proceedings, Part XXII 16 , pages 629–645. Springer, 2020. ii

    2020

  36. [44]

    Hierarchical mem- ory matching network for video object segmentation

    Hongje Seong, Seoung Wug Oh, Joon-Young Lee, Seong- won Lee, Suhyeon Lee, and Euntai Kim. Hierarchical mem- ory matching network for video object segmentation. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 12889–12898, 2021. ii

    2021

  37. [45]

    Hierarchical image saliency detection on extended cssd

    Jianping Shi, Qiong Yan, Li Xu, and Jiaya Jia. Hierarchical image saliency detection on extended cssd. IEEE transac- tions on pattern analysis and machine intelligence , 38(4): 717–729, 2015. ii

    2015

  38. [46]

    Denoising diffusion implicit models

    Jiaming Song, Chenlin Meng, and Stefano Ermon. Denoising diffusion implicit models. arXiv preprint arXiv:2010.02502, 2020. iii

    2010 arXiv

  39. [47]

    Improved techniques for training score-based generative models

    Yang Song and Stefano Ermon. Improved techniques for training score-based generative models. Advances in neural information processing systems, 33:12438–12448, 2020. iv

    2020

  40. [48]

    Unsupervised video object segmentation with online adversarial self-tuning

    Tiankang Su, Huihui Song, Dong Liu, Bo Liu, and Qingshan Liu. Unsupervised video object segmentation with online adversarial self-tuning. In Proceedings of the IEEE/CVF In- ternational Conference on Computer Vision, pages 688–698,

  41. [49]

    A unified transformer frame- work for group-based segmentation: Co-segmentation, co- saliency detection and video salient object detection

    Yukun Su, Jingliang Deng, Ruizhou Sun, Guosheng Lin, Hanjing Su, and Qingyao Wu. A unified transformer frame- work for group-based segmentation: Co-segmentation, co- saliency detection and video salient object detection. IEEE Transactions on Multimedia, 26:313–325, 2023. v

    2023

  42. [50]

    Raft: Recurrent all-pairs field transforms for optical flow

    Zachary Teed and Jia Deng. Raft: Recurrent all-pairs field transforms for optical flow. In Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23– 28, 2020, Proceedings, Part II 16, pages 402–419. Springer,

    2020

  43. [51]

    Learning to de- tect salient objects with image-level supervision

    Lijun Wang, Huchuan Lu, Yifan Wang, Mengyang Feng, Dong Wang, Baocai Yin, and Xiang Ruan. Learning to de- tect salient objects with image-level supervision. InProceed- ings of the IEEE conference on computer vision and pattern recognition, pages 136–145, 2017. iv

    2017

  44. [52]

    Consistent video saliency using local gradient flow optimization and global refinement

    Wenguan Wang, Jianbing Shen, and Ling Shao. Consistent video saliency using local gradient flow optimization and global refinement. IEEE Transactions on Image Processing, 24(11):4185–4196, 2015. ii, v, vi, vii

    2015

  45. [53]

    Video salient object detection via fully convolutional networks

    Wenguan Wang, Jianbing Shen, and Ling Shao. Video salient object detection via fully convolutional networks. IEEE Transactions on Image Processing, 27(1):38–49, 2017. ii

    2017

  46. [54]

    F3net: fusion, feedback and focus for salient object detection

    Jun Wei, Shuhui Wang, and Qingming Huang. F3net: fusion, feedback and focus for salient object detection. In Proceed- ings of the AAAI conference on artificial intelligence , pages 12321–12328, 2020. v

    2020

  47. [55]

    Cbam: Convolutional block attention module

    Sanghyun Woo, Jongchan Park, Joon-Young Lee, and In So Kweon. Cbam: Convolutional block attention module. In Proceedings of the European conference on computer vision (ECCV), pages 3–19, 2018. iv

    2018

  48. [56]

    Segformer: Simple and efficient design for semantic segmentation with transform- ers

    Enze Xie, Wenhai Wang, Zhiding Yu, Anima Anandkumar, Jose M Alvarez, and Ping Luo. Segformer: Simple and efficient design for semantic segmentation with transform- ers. Advances in neural information processing systems, 34: 12077–12090, 2021. iv, vii

    2021

  49. [57]

    Learning motion-appearance co- attention for zero-shot video object segmentation

    Shu Yang, Lu Zhang, Jinqing Qi, Huchuan Lu, Shuo Wang, and Xiaoxing Zhang. Learning motion-appearance co- attention for zero-shot video object segmentation. In Pro- ceedings of the IEEE/CVF international conference on com- puter vision, pages 1564–1573, 2021. iii

    2021

  50. [58]

    Anchor diffusion for un- supervised video object segmentation

    Zhao Yang, Qiang Wang, Luca Bertinetto, Weiming Hu, Song Bai, and Philip HS Torr. Anchor diffusion for un- supervised video object segmentation. In Proceedings of the IEEE/CVF international conference on computer vision, pages 931–940, 2019. v

    2019

  51. [59]

    Dragnuwa: Fine-grained control in video generation by integrating text, image, and trajectory

    Shengming Yin, Chenfei Wu, Jian Liang, Jie Shi, Houqiang Li, Gong Ming, and Nan Duan. Dragnuwa: Fine-grained control in video generation by integrating text, image, and trajectory. arXiv preprint arXiv:2308.08089, 2023. iii

    2023 arXiv

  52. [60]

    I2vgen-xl: High-quality image-to-video synthesis via cascaded diffusion models

    Shiwei Zhang, Jiayu Wang, Yingya Zhang, Kang Zhao, Hangjie Yuan, Zhiwu Qin, Xiang Wang, Deli Zhao, and Jingren Zhou. I2vgen-xl: High-quality image-to-video synthesis via cascaded diffusion models. arXiv preprint arXiv:2311.04145, 2023. iii

    2023 arXiv

  53. [61]

    Suppress and balance: A simple gated net- work for salient object detection

    Xiaoqi Zhao, Youwei Pang, Lihe Zhang, Huchuan Lu, and Lei Zhang. Suppress and balance: A simple gated net- work for salient object detection. InComputer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23– 28, 2020, Proceedings, Part II 16 , pages 35–51. Springer,

    2020

  54. [62]

    Learning discriminative feature with crf for unsupervised video object segmentation

    Mingmin Zhen, Shiwei Li, Lei Zhou, Jiaxiang Shang, Haoan Feng, Tian Fang, and Long Quan. Learning discriminative feature with crf for unsupervised video object segmentation. In Computer Vision–ECCV 2020: 16th European Confer- ence, Glasgow, UK, August 23–28, 2020, Proceedings,...

    2020

  55. [63]

    Motion-attentive transition for zero-shot video object segmentation

    Tianfei Zhou, Shunzhou Wang, Yi Zhou, Yazhou Yao, Jianwu Li, and Ling Shao. Motion-attentive transition for zero-shot video object segmentation. In Proceedings of the AAAI conference on artificial intelligence , pages 13066– 13073, 2020. iii

    2020

Pith tools

Reviewed August 12, 2026 · model on record in the stance chip above.