{"id":"4a07b14c-4790-4abf-af98-f25c3ce81687","arxiv_id":"2411.13975","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":4,"one_line_summary":"Diffusion-generated image-to-video transformations produce realistic optical flows that, when used as training data, push video salient object detection to new state-of-the-art results on DAVIS, FBMS, DAVSOD, and ViSal.","lead":"The authors train a video salient object detection model on synthetic training pairs made by feeding static images into an image-to-video diffusion model and estimating optical flow between source and generated frames. This augmentation reaches state-of-the-art scores on four standard benchmarks, suggesting generated flows can substitute for real video motion in training.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"No controlled warped-data baseline, so the central claim that diffusion-simulated flows are uniquely effective for VSOD training is not yet established.","rationale":"The central claim is causal: SVD-generated flows are more realistic than warping-based flows, and this is why RealFlow achieves SOTA. This claim has two parts: (a) the synthetic flows are more realistic, and (b) that realism is what drives downstream gains. Part (a) is supported only by qualitative figures; the Limitation explicitly notes artifacts. Part (b) has no direct test because the Simulated condition differs from any plausible warped baseline in data source, volume, and generation mechanism. A warped control is the minimal experiment that would separate 'more training data from images' from 'better motion cues from diffusion.' The paper is otherwise strong: same-network ablations show consistent trends, the method is reproducible in principle, and the authors are candid about artifacts. But as written, the stated motivation—diffusion beats basic image-to-video simulation—is asserted, not demonstrated. This is a correctness risk in the central claim, not a stylistic issue. My recommendation remains CONDITIONAL because the empirical results are promising and the missing control is straightforward to run; if the control shows no advantage, the claim would need substantial revision.","tokens_in":12895,"tokens_out":4539,"duration_ms":45958,"concrete_test":"Retrain the same two-stream VSOD network under four conditions: (1) real DAVIS/DAVSOD data, (2) SVD+RAFT simulated data from DUTS, (3) spatially warped simulated data from the same DUTS images using random affine and thin-plate-spline deformations with RAFT flows, and (4) mixed real + warped, keeping the same 2:1:1 ratio and all hyperparameters. Run each condition with at least three seeds and report mean/std S-measure on DAVIS2016, FBMS, DAVSOD, and ViSal. If conditions (3) or (4) are statistically indistinguishable from (2) or Mixed, the paper's central causal claim about diffusion realism fails.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The paper claims that image-to-video diffusion generates 'realistic' optical flows that improve VSOD training, unlike affine/spline warping. The only training-protocol ablation (Table 2) compares Real, Simulated, and Mixed data, but never a Warped condition: applying thin-plate-spline or affine deformations to the same DUTS source images, estimating flows with the same RAFT model, and training the same two-stream network. Without this control, the Simulated advantage over Real is confounded by training-set size and diversity: synthetic generation from 15.5k DUTS images yields far more samples than the DAVIS/DAVSOD training videos. The paper's own Limitation (Section 5) admits checkerboard flow artifacts and non-continuous motion, which weakens the 'realistic flow' premise; Section 4.5 shows qualitative examples only. If a simple warped augmentation matches RealFlow's scores, the core contribution reduces to generic data augmentation rather than diffusion-based motion realism, and the SOTA numbers lose their causal interpretation.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes a data-simulation strategy for video salient object detection (VSOD): starting from static images in the DUTS dataset, it uses Stable Video Diffusion to generate a short video from each image, estimates optical flows between the source image and each generated target frame with RAFT, and trains a two-stream encoder-decoder network (RGB plus flow) on a mixture of these synthetic image-flow pairs and real video data from DAVIS2016 and DAVSOD. The method, named RealFlow, is reported to achieve the highest S-measure on DAVIS2016 (94.5%), FBMS (92.6%), DAVSOD (80.3%), and ViSal (96.2%), and top F-measure on three of the four benchmarks. The paper also includes ablations on training protocol, backbone size, and input resolution, along with qualitative comparisons of simulated and real flows.","tokens_in":13053,"tokens_out":4354,"duration_ms":39386,"significance":"If the central claim is correct, the paper would demonstrate a practical way to leverage large-scale image datasets for motion-guided video tasks without costly video annotations, using off-the-shelf image-to-video diffusion and optical flow estimation. The proposed pipeline is simple and reproducible, and the authors commit to releasing code and models. The main experiments use standard benchmarks and metrics, and the ablation in Table 2 isolates the effect of simulated versus real training data to some degree. However, the causal interpretation that diffusion-based flow realism, rather than sheer training-data volume and diversity, drives the improvement is not yet supported, because the paper omits the obvious controlled baseline of training on flows from traditional warping of the same static images. The reported gains over prior state-of-the-art methods are small in several cases, and no variance estimates are provided, so the strength of the empirical claim is uncertain despite the strong headline numbers.","major_comments":[{"comment":"The central claim of the paper, stated in the Abstract and Introduction, is that diffusion-based image-to-video generation produces 'realistic' or 'plausible' optical flows that improve VSOD training, unlike affine or spline warping. The ablation in Table 2 only compares training on real video, simulated diffusion data, and a mixture of the two. It never includes a 'Warped' condition in which the same DUTS source images are deformed with thin-plate spline or affine transformations, flows are estimated with the same RAFT model, and the same two-stream network is trained. Without this control, the advantage of Simulated over Real is confounded by training-set size and diversity: the DUTS-based synthetic set yields many more training pairs (about 15,572 source images, each producing multiple flow pairs) than the roughly 91 real training videos in DAVIS2016 and DAVSOD. I request that the authors add this warped-data baseline and report the same metrics. If the warped baseline matches or exceeds the diffusion-based result, the contribution reduces to generic data augmentation rather than diffusion-specific motion realism, and the state-of-the-art numbers lose their current causal interpretation.","section":"Section 3.2, Table 2, and Section 5"},{"comment":"All results are reported as single-run numbers without error bars or multiple seeds. Differences with prior work are often small: on DAVIS2016 the S-measure is 94.5 vs. 93.5 for OAST, on DAVSOD it is 80.3 vs. 79.8 for TGFormer, and on ViSal it is 96.2 vs. 95.2 for TGFormer. These margins may be within run-to-run variation for a randomly initialized network, especially with a training process that involves stochastic data shuffling and diffusion sampling. The authors should report mean and standard deviation over at least three independent training runs for the main comparison (Table 1) and for the key ablation in Table 2, or otherwise demonstrate that the observed differences are reproducible and significant.","section":"Table 1 and Section 4.3"},{"comment":"The mixture ratio of simulated, DAVIS2016, and DAVSOD data is set to 2:1:1 with no sensitivity analysis and no justification. This ratio is load-bearing because it determines how much synthetic data is used relative to real data, and the comparison in Table 2 between 'Simulated' and 'Real' does not control for total sample count. I recommend sweeping the mixture ratio (e.g., 1:1:1, 3:1:1, and the current 2:1:1) and reporting the resulting scores. Additionally, the text in Section 3.2 is internally inconsistent about the total dataset size: it first says 'from N source images, we can obtain N*T paired data samples' and then states that a total of 15,572 data pairs is obtained from 15,572 DUTS images, which would be the case only if T=1. Please clarify the exact number of training pairs used.","section":"Section 3.3, 'Dataset construction' in Section 3.2"},{"comment":"The paper's own limitation statement concedes that generated frames 'often lack continuous motion' and that flow maps 'frequently exhibit checkerboard artifacts.' This directly weakens the premise, stated in the Introduction and Figure 1, that the simulated flows are realistic and capture independent object motion. The qualitative evidence in Figures 5 and 6 is not sufficient to establish that the simulated flow distribution is closer to real video flows than flows from warped images. The authors should either provide quantitative evidence that the simulated flows resemble real flows (for example, statistics of flow magnitude, distribution of flow boundaries, or a learned discriminator's accuracy) or demonstrate empirically that the checkerboard artifacts and non-continuous motion do not harm training, for instance by ablating post-processing of the flows. Without this, the mechanism behind the performance gain remains unclear.","section":"Section 5 (Limitation) and Section 4.5"}],"minor_comments":[{"comment":"The notation 'P T 1' and 'Q T 1' is typeset awkwardly; it should be P_1^T, and the subscripts/superscripts need to be cleaned up.","section":"Section 3.2, Eq. (2) and Eq. (3)"},{"comment":"The figures contain the watermark '© 2023 Adobe. All Rights Reserved. Adobe Confidential.' This is not part of a normal academic manuscript and should be removed; its presence suggests the figures were taken from an internal Adobe document or the manuscript template was not properly cleaned.","section":"Throughout, figure captions"},{"comment":"The Abstract claims 'state-of-the-art performance across all public benchmark datasets' without qualification, but Section 4.3 correctly notes that the F-measure is top on three of the four benchmarks; on FBMS, RealFlow's F-measure (90.6) is below OAST and TGFormer (91.9). Please align the Abstract's wording with the more precise statement in Section 4.3.","section":"Abstract and Section 4.3"},{"comment":"The dimensionality notation 'x ∈ RC×T HW' is malformed; it should be R^{C×T×H×W}, and 'repeat(zs, T)' should specify the repetition over the temporal dimension.","section":"Section 3.1, Eq. (1)"},{"comment":"The text cites 'deterministic DDIM sampler [47]' but reference [47] is Song and Ermon, 'Improved techniques for training score-based generative models.' The DDIM sampler is from Song et al., 'Denoising Diffusion Implicit Models' (2020). The citation should be corrected.","section":"References, [47]"}],"recommendation":"major_revision","confidential_remarks":"The presence of '© 2023 Adobe. All Rights Reserved. Adobe Confidential.' watermarks on the figures is a provenance red flag. The authors are from Yonsei University, not Adobe, and the watermark is unexplained; the editor may wish to ask the authors to clarify the origin of the figures and ensure they have the right to use them. This does not affect the technical assessment but should be resolved before publication."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Worth a look, and worth sending out for review, but the paper's headline claim outruns its evidence. The new thing here is concrete: instead of affine/spline warping to hallucinate video from static images, they use Stable Video Diffusion to generate target frames, run RAFT from the source to each generated frame, and train a bog-standard two-stream VSOD network on the resulting image-flow pairs plus DUTS masks. The results on four benchmarks are genuinely strong, and the cleanest ablation—simulated-only versus real-only training—shows the synthetic data carries most of the weight on three of four datasets. The limitations section is unusually honest about checkerboard flow artifacts and unrealistic motion dynamics.\n\nThe soft spot is that the paper never runs the comparison that its own abstract sets up. It says spatial warping 'fails to produce realistic optical flows' and that diffusion instead generates plausible ones, but there is no training protocol with warped DUTS images and the same RAFT flow estimation. The Simulated condition also has vastly more samples than the Real condition, so the 'simulated beats real' result is confounded by data volume. A reviewer should ask for a Warped condition matched for sample count, and ideally an equal-volume Real condition, before accepting the causal story. The S-measure improvements are real, but their interpretation as evidence for diffusion-specific realism is not yet established. The mixture ratio 2:1:1 and SVD sampling settings are also essentially untuned free parameters, and there are no error bars or multi-seed runs—minor in this field but worth noting. One editorial slip: the text says 15,572 data pairs from 15,572 DUTS images, when with T=14 frames it should be far more; presumably a typo.\n\nIf the missing baseline comes back showing warped flows do just as well, the paper still has a useful recipe: a new way to synthesize training pairs for dense video prediction. If it comes back showing diffusion helps, it's a stronger result. Either way, this deserves a serious referee. The writing is clear, the method is reproducible in principle (code link provided), and the honest limitations section is a good sign. My recommendation: send to peer review, with the missing controlled baseline as the major revision.","headline":"A simple, reproducible recipe for VSOD training data, with a real but fixable missing-baseline gap in the central claim.","tokens_in":13626,"tokens_out":2996,"would_cite":true,"duration_ms":28671,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Training on image-flow pairs synthesized by a video diffusion model plus a flow estimator achieves state-of-the-art video salient object detection on four benchmarks.","keywords":["video salient object detection","image-to-video generation","optical flow","diffusion models","synthetic data augmentation","two-stream network","Stable Video Diffusion","RAFT"],"falsifier":"Train the same network on the same number of simulated pairs built with traditional spatial warps, such as affine or spline transformations, instead of diffusion-generated frames; if that model matches or beats the diffusion-based one on the four benchmarks, the claim that diffusion-based generation is the source of the improvement is falsified.","tokens_in":12648,"feed_emoji":"🎬","tokens_out":5562,"duration_ms":52184,"temperature":0.7,"pith_summary":"The paper aims to show that image-to-video diffusion models can replace hand-crafted spatial warping as a way to synthesize training data for video salient object detection (VSOD), the task of locating and segmenting the most attention-grabbing object in every frame. The recipe starts from a static image, lets a diffusion model generate several plausible successor frames, and uses a pre-trained optical flow estimator to produce motion maps between the source and each generated frame; these synthetic image-flow pairs are then mixed with real video clips to train a simple two-stream network. On four standard VSOD benchmarks the trained model reaches the highest reported structure-measure scores. The wider promise is that realistic, object-aware motion cues can be produced from abundant static images, reducing the field's dependence on costly video annotations.","feed_headline":"Diffusion-generated image-flow pairs push video saliency to new records","feed_subtitle":"A simple two-stream network trained on synthetic image-flow pairs beats prior models on four video saliency benchmarks.","key_machinery":"The load-bearing machinery is a data-creation pipeline: a source image is fed to Stable Video Diffusion, which denoises latents conditioned on the source to produce 14 generated frames; a pre-trained RAFT model then estimates optical flow from the source to each generated frame, yielding paired samples with spatially aligned masks and flows. Because the diffusion model generates whole frames rather than warping pixels, the flows can separate foreground and background motion. The segmentation network is a two-stream encoder with MiT-b2 backbones for appearance and motion, lightweight attention fusion, and a progressive decoder; training mixes simulated data with real video data at a 2:1:1 ratio. This setup turns 10,553 DUTS training images plus 5,019 test images into 15,572 synthetic image-flow pairs.","core_discovery":"The central claim is that the bottleneck for flow-guided VSOD is not the segmentation architecture but the quality and quantity of motion cues in training data. Conventional image-to-video simulation via affine warps or splines produces optical flows that do not reflect each object's independent movement, because the warps move all pixels according to one global deformation. The paper claims that Stable Video Diffusion, an image-to-video diffusion model, transforms a static source image into several novel frames while preserving object-level context, so that optical flow maps estimated by RAFT between the source and these generated frames carry plausible, object-specific motion. Paired with DUTS images under a 2:1:1 mixture of simulated data, DAVIS 2016 training, and DAVSOD training, this data supplies enough realistic motion cues that a simple MiT-b2 two-stream network reaches new state-of-the-art S-measure results: 94.5% on DAVIS 2016, 92.6% on FBMS, 80.3% on DAVSOD, and 96.2% on ViSal.","pith_inferences":["Extension: the paper leaves implicit a test that would isolate whether the gain comes from diffusion-generated realism or simply from having 15,572 extra training pairs; ablating with the same volume of warped-image pairs would settle it.","Extension: the simulated flows could be evaluated directly—for example by comparing their statistics or warping error against real video flows—to give a quantitative proxy for transfer before training a network.","Extension: a natural follow-up is to chain generated frames, using frame t as the source for frame t+1, to produce longer-range motion and reduce the current reliance on a single static anchor frame.","Extension: the reported checkerboard artifacts in simulated flows suggest a filtering or augmentation strategy that treats these artifacts as hard negatives, which the paper does not explore."],"forward_implications":["Simulated image-flow pairs alone outperform training on real video data on FBMS, DAVSOD, and ViSal, and the mixed training set outperforms both on every benchmark.","Because each source image yields multiple paired samples, the synthetic dataset scales linearly with the number of static images, so large image collections can be converted into motion-annotated training data.","The gains are obtained with a simple two-stream backbone, suggesting that data quality can substitute for architectural complexity in flow-guided saliency.","The same recipe should transfer to other two-stream video tasks that rely on optical flow as input, since it produces paired RGB-flow samples without manual video annotation.","The paper's own limitation analysis implies that improving the generator's temporal consistency would further improve the simulated flows and downstream accuracy."],"supporting_citations":[{"why":"Supplies the image-to-video diffusion model that generates target frames from static source images.","marker":"[3]"},{"why":"Supplies the pre-trained optical flow estimator used to create the simulated flow maps between source and generated frames.","marker":"[50]"},{"why":"Supplies the DUTS static image collection from which the 15,572 synthetic image-flow pairs are built.","marker":"[51]"},{"why":"Supplies real training videos and the validation benchmark for DAVIS 2016.","marker":"[39]"},{"why":"Supplies real training videos and the test benchmark for DAVSOD.","marker":"[16]"},{"why":"Provides the FBMS test benchmark used for evaluation.","marker":"[34]"},{"why":"Provides the ViSal evaluation dataset.","marker":"[52]"}],"fun_headline_variants":["Diffusion-generated flows set new video saliency records","Synthetic flows from diffusion models boost video saliency","Object-aware flows from diffusion improve saliency detection","Diffusion transforms images into motion-rich video saliency data"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The entire transfer depends on the unstated assumption that RAFT optical flows computed between a real static image and a diffusion-generated frame are statistically close enough to real video flows that a network trained on them generalizes to real videos.","fun_headline_variants_meta":{"raw":{"variants":["Diffusion-generated flows set new video saliency records","Synthetic flows from diffusion models boost video saliency","Object-aware flows from diffusion improve saliency detection","Diffusion transforms images into motion-rich video saliency data"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000421,"raw_usage":{"total_tokens":2160,"prompt_tokens":934,"completion_tokens":1226,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":550,"completion_tokens_details":{"reasoning_tokens":1163}},"tokens_in":550,"tokens_out":1226,"duration_ms":9553,"temperature":1.0,"reasoning_tokens":1163,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-12T15:40:40.769461+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Train the same network on the same number of simulated pairs built with traditional spatial warps, such as affine or spline transformations, instead of diffusion-generated frames; if that model matches or beats the diffusion-based one on the four benchmarks, the claim that diffusion-based generation is the source of the improvement is falsified.","supporting_citations":[{"cited_title":"Learning to de- tect salient objects with image-level supervision","cited_arxiv_id":null,"evidence_quote":"Supplies the DUTS static image collection from which the 15,572 synthetic image-flow pairs are built."},{"cited_title":"Segmentation of moving objects by long term video analysis","cited_arxiv_id":null,"evidence_quote":"Provides the FBMS test benchmark used for evaluation."},{"cited_title":"Consistent video saliency using local gradient flow optimization and global refinement","cited_arxiv_id":null,"evidence_quote":"Provides the ViSal evaluation dataset."}],"review_version":1}