{"id":"277afe90-6f72-4eb0-abb4-9e28ed74c9b8","arxiv_id":"2412.08259","paper_version":2,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":3,"one_line_summary":"VSD2M, a 2.09 million sample bilingual sticker dataset with animated GIFs, plus a Spatial Temporal Interaction layer, improves animated sticker generation over standard video diffusion baselines.","lead":"The authors built a two-million sample dataset of social media stickers, including animated GIF stickers with action-aware text descriptions, and tested video generation models on creating new animated stickers. They also added a lightweight spatial temporal interaction layer that improved generated sticker quality in their comparisons.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Test-set contamination risk: VSD-R/VSD-C are selected from the same crawled pool with no reported deduplication, so Table 4 gains may reflect leakage rather than the STI layer.","rationale":"The reader's weakest assumption is that the single-run metric differences in Table 4 are reliable, supported by the absence of confidence intervals or significance tests. That is a valid measurement concern. However, I see a more fundamental premise that the reader did not explicitly flag: the integrity of the test set itself. The paper does not state that VSD-R and VSD-C are disjoint from the training pool, nor does it describe any deduplication step. Since the test sets are drawn from the same crawled collection using the same trigger words, the possibility of exact or near-duplicate leakage is concrete. If leakage exists, even a perfectly repeated multi-seed experiment would only quantify noise around a biased estimate; the entire benchmark would need re-evaluation. This concern is testable without requiring the authors to retrain models: a simple overlap analysis settles it. The dataset scale claim (2.09M vs 820K) remains plausible regardless, so I do not move to REJECT. The verdict stays CONDITIONAL, but with the additional condition that the authors release a deduplication analysis and exclude any leaked test samples. I therefore set verdict_should_be to UNCHANGED relative to the reader's conditional verdict, while noting that the justification differs: the decisive test is a data-integrity check, not merely adding error bars. This is an honest non-adversarial reading: the concern is about missing verification, not about an observed flaw, and it is fully addressable by the authors.","tokens_in":13685,"tokens_out":7648,"duration_ms":84969,"concrete_test":"Compute the overlap between the VSD-R/VSD-C test sets and the training split, both exact duplicates (e.g., MD5 hashes of GIF/JPEG bytes) and near-duplicates (e.g., perceptual hash distance below a fixed threshold, or CLIP embedding cosine similarity > 0.95). Report the overlap rate and re-run the Table 4 comparisons after excluding all overlapping test samples. If the overlap rate exceeds 1%, or if removing overlaps shifts the Ours-vs-I2VGen-XL FVD gap on VSD-R by more than 10 FVD points, the reported benchmark numbers must be recomputed and the qualitative conclusions revisited.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The paper's central claims — that VSD2M is a comprehensive benchmark and that the STI layer improves animated sticker generation — rest on the reliability of the quantitative comparisons in Table 4. A premise that is not explicitly secured is that the VSD-R and VSD-C test sets are cleanly held out from the training data. In the Data Construction section, the authors crawl 2.5M samples using 600K trigger words, filter to 2.1M, and then 'manually select from multiple pieces of samples corresponding to each top 500 trigger words in real and cartoon domain as test sets.' No train/test split is described before annotation, and no exact or near-duplicate removal (e.g., perceptual hashing, CLIP-based dedup) is reported anywhere in the paper. If any test GIFs are exact or near-duplicates of training GIFs — a common issue in web-scale crawled collections — the FVD metric, which measures distribution distance, would be artificially low, and VQA would be inflated for all methods. This is load-bearing because the reported advantage over I2VGen-XL on VSD-R is only 16 FVD points (2613.41 vs 2623.47) and 0.006 VQA points; a small overlap rate could erase or reverse these margins. Even the larger margin over VideoLDM on VSD-C (FVD 5513.64 vs 5822.87) could be partly attributable to leakage of near-duplicate training samples. The absence of a deduplication description makes this a concrete, testable risk rather than a speculative one.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"This paper introduces VSD2M, a vision-language dataset of about 2.09M static and animated stickers crawled from the web, annotated with English and Chinese captions, trigger words, OCR text, and a real/cartoon flag. The authors propose a Spatial Temporal Interaction (STI) layer that splits temporal modeling into a semantic-interaction branch (downsampled self-attention across frames) and a detail-preservation branch (frame convolution), and they insert it into VideoLDM. They compare VideoGPT, Make-A-Video, VideoLDM, VideoFactory, I2VGen-XL, and 'Ours' on two test sets (VSD-R and VSD-C) using FVD, CLIP similarity, and VQA, plus user studies. The paper claims that VSD2M is the largest multimodal sticker dataset to date and that the STI layer improves animated sticker generation.","tokens_in":14015,"tokens_out":4485,"duration_ms":46177,"significance":"If the quantitative results are reliable, the dataset is a useful community resource: it is roughly three times the size of Sticker820K, includes animated stickers (GIFs) with action-oriented captions, and provides a first systematic comparison of video-generation architectures on the animated sticker generation task. The STI layer is a simple, plausible modification for discrete low-frame-rate data, and the paper's user studies cover dimensions such as temporal consistency, visual quality, and user favorability beyond the automatic metrics. The strengths are the scope of the data-collection pipeline and the breadth of baselines, not machine-checked proofs or parameter-free derivations. However, the method contribution is not yet supported at the reported level of evidence: the key Table 4 margins are small, no confidence intervals or multiple seeds are reported, the test sets are drawn from the same crawl with no deduplication described, and the STI components are not ablated.","major_comments":[{"comment":"The test sets VSD-R and VSD-C are manually selected from samples corresponding to the top 500 trigger words in real and cartoon domains, drawn from the same 2.1M-sample crawled pool used for training, and no exact or near-duplicate removal (e.g., perceptual hashing or CLIP-based deduplication) is reported before or after the split. Because FVD measures distribution distance and VQA measures quality, any exact or near-duplicate overlap between the test GIFs and training GIFs would inflate all methods' scores; the reported margins over I2VGen-XL on VSD-R are only 16 FVD points (2613.41 vs 2623.47) and 0.006 VQA points. The paper should report the deduplication procedure and quantify the train/test overlap; without this, the benchmark numbers in Table 4 are not a clean test of the STI layer.","section":"Data Construction"},{"comment":"The central claim that 'Ours' is best rests on single-run metric values with no confidence intervals, multiple seeds, or significance tests. For example, on VSD-R, Ours (0.487 VQA, 2613.41 FVD) is separated from I2VGen-XL (0.481, 2623.47) and VideoFactory (0.480, 2638.97) by margins that are likely within run-to-run noise for FVD/VQA. Please provide error bars over at least three seeds and a paired significance test. In addition, the STI layer is only evaluated as a full replacement of VideoLDM's temporal layers; there is no ablation isolating the semantic-interaction branch, the detail-preservation branch, or the learnable balancing weights, so the attribution of the gains to the proposed components is not established.","section":"Experimental Setups / Table 4"},{"comment":"The description of the STI layer leaves the downsampling factor gamma, the frame convolution kernel size k, and the initialization or learning of the balancing weights unspecified, and the experiments never state the values used. Without these numbers, the proposed module cannot be reproduced or compared across backbones; please report the actual settings and, ideally, a sensitivity study over gamma and k. This is not merely a presentation issue because the paper motivates the design by 'appropriate gamma values' and 'kernel size k,' yet no configuration is given in the main text or the Appendix.","section":"STI Layer for Discrete Sticker Generation"}],"minor_comments":[{"comment":"There are several typographical errors, including 'the ASG filed' in the Introduction, 'layerll' before the Qualitative Comparison subsection, and 'an stronger sense of motion' in the Qualitative Comparison subsection; the manuscript needs proofreading.","section":"Throughout"},{"comment":"The number of labelers is inconsistent: the main text says 'nine labelers' while Appendix A.2 says 'ten people'; please report the exact protocol, the number of participants, and any inter-annotator agreement measure.","section":"User Preference / Appendix A.2"},{"comment":"The dataset and code are promised 'at link' with a placeholder URL; this should be replaced with a working link and the license or terms of the crawled data should be stated.","section":"Abstract and Introduction"},{"comment":"The row 'Ave Frames Numbers' reports 16.91, but the surrounding text and Figure 4(b) note that only multi-frame animated stickers are counted; please make the counting convention explicit in the table caption.","section":"Table 3"},{"comment":"Training hyperparameters such as learning rate, batch size, number of training steps, and optimizer are not reported, which limits the reproducibility of the baselines and of 'Ours'.","section":"Experimental Setups"}],"recommendation":"major_revision","confidential_remarks":"The paper's main value is the dataset plus a method; the dataset is potentially valuable, but the method claim needs stronger evidence. The small margins in Table 4 and the absence of deduplication are the main risks, and both are addressable in revision. I would encourage the editor to treat the promised dataset and code release as a condition of acceptance, especially since the current text only provides a placeholder link."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"The real contribution here is the dataset. VSD2M is the first large-scale vision-language sticker dataset to include animated GIFs, with about two million samples, action-aware captions, OCR, domain labels, and bilingual text. That fills a genuine gap: SER30K and Sticker820K are static-only and smaller, and TGIF is a GIF description dataset, not a generation benchmark. The paper also trains a set of standard video generation baselines on VSD2M and reports systematic comparisons, which is useful groundwork for a niche but active application area.\n\nThe STI layer is a reasonable but incremental idea: downsampled cross-frame self-attention for semantic interaction plus k×1×1 convolutions for detail preservation, inserted into VideoLDM's temporal layers. The evidence for it, however, is thin. Table 4 shows the best VQA and FVD numbers, but the margins over I2VGen-XL are tiny (0.006 VQA, 10–16 FVD points), no confidence intervals or multiple seeds are reported, and there are no ablations isolating the two branches. The claim that semantic interaction improves FVD while detail preservation improves VQA is asserted without experimental support.\n\nThe stress-test worry about test-set contamination is legitimate. VSD-R and VSD-C are selected from the same crawled pool as the training data, and no deduplication is described. Given the small margins, even modest overlap between training and test GIFs could flip the reported ranking. The authors need to either describe a clean held-out split or report near-duplicate removal (perceptual hashing, CLIP-based dedup) before those numbers can be trusted.\n\nOther soft spots: the STI layer is only tested inside VideoLDM, so transferability is unknown; the dataset and code are promised but not yet available; test set sizes and selection details are missing. The appendix does acknowledge some honest limitations about data collection and caption difficulty, which is to the authors' credit.\n\nOn balance, the dataset alone is worth refereeing. I would send this to peer review with instructions to demand dedup details, error bars, ablations, and artifact release. The method claim may not survive, but the benchmark contribution is likely to be useful to the community.","headline":"VSD2M is a genuinely useful large-scale animated sticker dataset, but the STI layer's performance claims rest on thin, single-run evidence and a test split that may overlap training data.","tokens_in":14555,"tokens_out":3916,"would_cite":false,"duration_ms":37794,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"The paper introduces VSD2M, a two-million-sample dataset of static stickers and animated GIFs with action-aware captions, and a spatial-temporal interaction layer that improves animated sticker generation over standard video diffusion…","keywords":["sticker dataset","animated sticker generation","vision-language benchmark","GIF generation","spatial temporal interaction","diffusion models","text-to-video"],"falsifier":"Run the same 'ours' and I2VGen-XL training setups at least five seeds each and compute the mean and standard deviation of VQA and FVD on VSD-R and VSD-C; if the difference between the two methods consistently lies within one standard deviation, the claimed benefit of the STI layer is not supported. Alternatively, an independent re-evaluation of the released checkpoints with a third-party FVD implementation that yields overlapping scores would falsify the reported margin.","tokens_in":13497,"feed_emoji":"🎨","tokens_out":4853,"duration_ms":43128,"temperature":0.7,"pith_summary":"The paper introduces VSD2M, a two-million-sample dataset of static stickers and animated GIFs, which it claims is the largest and most comprehensive multimodal sticker dataset to date. Each sample includes a description that captures action information, such as \"swaying in the wind,\" plus OCR text, trigger words, and a real/cartoon domain flag. The authors also propose a spatial-temporal interaction (STI) layer that splits temporal modeling into semantic interaction and detail preservation, and they show it improves video-generation baselines on the new benchmark. If the dataset delivers on its claims, the field gains a public benchmark for animated sticker generation, a task that previously had neither dedicated data nor standardized evaluation.","feed_headline":"Largest sticker dataset (2M) opens animated GIF generation","feed_subtitle":"New benchmark pairs 2.09M stickers with action-aware captions; a spatial-temporal layer beats diffusion baselines.","key_machinery":"The load-bearing object is the spatial-temporal interaction (STI) layer. It takes the feature tensor of a frame sequence, downsamples it spatially, runs self-attention across the flattened frame-and-space dimensions to let each frame attend to regions in other frames, then upsamples back; in parallel it applies convolutions with kernel size k×1×1 to preserve fine details, and learns a weight to balance the two branches. This replaces the temporal modeling in VideoLDM, targeting the paper's diagnosis that general video generators fail on stickers because their temporal receptive fields are too small and they rely on nearby frames, while sticker GIFs have large, discrete inter-frame changes. The dataset pipeline is the other essential piece: web-crawled stickers filtered by OCR, information entropy, and aspect ratio, then annotated by fine-tuned OCR, domain classification, video captioning, and an LLM summarizer to yield action-rich bilingual captions.","core_discovery":"On its own terms, the paper establishes two things. First, VSD2M contains 2.09 million stickers, about three times the size of the prior largest sticker dataset Sticker820K, and it is the first sticker dataset to include both static stickers and GIFs; its captions are generated by fine-tuning a video-language model on 330K manually annotated examples so that they describe actions, not just static content. Second, the proposed STI layer, inserted into the latent video diffusion model VideoLDM in place of its temporal layers, achieves the best VQA and FVD scores on both the real-domain test set VSD-R and the cartoon-domain test set VSD-C, with the paper attributing the gain to cross-frame semantic interaction plus detail-preserving convolutions.","pith_inferences":["Editor's inference: the STI layer's benefit may transfer to any temporally sparse generation task where entities change abruptly between frames, such as storyboard generation or frame-interpolated animations, not just stickers.","Editor's inference: the dataset's inclusion of OCR text and domain flags could support joint text-rendering and cartoon/real style control, an avenue the paper notes but does not explore in depth.","Editor's inference: the automatic-metric margins in Table 4 are small enough that an independent multi-seed replication is needed before the STI layer's gain is treated as settled; a negative result would leave the dataset contribution intact but weaken the method claim."],"forward_implications":["If VSD2M is as comprehensive as claimed, it becomes the standard training and evaluation set for animated sticker generation, replacing static-only sticker datasets.","The STI layer's separation of semantic interaction and detail preservation gives video-generation researchers a concrete architectural pattern to try on other discrete, low-frame-rate data such as anime, comic strips, or stop-motion.","The comparison shows diffusion-based methods clearly outperform the transformer-based VideoGPT on sticker generation, suggesting future work should start from diffusion backbones.","Because the captions encode actions, models trained on VSD2M can in principle be prompted to control motion, enabling text-to-sticker customization for chat applications."],"supporting_citations":[{"why":"Sticker820K is the prior largest sticker dataset that VSD2M claims to exceed in scale and modality.","marker":"(Zhao et al. 2023)"},{"why":"SER30K is a prior static sticker dataset used as a comparison in training-set experiments.","marker":"(Liu, Zhang, and Yang 2022)"},{"why":"VideoLDM is the base model whose temporal layers are replaced by the STI layer and which serves as the primary baseline.","marker":"(Blattmann et al. 2023)"},{"why":"FVD is the main distribution-consistency metric used in the quantitative comparisons.","marker":"(Unterthiner et al. 2018)"},{"why":"VideoLlama is fine-tuned on 330K manual annotations to generate the action-aware captions in VSD2M.","marker":"(Zhang, Li, and Bing 2023)"},{"why":"TGIF is a GIF-description dataset used as an alternative training set in Table 5 to demonstrate VSD2M's advantage.","marker":"(Li et al. 2016)"}],"fun_headline_variants":["2M sticker dataset boosts animated GIF generation","Largest sticker dataset for multi-frame animated stickers","Spatial-temporal interaction enhances animated sticker AI","VSD2M: 2M sticker benchmark for animated creation"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The claim that the STI layer improves sticker generation rests on single-run differences in automatic metrics—for example, 16 FVD points on the real test set—without confidence intervals or significance tests, so the whole method comparison could collapse if those differences are just run-to-run noise.","fun_headline_variants_meta":{"raw":{"variants":["2M sticker dataset boosts animated GIF generation","Largest sticker dataset for multi-frame animated stickers","Spatial-temporal interaction enhances animated sticker AI","VSD2M: 2M sticker benchmark for animated creation"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000233,"raw_usage":{"total_tokens":1512,"prompt_tokens":984,"completion_tokens":528,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":600,"completion_tokens_details":{"reasoning_tokens":466}},"tokens_in":600,"tokens_out":528,"duration_ms":5754,"temperature":1.0,"reasoning_tokens":466,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-11T18:01:06.676137+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run the same 'ours' and I2VGen-XL training setups at least five seeds each and compute the mean and standard deviation of VQA and FVD on VSD-R and VSD-C; if the difference between the two methods consistently lies within one standard deviation, the claimed benefit of the STI layer is not supported. Alternatively, an independent re-evaluation of the released checkpoints with a third-party FVD implementation that yields overlapping scores would falsify the reported margin.","supporting_citations":[{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"SER30K is a prior static sticker dataset used as a comparison in training-set experiments."}],"review_version":1}