Pith. sign in

REVIEW 3 major objections 5 minor 57 references

VSD2M: A Large-scale Vision-language Sticker Dataset for Multi-frame Animated Sticker Generation

T0 review · 3 major / 5 minor · reviewed 2026-08-11 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read The paper introduces VSD2M, a two-million-sample dataset of static stickers and animated GIFs with action-aware captions, and a spatial-temporal interaction layer that improves animated sticker generation over standard video diffusion…

desk verdict VSD2M is a genuinely useful large-scale animated sticker dataset, but the STI layer's performance claims rest on thin, single-run evidence and a test split that may overlap training data. read the letter →

arxiv 2412.08259 v2 pith:JO4GFE5M submitted 2024-12-11 cs.HC

Zhiqiang Yuan , Jiapei Zhang , Ying Deng , Yeshuang Zhu , Jie Zhou , Jinchao Zhang This is my paper · ORCID
classification cs.HC
keywords stickerdatasetanimatedgenerationvision-languagebenchmarkGIFspatialtemporalinteractiondiffusionmodelstext-to-video
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper introduces VSD2M, a two-million-sample dataset of static stickers and animated GIFs, which it claims is the largest and most comprehensive multimodal sticker dataset to date. Each sample includes a description that captures action information, such as "swaying in the wind," plus OCR text, trigger words, and a real/cartoon domain flag. The authors also propose a spatial-temporal interaction (STI) layer that splits temporal modeling into semantic interaction and detail preservation, and they show it improves video-generation baselines on the new benchmark. If the dataset delivers on its claims, the field gains a public benchmark for animated sticker generation, a task that previously had neither dedicated data nor standardized evaluation.

What carries the argument

The load-bearing object is the spatial-temporal interaction (STI) layer. It takes the feature tensor of a frame sequence, downsamples it spatially, runs self-attention across the flattened frame-and-space dimensions to let each frame attend to regions in other frames, then upsamples back; in parallel it applies convolutions with kernel size k×1×1 to preserve fine details, and learns a weight to balance the two branches. This replaces the temporal modeling in VideoLDM, targeting the paper's diagnosis that general video generators fail on stickers because their temporal receptive fields are too small and they rely on nearby frames, while sticker GIFs have large, discrete inter-frame changes. The dataset pipeline is the other essential piece: web-crawled stickers filtered by OCR, information entropy, and aspect ratio, then annotated by fine-tuned OCR, domain classification, video captioning, and an LLM summarizer to yield action-rich bilingual captions.

What would settle it

Run the same 'ours' and I2VGen-XL training setups at least five seeds each and compute the mean and standard deviation of VQA and FVD on VSD-R and VSD-C; if the difference between the two methods consistently lies within one standard deviation, the claimed benefit of the STI layer is not supported. Alternatively, an independent re-evaluation of the released checkpoints with a third-party FVD implementation that yields overlapping scores would falsify the reported margin.

Watch

Extended reading notes

Core claim

On its own terms, the paper establishes two things. First, VSD2M contains 2.09 million stickers, about three times the size of the prior largest sticker dataset Sticker820K, and it is the first sticker dataset to include both static stickers and GIFs; its captions are generated by fine-tuning a video-language model on 330K manually annotated examples so that they describe actions, not just static content. Second, the proposed STI layer, inserted into the latent video diffusion model VideoLDM in place of its temporal layers, achieves the best VQA and FVD scores on both the real-domain test set VSD-R and the cartoon-domain test set VSD-C, with the paper attributing the gain to cross-frame semantic interaction plus detail-preserving convolutions.

Load-bearing premise

The claim that the STI layer improves sticker generation rests on single-run differences in automatic metrics—for example, 16 FVD points on the real test set—without confidence intervals or significance tests, so the whole method comparison could collapse if those differences are just run-to-run noise.

Editorial extensions

If this is right

  • If VSD2M is as comprehensive as claimed, it becomes the standard training and evaluation set for animated sticker generation, replacing static-only sticker datasets.
  • The STI layer's separation of semantic interaction and detail preservation gives video-generation researchers a concrete architectural pattern to try on other discrete, low-frame-rate data such as anime, comic strips, or stop-motion.
  • The comparison shows diffusion-based methods clearly outperform the transformer-based VideoGPT on sticker generation, suggesting future work should start from diffusion backbones.
  • Because the captions encode actions, models trained on VSD2M can in principle be prompted to control motion, enabling text-to-sticker customization for chat applications.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Editor's inference: the STI layer's benefit may transfer to any temporally sparse generation task where entities change abruptly between frames, such as storyboard generation or frame-interpolated animations, not just stickers.
  • Editor's inference: the dataset's inclusion of OCR text and domain flags could support joint text-rendering and cartoon/real style control, an avenue the paper notes but does not explore in depth.
  • Editor's inference: the automatic-metric margins in Table 4 are small enough that an independent multi-seed replication is needed before the STI layer's gain is treated as settled; a negative result would leave the dataset contribution intact but weaken the method claim.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. This paper introduces VSD2M, a vision-language dataset of about 2.09M static and animated stickers crawled from the web, annotated with English and Chinese captions, trigger words, OCR text, and a real/cartoon flag. The authors propose a Spatial Temporal Interaction (STI) layer that splits temporal modeling into a semantic-interaction branch (downsampled self-attention across frames) and a detail-preservation branch (frame convolution), and they insert it into VideoLDM. They compare VideoGPT, Make-A-Video, VideoLDM, VideoFactory, I2VGen-XL, and 'Ours' on two test sets (VSD-R and VSD-C) using FVD, CLIP similarity, and VQA, plus user studies. The paper claims that VSD2M is the largest multimodal sticker dataset to date and that the STI layer improves animated sticker generation.

Significance. If the quantitative results are reliable, the dataset is a useful community resource: it is roughly three times the size of Sticker820K, includes animated stickers (GIFs) with action-oriented captions, and provides a first systematic comparison of video-generation architectures on the animated sticker generation task. The STI layer is a simple, plausible modification for discrete low-frame-rate data, and the paper's user studies cover dimensions such as temporal consistency, visual quality, and user favorability beyond the automatic metrics. The strengths are the scope of the data-collection pipeline and the breadth of baselines, not machine-checked proofs or parameter-free derivations. However, the method contribution is not yet supported at the reported level of evidence: the key Table 4 margins are small, no confidence intervals or multiple seeds are reported, the test sets are drawn from the same crawl with no deduplication described, and the STI components are not ablated.

major comments (3)
  1. [Data Construction] The test sets VSD-R and VSD-C are manually selected from samples corresponding to the top 500 trigger words in real and cartoon domains, drawn from the same 2.1M-sample crawled pool used for training, and no exact or near-duplicate removal (e.g., perceptual hashing or CLIP-based deduplication) is reported before or after the split. Because FVD measures distribution distance and VQA measures quality, any exact or near-duplicate overlap between the test GIFs and training GIFs would inflate all methods' scores; the reported margins over I2VGen-XL on VSD-R are only 16 FVD points (2613.41 vs 2623.47) and 0.006 VQA points. The paper should report the deduplication procedure and quantify the train/test overlap; without this, the benchmark numbers in Table 4 are not a clean test of the STI layer.
  2. [Experimental Setups / Table 4] The central claim that 'Ours' is best rests on single-run metric values with no confidence intervals, multiple seeds, or significance tests. For example, on VSD-R, Ours (0.487 VQA, 2613.41 FVD) is separated from I2VGen-XL (0.481, 2623.47) and VideoFactory (0.480, 2638.97) by margins that are likely within run-to-run noise for FVD/VQA. Please provide error bars over at least three seeds and a paired significance test. In addition, the STI layer is only evaluated as a full replacement of VideoLDM's temporal layers; there is no ablation isolating the semantic-interaction branch, the detail-preservation branch, or the learnable balancing weights, so the attribution of the gains to the proposed components is not established.
  3. [STI Layer for Discrete Sticker Generation] The description of the STI layer leaves the downsampling factor gamma, the frame convolution kernel size k, and the initialization or learning of the balancing weights unspecified, and the experiments never state the values used. Without these numbers, the proposed module cannot be reproduced or compared across backbones; please report the actual settings and, ideally, a sensitivity study over gamma and k. This is not merely a presentation issue because the paper motivates the design by 'appropriate gamma values' and 'kernel size k,' yet no configuration is given in the main text or the Appendix.
minor comments (5)
  1. [Throughout] There are several typographical errors, including 'the ASG filed' in the Introduction, 'layerll' before the Qualitative Comparison subsection, and 'an stronger sense of motion' in the Qualitative Comparison subsection; the manuscript needs proofreading.
  2. [User Preference / Appendix A.2] The number of labelers is inconsistent: the main text says 'nine labelers' while Appendix A.2 says 'ten people'; please report the exact protocol, the number of participants, and any inter-annotator agreement measure.
  3. [Abstract and Introduction] The dataset and code are promised 'at link' with a placeholder URL; this should be replaced with a working link and the license or terms of the crawled data should be stated.
  4. [Table 3] The row 'Ave Frames Numbers' reports 16.91, but the surrounding text and Figure 4(b) note that only multi-frame animated stickers are counted; please make the counting convention explicit in the table caption.
  5. [Experimental Setups] Training hyperparameters such as learning rate, batch size, number of training steps, and optimizer are not reported, which limits the reproducibility of the baselines and of 'Ours'.

Circularity Check

0 steps flagged · score 1.0 of 10

No circular derivation: VSD2M's claims rest on external baselines and standard metrics; test-set leakage concerns are data-integrity risks, not circularity.

full rationale

The paper's central contributions are dataset construction and an architectural layer, and neither reduces by construction to its inputs. The dataset is built from web crawling, filtering, annotation, and manual test-set selection; captions are produced by fine-tuned VideoLlama/LLaVA, but the benchmark claims are not derived from those captions. The STI layer is defined as an architecture modification and is compared against external baselines (VideoGPT, Make-A-Video, VideoLDM, VideoFactory, I2VGen-XL) using standard external metrics (FVD, CLIP, VQA), with all methods trained on the same VSD2M splits. No fitted parameter is renamed as a prediction, and no equation equates a claimed result with its input. The only notable concern is that VSD-R and VSD-C are selected from the same crawled pool with no reported deduplication, which could weaken the Table 4 comparison if leakage occurred; however, this is a data-quality and measurement risk, not a circular derivation. No load-bearing self-citation or imported uniqueness theorem appears. The mild score reflects the small-margin automatic-metric differences and the absence of significance testing, not circular reasoning.

Assumptions & free parameters 3 free parameters · 4 assumptions · 0 invented entities

The central numerical claims rest on dataset construction choices and evaluation assumptions rather than on derived constants. There are no invented physical entities. The main free parameters are the two STI branch hyperparameters and the learned branch weights, none of which are ablated. The axioms above are the load-bearing measurement and annotation assumptions.

free parameters (3)
  • Spatial-temporal downsampling factor γ in STI semantic interaction branch = not reported
    Chosen by hand; controls the size of the self-attention receptive field and computational cost. No ablation or sensitivity study is provided in section 'STI Layer for Discrete Sticker Generation'.
  • Frame convolution kernel size k in STI detail preservation branch = not reported
    Chosen by hand; determines how many frames are mixed to preserve details. The paper does not report its value or an ablation.
  • Learnable weights balancing semantic interaction and detail preservation branches = learned during training
    These weights are fit to the training data and determine the relative contribution of the two STI branches; their behavior across layers is not analyzed.
assumptions (4)
  • domain assumption Fine-tuned VideoLlama, LLaVA, and LLM summarization produce action-aware captions accurate enough to train text-conditioned sticker generation.
    The dataset's text condition is the main control signal; no caption quality metric or human agreement study is reported. This enters in Data Construction, item c.
  • domain assumption The single-run automatic metrics FVD, CLIP similarity, and Fast-VQA are reliable for ranking animated sticker generation models.
    No confidence intervals, seeds, or statistical tests are reported, yet the paper uses small metric gaps to rank methods. Enters in Experimental Setups, Metrics, and Table 4.
  • domain assumption The VSD-R and VSD-C test sets, manually selected from the top 500 trigger words per domain, are representative of the full dataset domains.
    Top trigger words may overrepresent common concepts and make the test sets easier than random samples; this is introduced in Data Construction.
  • domain assumption Standard video generation models can be transferred to animated sticker generation by fine-tuning on VSD2M.
    The whole benchmark relies on this transferability, which is asserted rather than derived in the Experimental Setups.

how reviews work

0 comments
Cite this review

Pith. "Pith review of VSD2M: A Large-scale Vision-language Sticker Dataset for Multi-frame Animated Sticker Generation." pith.science (2026). https://pith.science/paper/JO4GFE5M

@misc{pith2026241208259,
  author       = {Pith},
  title        = {Pith review of: VSD2M: A Large-scale Vision-language Sticker Dataset for Multi-frame Animated Sticker Generation},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/JO4GFE5M}},
  note         = {Machine review of arXiv:2412.08259}
}
read the original abstract

As a common form of communication in social media,stickers win users' love in the internet scenarios, for their ability to convey emotions in a vivid, cute, and interesting way. People prefer to get an appropriate sticker through retrieval rather than creation for the reason that creating a sticker is time-consuming and relies on rule-based creative tools with limited capabilities. Nowadays, advanced text-to-video algorithms have spawned numerous general video generation systems that allow users to customize high-quality, photo-realistic videos by only providing simple text prompts. However, creating customized animated stickers, which have lower frame rates and more abstract semantics than videos, is greatly hindered by difficulties in data acquisition and incomplete benchmarks. To facilitate the exploration of researchers in animated sticker generation (ASG) field, we firstly construct the currently largest vision-language sticker dataset named VSD2M at a two-million scale that contains static and animated stickers. Secondly, to improve the performance of traditional video generation methods on ASG tasks with discrete characteristics, we propose a Spatial Temporal Interaction (STI) layer that utilizes semantic interaction and detail preservation to address the issue of insufficient information utilization. Moreover, we train baselines with several video generation methods (e.g., transformer-based, diffusion-based methods) on VSD2M and conduct a detailed analysis to establish systemic supervision on ASG task. To the best of our knowledge, this is the most comprehensive large-scale benchmark for multi-frame animated sticker generation, and we hope this work can provide valuable inspiration for other scholars in intelligent creation.

Figures

Figures reproduced from arXiv: 2412.08259 by the authors.

Figure 1
Figure 1. Overview of data collection and processing, which can be divided into four stages: web crawling, data filtering, annotation and [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 2
Figure 2. Two samples of VSD2M, in which GIFs is framed for visualization. The [PITH_FULL_IMAGE:figures/full_fig_p003_2.png] view at source ↗
Figure 3
Figure 3. Word cloud distribution of the description in VSD2M, [PITH_FULL_IMAGE:figures/full_fig_p003_3.png] view at source ↗
Figures from the paper (6 more)
Figure 4
Figure 4. Figure 4: Visual analysis of VSD2M. (a) Frequency count of top 25 trigger words. (b) Statistics of frame number, note that we only count [PITH_FULL_IMAGE:figures/full_fig_p004_4.png]
Figure 5
Figure 5. Figure 5: Visual comparison for animated sticker generation between VideoLDM, VideoFactory, I2VGen-XL and ours. The text prompts are [PITH_FULL_IMAGE:figures/full_fig_p006_5.png]
Figure 6
Figure 6. Figure 6: The indicator changes of different models as the number [PITH_FULL_IMAGE:figures/full_fig_p007_6.png]
Figure 7
Figure 7. Figure 7: Manual comparison of sticker generated by different [PITH_FULL_IMAGE:figures/full_fig_p007_7.png]
Figure 9
Figure 9. Figure 9: More detailed subjective evaluation results, which are [PITH_FULL_IMAGE:figures/full_fig_p010_9.png]
Figure 8
Figure 8. Figure 8: Framework comparison of transformer-based and [PITH_FULL_IMAGE:figures/full_fig_p010_8.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

57 extracted references · 20 canonical work pages

  1. [1]

    Bain, M.; Nagrani, A.; Varol, G.; and Zisserman, A. 2021. Frozen in time: A joint video and image encoder for end-to-end retrieval. In Proceedings of the IEEE/CVF International Conference on Computer Vision, 1728--1738

    work page 2021

  2. [2]

    W.; Fidler, S.; and Kreis, K

    Blattmann, A.; Rombach, R.; Ling, H.; Dockhorn, T.; Kim, S. W.; Fidler, S.; and Kreis, K. 2023. Align your latents: High-resolution video synthesis with latent diffusion models. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 22563--22575

    2023

  3. [3]

    Brooks, T.; Hellsten, J.; Aittala, M.; Wang, T.-C.; Aila, T.; Lehtinen, J.; Liu, M.-Y.; Efros, A.; and Karras, T. 2022. Generating long videos of dynamic scenes. Advances in Neural Information Processing Systems, 35: 31769--31781

    work page 2022

  4. [4]

    Creswell, A.; White, T.; Dumoulin, V.; Arulkumaran, K.; Sengupta, B.; and Bharath, A. A. 2018. Generative adversarial networks: An overview. IEEE signal processing magazine, 35(1): 53--65

    2018

  5. [5]

    Du, Y.; Li, C.; Guo, R.; Yin, X.; Liu, W.; Zhou, J.; Bai, Y.; Yu, Z.; Yang, Y.; Dang, Q.; et al. 2020. Pp-ocr: A practical ultra lightweight ocr system. arXiv preprint arXiv:2009.09941

    arXiv 2020

  6. [6]

    Fei, Z.; Li, Z.; Zhang, J.; Feng, Y.; and Zhou, J. 2021. Towards expressive communication with internet memes: A new multimodal conversation dataset and benchmark. arXiv preprint arXiv:2109.01839

    arXiv 2021

  7. [7]

    Gao, S.; Chen, X.; Liu, C.; Liu, L.; Zhao, D.; and Yan, R. 2020. Learning to respond with stickers: A framework of unifying multi-modality in multi-turn dialog. In Proceedings of the Web Conference 2020, 1138--1148

    work page 2020

  8. [8]

    Ge, S.; Nah, S.; Liu, G.; Poon, T.; Tao, A.; Catanzaro, B.; Jacobs, D.; Huang, J.-B.; Liu, M.-Y.; and Balaji, Y. 2023. Preserve your own correlation: A noise prior for video diffusion models. In Proceedings of the IEEE/CVF International Conference on Computer Vision, 22930--22941

    2023

Show all 57 references
  1. [9]

    He, Y.; Xia, M.; Chen, H.; Cun, X.; Gong, Y.; Xing, J.; Zhang, Y.; Wang, X.; Weng, C.; Shan, Y.; et al. 2023. Animate-a-story: Storytelling with retrieval-augmented video generation. arXiv preprint arXiv:2307.06940

    2023 arXiv

  2. [10]

    Hong, W.; Ding, M.; Zheng, W.; Liu, X.; and Tang, J. 2022. Cogvideo: Large-scale pretraining for text-to-video generation via transformers. arXiv preprint arXiv:2205.15868

    2022 arXiv

  3. [11]

    Huang, F.; Li, X.; Zhang, S.; Zhang, J.; Chen, J.; and Zhai, Z. 2017. Overlapping community detection for multimedia social networks. IEEE Transactions on multimedia, 19(8): 1881--1893

    2017

  4. [12]

    Huang, Y.; Huang, J.; Liu, J.; Yan, M.; Dong, Y.; Lyu, J.; Chen, C.; and Chen, S. 2024. Wavedm: Wavelet-based diffusion models for image restoration. IEEE Transactions on Multimedia

    2024

  5. [13]

    Koonce, B.; and Koonce, B. 2021. EfficientNet. Convolutional Neural Networks with Swift for Tensorflow: Image Recognition and Dataset Categorization, 109--123

    2021

  6. [14]

    A.; et al

    Krishna, R.; Zhu, Y.; Groth, O.; Johnson, J.; Hata, K.; Kravitz, J.; Chen, S.; Kalantidis, Y.; Li, L.-J.; Shamma, D. A.; et al. 2017. Visual genome: Connecting language and vision using crowdsourced dense image annotations. International journal of computer vision, 123: 32--73

    2017

  7. [15]

    Lei, X.; Qian, X.; and Zhao, G. 2016. Rating prediction based on social sentiment from textual reviews. IEEE transactions on multimedia, 18(9): 1910--1921

    2016

  8. [16]

    Li, J.; Li, D.; Savarese, S.; and Hoi, S. 2023. Blip-2: Bootstrapping language-image pre-training with frozen image encoders and large language models. arXiv preprint arXiv:2301.12597

    2023 arXiv

  9. [17]

    Li, Y.; Song, Y.; Cao, L.; Tetreault, J.; Goldberg, L.; Jaimes, A.; and Luo, J. 2016. TGIF: A new dataset and benchmark on animated GIF description. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 4641--4650

    2016

  10. [18]

    Lin, T.-Y.; Maire, M.; Belongie, S.; Hays, J.; Perona, P.; Ramanan, D.; Doll \'a r, P.; and Zitnick, C. L. 2014. Microsoft coco: Common objects in context. In Computer Vision--ECCV 2014: 13th European Conference, Zurich, Switzerland, September 6-12, 2014, Proceedings, Part V 1...

    2014

  11. [19]

    Liu, J.; Yuan, H.; Yuan, Z.; Liu, L.; Lu, B.; and Yu, M. 2023. Visual transformer with stable prior and patch-level attention for single image dehazing. Neurocomputing, 551: 126535

    2023

  12. [20]

    Liu, S.; Zhang, X.; and Yang, J. 2022. SER30K: A large-scale dataset for sticker emotion recognition. In Proceedings of the 30th ACM International Conference on Multimedia, 33--41

    2022

  13. [21]

    Luo, Z.; Chen, D.; Zhang, Y.; Huang, Y.; Wang, L.; Shen, Y.; Zhao, D.; Zhou, J.; and Tan, T. 2023. VideoFusion: Decomposed Diffusion Models for High-Quality Video Generation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 10209--10218

    2023

  14. [22]

    Maaz, M.; Rasheed, H.; Khan, S.; and Khan, F. S. 2023. Video-ChatGPT: Towards Detailed Video Understanding via Large Vision and Language Models. arXiv preprint arXiv:2306.05424

    2023 arXiv

  15. [23]

    Mangrulkar, S.; Gugger, S.; Debut, L.; Belkada, Y.; Paul, S.; and Bossan, B. 2022. PEFT: State-of-the-art Parameter-Efficient Fine-Tuning methods. https://github.com/huggingface/peft

    2022

  16. [24]

    X.; and Min, M

    Ni, H.; Shi, C.; Li, K.; Huang, S. X.; and Min, M. R. 2023. Conditional Image-to-Video Generation with Latent Flow Diffusion Models. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 18444--18455

    2023

  17. [25]

    A.; Wang, L.; Cervantes, C

    Plummer, B. A.; Wang, L.; Cervantes, C. M.; Caicedo, J. C.; Hockenmaier, J.; and Lazebnik, S. 2015. Flickr30k entities: Collecting region-to-phrase correspondences for richer image-to-sentence models. In Proceedings of the IEEE international conference on computer vision, 2641--2649

    2015

  18. [26]

    Poria, S.; Hazarika, D.; Majumder, N.; Naik, G.; Cambria, E.; and Mihalcea, R. 2018. Meld: A multimodal multi-party dataset for emotion recognition in conversations. arXiv preprint arXiv:1810.02508

    2018 arXiv

  19. [27]

    Saito, M.; Matsumoto, E.; and Saito, S. 2017. Temporal generative adversarial nets with singular value clipping. In Proceedings of the IEEE international conference on computer vision, 2830--2839

    2017

  20. [28]

    Schuhmann, C.; Beaumont, R.; Vencu, R.; Gordon, C.; Wightman, R.; Cherti, M.; Coombes, T.; Katta, A.; Mullis, C.; Wortsman, M.; et al. 2022. Laion-5b: An open large-scale dataset for training next generation image-text models. Advances in Neural Information Processing Systems,...

    2022

  21. [29]

    Schuhmann, C.; Vencu, R.; Beaumont, R.; Kaczmarczyk, R.; Mullis, C.; Katta, A.; Coombes, T.; Jitsev, J.; and Komatsuzaki, A. 2021. Laion-400m: Open dataset of clip-filtered 400 million image-text pairs. arXiv preprint arXiv:2111.02114

    2021 arXiv

  22. [30]

    Sheynin, S.; Ashual, O.; Polyak, A.; Singer, U.; Gafni, O.; Nachmani, E.; and Taigman, Y. 2022. Knn-diffusion: Image generation via large-scale retrieval. arXiv preprint arXiv:2204.02849

    2022 arXiv

  23. [31]

    Siarohin, A.; Lathuili \`e re, S.; Tulyakov, S.; Ricci, E.; and Sebe, N. 2019. First order motion model for image animation. Advances in neural information processing systems, 32

    2019

  24. [32]

    Singer, U.; Polyak, A.; Hayes, T.; Yin, X.; An, J.; Zhang, S.; Hu, Q.; Yang, H.; Ashual, O.; Gafni, O.; et al. 2022. Make-a-video: Text-to-video generation without text-video data. arXiv preprint arXiv:2209.14792

    2022 arXiv

  25. [33]

    Skorokhodov, I.; Tulyakov, S.; and Elhoseiny, M. 2022. Stylegan-v: A continuous video generator with the price, image quality and perks of stylegan2. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 3626--3636

    2022

  26. [34]

    Sohl-Dickstein, J.; Weiss, E.; Maheswaranathan, N.; and Ganguli, S. 2015. Deep unsupervised learning using nonequilibrium thermodynamics. In International conference on machine learning, 2256--2265. PMLR

    2015

  27. [35]

    Song, J.; Meng, C.; and Ermon, S. 2020. Denoising diffusion implicit models. arXiv preprint arXiv:2010.02502

    2020 arXiv

  28. [36]

    P.; Kumar, A.; Ermon, S.; and Poole, B

    Song, Y.; Sohl-Dickstein, J.; Kingma, D. P.; Kumar, A.; Ermon, S.; and Poole, B. 2020. Score-based generative modeling through stochastic differential equations. arXiv preprint arXiv:2011.13456

    2020 arXiv

  29. [37]

    Soujanya, P.; Hazarika, D.; Majumder, N.; Naik, G.; Cambria, E.; and Mihalcea, R. 2018. A multimodal multi-party dataset for emotion recognition in conversations

    2018

  30. [38]

    Unterthiner, T.; Van Steenkiste, S.; Kurach, K.; Marinier, R.; Michalski, M.; and Gelly, S. 2018. Towards accurate generative models of video: A new metric & challenges. arXiv preprint arXiv:1812.01717

    2018 arXiv

  31. [39]

    N.; Kaiser, .; and Polosukhin, I

    Vaswani, A.; Shazeer, N.; Parmar, N.; Uszkoreit, J.; Jones, L.; Gomez, A. N.; Kaiser, .; and Polosukhin, I. 2017. Attention is all you need. Advances in neural information processing systems, 30

    2017

  32. [40]

    Wang, W.; Yang, H.; Tuo, Z.; He, H.; Zhu, J.; Fu, J.; and Liu, J. 2023 a . VideoFactory: Swap Attention in Spatiotemporal Diffusions for Text-to-Video Generation. arXiv preprint arXiv:2305.10874

    2023 arXiv

  33. [41]

    Wang, X.; Yuan, H.; Zhang, S.; Chen, D.; Wang, J.; Zhang, Y.; Shen, Y.; Zhao, D.; and Zhou, J. 2023 b . VideoComposer: Compositional Video Synthesis with Motion Controllability. arXiv preprint arXiv:2306.02018

    2023 arXiv

  34. [42]

    Wu, C.; Huang, L.; Zhang, Q.; Li, B.; Ji, L.; Yang, F.; Sapiro, G.; and Duan, N. 2021. Godiva: Generating open-domain videos from natural descriptions. arXiv preprint arXiv:2104.14806

    2021 arXiv

  35. [43]

    Wu, H.; Chen, C.; Hou, J.; Liao, L.; Wang, A.; Sun, W.; Yan, Q.; and Lin, W. 2022. Fast-vqa: Efficient end-to-end video quality assessment with fragment sampling. In European Conference on Computer Vision, 538--554. Springer

    2022

  36. [44]

    Z.; Ge, Y.; Wang, X.; Lei, S

    Wu, J. Z.; Ge, Y.; Wang, X.; Lei, S. W.; Gu, Y.; Shi, Y.; Hsu, W.; Shan, Y.; Qie, X.; and Shou, M. Z. 2023. Tune-a-video: One-shot tuning of image diffusion models for text-to-video generation. In Proceedings of the IEEE/CVF International Conference on Computer Vision, 7623--7633

    2023

  37. [45]

    Xiong, W.; Luo, W.; Ma, L.; Liu, W.; and Luo, J. 2018. Learning to generate time-lapse videos using multi-stage dynamic generative adversarial networks. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2364--2373

    2018

  38. [46]

    Yan, W.; Okumura, R.; James, S.; and Abbeel, P. 2022. Patch-based Object-centric Transformers for Efficient Video Generation. arXiv preprint arXiv:2206.04003

    2022 arXiv

  39. [47]

    Yan, W.; Zhang, Y.; Abbeel, P.; and Srinivas, A. 2021. Videogpt: Video generation using vq-vae and transformers. arXiv preprint arXiv:2104.10157

    2021 arXiv

  40. [48]

    Yang, A.; Xiao, B.; Wang, B.; Zhang, B.; Yin, C.; Lv, C.; Pan, D.; Wang, D.; Yan, D.; Yang, F.; et al. 2023. Baichuan 2: Open large-scale language models. arXiv preprint arXiv:2309.10305

    2023 arXiv

  41. [49]

    Yang, L.; Wang, H.; Tang, P.; and Li, Q. 2020. CaptionNet: A tailor-made recurrent neural network for generating image descriptions. IEEE Transactions on Multimedia, 23: 835--845

    2020

  42. [50]

    Yang, R.; Srivastava, P.; and Mandt, S. 2022. Diffusion probabilistic modeling for video generation. arXiv preprint arXiv:2203.09481

    2022 arXiv

  43. [51]

    Zhang, H.; Li, X.; and Bing, L. 2023. Video-llama: An instruction-tuned audio-visual language model for video understanding. arXiv preprint arXiv:2306.02858

    2023 arXiv

  44. [52]

    Zhang, S.; Wang, J.; Zhang, Y.; Zhao, K.; Yuan, H.; Qin, Z.; Wang, X.; Zhao, D.; and Zhou, J. 2023. I2vgen-xl: High-quality image-to-video synthesis via cascaded diffusion models. arXiv preprint arXiv:2311.04145

    2023 arXiv

  45. [53]

    D.; and Langlotz, C

    Zhang, Y.; Jiang, H.; Miura, Y.; Manning, C. D.; and Langlotz, C. P. 2022. Contrastive learning of medical visual representations from paired images and text. In Machine Learning for Healthcare Conference, 2--25. PMLR

    2022

  46. [54]

    Zhao, S.; Ge, Y.; Qi, Z.; Song, L.; Ding, X.; Xie, Z.; and Shan, Y. 2023. Sticker820K: Empowering Interactive Retrieval with Stickers. arXiv preprint arXiv:2306.06870

    2023 arXiv

  47. [55]

    Zhu, D.; Chen, J.; Shen, X.; Li, X.; and Elhoseiny, M. 2023. Minigpt-4: Enhancing vision-language understanding with advanced large language models. arXiv preprint arXiv:2304.10592

    2023 arXiv

  48. [56]

    , " * write output.state after.block = add.period write newline

    ENTRY address archivePrefix author booktitle chapter edition editor eid eprint howpublished institution isbn journal key month note number organization pages publisher school series title type volume year label extra.label sort.label short.list INTEGERS output.state before.all...

  49. [57]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 gl...

Pith tools

Reviewed August 11, 2026 · model on record in the stance chip above.