Pith. sign in

REVIEW 4 major objections 3 minor 93 references

From Image Captioning to Visual Storytelling

T0 review · 4 major / 3 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read Treating visual storytelling as a superset of image captioning—first caption each image, then turn the captions into a story—improves story quality and speeds up training compared with prior end-to-end approaches.

desk verdict The abstract proposes a plausible two-stage storytelling pipeline, but the supplied full text is an unrelated math paper, so the empirical claims and the 'ideality' metric are entirely unsupported. read the letter →

arxiv 2508.14045 v1 pith:ZNRXALNN submitted 2025-07-31 cs.CL cs.CV

Admitos Passadakis , Yingjin Song , Albert Gatt This is my paper
classification cs.CLcs.CV
keywords visualstorytellingimagecaptioningvision-to-languagelanguage-to-languageidealitymetricmultimodalgenerationnarrativecoherencetrainingefficiency
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper argues that visual storytelling should be decomposed into two stages: a vision-to-language model first produces a caption for each image in the sequence, and then a language-to-language model transforms those captions into a coherent narrative. The authors claim this division of labor yields better stories than prior approaches that generate a story directly from the image stream, while also cutting training time and making the system easier to reproduce. They also introduce 'ideality,' a metric that measures how far a generated story sits from an oracle model, and use it to estimate how human-like the stories are. The central bet is that separating grounding from narrative coherence is more effective than trying to solve both in one multimodal step.

What carries the argument

The load-bearing object is the two-stage pipeline: stage one uses a vision-to-language model to generate per-image captions; stage two uses a language-to-language model to convert the caption list into a coherent story. This architecture assumes captions carry the visual grounding and the text stage supplies narrative structure. The evaluation also relies on 'ideality,' a proposed metric that estimates the distance between a system's output and an oracle model's output, providing a proxy for how close a story comes to an ideal human-like narrative.

What would settle it

Train an end-to-end vision-to-story model on the same dataset with a comparable training budget and have human judges rate its stories for coherence and grounding; if it matches or beats the two-stage pipeline, the claim that the decomposition improves quality and speed would be undercut. Alternatively, systematically delete spatial-relation words from captions and measure whether story grounding scores drop substantially, showing the second stage cannot recover missing visual information.

Watch

Extended reading notes

Core claim

The paper's central claim is that visual storytelling can be profitably treated as a superset of image captioning. By first extracting captions from each input image with a vision-to-language model and then rewriting those captions into a fluent narrative with a language-to-language model, the system produces stories that are both grounded and coherent. The authors report that this two-stage integration positively impacts story quality and accelerates training relative to numerous prior studies, and they propose a new metric, 'ideality,' that simulates how far a system's outputs are from an oracle model, applying it to quantify human-likeness in visual storytelling.

Load-bearing premise

The two-stage pipeline assumes the captions produced from the images contain all the information the story needs, so the language-to-language stage can work without ever looking back at the original pictures; if captions omit spatial relations, scene dynamics, or salient details, story quality and grounding will degrade.

Editorial extensions

If this is right

  • If the two-stage approach is correct, visual storytelling systems can be built by reusing existing captioning models and text-generation models, lowering the barrier to entry for the task.
  • The reported training-time savings make the framework more scalable and reproducible, which could accelerate research on narrative generation for image sequences.
  • The ideality metric could serve as a model-agnostic evaluation tool for any generative multimodal task in which an oracle output can be defined or simulated.
  • The success of the decomposition suggests that separating visual grounding from narrative coherence is a useful design principle for other vision-and-language generation problems.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The paper's speed advantage is stated relative to prior studies, not necessarily against a strong end-to-end model trained with the same compute budget; a controlled comparison at equal compute would sharpen the claim.
  • Because the second stage never sees the images, the caption set defines an information ceiling—any story can only be as grounded as the captions that feed it, so improving caption fidelity (e.g., including spatial relations and dynamics) should directly raise story quality.
  • Ideality, if it truly measures distance to an oracle, could be extended to per-sentence grounding checks, revealing exactly where caption information is lost or distorted during narrative generation.
  • In low-resource settings where paired image-story datasets are scarce, the two-stage approach could shine because the captioning and text-rewriting components can be pretrained separately on larger, more abundant datasets.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 3 minor

Summary. The manuscript under review consists of an abstract that claims a two-stage visual-storytelling framework (first captions images, then converts captions into narratives), reports positive results on story quality and training speed, and introduces a new metric called "ideality." The full text supplied, however, is an unrelated mathematics paper titled "A Comparative Study of Some Wavelet and Sampling Operators on Various Features of an Image," which presents sampling Kantorovich, Gaussian, bilateral, and wavelet operators, their convergence properties, and image-denoising metrics such as MSE, SI, SSI, SMPI, and ENL. The full text contains no mention of image captioning, visual storytelling, narrative generation, the VIST dataset, baselines, or the proposed ideality metric. The abstract and the body therefore describe two entirely different research efforts, and no experimental or theoretical support for the abstract's claims appears anywhere in the submitted material.

Significance. If the abstract's claims were substantiated, a two-stage caption-then-narrate approach to visual storytelling, together with a new oracle-distance metric called ideality, could be a useful contribution to the multimodal generation literature. However, in the submitted form the paper provides no derivations, no experimental setup, no baselines, no numerical results, no error bars, no ablations, and no definition of the proposed metric. The full text is a self-contained mathematical study of approximation operators that is entirely disconnected from the abstract. Consequently, the manuscript cannot be evaluated as a research contribution, and whatever value the underlying ideas might have is not assessable from the supplied material.

major comments (4)
  1. [Abstract] The abstract asserts that "our multifarious evaluation" shows positive impact on story quality and accelerated training time, but the full text contains no evaluation of any visual storytelling system. Instead, the full text is a mathematics paper on wavelet and sampling operators, with quantitative results only for image-denoising metrics (MSE, SI, SSI, SMPI, ENL). There is no presentation of datasets, baselines, story-quality metrics, or training-time measurements, so the central empirical claim is entirely unsupported.
  2. [Abstract] The proposed "ideality" metric is never defined, formalized, or applied anywhere in the full text. The abstract describes it as a new metric/tool that simulates distance from an oracle model and emulates human-likeness, but the body contains no such definition, no theoretical characterization, and no experimental use. This leaves a load-bearing component of the claimed contribution completely unspecified.
  3. [Full text (entire document)] The full text of the submission is a completely different paper: it concerns sampling Kantorovich operators and their approximation properties, while the abstract concerns visual storytelling. No section of the body addresses image captioning, narrative generation, or the connection between them. This is not a matter of missing appendices or incomplete details; the subject matter of the two parts is disjoint, so the abstract's claims cannot be checked against the body.
  4. [Full text (Sections 1-3 and Tables 1-2)] The manuscript does not identify any prior visual-storytelling studies, and it does not compare against them, despite the abstract claiming that the proposed approach is "quite different compared to most of prior relevant studies" and that it "accelerates training time" relative to "numerous previous studies." Without a literature comparison, baseline results, or a controlled experimental protocol, these comparative claims are unverifiable from the submitted material.
minor comments (3)
  1. [Header/footline] The full text carries the arXiv identifier 2508.14043v1, while the manuscript under review is arXiv:2508.14045; this mismatch indicates that the wrong body text was assembled with the abstract.
  2. [Abstract] The phrase "multifarious evaluation" is vague; even if the body were the intended paper, a precise breakdown of evaluation dimensions would be necessary.
  3. [Section 1] In the full text, the sentence "In the last few years, the evolving research related to this article comes from the Department of Mathematics & Computer Science, University of Perugia" is incomplete and appears to be cut off, which further indicates a document-assembly problem.

Circularity Check

0 steps flagged · score 0.0 of 10

No assessable derivation chain; the supplied full text is an unrelated wavelet paper, so no circular step can be identified.

full rationale

The claimed paper, 'From Image Captioning to Visual Storytelling', is represented only by an abstract. The supplied full text is a different manuscript on sampling Kantorovich operators, wavelet approximations, and image-denoising metrics, with no mention of visual storytelling, image captioning, the VIST dataset, or the proposed 'ideality' metric. Consequently, there is no derivation chain, no fitted parameter, and no self-citation to examine for circularity. The abstract's claims that a two-stage pipeline improves story quality and training time, and that a new 'ideality' metric can emulate human-likeness, are unsupported by the provided body, but unsupported assertions are a matter of completeness and verification, not circularity. Without quoted equations or definitions linking the 'ideality' metric to the models used or to the evaluation data, no specific reduction of a result to its own inputs can be exhibited. Per the hard rules, circularity must be demonstrated with quoted text and a specific reduction; none exists here. The honest finding is therefore no significant circularity, with a score of 0.

Assumptions & free parameters 0 free parameters · 2 assumptions · 0 invented entities

From the abstract, the only identifiable premises are the sufficiency of captions as an intermediate representation and the validity of the 'ideality' metric as a proxy for human judgment. No free parameters are described, and no new physical or conceptual entities are introduced beyond the metric itself.

assumptions (2)
  • domain assumption Captions generated by the vision-to-language model contain sufficient information for story generation.
    The pipeline's second stage operates solely on captions; if captions omit scene context, narrative coherence and grounding would suffer. This assumption is implicit in the abstract's two-stage design.
  • domain assumption The 'ideality' metric can meaningfully measure distance to an oracle and emulate human-likeness.
    The metric's validity depends on the oracle model and the similarity notion; no evidence is given in the abstract.

how reviews work

0 comments
Cite this review

Pith. "Pith review of From Image Captioning to Visual Storytelling." pith.science (2026). https://pith.science/paper/ZNRXALNN

@misc{pith2026250814045,
  author       = {Pith},
  title        = {Pith review of: From Image Captioning to Visual Storytelling},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/ZNRXALNN}},
  note         = {Machine review of arXiv:2508.14045}
}
read the original abstract

Visual Storytelling is a challenging multimodal task between Vision & Language, where the purpose is to generate a story for a stream of images. Its difficulty lies on the fact that the story should be both grounded to the image sequence but also narrative and coherent. The aim of this work is to balance between these aspects, by treating Visual Storytelling as a superset of Image Captioning, an approach quite different compared to most of prior relevant studies. This means that we firstly employ a vision-to-language model for obtaining captions of the input images, and then, these captions are transformed into coherent narratives using language-to-language methods. Our multifarious evaluation shows that integrating captioning and storytelling under a unified framework, has a positive impact on the quality of the produced stories. In addition, compared to numerous previous studies, this approach accelerates training time and makes our framework readily reusable and reproducible by anyone interested. Lastly, we propose a new metric/tool, named ideality, that can be used to simulate how far some results are from an oracle model, and we apply it to emulate human-likeness in visual storytelling.

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

93 extracted references · 46 canonical work pages

  1. [1]

    URL: " 'urlintro :=

    ENTRY address author booktitle chapter edition editor howpublished institution journal key month note number organization pages publisher school series title type volume year eprint doi pubmed url lastchecked label extra.label sort.label short.list INTEGERS output.state before.all mid.sentence after.sentence after.block STRINGS urlintro eprinturl eprintpr...

  2. [2]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 global.max substring 't := if while FUNCTION word.in bbl.in capitalize " " * FUNCT...

  3. [3]

    Peter Anderson, Basura Fernando, Mark Johnson, and Stephen Gould. 2016. http://arxiv.org/abs/1607.08822 Spice: Semantic propositional image caption evaluation

    arXiv 2016

  4. [4]

    Peter Anderson, Xiaodong He, Chris Buehler, Damien Teney, Mark Johnson, Stephen Gould, and Lei Zhang. 2018. http://arxiv.org/abs/1707.07998 Bottom-up and top-down attention for image captioning and visual question answering

    arXiv 2018

  5. [5]

    Shuai Bai, Shusheng Yang, Jinze Bai, Peng Wang, Xingxuan Zhang, Junyang Lin, Xinggang Wang, Chang Zhou, and Jingren Zhou. 2023. http://arxiv.org/abs/2308.16890 Touchstone: Evaluating vision-language models by language models

    arXiv 2023

  6. [6]

    Satanjeev Banerjee and Alon Lavie. 2005. https://aclanthology.org/W05-0909/ METEOR : An automatic metric for MT evaluation with improved correlation with human judgments . In Proceedings of the ACL Workshop on Intrinsic & Extrinsic Evaluation Measures for Machine Translation and Summarization , pages 65--72, Ann Arbor, Michigan. Association for Computatio...

    2005

  7. [7]

    Hong Chen, Yifei Huang, Hiroya Takamura, and Hideki Nakayama. 2022 a . http://arxiv.org/abs/2102.02963 Commonsense knowledge aware concept selection for diverse and informative visual storytelling

    work page Pith review arXiv 2022

  8. [8]

    Wei Chen, Xuefeng Liu, and Jianwei Niu. 2022 b . https://doi.org/10.1109/TCSVT.2022.3183648 Sentistory: A multi-layered sentiment-aware generative model for visual storytelling . IEEE Transactions on Circuits and Systems for Video Technology

    arXiv 2022

Show all 93 references
  1. [9]

    Weiran Chen, Xin Li, Jiaqi Su, Guiqian Zhu, Ying Li, Yi Ji, and Chunping Liu. 2024. http://arxiv.org/abs/2403.11550 Tarn-vist: Topic aware reinforcement network for visual storytelling

    2024 arXiv

  2. [10]

    Cyril Chhun, Fabian M Suchanek, and Chlo \'e Clavel. 2024. Do language models enjoy their own stories? prompting large language models for automatic story evaluation. Transactions of the Association for Computational Linguistics, 12:1122--1142

    2024

  3. [11]

    Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2019. https://doi.org/10.18653/v1/N19-1423 BERT : Pre-training of deep bidirectional transformers for language understanding . In Proceedings of the 2019 Conference of the North A merican Chapter of the Associat...

    2019 doi

  4. [12]

    Lawrence Zitnick

    Jacob Devlin, Saurabh Gupta, Ross Girshick, Margaret Mitchell, and C. Lawrence Zitnick. 2015. http://arxiv.org/abs/1505.04467 Exploring nearest neighbor approaches for image captioning

    2015 arXiv

  5. [13]

    Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xiaohua Zhai, Thomas Unterthiner, Mostafa Dehghani, Matthias Minderer, Georg Heigold, Sylvain Gelly, Jakob Uszkoreit, and Neil Houlsby. 2021. http://arxiv.org/abs/2010.11929 An image is worth 16x16 words:...

    2021 arXiv

  6. [14]

    Ruichao Fan, Hanli Wang, Jinjing Gu, and Xianhui Liu. 2022. https://doi.org/10.1145/3469877.3490604 Visual storytelling with hierarchical bert semantic guidance . In Proceedings of the 3rd ACM International Conference on Multimedia in Asia, New York, USA. Association for Compu...

    2022

  7. [15]

    Le Fang, Tao Zeng, Chaochun Liu, Liefeng Bo, Wen Dong, and Changyou Chen. 2021. http://arxiv.org/abs/2101.00828 Transformer-based conditional variational autoencoder for controllable story generation

    2021 arXiv

  8. [16]

    Ali Farhadi, Mohsen Hejrati, Mohammad Amin Sadeghi, Peter Young, Cyrus Rashtchian, Julia Hockenmaier, and David Forsyth. 2010. Every picture tells a story: Generating sentences from images. In Computer Vision--ECCV 2010: 11th European Conference on Computer Vision, Heraklion, ...

    2010

  9. [17]

    Pablo Gerv \'a s. 2009. Computational approaches to storytelling and creativity. AI Magazine, 30(3):49--49

    2009

  10. [18]

    Diana Gonzalez-Rico and Gibran Fuentes-Pineda. 2018. http://arxiv.org/abs/1806.00738 Contextualize, show and tell: A neural visual storyteller

    2018 arXiv

  11. [19]

    Jian Guan, Fei Huang, Zhihao Zhao, Xiaoyan Zhu, and Minlie Huang. 2020. http://arxiv.org/abs/2001.05139 A knowledge-enhanced pretraining model for commonsense story generation

    2020 arXiv

  12. [20]

    Soyeon Caren Han, Siqu Long, Siwen Luo, Kunze Wang, and Josiah Poon. 2020. http://arxiv.org/abs/2010.03182 Victr: Visual information captured text representation for text-to-image multimodal tasks

    2020 arXiv

  13. [21]

    Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. 2015. http://arxiv.org/abs/1512.03385 Deep residual learning for image recognition

    2015 arXiv

  14. [22]

    Tavakoli, Michael Yang, Bodo Rosenhahn, and Nicolas Pugeault

    Sen He, Wentong Liao, Hamed R. Tavakoli, Michael Yang, Bodo Rosenhahn, and Nicolas Pugeault. 2020. http://arxiv.org/abs/2004.14231 Image captioning through image transformer

    2020 arXiv

  15. [23]

    Simao Herdade, Armin Kappeler, Kofi Boakye, and Joao Soares. 2020. http://arxiv.org/abs/1906.05963 Image captioning: Transforming objects into words

    2020 arXiv

  16. [24]

    Xudong Hong, Asad Sayeed, Khushboo Mehra, Vera Demberg, and Bernt Schiele. 2023. http://arxiv.org/abs/2301.08571 Visual writing prompts: Character-grounded story generation with curated image sequences

    2023 arXiv

  17. [25]

    Xudong Hong, Rakshith Shetty, Asad Sayeed, Khushboo Mehra, Vera Demberg, and Bernt Schiele. 2020. https://doi.org/10.18653/v1/2020.conll-1.34 Diverse and relevant visual storytelling with scene graph embeddings . In Proceedings of the 24th Conference on Computational Natural L...

    2020 doi

  18. [26]

    Chao-Chun Hsu, Zi-Yuan Chen, Chi-Yang Hsu, Chih-Chia Li, Tzu-Yuan Lin, Ting-Hao 'Kenneth' Huang, and Lun-Wei Ku. 2019 a . http://arxiv.org/abs/1912.01496 Knowledge-enriched visual storytelling

    2019 arXiv

  19. [27]

    Chi-Yang Hsu, Yun-Wei Chu, Ting-Hao 'Kenneth' Huang, and Lun-Wei Ku. 2021. http://arxiv.org/abs/2105.06950 Plot and rework: Modeling storylines for visual storytelling

    2021 arXiv

  20. [28]

    Ting-Yao Hsu, Chieh-Yang Huang, Yen-Chia Hsu, and Ting-Hao 'Kenneth' Huang. 2019 b . http://arxiv.org/abs/1906.01764 Visual story post-editing

    2019 arXiv

  21. [29]

    Junjie Hu, Yu Cheng, Zhe Gan, Jingjing Liu, Jianfeng Gao, and Graham Neubig. 2020. http://arxiv.org/abs/1909.05316 What makes a good story? designing composite rewards for visual storytelling

    2020 arXiv

  22. [30]

    Lun Huang, Wenmin Wang, Jie Chen, and Xiao-Yong Wei. 2019. http://arxiv.org/abs/1908.06954 Attention on attention for image captioning

    2019 arXiv

  23. [31]

    Lawrence Zitnick, Devi Parikh, Lucy Vanderwende, Galley Michel, and Mitchell Margaret

    Ting-Hao Kenneth Huang, Francis Ferraro, Nasrin Mostafazadeh, Ishan Misra, Aishwarya Agrawal, Jacob Devlin, Ross Girshick, Xiaodong He, Pushmeet Kohli, Dhruv Batra, C. Lawrence Zitnick, Devi Parikh, Lucy Vanderwende, Galley Michel, and Mitchell Margaret. 2016. https://doi.org/...

    2016 doi

  24. [32]

    Parag Jain, Priyanka Agrawal, Abhijit Mishra, Mohak Sukhwani, Anirban Laha, and Karthik Sankaranarayanan. 2017. http://arxiv.org/abs/1707.05501 Story generation from sequence of independent short descriptions

    2017 arXiv

  25. [33]

    Yunjae Jung, Dahun Kim, Sanghyun Woo, Kyungsu Kim, Sungjin Kim, and In So Kweon. 2020. Hide-and-tell: Learning to bridge photo streams for visual storytelling. In Proceedings of the AAAI Conference on Artificial Intelligence, 07, pages 11213--11220

    2020

  26. [34]

    Jeong-Hee Kim. 2015. Understanding narrative inquiry: The crafting and analysis of stories as research. Sage publications

    2015

  27. [35]

    Taehyeong Kim, Min-Oh Heo, Seonil Son, Kyoung-Wha Park, and Byoung-Tak Zhang. 2019. http://arxiv.org/abs/1805.10973 Glac net: Glocal attention cascading networks for multi-image cued story generation

    2019 arXiv

  28. [36]

    Kingma and Jimmy Ba

    Diederik P. Kingma and Jimmy Ba. 2017. http://arxiv.org/abs/1412.6980 Adam: A method for stochastic optimization

    2017 arXiv

  29. [37]

    Shamma, Michael S

    Ranjay Krishna, Yuke Zhu, Oliver Groth, Justin Johnson, Kenji Hata, Joshua Kravitz, Stephanie Chen, Yannis Kalantidis, Li-Jia Li, David A. Shamma, Michael S. Bernstein, and Fei-Fei Li. 2016. http://arxiv.org/abs/1602.07332 Visual genome: Connecting language and vision using cr...

    2016 arXiv

  30. [38]

    Mike Lewis, Yinhan Liu, Naman Goyal, Marjan Ghazvininejad, Abdelrahman Mohamed, Omer Levy, Veselin Stoyanov, and Luke Zettlemoyer. 2020. https://doi.org/10.18653/v1/2020.acl-main.703 BART : Denoising sequence-to-sequence pre-training for natural language generation, translatio...

    2020 doi

  31. [39]

    Weixin Liang, Yuhui Zhang, Hancheng Cao, Binglu Wang, Daisy Ding, Xinyu Yang, Kailas Vodrahalli, Siyu He, Daniel Smith, Yian Yin, Daniel McFarland, and James Zou. 2024. http://arxiv.org/abs/2310.01783 Can large language models provide useful feedback on research papers? a larg...

    2024 arXiv

  32. [40]

    Chin-Yew Lin. 2004. Rouge: A package for automatic evaluation of summaries. In Text summarization branches out, pages 74--81

    2004

  33. [41]

    Belongie, Lubomir D

    Tsung - Yi Lin, Michael Maire, Serge J. Belongie, Lubomir D. Bourdev, Ross B. Girshick, James Hays, Pietro Perona, Deva Ramanan, Piotr Doll \' a r, and C. Lawrence Zitnick. 2014. http://arxiv.org/abs/1405.0312 Microsoft COCO: common objects in context . CoRR, abs/1405.0312

    2014 arXiv

  34. [42]

    Danyang Liu and Frank Keller. 2023. http://arxiv.org/abs/2303.17647 Detecting and grounding important characters in visual stories

    2023 arXiv

  35. [43]

    Danyang Liu, Mirella Lapata, and Frank Keller. 2024. http://arxiv.org/abs/2409.13555 Generating visual stories with grounded and coreferent characters

    2024 arXiv

  36. [44]

    Hanqing Liu, Jiacheng Yang, Chia-Hao Chang, Wei Wang, Hai-Tao Zheng, Yong Jiang, Hui Wang, Rui Xie, and Wei Wu. 2023 a . Aog-lstm: An adaptive attention neural network for visual storytelling. Neurocomputing, 552:126486

    2023

  37. [45]

    Haotian Liu, Chunyuan Li, Qingyang Wu, and Yong Jae Lee. 2023 b . http://arxiv.org/abs/2304.08485 Visual instruction tuning

    2023 arXiv

  38. [46]

    Hugo Liu and Push Singh. 2004. Conceptnet—a practical commonsense reasoning tool-kit. BT technology journal, 22(4):211--226

    2004

  39. [47]

    Yu Liu, Jianlong Fu, Tao Mei, and Chang Wen Chen. 2017. Let your photos talk: generating narrative paragraph for photo stream via bidirectional attention recurrent neural networks. In Proceedings of the Thirty-First AAAI Conference on Artificial Intelligence, AAAI'17, page 144...

    2017

  40. [48]

    Ilya Loshchilov and Frank Hutter. 2019. http://arxiv.org/abs/1711.05101 Decoupled weight decay regularization

    2019 arXiv

  41. [49]

    Jiasen Lu, Dhruv Batra, Devi Parikh, and Stefan Lee. 2019. http://arxiv.org/abs/1908.02265 Vilbert: Pretraining task-agnostic visiolinguistic representations for vision-and-language tasks

    2019 arXiv

  42. [50]

    Jiasen Lu, Caiming Xiong, Devi Parikh, and Richard Socher. 2017. http://arxiv.org/abs/1612.01887 Knowing when to look: Adaptive attention via a visual sentinel for image captioning

    2017 arXiv

  43. [51]

    Ron Mokady, Amir Hertz, and Amit H. Bermano. 2021. http://arxiv.org/abs/2111.09734 Clipcap: CLIP prefix for image captioning . CoRR, abs/2111.09734

    2021 arXiv

  44. [52]

    Munan Ning, Yujia Xie, Dongdong Chen, Zeyin Song, Lu Yuan, Yonghong Tian, Qixiang Ye, and Li Yuan. 2023. http://arxiv.org/abs/2305.12943 Album storytelling with iterative story-aware captioning and large language models

    2023 arXiv

  45. [53]

    OpenAI. 2023. http://arxiv.org/abs/2303.08774 Gpt-4 technical report

    2023 arXiv

  46. [54]

    Kishore Papineni, Salim Roukos, Todd Ward, and Wei-Jing Zhu. 2002. Bleu: a method for automatic evaluation of machine translation. In Proceedings of the 40th annual meeting of the Association for Computational Linguistics, pages 311--318

    2002

  47. [55]

    Cesc C Park and Gunhee Kim. 2015. https://proceedings.neurips.cc/paper_files/paper/2015/file/17e62166fc8586dfa4d1bc0e1742c08b-Paper.pdf Expressing an image stream with a sequence of natural sentences . In Advances in Neural Information Processing Systems, volume 28. Curran Ass...

    2015

  48. [56]

    my way of telling a story

    Shrimai Prabhumoye, Khyathi Raghavi Chandu, Ruslan Salakhutdinov, and Alan W Black. 2019. http://arxiv.org/abs/1906.06401 "my way of telling a story": Persona based grounded story generation

    2019 arXiv

  49. [57]

    Mengshi Qi, Jie Qin, Di Huang, Zhiqiang Shen, Yi Yang, and Jiebo Luo. 2021. https://doi.org/10.1145/3474085.3475236 Latent memory-augmented graph transformer for visual storytelling . In Proceedings of the 29th ACM International Conference on Multimedia, page 4892–4901, New Yo...

    2021

  50. [58]

    Alec Radford. 2018. Improving language understanding by generative pre-training. Open AI

    2018

  51. [59]

    Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, et al. 2021. Learning transferable visual models from natural language supervision. In International conference on machine learni...

    2021

  52. [60]

    Alec Radford, Jeffrey Wu, Rewon Child, David Luan, Dario Amodei, Ilya Sutskever, et al. 2019. Language models are unsupervised multitask learners. OpenAI blog, 1(8):9

    2019

  53. [61]

    Colin Raffel, Noam Shazeer, Adam Roberts, Katherine Lee, Sharan Narang, Michael Matena, Yanqi Zhou, Wei Li, and Peter J. Liu. 2020. http://jmlr.org/papers/v21/20-074.html Exploring the limits of transfer learning with a unified text-to-text transformer . Journal of Machine Lea...

    2020

  54. [62]

    Hareesh Ravi, Kushal Kafle, Scott Cohen, Jonathan Brandt, and Mubbasir Kapadia. 2021. Aesop: Abstract encoding of stories, objects, and pictures. In Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), pages 2052--2063

    2021

  55. [63]

    Rennie, Etienne Marcheret, Youssef Mroueh, Jarret Ross, and Vaibhava Goel

    Steven J. Rennie, Etienne Marcheret, Youssef Mroueh, Jarret Ross, and Vaibhava Goel. 2017. http://arxiv.org/abs/1612.00563 Self-critical sequence training for image captioning

    2017 arXiv

  56. [64]

    Thibault Sellam, Dipanjan Das, and Ankur Parikh. 2020. https://doi.org/10.18653/v1/2020.acl-main.704 BLEURT : Learning robust metrics for text generation . In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 7881--7892, Online. Ass...

    2020 doi

  57. [65]

    Claude Elwood Shannon. 1948. A mathematical theory of communication. The Bell system technical journal, 27(3):379--423

    1948

  58. [66]

    Marko Smilevski, Ilija Lalkovski, and Gjorgji Madjarov. 2018. https://doi.org/10.1007/978-3-030-00825-3_13 Stories for Images-in-Sequence by Using Visual and Narrative Components , page 148–159. Springer International Publishing

    2018 doi

  59. [67]

    Yingjin Song, Denis Paperno, and Albert Gatt. 2024. http://arxiv.org/abs/2408.06259 Context-aware visual storytelling with visual prefix tuning and contrastive learning

    2024 arXiv

  60. [68]

    Jing Su, Qingyun Dai, Frank Guerin, and Mian Zhou. 2020. http://arxiv.org/abs/2012.02128 Bert-hlstms: Bert and hierarchical lstms for visual storytelling

    2020 arXiv

  61. [69]

    Yixuan Su, Tian Lan, Yan Wang, Dani Yogatama, Lingpeng Kong, and Nigel Collier. 2022. http://arxiv.org/abs/2202.06417 A contrastive framework for neural text generation

    2022 arXiv

  62. [70]

    Aditya K Surikuchi, Sandro Pezzelle, and Raquel Fernández. 2023. http://arxiv.org/abs/2310.17770 Groovist: A metric for grounding objects in visual storytelling

    2023 arXiv

  63. [71]

    Kumiko Tanaka and Shunsuke Aihara. 2015. https://doi.org/10.1162/COLI_a_00228 Computational constancy measures of T exts --- Y ule`s k and r \'e nyi`s entropy . Computational Linguistics, 41(3)

    2015 doi

  64. [72]

    Gomez, Lukasz Kaiser, and Illia Polosukhin

    Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N. Gomez, Lukasz Kaiser, and Illia Polosukhin. 2017. http://arxiv.org/abs/1706.03762 Attention is all you need . CoRR, abs/1706.03762

    2017 arXiv

  65. [73]

    Lawrence Zitnick, and Devi Parikh

    Ramakrishna Vedantam, C. Lawrence Zitnick, and Devi Parikh. 2015. http://arxiv.org/abs/1411.5726 Cider: Consensus-based image description evaluation

    2015 arXiv

  66. [74]

    Oriol Vinyals, Alexander Toshev, Samy Bengio, and Dumitru Erhan. 2015. http://arxiv.org/abs/1411.4555 Show and tell: A neural image caption generator

    2015 arXiv

  67. [75]

    Eileen Wang, Caren Han, and Josiah Poon. 2022. http://arxiv.org/abs/2205.03774 Rovist:learning robust metrics for visual storytelling

    2022 arXiv

  68. [76]

    Eileen Wang, Soyeon Caren Han, and Josiah Poon. 2024. http://arxiv.org/abs/2402.00319 Sco-vist: Social interaction commonsense knowledge-based visual storytelling

    2024 arXiv

  69. [77]

    Jing Wang, Jianlong Fu, Jinhui Tang, Zechao Li, and Tao Mei. 2018 a . Show, reward and tell: automatic generation of narrative paragraph from photo stream by adversarial training. In Proceedings of the Thirty-Second AAAI Conference on Artificial Intelligence and Thirtieth Inno...

    2018

  70. [78]

    Ruize Wang, Zhongyu Wei, Piji Li, Qi Zhang, and Xuanjing Huang. 2020. https://doi.org/10.1609/aaai.v34i05.6455 Storytelling from an image stream using scene graphs . Proceedings of the AAAI Conference on Artificial Intelligence, 34(05):9185--9192

    2020 doi

  71. [79]

    Xin Wang, Wenhu Chen, Yuan-Fang Wang, and William Yang Wang. 2018 b . http://arxiv.org/abs/1804.09160 No metrics are perfect: Adversarial reward learning for visual storytelling

    2018 arXiv

  72. [80]

    Yuechen Wang, Wengang Zhou, Zhenbo Lu, and Houqiang Li. 2023. https://doi.org/10.1145/3581783.3612179 Text-only training for visual storytelling . In Proceedings of the 31st ACM International Conference on Multimedia, New York, NY, USA. Association for Computing Machinery

    2023

  73. [81]

    Sean Wu, Michael Koo, Lesley Blum, Andy Black, Liyo Kao, Fabien Scalzo, and Ira Kurtz. 2023. http://arxiv.org/abs/2308.04709 A comparative study of open-source large language models, gpt-4 and claude 2: Multiple-choice test taking in nephrology

    2023 arXiv

  74. [82]

    Chunpu Xu, Min Yang, Chengming Li, Ying Shen, Xiang Ao, and Ruifeng Xu. 2021. Imagine, reason and write: Visual storytelling with graph knowledge and relational reasoning. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 35, pages 3022--3029

    2021

  75. [83]

    Kelvin Xu, Jimmy Ba, Ryan Kiros, Kyunghyun Cho, Aaron Courville, Ruslan Salakhutdinov, Richard Zemel, and Yoshua Bengio. 2016. http://arxiv.org/abs/1502.03044 Show, attend and tell: Neural image caption generation with visual attention

    2016 arXiv

  76. [84]

    Peng Xu, Mostofa Patwary, Mohammad Shoeybi, Raul Puri, Pascale Fung, Anima Anandkumar, and Bryan Catanzaro. 2020. https://doi.org/10.18653/v1/2020.emnlp-main.226 MEGATRON - CNTRL : Controllable story generation with external knowledge using large-scale language models . In Pro...

    2020 doi

  77. [85]

    Dingyi Yang and Qin Jin. 2023. https://doi.org/10.18653/v1/2023.acl-long.619 Attractive storyteller: Stylized visual storytelling with unpaired text . In Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 1105...

    2023 doi

  78. [86]

    Li Yang, Zhiding Xiao, Wenxin Huang, and Xian Zhong. 2025. https://aclanthology.org/2025.coling-main.266/ S tory LL a VA : Enhancing visual storytelling with multi-modal large language models . In Proceedings of the 31st International Conference on Computational Linguistics, p...

    2025

  79. [87]

    Pengcheng Yang, Fuli Luo, Peng Chen, Lei Li, Zhiyi Yin, Xiaodong He, and Xu Sun. 2019. Knowledgeable storyteller: A commonsense-driven generative model for visual storytelling. In IJCAI, volume 3, page 7

    2019

  80. [88]

    Xu Yang, Kaihua Tang, Hanwang Zhang, and Jianfei Cai. 2018. http://arxiv.org/abs/1812.02378 Auto-encoding scene graphs for image captioning

    2018 arXiv

  81. [89]

    Lili Yao, Nanyun Peng, Ralph Weischedel, Kevin Knight, Dongyan Zhao, and Rui Yan. 2019. http://arxiv.org/abs/1811.05701 Plan-and-write: Towards better automatic storytelling

    2019 arXiv

  82. [90]

    Ting Yao, Yingwei Pan, Yehao Li, and Tao Mei. 2018. http://arxiv.org/abs/1809.07041 Exploring visual relationship for image captioning

    2018 arXiv

  83. [91]

    Licheng Yu, Mohit Bansal, and Tamara L. Berg. 2017. http://arxiv.org/abs/1708.02977 Hierarchically-attentive rnn for album summarization and storytelling

    2017 arXiv

  84. [92]

    Youngjae Yu, Jiwan Chung, Heeseung Yun, Jongseok Kim, and Gunhee Kim. 2021. Transitional adaptation of pretrained models for visual storytelling. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 12658--12668

    2021

  85. [93]

    Jingyi Zhang, Jiaxing Huang, Sheng Jin, and Shijian Lu. 2024. http://arxiv.org/abs/2304.00685 Vision-language models for vision tasks: A survey

    2024 arXiv

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.