Pith. sign in

REVIEW 6 major objections 6 minor 63 references

UnMA-CapSumT: Unified and Multi-Head Attention-driven Caption Summarization Transformer

T0 review · 6 major / 6 minor · reviewed 2026-08-11 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read This paper claims that a transformer summarizer can combine factual, romantic, and humorous image captions into one coherent caption, using pointer-generator and coverage mechanisms to avoid repetition and out-of-vocabulary errors.

desk verdict The integrated caption-summarization task is genuinely new, but the paper never measures it end to end, so the central claim rests on an untested assumption. read the letter →

arxiv 2412.11836 v1 pith:3FFYFODF submitted 2024-12-16 cs.CV

Dhruv Sharma , Chhavi Dhiman , Dinesh Kumar This is my paper · ORCID
classification cs.CV
keywords imagecaptioningstylizedtextsummarizationtransformerpointer-generatornetworkcoveragemechanismfastattention
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper tries to establish that image captioning can be reframed as a summarization problem: instead of generating separate factual, romantic, and humorous captions for an image, a single sentence can integrate all three. It proposes a two-phase framework in which one model (MAA-FIC) produces a factual caption, another (SF-Bi-ALSTM) produces romantic and humorous captions, and a transformer summarizer (UnMHA-ST) compresses these into one coherent caption. If correct, this would give users one description that is simultaneously accurate and stylistically rich, removing the need to choose among competing captions. The claim is supported by component-wise results on Flickr8K and a FlickrStyle10K subset, plus ablations on the summarizer.

What carries the argument

The load-bearing component is the UnMHA-ST summarization transformer, which combines a standard multi-head attention encoder-decoder with a unified attention module that keeps the query as a sequence rather than collapsing it into a single vector, letting the model attend to both the input captions and the target summary. It is paired with fTA-WE, a fastText-based embedding that augments word vectors with subword information and an attention-weighted context vector, and with a pointer-generator network plus coverage mechanism that lets the decoder copy rare words from the source and penalizes repeatedly attending to the same positions.

What would settle it

A concrete test: run a held-out set of images through the full MAA-FIC + SF-Bi-ALSTM + UnMHA-ST pipeline, collect the generated summarized captions, and have human annotators judge whether each caption is simultaneously factually correct, romantic, and humorous. If the final summaries lose factual accuracy or fail to reflect both styles on a substantial fraction of images, especially images where the captioning models produce repeated or out-of-vocabulary words, the central claim would be refuted.

Watch

Extended reading notes

Core claim

The central claim is that a transformer-based abstractive summarizer, UnMHA-ST, can take the outputs of a factual captioner and a stylized captioner and produce a single caption that preserves factual content while incorporating romantic and humorous elements. The paper argues this works because the summarizer's unified attention module captures interactions across the caption sources, its multi-head attention captures different representation subspaces, and a pointer-generator with coverage avoids out-of-vocabulary and repetition errors. In the paper's own framing, this is the first framework to provide a description that integrates different captioning methods for the same image.

Load-bearing premise

The summarizer is trained on clean human-written paragraphs of factual, romantic, and humorous captions, but at inference it must summarize the imperfect outputs of the two captioning models; the paper does not evaluate the integrated pipeline end-to-end, so the central claim depends on the summarizer's performance transferring from clean training inputs to noisy model outputs.

Editorial extensions

If this is right

  • If the framework works as claimed, users would no longer have to choose between a factual caption and a stylized one; a single description could carry both content and tone.
  • The pointer-generator and coverage components are the mechanism the paper credits for reducing out-of-vocabulary and repetition problems in the generated summaries.
  • The summarizer's ROUGE results improve when the unified attention transformer is combined with fTA-WE embeddings, suggesting that subword-aware embeddings help summarize rare words.
  • Because the two captioning models are evaluated separately and outperform their listed baselines, the integrated framework would inherit better-quality factual and stylized inputs.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • A direct test of the paper's central claim would be an end-to-end evaluation on held-out images, measuring whether the final summaries remain factually correct and stylistically faithful when fed model-generated captions rather than human-written paragraphs; the paper does not provide this.
  • The same two-phase design could be applied to other multi-source description tasks, such as merging captions from different viewpoints or combining factual and emotional descriptions, though the paper does not explore these settings.
  • If fTA-WE's subword attention helps in this low-resource summarization setting, it may also benefit other tasks where rare or out-of-vocabulary words dominate, such as domain-specific summarization.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

6 major / 6 minor

Summary. The paper proposes a two-phase image captioning framework, UnMA-CapSumT. In phase 1, a factual caption is produced by MAA-FIC and romantic/humorous captions by SF-Bi-ALSTM. In phase 2, a transformer-based summarizer UnMHA-ST, using fastText-attention embeddings (fTA-WE) and a pointer-generator with coverage, is intended to merge these three captions into a single styled, coherent caption. The authors evaluate the components on Flickr8K and FlickrStyle10K and ablate the summarizer on a constructed paragraph corpus. The central claim is that the integrated framework generates a single caption combining factual, romantic, and humorous content.

Significance. The task of combining multiple style-specific captions into one coherent caption is a reasonable and underexplored direction; if the system worked end-to-end, it would be a useful step. The paper provides a public code repository, component-level comparisons against several baselines, and a clear ablation from a baseline transformer to the full summarizer. However, the claimed integration is never quantitatively evaluated: the summarizer is only tested on clean human-written triples, while the deployed pipeline feeds it noisy model outputs. Consequently, the significance of the central contribution is not established by the present experiments.

major comments (6)
  1. [Section 4.2.4 / Tables I-III / Figure 9] The central claim is not evaluated end-to-end. Table III evaluates UnMHA-ST on a constructed corpus of human-written factual/romantic/humorous reference captions, while Figure 9 shows the full pipeline only qualitatively. At inference the summarizer receives MAA-FIC and SF-Bi-ALSTM outputs that contain errors (e.g., "mobile" instead of "camera," Section 4.2.1) and have BLEU-4 of 26.8 and BLEU-1 around 29 (Tables I-II). Because the paper's title and abstract claim an integrated framework, the absence of a quantitative experiment that scores the output of the full pipeline against human-written combined captions leaves the core claim unsupported.
  2. [Section 3.1.2 / Eqs. (3)-(14)] The MAA attention equations contain undefined symbols and operations. For instance, Eq. (3) uses \otimes, \oplus, and \mathcal{I}^T without definitions; Eqs. (7)-(9) introduce \phi_c, \phi_s, and \mathcal{X} that are never specified; Eq. (14) mixes \mathcal{X} and \mathcal{W} with no clear relation to Eq. (10). These equations define one of the two caption generators, so they must be precise and reproducible.
  3. [Section 3.2.1 / Eqs. (18)-(30)] The SF-Bi-ALSTM formulation is internally inconsistent. Eq. (19) defines S_x = (1/N)|S S^T| with S on both sides and N undefined; Eq. (20) defines \tilde{x}_t as a gated version of the input, but Eq. (25) uses \tilde{x}_t as the input gate; the roles of the shared matrices Q_x, S_x, and \Lambda_x across romantic and humorous styles are not stated. Without a correct formulation, the romantic/humorous captioning results in Table II are not reproducible.
  4. [Section 3.3 / Eqs. (32)-(37)] The fTA-WE contribution is not fully specified. The text says the embedding uses key, query, and value matrices (as in Fig. 4), but Eqs. (35)-(37) do not contain a softmax attention weight or K/Q/V projections; c_v and c_v^{att} are both used without a clear relationship. This makes the proposed embedding difficult to implement and compare.
  5. [Section 3.4 / Eqs. (41)-(54)] Several terms in the summarizer equations are undefined or ambiguous. Eq. (48) uses h_t^* before it is defined; Eq. (54) contains an undefined index i and uses \Omega_t' without specifying whether it is the same as \Omega_t; the coverage vector in Eq. (52) is summed over \Omega_{t'}, but the coverage loss in Eq. (54) is not derived from this vector. These equations are central to the claimed solution to OOV and repetition, so they need to be rewritten carefully.
  6. [Section 4 / Table III] The construction of the summarization dataset is underspecified. The authors state that about 7000 paragraphs were used, but they do not explain how the target summary sentences were created, how train/test splits were made, or whether the same images appear in both captioning and summarization sets. Without this information, the ROUGE scores in Table III cannot be interpreted.
minor comments (6)
  1. [Abstract / Section 1] The abstract and Section 1 use both UnMA-CapSumT and UnMHA-ST; please spell out the relationship at first use and use consistent notation throughout.
  2. [Eq. (27)] Equation (27) uses C without definition; either define it or use a different symbol.
  3. [Section 4.2.3] The authors state that the generated descriptions have "strong correlation with human evaluation," but no human evaluation is reported; please add the evaluation or temper the claim.
  4. [Figure 7] Figure 7 does not clearly distinguish model-generated captions from reference captions; please label each caption with its source.
  5. [References / Table II] The reference list contains apparent errors: Refs. [22] and [58] refer to the same work by Jia et al., and Table II and the surrounding text assign StyleNet and SF-LSTM to inconsistent reference numbers; please check all citation numbers.
  6. [Section 4.1] The implementation details are incomplete for reproducibility: no random seed, no hardware details, no optimizer hyperparameters for the summarizer beyond epochs and batch size, and no description of how the 400K GloVe vocabulary was obtained or truncated.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: components are separately trained and tested on public benchmarks, and the end-to-end evaluation gap is a validation limitation, not a circular derivation.

full rationale

The claimed derivation chain is not circular. Phase-1 components (MAA-FIC and SF-Bi-ALSTM) are trained and evaluated on public Flickr8K and FlickrStyle10K corpora (Tables I, II) against external baselines; their outputs are not defined in terms of the final summarized caption. Phase-2 UnMHA-ST is a standard transformer plus pointer-generator and coverage architecture (Eqs. 38-55), trained on a constructed paragraph corpus and ablated in Table III; the ROUGE scores are computed on held-out summaries, not on the training fit. The central weakness is that no end-to-end experiment scores the full pipeline (summarizer fed with generated MAA-FIC/SF-Bi-ALSTM outputs); Section 4.2.4 and Fig. 9 provide only qualitative examples. That is a validation gap, not a circular reduction: the summarizer's training targets are not the same objects as its deployed inputs, so no prediction is forced by construction. The only self-citation is [26], used as a comparative baseline in Table I and related work; it does not carry the derivation of MAA-FIC or the integrated framework.

Assumptions & free parameters 6 free parameters · 5 assumptions · 0 invented entities

The central claim depends on several domain assumptions about the validity of the constructed summarization corpus and the transferability of the summarizer to noisy inputs. The free parameters are standard hyperparameters, but several (like lambda) are unreported, and no systematic search is described.

free parameters (6)
  • Learning rate for MAA-FIC and SF-Bi-ALSTM = 2e-5
    Chosen by hand; no learning rate search reported, and results may depend on it.
  • Batch sizes = 64 (captioning), 96 (stylized), 4 (summarizer)
    Chosen by hand; no justification for these values.
  • Number of training epochs = 70 (MAA-FIC), 60 (SF-Bi-ALSTM), 100 (UnMHA-ST)
    Chosen by hand; no early stopping or validation-based selection described.
  • Embedding dimension and vocabulary size = 300 embedding, 400K vocabulary
    Standard choices, but not justified for the proposed fTA-WE model.
  • Coverage loss balancing parameter lambda = not reported
    Equation (54) introduces lambda but its value is not given, affecting the final loss.
  • Context window and subword n-gram size in fTA-WE = not reported
    The context window b and n-gram size are not specified, yet they determine the subword embeddings.
assumptions (5)
  • domain assumption The constructed three-sentence paragraphs (factual, romantic, humorous) are valid ground truth for learning to produce a single combined caption.
    The paper assumes that fusing the three sentences into one is a well-defined task, but the target output is not defined independently.
  • domain assumption ROUGE scores reflect the quality of the generated summarized captions.
    ROUGE is used as the primary metric for summarization quality without human evaluation or correlation analysis.
  • domain assumption Faster R-CNN and Inception-V3 provide sufficiently accurate object and spatial features for captioning.
    The paper relies on pretrained detectors and does not analyze their failure modes (e.g., 'mobile' instead of 'camera').
  • standard math The transformer pointer-generator and coverage mechanism from See et al. work as expected when applied to caption fusion.
    The paper invokes [43] without re-deriving or verifying its behavior in this new context.
  • domain assumption The unified attention module from Daniluk et al. is applicable to the caption summarization setting.
    The authors cite [54] for the iterative attention idea, but do not test whether it benefits the caption domain.

how reviews work

0 comments
Cite this review

Pith. "Pith review of UnMA-CapSumT: Unified and Multi-Head Attention-driven Caption Summarization Transformer." pith.science (2026). https://pith.science/paper/3FFYFODF

@misc{pith2026241211836,
  author       = {Pith},
  title        = {Pith review of: UnMA-CapSumT: Unified and Multi-Head Attention-driven Caption Summarization Transformer},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/3FFYFODF}},
  note         = {Machine review of arXiv:2412.11836}
}
read the original abstract

Image captioning is the generation of natural language descriptions of images which have increased immense popularity in the recent past. With this different deep-learning techniques are devised for the development of factual and stylized image captioning models. Previous models focused more on the generation of factual and stylized captions separately providing more than one caption for a single image. The descriptions generated from these suffer from out-of-vocabulary and repetition issues. To the best of our knowledge, no such work exists that provided a description that integrates different captioning methods to describe the contents of an image with factual and stylized (romantic and humorous) elements. To overcome these limitations, this paper presents a novel Unified Attention and Multi-Head Attention-driven Caption Summarization Transformer (UnMA-CapSumT) based Captioning Framework. It utilizes both factual captions and stylized captions generated by the Modified Adaptive Attention-based factual image captioning model (MAA-FIC) and Style Factored Bi-LSTM with attention (SF-Bi-ALSTM) driven stylized image captioning model respectively. SF-Bi-ALSTM-based stylized IC model generates two prominent styles of expression- {romance, and humor}. The proposed summarizer UnMHA-ST combines both factual and stylized descriptions of an input image to generate styled rich coherent summarized captions. The proposed UnMHA-ST transformer learns and summarizes different linguistic styles efficiently by incorporating proposed word embedding fastText with Attention Word Embedding (fTA-WE) and pointer-generator network with coverage mechanism concept to solve the out-of-vocabulary issues and repetition problem. Extensive experiments are conducted on Flickr8K and a subset of FlickrStyle10K with supporting ablation studies to prove the efficiency and efficacy of the proposed framework.

Figures

Figures reproduced from arXiv: 2412.11836 by the authors.

Figure 1
Figure 1. Block Diagram Representation of the proposed 𝑈𝑛𝑀𝐴 −CapSum Transformer based Captioning Framework incorrect representation of the style. This may be due to poor learning knowledge of factual content and its associated linguistic styles. To the best of our knowledge, there are no such works in literature that provide single-sentence summarized descriptions incorporating factual, romantic, and humorous content. To addr… view at source ↗
Figure 2
Figure 2. Proposed Methodology: Phase-1: MAA-FIC module and Stylized Image Captioning Module, Phase 2: 𝑈𝑛𝑀𝐻𝐴 − 𝑆𝑇: Unified and Multi-head Attention based Summarization Transformer dimensional vector of top 𝑛-boxes as the region of objects. The localization of objects is performed to extract spatial relationships between objects. The 𝑛-objects detected using Faster R-CNN [49] for each image are fed to Inception-V3 [50] and out… view at source ↗
Figure 3
Figure 3. (a) Soft-attention Mechanism, (b) Adaptive Attention Mechanism, and (c) Proposed Modified Adaptive Attention where, 𝔻𝕋 is the 𝑡 − 𝑑𝑖𝑚𝑒𝑛𝑠𝑖𝑜𝑛𝑎𝑙 vector of the spatial location of each object. The features extracted from the Faster R-CNN module and Inception-V3 module are concatenated. Therefore, each concatenated feature vector consists of a detected object feature vector (Φ𝑖) and the feature vector obtained from the l… view at source ↗
Figures from the paper (3 more)
Figure 5
Figure 5. Figure 5: Proposed Unified Attention Module 𝒶𝑤𝑖 = exp (𝑣𝑤0 𝑇 𝑐𝑣) (35) 𝒸𝑣𝑎𝑡𝑡 = ∑ 𝒶𝑤𝑖 𝑖∈[−𝑏,𝑏]−{0} (∑ℯ𝑗∈𝕊𝑤 𝓊̃ℯ𝑗 ) (36) The probability of the masked word is given by: 𝓅(𝑤0|𝑊[−𝑏,𝑏]−{0}) ∝ exp (𝓊̃𝑤0 𝑇 𝒸𝑣𝑎𝑡𝑡 ) (37) The attended vector is linearly transformed to obtain the transformed…
Figure 6
Figure 6. Figure 6: (a) Accuracy and (b) Loss curves for the proposed MAA-FIC TABLE I RESULTS ON FLICKR8K DATASET Model B-1 B-2 B-3 B-4 M LSTM [21] 66.0 42.0 27.0 18.0 - g-LSTM [58] 64.7 45.9 31.8 21.6 20.60 Log Bilinear [59] 65.6 42.4 27.7 17.7 17.31 SCA-CNN [6] 68.2 49.6 35.9 25.8 - Har…
Figure 7
Figure 7. Figure 7: Qualitative results obtained for the proposed 𝑈𝑛𝑀𝐴 −CapSumT (a) UnMA-CapSumT Factual + Humorous (b) UnMA-CapSumT Factual + Romantic (c) UnMA-CapSumT Factual + Romantic + Humorous The qualitative results presented in Fig.7 for the generation of romantic and humorous pro…

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

63 extracted references · 54 canonical work pages

  1. [1]

    Bottom -up and top-down attention for image captioning and visual question answering,

    P. Anderson, X. He, C. Buehler, D. Teney, M. Johnson, S. Gould and L. Zhang, "Bottom -up and top-down attention for image captioning and visual question answering," in IEEE Conference on Computer Vision and Pattern Recognition, Salt Lake City, 2018

    work page 2018

  2. [2]

    Entangled transformer for image captioning,

    G. Li, L. Zhu, P. Liu and Y. Yang, "Entangled transformer for image captioning," in IEEE Conference on Computer Vision and Pattern Recognition, Long Beach, CA, 2019

    work page 2019

  3. [3]

    Automatic alt -text: Computer-generated image descriptions for blind users on a social network service,

    S. Wu, J. Wieland, O. Farivar and J. Schiller, "Automatic alt -text: Computer-generated image descriptions for blind users on a social network service," in ACM Conference on Computer-Supported Cooperative Work and Social Computing, Portland, Oregon, 2017

    work page 2017

  4. [4]

    Vizwiz grand challenge: Answering visual questions from blind people,

    D. Gurari, Q. Li, A. Stangl, A. Guo, C. Lin, K. Grauman, J. Luo and J. Bigham, "Vizwiz grand challenge: Answering visual questions from blind people," in e IEEE Conference on Computer Vision and Pattern Recognition, 2018

    work page 2018

  5. [5]

    VQA: Visual question answering,

    S. Antol, A. Agrawal, J. Lu, M. Mitchell, D. Batra, C. Zitnick and D. Parikh, "VQA: Visual question answering," in IEEE International Conference on Computer Vision, 2015

    work page 2015

  6. [6]

    SCA-CNN: Spatial and Channel-wise Attention in Convolutional Networks for Image Captioning

    L. Chen, H. Zhang, J. Xiao, L. Nie, J. Shao, W. Liu and T.-S. Chua, "SCA-CNN: Spatial and Channel-wise Attention in Convolutional Networks for Image Captioning," in arXiv:1611.05594v2, 2017

    work page Pith review arXiv 2017

  7. [7]

    Image captioning using DenseNet network and adaptive attention,

    Z. Deng, Z. Jiang, R. Lan, W. Huang and X. Luo, "Image captioning using DenseNet network and adaptive attention," Signal Processing: Image Communication, vol. 85, no. 12, July 2020

    work page 2020

  8. [8]

    High-Order Interaction Learning for Image Captioning,

    Y. Wang, N. Xu, A.-A. Liu, W. Li and Y. Zhang, "High-Order Interaction Learning for Image Captioning," IEEE TRANSACTIONS ON CIRCUITS AND SYSTEMS FOR VIDEO TECHNOLOGY, 2021

    work page 2021

Show all 63 references
  1. [9]

    Stylenet: Generating attractive visual captions with styles,

    C. Gan, Z. Gan, X. He and J. Gao, "Stylenet: Generating attractive visual captions with styles," in IEEE Conference on Computer Vision and Pattern Recognition, Honolulu, Hawaii, 2017

    2017

  2. [10]

    Similar Scenes arouse Similar Emotions: Parallel Data Augmentation for Stylized Image Captioning,

    G. Li, Y. Zhai, Z. Lin and Y. Zhang, "Similar Scenes arouse Similar Emotions: Parallel Data Augmentation for Stylized Image Captioning," in ACM Multimedia Conference, Virtual Event, China, 2021

    2021

  3. [11]

    “Factual

    T. Chen, Z. Zhang, Q. You, C. Fang, Z. Wang, H. Jin and J. Luo, "“Factual” or “Emotional”: Stylized Image Captioning with Adaptive Learning and Attention," in European Conference on Computer Vision, Munich, Germany, 2018

    2018

  4. [12]

    MSap: Multi -Style Image Captioning With Unpaired Stylized Text,

    L. Guo, J. Liu, P. Yao, J. Li and H. Lu, "MSap: Multi -Style Image Captioning With Unpaired Stylized Text," in 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) , 2019

    2019

  5. [13]

    SummaRuNNer: A Recurrent Neural Network based Sequence Model for Extractive Summarization of Documents,

    R. Nallapatti, F. Zhai and B. Zhou, "SummaRuNNer: A Recurrent Neural Network based Sequence Model for Extractive Summarization of Documents," in Thirty-First AAAI Conference on Artificial Intelligence , 2017

    2017

  6. [14]

    Abstractive text summarization using sequence-tosequence rnns and beyond,

    R. Nallapatti, B. Zhou, C. N. dos santos, C. Gulcehre and B. Xiang, "Abstractive text summarization using sequence-tosequence rnns and beyond," in arXiv:1602.06023, 2016

    2016 arXiv

  7. [15]

    From image descriptions to visual denotations: New similarity metrics for semantic inference over event descriptions,

    P. Young, A. Lai, M. Hodosh, and J. Hockenmaier, "From image descriptions to visual denotations: New similarity metrics for semantic inference over event descriptions," Transactions of the Association for Computational Linguistics, vol. 2, pp. 67-78, 2014

    2014

  8. [16]

    Corpus -guided sentence generation of natural images,

    Y. Yang, C. L. Teo, H. Daume and Y. Aloimono, "Corpus -guided sentence generation of natural images," in Empirical Methods in Natural Language Processing, Edinburgh United Kingdom, 2011

    2011

  9. [17]

    Generating image descriptions from computer vision detections,

    M. Mitchell, J. Dodge, A. Goyal, K. Yamaguchi, K. Stratos, X. Han, A. Mensch, A. Berg, T. Berg and H. Daume, "Generating image descriptions from computer vision detections," in Proceedings of the 13th Conference of the European Chapter of the Association for Computational Ling...

    2012

  10. [18]

    Every picture tells a story: Generating sentences from images,

    A. Farhadi, M. Hejrati, M. Sadeghi, P. Young, C. Rashtchian, J. Hockenmaier and D. Forsyth, "Every picture tells a story: Generating sentences from images," in Proceedings of the European Conference on Computer Vision, Crete, Greece , 2010

    2010

  11. [19]

    Framing image description as a ranking task: Data, models and evaluation metrics,

    M. Hodosh, P. Young and J. Hockenmaier, "Framing image description as a ranking task: Data, models and evaluation metrics," Journal of Artificial Intelligence Research, vol. 47, pp. 853-899, August 2013

    2013

  12. [20]

    Deep captioning with multimodal recurrent neural networks (m-rnn),

    J. Mao, W. Xu, Y. Yang, J. Wang, Z. Huang and A. Yuille, "Deep captioning with multimodal recurrent neural networks (m-rnn)," in arXiv:1412.6632, 2015

    2015 arXiv

  13. [21]

    Show and Tell: A Neural Image Caption Generator,

    O. Vinyals, A. Toshev, S. Bengio and D. Erhan, "Show and Tell: A Neural Image Caption Generator," in IEEE Conference on Computer Vision and Pattern Recognition (CVPR) , Boston, USA, 2015

    2015

  14. [22]

    Guiding the long -short term memory model for image caption generation,

    X. Jia, E. Gavves, B. Fernando and T. Tuytelaars, "Guiding the long -short term memory model for image caption generation," in IEEE International Conference on Computer Vision, Santiago, Chile, 2015

    2015

  15. [23]

    Long-Term Recurrent Convolutional Networks for Visual Recognition and Description,

    J. Donahue, L. Hendricks, M. Rohrbach, . S. Venugopalan, . S. Guadarrama, K. Saenko and . T. Darrell, " Long-Term Recurrent Convolutional Networks for Visual Recognition and Description," IEEE Transactions on Pattern Analysis and Machine Intelligence, pp. 677-691, 2015

    2015

  16. [24]

    Image Captioning with Deep Bidirectional LSTMs,

    C. Wang, H. Yang and C. Meinel, "Image Captioning with Deep Bidirectional LSTMs," ACM Transactions on Multimedia Computing, Communications, and Applications, vol. 14, no. 2s, pp. 1-20, 2018

    2018

  17. [25]

    Knowing When to Look: Adaptive Attention via A Visual Sentinel for Image Captioning,

    J. Lu, C. Xiong, D. Parikh and R. Socher, "Knowing When to Look: Adaptive Attention via A Visual Sentinel for Image Captioning," in arXiv:1612.01887v2, 2017

    2017 arXiv

  18. [26]

    Automated Image Caption Generation Framework using Adaptive Attention and Bi-LSTM,

    D. Sharma, C. Dhiman and D. Kumar, "Automated Image Caption Generation Framework using Adaptive Attention and Bi-LSTM," in IEEE Delhi Section Conference (DELCON), Delhi, 2022

    2022

  19. [27]

    Exploring region relationships implicitly: Image captioning with visual relationship attention,

    Z. Zhang, Q. Wu, Y. Wang and F. Chen, "Exploring region relationships implicitly: Image captioning with visual relationship attention," Image and Vision Computing, vol. 109, May 2021

    2021

  20. [28]

    Image captioning with semantic attention,

    Q. You, H. Jin, Z. Wang, C. Fang and J. Luo, "Image captioning with semantic attention," in IEEE Conference on Computer Vision and Pattern Recognition, Caesars Palace, 2016

    2016

  21. [29]

    DAA: Dual LSTMs with adaptive attention for image captioning,

    F. Xiao, X. Gong, Y. Zhang, Y. Shen, J. Li and X. Gao, "DAA: Dual LSTMs with adaptive attention for image captioning," Neurocomputing, vol. 364, pp. 322-329, 2019

    2019

  22. [30]

    Task -Adaptive Attention for Image Captioning,

    C. Yan, Y. Hao, L. Li, J. Yin, A. Liu, Z. Mao, Z. Chen and X. Gao, "Task -Adaptive Attention for Image Captioning," IEEE Transactions on Circuits and Systems for Video Technology, vol. 32, no. 1, pp. 43 -51, 2021

    2021

  23. [31]

    A New Attention -Based LSTM for Image Captioning,

    F. Xiao, W. Xue, Y. Shen and X. Gao, "A New Attention -Based LSTM for Image Captioning," Neural Process Letters, vol. 54, p. 3157–3171, Feburary 2022

    2022

  24. [32]

    Semstyle: Learning to generate stylised image captions using unaligned text,

    A. Mathews, L. Xie and X. He, "Semstyle: Learning to generate stylised image captions using unaligned text," in IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) , 2018

    2018

  25. [33]

    MemCap: Memorizing Style Knowledge for Image Captioning,

    W. Zhao, X. Wu and X. Zhang, "MemCap: Memorizing Style Knowledge for Image Captioning," in AAAI Conference on Artificial Intelligence, 2020

    2020

  26. [34]

    SentiCap: Generating Image Descriptions with Sentiments,

    A. P. Mathews, L. Xie and X. He, "SentiCap: Generating Image Descriptions with Sentiments," in AAAI'16: Proceedings of the Thirtieth AAAI Conference on Artificial Intelligence, Phoenix, Arizona, 2016

    2016

  27. [35]

    Engaging Image Captioning via Personality,

    K. Shuster, S. Humeau, H. Hu, A. Bordes and J. Weston, "Engaging Image Captioning via Personality," in arXiv:1810.10665, 2018

    2018 arXiv

  28. [36]

    Image Captioning with Inherent Sentiment,

    T. Li, Y. Hu and X. Wu, "Image Captioning with Inherent Sentiment," in IEEE International Conference on Multimedia and Expo (ICME), Shenzhen, China, 2021

    2021

  29. [37]

    Assessing shallow sentence scoring techniques and combinations for single and multi -document summarization,

    H. Oliveira, R. Ferreira, R. Lima, R. D. Lins, F. Freitas, M. Riss and S. J. Simske, "Assessing shallow sentence scoring techniques and combinations for single and multi -document summarization," Expert Systems with Applications, vol. 65, pp. 68-86, 2016

    2016

  30. [38]

    Summarization of changes in dynamic text collections using latent dirichlet allocation model,

    M. Kar, S. Nunes and C. Ribeiro, "Summarization of changes in dynamic text collections using latent dirichlet allocation model," Information Processing & Management, vol. 51, no. 6, pp. 809-833, 2015

    2015

  31. [39]

    Fully abstractive approach to guided summarization,

    P.-E. Genes and G. Lapalme, "Fully abstractive approach to guided summarization," in 50th Annual Meeting of the Association for Computational Linguistics, 2012

    2012

  32. [40]

    A bayesian method to incorporate background knowledge during automatic text summarization,

    A. Louis, "A bayesian method to incorporate background knowledge during automatic text summarization," in Proceedings of the 52nd Annual Meeting of the Association for Computational Linguistics , Baltimore, Maryland, 2014

    2014

  33. [41]

    Multi-document abstractive summarization using ilp based multi- sentence compression,

    S. Banarjee, P. Mitra and K. Sugiyama, "Multi-document abstractive summarization using ilp based multi- sentence compression," in Twenty-Fourth International Joint Conference on Artificial Intelligence, 2015

    2015

  34. [42]

    A Neural Attention Model for Abstractive Sentence Summarization,

    A. M. Rush, S. Chopra and J. Weston, "A Neural Attention Model for Abstractive Sentence Summarization," in Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing, Lisbon, Portugal, 2015

    2015

  35. [43]

    Get To The Point: Summarization with Pointer-Generator Networks,

    A. See, P. J. Liu and C. D. Manning, "Get To The Point: Summarization with Pointer-Generator Networks," in arXiv preprint arXiv:1704.04368, 2017

    2017 arXiv

  36. [44]

    A Unified Model for Extractive and Abstractive Summarization using Inconsistency Loss,

    W. -T. Hsu, C. -K. Lin, M. -Y. Lee, K. Min, J. Tang and M. Sun, "A Unified Model for Extractive and Abstractive Summarization using Inconsistency Loss," 56th Annual Meeting of the Association for Computational Linguistics, p. 132–141, 2018

    2018

  37. [45]

    Abstractive Text Summarization with Multi-Head Attention,

    J. Li, C. Zhang, X. Chen, Y. Cao, P. Liao and P. Zhang, "Abstractive Text Summarization with Multi-Head Attention," in International Joint Conference on Neural Networks (IJCNN), Budapest, Hungary, 2019

    2019

  38. [46]

    Dual Encoding for Abstractive Text Summarization,

    K. Yao, L. Zhang, D. Du, T. Luo, L. Tao and Y. Wu, "Dual Encoding for Abstractive Text Summarization," IEEE Transactions on Cybernetics, vol. 50, no. 3, pp. 985 - 996, 2020

    2020

  39. [47]

    Transformers and Pointer -Generator Networks for Abstractive Summarization,

    J. Deaton, A. Jacobs, K. Kenealy and A. See, "Transformers and Pointer -Generator Networks for Abstractive Summarization," pp. 1-9, 2019

    2019

  40. [48]

    Attention is all you need,

    A. Vaswani, N. Shazeer, N. Parmar, J. Uszkoreit, L. Jones, A. N. Gomez, L. Kaiser and . I. Polosukhin, "Attention is all you need," in NIPS, 2017

    2017

  41. [49]

    Faster R -CNN: Towards Real -Time Object Detection with Region Proposal Networks,

    S. Ren, K. He, G. Ross and J. Sun, "Faster R -CNN: Towards Real -Time Object Detection with Region Proposal Networks," IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 39, pp. 1137- 1149, 2017

    2017

  42. [50]

    Rethinking the Inception Architecture for Computer Vision,

    C. Szegedy, V. Vanhoucke, S. Loffe, J. Shlens and Z. Wojna, "Rethinking the Inception Architecture for Computer Vision," in IEEE Conference on Computer Vision and Pattern Recognition (CVPR) , Las Vegas, NV, USA, 2016

    2016

  43. [51]

    Multi -task Sequence to Sequence Learning,

    M.-T. Luong, Q. V. Le, I. Sutskever, O. Vinyals and L. KAiser, "Multi -task Sequence to Sequence Learning," in arXiv:1511.06114v4, 2015

    2015 arXiv

  44. [52]

    Enriching Word Vectors with Subword Information,

    P. Bojanowski, E. Grave, A. Joulin and T. Mikolov, "Enriching Word Vectors with Subword Information," in arXiv:1607.04606, 2017

    2017 arXiv

  45. [53]

    Neural Machine Translation by Jointly Learning to Align and Translate,

    D. Bahdanau, . K. Cho and Y. Bengio, "Neural Machine Translation by Jointly Learning to Align and Translate," in arXiv:1409.0473, 2014

    2014 arXiv

  46. [54]

    Frustratingly Short Attention Spans in Neural Language Modeling,

    M. Daniluk, T. Rocktäschel, J. Welbl and S. Riedel, "Frustratingly Short Attention Spans in Neural Language Modeling," in arXiv:1702.04521, 2017

    2017 arXiv

  47. [55]

    "Bleu: a method for automatic evaluation of machine,

    K. Papineni, S. Roukos, T. Ward and W.-J. Zhu, ""Bleu: a method for automatic evaluation of machine," in Proceedings of the 40th Annual Meeting on Association for Computational Linguistics , 2002

    2002

  48. [56]

    Meteor: An automatic Metric for MT Evaluation with Improved Correlation with Human Judgments,

    S. Banerjee and A. Lavie, "Meteor: An automatic Metric for MT Evaluation with Improved Correlation with Human Judgments," in Proceedings of the ACL Workshop on Intrinsic and Extrinsic Evaluation Measures for Machine Translation and/or Summarization, 2005

    2005

  49. [57]

    Adam: A method for stochastic,

    D. P. Kingma and J. Ba, "Adam: A method for stochastic," in arXiv:1412.6980, 2014

    2014 arXiv

  50. [58]

    Guiding the long -short term memory model for image caption Generation,

    X. Jia, E. Gavves, B. Fernando and T. Tuytelaars, "Guiding the long -short term memory model for image caption Generation," in arXiv:1509.04942, 2015

    2015 arXiv

  51. [59]

    Multimodal Neural Language Models,

    R. Kiros, R. Salakhutdinov and R. Zemel, "Multimodal Neural Language Models," in 31st International Conference on Machine Learning, 2014

    2014

  52. [60]

    Stimulus -driven and concept -driven analysis for image caption generation,

    S. Ding, S. Qu, Y. Xi and S. Wan, "Stimulus -driven and concept -driven analysis for image caption generation," Neurocomputing, vol. 398, pp. 520-530, 2020

    2020

  53. [61]

    Learning joint relationship attention network for image captioning,

    C. Wang and X. Gu, "Learning joint relationship attention network for image captioning," Expert Systems with Applications, vol. 211, p. 118474, 2023

    2023

  54. [62]

    Detach and Attach: Stylized Image Captioning without Paired Stylized Dataset,

    Y. Tan, Z. Lin, P. Fu, L. Wang, Y. Cao and W. Wang, "Detach and Attach: Stylized Image Captioning without Paired Stylized Dataset," in Proceedings of the 30th ACM International Conference on Multimedia, 2022

    2022

  55. [63]

    Learning Cooperative Neural Modules for Stylized Image Captioning,

    X. Wu, W. Zhao and J. Luo, "Learning Cooperative Neural Modules for Stylized Image Captioning," International Journal of Computer Vision, vol. 130, p. 2305–2320, 2022

    2022

Pith tools

Reviewed August 11, 2026 · model on record in the stance chip above.