Pith. sign in

REVIEW 3 major objections 4 minor 45 references

Show, Tell and Summarize: Dense Video Captioning Using Visual Cue Aided Sentence Summarization

T0 review · 3 major / 4 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read The paper claims dense video captioning improves when each event is split into 20 short segments, each segment is captioned, and a two-stage LSTM with hierarchical attention and visual features summarizes the segment sentences into one…

desk verdict A plausible but incomplete dense-video-captioning pipeline: the division-and-summarization idea is real, but the paper's comparisons are dated and the segment-captioner training is unspecified. read the letter →

arxiv 2506.20567 v1 pith:CAYTT2IN submitted 2025-06-25 cs.CV cs.AI

Zhiwang Zhang , Dong Xu , Wanli Ouyang , Chuanqi Tan This is my paper · ORCID
classification cs.CVcs.AI
keywords densevideocaptioningdivision-and-summarizationsentencesummarizationhierarchicalattentiontwo-stageLSTMActivityNetCaptionsC3Dvisualfeatures
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Dense video captioning asks a model to localize every event in a long, untrimmed video and describe each one in a sentence. This paper claims that the describing half can be decomposed: divide each detected event proposal into 20 uniformly sampled short segments, run an existing neural image/video captioner on each segment to get one sentence, and then summarize those 20 noisy sentences into the final event sentence with a two-stage LSTM network whose attention is guided by C3D visual features at both the encoder and decoder. The authors call this the division-and-summarization (DaS) framework and report that it beats previous dense captioning methods on ActivityNet Captions, including Meteor 10.71 and CIDEr-D 31.41 with ground-truth proposals and Meteor 10.33 with automatic proposals. If the claim holds, per-segment captioning plus visual-aided summarization is a valid recipe for dense video captioning, and the hard part of event description shifts partly toward the strength of the segment captioner.

What carries the argument

The load-bearing machinery is the division-and-summarization pipeline plus a hierarchical attention mechanism. The division module uniformly samples each event proposal into $N_m=20$ segments, extracts 500-dimensional PCA-reduced C3D features per segment, and uses the Show and Tell captioner [34] to produce one sentence per segment. The summarization module is a two-stage LSTM: the first-stage encoder-fusion sub-net fuses each word embedding with an attended visual feature at every time step; the new encoder-attention sub-net groups the resulting hidden states into 20 sentence groups and applies a first-level attention inside each group followed by a second-level attention across groups; and the decoder sub-net attends over the visual features again while generating the output sentence. A discriminative word-occurrence loss and self-critical reinforcement learning with Meteor reward are added to the cross-entropy objective.

What would settle it

Take an ActivityNet video whose ground-truth event is playing soccer, feed the summarizer twenty segment captions that all say 'playing volleyball' with unchanged C3D features, and check whether the output says soccer; since the summarizer cannot recover an action absent from every segment caption, a 'volleyball' output confirms that division plus summarization alone cannot fix a weak segment captioner.

Watch

Extended reading notes

Core claim

The central discovery is that dense video captioning does not need to be solved as one monolithic generation problem. On the paper's terms, the DaS framework, a division module that produces $N_m=20$ C3D-feature-driven sentences per event proposal via the Show and Tell captioner, followed by a summarization module with an encoder-fusion LSTM, an encoder-attention sub-net using a new hierarchical attention mechanism, and a decoder LSTM with visual feature fusion, outperforms the compared baselines on ActivityNet Captions validation and on the 2018 online test server, with Meteor 6.40 versus 4.99 for Bi-AFCG. The ablations attribute the gain to three components: summarizing generated sentences instead of averaging or picking the best one, the hierarchical attention that first attends within each segment's words and then across segments, and the use of visual features on both encoder and decoder sides. Adding self-critical reinforcement learning with Meteor as the reward further improves the result.

Load-bearing premise

The pipeline assumes that the 20 per-segment captions plus C3D features contain enough correct semantic content to reconstruct the true event sentence; if the segment captioner mislabels or omits the key action, the summarizer can only recombine the wrong content.

Editorial extensions

If this is right

  • If the DaS claim is correct, researchers can improve dense video captioning by improving the per-segment captioner or the segment sampling strategy; the summarization stage does not need to recover omitted actions.
  • Hierarchical attention over sentence-grouped hidden states is a transferable module: any task that summarizes a noisy set of sentences, such as frame-level captions or document snippets, could use the same two-level grouping.
  • Visual features matter on both sides of the summarizer: removing them from the encoder or decoder drops Meteor by roughly 0.1 points in the paper's ablation, so future extensions should keep visual cues rather than treating summarization as text-only.
  • The framework's gain is largest on videos with many shot boundaries, with Meteor 10.10 versus 9.46 for Bi-AFCG on videos with at least 10 shot changes, which is the regime where a direct whole-event captioner struggles.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Because the division module uniformly samples 20 segments regardless of event structure, the recipe leaves an obvious knob unturned: adaptive or saliency-weighted segment sampling could push the summarizer further without changing the two-stage LSTM at all.
  • The formulation inherits the base captioner's vocabulary and biases, so part of the reported gain may come from ensembling many weak segment captions rather than from genuinely new visual understanding; a test that swaps Show and Tell for a stronger captioner and watches the metrics scale would separate these effects.
  • The same divide-caption-summarize pattern could transfer to other long-form video-language tasks, such as video question answering or highlight generation, where short clip captions are easier to obtain than one accurate long description.
  • The automatic-proposal CIDEr-D gap is much smaller, 12.93 versus 12.68 for Bi-AFCG, than the ground-truth-proposal gap, suggesting that the summarization gains may partly be masked by proposal localization errors; joint refinement of proposals and summaries is a natural next step.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 4 minor

Summary. The paper proposes a division-and-summarization (DaS) framework for dense video captioning on the ActivityNet Captions dataset. For each event proposal, the framework uniformly samples Nm=20 segments, extracts C3D features, generates one sentence per segment using an existing captioning approach, and then summarizes the generated sentences with a two-stage LSTM network that incorporates a hierarchical attention mechanism and visual features at both encoder and decoder. Experiments report Meteor 10.71 and CIDEr-D 31.41 with ground-truth proposals (Table I) and Meteor 10.33 with automatic proposals (Table II), outperforming the listed 2018-era baselines on some metrics. Ablations in Tables V-VIII and qualitative examples in Figures 4-7 support the contributions of the summarization module, visual feature fusion, hierarchical attention, and reinforcement learning.

Significance. If the claims hold, the paper offers a simple and potentially effective recipe for dense video captioning: per-segment captioning followed by visual-cue-aided sentence summarization. The paper is honest in several respects: it states that no extra datasets (MSCOCO, MSVD) are used in the captioning step, that activity category labels are not used, and that the same proposals as Bi-AFCG are used for the automatic-proposal comparison. The ablation study is reasonably complete, covering the hierarchical attention, visual feature fusion at encoder/decoder, the discriminative loss weight, and the number of segments. The qualitative analysis with shot-boundary grouping is a nice attempt to demonstrate the method's behavior on rapidly changing scenes. However, the significance is substantially undercut by the unspecified training protocol of the segment-level captioner and by the absence of any comparison with methods published after 2018, which limits the strength of the 'outperforms existing methods' claim.

major comments (3)
  1. [III-A] The training of the segment-level captioner is unspecified, and this is load-bearing for the central comparison. Section III-A states only that 'we apply the existing captioning approach in [34]' and that 'only ActivityNet Captions Dataset is used in the captioning step,' but it never states what ground-truth text is used to train this captioner, which data split is used, whether the model is fine-tuned or trained from scratch, or how segment-level caption targets are formed from the event-level annotations. If the only available ground truth is the full event sentence and it is replicated for each of the Nm=20 sampled segments, then the division module effectively produces ~20 paraphrases of the event sentence, making the subsequent summarization task much easier than the single-pass captioning task faced by DCE, DVC, and Bi-AFCG. The repetitive segment captions shown in Figure 6(b) (e.g., multiple near-identical 'a group of people ... playing soccer' sentences) are consistent with this recipe. The authors must disclose the exact training data construction and training protocol before the comparison can be reproduced or deemed fair.
  2. [IV-C, IV-D, Tables I-II] The headline comparison is limited to baselines from 2018 and earlier, and the only online test-set comparison is Meteor against Bi-AFCG (Table IV). The ActivityNet Captions Challenge 2018 leaderboard includes many subsequent methods, and the dense video captioning literature has advanced considerably since 2019. The abstract and conclusion claim that the framework 'outperforms existing dense video captioning methods,' but with the current evidence this is too broad. Either the comparison should be expanded to include more recent methods, or the claims should be scoped explicitly to the compared baselines.
  3. [IV-C, Tables I-II, Table V] No variance or statistical significance is reported, and the evaluation consistency is unclear. The paper says it 'follows the evaluation method in [1]' but does not state whether the baseline numbers in Tables I and II are taken from the original papers or recomputed with the authors' evaluation code. The discrepancy between TA's Meteor 8.75 in Table I and TA's Meteor 9.14 in Table V (where TA is re-implemented by the authors) suggests that different evaluation conditions may have been used. The authors should clarify which numbers were recomputed and ensure that all compared methods, including the re-implemented TA, are evaluated with identical preprocessing and metric code.
minor comments (4)
  1. [Table I] The table header 'ACTIVITY NET' should be 'ACTIVITYNET' to match the dataset name.
  2. [IV-D] The sentence 'the results of our method are 10.71% (Meteor score) and 31.41% (CIDEr-D score)' is incorrect because Meteor and CIDEr-D are not percentages; it should read '10.71 (Meteor) and 31.41 (CIDEr-D).'
  3. [IV-B] The sentence 'The hidden state size and the dropout rate for all LSTM cells is 512 and 0.8, respectively' has a subject-verb agreement error; 'is' should be 'are.'
  4. [Figure 6] The caption for Figure 6(b) should explicitly state that the listed sentences are the raw outputs of the division module (the inputs to the summarization module), not the final predictions, to avoid confusion with the qualitative comparison in Figure 6(a).

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the DaS pipeline is an empirical system evaluated on an external benchmark, and its headline results are not forced by construction.

full rationale

The paper's central claim is an empirical performance result on ActivityNet Captions, obtained by a trainable pipeline rather than by an equation that rewrites its inputs as its outputs. The division module (Section III-A) feeds C3D features into the external captioner [34] to produce segment sentences S and visual features V; the summarization module (Section III-B) then encodes S and V with learned LSTM attention modules and decodes a new event sentence. The final loss in Eq. (14) is cross-entropy against ground-truth event words plus a discriminative loss, and evaluation is done on held-out validation/test captions (Tables I, II, IV). Nothing in Eqs. (1)-(14) defines the predicted sentence as equal, by construction, to the input segment sentences, the visual features, or the ground-truth caption. The model can fail or improve relative to its inputs: Figure 6(b) shows segment captions mislabeling 'soccer' as 'volleyball', while the final output says 'soccer', so the output is not forced to equal the input. There is no load-bearing self-citation chain: the proposal generator [2] and the captioner [34] are external works, and the only same-author reference [17] appears as a related-work citation for action localization, not as justification of the central claim or as a uniqueness theorem. The paper does leave an experimental-transparency gap: Section III-A states that the captioning step uses only ActivityNet Captions but does not specify how event-level ground-truth sentences are converted into segment-level supervision for [34]. That is a fairness and reproducibility concern, not circularity, because the reported numbers still depend on trained model behavior and external metric computation rather than reducing to a fitted parameter renamed as a prediction.

Assumptions & free parameters 5 free parameters · 4 assumptions · 0 invented entities

The paper introduces no new physical or conceptual entities; it builds an architecture from standard LSTM and attention components. The main load-bearing premises are that noisy per-segment captions plus C3D features suffice for final sentence generation, and that Meteor dominance outweighs BLEU-4 deficits.

free parameters (5)
  • Nm (number of sampled segments per proposal) = 20
    Set by hand and validated via sweep in Table VIII; affects the fixed 500-word input length.
  • Nk (max words per segment sentence) = 25
    Chosen to fix total input length; sentences longer than 25 words are truncated.
  • lambda_d (discriminative loss weight) = 0.1
    Empirically set; Table VII shows sensitivity across 0, 0.01, 0.1, and 1.
  • dropout rate for LSTM cells = 0.8
    Hyperparameter, no sensitivity analysis reported.
  • hidden state size = 512
    Hyperparameter, no sensitivity analysis reported.
assumptions (4)
  • domain assumption Per-segment generated sentences preserve enough semantic content to describe the whole event proposal.
    Stated in Section III before Equation (1); if the base captioner fails, the summarizer cannot recover the lost content.
  • domain assumption Uniformly sampling each proposal into Nm=20 fixed-length segments with C3D features retains the visual information needed for captioning.
    Division module, Section III-A; no analysis of sampling granularity beyond the Nm sweep.
  • domain assumption Meteor is a more important metric than BLEU-4 for dense video captioning.
    Used in Section IV-D to argue overall superiority despite lower BLEU-4; this is an evaluation-prior, not a proven fact.
  • standard math LSTM cell equations and the attention formula from [20] and [35] are correct and applicable.
    Equations (2)-(5); background machine learning machinery taken from cited literature.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Show, Tell and Summarize: Dense Video Captioning Using Visual Cue Aided Sentence Summarization." pith.science (2026). https://pith.science/paper/CAYTT2IN

@misc{pith2026250620567,
  author       = {Pith},
  title        = {Pith review of: Show, Tell and Summarize: Dense Video Captioning Using Visual Cue Aided Sentence Summarization},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/CAYTT2IN}},
  note         = {Machine review of arXiv:2506.20567}
}
read the original abstract

In this work, we propose a division-and-summarization (DaS) framework for dense video captioning. After partitioning each untrimmed long video as multiple event proposals, where each event proposal consists of a set of short video segments, we extract visual feature (e.g., C3D feature) from each segment and use the existing image/video captioning approach to generate one sentence description for this segment. Considering that the generated sentences contain rich semantic descriptions about the whole event proposal, we formulate the dense video captioning task as a visual cue aided sentence summarization problem and propose a new two stage Long Short Term Memory (LSTM) approach equipped with a new hierarchical attention mechanism to summarize all generated sentences as one descriptive sentence with the aid of visual features. Specifically, the first-stage LSTM network takes all semantic words from the generated sentences and the visual features from all segments within one event proposal as the input, and acts as the encoder to effectively summarize both semantic and visual information related to this event proposal. The second-stage LSTM network takes the output from the first-stage LSTM network and the visual features from all video segments within one event proposal as the input, and acts as the decoder to generate one descriptive sentence for this event proposal. Our comprehensive experiments on the ActivityNet Captions dataset demonstrate the effectiveness of our newly proposed DaS framework for dense video captioning.

Figures

Figures reproduced from arXiv: 2506.20567 by the authors.

Figure 1
Figure 1. Overview of our division and summarization framework. [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 2
Figure 2. Overview of our summarization module (best viewed in color). All the operations are shown in the blocks with different colors. (a): our summarization [PITH_FULL_IMAGE:figures/full_fig_p003_2.png] view at source ↗
Figure 3
Figure 3. Illustration of the visual and textual feature fusion modules (a): at the [PITH_FULL_IMAGE:figures/full_fig_p003_3.png] view at source ↗
Figures from the paper (2 more)
Figure 4
Figure 4. Figure 4: Generated sentences by our method DaS(HA) and the baseline method TA [20] for one video from the Activity Captions validation set. Correctly captured actions are highlighted by purple color, while incorrectly captured actions are highlighted by blue color. Videos Groun…
Figure 6
Figure 6. Figure 6: (a) Comparison between our method DaS(HA) and the baseline method TA [20]. Correctly captured actions are highlighted by purple color (b) The output of our division module (i.e., Nm = 20 sentences), which is also the textual input of our summarization module. F. Qualit…

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

45 extracted references · 35 canonical work pages

  1. [34]

    Show and tell: A neural image caption generator,

    O. Vinyals, A. Toshev, S. Bengio, and D. Erhan, “Show and tell: A neural image caption generator,” in Proceedings of the IEEE conference on computer vision and pattern recognition , 2015, pp. 3156–3164

    2015

  2. [1]

    Dense- captioning events in videos

    R. Krishna, K. Hata, F. Ren, L. Fei-Fei, and J. C. Niebles, “Dense- captioning events in videos.” in ICCV, 2017, pp. 706–715

    work page 2017

  3. [2]

    Bidirectional attentive fusion with context gating for dense video captioning,

    J. Wang, W. Jiang, L. Ma, W. Liu, and Y . Xu, “Bidirectional attentive fusion with context gating for dense video captioning,” in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition , 2018, pp. 7190–7198

    work page 2018

  4. [3]

    Jointly localizing and describing events for dense video captioning,

    Y . Li, T. Yao, Y . Pan, H. Chao, and T. Mei, “Jointly localizing and describing events for dense video captioning,” in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition , 2018, pp. 7492–7500

    work page 2018

  5. [4]

    End-to-end dense video captioning with masked transformer,

    L. Zhou, Y . Zhou, J. J. Corso, R. Socher, and C. Xiong, “End-to-end dense video captioning with masked transformer,” in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition , 2018, pp. 8739–8748

    work page 2018

  6. [5]

    Show, attend and tell: Neural image caption generation with visual attention,

    K. Xu, J. Ba, R. Kiros, K. Cho, A. Courville, R. Salakhudinov, R. Zemel, and Y . Bengio, “Show, attend and tell: Neural image caption generation with visual attention,” in International conference on machine learning , 2015, pp. 2048–2057

    work page 2015

  7. [6]

    Automatic annotation of human actions in video,

    O. Duchenne, I. Laptev, J. Sivic, F. Bach, and J. Ponce, “Automatic annotation of human actions in video,” in Computer Vision, 2009 IEEE 12th International Conference on . IEEE, 2009, pp. 1491–1498

    work page 2009

  8. [7]

    Fast temporal activity proposals for efficient detection of human actions in untrimmed videos,

    F. Caba Heilbron, J. Carlos Niebles, and B. Ghanem, “Fast temporal activity proposals for efficient detection of human actions in untrimmed videos,” in Proceedings of the IEEE conference on computer vision and pattern recognition, 2016, pp. 1914–1923

    work page 2016

Show all 45 references
  1. [8]

    Daps: Deep action proposals for action understanding,

    V . Escorcia, F. C. Heilbron, J. C. Niebles, and B. Ghanem, “Daps: Deep action proposals for action understanding,” in European Conference on Computer Vision. Springer, 2016, pp. 768–784

    2016

  2. [9]

    Sst: Single-stream temporal action proposals,

    S. Buch, V . Escorcia, C. Shen, B. Ghanem, and J. C. Niebles, “Sst: Single-stream temporal action proposals,” in 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) . IEEE, 2017, pp. 6373–6382

    2017

  3. [10]

    Weakly supervised dense video captioning,

    Z. Shen, J. Li, Z. Su, M. Li, Y . Chen, Y .-G. Jiang, and X. Xue, “Weakly supervised dense video captioning,” in The IEEE Conference on Computer Vision and Pattern Recognition (CVPR) , vol. 2, no. 7, 2017

    2017

  4. [11]

    Jointly modeling embedding and translation to bridge video and language,

    Y . Pan, T. Mei, T. Yao, H. Li, and Y . Rui, “Jointly modeling embedding and translation to bridge video and language,” in Proceedings of the IEEE conference on computer vision and pattern recognition , 2016, pp. 4594–4602

    2016

  5. [12]

    Less is more: Picking informative frames for video captioning,

    Y . Chen, S. Wang, W. Zhang, and Q. Huang, “Less is more: Picking informative frames for video captioning,” arXiv preprint arXiv:1803.01457, 2018

    2018 arXiv

  6. [13]

    Video captioning via hierarchical reinforcement learning,

    X. Wang, W. Chen, J. Wu, Y .-F. Wang, and W. Y . Wang, “Video captioning via hierarchical reinforcement learning,” in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition , 2018, pp. 4213–4222

    2018

  7. [14]

    Reconstruction network for video captioning,

    B. Wang, L. Ma, W. Zhang, and W. Liu, “Reconstruction network for video captioning,” in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition , 2018, pp. 7622–7631

    2018

  8. [15]

    Interpretable video captioning via trajectory structured localization,

    X. Wu, G. Li, Q. Cao, Q. Ji, and L. Lin, “Interpretable video captioning via trajectory structured localization,” in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition , 2018, pp. 6829–6837

    2018

  9. [16]

    Regularizing rnns for caption generation by reconstructing the past with the present,

    X. Chen, L. Ma, W. Jiang, J. Yao, and W. Liu, “Regularizing rnns for caption generation by reconstructing the past with the present,” arXiv preprint arXiv:1803.11439, 2018

    2018 arXiv

  10. [17]

    Improving action localization by progressive cross-stream cooperation,

    R. Su, W. Ouyang, L. Zhou, and D. Xu, “Improving action localization by progressive cross-stream cooperation,” in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition , 2019, pp. 12 016–12 025

    2019

  11. [18]

    Translating videos to natural language using deep recurrent neural networks,

    S. Venugopalan, H. Xu, J. Donahue, M. Rohrbach, R. Mooney, and K. Saenko, “Translating videos to natural language using deep recurrent neural networks,” arXiv preprint arXiv:1412.4729 , 2014

    2014 arXiv

  12. [19]

    Semantic compositional networks for visual captioning,

    Z. Gan, C. Gan, X. He, Y . Pu, K. Tran, J. Gao, L. Carin, and L. Deng, “Semantic compositional networks for visual captioning,” in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, vol. 2, 2017

    2017

  13. [20]

    Describing videos by exploiting temporal structure,

    L. Yao, A. Torabi, K. Cho, N. Ballas, C. Pal, H. Larochelle, and A. Courville, “Describing videos by exploiting temporal structure,” in Proceedings of the IEEE international conference on computer vision , 2015, pp. 4507–4515

    2015

  14. [21]

    Video captioning with transferred semantic attributes,

    Y . Pan, T. Yao, H. Li, and T. Mei, “Video captioning with transferred semantic attributes,” in CVPR, vol. 2, 2017, p. 3

    2017

  15. [22]

    Sequence to sequence learning with neural networks,

    I. Sutskever, O. Vinyals, and Q. V . Le, “Sequence to sequence learning with neural networks,” in Advances in neural information processing systems, 2014, pp. 3104–3112

    2014

  16. [23]

    Long-term recurrent convolutional IEEE TRANSACTIONS ON XXXX, VOL.XX, NO.XX, XXXX,XXXX 10 networks for visual recognition and description,

    J. Donahue, L. Anne Hendricks, S. Guadarrama, M. Rohrbach, S. Venu- gopalan, K. Saenko, and T. Darrell, “Long-term recurrent convolutional IEEE TRANSACTIONS ON XXXX, VOL.XX, NO.XX, XXXX,XXXX 10 networks for visual recognition and description,” in Proceedings of the IEEE confer...

    2015

  17. [24]

    Hierarchical recurrent neural encoder for video representation with application to captioning,

    P. Pan, Z. Xu, Y . Yang, F. Wu, and Y . Zhuang, “Hierarchical recurrent neural encoder for video representation with application to captioning,” in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2016, pp. 1029–1038

    2016

  18. [25]

    Summarization-based video caption via deep neural networks,

    G. Li, S. Ma, and Y . Han, “Summarization-based video caption via deep neural networks,” in Proceedings of the 23rd ACM international conference on Multimedia . ACM, 2015, pp. 1191–1194

    2015

  19. [26]

    Boosting video description generation by explicitly translating from frame-level captions,

    Y . Liu and Z. Shi, “Boosting video description generation by explicitly translating from frame-level captions,” in Proceedings of the 24th ACM international conference on Multimedia . ACM, 2016, pp. 631–634

    2016

  20. [27]

    Lexrank: Graph-based lexical centrality as salience in text summarization,

    G. Erkan and D. R. Radev, “Lexrank: Graph-based lexical centrality as salience in text summarization,” Journal of artificial intelligence research, vol. 22, pp. 457–479, 2004

    2004

  21. [28]

    Ask, attend and answer: Exploring question- guided spatial attention for visual question answering,

    H. Xu and K. Saenko, “Ask, attend and answer: Exploring question- guided spatial attention for visual question answering,” in European Conference on Computer Vision . Springer, 2016, pp. 451–466

    2016

  22. [29]

    An end-to-end spatio- temporal attention model for human action recognition from skeleton data

    S. Song, C. Lan, J. Xing, W. Zeng, and J. Liu, “An end-to-end spatio- temporal attention model for human action recognition from skeleton data.” in AAAI, vol. 1, no. 2, 2017, pp. 4263–4270

    2017

  23. [30]

    Image captioning with semantic attention,

    Q. You, H. Jin, Z. Wang, C. Fang, and J. Luo, “Image captioning with semantic attention,” in Proceedings of the IEEE conference on computer vision and pattern recognition , 2016, pp. 4651–4659

    2016

  24. [31]

    Stacked attention networks for image question answering,

    Z. Yang, X. He, J. Gao, L. Deng, and A. Smola, “Stacked attention networks for image question answering,” in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2016, pp. 21– 29

    2016

  25. [32]

    Residual attention network for image classification,

    F. Wang, M. Jiang, C. Qian, S. Yang, C. Li, H. Zhang, X. Wang, and X. Tang, “Residual attention network for image classification,” arXiv preprint arXiv:1704.06904, 2017

    2017 arXiv

  26. [33]

    Multiple object recognition with visual attention,

    J. Ba, V . Mnih, and K. Kavukcuoglu, “Multiple object recognition with visual attention,” arXiv preprint arXiv:1412.7755 , 2014

    2014 arXiv

  27. [35]

    Long short-term memory,

    S. Hochreiter and J. Schmidhuber, “Long short-term memory,” Neural computation, vol. 9, no. 8, pp. 1735–1780, 1997

    1997

  28. [36]

    Review networks for caption generation,

    Z. Yang, Y . Yuan, Y . Wu, W. W. Cohen, and R. R. Salakhutdinov, “Review networks for caption generation,” in Advances in Neural Information Processing Systems , 2016, pp. 2361–2369

    2016

  29. [37]

    Self- critical sequence training for image captioning,

    S. J. Rennie, E. Marcheret, Y . Mroueh, J. Ross, and V . Goel, “Self- critical sequence training for image captioning,” in CVPR, vol. 1, no. 2, 2017, p. 3

    2017

  30. [38]

    Sequence to sequence-video to text,

    S. Venugopalan, M. Rohrbach, J. Donahue, R. Mooney, T. Darrell, and K. Saenko, “Sequence to sequence-video to text,” in Proceedings of the IEEE international conference on computer vision , 2015, pp. 4534– 4542

    2015

  31. [39]

    Video paragraph captioning using hierarchical recurrent neural networks,

    H. Yu, J. Wang, Z. Huang, Y . Yang, and W. Xu, “Video paragraph captioning using hierarchical recurrent neural networks,” in Proceedings of the IEEE conference on computer vision and pattern recognition , 2016, pp. 4584–4593

    2016

  32. [40]

    Activitynet: A large-scale video benchmark for human activity under- standing,

    F. Caba Heilbron, V . Escorcia, B. Ghanem, and J. Carlos Niebles, “Activitynet: A large-scale video benchmark for human activity under- standing,” in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2015, pp. 961–970

    2015

  33. [41]

    Adam: A method for stochastic optimization,

    D. P. Kingma and J. Ba, “Adam: A method for stochastic optimization,” arXiv preprint arXiv:1412.6980 , 2014

    2014 arXiv

  34. [42]

    Bleu: a method for automatic evaluation of machine translation,

    K. Papineni, S. Roukos, T. Ward, and W.-J. Zhu, “Bleu: a method for automatic evaluation of machine translation,” in Proceedings of the 40th annual meeting on association for computational linguistics . Association for Computational Linguistics, 2002, pp. 311–318

    2002

  35. [43]

    Rouge: A package for automatic evaluation of summaries,

    C.-Y . Lin, “Rouge: A package for automatic evaluation of summaries,” in Text summarization branches out: Proceedings of the ACL-04 workshop, vol. 8. Barcelona, Spain, 2004

    2004

  36. [44]

    Meteor universal: Language specific translation evaluation for any target language,

    M. Denkowski and A. Lavie, “Meteor universal: Language specific translation evaluation for any target language,” in Proceedings of the ninth workshop on statistical machine translation , 2014, pp. 376–380

    2014

  37. [45]

    Cider: Consensus- based image description evaluation,

    R. Vedantam, C. Lawrence Zitnick, and D. Parikh, “Cider: Consensus- based image description evaluation,” in Proceedings of the IEEE confer- ence on computer vision and pattern recognition , 2015, pp. 4566–4575. Zhiwang Zhang received BE degree in school of Electrical and Infor...

    2015

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.