Pith. sign in

REVIEW 4 major objections 4 minor 42 references

A Better Way to Attend: Attention with Trees for Video Question Answering

T0 review · 4 major / 4 minor · reviewed 2026-08-14 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read This paper claims that video question answering improves when attention follows the syntax parse tree of the question, with the largest gains on long and complex questions.

desk verdict The tree-attention model is clearly specified and the ablations are internally consistent, but the headline numbers rest on self-harvested datasets that are never checked against the official benchmarks, so the SOTA claim overreaches. read the letter →

arxiv 1909.02218 v1 pith:SPHPWQAP submitted 2019-09-05 cs.CV cs.CLcs.LG

Hongyang Xue , Wenqing Chu , Zhou Zhao , Deng Cai This is my paper · ORCID
classification cs.CVcs.CLcs.LG
keywords videoquestionansweringtree-structuredmemorynetworkattentionmechanismsyntaxparsetreetemporalvisualandverbalwordshierarchicallongreasoning
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Video question answering needs to find the frames that answer a question, and most attention models encode the question as a flat sequence of words. This paper tries to establish that the attention computation should follow the syntax parse tree of the question instead: visual words such as nouns and most verbs attend to the video, verbal words such as linking verbs and conjunctions act only as sentence structure, and the two kinds of representation are merged recursively from the leaves to the root. The proposed model, the heterogeneous tree-structured memory network (HTreeMN), is reported to outperform flat attention baselines on two video-QA datasets, with the largest gains on long questions and on number and time questions. If the claim holds, it would show that syntactic structure is a practical asset for grounding language in video, not just for text-only NLP.

What carries the argument

The central object is the parse tree of the question sentence, with words at the leaves and phrase nodes (such as NP and VP) inside. A leaf is visual if its word has a high concreteness rating; visual leaves query the video frames with a temporal attention module and add the attended feature to the word embedding, while verbal leaves are transformed linearly without attention. All parent nodes sum their children's outputs through type-dependent linear layers, so information is composed from the leaves to the root, whose state is fed to a softmax classifier over candidate answers. A recursive rule decides which internal nodes attend: a node is verbal exactly when every child is verbal, which makes the attention hierarchy follow the sentence structure. This tree composition is what lets long questions keep their semantic structure instead of being collapsed into one linear embedding.

What would settle it

Run the three baselines and HTreeMN on the official published train/validation/test splits of YouTube-QA and TGIF-QA with identical visual features and answer vocabularies, and compare accuracy within each question-length bin; if the tree model does not beat the flat attention baselines on questions longer than fifteen words, the paper's central claim fails.

Watch

Extended reading notes

Core claim

The paper's central claim is that the syntactic parse tree of a question is the right skeleton for video attention. HTreeMN labels each word as visual or verbal using concreteness ratings, computes temporal attention over video frames only for the visual words, and then propagates joint word-video representations bottom-up through the tree, with separate linear transformations for the two word types. Intermediate phrase nodes are also classified recursively as verbal only when all of their children are verbal, and visual intermediate nodes receive their own attention, yielding a hierarchical attention mechanism. The full model is reported to outperform the simple attention baseline, the E-SS sequence-to-sequence model, and the E-SA word-level attention model on both YouTube-QA and TGIF-QA in accuracy, WUPS@0.0, and WUPS@0.9, with the clearest margin on long and complex questions. The conclusion states that these results are state-of-the-art for video question answering.

Load-bearing premise

The load-bearing assumption is that the two datasets used in the experiments, re-harvested by the authors following the method of reference [41], are equivalent enough to the original published benchmarks that the gains over E-SA and E-SS reflect model quality rather than differences in question generation or data splits.

Editorial extensions

If this is right

  • Video question answering systems should encode questions as parse trees rather than flat LSTM chains, particularly when questions are long and compositional.
  • Skipping attention for verbal words removes a source of noise from the attended video features, since verbal words lack direct visual correspondence.
  • The largest practical gains should appear on number and time questions, where the reported accuracy jumps by large margins over the flat baselines.
  • The tree-structured models are claimed to make better use of small training data, because semantic structure compensates for fewer examples.
  • The flat baselines should be expected to degrade as question length grows, while the tree model stays stable.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Because the parse tree comes from an off-the-shelf parser, the same visual/verbal split and recursive attention could transfer to other video-language tasks with long compositional queries, such as temporal grounding from natural language; the paper does not test this extension.
  • The reported gap between HTreeMN and the non-hierarchical HTreeMN-noh is small, suggesting that the hierarchical attention on intermediate nodes adds little once the leaf nodes are heterogeneous; a cheaper model might be enough, though the authors still report the full model.
  • The state-of-the-art claim is conditional on the harvested datasets matching the originals; re-running on official splits would settle whether the gains generalize outside the authors' own data.
  • The visual/verbal split could be learned end-to-end instead of fixed by a concreteness threshold, which might adapt better to specialized video domains.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 4 minor

Summary. The paper proposes HTreeMN, a heterogeneous tree-structured memory network for video question answering. The model encodes questions using syntax parse trees, distinguishes 'visual' from 'verbal' words, applies word-level attention only to visual leaves, and adds a hierarchical attention mechanism over intermediate tree nodes. The approach is evaluated on two self-harvested datasets, YouTube-QA and TGIF-QA, and the authors report that the full HTreeMN outperforms the E-SA, E-SS, and several ablated baselines, and conclude that it achieves state-of-the-art results on video question answering.

Significance. If the empirical results were validated on standard benchmarks, the tree-structured attention mechanism would be a clearly described and potentially useful alternative to chain-based word-level attention for video QA. The paper's strengths include explicit model equations (1)-(8), a clear ablation sequence (TreeMN to HTreeMN-noh to HTreeMN), and qualitative analysis of attention distributions. However, the central quantitative claim is not currently established because the experiments are conducted on author-harvested datasets rather than the official YouTube-QA/TGIF-QA benchmarks, and because the reported per-type gains are not accompanied by statistical significance testing.

major comments (4)
  1. [Section IV-A, Tables I-IV] The datasets are self-harvested following the method of [41], not the official YouTube-QA or TGIF-QA benchmarks. Because the baselines E-SA and E-SS were originally evaluated on the official datasets, any comparison on different question-generation templates, splits, or answer vocabularies is not a valid test of model superiority. The paper provides no evidence, such as overlap statistics, template distribution comparisons, or split alignment, that the harvested data are equivalent to [41]. Consequently, the global claims in Section IV-F ('our full HTreeMN model performs the best on both the two datasets') and Section V ('we achieve the state-of-the-art results') are not supported. The authors should either evaluate on the official datasets or provide a rigorous equivalence analysis and restrict all claims to 'on our harvested datasets.'
  2. [Section IV-F.2, Tables V-X] The per-question-type results are computed on very small test sets: for example, YouTube-QA Location has only 88 test questions and Time has 69; TGIF-QA Location has 520 and Time has 791. On these subsets, accuracy differences between HTreeMN and the next-best model are often within the binomial standard error. For instance, on YouTube-QA Location, HTreeMN accuracy is 0.2159 while HTreeMN-noh achieves 0.2386, a difference of about 2 percentage points, which is within the roughly 4 percentage-point standard error for a sample of 88. No error bars, confidence intervals, or multiple-seed experiments are reported anywhere. The claim that the tree models perform better on complex questions requires statistical support.
  3. [Section III-C vs Section IV-B.2] The word labeling procedure is described inconsistently. Section III-C states that the labeling is performed using the NLTK tool depending on word properties (nouns, verbs, adjectives, etc.), while Section IV-B.2 states that words are classified by the concreteness ratings of [3] with a threshold of 0.5. These are different methods and will generate different visual/verbal partitions, so the exact HTreeMN model is not reproducible from the paper. Please specify the actual algorithm used and report the fraction of visual versus verbal words, since this choice is load-bearing for the heterogeneous-node ablation.
  4. [Section V] The conclusion's claim of state-of-the-art results is not established by the experiments. The only baselines are the three models from [41] plus the authors' own ablations; no comparison to other published video QA methods is provided. For the claim to stand, the evaluation must be on the official benchmarks used by prior work, or the claim must be restricted to the compared baselines.
minor comments (4)
  1. [Equation (4)] Equation (4) uses 'n∈Rz' for the bias term, which should be 'b∈Rz'.
  2. [Abstract and Section IV-A] The abstract gives the code URL as https://github.com/ZJULearning/TreeAttention while Section IV-A gives https://github.com/xuehy/TreeAttention; please unify the two URLs.
  3. [Section III-C] The phrase 'For an verbal node' should be 'For a verbal node.'
  4. [Section IV-A] The sentence 'The size of the training set is almost equal to the size of both the validation and the testing set' is ambiguous; the training set is roughly equal to validation+test combined, not to each individually.

Circularity Check

0 steps flagged · score 0.0 of 10

No circular derivation: HTreeMN's reported results are empirical held-out evaluations, not identities, fits, or self-citation-forced conclusions.

full rationale

The paper's central claim is an empirical architecture comparison: HTreeMN outperforms Simple, E-SS, E-SA, TreeMN, and HTreeMN-noh on two video QA datasets. No equation in the model section defines the target result in terms of the input, and no parameter is fitted on test data; Section IV-E.2 states that early stopping on validation accuracy is used and the final model is selected by best validation accuracy, so the test-table numbers are not fitting artifacts. The comparison methods are either from the third-party [41] or ablations of the proposed model, so no prediction reduces by construction to a fitted input. The only notable weakness is dataset provenance: Section IV-A says the two datasets are 'harvested following the method in [41]' because the original dataset is 'not totally available till now,' which weakens comparability with published external benchmarks and is a validity or reproducibility concern, not a circularity. The paper does not rely on a self-citation chain: [41] is authored by other researchers and is used as a baseline/dataset reference, not as an unverified uniqueness theorem or ansatz smuggled in from the present authors. There is also no renaming of a known result into new coordinates. Accordingly, no circular step can be quoted with a specific equation-to-equation reduction, and the honest finding is no significant circularity.

Assumptions & free parameters 4 free parameters · 4 assumptions · 0 invented entities

The central claim rests on four load-bearing assumptions: parse tree quality, the hand-chosen visual/verbal split, equivalence of the self-harvested datasets to prior benchmarks, and the sufficiency of frozen VGGNet features. The model itself adds several hand-chosen preprocessing parameters. No new physical entities are introduced.

free parameters (4)
  • Concreteness threshold = 0.5
    Hand-chosen threshold on the Brysbaert concreteness ratings that determines whether a word is classified as visual or verbal. This rule directly controls which words attend to video, a core component of HTreeMN.
  • Number of sampled frames per video = 60 for YouTube-QA, 30 for TGIF-QA
    Chosen by hand based on typical video length; affects the granularity of temporal attention for all models equally.
  • Hidden representation size = 1024
    Chosen for all models and linear layers; a standard capacity choice, not tuned per model.
  • Answer vocabulary size K = 500 for YouTube-QA, 1000 for TGIF-QA
    Top K most frequent answers are selected as the classification space, following the prior work; affects all models equally.
assumptions (4)
  • domain assumption Syntax parse trees generated by StanfordParser are accurate enough to capture the semantic structure of the questions.
    The entire tree-structured composition depends on parse tree quality; incorrect parses would propagate wrong groupings.
  • ad hoc to paper The concreteness ratings and threshold 0.5 reliably separate visual words from verbal words for video attention.
    The visual/verbal split introduced in Section III-C relies on this external rating threshold, chosen by hand, with no validation that the split improves grounding beyond the reported ablations.
  • domain assumption The re-harvested YouTube-QA and TGIF-QA datasets are valid stand-ins for the datasets used in prior work.
    Section IV-A says the original dataset is not totally available and the authors constructed their own versions. Cross-paper comparisons to prior published numbers assume equivalence.
  • domain assumption VGGNet features from sampled frames provide sufficient visual information for answering.
    The model uses frozen VGGNet features and does not learn visual features end-to-end, so answer accuracy is bounded by this representation.

how reviews work

0 comments
Cite this review

Pith. "Pith review of A Better Way to Attend: Attention with Trees for Video Question Answering." pith.science (2026). https://pith.science/paper/SPHPWQAP

@misc{pith2026190902218,
  author       = {Pith},
  title        = {Pith review of: A Better Way to Attend: Attention with Trees for Video Question Answering},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/SPHPWQAP}},
  note         = {Machine review of arXiv:1909.02218}
}
read the original abstract

We propose a new attention model for video question answering. The main idea of the attention models is to locate on the most informative parts of the visual data. The attention mechanisms are quite popular these days. However, most existing visual attention mechanisms regard the question as a whole. They ignore the word-level semantics where each word can have different attentions and some words need no attention. Neither do they consider the semantic structure of the sentences. Although the Extended Soft Attention (E-SA) model for video question answering leverages the word-level attention, it performs poorly on long question sentences. In this paper, we propose the heterogeneous tree-structured memory network (HTreeMN) for video question answering. Our proposed approach is based upon the syntax parse trees of the question sentences. The HTreeMN treats the words differently where the \textit{visual} words are processed with an attention module and the \textit{verbal} ones not. It also utilizes the semantic structure of the sentences by combining the neighbors based on the recursive structure of the parse trees. The understandings of the words and the videos are propagated and merged from leaves to the root. Furthermore, we build a hierarchical attention mechanism to distill the attended features. We evaluate our approach on two datasets. The experimental results show the superiority of our HTreeMN model over the other attention models especially on complex questions. Our code is available on github. Our code is available at https://github.com/ZJULearning/TreeAttention

Figures

Figures reproduced from arXiv: 1909.02218 by the authors.

Figure 1
Figure 1. Overview of our HTreeMN model. The tree is the [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. The original parse tree of the question Who is walking a horse generated by StanfordParser [16]. of the sentences loses the syntactic property of the natural language which can be modeled with parse trees. C. Tree Networks There have been some works on tree-structured LSTMs for NLP tasks. In [26], a Tree-LSTM is proposed for modeling semantic relatedness and sentiment classification for natural language sentences. S… view at source ↗
Figure 4
Figure 4. A general TreeLSTM. A. Previous Works In this subsection, we briefly review the E-SA [41] and the TreeLSTM [26] models,. The E-SA [41] model is an attention-based model which consists of two-layer LSTMs (see [PITH_FULL_IMAGE:figures/full_fig_p003_4.png] view at source ↗
Figures from the paper (7 more)
Figure 5
Figure 5. Figure 5: The nodes of the memory network can be categorized [PITH_FULL_IMAGE:figures/full_fig_p004_5.png]
Figure 6
Figure 6. Figure 6: The heterogeneous tree-structured memory network [PITH_FULL_IMAGE:figures/full_fig_p005_6.png]
Figure 7
Figure 7. Figure 7: The Heterogeneous Tree-Structured Memory Network (HTreeMN) corresponds to the question in Figure 2. Double [PITH_FULL_IMAGE:figures/full_fig_p006_7.png]
Figure 8
Figure 8. Figure 8: The accuracy on questions of different complexity. We divide the lengths into 5 sets [PITH_FULL_IMAGE:figures/full_fig_p009_8.png]
Figure 9
Figure 9. Figure 9: We display several examples of long questions where the E-SA[41] model errs and our model succeeds. [PITH_FULL_IMAGE:figures/full_fig_p009_9.png]
Figure 10
Figure 10. Figure 10: Several failure examples. The errors are mainly caused by recognition errors. For example, the baby lions are recognized [PITH_FULL_IMAGE:figures/full_fig_p010_10.png]
Figure 11
Figure 11. Figure 11: Attention comparison for TreeMN and HTreeMN-noh. The curves show the attention distribution for each word on [PITH_FULL_IMAGE:figures/full_fig_p011_11.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

42 extracted references · 34 canonical work pages

  1. [41]

    Zeng, T.-H

    K.-H. Zeng, T.-H. Chen, C.-Y . Chuang, Y .-H. Liao, J. C. Niebles, and M. Sun. Leveraging video descriptions to learn video question answering. In AAAI, 2017

    work page 2017

  2. [3]

    Brysbaert, A

    M. Brysbaert, A. B. Warriner, and V . Kuperman. Concreteness ratings for 40 thousand generally known english word lemmas. Behavior Research Methods, 46(3):904, 2014

    work page 2014

  3. [1]

    Antol, A

    S. Antol, A. Agrawal, J. Lu, M. Mitchell, D. Batra, C. Lawrence Zitnick, and D. Parikh. Vqa: Visual question answering. In Proceedings of the IEEE International Conference on Computer Vision , pages 2425–2433, 2015

    work page 2015

  4. [2]

    S. Bird, E. Klein, and E. Loper. Natural language processing with Python: analyzing text with the natural language toolkit . ” O’Reilly Media, Inc.”, 2009

    work page 2009

  5. [4]

    K. Chen, J. Wang, L.-C. Chen, H. Gao, W. Xu, and R. Nevatia. Abc- cnn: An attention based convolutional neural network for visual question answering. arXiv preprint arXiv:1511.05960 , 2015

    arXiv 2015

  6. [5]

    Q. Chen, X. Zhu, Z. Ling, S. Wei, and H. Jiang. Enhancing and combining sequential and tree lstm for natural language inference. arXiv preprint arXiv:1609.06038, 2016

    arXiv 2016

  7. [6]

    A. Das, H. Agrawal, C. L. Zitnick, D. Parikh, and D. Batra. Human attention in visual question answering: Do humans and deep networks look at the same regions? arXiv preprint arXiv:1606.03556 , 2016

    arXiv 2016

  8. [7]

    Denil, L

    M. Denil, L. Bazzani, H. Larochelle, and N. de Freitas. Learning where to attend with deep architectures for image tracking. Neural computation, 24(8):2151–2184, 2012

    work page 2012

Show all 42 references
  1. [8]

    Eriguchi, K

    A. Eriguchi, K. Hashimoto, and Y . Tsuruoka. Tree-to-sequence atten- tional neural machine translation. arXiv preprint arXiv:1603.06075 , 2016

    2016 arXiv

  2. [9]

    Fellbaum

    C. Fellbaum. WordNet. Wiley Online Library, 1998

    1998

  3. [10]

    Z. Guo, L. Gao, J. Song, X. Xu, J. Shao, and H. T. Shen. Attention-based lstm with semantic consistency for videos captioning. In Proceedings of the 2016 ACM on Multimedia Conference , pages 357–361. ACM, 2016

    2016

  4. [11]

    K. M. Hermann, T. Kocisky, E. Grefenstette, L. Espeholt, W. Kay, M. Suleyman, and P. Blunsom. Teaching machines to read and comprehend. In Advances in Neural Information Processing Systems , pages 1693–1701, 2015

    2015

  5. [12]

    Ilievski, S

    I. Ilievski, S. Yan, and J. Feng. A focused dynamic attention model for visual question answering. arXiv preprint arXiv:1604.01485 , 2016

    2016 arXiv

  6. [13]

    Iyyer, J

    M. Iyyer, J. L. Boyd-Graber, L. M. B. Claudino, R. Socher, and H. Daum ´e III. A neural network for factoid question answering over paragraphs. In EMNLP, pages 633–644, 2014

    2014

  7. [14]

    Z. Jie, X. Liang, J. Feng, X. Jin, W. Lu, and S. Yan. Tree-structured reinforcement learning for sequential object localization. In Advances in Neural Information Processing Systems , pages 127–135, 2016

    2016

  8. [15]

    Kingma and J

    D. Kingma and J. Ba. Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 , 2014

    2014 arXiv

  9. [16]

    Klein and C

    D. Klein and C. D. Manning. Accurate unlexicalized parsing. In Pro- ceedings of the 41st Annual Meeting on Association for Computational Linguistics-V olume 1, pages 423–430. Association for Computational Linguistics, 2003

    2003

  10. [17]

    Li, M.-T

    J. Li, M.-T. Luong, D. Jurafsky, and E. Hovy. When are tree struc- tures necessary for deep learning of representations? arXiv preprint arXiv:1503.00185, 2015

    2015 arXiv

  11. [18]

    Y . Li, Y . Song, L. Cao, J. Tetreault, L. Goldberg, A. Jaimes, and J. Luo. Tgif: A new dataset and benchmark on animated gif description. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 4641–4650, 2016

    2016

  12. [19]

    J. Lu, J. Yang, D. Batra, and D. Parikh. Hierarchical question-image co-attention for visual question answering. In Advances In Neural Information Processing Systems , pages 289–297, 2016

    2016

  13. [20]

    Mazaheri, D

    A. Mazaheri, D. Zhang, and M. Shah. Video fill in the blank with merging lstms. arXiv preprint arXiv:1610.04062 , 2016

    2016 arXiv

  14. [21]

    V . Mnih, N. Heess, A. Graves, et al. Recurrent models of visual attention. In Advances in neural information processing systems , pages 2204– 2212, 2014

    2014

  15. [22]

    Pennington, R

    J. Pennington, R. Socher, and C. D. Manning. Glove: Global vectors for word representation. In EMNLP, volume 14, pages 1532–1543, 2014

    2014

  16. [23]

    K. J. Shih, S. Singh, and D. Hoiem. Where to look: Focus regions for visual question answering. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition , pages 4613–4621, 2016

    2016

  17. [24]

    Simonyan and A

    K. Simonyan and A. Zisserman. Very deep convolutional networks for large-scale image recognition. arXiv preprint arXiv:1409.1556 , 2014

    2014 arXiv

  18. [25]

    Sukhbaatar, J

    S. Sukhbaatar, J. Weston, R. Fergus, et al. End-to-end memory networks. IEEE TRANSACTIONS ON IMAGE PROCESSING. VOL. **, NO. **, JULY 2017 12 In Advances in neural information processing systems , pages 2440– 2448, 2015

    2017

  19. [26]

    K. S. Tai, R. Socher, and C. D. Manning. Improved semantic represen- tations from tree-structured long short-term memory networks. arXiv preprint arXiv:1503.00075, 2015

    2015 arXiv

  20. [27]

    Tapaswi, Y

    M. Tapaswi, Y . Zhu, R. Stiefelhagen, A. Torralba, R. Urtasun, and S. Fidler. Movieqa: Understanding stories in movies through question- answering. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition , pages 4631–4640, 2016

    2016

  21. [28]

    Teng and Y

    Z. Teng and Y . Zhang. Bidirectional tree-structured lstm with head lexicalization. arXiv preprint arXiv:1611.06788 , 2016

    2016 arXiv

  22. [29]

    K. Tu, M. Meng, M. W. Lee, T. E. Choe, and S.-C. Zhu. Joint video and text parsing for understanding events and answering queries. IEEE MultiMedia, 21(2):42–70, 2014

    2014

  23. [30]

    Venugopalan, M

    S. Venugopalan, M. Rohrbach, J. Donahue, R. Mooney, T. Darrell, and K. Saenko. Sequence to sequence-video to text. In Proceedings of the IEEE International Conference on Computer Vision , pages 4534–4542, 2015

    2015

  24. [31]

    Wu and M

    Z. Wu and M. Palmer. Verbs semantics and lexical selection. In Pro- ceedings of the 32nd annual meeting on Association for Computational Linguistics, pages 133–138. Association for Computational Linguistics, 1994

    1994

  25. [32]

    Xiong, S

    C. Xiong, S. Merity, and R. Socher. Dynamic memory networks for visual and textual question answering. arXiv, 1603, 2016

    2016

  26. [33]

    Xu and K

    H. Xu and K. Saenko. Ask, attend and answer: Exploring question- guided spatial attention for visual question answering. In European Conference on Computer Vision , pages 451–466. Springer, 2016

    2016

  27. [34]

    K. Xu, J. Ba, R. Kiros, K. Cho, A. C. Courville, R. Salakhutdinov, R. S. Zemel, and Y . Bengio. Show, attend and tell: Neural image caption generation with visual attention. In ICML, volume 14, pages 77–81, 2015

    2015

  28. [35]

    Z. Yang, X. He, J. Gao, L. Deng, and A. Smola. Stacked attention networks for image question answering. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition , pages 21–29, 2016

    2016

  29. [36]

    L. Yao, A. Torabi, K. Cho, N. Ballas, C. Pal, H. Larochelle, and A. Courville. Describing videos by exploiting temporal structure. In Proceedings of the IEEE international conference on computer vision , pages 4507–4515, 2015

    2015

  30. [37]

    Q. You, L. Cao, H. Jin, and J. Luo. Robust visual-textual sentiment analysis: When attention meets tree-structured recursive neural networks. In Proceedings of the 2016 ACM on Multimedia Conference , pages 1008–1017. ACM, 2016

    2016

  31. [38]

    Q. You, H. Jin, Z. Wang, C. Fang, and J. Luo. Image captioning with semantic attention. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition , pages 4651–4659, 2016

    2016

  32. [39]

    H. Yu, J. Wang, Z. Huang, Y . Yang, and W. Xu. Video paragraph captioning using hierarchical recurrent neural networks. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition , pages 4584–4593, 2016

    2016

  33. [40]

    L. Yu, E. Park, A. C. Berg, and T. L. Berg. Visual madlibs: Fill in the blank description generation and question answering. In Proceedings of the IEEE International Conference on Computer Vision , pages 2461– 2469, 2015

    2015

  34. [42]

    Y . Zhu, O. Groth, M. Bernstein, and L. Fei-Fei. Visual7w: Grounded question answering in images. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition , pages 4995–5004, 2016

    2016

Pith tools

Reviewed August 14, 2026 · model on record in the stance chip above.