Pith. sign in

REVIEW 3 major objections 5 minor 47 references

Replacing a hard emotion tree with a continuous graph of catalog and lexical nodes, and coupling generation with multi-level emotion supervision, enables emotional video captioning that is both error-resilient and multi-emotion-aware.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · deepseek-v4-flash

2026-08-03 14:41 UTC pith:64W7HGWB

load-bearing objection SAGML is a well-engineered soft-graph alternative to hard emotion-tree masking, but the paper's central error-resilience claim is not directly tested. the 3 major comments →

arxiv 2607.29045 v1 pith:64W7HGWB submitted 2026-07-31 cs.CV

Adaptive Emotional Video Captioning via Affective Heterogeneous Graph Reasoning and Multi-task Joint Learning

Junbo Wang , Liangyu Fu , Yuke Li , Xuecheng Wu , Zhiyong Wang This is my paper
classification cs.CV
keywords emotional video captioningaffective heterogeneous graphsoft gateemotion distribution learningmulti-task learninglarge language modelEmVidCapvision-language generation
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

This paper argues that the standard tree-structured emotion prior in emotional video captioning is too brittle: when the model mispredicts a coarse emotion category, the associated hard mask can permanently block the correct fine-grained emotion word, and mixed emotions cannot be represented. The proposed SAGML replaces the tree with a heterogeneous graph of catalog-level emotion categories and lexical-level emotion words, with edges encoding category proximity, category–word co-occurrence, and word–word similarity. The predicted catalog distribution is propagated over the graph to produce a strictly positive 'soft gate' that biases—but never deletes—lexical candidates during visual attention. The affective representation is fed to an instruction-tuned language model decoder whose prompt states are also supervised with catalog and lexical distribution losses. On the standard EmVidCap benchmark, SAGML reports consistent gains over existing methods, including +6.7 in Acc_sw, +7.7 in Acc_c, +9.0 in CIDEr, and +8.6 in CFS on the full split.

Core claim

The paper's central claim is that the emotion prior in EVC should be a weighted graph, not a discrete tree, and that the graph's output should modulate lexical selection continuously. SAGML builds an adjacency over 34 catalog emotion categories and 179 lexical emotion words from three relation types—catalog–catalog proximity, catalog–word corpus co-occurrence, and word–word embedding/co-occurrence similarity. Given a video, a catalog distribution P_c is predicted and converted into a lexical activation r_w = (P_c A_cc) A_cw, then into a logistic gate g with all entries positive. The gate's log is added to visual-to-lexical attention scores, so a low-prior word is down-weighted but not remove

What carries the argument

The affective heterogeneous graph: a two-level graph with catalog nodes (34 psychological emotion categories) and lexical nodes (179 daily emotion words), whose adjacency A_graph (Eq. 14) fuses catalog–catalog proximity, category–word corpus co-occurrence, and word–word similarity. The topology-driven soft gate: r_w = (P_c A_cc) A_cw, mapped through a linear layer and logistic activation to a strictly positive gate g (Eqs. 22–23). The graph-biased attention: log g is added to the visual-to-lexical attention logits (Eq. 25), making the prior a continuous bias rather than a mask. The prompt-level multi-task objective: catalog and lexical heads on the pooled prompt hidden states of the language

Load-bearing premise

The load-bearing premise is that the category–word co-occurrence statistics extracted from the training captions (the A_cw matrix) form a reliable bridge from a video's predicted catalog emotion to the correct lexical words; if those corpus statistics over-weight frequent emotion words or encode annotation biases, the soft gate will bias generation toward corpus-frequent emotions and the claimed error-resilience will not hold on genuinely new videos.

What would settle it

Take a video whose correct emotion word is rare or absent in the training captions, feed the model a deliberately wrong but plausible catalog category, and check whether the generated caption still contains the visually supported rare emotion word; if it is consistently suppressed, the soft gate is dominated by corpus statistics rather than being error-resilient.

Watch this falsifier — get emailed when new claim-graph text bears on it.

If this is right

  • On the full EmVidCap benchmark, SAGML surpasses the strongest comparison method by 6.7 points in Acc_sw, 7.7 in Acc_c, 9.0 in CIDEr, and 8.6 in CFS, with consistent gains on the small and long splits.
  • Because the soft gate is strictly positive, a wrong coarse-category prediction no longer isolates the correct lexical emotion word; it remains available for visual rescue.
  • The ablation of graph relations shows that the word–word adjacency is the largest contributor: without it, CIDEr drops from 76.9 to 52.1, so lexical topology is a primary driver of the gains.
  • Prompt-level catalog and lexical distribution supervision improves emotion accuracy and caption semantic quality simultaneously, indicating that multi-task emotion learning and LLM decoding are complementary.
  • The framework's gains are not pinned to one visual backbone: it outperforms prior methods with pre-extracted appearance-only, appearance+motion, and vision-language aligned features.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • A reader could test whether the same soft-gating mechanism transfers to other hierarchical prediction tasks (e.g., fine-grained object or action taxonomies) where hard pruning currently truncates alternatives.
  • The paper's error-resilience claim is validated only end-to-end on the same benchmark used to build the corpus statistics; one should not assume the gate's recoverability holds on out-of-corpus emotion words or annotation styles.
  • Since the word–word graph mixes embedding similarity with corpus co-occurrence, a natural ablative extension is to measure how much of the CIDEr gain comes from using the language model's own embeddings versus the external word vector and co-occurrence statistics.
  • The design suggests a general recipe: use a structured attribute prior as a soft bias in an LLM prompt and supervise the prompt states explicitly; this could be transferred to stylistic or attribute-constrained captioning beyond emotion.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. The paper proposes SAGML, an emotional video captioning framework that replaces the fixed tree-structured emotion prior of prior EVC methods with a heterogeneous affective graph containing catalog-level and lexical-level emotion nodes. A predicted catalog distribution is propagated through category-category and category-word matrices to form a continuous soft gate, which is added as a log-bias to visual-to-lexical attention. The visual and affective tokens are fed as a prefix into a LoRA-adapted Qwen2.5 decoder, with additional catalog- and lexical-level distribution heads. The model is trained jointly with caption generation and emotion distribution losses. Experiments on EmVidCap-S, EmVidCap-L, and EmVidCap report consistent improvements over published baselines, with the largest gains on the combined split.

Significance. The core idea—replacing hard top-K masking with a soft graph-derived gate—is well motivated, and the architecture is described in complete detail with explicit equations. If the reported results are reproducible, SAGML is a strong new baseline for EVC and one of the first to adapt an LLM decoder to this task with explicit emotion distribution supervision. The paper is transparent about hyperparameters and includes extensive ablations of graph relations, losses, visual features, and decoder choices. However, the empirical support for the central 'error-resilient soft gate' claim is incomplete, and the headline comparison with prior methods is confounded by the choice of CLIP features and Qwen backbone. The method itself is technically sound, but the experiments need sharpening to substantiate the main contribution.

major comments (3)
  1. [§III.F-G, §IV.G (Tables II and V)] The paper's central novelty is that the soft gate preserves recoverability of visually supported lexical alternatives compared to a hard catalog-to-word mask. This property is never directly tested. The ablations compare a no-gate variant to the soft gate (Table II) and remove individual graph relations (Table V), but there is no variant with a hard top-K mask in place of Eq. 23, nor any experiment that perturbs the predicted catalog distribution P_c to check whether correct lexical words survive. The only direct evidence is three qualitative examples in Fig. 3. Please add a hard-mask ablation and/or a catalog-error injection study; without it, the error-resilience claim is not supported by controlled evidence.
  2. [§IV.D, Tables I and III] The abstract claims SAGML surpasses the strongest comparison method by 9.0 CIDEr on EmVidCap. However, Table I transcribes results from heterogeneous settings (different visual features and decoders), and Table III shows that replacing CLIP with ResNet-152 drops SAGML's CIDEr from 76.9 to 65.8, i.e., below MM-ECPE's 67.9 on the same benchmark. Thus the CIDEr advantage is largely attributable to the CLIP backbone, not to the proposed graph. Please provide a comparison with prior EVC components under the same CLIP+Qwen backbone (or at least report the graph's marginal gain over a strong backbone-only baseline). If no such comparison is available, the SOTA claim in the abstract should be reworded.
  3. [§III.D, Eq. 9] The catalog-catalog adjacency A_cc is defined via a distance d_cyc over the fixed index order of the 34 categories. The paper describes this as 'affective proximity,' but the index order is not justified as psychologically meaningful, and the ablation in Table V shows that removing A_cc reduces CIDEr by 10.4 points. This makes the arbitrary index ordering a load-bearing part of the model. Please either justify the ordering (e.g., by reference to Plutchik's wheel), construct A_cc from data such as corpus co-occurrence of catalog labels, or explicitly discuss it as a flat cyclic prior.
minor comments (5)
  1. [Fig. 1 caption] Typo: 'Lexcial' should be 'Lexical'.
  2. [§IV.C] Typo: '0.1smoothing' should be '0.1 smoothing'.
  3. [Eq. 10] The notation eAcw is introduced without a clear definition; please use a consistent symbol such as \tilde A_cw or define it explicitly.
  4. [§IV.C / Contributions] The paper states 'we fine-tuned a large language model,' but the model is adapted with LoRA; consider saying 'parameter-efficiently adapted' to be precise.
  5. [§IV.E] No confidence intervals or multiple-seed runs are reported. Given the small test sets (516 videos in EmVidCap), please include variance estimates or explicitly acknowledge this limitation.

Circularity Check

0 steps flagged

No significant circularity: the graph statistics, soft gate, and EDL losses are standard supervised training components, and the benchmark results are evaluated on held-out references.

full rationale

The paper's derivation chain is self-contained with respect to the benchmark. The graph adjacency matrices A_cc, A_cw, and A_ww (Eqs. 8-13) are constructed from the fixed emotion lexicon, Qwen/GloVe embedding spaces, and training-caption co-occurrence statistics. The soft lexical gate (Eqs. 22-23) is a deterministic function of the model's own predicted catalog distribution P_c and these corpus-derived matrices; it is not fitted to the test reference captions. The caption-generation objective and the emotion-distribution losses (Eqs. 33-35) are supervised on training captions in the normal way, so the final metrics are computed against held-out references and are not forced by construction. No load-bearing uniqueness theorem or ansatz is imported from the authors' prior work; the only possible self-citation is a peripheral factual-captioning reference (M3, [12]) that is not used to justify SAGML's central soft-gating claim. The lack of a hard-mask ablation or catalog-error-injection experiment weakens direct empirical support for the error-resilience narrative, but that is an evidence-strength concern, not circularity.

Axiom & Free-Parameter Ledger

6 free parameters · 4 axioms · 0 invented entities

SAGML rests on hand-set graph constants and corpus-derived semantic relations. The architecture makes a qualitative claim about error recovery, but no formal proof or isolated experiment demonstrates that a visually supported word can actually overcome a low gate value. Performance claims are empirical and benchmark-bound.

free parameters (6)
  • Gaussian width sigma for catalog-catalog adjacency A_cc = 1.5
    Eq. 9; hand-set, no sensitivity analysis; controls smoothing over catalog index order.
  • Additive smoothing alpha for corpus adjacency = 0.1
    Eqs. 10 and 12; chosen by hand; prevents zero entries in A_cw and C_ww.
  • Lexical adjacency fusion weights beta_L, beta_G, beta_R = 0.5, 0.3, 0.2
    Eq. 13; hand-set weights for Qwen similarity, GloVe similarity, and corpus co-occurrence.
  • Cosine threshold tau for lexical edges = 0.5
    Eq. 11; selected empirically on the test split in Table VII; no held-out validation.
  • EDL loss weight lambda schedule = 0.2 for epochs 1-5, 0.5 after
    Section IV-C; hand-scheduled; no ablation reported.
  • LoRA rank/scaling/dropout = 8 / 16 / 0.05
    Section III-H; decoder adaptation hyperparameters not central to the graph claim but needed to reproduce.
axioms (4)
  • domain assumption The 34-category/179-word EmVidCap emotion lexicon and its category-word mappings are valid psychological priors.
    Sections III-A and IV-B; graph nodes and EDL targets inherit this lexicon; if the mappings are wrong, gate and labels share the error.
  • domain assumption Qwen2.5 token embeddings provide a semantically meaningful initialization for emotion nodes, and its prompt states carry affective evidence.
    Sections III-C and III-I; node features are averages of Qwen input embeddings, and EDL heads read pooled prompt states. This is not independently validated.
  • domain assumption Corpus co-occurrence and embedding cosine similarity in the training set approximate affective relations for unseen videos.
    Section III-D, Eqs. 10-13; graph topology is static from the training corpus; transfer to test videos is assumed.
  • domain assumption CLIP ViT-B/32 frame features alone capture the affect-relevant visual content.
    Section III-B; default has no motion stream. Motion variants in Table III show motion helps overlap metrics, but the central comparison relies on CLIP-only features.

pith-pipeline@v1.3.0-daily-deepseek · 18788 in / 13060 out tokens · 131119 ms · 2026-08-03T14:41:39.640773+00:00 · methodology

0 comments
read the original abstract

Emotional video captioning (EVC) aims to describe a video with both factual correctness and affective expressiveness. It requires a model to perceive subtle, ambiguous, and temporally varying emotional cues and translate them into natural language without weakening objective visual content. Existing methods have progressively introduced contextual attention, emotion interpretation, emotion priors, dynamic emotion perception and emotion-cause reasoning. Nevertheless, most of them still depend on either global emotion vectors or rigid hierarchical priors. In recent methods, the tree-structured emotion prior establishes a coarse-to-fine connection between psychological emotion categories and daily emotion words, but its hard subordinate masking may irreversibly suppress correct lexical emotions once the coarse category prediction is inaccurate. It is also limited in representing mixed or overlapping emotions that frequently occur in real videos. To address the issues, we propose SAGML, an adaptive EVC framework via affective heterogeneous graph and multi-task language modeling. Instead of treating the emotion prior as a discrete tree, SAGML constructs a soft affective heterogeneous graph containing catalog-level emotion nodes and lexical-level emotion word nodes. The soft gate is injected into video-to-emotion graph attention as a continuous bias, allowing visually supported lexical emotions to remain recoverable rather than being removed by a hard mask. The resulting affective representation is fed together with visual tokens into a causal language decoder, while dual catalog and lexical heads impose explicit emotion distribution learning on the prompt hidden states. The overall model is trained with a joint objective that combines autoregressive caption generation and emotion distribution supervision. SAGML provides an error-resilient and multi-emotion-aware baseline for EVC.

Figures

Figures reproduced from arXiv: 2607.29045 by Junbo Wang, Liangyu Fu, Xuecheng Wu, Yuke Li, Zhiyong Wang.

Figure 1
Figure 1. Figure 1: Comparison between the (a) previous EVC framework and (b) our [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. Figure 2: Overview of SAGML. Pre-extracted CLIP features are contextualized as visual tokens [PITH_FULL_IMAGE:figures/full_fig_p005_2.png] view at source ↗
Figure 3
Figure 3. Figure 3: Qualitative comparison between DCGN and SAGML on EmVidCap. SAGML more accurately associates emotions with visual events, preserves action [PITH_FULL_IMAGE:figures/full_fig_p009_3.png] view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

47 extracted references · 1 linked inside Pith

  1. [1]

    Long-term recurrent convolutional networks for visual recognition and description,

    J. Donahue, L. A. Hendricks, S. Guadarrama, M. Rohrbach, S. Venu- gopalan, K. Saenko, and T. Darrell, “Long-term recurrent convolutional networks for visual recognition and description,” inProceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2015, pp. 2625–2634

    2015

  2. [2]

    Sequence to sequence—video to text,

    S. Venugopalan, M. Rohrbach, J. Donahue, R. Mooney, T. Darrell, and K. Saenko, “Sequence to sequence—video to text,” inProceedings of the IEEE International Conference on Computer Vision, 2015, pp. 4534– 4542

    2015

  3. [3]

    Hierarchical recurrent neural encoder for video representation with application to captioning,

    P. Pan, Z. Xu, Y . Yang, F. Wu, and Y . Zhuang, “Hierarchical recurrent neural encoder for video representation with application to captioning,” inProceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2016, pp. 1029–1038

    2016

  4. [4]

    Describing videos by exploiting temporal structure,

    L. Yao, A. Torabi, K. Cho, N. Ballas, C. Pal, H. Larochelle, and A. Courville, “Describing videos by exploiting temporal structure,” in Proceedings of the IEEE International Conference on Computer Vision, 2015, pp. 4507–4515. JOURNAL OF LATEX CLASS FILES, VOL. 14, NO. 8, AUGUST 2021 12

    2015

  5. [5]

    Emotion expression with fact transfer for video description,

    H. Wang, P. Tang, Q. Li, and M. Cheng, “Emotion expression with fact transfer for video description,”IEEE Transactions on Multimedia, vol. 24, pp. 715–727, 2022

    2022

  6. [6]

    Contextual attention network for emotional video captioning,

    P. Song, D. Guo, J. Cheng, and M. Wang, “Contextual attention network for emotional video captioning,”IEEE Transactions on Multimedia, vol. 25, pp. 1858–1867, 2023

    2023

  7. [7]

    Emotional video captioning with vision-based emotion interpretation network,

    P. Song, D. Guo, X. Yang, S. Tang, and M. Wang, “Emotional video captioning with vision-based emotion interpretation network,”IEEE Transactions on Image Processing, vol. 33, pp. 1122–1135, 2024

    2024

  8. [8]

    Emotion- prior awareness network for emotional video captioning,

    P. Song, D. Guo, X. Yang, S. Tang, E. Yang, and M. Wang, “Emotion- prior awareness network for emotional video captioning,” inProceedings of the 31st ACM International Conference on Multimedia, 2023, pp. 589–600

    2023

  9. [9]

    HEART: Emotionally grounded video captioning via hierarchical emotion-aligned representation,

    T. Han, Y . Gong, S. Zhao, M. Tan, Z. Yu, and H. Yao, “HEART: Emotionally grounded video captioning via hierarchical emotion-aligned representation,”IEEE Transactions on Affective Computing, vol. 17, no. 2, pp. 1709–1720, 2026

    2026

  10. [10]

    Multi- round mutual emotion-cause pair extraction for emotion-attributed video captioning,

    C. Ye, W. Chen, P. Song, X. Liu, L. Zhang, and Z. Mao, “Multi- round mutual emotion-cause pair extraction for emotion-attributed video captioning,” inProceedings of the 33rd ACM International Conference on Multimedia, 2025, pp. 3320–3329

    2025

  11. [11]

    Dual-path collaborative generation network for emotional video captioning,

    C. Ye, W. Chen, J. Li, L. Zhang, and Z. Mao, “Dual-path collaborative generation network for emotional video captioning,” inProceedings of the 32nd ACM International Conference on Multimedia, 2024, pp. 496– 505

    2024

  12. [12]

    M3: Multimodal memory modelling for video captioning,

    J. Wang, W. Wang, Y . Huang, L. Wang, and T. Tan, “M3: Multimodal memory modelling for video captioning,” inProceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2018, pp. 7512–7520

    2018

  13. [13]

    Reconstruction network for video captioning,

    B. Wang, L. Ma, W. Zhang, and W. Liu, “Reconstruction network for video captioning,” inProceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2018, pp. 7622–7631

    2018

  14. [14]

    Semantic grouping network for video captioning,

    H. Ryu, S. Kang, H. Kang, and C. D. Yoo, “Semantic grouping network for video captioning,” inProceedings of the AAAI Conference on Artificial Intelligence, vol. 35, no. 3, 2021, pp. 2514–2522

    2021

  15. [15]

    SwinBERT: End-to-end transformers with sparse attention for video captioning,

    K. Lin, L. Li, C.-C. Lin, F. Ahmed, Z. Gan, Z. Liu, Y . Lu, and L. Wang, “SwinBERT: End-to-end transformers with sparse attention for video captioning,” inProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2022, pp. 17 949–17 958

    2022

  16. [16]

    Dense- captioning events in videos,

    R. Krishna, K. Hata, F. Ren, L. Fei-Fei, and J. C. Niebles, “Dense- captioning events in videos,” inProceedings of the IEEE International Conference on Computer Vision, 2017, pp. 706–715

    2017

  17. [17]

    SentiBank: Large-scale ontology and classifiers for detecting sentiment and emotions in visual content,

    D. Borth, T. Chen, R. Ji, and S.-F. Chang, “SentiBank: Large-scale ontology and classifiers for detecting sentiment and emotions in visual content,” inProceedings of the 21st ACM International Conference on Multimedia, 2013, pp. 459–460

    2013

  18. [18]

    SentiCap: Generating image de- scriptions with sentiments,

    A. Mathews, L. Xie, and X. He, “SentiCap: Generating image de- scriptions with sentiments,” inProceedings of the AAAI Conference on Artificial Intelligence, vol. 30, no. 1, 2016, pp. 3574–3580

    2016

  19. [19]

    StyleNet: Generating attractive visual captions with styles,

    C. Gan, Z. Gan, X. He, J. Gao, and L. Deng, “StyleNet: Generating attractive visual captions with styles,” inProceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2017, pp. 3137–3146

    2017

  20. [20]

    “factual

    T. Chen, Z. Zhang, Q. You, C. Fang, Z. Wang, H. Jin, and J. Luo, ““factual” or “emotional”: Stylized image captioning with adaptive learning and attention,” inProceedings of the European Conference on Computer Vision, 2018, pp. 519–535

    2018

  21. [21]

    Emotion-oriented cross-modal prompting and alignment for human- centric emotional video captioning,

    Y . Wang, Y . Liu, S. Zhou, Y . Huang, C. Tang, W. Zhou, and Z. Chen, “Emotion-oriented cross-modal prompting and alignment for human- centric emotional video captioning,”IEEE Transactions on Multimedia, vol. 27, pp. 3766–3780, 2025

    2025

  22. [22]

    Context based emotion recognition using EMOTIC dataset,

    R. Kosti, J. M. Alvarez, A. Recasens, and A. Lapedriza, “Context based emotion recognition using EMOTIC dataset,”IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 42, no. 11, pp. 2755– 2766, 2020

    2020

  23. [23]

    Predicting emotions in user-generated videos,

    Y .-G. Jiang, B. Xu, and X. Xue, “Predicting emotions in user-generated videos,” inProceedings of the AAAI Conference on Artificial Intelli- gence, vol. 28, no. 1, 2014, pp. 73–79

    2014

  24. [24]

    An end-to-end visual-audio attention network for emotion recognition in user-generated videos,

    S. Zhao, Y . Ma, Y . Gu, J. Yang, T. Xing, P. Xu, R. Hu, H. Chai, and K. Keutzer, “An end-to-end visual-audio attention network for emotion recognition in user-generated videos,” inProceedings of the AAAI Conference on Artificial Intelligence, vol. 34, no. 1, 2020, pp. 303–311

    2020

  25. [25]

    Affect2MM: Affective analysis of multimedia content using emotion causality,

    T. Mittal, P. Mathur, A. Bera, and D. Manocha, “Affect2MM: Affective analysis of multimedia content using emotion causality,” inProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recogni- tion, 2021, pp. 5661–5671

    2021

  26. [26]

    Learning visual emotion representations from web data,

    Z. Wei, J. Zhang, Z. Lin, J.-Y . Lee, N. Balasubramanian, M. Hoai, and D. Samaras, “Learning visual emotion representations from web data,” inProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2020, pp. 13 106–13 115

    2020

  27. [27]

    Plutchik,Emotion: A Psychoevolutionary Synthesis

    R. Plutchik,Emotion: A Psychoevolutionary Synthesis. New York, NY , USA: Harper & Row, 1980

    1980

  28. [28]

    Image emotion distribution learning with graph convolutional networks,

    T. He and X. Jin, “Image emotion distribution learning with graph convolutional networks,” inProceedings of the 2019 ACM International Conference on Multimedia Retrieval, 2019, pp. 382–390

    2019

  29. [29]

    A circular-structured rep- resentation for visual emotion distribution learning,

    J. Yang, J. Li, L. Li, X. Wang, and X. Gao, “A circular-structured rep- resentation for visual emotion distribution learning,” inProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2021, pp. 4237–4246

    2021

  30. [30]

    Semi-supervised classification with graph convolutional networks,

    T. N. Kipf and M. Welling, “Semi-supervised classification with graph convolutional networks,” inInternational Conference on Learning Rep- resentations, 2017

    2017

  31. [31]

    Modeling relational data with graph convolutional networks,

    M. Schlichtkrull, T. N. Kipf, P. Bloem, R. van den Berg, I. Titov, and M. Welling, “Modeling relational data with graph convolutional networks,” inThe Semantic Web, ser. Lecture Notes in Computer Science, vol. 10843. Springer, 2018, pp. 593–607

    2018

  32. [32]

    Graph attention networks,

    P. Veli ˇckovi´c, G. Cucurull, A. Casanova, A. Romero, P. Li `o, and Y . Bengio, “Graph attention networks,” inInternational Conference on Learning Representations, 2018

    2018

  33. [33]

    Learning transferable visual models from natural language supervi- sion,

    A. Radford, J. W. Kim, C. Hallacy, A. Ramesh, G. Goh, S. Agarwal, G. Sastry, A. Askell, P. Mishkin, J. Clark, G. Krueger, and I. Sutskever, “Learning transferable visual models from natural language supervi- sion,” inProceedings of the 38th International Conference on Machine Learning, ser. Proceedings of Machine Learning Research, vol. 139, 2021, pp. 8748–8763

    2021

  34. [34]

    Multimodal few-shot learning with frozen language models,

    M. Tsimpoukelli, J. Menick, S. Cabi, S. M. A. Eslami, O. Vinyals, and F. Hill, “Multimodal few-shot learning with frozen language models,” inAdvances in Neural Information Processing Systems, vol. 34, 2021, pp. 200–212

    2021

  35. [35]

    Flamingo: A visual language model for few-shot learning,

    J.-B. Alayrac, J. Donahue, P. Luc, A. Miech, I. Barr, Y . Hasson, K. Lenc, A. Mensch, K. Millican, M. Reynolds, R. Ring, E. Rutherford, S. Cabi, T. Han, Z. Gong, S. Samangooei, M. Monteiro, J. L. Menick, S. Borgeaud, A. Brock, A. Nematzadeh, S. Sharifzadeh, M. Bi ´nkowski, R. Barreira, O. Vinyals, A. Zisserman, and K. Simonyan, “Flamingo: A visual languag...

    2022

  36. [36]

    BLIP-2: Bootstrapping language- image pre-training with frozen image encoders and large language mod- els,

    J. Li, D. Li, S. Savarese, and S. Hoi, “BLIP-2: Bootstrapping language- image pre-training with frozen image encoders and large language mod- els,” inProceedings of the 40th International Conference on Machine Learning, ser. Proceedings of Machine Learning Research, vol. 202, 2023, pp. 19 730–19 742

    2023

  37. [37]

    Qwen2.5 technical report,

    Qwen, :, A. Yang, B. Yang, B. Zhang, B. Hui, B. Zheng, B. Yu, C. Li, D. Liu, F. Huang, H. Wei, H. Lin, J. Yang, J. Tu, J. Zhang, J. Yang, J. Yang, J. Zhou, J. Lin, K. Dang, K. Lu, K. Bao, K. Yang, L. Yu, M. Li, M. Xue, P. Zhang, Q. Zhu, R. Men, R. Lin, T. Li, T. Tang, T. Xia, X. Ren, X. Ren, Y . Fan, Y . Su, Y . Zhang, Y . Wan, Y . Liu, Z. Cui, Z. Zhang, ...

  38. [38]

    LoRA: Low-rank adaptation of large language models,

    E. J. Hu, Y . Shen, P. Wallis, Z. Allen-Zhu, Y . Li, S. Wang, L. Wang, and W. Chen, “LoRA: Low-rank adaptation of large language models,” inInternational Conference on Learning Representations, 2022

    2022

  39. [39]

    GloVe: Global vectors for word representation,

    J. Pennington, R. Socher, and C. Manning, “GloVe: Global vectors for word representation,” inProceedings of the 2014 Conference on Empirical Methods in Natural Language Processing, 2014, pp. 1532– 1543

    2014

  40. [40]

    Collecting highly parallel data for paraphrase evaluation,

    D. Chen and W. Dolan, “Collecting highly parallel data for paraphrase evaluation,” inProceedings of the 49th Annual Meeting of the Asso- ciation for Computational Linguistics: Human Language Technologies. Association for Computational Linguistics, 2011, pp. 190–200

    2011

  41. [41]

    BLEU: A method for automatic evaluation of machine translation,

    K. Papineni, S. Roukos, T. Ward, and W.-J. Zhu, “BLEU: A method for automatic evaluation of machine translation,” inProceedings of the 40th Annual Meeting of the Association for Computational Linguistics. Association for Computational Linguistics, 2002, pp. 311–318

    2002

  42. [42]

    METEOR: An automatic metric for MT evaluation with improved correlation with human judgments,

    S. Banerjee and A. Lavie, “METEOR: An automatic metric for MT evaluation with improved correlation with human judgments,” inPro- ceedings of the ACL Workshop on Intrinsic and Extrinsic Evaluation Measures for Machine Translation and/or Summarization. Association for Computational Linguistics, 2005, pp. 65–72

    2005

  43. [43]

    ROUGE: A package for automatic evaluation of summaries,

    C.-Y . Lin, “ROUGE: A package for automatic evaluation of summaries,” inText Summarization Branches Out. Association for Computational Linguistics, 2004, pp. 74–81. JOURNAL OF LATEX CLASS FILES, VOL. 14, NO. 8, AUGUST 2021 13

    2004

  44. [44]

    CIDEr: Consensus-based image description evaluation,

    R. Vedantam, C. L. Zitnick, and D. Parikh, “CIDEr: Consensus-based image description evaluation,” inProceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2015, pp. 4566–4575

    2015

  45. [45]

    Deep residual learning for image recognition,

    K. He, X. Zhang, S. Ren, and J. Sun, “Deep residual learning for image recognition,” inProceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2016, pp. 770–778

    2016

  46. [46]

    Can spatiotemporal 3D CNNs retrace the history of 2D CNNs and ImageNet?

    K. Hara, H. Kataoka, and Y . Satoh, “Can spatiotemporal 3D CNNs retrace the history of 2D CNNs and ImageNet?” inProceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2018, pp. 6546–6555

    2018

  47. [2025]

    Available: https://arxiv.org/abs/2412.15115

    [Online]. Available: https://arxiv.org/abs/2412.15115

    Pith/arXiv arXiv