REVIEW 4 major objections 6 minor 32 references
Enhanced Multimodal Aspect-Based Sentiment Analysis by LLM-Generated Rationales
T0 review · 4 major / 6 minor · reviewed 2026-08-07 · deepseek-v4-flash
Pith's one-line read The paper proposes LRSA, a framework that feeds LLM-generated text and image rationales into small BART-based models via a dual cross-attention fusion, and reports consistent F1 gains over the base models on two MABSA benchmarks.
desk verdict A plausible idea for injecting LLM rationales into small MABSA models, but the main F1 gains are not yet trustworthy because the paper never says how samples with missing rationales are handled. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The load-bearing mechanism is the dual cross-attention module, which operates on stacked pairs (image features, image rationale) and (text features, text rationale). By forming all query-key-value products across the two stacked components, each updated representation contains both an intra-component self-attention term and a cross-component attention term, so the fused image features can draw on reasoning expressed in the image rationale, and the rationale features can draw on the concrete visual content. The same encoder shares parameters for the image–text pair and the two rationales, which the authors argue makes fusion more cohesive; the decoder then concatenates the fused features with the rationale features to keep the external explanation visible at generation time.
What would settle it
Re-run the MABSA evaluation on the complete Twitter2015 and Twitter2017 test sets, including the samples that had no LLM-generated rationale (using, say, a captioning fallback or the original text alone), and compute F1 for LRSA and the VLP/AoM baselines on the same full set. If the F1 advantage drops below statistical significance or reverses, the claimed gains come from an easier evaluation set rather than from the rationales.
Extended reading notes
Core claim
On its own terms, the paper establishes a recipe: use a multimodal LLM (Gemini 1.5 Pro) to generate a text rationale (an explanation of the sentence) and an image rationale (a description of the picture), then inject both into a small BART-based MABSA model. The dual cross-attention module treats each modality's hidden states and its rationale's hidden states as stacked query/key/value pairs, so that image features attend to image-rationale features and text features attend to text-rationale features, and both update one another. The updated features are added to the original encoder outputs, and the decoder receives the concatenation of fused image/text features with the rationale features. The authors report that this consistently outperforms the two base small models (VLP and AoM) as well as a long list of prior multimodal and text-only methods, across MABSA, MATE, and MASC metrics, and they show in ablations that removing the dual cross-attention or the decoder-side concatenation degrades performance. The intended consequence is that pre-trained small models can remain the decision-makers, with LLM rationales acting as a low-cost, model-agnostic injection that generalizes to at least two different small-model architectures.
Load-bearing premise
The reported F1 gains are only meaningful if LRSA and the baselines are evaluated on exactly the same test samples; the paper does not state whether the roughly one percent of samples for which the LLM produced no rationale were excluded from the evaluation.
Editorial extensions
If this is right
- LRSA can be dropped onto any BART-based MABSA model without changing its core training objective; the authors demonstrate this on two different base models, VLP and AoM.
- LLM-generated rationales provide a transferable source of fine-grained multimodal information, so gains are not limited to one dataset or one small model.
- The prompt design and rationale length matter: task-related hints (but not full aspect/sentiment predictions) and rationale length twice the image–text pair yield the best results, suggesting a practical recipe for other tasks.
- Because the small model still makes the final predictions, the approach keeps deployment cost low relative to using an LLM directly for inference.
- The dual cross-attention module is general—it can fuse any pair of feature sequences—so the same architecture could inject other auxiliary text (captions, knowledge bases) into multimodal encoders.
Reading between the lines
- The paper does not specify how samples where the LLM refused to generate rationales are handled; if they are dropped from evaluation, the reported gains would be inflated. A fair extension would be to test on the full test sets using a fallback (e.g., captioning model or no rationale) to see whether the gains persist.
- Since the best prompt explicitly does not ask the LLM to identify aspects or sentiments, this hints that the value of rationales is their descriptive grounding, not their task-specific correctness; a testable extension is to compare against rationales generated by an image-captioning model to isolate the contribution of LLM reasoning.
- The ablation study shows that both the dual cross-attention and the decoder-side concatenation contribute to the gains, but the paper does not analyze why symmetric updating helps; an extension could probe whether a one-directional fusion (rationale to modality only) still yields similar gains.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The manuscript presents LRSA, a framework for multimodal aspect-based sentiment analysis that augments two BART-based small language models (VLP and AoM) with rationales generated by a multimodal LLM (Gemini 1.5 Pro). For each image-text pair, the LLM produces a text rationale and an image rationale, which are concatenated with the input into the encoder. A dual cross-attention module is used to fuse the image/text features with the rationale features, and the fused features are passed to the decoder. Experiments on the Twitter2015 and Twitter2017 datasets report gains over the reproduced VLP and AoM baselines: e.g., 1.6% and 1.2% F1 improvement over VLP on the two datasets, and 1.9% and 1.4% over AoM (Section 4.3, Table 2). The paper also presents ablations and a study of prompt design and rationale length.
Significance. If the reported results are reproducible, the paper's central idea—leveraging LLM-generated free-text rationales as auxiliary signal for fine-tuned small models—is a reasonable and potentially useful contribution to MABSA. The paper's strengths include a clear architecture diagram, public benchmark evaluation, and ablations that test the contribution of the fusion module. However, the headline gains are small (1-2% F1) and rest on experimental details that are not fully disclosed, so the significance of the empirical claim is currently uncertain.
major comments (4)
- [4.1, Table 1] Table 1 reports LLM-processed sample counts that are smaller than the original dataset sizes for every split (e.g., Twitter2015 test 668/674, Twitter2017 test 585/586). Section 4.1 states that 'a small subset of data for which rationales could not be generated' was excluded, but it never specifies whether those samples are dropped from LRSA's training/evaluation or imputed with placeholder rationales. Since the architecture requires rationale tokens for every input (Section 3.2), dropping is the natural reading. If the LRSA results in Table 2 are obtained on the reduced test sets while the VLP and AoM baseline numbers are obtained on the full test sets, the reported F1 gains are not a fair comparison. The paper must state the exact evaluation protocol for both LRSA and the baselines, and ideally report results on a matched subset or on the full sets with imputed rationales.
- [4.2] The text notes that the reproduced VLP and AoM models 'did not match the outcomes reported in the paper,' and Table 2 uses these reproduced results (marked with *) as the baselines. This is a serious concern because it means the comparison is not against the published state of the art but against the authors' own re-implementations, whose behavior is unexplained. The authors should provide details of the reproduction (e.g., library versions, preprocessing, rationale handling for baselines) and show that the reproductions are consistent with the original papers. Without this, it is unclear whether the 1-2% gains reflect a real improvement or a discrepancy in the reproduction.
- [3.3, Eqs. (6)-(9)] The derivation of the dual cross-attention module is mathematically invalid as written. Eq. (6) defines the attention output with a softmax, but Eq. (7) omits the softmax and scaling and then expands the product as if the attention were a simple bilinear operation. With the softmax, the denominator couples the H and HL blocks, so the decomposition in Eqs. (8)-(9) does not follow. If the actual implementation uses the standard scaled dot-product attention, the equations misdescribe it; if it uses the linearized form, that should be stated explicitly and its consequences considered. Either way, the current description prevents faithful reproduction.
- [Abstract vs. Introduction] The abstract claims evaluation on 'three widely-used benchmarks,' but Section 4.1 and the experiments use only Twitter2015 and Twitter2017. The Introduction (contribution 3) correctly says 'two benchmark datasets.' This inconsistency should be corrected, and if a third benchmark was used, it should be reported.
minor comments (6)
- [2.1] There are duplicated words: 'to to capture' and 'to to enhance' should be 'to capture' and 'to enhance'.
- [4.3] The MASC discussion contains confusing comparisons; for example, 'LRSA on VLP outperforms the AoM results by 0.5% in accuracy on Twitter2017' should be reworded to compare LRSA directly with AoM.
- [Table 3] Some entries are missing or inconsistent (e.g., JML MASC F1 on Twitter2017 is '-'), and the table formatting makes it hard to read; the authors should ensure all values and column alignments are complete.
- [4.5] The rationale-length experiment is not clearly described; it is unclear how the prompts control the rationale length and how 'corresponds to that of the text-image pair' is defined in practice.
- [4.1] The paper does not report the number of independent runs or standard deviations for the main results, which is important for improvements as small as 1-2% F1.
- [3.2] The visual feature extractor is said to be ResNet with reference [4] to DeepSentiBank, but the specific ResNet variant and whether it is fine-tuned are not stated.
Circularity Check
Twitter2015 headline F1 is a test-set-selected value, not an independent prediction; the rest of the evaluation retains independent content.
-
fitted input called prediction
[Section 4.5 (prompt and rationale-length selection, Table 6) vs Section 4.3 (main results, Table 2)]
"Our experiments were conducted using the Twitter2015 dataset and considered the same length of rationale as the experimental background. The result shown in Table 6, indicate that the second prompt yield the most favorable outcomes. ... For our experiments, we utilized the Twitter2015 dataset and considered the second prompt as the experimental background. The results, shown in Table 6, indicate that the best effect is achieved when the total length of the text-image rationale feature is twice the average length of the text-image pair feature."
The final LRSA-on-VLP Twitter2015 F1 of 68.2 in Table 2 is exactly the value selected as best in Table 6. The prompt template and rationale length were chosen by maximizing F1 on the Twitter2015 test set, and the same test set is then used to report the headline improvement over VLP (68.2 vs 66.6). Thus the claimed 1.6-point F1 gain on Twitter2015 is a test-set-selected statistic rather than an out-of-sample prediction. The Twitter2017 result and the MATE/MASC sub-results were not directly tuned on their respective test sets, so the framework retains independent empirical content, but the dataset used for selection is presented as a benchmark outcome without disclosing that selection.
full rationale
The paper does not derive its predictions from a closed-form fit, and there is no load-bearing self-citation or imported uniqueness theorem. The architecture is a standard BART encoder-decoder with cross-attention, and the LLM rationales are generated by an external model, so the core mechanism is not circular by construction. The one concrete reduction is methodological: the prompt type and rationale length were selected using F1 on the Twitter2015 test set, and the best selected value is then reported as the Twitter2015 result in the main table. This is a fitted-input-called-prediction pattern for that benchmark. The missing-rationale handling (Table 1 shows fewer processed samples than original dataset sizes) is a serious evaluation-comparability threat, but it is not circularity because nothing in the model's definition is equivalent to the reported metric; it belongs to correctness risk rather than circular derivation. Given one partial test-set-selection issue but substantial independent results on Twitter2017 and MATE/MASC, the overall circularity score is 5.
Assumptions & free parameters
free parameters (3)
- rationale length multiplier =
2x average text-image pair length
- LLM prompt template =
Type 2: prompt with task-related hints
- training hyperparameters =
lr=7e-5; batch size=16/18; epochs=35
assumptions (3)
- ad hoc to paper The dual cross-attention decomposition in Eqs. (8)-(9) describes the implemented attention module, even though softmax and scaling are omitted from the derivation.
- domain assumption Examples where the LLM declined to generate rationales can be excluded or imputed without changing benchmark difficulty.
- domain assumption Gemini-1.5 Pro rationales contain no label leakage and are generated only from image-text pairs.
Cite this review
Pith. "Pith review of Enhanced Multimodal Aspect-Based Sentiment Analysis by LLM-Generated Rationales." pith.science (2026). https://pith.science/paper/YXJGGUS4
@misc{pith2026250514499,
author = {Pith},
title = {Pith review of: Enhanced Multimodal Aspect-Based Sentiment Analysis by LLM-Generated Rationales},
year = {2026},
howpublished = {\url{https://pith.science/paper/YXJGGUS4}},
note = {Machine review of arXiv:2505.14499}
}
read the original abstract
There has been growing interest in Multimodal Aspect-Based Sentiment Analysis (MABSA) in recent years. Existing methods predominantly rely on pre-trained small language models (SLMs) to collect information related to aspects and sentiments from both image and text, with an aim to align these two modalities. However, small SLMs possess limited capacity and knowledge, often resulting in inaccurate identification of meaning, aspects, sentiments, and their interconnections in textual and visual data. On the other hand, Large language models (LLMs) have shown exceptional capabilities in various tasks by effectively exploring fine-grained information in multimodal data. However, some studies indicate that LLMs still fall short compared to fine-tuned small models in the field of ABSA. Based on these findings, we propose a novel framework, termed LRSA, which combines the decision-making capabilities of SLMs with additional information provided by LLMs for MABSA. Specifically, we inject explanations generated by LLMs as rationales into SLMs and employ a dual cross-attention mechanism for enhancing feature interaction and fusion, thereby augmenting the SLMs' ability to identify aspects and sentiments. We evaluated our method using two baseline models, numerous experiments highlight the superiority of our approach on three widely-used benchmarks, indicating its generalizability and applicability to most pre-trained models for MABSA.
Figures
Reference graph
Works this paper leans on
-
[1]
Anthropic: Model card and evaluations for claude models (2023), https://www- files.anthropic.com/production/images/Model-Card-Claude-2.pdf, accessed: 2024- 06-13
work page 2023
-
[2]
Chen, G., Tian, Y., Song, Y.: Joint aspect extraction and sentiment analysis with directional graph convolutional networks. In: COLING. pp. 272–279 (Dec 2020)
work page 2020
-
[3]
Chen, H., Zhai, Z., Feng, F., Li, R., Wang, X.: Enhanced multi-channel graph convolutional network for aspect sentiment triplet extraction. In: ACL. pp. 2974– 2985 (May 2022)
work page 2022
-
[4]
arXiv preprint arXiv:1410.8586 (2014)
Chen, T., Borth, D., Darrell, T., Chang, S.F.: Deepsentibank: Visual sentiment concept classification with deep convolutional neural networks. arXiv preprint arXiv:1410.8586 (2014)
arXiv 2014
-
[5]
arXiv preprint arXiv:1810.04805 (2018)
Devlin, J., Chang, M.W., Lee, K., Toutanova, K.: Bert: Pre-training of deep bidi- rectional transformers for language understanding. arXiv preprint arXiv:1810.04805 (2018)
arXiv 2018
-
[6]
Hu, M., Peng, Y., Huang, Z., Li, D., Lv, Y.: Open-domain targeted sentiment analysis via span-based extraction and classification. In: ACL. pp. 537–546. Florence, Italy (Jul 2019)
work page 2019
- [7]
-
[8]
In: Proceedings of the 29th ACM International Conference on Multimedia
Khan, Z., Fu, Y.: Exploiting bert for multimodal target sentiment classification through input space translation. In: Proceedings of the 29th ACM International Conference on Multimedia. p. 3034–3042. MM ’21 (2021)
work page 2021
Show all 32 references
-
[9]
Lewis, M., Liu, Y., Goyal, N., Ghazvininejad, M., Mohamed, A., Levy, O., Stoyanov, V., Zettlemoyer, L.: BART: Denoising sequence-to-sequence pre-training for natural language generation, translation, and comprehension. In: ACL. pp. 7871–7880 (Jul 2020)
2020
-
[10]
Knowledge-Based Systems 235, 107643 (2022)
Liang, B., Su, H., Gui, L., Cambria, E., Xu, R.: Aspect-based sentiment analysis via affective knowledge enhanced graph convolutional networks. Knowledge-Based Systems 235, 107643 (2022)
2022
-
[11]
Ling, Y., Yu, J., Xia, R.: Vision-language pre-training for multimodal aspect-based sentiment analysis. In: ACL. pp. 2149–2159 (May 2022)
2022
-
[12]
arXiv preprint arXiv:1907.11692 (2019)
Liu, Y., Ott, M., Goyal, N., Du, J., Joshi, M., Chen, D., Levy, O., Lewis, M., Zettlemoyer, L., Stoyanov, V.: Roberta: A robustly optimized bert pretraining approach. arXiv preprint arXiv:1907.11692 (2019)
2019 arXiv
-
[13]
Neurocomputing428, 195–205 (2021)
Lv, Y., Wei, F., Cao, L., Peng, S., Niu, J., Yu, S., Wang, C.: Aspect-level sentiment analysis using context and aspect memory network. Neurocomputing428, 195–205 (2021)
2021
-
[14]
Title Suppressed Due to Excessive Length 15
Ma, Y., Cao, Y., Hong, Y., Sun, A.: Large language model is not a good few- shot information extractor, but a good reranker for hard samples! arXiv preprint arXiv:2303.08559 (2023). Title Suppressed Due to Excessive Length 15
2023 arXiv
-
[15]
arXiv preprint arXiv:2111.09734 (2021)
Mokady, R., Hertz, A., Bermano, A.H.: Clipcap: Clip prefix for image captioning. arXiv preprint arXiv:2111.09734 (2021)
2021 arXiv
-
[16]
OpenAI: Chatgpt: Optimizing language models for dialogue (2022), https://openai.com/blog/chatgpt/, accessed: 2024-06-13
2022
-
[17]
AAAI.34(05), 8600–8607 (Apr 2020)
Peng, H., Xu, L., Bing, L., Huang, F., Lu, W., Si, L.: Knowing what, how and why: A near complete solution for aspect-based sentiment analysis. AAAI.34(05), 8600–8607 (Apr 2020)
2020
-
[18]
arXiv preprint arXiv:2403.05530 (2024)
Reid, M., Savinov, N., Teplyashin, D., Lepikhin, D., Lillicrap, T., Alayrac, J.b., Soricut, R., Lazaridou, A., Firat, O., Schrittwieser, J., et al.: Gemini 1.5: Unlocking multimodal understanding across millions of tokens of context. arXiv preprint arXiv:2403.05530 (2024)
2024 arXiv
-
[19]
In: LREC
Rietzler, A., Stabinger, S., Opitz, P., Engl, S.: Adapt or get left behind: Domain adaptation through BERT language model finetuning for aspect-target sentiment classification. In: LREC. pp. 4933–4941 (May 2020)
2020
-
[20]
AAAI35(15), 13860–13868 (May 2021)
Sun, L., Wang, J., Zhang, K., Su, Y., Weng, F.: Rpbert: A text-image relation propagation-based bert model for multimodal ner. AAAI35(15), 13860–13868 (May 2021)
2021
-
[21]
In: NAACL-HLT
Tian, Y., Chen, G., Song, Y.: Aspect-based sentiment analysis with type-aware graph convolutional networks and layer ensemble. In: NAACL-HLT. pp. 2910–2922 (Jun 2021)
2021
-
[22]
Wang, K., Shen, W., Yang, Y., Quan, X., Wang, R.: Relational graph attention network for aspect-based sentiment analysis. In: ACL. pp. 3229–3238 (Jul 2020)
2020
-
[23]
arXiv preprint arXiv:2206.07682 (2022)
Wei, J., Tay, Y., Bommasani, R., Raffel, C., Zoph, B., Borgeaud, S., Yogatama, D., Bosma, M., Zhou, D., Metzler, D., et al.: Emergent abilities of large language models. arXiv preprint arXiv:2206.07682 (2022)
2022 arXiv
-
[24]
In: NLPCC
Wu, H., Cheng, S., Wang, J., Li, S., Chi, L.: Multimodal aspect extraction with region-aware alignment network. In: NLPCC. pp. 145–156. Springer (2020)
2020
-
[25]
In: Proceedings of the 28th ACM International Conference on Multimedia
Wu, Z., Zheng, C., Cai, Y., Chen, J., Leung, H.f., Li, Q.: Multimodal representation with embedded visual guiding objects for named entity recognition in social media posts. In: Proceedings of the 28th ACM International Conference on Multimedia. p. 1038–1046. MM ’20 (2020)
2020
-
[26]
In: ACL-IJCNLP
Yan, H., Dai, J., Ji, T., Qiu, X., Zhang, Z.: A unified generative framework for aspect-based sentiment analysis. In: ACL-IJCNLP. pp. 2416–2429 (Aug 2021)
2021
-
[27]
Information Processing & Management 59(5), 103038 (2022)
Yang, L., Na, J.C., Yu, J.: Cross-modal multitask transformer for end-to-end multimodal aspect-based sentiment analysis. Information Processing & Management 59(5), 103038 (2022)
2022
-
[28]
Yang, X., Feng, S., Wang, D., Sun, Q., Wu, W., Zhang, Y., Hong, P., Poria, S.: Few- shot joint multimodal aspect-sentiment analysis based on generative multimodal prompt. In: ACL. pp. 11575–11589 (Jul 2023)
2023
-
[29]
In: IJCAI
Yu, J., Jiang, J.: Adapting bert for target-oriented multimodal sentiment classifica- tion. In: IJCAI. pp. 5408–5414 (7 2019)
2019
-
[30]
IEEE/ACM Transactions on Audio, Speech, and Language Processing28, 429–439 (2020)
Yu, J., Jiang, J., Xia, R.: Entity-sensitive attention and fusion network for entity- level multimodal sentiment classification. IEEE/ACM Transactions on Audio, Speech, and Language Processing28, 429–439 (2020)
2020
-
[31]
Yu, J., Jiang, J., Yang, L., Xia, R.: Improving multimodal named entity recognition via entity span detection with unified multimodal transformer. In: ACL. pp. 3342– 3352 (Jul 2020)
2020
-
[32]
Zhou, R., Guo, W., Liu, X., Yu, S., Zhang, Y., Yuan, X.: AoM: Detecting aspect- oriented information for multimodal aspect-based sentiment analysis. In: ACL. pp. 8184–8196 (Jul 2023)
2023
Reviewed August 7, 2026 · model on record in the stance chip above.
Discussion (0). Sign in to comment.