Pith. sign in

REVIEW 3 major objections 5 minor 46 references

Aesthetic Image Captioning with Saliency Enhanced MLLMs

T0 review · 3 major / 5 minor · reviewed 2026-08-05 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read The paper claims that feeding a gradient-derived aesthetic saliency map into a vision encoder through cross-attention lets multimodal LLMs produce state-of-the-art aesthetic image captions.

desk verdict A plausible incremental integration of aesthetic saliency into MLLMs for AIC, but the causal benefit of the saliency map is not established and the SOTA claim is overstated; worth a serious referee with revisions. read the letter →

arxiv 2509.04378 v3 pith:XPS6PH25 submitted 2025-09-04 cs.CV

Yilin Tao , Jiashui Huang , Huaze Xu , Ling Shao This is my paper
classification cs.CV
keywords aestheticimagecaptioningsaliencymultimodallargelanguagemodelscross-attentionLayerCAMvisiontransformeraestheticsbenchmarkevaluation
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Generating a caption that says why a photo is beautiful, not just what it shows, requires a model to look where the aesthetic content is. This paper argues that multimodal LLMs, even after fine-tuning on captioning data, do not know where to look, and that explicitly feeding them a map of aesthetically salient regions fixes this. The proposed framework, ASE-MLLM, computes that map from an aesthetics classifier's gradients, then fuses it into the vision encoder with a cross-attention layer before the language model generates the caption. On three standard aesthetic-captioning benchmarks, the saliency-guided model beats fine-tuning alone on nearly every metric and reports state-of-the-art results. If the mechanism holds, it offers a cheap, model-agnostic way to steer LLM captions toward the photographic qualities humans actually critique.

What carries the argument

The aesthetic saliency map M_c: a per-pixel heatmap produced by the IASM from EAT's aesthetic classifier via LayerCAM (positive-gradient-weighted activation at one chosen layer). It is the load-bearing signal, because IAS-ViT uses it as the query in a cross-attention layer whose keys and values come from the original image features, forcing the encoder's output tokens to be re-weighted toward regions of high aesthetic value before the LLM generates captions.

What would settle it

Take the trained ASE-MLLM, replace M_c with a randomly generated or fixed-center saliency map for a held-out split of DPC2022, and re-run captioning: if CIDEr stays near 0.072, the mechanism is not what carries the gain. Or compare M_c against human-annotated aesthetic regions on test images: if the map overlaps no better than a plain visual-saliency or object-detector map while the caption gains persist, the saliency explanation is not the source of the improvement.

Watch

Extended reading notes

Core claim

ASE-MLLM is claimed to be the first framework to integrate image aesthetic saliency into a multimodal large language model specifically for aesthetic image captioning. Its Image Aesthetic Saliency Module runs an input photo through the EAT aesthetic transformer, takes the classifier's score for the most salient aesthetic class, and uses LayerCAM to turn the positive gradients of that score into a spatial saliency map M_c. A modified vision encoder, IAS-ViT, then applies cross-attention with the saliency features as query and the original image features as key and value, so the model's visual tokens carry aesthetic-focus information into the language model. The paper reports that this combina

Load-bearing premise

The aesthetic saliency map taken from an aesthetics classifier's gradients reliably marks the regions that the target captions actually describe; if that map is noisy or misaligned with human aesthetic focus, the cross-attention fusion cannot deliver the reported gains.

Editorial extensions

If this is right

  • Retrofitting is cheap: the saliency branch is added to the existing ViT encoder, so any MLLM with a ViT-style image encoder could adopt the pattern without altering the language model.
  • The margin over fine-tuning is consistent across all three datasets and both base models, which suggests aesthetic saliency is complementary to, not a replacement for, task-specific fine-tuning.
  • Because results improved for two different base MLLMs (InternVL2-8B and Qwen-VL-Chat), the framework is claimed to generalize beyond a single architecture.
  • If the approach holds, aesthetic-relevant outputs beyond captioning—photo critiques, editing suggestions, aesthetic scoring rationales—could use the same saliency-steering mechanism.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The reported absolute gains are small (e.g., DPC2022 CIDEr rises from 0.068 to 0.072), so the practical significance depends on whether the added metric points reflect genuinely more aesthetic words rather than style shifts; a human-preference or attribute-level evaluation would settle this—the paper does not run one.
  • Because M_c is built from the argmax aesthetic class of a single classifier, images with diffuse or multi-focus aesthetics may get mislocalized saliency; a multi-class or distribution-weighted saliency aggregation is a direct, testable variant.
  • The same query-from-saliency, key/value-from-image cross-attention design transfers naturally to other attribute-focused captioning tasks (e.g., describing materials, defects, or emotions) whenever a cheap attribute saliency map exists.
  • The admitted weakness on RPCD's CIDEr and SPICE suggests the saliency benefit interacts with the style gap between the base model's language distribution and the dataset's colloquial comments—worth probing with style-aware prompts.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. The paper proposes ASE-MLLM, a framework for Aesthetic Image Captioning (AIC) that injects an externally computed aesthetic-saliency map into an MLLM's vision encoder. Specifically, an Image Aesthetic Saliency Module (IASM) computes a saliency map Mc from the EAT aesthetic-scoring model using LayerCAM (Eqs. 1–5), and an IAS-ViT encoder fuses Mc with the original image features via cross-attention. The framework is instantiated on InternVL2-8B and Qwen-VL-Chat. Experiments on DPC2022, PCCD, and RPCD report eleven captioning metrics. The core quantitative evidence is the ablation in Tables II and III, where fine-tuning plus IASC (IASM+IAS-ViT) outperforms fine-tuning alone on nearly all metrics across two base models, e.g., DPC2022 CIDEr 0.068→0.072 and METEOR 0.142→0.150. The paper also claims state-of-the-art performance over generic (non-fine-tuned) MLLMs and conventional AIC models in Table I.

Significance. The idea of making an MLLM explicitly attend to aesthetic saliency rather than generic visual saliency is timely and plausible, and the paper is, to my knowledge, the first to propose this specific integration for AIC. The internal ablation is a genuine strength: the IASC gain is replicated across two different base MLLMs and is visible across most of the eleven metrics, which suggests the effect is not a single lucky metric. The method itself is simple and easy to build on. However, the evidence is not yet sufficient to establish that the aesthetic content of Mc, rather than added parameters, architectural changes, or regularization, causes the improvement; no control saliency or cross-attention variant is tested, and no significance or variability analysis is provided. The SOTA claim is also overstated because the MLLM baselines in Table I were not fine-tuned. If the missing controls are supplied, the paper could make a solid contribution; in its current form the central causal claim remains under-supported.

major comments (3)
  1. [§III-B/C, Table II, Table III] The central claim—that integrating aesthetic saliency, not merely fine-tuning or extra parameters, improves AIC—is not causally established. The IASC ablation compares FT+IASC against FT, but IASC bundles IASM (Mc from EAT+LayerCAM, Eq. 5) with the new cross-attention block in IAS-ViT. There is no control using (i) a random or generic visual-saliency map in place of Mc, (ii) a cross-attention path fed only duplicated original features, or (iii) an otherwise identical architecture with the saliency branch removed but parameter count matched. Without such controls, the gains in Tables II/III could come from additional capacity, regularization, or attention dynamics rather than from the aesthetic content of Mc. Additionally, there is no direct validation that Mc localizes the regions named in the target captions; EAT is an aesthetic-scoring model, not a caption-grounding model. I recommend
  2. [§IV-D, Table I] The abstract and Section I claim state-of-the-art performance, but Table I compares the fine-tuned ASE-MLLM against MLLM baselines that, as stated in §IV-D, 'had not been fine-tuned.' Since this paper's own ablation shows that fine-tuning alone produces most of the gain (e.g., DPC2022 BLEU-4 0.011→0.144), Table I largely demonstrates the effect of fine-tuning, not the contribution of aesthetic saliency. To support a SOTA claim, the comparison set should include fine-tuned AIC-specific MLLM baselines (e.g., the same InternVL2-8B and Qwen-VL-Chat fine-tuned without IASC, and recent AIC methods such as AesExpert/UNIAA evaluated on shared metrics), rather than only zero-shot general-purpose MLLMs.
  3. [Table II, Table III] No error bars, multiple seeds, or significance tests are reported. The IASC improvements are small on several metrics (e.g., DPC2022 CIDEr 0.068→0.072, METEOR 0.142→0.150), and although the pattern across metrics and base models is suggestive, single-run differences of this size are not convincing without variance estimates or paired significance tests. The authors should report results over at least 3 seeds with means and standard deviations, or use bootstrap resampling for the test-set metrics, to establish that the reported margins are not within noise.
minor comments (5)
  1. [§IV-D] The text refers to 'our AES-MLLM framework' when discussing Qwen; the acronym should be ASE-MLLM. This typo appears in a methodological claim and should be corrected.
  2. [Eq. (5)] Equation (5) is garbled in the manuscript (')kAk ijRELUMc'). Please rewrite it as a clear linear combination of ReLU-activated saliency feature maps across channels. Also, the text says 'linear combination' but does not specify the weights used in that combination.
  3. [§IV-C] The evaluation section omits the fact that the ARIC comparison in Table I reports only metrics copied from the original paper, and that some metrics are missing (BLEU, ROUGE, etc.). This should be stated directly next to the table, not only in prose, to avoid misleading comparisons.
  4. [References] There are several reference formatting issues: [17] contains 'on.1' instead of 'no.1'; [3] lists an incorrect journal volume/issue; [5] uses 'INT. J. PATTERN RECOGN.' with an inconsistent format; and [13] is given as an ACM MM paper but the publisher appears as 'Proc. ACM. Int. Conf. Multimed.' The reference list should be cleaned up.
  5. [§IV-B] The implementation details do not state the hardware budget, training epochs, or total number of trainable parameters for the proposed modules. This makes it difficult to judge the 'efficient' claim for IASM and the computational overhead of the 24-block IAS-ViT.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the saliency signal is derived from an independent pretrained aesthetic model (EAT) and the ablation directly tests the proposed modules.

full rationale

The paper's central derivation chain is not circular. The aesthetic saliency map Mc is computed from the externally pretrained EAT model [13] using LayerCAM [44] (Eqs. 1–5); it is not derived from the AIC captions or from the captioning loss. The claimed improvement from adding IASC is empirically evaluated through ablations (Tables II and III) that compare fine-tuning with and without the proposed modules on held-out test splits. The paper does not fit a parameter to the target captions and then rename it as a prediction; the only performance-based choices are architectural hyperparameters (target layer, number of blocks), which are standard model selection rather than circularity. There are no self-citations that are load-bearing: the cited EAT, LayerCAM, InternVL, and other baselines are independent external works, and none of the citations are authored by the present paper's authors. The SOTA comparison is weakened by the statement that baseline MLLMs were not fine-tuned, but that is an experimental-design limitation, not circularity. No equation or claim reduces to its own input. Therefore, the derivation is self-contained with respect to circularity concerns.

Assumptions & free parameters 3 free parameters · 4 assumptions · 0 invented entities

The central claim depends on the quality of the saliency extraction, which in turn depends on EAT and LayerCAM as external tools, plus several empirically chosen hyperparameters. No new physical or conceptual entities are introduced.

free parameters (3)
  • EAT target layer for LayerCAM = not specified (selected by empirical testing)
    The paper states 'through empirical testing, we selected one target layer that yielded the best performance' (Sec. III-B). This choice affects the saliency map quality and is not tied to a principled criterion.
  • Number of IAS-ViT blocks = 24
    Sec. III-C: 'Through experimental tests, we adopted 24 IAS-ViT blocks... achieves an optimal balance between model performance and computational overhead.' This is a hand-chosen architecture dimension.
  • Query/key/value assignment in cross-attention = query from aesthetic features, key/value from original features
    Sec. III-C: 'Through experimental tests, we selected image aesthetic features to generate the query vector...' This choice is empirical and could be swapped.
assumptions (4)
  • domain assumption EAT model's internal representations are suitable for extracting aesthetic saliency.
    The IASM uses EAT's classifier score to define saliency via LayerCAM; if EAT's attention does not align with human aesthetic focus, the saliency features are unreliable. Sec. III-B.
  • domain assumption LayerCAM gradient-based localization faithfully identifies aesthetically salient regions.
    The method relies on positive gradients of the most salient class score to weight feature maps (Eq. 2-5). This is an assumption about the interpretability of EAT's classifier.
  • domain assumption The three AIC datasets and their predefined splits are representative and correctly processed.
    The paper follows original partitioning but does not describe preprocessing or filtering in detail; results depend on these datasets.
  • domain assumption Standard captioning metrics (BLEU, METEOR, CIDEr, etc.) capture aesthetic caption quality.
    The paper uses these metrics but also acknowledges in Sec. V that model captions may be valid yet not match annotations, indicating a known mismatch between metrics and aesthetic quality.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Aesthetic Image Captioning with Saliency Enhanced MLLMs." pith.science (2026). https://pith.science/paper/XPS6PH25

@misc{pith2026250904378,
  author       = {Pith},
  title        = {Pith review of: Aesthetic Image Captioning with Saliency Enhanced MLLMs},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/XPS6PH25}},
  note         = {Machine review of arXiv:2509.04378}
}
read the original abstract

Aesthetic Image Captioning (AIC) aims to generate textual descriptions of image aesthetics, becoming a key research direction in the field of computational aesthetics. In recent years, pretrained Multimodal Large Language Models (MLLMs) have advanced rapidly, leading to a significant increase in image aesthetics research that integrates both visual and textual modalities. However, most existing studies on image aesthetics primarily focus on predicting aesthetic ratings and have shown limited application in AIC. Existing AIC works leveraging MLLMs predominantly rely on fine-tuning methods without specifically adapting MLLMs to focus on target aesthetic content. To address this limitation, we propose the Aesthetic Saliency Enhanced Multimodal Large Language Model (ASE-MLLM), an end-to-end framework that explicitly incorporates aesthetic saliency into MLLMs. Within this framework, we introduce the Image Aesthetic Saliency Module (IASM), which efficiently and effectively extracts aesthetic saliency features from images. Additionally, we design IAS-ViT as the image encoder for MLLMs, this module fuses aesthetic saliency features with original image features via a cross-attention mechanism. To the best of our knowledge, ASE-MLLM is the first framework to integrate image aesthetic saliency into MLLMs specifically for AIC tasks. Extensive experiments demonstrated that our approach significantly outperformed traditional methods and generic MLLMs on current mainstream AIC benchmarks, achieving state-of-the-art (SOTA) performance.

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

46 extracted references · 33 canonical work pages

  1. [1]

    A model of aesthetic appreciationandaestheticjudgments,

    H. Leder, B. Belke, A. Oeberst, and D. Augustin, “A model of aesthetic appreciationandaestheticjudgments,”BRIT.J.PSYCHOL,vol.95,no.4, pp.489-508,Nov.2004,doi:https://doi.org/10.1348/0007126042369811

    doi:10.1348/0007126042369811 2004

  2. [2]

    Prospectsforacognitiveneuroscienceofvisualaesthetics,

    A.Chatterjee,“Prospectsforacognitiveneuroscienceofvisualaesthetics,” BulletinofPsychologyandtheArts, vol. 4, no. 2, pp. 55–60, 2003. doi: https://doi.org/10.1037/e514602010-003

    work page doi:10.1037/e514602010-003 2003

  3. [3]

    Comment-guided semantics-awareimageaestheticsassessment,

    Y. Niu, S. Chen, B. Song, Z. Chen, and W. Liu, “Comment-guided semantics-awareimageaestheticsassessment,”IEEETrans.CircuitsSyst. Video Technol., vol. 33, no. 3, pp. 1487-1492, Mar. 2022, doi: https://doi.org/10.1109/TCSVT.2022.3201510

    work page arXiv 2022

  4. [5]

    Confidence-based dynamic cross-modal memory network for image aesthetic assessment,

    X. Zhang, Y. Xiao, J. Peng, X. Gao, B. Hu, “Confidence-based dynamic cross-modal memory network for image aesthetic assessment,” INT. J. PATTERN RECOGN., vol. 14, no. 9, pp. 1-13, May. 2024, doi: https://doi.org/10.1016/j.patcog.2023.110227

    work page arXiv 2024

  5. [6]

    Aesclip: Multi-attribute contrastive learning for image aesthetics assessment,

    X.Sheng,L.Li,P.Chen,J.Wu,W.Dong,Y.Yang,L.Xu,Y.Li,andG. Shi, “Aesclip: Multi-attribute contrastive learning for image aesthetics assessment,” in Proc. ACM. Int. Conf. Multimed., October 2023, pp. 1117-1126,doi:https://doi.org/10.1145/3581783.361196

    arXiv 2023

  6. [7]

    Image aesthetics assessment with attribute-assisted multimodal memory network,

    L. Li, T. Zhu, P. Chen, Y. Yang, Y. Li, and W. Lin, “Image aesthetics assessment with attribute-assisted multimodal memory network,” IEEE Trans.CircuitsSyst.VideoTechnol., vol. 33, no. 12, pp.7413-7424, Dec. 2023,doi:https://doi.org/10.1109/TCSVT.2023.3272984

    work page arXiv 2023

  7. [8]

    Enriched Image Captioning based on Knowledge Divergence and Focus,

    A. Liu, Q. Wu, N. Xu, H. Tian, and L. Wang, “Enriched Image Captioning based on Knowledge Divergence and Focus,” IEEE Trans. Circuits Syst. Video Technol., Jan. 2025, doi: https://doi.org/10.1109/TCSVT.2024.3525158

    work page arXiv 2025

  8. [9]

    Aesthetic image captioning from weakly-labelled photographs,

    K. Ghosal, A. Rana, and A. Smolic, “Aesthetic image captioning from weakly-labelled photographs,” in Proceedings oftheIEEEConf.Comput. Vis. Pattern Recognit. Workshops, Aug. 2019, Seoul, South Korea, pp. 4550-4560,doi:https://doi.org/10.1109/ICCVW.2019.00556

    work page arXiv 2019

Show all 46 references
  1. [10]

    Generating aesthetic based critique for photographs,

    Y. Y. Yeo, J. See, L. Kuan W., and H. N. Goh, “Generating aesthetic based critique for photographs,” In IEEE Int. Conf. onImageProcess., Sep. 2021, pp. 2523-2527, doi: https://doi.org/10.1109/ICIP42928.2021.9506385

    2021

  2. [11]

    UNIAA: A UnifiedMulti-modal Image Aesthetic Assessment Baseline andBenchmark,

    Z. Zhou, Q. Wang, B. Lin, Y. Su, R. Chen, X. Tao, …, D. Zhang, “UNIAA: A UnifiedMulti-modal Image Aesthetic Assessment Baseline andBenchmark,”2024,arXiv:2404.09619

    2024 arXiv

  3. [12]

    Studyon the Image Aesthetic Assessment Based on Saliency and Attention,

    K. Kang, “Studyon the Image Aesthetic Assessment Based on Saliency and Attention,” M.S. thesis, China Univ. Mining Technol., Beijing, China,2021

    2021

  4. [13]

    Eat: An enhancer for aesthetics-oriented transformers,

    S. He, A. Ming, S. Zheng, H. Zhong, and H. Ma, “Eat: An enhancer for aesthetics-oriented transformers,” in Proc. ACM. Int. Conf. Multimed., pp. 1023-1032, Oct. 2023, doi: https://doi.org/10.1145/3581783.3611881

    2023

  5. [14]

    Multi-Modality Multi-Attribute Contrastive Pre-Training for Image Aesthetics Computing,

    Y. Huang, L. Li, P. Chen, H. Wu, W. Lin, and G. Shi, “Multi-Modality Multi-Attribute Contrastive Pre-Training for Image Aesthetics Computing,”IEEETrans.PatternAnal.Mach.Intell.,vol.47,no.2,pp. 1205 - 1218, (Nov. 2024, doi: https://doi.org/10.1109/TPAMI.2024.3492259

    2024

  6. [15]

    Aesthetic critiques generation for photos,

    K. Chang, K. Lu, and C. Chen, “Aesthetic critiques generation for photos,” inProc. IEEE/CVF Conf.Comput. Vis. PatternRecognit., pp. 3514-3523, Oct. 2017, Venice, Italy, doi: https://doi.org/10.1109/ICCV.2017.380

    2017 doi

  7. [16]

    Neural aesthetic image reviewer,

    W. Wang, S. Yang, W. Zhang, and J. Zhang, “Neural aesthetic image reviewer,”IETComput.Vis.,vol.13,no.8,pp.749-758,Nov.2019,doi: https://doi.org/10.1049/iet-cvi.2019.0361

    2019

  8. [17]

    Latent dirichlet allocation,

    D. M. Blei, A. Y. Ng, and M. I. Jordan, “Latent dirichlet allocation,”J. Mach.Learn.Res.,vol.3,on.1,pp.993–1022,Mar.2003

    2003

  9. [18]

    Aesthetic Attributes Assessment of Images,

    X.Jin,L.Wu,G.Zhao,X.Li,X.Zhang,S.Ge,D.Zou,B.Zhou,andX. Zhou, “Aesthetic Attributes Assessment of Images,” inProc.ACM.Int. Conf. Multimed., pp. 311 – 319, New York, NY, USA, doi: https://doi.org/10.1145/3343031.3350970..2019

    2019

  10. [19]

    Towards personalized aesthetic image caption,

    K. Xiong, L. Jiang, X. Dang, G. Wang, W. Ye, and Z. Qin, “Towards personalized aesthetic image caption,” inInt.JointConf.NeuralNetw., pp. 1-8, Jul. 2020, doi: https://doi.org/10.1109/IJCNN48605.2020.9206953

    2020

  11. [20]

    Understanding aesthetics with language: A photo critique dataset for aesthetic assessment,

    D. V. Nieto, L. Celona, and C. F. Labrador, “Understanding aesthetics with language: A photo critique dataset for aesthetic assessment,” in Conf.NeuralInf.Process.Syst.,pp.34148-34161,Nov.-Dec.2022,New York,USA. [21]Z.Zhong,F.Zhou,andG.Qiu,“Aestheticallyrelevantimagecaptioning...

    2022

  12. [30]

    Aesexpert: Towards multi-modality foundation model for image aesthetics perception,

    Y.Huang,X.Sheng,Z.Yang,Q.Yuan,Z.Duan,P.Chen,L.Li,W.Lin, and G. Shi, “Aesexpert: Towards multi-modality foundation model for image aesthetics perception,” in Proc. ACM. Int. Conf. Multimed., pp. 5911-5920, Oct. 2024, Melbourne, Victoria, Australia, doi: https://doi.org/10.1145/...

    2024

  13. [31]

    Saliency-enhanced image aesthetics class prediction,

    L. K. Wong, K. L. Low, “Saliency-enhanced image aesthetics class prediction,” in IEEE Int. Conf. Image Process., pp. 997-1000, Nov. 2009,doi:https://doi.org/10.1109/ICIP.2009.5413825

    2009

  14. [32]

    Research on Image Aesthetic Evaluation using Image Scene and Saliency Information,

    M. Zhao, “Research on Image Aesthetic Evaluation using Image Scene and Saliency Information,” M.S. thesis, South China Univ. Technol., Guangzhou,China,2017

    2017

  15. [33]

    Research on image quality assessment and improvement algorithm based on aesthetics,

    X. Li, “Research on image quality assessment and improvement algorithm based on aesthetics,” Ph. D. dissertation, Beijing Univ. Posts andTelecommun.,Beijing,China,2021

    2021

  16. [34]

    Imageaestheticsassessment based on multi-stream CNN architecture and saliency features,

    H.Takimoto,F.Omori,andA.Kanagawa,“Imageaestheticsassessment based on multi-stream CNN architecture and saliency features,” Appl. Artif. Intell., vol. 35, no. 1, pp. 25-40, May 2021, doi: https://doi.org/10.1080/08839514.2020.1839197

    2021

  17. [35]

    Co-salient object detectionwithco-representationpurification,

    Z. Zhu, Z. Zhang, Z. Lin, X. Sun, and M. Cheng, “Co-salient object detectionwithco-representationpurification,”IEEETrans.PatternAnal. 3 >REPLACETHISLINEWITHYOURMANUSCRIPTIDNUMBER(DOUBLE-CLICKHERETOEDIT)< Mach. Intell., vol. 45, no. 7, pp. 8193-8205, Jan. 2023, doi: https://doi...

    2023

  18. [36]

    Associating inter-image salient instances for weakly supervised semantic segmentation,

    R.Fan,Q.Hou,M.Cheng,G.Yu,R.R.Martin,andS.Hu,“Associating inter-image salient instances for weakly supervised semantic segmentation,”inProc.Eur.Conf.Comput.Vis.,pp.367-383,Sep.2018, Munich,Germany,doi:https://doi.org/10.1007/978-3-030-01240-3_23

    2018 doi

  19. [37]

    Texture-guidedsaliency distillingfor unsupervisedsalientobjectdetection,

    H.Zhou,B.Qiao,L.Yang,J.Lai,andX.Xie,“Texture-guidedsaliency distillingfor unsupervisedsalientobjectdetection,”inProc.IEEEConf. Comput. Vis. Pattern Recognit., pp. 7257-7267, Jun. 2023, Vancouver, Canada,doi:https://doi.org/10.1109/CVPR52729.2023.00701

    2023

  20. [38]

    Human detection using a mobile platform and novel features derived from a visual saliency mechanism,

    S. Montabone and A. Soto, “Human detection using a mobile platform and novel features derived from a visual saliency mechanism,” Image Vis. Comput., vol. 28, no. 3, pp. 391-402, 2010, doi: https://doi.org/10.1016/j.imavis.2009.06.006

    2010 doi

  21. [39]

    Rethinking Image AestheticsAssessment:Models,DatasetsandBenchmarks,

    S. He, Y. Zhang, R. Xie, D. Jiang, and A. Ming, “Rethinking Image AestheticsAssessment:Models,DatasetsandBenchmarks,”inInt.Joint Conf.Artif.Intell.,pp.942-948,Jul.2022,MesseWien,Vienna,Austria, doi:https://doi.org/10.24963/ijcai.2022/132

    2022 doi

  22. [40]

    ‘Special Relativity’ of Image Aesthetics Assessment: a Preliminary Empirical Perspective,

    R. Xie, A. Ming, S. He, Y. Xiao, and H. Ma, “‘Special Relativity’ of Image Aesthetics Assessment: a Preliminary Empirical Perspective,” in Proc.ACM.Int.Conf.Multimed.,pp.2554-2563,Oct.2024,Melbourne, Victoria,Australia,doi:https://doi.org/10.1145/3664647.3681172

    2024

  23. [41]

    How far are we to gpt-4v? closing the gap to commercial multimodal models with open-source suites,

    Z. Chen,W. Wang, H. Tian,S. Ye, Z. Gao, E. Cui,..., W. Wang, “How far are we to gpt-4v? closing the gap to commercial multimodal models with open-source suites,” Sci. China Inf. Sciences, vol. 67, no. 12, pp. 220101:1 – 220101:18, Dec. 2024, doi: https://doi.org/10.48550/ARXIV...

    2024 doi arXiv

  24. [42]

    AutoTokenizer.HuggingFaceTransformerslibrary.[Online].Available: https://huggingface.co/docs/transformers/main_classes/tokenizer

  25. [43]

    Internlm2 technical report,

    Z.Cai,M.Cao,H.Chen,K.Chen,K.Chen,X.Chen,X.Chen,Z.Chen, Z. Chen, P. Chu, et al, “Internlm2 technical report,” 2024, arXiv:2403.17297

    2024 arXiv

  26. [44]

    Layercam: Exploring hierarchical class activation maps for localization,

    P. Jiang, C. Zhang, Q. Hou, M. Cheng, and Y. Wei, “Layercam: Exploring hierarchical class activation maps for localization,” in IEEE Trans. Image Process., pp. 5875-5888, Jun. 2021, doi: https://doi.org/10.1109/TIP.2021.3089943

    2021

  27. [45]

    Bleu: a method for automatic evaluation of machine translation,

    K. Papineni, S. Roukos, T. Ward, and W. Zhu, “Bleu: a method for automatic evaluation of machine translation,” in Proc. Association Computational Linguistics, pp. 311-318, Jul. 2002, doi: https://doi.org/10.3115/1073083.1073135

    2002

  28. [46]

    Rouge: A package for automatic evaluation of summaries,

    C. Y. Lin, “Rouge: A package for automatic evaluation of summaries,” in Text summarization branches out, pp. 74-81, Jul. 2004, Barcelona, Spain,AssociationComputationalLinguistics

    2004

  29. [47]

    METEOR: An automatic metric for MT evaluation with improved correlation with human judgments,

    S. Banerjee and A. Lavie, “METEOR: An automatic metric for MT evaluation with improved correlation with human judgments,” in Proc. ACL workshop on intrinsic and extrinsic evaluation measures for machinetranslationand/orsummarization,pp.65–72,Jun.2005

    2005

  30. [48]

    Cider: Consensus-based imagedescriptionevaluation,

    R. Vedantam, C. L. Zitnick, and D. Parikh, “Cider: Consensus-based imagedescriptionevaluation,”inProc.IEEEConf.Comput.Vis.Pattern Recognit. Pp. 4566-4575, Jun. 2015, doi: https://doi.org/10.1109/CVPR.2015.7299087

    2015

  31. [49]

    Spice: Semantic propositional image caption evaluation,

    P. Anderson, B. Fernando, M. Johnson, and S. Gould, “Spice: Semantic propositional image caption evaluation,” in Proc. Eur. Conf. Comput. Vis., pp. 382–398, Oct. 2016, Springer International Publishing, doi: https://doi.org/10.1007/978-3-319-46454-1_24

    2016 doi

  32. [50]

    [Online]

    OpenAI, 2024. [Online]. Available: GPT-4o mini. Retrieved from https://openai.com

    2024

  33. [51]

    Qwen- VL: A versatile vision-language model for understanding, localization, textreading,andbeyond,

    J. Bai, S. Bai, P. Wang, S. Yang, J. Lin, S. Wang, and J. Zhou, “Qwen- VL: A versatile vision-language model for understanding, localization, textreading,andbeyond,”2023,arXiv:2308.12966

    2023 arXiv

  34. [52]

    Thellama3herdofmodels,

    A. Grattafiori, A. Dubey, A. Jauhri, A. Pandey, A. Kadian, A. A. Dahle,...,J.Jia,“Thellama3herdofmodels,”2024,arXiv:2407.21783

    2024 arXiv

  35. [53]

    Improved Baselines with Visual InstructionTuning,

    H. Liu, C. Li, Y. Li, Y. J. Lee, “Improved Baselines with Visual InstructionTuning,”inProc.IEEEConf.Comput.Vis.PatternRecognit., pp. 26296-26306, Jun. 2024, Seattle WA, USA, doi: https://doi.org/ 10.1109/CVPR52733.2024.02484

    2024

  36. [54]

    mPLUG- Owl3: Towards Long Image-Sequence Understanding in Multi-Modal LargeLanguageModels,

    J. Ye, H. Xu, H. Liu, A. Hu, M. Yan, Q. Qian, …, J. Zhou, “mPLUG- Owl3: Towards Long Image-Sequence Understanding in Multi-Modal LargeLanguageModels,”2024,arXiv:2408.04840

    2024 arXiv

  37. [55]

    Microsoftcococaptions:Datacollectionandevaluationserver,

    X.Chen,H.Fang,T.Y.Lin,R.Vedantam,S.Gupta,P.Dollar,andC.L. Zitnick,“Microsoftcococaptions:Datacollectionandevaluationserver,” 2015,arXiv:1504.00325

    2015 arXiv

  38. [56]

    Show and tell: A neural image caption generator,

    O. Vinyals, A. Toshev, S. Bengio, and D. Erhan, “Show and tell: A neural image caption generator,” in Proc. IEEE Conf. Comput. Vis. PatternRecognit.,pp.3156–3164,2015

    2015

Pith tools

Reviewed August 5, 2026 · model on record in the stance chip above.