Pith. sign in

REVIEW 4 major objections 6 minor 44 references

MMArt A Multi-Perspective Multimodal Dataset for Visual Art Understanding

T0 review · 4 major / 6 minor · reviewed 2026-08-12 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read The paper claims that describing a painting from four independent perspectives — narrative, formal, emotional, and historical — produces a dataset in which no single perspective is best for every art-understanding task.

desk verdict Worth a serious referee, but the paper oversells its complementarity evidence and needs controls for metadata leakage before its strong claims are citeable. read the letter →

arxiv 2608.10706 v1 pith:URYH3SRO submitted 2026-08-11 cs.CV cs.MM

Shuai Wang , Wangyuan Ding , Yixian Shen , Jia-Hong Huang , Stevan Rudinac , Monika Kackovic , Nachoem Wijnberg , Marcel Worring This is my paper · ORCID
classification cs.CVcs.MM
keywords visualartunderstandingmulti-perspectivedatasetvision-languagemodelsperspectivecomplementarityimageretrievaltext-to-imagereconstructionaffectivecomputinginterpretation
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper sets out to fix a data problem: art datasets each offer only one way of describing a painting, so models learn to caption surface content rather than interpret form, feeling, and context. To change that, the authors build MMArt, a dataset of 74,234 paintings in which every work carries four independent descriptions — what is depicted, how the work is formally composed, what it feels like, and why it looks and means as it does — plus a unified caption that merges the four. The central claim is that these perspectives are genuinely complementary and task-asymmetric: narrative text is the strongest retrieval query, formal analysis best preserves composition in image reconstruction, and historical context is the least replaceable perspective when any one is removed. Validation comes from two directions: a generative probe that reconstructs paintings from text and measures style, composition, and emotional agreement, and a discriminative probe that uses each perspective to retrieve the correct painting from the full collection. A reader should care because, if right, MMArt supplies the missing supervision for models that interpret artworks across multiple registers at once.

What carries the argument

The load-bearing mechanism is a four-perspective annotation scheme in which each painting is described independently by narrative, formal, emotional, and historical viewpoints, following a layered scheme of art interpretation; each perspective is produced by a model chosen for that interpretive register rather than by one generalist model. The other carrying element is a two-directional complementarity analysis. A generative probe feeds perspective text to text-to-image models and measures reconstruction fidelity with three complementary metrics (style and semantic similarity, compositional similarity, and emotional agreement), while a discriminative probe embeds all paintings into a fixed gallery and measures how well each perspective retrieves the originating painting. Together these probes convert the qualitative idea of 'different perspectives' into measurable task asymmetry that persists across multiple generators and embedding models.

What would settle it

Run the generative analysis with a text-to-image model never trained on paintings and a generic control prompt like 'a painting' for every artwork; if the control matches originals on the style and composition metrics as closely as the formal perspective does, the reconstruction probe is measuring generator priors rather than perspective content.

Watch

Extended reading notes

Core claim

The discovery is that different ways of describing a painting carry different, only partially overlapping information, and the ranking of perspectives flips between tasks. In the generative direction, formal descriptions reconstruct compositional style better than other single perspectives, and the unified caption achieves the highest overall fidelity on both style and composition metrics. In the discriminative direction, narrative descriptions retrieve the correct painting from the full gallery 44.0 percent of the time at rank one, while formal descriptions, despite their reconstruction strength, reach only 7.8 percent, and historical descriptions are nearly non-discriminative. Leave-one-out analysis shows historical context is the hardest perspective to replace across both tasks. The paper concludes from this task asymmetry that no single perspective suffices for all art-understanding tasks, which directly motivates the dataset's multi-perspective design.

Load-bearing premise

The complementarity conclusions rest on treating text-to-image reconstruction fidelity as a direct measure of how much visual information a perspective encodes; if the generators rely on their own style priors or obey some prompts better than others, the resulting task rankings would not actually be measuring the perspectives.

Editorial extensions

If this is right

  • Models can be trained with explicit perspective conditioning, so a user can ask for a formal analysis, an emotional reading, or a historical account of the same painting.
  • Art visual question answering can be organized around perspective, because different questions depend on different annotation registers.
  • Retrieval systems can select query perspective by task: narrative text for finding a specific painting, formal and historical text for browsing by style or period.
  • Retrieval-augmented generation over art gains query-side historical grounding, since historical descriptions carry facts that are not visible in the image.
  • Benchmark designers can use the unified caption as a single-text upper bound, since it slightly outperforms every single perspective on retrieval while matching the best reconstructive conditions.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The paper leaves implicit that the same reconstruction-plus-retrieval protocol could be reused as a generic diagnostic for any dataset that annotates one image in several registers, not only art.
  • Because all pairwise perspective embedding similarities are below 0.55, a testable extension is to measure whether substituting one perspective for another actually changes downstream question-answering answers; if it does not, some of the claimed distinctiveness may not translate into task-relevant gains.
  • A practical extension is to ablate the human affective utterances from the emotional-perspective pipeline; if judge-assessed fidelity drops sharply, that would demonstrate where human grounding contributes inside a synthetic annotation pipeline.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 6 minor

Summary. The manuscript introduces MMArt, a dataset of 74,234 WikiArt paintings annotated with four independently generated interpretive perspectives (narrative, formal, emotional, historical) plus a harmonized unified caption. The perspectives are produced with specialized models: Qwen3-VL for narrative, GalleryGPT for formal analysis, ArtEmis-conditioned Qwen3-VL for emotion, and ArtRAG for historical context. The paper validates the resource with semantic distinctiveness statistics, an LLM-as-judge quality assessment, a blinded human evaluation against Claude 4.5 Sonnet, and two complementarity analyses: text-to-image reconstruction and perspective-guided retrieval over the full gallery. The stated central contribution is that the two analyses show task-asymmetric usefulness and that no single perspective suffices for all tasks.

Significance. If the complementarity claims hold, MMArt would be a valuable and timely resource: it is the first large-scale dataset providing four aligned interpretive perspectives over the same image set, and the public release of data, code, and embeddings would facilitate perspective-conditioned training, art VQA, and retrieval-augmented generation. The paper has notable methodological strengths, including the use of specialized generation models, two independent text-to-image generators, two architecturally distinct embedding models, an external-family LLM judge, and human evaluation. The remaining concerns are about the evidential basis for the headline cross-task claims rather than about the dataset resource itself.

major comments (4)
  1. [Abstract; §4.1, Fig. 5] The abstract's claim that leave-one-out analysis 'confirms that historical descriptions are the least replaceable perspective across both tasks' is contradicted by Fig. 5. In the Qwen3-VL-Embedding column, removing narrative (FEH) drops R@1 from 44.0 to 17.6, while removing historical (NFE) leaves R@1 at 34.8 and removing formal (NEH) leaves R@1 at 40.1. The retrieval leave-one-out results therefore identify narrative, not historical, as the least replaceable perspective for retrieval. The statement is supportable for the reconstruction metrics in Fig. 4 but not for retrieval, and it must be corrected or re-evidenced.
  2. [§4.1, Eq. (3)] The emotion agreement metric δ_emot is partly circular for the emotional perspective. e_emot is generated by conditioning on ArtEmis human affective utterances for the same painting, and δ_emot measures agreement between the original and reconstructed images on the ArtEmis top-1 emotion label. A high score can arise simply because the text-to-image model realizes the emotion name carried in the prompt, independent of whether the reconstructed image's visual content carries affective information. Please report emotion agreement with held-out human annotations or an independent affect classifier, and at minimum explicitly acknowledge this conditioning overlap as a limitation.
  3. [§4.1, Figs. 4 and 5] The complementarity conclusions are confounded by annotation-source and metadata differences across perspectives. Historical texts are produced by ArtRAG with retrieved context and naturally contain artist names, dates, style, and movement terms; formal texts use vocabulary such as chiaroscuro and impasto; narrative texts emphasize concrete scene entities and spatial layout. Text-to-image and retrieval models may exploit these lexical and metadata priors rather than the visual information each perspective encodes. The paper should include null conditions or ablations, such as title-only prompts, metadata-only prompts, style-only prompts, and proper-noun-stripped perspectives, to show that the observed task asymmetry is not a prompt-prior artifact. This is load-bearing for the claim that 'no single perspective suffices for all tasks.'
  4. [§4.1, Fig. 4] The text states that all conditions are synthesized into ~80-word prompts so that differences in fidelity are attributable to perspective content rather than formatting, but the synthesis itself is performed by a prompt-following LLM. No check is reported that the synthesized prompts preserve the perspective-specific information or that the downstream generator actually follows the synthesized prompt. Reporting prompt-fidelity statistics or a small human inspection of the synthesized prompts would strengthen the diagnostic interpretation of the reconstruction results.
minor comments (6)
  1. [§4.3] The sentence 'We have nince annotators participated included researchers with graduate-level training in visual cultural and art understanding' contains a typo and ungrammatical phrasing; please revise and report the recruitment and rater-agreement statistics more precisely.
  2. [§4.1] The phrase 'which probing what visual information each perspective encodes' is ungrammatical; it should read 'probing which visual information each perspective encodes.'
  3. [Fig. 4] The figure caption and x-axis use 'Full' for what appears to be the unified caption condition, while the text sometimes refers to 'the full four-perspective condition'; please clarify whether the full condition is the unified caption or the concatenation of all four perspectives.
  4. [Eq. (1)] Equation (1) is missing a comma between e_hist and e_unif.
  5. [§3.3] The phrase 'elength under 30 words are nulled' should read 'explanations with length under 30 words are nulled.'
  6. [§4.1] The nine conditions are described as being evaluated on 'the same stratified 1,000 painting samples,' but retrieval is performed over the full 74k gallery; please state explicitly that the queries are the 1,000-painting subset while the gallery contains all 74,234 paintings.

Circularity Check

1 steps flagged · score 3.0 of 10

Emotional-fidelity metric is self-referential; core task-asymmetry results remain independent

  1. fitted input called prediction [Section 3.2 (Annotation Pipeline, Emotional response) and Section 4.1 (Eq. 3, delta_emot)]
    "Emotional perspectives are generated using Qwen3-VL-8B-Instruct [2], conditioned on the painting image together with human-written affective utterances from ArtEmis [1,23] ... delta_emot is CLIP zero-shot emotion agreement, measuring the proportion of paintings for which the regenerated image and the original share the same top-1 ArtEmis emotion label."

    e_emot is generated by conditioning on ArtEmis human utterances, and delta_emot scores reconstruction against the top-1 ArtEmis emotion label from the same dataset. The emotional perspective's 'affective fidelity' therefore measures whether the T2I model can echo the conditioning annotation, not whether the perspective independently captures affect. This makes the affective-tone component of Fig. 4 and the abstract's claim that historical descriptions carry strong affective signal partially self-referential. The CLIP/DINO style-composition metrics and the retrieval asymmetry (narrative R@1=44.0% vs formal R@1=7.8%) are independent of ArtEmis, so the central 'no single perspective suffices' claim is not reduced to this fit.

full rationale

The paper's central contribution is a dataset resource; the main complementarity conclusion is supported by retrieval experiments using two independent embedding models and by blinded human evaluation, neither of which is tied to the ArtEmis labels used to construct e_emot. The only concrete self-referential step is the emotion-agreement metric in the generative analysis, which uses ArtEmis both as conditioning input and as ground-truth labels. The semantic-distinctiveness check (Table 2) is weak because the prompts already enforce cross-perspective separation, but the paper does not rest the central claim on it alone. Self-citations to ArtRAG [37] and related prior work are tool choices, and the historical perspective is independently assessed by Gemma-3 and human raters, so no load-bearing self-citation chain is present. Overall, the derivation is largely self-contained, with one localized circular metric; score 3 reflects that partial, non-central circularity.

Assumptions & free parameters 5 free parameters · 5 assumptions · 0 invented entities

The central claim relies on hand-chosen thresholds and sample sizes, on Panofsky's interpretive framework, and on the validity of CLIP/DINOv3/text-to-image diagnostics. No new physical or theoretical entities are introduced.

free parameters (5)
  • Perspective length filter threshold = 30 words
    Descriptions shorter than 30 words are nulled; this hand-chosen threshold affects which paintings enter the experiment-ready subset.
  • Unified caption target length = ~150 words
    The harmonization prompt asks for approximately 150 words, affecting text length and retrieval or reconstruction inputs.
  • Top-k historical context documents = 5
    ArtRAG retrieves the top-5 context documents for the historical perspective, and this choice affects the historical text content.
  • Complementarity analysis sample size = 1,000 paintings
    All complementarity analyses use the same stratified 1,000 sample with no power analysis, so leave-one-out differences may be noisy.
  • Multi-perspective reconstruction prompt length = ~80 words
    Conditions are synthesized into an ~80-word prompt before text-to-image generation; length and format may affect reconstruction fidelity.
assumptions (5)
  • domain assumption Panofsky's framework: narrative, formal, emotional, and historical are distinct and non-reducible interpretive acts.
    Invoked in Section 3.1 to justify the four-perspective decomposition of every artwork annotation.
  • domain assumption CLIP text-embedding cosine similarity is a valid measure of semantic distinctiveness between perspectives.
    Section 3.3 uses pairwise similarity below 0.55 as evidence of non-redundancy, but CLIP embeddings are not calibrated for interpretive distinction.
  • domain assumption Text-to-image reconstruction fidelity measured by CLIP, DINOv3, and emotion agreement is a valid probe of perspective information content.
    Section 4.1 calls reconstruction a diagnostic probe, but no verification is given that generators follow the text rather than their priors.
  • domain assumption ArtEmis human affective utterances are valid grounding for the emotional perspective.
    Section 3.2 uses ArtEmis utterances as affective anchors, assuming crowd-sourced emotional labels transfer to the generated descriptions.
  • domain assumption Gemma-3-27B LLM-as-judge ratings are reliable quality measures.
    Section 3.3 uses a single judge model with no human calibration of the 1-5 fidelity, accuracy, and depth scores.

how reviews work

0 comments
Cite this review

Pith. "Pith review of MMArt A Multi-Perspective Multimodal Dataset for Visual Art Understanding." pith.science (2026). https://pith.science/paper/URYH3SRO

@misc{pith2026260810706,
  author       = {Pith},
  title        = {Pith review of: MMArt A Multi-Perspective Multimodal Dataset for Visual Art Understanding},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/URYH3SRO}},
  note         = {Machine review of arXiv:2608.10706}
}
read the original abstract

Recent vision-language models demonstrate impressive general visual understanding, yet their art interpretation remains shallow: they describe surface content but struggle with formal analysis, grounded historical interpretation, or affective characterization. We argue this is not only a model but also a dataset limitation. Existing art datasets are single perspective resources, where no dataset provides narrative, formal, emotional, and historical perspectives simultaneously for the same artworks. We introduce MMArt, a large-scale dataset of 74,234 WikiArt paintings, each annotated with four independently annotated perspectives plus a harmonized unified caption, produced by specialized vision-language models or human annotation and validated through complementary quality evaluations. Two complementarity analyses establish that perspectives encode genuinely distinct information. A generative analysis shows that formal analysis descriptions best preserve compositional style, and historical descriptions carry strong affective signal in reconstructed images. A discriminative retrieval analysis reveals task-asymmetry: narrative descriptions drive retrieval (R@1 = 44.0%), while formal descriptions, strongest for reconstruction, are nearly nondiscriminative at retrieval scale (R@1 = 7.8%). Leave-one-out analysis further confirms that historical descriptions are the least replaceable perspective across both tasks. Together, the two analyses establish that no single perspective suffices for all tasks, directly motivating MMArt multi-perspective design. The dataset, code, and additional information are available at https://shuaiwang97.github.io/MMArt/.

Figures

Figures reproduced from arXiv: 2608.10706 by the authors.

Figure 1
Figure 1. Overview of the MMArt dataset. Each painting is annotated with four source￾specialized perspectives: Narrative and Scene Interpretation, Formal Visual Analysis, Emotional Response, and Historical and Contextual Analysis, plus a harmonized uni￾fied caption. Perspective validity is evaluated through a text-to-image reconstruction and perspective guided retrieval experiment measuring fidelity. viewer encountering Verme… view at source ↗
Figure 2
Figure 2. Overview of the MMArt dataset construction pipeline. Each painting is pro￾cessed through four specialized vision-language models to produce independently anno￾tated perspectives (narrative, formal, emotional, historical), which are then harmonized into a unified caption. 3 MMArt Dataset Construction The four-perspective architecture is grounded in Panofsky’s framework [25] for layered art interpretation, which disti… view at source ↗
Figure 3
Figure 3. Top: Characteristic vocabularies for different perspectives, using TF-IDF weighted word clouds. Bottom: Chronological distribution and volume of art styles. analysis framework to probe precisely this: whether each perspective’s utility is task-asymmetric. Both analyses evaluate nine conditions: four singles (e narr , e form, e emot , e hist), four leave-one-out triples (each omitting one perspective), and the unifie… view at source ↗
Figures from the paper (5 more)
Figure 4
Figure 4. Figure 4: Reconstruction fidelity across nine perspective conditions for FLUX.2-Klein (blue) and Qwen-Image (red). Shaded regions separate singles, leave-one-out triples, and the full four-perspective condition. Error bars show ±1 standard deviation. plementary by design: CLIP’s…
Figure 5
Figure 5. Figure 5: Description to painting retrieval performance across perspective conditions from the full MMArt gallery. Perspective-Guided Painting Retrieval Results [PITH_FULL_IMAGE:figures/full_fig_p012_5.png]
Figure 6
Figure 6. Figure 6: Claude Monet, Fishing Boats off Pourville (Impressionism). The loose brushwork and coastal atmosphere are partially recovered under N and E, but the formal description best preserves the painting’s hazy light and compositional structure. NFEH produces the most faithful…
Figure 7
Figure 7. Figure 7: Rembrandt, Saskia as Flora (1634, Baroque). The portrait character is strongly recovered under F, especially in chiaroscuro and three-quarter pose, and under NFEH. The historical perspective drifts toward a generic Dutch Golden Age portrait, while E captures the contem…
Figure 8
Figure 8. Figure 8: shows that annotators consistently preferred MMArt descriptions over the Claude 4.5 Sonnet baseline. Across all 312 judgments, MMArt was se￾lected in 208 cases (67%), compared with 87 selections for Claude (28%) and 17 ties (5%). The preference holds for every perspect…

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

44 extracted references · 33 canonical work pages

  1. [1]

    In: 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) (2021)

    Achlioptas, P., Ovsjanikov, M., Haydarov, K., Elhoseiny, M., Guibas, L.: ArtEmis: Affective Language for Visual Art. In: 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) (2021)

    work page 2021

  2. [2]

    Bai, S., Cai, Y., Chen, R., Chen, K., Chen, X., Cheng, Z., Deng, L., Ding, W., Gao, C., Ge, C., Ge, W., Guo, Z., Huang, Q., Huang, J., Huang, F., Hui, B., Jiang, S., Li, Z., Li, M., Li, M., Li, K., Lin, Z., Lin, J., Liu, X., Liu, J., Liu, C., Liu, Y., Liu, D., Liu, S., Lu, D., Luo, R., Lv, C., Men, R., Meng, L., Ren, X., Ren, X., Song, S., Sun, Y., Tang, ...

    arXiv 2025

  3. [3]

    2021 IEEE/CVF International Conference on Computer Vision (ICCV) pp

    Bai, Z., Nakashima, Y., Garc ´ ıa, N.: Explain me the painting: Multi-topic knowl- edgeable art description generation. 2021 IEEE/CVF International Conference on Computer Vision (ICCV) pp. 5402–5412 (2021),https://api.semanticscholar. org/CorpusID:237490413

    work page 2021

  4. [4]

    Oxford Paperbacks (1988)

    Baxandall, M.: Painting and experience in fifteenth century Italy: a primer in the social history of pictorial style. Oxford Paperbacks (1988)

    work page 1988

  5. [5]

    In: Proceedings of the 32nd ACM International Conference on Multimedia

    Bin, Y., Shi, W., Ding, Y., Hu, Z., Wang, Z., Yang, Y., Ng, S.K., Shen, H.T.: Gallerygpt: Analyzing paintings with large multimodal models. In: Proceedings of the 32nd ACM International Conference on Multimedia. (2024)

    work page 2024

  6. [6]

    Black Forest Labs: Flux.2: Analyzing and enhancing the latent space of flux – rep- resentation comparison.https://bfl.ai/research/representation-comparison (2025)

    work page 2025

  7. [7]

    In: 2024 IEEE/CVF Winter Conference on Applications of Computer Vision (W ACV)

    Bleidt, T., Eslami, S., De Melo, G.: ArtQuest: Countering Hidden Language Biases in ArtVQA. In: 2024 IEEE/CVF Winter Conference on Applications of Computer Vision (W ACV). Waikoloa, HI, USA (2024)

    work page 2024

  8. [8]

    In: Del Bimbo, A., Cuc- chiara, R., Sclaroff, S., Farinella, G.M., Mei, T., Bertini, M., Escalante, H.J., Vez- zani, R

    Cetinic, E.: Iconographic image captioning for artworks. In: Del Bimbo, A., Cuc- chiara, R., Sclaroff, S., Farinella, G.M., Mei, T., Bertini, M., Escalante, H.J., Vez- zani, R. (eds.) Pattern Recognition. ICPR International Workshops and Challenges (2021)

    work page 2021

Show all 44 references
  1. [9]

    In: Eu- ropean Conference on Computer Vision (2024)

    Chen, L., Li, J., Dong, X., Zhang, P., He, C., Wang, J., Zhao, F., Lin, D.: Sharegpt4v: Improving large multi-modal models with better captions. In: Eu- ropean Conference on Computer Vision (2024)

    2024

  2. [10]

    arXiv preprint arXiv:2603.02435 (2026)

    Efthymiou, A., Rudinac, S., Kackovic, M., Wijnberg, N., Worring, M.: Vl- kge: Vision-language models meet knowledge graph embeddings. arXiv preprint arXiv:2603.02435 (2026)

    2026

  3. [11]

    In: Pro- ceedings of the 29th ACM International Conference on Multimedia (2021)

    Efthymiou, A., Rudinac, S., Kackovic, M., Worring, M., Wijnberg, N.: Graph neu- ral networks for knowledge enhanced visual representation of paintings. In: Pro- ceedings of the 29th ACM International Conference on Multimedia (2021)

    2021

  4. [12]

    In: Proceedings of the European Conference in Computer Vision Workshops (2018) 16 S

    Garcia, N., Vogiatzis, G.: How to read paintings: Semantic art understanding with multi-modal retrieval. In: Proceedings of the European Conference in Computer Vision Workshops (2018) 16 S. Wang et al

    2018

  5. [13]

    Garcia, N., Ye, C., Liu, Z., Hu, Q., Otani, M., Chu, C., Nakashima, Y., Mitamura, T.: A Dataset and Baselines for Visual Question Answering on Art (Aug 2020), http://arxiv.org/abs/2008.12520

    2020 arXiv

  6. [14]

    In: Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics

    Hayashi, K., Sakai, Y., Kamigaito, H., Hayashi, K., Watanabe, T.: Towards artwork explanation in large-scale vision language models. In: Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics

  7. [15]

    ACM Comput

    Ji, Z., Lee, N., Frieske, R., Yu, T., Su, D., Xu, Y., Ishii, E., Bang, Y.J., Madotto, A., Fung, P.: Survey of hallucination in natural language generation. ACM Comput. Surv. (Mar 2023)

    2023

  8. [16]

    In: Proceedings of the British Machine Vision Conference

    Karayev, S., Trentacoste, M., Han, H., Agarwala, A., Darrell, T., Hertzmann, A., Winnemoeller, H.: Recognizing image style. In: Proceedings of the British Machine Vision Conference. BMV A Press (2014)

    2014

  9. [17]

    Koukounas, A., Mastrapas, G., Eslami, S., Wang, B., Akram, M.K., G¨ unther, M., Mohr, I., Sturua, S., Wang, N., Xiao, H.: jina-clip-v2: Multilingual multimodal embeddings for text and images (2025),https://arxiv.org/abs/2412.08802

    2025 arXiv

  10. [18]

    Li, D., Wang, S., Zou, J., Tian, C., Nieuwburg, E., Sun, F., Kanoulas, E.: Paint4poem: A dataset for artistic visualization of classical chinese poems (2021), https://arxiv.org/abs/2109.11682

    2021 arXiv

  11. [19]

    Journal of Big Data11(1), 61 (2024)

    Li, J., Wang, S., Rudinac, S., Osseyran, A.: High-performance computing in health- care: An automatic literature analysis perspective. Journal of Big Data11(1), 61 (2024)

    2024

  12. [20]

    Li, M., Zhang, Y., Long, D., Chen, K., Song, S., Bai, S., Yang, Z., Xie, P., Yang, A., Liu, D., Zhou, J., Lin, J.: Qwen3-vl-embedding and qwen3-vl-reranker: A unified framework for state-of-the-art multimodal retrieval and ranking (2026),https: //arxiv.org/abs/2601.04720

    2026 arXiv

  13. [21]

    In: Advances in Neural Information Processing Systems

    Liu, H., Li, C., Wu, Q., Lee, Y.J.: Visual instruction tuning. In: Advances in Neural Information Processing Systems. vol. 36 (2023)

    2023

  14. [22]

    IEEE Transactions on Computational Social Systems (2024)

    Lu, Y., Guo, C., Dai, X., Wang, F.: Artcap: A dataset for image captioning of fine art paintings. IEEE Transactions on Computational Social Systems (2024)

    2024

  15. [23]

    In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition

    Mohamed, Y., Khan, F.F., Haydarov, K., Elhoseiny, M.: It is okay to not be okay: Overcoming emotional bias in affective image captioning by contrastive data collec- tion. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. pp. 21263–21272 (2022)

    2022

  16. [24]

    van Noord, N., Garcia, N.: The iconicity of the generated image (2025),https: //arxiv.org/abs/2509.16473

    2025

  17. [25]

    University of Chicago Press (1955), https://books.google.nl/books?id=Qsa00QEACAAJ

    Panofsky, E.: Meaning in the Visual Arts. University of Chicago Press (1955), https://books.google.nl/books?id=Qsa00QEACAAJ

    1955

  18. [26]

    Radford, A., Kim, J.W., Hallacy, C., Ramesh, A., Goh, G., Agarwal, S., Sastry, G., Askell, A., Mishkin, P., Clark, J., Krueger, G., Sutskever, I.: Learning transferable visual models from natural language supervision (2021),https://arxiv.org/abs/ 2103.00020

    2021 arXiv

  19. [27]

    Riccio, P., Doh, M., H¨ oltgen, B., Garcia, N., van Noord, N.: Gender artifacts from art history to text-to-image generation (2026),https://arxiv.org/abs/ 2606.05829

    2026 arXiv

  20. [28]

    2901–2903

    Riccio, P., Khan, S., Schaerf, L., Wang, S., Liu, T., Efthymiou, A., Garcia, N., van Noord, N.: Understanding Art & Culture, p. 2901–2903. Association for Computing Machinery (2026)

    2026

  21. [29]

    Schaerf, L., Purificato, A., Riccio, P., Silvestri, F., Garcia, N.: Art beyond se- mantics: Sheaf-informed contrastive learning for multi-relational representations (2026),https://arxiv.org/abs/2607.16321 Abbreviated paper title 17

    2026 arXiv

  22. [30]

    Shen, Y., Yang, Z., Bi, Q., Wang, C., Wang, S., Huang, J.H., Floros, G., Tiwari, P., Pathania, A.: Spectral-progressive thought flow for lightweight multimodal rea- soning (2026),https://arxiv.org/abs/2606.02842

    2026 arXiv

  23. [31]

    In: Proceedings of the 27th ACM International Conference on Multimedia (2019)

    Sheng, S., Moens, M.F.: Generating captions for images of ancient artworks. In: Proceedings of the 27th ACM International Conference on Multimedia (2019)

    2019

  24. [32]

    Sim´ eoni, O., Vo, H.V., Seitzer, M., Baldassarre, F., Oquab, M., Jose, C., Khalidov, V., Szafraniec, M., Yi, S., Ramamonjisoa, M., Massa, F., Haziza, D., Wehrstedt, L., Wang, J., Darcet, T., Moutakanni, T., Sentana, L., Roberts, C., Vedaldi, A., Tolan, J., Brandt, J., Couprie...

    2025 arXiv

  25. [33]

    In: Image Analysis and Processing – ICIAP 2019 (2019)

    Stefanini, M., Cornia, M., Baraldi, L., Corsini, M., Cucchiara, R.: Artpedia: A new visual-semantic dataset with visual and contextual sentences in the artistic domain. In: Image Analysis and Processing – ICIAP 2019 (2019)

    2019

  26. [34]

    ACM Trans

    Strezoski, G., Worring, M.: Omniart: A large-scale artistic benchmark. ACM Trans. Multimedia Comput. Commun. Appl. (2018)

    2018

  27. [35]

    Team, G.: Gemma 3 technical report (2025),https://arxiv.org/abs/2503.19786

    2025 arXiv

  28. [36]

    In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)

    Tong, S., Liu, Z., Zhai, Y., Ma, Y., LeCun, Y., Xie, S.: Eyes wide shut? exploring the visual shortcomings of multimodal llms. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). pp. 9568–9578 (June 2024)

    2024

  29. [37]

    In: Proceedings of the 33rd ACM International Conference on Multimedia

    Wang, S., Najdenkoska, I., Zhu, H., Rudinac, S., Kackovic, M., Wijnberg, N., Wor- ring, M.: Artrag: Retrieval-augmented generation with structured context for visual art understanding. In: Proceedings of the 33rd ACM International Conference on Multimedia. MM ’25 (2025)

    2025

  30. [38]

    In: MultiMedia Modeling (2024)

    Wang, S., Shen, J., Efthymiou, A., Rudinac, S., Kackovic, M., Wijnberg, N., Wor- ring, M.: Prototype-enhanced hypergraph learning for heterogeneous information networks. In: MultiMedia Modeling (2024)

    2024

  31. [39]

    In: ML Repro- ducibility Challenge 2021 (Fall Edition) (2022),https://openreview.net/forum? id=SVx46hzmhRK

    Wang, S., Verhagen, P., Zhuge, J., Shulev, V.: Replication study of DECAF: Gener- ating fair synthetic data using causally-aware generative networks. In: ML Repro- ducibility Challenge 2021 (Fall Edition) (2022),https://openreview.net/forum? id=SVx46hzmhRK

    2022

  32. [40]

    Wang, S., Zhang, D.W., Huang, J.H., Rudinac, S., Kackovic, M., Wijnberg, N., Worring, M.: Ada-hgnn: Adaptive sampling for scalable hypergraph neural net- works (2024),https://arxiv.org/abs/2405.13372

    2024 arXiv

  33. [41]

    In: Proceedings of the 2026 International Conference on Multimedia Retrieval

    Wang, S., Zhu, H., Huang, J., Shen, Y., Zeng, C., Rudinac, S., Kackovic, M., Wijn- berg, N., Worring, M.: A-mar: Agent-based multimodal art retrieval for fine-grained artwork understanding. In: Proceedings of the 2026 International Conference on Multimedia Retrieval. pp. 2246–...

    2026

  34. [42]

    Wu, C., Li, J., Zhou, J., Lin, J., Gao, K., Yan, K., ming Yin, S., Bai, S., Xu, X., Chen, Y., Chen, Y., Tang, Z., Zhang, Z., Wang, Z., Yang, A., Yu, B., Cheng, C., Liu, D., Li, D., Zhang, H., Meng, H., Wei, H., Ni, J., Chen, K., Cao, K., Peng, L., Qu, L., Wu, M., Wang, P., Yu,...

    2025 arXiv

  35. [43]

    In: Proceedings of conference on language modelling (2024)

    Yuan, Z., Xue, H., Wang, X., Liu, Y., Zhao, Z., Wang, K.: Artgpt-4: Towards artistic-understanding large vision-language models with enhanced adapter. In: Proceedings of conference on language modelling (2024)

    2024

  36. [44]

    In: Proceedings of the 2026 International Conference on Multimedia Retrieval

    Zeng, C., Jiang, Y., Gao, G., Wang, S., Danier, D., Zhu, B., Rudinac, S., Bull, D., Zhang, F.: Sam3-litetext: An anatomical study of the sam3 text encoder for efficient vision-language segmentation. In: Proceedings of the 2026 International Conference on Multimedia Retrieval. ...

    2026

Pith tools

Reviewed August 12, 2026 · model on record in the stance chip above.