Pith. sign in

REVIEW 3 major objections 6 minor 3 cited by

Analyzing and Boosting the Power of Fine-Grained Visual Recognition for Multi-modal Large Language Models

T0 review · 3 major / 6 minor · reviewed 2026-08-10 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read The paper argues that fine-grained recognition fails in MLLMs because object and category name representations are misaligned, and that attribute descriptions as a bridge lift average accuracy from 65.95 to 76.84 on six benchmarks.

desk verdict The diagnostic analysis is worth reading, but the headline 10.89-point gain is a zero-shot vs. fine-tuned comparison; the paper's own ablation narrows the proposed mechanism's contribution to about 0.7 points. read the letter →

arxiv 2501.15140 v3 pith:FFOCHXK7 submitted 2025-01-25 cs.CV cs.AIcs.CLcs.LG

Hulingxiao He , Geng Li , Zijun Geng , Jinglin Xu , Yuxin Peng This is my paper · ORCID
classification cs.CVcs.AIcs.CLcs.LG
keywords fine-grainedvisualrecognitionmulti-modallargelanguagemodelsobject-categoryalignmentattributedescriptionscontrastivelearninghardnegativesinstructiontuningFinedefics
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper asks why multimodal large language models misclassify subordinate categories such as dog breeds, bird species, and flower types, and locates the failure in a single part of the pipeline: the representation of a visual object and the representation of its category name are not aligned in the model's embedding space. The authors show that object information extraction and category knowledge are both adequate, so the bottleneck is object-category misalignment. They then build Finedefics, which generates a per-image attribute description in natural language and uses contrastive learning to bind the object representation to the description and the description to the category name, with similar-but-wrong categories serving as hard negatives. On six fine-grained recognition datasets, this raises the average accuracy of the base model Idefics2 from 65.95 to 76.84 and surpasses Qwen-VL-Chat by 9.43 points on average.

What carries the argument

The machinery is Attribute Augmented Alignment, a two-stage training recipe built on the base model Idefics2. For each training image, a pipeline of GPT-4 or LLaMA attribute discovery, BLIP-2 or LLaVA attribute extraction, and final LLM summarization produces a per-sample attribute description. The description is tokenized and passed through the LLM to obtain a global representation, taken from an appended EOS token, and this representation becomes the intermediate anchor in two contrastive losses: Object-Attribute Contrastive (OAC) and Attribute-Category Contrastive (ACC). A third Category-Category Contrastive (CCC) loss separates similar category names, and CLIP-mined hard negatives from the three most similar incorrect categories strengthen the negatives. Stage I optimizes these contrastive losses together with an attribute-description generation loss; Stage II fine-tunes on open-set QA and closed-set multiple-choice classification data.

What would settle it

Take the Finedefics training triples and swap each image's attribute description with the description of another image from the same subordinate category, while keeping object and category fixed; if accuracy on the six benchmarks stays near the reported 76.84, the object-to-attribute binding is not carrying discriminative information. A second check is to feed the constructed descriptions alone, with category names removed, to a text-only LLM and see whether it can already name the classes.

Watch

Extended reading notes

Core claim

The central discovery is that MLLMs lose fine-grained classification accuracy not because they fail to extract discriminative visual details and not because they lack category knowledge, but because visual object embeddings and category-name embeddings sit in different regions of the LLM's representation space. Linear probing on the base model Idefics2 reaches about 95% accuracy for object features on Oxford-IIIT Pet-37, close to the vision-language model SigLIP, while probing on LLM-generated category descriptions reaches 92.51%, showing knowledge is present; category names alone, however, are poorly separated. Finedefics closes the gap by inserting an attribute description between object and category: it trains object-attribute and attribute-category contrastive losses simultaneously, adds a category-category contrastive loss to separate similar category names, and mines hard negatives with CLIP. After this alignment, a classification-centered instruction-tuning stage adapts the model to the task. The reported result is 76.84 average accuracy across six FGVR datasets, a gain of 10.89 points over Idefics2 and 9.43 points over Qwen-VL-Chat.

Load-bearing premise

The method assumes the auto-generated attribute descriptions accurately describe the individual image and separate subordinate categories well enough to serve as the binding link, and this assumption is not verified sample by sample.

Editorial extensions

If this is right

  • Object-centric question answering and reasoning that depend on subordinate-level identification should inherit the accuracy gains, since the paper identifies FGVR as a prerequisite for those capabilities.
  • The alignment recipe transfers to other MLLMs: applying attribute augmented alignment to LLaVA 1.5 raises its average accuracy from 43.24 to 57.21 on the same six datasets.
  • Two-stage training matters: running alignment and instruction tuning together collapses performance to 25.42, so representation alignment must precede task tuning.
  • Replacing CLIP-mined hard negatives with simple negatives reduces the average gain from 76.84 to 74.26, so the choice of similar-but-wrong negatives is load-bearing for the method.
  • Per-sample attribute descriptions matter more than per-class tags: on Bird-200, Finedefics reaches 51.12, above a tag-based baseline of 50.48 and near the human-annotated upper bound of 52.52.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • A natural next test is whether the same contrastive binding works when the attribute descriptions are produced by a single open-source model rather than a GPT-4/VQA cascade, which would remove the dependence on proprietary generation.
  • The failure analysis suggests that any mechanism that moves category names closer to object representations in the LLM embedding space, such as richer textual category definitions or retrieval of exemplar captions, could substitute for the attribute pipeline; a controlled comparison would reveal whether descriptions are uniquely effective.
  • A testable way to isolate the alignment contribution is to swap each image's attribute description with one from another image of the same class: if accuracy stays near 76.84, the object-to-attribute binding is not carrying the discriminative information.
  • The two-stage structure implies that continual fine-grained recognition, where new categories arrive after training, would require rerunning the contrastive alignment stage, a direction the paper names as future work.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 6 minor

Summary. The paper studies why multimodal large language models (MLLMs) underperform on fine-grained visual recognition (FGVR). It argues, on the basis of representation probing and visualization, that the bottleneck is not object information extraction or category knowledge, but misalignment between visual object representations and category name representations in the LLM's representation space. The proposed method, Finedefics, builds on Idefics2 and uses automatically constructed per-sample attribute descriptions as an intermediate binding: Stage I performs contrastive learning on object-attribute and attribute-category pairs with hard negatives, and Stage II performs classification-centered instruction tuning on open-set and closed-set FGVR data. The paper reports average gains of +10.89% over Idefics2 and +9.43% over Qwen-VL-Chat across six FGVR datasets, and supports the method with a large set of ablations, including attribute type, hard negatives, training stage, description quality, and generalization to other MLLMs.

Significance. The diagnostic setup is thoughtful: linear probing of object and category features, t-SNE visualizations, and the explicit decomposition into information extraction, knowledge reserve, and alignment are useful ways to study MLLM FGVR failure. The idea of using attribute descriptions as a binding intermediate, with hard negatives mined from similar categories, is sensible and the paper ships code and extensive ablations. However, the central quantitative claim is not supported by the reported comparisons, because the main table compares a model fine-tuned on the evaluation datasets' training splits against zero-shot baselines. The paper's own same-setting control reduces the attributed gain from +10.89 points to roughly +0.71 points, fundamentally changing the significance of the proposed mechanism.

major comments (3)
  1. [Section 4.2, Table 2 and Appendix Table 6b] The headline comparison is confounded by supervised fine-tuning. Finedefics is fine-tuned on the training splits of the six evaluation datasets using both Stage I contrastive learning and Stage II classification-centered instruction tuning, while all Table 2 baselines are evaluated zero-shot. The paper's own same-setting control, the "I.T. only" row in Table 6b, reaches 76.13% average accuracy, only 0.71 points below Finedefics's 76.84%. Thus the advertised +10.89% and +9.43% margins over Idefics2 and Qwen-VL-Chat do not isolate the effect of attribute augmented alignment; they are largely attributable to in-domain supervised fine-tuning. The central claim that "benefiting from attribute augmented alignment, Finedefics outperforms existing MLLMs" is therefore unsupported as stated. The same-setting control should appear in the main results, and the claims should be revised accordingly, or the comparison should include same-setting fine-tuned baselines for the other models.
  2. [Section 4.3, Table 3a] The "Finetune" control is not an adequate baseline for isolating the method's contribution. According to the text, this variant is fine-tuned solely on additional open-set FGVR data, so it omits the closed-set multiple-choice data used at evaluation. Its reported 0.03 accuracy therefore reflects a format-following failure rather than the effect of the proposed training paradigm. The relevant control is the classification-centered instruction-tuning-only row in Table 6b (76.13%), which should be reported and discussed in the main text alongside Table 3a.
  3. [Section 3.1 and Appendix Table 6c] The attribute description construction pipeline is not verified for faithfulness. The method relies on GPT-4/LLaMA to select attribute keys and on BLIP-2/LLaVA to read attribute values, but no per-sample check is provided that the resulting descriptions are faithful to the image content or sufficiently discriminative across subordinate categories. The noisy-description ablation in Table 6c perturbs already-generated text; it does not test whether the upstream pipeline produces descriptions that actually describe the image. Since the paper's mechanism depends on attribute descriptions binding objects to categories, the paper should provide direct evidence of descriptor faithfulness, for example by human evaluation or by measuring how often the VQA-extracted attributes match ground-truth attributes, beyond the probing accuracy reported in Section A.1.
minor comments (6)
  1. [Section 3.1] The subsection heading contains a typo: "Userful Attribute Discovery" should be "Useful Attribute Discovery".
  2. [Equation (7)] The displayed formula for LCCC has a malformed "− log 1P" term; the intended numerator is not typeset correctly and should be fixed.
  3. [Table 3a] The row label "Finetune 0.03" is unexplained. The numeric suffix appears to be an accuracy value from the table, but the label should be self-explanatory (e.g., "Finetune (open-set only)").
  4. [Table 6e] The numbers in Table 6e are confusing relative to Table 2: Finedefics is reported as 51.12 on Bird-200 here, while Table 2 reports 57.61 on the same dataset. The text states that this experiment is trained solely on Bird-200; this restriction should be stated clearly in the table caption and in the surrounding text.
  5. [Section 2.3] The notation is inconsistent: after defining the category embedding sequence of length n, the text refers to "the category name ˆci_m"; the subscript should be consistent with the sequence length n.
  6. [References] Several references are incomplete, including the Zhai et al. entry (missing year) and the Chen et al. entry in the Related Work section (missing year and venue).

Circularity Check

1 steps flagged · score 1.0 of 10

No significant circularity; one minor self-confirmatory alignment metric.

  1. self definitional [Appendix A.2, Table 8 (object-category alignment quality)]
    "The object-category alignment quality is calculated as the mean cosine similarity between embeddings of visual objects and their corresponding category names of each class. We observe that Finedefics significantly increases the object-category alignment quality, which further demonstrates the effectiveness in boosting alignment."

    Stage I (Eqs. 5-8) explicitly maximizes cosine similarities Sim(o,a) and Sim(a,c) for positive object-attribute and attribute-category pairs. The Table 8 metric is the mean cosine similarity between object and category embeddings, which the contrastive objective is designed to increase (transitively through the attribute binding). Reporting that this metric rises after training is therefore a check that the loss was optimized, not an independent confirmation that attribute augmented alignment is the cause of the accuracy gain. The paper's central FGVR accuracy claims are separate, empirical results and do not reduce by construction.

full rationale

This is an empirical training paper. The central claims are accuracy improvements on six FGVR benchmarks, obtained by training on dataset training splits and evaluating on test splits; no parameter is fitted to the test result and then renamed as a prediction. The root-cause analysis (Section 2) uses linear probing and representation visualizations as diagnostics, not as equations that assume the conclusion. The attribute descriptions are built externally with GPT-4, LLaMA, BLIP-2, and LLaVA, and the hard negatives come from CLIP, so no load-bearing self-citation chain is present. The main comparison against zero-shot Idefics2/Qwen-VL-Chat is confounded by supervised fine-tuning, and the paper's own Table 6b shows instruction tuning alone nearly matches Finedefics, but that is a benchmarking/interpretation concern rather than circularity. The only circular-adjacent element is the Appendix A.2 alignment-quality metric, which essentially measures what Stage I optimizes; this is a minor self-confirmatory check and does not force the paper's main empirical results.

Assumptions & free parameters 1 free parameters · 4 assumptions · 0 invented entities

The paper's improvement rests on the constructed attribute descriptions and on a shared EOS-token embedding space; these are domain assumptions rather than fitted constants. No new physical or conceptual entities are introduced.

free parameters (1)
  • hard_negative_count = 3
    Number of similar-but-incorrect category samples mined with CLIP per image (Section 3.2); hand-chosen, not ablated.
assumptions (4)
  • domain assumption The EOS-token embeddings from the last LLM layer, produced by separately encoding object visual tokens, attribute descriptions, and category names, lie in a common space where cosine similarity measures semantic alignment.
    Section 3.2 applies contrastive losses on these embeddings; Table 8's alignment quality metric uses the same cosine similarity.
  • domain assumption The automatically generated per-sample attribute descriptions are sufficiently faithful and discriminative to bind objects to categories.
    Section 3.1 builds (object, attribute, category) triples from GPT-4/LLaMA plus BLIP-2/LLaVA outputs; Table 6c only tests synthetic noise, not systematic hallucination.
  • domain assumption The multiple-choice evaluation protocol built from the test sets is a valid and fair measure of FGVR for both zero-shot baselines and the fine-tuned model.
    Section 4.1 constructs multiple-choice tasks from test sets for all models; no evidence that this format is equally suited to zero-shot and fine-tuned models.
  • domain assumption The bottleneck for FGVR is object-category alignment rather than object information extraction or category knowledge.
    Section 2 reaches this via probing and t-SNE on limited categories; if the bottleneck varies by dataset, the proposed alignment may not generalize.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Analyzing and Boosting the Power of Fine-Grained Visual Recognition for Multi-modal Large Language Models." pith.science (2026). https://pith.science/paper/FFOCHXK7

@misc{pith2026250115140,
  author       = {Pith},
  title        = {Pith review of: Analyzing and Boosting the Power of Fine-Grained Visual Recognition for Multi-modal Large Language Models},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/FFOCHXK7}},
  note         = {Machine review of arXiv:2501.15140}
}
read the original abstract

Multi-modal large language models (MLLMs) have shown remarkable abilities in various visual understanding tasks. However, MLLMs still struggle with fine-grained visual recognition (FGVR), which aims to identify subordinate-level categories from images. This can negatively impact more advanced capabilities of MLLMs, such as object-centric visual question answering and reasoning. In our study, we revisit three quintessential capabilities of MLLMs for FGVR, including object information extraction, category knowledge reserve, object-category alignment, and position of the root cause as a misalignment problem. To address this issue, we present Finedefics, an MLLM that enhances the model's FGVR capability by incorporating informative attribute descriptions of objects into the training phase. We employ contrastive learning on object-attribute pairs and attribute-category pairs simultaneously and use examples from similar but incorrect categories as hard negatives, naturally bringing representations of visual objects and category names closer. Extensive evaluations across multiple popular FGVR datasets demonstrate that Finedefics outperforms existing MLLMs of comparable parameter sizes, showcasing its remarkable efficacy. The code is available at https://github.com/PKU-ICST-MIPL/Finedefics_ICLR2025.

Figures

Figures reproduced from arXiv: 2501.15140 by the authors.

Figure 1
Figure 1. Three quintessential capabilities of MLLMs for fine-grained visual recognition. Current [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 2
Figure 2. Object/Category/Object-category representation visualization of SigLIP and Idefics2. [PITH_FULL_IMAGE:figures/full_fig_p003_2.png] view at source ↗
Figure 3
Figure 3. An illustration of framework to build Finedefics. (a) Attribute Description Construction, [PITH_FULL_IMAGE:figures/full_fig_p005_3.png] view at source ↗
Figures from the paper (4 more)
Figure 4
Figure 4. Figure 4: Representation visualization of Finetune, CL (object-category) and Finedefics. [PITH_FULL_IMAGE:figures/full_fig_p009_4.png]
Figure 5
Figure 5. Figure 5: Confusion matrix of Oxford-IIIT Pet-37. from images, connects them with category names in the representation space, and then generates precise, fine-grained predictions. Furthermore, [PITH_FULL_IMAGE:figures/full_fig_p017_5.png]
Figure 6
Figure 6. Figure 6: Qualitative comparison on FGVR datasets, where green indicates correct predictions and [PITH_FULL_IMAGE:figures/full_fig_p018_6.png]
Figure 7
Figure 7. Figure 7: Error analysis examples. The left column shows the image for prediction, while the right [PITH_FULL_IMAGE:figures/full_fig_p018_7.png]

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 3 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. FineBadminton: A Multi-Level Dataset for Fine-Grained Badminton Video Understanding

    cs.MM 2025-08 conditional novelty 7.0 of 10

    A new badminton video dataset with action, tactic, and decision-level annotations, a 12-task benchmark, and a baseline showing hit-centric keyframes plus coordinate-guided compression improve MLLM performance.

  2. Learning Taxonomic Trees with Hierarchical Representation Regularization for Large Multimodal Models

    cs.CV 2026-07 conditional novelty 6.0 of 10

    HiR² extracts coarse-to-fine visual features from LMM layers and regularizes them with Lorentz entailment cones and unit-sphere dispersive loss, improving hierarchical consistency across models and fine-tuning methods.

  3. MCFNet: A Multimodal Collaborative Fusion Network for Fine-Grained Semantic Classification

    cs.CV 2025-05 conditional novelty 3.0 of 10

    MCFNet fuses ALBERT text features and ViT image features with dropout, L1/L2 regularization, hybrid self/cross attention, and multi-loss training, claiming state-of-the-art accuracy on Con-Text and Drink Bottle.

Reference graph

Works this paper leans on

25 extracted references · 5 canonical work pages · cited by 3 Pith papers

  1. [1]

    Phi-3 technical re- port: A highly capable language model locally on your phone

    Marah Abdin, Sam Ade Jacobs, Ammar Ahmad Awan, Jyoti Aneja, Ahmed Awadallah, Hany Awadalla, Nguyen Bach, Amit Bahree, Arash Bakhtiari, Harkirat Behl, et al. Phi-3 technical re- port: A highly capable language model locally on your phone. arXiv preprint arXiv:2404.14219,

    arXiv

  2. [3]

    As shown in Table 4, after employing our proposed method, LLaV A 1.5 gains an accuracy improvement by 13.97% on average, demonstrating the effectiveness and generalizability

    on another typical MLLM: LLaV A 1.5 (Liu et al., 2024b). As shown in Table 4, after employing our proposed method, LLaV A 1.5 gains an accuracy improvement by 13.97% on average, demonstrating the effectiveness and generalizability. Effects of attribute types. We analyze the effects of specific attribute types in FGVR tasks. Specif- ically, we selectively ...

    work page 2012

  3. [4]

    Shikra: Unleashing multimodal llm’s referential dialogue magic

    Keqin Chen, Zhao Zhang, Weili Zeng, Richong Zhang, Feng Zhu, and Rui Zhao. Shikra: Unleashing multimodal llm’s referential dialogue magic. arXiv preprint arXiv:2306.15195,

    arXiv

  4. [6]

    Instructblip: Towards general-purpose vision-language models with instruction tuning

    Wenliang Dai, Junnan Li, Dongxu Li, Anthony Meng Huat Tiong, Junqi Zhao, Weisheng Wang, Boyang Li, Pascale Fung, and Steven Hoi. Instructblip: Towards general-purpose vision-language models with instruction tuning. arXiv preprint arXiv:2402.03766,

    arXiv

  5. [7]

    Internlm-xcomposer2: Mastering free- form text-image composition and comprehension in vision-language large model

    Xiaoyi Dong, Pan Zhang, Yuhang Zang, Yuhang Cao, Bin Wang, Linke Ouyang, Xilin Wei, Songyang Zhang, Haodong Duan, Maosong Cao, et al. Internlm-xcomposer2: Mastering free- form text-image composition and comprehension in vision-language large model. arXiv preprint arXiv:2401.16420,

    arXiv

  6. [8]

    Palm-e: An embodied multi- modal language model

    Danny Driess, Fei Xia, Mehdi SM Sajjadi, Corey Lynch, Aakanksha Chowdhery, Brian Ichter, Ayzaan Wahid, Jonathan Tompson, Quan Vuong, Tianhe Yu, et al. Palm-e: An embodied multi- modal language model. arXiv preprint arXiv:2303.03378,

    arXiv

  7. [9]

    African or european swallow? benchmarking large vision-language models for fine-grained object classification

    Gregor Geigle, Radu Timofte, and Goran Glavaˇs. African or european swallow? benchmarking large vision-language models for fine-grained object classification. arXiv preprint arXiv:2406.14496,

    arXiv

  8. [12]

    Improved baselines with visual instruction tuning

    Haotian Liu, Chunyuan Li, Yuheng Li, and Yong Jae Lee. Improved baselines with visual instruction tuning. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recogni- tion, pp. 26296–26306, 2024a. Haotian Liu, Chunyuan Li, Qingyang Wu, and Yong Jae Lee. Visual instruction tuning. Advances in neural information processing systems, 36, ...

    arXiv

Show all 25 references
  1. [13]

    Video-chatgpt: Towards detailed video understanding via large vision and language models

    Muhammad Maaz, Hanoona Rasheed, Salman Khan, and Fahad Shahbaz Khan. Video-chatgpt: Towards detailed video understanding via large vision and language models. arXiv preprint arXiv:2306.05424,

    arXiv

  2. [14]

    Fine-grained visual classification of aircraft

    Subhransu Maji, Esa Rahtu, Juho Kannala, Matthew Blaschko, and Andrea Vedaldi. Fine-grained visual classification of aircraft. arXiv preprint arXiv:1306.5151,

    arXiv

  3. [15]

    Cats and dogs

    12 Published as a conference paper at ICLR 2025 Omkar M Parkhi, Andrea Vedaldi, Andrew Zisserman, and CV Jawahar. Cats and dogs. In CVPR, pp. 3498–3505. IEEE,

    2025

  4. [18]

    Open and efficient foundation language models

    H Touvron, T Lavril, G Izacard, X Martinet, MA Lachaux, T Lacroix, B Rozi`ere, N Goyal, E Ham- bro, F Azhar, et al. Open and efficient foundation language models. Preprint at arXiv. https://doi. org/10.48550/arXiv, 2302,

    doi

  5. [21]

    Sigmoid loss for language image pre-training

    Xiaohua Zhai, Basil Mustafa, Alexander Kolesnikov, and Lucas Beyer. Sigmoid loss for language image pre-training. Hang Zhang, Xin Li, and Lidong Bing. Video-llama: An instruction-tuned audio-visual language model for video understanding. arXiv preprint arXiv:2306.02858, 2023a....

    2014 arXiv

  6. [22]

    Internlm-xcomposer: A vision- language large model for advanced text-image comprehension and composition

    13 Published as a conference paper at ICLR 2025 Pan Zhang, Xiaoyi Dong Bin Wang, Yuhang Cao, Chao Xu, Linke Ouyang, Zhiyuan Zhao, Shuan- grui Ding, Songyang Zhang, Haodong Duan, Hang Yan, et al. Internlm-xcomposer: A vision- language large model for advanced text-image compreh...

    2025 arXiv

  7. [23]

    Minigpt-4: En- hancing vision-language understanding with advanced large language models

    Deyao Zhu, Jun Chen, Xiaoqian Shen, Xiang Li, and Mohamed Elhoseiny. Minigpt-4: En- hancing vision-language understanding with advanced large language models. arXiv preprint arXiv:2304.10592,

    arXiv

  8. [25]

    Then, these per-class sets of attribute-description pairs are assigned to each sample belonging to the class

    to acquire per-class attribute descriptions for each attribute tag used in our construction process, without using actual image samples. Then, these per-class sets of attribute-description pairs are assigned to each sample belonging to the class. Since for the same super categ...

    2025

  9. [26]

    Finedefics suc- cessfully captures the nuance of the object features, setting them apart from visually similar sub- ordinate categories. This confirms that Finedefics effectively captures fine-grained visual details 16 Published as a conference paper at ICLR 2025 Table 9: Comp...

    2025

  10. [2010]

    Vi- sual chatgpt: Talking, drawing and editing with visual foundation models

    Chenfei Wu, Shengming Yin, Weizhen Qi, Xiaodong Wang, Zecheng Tang, and Nan Duan. Vi- sual chatgpt: Talking, drawing and editing with visual foundation models. arXiv preprint arXiv:2303.04671,

    arXiv

  11. [2012]

    Kosmos-2: Grounding multimodal large language models to the world

    Zhiliang Peng, Wenhui Wang, Li Dong, Yaru Hao, Shaohan Huang, Shuming Ma, and Furu Wei. Kosmos-2: Grounding multimodal large language models to the world. arXiv preprint arXiv:2306.14824,

    arXiv

  12. [2013]

    Obelics: An open web-scale filtered dataset of interleaved image-text documents

    Hugo Laurenc ¸on, Lucile Saulnier, L´eo Tronchon, Stas Bekman, Amanpreet Singh, Anton Lozhkov, Thomas Wang, Siddharth Karamcheti, Alexander Rush, Douwe Kiela, et al. Obelics: An open web-scale filtered dataset of interleaved image-text documents. Advances in Neural Information...

    arXiv

  13. [2014]

    The caltech-ucsd birds-200-2011 dataset

    Catherine Wah, Steve Branson, Peter Welinder, Pietro Perona, and Serge Belongie. The caltech-ucsd birds-200-2011 dataset

    2011

  14. [2017]

    The many faces of robustness: A critical analysis of out-of-distribution generalization

    11 Published as a conference paper at ICLR 2025 Dan Hendrycks, Steven Basart, Norman Mu, Saurav Kadavath, Frank Wang, Evan Dorundo, Rahul Desai, Tyler Zhu, Samyak Parajuli, Mike Guo, Dawn Song, Jacob Steinhardt, and Justin Gilmer. The many faces of robustness: A critical analy...

    2025

  15. [2021]

    Pandagpt: One model to instruction-follow them all

    Yixuan Su, Tian Lan, Huayang Li, Jialu Xu, Yan Wang, and Deng Cai. Pandagpt: One model to instruction-follow them all. arXiv preprint arXiv:2305.16355,

    arXiv

  16. [2023]

    Qwen-vl: A frontier large vision-language model with versatile abilities

    Jinze Bai, Shuai Bai, Shusheng Yang, Shijie Wang, Sinan Tan, Peng Wang, Junyang Lin, Chang Zhou, and Jingren Zhou. Qwen-vl: A frontier large vision-language model with versatile abilities. arXiv preprint arXiv:2308.12966,

    arXiv

  17. [2024]

    Gpt-4 technical report

    Josh Achiam, Steven Adler, Sandhini Agarwal, Lama Ahmad, Ilge Akkaya, Florencia Leoni Ale- man, Diogo Almeida, Janko Altenschmidt, Sam Altman, Shyamal Anadkat, et al. Gpt-4 technical report. arXiv preprint arXiv:2303.08774,

    arXiv

Pith tools

Reviewed August 10, 2026 · model on record in the stance chip above.