Pith. sign in

REVIEW 4 major objections 5 minor 111 references

Visual Species Recognition with Large Multimodal Models as Post-Hoc Correctors

T0 review · 4 major / 5 minor · reviewed 2026-08-03 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read Few-shot species recognition climbs 6.4 accuracy points on average when a large multimodal model re-ranks the expert's top-5 guesses, prompted with confidence scores and a few example images per candidate.

desk verdict Useful, honest empirical paper on using LMMs to re-rank few-shot species classifiers; the headline +6.4 is real but needs a sharper statement of its limits, especially about LMM training-data overlap. read the letter →

arxiv 2512.15748 v2 pith:M25IYA4O submitted 2025-12-10 cs.LG cs.CV

Tian Liu , Anwesha Basu , James Caverlee , Shu Kong This is my paper
classification cs.LGcs.CV
keywords visualspeciesrecognitionfew-shotlearninglargemultimodalmodelspost-hoccorrectionin-contextre-rankingfine-grainedsoftmaxconfidence
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper asks whether large multimodal models (LMMs) can replace few-shot expert models at visual species recognition, and the answer is mostly no: LMMs prompted even with chain-of-thought or self-verification lag far behind a simple few-shot finetuned classifier (about 35.6% versus 59.2% mean accuracy). But the paper then establishes the complementary result that matters: when an LMM is handed the expert's top-5 candidate species, their softmax confidence scores, and a few example images of each candidate, it can re-rank the shortlist and recover the correct species in many cases where the expert's top guess was wrong. The proposed Post-hoc Correction (POC) is training-free, validation-free, and works across FSL methods, visual encoders, and LMMs, lifting mean accuracy to 65.6% — a 6.4-point gain averaged over five benchmarks covering birds, insects, weeds, mollusks, and fungi. A sympathetic reader would care because species-level annotation is expensive and scarce, so a plug-and-play correction layer that upgrades existing few-shot classifiers has direct value for ecology, conservation, and evolutionary biology.

What carries the argument

The load-bearing object is the POC multimodal prompt: the test image, the expert model's top-5 species names each labeled with its softmax confidence, a stitched grid of few-shot training images for each candidate, and an instruction to rank the five candidates from most to least likely. The paper shows the prompt does three separable jobs: the shortlist narrows the LMM to a manageable closed set (open-vocabulary prompting fails badly); the few-shot images supply fine-grained visual context that text attributes and taxonomy do not; and the confidence scores act as an adaptive trust signal, letting the LMM defer to the expert when it is confident and overrule it when it is not. The final ingr

What would settle it

Shuffle the softmax confidence scores before prompting and re-measure POC's accuracy. If the 6.4-point gain survives score-scrambling, confidence is not the adaptive signal claimed; if accuracy falls back toward the paper's no-confidence ablation (60.5%), the confidence component is load-bearing. A complementary check: on any taxon group where the expert's top-5 accuracy approximates its top-1 accuracy, the paper predicts POC gains near zero — directly testable with the released code.

Watch

Extended reading notes

Core claim

Stated the way the authors would: contemporary large multimodal models do not rival few-shot expert models at species recognition when asked open-endedly — in this study the best prompting strategy reaches 35.6% mean accuracy against the expert's 59.2% — but they have a complementary, underused skill. When given the expert's top-5 candidate species along with softmax confidences and a few reference images per candidate, an LMM can identify the correct species in a substantial fraction of the cases where the expert's top-1 was wrong. The resulting re-ranking pipeline, POC, raises the expert's mean accuracy from 59.2% to 65.6% across five benchmarks with no training, no validation set, and no

Load-bearing premise

The load-bearing premise is that the true species already appears among the expert model's top-k predictions: POC only re-ranks the shortlist, so if the expert omitted the right answer there is nothing to recover, and the paper's own data show the headroom shrinks to near zero for very strong experts (BioCLIP gains only +0.3).

Editorial extensions

If this is right

  • Any existing few-shot species classifier can be upgraded by wrapping it in POC: the paper reports consistent gains over prompt-learning, adapter-learning, linear-probing, and finetuning-based FSL methods, with the largest gains on the weakest experts.
  • The method pays off most where data are scarcest: the paper's 4-shot and 8-shot settings show average gains of roughly +13 and +11 accuracy points, about double the 16-shot headline gain.
  • Correction quality scales with the LMM: all three LMMs tested improve the expert, and the strongest yields the largest gains, so the method gains value for free as LMMs improve.
  • Softmax confidence scores are the adaptive ingredient: adding them recovers datasets where the LMM alone was worse than the expert, and confidence-aided prompting beats any fixed confidence-thresholding rule.
  • The gain is bounded by the expert's top-k recall: accuracy rises with k and saturates near k=10 in the paper's sweep, and gains shrink to nearly nothing for vastly stronger experts whose top-5 accuracy is already near top-1.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The paper's story implies POC is a general re-ranking layer rather than a species-specific trick: the same shortlist prompt should transfer to other closed-set fine-grained tasks — minerals, artifacts, cultivars — wherever a cheap expert model exists. The paper does not test this.
  • A sharper test of the adaptive-confidence explanation than the paper runs: corrupt the softmax scores and measure the gain that remains, separating the hypothesis that confidence values inform re-ranking from the hypothesis that the LMM ignores numbers and uses only images and names.
  • For strong experts, the binding constraint is candidate recall, not re-ranking skill: the near-zero BioCLIP gain suggests the next step is widening the pool (confusion-neighbor retrieval, or k up to 10 where the paper's own sweep still shows gains rising) before asking the LMM to re-rank.
  • A deployment consideration the paper notes but does not optimize: at roughly 8 GPU-hours per 4,000 images with a 7-billion-parameter LMM, practical use in field ecology would need batched, cached, or distilled re-ranking.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. The paper studies few-shot visual species recognition (VSR) and compares two paradigms: training an FSL expert model (e.g., few-shot finetuning a CLIP encoder) and prompting large multimodal models (LMMs) with open-vocabulary, chain-of-thought, self-verification, or in-context prompts. The authors find that LMMs underperform FSL experts on five VSR benchmarks, but that the correct species is often in the expert's top-k predictions. They propose Post-hoc Correction (POC), a training-free method that prompts an LMM with the expert's top-k species names, their softmax confidence scores, and corresponding few-shot visual examples, asking the LMM to re-rank the candidates. Across five benchmarks, POC improves the Few-shot FT expert by 6.4 accuracy points on average, and consistently improves a range of FSL methods, pretrained backbones, and LMMs. The paper also analyzes confusion matrices, class-level gains, and the effect of k, and releases code.

Significance. If the central claim holds, POC is a practical plug-and-play module for upgrading existing few-shot species classifiers without retraining or validation, which could benefit ecology and biodiversity monitoring. The paper's strengths include a broad evaluation across five benchmarks, multiple FSL baselines, five backbones, and three LMMs, with code released. The consistent gains—especially on weaker FSL methods—are encouraging. However, the external validity of the headline result is threatened by possible LMM training-data contamination, and the claim of being 'validation-free' is weakened by the test-set-based selection of the only hyperparameter k. These issues, together with the absence of a direct comparison to the closely related RAR method, need to be resolved before the result can be fully trusted.

major comments (4)
  1. [Sec. 4.2 / Fig. 9 / Supp. Tab. 7] The paper claims POC is 'validation-free' and 'requires no manual intervention,' yet k=5—the only hyperparameter—is chosen after a test-set sweep. Fig. 9 shows k=10 gives a higher mean accuracy (66.5 vs. 65.6) and higher Aves accuracy (72.4 vs. 69.4). This is load-bearing for the plug-and-play claim. Please fix k a priori, justify k=5 with a non-test criterion, or report k=10 as the primary result.
  2. [Sec. 4.1 / Fig. 5 caption / Supp. Tab. 8] BioCLIP leakage is acknowledged, but no leakage analysis is provided for Qwen, GLM, or GPT-5-Mini, all trained on web-scale data. The benchmarks derive from iNaturalist, Species196, and FungiTastic, which are public web datasets. Low open-vocabulary accuracy does not rule out memorization in the re-ranking setting, where candidates and exemplars are supplied. Please add a leakage probe (e.g., test on a private or novel split, or randomize candidate lists) to show the gains reflect visual comparison, not memorized instances.
  3. [Sec. 2 / Sec. 3.2 / Ref. [46]] The closest prior method, RAR [46], is cited only as inspiration for the re-ranking strategy and is never compared or discussed in related work. Since POC shares the retrieve-then-rank paradigm, the paper should include RAR as a baseline in Tab. 3, or clearly state what POC adds (confidence scores, few-shot exemplars) and demonstrate the increment experimentally. Without this, the novelty and advantage over the closest baseline are not established.
  4. [Tab. 3] Tab. 3 reports single accuracy values for each FSL method and dataset, with no standard deviations or significance tests, despite the claim of consistent improvement. Fig. 5 and Supp. Tab. 8 provide three-seed standard deviations for backbones, but the main comparison across FSL methods lacks them. Please add multiple seeds and confidence intervals, especially for the headline +6.4 average, to support the claim that POC 'significantly improves' all methods.
minor comments (5)
  1. [Title/Abstract] The arXiv metadata title differs from the title on the first page ('Visual Species Recognition with Large Multimodal Models as Post-Hoc Correctors' vs. 'Surely Large Multimodal Models (Don't) Excel in Visual Species Recognition?'). Please align these.
  2. [Contributions] The abstract says '6.4 accuracy points' while the contributions say '6% average accuracy gains.' Use consistent terminology.
  3. [Sec. 3 / typo] Line: 'we review estabilished approaches' — typo for 'established.'
  4. [Tab. 2 / Fungi] POC with top-5 names alone drops Fungi accuracy to 16.2 from the expert's 29.9. A one-sentence explanation of this degradation would help readers understand why the final prompt is needed.
  5. [Sec. 1 / contribution 1] The claim of 'first attempt to study LMMs for VSR' is stated strongly; consider softening given Open-INsect [17] and other recent biodiversity benchmarks.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: POC is an empirical re-ranking pipeline whose inputs are distinct from its outputs; no fitted quantity is renamed as a prediction.

full rationale

The paper makes no formal derivation and fits no parameter to reported accuracies. POC's output is an LMM's re-ranked top-1 from the expert's top-k list, which is not the expert's top-1 by construction; the paper documents changed predictions and reports both expert and POC accuracies. The known ceiling (correct species must be in the expert's top-k) is acknowledged via Fig. 3 and Fig. 9. The ablations select prompt components on test benchmarks, a methodological concern about selection on test data, but not a fitted input called a prediction. The self-citation to [45] supplies the expert recipe, yet the generalization claim is validated across eight FSL methods from other groups, multiple backbones, and three LMMs, so that citation is not load-bearing in the circularity sense. Possible LMM pretraining contamination is an external-validity risk, not a reduction of results to inputs by definition; the paper even discloses BioCLIP leakage. No circular step is exhibited.

Assumptions & free parameters 2 free parameters · 4 assumptions · 0 invented entities

No new entities or formal axioms are introduced; the method is a prompting pipeline built on existing models. The ledger is small because the paper is empirical rather than derivational: the main hidden costs are the hand-chosen k and benchmark-filtering rules plus the top-k-containment assumption.

free parameters (2)
  • k (number of expert top-k predictions in POC prompt) = 5 (default; k=10 yields 66.5 avg vs 65.6 for k=5)
    The only POC hyperparameter; chosen without a validation set, and test-set sweeps (Fig.9) show k=5 is not optimal, so the headline +6.4 may reflect a hand-picked operating point.
  • class sampling filter (>=20 images, no ambiguous names) = 20 images; ambiguous-name exclusion applied to Species196
    Benchmark construction excludes classes with ambiguous names or too few images (Supp. Tab.4), which changes task difficulty; this is a hand-chosen data-selection rule affecting all numbers.
assumptions (4)
  • domain assumption Top-k containment: the correct species is usually inside the expert's top-k predictions
    Stated in Sec. 3.1.3 and shown in Fig.3; if the true label is outside the shortlist, POC cannot succeed.
  • domain assumption Smaller candidate sets make LMM selection more reliable
    Inferred from ZS-ICL gains on Weeds/Mollusca (Tab.1); this motivates POC but is not proven in general.
  • domain assumption Few-shot FT [45] hyperparameters transfer without tuning
    The paper adopts hyperparameters from a same-author paper without validation (Supp. Sec. B); the expert baselines depend on this transfer.
  • domain assumption Web-scale LMM pretraining does not introduce serious benchmark leakage
    No test is reported for overlap between LMM training data and the benchmark images; only BioCLIP leakage is acknowledged (Fig.5 caption).

how reviews work

0 comments
Cite this review

Pith. "Pith review of Visual Species Recognition with Large Multimodal Models as Post-Hoc Correctors." pith.science (2026). https://pith.science/paper/M25IYA4O

@misc{pith2026251215748,
  author       = {Pith},
  title        = {Pith review of: Visual Species Recognition with Large Multimodal Models as Post-Hoc Correctors},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/M25IYA4O}},
  note         = {Machine review of arXiv:2512.15748}
}
read the original abstract

Visual Species Recognition (VSR) is a fundamental task in scientific disciplines that require species-level identification, including ecology, palynology, evolutionary biology, systematics, and phylogenetics. Automating VSR through machine learning can significantly accelerate these efforts. However, species-level annotation requires extensive domain expertise, making large-scale labeled datasets difficult to obtain. Consequently, few-shot learning (FSL) is a practical paradigm, where an expert model is trained using only a few labeled examples. Meanwhile, Large Multimodal Models (LMMs) have demonstrated unprecedented zero-shot visual recognition capabilities, raising the question of whether they can serve as an alternative to FSL expert models for VSR. We start this work with a systematic comparison between FSL expert models and LMMs, revealing that, despite advanced prompting strategies, contemporary LMMs significantly underperform FSL expert models. Interestingly, we find that LMMs possess a complementary strength: given an image and a shortlist of candidate species generated by an expert model, LMMs can often recover the correct label when the expert model's top prediction is incorrect. Motivated by this, we propose Post-hoc Correction (POC), a simple training-free framework that leverages an LMM to post-process an expert model's top predictions. We develop a multimodal prompting strategy to enable POC to improve FSL expert models by 6.4 accuracy points, averaged over five VSR benchmarks. We show that POC generalizes across diverse FSL methods, visual encoders, and LMMs, making it a practical and effective framework for VSR.

Figures

Figures reproduced from arXiv: 2512.15748 by the authors.

Figure 1
Figure 1. Overview of methods on Visual Species Recognition (VSR). We compare the results of (a) Large Multimodal Models (LMMs; e.g., Qwen-2.5-VL-7B-Instruct [9]) under various prompt￾ing strategies [38, 87], and (b) a few-shot learned (FSL) “expert” model obtained by finetuning a Vision-Language Model’s (VLM; e.g., CLIP [64]) visual encoder on few-shot data [45]. Despite being pretrained on web-scale data, LMMs struggle in V… view at source ↗
Figure 2
Figure 2. ). Development Methodology. Considering the high cost of species-level annotation, we follow a realistic FSL setup [45, 69], which provides only a few labeled images per species (e.g., 4, 8, or 16 shots) and eschews a validation set. With few-shot labeled data, one can train an expert model via Top-1 prediction w/ conf. Top-2 prediction w/ conf. Top-3 prediction w/ conf. Aves Insecta Mollusca Weeds Fungi Long-legged… view at source ↗
Figure 3
Figure 3. Top-k accuracies of the FSL expert model. As expected, the top-5 accuracy is substantially higher than the top-1 accuracy, since larger k values naturally yield higher scores. The large gap between top-1 and top-5 metrics indicates that even when the expert model’s top-1 prediction is incorrect, the correct label often appears among the top-5 predictions (see visual examples in [PITH_FULL_IMAGE:figures/full_fig_p00… view at source ↗
Figures from the paper (13 more)
Figure 4
Figure 4. Figure 4: Post-hoc Correction (POC) workflow. POC combines a few-shot learned expert model (e.g., finetuning a VLM’s visual encoder [45]) with an LMM for better VSR. Specifically, for a test image, the expert model predicts the top-k species along with their corresponding softma…
Figure 5
Figure 5. Figure 5: Comparison of mean accuracy averaged across five benchmarks using various pretrained backbones. Following [45], We train an expert model (termed “Few-shot FT”) by fine￾tuning different pretrained visual encoders on 16-shot labeled data sampled with three random seeds, …
Figure 7
Figure 7. Figure 7: Visualization of confusion matrix on the Aves benchmark (200 classes) of (A) FSL expert model [45], where diagonal and off-diagonal dots represent correct and incorrect predictions, respectively. (B) shows the difference between this confusion matrix and the one after …
Figure 8
Figure 8. Figure 8: POC improves on both Aves’s classes, where the few [PITH_FULL_IMAGE:figures/full_fig_p008_8.png]
Figure 12
Figure 12. Figure 12: Example of open-vocabulary prompting with Self￾Verification [87] on Aves benchmark. This prompt keeps the question identical to the open-vocabulary version but asks the LMM to provide a brief “Explanation”. The goal is to encourage LMM for self-verification. However, …
Figure 11
Figure 11. Figure 11: Example of open-vocabulary prompt with Chain-of￾Thought (CoT) [38] on Aves benchmark. This template extends the open-vocabulary setting by requiring the LMM to output its reasoning steps before giving the final prediction. Although CoT prompting has achieved great gai…
Figure 14
Figure 14. Figure 14: Example of our final POC prompt on Aves benchmark. This prompt queries an LMM with the expert model’s top-5 predicted species, along with their softmax confidences and stitched few-shot images. The instructions request the LMM to match the test image against the provi…
Figure 15
Figure 15. Figure 15: Improvement of POC over expert model on binned classes for each benchmark. POC improves on both classes, where the few-shot finetuned expert [45] performs well and those it struggles. The expert model and LMM are the same as in Tab. 5. 18 [PITH_FULL_IMAGE:figures/ful…
Figure 16
Figure 16. Figure 16: Visualization of confusion matrix on the Mollusca benchmark (7 classes) of (A) FSL expert model [45], where diagonal and off-diagonal dots represent correct and incorrect predictions, respectively. (B) shows the difference between this confusion matrix and the one aft…
Figure 17
Figure 17. Figure 17: Visualization of confusion matrix on the Aves benchmark (200 classes) of (A) FSL expert model [45], where diagonal and off-diagonal dots represent correct and incorrect predictions, respectively. (B) shows the difference between this confusion matrix and the one after…
Figure 18
Figure 18. Figure 18: Visualization of confusion matrix on the Insecta benchmark (78 classes) of (A) FSL expert model [45], where diagonal and off-diagonal dots represent correct and incorrect predictions, respectively. (B) shows the difference between this confusion matrix and the one aft…
Figure 19
Figure 19. Figure 19: Visualization of confusion matrix on the Fungi benchmark (196 classes) of (A) FSL expert model [45], where diagonal and off-diagonal dots represent correct and incorrect predictions, respectively. (B) shows the difference between this confusion matrix and the one afte…
Figure 20
Figure 20. Figure 20: Visualization of confusion matrix on the Weeds benchmark (20 classes) of (A) FSL expert model [45], where diagonal and off-diagonal dots represent correct and incorrect predictions, respectively. (B) shows the difference between this confusion matrix and the one after…

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

111 extracted references · 15 linked inside Pith

  1. [46]

    Rar: Retrieving and ranking augmented mllms for visual recognition.arXiv preprint arXiv:2403.13805, 2024

    Ziyu Liu, Zeyi Sun, Yuhang Zang, Wei Li, Pan Zhang, Xi- aoyi Dong, Yuanjun Xiong, Dahua Lin, and Jiaqi Wang. Rar: Retrieving and ranking augmented mllms for visual recognition.arXiv preprint arXiv:2403.13805, 2024. 6

    arXiv 2024

  2. [1]

    Gpt-4 technical report.arXiv preprint arXiv:2303.08774,

    Josh Achiam, Steven Adler, Sandhini Agarwal, Lama Ah- mad, Ilge Akkaya, Florencia Leoni Aleman, Diogo Almeida, Janko Altenschmidt, Sam Altman, Shyamal Anadkat, et al. Gpt-4 technical report.arXiv preprint arXiv:2303.08774,

    arXiv

  3. [2]

    Deep learning approaches to the phylogenetic placement of extinct pollen morphotypes.PNAS nexus, 3(1):pgad419,

    Marc-Élie Adaïmé, Shu Kong, and Surangi W Punyasena. Deep learning approaches to the phylogenetic placement of extinct pollen morphotypes.PNAS nexus, 3(1):pgad419,

  4. [3]

    Pollen morphology, deep learning, phylogenetics, and the evolution of environ- mental adaptations in podocarpus.New Phytologist, 2025

    Marc-Élie Adaimé, Michael A Urban, Shu Kong, Carlos Jaramillo, and Surangi W Punyasena. Pollen morphology, deep learning, phylogenetics, and the evolution of environ- mental adaptations in podocarpus.New Phytologist, 2025. 2

    2025

  5. [4]

    Many-shot in-context learning

    Rishabh Agarwal, Avi Singh, Lei Zhang, Bernd Bohnet, Luis Rosias, Stephanie Chan, Biao Zhang, Ankesh Anand, Zaheer Abbas, Azade Nova, et al. Many-shot in-context learning. Advances in Neural Information Processing Systems, 37: 76930–76966, 2024. 2, 3, 4

    2024

  6. [5]

    Flamingo: a visual language model for few-shot learning

    Jean-Baptiste Alayrac, Jeff Donahue, Pauline Luc, Antoine Miech, Iain Barr, Yana Hasson, Karel Lenc, Arthur Mensch, Katherine Millican, Malcolm Reynolds, et al. Flamingo: a visual language model for few-shot learning. InAdvances in Neural Information Processing Systems (NeurIPS), 2022. 2

    2022

  7. [6]

    Vqa: Visual question answering

    Stanislaw Antol, Aishwarya Agrawal, Jiasen Lu, Margaret Mitchell, Dhruv Batra, C Lawrence Zitnick, and Devi Parikh. Vqa: Visual question answering. InProceedings of the IEEE international conference on computer vision (ICCV), 2015. 2, 3

    2015

  8. [7]

    Bach, Jesse E

    Gonzalo Araujo, Ariana Agustines, Steffen S. Bach, Jesse E. M. Cochran, Emilio De La Parra-Galván, Rafael De La Parra-Venegas, Stella Diamant, Alistair Dove, Steve Fox, Rachel T. Graham, Sofia M. Green, Jonathan R. Green, Royale S. Hardenstine, Alex Hearn, Mahardika R. Hi- mawan, Rhys Hobbs, Jason Holmberg, Ibrahim Shameel, Mohammed Y . Jaidah, Jessica La...

    2022

Show all 111 references
  1. [8]

    Qwen technical report.arXiv preprint arXiv:2309.16609, 2023

    Jinze Bai, Shuai Bai, Yunfei Chu, Zeyu Cui, Kai Dang, Xi- aodong Deng, Yang Fan, Wenbin Ge, Yu Han, Fei Huang, Binyuan Hui, Luo Ji, Mei Li, Junyang Lin, Runji Lin, Day- iheng Liu, Gao Liu, Chengqiang Lu, Keming Lu, Jianxin Ma, Rui Men, Xingzhang Ren, Xuancheng Ren, Chuanqi Tan...

    2023 arXiv

  2. [9]

    Qwen2.5-vl technical report, 2025

    Shuai Bai, Keqin Chen, Xuejing Liu, Jialin Wang, Wenbin Ge, Sibo Song, Kai Dang, Peng Wang, Shijie Wang, Jun Tang, Humen Zhong, Yuanzhi Zhu, Mingkun Yang, Zhaohai Li, Jianqiang Wan, Pengfei Wang, Wei Ding, Zheren Fu, Yiheng Xu, Jiabo Ye, Xi Zhang, Tianbao Xie, Zesen Cheng, Han...

    2025

  3. [10]

    Scaling Biodiversity Monitoring for the Data Age.XRDS: Crossroads, The ACM Magazine for Students, 27(4):14–18, 2021

    Sara Beery. Scaling Biodiversity Monitoring for the Data Age.XRDS: Crossroads, The ACM Magazine for Students, 27(4):14–18, 2021. 1, 2

    2021

  4. [11]

    The iWildCam 2020 competition dataset.arXiv preprint arXiv:2004.10340,

    Sara Beery, Elijah Cole, and Arvi Gjoka. The iWildCam 2020 competition dataset.arXiv preprint arXiv:2004.10340,

    2020 arXiv

  5. [12]

    The iWildCam 2021 competition dataset.arXiv preprint arXiv:2105.03494, 2021

    Sara Beery, Arushi Agarwal, Elijah Cole, and Vighnesh Birodkar. The iWildCam 2021 competition dataset.arXiv preprint arXiv:2105.03494, 2021. 1, 2

    2021 arXiv

  6. [13]

    Deep learning as a tool for ecology and evolution

    Marek L Borowiec, Rebecca B Dikow, Paul B Frandsen, Alexander McKeeken, Gabriele Valentini, and Alexander E White. Deep learning as a tool for ecology and evolution. Methods in Ecology and Evolution, 13(8):1640–1660, 2022. 1

    2022

  7. [14]

    Lan- guage models are few-shot learners

    Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared D Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, et al. Lan- guage models are few-shot learners. InAdvances in neural information processing systems (NeurIPS), 2020. 2, 3, 15

    2020

  8. [15]

    Emerging properties in self-supervised vision transformers

    Mathilde Caron, Hugo Touvron, Ishan Misra, Hervé Jé- gou, Julien Mairal, Piotr Bojanowski, and Armand Joulin. Emerging properties in self-supervised vision transformers. InProceedings of the IEEE International Conference on Computer Vision, pages 9650–9660, 2021. 2

    2021

  9. [16]

    Big self-supervised models are strong semi-supervised learners.Advances in Neural Information Processing Systems (NeurIPS), 2020

    Ting Chen, Simon Kornblith, Kevin Swersky, Mohammad Norouzi, and Geoffrey E Hinton. Big self-supervised models are strong semi-supervised learners.Advances in Neural Information Processing Systems (NeurIPS), 2020. 2 9

    2020

  10. [17]

    Christian Schmidt, Aditya Jain, Yves Basset, Sara Beery, Maxim Larrivée, and David Rol- nick

    Yuyan Chen, Nico Lang, B. Christian Schmidt, Aditya Jain, Yves Basset, Sara Beery, Maxim Larrivée, and David Rol- nick. Open-insect: Benchmarking open-set recognition of novel species in biodiversity monitoring. InThe Thirty- ninth Annual Conference on Neural Information Proce...

    2025

  11. [18]

    Internvl: Scaling up vision foundation models and aligning for generic visual-linguistic tasks

    Zhe Chen, Jiannan Wu, Wenhai Wang, Weijie Su, Guo Chen, Sen Xing, Muyan Zhong, Qinglong Zhang, Xizhou Zhu, Lewei Lu, et al. Internvl: Scaling up vision foundation models and aligning for generic visual-linguistic tasks. In Proceedings of the IEEE/CVF Conference on Computer Vis...

    2024

  12. [19]

    Reproducible scaling laws for contrastive language-image learning

    Mehdi Cherti, Romain Beaumont, Ross Wightman, Mitchell Wortsman, Gabriel Ilharco, Cade Gordon, Christoph Schuh- mann, Ludwig Schmidt, and Jenia Jitsev. Reproducible scaling laws for contrastive language-image learning. In IEEE/CVF Conference on Computer Vision and Pattern Reco...

    2023

  13. [20]

    Imagenet: A large-scale hierarchical image database

    Jia Deng, Wei Dong, Richard Socher, Li-Jia Li, Kai Li, and Li Fei-Fei. Imagenet: A large-scale hierarchical image database. InIEEE/CVF Computer Society Conference on Computer Vision and Pattern Recognition (CVPR), 2009. 7, 14, 18

    2009

  14. [21]

    Discovering localized attributes for fine-grained recog- nition

    Kun Duan, Devi Parikh, David Crandall, and Kristen Grau- man. Discovering localized attributes for fine-grained recog- nition. In2012 IEEE conference on computer vision and pattern recognition, pages 3474–3481. IEEE, 2012. 2

    2012

  15. [22]

    Ezray, Drew C

    Briana D. Ezray, Drew C. Wham, Carrie E. Hill, and Heather M. Hines. Unsupervised machine learning reveals mimicry complexes in bumblebees occur along a perceptual continuum.Proceedings of the Royal Society B: Biological Sciences, 286(1910):20191501, 2019. 2

    1910

  16. [23]

    open world

    Jennifer T Feng, Sandeep Puthanveetil Satheesan, Shu Kong, Timme H Donders, and Surangi W Punyasena. Addressing the “open world”: detecting and segmenting pollen on pa- lynological slides with deep learning.Paleobiology, 51(3): 394–407, 2025. 2

    2025

  17. [24]

    Clip-adapter: Better vision-language models with feature adapters.International Journal of Computer Vision, 132(2): 581–595, 2024

    Peng Gao, Shijie Geng, Renrui Zhang, Teli Ma, Rongyao Fang, Yongfeng Zhang, Hongsheng Li, and Yu Qiao. Clip-adapter: Better vision-language models with feature adapters.International Journal of Computer Vision, 132(2): 581–595, 2024. 2, 4, 6, 19

    2024

  18. [25]

    White, James Balhoff, Wasila M Dahdul, Daniel Rubenstein, Hilmar Lapp, Tanya Berger-Wolf, Wei-Lun Chao, and Yu Su

    Jianyang Gu, Samuel Stevens, Elizabeth G Campolongo, Matthew J Thompson, Net Zhang, Jiaman Wu, Andrei Kopanev, Zheda Mai, Alexander E. White, James Balhoff, Wasila M Dahdul, Daniel Rubenstein, Hilmar Lapp, Tanya Berger-Wolf, Wei-Lun Chao, and Yu Su. BioCLIP 2: Emer- gent prope...

    2025

  19. [26]

    Oskar L. P. Hansen, Jens-Christian Svenning, Kent Olsen, Steen Dupont, Beulah H. Garner, Alexandros Iosifidis, Ben- jamin W. Price, and Toke T. Høye. Species-level image clas- sification with convolutional neural network enables insect identification from habitus images.Ecolog...

    2020

  20. [27]

    Deep residual learning for image recognition

    Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition. InIEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2016. 7, 18

    2016

  21. [28]

    Momentum contrast for unsupervised visual repre- sentation learning

    Kaiming He, Haoqi Fan, Yuxin Wu, Saining Xie, and Ross Girshick. Momentum contrast for unsupervised visual repre- sentation learning. InProceedings of the IEEE/CVF Confer- ence on Computer Vision and Pattern Recognition (CVPR),

  22. [29]

    Species196: A one-million semi-supervised dataset for fine-grained species recognition

    Wei He, Kai Han, Ying Nie, Chengcheng Wang, and Yunhe Wang. Species196: A one-million semi-supervised dataset for fine-grained species recognition. InAdvances in Neural Information Processing Systems, 2024. 6, 14

    2024

  23. [30]

    Distill- ing the knowledge in a neural network.arXiv preprint arXiv:1503.02531, 2015

    Geoffrey Hinton, Oriol Vinyals, and Jeff Dean. Distill- ing the knowledge in a neural network.arXiv preprint arXiv:1503.02531, 2015. 2

    2015 arXiv

  24. [31]

    Deep learning on butterfly phenotypes tests evolution’s oldest mathematical model.Science advances, 5(8):eaaw4967, 2019

    Jennifer F Hoyal Cuthill, Nicholas Guttenberg, Sophie Ledger, Robyn Crowther, and Blanca Huertas. Deep learning on butterfly phenotypes tests evolution’s oldest mathematical model.Science advances, 5(8):eaaw4967, 2019. 2

    2019

  25. [32]

    Deep learning and computer vision will transform entomology

    Toke T Høye, Johanna Ärje, Kim Bjerge, Oskar LP Hansen, Alexandros Iosifidis, Florian Leese, Hjalte MR Mann, Kris- tian Meissner, Claus Melvad, and Jenni Raitoharju. Deep learning and computer vision will transform entomology. Proceedings of the National Academy of Sciences, 1...

    2021

  26. [33]

    Multimodal learning and reasoning for visual question answering

    Ilija Ilievski and Jiashi Feng. Multimodal learning and reasoning for visual question answering. InAdvances in Neural Information Processing Systems (NeurIPS), 2017. 2, 3

    2017

  27. [34]

    Scaling up visual and vision-language representa- tion learning with noisy text supervision

    Chao Jia, Yinfei Yang, Ye Xia, Yi-Ting Chen, Zarana Parekh, Hieu Pham, Quoc Le, Yun-Hsuan Sung, Zhen Li, and Tom Duerig. Scaling up visual and vision-language representa- tion learning with noisy text supervision. InInternational Conference on Machine Learning (ICML), 2021. 2

    2021

  28. [35]

    Vi- sual prompt tuning

    Menglin Jia, Luming Tang, Bor-Chun Chen, Claire Cardie, Serge Belongie, Bharath Hariharan, and Ser-Nam Lim. Vi- sual prompt tuning. InEuropean Conference on Computer Vision (ECCV), 2022. 4

    2022

  29. [36]

    Many-shot in-context learning in multimodal founda- tion models.URL https://arxiv

    Yixing Jiang, Jeremy Irvin, Ji Hun Wang, Muham- mad Ahmed Chaudhry, Jonathan H Chen, and Andrew Y Ng. Many-shot in-context learning in multimodal founda- tion models.URL https://arxiv. org/abs/2405.09798, 2024. 2, 3, 4, 5, 14

    2024 arXiv

  30. [37]

    Maple: Multi-modal prompt learning

    Muhammad Uzair Khattak, Hanoona Rasheed, Muhammad Maaz, Salman Khan, and Fahad Shahbaz Khan. Maple: Multi-modal prompt learning. InIEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2023. 2, 4, 6, 19

    2023

  31. [38]

    Large language models are zero-shot reasoners.Advances in neural information pro- cessing systems (NeurIPS), 2022

    Takeshi Kojima, Shixiang Shane Gu, Machel Reid, Yutaka Matsuo, and Yusuke Iwasawa. Large language models are zero-shot reasoners.Advances in neural information pro- cessing systems (NeurIPS), 2022. 1, 2, 3, 4, 5, 6, 14, 15

    2022

  32. [39]

    Low-rank bilinear pool- ing for fine-grained classification

    Shu Kong and Charless Fowlkes. Low-rank bilinear pool- ing for fine-grained classification. InProceedings of the IEEE conference on computer vision and pattern recogni- tion, pages 365–374, 2017. 2

    2017

  33. [40]

    Grounded language-image pre-training

    Liunian Harold Li, Pengchuan Zhang, Haotian Zhang, Jian- wei Yang, Chunyuan Li, Yiwu Zhong, Lijuan Wang, Lu 10 Yuan, Lei Zhang, Jenq-Neng Hwang, et al. Grounded language-image pre-training. InProceedings of the IEEE/CVF conference on computer vision and pattern recog- nition (...

    2022

  34. [41]

    Bilinear cnn models for fine-grained visual recognition

    Tsung-Yu Lin, Aruni RoyChowdhury, and Subhransu Maji. Bilinear cnn models for fine-grained visual recognition. In Proceedings of the IEEE international conference on com- puter vision, pages 1449–1457, 2015. 2

    2015

  35. [42]

    Multimodality helps unimodality: Cross- modal few-shot learning with multimodal models

    Zhiqiu Lin, Samuel Yu, Zhiyi Kuang, Deepak Pathak, and Deva Ramanan. Multimodality helps unimodality: Cross- modal few-shot learning with multimodal models. In IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2023. 1, 2, 4, 6, 19

    2023

  36. [43]

    Visual instruction tuning.Advances in neural information processing systems (NeurIPS), 2023

    Haotian Liu, Chunyuan Li, Qingyang Wu, and Yong Jae Lee. Visual instruction tuning.Advances in neural information processing systems (NeurIPS), 2023. 2, 3

    2023

  37. [44]

    Democratizing fine-grained visual recognition with large language models

    Mingxuan Liu, Subhankar Roy, Wenjing Li, Zhun Zhong, Nicu Sebe, and Elisa Ricci. Democratizing fine-grained visual recognition with large language models. InThe Twelfth International Conference on Learning Represen- tations (ICLR), 2024. 3, 6, 7, 18, 19

    2024

  38. [45]

    Few-shot recognition via stage-wise retrieval-augmented finetuning

    Tian Liu, Huixin Zhang, Shubham Parashar, and Shu Kong. Few-shot recognition via stage-wise retrieval-augmented finetuning. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2025. 1, 2, 3, 4, 5, 6, 7, 8, 14, 15, 17, 18, 19, 20, 21

    2025

  39. [47]

    Rsvp: Reasoning segmentation via visual prompting and multi-modal chain-of-thought

    Yi Lu, Jiawang Cao, Yongliang Wu, Bozheng Li, Licheng Tang, Yangguang Ji, Chong Wu, Jay Wu, and Wenbo Zhu. Rsvp: Reasoning segmentation via visual prompting and multi-modal chain-of-thought. InProceedings of the 63th Annual Meeting of the Association for Computational Lin- gui...

    2025

  40. [48]

    Computer vision, machine learning, and the promise of phenomics in ecology and evo- lutionary biology.Frontiers in Ecology and Evolution, 9: 642774, 2021

    Moritz D Lürig, Seth Donoughe, Erik I Svensson, Arthur Porto, and Masahito Tsuboi. Computer vision, machine learning, and the promise of phenomics in ecology and evo- lutionary biology.Frontiers in Ecology and Evolution, 9: 642774, 2021. 1

    2021

  41. [49]

    Lessons learned from a unifying empirical study of parameter-efficient transfer learning (petl) in visual recognition

    Zheda Mai, Ping Zhang, Cheng-Hao Tu, Hong-You Chen, Li Zhang, and Wei-Lun Chao. Lessons learned from a unifying empirical study of parameter-efficient transfer learning (petl) in visual recognition. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recogn...

    2025

  42. [50]

    En- hancing clip with gpt-4: Harnessing visual descriptions as prompts

    Mayug Maniparambil, Chris V orster, Derek Molloy, Noel Murphy, Kevin McGuinness, and Noel E O’Connor. En- hancing clip with gpt-4: Harnessing visual descriptions as prompts. InProceedings of the IEEE/CVF international conference on computer vision (CVPR), 2023. 3

    2023

  43. [51]

    Visual classification via description from large language models

    Sachit Menon and Carl V ondrick. Visual classification via description from large language models. InInternational Conference on Learning Representations (ICLR), 2022. 2, 3, 6

    2022

  44. [52]

    Re- thinking the role of demonstrations: What makes in-context learning work? InEMNLP, 2022

    Sewon Min, Xinxi Lyu, Ari Holtzman, Mikel Artetxe, Mike Lewis, Hannaneh Hajishirzi, and Luke Zettlemoyer. Re- thinking the role of demonstrations: What makes in-context learning work? InEMNLP, 2022. 2, 3

    2022

  45. [53]

    Compositional chain of thought prompting for large multimodal models

    Chancharik Mitra, Brandon Huang, Trevor Darrell, and Roei Herzig. Compositional chain of thought prompting for large multimodal models. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2024. 3

    2024

  46. [54]

    Norman, Jason A

    Bradley M. Norman, Jason A. Holmberg, Zaven Arzouma- nian, Samantha D. Reynolds, Rory P. Wilson, Dani Rob, Simon J. Pierce, Adrian C. Gleiss, Rafael De La Parra, Beatriz Galvan, Deni Ramirez-Macias, David Robinson, Steve Fox, Rachel Graham, David Rowat, Matthew Poten- ski, Mar...

    2017

  47. [55]

    Inaturalist.Science Scope, 41(7):12–13, 2018

    Jill Nugent. Inaturalist.Science Scope, 41(7):12–13, 2018. 2

    2018

  48. [56]

    Gpt-5 system card

    OpenAI. Gpt-5 system card. Technical report, OpenAI,

  49. [57]

    DINOv2: learning robust visual features without supervision.arXiv preprint 2304.07193, 2023

    Maxime Oquab, Timothée Darcet, Théo Moutakanni, Huy V o, Marc Szafraniec, Vasil Khalidov, Pierre Fernandez, Daniel Haziza, Francisco Massa, Alaaeldin El-Nouby, Mah- moud Assran, Nicolas Ballas, Wojciech Galuba, Russell Howes, Po-Yao Huang, Shang-Wen Li, Ishan Misra, Michael Ra...

    2023 arXiv

  50. [58]

    Prompting scientific names for zero-shot species recognition

    Shubham Parashar, Zhiqiu Lin, Yanan Li, and Shu Kong. Prompting scientific names for zero-shot species recognition. InConference on Empirical Methods in Natural Language Processing (EMNLP), 2023. 2, 3, 4, 6

    2023

  51. [59]

    The neglected tails of vision-language models

    Shubham Parashar, Zhiqiu Lin, Tian Liu, Xiangjue Dong, Yanan Li, Deva Ramanan, James Caverlee, and Shu Kong. The neglected tails of vision-language models. InIEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2024. 1, 2, 3, 4, 14

    2024

  52. [60]

    Fungitastic: A multi-modal dataset and benchmark for image categorization

    Lukas Picek, Klara Janouskova, V ojtech Cermak, and Jiri Matas. Fungitastic: A multi-modal dataset and benchmark for image categorization. In2025 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW), pages 2037–2047. IEEE, 2025. 2, 6, 14

  53. [61]

    Catarina Pinho, Antigoni Kaliontzopoulou, Carlos A Fer- reira, and João Gama. Identification of morphologically cryptic species with computer vision models: wall lizards (Squamata: Lacertidae: Podarcis) as a case study.Zoologi- cal Journal of the Linnean Society, 198(1):184–20...

    2022

  54. [62]

    What does a platypus look like? generating customized prompts 11 for zero-shot image classification

    Sarah Pratt, Ian Covert, Rosanne Liu, and Ali Farhadi. What does a platypus look like? generating customized prompts 11 for zero-shot image classification. InProceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), 2023. 3

    2023

  55. [63]

    Automated identifi- cation of diverse neotropical pollen samples using convolu- tional neural networks.Methods in Ecology and Evolution, 13(9):2049–2064, 2022

    Surangi W Punyasena, Derek S Haselhorst, Shu Kong, Char- less C Fowlkes, and J Enrique Moreno. Automated identifi- cation of diverse neotropical pollen samples using convolu- tional neural networks.Methods in Ecology and Evolution, 13(9):2049–2064, 2022. 2

  56. [64]

    Learn- ing transferable visual models from natural language super- vision

    Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, et al. Learn- ing transferable visual models from natural language super- vision. InInternational Conference on Machine Learning ...

    2021

  57. [65]

    Ingrid C Romero, Shu Kong, Charless C Fowlkes, Carlos Jaramillo, Michael A Urban, Francisca Oboh-Ikuenobe, Car- los D’apolito, and Surangi W Punyasena. Improving the taxonomy of fossil pollen using convolutional neural net- works and superresolution microscopy.Proceedings of t...

    2020

  58. [66]

    Im- proved zero-shot classification by adapting vlms with text descriptions

    Oindrila Saha, Grant Van Horn, and Subhransu Maji. Im- proved zero-shot classification by adapting vlms with text descriptions. InIEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2024. 3, 6

    2024

  59. [67]

    Laion-400m: Open dataset of clip-filtered 400 million image-text pairs

    Christoph Schuhmann, Richard Vencu, Romain Beaumont, Robert Kaczmarczyk, Clayton Mullis, Aarush Katta, Theo Coombes, Jenia Jitsev, and Aran Komatsuzaki. Laion-400m: Open dataset of clip-filtered 400 million image-text pairs. arXiv:2111.02114, 2021. 7, 18

    2021 arXiv

  60. [68]

    Conceptual captions: A cleaned, hypernymed, im- age alt-text dataset for automatic image captioning

    Piyush Sharma, Nan Ding, Sebastian Goodman, and Radu Soricut. Conceptual captions: A cleaned, hypernymed, im- age alt-text dataset for automatic image captioning. InPro- ceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers...

    2018

  61. [69]

    A closer look at the few-shot adaptation of large vision-language models

    Julio Silva-Rodriguez, Sina Hajimiri, Ismail Ben Ayed, and Jose Dolz. A closer look at the few-shot adaptation of large vision-language models. InIEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2024. 1, 2, 3, 4, 6, 19

    2024

  62. [70]

    Scaling-up camera traps: monitoring the planet’s biodi- versity with networks of remote sensors.Frontiers in Ecology and the Environment, 15(1):26–34, 2017

    Robin Steenweg, Mark Hebblewhite, Roland Kays, Jorge Ahumada, Jason T Fisher, Cole Burton, Susan E Townsend, Chris Carbone, J Marcus Rowcliffe, Jesse Whittington, Jedediah Brodie, J Andrew Royle, Adam Switalski, An- thony P Clevenger, Nicole Heim, and Lindsey N Rich. Scaling-u...

    2017

  63. [71]

    BioCLIP: A vision foundation model for the tree of life

    Samuel Stevens, Jiaman Wu, Matthew J Thompson, Eliza- beth G Campolongo, Chan Hee Song, David Edward Car- lyn, Li Dong, Wasila M Dahdul, Charles Stewart, Tanya Berger-Wolf, Wei-Lun Chao, and Yu Su. BioCLIP: A vision foundation model for the tree of life. InProceedings of the I...

    2024

  64. [72]

    The semi- supervised inaturalist-aves challenge at fgvc7 workshop

    Jong-Chyi Su and Subhransu Maji. The semi- supervised inaturalist-aves challenge at fgvc7 workshop. arXiv:2103.06937, 2021. 6, 14

    2021 arXiv

  65. [73]

    A real- istic evaluation of semi-supervised learning for fine-grained classification

    Jong-Chyi Su, Zezhou Cheng, and Subhransu Maji. A real- istic evaluation of semi-supervised learning for fine-grained classification. InIEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2021. 1, 2, 7, 14, 18

    2021

  66. [74]

    Gemini 2.5: Pushing the frontier with ad- vanced reasoning, multimodality, long context, and next generation agentic capabilities, 2025

    Gemini Team. Gemini 2.5: Pushing the frontier with ad- vanced reasoning, multimodality, long context, and next generation agentic capabilities, 2025. 4

    2025

  67. [75]

    Gemini: a family of highly capable multimodal models.arXiv preprint arXiv:2312.11805, 2023

    Gemini Team, Rohan Anil, Sebastian Borgeaud, Jean- Baptiste Alayrac, Jiahui Yu, Radu Soricut, Johan Schalkwyk, Andrew M Dai, Anja Hauth, Katie Millican, et al. Gemini: a family of highly capable multimodal models.arXiv preprint arXiv:2312.11805, 2023. 2, 3, 4

    2023 arXiv

  68. [76]

    Glm-4.5v and glm-4.1v-thinking: Towards ver- satile multimodal reasoning with scalable reinforcement learning, 2025

    V Team. Glm-4.5v and glm-4.1v-thinking: Towards ver- satile multimodal reasoning with scalable reinforcement learning, 2025. 4, 7

    2025

  69. [77]

    Bird distribution modelling using remote sensing and citizen science data.arXiv preprint arXiv:2305.01079, 2023

    Mélisande Teng, Amna Elmustafa, Benjamin Akera, Hugo Larochelle, and David Rolnick. Bird distribution modelling using remote sensing and citizen science data.arXiv preprint arXiv:2305.01079, 2023. 1, 2

    2023 arXiv

  70. [78]

    Eyes wide shut? exploring the visual shortcomings of multimodal llms

    Shengbang Tong, Zhuang Liu, Yuexiang Zhai, Yi Ma, Yann LeCun, and Saining Xie. Eyes wide shut? exploring the visual shortcomings of multimodal llms. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2024. 7

    2024

  71. [79]

    Llama: Open and efficient foundation language models

    Hugo Touvron, Thibaut Lavril, Gautier Izacard, Xavier Mar- tinet, Marie-Anne Lachaux, Timothée Lacroix, Baptiste Rozière, Naman Goyal, Eric Hambro, Faisal Azhar, et al. Llama: Open and efficient foundation language models. arXiv preprint arXiv:2302.13971, 2023. 3

    2023 arXiv

  72. [80]

    Perspectives in machine learning for wildlife conservation.Nature communications, 13(1):792,

    Devis Tuia, Benjamin Kellenberger, Sara Beery, Blair R Costelloe, Silvia Zuffi, Benjamin Risse, Alexander Mathis, Mackenzie W Mathis, Frank van Langevelde, Tilo Burghardt, et al. Perspectives in machine learning for wildlife conservation.Nature communications, 13(1):792,

  73. [81]

    iNat Challenge 2021 - FGVC8, 2021

    Grant Van Horn and Oisin Mac Aodha. iNat Challenge 2021 - FGVC8, 2021. 3, 7, 14, 17, 18

    2021

  74. [82]

    The inaturalist species classification and detection dataset

    Grant Van Horn, Oisin Mac Aodha, Yang Song, Yin Cui, Chen Sun, Alex Shepard, Hartwig Adam, Pietro Perona, and Serge Belongie. The inaturalist species classification and detection dataset. InIEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2018. 6, 7

    2018

  75. [83]

    Enabling val- idation for robust few-shot recognition.arXiv preprint arXiv:2506.04713, 2025

    Hanxin Wang, Tian Liu, and Shu Kong. Enabling val- idation for robust few-shot recognition.arXiv preprint arXiv:2506.04713, 2025. 1

    2025

  76. [84]

    Internvl3.5: Advancing open-source multimodal models in versatility, reasoning, and efficiency

    Weiyun Wang, Zhangwei Gao, Lixin Gu, Hengjun Pu, Long Cui, Xingguang Wei, Zhaoyang Liu, Linglin Jing, Sheng- long Ye, Jie Shao, et al. Internvl3.5: Advancing open-source multimodal models in versatility, reasoning, and efficiency. arXiv preprint arXiv:2508.18265, 2025. 3, 4

    2025 arXiv

  77. [85]

    Self-consistency improves chain of thought reason- ing in language models

    Xuezhi Wang, Jason Wei, Dale Schuurmans, Quoc Le, Ed Chi, Sharan Narang, Aakanksha Chowdhery, and Denny Zhou. Self-consistency improves chain of thought reason- ing in language models. InInternational Conference on Learning Representations (ICLR), 2023. 3 12

    2023

  78. [86]

    Chain-of-thought prompting elicits reasoning in large lan- guage models

    Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Fei Xia, Ed Chi, Quoc V Le, Denny Zhou, et al. Chain-of-thought prompting elicits reasoning in large lan- guage models. InAdvances in neural information processing systems (NeurIPS), 2022. 3

    2022

  79. [87]

    Large language models are better reasoners with self-verification

    Yixuan Weng, Minjun Zhu, Fei Xia, Bin Li, Shizhu He, Shengping Liu, Bin Sun, Kang Liu, and Jun Zhao. Large language models are better reasoners with self-verification. InThe 2023 Conference on Empirical Methods in Natural Language Processing (EMNLP), 2023. 1, 2, 4, 5, 14, 15

    2023

  80. [88]

    Robust fine-tuning of zero-shot models

    Mitchell Wortsman, Gabriel Ilharco, Jong Wook Kim, Mike Li, Simon Kornblith, Rebecca Roelofs, Raphael Gon- tijo Lopes, Hannaneh Hajishirzi, Ali Farhadi, Hongseok Namkoong, and Ludwig Schmidt. Robust fine-tuning of zero-shot models. InIEEE/CVF Conference on Computer Vision and ...

    2022

  81. [89]

    Pro- tect: Prompt tuning for taxonomic open set classification

    Tz-Ying Wu, Chih-Hui Ho, and Nuno Vasconcelos. Pro- tect: Prompt tuning for taxonomic open set classification. InIEEE/CVF Conference on Computer Vision and Pattern Recognition, 2024. 6

    2024

  82. [90]

    Demysti- fying CLIP data

    Hu Xu, Saining Xie, Xiaoqing Ellen Tan, Po-Yao Huang, Russell Howes, Vasu Sharma, Shang-Wen Li, Gargi Ghosh, Luke Zettlemoyer, and Christoph Feichtenhofer. Demysti- fying CLIP data. InInternational Conference on Learning Representations (ICLR), 2024. 2

    2024

  83. [91]

    Learning concise and descriptive attributes for visual recognition

    An Yan, Yu Wang, Yiwu Zhong, Chengyu Dong, Zexue He, Yujie Lu, William Yang Wang, Jingbo Shang, and Ju- lian McAuley. Learning concise and descriptive attributes for visual recognition. InProceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), 2023. 3

    2023

  84. [92]

    Qwen2.5 technical report.arXiv preprint arXiv:2412.15115, 2024

    An Yang, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chengyuan Li, Dayiheng Liu, Fei Huang, Haoran Wei, Huan Lin, Jian Yang, Jianhong Tu, Jianwei Zhang, Jianxin Yang, Jiaxi Yang, Jingren Zhou, Junyang Lin, Kai Dang, Keming Lu, Keqin Bao, Kexin Yang, Le Yu, Me...

    2024 arXiv

  85. [93]

    An empirical study of gpt-3 for few-shot knowledge-based vqa

    Zhengyuan Yang, Zhe Gan, Jianfeng Wang, Xiaowei Hu, Yumao Lu, Zicheng Liu, and Lijuan Wang. An empirical study of gpt-3 for few-shot knowledge-based vqa. InPro- ceedings of the AAAI conference on artificial intelligence (AAAI), 2022. 3

    2022

  86. [94]

    Visual- language prompt tuning with knowledge-guided context op- timization

    Hantao Yao, Rui Zhang, and Changsheng Xu. Visual- language prompt tuning with knowledge-guided context op- timization. InIEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2023. 2

    2023

  87. [95]

    Image captioning with semantic attention

    Quanzeng You, Hailin Jin, Zhaowen Wang, Chen Fang, and Jiebo Luo. Image captioning with semantic attention. In Proceedings of the IEEE conference on computer vision and pattern recognition (CVPR), 2016. 2, 3

    2016

  88. [96]

    Task residual for tuning vision-language models

    Tao Yu, Zhihe Lu, Xin Jin, Zhibo Chen, and Xinchao Wang. Task residual for tuning vision-language models. InIEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2023. 2

    2023

  89. [97]

    Part-based r-cnns for fine-grained category detection

    Ning Zhang, Jeff Donahue, Ross Girshick, and Trevor Dar- rell. Part-based r-cnns for fine-grained category detection. InEuropean conference on computer vision, pages 834–849. Springer, 2014. 2

    2014

  90. [98]

    Revisiting semi-supervised learning in the era of foundation models.Advances in Neural Information Processing Systems (NeurIPS), 2025

    Ping Zhang, Zheda Mai, Quang-Huy Nguyen, and Wei- Lun Chao. Revisiting semi-supervised learning in the era of foundation models.Advances in Neural Information Processing Systems (NeurIPS), 2025. 2, 7

    2025

  91. [99]

    Tip- adapter: Training-free adaption of clip for few-shot clas- sification

    Renrui Zhang, Rongyao Fang, Peng Gao, Wei Zhang, Kun- chang Li, Jifeng Dai, Yu Qiao, and Hongsheng Li. Tip- adapter: Training-free adaption of clip for few-shot clas- sification. InEuropean Conference on Computer Vision (ECCV), 2022. 2, 4, 6, 19

    2022

  92. [100]

    What makes good examples for visual in-context learning?Advances in Neural Information Processing Systems (NeurIPS), 2023

    Yuanhan Zhang, Kaiyang Zhou, and Ziwei Liu. What makes good examples for visual in-context learning?Advances in Neural Information Processing Systems (NeurIPS), 2023. 2, 3

    2023

  93. [101]

    Finer-cam: Fine-grained visual interpretability through class-specific gradient refinements

    Ziheng Zhang, Jianyang Gu, Arpita Chowdhury, Zheda Mai, David Carlyn, Tanya Berger-Wolf, Yu Su, and Wei- Lun Chao. Finer-cam: Fine-grained visual interpretability through class-specific gradient refinements. InProceedings of the IEEE/CVF Conference on Computer Vision and Pat- ...

    2025

  94. [102]

    Conditional prompt learning for vision-language models

    Kaiyang Zhou, Jingkang Yang, Chen Change Loy, and Zi- wei Liu. Conditional prompt learning for vision-language models. InIEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2022. 4

    2022

  95. [103]

    Learning to prompt for vision-language models.In- ternational Journal of Computer Vision, 130(9):2337–2348,

    Kaiyang Zhou, Jingkang Yang, Chen Change Loy, and Ziwei Liu. Learning to prompt for vision-language models.In- ternational Journal of Computer Vision, 130(9):2337–2348,

  96. [106]

    Semipalmated Plover (Charadrius semipalmatus)

  97. [107]

    Violet-green Swallow (Tachycineta thalassina)

  98. [108]

    Northern Rough-winged Swallow (Stelgidopteryx serripennis)

  99. [111]

    We then run POC with different LMMs, including the open-sourced ones like GLM-4.1V-9B-Thinking (GLM), Qwen-2.5- VL-7B (Qwen), and the closed-source GPT-5-Mini (GPT)

    ViT-B/32 model’s visual encoder on 16-shot labeled data from each dataset. We then run POC with different LMMs, including the open-sourced ones like GLM-4.1V-9B-Thinking (GLM), Qwen-2.5- VL-7B (Qwen), and the closed-source GPT-5-Mini (GPT). Results show that all of these LMMs ...

  100. [198]

    Eastern Whip-poor-will (Antrostomus vociferus)

  101. [199]

    All w/ Conf

    Magellanic Penguin (Spheniscus magellanicus) Respond strictly in the following format with no additional text: Most Likely: [Common Name (Scientific name)] Example of ICL Prompt with All Classes on Aves Benchmark Figure 13.Example of Zero-shot In-Context Learning (ZS- ICL) Pro...

  102. [2022]

    Steps:" section, then the single

    2, 4, 6, 19 13 Surely Large Multimodal Models (Don’t) Excel in Visual Species Recognition? Supplementary Material Outline This document supports our main paper with detailed results and comprehensive analyses. The document is organized as follows: • Section Aprovides a detaile...

  103. [2025]

    Version published on August 13, 2025. 3, 4, 7

    2025

Pith tools

Reviewed August 3, 2026 · model on record in the stance chip above.