Pith. sign in

REVIEW 4 major objections 4 minor 77 references

EvoVLMA: Evolutionary Vision-Language Model Adaptation

T0 review · 4 major / 4 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read An LLM-driven evolutionary search over Python code automatically designs training-free adaptation algorithms for vision-language models, improving on the human-designed Tip-Adapter, APE, and GDA baselines across few-shot classification…

desk verdict Honest, well-executed application of LLM-based code evolution to training-free VLM adaptation; the main weaknesses are missing error bars and an under-tested holdout composition, not the idea. read the letter →

arxiv 2508.01558 v1 pith:YEYDIQTO submitted 2025-08-03 cs.CV

Kun Ding , Ying Wang , Shiming Xiang This is my paper
classification cs.CV
keywords vision-languagemodelsfew-shotclassificationtraining-freeadaptationcodegenerationevolutionaryalgorithmfeatureselectionCLIP
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

EvoVLMA tries to show that adapting a pre-trained vision-language model to a new task can itself be automated: rather than a human expert deciding which feature channels matter and how to turn similarities into class scores, a large language model writes candidate Python functions and an evolutionary loop keeps the ones that score best on holdout data. Training-free here means the adapted model is not fine-tuned with gradients. Starting from the public implementations of three known training-free methods, Tip-Adapter, APE, and GDA, the search improves on all of them on the eleven few-shot classification datasets, with the largest single gain being +1.91 accuracy points over APE at 8-shot. The paper also reports that the discovered algorithms transfer to other CLIP visual backbones and to domain generalization, and that the population-based search beats both joint search and single-solution iterative refinement. A sympathetic reader would take the central claim to be that LLM-based code evolution can replace part of the human design loop for training-free model adaptation.

What carries the argument

The mechanism that carries the argument is a two-stage, LLM-assisted evolutionary loop over algorithm code. Each individual is a pair of natural-language thoughts plus a Python implementation of either `feat_selection` or `compute_logits`; crossover and mutation are prompts that ask the LLM to combine or modify one or two parent algorithms. The fitness function is classification accuracy on five holdout datasets, using the best hyper-parameter setting from a small grid, which lets the search compare candidates without touching the downstream test sets. Two engineering pieces make the loop practical: a lookup-table conversion that rewrites operations such as matrix inverse to run in half precision, and web-service-based execution with a process monitor that restarts GPU workers when generated code crashes. Together these turn an open-ended code search into a stable, parallelizable optimization loop.

What would settle it

Run the same search a second time with a different holdout set of five datasets chosen to be distributionally far from the eleven downstream benchmarks, then test the evolved algorithm on the same eleven benchmarks; if the gain over APE or GDA disappears or turns negative, the holdout fitness function is the load-bearing assumption. A quicker check is to take the published searched code and evaluate it on held-out datasets that were used for neither search nor downstream validation, such as medical or satellite image sets not in the paper's list.

Watch

Extended reading notes

Core claim

On its own terms, the paper claims that training-free VLM adaptation reduces to two functions: feature selection, which chooses which channels of the frozen image and text features to keep, and logits computation, which turns similarities between test, train, and class-text features into class scores. EvoVLMA initializes a population with existing code, then repeatedly asks an LLM to produce variants through crossover and mutation, evaluates each candidate by running it on holdout datasets at 1, 2, 4, 8, and 16 shots and taking the best accuracy over a small hyper-parameter grid, and keeps the top-ranked algorithms. The two stages are run sequentially, one function fixed while the other evolves, which the paper says works better than trying to evolve both together. In the reported experiments, the evolved algorithms outperform their manual starting points in most few-shot settings and on average, the gains carry over to domain generalization, and the found feature-selection code runs much faster than the APE selector it replaces.

Load-bearing premise

The load-bearing premise is that the accuracy measured on the five holdout datasets (CIFAR-100, Fashion-MNIST, ObjectNet, UcMerced, and UCSD-Birds) at 1, 2, 4, 8, and 16 shots predicts accuracy on the eleven downstream datasets; if the holdout selection is not representative, evolution will pick algorithms that fit the holdouts and the reported gains will not transfer.

Editorial extensions

If this is right

  • If the central claim holds, training-free VLM adaptation no longer has to start from a human-designed rule; seeding an LLM-based search with existing code yields improved algorithms automatically.
  • The reported +1.91 point gain over APE at 8-shot is not an isolated number: average few-shot accuracy improves over all three initializations, and most shot counts improve as well.
  • The discovered algorithms transfer across visual backbones (ResNet-101 and ViT-B/16 in the paper), suggesting the search finds feature-selection and logits patterns that are not tied to one feature distribution.
  • Because the searched feature-selection code is faster than APE's hand-written version, the accuracy gain does not come at the price of slower inference.
  • Two-stage sequential search is the right decomposition: the paper reports it beats joint searching and iterative single-solution refinement at comparable token cost.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Beyond the paper's claims, the same search loop could be pointed at other components of training-free adaptation, such as cache weighting rules, text-side feature transformations, or fusion coefficients, once they are isolated as functions with a measurable fitness.
  • A natural extension is to put backbone identity or dataset family inside the fitness function, so the search is explicitly optimized to produce code that transfers rather than code that transfers only as a side effect.
  • The paper's error-rate table implies the method's ceiling depends strongly on code-execution reliability, which suggests that pairing the loop with cheaper or more reliable code generators is a direct way to scale the search.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 4 minor

Summary. The paper proposes EvoVLMA, an evolutionary LLM-based method that searches over Python code implementations of two components of training-free VLM adaptation: feature selection and logits computation. Starting from manually designed algorithms (Tip-Adapter, APE, GDA), EvoVLMA uses LLM-driven crossover and mutation, low-precision code conversion, and fault-tolerant web-based code execution, with a fitness function based on accuracy on five holdout datasets. A two-stage strategy searches the two algorithmic components sequentially. On 11 downstream few-shot classification datasets and two domain generalization targets, the searched algorithms improve over the manual baselines in most configurations, with the headline result of +1.91 accuracy points over APE at 8-shot. The paper also reports ablations on search strategy, initialization, LLM choice, number of holdout datasets, iteration count, backbone transfer, and computational cost.

Significance. If the results hold up, EvoVLMA is a valuable demonstration of LLM-assisted automatic algorithm design for a concrete CV problem. The two-stage decomposition of the code search and the practical engineering contributions (process monitoring, fp16 code conversion, parallel web-based evaluation) are useful and likely reusable. The paper is honest about the limitations of current LLMs and provides code. However, the reported gains are modest and uneven: they are largest for APE initialization (average +1.25/+1.13), small for GDA (average +0.38/+0.04), and some per-shot results are negative. The main scientific value is the framework rather than any single discovered algorithm.

major comments (4)
  1. [Sec. 4.3, Table 6] The proxy validity of the fitness function is not established. The only holdout ablation, Table 6, varies the number of holdout datasets (C+F vs C+F+O+M+B), not the composition of the holdout set. Consequently, the experiments cannot detect overfitting to the particular choice of CIFAR100, FashionMNIST, ObjectNet, UcMerced, and UCSDBirds. I request leave-one-out or random-subset experiments over the five holdout datasets, reporting the downstream accuracy of the algorithms selected under each composition. Additionally, the GDA rows in Table 6 appear inconsistent with Table 13: the C+F GDA entries (69.98 and 70.05) disagree with the corresponding C+F per-shot averages in Table 13 (70.17 and 69.83), and the corrected values would make the C+F GDA results identical to the C+F+O+M+B results. This suggests either a transcription error or that the GDA search was not rerun under the two-dataset condition; the authors should clarify.
  2. [Sec. 4.2, Table 2] The evolutionary search is stochastic (LLM sampling, random parent selection), but every configuration is searched only once. The statement in Sec. 4.1 that 'the average results over three seeds are reported' applies to downstream few-shot data sampling, not to independent runs of the evolutionary search. Without repeated searches, the headline gain of +1.91 points (APE logit→fs, 8-shot) could be within run-to-run noise. I ask the authors to report results from at least three independent search runs for the main configurations (or at minimum for the APE initialization), giving mean and standard deviation or min/max of downstream accuracy, and to state whether the reported algorithms are the best run or a representative run.
  3. [Sec. 3.4, Eq. (1)] The fitness equation is written as `fitness = 1 - arg max_{θ∈S} Acc(...)`, which is not mathematically meaningful because `arg max` returns a parameter vector, not a scalar. It should presumably be `1 - max_{θ∈S} Acc(...)`. Please correct this. More substantively, taking the max over θ on the holdout test set makes the fitness an optimistic estimate and can select algorithms that fit idiosyncrasies of those particular test sets. The coarse grid S = {0.5}^2 x {0.7d} x {0.1,1,10}^3 mitigates but does not eliminate this risk; the paper should discuss this explicitly and provide evidence that the selected algorithms are stable under changes to S.
  4. [Table 2, Sec. 4.2] No error bars, confidence intervals, or significance tests are reported for the downstream accuracy differences. Many of the improvements are small; for example, GDA fs→logit at 8-shot is +0.27 and at 16-shot is -0.02, TIP fs→logit at 1-shot is +0.23, and several GDA per-shot results are negative. Without a measure of variability across the three seeds that are averaged, it is difficult to conclude that these small differences are meaningful rather than noise. Please report per-seed results or standard deviations for at least the main few-shot comparisons.
minor comments (4)
  1. [Sec. 4.1] The description of fitness evaluation says accuracy is averaged over holdout datasets and shots, but it is not stated whether the few-shot train/test splits are fixed across generations of the search or resampled; if they are resampled, the fitness values will contain additional noise that should be discussed.
  2. [Table 3] The column heading 'iteration' is ambiguous; the table actually compares population-based evolutionary search with individual-based iterative refinement. Please rename the column to something like 'Search method'.
  3. [Figure 4] The backbone transfer curves would be more informative if they included error bars, as the differences between methods are small in some shot regimes.
  4. [Table 2 caption] The caption says 'The bold numbers represent the accuracy gap', but in the table the bold numbers are the differences (positive values), while negative differences are not bold. Please clarify the formatting.

Circularity Check

0 steps flagged · score 2.0 of 10

No significant circularity: searched algorithms are validated on downstream datasets disjoint from the holdout fitness sets, against external baselines with symmetric Optuna tuning; the few self-citations are non-load-bearing.

full rationale

EvoVLMA's derivation chain is self-contained and not circular. The fitness function (Sec. 3.4) selects algorithms by accuracy on five holdout datasets (CIFAR100, FashionMNIST, ObjectNet, UcMerced, UCSDBirds), while the central claim — that found algorithms outperform manual ones by up to +1.91 points (Table 2) — is evaluated on eleven disjoint downstream datasets plus ImageNetV2/ImageNet-Sketch, whose test labels never enter the search. Both manual and searched algorithms receive the same Optuna hyper-parameter tuning on downstream validation sets (Sec. 4.1), so the comparison is symmetric and not forced by construction. Evidence against any fitting-by-construction account is internal: reported gains vary in sign across settings (GDA fs→logit gains +1.01 at 1-shot but -0.02 at 16-shot; APE logit→fs loses 0.75 points on ImageNet-Sketch), which would not occur if the 'prediction' were definitionally equal to the search objective. The four self-citations ([14], [15], [16], [38]) appear only in related-work or introduction contexts and carry none of the load; no uniqueness theorem or ansatz is imported from prior work by these authors, and initialization uses external baselines (Tip-Adapter, APE, GDA). The defensible concern — that holdout fitness is only a proxy, and Table 6 varies dataset count rather than composition — is a generalization and robustness risk, not a circular reduction, and per the review rules belongs under correctness risk rather than circularity.

Assumptions & free parameters 3 free parameters · 4 assumptions · 0 invented entities

The central claim rests on the choice of holdout datasets, the decomposition of adaptation into two functions, and the reliability of LLM-generated code. The hyper-parameters of the searched algorithms are fitted via Optuna on downstream validation splits, so the reported improvements are not parameter-free predictions.

free parameters (3)
  • alpha0, alpha1, alpha2 = vary per dataset, ranges [1e-9, 100]
    Logits fusion weights in the searched algorithms, tuned with Optuna on the validation split of each downstream dataset (Sec 4.1, Table 10).
  • w0, w1, topk = w0=w1=0.5 fixed in downstream; topk in (0, d]
    Feature selection weights and number of kept channels. topk is tuned with Optuna on downstream validation; w0 and w1 are fixed to 0.5 during downstream tuning (Table 10).
  • Search-space grid for fitness = {0.5}^2, {0.7d}, {0.1, 1, 10}^3
    The hyper-parameter search space S used in the fitness function during evolution, chosen by hand (Sec 4.1). This grid defines the maximization in the fitness computation.
assumptions (4)
  • domain assumption Training-free VLM adaptation can be decomposed into feature selection and logits computation.
    The method fixes one component while searching the other (Sec 3, Sec 3.1). If some good algorithms require tight coupling of the two parts, they are excluded from the search space.
  • domain assumption The five holdout datasets (CIFAR100, FashionMNIST, ObjectNet, UcMerced, UCSDBirds) are representative of downstream evaluation datasets.
    Fitness is computed on these datasets (Sec 4.1). If they do not correlate with ImageNet, Caltech101, etc., the search would not transfer to downstream tasks.
  • domain assumption Existing algorithms (Tip-Adapter, APE, GDA) are sensible starting points that cover the useful region of algorithm space.
    The initial population is built from their code (Sec 3.1). If these starting points are far from the optimal region, the search may be trapped.
  • domain assumption LLM-generated code, after parsing and fp16 conversion, correctly implements the intended algorithm.
    The method relies on executable code. Appendix A.5.6 shows examples where code does not match the thoughts, so this assumption holds only partially and is mitigated by fitness selection.

how reviews work

0 comments
Cite this review

Pith. "Pith review of EvoVLMA: Evolutionary Vision-Language Model Adaptation." pith.science (2026). https://pith.science/paper/YEYDIQTO

@misc{pith2026250801558,
  author       = {Pith},
  title        = {Pith review of: EvoVLMA: Evolutionary Vision-Language Model Adaptation},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/YEYDIQTO}},
  note         = {Machine review of arXiv:2508.01558}
}
read the original abstract

Pre-trained Vision-Language Models (VLMs) have been exploited in various Computer Vision tasks (e.g., few-shot recognition) via model adaptation, such as prompt tuning and adapters. However, existing adaptation methods are designed by human experts, requiring significant time cost and experience. Inspired by recent advances in Large Language Models (LLMs) based code generation, we propose an Evolutionary Vision-Language Model Adaptation (EvoVLMA) method to automatically search training-free efficient adaptation algorithms for VLMs. We recognize feature selection and logits computation as the key functions in training-free VLM adaptation, and propose a two-stage LLM-assisted evolutionary algorithm for optimizing these parts in a sequential manner, effectively addressing the challenge posed by the expansive search space through a divide-and-conquer strategy. Besides, to enhance the stability and efficiency of searching process, we propose low-precision code conversion, web based code execution and process monitoring, leading to a highly effective automatic algorithm design system. Extensive experiments demonstrate that the algorithms found by EvoVLMA can obtain promising results compared to previous manually-designed ones. More specifically, in the 8-shot image classification setting, the classical APE algorithm can be improved by 1.91 points in recognition accuracy. This research opens new possibilities for automating the optimization of adaptation algorithms of pre-trained multimodal models. Code is available at: https://github.com/kding1225/EvoVLMA

Figures

Figures reproduced from arXiv: 2508.01558 by the authors.

Figure 1
Figure 1. Main idea of this work. (a) The manual algorithm [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. The flowchart of EvoVLMA. Top-left panel: the workflow of automatic algorithm designing; right panel: the imple [PITH_FULL_IMAGE:figures/full_fig_p003_2.png] view at source ↗
Figure 3
Figure 3. Searching process, found algorithms and code snippets. (a) Object curve and found logits computation algorithm [PITH_FULL_IMAGE:figures/full_fig_p006_3.png] view at source ↗
Figures from the paper (12 more)
Figure 4
Figure 4. Figure 4: Effect of different visual backbones. Left to right: [PITH_FULL_IMAGE:figures/full_fig_p008_4.png]
Figure 5
Figure 5. Figure 5: Initial algorithms for two-stage searching. [PITH_FULL_IMAGE:figures/full_fig_p013_5.png]
Figure 6
Figure 6. Figure 6: Initial algorithms for joint searching [PITH_FULL_IMAGE:figures/full_fig_p014_6.png]
Figure 7
Figure 7. Figure 7: Prompts of crossover and mutation for searching feature selection and logits computation algorithms. [PITH_FULL_IMAGE:figures/full_fig_p015_7.png]
Figure 8
Figure 8. Figure 8: Prompts of crossover and mutation for joint searching. The prompt template is the same to that in Fig. 7. [PITH_FULL_IMAGE:figures/full_fig_p015_8.png]
Figure 9
Figure 9. Figure 9: Visualization of searching process of joint searching. The logits computation algorithm is initialized with Tip-Adapter [PITH_FULL_IMAGE:figures/full_fig_p016_9.png]
Figure 10
Figure 10. Figure 10: Comparison of per-dataset few-shot classification results. [PITH_FULL_IMAGE:figures/full_fig_p016_10.png]
Figure 11
Figure 11. Figure 11: Visualization of searching process and results (with Tip-Adapter based initialization). Top row: logits computation [PITH_FULL_IMAGE:figures/full_fig_p017_11.png]
Figure 12
Figure 12. Figure 12: Visualization of searching process and results (with Tip-Adapter based initialization). Top row: feature selection [PITH_FULL_IMAGE:figures/full_fig_p017_12.png]
Figure 13
Figure 13. Figure 13: Visualization of searching process and results (with APE based initialization). Top row: logits computation (first [PITH_FULL_IMAGE:figures/full_fig_p018_13.png]
Figure 14
Figure 14. Figure 14: Visualization of searching process and results (with APE based initialization). Top row: feature selection (first stage), [PITH_FULL_IMAGE:figures/full_fig_p018_14.png]
Figure 15
Figure 15. Figure 15: Visualization of searching process and results (with GDA based initialization). Top row: feature selection (first stage), [PITH_FULL_IMAGE:figures/full_fig_p019_15.png]

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

77 extracted references · 60 canonical work pages

  1. [1]

    Virginia Aglietti, Ira Ktena, Jessica Schrouff, Eleni Sgouritsa, Francisco J. R. Ruiz, Alan Malek, Alexis Bellot, and Silvia Chiappa. 2024. FunBO: Discovering Acquisi- tion Functions for Bayesian Optimization with FunSearch. CoRR abs/2406.04824 (2024)

    arXiv 2024

  2. [2]

    Takuya Akiba, Shotaro Sano, Toshihiko Yanase, Takeru Ohta, and Masanori Koyama. 2019. Optuna: A Next-generation Hyperparameter Optimization Frame- work. In ACM SIGKDD. 2623–2631

    work page 2019

  3. [3]

    Jinheon Baek, Sujay Kumar Jauhar, Silviu Cucerzan, and Sung Ju Hwang. 2025. ResearchAgent: Iterative Research Idea Generation over Scientific Literature with Large Language Models. In NAACL-HLT. 6709–6738

    work page 2025

  4. [4]

    Andrei Barbu, David Mayo, Julian Alverio, William Luo, Christopher Wang, Dan Gutfreund, Josh Tenenbaum, and Boris Katz. 2019. ObjectNet: A large-scale bias-controlled dataset for pushing the limits of object recognition models. In NeurIPS. 9448–9458

    work page 2019

  5. [5]

    Lukas Bossard, Matthieu Guillaumin, and Luc Van Gool. 2014. Food-101 - Mining Discriminative Components with Random Forests. In ECCV. 446–461

    work page 2014

  6. [6]

    Angelica Chen, David Dohan, and David R. So. 2023. EvoPrompting: Language Models for Code-Level Neural Architecture Search. In NeurIPS. 7787–7817

    work page 2023

  7. [7]

    Jun Chen, Deyao Zhu, Guocheng Qian, Bernard Ghanem, Zhicheng Yan, Chenchen Zhu, Fanyi Xiao, Sean Chang Culatana, and Mohamed Elhoseiny

    work page

  8. [8]

    Ting Chen, Simon Kornblith, Mohammad Norouzi, and Geoffrey E. Hinton. 2020. A Simple Framework for Contrastive Learning of Visual Representations. In ICML, Vol. 119. 1597–1607

    work page 2020

Show all 77 references
  1. [9]

    Zhe Chen, Yuchen Duan, Wenhai Wang, Junjun He, Tong Lu, Jifeng Dai, and Yu Qiao. 2023. Vision Transformer Adapter for Dense Predictions. InICLR. 20 pages

    2023

  2. [10]

    Mircea Cimpoi, Subhransu Maji, Iasonas Kokkinos, Sammy Mohamed, and An- drea Vedaldi. 2014. Describing Textures in the Wild. In CVPR. 3606–3613

    2014

  3. [11]

    DeepSeek-AI, Aixin Liu, Bei Feng, and et al. 2024. DeepSeek-V3 Technical Report. CoRR abs/2412.19437 (2024)

    2024 arXiv

  4. [12]

    Mostafa Dehghani, Josip Djolonga, Basil Mustafa, and et al. 2023. Scaling Vision Transformers to 22 Billion Parameters. In ICML, Vol. 202. 7480–7512

    2023

  5. [13]

    Jia Deng, Wei Dong, Richard Socher, Li-Jia Li, Kai Li, and Li Fei-Fei. 2009. Ima- geNet: A large-scale hierarchical image database. In CVPR. 248–255

    2009

  6. [14]

    Kun Ding, Ying Wang, Pengzhang Liu, Qiang Yu, Haojian Zhang, Shiming Xiang, and Chunhong Pan. 2024. Multi-task prompt tuning with soft context sharing for vision–language models. Neurocomputing 603 (2024), 128290

    2024

  7. [15]

    Kun Ding, Qiang Yu, Haojian Zhang, Gaofeng Meng, and Shiming Xiang. 2024. Calibrated Cache Model for Few-Shot Vision-Language Model Adaptation. CoRR abs/2410.08895 (2024)

    2024 arXiv

  8. [16]

    Kun Ding, Haojian Zhang, Qiang Yu, Ying Wang, Shiming Xiang, and Chunhong Pan. 2024. Weak distribution detectors lead to stronger generalizability of vision- language prompt tuning. In AAAI, Vol. 38. 1528–1536

    2024

  9. [17]

    Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xi- aohua Zhai, Thomas Unterthiner, Mostafa Dehghani, Matthias Minderer, Georg Heigold, Sylvain Gelly, Jakob Uszkoreit, and Neil Houlsby. 2021. An Image is Worth 16x16 Words: Transformers for Image Recogn...

    2021

  10. [18]

    Cheng Han, Qifan Wang, Yiming Cui, Zhiwen Cao, Wenguan Wang, Siyuan Qi, and Dongfang Liu. 2023. E2VPT: An Effective and Efficient Approach for Visual Prompt Tuning. In ICCV. 17445–17456

    2023

  11. [19]

    Hao Hao, Xiaoqun Zhang, and Aimin Zhou. 2024. Large language models as surrogate models in evolutionary algorithms: A preliminary study. Swarm Evol. Comput. 91 (2024), 101741

    2024

  12. [20]

    Girshick

    Kaiming He, Xinlei Chen, Saining Xie, Yanghao Li, Piotr Dollár, and Ross B. Girshick. 2022. Masked Autoencoders Are Scalable Vision Learners. In CVPR. 15979–15988

    2022

  13. [21]

    Patrick Helber, Benjamin Bischke, Andreas Dengel, and Damian Borth. 2019. EuroSAT: A Novel Dataset and Deep Learning Benchmark for Land Use and Land Cover Classification. IEEE J. Sel. Top. Appl. Earth Obs. Remote. Sens. 12, 7 (2019), 2217–2226

    2019

  14. [22]

    Wenlong Huang, Pieter Abbeel, Deepak Pathak, and Igor Mordatch. 2022. Lan- guage Models as Zero-Shot Planners: Extracting Actionable Knowledge for Em- bodied Agents. In ICML, Vol. 162. 9118–9147

    2022

  15. [23]

    Jiho Jang, Chaerin Kong, Donghyeon Jeon, Seonhoon Kim, and Nojun Kwak. 2023. Unifying Vision-Language Representation Space with Single-Tower Transformer. In AAAI. 980–988

    2023

  16. [24]

    Ganesh Jawahar, Muhammad Abdul-Mageed, Laks V. S. Lakshmanan, and Dujian Ding. 2024. LLM Performance Predictors are good initializers for Architecture Search. In ACL. 10540–10560

    2024

  17. [25]

    Le, Yun-Hsuan Sung, Zhen Li, and Tom Duerig

    Chao Jia, Yinfei Yang, Ye Xia, Yi-Ting Chen, Zarana Parekh, Hieu Pham, Quoc V. Le, Yun-Hsuan Sung, Zhen Li, and Tom Duerig. 2021. Scaling Up Visual and Vision-Language Representation Learning With Noisy Text Supervision. InICML, Vol. 139. 4904–4916

    2021

  18. [26]

    Belongie, Bharath Hariharan, and Ser-Nam Lim

    Menglin Jia, Luming Tang, Bor-Chun Chen, Claire Cardie, Serge J. Belongie, Bharath Hariharan, and Ser-Nam Lim. 2022. Visual Prompt Tuning. In ECCV, Vol. 13693. 709–727

    2022

  19. [27]

    Juyong Jiang, Fan Wang, Jiasi Shen, Sungju Kim, and Sunghun Kim. 2024. A survey on large language models for code generation. CoRR abs/2406.00515 (2024)

    2024 arXiv

  20. [28]

    Shibo Jie, Zhi-Hong Deng, Shixuan Chen, and Zhijuan Jin. 2024. Convolutional Bypasses Are Better Vision Transformer Adapters. In ECAI, Vol. 392. 202–209

    2024

  21. [29]

    Jonathan Krause, Michael Stark, Jia Deng, and Li Fei-Fei. 2013. 3D Object Repre- sentations for Fine-Grained Categorization. In ICCVW. 554–561

    2013

  22. [30]

    Alex Krizhevsky, Geoffrey Hinton, et al. 2009. Learning multiple layers of features from tiny images. (2009)

    2009

  23. [31]

    Fei-Fei Li, Robert Fergus, and Pietro Perona. 2007. Learning generative visual models from few training examples: An incremental Bayesian approach tested on 101 object categories. Comput. Vis. Image Underst. 106, 1 (2007), 59–70

    2007

  24. [32]

    Yutong Li, Lu Chen, Aiwei Liu, Kai Yu, and Lijie Wen. 2025. ChatCite: LLM Agent with Human Workflow Guidance for Comparative Literature Summary. In COLING. 3613–3630

    2025

  25. [33]

    Kevin Lin, Christopher Agia, Toki Migimatsu, Marco Pavone, and Jeannette Bohg

  26. [34]

    Fei Liu, Xialiang Tong, Mingxuan Yuan, Xi Lin, Fu Luo, Zhenkun Wang, Zhichao Lu, and Qingfu Zhang. 2024. Evolution of Heuristics: Towards Efficient Automatic Algorithm Design Using Large Language Model. In ICML. 32201–32223

    2024

  27. [35]

    Text2Motion: from natural language instructions to feasible plans. Auton. Robots 47, 8 (2023), 1345–1365

    2023

  28. [36]

    Shengcai Liu, Caishun Chen, Xinghua Qu, Ke Tang, and Yew-Soon Ong. 2024. Large language models as evolutionary optimizers. In IEEE Congress on Evolu- tionary Computation (CEC). 1–8

    2024

  29. [37]

    Fei Liu, Yiming Yao, Ping Guo, Zhiyuan Yang, Zhe Zhao, Xi Lin, Xialiang Tong, Mingxuan Yuan, Zhichao Lu, Zhenkun Wang, and Qingfu Zhang. 2024. A Systematic Survey on Large Language Models for Algorithm Design. CoRR abs/2410.14716 (2024)

    2024

  30. [38]

    Kai Lu, Shixiong Xu, Jinqiu Li, Kun Ding, and Gaofeng Meng. 2025. Agent Reviewers: Domain-specific Multimodal Agents with Shared Memory for Paper Review. In ICML

    2025

  31. [39]

    Chris Lu, Samuel Holt, Claudio Fanconi, Alex Chan, Jakob Foerster, Mihaela van der Schaar, and Robert Lange. 2024. Discovering preference optimization algorithms with and for large language models. NeurIPS 37 (2024), 86528–86573

    2024

  32. [40]

    Blaschko, and Andrea Vedaldi

    Subhransu Maji, Esa Rahtu, Juho Kannala, Matthew B. Blaschko, and Andrea Vedaldi. 2013. Fine-Grained Visual Classification of Aircraft. CoRR abs/1306.5151 (2013)

    2013 arXiv

  33. [41]

    Ziming Luo, Zonglin Yang, Zexin Xu, Wei Yang, and Xinya Du. 2025. LLM4SR: A Survey on Large Language Models for Scientific Research. CoRR abs/2501.04306 (2025)

    2025 arXiv

  34. [42]

    Maria-Elena Nilsback and Andrew Zisserman. 2008. Automated Flower Classifi- cation over a Large Number of Classes. In ICVGIP. 722–729

    2008

  35. [43]

    Jinzhu Mao, Dongyun Zou, Li Sheng, Siyi Liu, Chen Gao, Yue Wang, and Yong Li

  36. [44]

    Parkhi, Andrea Vedaldi, Andrew Zisserman, and C

    Omkar M. Parkhi, Andrea Vedaldi, Andrew Zisserman, and C. V. Jawahar. 2012. Cats and dogs. In CVPR. 3498–3505

    2012

  37. [45]

    Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, Gretchen Krueger, and Ilya Sutskever. 2021. Learning Transferable Visual Models From Natural Language Supervision. In ICML, Vol. ...

    2021

  38. [46]

    Maxime Oquab, Timothée Darcet, Théo Moutakanni, and et al. 2024. DINOv2: Learning Robust Visual Features without Supervision. Trans. Mach. Learn. Res. 2024 (2024)

    2024

  39. [47]

    Benjamin Recht, Rebecca Roelofs, Ludwig Schmidt, and Vaishaal Shankar. 2019. Do ImageNet Classifiers Generalize to ImageNet?. In ICML. 5389–5400

    2019

  40. [48]

    Pawan Kumar, Emilien Dupont, Francisco J

    Bernardino Romera-Paredes, Mohammadamin Barekatain, Alexander Novikov, Matej Balog, M. Pawan Kumar, Emilien Dupont, Francisco J. R. Ruiz, Jordan S. Ellenberg, Pengming Wang, Omar Fawzi, Pushmeet Kohli, and Alhussein Fawzi

  41. [49]

    Yongming Rao, Wenliang Zhao, Guangyi Chen, Yansong Tang, Zheng Zhu, Guan Huang, Jie Zhou, and Jiwen Lu. 2022. DenseCLIP: Language-Guided Dense Prediction with Context-Aware Prompting. In CVPR. 18061–18070

    2022

  42. [50]

    Dhruv Shah, Michael Robert Equi, Blazej Osinski, Fei Xia, Brian Ichter, and Sergey Levine. 2023. Navigation with Large Language Models: Semantic Guesswork as a Heuristic for Planning. In CoRL, Vol. 229. 2683–2699

    2023

  43. [51]

    Amanpreet Singh, Ronghang Hu, Vedanuj Goswami, Guillaume Couairon, Woj- ciech Galuba, Marcus Rohrbach, and Douwe Kiela. 2022. FLAVA: A Foundational Language And Vision Alignment Model. In CVPR. 15617–15629

    2022

  44. [52]

    Nature 625, 7995 (2024), 468–475

    Mathematical discoveries from program search with large language models. Nature 625, 7995 (2024), 468–475

    2024

  45. [53]

    Fawaz Sammani and Nikos Deligiannis. 2024. Interpreting and Analysing CLIP’s Zero-Shot Image Classification via Mutual Knowledge. InNeurIPS, Vol. 37. 39597– 39631. MM ’25, October 27–31, 2025, Dublin, Ireland Kun Ding, Ying Wang, Shiming Xiang

    2024

  46. [54]

    Michael Tschannen, Basil Mustafa, and Neil Houlsby. 2023. CLIPPO: Image-and- Language Understanding from Pixels Only. In CVPR. 11006–11017

    2023

  47. [55]

    Niki van Stein and Thomas Bäck. 2024. LLaMEA: A large language model evolu- tionary algorithm for automatically generating metaheuristics. IEEE Transactions on Evolutionary Computation (2024)

    2024

  48. [56]

    Khurram Soomro, Amir Roshan Zamir, and Mubarak Shah. 2012. UCF101: A Dataset of 101 Human Actions Classes From Videos in The Wild. CoRR abs/1212.0402 (2012)

    2012 arXiv

  49. [57]

    Patara Trirat, Wonyong Jeong, and Sung Ju Hwang. 2025. AutoML-Agent: A Multi-Agent LLM Framework for Full-Pipeline AutoML. In ICML

    2025

  50. [58]

    Han Xiao, Kashif Rasul, and Roland Vollgraf. 2017. Fashion-MNIST: a Novel Image Dataset for Benchmarking Machine Learning Algorithms. CoRR abs/1708.07747 (2017)

    2017 arXiv

  51. [59]

    Ehinger, Aude Oliva, and Antonio Torralba

    Jianxiong Xiao, James Hays, Krista A. Ehinger, Aude Oliva, and Antonio Torralba

  52. [60]

    Lipton, and Eric P

    Haohan Wang, Songwei Ge, Zachary C. Lipton, and Eric P. Xing. 2019. Learning Robust Global Representations by Penalizing Local Predictive Power. In NeurIPS. 10506–10518

    2019

  53. [61]

    Zhengbo Wang, Jian Liang, Lijun Sheng, Ran He, Zilei Wang, and Tieniu Tan

  54. [62]

    A Hard-to-Beat Baseline for Training-free CLIP-based Adaptation. InICLR

  55. [63]

    Haoran Ye, Jiarui Wang, Zhiguang Cao, Federico Berto, Chuanbo Hua, Haeyeon Kim, Jinkyoo Park, and Guojie Song. 2024. ReEvo: Large Language Models as Hyper-Heuristics with Reflective Evolution. NeurIPS 37 (2024), 43571–43608

    2024

  56. [64]

    Jiahui Yu, Zirui Wang, Vijay Vasudevan, Legg Yeung, Mojtaba Seyedhosseini, and Yonghui Wu. 2022. CoCa: Contrastive Captioners are Image-Text Foundation Models. Trans. Mach. Learn. Res. 2022 (2022)

    2022

  57. [65]

    Junhua Zeng, Chao Li, Zhun Sun, Qibin Zhao, and Guoxu Zhou. 2024. tnGPS: dis- covering unknown tensor network structure search algorithms via large language models (LLMs). In ICML. 58329–58347

    2024

  58. [66]

    Chengrun Yang, Xuezhi Wang, Yifeng Lu, Hanxiao Liu, Quoc V Le, Denny Zhou, and Xinyun Chen. 2023. Large language models as optimizers. CoRR abs/2309.03409 (2023)

    2023 arXiv

  59. [67]

    Hantao Yao, Rui Zhang, and Changsheng Xu. 2023. Visual-Language Prompt Tuning with Knowledge-Guided Context Optimization. In CVPR. 6757–6767

    2023

  60. [68]

    Yiming Yao, Fei Liu, Ji Cheng, and Qingfu Zhang. 2024. Evolve Cost-Aware Acquisition Functions Using Large Language Models. In PPSN, Vol. 15149. 374– 390

    2024

  61. [69]

    Kaiyang Zhou, Jingkang Yang, Chen Change Loy, and Ziwei Liu. 2022. Learning to Prompt for Vision-Language Models. Int. J. Comput. Vis. 130, 9 (2022), 2337–2348

    2022

  62. [70]

    Xiangyang Zhu, Renrui Zhang, Bowei He, Aojun Zhou, Dong Wang, Bin Zhao, and Peng Gao. 2023. Not All Features Matter: Enhancing Few-shot CLIP with Adaptive Prior Refinement. In ICCV. 2605–2615. EvoVLMA: Evolutionary Vision-Language Model Adaptation MM ’25, October 27–31, 2025, ...

    2023

  63. [72]

    Renrui Zhang, Wei Zhang, Rongyao Fang, Peng Gao, Kunchang Li, Jifeng Dai, Yu Qiao, and Hongsheng Li. 2022. Tip-Adapter: Training-Free Adaption of CLIP for Few-Shot Classification. In ECCV, Vol. 13695. 493–510

    2022

  64. [73]

    Shujian Zhang, Chengyue Gong, Lemeng Wu, Xingchao Liu, and Mingyuan Zhou

  65. [74]

    CoRR abs/2305.02499 (2023)

    Automl-gpt: Automatic machine learning with gpt. CoRR abs/2305.02499 (2023)

    2023 arXiv

  66. [75]

    Shenao Zhang, Sirui Zheng, Shuqi Ke, Zhihan Liu, Wanxin Jin, Jianbo Yuan, Yingxiang Yang, Hongxia Yang, and Zhaoran Wang. 2024. How Can LLM Guide RL? A Value-Based Approach. CoRR abs/2402.16181 (2024)

    2024 arXiv

  67. [2010]

    SUN database: Large-scale scene recognition from abbey to zoo. In CVPR. 3485–3492

  68. [2023]

    Exploring Open-Vocabulary Semantic Segmentation from CLIP Vision Encoder Distillation Only. In ICCV. 699–710

  69. [2024]

    CoRR abs/2403.03962 (2024)

    Identify Critical Nodes in Complex Network with Large Language Models. CoRR abs/2403.03962 (2024)

    2024 arXiv

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.