Pith. sign in

REVIEW 2 major objections 6 minor 128 references

COBRA: COmBinatorial Retrieval Augmentation for Few-Shot Adaptation

T0 review · 2 major / 6 minor · reviewed 2026-08-11 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read Standard retrieval-augmented few-shot adaptation picks redundant images one at a time; COBRA instead selects a diverse, class-balanced set, and this consistently improves accuracy across image classification tasks and few-shot methods.

desk verdict Useful diversity-aware retrieval for few-shot CLIP, but per-shot hyperparameter tuning for COBRA weakens the 'consistent' claim; worth a careful referee. read the letter →

arxiv 2412.17684 v2 pith:4ZZMRZBS submitted 2024-12-23 cs.LG

Arnav M. Das , Gantavya Bhatt , Lilly Kumari , Sahil Verma , Jeff Bilmes This is my paper · ORCID
classification cs.LG
keywords retrievalaugmentationfew-shotlearningcombinatorialmutualinformationsubmodularoptimizationdiversity-awareselectionCLIPimageclassificationLAION-2B
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper argues that standard retrieval-augmented few-shot adaptation picks auxiliary training images one at a time and therefore wastes budget on redundant, near-duplicate samples. It shows that the prevalent nearest-neighbor objective is a special case of combinatorial mutual information (CMI), a set-level scoring family, and proposes COBRA, which uses a facility-location CMI measure that rewards images for being both similar to the target classes and different from each other. On six image classification datasets, four few-shot CLIP adaptation methods, and several CLIP backbones, COBRA consistently outperforms Sim-Score, CLIP-Score, random retrieval, and synthetically generated training images when retrieving from LAION-2B. The practical claim is that diversity-aware set selection is a cheap, reliable upgrade to retrieval-augmented adaptation.

What carries the argument

The load-bearing object is combinatorial mutual information, $I_f(A;B)=f(A)+f(B)-f(A\cup B)$ for a submodular function $f$, which measures how much information a candidate set carries about the target set. COBRA instantiates it with the facility location function so that each auxiliary image contributes $\min(\max_{j\in A} w_{ij}, \max_{j\in V^{\mathrm{tar}}} w_{ij})$: the first max rewards similarity to the already-selected set, the second rewards similarity to the target, and the min forces each selected image to cover a target-relevant region the current set has not saturated. A soft class-balancing term keeps pseudo-label counts close to uniform without hard constraints, and an optional quality score ($\mu q(A)$) blends in per-sample relevance. The whole objective is monotone submodular, so the greedy algorithm with a priority queue maximizes it within a $(1-1/e)$ factor of the optimum.

What would settle it

A direct test is to rerun the 16-shot ImageNet experiment with auxiliary pseudo-labels replaced by labels from a high-accuracy oracle classifier on the same LAION pool. If COBRA's accuracy advantage over Sim-Score disappears or reverses when the labels are corrected, the reported gains are driven by the label-construction step rather than by diversity-aware selection; if the advantage grows, the FLMI diversity term is doing the work.

Watch

Extended reading notes

Core claim

The paper's central claim is that the retrieval step in few-shot CLIP adaptation is a subset-selection problem, not a ranking problem, and that treating it as such improves accuracy. Concretely, it proves that maximizing the sum of similarities to the target set, the objective behind Sim-Score and CLIP-Score, equals maximizing graph-cut mutual information, a specific CMI with no diversity term. COBRA instead maximizes the FLMI objective, $\sum_{i\in V}\min(\max_{j\in A} w_{ij}, \max_{j\in V^{\mathrm{tar}}} w_{ij})$, which selects auxiliary images that are relevant to the target while covering it from different directions, plus a soft class-balancing term $\sum_{u=1}^C \log(1+m_u(A))/C$ and an optional quality score. Because the objective is monotone submodular, greedy selection carries a $(1-1/e)$ approximation guarantee. The experiments support the claim that this set-level selection consistently improves downstream few-shot accuracy over per-sample retrieval.

Load-bearing premise

COBRA's gains rest on the assumption that caption-derived pseudo-labels and CLIP cosine similarity correctly identify which auxiliary images are relevant to the target classes and how different those images are from one another; if captions are wrong or CLIP features miss class-relevant variation, both the relevance and diversity terms in the objective are miscalibrated.

Editorial extensions

If this is right

  • On the six tested datasets, COBRA ranks first on average across 1-16 shots, with the largest improvements on ImageNet and ImageNet-V2, where intraclass diversity is high.
  • COBRA's retrieval advantage transfers across four few-shot adapters (linear probe, CLIP-Adapter, Tip-Adapter-F, and CaFo) and across ViT-B/32, ViT-B/16, and ViT-L/14 backbones.
  • Adding soft class balancing to Sim-Score and CLIP-Score improves them, but they still trail COBRA, isolating FLMI's diversity term as the source of the gain.
  • Retrieving eight times more samples with the baselines does not close the gap with COBRA, so the advantage is not simply a budget effect.
  • Outside vision, COBRA also outperforms Sim-Score at selecting in-context exemplars for a language model on MRPC, SST-2, RTE, and TREC.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Because COBRA only needs a pairwise similarity matrix and pseudo-labels, the same FLMI objective could be dropped into retrieval-augmented generation for text, code, or molecular search without retraining the embedder; the key requirement is a way to encode pairwise similarity among candidate items.
  • The Flowers-102 null result suggests a testable scaling law: the COBRA advantage should grow with the target's intraclass variability. A controlled synthetic target with tunable intraclass variance would let a reader verify whether diversity is the operative mechanism.
  • The component analysis shows the quality term can slightly hurt on ImageNet while helping on Flowers, implying that $\mu$ should be tuned per dataset rather than treated as a universal constant; a practical recipe would default to pure FLMI and add quality only when the pool is noisy.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

2 major / 6 minor

Summary. The paper proposes COBRA, a set-based retrieval strategy for few-shot image classification that augments a small labeled target set with samples from a large web-scale pool (LAION-2B). It first shows that standard nearest-neighbor retrieval objectives (Sim-Score, CLIP-Score) are instances of combinatorial mutual information (CMI) instantiated with a graph-cut function, then proposes an alternative CMI based on facility location (FLMI), augmented with a soft class-balancing term and an optional quality score. Empirical comparisons are carried out over six target datasets, several few-shot adaptation methods, multiple CLIP backbones, and in-context learning tasks, with the headline claim that COBRA 'consistently outperforms' prior retrieval strategies.

Significance. The conceptual framing is attractive: unifying existing retrieval methods under CMI is elegant and gives the diversity/relevance tradeoff a principled home. The theoretical derivations in Lemma 2.2 and Corollaries 2.3-2.4 are correct, and the greedy optimization inherits the standard 1 - 1/e guarantee. The empirical study is broad, including six datasets, four few-shot learners, three backbones, and an ICL transfer, and the paper is transparent about the Flowers-102 non-difference and the relevance assumption in Section 11. However, the central comparative claim is currently undermined by the per-condition selection of COBRA's hyperparameters, which gives COBRA model-selection freedom that the baselines do not receive; the result is therefore conditional on a fairer comparison.

major comments (2)
  1. [§18.3, Tables 7–11, Table 12] The headline comparative claim is compromised by an asymmetry in hyperparameter tuning. COBRA's sparsity parameter k and optional quality weight µ are reported per dataset, per shot count, and per backbone (Tables 7–11 and Table 12), while Sim-Score, CLIP-Score, Random, and SDXL-Aug have no retrieval-side hyperparameters of their own. The appendix statement that 'we then fix these hyperparameters across all retrieval algorithms' (Section 18.3) refers only to the optimizer hyperparameters in Table 6, not to COBRA's k and µ. Because Figure 10b shows that varying µ interpolates between pure COBRA and Sim-Score, the per-cell choice of µ can position COBRA at the best-performing point on a continuum without giving Sim-Score any equivalent tuning. No held-out validation split or a-priori selection rule for k/µ is documented; the sentence about sweeping 'while training on only the samples from the target dataset' indicates selection on the training labels, which risks overfitting. To support 'consistently outperforms', the authors should fix k/µ per dataset (or per task family) using a principled, a-priori choice, or provide baselines with comparable tuning, and re-run the main comparisons.
  2. [§3, §18.2, §11] The method's two core terms both rest on the pseudo-labels obtained by string-matching class names in LAION captions: the similarity matrix W is zeroed between images with different pseudo-labels (Section 18.2), and the soft class-balancing term in Eq. (7) counts pseudo-label membership. The paper does not stress-test the reliability of these pseudo-labels, even though LAION captions are notoriously noisy. If a caption contains a class name but depicts something else, the masked W can remove legitimate edges and the class-balancing term can balance over the wrong partition. This is a correctness risk that is specific to COBRA's set objective, because COBRA's diversity term operates exclusively on the subgraph induced by each predicted class, whereas the scoring baselines use class membership only to aggregate per-class relevance. I request a robustness experiment in which the pseudo-labels are corrupted or replaced by a CLIP-based classifier's predictions, measuring the relative performance drop of COBRA versus Sim-Score and CLIP-Score; without such an experiment, the method's advantage over the string-matching pipeline is not established.
minor comments (6)
  1. [§3 (Definition 3.3 vs Lemma 3.2)] The objective in Eq. (7) uses log(1 + m_u(A)) but Lemma 3.2 is stated and proved for log(m_u(A)); the relation between the lemma and the implemented class-balancing term should be clarified, particularly for low-shot settings where zero counts occur.
  2. [Figure 10b] The x-axis labels are 0, 0.2, 0.4, 0.8, 1.0, skipping the evaluated 0.6 value; the axis should be corrected or the skipped point explained.
  3. [§18.3] The sentence 'We then fix these hyperparameters across all retrieval algorithms' is ambiguous and contradicted by Tables 7–11; please state explicitly that only the learning-algorithm hyperparameters in Table 6 are fixed across retrieval methods.
  4. [§4.1] The claim of 'no statistically significant difference' on Flowers102 is not supported by a formal test; with three seeds, please either run a paired significance test or phrase the statement in terms of overlapping confidence intervals.
  5. [§20.3] The qualitative discussion of Flowers102 refers to 'Fig. 12c', but the figure showing the Air Plant class is Fig. 16c; the cross-reference should be corrected.
  6. [§4.2] The statement that log det-MI was scaled to 'the largest sizes ever used' is not backed by any description of the kernel and the Woodbury matrix trick used; a brief reproducibility note would be valuable.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: COBRA's derivation is self-contained, and its empirical claims rest on external benchmarks rather than on its own inputs.

full rationale

The paper's core derivation is a mathematical generalization result, not a fitted prediction. Lemma 2.2 proves that the graph-cut CMI reduces to the nearest-neighbor retrieval objective, and Corollaries 2.3–2.5 instantiate Sim-Score and CLIP-Score as special cases; this is an equivalence shown from the paper's own equations, not a conclusion that presupposes COBRA's superiority. The COBRA objective in Definition 3.3 is defined a priori from FLMI, a soft class-balancing term, and an optional quality term, and it is then evaluated on held-out downstream accuracy, so no fitted parameter is renamed as a prediction. Self-citations to prior submodularity and CMI work are present but not load-bearing in a circular sense: the definition of FLMI is restated in the paper, and the greedy constant-factor guarantee is attributed to the classical external result of Nemhauser et al. (1978), while the Submarine software citation [10] is an implementation tool, not the source of the empirical advantage. The acknowledged limitation in Section 11 (auxiliary pool must contain relevant samples) is an assumption about the setting, not a hidden restatement of the conclusion. The per-shot tuning of COBRA's k and µ documented in Appendix 18.3 is a legitimate experimental-fairness concern, because it may give COBRA additional model-selection freedom relative to baselines, but it does not make the reported accuracy numbers equivalent by construction to the retrieval objective. The paper is self-contained against external benchmarks across multiple datasets, shots, backbones, and few-shot adapters, so no circular step is exhibited.

Assumptions & free parameters 3 free parameters · 4 assumptions · 0 invented entities

COBRA contributes a selection objective and no new physical or model entities. It relies on standard submodular optimization theory plus three domain assumptions common to retrieval-augmented vision: CLIP similarity validity, caption pseudo-label reliability, and pool relevance. Its free parameters are hyperparameters tuned per setting.

free parameters (3)
  • k (sparsity neighbors) = varies per dataset and shot; e.g., Imagenet k in {5, 8, 32, 512}, Flowers102 k in {8, 16, 128}, ranges in Tables 7 to 11
    Controls the number of nearest neighbors retained in the sparse similarity matrix; it affects FLMI saturation and is chosen by hyperparameter sweep per setting, not derived.
  • lambda (soft class-balancing weight) = 1 for vision experiments; ICL values 0, 0.01, or 0.1
    Weights the class-balancing term in the COBRA objective; fixed to 1 in main vision experiments and tuned in the ICL experiments.
  • mu (quality score weight) = 0, 0.2, or 0.5 depending on dataset and shot; ICL values 0 or 1e-4
    Trade-off between the FLMI objective and the optional quality (Sim-Score) term; tuned in experiments, and the sensitivity analysis shows mu=1 recovers Sim-Score.
assumptions (4)
  • standard math Greedy maximization of a monotone nondecreasing submodular function under a cardinality constraint has a 1-1/e approximation guarantee.
    Used to justify optimizing the COBRA objective with the greedy algorithm (Section 2 and Algorithm 1).
  • domain assumption CLIP cosine similarity captures task-relevant semantic similarity between images.
    The similarity matrix W is built from CLIP image and text embeddings (Section 18.2); if CLIP features miss class-relevant variation, both the relevance term and the diversity term in COBRA are miscalibrated.
  • domain assumption Caption string matching produces reliable pseudo-labels for LAION images.
    The prefiltering step discards captions that do not contain a class name and uses the class name as the label (Section 4 and Section 18.1); wrong or missing captions would mislabel images and harm class-conditioned sparsification and class balancing.
  • domain assumption The auxiliary pool contains samples relevant to the target task.
    Stated as a limitation in Section 11; if the pool lacks relevant samples, retrieval cannot improve performance, with or without diversity.

how reviews work

0 comments
Cite this review

Pith. "Pith review of COBRA: COmBinatorial Retrieval Augmentation for Few-Shot Adaptation." pith.science (2026). https://pith.science/paper/4ZZMRZBS

@misc{pith2026241217684,
  author       = {Pith},
  title        = {Pith review of: COBRA: COmBinatorial Retrieval Augmentation for Few-Shot Adaptation},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/4ZZMRZBS}},
  note         = {Machine review of arXiv:2412.17684}
}
read the original abstract

Retrieval augmentation, the practice of retrieving additional data from large auxiliary pools, has emerged as an effective technique for enhancing model performance in the low-data regime. Prior approaches have employed only nearest-neighbor based strategies for data selection, which retrieve auxiliary samples with high similarity to instances in the target task. However, these approaches are prone to selecting highly redundant samples, since they fail to incorporate any notion of diversity. In our work, we first demonstrate that data selection strategies used in prior retrieval-augmented few-shot adaptation settings can be generalized using a class of functions known as Combinatorial Mutual Information (CMI) measures. We then propose COBRA (COmBinatorial Retrieval Augmentation), which employs an alternative CMI measure that considers both diversity and similarity to a target dataset. COBRA consistently outperforms previous retrieval approaches across image classification tasks and few-shot learning techniques when used to retrieve samples from LAION-2B. COBRA introduces negligible computational overhead to the cost of retrieval while providing significant gains in downstream model performance.

Figures

Figures reproduced from arXiv: 2412.17684 by the authors.

Figure 1
Figure 1. COBRA uses the target pool to select diverse and relevant [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. COBRA retrieves samples from LAION that are diverse and relevant to the target dataset. In contrast, Sim-Score retrieves redundant samples and sometimes even exact duplicates (shown in red boxes). Additional visualizations are presented in Section 20. tion class. To address the deficiencies of nearest-neighbor based retrieval, we propose COmBinatorial Retrieval Aug￾mentation (COBRA) which uses an alternative CMI fun… view at source ↗
Figure 3
Figure 3. 2D Example We consider a simple example where |Dtar| = 64 and |Daux| = 25000. From D aux, we retrieve a subset of size 128 based on D tar. COBRA (left) effectively covers the target D tar, on the other hand, Sim-Score (right) selects clumpy ex￾amples, as highlighted by the bounding boxes. Refer to Section 17 for more details. Submodular Functions Submodular functions, defined over a ground set V , must satisfy f(A ∪… view at source ↗
Figures from the paper (14 more)
Figure 4
Figure 4. Figure 4: Aggregated Ranking (↓) Average ranking of each re￾trieval strategy (lower is better) across different levels of data scarcity, six datasets, and three random seeds. In over 90 experimen￾tal settings, COBRA generally outperforms any baseline we test. retrieval strategie…
Figure 5
Figure 5. Figure 5: Results Across Target Datasets We compare various retrieval strategies over LAION-2B as D aux using Tip-Adapter-F [106] to adapt CLIP to a small target dataset. We find that COBRA generally outperforms all other retrieval strategies, which is also reflected in mean-ran…
Figure 6
Figure 6. Figure 6: Imagenet Results with Different Few-Shot Techniques After retrieving samples from LAION-2B for Imagenet, we train either a Linear Probe, Clip-Adapter [35], Tip-Adapter-F [106], and CaFo [107] on top of a pretrained CLIP model. We find that COBRA-based retrieval consist…
Figure 7
Figure 7. Figure 7: Sample Efficiency We demonstrate that retrieving more samples does not necessarily improve the performance of Sim-Score and CLIP-Score and that COBRA maintains its edge even when retrieving 8x fewer samples. 1 2 4 8 16 # of Training Samples per Class 69 70 71 72 73 Acc…
Figure 8
Figure 8. Figure 8: Results of using COBRA with additional few-shot adaptation strategies [PITH_FULL_IMAGE:figures/full_fig_p019_8.png]
Figure 9
Figure 9. Figure 9: 2D Example We consider a simple example where |Dtar| = 64 and |Daux| = 25000. From D aux, we retrieve a subset of size 128 based on D tar. COBRA (left) effectively covers the target D tar, on the other hand, Sim-Score (right) selects clumpy samples, as highlighted by t…
Figure 10
Figure 10. Figure 10: Hyperparameter sensitivity analysis of COBRA for [PITH_FULL_IMAGE:figures/full_fig_p024_10.png]
Figure 11
Figure 11. Figure 11: Visual Comparison of Retrieval Methods for Vultures (Imagenet). 25 [PITH_FULL_IMAGE:figures/full_fig_p025_11.png]
Figure 12
Figure 12. Figure 12: Visual Comparison of Retrieval Methods for Hoop Skirt (Imagenet). 20.2. Text Example See [PITH_FULL_IMAGE:figures/full_fig_p026_12.png]
Figure 13
Figure 13. Figure 13: Visual Comparison of Retrieval Methods for Warthog (Imagenet). 27 [PITH_FULL_IMAGE:figures/full_fig_p027_13.png]
Figure 14
Figure 14. Figure 14: Visual Comparison of Retrieval Methods for Wheaton Terrier (Imagenet) [PITH_FULL_IMAGE:figures/full_fig_p028_14.png]
Figure 15
Figure 15. Figure 15: ICL Qualitative Example An example of COBRA vs Sim-Score on the TREC dataset [55] 28 [PITH_FULL_IMAGE:figures/full_fig_p028_15.png]
Figure 16
Figure 16. Figure 16: Visual Comparison of Retrieval Methods for Class Air Plant (Flowers-102). We observe there is very little difference in the train and test distributions in Flowers-102. Therefore, the utility of diversity in retrieval is diminished so we do not see statistically signi…
Figure 17
Figure 17. Figure 17: Analysis using [31] w/ Tip-Adapter-F 30 [PITH_FULL_IMAGE:figures/full_fig_p030_17.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

128 extracted references · 59 canonical work pages

  1. [1]

    Gone fishing: Neural active learning with fisher embeddings

    Jordan Ash, Surbhi Goel, Akshay Krishnamurthy, and Sham Kakade. Gone fishing: Neural active learning with fisher embeddings. Advances in Neural Information Processing Systems, 34:8927–8939, 2021. 15

    2021

  2. [2]

    Deep batch active learning by diverse, uncertain gradient lower bounds

    Jordan T Ash, Chicheng Zhang, Akshay Krishnamurthy, John Langford, and Alekh Agarwal. Deep batch active learning by diverse, uncertain gradient lower bounds. arXiv preprint arXiv:1906.03671, 2019. 1, 6, 15

    arXiv 1906

  3. [3]

    Training con- nectionist networks with queries and selective sampling

    Les Atlas, David Cohn, and Richard Ladner. Training con- nectionist networks with queries and selective sampling. Advances in neural information processing systems, 2, 1989. 15

    1989

  4. [4]

    Leaving reality to imagi- nation: Robust classification via generated datasets

    Hritik Bansal and Aditya Grover. Leaving reality to imagi- nation: Robust classification via generated datasets. arXiv preprint arXiv:2302.02503, 2023. 5

    arXiv 2023

  5. [5]

    The power of ensembles for active learning in image classification

    William H Beluch, Tim Genewein, Andreas Nürnberger, and Jan M Köhler. The power of ensembles for active learning in image classification. In Proceedings of the IEEE conference on computer vision and pattern recognition , pages 9368– 9377, 2018. 15

    2018

  6. [6]

    The fifth PASCAL recog- nizing textual entailment challenge

    Luisa Bentivogli, Bernardo Magnini, Ido Dagan, Hoa Trang Dang, and Danilo Giampiccolo. The fifth PASCAL recog- nizing textual entailment challenge. In Proceedings of the Second Text Analysis Conference, TAC 2009, Gaithersburg, Maryland, USA, November 16-17, 2009. NIST, 2009. 14

    2009

  7. [7]

    Truong, Stephen Mussmann, Yinglun Zhu, Jeff A

    Gantavya Bhatt, Yifang Chen, Arnav Mohanty Das, Jifan Zhang, Sang T. Truong, Stephen Mussmann, Yinglun Zhu, Jeff A. Bilmes, Simon S. Du, Kevin G. Jamieson, Jordan T. Ash, and Robert D. Nowak. An experimental design frame- work for label-efficient supervised finetuning of large lan- guage models. In ACL (Findings), pages 6549–6560, 2024. 1, 3, 5

    2024

  8. [8]

    Deep submodular peripteral networks

    Gantavya Bhatt, Arnav Mohanty Das, and Jeff Bilmes. Deep submodular peripteral networks. In The Thirty-eighth An- nual Conference on Neural Information Processing Systems,

Show all 128 references
  1. [9]

    Submodularity in machine learning and artifi- cial intelligence

    Jeff Bilmes. Submodularity in machine learning and artifi- cial intelligence. arXiv preprint arXiv:2202.00132, 2022. 1, 3

    2022 arXiv

  2. [10]

    Submarine: SUBModularity for ARtificial INtelligencE and machine learning

    Jeff Bilmes. Submarine: SUBModularity for ARtificial INtelligencE and machine learning. Online Software System,

  3. [11]

    Retrieval-augmented diffusion models

    Andreas Blattmann, Robin Rombach, Kaan Oktay, Jonas Müller, and Björn Ommer. Retrieval-augmented diffusion models. In Advances in Neural Information Processing Systems, pages 15309–15324. Curran Associates, Inc., 2022. 2

    2022

  4. [12]

    Food-101 – mining discriminative components with random forests

    Lukas Bossard, Matthieu Guillaumin, and Luc Van Gool. Food-101 – mining discriminative components with random forests. In Computer Vision – ECCV 2014, pages 446–461, Cham, 2014. Springer International Publishing. 5

    2014

  5. [13]

    Tom B. Brown, Benjamin Mann, Nick Ryder, Melanie Sub- biah, Jared Kaplan, Prafulla Dhariwal, Arvind Neelakan- tan, Pranav Shyam, Girish Sastry, Amanda Askell, Sand- hini Agarwal, Ariel Herbert-V oss, Gretchen Krueger, Tom Henighan, Rewon Child, Aditya Ramesh, Daniel M. Ziegler...

    2020

  6. [14]

    Synartifact: Classifying and alleviat- ing artifacts in synthetic images via vision-language model

    Bin Cao, Jianhao Yuan, Yexin Liu, Jian Li, Shuyang Sun, Jing Liu, and Bo Zhao. Synartifact: Classifying and alleviat- ing artifacts in synthetic images via vision-language model. arXiv preprint arXiv:2402.18068, 2024. 6

    2024 arXiv

  7. [15]

    The use of mmr, diversity-based reranking for reordering documents and pro- ducing summaries

    Jaime Carbonell and Jade Goldstein. The use of mmr, diversity-based reranking for reordering documents and pro- ducing summaries. In Proceedings of the 21st Annual Inter- national ACM SIGIR Conference on Research and Devel- opment in Information Retrieval, page 335–336, New Yo...

    1998

  8. [16]

    Fast greedy map inference for determinantal point process to improve recommendation diversity

    Laming Chen, Guoxin Zhang, and Eric Zhou. Fast greedy map inference for determinantal point process to improve recommendation diversity. Advances in Neural Information Processing Systems, 31, 2018. 6

    2018

  9. [17]

    Lift yourself up: Retrieval-augmented text generation with self-memory

    Xin Cheng, Di Luo, Xiuying Chen, Lemao Liu, Dongyan Zhao, and Rui Yan. Lift yourself up: Retrieval-augmented text generation with self-memory. Advances in Neural Infor- mation Processing Systems, 36, 2024. 8

    2024

  10. [18]

    Cimpoi, S

    M. Cimpoi, S. Maji, I. Kokkinos, S. Mohamed, , and A. Vedaldi. Describing textures in the wild. In Proceedings of the IEEE Conf. on Computer Vision and Pattern Recognition (CVPR), 2014. 5

    2014

  11. [19]

    Batch active learning at scale

    Gui Citovsky, Giulia DeSalvo, Claudio Gentile, Lazaros Karydas, Anand Rajagopalan, Afshin Rostamizadeh, and Sanjiv Kumar. Batch active learning at scale. Advances in Neural Information Processing Systems, 34:11933–11944,

  12. [20]

    Berg, Robert Nowak, Roshan Sumbaly, Matei Zaharia, and I

    Cody Coleman, Edward Chou, Julian Katz-Samuels, Sean Culatana, Peter Bailis, Alexander C. Berg, Robert Nowak, Roshan Sumbaly, Matei Zaharia, and I. Zeki Yalniz. Simi- larity search for efficient active learning and search of rare concepts, 2021. 15

    2021

  13. [21]

    Support-vector net- works

    Corinna Cortes and Vladimir Vapnik. Support-vector net- works. Machine learning, 20:273–297, 1995. 19

    1995

  14. [22]

    Cover and Joy A

    Thomas M. Cover and Joy A. Thomas. Elements of Infor- mation Theory 2nd Edition (Wiley Series in Telecommuni- cations and Signal Processing). Wiley-Interscience, 2006. 3

    2006

  15. [23]

    Gao, Rui Yang, and Jeff Bilmes

    Arnav Mohanty Das, Gantavya Bhatt, Megh Manoj Bhalerao, Vianne R. Gao, Rui Yang, and Jeff Bilmes. Ac- celerating batch active learning using continual learning techniques. Transactions on Machine Learning Research,

  16. [24]

    Imagenet: A large-scale hierarchical image database

    Jia Deng, Wei Dong, Richard Socher, Li-Jia Li, Kai Li, and Li Fei-Fei. Imagenet: A large-scale hierarchical image database. In 2009 IEEE conference on computer vision and pattern recognition, pages 248–255. Ieee, 2009. 5

    2009

  17. [25]

    Jointly optimising relevance and diversity in image retrieval

    Thomas Deselaers, Tobias Gass, Philippe Dreuw, and Her- mann Ney. Jointly optimising relevance and diversity in image retrieval. In Proceedings of the ACM International Conference on Image and Video Retrieval, New York, NY , USA, 2009. Association for Computing Machinery. 8

    2009

  18. [26]

    Dolan and Chris Brockett

    William B. Dolan and Chris Brockett. Automatically con- structing a corpus of sentential paraphrases. In Proceed- ings of the Third International Workshop on Paraphrasing (IWP2005), 2005. 14

    2005

  19. [27]

    Adversarial active learning for deep networks: a margin based approach

    Melanie Ducoffe and Frederic Precioso. Adversarial active learning for deep networks: a margin based approach. arXiv preprint arXiv:1802.09841, 2018. 15

    2018 arXiv

  20. [28]

    Not all features matter: Enhancing few-shot clip with adaptive prior refinement

    Xiangyang Zhu et al. Not all features matter: Enhancing few-shot clip with adaptive prior refinement. In ICCV, 2023. 18, 19

    2023

  21. [29]

    Amu-tuning: Effective logit bias for clip-based few-shot learning

    Yuwei Tang et al. Amu-tuning: Effective logit bias for clip-based few-shot learning. In CVPR, 2024. 18, 19

    2024

  22. [30]

    Model- agnostic meta-learning for fast adaptation of deep networks

    Chelsea Finn, Pieter Abbeel, and Sergey Levine. Model- agnostic meta-learning for fast adaptation of deep networks. In Proceedings of the 34th International Conference on Machine Learning, pages 1126–1135. PMLR, 2017. 15

    2017

  23. [31]

    The vendi score: A diversity evaluation metric for machine learning

    Dan Friedman and Adji Dieng. The vendi score: A diversity evaluation metric for machine learning. TMLR, 2022. 30

    2022

  24. [32]

    Submodular functions and optimization

    Satoru Fujishige. Submodular functions and optimization. Elsevier, 2005. 3

    2005

  25. [33]

    Datacomp: In search of the next generation of multimodal datasets, 2023

    Samir Yitzhak Gadre, Gabriel Ilharco, Alex Fang, Jonathan Hayase, Georgios Smyrnis, Thao Nguyen, Ryan Marten, Mitchell Wortsman, Dhruba Ghosh, Jieyu Zhang, Eyal Or- gad, Rahim Entezari, Giannis Daras, Sarah Pratt, Vivek Ramanujan, Yonatan Bitton, Kalyani Marathe, Stephen Mussm...

    2023

  26. [34]

    Deep bayesian active learning with image data

    Yarin Gal, Riashat Islam, and Zoubin Ghahramani. Deep bayesian active learning with image data. In International Conference on Machine Learning, pages 1183–1192. PMLR,

  27. [35]

    Clip-adapter: Better vision-language models with feature adapters, 2021

    Peng Gao, Shijie Geng, Renrui Zhang, Teli Ma, Rongyao Fang, Yongfeng Zhang, Hongsheng Li, and Yu Qiao. Clip-adapter: Better vision-language models with feature adapters, 2021. 2, 5, 6, 7, 15

    2021

  28. [36]

    Deep active learning over the long tail

    Yonatan Geifman and Ran El-Yaniv. Deep active learning over the long tail. arXiv preprint arXiv:1711.00941, 2017. 15

    2017 arXiv

  29. [37]

    The unmet promise of synthetic training images: Using retrieved real images performs better, 2024

    Scott Geng, Cheng-Yu Hsieh, Vivek Ramanujan, Matthew Wallingford, Chun-Liang Li, Pang Wei Koh, and Ranjay Krishna. The unmet promise of synthetic training images: Using retrieved real images performs better, 2024. 1, 5, 8

    2024

  30. [38]

    Bilmes, and Himanshu Asnani

    Rishabh Iyer, Ninad A Khargonkar, Jeffrey A. Bilmes, and Himanshu Asnani. Submodular combinatorial information measures with applications in machine learning. InThe 32nd International Conference on Algorithmic Learning Theory, Virtual Conference, 2021. 1, 3, 4

    2021

  31. [39]

    Generalized submodular information measures: Theoretical properties, examples, optimization algorithms, and applications

    Rishabh Iyer, Ninad Khargonkar, Jeff Bilmes, and Himanshu Asnani. Generalized submodular information measures: Theoretical properties, examples, optimization algorithms, and applications. IEEE Transactions on Information Theory, 68(2):752 – 781, 2022. 3

    2022

  32. [40]

    Leveraging passage retrieval with generative models for open domain question answering

    Gautier Izacard and Édouard Grave. Leveraging passage retrieval with generative models for open domain question answering. In Proceedings of the 16th Conference of the European Chapter of the Association for Computational Linguistics: Main Volume, pages 874–880, 2021. 8

    2021

  33. [41]

    Atlas: Few-shot learning with retrieval augmented language mod- els, 2022

    Gautier Izacard, Patrick Lewis, Maria Lomeli, Lucas Hos- seini, Fabio Petroni, Timo Schick, Jane Dwivedi-Yu, Ar- mand Joulin, Sebastian Riedel, and Edouard Grave. Atlas: Few-shot learning with retrieval augmented language mod- els, 2022. 1, 2, 8

    2022

  34. [42]

    Task agnostic meta-learning for few-shot learning

    Muhammad Abdullah Jamal and Guo-Jun Qi. Task agnostic meta-learning for few-shot learning. In 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 11711–11719, 2019. 15

    2019

  35. [43]

    Jegelka and J

    S. Jegelka and J. Bilmes. Cooperative cuts: Graph cuts with submodular edge weights. Technical Report 189, Max Planck Institute for Biological Cybernetics, Tuebingen, Ger- many, 2010. 3

    2010

  36. [44]

    Submodularity beyond submodular energies: Coupling edges in graph cuts

    Stefanie Jegelka and Jeff Bilmes. Submodularity beyond submodular energies: Coupling edges in graph cuts. In CVPR 2011, pages 1897–1904, 2011. 3

    2011

  37. [45]

    Le, Yunhsuan Sung, Zhen Li, and Tom Duerig

    Chao Jia, Yinfei Yang, Ye Xia, Yi-Ting Chen, Zarana Parekh, Hieu Pham, Quoc V . Le, Yunhsuan Sung, Zhen Li, and Tom Duerig. Scaling up visual and vision-language representa- tion learning with noisy text supervision, 2021. 15

    2021

  38. [46]

    Billion- scale similarity search with GPUs

    Jeff Johnson, Matthijs Douze, and Hervé Jégou. Billion- scale similarity search with GPUs. IEEE Transactions on Big Data, 7(3):535–547, 2019. 5

    2019

  39. [47]

    Data-efficient contrastive self-supervised learning: Most beneficial exam- ples for supervised learning contribute the least

    Siddharth Joshi and Baharan Mirzasoleiman. Data-efficient contrastive self-supervised learning: Most beneficial exam- ples for supervised learning contribute the least. In Pro- ceedings of the 40th International Conference on Machine Learning, pages 15356–15370. PMLR, 2023. 3

    2023

  40. [48]

    Large language models struggle to learn long-tail knowledge

    Nikhil Kandpal, Haikang Deng, Adam Roberts, Eric Wal- lace, and Colin Raffel. Large language models struggle to learn long-tail knowledge. In International Conference on Machine Learning, pages 15696–15707. PMLR, 2023. 8

    2023

  41. [49]

    Orient: Submodular mutual information measures for data subset selection under distribution shift

    Athresh Karanam, Krishnateja Killamsetty, Harsha Kokel, and Rishabh Iyer. Orient: Submodular mutual information measures for data subset selection under distribution shift. In Advances in Neural Information Processing Systems, pages 31796–31808. Curran Associates, Inc., 2022. 3

    2022

  42. [50]

    Similar: Submodular information mea- sures based active learning in realistic scenarios

    Suraj Kothawade, Nathan Beck, Krishnateja Killamsetty, and Rishabh Iyer. Similar: Submodular information mea- sures based active learning in realistic scenarios. In Ad- vances in Neural Information Processing Systems , pages 18685–18697. Curran Associates, Inc., 2021. 1

    2021

  43. [51]

    Similar: Submodular information mea- 10 sures based active learning in realistic scenarios

    Suraj Kothawade, Nathan Beck, Krishnateja Killamsetty, and Rishabh Iyer. Similar: Submodular information mea- 10 sures based active learning in realistic scenarios. In Ad- vances in Neural Information Processing Systems , pages 18685–18697. Curran Associates, Inc., 2021. 3, 15

    2021

  44. [52]

    Prism: A rich class of pa- rameterized submodular information measures for guided subset selection, 2022

    Suraj Kothawade, Vishal Kaushal, Ganesh Ramakrishnan, Jeff Bilmes, and Rishabh Iyer. Prism: A rich class of pa- rameterized submodular information measures for guided subset selection, 2022. 1, 4, 6

    2022

  45. [53]

    Determinantal point pro- cesses for machine learning

    Alex Kulesza, Ben Taskar, et al. Determinantal point pro- cesses for machine learning. Foundations and Trends® in Machine Learning, 5(2–3):123–286, 2012. 6

    2012

  46. [54]

    Retrieval-augmented generation for knowledge-intensive nlp tasks

    Patrick Lewis, Ethan Perez, Aleksandra Piktus, Fabio Petroni, Vladimir Karpukhin, Naman Goyal, Heinrich Küt- tler, Mike Lewis, Wen-tau Yih, Tim Rocktäschel, et al. Retrieval-augmented generation for knowledge-intensive nlp tasks. Advances in Neural Information Processing Syste...

    2020

  47. [55]

    Learning question classifiers

    Xin Li and Dan Roth. Learning question classifiers. In COLING 2002: The 19th International Conference on Com- putational Linguistics, 2002. 14, 28

    2002

  48. [56]

    A class of submodular functions for document summarization

    Hui Lin and Jeff Bilmes. A class of submodular functions for document summarization. In Proceedings of the 49th Annual Meeting of the Association for Computational Lin- guistics: Human Language Technologies, pages 510–520, Portland, Oregon, USA, 2011. Association for Computa- ...

    2011

  49. [57]

    Learning mixtures of submodular shells with application to document summarization

    Hui Lin and Jeff A Bilmes. Learning mixtures of submodular shells with application to document summarization. arXiv preprint arXiv:1210.4871, 2012. 3

    2012 arXiv

  50. [58]

    Learning customized visual models with retrieval-augmented knowledge, 2023

    Haotian Liu, Kilho Son, Jianwei Yang, Ce Liu, Jianfeng Gao, Yong Jae Lee, and Chunyuan Li. Learning customized visual models with retrieval-augmented knowledge, 2023. 1, 2, 3, 6, 8, 15

    2023

  51. [59]

    Jiachang Liu, Dinghan Shen, Yizhe Zhang, Bill Dolan, Lawrence Carin, and Weizhu Chen. What makes good in- context examples for GPT-3? In Proceedings of Deep Learning Inside Out (DeeLIO 2022): The 3rd Workshop on Knowledge Extraction and Integration for Deep Learning Architectu...

    2022

  52. [60]

    Submodular feature selection for high- dimensional acoustic score spaces

    Yuzong Liu, Kai Wei, Katrin Kirchhoff, Yisong Song, and Jeff Bilmes. Submodular feature selection for high- dimensional acoustic score spaces. In 2013 IEEE Interna- tional Conference on Acoustics, Speech and Signal Process- ing, pages 7184–7188, 2013. 3

    2013

  53. [61]

    Fine-grained visual clas- sification of aircraft, 2013

    Subhransu Maji, Esa Rahtu, Juho Kannala, Matthew Blaschko, and Andrea Vedaldi. Fine-grained visual clas- sification of aircraft, 2013. 5

    2013

  54. [62]

    Active learning principles for in-context learning with large language models

    Katerina Margatina, Timo Schick, Nikolaos Aletras, and Jane Dwivedi-Yu. Active learning principles for in-context learning with large language models. In Findings of the Association for Computational Linguistics: EMNLP 2023, pages 5011–5034, Singapore, 2023. Association for Co...

    2023

  55. [63]

    Accelerated greedy algorithms for maximiz- ing submodular set functions

    Michel Minoux. Accelerated greedy algorithms for maximiz- ing submodular set functions. In Optimization Techniques: Proceedings of the 8th IFIP Conference on Optimization Techniques Würzburg, September 5–9, 1977, pages 234–243. Springer, 2005. 3, 17

    1977

  56. [64]

    Coresets for data-efficient training of machine learning mod- els

    Baharan Mirzasoleiman, Jeff Bilmes, and Jure Leskovec. Coresets for data-efficient training of machine learning mod- els. In International Conference on Machine Learning , pages 6950–6960. PMLR, 2020. 3

    2020

  57. [65]

    An analysis of approximations for maximizing sub- modular set functions—i

    George L Nemhauser, Laurence A Wolsey, and Marshall L Fisher. An analysis of approximations for maximizing sub- modular set functions—i. Mathematical programming, 14: 265–294, 1978. 2, 3, 17

    1978

  58. [66]

    On first- order meta-learning algorithms

    Alex Nichol, Joshua Achiam, and John Schulman. On first- order meta-learning algorithms. CoRR, abs/1803.02999,

    arXiv

  59. [67]

    Automated flower classification over a large number of classes

    Maria-Elena Nilsback and Andrew Zisserman. Automated flower classification over a large number of classes. In Indian Conference on Computer Vision, Graphics and Image Processing, 2008. 5

    2008

  60. [68]

    Direct: Deep active learning under imbalance and label noise

    Shyam Nuggehalli, Jifan Zhang, Lalit Jain, and Robert Nowak. Direct: Deep active learning under imbalance and label noise. arXiv preprint arXiv:2312.09196, 2023. 15

    2023 arXiv

  61. [69]

    Oreshkin, Pau Rodríguez López, and Alexandre Lacoste

    Boris N. Oreshkin, Pau Rodríguez López, and Alexandre Lacoste. TADAM: task dependent adaptive metric for im- proved few-shot learning. CoRR, abs/1805.10123, 2018. 15

    2018 arXiv

  62. [70]

    Check your facts and try again: Im- proving large language models with external knowledge and automated feedback

    Baolin Peng, Michel Galley, Pengcheng He, Hao Cheng, Yujia Xie, Yu Hu, Qiuyuan Huang, Lars Liden, Zhou Yu, Weizhu Chen, et al. Check your facts and try again: Im- proving large language models with external knowledge and automated feedback. arXiv preprint arXiv:2302.12813,

    arXiv

  63. [71]

    Sdxl: Improving latent diffusion mod- els for high-resolution image synthesis

    Dustin Podell, Zion English, Kyle Lacey, Andreas Blattmann, Tim Dockhorn, Jonas Müller, Joe Penna, and Robin Rombach. Sdxl: Improving latent diffusion mod- els for high-resolution image synthesis. arXiv preprint arXiv:2307.01952, 2023. 5

    2023 arXiv

  64. [72]

    What does a platypus look like? generating customized prompts for zero- shot image classification

    Sarah Pratt, Rosanne Liu, and Ali Farhadi. What does a platypus look like? generating customized prompts for zero- shot image classification. arXiv preprint arXiv:2209.03320,

    arXiv

  65. [73]

    Learning transferable visual models from natural language supervision, 2021

    Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, Gretchen Krueger, and Ilya Sutskever. Learning transferable visual models from natural language supervision, 2021. 2, 4, 5, 6, 15, 20

    2021

  66. [74]

    Meta-learning with implicit gradients, 2019

    Aravind Rajeswaran, Chelsea Finn, Sham Kakade, and Sergey Levine. Meta-learning with implicit gradients, 2019. 15

    2019

  67. [75]

    In-context retrieval-augmented language models

    Ori Ram, Yoav Levine, Itay Dalmedigos, Dor Muhlgay, Amnon Shashua, Kevin Leyton-Brown, and Yoav Shoham. In-context retrieval-augmented language models. Transac- tions of the Association for Computational Linguistics, 11: 1316–1331, 2023. 1, 2, 8

    2023

  68. [76]

    Do imagenet classifiers generalize to imagenet? CoRR, abs/1902.10811, 2019

    Benjamin Recht, Rebecca Roelofs, Ludwig Schmidt, and Vaishaal Shankar. Do imagenet classifiers generalize to imagenet? CoRR, abs/1902.10811, 2019. 5

    1902 arXiv

  69. [77]

    Sentence-bert: Sentence embeddings using siamese bert-networks

    Nils Reimers and Iryna Gurevych. Sentence-bert: Sentence embeddings using siamese bert-networks. arXiv preprint arXiv:1908.10084, 2019. 14 11

    1908 arXiv

  70. [78]

    Learn- ing to retrieve prompts for in-context learning, 2022

    Ohad Rubin, Jonathan Herzig, and Jonathan Berant. Learn- ing to retrieve prompts for in-context learning, 2022. 14

    2022

  71. [79]

    Akanksha Saran, Safoora Yousefi, Akshay Krishnamurthy, John Langford, and Jordan T. Ash. Streaming active learning with deep neural networks, 2023. 6

    2023

  72. [80]

    Laion-5b: An open large-scale dataset for training next generation image-text models, 2022

    Christoph Schuhmann, Romain Beaumont, Richard Vencu, Cade Gordon, Ross Wightman, Mehdi Cherti, Theo Coombes, Aarush Katta, Clayton Mullis, Mitchell Worts- man, Patrick Schramowski, Srivatsa Kundurthy, Katherine Crowson, Ludwig Schmidt, Robert Kaczmarczyk, and Jenia Jitsev. Lai...

    2022

  73. [81]

    Active learning for convolu- tional neural networks: A core-set approach

    Ozan Sener and Silvio Savarese. Active learning for convolu- tional neural networks: A core-set approach. arXiv preprint arXiv:1708.00489, 2017. 1

    2017 arXiv

  74. [82]

    Active learning for con- volutional neural networks: A core-set approach

    Ozan Sener and Silvio Savarese. Active learning for con- volutional neural networks: A core-set approach. In In- ternational Conference on Learning Representations, 2018. 15

    2018

  75. [83]

    From theories to queries: Active learning in practice

    Burr Settles. From theories to queries: Active learning in practice. In Active Learning and Experimental Design workshop In conjunction with AISTATS 2010, pages 1–18, Sardinia, Italy, 2011. PMLR. 15

    2010

  76. [84]

    On greedy maximization of entropy

    Dravyansh Sharma, Ashish Kapoor, and Amit Deshpande. On greedy maximization of entropy. In Proceedings of the 32nd International Conference on Machine Learning, pages 1330–1338, Lille, France, 2015. PMLR. 6

    2015

  77. [85]

    Replug: Retrieval-augmented black-box language mod- els

    Weijia Shi, Sewon Min, Michihiro Yasunaga, Minjoon Seo, Rich James, Mike Lewis, Luke Zettlemoyer, and Wen-tau Yih. Replug: Retrieval-augmented black-box language mod- els. arXiv preprint arXiv:2301.12652, 2023. 1, 2, 8

    2023 arXiv

  78. [86]

    Retrieval augmentation reduces hallucination in conversation

    Kurt Shuster, Spencer Poff, Moya Chen, Douwe Kiela, and Jason Weston. Retrieval augmentation reduces hallucination in conversation. In Findings of the Association for Compu- tational Linguistics: EMNLP 2021, pages 3784–3803, 2021. 8

    2021

  79. [87]

    Jake Snell, Kevin Swersky, and Richard S. Zemel. Prototyp- ical networks for few-shot learning, 2017. 15

    2017

  80. [88]

    Manning, Andrew Ng, and Christopher Potts

    Richard Socher, Alex Perelygin, Jean Wu, Jason Chuang, Christopher D. Manning, Andrew Ng, and Christopher Potts. Recursive deep models for semantic compositionality over a sentiment treebank. In Proceedings of the 2013 Confer- ence on Empirical Methods in Natural Language Proc...

    2013

  81. [89]

    Smith, and Tao Yu

    Hongjin Su, Jungo Kasai, Chen Henry Wu, Weijia Shi, Tianlu Wang, Jiayi Xin, Rui Zhang, Mari Ostendorf, Luke Zettlemoyer, Noah A. Smith, and Tao Yu. Selective anno- tation makes language models better few-shot learners. In The Eleventh International Conference on Learning Repre...

    2023

  82. [90]

    Flood Sung, Yongxin Yang, Li Zhang, Tao Xiang, Philip H. S. Torr, and Timothy M. Hospedales. Learning to com- pare: Relation network for few-shot learning, 2018. 15

    2018

  83. [91]

    A comparative study of diversity methods for hybrid text and image retrieval approaches

    Sabrina Tollari, Philippe Mulhem, Marin Ferecatu, Hervé Glotin, Marcin Detyniecki, Patrick Gallinari, Hichem Sahbi, and Zhong-Qiu Zhao. A comparative study of diversity methods for hybrid text and image retrieval approaches. In Evaluating Systems for Multilingual and Multimoda...

    2009

  84. [92]

    Sus-x: Training-free name-only transfer of vision-language models, 2023

    Vishaal Udandarao, Ankush Gupta, and Samuel Albanie. Sus-x: Training-free name-only transfer of vision-language models, 2023. 7, 8, 18

    2023

  85. [93]

    Matching networks for one shot learning

    Oriol Vinyals, Charles Blundell, Timothy Lillicrap, koray kavukcuoglu, and Daan Wierstra. Matching networks for one shot learning. In Advances in Neural Information Pro- cessing Systems. Curran Associates, Inc., 2016. 15

    2016

  86. [94]

    Neural priming for sample- efficient adaptation

    Matthew Wallingford, Vivek Ramanujan, Alex Fang, Aditya Kusupati, Roozbeh Mottaghi, Aniruddha Kembhavi, Lud- wig Schmidt, and Ali Farhadi. Neural priming for sample- efficient adaptation. In Thirty-seventh Conference on Neural Information Processing Systems, 2023. 1, 2, 3, 5, ...

    2023

  87. [95]

    GPT-J-6B: A 6 Billion Parameter Autoregressive Language Model

    Ben Wang and Aran Komatsuzaki. GPT-J-6B: A 6 Billion Parameter Autoregressive Language Model. https:// github.com/kingoflolz/mesh- transformer- jax, 2021. 14

    2021

  88. [96]

    Diversity measurement and subset selection for instruction tuning datasets

    Peiqi Wang, Yikang Shen, Zhen Guo, Matthew Stallone, Yoon Kim, Polina Golland, and Rameswar Panda. Diversity measurement and subset selection for instruction tuning datasets. arXiv preprint arXiv:2402.02318, 2024. 6

    2024 arXiv

  89. [97]

    Augmenting black-box llms with medical textbooks for clinical question answering

    Yubo Wang, Xueguang Ma, and Wenhu Chen. Augmenting black-box llms with medical textbooks for clinical question answering. arXiv preprint arXiv:2309.02233, 2023. 8

    2023 arXiv

  90. [98]

    Submodularity in data subset selection and active learning

    Kai Wei, Rishabh Iyer, and Jeff Bilmes. Submodularity in data subset selection and active learning. In International conference on machine learning, pages 1954–1963. PMLR,

    1954

  91. [99]

    Diverse diffusion: Enhancing image diversity in text-to- image generation

    Mariia Zameshina, Olivier Teytaud, and Laurent Najman. Diverse diffusion: Enhancing image diversity in text-to- image generation. arXiv preprint arXiv:2310.12583, 2023. 6

    2023 arXiv

  92. [100]

    Zancato, A

    L. Zancato, A. Achille, T. Liu, M. Trager, P. Perera, and S. Soatto. Train/test-time adaptation with retrieval. In 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 15911–15921, Los Alamitos, CA, USA, 2023. IEEE Computer Society. 1, 2, 3, 5, 6, 8

    2023

  93. [101]

    Keyword-based diverse image retrieval with variational multiple instance graph

    Yawen Zeng, Yiru Wang, Dongliang Liao, Gongfu Li, Weijie Huang, Jin Xu, Da Cao, and Hong Man. Keyword-based diverse image retrieval with variational multiple instance graph. IEEE Transactions on Neural Networks and Learning Systems, 34(12):10528–10537, 2023. 8

    2023

  94. [102]

    Repocoder: Repository-level code completion through iter- ative retrieval and generation

    Fengji Zhang, Bei Chen, Yue Zhang, Jacky Keung, Jin Liu, Daoguang Zan, Yi Mao, Jian-Guang Lou, and Weizhu Chen. Repocoder: Repository-level code completion through iter- ative retrieval and generation. In Proceedings of the 2023 Conference on Empirical Methods in Natural Langu...

    2023

  95. [103]

    Galaxy: Graph-based active learning at the extreme

    Jifan Zhang, Julian Katz-Samuels, and Robert Nowak. Galaxy: Graph-based active learning at the extreme. In In- ternational Conference on Machine Learning, pages 26223– 26238. PMLR, 2022. 15 12

    2022

  96. [104]

    Labelbench: A comprehensive framework for benchmarking adaptive label- efficient learning

    Jifan Zhang, Gregory Canal, Yinglun Zhu, Robert D Nowak, Yifang Chen, Arnav M Das, Gantavya Bhatt, Stephen Muss- mann, Jeffrey Bilmes, Simon S Du, et al. Labelbench: A comprehensive framework for benchmarking adaptive label- efficient learning. arXiv preprint arXiv:2306.09910,...

    2023 arXiv

  97. [105]

    Algorithm selection for deep active learning with imbal- anced datasets, 2023

    Jifan Zhang, Shuai Shao, Saurabh Verma, and Robert Nowak. Algorithm selection for deep active learning with imbal- anced datasets, 2023. 15

    2023

  98. [106]

    Tip-adapter: Training-free clip-adapter for better vision- language modeling, 2021

    Renrui Zhang, Rongyao Fang, Wei Zhang, Peng Gao, Kunchang Li, Jifeng Dai, Yu Qiao, and Hongsheng Li. Tip-adapter: Training-free clip-adapter for better vision- language modeling, 2021. 2, 5, 6, 7, 15

    2021

  99. [107]

    Prompt, generate, then cache: Cascade of foundation models makes strong few-shot learners

    Renrui Zhang, Xiangfei Hu, Bohao Li, Siyuan Huang, Han- qiu Deng, Hongsheng Li, Yu Qiao, and Peng Gao. Prompt, generate, then cache: Cascade of foundation models makes strong few-shot learners. arXiv preprint arXiv:2303.02151,

    arXiv

  100. [108]

    Keyword-based diverse image retrieval by semantics-aware contrastive learn- ing and transformer, 2023

    Minyi Zhao, Jinpeng Wang, Dongliang Liao, Yiru Wang, Huanzhong Duan, and Shuigeng Zhou. Keyword-based diverse image retrieval by semantics-aware contrastive learn- ing and transformer, 2023. 8

    2023

  101. [109]

    Conditional prompt learning for vision-language models

    Kaiyang Zhou, Jingkang Yang, Chen Change Loy, and Zi- wei Liu. Conditional prompt learning for vision-language models. In IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2022. 2, 15

    2022

  102. [110]

    Learning to prompt for vision-language models

    Kaiyang Zhou, Jingkang Yang, Chen Change Loy, and Zi- wei Liu. Learning to prompt for vision-language models. International Journal of Computer Vision (IJCV), 2022. 2, 15

    2022

  103. [111]

    Docprompting: Generating code by retriev- ing the docs

    Shuyan Zhou, Uri Alon, Frank F Xu, Zhengbao Jiang, and Graham Neubig. Docprompting: Generating code by retriev- ing the docs. In The Eleventh International Conference on Learning Representations, 2022. 8 13 COBRA: COmBinatorial Retrieval Augmentation for Few-Shot Adaptation Su...

    2022

  104. [113]

    Summary of Notations We provide a list of notations in the paper in the Tab. 4. Notation Meaning Daux Auxiliary dataset Dtar Target dataset V aux Daux indices V tar Dtar indices X Domain of images Y Domain of labels Z Domain of Daux elements W Pairwise similarity matrix wij El...

  105. [114]

    In this section, we explore its use within the framework of in-context learning (ICL) with large language models (LLMs)

    Application to In-Context Learning While our work primarily focuses on vision-related applications, COBRA is a versatile retrieval strategy that can be applied across a wide range of settings. In this section, we explore its use within the framework of in-context learning (ICL...

  106. [115]

    Additional Related Work Few-shot Learning Few-shot learning techniques are designed to enhance the data efficiency of neural networks, enabling them to generalize effectively from a limited amount of data. Older lines of research investigate using learned metrics to 14 quantif...

  107. [116]

    However, this assumption may not hold for highly novel tasks or specialized domains

    Limitations A key limitation of COBRA is that it assumes that the auxiliary dataset has some samples that are relevant to the target task. However, this assumption may not hold for highly novel tasks or specialized domains. Furthermore, diverse retrieval may not offer signific...

  108. [117]

    Lemma 12.1 (Graph Cut Mutual Information )

    Background and Lemmas from the Main Paper In this section, we re-write the lemmas from the main paper (Section 2 and Section 3 to be precise). Lemma 12.1 (Graph Cut Mutual Information ). Let G = (V, E) be a graph with edge weights defined with symmetric W = [w]i,j ∈ R+ ∪ {0}. ...

  109. [118]

    For our experiments, we use sparse matrices constructed using FAISS with r nearest neighbors in each row, where r << M+ N; this makes our space complexity O(r(M + N ))

    Complexity Analysis For simplicity, let M (|V tar| according to the main paper) be the total number of training samples provided and N (|V aux| according to the main paper) be the size of an auxiliary dataset. For our experiments, we use sparse matrices constructed using FAISS...

  110. [119]

    Submodular Maximization In this section, we provide the pseudocode for maximizing submodular function. Since COBRA uses a monotone non- decreasing and normalized ( f (∅) = 0) submodular function (a.k.a polymatroidal functions) for objective, we outline the greedy algorithm in ...

  111. [120]

    Below, we describe each additional approach

    Additional Baseline Details In this section, we compare to other retrieval baselines. Below, we describe each additional approach. Note that, similar to the main paper, every additional baseline is conducted on Imagenet-1K. 17 Algorithm 1 Greedy Algorithm for Maximizing polyma...

  112. [121]

    We demonstrate in Figure 8 that COBRA continues to outperform the next best baseline

    Additional Few-Shot Learning Strategies In this section, we include results that test various retrieval strategies for other, more recent, few-shot learning strategies [28, 29]. We demonstrate in Figure 8 that COBRA continues to outperform the next best baseline. 18 16 32 64 1...

  113. [122]

    For Dtar we use 16 samples from each of 4 different Gaussian, making it 64 samples per class

    Toy Example Setup Here we describe the toy example setup from the main paper in greater detail. For Dtar we use 16 samples from each of 4 different Gaussian, making it 64 samples per class. For the Daux we sample 25000 points from a mixture of Gaussian such that the Daux is he...

  114. [123]

    Lastly, we do not use any optional quality function

    on 64 samples from Dtar and use its predicted labels on Daux to associate each of its samples with one of the 4 labels. Lastly, we do not use any optional quality function. With the setup above, retrieval is performed to fetch 128 samples from Daux, and is shown in Fig.9

  115. [124]

    19 Feature 1 Feature 2 Retrieving using COBRA aux tar COBRA Feature 1 Retrieving using Sim-Score aux tar Sim-Score Figure 9

    Implementation Details For convenience, we again describe the auxiliary data collection procedure here. 19 Feature 1 Feature 2 Retrieving using COBRA aux tar COBRA Feature 1 Retrieving using Sim-Score aux tar Sim-Score Figure 9. 2D Example We consider a simple example where |D...

  116. [125]

    Collect Target DatasetDtar: We collect a small target dataset by sampling a standard image classification dataset uniformly at random, retaining 1-16 images per class

  117. [126]

    This stage circumvents the need to compute features for the full auxiliary pool while filtering out images that are unlikely to contain any semantically relevant information

    Collect Auxilary Dataset Daux: We follow the prefiltering step proposed by [94] and use string matching to discard images with captions that do not contain the name of any class name in the target dataset. This stage circumvents the need to compute features for the full auxili...

  118. [127]

    Sensitivity Analysis The hyperparameter associated with COBRA is the sparsity of the similarity matrix, which is governed by the top-k entries in each row. To this end, for our first sensitivity analysis, we study the downstream accuracy for the 16-shot case with Imagenet as D...

  119. [128]

    Air Plant

    More Qualitative Results 20.1. Imagenet Results (a) Target Images (b) COBRA Images (c) Sim-Score Images (d) CLIP-Score Images (e) SDXL Images (f) Random Images Figure 11. Visual Comparison of Retrieval Methods for Vultures (Imagenet). 25 (a) Target Images (b) COBRA Images (c) ...

  120. [2025]

    https://submarine.page. 5, 6

Pith tools

Reviewed August 11, 2026 · model on record in the stance chip above.