Pith. sign in

REVIEW 3 major objections 6 minor 55 references

Pick the class center, beat the fancy baselines

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · glm-5.2

2026-07-08 21:14 UTC pith:B45KTJ3W

load-bearing objection Simple, useful baseline for few-shot KD coreset selection, but the 'consistently surpasses' claim is overstated for pre-trained students. the 3 major comments →

arxiv 2607.05891 v1 pith:B45KTJ3W submitted 2026-07-07 cs.CV cs.AIcs.LG

Few-Medoids: An Embarrassingly Simple Coreset Selection Method for Few-Shot Knowledge Distillation

Cemil-Andrei Dilmac , Florinel-Alin Croitoru , Radu Tudor Ionescu This is my paper
classification cs.CV cs.AIcs.LG
keywords coreset selectionknowledge distillationfew-shot learningmedoiddata pruningimage classificationlatent spacemodel compression
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper introduces few-medoids, a method for selecting a small, representative subset of training data (a coreset) for few-shot knowledge distillation. The approach is straightforward: within the teacher model's latent feature space, for each class, compute the average Euclidean distance from every sample to all other same-class samples, and select the samples with the lowest average distance. These geometrically central samples (medoids) are then used to train the student model. The author demonstrates across four image classification datasets and three teacher-student architecture pairs that this centrality criterion consistently outperforms random selection, herding, k-center Greedy, and PCA-guided matching, particularly when the student model is trained from scratch. The paper's central claim is that geometric centrality in the teacher's representation space is a superior and simpler proxy for sample importance in distillation than existing diversity- or mean-matching-based heuristics.

Core claim

The core discovery is that, for few-shot knowledge distillation, selecting samples that sit closest to the geometric center of their class distribution in the teacher's latent space yields better student performance than established coreset selection baselines. This advantage is most pronounced in the low-to-medium shot regime (1 to 32 samples per class) and when the student network is initialized from scratch. The method requires no training dynamics, no gradient computation, and no iterative optimization—it is a single-pass ranking by average pairwise L2 distance within each class.

What carries the argument

few-medoids: a per-class sample ranking computed as the average Euclidean (L2) distance between a sample's teacher-extracted feature vector and all other same-class feature vectors; samples with the lowest average distance are selected for the coreset.

Load-bearing premise

The method assumes that geometric centrality in the teacher's latent space is the right criterion for selecting distillation samples, but the paper provides no theoretical justification for why this should be the case, and the advantage disappears or reverses when the student model is pre-trained rather than trained from scratch.

What would settle it

If, in a broader range of teacher-student architecture pairs or datasets, few-medoids consistently underperforms random selection or herding—particularly for pre-trained students—the claim that centrality is a universally superior selection criterion would not hold.

Watch this falsifier. Get emailed when new claim-graph text bears on it.

If this is right

  • Researchers using few-shot knowledge distillation can adopt few-medoids as a default, low-cost coreset selection method, potentially replacing more computationally expensive or less reliable baselines like herding and k-center Greedy.
  • The result suggests that for distillation, representativeness (being close to the class center) may matter more than diversity (covering the spread of the class), challenging the assumptions behind coverage-maximizing selection methods.
  • The finding that the method's advantage depends on whether the student is trained from scratch or pre-trained indicates that the optimal coreset selection strategy is contingent on the student's initialization state, opening a direction for adaptive selection methods.
  • The simplicity and training-free nature of the method make it directly applicable to resource-constrained settings where reducing both training data and model size is necessary.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

3 major / 6 minor

Summary. The paper introduces few-medoids, a training-free coreset selection method for few-shot knowledge distillation (KD). The method selects, per class, the k samples with the lowest average L2 distance to all same-class samples in the teacher's frozen feature space. The authors evaluate the method on four datasets (CIFAR-10, CIFAR-100, Oxford Flowers 102, Food-101) and three teacher-student architecture pairs (ResNet-34→ResNet-18, ViT-B/16→ResNet-50, ViT-B/16→ViT-Small), comparing against random selection, herding, k-center Greedy, and a custom PCA-guided matching baseline. The central claim is that few-medoids consistently surpasses these baselines and can serve as a drop-in replacement for commonly used methods.

Significance. The paper's primary strength is the simplicity and parameter-free nature of few-medoids: it requires no fitted constants, uses only the teacher's frozen features, and is trivially reproducible. The authors release code publicly, which is commendable. The experimental matrix is reasonably broad (four datasets, three architecture pairs, eight values of k, five seeds). The observation that geometric centrality in the teacher's latent space helps from-scratch students more than pre-trained students is interesting and, if properly scoped and investigated, could be a useful empirical finding for the community.

major comments (3)
  1. The abstract states that few-medoids 'is able to consistently surpass the random selection baseline, as well as the other coreset selection strategies,' and the conclusion recommends it as a 'drop-in replacement for commonly-used baselines.' However, the paper's own ViT-B/16→ViT-Small results (Tables 3–6) contradict this: herding dominates for k≥4 across all four datasets, often by large margins (e.g., CIFAR-10 k=64: herding 94.53 vs. few-medoids 86.94; CIFAR-100 k=128: herding 84.18 vs. 77.57; Food-101 k=128: herding 86.86 vs. 83.82). The text in §4 acknowledges this exception, but the abstract and conclusion do not. The central claim's scope is broader than the evidence supports. The abstract and conclusion should be revised to state that the advantage holds primarily for students trained from scratch, or the 'consistently surpasses' claim should be withdrawn.
  2. The paper offers a post-hoc explanation for the ViT-B/16→ViT-Small results, speculating that few-medoids excels when the student is trained from scratch because 'its initial latent space is not semantically organized a priori' (§4). This is a load-bearing mechanistic claim—distinguishing the from-scratch and pre-trained regimes is central to scoping the contribution—but no experiment is provided to test it. A controlled experiment (e.g., initializing ViT-Small from scratch, or ablating the student initialization for a fixed teacher-student pair) would verify whether the gains are attributable to the selection method or to the training dynamics. Without this, the explanation remains speculative and the contribution's scope is unclear.
  3. No statistical significance tests are provided. Several of the reported wins in the from-scratch settings have margins within one standard deviation (e.g., Table 3, CIFAR-10 ResNet-34→ResNet-18, k=1: few-medoids 18.05±1.23 vs. herding 17.17±1.46; Table 5, Oxford Flowers 102, k=8: random 51.69±0.73 vs. few-medoids 51.44±0.59). Given that the paper claims 'consistent' superiority, at least a paired test across seeds for the key comparisons would clarify which wins are real rather than noise. This is particularly important for Oxford Flowers 102, where the authors themselves note the rankings are 'unstable' (§4).
minor comments (6)
  1. Table 1: Oxford Flowers 102 lists 'Images per class' for the train split as '10' but the total training images as 1,020. This is consistent (102 classes × 10 = 1,020), but the formatting is confusing because the other datasets show per-class counts that are more intuitive. A brief note clarifying this constraint would help.
  2. §4, Hyperparameters: The learning rate 'varies between 5·10^{-2} and 10^{-4},' but it is unclear which architecture uses which rate. Specifying the per-architecture learning rate would improve reproducibility.
  3. Figure 1: The y-axis ranges differ across subfigures (e.g., panel (a) starts at 20, panel (b) starts at 0), which can visually exaggerate or compress differences. Consider using a consistent axis range or clearly marking the broken axis.
  4. §2, Related Work: The distinction between few-medoids and the medium-difficulty approach of Chen et al. [6] is discussed qualitatively but not empirically. Including medium-difficulty selection as a baseline would strengthen the comparison, or the authors should explain why they chose not to include it.
  5. Algorithm 1, line 4: The average distance is computed as (1/m) * sum over j=1..m, which includes the distance from x_i to itself (zero). This does not affect the ranking but is technically a minor bias. The normalization could use m−1.
  6. The term 'embarrassingly simple' appears in the title, abstract, and body. While not incorrect, it is informal for a journal submission. Consider 'A Simple Coreset Selection Method' or similar. This would improve readability.

Simulated Author's Rebuttal

3 responses · 0 unresolved

We thank the referee for a careful and constructive review. The referee raises three major points: (1) the abstract and conclusion overstate the scope of our 'consistently surpasses' claim relative to the ViT-B/16→ViT-Small results, (2) the mechanistic explanation for the from-scratch vs. pre-trained student distinction is speculative without a controlled ablation, and (3) no statistical significance tests are provided despite some margins falling within one standard deviation. We agree with all three points and will revise the manuscript accordingly.

read point-by-point responses

  1. Referee: The abstract and conclusion overstate the 'consistently surpasses' claim, which is contradicted by the ViT-B/16→ViT-Small results where herding dominates for k≥4.

    Authors: The referee is correct. Our abstract and conclusion currently state that few-medoids 'consistently surpasses' all baselines and recommend it as a 'drop-in replacement,' but the ViT-B/16→ViT-Small results in Tables 3–6 clearly show herding dominating for k≥4 across all four datasets, often by large margins. This is a genuine overstatement of scope. We will revise both the abstract and conclusion to explicitly scope the claim: few-medoids consistently outperforms baselines when the student is trained from scratch (ResNet-34→ResNet-18, ViT-B/16→ResNet-50), while for pre-trained students (ViT-B/16→ViT-Small), herding is generally superior for k≥4 and few-medoids is competitive only for k≤2. The phrase 'drop-in replacement' will be qualified or removed. revision: yes

  2. Referee: The post-hoc explanation for the ViT-B/16→ViT-Small results (that few-medoids excels for from-scratch students because their latent space is not semantically organized a priori) is a load-bearing mechanistic claim with no controlled experiment to test it.

    Authors: We agree that this explanation is currently speculative. The distinction between from-scratch and pre-trained student regimes is central to scoping our contribution, and a controlled experiment is the right way to test it. We will conduct an ablation in which we initialize ViT-Small from scratch (rather than from pre-trained weights) for a fixed teacher (ViT-B/16), keeping all other hyperparameters constant, and compare few-medoids against herding under both initialization regimes. This will directly test whether the advantage of few-medoids is attributable to the selection method interacting with training dynamics, or to the student's initialization. If the experiment confirms our hypothesis, we will report it as empirical evidence rather than speculation; if it does not, we will revise the explanation accordingly. In either case, we will temper the language in §4 from a mechanistic claim to a hypothesis pending the ablation results. revision: yes

  3. Referee: No statistical significance tests are provided, and several reported wins have margins within one standard deviation, particularly on Oxford Flowers 102 where rankings are described as 'unstable.'

    Authors: This is a fair point. Several of our reported margins are indeed within one standard deviation, and we should not claim 'consistent' superiority without formal testing. We will add paired statistical tests (e.g., paired t-tests or Wilcoxon signed-rank tests across the five seeds) for the key comparisons between few-medoids and each baseline, for each (dataset, architecture pair, k) setting. We will report p-values alongside the existing accuracy tables or in a supplementary table. We will also explicitly acknowledge in the text which wins are statistically significant and which are not, particularly for Oxford Flowers 102 where we already note instability. This will make the empirical claims more precise and honest about where the evidence is strong versus marginal. revision: yes

Circularity Check

0 steps flagged

No circularity: few-medoids is a parameter-free geometric selection method with no fitted constants and no self-citation load-bearing chain.

full rationale

The paper's central method, few-medoids, is defined in Algorithm 1 and Eq. (1) as selecting per-class samples with the lowest average L2 distance to all same-class samples in the teacher's frozen latent space. No parameters are fitted to the evaluation data: the method is parameter-free (using only the teacher's frozen features and L2 distance), and the KD hyperparameters (τ=4, λ1=0.6, λ2=0.4) are fixed across all experiments and not tuned per dataset or per method. The 'prediction' (which samples form the best coreset) is not equivalent to a fitted quantity by construction. Self-citations exist (Croitoru et al. [7], Ionescu et al. [16, 22]) but they are contextual references to prior work on diffusion models and few-shot distillation, not load-bearing justifications for the few-medoids method itself. The method is defined from first principles (geometric centrality via average L2 distance) and evaluated against external baselines (random, herding, k-center Greedy, PCA-guided matching) on independent benchmarks. The derivation chain is self-contained with no circular reduction.

Axiom & Free-Parameter Ledger

4 free parameters · 4 axioms · 0 invented entities

The method introduces no new entities, particles, forces, or postulated objects. It operates entirely within the standard KD framework using existing teacher features and Euclidean geometry. The free parameters (τ, λ1, λ2) are standard KD hyperparameters fixed across all experiments, not fitted to the evaluation data. The axiom set is small and consists of domain assumptions about the usefulness of teacher-feature-space geometry, which are tested empirically rather than assumed circularly.

free parameters (4)
  • τ (distillation temperature) = 4
    Set globally across all experiments; not tuned per dataset or model pair.
  • λ1 (KD loss weight) = 0.6
    Set globally; controls the relative weight of distillation vs. cross-entropy.
  • λ2 (CE loss weight) = 0.4
    Set globally; controls the relative weight of cross-entropy vs. distillation.
  • k (samples per class) = {1,2,4,8,16,32,64,128}
    Swept over a range; not a fitted constant but an experimental variable.
axioms (4)
  • domain assumption Euclidean distance in the teacher's latent feature space is a meaningful metric for sample centrality.
    Invoked in §3, Algorithm 1, line 4. The entire method depends on L2 distance being the right metric; no alternative metrics are tested.
  • domain assumption Samples near the class centroid in teacher-feature-space provide a better distillation signal than diverse or random samples.
    Invoked in §1 and §5. This is the core hypothesis; it is tested empirically but not derived from theory.
  • standard math Standard KD loss (Hinton et al. [21]) with fixed temperature and loss weights is an appropriate distillation framework.
    Invoked in §3, Eqs. (2)-(4). Standard KD formulation; not a paper-specific assumption.
  • domain assumption The teacher fine-tuned on the target dataset provides useful features for coreset selection.
    Invoked in §4, 'Underlying neural models' paragraph. The teacher is fine-tuned on the full target dataset before feature extraction; this is standard but assumes the teacher's features are informative for the target task.

pith-pipeline@v1.1.0-glm · 18081 in / 2873 out tokens · 428305 ms · 2026-07-08T21:14:28.760361+00:00 · methodology

0 comments
read the original abstract

Coreset selection aims to identify a small and highly representative subset of a massive dataset for efficient model training. The problem remains challenging even in the few-shot knowledge distillation (KD) setup, where a full-scale pre-trained teacher informs the student network. Typical sample selection strategies often struggle to surpass the random selection baseline. In this paper, we showcase few-medoids, an embarrassingly simple coreset selection strategy that chooses the samples closest to the centroid (average image) of each class. We present extensive KD experiments on four datasets, covering a wide range of image classification problems, and three teacher-student model pairs, comprising both convolutional and transformer networks. Although the proposed method is embarrassingly simple, our empirical results indicate that few-medoids is able to consistently surpass the random selection baseline, as well as the other coreset selection strategies. We therefore consider that few-medoids can be used as a drop-in replacement for commonly-used baselines (e.g. herding or k-center Greedy), in future research on coreset selection. To reproduce the reported results, we publicly release our code at https://github.com/CemilAndreiDilmac/Few-Shot-KD-Coreset.

Figures

Figures reproduced from arXiv: 2607.05891 by Cemil-Andrei Dilmac, Florinel-Alin Croitoru, Radu Tudor Ionescu.

Figure 1
Figure 1. Figure 1: Quantitative comparison for the ResNet-34 [PITH_FULL_IMAGE:figures/full_fig_p010_1.png] view at source ↗
Figure 2
Figure 2. Figure 2: t-SNE visualization of ResNet-18 embeddings ( [PITH_FULL_IMAGE:figures/full_fig_p010_2.png] view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

55 extracted references · 55 canonical work pages · 4 internal anchors

  1. [1]

    Geometric approximation via coresets, in: Combinatorial and Computational Geom- etry

    Agarwal, P.K., Har-Peled, S., Varadarajan, K.R., 2005. Geometric approximation via coresets, in: Combinatorial and Computational Geom- etry. Cambridge University Press, pp. 1–30

    work page 2005

  2. [2]

    Data pruning and neural scaling laws: fundamental limitations of score-based algorithms

    Ayed, F., Hayou, S., 2023. Data pruning and neural scaling laws: fundamental limitations of score-based algorithms. Transactions on Machine Learning Research

    work page 2023

  3. [3]

    Distilling the knowledge in data pruning, in: Proceedings of ICML, pp

    Baruch, E.B., Botach, A., Kviatkovsky, I., Aggarwal, M., Medioni, G., 2025. Distilling the knowledge in data pruning, in: Proceedings of ICML, pp. 3659–3676

    work page 2025

  4. [4]

    Food-101 – Mining Discriminative Components with Random Forests, in: Proceedings of ECCV , pp

    Bossard, L., Guillaumin, M., Van Gool, L., 2014. Food-101 – Mining Discriminative Components with Random Forests, in: Proceedings of ECCV , pp. 446–461

    work page 2014

  5. [5]

    Super-samples from kernel herding, in: Proceedings of UAI, pp

    Chen, Y ., Welling, M., Smola, A., 2010. Super-samples from kernel herding, in: Proceedings of UAI, pp. 109–116

    work page 2010

  6. [6]

    Medium-difficulty samples constitute smoothed decision boundary for knowledge distillation on pruned datasets, in: Proceedings of ICLR

    Chen, Y ., Xu, X., de Hoog, F., Liu, J., Wang, S., 2025. Medium-difficulty samples constitute smoothed decision boundary for knowledge distillation on pruned datasets, in: Proceedings of ICLR

    work page 2025

  7. [7]

    Diffusion models in vision: A survey

    Croitoru, F.A., Hondru, V ., Ionescu, R.T., Shah, M., 2023. Diffusion models in vision: A survey. IEEE Tranactions on Pattern Analysis and Machine Intelligence 45, 10850–10869

    work page 2023

  8. [8]

    DA-NAS: Data Adapted Pruning for Efficient Neural Architecture Search, in: Proceedings of ECCV , pp

    Dai, X., Chen, D., Liu, M., Chen, Y ., Yuan, L., 2020. DA-NAS: Data Adapted Pruning for Efficient Neural Architecture Search, in: Proceedings of ECCV , pp. 584–600

    work page 2020

  9. [9]

    An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale, in: Proceedings of ICLR

    Dosovitskiy, A., Beyer, L., Kolesnikov, A., Weissenborn, D., Zhai, X., Unterthiner, T., Dehghani, M., Minderer, M., Heigold, G., Gelly, S., Uszkoreit, J., Houlsby, N., 2021. An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale, in: Proceedings of ICLR

    work page 2021

  10. [10]

    Adversarial Active Learning for Deep Networks: a Margin Based Approach

    Ducoffe, M., Precioso, F., 2018. Adversarial active learning for deep networks: a margin based approach. arXiv preprint arXiv:1802.09841

    work page internal anchor Pith review Pith/arXiv arXiv 2018

  11. [11]

    MMTEB: Massive multilingual text embedding benchmark, in: Proceedings of ICLR

    Enevoldsen, K., Chung, I., Kerboua, I., Kardos, M., Mathur, A., Stap, D., Gala, J., Siblini, W., et al., 2025. MMTEB: Massive multilingual text embedding benchmark, in: Proceedings of ICLR

    work page 2025

  12. [12]

    GIO: Gradient Information Optimization for Training Dataset Selection, in: Proceedings of ICLR

    Everaert, D., Potts, C., 2024. GIO: Gradient Information Optimization for Training Dataset Selection, in: Proceedings of ICLR

    work page 2024

  13. [13]

    LLMCarbon: Modeling the end-to-end carbon footprint of large language models, in: Proceedings of ICLR

    Faiz, A., Kaneda, S., Wang, R., Osi, R.C., Sharma, P., Chen, F., Jiang, L., 2024. LLMCarbon: Modeling the end-to-end carbon footprint of large language models, in: Proceedings of ICLR. 7 Table 5: Few-shot KD test accuracy (%) on Oxford Flowers 102, fork∈ {1,2,4,8}. Average accuracy rates and standard deviations are computed over five runs with different s...

    work page 2024

  14. [14]

    Efficient knowledge distillation from an ensemble of teachers, in: Proceedings of INTERSPEECH, pp

    Fukuda, T., Suzuki, M., Kurata, G., Thomas, S., Cui, J., Ramabhadran, B., 2017. Efficient knowledge distillation from an ensemble of teachers, in: Proceedings of INTERSPEECH, pp. 3697–3701

    work page 2017

  15. [15]

    Zero-shot coreset selection via iterative subspace sampling, in: Proceedings of W ACV , pp

    Griffin, B.A., Marks, J., Corso, J.J., 2026. Zero-shot coreset selection via iterative subspace sampling, in: Proceedings of W ACV , pp. 2114–2124

    work page 2026

  16. [16]

    Weight copy and low-rank adaptation for few-shot distillation of vision transformers, in: Proceedings of W ACV , pp

    Grigore, D.N., Georgescu, M.I., Justo, J.A., Johansen, T., Ionescu, A.I., Ionescu, R.T., 2025. Weight copy and low-rank adaptation for few-shot distillation of vision transformers, in: Proceedings of W ACV , pp. 7368–7378

    work page 2025

  17. [17]

    DeepCore: A Comprehensive Library for Coreset Selection in Deep Learning, in: Proceedings of DEXA, pp

    Guo, C., Zhao, B., Bai, Y ., 2022. DeepCore: A Comprehensive Library for Coreset Selection in Deep Learning, in: Proceedings of DEXA, pp. 181–195

    work page 2022

  18. [18]

    FedCS: Coreset Selection for Federated Learning, in: Proceedings of CVPR, pp

    Hao, C., Xie, W., Li, D., Qin, H., Ye, H., Fang, L., Li, Y ., 2025. FedCS: Coreset Selection for Federated Learning, in: Proceedings of CVPR, pp. 15434–15443

    work page 2025

  19. [19]

    Deep Residual Learning for Image Recognition, in: Proceedings of CVPR, pp

    He, K., Zhang, X., Ren, S., Sun, J., 2016. Deep Residual Learning for Image Recognition, in: Proceedings of CVPR, pp. 770–778

    work page 2016

  20. [20]

    Large-scale dataset pruning with dynamic uncertainty, in: Proceedings of CVPR, pp

    He, M., Yang, S., Huang, T., Zhao, B., 2024. Large-scale dataset pruning with dynamic uncertainty, in: Proceedings of CVPR, pp. 7713–7722

    work page 2024

  21. [21]

    Distilling the Knowledge in a Neural Network

    Hinton, G., Vinyals, O., Dean, J., 2015. Distilling the Knowledge in a Neural Network. arXiv preprint arXiv:1503.02531

    work page internal anchor Pith review Pith/arXiv arXiv 2015

  22. [22]

    Towards few-call model stealing via active self-paced knowledge distillation and diffusion-based image generation

    Hondru, V ., Ionescu, R.T., 2025. Towards few-call model stealing via active self-paced knowledge distillation and diffusion-based image generation. Artificial Intelligence Review 58, 254

    work page 2025

  23. [23]

    Active learning by learning, in: Proceedings of AAAI, pp

    Hsu, W.N., Lin, H.T., 2015. Active learning by learning, in: Proceedings of AAAI, pp. 2659–2665

    work page 2015

  24. [24]

    GLISTER: Generalization based Data Subset Selection for Efficient and Robust Learning, in: Proceedings of AAAI, pp

    Killamsetty, K., Sivasubramanian, D., Ramakrishnan, G., Iyer, R., 2021. GLISTER: Generalization based Data Subset Selection for Efficient and Robust Learning, in: Proceedings of AAAI, pp. 8110–8118

    work page 2021

  25. [25]

    Learning Multiple Layers of Features from Tiny Images

    Krizhevsky, A., Hinton, G., 2009. Learning Multiple Layers of Features from Tiny Images. Technical Report 0. University of Toronto. Toronto, Ontario. URL:https://www.cs.toronto.edu/ ~kriz/learning-features-2009-TR.pdf

    work page 2009

  26. [26]

    Coreset selection for object detection, in: Proceedings of CVPRW, pp

    Lee, H., Kim, S., Lee, J., Yoo, J., Kwak, N., 2024. Coreset selection for object detection, in: Proceedings of CVPRW, pp. 7682–7691

    work page 2024

  27. [27]

    Making AI Less ‘Thirsty’

    Li, P., Yang, J., Islam, M.A., Ren, S., 2025. Making AI Less ‘Thirsty’. Communications of the ACM 68, 54–61

    work page 2025

  28. [28]

    Swin Transformer: Hierarchical Vision Transformer using Shifted Windows, in: Proceedings of ICCV , pp

    Liu, Z., Lin, Y ., Cao, Y ., Hu, H., Wei, Y ., Zhang, Z., Lin, S., Guo, B., 2021. Swin Transformer: Hierarchical Vision Transformer using Shifted Windows, in: Proceedings of ICCV , pp. 9992–10002

    work page 2021

  29. [29]

    Large multimodal models for low-resource languages: a survey

    Lupas ¸cu, M., Rogoz, A.C., Stupariu, M.S., Ionescu, R.T., 2026. Large multimodal models for low-resource languages: a survey. Information Fusion 131, 104189

    work page 2026

  30. [30]

    D2 Pruning: Message Passing for Balancing Diversity and Difficulty in Data Pruning, in: Proceedings of ICLR

    Maharana, A., Yadav, P., Bansal, M., 2024. D2 Pruning: Message Passing for Balancing Diversity and Difficulty in Data Pruning, in: Proceedings of ICLR

    work page 2024

  31. [31]

    Coresets for data-efficient training of machine learning models, in: Proceedings of ICML, pp

    Mirzasoleiman, B., Bilmes, J., Leskovec, J., 2020. Coresets for data-efficient training of machine learning models, in: Proceedings of ICML, pp. 6950–6960

    work page 2020

  32. [32]

    A coreset selection of coreset selection literature: Introduction and recent advances

    Moser, B.B., Shanbhag, A.S., Frolov, S., Raue, F., Folz, J., Dengel, A., 2026. A coreset selection of coreset selection literature: Introduction and recent advances. arXiv preprint arXiv:2505.17799

    work page arXiv 2026

  33. [33]

    MTEB: Massive text embedding benchmark, in: Proceedings of ACL, pp

    Muennighoff, N., Tazi, N., Magne, L., Reimers, N., 2023. MTEB: Massive text embedding benchmark, in: Proceedings of ACL, pp. 2014– 2037

    work page 2023

  34. [34]

    Automated Flower Classification over a Large Number of Classes, in: Proceedings of ICVGIP, pp

    Nilsback, M.E., Zisserman, A., 2008. Automated Flower Classification over a Large Number of Classes, in: Proceedings of ICVGIP, pp. 722–729

    work page 2008

  35. [35]

    Active learning by feature mixing, in: Proceedings of CVPR, pp

    Parvaneh, A., Abbasnejad, E., Teney, D., Haffari, G.R., van den Hengel, A., Shi, J.Q., 2022. Active learning by feature mixing, in: Proceedings of CVPR, pp. 12237–12246. 8 Table 6: Few-shot KD test accuracy (%) on Food-101, fork∈ {1,2,4,8,16,32,64,128}. Average accuracy rates and standard deviations are computed over five runs with different seeds.Bold gr...

    work page 2022

  36. [36]

    Deep learning on a data diet: Finding important examples early in training, in: Proceedings of NeurIPS, pp

    Paul, M., Ganguli, S., Dziugaite, G.K., 2021. Deep learning on a data diet: Finding important examples early in training, in: Proceedings of NeurIPS, pp. 20596–20607

    work page 2021

  37. [37]

    Direct Preference Optimization: your language model is secretly a reward model, in: Proceedings of NeurIPS, pp

    Rafailov, R., Sharma, A., Mitchell, E., Ermon, S., Manning, C.D., Finn, C., 2023. Direct Preference Optimization: your language model is secretly a reward model, in: Proceedings of NeurIPS, pp. 53728–53741

    work page 2023

  38. [38]

    iCaRL: Incremental Classifier and Representation Learning, in: Proceedings of CVPR, pp

    Rebuffi, S.A., Kolesnikov, A., Sperl, G., Lampert, C.H., 2017. iCaRL: Incremental Classifier and Representation Learning, in: Proceedings of CVPR, pp. 5533–5542

    work page 2017

  39. [39]

    High-resolution image synthesis with latent diffusion models, in: Proceedings of CVPR, pp

    Rombach, R., Blattmann, A., Lorenz, D., Esser, P., Ommer, B., 2022. High-resolution image synthesis with latent diffusion models, in: Proceedings of CVPR, pp. 10684–10695

    work page 2022

  40. [40]

    Green AI

    Schwartz, R., Dodge, J., Smith, N.A., Etzioni, O., 2020. Green AI. Communications of the ACM 63, 54–63

    work page 2020

  41. [41]

    Active learning for convolutional neural networks: A core-set approach, in: Proceedings of ICLR

    Sener, O., Savarese, S., 2018. Active learning for convolutional neural networks: A core-set approach, in: Proceedings of ICLR

    work page 2018

  42. [42]

    Beyond neural scaling laws: Beating power law scaling via data pruning, in: Proceedings of NeurIPS, pp

    Sorscher, B., Geirhos, R., Shekhar, S., Ganguli, S., Morcos, A.S., 2022. Beyond neural scaling laws: Beating power law scaling via data pruning, in: Proceedings of NeurIPS, pp. 19523–19536

    work page 2022

  43. [43]

    Data pruning by information maximization, in: Proceedings of ICLR

    Tan, H., Wu, S., Huang, W., Zhao, S., QI, X., 2025. Data pruning by information maximization, in: Proceedings of ICLR

    work page 2025

  44. [44]

    An Empirical Study of Example Forgetting during Deep Neural Network Learning

    Toneva, M., Sordoni, A., Tachet des Combes, R., Trischler, A., Bengio, Y ., Gordon, G.J., 2018. An empirical study of example forgetting during deep neural network learning. arXiv preprint arXiv:1812.05159

    work page internal anchor Pith review Pith/arXiv arXiv 2018

  45. [45]

    LLaMA: Open and Efficient Foundation Language Models

    Touvron, H., Lavril, T., Izacard, G., Martinet, X., Lachaux, M.A., Lacroix, T., Rozi `ere, B., Goyal, N., Hambro, E., Azhar, F., Rodriguez, A., Joulin, A., Grave, E., Lample, G., 2023. Llama: Open and efficient foundation language models. arXiv preprint arXiv 2302.13971

    work page internal anchor Pith review Pith/arXiv arXiv 2023

  46. [46]

    Efficient Few-Shot Learning for Edge AI via Knowledge Distillation on MobileViT

    Tsuyuki, S., Bensaid, R., Morlier, J., L ´eonardon, M., Onizawa, N., Gripon, V ., Hanyu, T., 2026. Efficient Few-Shot Learning for Edge AI via Knowledge Distillation on MobileViT. arXiv preprint arXiv:2603.26145

    work page arXiv 2026

  47. [47]

    Attention is all you need, in: Proceedings of NeurIPS, pp

    Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A.N., Kaiser, L.u., Polosukhin, I., 2017. Attention is all you need, in: Proceedings of NeurIPS, pp. 6000–6010

    work page 2017

  48. [48]

    DRoP: Distributionally Robust Data Pruning, in: Proceedings of ICLR

    Vysogorets, A., Ahuja, K., Kempe, J., 2025. DRoP: Distributionally Robust Data Pruning, in: Proceedings of ICLR

    work page 2025

  49. [49]

    Herding dynamical weights to learn, in: Proceedings of ICML, pp

    Welling, M., 2009. Herding dynamical weights to learn, in: Proceedings of ICML, pp. 1121–1128

    work page 2009

  50. [50]

    Moderate Coreset: A Universal Method of Data Selection for Real-World Data- Efficient Deep Learning, in: Proceedings of ICLR

    Xia, X., Liu, J., Yu, J., Shen, X., Han, B., Liu, T., 2023. Moderate Coreset: A Universal Method of Data Selection for Real-World Data- Efficient Deep Learning, in: Proceedings of ICLR

    work page 2023

  51. [51]

    Dataset Pruning: Reducing Training Data by Examining Generalization Influence, in: Proceedings of ICLR

    Yang, S., Xie, Z., Peng, H., Xu, M., Sun, M., Li, P., 2023. Dataset Pruning: Reducing Training Data by Examining Generalization Influence, in: Proceedings of ICLR

    work page 2023

  52. [52]

    Decoupled knowledge distillation, in: Proceedings of CVPR, pp

    Zhao, B., Cui, Q., Song, R., Qiu, Y ., Liang, J., 2022. Decoupled knowledge distillation, in: Proceedings of CVPR, pp. 11953–11962

    work page 2022

  53. [53]

    Coverage-centric coreset selection for high pruning rates, in: Proceedings of ICLR

    Zheng, H., Liu, R., Lai, F., Prakash, A., 2023. Coverage-centric coreset selection for high pruning rates, in: Proceedings of ICLR

    work page 2023

  54. [54]

    ELFS: Label-free coreset selection with proxy training dynamics, in: Proceedings of ICLR

    Zheng, H., Tsai, E., Lu, Y ., Sun, J., Bartoldson, B.R., Kailkhura, B., Prakash, A., 2025. ELFS: Label-free coreset selection with proxy training dynamics, in: Proceedings of ICLR. 9 1 2 4 8 16 32 64 128 k (samples per class) 20 30 40 50 60 70 80 90T est Accuracy (%) Full Training (94.03%) Random K-Center Herding PCA-Guided Matching Few-Medoids (a) Few-sh...

    work page 2025

  55. [55]

    Few-medoids (right) generates a more discriminative latent space than random coreset selection (left)

    Different colors represent different food categories. Few-medoids (right) generates a more discriminative latent space than random coreset selection (left). Best viewed in color. 10

    work page