Pith. sign in

REVIEW 4 major objections 5 minor 26 references

Prototypical networks trained on biology-pretrained embeddings exceed the FungiCLEF 2025 baseline by more than 30 points in Recall@5.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Using prototypical networks with BioCLIP embeddings and validation-set support, the authors exceed FungiCLEF 2025 baselines by over 30 points in Recall@5.

T0 review reviewed 2026-08-04 challenge →

load-bearing objection Large gain on FungiCLEF 2025, but headline 30-point claim rests on using validation labels at inference; worth a referee with a protocol check. the 4 major comments →

arxiv 2509.11020 v1 pith:QZXTDLUM submitted 2025-09-14 cs.CV

Improving Fungi Prototype Representations for Few-Shot Classification

Abdarahmane Traore , \'Eric Hervet , Andy Couturier This is my paper
classification cs.CV
keywords few-shot classificationprototypical networksfungi recognitionFungiCLEF 2025Recall@5rare speciesepisodic trainingvision-language pretraining
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper claims that few-shot fungal species recognition improves strongly when images are mapped to a learned embedding space and each species is represented by the average embedding of its few labeled photos, called a prototype. On the FungiCLEF 2025 challenge, where most species have only one to four training images and roughly a fifth of observations are rare species, this prototypical-network approach beats the organizers' retrieval baselines by more than 30 percentage points in Recall@5 on both leaderboards. The authors compare prototype averaging, caption-supervised finetuning, and episodic prototypical networks, and find that episodic training with a biology-pretrained encoder works best. They also report that adding validation-set images to the support set helps, especially for rare classes. A reader should care because reliable top-5 identification of rare fungi from field photos would directly support biodiversity monitoring and citizen-science platforms.

Core claim

The paper's central claim is that episodic training of a prototypical network—sampling episodes of 75 species with 3 support images each—produces prototypes that separate the 2,427-species embedding space well enough that the true species lands in the top five predictions for roughly 64% of public-leaderboard queries and 58% of private-leaderboard queries. The authors attribute the gain to learning a representation where visually similar, taxonomically related species sit close together, so that when the exact species is missed, a close relative is still in the top five. The best configuration uses a vision encoder pretrained on biological image–text data, stochastic weight averaging, and a

What carries the argument

The central mechanism is the prototypical network: in each training episode, K species (called ways) and S support photos per species (called shots) are sampled; a shared encoder embeds all images, each class prototype is the mean of its support embeddings, and query images are classified by nearest prototype. The paper's best model uses K=75 ways, S=3 shots, Q=1 query, a biology-pretrained vision encoder, and 750 training episodes with stochastic weight averaging; at inference, prototypes are built from training plus validation images and queries are scored by cosine similarity. This episodic objective forces the embedding to place same-species images close together and different-species im

Load-bearing premise

The load-bearing premise, stated in Section 4, is that validation-split images and labels can be added to the support set when forming prototypes at inference time; if the competition rules or a real deployment forbid using those labels, the reported 30-point margin may shrink or disappear.

What would settle it

Retrain the paper's best configuration with the support set restricted to the training split only, exactly matching the official baseline protocol, and measure Recall@5 on the same private test set. If the score falls from about 0.58 to roughly the 0.33 level of the embedding-retrieval baseline, the claimed lead is caused by validation-label augmentation rather than by prototypical-network training.

Watch this falsifier. Get emailed when new claim-graph text bears on it.

If this is right

  • If the claim holds, an off-the-shelf biology-pretrained encoder plus a modest amount of episodic training can reach top-5 accuracy above 60% in a 2,427-class few-shot task, without per-class fine-tuning datasets.
  • Adding newly labeled images to the support set—even images not used during training—improves rare-class prototypes, so a deployed system could grow more accurate as citizen scientists contribute identifications.
  • The consistent ranking of methods from public to private splits suggests the improvement is not leaderboard overfitting, so the same configuration should transfer to unseen locations and observers.
  • High-quality captions from large language models improve caption-supervised finetuning, but less than the prototypical-network mechanism, implying that embedding geometry matters more than caption supervision alone for this task.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • Editorial extension: the paper leaves the size of the validation-augmentation effect unquantified as a standalone ablation; isolating it would show how much of the 30-point gain comes from the prototypical network itself versus access to extra labeled examples.
  • Editorial extension: because the embedding groups morphologically similar species, reranking top-5 candidates with metadata such as substrate, habitat, season, or location is a natural next step the paper does not test.
  • Editorial extension: the same prototype-based recipe is likely to transfer to other long-tailed citizen-science domains such as insects, plants, or marine invertebrates, since the method makes no fungi-specific assumption beyond the pretrained encoder's biological grounding.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. The paper describes a few-shot fungi classification system for the FungiCLEF 2025 challenge. The authors compare three families of methods: prototype averaging with frozen pretrained encoders (BioCLIP, OpenCLIP variants), CLIP finetuning with generated captions, and episodic prototypical networks. Their best result is obtained with a BioCLIP-based prototypical network trained with K=75, S=3, Q=1 on the combined train/validation split, achieving Recall@5 of 0.64159 on the public leaderboard and 0.58473 on the private leaderboard. The paper claims that this exceeds the competition's FAISS prototype baseline by more than 30 percentage points on both leaderboards. The authors also provide UMAP visualizations of the learned prototype space and release source code.

Significance. If the central claim holds, the paper demonstrates a practical and simple prototypical-network recipe that substantially improves few-shot recognition on a realistic, long-tailed biodiversity dataset. The public code and the systematic comparison across prototype averaging, finetuned CLIP, and episodic training are useful contributions to the working-notes literature. However, the headline >30-point gain is achieved only when validation labels are used to construct support prototypes at inference, while the organizers' baselines were restricted to the train split. Whether this is permitted by the competition protocol is not established, and the ablation does not isolate the contribution of the validation data. These issues make the central claim currently under-supported, though plausibly fixable with additional experiments and clarification.

major comments (4)
  1. [Section 4, Table 3] The claim of 'more than 30 percentage points' on both leaderboards depends on the train/val support set. The best row (K=75, train/val) gives 0.64159 PB and 0.58473 PR, which are 30.97pp and 31.82pp above the FAISS prototype baseline (0.33185 PB, 0.26649 PR). The closest train-only row (K=70) gives 0.61061 PB and 0.57179 PR, i.e., only 27.88pp above baseline on PB. The comparison against the organizers' baselines is therefore not controlled, since the paper states the organizers restricted baselines to the train split only. The authors should (a) cite the relevant competition rule that permits using validation labels at inference, or explicitly state whether this was allowed; (b) report train-only and train/val comparisons under identical K/S/Q; and (c) if validation labels are not permitted, revise the headline claim.
  2. [Section 4.3, Table 3] The ablation intended to show the benefit of validation data is confounded. The train-only row uses K=70 and 500 episodes, while the train/val row uses K=75 and 750 episodes. The observed gain could be due to increased K, longer training, or the validation images. A controlled comparison with the same K, S, Q, and number of episodes is needed to attribute the improvement to the validation split. As written, the paper does not demonstrate the marginal value of validation data.
  3. [Section 4.1, Table 1] The text states that 'augmenting prototypes with both training and validation data led to improved Recall@5 on both PB and PR,' but Table 1 does not show train-only versus train/val results for prototype averaging. Without that comparison, the claim is unsupported. The table should include both configurations, or the text should be revised.
  4. [Section 4.3, Tables 1-3] All results are single-run leaderboard scores with no error bars, multiple seeds, or variance estimates. Hyperparameters (K, S, Q, training length) appear to have been selected after observing leaderboard results. Given that some PB/PR differences are small (e.g., ViT-B-16-SigLIP-512: 0.53539 vs 0.53298), the reported margins and the >30pp claim should be accompanied by a stability analysis or at least a statement of protocol for hyperparameter selection. This is particularly important because the central claim is a quantitative margin over a baseline.
minor comments (5)
  1. [Section 3.2] Sentence 'For the main evaluation, sets k=5' is grammatically incomplete; should read 'the main evaluation metric uses k=5' or similar.
  2. [Section 4.1 / Table 1] The statement that validation augmentation improved results is not visible in the table; consider adding explicit columns for 'train' and 'train+val' for each model.
  3. [Figure 3] The UMAP visualization is qualitative; the text should note that it serves as an illustration, not as quantitative evidence of clustering quality.
  4. [References] Reference [10] appears malformed (URL and title are garbled). Several URLs have OCR artifacts (e.g., 'envel⌢pe-⌢peneat4651@umoncton.ca').
  5. [Throughout] Occasional capitalization/formatting inconsistencies: 'Menispora Caesia' should be 'Menispora caesia'; 'Recall@k' and 'Recall@K' are used interchangeably.

Circularity Check

0 steps flagged

No significant circularity: the reported gains are measured against external baselines and leaderboards, not derived from the paper's own assumptions.

full rationale

The paper reports a few-shot classification system and compares it against external baselines (BioCLIP+FAISS+NN, BioCLIP+FAISS+Prototypes) and external leaderboards (public and private). The central claim—exceeding the competition baseline by more than 30 percentage points in Recall@5—is an empirical result evaluated on held-out test data, not a quantity derived from the paper's own definitions or fitted parameters. The method is a standard prototypical network with a pretrained BioCLIP encoder, following Snell et al. and Radford et al.; these are independent, established methods, and the paper does not invoke a uniqueness theorem or a self-citation chain to justify its design. The main methodological weakness is that the best configuration ('train/val') uses validation split images and labels to construct class prototypes at inference, while the organizers' baselines used only training data. This is a legitimate concern about protocol permissibility and fair comparison, but it is not circularity: the paper does not define its prediction target in terms of the validation labels, nor does it fit a parameter to the leaderboard and then repackage that fit as a prediction. The reported differences between configurations are empirical outcomes, not identities forced by construction. Hyperparameter choices (K, S, Q, training length) may have been influenced by public leaderboard feedback, but this is a post-hoc selection issue, not a circular derivation. No step in the paper reduces, by equation or self-citation, to its own inputs. Therefore the circularity score is 0.

Axiom & Free-Parameter Ledger

2 free parameters · 2 axioms · 0 invented entities

The method rests on standard pretrained representations and episodic training assumptions, with no new theory. The main choices are hyperparameters and the decision to use validation data for prototypes.

free parameters (2)
  • Episodic training configuration (K, S, Q) = K=75, S=3, Q=1
    Chosen by experimentation to maximize Recall@5 on the public leaderboard; not derived from theory.
  • Training schedule hyperparameters = LR 1e-5, SWA LR 5e-5, 750 episodes, SWA start at 100 episodes before end
    Manually set based on common practice and validation performance; these are fitted to the task.
axioms (2)
  • domain assumption Pretrained BioCLIP encoder trained on TreeOfLife-10M provides useful embeddings for fungi species.
    The method relies on the pretrained encoder providing discriminative features for fungi. This is plausible but not proven in this paper.
  • domain assumption Episodic training with high K and fixed S=3, Q=1 generalizes to the FungiCLEF test distribution.
    The paper assumes episodic training with high K improves performance, following prior work, but does not establish this for the FungiCLEF data.

reviewed 2026-08-04 · how reviews work

0 comments
Cite this review

Pith. "Pith review of Improving Fungi Prototype Representations for Few-Shot Classification." pith.science (2026). https://pith.science/paper/QZXTDLUM

@misc{pith2026250911020,
  author       = {Pith},
  title        = {Pith review of: Improving Fungi Prototype Representations for Few-Shot Classification},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/QZXTDLUM}},
  note         = {Machine review of arXiv:2509.11020}
}
Share X Bluesky LinkedIn Reddit HN
read the original abstract

The FungiCLEF 2025 competition addresses the challenge of automatic fungal species recognition using realistic, field-collected observational data. Accurate identification tools support both mycologists and citizen scientists, greatly enhancing large-scale biodiversity monitoring. Effective recognition systems in this context must handle highly imbalanced class distributions and provide reliable performance even when very few training samples are available for many species, especially rare and under-documented taxa that are often missing from standard training sets. According to competition organizers, about 20\% of all verified fungi observations, representing nearly 20,000 instances, are associated with these rarely recorded species. To tackle this challenge, we propose a robust deep learning method based on prototypical networks, which enhances prototype representations for few-shot fungal classification. Our prototypical network approach exceeds the competition baseline by more than 30 percentage points in Recall@5 on both the public (PB) and private (PR) leaderboards. This demonstrates strong potential for accurately identifying both common and rare fungal species, supporting the main objectives of FungiCLEF 2025.

Figures

Figures reproduced from arXiv: 2509.11020 by Abdarahmane Traore, Andy Couturier, \'Eric Hervet.

Figure 1
Figure 1. Figure 1: Overview of the proposed architecture pipeline. For each class, a set of 𝑆 support images and one query image 𝑞 are encoded using the BioCLIP visual encoder to obtain image embeddings. Embeddings from the support set are averaged to form class prototypes. The distance between the query embedding and each class prototype is computed using the negative L2 metric, followed by a softmax operation to obtain cla… view at source ↗
Figure 2
Figure 2. Figure 2: Samples of Four Fungi Classes: Three representative images from each of four fungi classes (Agaricus arvensis, Crocicreas furvum, Geastrum elegans, and Menispora caesia), with one class per column. 3.2. Evaluation Protocol The primary evaluation metric for this competition is Recall@k, which measures the proportion of samples where the true species label appears among the top 𝑘 predicted labels for each ob… view at source ↗
Figure 3
Figure 3. Figure 3: UMAP embedding space of 2,427 fungal species prototypes from our best-performing model. The dense cluster region (red dashed rectangle) and mixed region (orange dashed rectangle) highlight areas of particular interest. Representative species from three automatically identified clusters (K-means) are marked with star, square, and triangle symbols. These regions demonstrate different embedding characteristic… view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

26 extracted references · 1 canonical work pages · 1 internal anchor

  1. [1]

    A. Joly, L. Picek, S. Kahl, H. Goëau, LifeCLEF 2024 teaser: challenges on species distribution prediction and identification, in: European Conference on Information Retrieval, Springer, 2024, pp. 18–25

    2024

  2. [2]

    Picek, M

    L. Picek, M. Šulc, J. Matas, J. Heilmann-Clausen, et al., Overview of FungiCLEF 2024: Revisiting fungi species recognition beyond 0-1 cost, in: Proceedings of the Working Notes of CLEF 2024 – Conference and Labs of the Evaluation Forum, volume 3740 ofCEUR Workshop Proceedings, 2024

    2024

  3. [3]

    Picek, S

    L. Picek, S. Kahl, H. Goëau, L. Adam, T. Larcher, C. Leblanc, M. Servajean, K. Janoušková, J. Matas, V. Čermák, K. Papafitsoros, R. Planqué, W.-P. Vellinga, H. Klinck, T. Denton, J. S. Cañas, G. Martel- lucci, F. Vinatier, P. Bonnet, A. Joly, Overview of lifeclef 2025: Challenges on species presence prediction and identification, and individual animal ide...

    2025

  4. [4]

    Picek, M

    L. Picek, M. Šulc, J. Matas, Overview of fungiclef 2024: Revisiting fungi species recognition beyond 0-1 cost, in: Working Notes of CLEF 2024 - Conference and Labs of the Evaluation Forum, volume 3740 ofCEUR Workshop Proceedings, CEUR-WS.org, 2024. URL: https://ceur-ws.org/Vol-3740/ paper-185.pdf

    2024

  5. [5]

    Picek, M

    L. Picek, M. Šulc, R. Chamidullin, J. Matas, Overview of fungiclef 2023: Fungi recognition beyond 0-1 cost, in: Working Notes of CLEF 2023 - Conference and Labs of the Evaluation Forum, volume 3495 ofCEUR Workshop Proceedings, CEUR-WS.org, 2023. URL: https://ceur-ws.org/Vol-3495/ paper-153.pdf

    2023

  6. [6]

    Picek, M

    L. Picek, M. Šulc, J. Matas, J. Heilmann-Clausen, Overview of fungiclef 2022: Fungi recognition as an open set classification problem, in: Working Notes of CLEF 2022 - Conference and Labs of the Evaluation Forum, volume 3180 ofCEUR Workshop Proceedings, CEUR-WS.org, 2022. URL: https://ceur-ws.org/Vol-3180/paper-157.pdf

    2022

  7. [7]

    Janoušková, J

    K. Janoušková, J. Matas, L. Picek, Overview of FungiCLEF 2025: Few-shot classification with rare fungi species, in: Working Notes of CLEF 2025 - Conference and Labs of the Evaluation Forum, 2025

    2025

  8. [8]

    Ozsari, E

    S. Ozsari, E. Kumru, F. Ekinci, I. Akata, M. S. Guzel, K. Acici, E. Ozcan, T. Asuroglu, Deep learning- based classification of macrofungi: Comparative analysis of advanced models for accurate fungi identification, Sensors 24 (2024) 7189

    2024

  9. [9]

    M. Arik, U. Kavas, B. Sezer, A mobile application for mushroom identification using deep learning and transfer learning, International Journal of Innovative Computing 13 (2023) 26–33

    2023

  10. [10]

    Kocagül, Classification of organisms: Pathogenicity classification of fungi (august 2023) classification lists, 2023

    M. Kocagül, Classification of organisms: Pathogenicity classification of fungi (august 2023) classification lists, 2023. URL: https://cogem.net/en/publication/ classification-of-organisms-pathogenicity-classification-of-fungi/, cGM/230828-03

    2023

  11. [11]

    Gaikwad, M

    A. Gaikwad, M. S. Bote, A novel computational framework for precision diagnosis and subtype discovery of plant with lesion, Frontiers in Plant Science 12 (2021) 789630

    2021

  12. [12]

    FungiTastic: A multi-modal dataset and benchmark for image categorization

    L. Picek, K. Janoušková, V. Čermák, J. Matas, Fungitastic: A multi-modal dataset and benchmark for image categorization (2024). doi:10.48550/ARXIV.2408.13632.arXiv:2408.13632

    work page internal anchor Pith review Pith/arXiv arXiv doi:10.48550/arxiv.2408.13632 2024

  13. [13]

    Ouhami, B

    M. Ouhami, B. El Asri, M. El Alami, Few-shot disease recognition algorithm based on supervised contrastive learning, Frontiers in Plant Science 13 (2022) 974343

    2022

  14. [14]

    Alfarrarjeh, M

    M. Alfarrarjeh, M. Ouhami, B. Bánhelyi, Analysis of few-shot techniques for fungal plant disease classification and evaluation of clustering capabilities over real datasets, Sensors 24 (2024) 1117

    2024

  15. [15]

    X. Su, H. Wang, B. Xu, Rare fungi species recognition based on transfer learning and vision transformer, in: Proceedings of the 2024 4th International Conference on Computer Science and Intelligent Control (CSIC 2024), ACM, 2024, pp. 535–540

    2024

  16. [16]

    Radford, J

    A. Radford, J. W. Kim, C. Hallacy, A. Ramesh, G. Goh, S. Agarwal, G. Sastry, A. Askell, P. Mishkin, J. Clark, G. Krueger, I. Sutskever, Learning transferable visual models from natural language supervision (2021). doi:10.48550/ARXIV.2103.00020.arXiv:2103.00020

    Pith/arXiv arXiv doi:10.48550/arxiv.2103.00020 2021

  17. [17]

    Snell, K

    J. Snell, K. Swersky, R. S. Zemel, Prototypical networks for few-shot learning (2017). doi: 10. 48550/ARXIV.1703.05175.arXiv:1703.05175

    Pith/arXiv arXiv doi:10.48550/arxiv.1703.05175 2017

  18. [18]

    Cherti, R

    M. Cherti, R. Beaumont, R. Wightman, M. Wortsman, G. Ilharco, C. Gordon, C. Schuhmann, L. Schmidt, J. Jitsev, Reproducible scaling laws for contrastive language-image learning, in: 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), IEEE, 2022, pp. 2818–2829. doi:10.1109/cvpr52729.2023.00276.arXiv:2212.07143

    arXiv 2023

  19. [19]

    Stevens, J

    S. Stevens, J. Wu, M. J. Thompson, E. G. Campolongo, C. H. Song, D. E. Carlyn, L. Dong, W. M. Dahdul, C. Stewart, T. Berger-Wolf, W.-L. Chao, Y. Su, Bioclip: A vision foundation model for the tree of life (2023). doi:10.48550/ARXIV.2311.18803.arXiv:2311.18803

    Pith/arXiv arXiv doi:10.48550/arxiv.2311.18803 2023

  20. [20]

    Douze, A

    M. Douze, A. Guzhva, C. Deng, J. Johnson, G. Szilvasy, P.-E. Mazaré, M. Lomeli, L. Hosseini, H. Jégou, The faiss library (2024).arXiv:2401.08281

    Pith/arXiv arXiv 2024

  21. [21]

    Deitke, C

    M. Deitke, C. Clark, S. Lee, R. Tripathi, Y. Yang, J. S. Park, M. Salehi, N. Muennighoff, K. Lo, L. Soldaini, J. Lu, T. Anderson, E. Bransom, K. Ehsani, H. Ngo, Y. Chen, A. Patel, M. Yatskar, C. Callison-Burch, A. Head, R. Hendrix, F. Bastani, E. VanderBilt, N. Lambert, Y. Chou, A. Chheda, J. Sparks, S. Skjonsberg, M. Schmitz, A. Sarnat, B. Bischoff, P. W...

    Pith/arXiv arXiv doi:10.48550/arxiv.2409.17146 2024

  22. [22]

    OpenAI, GPT-4 technical report, arXiv preprint arXiv:2303.08774 (2023)

    Pith/arXiv arXiv 2023

  23. [23]

    T. Wolf, L. Debut, V. Sanh, J. Chaumond, C. Delangue, A. Moi, P. Cistac, T. Rault, R. Louf, M. Fun- towicz, J. Davison, S. Shleifer, P. von Platen, C. Ma, Y. Jernite, J. Plu, C. Xu, T. L. Scao, S. Gugger, M. Drame, Q. Lhoest, A. M. Rush, Transformers: State-of-the-art natural language processing, in: Proceedings of the 2020 Conference on Empirical Methods...

    2020

  24. [24]

    P. Wang, S. Bai, S. Tan, S. Wang, Z. Fan, J. Bai, K. Chen, X. Liu, J. Wang, W. Ge, Y. Fan, K. Dang, M. Du, X. Ren, R. Men, D. Liu, C. Zhou, J. Zhou, J. Lin, Qwen2-vl: Enhancing vision-language model’s perception of the world at any resolution (2024). doi: 10.48550/ARXIV.2409.12191. arXiv:2409.12191

    Pith/arXiv arXiv doi:10.48550/arxiv.2409.12191 2024

  25. [25]

    E. J. Hu, Y. Shen, P. Wallis, Z. Allen-Zhu, Y. Li, S. Wang, L. Wang, W. Chen, Lora: Low-rank adaptation of large language models, in: International Conference on Learn- ing Representations (ICLR), 2022. URL: https://www.microsoft.com/en-us/research/publication/ lora-low-rank-adaptation-of-large-language-models/

    2022

  26. [26]

    Izmailov, D

    P. Izmailov, D. Podoprikhin, T. Nolte, D. Hoffman, D. P. Vetrov, A. G. Wilson, Obtaining better gen- eralization with stochastic weight averaging, in: International Conference on Machine Learning, PMLR, 2018, pp. 2220–2229

    2018

This paper was first reviewed by deepseek-v4-flash on August 4, 2026.