{"id":"d6c49e42-d1e7-4c76-a454-ce43715e7f80","arxiv_id":"2507.20036","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":3.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":1,"one_line_summary":"Averaging a small number of audio embeddings per class outperforms zero-shot text-embedding classification for CLAP-based audio classification.","lead":"This paper shows that using a few labeled audio samples per class, rather than text prompts, to build class reference embeddings improves audio classification with CLAP. The gain is up to 10 percentage points over zero-shot text-based classification on three datasets.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The claimed few-shot gains on ESC-50 and FSD50K depend on externally imported zero-shot baselines; until those are re-run with the same CLAP checkpoint and prompts, the 2–10 pp margin is not controlled.","rationale":"This is the single most load-bearing concern because the paper's headline is explicitly comparative ('typically outperforms the zero-shot baseline'), and two of the three datasets' baselines are not measured in the paper. The authors' own BBL footnote demonstrates that zero-shot scores depend on prompt and label wording, making the uncontrolled text side of the comparison material. The FSD50K least-overlap support selection is a second uncontrolled choice that makes the FS side optimistic, and it should be tested in the same controlled experiment. I do not see an internal inconsistency or a reason to reject: the AVG method is simple, the BBL re-run gives some in-paper support, and the proposed check could confirm the comparison. The reader's CONDITIONAL verdict already encodes this concern; nothing in my stress test moves it, so the verdict remains CONDITIONAL and the paper should be revised to include the controlled baseline reproduction before acceptance.","tokens_in":7898,"tokens_out":7969,"duration_ms":99958,"concrete_test":"Run a controlled reproduction: with the CLAP 2023 checkpoint, compute ESC-50 5-fold zero-shot accuracy and FSD50K mAP using standard class-name prompts and the same scoring code as the FS runs, then repeat FSD50K FS with randomly drawn support sets in place of least-overlap selection. If the reproduced zero-shot numbers are within 0.01 of 0.948 and 0.485, and the random-support FS mAP remains above the reproduced baseline, the central claim stands; otherwise, report the corrected margins.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central comparison in Table 1 is not fully controlled. The FS rows are computed by the authors with CLAP 2023, while the zero-shot rows for ESC-50 and FSD50K are \"obtained from the respective papers\" (Table 1 caption) and may use a different prompt template, label vocabulary, or evaluation protocol. The margin on ESC-50 is only 0.970 - 0.948 = 0.022, and on FSD50K it is 0.579 - 0.485 = 0.094; a modest upward shift in the true zero-shot baseline would erode or eliminate the ESC-50 gain. The paper's own BBL footnote ('low rev engines' -> 'idling tractor') shows that zero-shot scores are highly sensitive to text-side label mapping, so an external baseline cannot be assumed interchangeable. A second uncontrolled advantage goes the other way: FSD50K support files are \"specifically choose[n] those with the least class overlap\" (Section 4.2), a selection rule with no analogue in the zero-shot arm, so part of the FSD50K gain may reflect easier support examples rather than the few-shot principle. The K-selection for MI variants on the evaluation set (Table 1 caption) further inflates some FS rows, though the plain AVG result does not depend on it.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes replacing text-derived class embeddings with class-averaged audio embeddings in a few-shot setting for audio classification. Using CLAP 2023 audio embeddings, the authors compute per-class reference vectors from a small number of labeled support examples and classify by nearest-vector matching, optionally preceded by linear discriminant analysis or mutual-information feature selection. They evaluate on an internal BBL dataset, ESC-50, and FSD50K, and report that the averaged-embedding method (FS AVG) exceeds zero-shot CLAP23 by roughly 2 to 10 percentage points, while LDA-based variants are less competitive. The paper also studies the effect of the number of support samples and the role of mutual-information dimensionality reduction.","tokens_in":8035,"tokens_out":3943,"duration_ms":53641,"significance":"If the reported margins are reliable, the central observation is practically useful: a parameter-free average of CLAP audio embeddings from a handful of labeled samples can replace text embeddings for closed-set classification, avoiding prompt engineering and fine-tuning costs. The paper's AVG method is simple, deterministic, and easy to reproduce in principle, and the comparison to a fine-tuned CLAP model in Figure 1 is a useful calibration. The paper is also honest in acknowledging the fragility of text-based labels, e.g., the BBL label remappings in the Section 5.1 footnote. However, the quantitative claim that few-shot 'typically outperforms' zero-shot is not yet fully supported because the zero-shot baselines on two of three datasets are imported from external papers rather than recomputed under the same protocol, and because one few-shot variant selects support samples on a basis that has no zero-shot analogue. The practical significance therefore depends on completing the controlled comparison.","major_comments":[{"comment":"The zero-shot rows for ESC-50 and FSD50K are taken from external papers, as stated in the table caption, while the few-shot rows are computed by the authors with their own CLAP version, prompt choices, and evaluation pipeline. The claimed margins are small on ESC-50 (0.970 vs. 0.948, i.e., 2.2 pp) and substantial on FSD50K (0.579 vs. 0.485, i.e., 9.4 pp), but they cannot be interpreted as controlled comparisons unless the zero-shot baseline is rerun with the identical CLAP checkpoint and text prompts. The paper's own note that BBL labels had to be manually remapped (e.g., 'low rev engines' to 'idling tractor') demonstrates how sensitive zero-shot scores are to text-side choices. Please rerun CLAP23 zero-shot on ESC-50 and FSD50K under the same protocol, or explicitly justify why the externally reported numbers are directly comparable.","section":"Table 1"},{"comment":"For FSD50K, the few-shot support samples are 'specifically choose[n] those with the least class overlap with other files'. This selection rule has no analogue in the zero-shot arm and can make the few-shot task easier by removing ambiguous recordings. Since FSD50K is multi-label and the reported gain is 9.4 pp, the result may reflect support-set selection rather than the few-shot principle. Please report results with random support selection, or justify that the overlap-based selection does not systematically inflate the FSD50K numbers.","section":"Section 4.2"},{"comment":"The caption states that for MI-based methods 'we use the best K for a given method/dataset pair (see Figure 3)'. Choosing K on the evaluation set is a form of test-set selection, and it overstates the MI+AVG and MI+LDA results. This does not affect the plain AVG rows, which are the central claim, but the MI rows are reported as results rather than as an oracle upper bound. Please use a fixed K or select K on a held-out split, and report the corresponding performance for all K values.","section":"Table 1 caption and Figure 3"},{"comment":"The BBL zero-shot baseline, although rerun by the authors, required manual adjustments to align class names with CLAP's text vocabulary. The paper does not report the exact prompt template or the full list of label mappings used for BBL, ESC-50, or FSD50K. Since the central claim is that audio-based few-shot representations are more robust than text-based ones, the exact text protocol is a load-bearing part of the comparison. Please provide the prompt templates, the label mappings, and the specific CLAP 2023 checkpoint identifier used in the experiments.","section":"Section 5.1"}],"minor_comments":[{"comment":"The abstract says few-shot classification 'typically outperforms' the zero-shot baseline, but LDA with |Ec|=10 underperforms zero-shot in Table 1; please qualify the claim to the averaged-embedding method or to the evaluated regimes.","section":"Abstract and Section 5.1"},{"comment":"The notation for the class-conditional embedding set is inconsistent: the text uses 'E c' while the equation uses 'Ec'. Please standardize the notation.","section":"Equation (2)"},{"comment":"The mutual-information feature-selection procedure is underspecified: no MI estimator, binning strategy, or tie-handling rule is given, which makes the MI+AVG and MI+LDA results hard to reproduce.","section":"Section 4.2"},{"comment":"The confusion-matrix comparison in Figure 2 does not state which support-set size |Ec| is used for the FS AVG result; please add this information to the caption.","section":"Figure 2"},{"comment":"The scaling behaviour as a function of |Ec| is only shown for ESC-50; showing the same curve for BBL or FSD50K would make the few-shot data-efficiency claim more general.","section":"Figure 1"},{"comment":"Reference [26] is cited for the curse of dimensionality but appears to be a book on dynamic programming; please replace it with an appropriate citation.","section":"Reference [26]"}],"recommendation":"major_revision","confidential_remarks":"The paper is a short empirical study with a practical, easily deployable method. The main concern is not the method itself but the validity of the headline comparison: two of the three zero-shot baselines are externally imported, and one few-shot variant uses a support-selection heuristic that favours clean examples. These are fixable within the scope of the paper by rerunning the baselines and repeating the FSD50K experiment with random support samples. If the authors can provide those controlled results, the paper would be suitable for publication; as it stands, the 2-10 pp margin is not fully substantiated."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"James Taylor and Wolfgang Mack report a simple, useful result: for CLAP-based audio classification, averaging the embeddings of 10–50 labeled audio examples per class beats zero-shot text-prompt classification on BBL, ESC-50, and FSD50K by 2–10 percentage points. The method is just prototypical few-shot classification applied to CLAP features, and the paper doesn't cite that literature, but the practical message is clear and the experiments are mostly straightforward.\n\nWhat's genuinely good: The paper is honest about the fragility of text prompts—they manually remapped labels like 'low rev engines' to 'idling tractor' for BBL, which demonstrates the sensitivity they're trying to mitigate. Figure 1 shows how performance scales with support size, with error bars. The BBL comparison is internally controlled. The AVG result is simple enough to reproduce without code.\n\nThe soft spots are real but addressable. The zero-shot baselines for ESC-50 and FSD50K are taken from the original papers rather than re-run with the same CLAP checkpoint and prompts. The ESC-50 margin is only 0.970 vs. 0.948, so a slightly different zero-shot protocol could erase the gain. That's a controlled-comparison problem, and the stress-test is right to flag it. For FSD50K, the support samples are deliberately chosen as those with least class overlap, which gives the few-shot arm an advantage that has no zero-shot analogue; the 9-point mAP gain may partially reflect easier support examples. The MI feature-count K is chosen on the evaluation set (Table 1 caption, Figure 3), which inflates the MI rows; the plain AVG result doesn't depend on K, so that's a minor issue. And there's no code release or detailed FSD50K mAP protocol, which limits reproducibility.\n\nThese aren't fatal. The central AVG claim is plausible and consistent with the broader few-shot literature; I'd expect a properly controlled re-run to still show gains on BBL, and possibly smaller gains elsewhere. The paper doesn't overclaim—it explicitly notes that fine-tuning still beats few-shot on ESC-50. The novelty is low, but for practitioners who need a cheap upgrade to deployed zero-shot systems, the finding is valuable.\n\nI'd send it to peer review, but the authors should re-run the external baselines (or at minimum add a sensitivity analysis with different prompts and checkpoints), randomize the FSD50K support selection, and release code. The right venue is a workshop or a practice-oriented conference session, not a flagship methods paper.\n\nI'd bring it to the reading group as a data point on how few-shot baselines should be controlled, but I wouldn't cite it in my own work. The authors are thinking honestly; the problems are in the evaluation protocol, not in the logic.","headline":"Per-class averaged CLAP embeddings beat zero-shot prompts, but uncontrolled external baselines and biased support selection make the ESC-50 and FSD50K margins unproven.","tokens_in":8655,"tokens_out":3502,"would_cite":false,"duration_ms":38862,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Defining sound classes by a few averaged audio examples beats defining them by text prompts.","keywords":["few-shot learning","zero-shot audio classification","CLAP embeddings","prototype averaging","audio classification","text embeddings","mutual information feature selection","sound event classification"],"falsifier":"A direct check is to reproduce the ESC-50 and FSD50K zero-shot baselines with the same CLAP-23 checkpoint and the paper's evaluation protocol; if the reproduced zero-shot accuracy reaches 0.970 on ESC-50 or mAP reaches 0.579 on FSD50K, the claimed few-shot advantage collapses.","tokens_in":7574,"feed_emoji":"🔊","tokens_out":9527,"duration_ms":105139,"temperature":0.7,"pith_summary":"Few-shot classification here means replacing the text prompt that a contrastive audio-language model uses to define a class with a prototype built by averaging a few real audio recordings of that class. The paper claims this substitution consistently beats zero-shot classification: with 50 examples per class, averaged CLAP audio embeddings reach 0.716 accuracy on the internal BBL set versus 0.623 for CLAP23 text prompts, 0.970 on ESC-50 versus 0.948, and 0.579 mAP on FSD50K versus 0.485. The practical payoff is that a small handful of labeled recordings removes the need to hunt for the best text description of a sound class. The paper also reports that simple averaging beats linear-discriminant classifiers and that mutual-information feature selection gives little extra.","feed_headline":"Few-shot beats zero-shot audio classification with 10-50 samples","feed_subtitle":"Averaging a few audio embeddings per class raises accuracy by 2 to 10 percentage points over text-prompt zero-shot.","key_machinery":"The central object is the per-class averaged audio embedding, a prototype $e_c$ computed from a small set $E_c$ of CLAP audio embeddings for class $c$ via $e_c = \\sum_{e_i \\in E_c} w_i e_i$, with either uniform weights (arithmetic mean) or $\\ell^2$-normalized weights. This prototype replaces the text embedding that zero-shot classification compares against; test embeddings are assigned by minimum distance, using cosine similarity (consistent with CLAP's training) rather than mean squared error. The paper also tests LDA classifiers and mutual-information feature selection as alternatives, but these are secondary: simple prototype averaging is the mechanism that carries the claimed gains.","core_discovery":"The paper's discovery is that audio-derived class prototypes are a better reference than text-derived embeddings for contrastive audio classification. Starting from a pretrained CLAP audio encoder, each class is represented by the average of its audio embeddings from a small development set, and test clips are classified by cosine similarity to the closest prototype. Across three datasets this few-shot recipe beats the same model family's zero-shot text classification by 2 to 10 percentage points, with larger margins where text descriptions are confusing, for example improving 'car' accuracy on BBL from near zero to 45 percent. LDA underperforms the average, which the paper attributes to the curse of dimensionality in 1024-dimensional embeddings; fine-tuning CLAP still slightly edges out the few-shot method on ESC-50, but at far higher adaptation cost.","pith_inferences":["If the gains survive controlled baseline reproduction, collecting a few recordings per class could replace manual prompt selection and label-to-vocabulary remapping in deployed zero-shot audio systems.","The result is contingent on the quality of the underlying audio encoder; with weaker or domain-mismatched embeddings, averaged prototypes will likely gain less, so the recipe is an add-on to representation quality rather than a substitute.","A natural extension is to compare prototype averaging against linear probing or learned nearest-centroid classifiers on the same CLAP embeddings to isolate how much of the gain comes from the prototype construction itself.","The prototype idea could generalize to hierarchical or overlapping-sound settings by averaging at multiple ontology levels or using mixtures of prototypes; the paper only evaluates flat, largely non-overlapping labels."],"forward_implications":["With as few as 10 audio examples per class, averaged prototypes already beat the CLAP23 zero-shot baseline on all three datasets.","On ESC-50, fewer than five examples per class are enough for the averaged-prototype method to outperform the zero-shot text baseline.","Raising the number of examples per class from 10 to 50 increases accuracy for the averaging method, while LDA-based classifiers need more examples and stay below averaging.","Mutual-information feature selection gives only a small boost to averaging and helps LDA only at large feature budgets, so simple averaging carries the result.","Because only the audio encoder is used at adaptation time, new classes can be added by supplying a few recordings, without retraining or fine-tuning the contrastive model."],"supporting_citations":[{"why":"Supplies the CLAP contrastive audio-text model whose audio encoder generates the embeddings used for both the few-shot prototypes and the zero-shot text comparison.","marker":"[19]"},{"why":"Provides the CLAP-23 zero-shot classification protocol and baseline that the few-shot methods are compared against.","marker":"[22]"},{"why":"Provides the ESC-50 dataset and its 5-fold evaluation protocol used in the experiments.","marker":"[31]"},{"why":"Provides the FSD50K dataset and the mAP evaluation protocol used in the experiments.","marker":"[32]"},{"why":"Supplies the AudioSet audio from which the internal BBL dataset's signals are drawn.","marker":"[30]"},{"why":"Supplies the mutual-information feature-selection method used in the MI+AVG and MI+LDA variants.","marker":"[28]"},{"why":"Motivates the dimensionality-reduction preprocessing that the paper applies before LDA.","marker":"[27]"}],"fun_headline_variants":["Audio prototypes beat text prompts for few-shot classification","Few-shot audio: class averages outperform text embeddings","Averaged audio embeddings lift classification over zero-shot","From zero-shot to few-shot: audio prototypes win","From near zero to 45%: audio prototypes fix confusing classes"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The comparison assumes the zero-shot baselines quoted for ESC-50 and FSD50K were measured with the same CLAP checkpoint, prompt wording, and evaluation protocol as the few-shot results, but those baselines come from external papers and only the BBL baseline was re-run, with manual label remapping.","fun_headline_variants_meta":{"raw":{"variants":["Audio prototypes beat text prompts for few-shot classification","Few-shot audio: class averages outperform text embeddings","Averaged audio embeddings lift classification over zero-shot","From zero-shot to few-shot: audio prototypes win","From near zero to 45%: audio prototypes fix confusing classes"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.001008,"raw_usage":{"total_tokens":4190,"prompt_tokens":803,"completion_tokens":3387,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":419,"completion_tokens_details":{"reasoning_tokens":3324}},"tokens_in":419,"tokens_out":3387,"duration_ms":26048,"temperature":1.0,"reasoning_tokens":3324,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-06T13:50:15.051834+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"A direct check is to reproduce the ESC-50 and FSD50K zero-shot baselines with the same CLAP-23 checkpoint and the paper's evaluation protocol; if the reproduced zero-shot accuracy reaches 0.970 on ESC-50 or mAP reaches 0.579 on FSD50K, the claimed few-shot advantage collapses.","supporting_citations":[{"cited_title":"Towards Robust Few-shot Class Incremental Learning in Audio Classification using Con- trastive Representation,","cited_arxiv_id":null,"evidence_quote":"Supplies the CLAP contrastive audio-text model whose audio encoder generates the embeddings used for both the few-shot prototypes and the zero-shot text comparison."},{"cited_title":"Learning Transferable Visual Models From Natu- ral Language Supervision,","cited_arxiv_id":null,"evidence_quote":"Provides the CLAP-23 zero-shot classification protocol and baseline that the few-shot methods are compared against."},{"cited_title":"Audio Flamingo: A Novel Audio Language Model with Few- Shot Learning and Dialogue Abilities,","cited_arxiv_id":null,"evidence_quote":"Provides the ESC-50 dataset and its 5-fold evaluation protocol used in the experiments."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Provides the FSD50K dataset and the mAP evaluation protocol used in the experiments."},{"cited_title":"Pengi: An Audio Language Model for Audio Tasks,","cited_arxiv_id":null,"evidence_quote":"Supplies the AudioSet audio from which the internal BBL dataset's signals are drawn."},{"cited_title":"Natural Language Supervision For General-Purpose Audio Representations,","cited_arxiv_id":null,"evidence_quote":"Supplies the mutual-information feature-selection method used in the MI+AVG and MI+LDA variants."},{"cited_title":"Large-Scale Contrastive Language-Audio Pretrain- ing with Feature Fusion and Keyword-to-Caption Augmentation,","cited_arxiv_id":null,"evidence_quote":"Motivates the dimensionality-reduction preprocessing that the paper applies before LDA."}],"review_version":1}