{"id":"6d70719d-1e30-43f8-b896-255827c092a1","arxiv_id":"2607.06063","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":4.0,"correctness_risk":"unknown","formal_verification":"none","parameter_count":12,"one_line_summary":"CARE-DPP combines class-balanced uncertainty, annealed embedding novelty, and DPP-based batch diversification for bioacoustic active learning, achieving 0.50 mean AULC versus 0.46 for CoreSet.","lead":"This paper introduces CARE-DPP, a batch active-learning method for bioacoustic species classification that combines class-balanced uncertainty, embedding novelty, and determinantal point process (DPP) batch diversification. A smart generalist might read it to learn how DPP-based sampling can reduce annotation effort in large-scale ecological monitoring.","discovery_kind":"unclear","skeptic_critique":{"model":"glm-5.2","headline":"The ablation attributing the largest gain to DPP does not specify what replaces DPP selection, making it impossible to distinguish DPP's diversity benefit from a degenerate fallback.","rationale":"The reader correctly identified the most load-bearing concern: the DPP ablation's unspecified fallback and lack of significance testing. This is a real gap, but it is addressable and does not invalidate the method itself — CARE-DPP is well-specified, reproducible, and the main result (0.50 AULC) is reported with 5 repeats. The concern affects the strength of the attribution claim ('DPP is the largest contributor') more than the headline performance claim. The verdict should remain CONDITIONAL: the method appears to work, but the specific claim about which component drives the gain is not yet securely established. The paper would strengthen considerably by (a) stating the fallback selection strategy explicitly, (b) reporting standard deviations in Table 3, and (c) ideally testing at least two fallbacks (random-from-pool and greedy-top-k) to separate the diversity benefit from the quality-selection benefit. The comparison against official baselines (Table 2) is less concerning than the reader suggests, since all methods run under the same BaseAL framework with fixed embeddings and training protocol, though direct re-implementation under identical seeds would be ideal.","tokens_in":5663,"tokens_out":2220,"duration_ms":128728,"concrete_test":"Re-run the 'No DPP' ablation with an explicit, stated fallback: greedy selection of the top-B_t candidates by quality score q_i from the same candidate pool (same exploration fraction, same adaptive batch schedule). Report per-dataset means ± standard deviations across all 5 repeats. If the greedy-quality fallback achieves mean AULC within 0.02 of the full method (≥0.48), the claim that DPP is the largest contributor weakens substantially, as the gain would be attributable to quality-based candidate filtering rather than batch-level diversity.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim that 'DPP batch diversification is the largest contributor' rests entirely on the 'No DPP batch selection' row in Table 3 (mean AULC drops from 0.5017 to 0.4639). However, the paper never specifies what selection mechanism replaces DPP. If the fallback is random sampling from the candidate pool, the ablation conflates 'DPP diversity' with 'using quality scores for selection at all' — a much weaker comparison than intended. A fairer fallback would be greedy top-k by quality score q_i from the same candidate pool, which would isolate the diversity contribution of DPP from its quality-weighting role. Additionally, Table 3 reports only means across 5 repeats with no standard deviations or significance tests, so we cannot assess whether the 0.04 mean drop is robust or driven by a single dataset (the drop is concentrated on HSN: 0.608→0.541, and UHH: 0.4335→0.3697, while ATBFL barely moves: 0.4652→0.4634). Notably, the 'No DPP' mean of 0.4639 nearly matches the official CoreSet baseline of 0.4600, which raises the possibility that the fallback is functionally similar to CoreSet — making the comparison less about DPP's added value and more about 'CARE-DPP without its key component equals the baseline.'","agreement_with_reader":"agree"},"referee_report":{"model":"glm-5.2","summary":"This technical report presents CARE-DPP, a batch active-learning acquisition method for the BioDCASE 2026 challenge. The method combines class-balanced multilabel uncertainty, cosine embedding novelty with annealed weights, a candidate pool mixing top-quality and random exploration samples, and DPP-based greedy batch selection via pivoted Cholesky. An adaptive acquisition schedule uses smaller batches early and larger batches later. Evaluated on four bioacoustic datasets (three BirdSet subsets and ATBFL) over five repeats, CARE-DPP achieves a mean development AULC of 0.50 versus 0.46 for the official CoreSet baseline. An ablation study attributes the largest gains to DPP batch selection and the adaptive acquisition schedule.","tokens_in":5936,"tokens_out":1574,"duration_ms":119433,"significance":"The method is clearly and completely specified, including all equations, hyperparameter schedules, and Algorithm 1, making it reproducible. The use of DPP for batch active learning in bioacoustics is a reasonable and well-motivated application of an established technique. The evaluation against four official baselines on four datasets provides useful evidence. The per-dataset ablation breakdown (Table 3) is informative and goes beyond reporting only aggregate means. The submission includes reproducible code and five-repeat result exports, which is commendable.","major_comments":[{"comment":"§4.2, Table 3, 'No DPP batch selection' row: The paper does not specify what selection mechanism replaces DPP in this ablation. This is load-bearing for the central claim that 'DPP batch diversification is the largest contributor.' If the fallback is random sampling from the candidate pool, the ablation conflates DPP's diversity benefit with the benefit of using quality scores for selection at all. A fairer fallback would be greedy top-k selection by quality score q_i from the same candidate pool, which would isolate the diversity contribution of DPP from its quality-weighting role. The paper should specify the fallback and, ideally, include this stronger baseline.","section":null},{"comment":"§4.2, Table 3: No standard deviations or significance tests are reported across the five repeats, despite the paper stating that five independent repeats were conducted (§4) and that submitted YAML files contain per-cycle means and standard deviations. The mean AULC drop attributed to removing DPP (0.5017 to 0.4639) is concentrated on HSN (0.608 to 0.541) and UHH (0.4335 to 0.3697), while ATBFL barely moves (0.4652 to 0.4634) and POW drops moderately (0.5002 to 0.4813). Without variance estimates, it is unclear whether the 0.04 mean difference is robust or driven by one or two datasets. Adding standard deviations (or at minimum, per-dataset repeat-level variance) would substantially strengthen the ablation claims.","section":null}],"minor_comments":[{"comment":"§3, paragraph below Eq. (6): The text states weights were 'fixed after a coarse validation sweep' but does not specify the validation split used, the range of values explored, or the selection criterion. A brief note on which dataset(s) the sweep was performed on would help assess generalization risk.","section":null},{"comment":"Table 1: The 'Labels/sample' column is useful but the caption could clarify whether these are averages over the development pool or the full dataset.","section":null},{"comment":"§3.3, Eq. (7): The candidate pool size formula M_t = min{|U_t|, max(30B_t, 1500)} could benefit from a brief justification for the specific constants (30, 1500), especially since the pool size directly affects DPP computational cost and selection quality.","section":null},{"comment":"Figure 1 is referenced but not visible in the provided text; ensure it clearly labels all ablation variants and uses consistent ordering with Table 3.","section":null},{"comment":"§5, final paragraph: The mention of 'additional candidate-pool variants were explored but did not yield robust improvements' without reporting them is acceptable for a technical report but could be briefly footnoted with the variants tried, to save future researchers from repeating the same exploration.","section":null}],"recommendation":"major_revision","confidential_remarks":"This is a challenge technical report, so some brevity is expected. However, the two major issues — the unspecified DPP fallback and the absence of variance estimates — directly affect the paper's central ablation claim. Both are fixable within the manuscript's scope: specifying the fallback and adding standard deviations to Table 3. If the authors can show that the DPP ablation holds against a greedy top-k fallback with variance estimates, the claim would be well-supported. The paper is otherwise a solid contribution to the challenge."},"author_rebuttal":{"model":"glm-5.2","summary":"We thank the referee for the careful reading and constructive comments. Both major points are well-taken and addressable in revision. We will (1) specify the DPP ablation fallback and add a greedy top-k baseline, and (2) add standard deviations across the five repeats to the ablation table. We agree with both points and will revise accordingly.","responses":[{"response":"The referee is correct on both counts. The current ablation uses random sampling from the candidate pool as the fallback, and we agree this does not isolate DPP's diversity contribution from its quality-weighting role. This is a genuine gap in the ablation design. We will (1) explicitly state the fallback mechanism in the revised Table 3 caption and §4.2, and (2) add a new ablation row using greedy top-k selection by quality score q_i from the same candidate pool, which isolates the diversity contribution of DPP from its quality-weighting role. This stronger baseline will allow a fairer attribution of the gains to batch-level non-redundancy specifically.","revision_made":"yes","referee_comment":"The paper does not specify what selection mechanism replaces DPP in the 'No DPP batch selection' ablation. If the fallback is random sampling from the candidate pool, the ablation conflates DPP's diversity benefit with the benefit of using quality scores for selection at all. A fairer fallback would be greedy top-k selection by quality score q_i from the same candidate pool."},{"response":"The referee is right that variance estimates are necessary to assess the robustness of the ablation differences, and we have no reason not to report them. The per-repeat data is already available in the submitted YAML exports. We will add standard deviations across the five repeats to Table 3 for every cell (both per-dataset and mean columns). We will also add a brief note in §4.2 discussing the per-dataset variability pattern the referee identifies, particularly the contrast between the large DPP-removal effects on HSN and UHH versus the negligible effect on ATBFL.","revision_made":"yes","referee_comment":"No standard deviations or significance tests are reported across the five repeats in Table 3, despite the paper stating five independent repeats were conducted. Without variance estimates, it is unclear whether the 0.04 mean difference attributed to removing DPP is robust or driven by one or two datasets."}],"tokens_in":5537,"tokens_out":516,"duration_ms":88234,"standing_objections":[]},"desk_editor":{"model":"glm-5.2","letter":"Short version: this is a well-specified challenge entry that combines known components (DPP batch AL, class-balanced uncertainty, embedding novelty) into a working pipeline for bioacoustics. The 0.50 vs 0.46 AULC improvement over CoreSet is real and the method is clearly described. The main soft spot is the ablation: the stress-test concern about the 'No DPP' row lands, and it matters more than the reader flagged it. But the paper is still a useful contribution that deserves a serious read. The reader's CONDITIONAL verdict is about right, maybe slightly harsh given that this is explicitly a challenge technical report with reproducible exports. The novelty score of 4.0 is accurate — this is a legitimate combination of known ideas in a new domain, not a new idea itself. The soundness score of 6.0 is fair but could be 5.0 given the ablation gap. The circularity burden is correctly assessed at 2.0 — self-citation is minimal and the DPP formulation is standard, properly attributed to Kulesza & Taskar and Biyik et al. The free-parameter list is long (12+ hyperparameters), which is a legitimate concern for a method paper, though the paper is transparent that these came from a coarse validation sweep and were then fixed. The stress-test concern is the real issue. The 'No DPP batch selection' ablation row drops from 0.5017 to 0.4639, and the paper never specifies what replaces DPP. If the fallback is random sampling from the candidate pool, the ablation conflates 'DPP diversity' with 'using quality scores for selection at all.' A fairer comparison would be greedy top-k by quality score, isolating DPP's diversity contribution from its quality-weighting role. The fact that the 'No DPP' mean (0.4639) nearly matches the CoreSet baseline (0.4600) is suspicious and supports the concern that the fallback may be functionally degenerate. Additionally, no standard deviations or significance tests are reported in Table 3, and the DPP gain is concentrated on two of four datasets (HSN and UHH), so we can't tell if the 0.04 mean is robust or driven by those two. The reader caught the missing significance tests but under-weighted the unspecified-fallback problem. The stress-test note is correct to flag this as the central weakness. That said, the paper does several things well. The method is fully specified — all equations, schedules, and hyperparameters are given. The ablation isolates each component individually, which is more than most challenge submissions do. The per-dataset breakdown in Table 3 is genuinely informative, showing that different components help on different datasets. The submitted YAML exports with per-cycle means and standard deviations are a plus for reproducibility. The adaptive acquisition schedule (smaller batches early) is a sensible design choice and the ablation supports it. This paper is for researchers working on active learning for bioacoustics or ecological monitoring who want a practical, well-tested acquisition method. It's also useful as a baseline for future BioDCASE challenge entries. It doesn't resolve a long-open question but it's honest, competent work. The ablation gap is addressable in revision — specify the fallback, add standard deviations, and ideally run the greedy top-k comparison. If the authors do that and the DPP gain holds, the central claim is solid. Recommend accepting for peer review with a request to address the ablation specification and add variance reporting.","headline":"Solid challenge submission combining DPP batch selection with annealed uncertainty-novelty for bioacoustic active learning. The method works and the ablation is informative, but the key ablation has a gap that weakens the central attribution claim.","tokens_in":6467,"tokens_out":844,"would_cite":false,"duration_ms":119104,"reading_group":"no","serious_thinker":"no","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"glm-5.2","headline":"DPP-driven active learning cuts bioacoustic annotation costs","keywords":["active learning","determinantal point process","bioacoustics","batch selection","class imbalance","annotation efficiency","embedding novelty"],"falsifier":"If the unspecified fallback used when DPP is removed is a degenerate or poorly-tuned selection method, the attributed gain to DPP may be overstated; a properly tuned alternative batch diversity method might close the gap.","tokens_in":5817,"feed_emoji":"🦜","tokens_out":654,"duration_ms":193087,"temperature":0.7,"pith_summary":"The paper introduces CARE-DPP, an active learning method for bioacoustic classification that reduces annotation effort by intelligently selecting which audio samples to label. The core mechanism is a determinantal point process (DPP), a mathematical object that selects diverse, non-redundant subsets of data points. By combining DPP-based batch diversification with an annealed schedule that shifts from geometric coverage early to classifier uncertainty late, the method achieves a mean AULC of 0.50 across four datasets, outperforming the CoreSet baseline of 0.46. The central claim is that enforcing batch-level diversity via DPP is the single most important factor for improving label efficiency in this setting.","feed_headline":"DPP batch selection beats CoreSet in bioacoustic active learning","feed_subtitle":"A determinantal point process for diverse annotation batches lifts label efficiency to 0.50 AULC, with diversity as the biggest win.","key_machinery":"Determinantal point process (DPP) batch selection: a quality-weighted kernel that greedily maximizes the log-determinant of selected sample embeddings, rewarding high individual quality while penalizing redundancy within the batch.","core_discovery":"The key finding is that using a determinantal point process to select diverse, non-redundant annotation batches yields the largest performance gains in a bioacoustic active learning loop, more so than class-balanced uncertainty, annealed novelty, or exploration strategies. Removing DPP selection drops mean AULC from 0.50 to 0.46, with the steepest losses on datasets with heterogeneous embedding spaces (HSN and UHH).","pith_inferences":[],"forward_implications":["DPP-based batch selection could be directly transferred to other active learning domains with high embedding heterogeneity, such as medical imaging or remote sensing.","The finding that adaptive batch sizing (small early, large late) matters suggests that frequent model updates early in training are critical when initial predictions are unreliable.","The dataset-dependent effects of annealing and class balancing indicate that a one-size-fits-all active learning strategy may be suboptimal across bioacoustic domains with different label densities and class distributions.","The decreasing random exploration fraction provides a template for balancing exploitation of model scores with exploration of the feature space in other pool-based active learning settings."],"fun_headline_variants":["DPP batch diversity drives label efficiency in bioacoustics","Diverse batch selection lifts bioacoustic active learning gains","DPP-based annotation batches beat CoreSet baseline","Non-redundant batch selection key to bioacoustic label efficiency","DPP batch diversification lifts bioacoustic AULC to 0.50"],"cache_read_input_tokens":0,"weakest_assumption_plain":"The claim that DPP is the largest contributor rests on a single ablation configuration where DPP is replaced by an unspecified fallback, without reporting statistical significance across the five repeats or testing sensitivity to the fallback choice.","fun_headline_variants_meta":{"raw":{"variants":["DPP batch diversity drives label efficiency in bioacoustics","Diverse batch selection lifts bioacoustic active learning gains","DPP-based annotation batches beat CoreSet baseline","Non-redundant batch selection key to bioacoustic label efficiency","DPP batch diversification lifts bioacoustic AULC to 0.50"]},"model":"glm-5.2","effort":"high","cost_usd":0.0,"raw_usage":{"total_tokens":651,"prompt_tokens":579,"completion_tokens":72,"prompt_tokens_details":null},"tokens_in":579,"tokens_out":72,"duration_ms":50233,"temperature":1.0,"reasoning_tokens":null,"cache_read_input_tokens":0,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-07-08T17:30:31.868875+00:00","model_set":{"reader":"glm-5.2"},"falsifier":"If the unspecified fallback used when DPP is removed is a degenerate or poorly-tuned selection method, the attributed gain to DPP may be overstated; a properly tuned alternative batch diversity method might close the gap.","supporting_citations":[],"review_version":1}