{"id":"53c199fa-a1bc-4e79-8b30-05979faca519","arxiv_id":"2507.00049","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":4,"one_line_summary":"ADADEDUP adaptively prunes object detection datasets by combining semantic clustering with proxy-model loss feedback, matching full-data mAP at 20% pruning.","lead":"ADADEDUP is a data pruning method that removes redundant training images while keeping the ones a model actually needs. It can cut object detection training data by 20 percent with almost no loss in accuracy, saving significant compute.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Eq. 2's unnormalized loss sums make the adaptive signal a function of per-cluster pruning ratio, not of sample informativeness.","rationale":"The reader flagged a plausible biasing mechanism: a proxy trained on kept samples may assign artificially low loss to those same samples, corrupting Δℓ_i. I found a more immediate and more easily checkable confound: Eq. 2 aggregates losses as sums over subsets of unequal size, so even with unbiased per-sample losses the sign of Δℓ_i is strongly tied to the initial per-cluster pruning ratio γ_i. This is load-bearing because the adaptive re-pruning step is the paper's core novelty; if the signal is dominated by subset sizes, the claimed 'model-informed feedback' is not doing the advertised work. A fix is straightforward (use per-sample means or a proper normalization), and the open-source code can settle the question empirically. I therefore recommend CONDITIONAL rather than REJECT: the paper should either correct Eq. 2 and Algorithm 1, or provide an ablation showing that the sum form is not responsible for the reported gains. If the test shows the effect is an artifact, the central claims would need substantial revision. My concern overlaps with the reader's but is distinct, hence partial agreement.","tokens_in":16490,"tokens_out":7115,"duration_ms":90375,"concrete_test":"Run the released code on a subset of Waymo (or COCO) with K=10 and a 20%/40% pruning budget. Compute Δℓ_i as written (sums) and also δ_i = mean loss on kept minus mean loss on pruned per cluster. Test whether sign(Δℓ_i) matches sign(1−2γ_i) in a majority of clusters (expected if the sum artifact dominates). Then retrain with δ_i as the signal, keeping all other hyperparameters and the budget normalization identical, and compare pruned-set composition and final mAP. If mAP changes beyond the reported std or per-cluster keep counts shift by more than 5%, the published adaptive step is not measuring the claimed quantity.","verdict_should_be":"CONDITIONAL","load_bearing_attack":"Section 3.3.2 defines Δℓ_i = ℓ^s_i − ℓ^p_i with ℓ^s_i and ℓ^p_i as sums over kept and pruned samples in cluster c_i. These subsets have different sizes. For any cluster, if average losses on kept and pruned samples are equal, Δℓ_i = |c_i|(1−2γ_i) times that average, where γ_i is the initial pruning ratio. Hence Δℓ_i > 0 whenever γ_i < 0.5 and Δℓ_i < 0 whenever γ_i > 0.5. The sign that drives the adaptive update (prune more vs. less) is therefore determined largely by how many samples were initially pruned, not by whether the pruned samples are redundant. No per-sample averaging or count normalization appears in Eq. 2; the later budget normalization in Algorithm 1 rescales values but does not remove this confound. Because the entire method rests on Δℓ_i as a 'differential loss signal' indicating relative information content, the central empirical claims (e.g., 20% data efficiency with near-original mAP, 54% loss reduction vs. random) are not attributable to the proposed mechanism unless the signal is computed per sample or otherwise corrected. This is an internal inconsistency, not a disagreement with prior work.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"AdaDeDup proposes a two-stage hybrid data-pruning method for object detection. Stage 1 clusters the dataset with VLM features and applies an initial density-based pruning. Stage 2 trains a proxy model on the kept subset, computes per-cluster differential loss signals by summing losses on kept versus pruned samples, and uses these signals to adjust each cluster's pruning ratio before re-pruning from the full dataset. Experiments on Waymo, COCO, and nuScenes with BEVFormer-S and Faster R-CNN report that AdaDeDup outperforms random downsampling, CLIP-DeDup, and VLM-SSE, achieving near-original mAP at 20% pruning on Waymo and reducing performance loss substantially relative to baselines.","tokens_in":16758,"tokens_out":5164,"duration_ms":49546,"significance":"If the proposed mechanism is sound, AdaDeDup is a practically relevant contribution: it targets the under-explored setting of data pruning for object detection, combines density-based and model-based signals at cluster level, and is evaluated on three large-scale benchmarks with standard deviations over multiple runs and open-sourced code. However, the central adaptive signal is confounded by cluster size and initial pruning ratio, so the current experiments do not establish the claimed mechanism. The method may still work as a heuristic, but the paper's theoretical framing and the attribution of the empirical gains need correction. The manuscript's own limitations section acknowledges sensitivity to hyperparameters but does not address this confounding.","major_comments":[{"comment":"The differential signal Δℓ_i = ℓ_i^s − ℓ_i^p is computed as the difference of unnormalized sums over kept and pruned subsets that have different sizes. If the per-sample average loss in cluster c_i is μ, then Δℓ_i = μ |c_i| (1 − 2γ_i), where γ_i is the initial pruning ratio. Consequently, the sign of Δℓ_i is determined by whether γ_i is below or above 1/2, not by whether the pruned samples are redundant. Since the update γ'_i = γ_i + β·α_i·Δℓ_i increases γ_i when Δℓ_i > 0 and decreases it when Δℓ_i < 0, the signal drives every cluster's pruning ratio toward 1/2 regardless of model feedback. The reported experiments therefore do not identify the proposed model-informed adaptation as the source of the gains. The manuscript should compute ℓ_i^s and ℓ_i^p as per-sample averages (or otherwise normalize by subset size), rerun the experiments, and reinterpret the results.","section":"§3.3.2, Eq. (2) and Algorithm 1, lines 8-9"},{"comment":"The claimed zero-order gradient approximation, ℓ(θ*(W_s), s_p) − ℓ(θ*(W_s), s_k) as a proxy for ∂J(W_s)/∂(W_s)_p, is not derived, and it is not what Algorithm 1 computes: Algorithm 1 compares aggregated losses over entire clusters rather than a kept sample near each pruned sample. Without a formal connection between the aggregate Δℓ_i and the policy gradient ∂J/∂λ_c, the theoretical framing in Section 3.2 is not supported by the implementation. Please either provide a rigorous derivation or state explicitly that the update is a heuristic motivated by, but not equivalent to, a zero-order gradient.","section":"§3.2, Step 2, and Algorithm 1"},{"comment":"The normalization step is underspecified. The text says the scaled differences are 'typically normalized' and Algorithm 1 states that the adjusted values must satisfy ∑ |c_i|(γ_i + β·Δℓ_i*) = n − m while respecting 0 ≤ γ'_i ≤ 1, but no concrete normalization or rounding procedure is given. The final selected set depends on this normalization, and feasibility after clipping is not established. Specify the exact normalization rule, the order of clipping versus rescaling, and how rounding of k_i is handled.","section":"§3.3.2 and Algorithm 1, step 14"},{"comment":"Because the proxy model is trained only on the kept subset D_s^(0), the loss difference ℓ_i^s − ℓ_i^p reflects the model's familiarity with the kept samples as much as any intrinsic redundancy of the pruned samples. A proxy trained on a random subset of the same size would also produce nonzero per-cluster differences, so the adaptive signal may be dominated by overfitting to the kept set. The paper does not test this possibility. An ablation with a proxy trained on a random subset of equal size, or another control, is needed to support the claim that Δℓ_i measures relative information content.","section":"§3.3.2, Step 1"}],"minor_comments":[{"comment":"The proxy model training appears inside the first for loop, which would train the proxy K times; move it before the loop or fix the indentation.","section":"Algorithm 1, lines 4-6"},{"comment":"GLISTER is cited as [14], but reference [14] is GoodCore; the correct GLISTER citation appears to be [37]. Please fix the citation.","section":"Section 2 and Section 3.1"},{"comment":"The heading 'Parameterizing the pruning policy to narrow the decision space' is repeated from item 1; the heading for item 2 should be 'Zero-order estimation for the policy gradient'.","section":"Section 3.2, item 2"},{"comment":"'A V datasets' should be 'AV datasets'.","section":"Appendix D.1"},{"comment":"The GPU name 'NVIDIA A5880 Ada' appears to be a typo for 'NVIDIA A6000 Ada' or the correct product name.","section":"Appendix C.1"},{"comment":"The word 'perserved' should be 'preserved'.","section":"Broader Impact Statement"}],"recommendation":"major_revision","confidential_remarks":"The empirical results may survive a corrected normalization, but the central mechanism as written is not identified by the experiments. I recommend requesting the per-sample averaging fix and the control ablation before further consideration. The paper would also benefit from a tighter connection between the stated zero-order gradient motivation and the implemented algorithm."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Colleague,\n\nThe core idea here is genuinely new: after an initial density-based prune, train a proxy model and compare losses on kept vs pruned samples per cluster to adjust cluster-specific thresholds. That is a sensible way to combine the two pruning families, and the experiments on Waymo, COCO, and nuScenes are the most thorough I've seen for object-detection pruning.\n\nBut there is a problem with the signal itself. In Eq. (2), ℓ^s_i and ℓ^p_i are sums, not averages. If the average losses on kept and pruned samples are equal, Δℓ_i = |c_i|(1−2γ_i) times that average. So the sign that drives the update is mostly determined by whether the initial pruning ratio γ_i is below or above 0.5, not by whether pruned samples are informative. Clusters pruned less than half get a positive Δ and are pruned more; clusters pruned more than half get a negative Δ and are pruned less. The method pushes every cluster toward 50% pruning. The authors interpret Δℓ_i as \"kept are harder\" or \"pruned are easier\", but with sums that interpretation does not follow. This directly contradicts their motivating example: cluster #1 has a high pruning ratio yet is called redundant, but the formula would drive it toward less pruning.\n\nThis confound is load-bearing. The paper's central claim—that the adaptive step recovers informative samples—is not supported by the stated equations. The empirical improvements over baselines might still be real; the budget reallocation could help even if the direction is partly accidental. But the mechanism as described is not what the results demonstrate.\n\nOther soft spots: Algorithm 1's normalization step (step 14) is underspecified; the hyperparameters (K, β, α+, α−) are not ablated; and Appendix A has an unresolved \"[cite]\" placeholder. The proxy-overfitting worry the reader raised is real but secondary.\n\nThe positive side: the benchmarks are solid, the nuScenes table is clean, and the code is open. This deserves a serious referee, but the authors need to fix the signal (per-sample averaging or a clear justification of the sum form) and add an ablation showing the adaptive step beats the initial density-only prune. Without that, I wouldn't cite it as a method.","headline":"The adaptive signal in Eq. (2) is confounded by pruning ratio and cluster size, so the paper's central mechanism is not supported, but the empirical study is solid enough to warrant a careful revision.","tokens_in":17294,"tokens_out":5285,"would_cite":false,"duration_ms":51211,"reading_group":"yes","serious_thinker":"no","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"AdaDeDup claims hybrid density-plus-model pruning trains object detectors on 20% less data with almost no accuracy loss, cutting performance loss by at least 54% versus random downsampling on Waymo.","keywords":["data pruning","object detection","density-based pruning","proxy model","cluster-adaptive selection","data efficiency","semantic deduplication","autonomous driving"],"falsifier":"A decisive check: train the proxy on the initially kept subset, then retrain a second proxy on the kept subset plus a random half of the pruned samples, and see whether the per-cluster loss-gap ranking of which clusters deserve less pruning survives. If the ranking flips for many clusters, or if AdaDeDup's mAP gain over its density baseline disappears when the swapped-back 5–10% of samples are chosen randomly instead of by the loss gap, the adaptive signal is not the cause of the reported results.","tokens_in":16304,"feed_emoji":"✂️","tokens_out":19318,"duration_ms":147849,"temperature":0.7,"pith_summary":"The paper claims that the two competing families of data pruning can be merged into one adaptive rule that works for large-scale object detection. Density-based pruning removes near-duplicate images cheaply but is blind to what the detector actually needs, while model-based scoring tracks the task but is costly and can over-select hard or noisy samples. AdaDeDup first prunes inside semantic clusters, then trains a small proxy model on the kept images and measures, per cluster, whether the pruned images were easier or harder than the kept ones using the differential loss signal $\\Delta\\ell_i$. Clusters whose pruned images were easy get pruned harder; clusters whose pruned images were hard get some samples restored. On Waymo, COCO, and nuScenes the paper reports near-original mean average precision at 15–20% pruning and reductions in performance loss of at least 54% (Waymo) and 66% (COCO) relative to random downsampling, so the practical stake is training accurate detectors on noticeably less data and compute.","feed_headline":"Hybrid pruning drops 20% of detection data with near-zero mAP loss","feed_subtitle":"Prune 20% of training frames, keep nearly full detection accuracy, and beat random sampling by over half the mAP loss.","key_machinery":"The carrying object is the differential loss signal $\\Delta\\ell_i = \\ell_i^s - \\ell_i^p$ (Eq. 2): for each cluster $c_i$, the proxy model's summed loss on initially kept samples minus its summed loss on initially pruned samples. The paper treats this scalar as a zero-order estimate of the gradient of the selection objective with respect to the cluster's pruning parameter, converting an NP-hard bi-level subset-selection problem into one threshold update $\\gamma'_i = \\mathrm{clip}(\\gamma_i + \\beta \\tilde{\\Delta\\ell}_i, 0, 1)$, with the per-cluster updates normalized so the total data budget is preserved. The sign of $\\Delta\\ell_i$ decides whether cluster $c_i$ is pruned more or less; its magnitude, scaled by $\\alpha_+$ or $\\alpha_-$, sets how far the cluster-specific density threshold moves in that single step.","core_discovery":"The central claim is that a signed per-cluster loss gap, $\\Delta\\ell_i = \\ell_i^s - \\ell_i^p$, computed by a proxy model trained only on the initially kept subset, is a usable signal for how aggressively each semantic cluster should be pruned. If within a cluster the kept samples carry higher average loss than the pruned ones, the pruned ones are judged redundant and the cluster's pruning ratio rises; if the pruned samples carry higher loss, informative content was discarded and the ratio falls. The adjustment is a single one-shot update that changes the selection status of roughly 5–10% of samples, avoiding the repeated retraining of bi-level optimization. The paper argues that this cluster-adaptive hybrid beats pure density-based deduplication (CLIP-DeDup and VLM-SSE) and random downsampling on Waymo, COCO, and nuScenes, reaching near-full-data mAP at 20% pruning on Waymo and 10% on COCO, and matching random downsampling's accuracy with 15–20% less data.","pith_inferences":["The per-cluster loss gap is a coarse influence estimate: it replaces expensive inverse-Hessian computations with one forward pass of a small proxy. The paper does not test whether the signal remains reliable when the proxy is far smaller than the target model, even though its own proxies already train on at most 10–30k samples.","The main untested risk is self-confirmation: because the proxy trains only on kept samples, a negative gap may reflect overfitting to those samples rather than true redundancy. A direct test would retrain the proxy on a mix of kept and pruned samples and check whether cluster-level decisions flip.","Waymo and nuScenes are scene-structured: removing one front-view image discards the whole multi-camera scene, so the effective pruning decision is per scene while the loss gap is computed per image; decoupling these two levels is a refinement the paper leaves open.","The paper lists sensitivity to cluster count and to the scaling constants $\\alpha_+$, $\\alpha_-$, $\\beta$ as a limitation; normalizing the gap by cluster size or intra-cluster density variance is a testable way to stabilize the adaptation across 10, 100, and 500 clusters."],"forward_implications":["At 20% pruning on Waymo and 10% on COCO, models trained on the pruned subset reach nearly the same mean average precision as full-data training, so a fifth of a large detection dataset can be dropped without a retraining-quality penalty.","The adaptation is a single one-shot update that reallocates each cluster's pruning budget, so no line search or repeated retraining is needed; the paper reports choosing the update size so that roughly 5–10% of sample selections change.","Because results are reported with BEVFormer-S on driving benchmarks and Faster R-CNN on COCO, the mechanism appears to transfer across scene-structured multi-camera data and generic image collections.","The largest relative gains over random downsampling appear at higher pruning ratios (at least 54% loss reduction on Waymo up to 40% pruning; at least 66% on COCO up to 20% pruning), so the method's advantage grows as the data budget tightens."],"supporting_citations":[{"why":"Defines VLM-SSE, the semantic-deduplication baseline AdaDeDup extends and compares against; also supplies the specialized autonomous-driving caption prompt used for Waymo and nuScenes.","marker":"[5]"},{"why":"SemDeDup, the semantic-deduplication foundation showing that embedding-near duplicates can be removed, which motivates the initial density-based pruning stage.","marker":"[2]"},{"why":"Selection via Proxy, the basis for using a cheaper proxy model to guide data selection instead of repeatedly training the full target model.","marker":"[12]"},{"why":"The Waymo Open Dataset, the primary benchmark behind the headline claims of near-full performance at 20% pruning and at least 54% loss reduction versus random downsampling.","marker":"[16]"},{"why":"COCO 2017, the generic object-detection benchmark where AdaDeDup claims near-zero loss at 10% pruning and at least 66% loss reduction at 20% pruning.","marker":"[18]"},{"why":"nuScenes, the third benchmark where AdaDeDup reports a 68% performance-loss reduction versus random downsampling at 30% pruning.","marker":"[15]"},{"why":"BEVFormer, the detector trained on Waymo and nuScenes, and the source of the 2Hz subsampled Waymo split used in the experiments.","marker":"[29]"},{"why":"Faster R-CNN, the detector trained on COCO, showing the method transfers beyond a single object-detection architecture.","marker":"[31]"},{"why":"CLIP's ViT-L/14 embeddings define the CLIP-DeDup density baseline for Waymo and nuScenes that AdaDeDup must beat.","marker":"[33]"}],"fun_headline_variants":["Cluster-adaptive pruning: 20% less data, near-full detection mAP","Adaptive hybrid pruning cuts detection data by 20% with minimal mAP drop","One-shot cluster feedback prunes 20% of detection data, preserving mAP","Cluster-wise loss gaps guide adaptive pruning for object detection"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The method assumes that the difference in a proxy model's loss between kept and pruned images inside a cluster tells you how much real information the pruning threw away, even though the proxy was trained only on the kept images and might simply be more comfortable with them.","fun_headline_variants_meta":{"raw":{"variants":["Cluster-adaptive pruning: 20% less data, near-full detection mAP","Adaptive hybrid pruning cuts detection data by 20% with minimal mAP drop","One-shot cluster feedback prunes 20% of detection data, preserving mAP","Cluster-wise loss gaps guide adaptive pruning for object detection"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000642,"raw_usage":{"total_tokens":2989,"prompt_tokens":1015,"completion_tokens":1974,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":631,"completion_tokens_details":{"reasoning_tokens":1893}},"tokens_in":631,"tokens_out":1974,"duration_ms":14952,"temperature":1.0,"reasoning_tokens":1893,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-06T22:58:03.318990+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"A decisive check: train the proxy on the initially kept subset, then retrain a second proxy on the kept subset plus a random half of the pruned samples, and see whether the per-cluster loss-gap ranking of which clusters deserve less pruning survives. If the ranking flips for many clusters, or if AdaDeDup's mAP gain over its density baseline disappears when the swapped-back 5–10% of samples are chosen randomly instead of by the loss gap, the adaptive signal is not the cause of the reported results.","supporting_citations":[{"cited_title":"Sse: Multimodal semantic data selection and enrichment for industrial-scale data assimilation.ACM SIGKDD International Conference on Knowledge Discovery and Data Mining (KDD), 2025","cited_arxiv_id":null,"evidence_quote":"Defines VLM-SSE, the semantic-deduplication baseline AdaDeDup extends and compares against; also supplies the specialized autonomous-driving caption prompt used for Waymo and nuScenes."},{"cited_title":"Faster r-cnn: Towards real-time object detection with region proposal networks.IEEE transactions on pattern analysis and machine intelligence, 39(6):1137–1149, 2016","cited_arxiv_id":null,"evidence_quote":"Faster R-CNN, the detector trained on COCO, showing the method transfers beyond a single object-detection architecture."}],"review_version":1}