{"id":"9389487c-5f30-43d3-ac7b-e236849487ff","arxiv_id":"2501.14253","paper_version":2,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":4,"one_line_summary":"DRCS derives an upper bound on worst-case validation error under covariate shift and greedily chooses a coreset that minimizes this bound.","lead":"The authors propose DRCS, a method that selects a small, robust training subset when the test distribution may shift away from training. It derives and minimizes an upper bound on the worst-case validation error under a weight-perturbation model, and tests it on tabular benchmarks and CIFAR-10.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Theorem 3.3's bound is false for the stated W′: Lemma C.6 silently imposes the constraint 1⊤w′=n′, and a valid w′=(0.6,1.3) with Q=0.5 exceeds the claimed bound.","rationale":"The reader's weakest_assumption identifies exactly the sum-constraint issue: the bound is proven only for the ball intersected with 1⊤w′=n′, while the stated W′ omits that constraint. Our stress test supplies a concrete numerical violation, confirming that the concern is not merely a typesetting slip but a genuine correctness gap in the central theorem as stated. Because the flaw is repairable by adding Eq. 36 to W′ and normalizing validation weights in the experiments, the conditional verdict is unchanged: this is a major revision, not an outright rejection. We see no other load-bearing objection: the parameter-ball argument from Hanada et al. is a legitimate basis, the L2-ball ambiguity model is a standard distributionally robust choice, and the missing error bars are a reporting weakness rather than a threat to the main claim. The paper's technical contribution—a duality-gap-based upper bound that can be minimized greedily—remains meaningful once the constraint is made explicit.","tokens_in":29645,"tokens_out":17155,"duration_ms":156373,"concrete_test":"Run the following exact check: set n′=2, Q=0.5, ζ=(1,0), and compute the claimed upper bound from Eq. 18. Then evaluate VaEr for w′=(0.6,1.3), which satisfies Eq. 10. If VaEr exceeds the bound, Lemma C.6 and Theorem 3.3 are invalid for W′ as defined. As a complementary check, re-derive Lemma C.6 without Eq. 36; the maximizer is no longer Eq. 50 and the closed-form Eq. 37 changes, confirming the missing constraint. If the authors instead intend W′ to include 1⊤w′=n′, they must state it in Eq. 10 and rerun the experimental guarantees with validation weights normalized to sum n′.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The load-bearing assumption is hidden in the proof, not the theorem statement. Theorem 3.3 (Eq. 18) claims to upper-bound the worst-case weighted validation error for the ambiguity set W′ defined in Eq. 10 as the bare L2 ball {w′ : ||w′−1_n′||_2 ≤ Q}. The proof, however, relies on Lemma C.6, whose constrained optimization problem (Eqs. 41–43) explicitly adds the sum constraint 1⊤_n′ w′ = n′ (Eq. 36). The closed form in Eq. 37, and hence Eq. 18, is the solution of that constrained problem, not of the stated one. This is not cosmetic: because the validation error VaEr = 1 − ζ⊤w′/1⊤w′ is scale-invariant, the L2 ball without the sum constraint contains different rays than the ball intersected with the hyperplane. Concretely, take n′=2, Q=0.5, ζ(v)=(1,0). Equation 18 gives WrVaErUB = 1 − (1 − 0.5·√(1−1/2))/2 ≈ 0.6768. The weight vector w′=(0.6,1.3) lies in the stated W′ (distance √(0.4²+0.3²)=0.5), yet VaEr = 1 − 0.6/1.9 ≈ 0.6842 > 0.6768. Thus the theorem as stated is false; the guarantee holds only for a smaller, unstated ambiguity set. Moreover, the experiments set validation weights to a for positives and 1 for negatives (Section 5.1), so Σw′ = n′ + n′_+(a−1) ≠ n′; the plotted 'Guarantee' curves are therefore not guaranteed lower bounds for the actual validation-weight perturbations used.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes Distributionally Robust Coreset Selection (DRCS) for choosing a training-data subset that performs well under an unknown covariate shift. The shift is modeled by L2 balls around uniform importance weights for both the training set (W, Eq. 5) and the validation set (W', Eq. 10). The central theoretical contribution is Theorem 3.3, which claims an upper bound WrVaErUB(v) on the worst-case weighted validation error, obtained by combining a model-parameter bound from a duality gap with a maximization over validation weights. The proposed algorithm greedily removes training instances that minimize this bound. Experiments on tabular data, CIFAR-10 features, and an NTK variant compare DRCS with standard coreset baselines and report both accuracy and a theoretical \"Guarantee\" curve.","tokens_in":30088,"tokens_out":7177,"duration_ms":69780,"significance":"If the bound in Theorem 3.3 were valid as stated, the paper would provide a useful pre-retraining certification tool for coresets under importance-weight uncertainty, and the greedy selection procedure would be a practical way to use that certificate. The paper is commendably concrete: it gives a complete proof outline, explicit algorithms, and extensive experimental comparisons, and the bound is derived from a duality gap rather than fitted to validation labels. However, the main theorem is currently stated for an ambiguity set that does not match the set used in the proof, and the experiments use validation weights that violate the proof's extra constraint. Because these issues affect the truth of the central guarantee and the validity of the plotted guarantee curves, the paper needs substantial revision before the claims can be accepted.","major_comments":[{"comment":"The claimed upper bound is not established for the ambiguity set W' defined in Eq. (10). The theorem states Eq. (18) for W'={w' : ||w'-1_{n'}||_2 <= Q}, but Lemma C.6 introduces an additional constraint 1^T w'=n' (Eq. 36), and the closed form in Eq. (37) is derived from that constrained problem in Appendix C.2. The extra constraint is not cosmetic because VaEr is scale-invariant: the L2 ball without the sum constraint contains different rays from the ball intersected with the hyperplane. Concretely, take n'=2, Q=0.5, and zeta=(1,0). Equation (18) gives WrVaErUB = 1 - (1 - 0.5*sqrt(1-1/2))/2 ≈ 0.6768. The vector w'=(0.6,1.3) lies in the stated W' (distance 0.5 from (1,1)) and yields VaEr = 1 - 0.6/1.9 ≈ 0.6842 > 0.6768. Thus Theorem 3.3 is false as stated. The problem definition in Eq. (10) and the theorem statement must either include the sum-to-n' constraint, or be replaced by a guarantee that is actually valid for the unconstrained L2 ball.","section":"Theorem 3.3 and Appendix C.1 (Lemma C.6)"},{"comment":"The experimental \"Guarantee\" curves are not lower bounds for the validation-weight perturbations actually used in the experiments. Section 5.1 sets validation weights to a for positive instances and 1 for negative instances, so 1^T w' = n' + n'_+(a-1), which is not equal to n' when a != 1. Since Eq. (18) and Lemma C.6 rely on the sum-to-n' constraint, the plotted guarantee does not cover the perturbations used in the accuracy comparisons. The authors should either normalize the validation weights so that their sum is n', recompute the guarantee curves under a correctly constrained W', or explicitly restrict the claim to a different perturbation model.","section":"Section 5.1 and Figures 4, 5"},{"comment":"There is a mismatch between the stated assumption and the formulas when the regularization is not L2. The theorem assumes that rho in P_{1_n,1_n} is mu-strongly convex, but Eq. (19) and Lemma C.2 use lambda, which is only defined for the L2-regularization case rho(beta) = (lambda/2)||beta||^2. For a general strongly convex rho, Lemma A.4 would give a radius of sqrt(2/mu)(P-P*), not sqrt(2/lambda) DG. The theorem should either be stated specifically for L2 regularization or the strong-convexity parameter should be used consistently throughout.","section":"Theorem 3.3 and Eq. (19)"}],"minor_comments":[{"comment":"The manuscript contains many typos and formatting errors, including \"calucuration\", \"mehods\", \"valdation\", \"varidation\", and \"Fisrt\". A careful proofread is needed before resubmission.","section":"General"},{"comment":"The caption says the vertical axis is the weighted validation accuracy minimized with respect to w' by using Eq. (43), but Eq. (43) is about the bound and depends on the sum-to-n' constraint. The relationship between the plotted quantity and Eq. (43) should be clarified.","section":"Figure 4 caption and Eq. (43)"},{"comment":"The set W' as defined is an L2 ball in R^{n'} and does not restrict w' to positive entries or exclude zero-sum vectors, although VaEr in Eq. (9) divides by the sum of w'. The paper should state positivity and rule out zero-sum weights, or explicitly restrict Q so that such cases cannot occur.","section":"Eq. (10) and Eq. (9)"},{"comment":"The text says Algorithm 1 removes the instance with the smallest DGi value, but the selection criterion is the overall upper bound WrVaErUB in Eq. (18), which also depends on the validation set through zeta. The pseudocode and the surrounding explanation should state whether DGi is being used as a surrogate and how zeta is updated after each removal.","section":"Section 5.2 / Algorithm 1"}],"recommendation":"major_revision","confidential_remarks":"The main theorem is false as stated, but the proof strategy is reusable once the sum-to-n' constraint is made explicit in the ambiguity set. The experiments currently do not respect that constraint, so the guarantee plots overstate the theoretical support. I would support a major revision that corrects the W' definition, updates the theorem and the experimental protocol accordingly, and re-runs the affected experiments. I would not accept the current version."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Quick take: the paper proposes a distributionally robust coreset selection method that certifies a worst-case validation error before retraining, using the duality-gap parameter ball. The idea is reasonable and the extension of Hanada et al.'s bound to coresets is new. But the main bound as stated is not correct: the proof assumes 1⊤w′ = n′ for validation weights, which is absent from the problem's W′. This is a real, load-bearing flaw, not a typo.\n\nWhat's new and good: DRCS is the first coreset-selection method I know with a worst-case validation-error guarantee under unknown covariate shift, at least for convex losses with strong convexity. The greedy algorithms are sensible, and the paper is honest about limitations (binary classification, convex models, NTK approximation for deep learning). The heavy reliance on Hanada et al. (2023, 2024) is fine; those results are published separately and the parameter-ball machinery is genuinely reused rather than re-derived ad hoc.\n\nThe soft spot is central. Lemma C.6 solves max_w′ VaEr under an explicit sum constraint, but W′ in Eq. (10) has no such constraint. Because VaEr is scale-invariant, the true worst-case over the stated ball can be higher. A concrete violation: take n′=2, Q=0.5, ζ=(1,0). Eq. (18) gives about 0.6768, but w′=(0.6,1.3) is in W′ and gives VaEr≈0.6842. The theorem as written is false. The fix is simple on paper—define W′ with the sum constraint—but the experiments also need rethinking: validation weights are set to a for positives and 1 for negatives, so their sum is not n′ unless a=1. The plotted 'Guarantee' curves are therefore not lower bounds on the actual evaluated accuracy.\n\nMinor issues: the square-root bracket in Eq. (19) is ambiguous (probably a typesetting error), and there are no error bars or repeated runs. These are fixable and secondary.\n\nWho it is for: readers in coreset selection, distributionally robust learning, and safe screening will care about the problem framing. The paper deserves referee time only if the authors are willing to correct the theorem and align the experiments with the corrected W′. I would not accept it as is, but I would not desk-reject it either—a good referee could help them fix a straightforward missing constraint. Worth a careful reading, not a citation yet.","headline":"Useful idea, false as written: the DRCS bound's proof assumes a sum-to-n' constraint on validation weights that is not in the stated ambiguity set.","tokens_in":30650,"tokens_out":6577,"would_cite":false,"duration_ms":58309,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Coreset selection can be made robust to unknown covariate shift by minimizing an analytically computable bound on worst-case validation error, rather than the error itself.","keywords":["coreset selection","distributionally robust optimization","covariate shift","worst-case validation error","duality gap","importance weighting","neural tangent kernel","data pruning"],"falsifier":"On a small binary dataset, fix a coreset $v$, compute $\\mathrm{WrVaEr}^{\\mathrm{UB}}(v)$ from Eq. (18), then sample or enumerate weight vectors $w\\in W$, retrain the logistic model for each, and evaluate weighted validation error under weights $w'\\in W'$; if any retrained model exceeds the computed bound, Theorem 3.3 is false. A lighter empirical check is to build a test set whose density ratio has L2 distance from uniform larger than $S$ and see whether the measured error exceeds the certificate.","tokens_in":29478,"feed_emoji":"🎯","tokens_out":9770,"duration_ms":80335,"temperature":0.7,"pith_summary":"This paper tries to establish that coreset selection can be made robust to unknown covariate shift by minimizing an analytically computable upper bound on the worst-case weighted validation error, rather than the true worst-case error, which is intractable. The bound is derived for binary classification with strongly convex regularized objectives, and the selection procedure greedily removes the training instance that most shrinks the bound. Because the bound is computed before retraining, every candidate subset carries a certificate of guaranteed worst-case accuracy within the assumed shift set. The paper also argues the method extends to deep networks through neural tangent kernels and fixed feature extractors, and reports experiments showing the selected subsets maintain higher weighted validation accuracy than standard coreset baselines under the shifts tested.","feed_headline":"One bound certifies coresets before retraining under covariate shift","feed_subtitle":"Deleting instances that shrink the worst-case bound keeps small subsets accurate when test data shifts","key_machinery":"The load-bearing object is the duality gap $DG(v,w)$, the difference between the weighted training objective at the full-data solution and the weighted dual objective at the full-data dual solution. Under $\\mu$-strong convexity of the regularizer, this gap controls the distance from the full-data solution to the parameters after retraining on the coreset: $\\beta^*(v,w)\\in B_{v,w}=\\{\\beta:\\|\\beta-\\beta^*_{1_n,1_n}\\|_2\\le \\sqrt{2/\\lambda}\\,DG(v,w)\\}$. The proof pushes this parameter sphere through the validation set, labeling each validation point surely correct, surely incorrect, or unknown according to whether its signed margin is strictly positive, negative, or straddling zero over the whole sphere; the worst case counts every unknown point as an error. Maximizing the resulting error over the training-weight ball $W=\\{w:\\|w-1_n\\|_2\\le S\\}$ and the validation-weight ball $W'=\\{w':\\|w'-1_{n'}\\|_2\\le Q\\}$ yields the closed-form upper bound, and for L2 regularization the inner maximization $\\max_{w\\in W}DG(v,w)$ reduces to a tractable convex quadratic optimization problem.","core_discovery":"The central claim is Theorem 3.3: for a coreset vector $v$, the worst-case weighted validation error $\\mathrm{WrVaEr}(v)=\\max_{w\\in W}\\mathrm{VaEr}(v,w)$ is bounded above by $\\mathrm{WrVaEr}^{\\mathrm{UB}}(v)$ given in Eq. (18). The bound is built from a set $\\zeta(v)$ of validation points judged “surely correct” by comparing each point's margin against the radius $\\sqrt{2/\\lambda}\\,\\max_{w\\in W}DG(v,w)$, where $DG(v,w)=P_{v,w}(\\beta^*_{1_n,1_n})-D_{v,w}(\\alpha^*_{1_n,1_n})$ is the duality gap between the weighted primal objective evaluated at the full-data solution and the weighted dual objective at the full-data dual solution. Strong convexity of the regularizer ensures that retrained parameters $\\beta^*(v,w)$ lie inside a sphere centered at $\\beta^*_{1_n,1_n}$ with that radius, so a validation point is surely correct only if its signed margin exceeds the maximal possible disturbance. The paper then turns the bound into a selection rule: remove, one at a time, the training instance that most reduces the bound, using three greedy variants that trade recomputation cost against fidelity.","pith_inferences":["Switching the uncertainty set from an L2 ball of importance weights to a Wasserstein or KL ball would require a new worst-case computation for $\\max_{w\\in W}DG(v,w)$, since tractability here relies on the quadratic structure that L2 regularization creates.","The three greedy rules are heuristics; an experiment comparing them with exhaustive search on small $n$ would show how much of the bound-minimizing objective is lost by greediness.","The guarantee is for the validation set inside the assumed shift set, so its practical force depends on that set actually containing the deployment distribution; a shift outside the L2 balls is invisible to the certificate."],"forward_implications":["Any fixed coreset can be audited before retraining: evaluating Eq. (18) gives a guaranteed worst-case weighted validation accuracy over the chosen shift balls.","Removing the instance that most reduces the bound gives a concrete construction of a robust subset, and the three proposed greedy algorithms trade recomputation cost against fidelity to that rule.","The bound and selection procedure apply to binary classification with any strongly convex regularized objective, including kernel logistic regression and SVM.","For deep networks, approximating the model with a neural tangent kernel or a fixed feature extractor brings the selection into the same convex regime.","The achievable deletion ratio and the tightness of the guarantee are governed by the regularization strength $\\lambda$, with stronger regularization yielding tighter parameter bounds but weaker underlying models."],"supporting_citations":[{"why":"Supplies the model-parameter bound: after retraining, beta*(v,w) lies in an L2 hypersphere whose radius is governed by the duality gap.","marker":"Hanada et al. (2023)"},{"why":"Provides the strong-convexity inequality used to turn objective suboptimality into a parameter-distance bound (Lemma A.4).","marker":"Ndiaye et al. (2015)"},{"why":"Supplies Fenchel's duality theorem and convex-analysis facts used to derive the dual problem and the duality gap.","marker":"Rockafellar (1970)"},{"why":"Gives the generalized representer theorem that lets the method be kernelized.","marker":"Schölkopf et al. (2001)"},{"why":"Provides the neural-tangent-kernel machinery used to extend the bound to deep networks.","marker":"Novak et al. (2020)"},{"why":"Supplies the LIBSVM tabular datasets used in the main experiments.","marker":"Chang & Lin (2011)"}],"fun_headline_variants":["Worst-case bound guides robust data subset choice","Coresets that survive unknown test shifts","Pick data points by shrinking a worst-case error bound","Robust coreset selection for unknown future distributions","Bound-driven coreset selection fights covariate shift"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The certificate holds only if the true deployment distribution corresponds to importance weights inside an L2 ball around the uniform training and validation weights, and the proof of the bound also requires the validation weights to sum to $n'$; shifts outside those balls, or validation-weight choices violating the sum constraint, are not covered by the guarantee.","fun_headline_variants_meta":{"raw":{"variants":["Worst-case bound guides robust data subset choice","Coresets that survive unknown test shifts","Pick data points by shrinking a worst-case error bound","Robust coreset selection for unknown future distributions","Bound-driven coreset selection fights covariate shift"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000585,"raw_usage":{"total_tokens":2780,"prompt_tokens":1007,"completion_tokens":1773,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":623,"completion_tokens_details":{"reasoning_tokens":1702}},"tokens_in":623,"tokens_out":1773,"duration_ms":11808,"temperature":1.0,"reasoning_tokens":1702,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-10T15:15:44.039020+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"On a small binary dataset, fix a coreset $v$, compute $\\mathrm{WrVaEr}^{\\mathrm{UB}}(v)$ from Eq. (18), then sample or enumerate weight vectors $w\\in W$, retrain the logistic model for each, and evaluate weighted validation error under weights $w'\\in W'$; if any retrained model exceeds the computed bound, Theorem 3.3 is false. A lighter empirical check is to build a test set whose density ratio has L2 distance from uniform larger than $S$ and see whether the measured error exceeds the certificate.","supporting_citations":[{"cited_title":"splice”). • In the table, the column “d","cited_arxiv_id":null,"evidence_quote":"Supplies the LIBSVM tabular datasets used in the main experiments."}],"review_version":1}