{"id":"41215705-4696-438f-8df0-bb01cd5c1be8","arxiv_id":"2509.10367","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":4.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":0,"one_line_summary":"Dataset condensation is reframed as minimizing distribution discrepancies, and existing methods are sorted into a taxonomy; no new algorithm or experiments are provided.","lead":"This paper proposes a mathematical framework that treats dataset condensation as matching two probability distributions with discrepancy measures, and uses that framework to organize existing condensation methods into a taxonomy. A general reader might care because it gives the field a shared language for comparing methods that have been developed heuristically.","discovery_kind":"review","skeptic_critique":{"model":"deepseek-v4-flash","headline":"GD≤2D holds only when the discrepancy hypothesis class contains the trained models h*_μ1 and h*_μ2; practical distribution matching uses a small random-batch hypothesis class that excludes them, so the bound does not transfer to the methods the framework classifies.","rationale":"The reader's weakest_assumption identifies the same gap in Section 3 and Section 5.1.4: the theoretical bound assumes H contains both trained models, but the practical approximation of H by random/pretrained networks generally does not contain them. My read agrees. The central claim of the paper is that existing DC methods can be unified under a discrepancy-minimization framework where minimizing D bounds GD. For that claim to be substantiated, the method actually used to minimize D must contain the trained model in its hypothesis class, or at least the approximation must not undercut the bound. Since the practical H_batch is a strict, finite, randomly-sampled subset of the full H, the practical objective is a lower bound on the true D, and the guarantee fails in the direction needed. This is a genuine correctness risk for the paper's theoretical framing, separate from the taxonomy itself. The taxonomy may still be useful, but the conditional verdict is appropriate: the authors should either show that the practical H_batch sufficiently approximates the full class (e.g., via concentration or universality) or restrict their theoretical claims to the idealized setting. I see no reason to move to REJECT because the formal bound is correct under its stated assumption; the issue is the applicability to the methods surveyed. Hence CONDITIONAL, matching the reader's verdict.","tokens_in":27803,"tokens_out":5718,"duration_ms":60437,"concrete_test":"Run a standard distribution matching condensation (Zhao and Bilen 2023) on CIFAR-10 with M=1 image/class. After optimization, train h*_S on the synthetic set and evaluate E_practical = max_{h∈H_batch} |φ(h, μ_T)-φ(h, μ_S)| over the same random-batch models used during condensation, and E_star = |φ(h*_S, μ_T)-φ(h*_S, μ_S)|. If E_star > 2·E_practical (or even E_star > E_practical) for a majority of random seeds, then the practical discrepancy does not bound the term involving the trained model, so GD ≤ 2D cannot hold with H_batch as the hypothesis class. Report the distribution of E_star/E_practical over at least 10 seeds.","verdict_should_be":"CONDITIONAL","load_bearing_attack":"The central theoretical result is GD(μ1, μ2; H) ≤ 2·D(μ1, μ2; H) (Section 3), asserted under 'assuming H contains both h*_μ1 and h*_μ2'. In the formal DC problem, h*_μ2 is defined as an element of H, so the assumption is automatically satisfied for the idealized hypothesis class. The load-bearing issue is that the practical algorithms catalogued in Section 5.1.4 (distribution matching, gradient matching, higher-order moment matching, etc.) do not optimize D over that full class H. Instead, they approximate the supremum over H by 'a batch of randomly initialized or pretrained models' (Section 5.1.4). This batch is a tiny finite subset H_batch. The model actually trained on the synthetic set, h*_S, is essentially never in H_batch. Consequently, the practical objective minimizes sup_{h∈H_batch} |φ(h, μ_T)-φ(h, μ_S)|, which is a lower bound on the true D(μ_T, μ_S; H). Minimizing a lower bound does not drive D small, and the chain |φ(h*_S, μ_T)-φ(h*_S, μ_S)| ≤ D(μ_T, μ_S; H) cannot be invoked because h*_S ∉ H_batch. Thus the theoretical guarantee GD ≤ 2D does not apply to the very methods presented as instances of the discrepancy-minimization framework. The paper explicitly notes the containment assumption, but it never reconciles it with the practical approximation; the taxonomy remains descriptive, while the claimed principled foundation for these methods is unsupported by the bound. This is not an internal inconsistency in the proof, but a theory–practice gap that undermines the central claim that minimizing the distribution discrepancy used in practice bounds the generalization loss difference.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes a unified discrepancy-based framework for dataset condensation (DC). It introduces several discrepancy notions—generalization discrepancy (GD), value discrepancy (VD), parameter discrepancy (PD), distribution discrepancy (DD), and characteristic discrepancy (CD)—and derives a hierarchy of upper bounds GD ≤ L·VD ≤ LC·PD and GD ≤ 2·D, where D is an IPM-type discrepancy over a hypothesis class H that contains the trained models. The framework is then used to organize a broad taxonomy of existing DC methods according to the discrepancy metric, the space in which matching/optimization is performed, and additional tricks such as augmentation and regularization. The paper also extends the discussion to privacy- and robustness-aware condensation. The central theoretical bound is sound under its stated containment assumption: the displayed inequality chain in Section 3 is justified because the optimality of h*_μ1 and h*_μ2 makes both the base difference and the added term nonnegative.","tokens_in":28222,"tokens_out":9943,"duration_ms":114494,"significance":"If taken as a high-level organizing framework, the paper provides a useful common language for comparing dataset condensation methods and clarifies that many existing heuristics can be viewed as variants of distributional discrepancy minimization. Its strengths are the breadth of covered methods, the explicit formalization of DC as discrepancy minimization, and the extension to multi-objective settings such as privacy and robustness. The paper does not provide new algorithms or empirical validation, which is acceptable for a framework/taxonomy paper. However, the normative claim that existing practical methods inherit the theoretical generalization bound is not established; the practical algorithms optimize a lower-bound approximation of the discrepancy over a small hypothesis class, so the theoretical guarantee does not transfer as stated.","major_comments":[{"comment":"The generalization surrogate bound GD(μ1, μ2; H) ≤ 2·D(μ1, μ2; H) in Section 3 requires h*_μ1, h*_μ2 ∈ H. In the practical distribution matching, gradient matching, and higher-order moment matching methods described in Section 5.1.4, the supremum over H is replaced by a batch of randomly initialized or pretrained models H_batch. The model actually trained on the synthetic data, h*_S, is generally not in H_batch. Since D(·, ·; H_batch) ≤ D(·, ·; H), minimizing the batch discrepancy need not drive the true discrepancy small, and the chain |φ(h*_S, μ_T) − φ(h*_S, μ_S)| ≤ D(μ_T, μ_S; H) cannot be invoked. The paper should either state explicit conditions on H_batch under which the bound transfers, or soften the claim that these methods are principled instances of the theoretical framework beyond a descriptive taxonomy.","section":"§5.1.4 and §3"},{"comment":"The text states that both MMD and the Wasserstein metric are special cases of IPM. This is true for W_1 via the Kantorovich–Rubinstein duality, but false for p > 1: W_p is not generally representable as sup_{h∈H} |E_{μ1} h − E_{μ2} h| for a fixed function class. Proposition 2.1(3) also asserts injectivity for W_p for all p∈[1,∞], which is true as a metric property but does not make W_p an IPM. Since the paper's discrepancy definition and the taxonomy (e.g., the Wasserstein-distance DC method of Liu et al.) only need p = 1, the characterization should be restricted to W_1 or the IPM claim should be reformulated.","section":"§2, canonical metrics and Prop. 2.1(3)"}],"minor_comments":[{"comment":"The text refers to 'Theorem 2.1' but only Proposition 2.1 exists. The reference should be corrected.","section":"§4, first sentence"},{"comment":"The proof claims that the indicator 1_U of an arbitrary open set U can be uniformly approximated by bounded continuous functions. This is false in general; a continuous function cannot be uniformly close to a discontinuous indicator unless the set is clopen. The injectivity result itself is standard and can be proven correctly, but the proof as written needs revision.","section":"§2, Proposition 2.1 proof"},{"comment":"In the gradient matching and higher-order moment matching objectives, expressions such as ∇h(T) are used without defining whether ∇h(D) denotes the gradient of the empirical loss averaged over D or something else. Please define this notation precisely.","section":"§5.1.4, notation"},{"comment":"The displayed definition of the outer loss, '˜L(h*_S, S) := L(h*_S, T)', seems to use the wrong argument (S rather than T). It should likely be a function of S that evaluates the model's loss on T; clarify or correct.","section":"§5.1.1, CIG"},{"comment":"In the last row, the second column is labeled 'Latent space R^n' and the minimization is over Z⊆R^n; this appears to be a typo for R^m, since the other latent-space entries optimize over latent vectors. Please check the row/column labels.","section":"Table 3"}],"recommendation":"major_revision","confidential_remarks":"The paper is essentially a survey and taxonomy with a light theoretical wrapper. The central GD bound is correct under the stated assumption, contrary to the reader's initial concern, but the theory–practice gap in Section 5.1.4 is real and should be addressed before the paper can claim to provide a 'principled foundation' for practical DC methods. The Wasserstein/IPM mischaracterization is also easy to fix. With these revisions the paper would be a useful contribution to the DC literature."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"This paper is best read as a survey with a unifying vocabulary: it sorts dataset condensation methods by the discrepancy they minimize (GD, VD, PD, CD, DD) and by the space they match and optimize in. That organizational work is genuinely useful—someone entering the field gets a clear map, and the change-of-space discussion via push-forwards is a nice conceptual frame. The literature coverage is broad and the taxonomy in Table 4 is a reasonable reading of the cited methods. There are no experiments, which is fine for this kind of paper.\n\nThe problems are mathematical and load-bearing. The proof of GD ≤ 2D contains an invalid inequality: inside the absolute value, the paper replaces φ(h*_μ2, μ2) − φ(h*_μ1, μ1) with φ(h*_μ1, μ2) − φ(h*_μ1, μ1). Since h*_μ2 minimizes over μ2, the first quantity is ≤ the second, but replacing a term inside an absolute value with a larger term can decrease the absolute value. The inequality chain is simply not valid. Also, the paper states that MMD and the Wasserstein metric are special cases of IPMs, but only the 1-Wasserstein has the Lipschitz dual; W_p for p > 1 is not an IPM in that sense.\n\nThe stress-test critique also lands. The theoretical bound assumes H contains both h*_μ1 and h*_μ2, which is fine in the idealized statement. But the practical distribution matching methods in Section 5.1.4 approximate the supremum over H by a small batch of random or pretrained models, H_batch. That batch generally does not contain the model eventually trained on the synthetic data. Minimizing sup over H_batch is minimizing a lower bound on the true D, and the chain |φ(h*_S, μ_T) − φ(h*_S, μ_S)| ≤ D(μ_T, μ_S; H) cannot be invoked when h*_S ∉ H_batch. The paper notes the containment assumption but never squares it with its own description of the practical algorithms. So the claimed principled foundation for these methods is not supported by the bound; the taxonomy remains descriptive.\n\nThere are also minor typos and a section on privacy/robustness that is really a list of related work rather than a consequence of the framework.\n\nWho is this for? Researchers who want a one-stop map of dataset condensation methods and a common terminology. Not for someone seeking a new theorem or a rigorous foundation. The paper deserves a serious referee because the taxonomy is useful and the flaws are fixable, but it needs major revision: correct the proof, fix the Wasserstein claim, and be upfront about the gap between the theory and the practical surrogates. If the authors cannot repair the bound, they should reframe the paper as a survey and stop presenting the bound as the theoretical justification for the methods it classifies.","headline":"Useful taxonomy of dataset condensation as discrepancy minimization, but the central proof is flawed and the theory does not actually cover the practical methods it classifies.","tokens_in":789,"tokens_out":1020,"would_cite":false,"duration_ms":46315,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Dataset condensation is best understood not as a bag of heuristics but as the problem of choosing a synthetic distribution that minimizes a statistical discrepancy against the original data, and the paper proves that this discrepancy direct","keywords":["dataset condensation","distribution discrepancy","integral probability metric","generalization discrepancy","distribution matching","gradient matching","dataset distillation","coreset selection"],"falsifier":"On a fixed binary classification task, take a batch of random feature networks as H, and construct a synthetic set whose empirical MMD (or gradient-matching loss) to the real data is very small but whose freshly trained model loses more than twice that discrepancy in accuracy on the real test set; this would show the bound's practical surrogate fails exactly where the assumption on H is violated.","tokens_in":27687,"feed_emoji":"🎯","tokens_out":5193,"duration_ms":52921,"temperature":0.7,"pith_summary":"This paper argues that the many heuristic methods for dataset condensation—building a small synthetic dataset whose trained models match the accuracy of models trained on the full data—are all instances of one formal problem: choose a synthetic distribution that minimizes a discrepancy against the original distribution. It defines the generalization discrepancy as the drop in test performance when training on synthetic rather than real data, and shows this drop is bounded by twice the distribution discrepancy, making discrepancy minimization a principled surrogate for the original bi-level objective. From this it derives a hierarchy of surrogate measures—value discrepancy and parameter discrepancy—and shows how existing condensation methods correspond to choosing one of these discrepancies together with a space (input or latent) in which to match. The payoff is a common vocabulary for comparing methods, an explicit statement of their shared assumptions, and a route to extending condensation beyond accuracy toward robustness and privacy.","feed_headline":"A single discrepancy bound explains dataset condensation methods","feed_subtitle":"Generalization loss is bounded by distribution discrepancy, unifying distillation, gradient, and trajectory matching.","key_machinery":"The load-bearing construction is a family of distribution discrepancies over a hypothesis class H. Generalization discrepancy (GD) compares the test loss of the model trained on synthetic data with that of the model trained on real data, both evaluated on the real distribution. The paper bounds GD by 2·D, where D is an integral probability metric or MMD/Wasserstein distance; it then introduces value discrepancy (VD, the sup-norm difference between trained models) and parameter discrepancy (PD, the difference in trained parameters), giving the chain GD ≤ L·VD ≤ LC·PD. The other essential piece is the change-of-space machinery: matching in a latent space is valid only when the encoder is injec","core_discovery":"The central claim is that dataset condensation should be formalized as a bi-level optimization problem in which the outer loop seeks a synthetic distribution µ2 whose optimally trained model performs well on the original distribution µ1. The paper proves GD(µ1, µ2; H) ≤ 2·D(µ1, µ2; H), where GD is the absolute difference between the generalization losses of the two optimal models and D is a model-agnostic distribution discrepancy such as an integral probability metric. It further derives GD ≤ L·VD ≤ LC·PD, so matching learned outputs or parameters is a legitimate surrogate. The paper then organizes a wide range of existing DC methods—distribution matching, gradient matching, trajectory match","pith_inferences":["The bound's assumption that H contains both trained models is typically unmet in practical distribution matching, which samples random or pretrained networks; this suggests the guarantee should be read as a design principle rather than a certificate for a specific trained model.","The taxonomy implies a testable ordering: methods that minimize a task-coupled discrepancy (GD/VD/PD) should control generalization more reliably than methods that minimize a purely distributional discrepancy such as characteristic discrepancy, which carries no generalization guarantee.","A natural benchmark would compare methods by fixing the discrepancy metric and data space, isolating the effect of auxiliary regularization terms (diversity, contrastive, curvature) rather than comparing whole pipelines."],"forward_implications":["Minimizing any distribution discrepancy—IPM, MMD, or Wasserstein—between synthetic and real empirical distributions is a surrogate for minimizing the generalization gap, up to the factor-2 bound.","Feature matching, gradient matching, and trajectory matching are not separate ideas; they are the same condensation objective expressed through value or parameter discrepancy.","Condensation in a latent space incurs a fidelity cost: without an injective encoder or perfect reconstruction, the induced discrepancy may no longer be a true metric, so some information loss is unavoidable.","The same discrepancy-minimization formulation handles objectives beyond generalization, such as adversarial robustness (by changing the loss to an adversarial loss) and differential privacy (by adding calibrated noise during matching)."],"fun_headline_variants":["One bound unifies all dataset condensation methods","Dataset condensation: one discrepancy bound to rule them all","A single inequality explains condensation methods","How dataset condensation works: one bound","Discrepancy bound ties together condensation techniques"],"cache_read_input_tokens":2304,"weakest_assumption_plain":"The generalization bound assumes that the hypothesis class H used to define the discrepancy contains both the model trained on the synthetic distribution and the model trained on the original distribution; practical methods replace H with a batch of random or pretrained networks, which generally does not contain the final trained model.","fun_headline_variants_meta":{"raw":{"variants":["One bound unifies all dataset condensation methods","Dataset condensation: one discrepancy bound to rule them all","A single inequality explains condensation methods","How dataset condensation works: one bound","Discrepancy bound ties together condensation techniques"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000135,"raw_usage":{"total_tokens":954,"prompt_tokens":694,"completion_tokens":260,"prompt_tokens_details":{"cached_tokens":256},"prompt_cache_hit_tokens":256,"prompt_cache_miss_tokens":438,"completion_tokens_details":{"reasoning_tokens":195}},"tokens_in":438,"tokens_out":260,"duration_ms":4201,"temperature":1.0,"reasoning_tokens":195,"cache_read_input_tokens":256,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-04T17:52:10.459125+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"On a fixed binary classification task, take a batch of random feature networks as H, and construct a synthetic set whose empirical MMD (or gradient-matching loss) to the real data is very small but whose freshly trained model loses more than twice that discrepancy in accuracy on the real test set; this would show the bound's practical surrogate fails exactly where the assumption on H is violated.","supporting_citations":[],"review_version":1}