{"id":"366ad754-163c-4d79-b17c-83f91c754b28","arxiv_id":"2606.21260","paper_version":1,"verdict":"UNVERDICTED","confidence":"LOW","novelty_score":5.0,"correctness_risk":"unknown","formal_verification":"none","parameter_count":0,"one_line_summary":"Derives an optimal subsampling probability for Horvitz-Thompson reweighted kernel ridge or SVM estimators in RKHS and validates it numerically.","lead":"This paper studies subsampling for kernel-based supervised learning by reweighting the empirical risk with Horvitz-Thompson weights and derives an optimal subsampling scheme based on the trace of the covariance operator that can be implemented via plug-in. A smart generalist might read it to learn practical ways to cut computation and energy use when training on very large datasets while preserving statistical properties.","discovery_kind":"new_method","skeptic_critique":{"model":"grok-4.3","headline":"Optimality of trace-based subsampling requires explicit domination of asymptotic covariance by that term","rationale":"The reader's weakest_assumption correctly isolates the unverified step from asymptotics to optimality criterion. No stronger internal inconsistency appears from the supplied abstract and claim description; the concern is therefore the same one.","tokens_in":1649,"tokens_out":269,"duration_ms":15010,"concrete_test":"Extract the asymptotic expansion of the estimator (likely in the section deriving the limiting distribution) and verify whether the covariance is exactly or asymptotically proportional to the trace term alone; recompute the optimal probabilities under the full covariance expression and check whether they differ from the trace-only rule by more than a constant factor.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim derives an optimal subsampling distribution from the asymptotics of the Horvitz-Thompson reweighted ERM in the RKHS. This requires that the asymptotic covariance (or its leading term) is dominated by the trace of the covariance operator in a form that can be directly minimized w.r.t. the inclusion probabilities. The abstract states this link but supplies no explicit expansion or bound showing the trace term controls the variance uniformly over sampling schemes; without that, the optimality criterion does not necessarily follow from the asymptotics.","agreement_with_reader":"agree"},"referee_report":{"model":"grok-4.3","summary":"The manuscript analyzes subsampling for supervised learning when the hypothesis class is a reproducing kernel Hilbert space. It considers the Horvitz-Thompson reweighted empirical risk minimizer, derives its asymptotic properties, and uses those properties to identify an optimal subsampling distribution that depends on the trace of the covariance operator; the distribution is then estimated by plug-in. A numerical study on synthetic and real data is included to illustrate practicality.","tokens_in":1770,"tokens_out":431,"duration_ms":19694,"significance":"If the asymptotic expansion is valid and the trace term indeed governs the leading variance uniformly across sampling schemes, the work supplies a statistically grounded, computationally attractive subsampling rule for kernel methods. The plug-in implementation and the explicit link between asymptotics and sampling probabilities would be a useful contribution to large-scale nonparametric learning.","major_comments":[{"comment":"Abstract (and the paragraph on asymptotic properties): the optimality claim rests on the assertion that the trace of the covariance operator dominates the asymptotic covariance of the reweighted estimator. No explicit expansion, bound, or set of conditions (e.g., on the kernel, the sampling probabilities, or the regression function) is supplied in the abstract showing that this domination holds uniformly; without that step the optimality criterion does not necessarily follow from the asymptotics.","section":"Abstract"},{"comment":"The transition from the asymptotic covariance expression to the trace-based objective appears to treat the trace as an external functional that can be minimized directly with respect to the inclusion probabilities. If the covariance expansion contains additional terms that also depend on the sampling scheme, the claimed optimality may be incomplete; the manuscript should state the precise domination argument and any uniformity conditions.","section":"asymptotic analysis section"}],"minor_comments":[{"comment":"Notation for the Horvitz-Thompson weights and the RKHS inner product should be introduced once and used consistently; several symbols appear without prior definition in the abstract.","section":null}],"recommendation":"major_revision","confidential_remarks":null},"author_rebuttal":{"model":"grok-4.3","summary":"We thank the referee for the thoughtful and constructive report. The comments highlight the need for greater explicitness regarding the domination argument in both the abstract and the asymptotic analysis. We address each point below and will revise the manuscript accordingly to strengthen the presentation.","responses":[{"response":"We agree that the abstract, being a concise summary, does not reproduce the full technical conditions. The asymptotic expansion and the uniform domination of the trace term are established in Theorem 2 under Assumptions 1--3 (bounded kernel, moment conditions on the regression function, and sampling probabilities bounded away from zero and one). These ensure the remainder is o(1) uniformly over admissible sampling schemes. To address the concern, we will revise the abstract to include a brief reference to these conditions and the resulting optimality criterion.","revision_made":"yes","referee_comment":"[Abstract] Abstract (and the paragraph on asymptotic properties): the optimality claim rests on the assertion that the trace of the covariance operator dominates the asymptotic covariance of the reweighted estimator. No explicit expansion, bound, or set of conditions (e.g., on the kernel, the sampling probabilities, or the regression function) is supplied in the abstract showing that this domination holds uniformly; without that step the optimality criterion does not necessarily follow from the asymptotics."},{"response":"The asymptotic covariance derived in Theorem 1 takes the form trace(Σ_p) + R_n, where the remainder R_n is shown to be o_p(1) uniformly when the inclusion probabilities satisfy inf p_i ≥ c > 0 and the kernel is continuous and bounded. Because the leading term depends on the sampling distribution only through the trace functional, minimization with respect to the inclusion probabilities is valid. We will insert an explicit remark immediately after Theorem 1 stating the domination argument, the uniformity conditions, and why no other sampling-dependent terms enter at the leading order.","revision_made":"yes","referee_comment":"[asymptotic analysis section] The transition from the asymptotic covariance expression to the trace-based objective appears to treat the trace as an external functional that can be minimized directly with respect to the inclusion probabilities. If the covariance expansion contains additional terms that also depend on the sampling scheme, the claimed optimality may be incomplete; the manuscript should state the precise domination argument and any uniformity conditions."}],"tokens_in":1280,"tokens_out":504,"duration_ms":13902,"standing_objections":[]},"desk_editor":{"model":"grok-4.3","letter":"The main takeaway is that the authors analyze the asymptotics of a Horvitz-Thompson reweighted empirical risk minimizer in an RKHS and extract an optimal subsampling distribution tied to the trace of the covariance operator, then implement it via plug-in. They also run a numerical study on synthetic and real datasets to check practicality.\n\nWhat stands out as new is the specific combination of the reweighting with an optimality criterion based on that trace in the RKHS setting. The numerical experiments add some value by showing the method can be applied without too much extra cost.\n\nThe soft spot sits in the move from the asymptotic expansion to the optimality claim. The abstract states that the properties reveal the optimal scheme regarding the trace, yet it is not obvious from the given material whether the covariance is controlled by that term alone or whether other contributions could alter the minimizing distribution. If the full proofs contain a clean domination argument or uniform bound, that concern disappears; otherwise the optimality result rests on an unverified step.\n\nThis work is for people already working on scalable nonparametric methods or subsampling schemes in kernel settings. A reader who wants a theoretically motivated way to pick inclusion probabilities for large kernel problems will get something concrete, though the advance is incremental rather than foundational.\n\nIt is worth sending to peer review. The core construction is coherent and the practical angle is present, so referees can check the asymptotics and the experiments in detail.","headline":"The paper derives a trace-based optimal subsampling rule for Horvitz-Thompson reweighted kernel ERM from asymptotics and backs it with numerics, but the step showing the trace dominates the asymptotic variance needs explicit bounds to hold up.","tokens_in":2253,"tokens_out":385,"would_cite":false,"duration_ms":19758,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"grok-4.3","headline":"Studying asymptotics of the Horvitz-Thompson reweighted empirical risk minimizer in an RKHS reveals an optimal subsampling scheme based on the trace of the covariance operator that works via plug-in.","keywords":["subsampling","reproducing kernel Hilbert space","Horvitz-Thompson estimator","empirical risk minimization","asymptotic analysis","optimal sampling","plug-in estimator","supervised learning"],"falsifier":"If the plug-in estimator of the trace-based subsampling probabilities fails to produce lower estimation error than uniform subsampling on large-scale data sets, the claimed optimality would be refuted.","tokens_in":2555,"feed_emoji":"","tokens_out":632,"duration_ms":19435,"temperature":0.7,"pith_summary":"The paper focuses on subsampling to lower the cost of training nonparametric models whose hypotheses lie in a reproducing kernel Hilbert space. It considers the Horvitz-Thompson reweighted version of empirical risk minimization and derives its asymptotic behavior. From that analysis the authors obtain an explicit optimal subsampling distribution expressed in terms of the trace of the covariance operator. The distribution is shown to be realizable by a plug-in estimator that does not require knowledge of the unknown measure. Experiments on synthetic and real data illustrate that the resulting procedure reduces computation while retaining statistical performance.","feed_headline":"Optimal kernel subsampling follows from covariance trace","feed_subtitle":"Asymptotic study of reweighted risk minimizers produces a plug-in scheme tied to the trace of the covariance operator.","key_machinery":"Horvitz-Thompson reweighted empirical risk minimizer in a reproducing kernel Hilbert space, whose asymptotic covariance supplies the optimality criterion via the trace of the covariance operator.","core_discovery":"By studying the asymptotic properties of the Horvitz-Thompson reweighted empirical risk minimizer in an RKHS, the authors reveal an optimal subsampling scheme regarding the trace of the covariance operator and show that it can be used via plug-in.","pith_inferences":["The same trace-based criterion could be tested in other nonparametric settings that admit a covariance operator.","Implementation would allow direct comparison of variance reduction against uniform or leverage-score sampling on benchmark data sets.","The method supplies a concrete route to lower energy use in kernel-based training pipelines by reducing the number of kernel evaluations."],"forward_implications":["Subsampling probabilities are chosen proportional to the trace terms of the covariance operator for asymptotic optimality.","The optimal scheme is realized by a plug-in estimator that replaces unknown quantities with empirical estimates.","The procedure lowers computational cost for a fixed sample size while preserving the asymptotic efficiency of the full-data estimator.","Numerical comparisons on synthetic and real-world data confirm practical gains over standard subsampling."],"fun_headline_variants":["Kernel subsampling optimized by covariance trace","Covariance trace optimizes kernel subsampling","Optimal subsampling in RKHS via covariance trace","Trace of covariance guides plug-in kernel subsampling"],"cache_read_input_tokens":2112,"weakest_assumption_plain":"The asymptotic covariance of the estimator is dominated by the trace of the covariance operator in a way that directly yields an optimal subsampling distribution.","fun_headline_variants_meta":{"raw":{"variants":["Kernel subsampling optimized by covariance trace","Covariance trace optimizes kernel subsampling","Optimal subsampling in RKHS via covariance trace","Trace of covariance guides plug-in kernel subsampling"]},"model":"grok-4.3","cost_usd":0.004049,"raw_usage":{"total_tokens":2004,"prompt_tokens":554,"num_sources_used":0,"completion_tokens":53,"cost_in_usd_ticks":40487000,"prompt_tokens_details":{"text_tokens":554,"audio_tokens":0,"image_tokens":0,"cached_tokens":256},"completion_tokens_details":{"audio_tokens":0,"reasoning_tokens":1397,"accepted_prediction_tokens":0,"rejected_prediction_tokens":0}},"tokens_in":554,"tokens_out":53,"duration_ms":12143,"temperature":1.0,"reasoning_tokens":1397,"cache_read_input_tokens":256,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-06-26T12:45:57.318048+00:00","model_set":{"reader":"grok-4.3"},"falsifier":"If the plug-in estimator of the trace-based subsampling probabilities fails to produce lower estimation error than uniform subsampling on large-scale data sets, the claimed optimality would be refuted.","supporting_citations":[],"review_version":1}