{"id":"2f6e207e-1c1e-4f89-8cef-4e4e321737db","arxiv_id":"2607.18524","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":7,"one_line_summary":"Geometry-only leverage scoring reaches full-dataset accuracy for linear ACE potentials with 30-40% of the DFT labels (2-3x less) on ASSYST pools, including defect-level tests.","lead":"This paper asks which atomic configurations deserve the expensive quantum-mechanics calculations when training machine-learned interatomic potentials, using only geometry. It finds that leverage-based picks reach the same accuracy with about a third of the data as random picks, including for defects.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Eq. (21) sums leverage over rows; ASSYST cells span 1–10 atoms (4–31 rows), so block-CUR's gain may be cell-size selection rather than descriptor-space coverage.","rationale":"The reader's weakest-assumption analysis already targeted Eq. (21) and the cell-size justification, and I agree that this is the most load-bearing point. The paper is otherwise carefully designed: label-driven baselines have strictly more information, hyperparameters are fixed, and independent DFT validation is present. But the central claim is not merely that some geometry-only scheme saves DFT effort; it is that leverage scores capture descriptor-spanning information. Because Eq. (21) is unnormalized and ASSYST explicitly contains 1–10 atom cells, the configuration score is confounded with cell size. This affects the interpretation of the main learning-curve advantage and the defect-fidelity results (Table II). The proposed size-matched random baseline is a concrete, inexpensive check that would resolve the ambiguity. Since this is a gap in evidence rather than an internal inconsistency, the appropriate verdict remains CONDITIONAL, matching the reader's original assessment.","tokens_in":16670,"tokens_out":5175,"duration_ms":65177,"concrete_test":"Re-run the Al learning-curve comparison (Fig. 2) with a stratified random baseline that matches block-CUR's selected cell-size distribution at each labeled fraction: sample uniformly within N-atom strata in the same proportion as the cells chosen by block-CUR. If the size-matched random learning curve overlaps block-CUR's energy/force RMSE, the advertised advantage is driven by cell-size selection through Eq. (21). If block-CUR remains clearly better under this control, the concern is refuted.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central mechanistic claim—that geometry-only leverage sampling improves data efficiency by spanning descriptor space—rests on the configuration-level score in Eq. (21), which sums row-level regularized leverage over all energy and force rows of a configuration. But ASSYST cells range from 1 to 10 atoms (Table I), so |I_R| = 1 + 3N ranges from 4 to 31, an eightfold variation. Summed leverage will tend to grow with cell size unless per-row leverage scores are strongly anti-correlated with N. The text justifies Eq. (21) by asserting that 'variations in |I_R| are limited,' which is contradicted by the paper's own Table I. If larger cells dominate the ranking, block-CUR preferentially selects configurations with more force rows per DFT calculation, which can improve force accuracy and defect-related observables without any true descriptor-space coverage advantage. Random sampling draws uniformly from the pool, so its cell-size distribution mirrors the pool, while block-CUR's selected subsets are enriched in high-|I_R| cells. Thus the reported 2–3x reduction (Fig. 2) and the large defect-energy improvements (Table II) could be partly an artifact of the unnormalized aggregation in Eq. (21), not evidence for leverage-guided geometric coverage. A size-matched random baseline would separate these explanations; without it, the central interpretation is not secure.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes using regularized leverage scores computed from the ACE design matrix to select subsets of ASSYST-generated configurations for DFT labeling, without access to energy or force labels. It compares two leverage-guided strategies, CUR and block-CUR, against random, energy-based, and force-based selection under an iterative labeling protocol for elemental Al, Cu, and Al-Cu alloys. The central claim is that leverage-guided selection reaches plateau-level accuracy at labeled fractions of roughly 30-40% for Al, versus 75-80% for random sampling, corresponding to a 2-3x reduction in DFT-labeled configurations, while preserving defect-level transferability.","tokens_in":16889,"tokens_out":4928,"duration_ms":57963,"significance":"If the central claim holds, this is a practically valuable result: it offers a label-free, geometry-only criterion for reducing DFT labeling cost in linear ACE model training, which is directly relevant to MLIP workflow design. The study has several strengths: a fixed ACE architecture across all methods, five-seed repeated runs, hold-out test sets, an independent 360-config DFT validation set for Al, and label-driven baselines that have strictly more information than the label-free methods. The Al learning-curve numbers (30-40% vs 75-80%) are striking. However, the mechanistic interpretation and the workload-reduction claim depend on the unnormalized configuration-level leverage definition in Eq. (21), which is not adequately justified and requires an additional control experiment before the central claim is secure.","major_comments":[{"comment":"The definition L_R = Σ_{i∈I_R} h_i is justified by the statement that 'variations in |I_R| are limited'. Table I contradicts this: ASSYST cells contain 1–10 atoms per element, so |I_R| = 1 + 3N_R spans 4 to 31, an eight-fold variation. Since block-CUR samples with probability proportional to L_R (Eq. 23), it will systematically favor configurations with more force rows unless per-row leverages are strongly anti-correlated with N_R. No such anti-correlation is shown. Consequently, the block-CUR plateau advantage in Fig. 2 and the defect-energy gains in Table II could be, at least in part, a cell-size selection effect rather than evidence of descriptor-space coverage. A size-matched random baseline (random sampling conditioned on the same cell-size distribution as the leverage-selected set) or normalization of L_R by |I_R|, together with a report of selected cell-size distributions, is nee","section":"Sec. II.D.1, Eq. (21), Table I"},{"comment":"The reported 2–3x reduction is stated in terms of the number of DFT-labeled configurations (f_lab). DFT cost, however, scales with the number of atoms in the cell, not simply with configuration count. If leverage selection enriches for larger cells (see major comment above), the reduction in total DFT workload may be smaller than claimed. The paper should report, at the plateau for each method, the total number of atoms labeled or an estimated DFT cost, not only the configuration fraction. This is load-bearing for the practical claim that leverage-guided selection reduces DFT labeling cost by 2–3x.","section":"Sec. III.E, Abstract"}],"minor_comments":[{"comment":"The hyperparameter scan uses a fixed 20% validation split to select the ACE architecture; the learning-curve protocol then reserves 20% of the data as an 'independent' test set. Please clarify whether these are the same 20% or disjoint splits. If the same split is reused for hyperparameter selection and testing, the absolute error levels may be optimistic, although the relative comparison between methods would remain valid.","section":"Sec. III.A/B"},{"comment":"The text says force-based selection 'selecting configurations solely by large force magnitudes does not efficiently span descriptor space', but Eq. (25) ranks by force residual, not by force magnitude. Please align the wording with the actual selection criterion.","section":"Sec. III.B, Eq. (25)"},{"comment":"Code availability states that a link 'will be added upon publication'. For reproducibility of the specific leverage computations and the iterative protocol, it would be helpful to make the code available with the submitted version or to include sufficient pseudocode in an appendix.","section":"Sec. V"},{"comment":"Minor typographical/formatting issues: 'm eV/atom' and 'm eV/Å' should be 'meV/atom' and 'meV/Å'. Also, 'Global E' and 'Global F' columns are not explicitly defined in the table caption; please clarify the metric.","section":"Table II"}],"recommendation":"major_revision","confidential_remarks":"The core idea is attractive and the experimental protocol is careful. The main issue is not the protocol itself but the definition of configuration-level leverage in Eq. (21), whose stated justification is contradicted by the paper's own Table I. The authors need to add a size-matched random baseline or normalize by block size, and also report the DFT workload in terms of atoms rather than configurations. If those additions confirm the reported trends, the paper would be a solid contribution. I would not reject now because the concern is fixable within the manuscript's scope."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"The paper is worth a serious look. It tests a known idea—leverage/CUR row selection—in a practically important setting (ASSYST pools for linear ACE) and does so with a fair protocol: fixed architecture, five seeds, label-driven baselines that have strictly more information, and independent DFT validation. The Al learning-curve result (block-CUR plateau at 30–40% labeled fraction vs. 75–80% for random) is a genuine empirical finding, and the defect transfer tests are a useful addition beyond global RMSE. The internal consistency checks out: the numbers in the text match the figures.\n\nThe main soft spot is the one the stress test flags. Equation (21) sums row-level leverage over all energy and force rows of a configuration. Table I says ASSYST cells range from 1 to 10 atoms, so |I_R| spans 4 to 31. The text justifies the sum by saying variations in |I_R| are limited, but the paper's own table contradicts that. Larger cells will tend to produce larger summed leverage, so block-CUR likely enriches the selected subset in bigger cells. That could improve force accuracy and defect energetics simply because you get more force labels per DFT calculation, not because of better descriptor-space coverage. A size-matched random baseline—random selection with the same cell-size distribution as block-CUR—would separate the two mechanisms. Without it, the central interpretation is not secure.\n\nTwo smaller issues. No code or data is released, so the large Table II gaps (Cu vacancy error 0.041 vs 0.246 eV) cannot be checked. And the abstract's 2–3x claim is really an Al-specific result; Cu saturates at 40–50% with no random baseline quoted, so the transfer claim is thinner.\n\nNone of this is fatal. The controlled comparison is a real contribution, and the label-free nature of the selection is correctly emphasized. But the paper would be much stronger with the size-matched baseline and a public repository. I would send it to peer review, asking for those two things. A reader who uses linear ACE on ASSYST pools should see this; just don't take the geometry-coverage story at face value yet.","headline":"A careful empirical study of leverage-based subset selection for linear ACE, but the cell-size confound in Eq. (21) and missing code/data keep it from being fully convincing.","tokens_in":17548,"tokens_out":1706,"would_cite":true,"duration_ms":22237,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Label-free leverage sampling cuts ACE training cost two- to threefold while preserving defect accuracy.","keywords":["linear ACE potentials","leverage scores","CUR matrix sampling","ASSYST structure pools","DFT dataset efficiency","interatomic potential transferability","subset selection","labeled fraction"],"falsifier":"Compute configuration-level leverage scores for the ASSYST Al pool and check whether the highest-ranked configurations are disproportionately the largest cells (8–10 atoms). If so, repeat the training comparison with a cell-size-stratified pool that contains equal numbers of each cell size; if the leverage advantage over random sampling disappears or shrinks, the reported gains are a cell-size artifact rather than a descriptor-coverage effect.","tokens_in":16360,"feed_emoji":"⚛️","tokens_out":6420,"duration_ms":57015,"temperature":0.7,"pith_summary":"This paper argues that for linear ACE interatomic potentials trained on systematically generated ASSYST structure pools, the bottleneck of DFT labeling can be cut by choosing training configurations with statistical leverage scores that depend only on atomic geometry and the ACE basis, not on any energy or force labels. In elemental Al, leverage-guided subsets reach the same plateau accuracy as random sampling at about 30–40% labeled fraction instead of 75–80%, an effective two- to threefold reduction in DFT cost, with similar behavior for Cu and Al–Cu. The paper further shows that leverage-selected subsets preserve or improve defect-level accuracy—vacancy formation, grain-boundary, and dilute-alloy energetics—so data efficiency and transferability are not in conflict. A sympathetic reader cares because DFT labeling is the dominant expense in building machine-learned potentials, and this offers a practical, offline way to reduce it before any quantum-mechanical calculation.","feed_headline":"Label-free leverage sampling cuts ACE training cost 2-3x","feed_subtitle":"Selecting structures by descriptor impact reaches full ACE accuracy at 30-40% of the DFT labels.","key_machinery":"Regularized leverage scores are the diagonal entries of the hat matrix H_λ = Ã(βÃᵀà + Σ₀⁻¹)⁻¹Ãᵀ, where à is the weighted ACE design matrix (one row per energy, force component, or stress observable) and Σ₀⁻¹ is the smoothness-prior precision from Bayesian linear regression. These scores are label-free: they are computed from the geometry and the chosen basis before any DFT labels exist. Configuration-level leverage L_R = Σ_{i∈I_R} h_i aggregates the row scores over all observables of a configuration, and CUR/block-CUR sample configurations with probability proportional to L_R (or its energy-only analog), realizing a geometry-driven approximation to optimal experimental design without explici","core_discovery":"The central claim is that configuration-level regularized leverage scores, defined as the sum over all energy and force rows of the diagonal of the Tikhonov-regularized hat matrix of the weighted ACE design matrix, provide a label-free ranking of ASSYST configurations by information content. Sampling without replacement proportional to these scores (CUR for energy-only, block-CUR for energy+force) yields training subsets that reach the test-error plateau at f_lab ≈ 0.30–0.40 for Al, whereas random sampling requires f_lab ≈ 0.75–0.80 — a two- to threefold reduction in DFT-labeled configurations. In Cu and Al–Cu, the same selection converges faster and, notably, produces lower vacancy-formatio","pith_inferences":["The configuration-level sum of row leverages may be biased by cell size: ASSYST pools contain cells from 1 to 10 atoms, so a 10-atom cell contributes 31 rows while a 1-atom cell contributes 4 rows; normalizing by row count or atoms would test whether the ranking advantage reflects descriptor coverage or simply larger cells.","The same leverage machinery extends naturally to any linear or linearized MLIP with a design matrix and a regularized hat matrix (e.g., linear GAP or MTP), and to the last-layer features of neural network potentials, though these extensions are untested.","A hybrid scheme that uses leverage scores to build the initial diverse subset and then switches to active-learning uncertainty sampling might yield further reductions, since leverage covers the pool while active learning targets the current model's weak spots.","The reported 2–3x compression is measured at the accuracy plateau within one ACE architecture (ν=3, td=17, r_cut=6.5 Å); sweeping body order and basis size could reveal whether the compression factor grows or shrinks with model complexity."],"forward_implications":["For elemental Al and Cu, training on leverage-selected subsets can reach the accuracy plateau with roughly 30–50% of the labeled configurations that random sampling would require, directly reducing DFT workload by a factor of two to three.","Because leverage scores require no labels, workflows can be reordered to filter-then-label: generate a large ASSYST pool, rank by leverage, and run DFT only on the selected subset.","Defect-level transferability is preserved at reduced training size: leverage-selected Cu models cut vacancy-formation error from about 0.25 eV to 0.04 eV and grain-boundary error by about a third compared with baselines.","For ordered Al–Cu intermetallics, leverage subsets at 25% labeled data match 50% baselines on antisite and vacancy errors; in dilute Cu-in-Al, CUR at 25% achieves vacancy-formation RMSE below the 50% baselines.","Leverage-based sampling offers an offline, non-iterative alternative to online active learning, avoiding repeated retraining and DFT-in-the-loop while still concentrating labels in informative regions of descriptor space."],"fun_headline_variants":["Leverage-guided picks cut ACE training data 2-3x","CUR sampling: same ACE accuracy with 40% fewer labels","Label-free subset selection halves ACE DFT cost","Leverage scores pick 30% of structures, full accuracy"],"cache_read_input_tokens":2304,"weakest_assumption_plain":"The selection ranking assumes that summing per-observable leverage scores over a configuration is unbiased because ASSYST cells have comparable atom counts, but the pools actually span 1 to 10 atoms per element, so configurations with more atoms contribute more rows and may dominate the ranking purely by size.","fun_headline_variants_meta":{"raw":{"variants":["Leverage-guided picks cut ACE training data 2-3x","CUR sampling: same ACE accuracy with 40% fewer labels","Label-free subset selection halves ACE DFT cost","Leverage scores pick 30% of structures, full accuracy"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.00059,"raw_usage":{"total_tokens":2613,"prompt_tokens":764,"completion_tokens":1849,"prompt_tokens_details":{"cached_tokens":256},"prompt_cache_hit_tokens":256,"prompt_cache_miss_tokens":508,"completion_tokens_details":{"reasoning_tokens":1778}},"tokens_in":508,"tokens_out":1849,"duration_ms":12790,"temperature":1.0,"reasoning_tokens":1778,"cache_read_input_tokens":256,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-01T15:09:48.062823+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Compute configuration-level leverage scores for the ASSYST Al pool and check whether the highest-ranked configurations are disproportionately the largest cells (8–10 atoms). If so, repeat the training comparison with a cell-size-stratified pool that contains equal numbers of each cell size; if the leverage advantage over random sampling disappears or shrinks, the reported gains are a cell-size artifact rather than a descriptor-coverage effect.","supporting_citations":[],"review_version":1}