{"id":"2009aac0-b35b-40de-b7a4-095c31c6523a","arxiv_id":"2501.16398","paper_version":2,"verdict":"REJECT","confidence":"MODERATE","novelty_score":4.0,"correctness_risk":"high","formal_verification":"none","parameter_count":5,"one_line_summary":"DV-LAE reduces machine learning potential datasets by comparing local atomic environments with binary difference vectors, at the cost of a noticeable accuracy increase in some systems.","lead":"This paper introduces DV-LAE, a method that turns atomic structures into simple binary codes by comparing histograms of local atomic environment descriptors to a reference structure. The authors show it can shrink machine learning potential training sets and cut training time, but the accuracy loss in the iron-hydrogen example is larger than the abstract suggests.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The central 'minimal compromise' claim is contradicted by the paper's own Fe/H accuracy numbers, and the later SOAP comparison reports a different force RMSE for the same optimized dataset.","rationale":"The reader correctly identifies that the Fe/H results are at odds with 'highly similar accuracy', and I agree the verdict should be rejection. However, the reader's stated weakest assumption (reference-structure choice) is not the most load-bearing problem. The more immediate failure is internal: the same optimized 8,691-structure Fe/H dataset appears with force RMSE 3.63 meV/Å in §2.3 and 2.07 meV/Å in §2.4, and the §2.4 value is even better than the full-dataset value from §2.3. Without a shared test set or checkpoint, the central quantitative claim cannot be verified. A single retraining experiment on the released code would resolve which number is correct. The reference-structure robustness remains a legitimate secondary concern, but the accuracy contradiction is sufficient and prior.","tokens_in":13840,"tokens_out":5027,"duration_ms":44763,"concrete_test":"Using the provided GitHub repository, retrain the DV-LAE model on the 8,691-structure α-Fe/H subset with the same hyperparameters, network architecture, and test set used in §2.3 for 100 iterations, then compute energy and force RMSE. If the force RMSE is approximately 3.63 meV/Å, the §2.4 value of 2.07 meV/Å is unreconciled and the 'minimal compromise' claim is unsupported; if it is approximately 2.07 meV/Å, then §2.3's reported optimized accuracy is wrong. Either result resolves the contradiction, while a third value would indicate uncontrolled variability in training or evaluation.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim is that DV-LAE pruning yields 'minimal compromise' in accuracy. The strongest quantitative evidence for a binary system is in §2.3: on α-Fe/H, reducing the dataset from 19,743 to 8,691 structures increases energy RMSE from 0.312 to 0.375 meV/atom (+20%) and force RMSE from 2.25 to 3.63 meV/Å (+61%). A 61% force-error increase after removing 56% of the data does not support 'highly similar accuracy' or 'minimal compromise'. In §2.4, however, the same DV-LAE method on the same Fe-H dataset is reported to give energy error 0.339 meV/atom and force error 2.07 meV/Å, i.e., better than the full-dataset values in §2.3. No explanation reconciles why the optimized model in one section has force RMSE 3.63 meV/Å and in another 2.07 meV/Å. Because the central claim rests on these reported accuracies, the inconsistency is not cosmetic; the empirical case for the headline claim is internally contradictory and cannot be independently checked from the text. The lack of a specified screening threshold or selection rule in the t-SNE procedure compounds this, but the accuracy-number conflict alone is sufficient to block the claim as stated.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper introduces DV-LAE (Difference Vectors based on Local Atomic Environments), a method that converts atomic environment descriptors (G-functions) into per-structure histograms, compares these histograms against a single reference structure via XOR operations to obtain binary difference vectors, and uses t-SNE on these vectors to visualize dataset redundancy. The authors apply this pipeline to prune MLP training sets for high-pressure hydrogen, α-Fe/H, magnesium hydride, and carbon systems, reporting training-time reductions and claiming 'minimal compromise' in accuracy. They further compare DV-LAE with SOAP-based screening and propose using DV-LAE visualizations to identify out-of-distribution data.","tokens_in":14093,"tokens_out":4216,"duration_ms":39154,"significance":"If the claims were fully supported, the method would offer a cheap, interpretable data-pruning tool for MLP construction, with the attractive feature of visual redundancy detection. The code and data are made available on GitHub, which is a strength. However, the paper's core quantitative claim is undermined by an internal contradiction in the reported accuracy numbers for the same dataset (§2.3 vs. §2.4), and the screening procedure is not specified at a level that permits reproduction. As presented, the significance of the contribution is therefore not established.","major_comments":[{"comment":"The two sections report inconsistent accuracy for the same DV-LAE-optimized Fe/H dataset (8,691 structures). In §2.3, the optimized set yields energy RMSE 0.375 meV/atom and force RMSE 3.63 meV/Å, compared with full-dataset values of 0.312 meV/atom and 2.25 meV/Å. In §2.4, the same method on the same dataset is reported to give energy error 0.339 meV/atom and force error 2.07 meV/Å — i.e., better than the full dataset. No explanation is provided for this discrepancy, and the two sets of numbers cannot both describe the same trained model. Because the abstract's 'minimal compromise' claim rests on these accuracy comparisons, the central empirical claim is internally contradictory and cannot be independently checked from the text.","section":"§2.3 vs. §2.4, Figs. 4c-d and 5b"},{"comment":"The comparison between DV-LAE and SOAP-based screening is not adequately controlled. The SOAP method with threshold 0.1 retains 17,900 structures, while DV-LAE retains 8,691; reporting only final errors (SOAP: 3.11 meV/atom, 11.06 meV/Å; DV-LAE: 0.339 meV/atom, 2.07 meV/Å) conflates the screening method with the resulting dataset size. The reader cannot determine whether the improvement is due to DV-LAE's selection criterion or merely to the different number of retained structures, and the claim that DV-LAE 'significantly outperforms' SOAP is therefore not demonstrated.","section":"§2.4, Fig. 5b"},{"comment":"The data screening step is under-specified. The text states that 'leveraging the outcomes of the visualization, we executed a targeted data screening' and that a subset is selected 'strategically,' but no algorithmic criterion is given for which structures are removed or retained based on the t-SNE plot or the difference vectors. The number of histogram bins N is deferred to Figures S6-S7, and no screening threshold is stated for the t-SNE-based selection. As a result, the experiments cannot be reproduced from the paper alone, and the method's claimed advantage over manual or threshold-based screening cannot be evaluated.","section":"Methods, Figure 1 step ⑥"},{"comment":"The difference vectors are defined relative to a single reference structure 'encompassing all elemental species in the target system.' This choice is arbitrary and is never tested for sensitivity. A different reference structure would change every difference vector and could alter the screening outcome, yet no experiment or argument shows that the results are robust to this choice. Since the reference structure is a load-bearing component of the descriptor, the method's validity is not established without such a test.","section":"Methods, first paragraph of DV-LAE construction"}],"minor_comments":[{"comment":"The same paragraph reports the data reduction as both '~55.98%' and '~56.46%' for the same α-Fe/H dataset; one of these is likely a typo and should be corrected.","section":"§2.3, second paragraph after Fig. 4"},{"comment":"The elastic constant is labeled 'C14,' but for a cubic crystal the off-diagonal elastic constant is conventionally denoted C44. This notation should be checked and corrected.","section":"Table 1 and §2.3"},{"comment":"Several references are duplicated: [15] and [16] are the same Behler–Parrinello paper, and [40] also duplicates it; [33] and [41] are the same Behler paper. These should be consolidated.","section":"References"},{"comment":"The text refers to 'the 20 highest error prediction points (red diamond),' but the figure shows multiple points; the caption should use the plural 'red diamonds' for grammatical consistency.","section":"§2.5, Figure 6e-f captions"},{"comment":"The sentence 'with our method, the energy error decreased to 0.339 meV/atom' is misleading because the comparison baseline is the SOAP-screened model, not the full dataset; in §2.3 the full-dataset energy error is already lower (0.312 meV/atom).","section":"§2.4, Figure 5b"},{"comment":"The cutoff function is not rendered clearly in the manuscript; the piecewise definition and the variable x should be typeset properly so the formula can be read unambiguously.","section":"Equation (4)"}],"recommendation":"reject","confidential_remarks":"The manuscript contains a direct internal contradiction in the headline accuracy numbers for the Fe/H system, and the screening method is not algorithmically specified. Even if the numbers were corrected, the abstract's claim of 'minimal compromise' is contradicted by the 61% increase in force RMSE reported in §2.3. I see no straightforward revision within the paper's current scope that would resolve these issues without re-running the experiments and rewriting the central claims."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Two things to know. First, there is a real kernel of an idea here: a binary difference vector built from histogram-binned Behler–Parrinello symmetry functions, XOR'd against a single reference structure, then t-SNE'd. I have not seen that exact representation before, and it is cheap enough to be useful as a first-pass redundancy filter. Second, the paper's own numbers do not support its headline claim of 'minimal compromise' on the α-Fe/H system, and one section's numbers contradict another's.\n\nWhat the paper does well: the hydrogen result is honest and reproducible in spirit — 31.8% data reduction for a ~2-3% RMSE increase is a modest but real win. The OOD visualization idea, checking where high-error prediction points land in the DV-LAE embedding, is a sensible qualitative tool. Code and structures are on GitHub, which is good practice.\n\nWhere it goes wrong: in §2.3, removing 56% of the Fe/H training data raises force RMSE from 2.25 to 3.63 meV/Å (61%) and energy RMSE from 0.312 to 0.375 meV/atom (20%). That is not 'highly similar accuracy' by ordinary standards. Then §2.4 reports the same DV-LAE-optimized Fe/H dataset as having force error 2.07 meV/Å and energy error 0.339 meV/atom — better than the full-dataset numbers in §2.3. The text never reconciles this. One of these numbers is wrong, or the test sets differ, and either way the central empirical claim cannot be checked.\n\nThe other soft spots are secondary but real: the reference structure is arbitrary and its influence is never tested; the t-SNE screening cutoff is a human eyeball decision, not a stated rule; no error bars or repeated runs anywhere.\n\nBottom line: the descriptor is worth a look, the evidence is not yet at the level claimed. This needs a revision that fixes the accuracy table, specifies the screening rule, and tests sensitivity to the reference structure. I would send it to peer review — the kernel is new and the code exists — but I would expect heavy revision.","headline":"The DV-LAE descriptor is a genuinely new cheap structural fingerprint, but the Fe/H accuracy numbers contradict the 'minimal compromise' claim and even contradict each other across sections.","tokens_in":14709,"tokens_out":2080,"would_cite":false,"duration_ms":18688,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Difference vectors built from local atomic environments can cut machine-learning-potential training sets by more than half while keeping energy and force errors close to full-data values.","keywords":["machine learning potentials","dataset optimization","local atomic environment","difference vectors","t-SNE visualization","data redundancy","out-of-distribution detection","Behler-Parrinello symmetry functions"],"falsifier":"Re-run the DV-LAE screening on the hydrogen or Fe-H dataset with several alternative reference structures (a random configuration, a different supercell, or a deliberately atypical defect configuration) and compare the selected subsets and the resulting energy/force RMSEs; large swings in either would show the method's output is an artifact of the reference choice.","tokens_in":13583,"feed_emoji":"⚛️","tokens_out":8453,"duration_ms":67765,"temperature":0.7,"pith_summary":"This paper proposes DV-LAE (Difference Vectors based on Local Atomic Environments), a method for detecting redundancy in the datasets used to train machine-learning potentials. It represents each atomic structure by the histograms of its local-environment descriptors, then compares those histograms to a reference structure with a per-bin XOR operation to form a compact binary difference vector. The authors claim this screening works across high-pressure hydrogen, iron–hydrogen binaries, magnesium hydrides, and carbon allotropes, with quantitative reductions of about 32% and 56% demonstrated on the hydrogen and α-Fe/H datasets while energy and force errors rise by only a few percent. The same representation, projected with t-SNE, separates phases and makes out-of-distribution structures visible as peripheral clusters.","feed_headline":"XOR histograms of atomic environments cut training data by 56%","feed_subtitle":"For machine-learning potentials, a fast binary comparison flags redundant structures—training runs twice as fast with nearly unchanged…","key_machinery":"The mechanism is the difference vector D: for each descriptor type (radial and angular Behler–Parrinello symmetry functions), a histogram with N bins is built from all atoms in a structure; each bin count is XOR-ed with the corresponding bin of a chosen reference structure, yielding 1 if the counts differ and 0 if they match; the per-descriptor binary vectors are concatenated into one vector. The XOR step converts continuous descriptor distributions into a compact code that registers presence of descriptor values rather than exact counts, and t-SNE then projects these codes for visual clustering. This construction is what lets the method screen a 19,743-structure dataset in a fraction of the time needed by SOAP-based similarity screening.","core_discovery":"The central claim is that structural redundancy in MLP training sets can be identified from a cheap, size-uniform, binary descriptor rather than from expensive all-to-all comparisons of high-dimensional atomic descriptors. The authors show that DV-LAE preserves physically meaningful similarity—primitive cells and their supercells overlap in feature space, unlike zero-padded G-function descriptors—and that datasets pruned by this method train potentials with near-full-dataset accuracy. In the α-Fe/H binary system the pruned set has 8,691 structures instead of 19,743 (about 56% fewer), training time per iteration drops by 55.8%, yet phonon spectra and band structures remain consistent and vacancy formation energies stay within 0.1 eV of DFT values.","pith_inferences":["A natural test is to replace the single reference structure with an ensemble of references and aggregate the XOR codes; if the pruned subsets stabilize, the method is stable rather than dependent on that single choice.","Because the XOR code discards magnitude information, it might miss distinctions among environments that occupy the same histogram bins but with very different counts; a graded or weighted histogram distance could recover that information.","The same histogram-XOR construction should transfer to other local descriptors such as SOAP or equivariant embeddings, potentially making the screening model-agnostic; the paper does not test this."],"forward_implications":["MLP training datasets can be cut by roughly a third to more than half with only slight RMSE increases, directly reducing both data-generation and training costs.","The screening cost is low enough to rerun as new structures arrive, so the method can support iterative dataset construction and active-learning loops.","For Fe-H, the pruned-dataset potential preserves lattice constants, elastic constants, phonon spectra, and band structure, indicating that key physical properties survive aggressive pruning.","The t-SNE map of difference vectors can act as a reliability check: high-error predictions sit at the periphery, making it easier to spot out-of-distribution configurations during simulation."],"supporting_citations":[{"why":"Supplies the high-pressure hydrogen training set (25,440 structures) used for the single-element demonstration.","marker":"34"},{"why":"Supplies the α-Fe/H raw dataset (19,743 structures) used for the binary-system demonstration and the 56% pruning result.","marker":"35"},{"why":"Provides the reference Fe-H MLP results (REF) against which lattice constants, vacancy formation energies, and elastic constants are compared.","marker":"36"},{"why":"Defines the SOAP descriptor whose similarity-based screening serves as the baseline compared with DV-LAE in Figure 5.","marker":"37"},{"why":"Defines the atom-centered symmetry functions whose histograms are the raw material for the difference vectors.","marker":"33"},{"why":"Motivates the 'less is more' principle that justifies recording the presence of descriptor values across histogram bins rather than exact counts.","marker":"42"}],"fun_headline_variants":["Difference vectors cut ML dataset by 56% and speed training","Atomic difference vectors reveal redundant data, cut training time","Cut ML dataset 56% with local difference vectors, keep accuracy","Histogram vectors of atomic environments prune 56% of ML data"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that a single reference structure, chosen by the authors, is representative enough that XOR comparisons against it reveal which structures in the dataset are redundant; the paper does not test how much the screening outcome depends on that particular reference.","fun_headline_variants_meta":{"raw":{"variants":["Difference vectors cut ML dataset by 56% and speed training","Atomic difference vectors reveal redundant data, cut training time","Cut ML dataset 56% with local difference vectors, keep accuracy","Histogram vectors of atomic environments prune 56% of ML data"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000654,"raw_usage":{"total_tokens":2988,"prompt_tokens":925,"completion_tokens":2063,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":541,"completion_tokens_details":{"reasoning_tokens":1992}},"tokens_in":541,"tokens_out":2063,"duration_ms":14636,"temperature":1.0,"reasoning_tokens":1992,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-10T13:57:26.800635+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Re-run the DV-LAE screening on the hydrogen or Fe-H dataset with several alternative reference structures (a random configuration, a different supercell, or a deliberately atypical defect configuration) and compare the selected subsets and the resulting energy/force RMSEs; large swings in either would show the method's output is an artifact of the reference choice.","supporting_citations":[{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Supplies the high-pressure hydrogen training set (25,440 structures) used for the single-element demonstration."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Supplies the α-Fe/H raw dataset (19,743 structures) used for the binary-system demonstration and the 56% pruning result."},{"cited_title":"& Ogata, S","cited_arxiv_id":null,"evidence_quote":"Provides the reference Fe-H MLP results (REF) against which lattice constants, vacancy formation energies, and elastic constants are compared."},{"cited_title":"L., Gasparotto, P., Csá nyi, G","cited_arxiv_id":null,"evidence_quote":"Defines the SOAP descriptor whose similarity-based screening serves as the baseline compared with DV-LAE in Figure 5."},{"cited_title":"Atom -centered symmetry functions for constructing high -dimensional neural network potentials","cited_arxiv_id":null,"evidence_quote":"Defines the atom-centered symmetry functions whose histograms are the raw material for the difference vectors."}],"review_version":1}