Pith. sign in

REVIEW 4 major objections 6 minor 41 references

Data-Efficient Machine Learning Potentials via Difference Vectors Based on Local Atomic Environments

T0 review · 4 major / 6 minor · reviewed 2026-08-10 · deepseek-v4-flash

Pith's one-line read Difference vectors built from local atomic environments can cut machine-learning-potential training sets by more than half while keeping energy and force errors close to full-data values.

desk verdict The DV-LAE descriptor is a genuinely new cheap structural fingerprint, but the Fe/H accuracy numbers contradict the 'minimal compromise' claim and even contradict each other across sections. read the letter →

arxiv 2501.16398 v2 pith:SKJOCAPF submitted 2025-01-27 cs.LG physics.atom-ph

classification cs.LGphysics.atom-ph
keywords machinelearningpotentialsdatasetoptimizationlocalatomicenvironmentdifferencevectorst-SNEvisualizationdataredundancyout-of-distributiondetectionBehler-Parrinellosymmetryfunctions
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper proposes DV-LAE (Difference Vectors based on Local Atomic Environments), a method for detecting redundancy in the datasets used to train machine-learning potentials. It represents each atomic structure by the histograms of its local-environment descriptors, then compares those histograms to a reference structure with a per-bin XOR operation to form a compact binary difference vector. The authors claim this screening works across high-pressure hydrogen, iron–hydrogen binaries, magnesium hydrides, and carbon allotropes, with quantitative reductions of about 32% and 56% demonstrated on the hydrogen and α-Fe/H datasets while energy and force errors rise by only a few percent. The same representation, projected with t-SNE, separates phases and makes out-of-distribution structures visible as peripheral clusters.

What carries the argument

The mechanism is the difference vector D: for each descriptor type (radial and angular Behler–Parrinello symmetry functions), a histogram with N bins is built from all atoms in a structure; each bin count is XOR-ed with the corresponding bin of a chosen reference structure, yielding 1 if the counts differ and 0 if they match; the per-descriptor binary vectors are concatenated into one vector. The XOR step converts continuous descriptor distributions into a compact code that registers presence of descriptor values rather than exact counts, and t-SNE then projects these codes for visual clustering. This construction is what lets the method screen a 19,743-structure dataset in a fraction of the time needed by SOAP-based similarity screening.

What would settle it

Re-run the DV-LAE screening on the hydrogen or Fe-H dataset with several alternative reference structures (a random configuration, a different supercell, or a deliberately atypical defect configuration) and compare the selected subsets and the resulting energy/force RMSEs; large swings in either would show the method's output is an artifact of the reference choice.

Watch

Extended reading notes

Core claim

The central claim is that structural redundancy in MLP training sets can be identified from a cheap, size-uniform, binary descriptor rather than from expensive all-to-all comparisons of high-dimensional atomic descriptors. The authors show that DV-LAE preserves physically meaningful similarity—primitive cells and their supercells overlap in feature space, unlike zero-padded G-function descriptors—and that datasets pruned by this method train potentials with near-full-dataset accuracy. In the α-Fe/H binary system the pruned set has 8,691 structures instead of 19,743 (about 56% fewer), training time per iteration drops by 55.8%, yet phonon spectra and band structures remain consistent and vacancy formation energies stay within 0.1 eV of DFT values.

Load-bearing premise

The load-bearing premise is that a single reference structure, chosen by the authors, is representative enough that XOR comparisons against it reveal which structures in the dataset are redundant; the paper does not test how much the screening outcome depends on that particular reference.

Editorial extensions

If this is right

  • MLP training datasets can be cut by roughly a third to more than half with only slight RMSE increases, directly reducing both data-generation and training costs.
  • The screening cost is low enough to rerun as new structures arrive, so the method can support iterative dataset construction and active-learning loops.
  • For Fe-H, the pruned-dataset potential preserves lattice constants, elastic constants, phonon spectra, and band structure, indicating that key physical properties survive aggressive pruning.
  • The t-SNE map of difference vectors can act as a reliability check: high-error predictions sit at the periphery, making it easier to spot out-of-distribution configurations during simulation.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • A natural test is to replace the single reference structure with an ensemble of references and aggregate the XOR codes; if the pruned subsets stabilize, the method is stable rather than dependent on that single choice.
  • Because the XOR code discards magnitude information, it might miss distinctions among environments that occupy the same histogram bins but with very different counts; a graded or weighted histogram distance could recover that information.
  • The same histogram-XOR construction should transfer to other local descriptors such as SOAP or equivariant embeddings, potentially making the screening model-agnostic; the paper does not test this.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 6 minor

Summary. The paper introduces DV-LAE (Difference Vectors based on Local Atomic Environments), a method that converts atomic environment descriptors (G-functions) into per-structure histograms, compares these histograms against a single reference structure via XOR operations to obtain binary difference vectors, and uses t-SNE on these vectors to visualize dataset redundancy. The authors apply this pipeline to prune MLP training sets for high-pressure hydrogen, α-Fe/H, magnesium hydride, and carbon systems, reporting training-time reductions and claiming 'minimal compromise' in accuracy. They further compare DV-LAE with SOAP-based screening and propose using DV-LAE visualizations to identify out-of-distribution data.

Significance. If the claims were fully supported, the method would offer a cheap, interpretable data-pruning tool for MLP construction, with the attractive feature of visual redundancy detection. The code and data are made available on GitHub, which is a strength. However, the paper's core quantitative claim is undermined by an internal contradiction in the reported accuracy numbers for the same dataset (§2.3 vs. §2.4), and the screening procedure is not specified at a level that permits reproduction. As presented, the significance of the contribution is therefore not established.

major comments (4)
  1. [§2.3 vs. §2.4, Figs. 4c-d and 5b] The two sections report inconsistent accuracy for the same DV-LAE-optimized Fe/H dataset (8,691 structures). In §2.3, the optimized set yields energy RMSE 0.375 meV/atom and force RMSE 3.63 meV/Å, compared with full-dataset values of 0.312 meV/atom and 2.25 meV/Å. In §2.4, the same method on the same dataset is reported to give energy error 0.339 meV/atom and force error 2.07 meV/Å — i.e., better than the full dataset. No explanation is provided for this discrepancy, and the two sets of numbers cannot both describe the same trained model. Because the abstract's 'minimal compromise' claim rests on these accuracy comparisons, the central empirical claim is internally contradictory and cannot be independently checked from the text.
  2. [§2.4, Fig. 5b] The comparison between DV-LAE and SOAP-based screening is not adequately controlled. The SOAP method with threshold 0.1 retains 17,900 structures, while DV-LAE retains 8,691; reporting only final errors (SOAP: 3.11 meV/atom, 11.06 meV/Å; DV-LAE: 0.339 meV/atom, 2.07 meV/Å) conflates the screening method with the resulting dataset size. The reader cannot determine whether the improvement is due to DV-LAE's selection criterion or merely to the different number of retained structures, and the claim that DV-LAE 'significantly outperforms' SOAP is therefore not demonstrated.
  3. [Methods, Figure 1 step ⑥] The data screening step is under-specified. The text states that 'leveraging the outcomes of the visualization, we executed a targeted data screening' and that a subset is selected 'strategically,' but no algorithmic criterion is given for which structures are removed or retained based on the t-SNE plot or the difference vectors. The number of histogram bins N is deferred to Figures S6-S7, and no screening threshold is stated for the t-SNE-based selection. As a result, the experiments cannot be reproduced from the paper alone, and the method's claimed advantage over manual or threshold-based screening cannot be evaluated.
  4. [Methods, first paragraph of DV-LAE construction] The difference vectors are defined relative to a single reference structure 'encompassing all elemental species in the target system.' This choice is arbitrary and is never tested for sensitivity. A different reference structure would change every difference vector and could alter the screening outcome, yet no experiment or argument shows that the results are robust to this choice. Since the reference structure is a load-bearing component of the descriptor, the method's validity is not established without such a test.
minor comments (6)
  1. [§2.3, second paragraph after Fig. 4] The same paragraph reports the data reduction as both '~55.98%' and '~56.46%' for the same α-Fe/H dataset; one of these is likely a typo and should be corrected.
  2. [Table 1 and §2.3] The elastic constant is labeled 'C14,' but for a cubic crystal the off-diagonal elastic constant is conventionally denoted C44. This notation should be checked and corrected.
  3. [References] Several references are duplicated: [15] and [16] are the same Behler–Parrinello paper, and [40] also duplicates it; [33] and [41] are the same Behler paper. These should be consolidated.
  4. [§2.5, Figure 6e-f captions] The text refers to 'the 20 highest error prediction points (red diamond),' but the figure shows multiple points; the caption should use the plural 'red diamonds' for grammatical consistency.
  5. [§2.4, Figure 5b] The sentence 'with our method, the energy error decreased to 0.339 meV/atom' is misleading because the comparison baseline is the SOAP-screened model, not the full dataset; in §2.3 the full-dataset energy error is already lower (0.312 meV/atom).
  6. [Equation (4)] The cutoff function is not rendered clearly in the manuscript; the piecewise definition and the variable x should be typeset properly so the formula can be read unambiguously.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the DV-LAE descriptor construction and dataset screening are empirical and self-contained, not derived from the MLP accuracy claims.

full rationale

The paper's central derivation chain is empirical rather than circular. The DV-LAE difference vector is constructed from histogram statistics of Behler-Parrinello symmetry functions followed by an XOR comparison against a single reference structure (Methods, Eqs. 5–7). These difference vectors are not fitted to energies or forces; they are defined directly from structural descriptors. The subsequent t-SNE visualization and manual data screening are heuristic data-selection procedures, not predictions derived from the fitted model. The accuracy claims (e.g., hydrogen energy RMSE 0.566 vs. 0.577 meV/atom, and Fe/H force RMSE 2.25 vs. 3.63 meV/Å) are reported as empirical comparisons between MLPs trained on full and optimized datasets, so they are not circular in the sense of reducing to the method's own inputs. No self-citation is load-bearing: the cited prior works (Refs. 34, 35, 36) provide the datasets and baseline MLPs but the screening method itself is not justified by those citations. The paper does not invoke a uniqueness theorem or rename a known result. Concerns such as the arbitrary choice of reference structure, unspecified screening threshold, and the internal inconsistency of the Fe/H force-error numbers between Sections 2.3 and 2.4 are reproducibility and correctness issues, not circularity. Therefore the appropriate circularity score is 0.

Assumptions & free parameters 5 free parameters · 4 assumptions · 0 invented entities

The central claim rests on a chain of heuristic choices: the reference structure, histogram bin count, t-SNE settings, and the manual screening threshold. None are derived, and their influence on the result is not studied. The method also assumes without evidence that t-SNE overlap indicates data redundancy.

free parameters (5)
  • Number of histogram bins N = Not reported in main text
    The binary difference vector is constructed by binning each descriptor into N intervals; N controls resolution and is chosen via supplementary figures rather than derived.
  • t-SNE perplexity = Not reported
    t-SNE embeddings depend strongly on perplexity; without reporting it, the visual redundancy assessment is not reproducible.
  • Reference structure = Not specified
    The XOR comparison requires a reference structure, but the paper gives no criterion for selecting it and no robustness test.
  • Data screening cutoff = Not specified
    The paper describes 'targeted data screening' based on t-SNE density but provides no numerical threshold for removing structures.
  • SOAP similarity threshold = 0.1
    Used only in the baseline comparison, not in the central DV-LAE method.
assumptions (4)
  • domain assumption Histogram bin overlap between structures indicates data redundancy.
    The screening removes structures whose t-SNE points overlap, assuming these overlaps mean the structures are redundant for MLP training. This is not proven.
  • domain assumption t-SNE density reflects data redundancy.
    The paper uses dense t-SNE clusters as evidence of redundancy, but t-SNE distorts densities and distances, so this is an untested assumption.
  • ad hoc to paper A single reference structure can represent all elemental combinations in a multi-element system.
    The method requires a reference structure 'encompassing all elemental species', but no criterion is given for constructing it or testing its representativeness.
  • standard math The Behler-Parrinello symmetry functions with the chosen parameter set sufficiently describe local atomic environments for screening.
    This is inherited from prior MLP literature and is not tested within the paper.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Data-Efficient Machine Learning Potentials via Difference Vectors Based on Local Atomic Environments." pith.science (2026). https://pith.science/paper/SKJOCAPF

@misc{pith2026250116398,
  author       = {Pith},
  title        = {Pith review of: Data-Efficient Machine Learning Potentials via Difference Vectors Based on Local Atomic Environments},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/SKJOCAPF}},
  note         = {Machine review of arXiv:2501.16398}
}
abstract

Constructing efficient and diverse datasets is essential for the development of accurate machine learning potentials (MLPs) in atomistic simulations. However, existing approaches often suffer from data redundancy and high computational costs. Herein, we propose a new method--Difference Vectors based on Local Atomic Environments (DV-LAE)--that encodes structural differences via histogram-based descriptors and enables visual analysis through t-SNE dimensionality reduction. This approach facilitates redundancy detection and dataset optimization while preserving structural diversity. We demonstrate that DV-LAE significantly reduces dataset size and training time across various materials systems, including high-pressure hydrogen, iron-hydrogen binaries, magnesium hydrides, and carbon allotropes, with minimal compromise in prediction accuracy. For instance, in the $\alpha$-Fe/H system, maintaining a highly similar MLP accuracy, the dataset size was reduced by 56%, and the training time per iteration dropped by over 50%. Moreover, we show how visualizing the DV-LAE representation aids in identifying out-of-distribution data by examining the spatial distribution of high-error prediction points, providing a robust reliability metric for new structures during simulations. Our results highlight the utility of local environment visualization not only as an interpretability tool but also as a practical means for accelerating MLP development and ensuring data efficiency in large-scale atomistic modeling.

Figures

Figures reproduced from arXiv: 2501.16398 by the authors.

Figure 1
Figure 1. Schematic workflow for constructing structural difference vectors and optimizing datasets in a binary system. ① Calculate Atomic Environment Features: for each atom in the [PITH_FULL_IMAGE:figures/full_fig_p005_1.png] view at source ↗
Figure 2
Figure 2. The representation of supercell structures using G-function and difference-vector descriptors and comparison of their effectiveness through PCA and t-SNE visualization. (a) Schematic illustration of the primitive cell to supercell transformation via G-function descriptors with dimensionless zero-padding. (b) G-function-based PCA projection visualization for supercell dataset. (c) Difference-vector-based PCA projecti… view at source ↗
Figure 3
Figure 3. Results of the dataset optimization process, comparing the original and [PITH_FULL_IMAGE:figures/full_fig_p011_3.png] view at source ↗
Figures from the paper (1 more)
Figure 4
Figure 4. Figure 4: Application and validation of the DV-LAE optimization method for the α-Fe/H binary system dataset, highlighting the significant reduction in training time and dataset size achieved while preserving prediction accuracy and key physical properties. (a) The t-SNE visualiz…

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

41 extracted references · 37 canonical work pages

  1. [1]

    Sholl, D. S. & Steckel, J. A. Density Functional Theory: A Practical Introduction. (John Wiley & Sons, 2022)

  2. [2]

    Taking materials dynamics to new extremes using machine learning interatomic potentials

    Yang Yang et al. Taking materials dynamics to new extremes using machine learning interatomic potentials. Journal of Materials Informatics 1, 10 (2021)

  3. [3]

    Liu, Z.-Q. et al. FT2DP: large atomic model fine-tuned machine learning potential for accelerating atomistic simulation of iron -based Fischer -Tropsch synthesis. Journal of Materials Informatics 5, (2025)

  4. [4]

    Wu, X., Kang, F., Duan, W. & Li, J. Density functional theory calculations: A powerful tool to simulate and design high -performance energy storage and conversion materials. Progress in Natural Science: Materials International 29, 247–255 (2019)

  5. [5]

    Perspective on density functional theory

    Burke, K. Perspective on density functional theory. The Journal of chemical physics 136, (2012)

  6. [6]

    & Von Lilienfeld, O

    Rupp, M., Ramakrishnan, R. & Von Lilienfeld, O. A. Machine learning for quantum mechanical properties of atoms in molecules. The Journal of Physical Chemistry Letters 6, 3309–3313 (2015)

  7. [7]

    & Kim, C

    Ramprasad, R., Batra, R., Pilania, G., Mannodi -Kanakkithodi, A. & Kim, C. Machine learning in materials informatics: recent applications and prospects. npj Computational Materials 3, 54 (2017)

  8. [8]

    & Tkatchenko, A

    Poltavsky, I. & Tkatchenko, A. Machine learning force fields: Recent advances and remaining challenges. The journal of physical chemistry letters 12, 6551–6564 (2021)

Show all 41 references
  1. [9]

    Unke, O. T. et al. Machine learning force fields. Chemical Reviews 121, 10142–10186 (2021)

  2. [10]

    Gubernatis, J. E. & Lookman, T. Machine learning in materials design and discovery: Examples from the present and suggestions for the future. Physical Review Materials 2, 120301 (2018)

  3. [11]

    Mailoa, J. P. et al. A fast neural network approach for direct covariant forces prediction in complex multi-element extended systems. Nat Mach Intell 1, 471–479 (2019)

  4. [12]

    Perspective: Machine learning potentials for atomistic simulations

    Behler, J. Perspective: Machine learning potentials for atomistic simulations. The Journal of Chemical Physics 145, 170901 (2016)

  5. [13]

    P., Kermode, J., Bernstein, N

    Bartó k, A. P., Kermode, J., Bernstein, N. & Csá nyi, G. Machine learning a general -purpose interatomic potential for silicon. Physical Review X 8, 041048 (2018)

  6. [14]

    Deringer, V. L. & Csá nyi, G. Machine learning based interatomic potential for amorphous carbon. Physical Review B 95, 094203 (2017)

  7. [16]

    & Parrinello, M

    Behler, J. & Parrinello, M. Generalized neural -network representation of high -dimensional potential-energy surfaces. Physical review letters 98, 146401 (2007)

  8. [17]

    Zhang, L., Han, J., Wang, H., Car, R. & E, W. Deep potential molecular dynamics: a scalable model with the accuracy of quantum mechanics. Physical review letters 120, 143001 (2018)

  9. [18]

    Deep potential: A general representation of a many -body potential energy surface

    Han, J., Zhang, L., Car, R., & others. Deep potential: A general representation of a many -body potential energy surface. arXiv preprint arXiv:1707.01478 (2017)

  10. [19]

    P., Simm, G., Ortner, C

    Batatia, I., Kovacs, D. P., Simm, G., Ortner, C. & Csá nyi, G. MACE: Higher order equivariant message passing neural networks for fast and accurate force fields. Advances in neural information processing systems 35, 11423–11436 (2022)

  11. [20]

    Musaelian, A. et al. Learning local equivariant representations for large-scale atomistic dynamics. Nature Communications 14, 579 (2023)

  12. [21]

    & Vasilakos, A

    Zhou, L., Pan, S., Wang, J. & Vasilakos, A. V. Machine learning on big data: Opportunities and challenges. Neurocomputing 237, 350–361 (2017)

  13. [22]

    Jordan, M. I. & Mitchell, T. M. Machine learning: Trends, perspectives, and prospects. Science 349, 255–260 (2015)

  14. [23]

    Wei, J. et al. Machine learning in materials science. InfoMat 1, 338–358 (2019)

  15. [24]

    The development and comparison of molecular dynamics simulation and Monte Carlo simulation

    Chen, J. The development and comparison of molecular dynamics simulation and Monte Carlo simulation. in IOP Conference Series: Earth and Environmental Science vol. 128 012110 (IOP Publishing, 2018)

  16. [25]

    & Beratan, D

    Rupakheti, C., Virshup, A., Yang, W. & Beratan, D. N. Strategy To Discover Diverse Optimal Molecules in the Small Molecule Universe. J. Chem. Inf. Model. 55, 529–537 (2015)

  17. [26]

    Mu, S. et al. Uncovering electron scattering mechanisms in NiFeCoCrMn derived concentrated solid solution and high entropy alloys. npj Computational Materials 5, 1 (2019)

  18. [27]

    Bowman, J. M. et al. The MD17 datasets from the perspective of datasets for gas -phase “small” molecule potentials. J. Chem. Phys. 156, 240901 (2022)

  19. [28]

    Zhang, L., Lin, D.-Y., Wang, H., Car, R. & E, W. Active learning of uniformly accurate interatomic potentials for materials simulation. Physical Review Materials 3, 023804 (2019)

  20. [29]

    Greenacre, M. et al. Principal component analysis. Nature Reviews Methods Primers 2, 100 (2022)

  21. [30]

    & Hinton, G

    Van der Maaten, L. & Hinton, G. Visualizing data using t -SNE. Journal of machine learning research 9, (2008)

  22. [31]

    & Johnson, I

    Wattenberg, M., Vié gas, F. & Johnson, I. How to use t-SNE effectively. Distill 1, e2 (2016)

  23. [32]

    Zhang, Y. et al. DP-GEN: A concurrent learning platform for the generation of reliable deep learning based potential energy models. Comput Phys Commun 253, 107206 (2020)

  24. [33]

    Atom -centered symmetry functions for constructing high -dimensional neural network potentials

    Behler, J. Atom -centered symmetry functions for constructing high -dimensional neural network potentials. The Journal of chemical physics 134, (2011)

  25. [34]

    Cheng, B., Mazzola, G., Pickard, C. J. & Ceriotti, M. Evidence for supercritical behaviour of high- pressure liquid hydrogen. Nature 585, 217–220 (2020)

  26. [35]

    Meng, F. -S. et al. General-purpose neural network interatomic potential for the α -iron and hydrogen binary system: Toward atomic -scale understanding of hydrogen embrittlement. Physical Review Materials 5, 113606 (2021)

  27. [36]

    & Ogata, S

    Zhang, S., Meng, F., Fu, R. & Ogata, S. Highly efficient and transferable interatomic potentials for α-iron and α-iron/hydrogen binary systems using deep neural networks. Computational Materials Science 235, 112843 (2024)

  28. [37]

    L., Gasparotto, P., Csá nyi, G

    Rowe, P., Deringer, V. L., Gasparotto, P., Csá nyi, G. & Michaelides, A. An accurate and transferable machine learning potential for carbon. The Journal of Chemical Physics 153, 034702 (2020)

  29. [38]

    Silverman, B. W. Density Estimation for Statistics and Data Analysis. (Routledge, 2018)

  30. [39]

    S., Fu, N., Dong, R., Hu, M

    Omee, S. S., Fu, N., Dong, R., Hu, M. & Hu, J. Structure -based out -of-distribution (OOD) materials property prediction: a benchmark study. npj Comput Mater 10, 144 (2024)

  31. [40]

    & Parrinello, M

    Behler, J. & Parrinello, M. Generalized Neural -Network Representation of High -Dimensional Potential-Energy Surfaces. Phys. Rev. Lett. 98, 146401 (2007)

  32. [41]

    Atom -centered symmetry functions for constructing high -dimensional neural network potentials

    Behler, J. Atom -centered symmetry functions for constructing high -dimensional neural network potentials. The Journal of Chemical Physics 134, 074106 (2011)

  33. [42]

    S., Nebgen, B., Lubbers, N., Isayev, O

    Smith, J. S., Nebgen, B., Lubbers, N., Isayev, O. & Roitberg, A. E. Less is more: Sampling chemical space with active learning. J. Chem. Phys. 148, 241733 (2018)

Pith tools

Reviewed August 10, 2026 · model on record in the stance chip above.