{"id":"2f000f3a-2453-4892-b514-df8f708327d6","arxiv_id":"2601.01185","paper_version":2,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":4.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":5,"one_line_summary":"AutoPot automates active-learning construction of moment tensor potentials, demonstrated on W and Mo-Ta, but its on-the-fly abort logic is unreachable and the claimed 'couple of hours' runtime is never measured.","lead":"AutoPot is a new workflow tool that automatically picks training data for machine-learned interatomic potentials, tested by building tungsten and molybdenum-tantalum potentials. The paper shows the tool can produce accurate-looking potentials, but its headline claims about speed and on-the-fly sampling include a logical error and unquantified timing.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"AutoPot's stopping criterion rests on the D-optimality grade γ being a faithful error proxy; the paper's own Section 6 concedes γ is not a good quantification of true error, and no calibration test is provided.","rationale":"The reader's weakest assumption is exactly the reliability of γ as a proxy for true error, and the paper's own Section 6 acknowledges this limitation. This is the most load-bearing concern because the entire automation claim depends on the stopping criterion: if γ fails to flag important neighborhoods, AutoPot terminates prematurely while still claiming convergence. The paper does provide some independent support — the W MTP trained on bulk/stacking-fault configurations predicts dislocation-core properties and the Peierls barrier in reasonable agreement with a reference MTP and DFT, and the Mo-Ta MTP matches DFT for elastic constants and stacking fault energies across compositions. These benchmarks give partial evidence that γ-based selection can work in practice. However, none of these tests directly calibrates γ against true errors, and the benchmark properties may not be sensitive to local errors that would matter in other applications (e.g., high-temperature dynamics or defect migration). The 'couple of hours' runtime claim and the Algorithm 2.2 abort-condition ordering bug are secondary concerns: they are concrete and fixable, but the reliability of the convergence criterion is the scientific crux. Since the reader's verdict is already CONDITIONAL and the concern aligns with that, no verdict change is needed.","tokens_in":17467,"tokens_out":4744,"duration_ms":54920,"concrete_test":"Take the trained Mo-Ta MTP from §5.2 and the held-out DFT setup from Appendix B. Generate 50–100 configurations from (i) MD at 2000 K, and (ii) compositions not explicitly present in the training set (e.g., Ta-rich or Mo-rich beyond the Dirichlet samples). For each, compute γ (Eq. 6) and DFT forces/energies. Plot force MAE versus γ and count configurations with γ ≤ 2 whose force error exceeds the training tolerance (≈10 meV/Å). If the count is significant, the γ<2 stopping rule is unsafe; if not, the concern is settled.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim is that AutoPot 'fully automatically' constructs highly accurate MTPs. The loop terminates when no sampled configuration has extrapolation grade γ > γmin (Section 2.3, Algorithm 2.2), with γmin in the range 1–2. This is only sound if γ ≤ 2 implies the MTP error is acceptable. Section 6 concedes that γ 'is an error indicator and guarantees a stable potential but is not necessarily a good quantification of the true error.' The paper never calibrates γ against actual energy/force errors on held-out configurations, nor tests whether the stopping threshold transfers to temperatures, strain states, or chemistries beyond the training candidates. In particular, the Mo-Ta potential is trained on bulk and stacking-fault configurations at ±3% strain and MD at 100 K; the claimed 'wide configuration and chemical space' (alloy compositions) is validated only on lattice constants, elastic constants, and stacking fault energies — quantities that may be insensitive to local environment errors. If γ is miscalibrated, AutoPot terminates with an undertrained potential while still reporting convergence. This is not merely a disagreement with consensus; it is an internal admission (Section 6) that the stopping criterion is not error-calibrated.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper presents AutoPot, an open-source workflow for automating the construction of Moment Tensor Potentials (MTPs) through active learning. AutoPot is built on the BlackDynamite parametric-study framework and the Motoko event-based orchestrator, and it implements two selection protocols: pre-selection from a large candidate set (Algorithm 2.1) and on-the-fly selection from molecular dynamics simulations (Algorithm 2.2), both using the D-optimality extrapolation grade γ. The workflow is demonstrated by constructing MTPs for bcc W and for Mo-Ta random alloys, with validation against DFT for lattice constants, elastic constants, stacking-fault energies, and, for W, screw-dislocation core structure and Peierls barrier. The central claim is that AutoPot enables fully automatic, reproducible construction of accurate MTPs within a couple of hours with minimal user input.","tokens_in":17853,"tokens_out":7933,"duration_ms":222524,"significance":"The automation and provenance-tracking of MLIP training protocols is a genuine need in computational materials science, and the paper's architecture (persistent task storage, asynchronous orchestration, Python-based tasks) is a reasonable response. If the workflow is robust, it lowers the barrier for non-experts and improves reproducibility. The two demonstrations show that the produced potentials match DFT or reference MTPs well for the tested properties, which is encouraging. The paper explicitly provides open-source repositories, which is a strength. However, the validity of the stopping criterion and the quantitative performance claim ('couple of hours') are not demonstrated, and one of the two presented algorithms contains a logical error. These issues need to be resolved before the paper's claims can be fully accepted.","major_comments":[{"comment":"The conditional in the MD selection loop is logically inverted. If γmin < γmax (the natural reading, also implied by the names 'min' and 'max'), then the condition 'γ* > γmax' is never reached for γ* > γmin, because any γ* > γmax already satisfies γ* > γmin and is captured by the first branch. Thus the abort-on-risky-configuration branch is unreachable. If instead γmin > γmax, the naming is confusing and the abort branch is reachable only for γmax < γ* ≤ γmin, again not the intended semantics. This is a load-bearing defect: the safety mechanism that prevents continuing MD in unsafe extrapolation regions does nothing as written. Even if the implementation in the repository is correct, the pseudocode misrepresents the algorithm and could mislead users.","section":"Algorithm 2.2, lines 7–11"},{"comment":"The claim that AutoPot constructs 'highly accurate MTPs ... fully automatically within a couple of hours' is unsupported by any timing or scaling data. No wall-clock times, number of VASP single points, degree of parallelism, or speedups are reported. The paper mentions that the examples ran on VSC-5, but that is not sufficient to substantiate a quantitative performance claim. The authors should either provide timing measurements for the W and Mo-Ta cases (including the number of DFT calculations and the parallelization achieved) or soften the claim to something qualitative such as 'rapid'.","section":"Abstract and §5.2"},{"comment":"The stopping criterion of the workflow is that no configuration has extrapolation grade γ above γmin (typically 1–2). The paper's own Section 6 states that γ 'is an error indicator and guarantees a stable potential but is not necessarily a good quantification of the true error.' This is a frank admission, but it means that the central 'fully automatic' claim rests on an uncalibrated proxy. The paper reports no test of whether γ ≤ γmin corresponds to acceptable energy/force errors on held-out configurations, nor any sensitivity analysis of the choice of γmin. Without such a calibration, a user cannot know whether AutoPot terminates with an under-trained potential in a new application. I recommend adding either (i) a calibration plot of γ against actual errors on a held-out set for the demonstrated systems, or (ii) a comparison of potentials produced with different γmin values (e.g., 1, 2,","section":"§2.3 and §6"}],"minor_comments":[{"comment":"The text says 'The MTP training errors are given in Table 2' and 'the agreement with DFT ... as shown in Table 2', but Table 2 is the W property table; the training errors are in Table 1, and the Mo-Ta property comparison is in Figure 6. Please correct these cross-references.","section":"§5.2, Table 1 and Figure 6"},{"comment":"The pseudocode of Algorithm 2.1 adds every candidate with γ > γthreshold to T_selected, while the preceding paragraph argues that in practice it suffices to add only the candidate with the highest extrapolation grade. This is a real algorithmic difference, not just a simplification. Clarify which behavior is implemented in AutoPot and, if only the highest is added, align the pseudocode with the implementation.","section":"Algorithm 2.1 and §2.3 text"},{"comment":"The sentence 'We remark that the training set typically contains many more configurations than parameters, making A overdetermined' is imprecise: A as defined in Eq. (5) is an m×m matrix built from m active configurations. What is meant is that the full training set is overdetermined and maxvol selects the m most representative rows. Suggest rewording.","section":"§2.2, after Eq. (5)"},{"comment":"The instruction 'Add the most representative configurations from T_cand to the training set' is vague. How is 'most representative' determined? Since Algorithm 2.1 is then invoked with γthreshold = γmin, and all configurations in T_cand have γ > γmin by construction, it is unclear how any filtering occurs. Please specify the criterion used.","section":"Algorithm 2.2, line 15"},{"comment":"The sentence 'The D-optimality criterion is an error indicator and guarantees a stable potential' conflates two different levels of certainty. An 'error indicator' does not 'guarantee' stability in a mathematical sense. Given the same paragraph immediately notes that γ is not a good quantification of the true error, please soften 'guarantees' to 'is intended to support' or similar.","section":"§6, first paragraph"}],"recommendation":"major_revision","confidential_remarks":"The paper is well within the scope of physics.comp-ph and the software is a useful contribution. The main concerns are the Algorithm 2.2 logic error, the unsupported 'couple of hours' claim, and the lack of calibration for the stopping criterion; all three are addressable in revision. The authors appear to be the developers of the underlying MTP methods, which explains a number of self-citations, but that is not inappropriate in this context."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"AutoPot is a real software contribution: it packages active-learning loops for MTPs into a persistent, queryable workflow manager (Motoko) and demonstrates the whole thing on two nontrivial systems. That's worth having. The authors are honest that the scientific ingredients—D-optimality selection, on-the-fly sampling—are prior work; the new part is the orchestration and data model. The validation is solid for a software paper: the W potential reproduces the reference MTP's dislocation core and Peierls barrier, and the Mo-Ta potential tracks DFT across the composition range. Training errors are in the expected range. Code and data are on GitLab, which counts for a lot.\n\nThe soft spots, in order of seriousness.\n\nFirst, Algorithm 2.2 has a real ordering bug: the `else if γ > γmax` branch is unreachable because any γ exceeding γmax also exceeds γmin (assuming γmax > γmin, which is the intended relation). So the 'abort the simulation' safeguard never fires. That undermines one of the two claimed selection mechanisms, though in practice the MD sampling still worked because the potentials were adequate. This should be fixed and re-tested.\n\nSecond, the 'fully automatically within a couple of hours' claim is unmeasured. No timing data anywhere. I'd either add wall-clock timings or drop the phrase.\n\nThird, and most substantive: the stopping criterion is γ ≤ 2, and the authors themselves concede in Section 6 that γ is not necessarily a good quantification of true error. The paper never calibrates γ against held-out energy/force errors, nor tests whether the threshold transfers to other temperatures or chemistries. For the two demonstrations the benchmarks are reassuring, but the transferability claim rests on a proxy. This limitation is stated, not hidden, which I respect. For a first software release it's acceptable; for a claim of general 'fully automatic' construction it needs either calibration data or more cautious language.\n\nThe property comparisons for alloys lack error bars, but that's minor for a demonstration.\n\nOverall: this is an honest, useful software paper. The bug and the missing timing are fixable; the γ caveat is an acknowledged limitation, not a fatal flaw. I'd send it to peer review.","headline":"AutoPot is a useful, well-demonstrated workflow tool for automated MTP active learning; the stopping-criterion caveat and a bug in Algorithm 2.2 need fixing, but the software deserves peer review.","tokens_in":18299,"tokens_out":3942,"would_cite":true,"duration_ms":38919,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"The paper claims that a fully automated active-learning workflow can now train quantum-accurate machine-learning interatomic potentials in a couple of hours, with no human in the loop and with every intermediate calculation archived.","keywords":["active learning","machine-learning interatomic potentials","moment tensor potentials","D-optimality criterion","extrapolation grade","workflow orchestration","reproducibility","random alloys"],"falsifier":"Run a molecular dynamics simulation at, say, 2000 K for tungsten using the AutoPot-trained potential, compute the DFT energy of a few sampled snapshots, and check whether any snapshot with γ<2 deviates by more than, say, 50 meV/atom in energy. If such a snapshot exists, the D-optimality gate failed to select it.","tokens_in":17407,"feed_emoji":"⚛️","tokens_out":7900,"duration_ms":76330,"temperature":0.7,"pith_summary":"This paper introduces AutoPot, a software platform that claims to automate the entire active-learning protocol for machine-learning interatomic potentials. The workflow selects uncertain atomic configurations from large candidate sets or from running molecular dynamics, launches density functional theory single-point calculations that add these configurations to the training set, retrains the potential, and repeats until no new configuration is deemed uncertain. The central claim is that a user can go from a small initial training set and a pool of candidates to a production-ready moment tensor potential within a couple of hours, with minimal manual input, because the event-driven orchestrator runs thousands of tasks in parallel and preserves every intermediate dataset for reproducibility. The paper demonstrates this for tungsten and for molybdenum-tantalum random alloys: the resulting potentials are trained from only 123 and 389 configurations, respectively, and predict elastic constants, stacking-fault energies, dislocation core structures, and Peierls barriers within a few percent of density functional theory reference values. A sympathetic reader would care because the bottleneck of building high-quality training sets, currently requiring expert attention and careful protocols, is removed.","feed_headline":"Automated active learning builds atomistic potentials in hours","feed_subtitle":"A parallel workflow selects the right training data, checks it with quantum mechanics, and retrains until done.","key_machinery":"The extrapolation grade γ (defined in Eq. 6) is the scalar that carries the entire active-learning logic: for a candidate configuration, γ = max_i |c_i|, where c is the row vector of the candidate's derivative with respect to the MTP parameters multiplied by the inverse of the Jacobian matrix A of the active set. It classifies a configuration as interpolating (γ≤1), accurately extrapolating (1<γ≤2), still reliable (2<γ≤10), or risky (γ>10), and AutoPot aborts MD simulations or spawns DFT calculations accordingly. The maxvol algorithm selects the m most linearly independent neighborhoods to build A efficiently. Around this quantity, the paper builds an event-based asynchronous orchestrator th","core_discovery":"The central claim is that constructing a quantum-accurate machine-learning interatomic potential can be made fully automatic by tying together, in a parallelized event-driven workflow, the four steps of active learning—configuration selection, reference quantum-mechanical evaluation, retraining, and convergence checking—and by storing every dataset and metadata so the process can be queried, reused, and reproduced. The decision of which configurations need quantum-mechanical evaluation is driven by the extrapolation grade γ, a scalar derived from the D-optimality criterion: γ measures, effectively, how far a candidate atomic neighborhood is from the current training set. Configurations above","pith_inferences":["The paper's convergence claim leans on the extrapolation grade γ being a faithful error oracle; since the authors themselves note that γ is not a good quantification of the true error, a natural next test is to couple AutoPot with calibrated uncertainty estimates to see whether the automatically chosen training set still misses rare but important neighborhoods.","The orchestration pattern—event-based, parallel, data-preserving—appears portable to other active-learning problems, such as training continuum models from atomistic data, which the paper explicitly names as future work.","A concrete stress test of the 'fully automatic in a couple of hours' claim would be running the identical protocol on a ternary or magnetic alloy; the growth in configuration space and the need for multiple γ thresholds may push the runtime well beyond the two-hour mark, revealing how much of the claim depends on system simplicity.","The demonstrated benchmarks focus on ground-state and low-temperature properties; long high-temperature runs or radiation-damage conditions would be a stronger test of whether the automatically assembled training set ensures simulation stability far from the initial candidate distribution."],"forward_implications":["A user can start from a handful of initial configurations and a set of candidate structures and receive a converged moment tensor potential in about two hours, without manually adjusting thresholds or restarting simulations.","Every task input, output, and parameter value is stored in a queryable object database, so a finished training run can be reconstructed exactly, or a branch of the workflow can be rerun without recomputing unchanged parts.","Because tasks are defined as plain Python functions, the same orchestrator can accommodate other potential classes and other quantum-chemistry packages, making the protocol a testbed for comparing active-learning strategies.","The on-the-fly sampling stage can monitor thousands of independent molecular dynamics runs, automatically requesting fresh quantum-mechanical reference data for any trajectory that begins to extrapolate, and aborting those that go too far."],"fun_headline_variants":["AutoPot automates atomistic potential building in parallel","Parallel workflow fully automates ML potential construction","Automated pipeline builds quantum-accurate interatomic potentials","Self-driving workflow trains atomistic potentials end-to-end","AutoPot: turnkey active learning for machine-learned potentials"],"cache_read_input_tokens":2304,"weakest_assumption_plain":"The load-bearing premise is that the extrapolation grade γ reliably flags every atomic neighborhood where the potential is inaccurate; if a configuration has a low γ but the potential's energy and forces there are nevertheless far from quantum-mechanical results, AutoPot reports convergence and hands over an undertrained potential.","fun_headline_variants_meta":{"raw":{"variants":["AutoPot automates atomistic potential building in parallel","Parallel workflow fully automates ML potential construction","Automated pipeline builds quantum-accurate interatomic potentials","Self-driving workflow trains atomistic potentials end-to-end","AutoPot: turnkey active learning for machine-learned potentials"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000556,"raw_usage":{"total_tokens":2522,"prompt_tokens":820,"completion_tokens":1702,"prompt_tokens_details":{"cached_tokens":256},"prompt_cache_hit_tokens":256,"prompt_cache_miss_tokens":564,"completion_tokens_details":{"reasoning_tokens":1625}},"tokens_in":564,"tokens_out":1702,"duration_ms":12780,"temperature":1.0,"reasoning_tokens":1625,"cache_read_input_tokens":256,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-03T12:52:24.250923+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run a molecular dynamics simulation at, say, 2000 K for tungsten using the AutoPot-trained potential, compute the DFT energy of a few sampled snapshots, and check whether any snapshot with γ<2 deviates by more than, say, 50 meV/atom in energy. If such a snapshot exists, the D-optimality gate failed to select it.","supporting_citations":[],"review_version":1}