{"id":"45e7d502-8a78-468f-8bad-51bdf60798a5","arxiv_id":"2505.18162","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":4.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":3,"one_line_summary":"Two active-learning rounds with gradient boosting and particle swarm optimization reduced cathode capacity prediction error from 2.5% to 1.5%, but no baseline comparison supports the acceleration claim.","lead":"This paper applies an iterative machine learning loop to guide battery cathode experiments, claiming to find good materials faster and with fewer tries. The two-round demonstration improved accuracy but lacks a comparison against conventional experimentation, so the main claim is not yet established.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"No same-budget baseline means the acceleration claim is unsupported; error reduction is measured on model-selected points, not a representative test set.","rationale":"The reader's verdict is CONDITIONAL, and the primary basis is the absence of a same-budget baseline comparison and lack of code/data. My stress-test analysis reaches the same conclusion through a slightly different route: the paper's own abstract and conclusion make a comparative claim ('markedly reduces the total number of experimental cycles necessary'), but the experimental design contains no control arm. This is the single most load-bearing weakness because it directly undermines the central claim. A secondary issue strengthens the concern: the predictive-error metric is evaluated on non-representative, model-selected points, so the reported improvement (2.5% to 1.5%) does not establish improved global model accuracy. Together, these issues mean the paper is a useful case study, but its headline acceleration claim requires additional evidence.\n\nThe reader's formal 'weakest_assumption' focused on historical dataset consistency (mixing, furnace, gas flow), which is also a real concern. However, I consider the missing control arm more load-bearing because even if the dataset is perfectly consistent, the relative acceleration claim remains unsupported. Thus my agreement is 'partial': I share the reader's conditional stance and many of the same requested additions, but I identify a different weakest link. The recommended verdict is UNCHANGED because the reader already issued CONDITIONAL; my analysis reinforces, rather than redirects, that decision.","tokens_in":7756,"tokens_out":3207,"duration_ms":34004,"concrete_test":"Using the released dataset (or, failing that, a simulation with the same GBM pipeline and historical data), compare the two-shot AL selection against three same-budget controls: (i) 20 experiments chosen uniformly at random in the ~94% Ni region, (ii) 20 experiments chosen by a domain expert following standard heuristic rules, and (iii) a one-shot batch of 20 PSO-maximizing configurations without intermediate retraining. If any control reaches a comparable 223–229 mAh/g capacity window, or if the AL error reduction is not reproduced when evaluated on a held-out test set sampled uniformly across the full design space, the central acceleration claim fails. Additionally, recompute the MAE of the final model on the original zero-shot test set before and after adding the 20 new points to quantify true global model improvement.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim is that iterative active learning 'markedly reduces the total number of experimental cycles necessary' (Abstract; Conclusion). This is a relative claim, yet the paper provides no same-budget comparison against any alternative selection strategy (e.g., OFAT, random sampling, or expert choice). Two AL iterations (20 experiments) are reported, but there is no evidence that the same 20 experiments chosen randomly or by a domain expert would not achieve a similar 223–229 mAh/g window. Without such a control, the observed convergence could simply reflect the fact that the previously unexplored ~94% Ni region naturally yields high capacities, independent of the ML guidance.\n\nA second, compounding issue is the error metric. The reported improvement in mean absolute predictive error (from ~2.5% to ~1.5%, Results, 'Iterative Active Learning') is computed on the ten AL-proposed experiments in each round. These points are not a representative sample of the design space; they are concentrated in the narrow high-capacity region that the model (via PSO, Methods) was explicitly optimizing. Thus, the error decrease shows local calibration in a targeted region, not improved global predictive accuracy. A valid evaluation would use a held-out test set sampled uniformly from the parameter space, or at least report the model's error on the original dataset after retraining with the 20 new points.\n\nFinally, there is a mismatch between the stated active-learning mechanism and the implementation. The Methods say the AL loop identifies 'regions of high predictive uncertainty or sparse coverage,' but the actual selection procedure is PSO maximizing predicted initial discharge capacity over 1000 iterations (Methods, 'Domain Adaptation Framework using Particle Swarm Optimization'). The experiments were chosen for predicted performance, not for uncertainty or coverage.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper reports an iterative machine-learning framework for NCM cathode optimization at industrial scale. The authors compile a multi-year heuristic-driven dataset, train a gradient boosting model to predict initial discharge capacity, then run two active-learning iterations in which the model proposes ten new experiments per round via particle swarm optimization. After the first iteration the mean absolute predictive error drops from about 2.5% to about 1.5%, and experimentally measured capacities converge to 223–229 mAh/g in the second round. The central claims are that active learning markedly reduces the number of experimental cycles needed and that incorporating failed or suboptimal experiments improves model robustness and reduces anthropogenic bias.","tokens_in":8017,"tokens_out":1805,"duration_ms":17708,"significance":"If the acceleration claim were supported, this would be a valuable demonstration of ML-guided experimentation on real industrial-scale battery processing, with out-of-sample validation on genuinely new synthesis experiments. The paper's explicit use of negative data, its attention to heuristic biases in historical records, and its real kiln-scale synthesis conditions are strengths that distinguish it from synthetic or small-batch academic studies. However, the significance is currently limited because the central acceleration claim is not tested against any same-budget comparator, and the reported error reduction is measured on model-selected points rather than a representative test set. The work is a useful case study but does not yet quantitatively establish that active learning outperforms standard baselines.","major_comments":[{"comment":"The central claim that active learning 'markedly reduces the total number of experimental cycles necessary' is a relative claim, but the paper provides no same-budget baseline. There is no comparison against random selection, OFAT, or expert-driven selection using the same 20 new experiments. Without such a control, the observed convergence to 223–229 mAh/g could simply reflect that the previously unexplored ~94% Ni region naturally yields high capacities. Please add a baseline comparison or explicitly reframe the claim to describe what was demonstrated (e.g., convergence under one specific AL policy) rather than claiming a general acceleration relative to unspecified alternatives.","section":"Abstract; Results, 'Iterative Active Learning'; Conclusion"},{"comment":"The reported reduction in mean absolute predictive error, from ~2.5% to ~1.5%, is computed on the ten AL-proposed experiments in each round. These points are not a representative sample of the design space: they are concentrated in the narrow high-capacity region that the model (via PSO) was explicitly optimizing. The error decrease therefore shows local calibration in a targeted region, not improved global predictive accuracy. Please evaluate the model on a held-out test set sampled uniformly from the parameter space, or at least report the model's error on the original dataset after retraining with the 20 new points, so the reader can distinguish local refinement from genuine generalization.","section":"Results, 'Iterative Active Learning'; Figure 4"},{"comment":"There is a mismatch between the stated active-learning mechanism and the implemented acquisition strategy. The text says the GBM model 'identified regions of high predictive uncertainty or sparse coverage,' but the PSO procedure described maximizes predicted initial discharge capacity only, with no explicit uncertainty term or coverage criterion in the acquisition function. As written, the method is greedy optimization under the current surrogate model, not uncertainty-based active learning. Please clarify the exact acquisition function used, including how uncertainty or sparsity entered the selection of the ten experimental conditions.","section":"Methods, 'Domain Adaptation Framework using Particle Swarm Optimization'"},{"comment":"The validity of the entire modeling exercise depends on the assertion that 'consistent experimental conditions were meticulously maintained wherever possible' across a multi-year dataset. Unintentional variations in mixing homogeneity, furnace heat distribution, gas flow, or raw-material lots could be confounded with the recorded design variables, and the paper provides no batch-level or date-level metadata to verify this assumption. Please provide evidence of process stability (e.g., control charts of residual lithium, pH, tap density, or XRD metrics that were monitored but not used in ML) or discuss how potential drifts were handled.","section":"Results and discussion, 'Anthropogenic Bias...'; Methods, 'Experimental Methods'"}],"minor_comments":[{"comment":"The phrase 'iterative, iterative ML-driven active learning framework' contains a duplicated word; one 'iterative' should be removed.","section":"Introduction, first paragraph"},{"comment":"The table is referred to as 'Table A' in the Results section but is numbered 'Table 1' in the caption; please make the in-text reference consistent.","section":"Table 1 (labeled 'Table A' in text)"},{"comment":"The red points are described as 'domain expert-based experimental results' but the source of these data and how they were generated is not explained in the main text or methods; please clarify whether they come from the historical dataset or new expert-selected experiments.","section":"Figure 5"},{"comment":"Several references are incomplete or nonstandard (e.g., reference 1 lacks author and title, references 10 and 38 lack full bibliographic details). Please format all references consistently.","section":"References"},{"comment":"The manuscript does not state whether the dataset and code are available. Given the emphasis on data-driven methods and the importance of reproducibility, a data availability statement (even if data cannot be public due to industrial confidentiality) is needed.","section":"General"}],"recommendation":"major_revision","confidential_remarks":"The paper is a plausible industrial case study, but the main quantitative claim requires a same-budget baseline and a more careful evaluation protocol before it can be accepted. The absence of an uncertainty-aware acquisition function is also concerning given the 'active learning' framing. I would encourage the editor to request a revision that adds a baseline comparison and clarifies the acquisition criterion, rather than rejecting outright, because the underlying out-of-sample validation on new experiments is a genuine strength."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"The paper is a genuine industrial case study: two active-learning cycles on NCM cathode synthesis, where the new experiments are truly out-of-sample. That is its real strength. The authors also make a good empirical point about anthropogenic bias in their historical dataset—the skewed coating-temperature distribution in Figure 2 is a nice illustration of heuristic fixation. Including failed experiments is sensible and consistent with the cited literature.\n\nThe central claim, though, is not supported. The abstract says active learning 'markedly reduces the total number of experimental cycles necessary,' but there is no same-budget baseline: no random, OFAT, or expert-selection comparison. Without it, the observed convergence of the ten two-shot experiments to 223–229 mAh/g could simply reflect that the previously sparse 94% Ni region happens to be high-capacity. The ML guidance is not shown to be the cause.\n\nThe error metric also has a problem. The reported drop from ~2.5% to ~1.5% is computed on the ten points the model selected in each iteration. Those points are concentrated in a narrow high-capacity region, so the improvement shows local calibration, not better global prediction. A held-out test set sampled more broadly, or at least error on the original dataset after retraining, would be needed.\n\nThere is also an internal mismatch. The Methods say the active-learning loop identifies regions of high predictive uncertainty or sparse coverage, but the actual selection is PSO maximizing predicted discharge capacity over 1000 iterations. That is exploitation, not exploration. The prose and the implementation disagree, and this matters for how the results are framed.\n\nWeaker issues: the role of failed data is asserted, not ablated; no code or data are released; and the historical dataset's consistency relies on 'meticulously maintained wherever possible' without batch-level metadata. The last concern is minor because the new experiments were run under controlled conditions, but it limits confidence in the zero-shot model.\n\nWho gets value from this? Practitioners in industrial battery materials who want a template for applying GBM plus PSO to their own data, and anyone teaching active-learning evaluation pitfalls. It deserves a serious referee, because the dataset and demonstration are real and the weaknesses are addressable. But the authors need to add baselines, fix the selection-method description, and either release code or report detailed model settings. If those changes are made, the paper could be a solid case study; as it stands, the acceleration headline goes beyond what the data show.","headline":"A useful industrial case study whose acceleration claim is undercut by missing baselines and an error metric measured on model-selected points.","tokens_in":8624,"tokens_out":2566,"would_cite":false,"duration_ms":26135,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Two rounds of active-learning-chosen experiments reduced mean absolute prediction error for NCM cathode capacity from roughly 2.5% to 1.5% and converged measured capacities to a 223–229 mAh/g window.","keywords":["active learning","machine learning","battery materials","NCM cathode","gradient boosting","particle swarm optimization","anthropogenic bias","failed experiments"],"falsifier":"Run a matched control in which the same zero-shot GBM proposes two rounds of ten experiments chosen uniformly at random from the same candidate grid; if random selection achieves comparable error reduction and a comparable 223–229 mAh/g convergence, the active-learning loop is not what drives the improvement.","tokens_in":7557,"feed_emoji":"🔋","tokens_out":8682,"duration_ms":75785,"temperature":0.7,"pith_summary":"An iterative machine-learning workflow that uses active learning to choose the next experiments can break the bottlenecks of data scarcity and human bias in battery cathode development. The paper applies the workflow to years of NCM cathode synthesis records and reports that two active-learning rounds, adding ten new experiments each, reduced the mean absolute prediction error for initial discharge capacity from roughly 2.5% to 1.5% and pulled measured capacities into a 223–229 mAh/g window. It argues that the improvement comes from deliberately retraining on suboptimal and failed outcomes, which traditional Edisonian practice discards, and from widening parameter distributions that expert heuristics had compressed. If the claim holds, the approach offers a practical way to explore high-dimensional manufacturing parameter spaces with a handful of targeted experiments.","feed_headline":"Two active-learning rounds cut cathode error from 2.5% to 1.5%","feed_subtitle":"Just 20 ML-chosen experiments pushed NCM capacities into a tight 223–229 mAh/g window.","key_machinery":"The mechanism that carries the argument is an active-learning loop: a GBM, an ensemble model that combines many weak decision trees, predicts initial discharge capacity from composition, dopant, calcination, and coating variables; a Particle Swarm Optimizer (PSO) searches that space in discrete steps (5 °C calcination steps, 30-minute coating-time steps, small dopant increments) and returns the top ten predicted configurations; those ten are synthesized and tested; and all results, including failures, are folded back into the GBM for the next round. The loop does two things at once: it widens the parameter distributions the model has seen, and it feeds the model boundary information from suboptimal runs, which the authors identify as the key to sharper predictions.","core_discovery":"The authors' central claim is that an iterative active-learning loop, built on a Gradient Boosting Machine (GBM) and a Particle Swarm Optimizer, can systematically explore an industrial NCM cathode design space that years of heuristic experimentation left biased and incomplete. Starting from a zero-shot GBM trained only on historical records, the model targets the underrepresented 94% Ni composition and proposes ten conditions; validating them gives capacities of 215–229 mAh/g with about 2.5% average predictive error. After retraining on all ten outcomes, a second round proposes ten further conditions whose measured capacities converge to 223–229 mAh/g while average error drops to about 1.5%; the highest predicted value of about 229 mAh/g was not itself reached, but the measurements cluster tightly around it. The authors attribute the gain to the deliberate inclusion of failed and boundary experiments in each retraining step, which supplies the model with the negative data it needs to locate performance boundaries rather than merely reproducing known successes.","pith_inferences":["A direct control experiment would test whether the gain comes from active selection or simply from adding any new data: run the same two rounds with randomly chosen conditions and compare the final error and capacity window.","The paper's emphasis on negative data implies a broader data-management lesson: industrial archives that discard failed runs are not merely incomplete, they are biased in a way that no amount of successful examples can correct.","The 223–229 mAh/g convergence may reflect the optimizer exploiting the model's known high-capacity region rather than a proven global optimum; three or more iterations, or a comparison against a different optimizer, would be needed to separate these.","Applying the identical workflow to a different cathode chemistry, for example a lithium-rich or cobalt-free composition, would test whether two iterations give a comparable error tightening outside the NCM family."],"forward_implications":["Two ten-experiment rounds sufficed to shrink the mean absolute prediction error from roughly 2.5% to 1.5%, so the workflow can replace broad one-factor-at-a-time sweeps with a small number of targeted experiments.","By deliberately proposing conditions in sparse regions, such as 94% Ni and coating temperatures far from the heuristic cluster, the loop counteracts researcher fixation on historically successful parameter values.","Retraining on suboptimal outcomes is presented as essential: without the failed runs, the model would keep interpolating known successes instead of learning performance boundaries.","The paper points to expanding the dataset to additional variables and multiple laboratories as the natural follow-up, since the data-preparation and retraining steps are not specific to one chemistry.","Predictive accuracy and the spread of experimental outcomes both improve across iterations, indicating that reliability, not just peak predicted performance, is gained from each retraining cycle."],"supporting_citations":[{"why":"Documents how anthropogenic biases in chemical reaction data hinder exploratory synthesis, which grounds the paper's claim that heuristic-driven records are skewed.","marker":"[32]"},{"why":"Shows failed experiments can enable machine-learning-assisted materials discovery, the direct precedent for retraining on suboptimal NCM outcomes.","marker":"[25]"},{"why":"Argues that failed experiments are necessary for machine learning of chemical reactivity, supporting the paper's central retraining-on-negative-data step.","marker":"[26]"},{"why":"Supplies the active-learning and adaptive-sampling methodology that the paper adapts to choose which NCM experiments to run next.","marker":"[27]"},{"why":"Demonstrates in battery production design that modest numbers of well-chosen machine-learning experiments improve predictive accuracy, motivating the one-shot and two-shot cycle design.","marker":"[31]"},{"why":"Support the choice of gradient boosting as the baseline model for handling noisy, skewed parameter data.","marker":"[39-41]"},{"why":"Describe data scarcity and incomplete metadata as obstacles for machine learning in battery materials, the problems the active-learning loop is meant to overcome.","marker":"[22,23]"}],"fun_headline_variants":["ML cuts cathode error 40% in two rounds","Active learning finds better cathodes in 20 tests","Two ML rounds tighten NCM capacity to 223–229 mAh/g","Iterative ML slashes battery testing, boosts capacity"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The paper assumes the multi-year historical dataset was produced under effectively constant equipment and process conditions, with no batch- or date-level metadata supplied to verify this; if unrecorded drift in mixing, furnace heat, or gas flow occurred, the model may attribute capacity differences to the wrong design variables.","fun_headline_variants_meta":{"raw":{"variants":["ML cuts cathode error 40% in two rounds","Active learning finds better cathodes in 20 tests","Two ML rounds tighten NCM capacity to 223–229 mAh/g","Iterative ML slashes battery testing, boosts capacity"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000156,"raw_usage":{"total_tokens":1200,"prompt_tokens":910,"completion_tokens":290,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":526,"completion_tokens_details":{"reasoning_tokens":222}},"tokens_in":526,"tokens_out":290,"duration_ms":3181,"temperature":1.0,"reasoning_tokens":222,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-15T22:15:25.007042+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run a matched control in which the same zero-shot GBM proposes two rounds of ten experiments chosen uniformly at random from the same candidate grid; if random selection achieves comparable error reduction and a comparable 223–229 mAh/g convergence, the active-learning loop is not what drives the improvement.","supporting_citations":[{"cited_title":"Anthropogenic biases in chemical reaction data hinder exploratory inorganic synthesis","cited_arxiv_id":null,"evidence_quote":"Documents how anthropogenic biases in chemical reaction data hinder exploratory synthesis, which grounds the paper's claim that heuristic-driven records are skewed."},{"cited_title":"Machine-learning-assisted materials discovery using failed experiments","cited_arxiv_id":null,"evidence_quote":"Shows failed experiments can enable machine-learning-assisted materials discovery, the direct precedent for retraining on suboptimal NCM outcomes."},{"cited_title":"Machine learning for chemical reactivity: the importance of failed experiments","cited_arxiv_id":null,"evidence_quote":"Argues that failed experiments are necessary for machine learning of chemical reactivity, supporting the paper's central retraining-on-negative-data step."},{"cited_title":"Active learning in materials science with emphasis on adaptive sampling using uncertainties for targeted design","cited_arxiv_id":null,"evidence_quote":"Supplies the active-learning and adaptive-sampling methodology that the paper adapts to choose which NCM experiments to run next."},{"cited_title":"Battery production design using multi -output machine learning models","cited_arxiv_id":null,"evidence_quote":"Demonstrates in battery production design that modest numbers of well-chosen machine-learning experiments improve predictive accuracy, motivating the one-shot and two-shot cycle design."}],"review_version":1}