REVIEW 3 major objections 6 minor 19 references
Optimal Design of Experiment for Electrochemical Parameter Identification of Li-ion Battery via Deep Reinforcement Learning
T0 review · 3 major / 6 minor · reviewed 2026-08-15 · deepseek-v4-flash
Pith's one-line read The paper claims that a deep reinforcement learning policy, trained with a squared-voltage-sensitivity reward, designs a 30-minute current excitation that yields higher Fisher information and lower parameter estimation errors for the…
desk verdict A careful simulation study of TD3-based optimal experiment design for battery rate constants, but the headline comparison with NMPC is tilted by horizon and the validation stays inside the training model; worth a serious look if you work on battery OED. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The load-bearing object is the single-step reward $r_k = (\partial V_k/\partial \theta_i)^2$, penalized by a constant $M$ when constraints are violated, combined with the TD3 actor-critic loop that learns a policy mapping cell terminal voltage and cathode surface concentration to current. This reward is not an approximation of Fisher information: under i.i.d. Gaussian measurement noise, $FI = \sigma_y^{-2} \sum_k (\partial y_k/\partial \theta)^2$, so maximizing cumulative squared sensitivity is exactly maximizing FI for the target parameter. The analytic sensitivity expressions $\partial V/\partial k_p = -\partial \eta_p/\partial i_{0p} \cdot \partial i_{0p}/\partial k_p$ and $\partial V/\partial k_n = \partial \eta_n/\partial i_{0n} \cdot \partial i_{0n}/\partial k_n$, computed from the E-ECM's Butler-Volmer and exchange-current equations, are what let both DRL and NMPC evaluate the reward. The E-ECM itself is the shared simulator that generates the voltage data used for training, excitation testing, and parameter estimation.
What would settle it
Run the DRL-designed and NMPC-designed 1800-second profiles on a physical 30 Ah NMC-graphite cell, estimate $k_p$ and $k_n$ from the measured voltage traces, and compare the residuals and parameter covariance against values obtained from the same cell with a long RCID test. If the learned profile does not produce lower estimation uncertainty than NMPC on real data, the paper's central claim fails; a cheaper check is to compare the model's predicted voltage sensitivity at high SoC and high C-rate with the measured sensitivity of a real cell.
Extended reading notes
Core claim
On the paper's own terms, the central discovery is that TD3, an actor-critic deep reinforcement learning algorithm, can solve the optimal experimental design problem for battery parameter identification better than a receding-horizon optimizer. The reward at each step is the squared sensitivity of terminal voltage to the target rate constant, with a penalty for violating voltage constraints; maximizing this reward over an episode maximizes Fisher information, since FI is a scaled sum of those squared sensitivities. The trained policy learns to push the cell into the regions where the model's voltage is most sensitive: high state of charge with high C-rate pulses for the anode rate constant $k_n$, and low state of charge with high C-rate pulses for the cathode rate constant $k_p$. In the reported simulations, this produces an average FI of 2.90e12 $V^{2}$ for $k_p$ and 4.86e12 $V^{2}$ for $k_n$, against 1.33e12 and 1.65e12 for NMPC, and median estimation errors of 0.244% and 0.233%, against 0.340% and 0.304% for NMPC, all within an 1800 s experiment. The paper also reports that the trained policy generates each current command in about 5.2 ms, versus about 5 s per step for NMPC, after a roughly 24-hour training phase.
Load-bearing premise
The battery model used to train the policy and to produce the synthetic test data must faithfully represent the real 30 Ah NMC-graphite cell; the paper validates only against that same model.
Editorial extensions
If this is right
- An 1800-second reinforcement-learned excitation replaces a 102,212-second RCID test for rate-constant identification, cutting experiment time by more than fifty times.
- Once trained, the policy evaluates a current command in milliseconds, so the same approach could close the loop in real time during a characterization test, something the NMPC's seconds-per-step computation makes impractical.
- Because the reward can be pointed at any model parameter by swapping the sensitivity in the objective, the procedure transfers to other electrochemical parameters beyond $k_p$ and $k_n$.
- Higher Fisher information implies a lower Cramér-Rao bound on unbiased estimators, so the identified rate constants should be less sensitive to measurement noise under the learned excitation.
- The policy learns qualitatively different profiles for different parameters (high-SoC pulsing for $k_n$, low-SoC depletion for $k_p$), showing that physics-guided excitation rather than a universal test is what delivers the gain.
Reading between the lines
- Because the validation uses synthetic voltage from the same E-ECM that trained the policy, the 0.24% error is an in-model result; on hardware, model mismatch and current-tracking error will raise both DRL and NMPC errors, and the practical gap may shrink.
- The reward optimizes one parameter at a time; a D-optimal reward that accounts for the cross-sensitivity between $k_p$ and $k_n$ could yield a single profile that jointly identifies both, which the paper does not test.
- The 24-hour training cost only pays off if the policy is reused on many cells or chemistries; whether a policy trained on one cell transfers to another with the same chemistry is a natural, untested extension.
- The NMPC comparison uses a 20-second prediction horizon; a longer-horizon NMPC could recover part of the gap, so part of the DRL advantage may be horizon rather than the learning algorithm itself.
Signed reviews
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The manuscript proposes a TD3-based deep reinforcement learning framework for optimal experimental design in electrochemical parameter identification of a Li-ion battery. The target parameters are the anode and cathode rate constants k_n and k_p in an E-ECM model. The DRL policy generates an 1800 s current excitation at 1 s resolution, with a reward equal to the squared voltage sensitivity with respect to the target parameter, subject to current and voltage constraints. The optimized excitations are compared against an NMPC design with a 20 s prediction horizon and against conventional test profiles (1C discharge, RCID, drive cycle). The authors report that the DRL profiles achieve the highest average Fisher information and the lowest median single-parameter least-squares estimation errors in simulation, and they claim superior information content with reduced experimental time and computational burden. All validation is performed on synthetic data generated from the same E-ECM model used for training.
Significance. If the claims held, the paper would provide a useful demonstration that DRL can generate informative excitation profiles for battery parameter identification, with a clear sensitivity-based interpretation and quantitative comparisons against multiple baselines. The strengths are the explicit sensitivity derivation, the inclusion of several conventional test procedures, and the measurement of online computation times. However, the central comparison with NMPC is structurally favorable to DRL because of the NMPC's short prediction horizon, the headline FI metric is the training objective itself, and all validation is noiseless, single-parameter, and synthetic. These issues must be addressed before the superiority claim can be considered established.
major comments (3)
- [III-B; IV-B] The NMPC baseline is given a prediction horizon of only 20 s (Section III-B), while the DRL policy is trained over the full 1800 s episode (Section IV-A). Section IV-B explicitly attributes the NMPC's lower FI for k_n to its "limited preview information" and lack of prior knowledge of the maximum sensitivity region at high SoC. This makes the comparison structurally favorable to DRL: an NMPC with a longer horizon, or with a terminal cost that encodes the sensitivity map, could plausibly match or exceed the DRL performance. The claim of superiority over NMPC in the abstract and Table II is therefore not established by the reported experiments.
- [IV-C; Eq. (12)] The parameter estimation validation is performed without measurement noise, using synthetic voltage data generated from the same E-ECM model that defines the training environment, and estimating one parameter at a time. Since the FI in Eq. (2) assumes i.i.d. Gaussian measurement noise and the Cramér-Rao bound in Eq. (3) is a statement about estimator variance under noise, this noiseless single-parameter setup does not test the mechanism by which FI is claimed to translate into estimation accuracy. The paper also defers hardware-in-the-loop validation to future work (Section V). Additional evidence, such as noisy data, joint multi-parameter estimation, or model mismatch, is needed to support the generalization claim.
- [III-A; Eq. (9); Table II] The average FI values in Table II are not an independent confirmation of the method's quality: the reward r_k in Eq. (9) is exactly the squared sensitivity (∂V_k/∂θ_i)^2, and FI in Eq. (2) is a constant times the sum of these per-step rewards. Therefore the DRL agent is trained to maximize the reported headline metric, so the higher FI for the DRL design is partly a restatement of the training objective. The lower parameter estimation errors are a separate and more meaningful outcome, but they are currently reported only for a noiseless single-parameter setting; the paper should present estimation errors as the primary evidence and treat the FI comparison as a consistency check.
minor comments (6)
- [Eq. (6)] In Eq. (6), the denominator appears to contain "i0,i" with a misplaced comma; this should be i_{0,i}, and the subscript convention should be checked throughout.
- [III-A] The statement that TD3 is "model-free and does not rely on an explicit mathematical model" is misleading because training uses the E-ECM simulator; the authors should clarify that the policy does not use an internal model at deployment, not that no model is used during training.
- [Table I] In Table I, the entry "Exploration Noise and Model 9 A and Ornstein-Uhlenbeck" is not a complete description; the authors should specify the noise standard deviation and the Ornstein-Uhlenbeck parameters.
- [Fig. 1] The sensitivity axis in Fig. 1 is labeled with units V^2, but the plotted quantity is (∂V/∂θ)^2; please state the parameter units or note that the values are normalized.
- [Eq. (12)] In Eq. (12), the notation \tilde V_k(\tilde θ_i, I_k) is used but \tilde V_k is not formally defined in the text; please define the perturbed-model voltage output explicitly.
- [Table II] The conventional test durations are very different from the DRL test duration (for example, RCID lasts 102,212 s), so the claim of reduced experimental time versus conventional tests is partly an artifact of the chosen test lengths; this should be acknowledged in the discussion.
Circularity Check
The DRL method's higher Fisher information is the training objective itself (the reward in Eq. 9 is the FI integrand of Eq. 2), so the FI component of the headline is circular; the lower parameter-estimation errors remain an independent simulation result.
-
fitted input called prediction
[Sec. II-B Eq. (2); Sec. III-A Eq. (9); Sec. IV-B Table II]
"FI= 1/σ_y^2 ∑_{k=1}^N (∂ y_k/∂ θ)^2 ... r_k = { (∂V_k/∂ θ_i)^2, normal condition; M, constraint violation } ... The first term maximizes the FI for the considered parameter ... the DRL approach achieves significantly higher average FI for k_n and k_p compared to the NMPC design."
The per-step reward in Eq. (9) is exactly the squared voltage sensitivity (∂V_k/∂θ_i)^2 that is summed in the FI definition of Eq. (2), and the paper explicitly states that the reward's first term 'maximizes the FI for the considered parameter.' Therefore the TD3 policy is trained to maximize a discounted sum of precisely the quantities whose total is reported as average Fisher information. Reporting that the DRL profile 'achieves significantly higher average FI' restates the training objective: the policy was selected for high values of this same metric, so its FI is a property of the fitted policy rather than an independent prediction. The NMPC baseline optimizes the same integrand over only a 20 s horizon, making the DRL FI advantage largely preordained.
full rationale
The paper's central claim has two components. The first, higher Fisher information, reduces by construction: Eq. (9) defines the DRL reward as the squared voltage sensitivity, while Eq. (2) defines FI as the sum of those same squared sensitivities (scaled by 1/σ_y^2). Thus the high-FI result is the policy's own training objective, a form of fitted input called prediction. The second component, lower median parameter estimation errors (0.24% for k_p and 0.23% for k_n), is not circular: the least-squares estimator in Eq. (12) is run on synthetic voltage data generated from the same E-ECM, and no constants are fitted to force the DRL profile to win; it is a legitimate, though simulation-only, comparison. Self-citations are present but not load-bearing in a way that imports an unverified uniqueness theorem: [14] supplies the E-ECM simulation environment, [17] motivates the choice of k_p and k_n, and [19] provides a drive-cycle input; these are normal uses of prior work. The admitted lack of hardware-in-the-loop validation (Sec. V) is a model-fidelity limitation, not a circularity. The NMPC's 20 s horizon versus DRL's full 1800 s episode is an experimental-design fairness concern affecting the FI comparison, but it is an evaluation artifact rather than a definitional circularity. Because one of the two headline results reduces to the training objective, the overall circularity score is 6 (partial circularity); the independent estimation-error result prevents a higher score.
Assumptions & free parameters
free parameters (4)
- NMPC prediction horizon H =
20 s
- Constraint violation penalty M =
-5
- TD3 discount factor gamma =
0.99
- TD3 exploration noise amplitude =
9 A (Ornstein-Uhlenbeck)
assumptions (5)
- domain assumption The E-ECM terminal voltage equation (1) with the reduced-order dynamics from [14] accurately reproduces the cell's voltage response for the experimental conditions studied.
- domain assumption Only the kinetic overpotential terms in Eq. (4) depend on the rate constants, and all other contributions cancel in the sensitivity expression of Eq. (5).
- domain assumption Measurement noise is i.i.d. Gaussian with known variance, justifying the Fisher information formula in Eq. (2) as 1/sigma^2 times the sum of squared sensitivities.
- domain assumption Single-parameter estimation with all other parameters fixed is representative of the parameter identification problem.
- ad hoc to paper TD3 training with up to 10,000 episodes converges to a near-optimal policy for the FI reward.
Cite this review
Pith. "Pith review of Optimal Design of Experiment for Electrochemical Parameter Identification of Li-ion Battery via Deep Reinforcement Learning." pith.science (2026). https://pith.science/paper/EZBJUSMG
@misc{pith2026250619146,
author = {Pith},
title = {Pith review of: Optimal Design of Experiment for Electrochemical Parameter Identification of Li-ion Battery via Deep Reinforcement Learning},
year = {2026},
howpublished = {\url{https://pith.science/paper/EZBJUSMG}},
note = {Machine review of arXiv:2506.19146}
}
read the original abstract
Accurate parameter estimation in electrochemical battery models is essential for monitoring and assessing the performance of lithium-ion batteries (LiBs). This paper presents a novel approach that combines deep reinforcement learning (DRL) with an optimal experimental design (OED) framework to identify key electrochemical parameters of LiB cell models. The proposed method utilizes the twin delayed deep deterministic policy gradient (TD3) algorithm to optimize input excitation, thereby increasing the sensitivity of the system response to electrochemical parameters. The performance of this DRL-based approach is evaluated against a nonlinear model predictive control (NMPC) method and conventional tests. Results indicate that the DRL-based method provides superior information content, reflected in higher Fisher information (FI) values and lower parameter estimation errors compared to the NMPC design and conventional test practices. Additionally, the DRL approach offers a substantial reduction in experimental time and computational resources.
Figures
Reference graph
Works this paper leans on
-
[1]
M. Andersson, M. Streb, J. Y . Ko, V . L¨ofqvist Klass, M. Klett, H. Ek- str¨om, M. Johansson, and G. Lindbergh, “Parametrization of physics- based battery models from input–output data: A review of method- ology and current research,”Journal of Power Sources, vol. 521, p. 230859, 2022
work page 2022
-
[2]
Global sensitivity methods for design of experiments in lithium-ion battery context,
A. Pozzi, X. Xie, D. Raimondo, and R. Schenkendorf, “Global sensitivity methods for design of experiments in lithium-ion battery context,”IFAC-PapersOnLine, vol. 53, no. 2, pp. 7248–7255, 2020. 21st IFAC World Congress
work page 2020
-
[3]
C.-H. Chen, F. B. Planella, K. O’regan, D. Gastol, W. D. Widanage, and E. Kendrick, “Development of experimental techniques for pa- rameterization of multi-scale lithium-ion battery models,”Journal of The Electrochemical Society, vol. 167, no. 8, p. 080534, 2020
work page 2020
-
[4]
Modeling and estimation for advanced battery management,
X. Lin, Y . Kim, S. Mohan, J. B. Siegel, and A. G. Stefanopoulou, “Modeling and estimation for advanced battery management,”Annual Review of Control, Robotics, and Autonomous Systems, vol. 2, no. V ol- ume 2, 2019, pp. 393–426, 2019
work page 2019
-
[5]
Q. Lai, H. J. Ahn, Y . Kim, Y . N. Kim, and X. Lin, “New data optimization framework for parameter estimation under uncertainties with application to lithium-ion battery,”Applied Energy, vol. 295, p. 117034, 2021
work page 2021
-
[6]
Optimal ex- perimental design for parameterization of an electrochemical lithium- ion battery model,
S. Park, D. Kato, Z. Gima, R. Klein, and S. Moura, “Optimal ex- perimental design for parameterization of an electrochemical lithium- ion battery model,”Journal of The Electrochemical Society, vol. 165, no. 7, p. A1309, 2018
work page 2018
-
[7]
Q. Lai, H. J. Ahn, G. Kim, W. T. Joe, and X. Lin, “Optimization of current excitation for identification of battery electrochemical parameters based on analytic sensitivity expression,” in2020 American control conference (ACC), pp. 346–351, IEEE, 2020
work page 2020
-
[8]
R. Huang, J. Fogelquist, and X. Lin, “Input excitation optimization for estimating battery electrochemical parameters using reinforcement learning,” in2022 IEEE Vehicle Power and Propulsion Conference (VPPC), pp. 1–6, 2022
work page 2022
Show all 19 references
-
[9]
Reinforcement learning of optimal input excitation for parameter estimation with application to li-ion battery,
R. Huang, J. Fogelquist, and X. Lin, “Reinforcement learning of optimal input excitation for parameter estimation with application to li-ion battery,”IEEE Transactions on Industrial Informatics, vol. 19, no. 11, pp. 11160–11170, 2023
2023
-
[10]
Excitation optimization for estimat- ing battery health parameters using reinforcement learning considering information content and bias,
R. Huang, M. Jones, and X. Lin, “Excitation optimization for estimat- ing battery health parameters using reinforcement learning considering information content and bias,” in2023 American Control Conference (ACC), pp. 3093–3098, IEEE, 2023
2023
-
[11]
Deep reinforcement learning versus evolution strategies: A comparative survey,
A. Y . Majid, S. Saaybi, V . Francois-Lavet, R. V . Prasad, and C. Ver- hoeven, “Deep reinforcement learning versus evolution strategies: A comparative survey,”IEEE Transactions on Neural Networks and Learning Systems, vol. 35, no. 9, pp. 11939–11957, 2024
2024
-
[12]
Addressing function approxi- mation error in actor-critic methods,
S. Fujimoto, H. Hoof, and D. Meger, “Addressing function approxi- mation error in actor-critic methods,” inInternational conference on machine learning, pp. 1587–1596, PMLR, 2018
2018
-
[13]
Improving aging identifiability of lithium-ion batteries using deep reinforcement learning,
H. Chun, K. Yoon, J. Kim, and S. Han, “Improving aging identifiability of lithium-ion batteries using deep reinforcement learning,”IEEE Transactions on Transportation Electrification, vol. 9, no. 1, pp. 995– 1007, 2023
2023
-
[14]
Physics-based equivalent circuit model for lithium-ion cells via reduction and ap- proximation of electrochemical model,
D. Seals, P. Ramesh, M. D’Arpino, and M. Canova, “Physics-based equivalent circuit model for lithium-ion cells via reduction and ap- proximation of electrochemical model,”SAE International Journal of Advances and Current Practices in Mobility, vol. 4, no. 2022-01-0701, pp. 115...
2022
-
[15]
E. L. Lehmann and G. Casella,Theory of point estimation. Springer Science & Business Media, 2006
2006
-
[16]
Geometry of the cramer-rao bound,
L. L. Scharf and L. T. McWhorter, “Geometry of the cramer-rao bound,”Signal Processing, vol. 31, no. 3, pp. 301–311, 1993
1993
-
[17]
Parameter identification for electrochem- ical models of lithium-ion batteries using sensitivity analysis,
C. Dangwal and M. Canova, “Parameter identification for electrochem- ical models of lithium-ion batteries using sensitivity analysis,”ASME Letters in Dynamic Systems and Control, vol. 1, no. 4, p. 041014, 2021
2021
-
[18]
Electro- thermal battery model identification for automotive applications,
Y . Hu, S. Yurkovich, Y . Guezennec, and B. Yurkovich, “Electro- thermal battery model identification for automotive applications,” Journal of Power Sources, vol. 196, no. 1, pp. 449–457, 2011
2011
-
[19]
Parameter identification for electrochemical models of lithium-ion batteries using bayesian optimization,
J. Pi, S. F. da Silva, M. F. Ozkan, A. Gupta, and M. Canova, “Parameter identification for electrochemical models of lithium-ion batteries using bayesian optimization,”IFAC-PapersOnLine, vol. 58, no. 28, pp. 180– 185, 2024. The 4th Modeling, Estimation, and Control Conference – 2024
2024
Reviewed August 15, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.