Pith. sign in

REVIEW 3 major objections 6 minor 19 references

Optimal Design of Experiment for Electrochemical Parameter Identification of Li-ion Battery via Deep Reinforcement Learning

T0 review · 3 major / 6 minor · reviewed 2026-08-15 · deepseek-v4-flash

Pith's one-line read The paper claims that a deep reinforcement learning policy, trained with a squared-voltage-sensitivity reward, designs a 30-minute current excitation that yields higher Fisher information and lower parameter estimation errors for the…

desk verdict A careful simulation study of TD3-based optimal experiment design for battery rate constants, but the headline comparison with NMPC is tilted by horizon and the validation stays inside the training model; worth a serious look if you work on battery OED. read the letter →

arxiv 2506.19146 v1 pith:EZBJUSMG submitted 2025-06-23 eess.SY cs.SY

classification eess.SYcs.SY
keywords optimalexperimentaldesigndeepreinforcementlearningTD3electrochemicalparameteridentificationlithium-ionbatteryFisherinformationmodelpredictivecontrolsensitivityanalysis
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper is trying to establish that the current profile used to identify a lithium-ion battery's electrochemical parameters can itself be learned: a deep reinforcement learning agent can design a short, information-rich excitation that makes the parameters easier to estimate from voltage measurements. If true, battery characterization would not have to choose between long standard tests and expensive online optimization, because the learned policy would instantly output a current profile that maximizes the Fisher information of the response with respect to the rate constants $k_p$ and $k_n$. Using a computationally efficient electrochemical equivalent circuit model of a 30 Ah NMC-graphite cell, the authors compare this learned excitation with a nonlinear model-predictive controller and with conventional tests, reporting that the reinforcement-learned profile gives the highest Fisher information and the lowest single-parameter least-squares estimation errors in an 1800-second experiment. The practical payoff would be faster, cheaper, and more consistent parameter identification for physics-based battery models.

What carries the argument

The load-bearing object is the single-step reward $r_k = (\partial V_k/\partial \theta_i)^2$, penalized by a constant $M$ when constraints are violated, combined with the TD3 actor-critic loop that learns a policy mapping cell terminal voltage and cathode surface concentration to current. This reward is not an approximation of Fisher information: under i.i.d. Gaussian measurement noise, $FI = \sigma_y^{-2} \sum_k (\partial y_k/\partial \theta)^2$, so maximizing cumulative squared sensitivity is exactly maximizing FI for the target parameter. The analytic sensitivity expressions $\partial V/\partial k_p = -\partial \eta_p/\partial i_{0p} \cdot \partial i_{0p}/\partial k_p$ and $\partial V/\partial k_n = \partial \eta_n/\partial i_{0n} \cdot \partial i_{0n}/\partial k_n$, computed from the E-ECM's Butler-Volmer and exchange-current equations, are what let both DRL and NMPC evaluate the reward. The E-ECM itself is the shared simulator that generates the voltage data used for training, excitation testing, and parameter estimation.

What would settle it

Run the DRL-designed and NMPC-designed 1800-second profiles on a physical 30 Ah NMC-graphite cell, estimate $k_p$ and $k_n$ from the measured voltage traces, and compare the residuals and parameter covariance against values obtained from the same cell with a long RCID test. If the learned profile does not produce lower estimation uncertainty than NMPC on real data, the paper's central claim fails; a cheaper check is to compare the model's predicted voltage sensitivity at high SoC and high C-rate with the measured sensitivity of a real cell.

Watch

Extended reading notes

Core claim

On the paper's own terms, the central discovery is that TD3, an actor-critic deep reinforcement learning algorithm, can solve the optimal experimental design problem for battery parameter identification better than a receding-horizon optimizer. The reward at each step is the squared sensitivity of terminal voltage to the target rate constant, with a penalty for violating voltage constraints; maximizing this reward over an episode maximizes Fisher information, since FI is a scaled sum of those squared sensitivities. The trained policy learns to push the cell into the regions where the model's voltage is most sensitive: high state of charge with high C-rate pulses for the anode rate constant $k_n$, and low state of charge with high C-rate pulses for the cathode rate constant $k_p$. In the reported simulations, this produces an average FI of 2.90e12 $V^{2}$ for $k_p$ and 4.86e12 $V^{2}$ for $k_n$, against 1.33e12 and 1.65e12 for NMPC, and median estimation errors of 0.244% and 0.233%, against 0.340% and 0.304% for NMPC, all within an 1800 s experiment. The paper also reports that the trained policy generates each current command in about 5.2 ms, versus about 5 s per step for NMPC, after a roughly 24-hour training phase.

Load-bearing premise

The battery model used to train the policy and to produce the synthetic test data must faithfully represent the real 30 Ah NMC-graphite cell; the paper validates only against that same model.

Editorial extensions

If this is right

  • An 1800-second reinforcement-learned excitation replaces a 102,212-second RCID test for rate-constant identification, cutting experiment time by more than fifty times.
  • Once trained, the policy evaluates a current command in milliseconds, so the same approach could close the loop in real time during a characterization test, something the NMPC's seconds-per-step computation makes impractical.
  • Because the reward can be pointed at any model parameter by swapping the sensitivity in the objective, the procedure transfers to other electrochemical parameters beyond $k_p$ and $k_n$.
  • Higher Fisher information implies a lower Cramér-Rao bound on unbiased estimators, so the identified rate constants should be less sensitive to measurement noise under the learned excitation.
  • The policy learns qualitatively different profiles for different parameters (high-SoC pulsing for $k_n$, low-SoC depletion for $k_p$), showing that physics-guided excitation rather than a universal test is what delivers the gain.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Because the validation uses synthetic voltage from the same E-ECM that trained the policy, the 0.24% error is an in-model result; on hardware, model mismatch and current-tracking error will raise both DRL and NMPC errors, and the practical gap may shrink.
  • The reward optimizes one parameter at a time; a D-optimal reward that accounts for the cross-sensitivity between $k_p$ and $k_n$ could yield a single profile that jointly identifies both, which the paper does not test.
  • The 24-hour training cost only pays off if the policy is reused on many cells or chemistries; whether a policy trained on one cell transfers to another with the same chemistry is a natural, untested extension.
  • The NMPC comparison uses a 20-second prediction horizon; a longer-horizon NMPC could recover part of the gap, so part of the DRL advantage may be horizon rather than the learning algorithm itself.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 6 minor

Summary. The manuscript proposes a TD3-based deep reinforcement learning framework for optimal experimental design in electrochemical parameter identification of a Li-ion battery. The target parameters are the anode and cathode rate constants k_n and k_p in an E-ECM model. The DRL policy generates an 1800 s current excitation at 1 s resolution, with a reward equal to the squared voltage sensitivity with respect to the target parameter, subject to current and voltage constraints. The optimized excitations are compared against an NMPC design with a 20 s prediction horizon and against conventional test profiles (1C discharge, RCID, drive cycle). The authors report that the DRL profiles achieve the highest average Fisher information and the lowest median single-parameter least-squares estimation errors in simulation, and they claim superior information content with reduced experimental time and computational burden. All validation is performed on synthetic data generated from the same E-ECM model used for training.

Significance. If the claims held, the paper would provide a useful demonstration that DRL can generate informative excitation profiles for battery parameter identification, with a clear sensitivity-based interpretation and quantitative comparisons against multiple baselines. The strengths are the explicit sensitivity derivation, the inclusion of several conventional test procedures, and the measurement of online computation times. However, the central comparison with NMPC is structurally favorable to DRL because of the NMPC's short prediction horizon, the headline FI metric is the training objective itself, and all validation is noiseless, single-parameter, and synthetic. These issues must be addressed before the superiority claim can be considered established.

major comments (3)
  1. [III-B; IV-B] The NMPC baseline is given a prediction horizon of only 20 s (Section III-B), while the DRL policy is trained over the full 1800 s episode (Section IV-A). Section IV-B explicitly attributes the NMPC's lower FI for k_n to its "limited preview information" and lack of prior knowledge of the maximum sensitivity region at high SoC. This makes the comparison structurally favorable to DRL: an NMPC with a longer horizon, or with a terminal cost that encodes the sensitivity map, could plausibly match or exceed the DRL performance. The claim of superiority over NMPC in the abstract and Table II is therefore not established by the reported experiments.
  2. [IV-C; Eq. (12)] The parameter estimation validation is performed without measurement noise, using synthetic voltage data generated from the same E-ECM model that defines the training environment, and estimating one parameter at a time. Since the FI in Eq. (2) assumes i.i.d. Gaussian measurement noise and the Cramér-Rao bound in Eq. (3) is a statement about estimator variance under noise, this noiseless single-parameter setup does not test the mechanism by which FI is claimed to translate into estimation accuracy. The paper also defers hardware-in-the-loop validation to future work (Section V). Additional evidence, such as noisy data, joint multi-parameter estimation, or model mismatch, is needed to support the generalization claim.
  3. [III-A; Eq. (9); Table II] The average FI values in Table II are not an independent confirmation of the method's quality: the reward r_k in Eq. (9) is exactly the squared sensitivity (∂V_k/∂θ_i)^2, and FI in Eq. (2) is a constant times the sum of these per-step rewards. Therefore the DRL agent is trained to maximize the reported headline metric, so the higher FI for the DRL design is partly a restatement of the training objective. The lower parameter estimation errors are a separate and more meaningful outcome, but they are currently reported only for a noiseless single-parameter setting; the paper should present estimation errors as the primary evidence and treat the FI comparison as a consistency check.
minor comments (6)
  1. [Eq. (6)] In Eq. (6), the denominator appears to contain "i0,i" with a misplaced comma; this should be i_{0,i}, and the subscript convention should be checked throughout.
  2. [III-A] The statement that TD3 is "model-free and does not rely on an explicit mathematical model" is misleading because training uses the E-ECM simulator; the authors should clarify that the policy does not use an internal model at deployment, not that no model is used during training.
  3. [Table I] In Table I, the entry "Exploration Noise and Model 9 A and Ornstein-Uhlenbeck" is not a complete description; the authors should specify the noise standard deviation and the Ornstein-Uhlenbeck parameters.
  4. [Fig. 1] The sensitivity axis in Fig. 1 is labeled with units V^2, but the plotted quantity is (∂V/∂θ)^2; please state the parameter units or note that the values are normalized.
  5. [Eq. (12)] In Eq. (12), the notation \tilde V_k(\tilde θ_i, I_k) is used but \tilde V_k is not formally defined in the text; please define the perturbed-model voltage output explicitly.
  6. [Table II] The conventional test durations are very different from the DRL test duration (for example, RCID lasts 102,212 s), so the claim of reduced experimental time versus conventional tests is partly an artifact of the chosen test lengths; this should be acknowledged in the discussion.

Circularity Check

1 steps flagged · score 6.0 of 10

The DRL method's higher Fisher information is the training objective itself (the reward in Eq. 9 is the FI integrand of Eq. 2), so the FI component of the headline is circular; the lower parameter-estimation errors remain an independent simulation result.

  1. fitted input called prediction [Sec. II-B Eq. (2); Sec. III-A Eq. (9); Sec. IV-B Table II]
    "FI= 1/σ_y^2 ∑_{k=1}^N (∂ y_k/∂ θ)^2 ... r_k = { (∂V_k/∂ θ_i)^2, normal condition; M, constraint violation } ... The first term maximizes the FI for the considered parameter ... the DRL approach achieves significantly higher average FI for k_n and k_p compared to the NMPC design."

    The per-step reward in Eq. (9) is exactly the squared voltage sensitivity (∂V_k/∂θ_i)^2 that is summed in the FI definition of Eq. (2), and the paper explicitly states that the reward's first term 'maximizes the FI for the considered parameter.' Therefore the TD3 policy is trained to maximize a discounted sum of precisely the quantities whose total is reported as average Fisher information. Reporting that the DRL profile 'achieves significantly higher average FI' restates the training objective: the policy was selected for high values of this same metric, so its FI is a property of the fitted policy rather than an independent prediction. The NMPC baseline optimizes the same integrand over only a 20 s horizon, making the DRL FI advantage largely preordained.

full rationale

The paper's central claim has two components. The first, higher Fisher information, reduces by construction: Eq. (9) defines the DRL reward as the squared voltage sensitivity, while Eq. (2) defines FI as the sum of those same squared sensitivities (scaled by 1/σ_y^2). Thus the high-FI result is the policy's own training objective, a form of fitted input called prediction. The second component, lower median parameter estimation errors (0.24% for k_p and 0.23% for k_n), is not circular: the least-squares estimator in Eq. (12) is run on synthetic voltage data generated from the same E-ECM, and no constants are fitted to force the DRL profile to win; it is a legitimate, though simulation-only, comparison. Self-citations are present but not load-bearing in a way that imports an unverified uniqueness theorem: [14] supplies the E-ECM simulation environment, [17] motivates the choice of k_p and k_n, and [19] provides a drive-cycle input; these are normal uses of prior work. The admitted lack of hardware-in-the-loop validation (Sec. V) is a model-fidelity limitation, not a circularity. The NMPC's 20 s horizon versus DRL's full 1800 s episode is an experimental-design fairness concern affecting the FI comparison, but it is an evaluation artifact rather than a definitional circularity. Because one of the two headline results reduces to the training objective, the overall circularity score is 6 (partial circularity); the independent estimation-error result prevents a higher score.

Assumptions & free parameters 4 free parameters · 5 assumptions · 0 invented entities

The central results rest on the E-ECM model and its nominal parameters from prior publications, on the sensitivity simplification in Eq. (5), and on a set of hand-chosen training and comparison hyperparameters. No new physical entities are introduced. The main uncharged inputs are model fidelity and the NMPC horizon, both of which materially affect the claimed DRL advantage.

free parameters (4)
  • NMPC prediction horizon H = 20 s
    Chosen by hand and not optimized. It limits NMPC look-ahead to 20 steps while DRL optimizes the full 1800 s episode, giving DRL a structural advantage in the comparison (Section III-B and IV-B).
  • Constraint violation penalty M = -5
    Hand-picked reward penalty in Eq. (9). It shapes how aggressively the DRL policy approaches voltage and current limits.
  • TD3 discount factor gamma = 0.99
    Hand-chosen hyperparameter in Table I. It controls how much the policy values future Fisher information rewards.
  • TD3 exploration noise amplitude = 9 A (Ornstein-Uhlenbeck)
    Hand-chosen exploration parameter in Table I. It affects policy convergence and the final excitation profile.
assumptions (5)
  • domain assumption The E-ECM terminal voltage equation (1) with the reduced-order dynamics from [14] accurately reproduces the cell's voltage response for the experimental conditions studied.
    Invoked in Section II-A and used to generate all training and validation data. No comparison against a higher-fidelity model or hardware is made, so this is a load-bearing modeling premise.
  • domain assumption Only the kinetic overpotential terms in Eq. (4) depend on the rate constants, and all other contributions cancel in the sensitivity expression of Eq. (5).
    Used to compute the FI reward. It is valid within the E-ECM's structure but is not separately verified against the full electrochemical model.
  • domain assumption Measurement noise is i.i.d. Gaussian with known variance, justifying the Fisher information formula in Eq. (2) as 1/sigma^2 times the sum of squared sensitivities.
    Stated in Section II-B. The later estimation study does not actually add noise to the synthetic data, so the noise model is only used to motivate FI, not to generate the reported errors.
  • domain assumption Single-parameter estimation with all other parameters fixed is representative of the parameter identification problem.
    Section IV-C estimates only k_p or k_n separately, avoiding joint identifiability issues that a real multi-parameter calibration would face.
  • ad hoc to paper TD3 training with up to 10,000 episodes converges to a near-optimal policy for the FI reward.
    The paper provides no convergence certificate or ablation over seeds, so the optimality of the learned policy is assumed (Section III-A and IV-B).

how reviews work

0 comments
Cite this review

Pith. "Pith review of Optimal Design of Experiment for Electrochemical Parameter Identification of Li-ion Battery via Deep Reinforcement Learning." pith.science (2026). https://pith.science/paper/EZBJUSMG

@misc{pith2026250619146,
  author       = {Pith},
  title        = {Pith review of: Optimal Design of Experiment for Electrochemical Parameter Identification of Li-ion Battery via Deep Reinforcement Learning},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/EZBJUSMG}},
  note         = {Machine review of arXiv:2506.19146}
}
read the original abstract

Accurate parameter estimation in electrochemical battery models is essential for monitoring and assessing the performance of lithium-ion batteries (LiBs). This paper presents a novel approach that combines deep reinforcement learning (DRL) with an optimal experimental design (OED) framework to identify key electrochemical parameters of LiB cell models. The proposed method utilizes the twin delayed deep deterministic policy gradient (TD3) algorithm to optimize input excitation, thereby increasing the sensitivity of the system response to electrochemical parameters. The performance of this DRL-based approach is evaluated against a nonlinear model predictive control (NMPC) method and conventional tests. Results indicate that the DRL-based method provides superior information content, reflected in higher Fisher information (FI) values and lower parameter estimation errors compared to the NMPC design and conventional test practices. Additionally, the DRL approach offers a substantial reduction in experimental time and computational resources.

Figures

Figures reproduced from arXiv: 2506.19146 by the authors.

Figure 1
Figure 1. Sensitivity (squared) of voltage output on [PITH_FULL_IMAGE:figures/full_fig_p004_1.png] view at source ↗
Figure 2
Figure 2. Comparison of DRL and NMPC based design of [PITH_FULL_IMAGE:figures/full_fig_p005_2.png] view at source ↗
Figure 4
Figure 4. Parameter estimation absolute error distribution for [PITH_FULL_IMAGE:figures/full_fig_p005_4.png] view at source ↗
Figures from the paper (1 more)
Figure 5
Figure 5. Figure 5: Parameter estimation absolute error distribution for [PITH_FULL_IMAGE:figures/full_fig_p006_5.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

19 extracted references · 17 canonical work pages

  1. [1]

    Parametrization of physics- based battery models from input–output data: A review of method- ology and current research,

    M. Andersson, M. Streb, J. Y . Ko, V . L¨ofqvist Klass, M. Klett, H. Ek- str¨om, M. Johansson, and G. Lindbergh, “Parametrization of physics- based battery models from input–output data: A review of method- ology and current research,”Journal of Power Sources, vol. 521, p. 230859, 2022

  2. [2]

    Global sensitivity methods for design of experiments in lithium-ion battery context,

    A. Pozzi, X. Xie, D. Raimondo, and R. Schenkendorf, “Global sensitivity methods for design of experiments in lithium-ion battery context,”IFAC-PapersOnLine, vol. 53, no. 2, pp. 7248–7255, 2020. 21st IFAC World Congress

  3. [3]

    Development of experimental techniques for pa- rameterization of multi-scale lithium-ion battery models,

    C.-H. Chen, F. B. Planella, K. O’regan, D. Gastol, W. D. Widanage, and E. Kendrick, “Development of experimental techniques for pa- rameterization of multi-scale lithium-ion battery models,”Journal of The Electrochemical Society, vol. 167, no. 8, p. 080534, 2020

  4. [4]

    Modeling and estimation for advanced battery management,

    X. Lin, Y . Kim, S. Mohan, J. B. Siegel, and A. G. Stefanopoulou, “Modeling and estimation for advanced battery management,”Annual Review of Control, Robotics, and Autonomous Systems, vol. 2, no. V ol- ume 2, 2019, pp. 393–426, 2019

  5. [5]

    New data optimization framework for parameter estimation under uncertainties with application to lithium-ion battery,

    Q. Lai, H. J. Ahn, Y . Kim, Y . N. Kim, and X. Lin, “New data optimization framework for parameter estimation under uncertainties with application to lithium-ion battery,”Applied Energy, vol. 295, p. 117034, 2021

  6. [6]

    Optimal ex- perimental design for parameterization of an electrochemical lithium- ion battery model,

    S. Park, D. Kato, Z. Gima, R. Klein, and S. Moura, “Optimal ex- perimental design for parameterization of an electrochemical lithium- ion battery model,”Journal of The Electrochemical Society, vol. 165, no. 7, p. A1309, 2018

  7. [7]

    Optimization of current excitation for identification of battery electrochemical parameters based on analytic sensitivity expression,

    Q. Lai, H. J. Ahn, G. Kim, W. T. Joe, and X. Lin, “Optimization of current excitation for identification of battery electrochemical parameters based on analytic sensitivity expression,” in2020 American control conference (ACC), pp. 346–351, IEEE, 2020

  8. [8]

    Input excitation optimization for estimating battery electrochemical parameters using reinforcement learning,

    R. Huang, J. Fogelquist, and X. Lin, “Input excitation optimization for estimating battery electrochemical parameters using reinforcement learning,” in2022 IEEE Vehicle Power and Propulsion Conference (VPPC), pp. 1–6, 2022

Show all 19 references
  1. [9]

    Reinforcement learning of optimal input excitation for parameter estimation with application to li-ion battery,

    R. Huang, J. Fogelquist, and X. Lin, “Reinforcement learning of optimal input excitation for parameter estimation with application to li-ion battery,”IEEE Transactions on Industrial Informatics, vol. 19, no. 11, pp. 11160–11170, 2023

  2. [10]

    Excitation optimization for estimat- ing battery health parameters using reinforcement learning considering information content and bias,

    R. Huang, M. Jones, and X. Lin, “Excitation optimization for estimat- ing battery health parameters using reinforcement learning considering information content and bias,” in2023 American Control Conference (ACC), pp. 3093–3098, IEEE, 2023

  3. [11]

    Deep reinforcement learning versus evolution strategies: A comparative survey,

    A. Y . Majid, S. Saaybi, V . Francois-Lavet, R. V . Prasad, and C. Ver- hoeven, “Deep reinforcement learning versus evolution strategies: A comparative survey,”IEEE Transactions on Neural Networks and Learning Systems, vol. 35, no. 9, pp. 11939–11957, 2024

  4. [12]

    Addressing function approxi- mation error in actor-critic methods,

    S. Fujimoto, H. Hoof, and D. Meger, “Addressing function approxi- mation error in actor-critic methods,” inInternational conference on machine learning, pp. 1587–1596, PMLR, 2018

  5. [13]

    Improving aging identifiability of lithium-ion batteries using deep reinforcement learning,

    H. Chun, K. Yoon, J. Kim, and S. Han, “Improving aging identifiability of lithium-ion batteries using deep reinforcement learning,”IEEE Transactions on Transportation Electrification, vol. 9, no. 1, pp. 995– 1007, 2023

  6. [14]

    Physics-based equivalent circuit model for lithium-ion cells via reduction and ap- proximation of electrochemical model,

    D. Seals, P. Ramesh, M. D’Arpino, and M. Canova, “Physics-based equivalent circuit model for lithium-ion cells via reduction and ap- proximation of electrochemical model,”SAE International Journal of Advances and Current Practices in Mobility, vol. 4, no. 2022-01-0701, pp. 115...

  7. [15]

    E. L. Lehmann and G. Casella,Theory of point estimation. Springer Science & Business Media, 2006

  8. [16]

    Geometry of the cramer-rao bound,

    L. L. Scharf and L. T. McWhorter, “Geometry of the cramer-rao bound,”Signal Processing, vol. 31, no. 3, pp. 301–311, 1993

  9. [17]

    Parameter identification for electrochem- ical models of lithium-ion batteries using sensitivity analysis,

    C. Dangwal and M. Canova, “Parameter identification for electrochem- ical models of lithium-ion batteries using sensitivity analysis,”ASME Letters in Dynamic Systems and Control, vol. 1, no. 4, p. 041014, 2021

  10. [18]

    Electro- thermal battery model identification for automotive applications,

    Y . Hu, S. Yurkovich, Y . Guezennec, and B. Yurkovich, “Electro- thermal battery model identification for automotive applications,” Journal of Power Sources, vol. 196, no. 1, pp. 449–457, 2011

  11. [19]

    Parameter identification for electrochemical models of lithium-ion batteries using bayesian optimization,

    J. Pi, S. F. da Silva, M. F. Ozkan, A. Gupta, and M. Canova, “Parameter identification for electrochemical models of lithium-ion batteries using bayesian optimization,”IFAC-PapersOnLine, vol. 58, no. 28, pp. 180– 185, 2024. The 4th Modeling, Estimation, and Control Conference – 2024

Pith tools

Reviewed August 15, 2026 · model on record in the stance chip above.