REVIEW 4 major objections 4 minor 14 references
Dual Heuristic Dynamic Programing Control of Grid-Connected Synchronverters
T0 review · 4 major / 4 minor · reviewed 2026-08-14 · deepseek-v4-flash
Pith's one-line read This paper claims that a dual heuristic dynamic programming controller drives a grid-connected synchronverter toward active and reactive power references more accurately than a PI controller or a neural network predictive controller…
desk verdict A plausible incremental DHP-synchronverter application whose main results rest on an unvalidated system-model network and visual-only comparisons. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The load-bearing mechanism is the dual heuristic dynamic programming architecture: a critic neural network estimates the derivative of the cost-to-go $\lambda(t)=\partial J(t)/\partial X(t)$ directly, and equation (11) propagates this derivative through the system model via the chain rule, including the term $(\partial X_i(t+1)/\partial u_k(t))(\partial u_k(t)/\partial X_j(t))$ that allows the action network to be trained by backpropagation. Because the synchronverter parameters are treated as uncertain, a pretrained fully connected system neural network with two hidden layers of five nodes supplies the required derivatives. The action network, a feedforward net with two hidden layers of eight nodes, outputs the peak inverter voltage $E$; the control objective is a utility $U(k)=\sqrt{K_P e_P^2+K_Q e_Q^2+K_f e_f^2}$, with $K_f=0$ in the reported simulations.
What would settle it
Train the system neural network on data from one line impedance and test the DHP controller on the other grid, then measure active- and reactive-power tracking error: if the DHP trajectory is no longer better than NNPC or PI, the claimed generality under unknown parameters is refuted. A hardware-in-the-loop test with a step change in line resistance mid-run would also show whether the pretrained model copes with a non-inductive condition it was not trained on.
Extended reading notes
Core claim
On the paper's own terms, the central discovery is that a well-trained DHP controller produces a control trajectory more optimal than either a PI-based synchronverter or an NNPC, measured against a utility function of active-power, reactive-power, and frequency errors. In inductive grids, DHP and NNPC both track references better than PI; in resistive grids the improvement is called tremendous, especially for reactive power, and DHP is slightly better than NNPC because DHP searches the entire acceptable domain for the inverter voltage magnitude rather than choosing among ten candidate values. The controller uses active power as an additional input so the same design works in both inductive and resistive grids.
Load-bearing premise
The whole design rests on the pretrained system neural network accurately modeling the synchronverter's dynamics, including the derivatives used in equation (11), even though it is trained on data from a PI-controlled system and is never validated against the true model.
Editorial extensions
If this is right
- A single DHP-based synchronverter controller can replace the PI voltage-magnitude loop and operate in both inductive and resistive grids without re-tuning for line impedance.
- Because the system model is learned by a neural network, the controller can in principle handle parameter uncertainty and changing operating points such as different power angles.
- Resistive-grid reactive-power tracking, where the decoupling approximation $Q\approx E(E-V)/(2X_{eq})$ fails, is the case where the neural controllers show their largest advantage over PI.
- The training procedure is feasible with data generated by a PI-controlled system: 10,000 samples train the system network, followed by critic and action training over a 1000-second horizon.
Reading between the lines
- The paper does not demonstrate online adaptation: the system network is pretrained, not continuously updated, so the uncertainty handling shown here is robustness to fixed unknown parameters rather than genuine online learning; retraining the system network during operation would be a natural test of the claimed generality.
- The NNPC comparison uses only ten candidate voltage choices, so the reported slight DHP edge may be tied to that implementation detail; allowing NNPC a continuous search could narrow or reverse the difference.
- A testable extension the paper leaves implicit is whether DHP re-plans its voltage trajectory during grid faults, frequency excursions, or mid-run impedance steps, since the cost-to-go minimization suggests it should.
Signed reviews
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper proposes a dual heuristic dynamic programming (DHP) controller for a grid-connected synchronverter, a virtual-synchronous-generator-based inverter. The controller comprises three neural networks: a pretrained system model, an action network that outputs the inverter voltage magnitude, and a critic network that estimates the derivative of the cost-to-go with respect to states. The cost function includes weighted errors in active power, reactive power, and frequency. The authors compare the DHP controller with a conventional PI controller and a neural network predictive controller (NNPC) in simulations of inductive and resistive grids, and claim that a well-trained DHP performs on a more optimal trajectory than the other two controllers. Results are presented as time-domain plots.
Significance. If validated, the application of an adaptive-critic DHP scheme to synchronverter control would be a useful contribution, since conventional linear controllers degrade in resistive or uncertain grids and DHP offers a model-based reinforcement-learning approach that can handle nonlinearities. The comparative simulation framework against PI and NNPC is appropriate in principle. However, the central claim is not yet fully supported by the evidence: the system neural network that supplies critical derivatives is never validated, quantitative performance metrics are absent, and a key training parameter (gamma = 1 in Table I) contradicts the boundedness requirement in Eq. (6). Strengths include the clear problem motivation and the use of three baseline-comparison cases (inductive and resistive grids). The paper does not ship machine-checked proofs, reproducible code, or statistical error analysis, so the current evidence rests entirely on unquantified simulation plots.
major comments (4)
- [III-A, Eq. (11), and IV] The DHP training depends on the partial derivatives dX_i(t+1)/dX_j(t) and dX_i(t+1)/du_k(t), which are supplied by a pretrained system neural network (Section III-A). Section IV states that this network is trained on 10,000 samples generated while a PI controls the synchronverter. No training curves, holdout errors, or checks that the states visited during DHP evaluation lie within the training distribution are reported. Because these Jacobians form the critic target in Eq. (11), an inaccurate system model can corrupt the cost-to-go and make the observed tracking improvement an artifact of model bias rather than a genuine property of DHP. This is load-bearing for the central claim and must be addressed with a validation study.
- [Table I and Eq. (6)] The discount factor is set to gamma = 1 in Table I, whereas Eq. (6) requires 0 < gamma < 1 to guarantee a bounded cost-to-go. With gamma = 1, the infinite-horizon sum may diverge, and the paper does not explain how the training was actually implemented (e.g., a finite horizon truncation or a modified objective). This inconsistency leaves the training procedure underspecified and makes it difficult to rule out implementation artifacts.
- [III-A, utility function] The utility function U(k) = sqrt(kP*eP^2 + kQ*eQ^2 + kf*ef^2) is not differentiable at zero error, yet the DHP critic and action updates require derivatives of U with respect to the states (see Eqs. (8)-(11)). The singular gradient at eP = eQ = ef = 0 can cause numerical issues during training, especially because the controller aims to drive tracking errors to zero. The authors should either use a smooth utility (e.g., weighted squared errors) or explain how the nondifferentiability is handled in the simulations.
- [Section IV, Figs. 8 and 9] The central claim that DHP outperforms PI and NNPC is supported only by time-domain plots, with no quantitative metrics such as RMS tracking error, settling time, overshoot, or steady-state error. The statements 'slightly better' and 'tremendously better' are not backed by numbers, and no error bars or statistical significance are reported. Please provide a quantitative comparison table for all operating points and both grid types.
minor comments (4)
- [Section IV] The two subsections in Section IV are both labeled 'A' (Inductive grid and Resistive grid); the second should be labeled 'B'.
- [Abstract and Conclusion] The phrase 'more optimal' is not standard English or technical usage; consider 'with better tracking performance' or 'with lower cumulative cost'.
- [Introduction and Conclusion] Typos: 'phase-luck-loop' should be 'phase-locked loop' in the introduction, and 'synchroinverter' in the conclusion should be 'synchronverter'.
- [Section III-B] The objective function zeta = sum_k dJ(k+1)/du(k) is written without limits on the summation; please specify the summation range.
Circularity Check
No circularity: DHP is trained against an explicit cost, but the PI and NNPC comparisons are external, and the system-model NN is an unvalidated assumption rather than a self-referential input.
full rationale
The paper's central comparison is between DHP, a PI controller, and an NNPC, all evaluated on the same power-tracking task. DHP is indeed trained to minimize a cost function whose terms are active-power error, reactive-power error, and frequency error, and the paper's 'more optimal' phrasing refers to that same cost. However, the PI and NNPC baselines are not constructed to minimize that exact cost in the same way (NNPC does use the same cost but with a restricted set of ten candidate controls), so the reported superiority is not forced by definition. No equation in the paper reduces the claimed result to its own inputs: the critic training uses the system-model neural network's Jacobians, the system model is trained on PI-generated plant data, and the subsequent evaluation uses the actual plant simulation. The lack of validation of the system-model neural network, the limited state coverage of PI-generated training data, and the Table I inconsistency (gamma = 1 despite Eq. (6) requiring 0 < gamma < 1) are correctness and reproducibility concerns, not circularity. The only self-citation, reference [8], appears in a general neural-network application sentence and is not load-bearing for the controller derivation. Therefore, no significant circularity is present.
Assumptions & free parameters
free parameters (5)
- Cost function weights [kP, kQ, kf] =
[1, 1, 0]
- Discount factor gamma =
1
- Sampling time T_s =
1 ms
- Neural network sizes =
System NN: 5 nodes per hidden layer; Action NN: 8 nodes per hidden layer
- NNPC candidate set size =
10
assumptions (4)
- domain assumption The synchronverter is connected to a fixed DC source so it can inject arbitrary additional power during transients.
- domain assumption The grid can be modeled as a stiff voltage source with known equivalent reactance and resistance.
- ad hoc to paper The system neural network accurately approximates the plant dynamics and provides correct derivatives for DHP training.
- domain assumption The cost function U(k) is a valid measure of controller performance for a grid-connected synchronverter.
Cite this review
Pith. "Pith review of Dual Heuristic Dynamic Programing Control of Grid-Connected Synchronverters." pith.science (2026). https://pith.science/paper/FFCE6VBS
@misc{pith2026190805191,
author = {Pith},
title = {Pith review of: Dual Heuristic Dynamic Programing Control of Grid-Connected Synchronverters},
year = {2026},
howpublished = {\url{https://pith.science/paper/FFCE6VBS}},
note = {Machine review of arXiv:1908.05191}
}
read the original abstract
In this paper a new approach to control a grid-connected synchronverter by using a dual heuristic dynamic programing (DHP) design is presented. The disadvantages of conventional synchronverter controller such as the challenges to cope with nonlinearity, uncertainties, and non-inductive grids are discussed.To deal with the aforementioned challenges a neural network based adaptive critic design is introduced to optimize the associated cost function. The characteristic of the neural networks facilitates the performance under uncertainties and unknown parameters (for example different power angles). The proposed DHP design includes three neural networks: system NN, action NN, and critic NN. The simulation results compare the performance of the proposed DHP with a traditional PI-based design and with a neural network predictive controller. It is shown a well trained DHP design performs in a trajectory, which is more optimal compared to the other two controllers.
Figures
Figures from the paper (2 more)
Reference graph
Works this paper leans on
-
[1]
Enabling Extreme Fast Charging with Energy Storage
Therefore, (2) and (3) can be written as 𝑃 ≈ 𝐸𝑉 2𝑋𝑒𝑞 𝛿 (4) 𝑄 ≈ 𝐸 2𝑋𝑒𝑞 (E − V) (5) Equation (4) and (5) clarify that in inductive grids, the active power is proportional to the power angle and the reactive power is proportional to the inverter voltage . In this case, the conventional VSG controller performance is acceptable; nonetheless, in low-voltage gri...
-
[2]
Overview of Control and Grid Synchronization for Distributed Power Generation Systems,
F. Blaabjerg, R. Teodorescu, M. Liserre, and A. Timbus, “Overview of Control and Grid Synchronization for Distributed Power Generation Systems,” IEEE Transactions on Industrial Electronics, vol. 53, no. 5, pp. 1398–1409, 2006
work page 2006
-
[3]
Phase -locked loop techniques. A survey,
G.-C. Hsieh and J. Hung, “Phase -locked loop techniques. A survey,” IEEE Transactions on Industrial Electronics, vol. 43, no. 6, pp. 609–615, 1996
work page 1996
-
[4]
Impact of Distributed Generations With Energy Storage Devices on the Electric Grid,
A. K. Srivastava, A. A. Kumar, and N. N. Schulz, “Impact of Distributed Generations With Energy Storage Devices on the Electric Grid,” IEEE Systems Journal, vol. 6, no. 1, pp. 110–117, 2012
work page 2012
-
[5]
H.-P. Beck and R. Hesse, “Virtual synchronous machine,” in Proc. 9th Int. Conf. Elect. Power Qual. Utilisation, 2007, pp. 1–6
work page 2007
-
[6]
Equivalence of virtual synchronous machines and frequency -droops for converter -based microgrids,
S. D’Arco and J. A. Suul, “Equivalence of virtual synchronous machines and frequency -droops for converter -based microgrids,” IEEE Trans. Smart Grid, vol. 5, no. 1, pp. 394–395, Jan. 2014
work page 2014
-
[7]
N. Zhang and D. Wunsch, “A comparison of dual heuristic programming (DHP) and neural network based stochastic optimization approach on collective robotic search problem,” Proceedings of the International Joint Conference on Neural Networks, 2003
work page 2003
-
[8]
Dual heuristic programming excitation neurocontrol for generators in a multimachine power system,
G. Venayagamoorthy, R. Harley, and D. Wunsch, “Dual heuristic programming excitation neurocontrol for generators in a multimachine power system,” Conference Record of the 2001 IEEE Industry Applications Conference. 36th IAS Annual Meeting (Cat. No.01CH37248)
work page 2001
Show all 14 references
-
[9]
Estimating the Rating of the Reviews Based on the Text,
M. Kavousi and S. Saadatmand, “Estimating the Rating of the Reviews Based on the Text,” Data Anal ytics and Learning Lecture Notes in Networks and Systems, pp. 257–267, 2018
2018
-
[10]
Analysis of intelligent Wide Area System-Centric Controller and Observer for power system stabilization using DHP control architecture,
S. Kamalasadan and A. Manickam, “Analysis of intelligent Wide Area System-Centric Controller and Observer for power system stabilization using DHP control architecture,” 2012 IEEE Power and Energy Society General Meeting, 2012
2012
-
[11]
DHP Adaptive Critic based control of STATCOM in power system,
M. R. Tousi and S. H. Hosseinian, “DHP Adaptive Critic based control of STATCOM in power system,” 2008 Third International Conference on Electric Utility Deregulation and Restructuring and Power Technologies, 2008
2008
-
[12]
Novel comprehensive control framework for incorporating VSCs to smart power grids using bidirectional synchronous-VSC,
M. Ashabani and Y. A. R. I. Mohamed, “Novel comprehensive control framework for incorporating VSCs to smart power grids using bidirectional synchronous-VSC,” IEEE Trans. Power Syst., vol. 29, no. 2, pp. 943–957, Mar. 2014
2014
-
[13]
Wind farms with HVDC delivery in inertial response and primary frequency control,
M. Zhixin, F. Lingling, D. Osborn, and S. Yuvarajan, “Wind farms with HVDC delivery in inertial response and primary frequency control,” IEEE Trans. Energy Convers., vol. 25, no. 4, pp. 1171–1178, Dec. 2010
2010
-
[14]
Approximate dynamic programming for real -time control and neu ral modeling,
P. Werbos, “Approximate dynamic programming for real -time control and neu ral modeling,” in Handbook of Intelligent Control: Neural, Fuzzy, and Adaptive Approaches, White and Sofge, Eds. New York: Van Nostrand Reinhold, 1992, pp. 493–525. (a) (b) (c) (d) (e) Fig. 9. DHP,PI, a...
1992
Reviewed August 14, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.