Pith. sign in

REVIEW 4 major objections 4 minor 14 references

Dual Heuristic Dynamic Programing Control of Grid-Connected Synchronverters

T0 review · 4 major / 4 minor · reviewed 2026-08-14 · deepseek-v4-flash

Pith's one-line read This paper claims that a dual heuristic dynamic programming controller drives a grid-connected synchronverter toward active and reactive power references more accurately than a PI controller or a neural network predictive controller…

desk verdict A plausible incremental DHP-synchronverter application whose main results rest on an unvalidated system-model network and visual-only comparisons. read the letter →

arxiv 1908.05191 v1 pith:FFCE6VBS submitted 2019-08-14 eess.SY cs.NEcs.SY

classification eess.SYcs.NEcs.SY
keywords dualheuristicdynamicprogrammingsynchronvertervirtualsynchronousgeneratoradaptivecriticdesignneuralnetworkpredictivecontrolgrid-connectedinverterresistivegridreactivepowertracking
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper argues that a synchronverter, an inverter that mimics the inertia of a synchronous generator, can be controlled more accurately by a dual heuristic dynamic programming (DHP) neural controller than by a conventional PI controller or a neural network predictive controller (NNPC). The DHP design optimizes a cost function of active-power, reactive-power, and frequency errors over time using three neural networks: one to model the system, one to choose the inverter voltage, and one to estimate the gradient of the cost-to-go. The claimed payoff is that the same controller tracks power references well in both inductive and resistive grids, where the usual decoupling of active and reactive power breaks down. If true, the result offers a path to grid-connected inverters that retain the stabilizing inertia of a virtual synchronous generator without requiring exact line parameters.

What carries the argument

The load-bearing mechanism is the dual heuristic dynamic programming architecture: a critic neural network estimates the derivative of the cost-to-go $\lambda(t)=\partial J(t)/\partial X(t)$ directly, and equation (11) propagates this derivative through the system model via the chain rule, including the term $(\partial X_i(t+1)/\partial u_k(t))(\partial u_k(t)/\partial X_j(t))$ that allows the action network to be trained by backpropagation. Because the synchronverter parameters are treated as uncertain, a pretrained fully connected system neural network with two hidden layers of five nodes supplies the required derivatives. The action network, a feedforward net with two hidden layers of eight nodes, outputs the peak inverter voltage $E$; the control objective is a utility $U(k)=\sqrt{K_P e_P^2+K_Q e_Q^2+K_f e_f^2}$, with $K_f=0$ in the reported simulations.

What would settle it

Train the system neural network on data from one line impedance and test the DHP controller on the other grid, then measure active- and reactive-power tracking error: if the DHP trajectory is no longer better than NNPC or PI, the claimed generality under unknown parameters is refuted. A hardware-in-the-loop test with a step change in line resistance mid-run would also show whether the pretrained model copes with a non-inductive condition it was not trained on.

Watch

Extended reading notes

Core claim

On the paper's own terms, the central discovery is that a well-trained DHP controller produces a control trajectory more optimal than either a PI-based synchronverter or an NNPC, measured against a utility function of active-power, reactive-power, and frequency errors. In inductive grids, DHP and NNPC both track references better than PI; in resistive grids the improvement is called tremendous, especially for reactive power, and DHP is slightly better than NNPC because DHP searches the entire acceptable domain for the inverter voltage magnitude rather than choosing among ten candidate values. The controller uses active power as an additional input so the same design works in both inductive and resistive grids.

Load-bearing premise

The whole design rests on the pretrained system neural network accurately modeling the synchronverter's dynamics, including the derivatives used in equation (11), even though it is trained on data from a PI-controlled system and is never validated against the true model.

Editorial extensions

If this is right

  • A single DHP-based synchronverter controller can replace the PI voltage-magnitude loop and operate in both inductive and resistive grids without re-tuning for line impedance.
  • Because the system model is learned by a neural network, the controller can in principle handle parameter uncertainty and changing operating points such as different power angles.
  • Resistive-grid reactive-power tracking, where the decoupling approximation $Q\approx E(E-V)/(2X_{eq})$ fails, is the case where the neural controllers show their largest advantage over PI.
  • The training procedure is feasible with data generated by a PI-controlled system: 10,000 samples train the system network, followed by critic and action training over a 1000-second horizon.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The paper does not demonstrate online adaptation: the system network is pretrained, not continuously updated, so the uncertainty handling shown here is robustness to fixed unknown parameters rather than genuine online learning; retraining the system network during operation would be a natural test of the claimed generality.
  • The NNPC comparison uses only ten candidate voltage choices, so the reported slight DHP edge may be tied to that implementation detail; allowing NNPC a continuous search could narrow or reverse the difference.
  • A testable extension the paper leaves implicit is whether DHP re-plans its voltage trajectory during grid faults, frequency excursions, or mid-run impedance steps, since the cost-to-go minimization suggests it should.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 4 minor

Summary. The paper proposes a dual heuristic dynamic programming (DHP) controller for a grid-connected synchronverter, a virtual-synchronous-generator-based inverter. The controller comprises three neural networks: a pretrained system model, an action network that outputs the inverter voltage magnitude, and a critic network that estimates the derivative of the cost-to-go with respect to states. The cost function includes weighted errors in active power, reactive power, and frequency. The authors compare the DHP controller with a conventional PI controller and a neural network predictive controller (NNPC) in simulations of inductive and resistive grids, and claim that a well-trained DHP performs on a more optimal trajectory than the other two controllers. Results are presented as time-domain plots.

Significance. If validated, the application of an adaptive-critic DHP scheme to synchronverter control would be a useful contribution, since conventional linear controllers degrade in resistive or uncertain grids and DHP offers a model-based reinforcement-learning approach that can handle nonlinearities. The comparative simulation framework against PI and NNPC is appropriate in principle. However, the central claim is not yet fully supported by the evidence: the system neural network that supplies critical derivatives is never validated, quantitative performance metrics are absent, and a key training parameter (gamma = 1 in Table I) contradicts the boundedness requirement in Eq. (6). Strengths include the clear problem motivation and the use of three baseline-comparison cases (inductive and resistive grids). The paper does not ship machine-checked proofs, reproducible code, or statistical error analysis, so the current evidence rests entirely on unquantified simulation plots.

major comments (4)
  1. [III-A, Eq. (11), and IV] The DHP training depends on the partial derivatives dX_i(t+1)/dX_j(t) and dX_i(t+1)/du_k(t), which are supplied by a pretrained system neural network (Section III-A). Section IV states that this network is trained on 10,000 samples generated while a PI controls the synchronverter. No training curves, holdout errors, or checks that the states visited during DHP evaluation lie within the training distribution are reported. Because these Jacobians form the critic target in Eq. (11), an inaccurate system model can corrupt the cost-to-go and make the observed tracking improvement an artifact of model bias rather than a genuine property of DHP. This is load-bearing for the central claim and must be addressed with a validation study.
  2. [Table I and Eq. (6)] The discount factor is set to gamma = 1 in Table I, whereas Eq. (6) requires 0 < gamma < 1 to guarantee a bounded cost-to-go. With gamma = 1, the infinite-horizon sum may diverge, and the paper does not explain how the training was actually implemented (e.g., a finite horizon truncation or a modified objective). This inconsistency leaves the training procedure underspecified and makes it difficult to rule out implementation artifacts.
  3. [III-A, utility function] The utility function U(k) = sqrt(kP*eP^2 + kQ*eQ^2 + kf*ef^2) is not differentiable at zero error, yet the DHP critic and action updates require derivatives of U with respect to the states (see Eqs. (8)-(11)). The singular gradient at eP = eQ = ef = 0 can cause numerical issues during training, especially because the controller aims to drive tracking errors to zero. The authors should either use a smooth utility (e.g., weighted squared errors) or explain how the nondifferentiability is handled in the simulations.
  4. [Section IV, Figs. 8 and 9] The central claim that DHP outperforms PI and NNPC is supported only by time-domain plots, with no quantitative metrics such as RMS tracking error, settling time, overshoot, or steady-state error. The statements 'slightly better' and 'tremendously better' are not backed by numbers, and no error bars or statistical significance are reported. Please provide a quantitative comparison table for all operating points and both grid types.
minor comments (4)
  1. [Section IV] The two subsections in Section IV are both labeled 'A' (Inductive grid and Resistive grid); the second should be labeled 'B'.
  2. [Abstract and Conclusion] The phrase 'more optimal' is not standard English or technical usage; consider 'with better tracking performance' or 'with lower cumulative cost'.
  3. [Introduction and Conclusion] Typos: 'phase-luck-loop' should be 'phase-locked loop' in the introduction, and 'synchroinverter' in the conclusion should be 'synchronverter'.
  4. [Section III-B] The objective function zeta = sum_k dJ(k+1)/du(k) is written without limits on the summation; please specify the summation range.

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity: DHP is trained against an explicit cost, but the PI and NNPC comparisons are external, and the system-model NN is an unvalidated assumption rather than a self-referential input.

full rationale

The paper's central comparison is between DHP, a PI controller, and an NNPC, all evaluated on the same power-tracking task. DHP is indeed trained to minimize a cost function whose terms are active-power error, reactive-power error, and frequency error, and the paper's 'more optimal' phrasing refers to that same cost. However, the PI and NNPC baselines are not constructed to minimize that exact cost in the same way (NNPC does use the same cost but with a restricted set of ten candidate controls), so the reported superiority is not forced by definition. No equation in the paper reduces the claimed result to its own inputs: the critic training uses the system-model neural network's Jacobians, the system model is trained on PI-generated plant data, and the subsequent evaluation uses the actual plant simulation. The lack of validation of the system-model neural network, the limited state coverage of PI-generated training data, and the Table I inconsistency (gamma = 1 despite Eq. (6) requiring 0 < gamma < 1) are correctness and reproducibility concerns, not circularity. The only self-citation, reference [8], appears in a general neural-network application sentence and is not load-bearing for the controller derivation. Therefore, no significant circularity is present.

Assumptions & free parameters 5 free parameters · 4 assumptions · 0 invented entities

The paper introduces no new physical entities or forces. The free parameters are the cost function weights, discount factor, sampling time, and network architecture choices. The main load-bearing assumption is that the pretrained system neural network accurately models the synchronverter, which is not validated. The fixed DC source and stiff grid assumptions limit the scope to a simplified simulation environment.

free parameters (5)
  • Cost function weights [kP, kQ, kf] = [1, 1, 0]
    Hand-chosen weights in the utility function U(k) that define the relative importance of active power error, reactive power error, and frequency error. The frequency weight is set to zero, so frequency error does not directly enter the cost.
  • Discount factor gamma = 1
    Set to 1 in Table I, which contradicts the requirement in equation (6) that 0 < gamma < 1 for a bounded infinite-horizon cost. The training horizon is finite (1000s), so the cost may still be bounded in practice.
  • Sampling time T_s = 1 ms
    Simulation step used for the discrete-time controller and neural network updates.
  • Neural network sizes = System NN: 5 nodes per hidden layer; Action NN: 8 nodes per hidden layer
    Architecture choices for the three networks (system, action, critic) that are not justified by any analysis.
  • NNPC candidate set size = 10
    The NNPC selects the best control among ten discrete voltage choices, which is an ad hoc design choice for comparison.
assumptions (4)
  • domain assumption The synchronverter is connected to a fixed DC source so it can inject arbitrary additional power during transients.
    Stated in Section II-A. This limits applicability to solar PV without storage, as noted by the authors.
  • domain assumption The grid can be modeled as a stiff voltage source with known equivalent reactance and resistance.
    The power flow equations in Section II-B assume a two-bus system with an ideal grid. Grid dynamics and impedance uncertainty are not modeled.
  • ad hoc to paper The system neural network accurately approximates the plant dynamics and provides correct derivatives for DHP training.
    The entire DHP training depends on the accuracy of the pretrained system model, but no validation results are shown.
  • domain assumption The cost function U(k) is a valid measure of controller performance for a grid-connected synchronverter.
    The weights are chosen arbitrarily, and the frequency error weight is set to zero, so the 'optimal' claim is relative to this specific cost.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Dual Heuristic Dynamic Programing Control of Grid-Connected Synchronverters." pith.science (2026). https://pith.science/paper/FFCE6VBS

@misc{pith2026190805191,
  author       = {Pith},
  title        = {Pith review of: Dual Heuristic Dynamic Programing Control of Grid-Connected Synchronverters},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/FFCE6VBS}},
  note         = {Machine review of arXiv:1908.05191}
}
read the original abstract

In this paper a new approach to control a grid-connected synchronverter by using a dual heuristic dynamic programing (DHP) design is presented. The disadvantages of conventional synchronverter controller such as the challenges to cope with nonlinearity, uncertainties, and non-inductive grids are discussed.To deal with the aforementioned challenges a neural network based adaptive critic design is introduced to optimize the associated cost function. The characteristic of the neural networks facilitates the performance under uncertainties and unknown parameters (for example different power angles). The proposed DHP design includes three neural networks: system NN, action NN, and critic NN. The simulation results compare the performance of the proposed DHP with a traditional PI-based design and with a neural network predictive controller. It is shown a well trained DHP design performs in a trajectory, which is more optimal compared to the other two controllers.

Figures

Figures reproduced from arXiv: 1908.05191 by the authors.

Figure 1
Figure 1. Conventiona KSi l Virtual Synchronous Generator (VSG) block diagram Power & Frequency meter Virtual Inertia control Reactive power controller Energy Source XF Iinv P Vinv Q δ E PWM Vref Qref Pref Vgrid Grid XL RL ωg [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 3
Figure 3. Equivalent circuit diagram of a grid connected VSG [PITH_FULL_IMAGE:figures/full_fig_p003_3.png] view at source ↗
Figure 6
Figure 6. Fully connected feedforward neural network [PITH_FULL_IMAGE:figures/full_fig_p004_6.png] view at source ↗
Figures from the paper (2 more)
Figure 7
Figure 7. Figure 7: DHP-based synchronverter block diagram Power meter Energy Source XF Iinv Vinv P Q ω i Pref Grid XL RL Virtual Inertia control Network input generator Qref δ PWM Action NN DHP Controller Model NN E Critic NN Critic NN Utility X(t+Ts) λ (t) λ (t+Ts) X(t) [PITH_FULL_IMAG…
Figure 8
Figure 8. Figure 8: DHP,PI, and NNPC designs performance for a VSG connected to [PITH_FULL_IMAGE:figures/full_fig_p005_8.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

14 extracted references · 14 canonical work pages

  1. [1]

    Enabling Extreme Fast Charging with Energy Storage

    Therefore, (2) and (3) can be written as 𝑃 ≈ 𝐸𝑉 2𝑋𝑒𝑞 𝛿 (4) 𝑄 ≈ 𝐸 2𝑋𝑒𝑞 (E − V) (5) Equation (4) and (5) clarify that in inductive grids, the active power is proportional to the power angle and the reactive power is proportional to the inverter voltage . In this case, the conventional VSG controller performance is acceptable; nonetheless, in low-voltage gri...

  2. [2]

    Overview of Control and Grid Synchronization for Distributed Power Generation Systems,

    F. Blaabjerg, R. Teodorescu, M. Liserre, and A. Timbus, “Overview of Control and Grid Synchronization for Distributed Power Generation Systems,” IEEE Transactions on Industrial Electronics, vol. 53, no. 5, pp. 1398–1409, 2006

  3. [3]

    Phase -locked loop techniques. A survey,

    G.-C. Hsieh and J. Hung, “Phase -locked loop techniques. A survey,” IEEE Transactions on Industrial Electronics, vol. 43, no. 6, pp. 609–615, 1996

  4. [4]

    Impact of Distributed Generations With Energy Storage Devices on the Electric Grid,

    A. K. Srivastava, A. A. Kumar, and N. N. Schulz, “Impact of Distributed Generations With Energy Storage Devices on the Electric Grid,” IEEE Systems Journal, vol. 6, no. 1, pp. 110–117, 2012

  5. [5]

    Virtual synchronous machine,

    H.-P. Beck and R. Hesse, “Virtual synchronous machine,” in Proc. 9th Int. Conf. Elect. Power Qual. Utilisation, 2007, pp. 1–6

  6. [6]

    Equivalence of virtual synchronous machines and frequency -droops for converter -based microgrids,

    S. D’Arco and J. A. Suul, “Equivalence of virtual synchronous machines and frequency -droops for converter -based microgrids,” IEEE Trans. Smart Grid, vol. 5, no. 1, pp. 394–395, Jan. 2014

  7. [7]

    A comparison of dual heuristic programming (DHP) and neural network based stochastic optimization approach on collective robotic search problem,

    N. Zhang and D. Wunsch, “A comparison of dual heuristic programming (DHP) and neural network based stochastic optimization approach on collective robotic search problem,” Proceedings of the International Joint Conference on Neural Networks, 2003

  8. [8]

    Dual heuristic programming excitation neurocontrol for generators in a multimachine power system,

    G. Venayagamoorthy, R. Harley, and D. Wunsch, “Dual heuristic programming excitation neurocontrol for generators in a multimachine power system,” Conference Record of the 2001 IEEE Industry Applications Conference. 36th IAS Annual Meeting (Cat. No.01CH37248)

Show all 14 references
  1. [9]

    Estimating the Rating of the Reviews Based on the Text,

    M. Kavousi and S. Saadatmand, “Estimating the Rating of the Reviews Based on the Text,” Data Anal ytics and Learning Lecture Notes in Networks and Systems, pp. 257–267, 2018

  2. [10]

    Analysis of intelligent Wide Area System-Centric Controller and Observer for power system stabilization using DHP control architecture,

    S. Kamalasadan and A. Manickam, “Analysis of intelligent Wide Area System-Centric Controller and Observer for power system stabilization using DHP control architecture,” 2012 IEEE Power and Energy Society General Meeting, 2012

  3. [11]

    DHP Adaptive Critic based control of STATCOM in power system,

    M. R. Tousi and S. H. Hosseinian, “DHP Adaptive Critic based control of STATCOM in power system,” 2008 Third International Conference on Electric Utility Deregulation and Restructuring and Power Technologies, 2008

  4. [12]

    Novel comprehensive control framework for incorporating VSCs to smart power grids using bidirectional synchronous-VSC,

    M. Ashabani and Y. A. R. I. Mohamed, “Novel comprehensive control framework for incorporating VSCs to smart power grids using bidirectional synchronous-VSC,” IEEE Trans. Power Syst., vol. 29, no. 2, pp. 943–957, Mar. 2014

  5. [13]

    Wind farms with HVDC delivery in inertial response and primary frequency control,

    M. Zhixin, F. Lingling, D. Osborn, and S. Yuvarajan, “Wind farms with HVDC delivery in inertial response and primary frequency control,” IEEE Trans. Energy Convers., vol. 25, no. 4, pp. 1171–1178, Dec. 2010

  6. [14]

    Approximate dynamic programming for real -time control and neu ral modeling,

    P. Werbos, “Approximate dynamic programming for real -time control and neu ral modeling,” in Handbook of Intelligent Control: Neural, Fuzzy, and Adaptive Approaches, White and Sofge, Eds. New York: Van Nostrand Reinhold, 1992, pp. 493–525. (a) (b) (c) (d) (e) Fig. 9. DHP,PI, a...

Pith tools

Reviewed August 14, 2026 · model on record in the stance chip above.