{"id":"df4dfb1c-db75-4243-baf1-4ce272874645","arxiv_id":"1908.05191","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":4.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":5,"one_line_summary":"A dual heuristic dynamic programming controller improves power tracking for grid-connected synchronverters compared with PI and neural network predictive control in simulation.","lead":"This paper applies a neural network control method called dual heuristic dynamic programming to regulate a grid-connected synchronverter, a virtual synchronous generator. Simulations suggest it tracks power references better than a PI controller and a neural predictive controller in both inductive and resistive grids.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The DHP claim rests on an unvalidated system model NN: Eq. (11) feeds its Jacobians into the critic target, and the model is trained only on PI-controlled data, so the reported superiority may reflect model bias rather than DHP itself.","rationale":"The reader's weakest-assumption analysis identified exactly the same load-bearing point: the pretrained system NN is unvalidated and trained on PI-controlled data, yet it provides the Jacobians in Eq. (11) that drive critic and action updates. My stress-test confirms that this is the most critical gap in the paper's argument because the entire optimality claim is mediated by these derivatives. The proposed concrete test isolates the effect of the learned model by replacing it with the exact analytic model; if the DHP advantage disappears, the claim falls, and if it persists, the model-accuracy concern is demoted. No independent code, data, or quantitative metrics are provided, so the simulation results cannot be checked as reported. I therefore maintain the reader's CONDITIONAL verdict: the idea is plausible, but the paper must supply model validation, reproducible results, and clearer training details before the claim can be accepted.","tokens_in":8018,"tokens_out":3901,"duration_ms":40919,"concrete_test":"Re-run the resistive-grid comparison of Section IV with Eq. (11)'s Jacobians computed from the exact analytic synchronverter model (swing equation and P/Q power-flow equations) instead of the trained system NN, keeping all other training details identical. If the DHP-vs-NNPC margin shrinks or the DHP tracking error changes materially, the reported superiority is an artifact of the learned model; if the margin persists, the system-model concern is not the decisive factor.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim that DHP outperforms PI and NNPC depends on the critic training signal computed in Eq. (11), which requires the partial derivatives ∂X_i(t+1)/∂X_j(t) and ∂X_i(t+1)/∂u_k(t). These derivatives are supplied by a pretrained system neural network (Section III-A) that is never validated. Section IV states that this network is trained on 10,000 samples generated while 'a PI controls the synchronverter'; no training curves, holdout errors, or checks on the DHP-visited state distribution are reported. If the DHP action network moves the system into regions not covered by PI-generated data, the system NN extrapolates, and the critic receives incorrect targets. The resulting control may minimize a corrupted cost-to-go, so the observed tracking improvement over NNPC and PI could be an artifact of model bias rather than a genuine property of DHP. Since the paper provides no quantitative comparison and no reproducibility artifacts, the load-bearing assumption of an accurate system model is unsupported. The discount-factor inconsistency in Table I (γ = 1 despite Eq. (6) requiring 0 < γ < 1) is secondary, but it reinforces that the training details are not sufficiently precise to rule out implementation artifacts.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes a dual heuristic dynamic programming (DHP) controller for a grid-connected synchronverter, a virtual-synchronous-generator-based inverter. The controller comprises three neural networks: a pretrained system model, an action network that outputs the inverter voltage magnitude, and a critic network that estimates the derivative of the cost-to-go with respect to states. The cost function includes weighted errors in active power, reactive power, and frequency. The authors compare the DHP controller with a conventional PI controller and a neural network predictive controller (NNPC) in simulations of inductive and resistive grids, and claim that a well-trained DHP performs on a more optimal trajectory than the other two controllers. Results are presented as time-domain plots.","tokens_in":8311,"tokens_out":3889,"duration_ms":36766,"significance":"If validated, the application of an adaptive-critic DHP scheme to synchronverter control would be a useful contribution, since conventional linear controllers degrade in resistive or uncertain grids and DHP offers a model-based reinforcement-learning approach that can handle nonlinearities. The comparative simulation framework against PI and NNPC is appropriate in principle. However, the central claim is not yet fully supported by the evidence: the system neural network that supplies critical derivatives is never validated, quantitative performance metrics are absent, and a key training parameter (gamma = 1 in Table I) contradicts the boundedness requirement in Eq. (6). Strengths include the clear problem motivation and the use of three baseline-comparison cases (inductive and resistive grids). The paper does not ship machine-checked proofs, reproducible code, or statistical error analysis, so the current evidence rests entirely on unquantified simulation plots.","major_comments":[{"comment":"The DHP training depends on the partial derivatives dX_i(t+1)/dX_j(t) and dX_i(t+1)/du_k(t), which are supplied by a pretrained system neural network (Section III-A). Section IV states that this network is trained on 10,000 samples generated while a PI controls the synchronverter. No training curves, holdout errors, or checks that the states visited during DHP evaluation lie within the training distribution are reported. Because these Jacobians form the critic target in Eq. (11), an inaccurate system model can corrupt the cost-to-go and make the observed tracking improvement an artifact of model bias rather than a genuine property of DHP. This is load-bearing for the central claim and must be addressed with a validation study.","section":"III-A, Eq. (11), and IV"},{"comment":"The discount factor is set to gamma = 1 in Table I, whereas Eq. (6) requires 0 < gamma < 1 to guarantee a bounded cost-to-go. With gamma = 1, the infinite-horizon sum may diverge, and the paper does not explain how the training was actually implemented (e.g., a finite horizon truncation or a modified objective). This inconsistency leaves the training procedure underspecified and makes it difficult to rule out implementation artifacts.","section":"Table I and Eq. (6)"},{"comment":"The utility function U(k) = sqrt(kP*eP^2 + kQ*eQ^2 + kf*ef^2) is not differentiable at zero error, yet the DHP critic and action updates require derivatives of U with respect to the states (see Eqs. (8)-(11)). The singular gradient at eP = eQ = ef = 0 can cause numerical issues during training, especially because the controller aims to drive tracking errors to zero. The authors should either use a smooth utility (e.g., weighted squared errors) or explain how the nondifferentiability is handled in the simulations.","section":"III-A, utility function"},{"comment":"The central claim that DHP outperforms PI and NNPC is supported only by time-domain plots, with no quantitative metrics such as RMS tracking error, settling time, overshoot, or steady-state error. The statements 'slightly better' and 'tremendously better' are not backed by numbers, and no error bars or statistical significance are reported. Please provide a quantitative comparison table for all operating points and both grid types.","section":"Section IV, Figs. 8 and 9"}],"minor_comments":[{"comment":"The two subsections in Section IV are both labeled 'A' (Inductive grid and Resistive grid); the second should be labeled 'B'.","section":"Section IV"},{"comment":"The phrase 'more optimal' is not standard English or technical usage; consider 'with better tracking performance' or 'with lower cumulative cost'.","section":"Abstract and Conclusion"},{"comment":"Typos: 'phase-luck-loop' should be 'phase-locked loop' in the introduction, and 'synchroinverter' in the conclusion should be 'synchronverter'.","section":"Introduction and Conclusion"},{"comment":"The objective function zeta = sum_k dJ(k+1)/du(k) is written without limits on the summation; please specify the summation range.","section":"Section III-B"}],"recommendation":"major_revision","confidential_remarks":"The manuscript is a simulation study with a plausible idea but the evidence is currently insufficient for a journal publication. The lack of quantitative comparison and the absence of validation for the system neural network are serious, and the gamma inconsistency needs a clear resolution. If the authors add a model-validation study, quantitative metrics, and clarify the training procedure, the paper could become acceptable."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Short version: this is a plausible, incremental application of DHP to synchronverter voltage control, and the comparison against PI and NNPC is interesting. The thing to know before you rely on any of the plots is that the system-model network that produces the derivatives for critic training is never validated, and the paper gives no numbers, only trajectories.\n\nWhat's genuinely new: the specific combination of DHP with a virtual-inertia loop, using active power as an additional input so the controller works in resistive grids, and the head-to-head with NNPC. That's a legitimate extension, not a breakthrough. The paper does a clear job of explaining why the standard P/δ, Q/V decoupling fails when line resistance is significant, and the power-flow setup is standard and usable.\n\nThe soft spots are real but not disqualifying. The biggest is the pretrained system NN used in Eq. (11). It is trained on 10,000 samples from a PI-controlled system, and there is no holdout error, no test on states DHP actually visits, no comparison with the known plant model. If that network extrapolates badly, the critic sees wrong targets and the whole DHP advantage could be an artifact. The authors state the assumption, but they don't support it. Second, every comparison is visual. Phrases like \"slightly better\" and \"tremendously better\" have no RMSE or settling-time numbers behind them. That makes it hard to judge whether the DHP edge over NNPC is real or just a specific simulation setup. Third, Table I lists γ = 1 while Eq. (6) requires 0 < γ < 1. Probably a typo, but together with the missing training details it suggests the simulations aren't described tightly enough for reproduction. The sqrt utility function also has an undefined derivative at zero, but that's minor since the errors don't seem to sit exactly at zero.\n\nNone of this makes the paper incoherent. The DHP derivation is standard, the comparison controllers are external, and the claimed improvement over PI in resistive grids is consistent with what you'd expect from a nonlinear adaptive controller. The \"more optimal\" language is too strong for a paper with no formal optimality guarantee, but the engineering claim is plausible.\n\nThis paper will be useful to people working on synchronverter control or applying adaptive critics in power electronics, especially if they want a starting point for a more rigorous study. It deserves a serious referee, but the referee should ask for validation of the system model, quantitative metrics, and corrected training details. I'd engage with it conditionally.","headline":"A plausible incremental DHP-synchronverter application whose main results rest on an unvalidated system-model network and visual-only comparisons.","tokens_in":8821,"tokens_out":2802,"would_cite":false,"duration_ms":27955,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"This paper claims that a dual heuristic dynamic programming controller drives a grid-connected synchronverter toward active and reactive power references more accurately than a PI controller or a neural network predictive controller…","keywords":["dual heuristic dynamic programming","synchronverter","virtual synchronous generator","adaptive critic design","neural network predictive control","grid-connected inverter","resistive grid","reactive power tracking"],"falsifier":"Train the system neural network on data from one line impedance and test the DHP controller on the other grid, then measure active- and reactive-power tracking error: if the DHP trajectory is no longer better than NNPC or PI, the claimed generality under unknown parameters is refuted. A hardware-in-the-loop test with a step change in line resistance mid-run would also show whether the pretrained model copes with a non-inductive condition it was not trained on.","tokens_in":7824,"feed_emoji":"⚡","tokens_out":4059,"duration_ms":39009,"temperature":0.7,"pith_summary":"This paper argues that a synchronverter, an inverter that mimics the inertia of a synchronous generator, can be controlled more accurately by a dual heuristic dynamic programming (DHP) neural controller than by a conventional PI controller or a neural network predictive controller (NNPC). The DHP design optimizes a cost function of active-power, reactive-power, and frequency errors over time using three neural networks: one to model the system, one to choose the inverter voltage, and one to estimate the gradient of the cost-to-go. The claimed payoff is that the same controller tracks power references well in both inductive and resistive grids, where the usual decoupling of active and reactive power breaks down. If true, the result offers a path to grid-connected inverters that retain the stabilizing inertia of a virtual synchronous generator without requiring exact line parameters.","feed_headline":"DHP neural controller beats PI and NNPC in synchronverter tracking","feed_subtitle":"A dual-heuristic dynamic programming controller keeps active and reactive power on target in resistive grids.","key_machinery":"The load-bearing mechanism is the dual heuristic dynamic programming architecture: a critic neural network estimates the derivative of the cost-to-go $\\lambda(t)=\\partial J(t)/\\partial X(t)$ directly, and equation (11) propagates this derivative through the system model via the chain rule, including the term $(\\partial X_i(t+1)/\\partial u_k(t))(\\partial u_k(t)/\\partial X_j(t))$ that allows the action network to be trained by backpropagation. Because the synchronverter parameters are treated as uncertain, a pretrained fully connected system neural network with two hidden layers of five nodes supplies the required derivatives. The action network, a feedforward net with two hidden layers of eight nodes, outputs the peak inverter voltage $E$; the control objective is a utility $U(k)=\\sqrt{K_P e_P^2+K_Q e_Q^2+K_f e_f^2}$, with $K_f=0$ in the reported simulations.","core_discovery":"On the paper's own terms, the central discovery is that a well-trained DHP controller produces a control trajectory more optimal than either a PI-based synchronverter or an NNPC, measured against a utility function of active-power, reactive-power, and frequency errors. In inductive grids, DHP and NNPC both track references better than PI; in resistive grids the improvement is called tremendous, especially for reactive power, and DHP is slightly better than NNPC because DHP searches the entire acceptable domain for the inverter voltage magnitude rather than choosing among ten candidate values. The controller uses active power as an additional input so the same design works in both inductive and resistive grids.","pith_inferences":["The paper does not demonstrate online adaptation: the system network is pretrained, not continuously updated, so the uncertainty handling shown here is robustness to fixed unknown parameters rather than genuine online learning; retraining the system network during operation would be a natural test of the claimed generality.","The NNPC comparison uses only ten candidate voltage choices, so the reported slight DHP edge may be tied to that implementation detail; allowing NNPC a continuous search could narrow or reverse the difference.","A testable extension the paper leaves implicit is whether DHP re-plans its voltage trajectory during grid faults, frequency excursions, or mid-run impedance steps, since the cost-to-go minimization suggests it should."],"forward_implications":["A single DHP-based synchronverter controller can replace the PI voltage-magnitude loop and operate in both inductive and resistive grids without re-tuning for line impedance.","Because the system model is learned by a neural network, the controller can in principle handle parameter uncertainty and changing operating points such as different power angles.","Resistive-grid reactive-power tracking, where the decoupling approximation $Q\\approx E(E-V)/(2X_{eq})$ fails, is the case where the neural controllers show their largest advantage over PI.","The training procedure is feasible with data generated by a PI-controlled system: 10,000 samples train the system network, followed by critic and action training over a 1000-second horizon."],"supporting_citations":[{"why":"Defines approximate dynamic programming and the adaptive critic architecture from which the DHP design is derived.","marker":"[13]"},{"why":"Introduces the virtual synchronous machine concept that the synchronverter controller emulates.","marker":"[4]"},{"why":"Establishes the relationship between virtual synchronous machines and frequency droop control used to justify inertia emulation.","marker":"[5]"},{"why":"Supplies the inductive-grid active and reactive power approximations that motivate the need for adaptive control when lines are resistive.","marker":"[11]"},{"why":"Also used for the inductive-grid power-flow approximation in the VSG model.","marker":"[12]"},{"why":"Shows dual heuristic programming applied to generator excitation, providing a precedent for DHP in power systems.","marker":"[7]"}],"fun_headline_variants":["DHP control outperforms PI and NNPC for synchronverters","Neural DHP beats PI and predictive control in grid tracking","DHP synchronverter controller wins on optimal trajectory","DHP handles resistive grids better than PI or NNPC","Adaptive critic DHP beats PI and NNPC in synchronverters"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The whole design rests on the pretrained system neural network accurately modeling the synchronverter's dynamics, including the derivatives used in equation (11), even though it is trained on data from a PI-controlled system and is never validated against the true model.","fun_headline_variants_meta":{"raw":{"variants":["DHP control outperforms PI and NNPC for synchronverters","Neural DHP beats PI and predictive control in grid tracking","DHP synchronverter controller wins on optimal trajectory","DHP handles resistive grids better than PI or NNPC","Adaptive critic DHP beats PI and NNPC in synchronverters"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000748,"raw_usage":{"total_tokens":3264,"prompt_tokens":812,"completion_tokens":2452,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":428,"completion_tokens_details":{"reasoning_tokens":2367}},"tokens_in":428,"tokens_out":2452,"duration_ms":15724,"temperature":1.0,"reasoning_tokens":2367,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-14T13:19:56.002540+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Train the system neural network on data from one line impedance and test the DHP controller on the other grid, then measure active- and reactive-power tracking error: if the DHP trajectory is no longer better than NNPC or PI, the claimed generality under unknown parameters is refuted. A hardware-in-the-loop test with a step change in line resistance mid-run would also show whether the pretrained model copes with a non-inductive condition it was not trained on.","supporting_citations":[{"cited_title":"Wind farms with HVDC delivery in inertial response and primary frequency control,","cited_arxiv_id":null,"evidence_quote":"Defines approximate dynamic programming and the adaptive critic architecture from which the DHP design is derived."},{"cited_title":"Impact of Distributed Generations With Energy Storage Devices on the Electric Grid,","cited_arxiv_id":null,"evidence_quote":"Introduces the virtual synchronous machine concept that the synchronverter controller emulates."},{"cited_title":"Virtual synchronous machine,","cited_arxiv_id":null,"evidence_quote":"Establishes the relationship between virtual synchronous machines and frequency droop control used to justify inertia emulation."},{"cited_title":"DHP Adaptive Critic based control of STATCOM in power system,","cited_arxiv_id":null,"evidence_quote":"Supplies the inductive-grid active and reactive power approximations that motivate the need for adaptive control when lines are resistive."},{"cited_title":"Novel comprehensive control framework for incorporating VSCs to smart power grids using bidirectional synchronous-VSC,","cited_arxiv_id":null,"evidence_quote":"Also used for the inductive-grid power-flow approximation in the VSG model."},{"cited_title":"A comparison of dual heuristic programming (DHP) and neural network based stochastic optimization approach on collective robotic search problem,","cited_arxiv_id":null,"evidence_quote":"Shows dual heuristic programming applied to generator excitation, providing a precedent for DHP in power systems."}],"review_version":1}