Pith. sign in

REVIEW 3 major objections 6 minor 34 references

ARM-IRL: Adaptive Resilience Metric Quantification Using Inverse Reinforcement Learning

T0 review · 3 major / 6 minor · reviewed 2026-08-10 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read The paper claims that adversarial inverse reinforcement learning can turn heuristic expert demonstrations into a state-dependent adaptive resilience metric that outperforms a fixed-reward forward RL baseline on cyber-physical restoration…

desk verdict AIRL applied to learn resilience rewards in power distribution is a plausible engineering contribution, but the 'adaptive resilience metric' label overreaches: the learned reward is never validated independently of the heuristic demonstrations that produced it. read the letter →

arxiv 2501.12362 v1 pith:BVNYLFZO submitted 2025-01-21 eess.SY cs.SY

Abhijeet Sahu , Venkatesh Venkataramanan , Richard Macwan This is my paper · ORCID
classification eess.SYcs.SY
keywords inversereinforcementlearningadaptiveresiliencemetriccyber-physicalsystemspowerdistributionnetworkreconfigurationreroutingadversarialIEEE123-bussystemimitation
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper tries to establish that the resilience of a cyber-physical power system can be captured by a single adaptive metric learned from data instead of a hand-weighted static formula. The authors model each restoration task as a Markov decision process with an unknown reward, and use adversarial inverse reinforcement learning (AIRL) to recover that reward from demonstrations produced by heuristic algorithms. They claim that the policy trained on the learned reward reaches its goal in fewer average steps than a forward reinforcement learning baseline (PPO) on three tasks: communication rerouting under denial-of-service, distribution feeder reconfiguration after outages, and combined cyber-physical critical-load restoration on the IEEE 123-bus system. If correct, this would give grid operators a way to derive response and recovery objectives from observed operator behavior without specifying weights in advance.

What carries the argument

The load-bearing mechanism is adversarial inverse reinforcement learning (AIRL), a GAN-style method that learns the reward function while training a policy. A discriminator network is trained to distinguish expert state-action pairs from generator rollouts, and the generator, a PPO policy, is trained to maximize the reward implied by the discriminator; the discriminator's output is interpreted as the resilience metric. AIRL's design choice of parameterizing the reward as a function of state rather than action is what the paper credits for producing a disentangled reward that is less tied to environment dynamics. Expert demonstrations come from two heuristic sources: Algorithm 2, which reroutes traffic around compromised routers by choosing the next hop with the lowest packet drop rate, and a spanning-tree search method that produces switching sequences for distribution network reconfiguration.

What would settle it

Run the same AIRL pipeline with demonstrations from a different expert source—for example, recorded operator decisions or a high-fidelity network emulator—and compare the learned reward surfaces and policy episode lengths. If the AIRL policy no longer beats PPO, or the reward function changes drastically with the demonstrator, then the adaptive resilience metric is an artifact of the particular heuristic rather than a general resilience objective.

Watch

Extended reading notes

Core claim

The central claim, stated on the paper's own terms, is that AIRL can learn an adaptive resilience metric—a state- and time-dependent reward function represented by a neural network—from heuristic expert demonstrations, and that the resulting policy performs better than forward RL. Across the rerouting, reconfiguration, and combined cyber-physical problems, the AIRL-trained agent reduces the average number of steps to reach the goal state compared with a PPO baseline, and it does so with fewer transition samples than GAIL. The learned reward functions can be visualized against selected state-action pairs, showing, for example, that the reward for a router action rises as the packet drop rate of a compromised router increases, and that the reconfiguration reward grows as more critical loads are restored. The paper also reports that AIRL is more robust than GAIL in the combined cyber-physical setting, where GAIL's performance deteriorates.

Load-bearing premise

The load-bearing premise is that the heuristic algorithms used to generate expert demonstrations are a faithful stand-in for a real operator's resilience objectives; if those demonstrations do not encode true resilience preferences, the learned reward is an artifact of the heuristics rather than a valid resilience metric.

Editorial extensions

If this is right

  • If the learned reward is a valid adaptive resilience metric, grid operators can obtain response policies from demonstrations without committing to a fixed weighted sum of resilience indices.
  • AIRL's sample efficiency relative to GAIL means the approach can be applied to larger feeder and communication networks where collecting expert trajectories is expensive.
  • The learned reward can be visualized as a function of state and action, offering operators a view of what the agent treats as resilient behavior rather than a black-box policy.
  • The same AIRL framework extends to other cyber-physical control problems, such as volt-var control, automatic generation control, and automatic voltage regulation, as the conclusion states.
  • The combined cyber-physical MDP shows that a single learned reward can coordinate actions across two asynchronous simulators to reach a joint goal.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • This is an extension of the disentanglement claim: if the learned reward is truly a function of state only, it should transfer to larger network topologies without retraining; the paper does not test this directly.
  • Because the 'expert' is a heuristic, the learned metric inherits the heuristic's notion of resilient behavior; a practical validation—again beyond the paper—would retrain on logged operator actions from a utility and compare the reward surfaces.
  • The comparison with PPO is not purely algorithmic: PPO is both the baseline and the generator inside AIRL, so part of the gain may come from the learned reward shaping PPO's exploration rather than from AIRL's training objective alone; this is an interpretation, not a paper claim.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 6 minor

Summary. The paper proposes ARM-IRL, an approach that uses adversarial inverse reinforcement learning (AIRL) to learn an "adaptive resilience metric" in the form of a reward function from expert demonstrations, and then uses the learned reward to train a restoration policy. The method is evaluated on three cyber-physical tasks: communication network rerouting, distribution feeder reconfiguration, and a combined cyber-physical critical-load restoration problem on an IEEE 123-bus system with a SimPy/OpenDSS co-simulation environment. The reported results show that policies trained with AIRL achieve lower average episode lengths than forward RL (PPO), GAIL, DAgger, behavioral cloning, and random baselines in these tasks. The paper also visualizes the learned reward functions as functions of selected state and action pairs.

Significance. If the central claim were established, the work would provide a useful data-driven alternative to static, hand-weighted resilience metrics for power distribution cyber-physical systems. The authors build a nontrivial co-simulation environment, provide a Code Ocean repository, and compare several imitation and inverse RL methods, which is a strength. However, the evidence supports the weaker claim that AIRL can imitate heuristic expert demonstrations and reduce episode length; it does not establish that the learned reward is a valid resilience metric, because the identification step is never tested. The paper is therefore more a demonstration of AIRL for these restoration tasks than a validated resilience-quantification method.

major comments (3)
  1. [§IV.B.5 and §VI] The central claim that the reward recovered by AIRL is an "adaptive resilience metric" is not tested. As the paper itself notes in §IV.B.5, AIRL recovers the reward only up to potential-based shaping, so many reward functions can rationalize the same expert demonstrations. All of the quantitative evidence in Figs. 9, 11, and 15 is average episode length on the same training distribution, which can be explained by imitation of the heuristic demonstrations alone. To support the metric claim, the learned reward would need to be validated externally, for example by transfer to held-out threat types, contingencies, load profiles, or by comparison against a separately defined resilience index. Without such a test, the paper supports an imitation-learning result, not a resilience-metric result.
  2. [§V.B and §VII] The expert demonstrations are generated by the authors' own heuristics — Algorithm 2 for rerouting and spanning-tree search [32] for reconfiguration — and §VII concedes that these heuristics "might not be optimal" in the combined cyber-physical environment. Since IRL assumes the demonstrations come from an optimal policy (Eq. 4), the learned reward is, by construction, an encoding of those heuristics rather than an independently grounded model of operator resilience objectives. This is a load-bearing issue for the title and abstract claims, and it is not resolved by the sentence in §VII that real operator demonstrations could be incorporated in the future.
  3. [§VI, Figs. 7, 9, 11, and 15] The empirical comparison lacks the statistical detail needed to support the reported gains. The figures plot average episode length with no variance, no number of seeds, and no confidence intervals, and the baseline label "random PPO agents" is ambiguous: it is not clear whether the PPO policy is random before training, trained with random weights, or evaluated with random exploration. Hyperparameters for the PPO generator and for the other baselines are not specified, and the sample counts ("30K transition trajectories", "150K transition samples") are used inconsistently. The claim that AIRL "is better than the expert" in §VI.C.4 appears to rest on a single scalar comparison without uncertainty quantification.
minor comments (6)
  1. [References] Reference [18] appears to be identical to reference [1]; the duplicate should be removed and the in-text citation corrected.
  2. [§V.A and Fig. 2] The component labeled "Optimization Solver/Outing" appears to contain a typo; "Outing" should likely be "Routing" or a similarly meaningful word.
  3. [§VI.A.4] The text alternates among "transition trajectories", "transition samples", and "training samples"; these terms should be used consistently so the reader can compare sample counts across methods.
  4. [Table I] The table header is confusing: the columns labeled "R1,R2" contain encoded action values rather than clear column names; please add explicit column headings and a description of the encoding.
  5. [Fig. 10] The captions for Fig. 10 describe reward surfaces but do not state which network size, training run, or training sample count produced the visualized reward function; this information is needed for reproducibility.
  6. [§III.A and Eq. (3)] In Eq. (3), the index i is not defined, and the text does not say which candidate metrics R_i are included in the linear adaptive resilience metric; please define the index and the candidate set.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the reported episode-length gains are independent empirical comparisons, and the learned reward is explicitly a fitted quantity whose heuristic dependence is acknowledged as a limitation rather than hidden as a prediction.

full rationale

The paper's derivation chain is: heuristic expert demonstrations (Alg. 2 for rerouting, spanning-tree search [32] for reconfiguration) feed AIRL, which learns a reward function; that reward is then used to train a PPO policy; the policy is evaluated by average episode length against PPO, GAIL, DAgger, behavioral cloning, and random baselines. The episode-length comparisons are genuine empirical measurements on the same task distribution and do not reduce by construction to the training demonstrations; the paper even reports learned policies outperforming the expert (Sec. VI.C.4: 'Training AIRL with 150K transition samples, the performance is better than the expert'). The 'adaptive resilience metric' is the reward network output by AIRL, so its content is necessarily tied to the heuristic demonstrations used to fit it; however, the paper presents this as a learned quantity and explicitly acknowledges in Sec. VII that the heuristic and spanning-tree demonstrations 'might not be optimal while considering the combined cyber-physical environment.' That is a stated external-validity limitation, not a circular step: no equation is defined in terms of the target claim, no fitted parameter is renamed as a prediction of the same quantity, and no load-bearing conclusion depends on an unverified self-citation. AIRL's reward-recovery background is cited to Fu et al. [30], and the same authors' prior work [24] supplies only the simulator interface. The central performance claim is therefore self-contained against the baselines used.

Assumptions & free parameters 4 free parameters · 4 assumptions · 0 invented entities

The central claim rests on the assumption that heuristic demonstrations are a good proxy for expert resilience behavior, and that the learned reward function constitutes a meaningful resilience metric. The paper provides no external validation for these assumptions, and the simulation environment is acknowledged as low fidelity.

free parameters (4)
  • Goal packet threshold N_g = 5
    Set to 5 for both DCs to define the goal state for rerouting and combined tasks; hand-chosen, not learned.
  • Physical reward constants in Eq. (1) = 20 and -1
    Hand-crafted reward for network reconfiguration: +20 when all critical loads restored, -1 per unrestored load.
  • Combined reward constants in Eq. (2) = Rc+Rp, Rp, Rc
    Hand-designed reward engineering to avoid rewarding the agent for reaching only one goal; constants are not learned.
  • Discriminator network architecture = 2 hidden layers, 32 neurons each
    Chosen by hand for AIRL; no architecture search reported.
assumptions (4)
  • domain assumption Expert demonstrations generated by heuristic algorithms (Alg. 2 for rerouting, spanning tree search [32] for reconfiguration) represent near-optimal expert behavior.
    The whole IRL framework assumes demonstrations come from an optimal policy. The paper uses its own heuristics as a proxy for human operators, which may not be optimal or may not capture true resilience preferences. Section V.B.
  • domain assumption Average episode length is a valid measure of resilience and performance for these tasks.
    The paper evaluates all methods by average number of steps to reach the goal state, implicitly equating shorter episodes with better resilience. This is a modeling choice, not independently justified. Section VI.
  • domain assumption The SimPy-based cyber network model accurately captures packet drop and delay dynamics relevant to resilience.
    The paper uses a lightweight SimPy model for speed and admits it is lower fidelity than Mininet, CORE, or NS-3. Section VII.
  • ad hoc to paper The learned reward function is a resilience metric that can generalize across states and scenarios.
    The paper labels the learned reward as an 'adaptive resilience metric' without validating it against any existing resilience quantification framework. This is central to the paper's contribution but is not independently grounded. Sections IV, V, and VII.

how reviews work

0 comments
Cite this review

Pith. "Pith review of ARM-IRL: Adaptive Resilience Metric Quantification Using Inverse Reinforcement Learning." pith.science (2026). https://pith.science/paper/BVNYLFZO

@misc{pith2026250112362,
  author       = {Pith},
  title        = {Pith review of: ARM-IRL: Adaptive Resilience Metric Quantification Using Inverse Reinforcement Learning},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/BVNYLFZO}},
  note         = {Machine review of arXiv:2501.12362}
}
read the original abstract

Resilience of safety-critical systems is gaining importance, particularly with the increasing number of cyber and physical threats. Cyber-physical threats are becoming increasingly prevalent, as digital systems are ubiquitous in critical infrastructure. The challenge with determining the resilience of cyber-physical systems is identifying a set of resilience metrics that can adapt to the changing states of the system. A static resilience metric can lead to an inaccurate estimation of system state, and can result in unintended consequences against cyber threats. In this work, we propose a data-driven method for adaptive resilience metric learning. The primary goal is to learn a single resilience metric by formulating an inverse reinforcement learning problem that learns a reward or objective from a set of control actions from an expert. It learns the structure or parameters of the reward function based on information provided by expert demonstrations. Most prior work has considered static weights or theories from fuzzy logic to formulate a single resilience metric. Instead, this work learns the resilience metric, represented as reward function, using adversarial inverse reinforcement learning, to determine the optimal policy through training the generator discriminator in parallel. We evaluate our proposed technique in scenarios such as optimal communication network rerouting, power distribution network reconfiguration, and a combined cyber-physical restoration of critical load using the IEEE 123-bus system.

Figures

Figures reproduced from arXiv: 2501.12362 by the authors.

Figure 1
Figure 1. GAN architecture for policy and reward learning. [PITH_FULL_IMAGE:figures/full_fig_p006_1.png] view at source ↗
Figure 2
Figure 2. Overall architecture of ARM-IRL. A. System Architecture The different components of the architecture are: 1. Envi￾ronment Model: The MDP model for the network reconfig￾uration using OpenDSS and the router rerouting within the SimPy-based simulator [24] is defined here. 2. RL Environ￾ment Engine: This is the interface for interacting with the OpenDSS and SimPy simulator for creating the MDP model. 3. Static Resilienc… view at source ↗
Figure 3
Figure 3. IEEE 123 test system segregated to two zones. [PITH_FULL_IMAGE:figures/full_fig_p007_3.png] view at source ↗
Figures from the paper (12 more)
Figure 4
Figure 4. Figure 4: Communication network, N6, where all the six routers are controllable and DoS attack is performed at any of the routers = R3, R4, R5 [PITH_FULL_IMAGE:figures/full_fig_p008_4.png]
Figure 5
Figure 5. Figure 5: Communication network, N8, where three out of eight routers are controllable 1) Can we reduce the episode lengths for variable-length MDPs by training a policy using expert demonstrations compared to forward RL techniques? 2) Is the approach of imitation learning throu…
Figure 6
Figure 6. Figure 6: Evaluation of the behavioral cloning for cyber network of [Left] N6 and [Right] N8 with two unique action space [PITH_FULL_IMAGE:figures/full_fig_p009_6.png]
Figure 7
Figure 7. Figure 7: Evaluation of the DAgger Algorithm for cyber network of [Left] N6 and [Right] N8 with two unique action space [PITH_FULL_IMAGE:figures/full_fig_p009_7.png]
Figure 8
Figure 8. Figure 8: Evaluation of the GAIL method for cyber network of [Left] N6 and [Right] N8 with two unique action space [PITH_FULL_IMAGE:figures/full_fig_p009_8.png]
Figure 9
Figure 9. Figure 9: Evaluation of the AIRL method for cyber network of [Left] N6 and [Right] N8 with two unique action space. B. Physical-Side Resilience Metric Learning For the power system resilience metric learning, we con￾sidered variable-length episode-based network reconfiguration p…
Figure 10
Figure 10. Figure 10: [Left] Reward as a function of the packet drop rate at the router R3 and the action taken at router R1 and R2. [Middle] [PITH_FULL_IMAGE:figures/full_fig_p010_10.png]
Figure 11
Figure 11. Figure 11: Comparison of (a) the behavior cloning technique with random and other behavioral cloning agents with training under [PITH_FULL_IMAGE:figures/full_fig_p011_11.png]
Figure 12
Figure 12. Figure 12: Evaluation of Behavioral Cloning method for the cyber-physical critical load restoration on [Left] N6 and [Right] N8 network [PITH_FULL_IMAGE:figures/full_fig_p011_12.png]
Figure 13
Figure 13. Figure 13: Evaluation of DAgger method for the cyber-physical critical load restoration on [Left] N6 and [Right] N8 network. might not be optimal while considering the combined cyber￾physical environment. b) The SimPy-based cyber environment considered is not as high fidelity in…
Figure 14
Figure 14. Figure 14: Evaluation of GAIL method for the cyber-physical critical load restoration on [Left] N6 and [Right] N8 network [PITH_FULL_IMAGE:figures/full_fig_p012_14.png]
Figure 15
Figure 15. Figure 15: Evaluation of AIRL method for the cyber-physical critical load restoration on [Left] N6 and [Right] N8 network. the article do not necessarily represent the views of the DOE or the U.S. Government. REFERENCES [1] G. Murino, A. Armando, and A. Tacchella, “Resilience of…

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

34 extracted references · 29 canonical work pages

  1. [32]

    Distribution system restoration with microgrids using spanning tree search,

    J. Li, X.-Y . Ma, C.-C. Liu, and K. P. Schneider, “Distribution system restoration with microgrids using spanning tree search,” IEEE Transac- tions on Power Systems , vol. 29, no. 6, pp. 3021–3029, 2014

    work page 2014

  2. [2]

    Power system resilience: Current practices, challenges, and future directions,

    N. Bhusal, M. Abdelmalak, M. Kamruzzaman, and M. Benidris, “Power system resilience: Current practices, challenges, and future directions,” IEEE Access, vol. 8, pp. 18 064–18 086, 2020

    2020

  3. [3]

    Enabling systems engineers and program managers to select the most useful assessment methods,

    D. J. Bordeau, R. D. Graubart, R. M. McQuaid, and J. Woodill, “Enabling systems engineers and program managers to select the most useful assessment methods,” Sept 2018

    work page 2018

  4. [4]

    Cyber resiliency metrics and scoring in practice,

    ——, “Cyber resiliency metrics and scoring in practice,” Sept 2018

    work page 2018

  5. [5]

    Resilience metrics for cyber systems,

    I. Linkov, D. A. Eisenberg, K. Plourde, T. P. Seager, J. Allen, and A. Kott, “Resilience metrics for cyber systems,” Environment Systems and Decisions, vol. 33, pp. 471–476, 2013

    work page 2013

  6. [6]

    Cp-tram: Cyber-physical transmission resiliency assessment metric,

    Tushar, V . Venkataramanan, A. Srivastava, and A. Hahn, “Cp-tram: Cyber-physical transmission resiliency assessment metric,” IEEE Trans- actions on Smart Grid , vol. 11, no. 6, pp. 5114–5123, 2020

    work page 2020

  7. [7]

    CP-SAM: Cyber- physical security assessment metric for monitoring microgrid resiliency,

    V . Venkataramanan, A. Hahn, and A. Srivastava, “CP-SAM: Cyber- physical security assessment metric for monitoring microgrid resiliency,” IEEE Transactions on Smart Grid , vol. 11, no. 2, pp. 1055–1065, 2020

    work page 2020

  8. [8]

    Scalabil- ity in Multiobjective Optimization (Dagstuhl Seminar 20031),

    C. M. Fonseca, K. Klamroth, G. Rudolph, and M. M. Wiecek, “Scalabil- ity in Multiobjective Optimization (Dagstuhl Seminar 20031),” Dagstuhl Reports, vol. 10, no. 1, pp. 52–129, 2020

    work page 2020

Show all 34 references
  1. [9]

    Reinforcement learning for feedback-enabled cyber resilience,

    Y . Huang, L. Huang, and Q. Zhu, “Reinforcement learning for feedback-enabled cyber resilience,” 2021. [Online]. Available: https://arxiv.org/abs/2107.00783

    2021 arXiv

  2. [10]

    To improve cyber resilience, measure it,

    A. Kott and I. Linkov, “To improve cyber resilience, measure it,” Computer, vol. 54, no. 2, pp. 80–85, 2021

    2021

  3. [11]

    Metrics and quantification of operational and infrastruc- ture resilience in power systems,

    M. Panteli, P. Mancarella, D. N. Trakas, E. Kyriakides, and N. D. Hatziargyriou, “Metrics and quantification of operational and infrastruc- ture resilience in power systems,” IEEE Transactions on Power Systems, vol. 32, no. 6, pp. 4732–4742, 2017

    2017

  4. [12]

    Microgrids as a resilience resource and strategies used by microgrids for enhancing resilience,

    A. Hussain, V .-H. Bui, and H.-M. Kim, “Microgrids as a resilience resource and strategies used by microgrids for enhancing resilience,” Applied Energy, vol. 240, pp. 56–72, 2019

    2019

  5. [13]

    Resilient scheduling of networked microgrids against real-time fail- ures,

    H. S. Fesagandis, M. Jalali, K. Zare, M. Abapour, and H. Karimipour, “Resilient scheduling of networked microgrids against real-time fail- ures,” IEEE Access, vol. 9, pp. 21 443–21 456, 2021

    2021

  6. [14]

    Quantitative analysis of power systems resilience: Standardization, categorizations, and challenges,

    A. Umunnakwe, H. Huang, K. Oikonomou, and K. Davis, “Quantitative analysis of power systems resilience: Standardization, categorizations, and challenges,” Renewable and Sustainable Energy Reviews , vol. 149, p. 111252, 2021

    2021

  7. [15]

    Quantifying the system-level resilience of thermal power generation to extreme temper- atures and water scarcity,

    Y . Zhou, M. Panteli, B. Wang, and P. Mancarella, “Quantifying the system-level resilience of thermal power generation to extreme temper- atures and water scarcity,” IEEE Systems Journal , vol. 14, no. 1, pp. 749–759, 2020

    2020

  8. [16]

    Optimizing dynamics of integrated food–energy–water systems under the risk of climate change,

    M. Memarzadeh, S. Moura, and A. Horvath, “Optimizing dynamics of integrated food–energy–water systems under the risk of climate change,” Environmental Research Letters, vol. 14, no. 7, p. 074010, jul

  9. [17]

    Performance- based cyber resilience metrics: An applied demonstration toward moving target defense,

    S. Hossain-McKenzie, C. Lai, A. Chavez, and E. Vugrin, “Performance- based cyber resilience metrics: An applied demonstration toward moving target defense,” in IECON 2018 - 44th Annual Conference of the IEEE Industrial Electronics Society , 2018, pp. 766–773

    2018

  10. [18]

    Resilience of cyber-physical systems: an experimental appraisal of quantitative measures,

    G. Murino, A. Armando, and A. Tacchella, “Resilience of cyber-physical systems: an experimental appraisal of quantitative measures,” in 2019 11th International Conference on Cyber Conflict , vol. 900, pp. 1–19

    2019

  11. [19]

    To- wards a resilience metric framework for cyber-physical systems,

    I. Friedberg, K. McLaughlin, P. Smith, and M. Wurzenberger, “To- wards a resilience metric framework for cyber-physical systems,” in Proceedings of the 4th International Symposium for ICS & SCADA Cyber Security Research 2016 , ser. ICS-CSR ’16. Swindon, GBR: BCS Learning & Dev...

    2016

  12. [20]

    Emotional deep learning programming controller for automatic voltage control of power systems,

    L. Yin, C. Zhang, Y . Wang, F. Gao, J. Yu, and L. Cheng, “Emotional deep learning programming controller for automatic voltage control of power systems,” IEEE Access, vol. 9, pp. 31 880–31 891, 2021

    2021

  13. [21]

    A data-driven method for fast ac optimal power flow solutions via deep reinforcement learning,

    Y . Zhou, B. Zhang, C. Xu, T. Lan, R. Diao, D. Shi, Z. Wang, and W.-J. Lee, “A data-driven method for fast ac optimal power flow solutions via deep reinforcement learning,” Journal of Modern Power Systems and Clean Energy, vol. 8, no. 6, pp. 1128–1139, 2020

    2020

  14. [22]

    Online learning and distributed control for residential demand response,

    X. Chen, Y . Li, J. Shimada, and N. Li, “Online learning and distributed control for residential demand response,” IEEE Transactions on Smart Grid, vol. 12, no. 6, pp. 4843–4853, 2021

    2021

  15. [23]

    Self-organizing map-based resilience quan- tification and resilient control of distribution systems under extreme events,

    K. Utkarsh and F. Ding, “Self-organizing map-based resilience quan- tification and resilient control of distribution systems under extreme events,” IEEE Transactions on Smart Grid , vol. 13, no. 3, pp. 1923– 1937, 2022

    1923

  16. [24]

    Reinforcement learning environment for cyber-resilient power distribution system,

    A. Sahu, V . Venkataramanan, and R. Macwan, “Reinforcement learning environment for cyber-resilient power distribution system,”IEEE Access, pp. 1–1, 2023

    2023

  17. [25]

    An irl approach for cyber-physical attack intention prediction and recovery,

    M. Elnaggar and N. Bezzo, “An irl approach for cyber-physical attack intention prediction and recovery,” in 2018 Annual American Control Conference (ACC), 2018, pp. 222–227. 13

    2018

  18. [26]

    A survey of inverse reinforcement learning,

    S. Adams, T. Cody, and P. Beling, “A survey of inverse reinforcement learning,” Artificial Intelligence Review , 02 2022

    2022

  19. [27]

    Bayesian inverse reinforcement learn- ing,

    D. Ramachandran and E. Amir, “Bayesian inverse reinforcement learn- ing,” in Proceedings of the 20th International Joint Conference on Artifical Intelligence, ser. IJCAI’07. San Francisco, CA, USA: Morgan Kaufmann Publishers Inc., 2007, p. 2586–2591

    2007

  20. [28]

    A reduction of imitation learning and structured prediction to no-regret online learning,

    S. Ross, G. J. Gordon, and J. A. Bagnell, “A reduction of imitation learning and structured prediction to no-regret online learning,” 2010. [Online]. Available: https://arxiv.org/abs/1011.0686

    2010 arXiv

  21. [29]

    Generative adversarial imitation learning,

    J. Ho and S. Ermon, “Generative adversarial imitation learning,” 2016. [Online]. Available: https://arxiv.org/abs/1606.03476

    2016 arXiv

  22. [30]

    Learning robust rewards with adversarial inverse reinforcement learning,

    J. Fu, K. Luo, and S. Levine, “Learning robust rewards with adversarial inverse reinforcement learning,” 2017. [Online]. Available: https://arxiv.org/abs/1710.11248

    2017 arXiv

  23. [31]

    ARM-IRL: Adaptive Resilience Metric Quantification Using Inverse Reinforcement Learning,

    A. Sahu, “ARM-IRL: Adaptive Resilience Metric Quantification Using Inverse Reinforcement Learning,” Apr 2024. [Online]. Available: https://codeocean.com/capsule/4520283/tree

    2024

  24. [33]

    Con- vex neural networks,

    Y . Bengio, N. Roux, P. Vincent, O. Delalleau, and P. Marcotte, “Con- vex neural networks,” in Advances in Neural Information Processing Systems, Y . Weiss, B. Sch¨olkopf, and J. Platt, Eds. MIT Press, 2005. Abhijeet Sahu received the M.S and Ph.D. degrees in electrical engine...

    2005

  25. [2011]

    He is currently a Senior Research Engineer working in cybersecurity at the National Renewable Energy Laboratory (NREL). His research interests include network security, cyber-physical modeling for intrusion detection and response, and artificial intelligence for cyber-physical...

    2015

  26. [2019]

    Available: https://doi.org/10.1088/1748-9326/ab2104

    [Online]. Available: https://doi.org/10.1088/1748-9326/ab2104

    doi

Pith tools

Reviewed August 10, 2026 · model on record in the stance chip above.