Pith. sign in

REVIEW 5 major objections 5 minor 119 references

Data-Driven Policy Mapping for Safe RL-based Energy Management Systems

T0 review · 5 major / 5 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read This paper claims that clustering building load profiles, LSTM forecasting, and masked Proximal Policy Optimization reduce building operating costs by up to 15 percent for some building types while keeping policies transferable to new…

desk verdict An integration of clustering, LSTM forecasting, and masked PPO that looks sensible for BEMS, but the headline transfer-to-new-buildings claim is not actually tested. read the letter →

arxiv 2506.16352 v1 pith:SKF3IJDC submitted 2025-06-19 cs.LG

Theo Zangato , Aomar Osmani , Pegah Alizadeh This is my paper
classification cs.LG
keywords ReinforcementLearningEnergyManagementSystemTime-seriesClusteringGeneralizationLSTMforecastingActionmaskingCityLearn
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper tries to establish that a building energy management system can avoid retraining a reinforcement-learning controller for every building by reusing a small library of cluster-specific policies. It claims that grouping buildings by the shape of their non-shiftable load curves produces groups within which one trained policy is near-optimal for all members. A one-week sample of a new building's load is enough to assign it to the right group, after which an LSTM-augmented, action-masked Proximal Policy Optimization agent manages its storage. On the CityLearn benchmark the authors report operating-cost reductions of up to 15% for some building types, stable carbon metrics, and continued operation under stochastic tariff changes without retraining. The authors themselves note that buildings whose profile matches no cluster, or whose behavior drifts over time, would still need new training or a reassignment mechanism.

What carries the argument

The load-profile clustering pipeline is the central object: for each building's non-shiftable load time series it computes the derivative, applies an FFT, and compares the frequency-domain series with dynamic time warping, after which Ward's hierarchical clustering groups buildings by minimizing within-cluster variance. Each cluster is represented by a reference series, and classification of a new building reduces to the minimum DTW distance between that building's short load snapshot and the reference series. Policy learning is carried by masked Proximal Policy Optimization (PPO), where the mask is rebuilt every timestep from physical bounds $\beta_1$ and $\beta_2$ on the storage device's feasible charge and discharge interval. The LSTM module augments the agent's observation with forecasts of price, solar generation, and load, giving the policy lookahead.

What would settle it

Take a set of held-out buildings and assign each a policy by one-week DTW classification, then compare each building's yearly operating cost against an oracle policy trained on that building's own full-year data; the paper's claim predicts the gap stays small across many buildings. A concrete failure mode would be a seasonal mismatch: a winter snapshot assigning a building to the wrong cluster while a summer snapshot assigns it correctly, with the wrong assignment measurably raising cost.

Watch

Extended reading notes

Core claim

On the paper's own terms, the discovery is that consumption-behaviour clustering turns RL-based building energy management from a per-building modelling problem into a per-cluster policy problem. Non-shiftable loads, transformed through derivative and Fast Fourier Transform (FFT) preprocessing and compared with dynamic time warping, separate buildings into three groups, and hierarchical clustering with Ward's linkage picks the groups. Each group receives one masked PPO policy trained across randomized members, and a new building is classified by its minimum DTW distance to cluster reference series using only one week of data. The LSTM forecast module supplies lookahead values for solar generation, price, and load, and the mask enforces charge and discharge limits derived from battery and grid physics. The claim is that the assigned policies reduce yearly operating cost by up to 15% for certain building types while keeping carbon emissions close to the no-storage baseline, and that they tolerate perturbed tariffs without retraining.

Load-bearing premise

The load-bearing premise is that grouping buildings by the shape of their non-shiftable load curves yields groups within which a single trained RL policy is near-optimal for every member, and that a one-week load snapshot is enough to place a new building in the correct group for that pre-trained policy to transfer.

Editorial extensions

If this is right

  • A BEMS operator can deploy one trained policy per consumption cluster instead of retraining a controller for each building.
  • A new building can receive a working policy after one week of load observation, avoiding long data-collection and training delays.
  • Storage policies trained under one tariff regime keep performing when prices and carbon costs are perturbed stochastically, with only the mask bounds needing adjustment.
  • Action masking confines exploration to physically feasible charging and discharging ranges, which protects equipment and speeds learning.
  • Cost reductions of up to 15% coexist with roughly stable carbon emissions, so the savings are not achieved by emitting more.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • A natural stress test is seasonal: a one-week winter snapshot may misassign a building whose summer load shape differs, which would motivate season-conditioned reference series or a longer classification window.
  • The same cluster-then-transfer recipe could apply to other building control tasks, such as electric vehicle charging or HVAC setpoints, whenever a stable non-shiftable load signature defines the grouping.
  • The reported tolerance to tariff noise suggests a stronger claim worth testing directly: masked policies may be robust to distribution shifts in reward coefficients generally, as long as the transition dynamics stay fixed.
  • Because the authors flag behavior drift as open, a concrete extension would be a drift detector on the DTW distance that triggers cluster reassignment or incremental policy adaptation.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

5 major / 5 minor

Summary. The paper proposes a Building Energy Management System (BEMS) that combines hierarchical clustering of non-shiftable load (NSL) time series, LSTM-based forecasting of relevant environmental variables, and a masked PPO reinforcement learning agent. The claimed contributions are policy generalization across buildings via cluster assignment, transfer of pre-trained policies to new buildings after only one week of data, safe action selection through domain-informed masking, and adaptation to stochastic tariff changes without retraining. The experiments are carried out in the CityLearn simulator, reporting cost reductions up to about 15% for certain building types and comparisons with SAC and a rule-based controller.

Significance. If the transfer and adaptability claims were fully supported, this would be a practically relevant contribution to scalable RL-based building energy management. The paper has clear strengths: it uses a realistic simulator (CityLearn), provides forecast accuracy numbers for the LSTM module (Table 1), and compares against a strong continuous RL baseline (SAC). However, the central transfer claim is not backed by a true held-out evaluation, several equations contain undefined quantities or ambiguities, and the reported results lack confidence intervals. These gaps currently prevent the central claim from being considered fully established.

major comments (5)
  1. [Section 6.2.3 / Table 2] The central transfer claim is not evaluated. The paper states that 'we randomized the building selection within each cluster for every episode' and reports results on 'test set buildings' in Table 2, but it never defines the train/test split nor states whether any building was excluded from clustering or from policy training. Consequently, Table 2 may simply measure in-distribution performance rather than transfer to new buildings. The one-week clustering consistency shown in Figure 12 does not establish that the assigned cluster policy is near-optimal for held-out buildings. Please add a leave-one-building-out or fully held-out evaluation and report per-building costs for the assigned policy.
  2. [Section 4.2, Eq. (27)] The reward function in Eq. (27) contains an undefined hyper-parameter α in the discharge-cost term, and the second term's parenthesization is ambiguous. Since the reward is the training objective, this prevents exact reproduction of the method. Define α and rewrite the expression unambiguously, with all terms clearly parenthesized.
  3. [Section 4.2, Eq. (24) and Section 5.4, Eqs. (44)-(45)] The action-space formula in Eq. (24) uses v_z, described only as 'the maximum feasible energy input under the current conditions' but never defined. In addition, Section 5.4 gives different bounds, β1 and β2 in Eqs. (44)-(45), that are not shown to be equivalent to Eq. (24); for example, the upper bound differs by a factor 1/η_t. Please specify which action bounds were actually used in the experiments and reconcile the two formulations, as this is load-bearing for both safety and performance claims.
  4. [Section 6.2.3, Table 2] Table 2 reports normalized costs and emissions without confidence intervals, number of random seeds, or any measure of variability. Given the stochasticity of PPO training and the large error bars visible in Table 3, single-run results are not sufficient to support the claimed cost reductions of up to 15%. Report means and standard deviations over multiple seeds and state precisely the baseline used for normalization.
  5. [Section 6.3, Table 3] Under stochastic tariffs, the reported financial costs are 1.005, 1.03, and 1.01 with ±0.1 error bars, i.e., at or slightly above the no-storage baseline. This does not demonstrate cost-effective adaptation without retraining; it only shows that costs do not diverge catastrophically. Add a proper comparison, such as a retrained agent or an upper-bound controller, or reinterpret the claim as one of bounded performance rather than cost savings.
minor comments (5)
  1. [Section 6.1] Please specify the number of buildings per cluster and how the train/test partition was created (e.g., by building, by cluster, or by time). This is needed to interpret Tables 2 and 3.
  2. [Section 6.2.1, Figure 10] The number of clusters w=3 is chosen post hoc, and the text acknowledges that the silhouette criterion suggests w=2. Include a sensitivity analysis of w on the final policy cost to show that the reported results are not an artifact of this choice.
  3. [Section 4, Eqs. (18)-(19)] There is a sign inconsistency for E_pv: Eq. (18) adds E_pv to the total consumption, while Eq. (19) subtracts it. If E_pv is generation, it should be subtracted in the total energy balance equation.
  4. [Section 5.3] The LSTM module is based on the authors' prior work [115], and the text mentions 'the last o observations' and a prediction horizon N, but the values of o, N, and other training details are not given. Please add these details for reproducibility.
  5. [Throughout] There are several typos and stylistic issues, including 'Locig' for 'Logic' (Section 3), 'sytems' for 'systems' (Section 4.2), 'prevision' for 'precision' (Figure 8 caption), and 'their are' for 'they are' (Section 1).

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the cost-reduction and transfer claims rest on external CityLearn experiments with independent baselines, and the only self-citation is a supporting LSTM module evaluated against a non-circular baseline.

full rationale

The paper's derivation chain is: cluster non-shiftable load profiles using FFT/derivative/DTW plus hierarchical clustering; train one masked-PPO policy per cluster in CityLearn; assign a new building to a cluster by DTW distance; measure cost and emissions against no-storage, SAC, random, and rule-based baselines. None of these steps defines the claimed outcome into existence. The clustering distance is computed from raw load data, not from policy performance, and the policies are trained with an external algorithm (maskable PPO) in an external simulator (CityLearn). The reported savings in Table 2 are empirical comparisons with baseline controllers, not quantities reconstructed from fitted parameters. The only self-citation is the LSTM forecasting module 'following the work of [115]' (Section 5.3), but that module is independently evaluated in Table 1 against a one-hour-lag baseline, and it is a supporting component rather than the source of the central generalization claim. The one-week transfer claim is supported only by cluster-structure stability in Figure 12 and lacks a held-out building cost evaluation; this is an evaluation gap, not a circular step. The Discussion also explicitly acknowledges that buildings outside existing clusters would require retraining. Undefined quantities in Eqs. (24) and (27) harm reproducibility but do not create a definitional equivalence between inputs and outputs. Overall, the central claims are self-contained against external benchmarks, with no load-bearing self-citation or fitted-input-renamed-as-prediction pattern.

Assumptions & free parameters 3 free parameters · 4 assumptions · 0 invented entities

The central claim rests on four assumptions: cluster-to-policy transfer validity, simulator fidelity, correctness of the reward, and correctness of the action-masking constraints. No code or data is released, and some reward parameters are unreported, so the burden on these assumptions is high.

free parameters (3)
  • Number of clusters w = 3
    Chosen via silhouette and inconsistency scores plus expert judgment; silhouette alone suggested 2 clusters, so the choice is post hoc and directly determines the number of trained policies.
  • Reward hyper-parameter ζ = not reported
    Introduced in Eq. 27 to balance charging and discharging cost terms; no value is given, so the training objective is incompletely specified.
  • Undefined parameter α in reward = undefined
    Appears in Eq. 27 as (1-α) but is never defined; either a typo or an unreported free parameter affecting the policy objective.
assumptions (4)
  • domain assumption Buildings in the same NSL cluster share similar optimal storage control policies
    The policy mapping in Section 5.2 assumes that cluster similarity in non-shiftable load implies transferability of a trained RL policy; this is never tested directly.
  • domain assumption CityLearn simulator faithfully represents real-world building energy dynamics
    All empirical results come from the CityLearn environment; real-world validity is assumed, not demonstrated.
  • ad hoc to paper The reward function in Eq. 27 correctly encodes the operating-cost minimization objective
    The reward has an undefined α and an unreported ζ, yet policies are trained to optimize it; replication requires resolving these.
  • domain assumption The action masking bounds in Eqs. 44-45 are correct and complete
    The safety claim rests on these bounds, but the formulas are ambiguous (e.g., β2 could be negative) and are not validated against the simulator's actual constraints.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Data-Driven Policy Mapping for Safe RL-based Energy Management Systems." pith.science (2026). https://pith.science/paper/SKF3IJDC

@misc{pith2026250616352,
  author       = {Pith},
  title        = {Pith review of: Data-Driven Policy Mapping for Safe RL-based Energy Management Systems},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/SKF3IJDC}},
  note         = {Machine review of arXiv:2506.16352}
}
read the original abstract

Increasing global energy demand and renewable integration complexity have placed buildings at the center of sustainable energy management. We present a three-step reinforcement learning(RL)-based Building Energy Management System (BEMS) that combines clustering, forecasting, and constrained policy learning to address scalability, adaptability, and safety challenges. First, we cluster non-shiftable load profiles to identify common consumption patterns, enabling policy generalization and transfer without retraining for each new building. Next, we integrate an LSTM based forecasting module to anticipate future states, improving the RL agents' responsiveness to dynamic conditions. Lastly, domain-informed action masking ensures safe exploration and operation, preventing harmful decisions. Evaluated on real-world data, our approach reduces operating costs by up to 15% for certain building types, maintains stable environmental performance, and quickly classifies and optimizes new buildings with limited data. It also adapts to stochastic tariff changes without retraining. Overall, this framework delivers scalable, robust, and cost-effective building energy management.

Figures

Figures reproduced from arXiv: 2506.16352 by the authors.

Figure 1
Figure 1. Example of a BEMS in a commercial/residential building [29]. method for Reinforcement Learning-based EMS agents. Finally we discuss our approach and conclude in Section 7. 2. BEMS Formulation This section provides a clear definition of a BEMS and a detailed modelling of its main com￾ponents, which are recurrent in the literature. Each component has its own characteristics and interacts in a specific way with all or … view at source ↗
Figure 2
Figure 2. Single (left) and double-diode (right) PV models representing the current produced by the cell and internal [PITH_FULL_IMAGE:figures/full_fig_p007_2.png] view at source ↗
Figure 3
Figure 3. Typical PV grid connected system [42]. 7 [PITH_FULL_IMAGE:figures/full_fig_p007_3.png] view at source ↗
Figures from the paper (12 more)
Figure 4
Figure 4. Figure 4: Representation of the wind power curve [57]. [PITH_FULL_IMAGE:figures/full_fig_p010_4.png]
Figure 5
Figure 5. Figure 5: Typical OCV-SOC curve of a Li-ion battery. [PITH_FULL_IMAGE:figures/full_fig_p012_5.png]
Figure 6
Figure 6. Figure 6: Maximum charge and discharge current for various values of di [PITH_FULL_IMAGE:figures/full_fig_p013_6.png]
Figure 7
Figure 7. Figure 7: Decentralized, centralized and hybrid EMS representations. [PITH_FULL_IMAGE:figures/full_fig_p014_7.png]
Figure 8
Figure 8. Figure 8: The overall framework of the proposed approach. The clustering module is used on historical data to extract [PITH_FULL_IMAGE:figures/full_fig_p022_8.png]
Figure 9
Figure 9. Figure 9: Citylearn environment overview [28]. 6. Experiment 6.1. Setting Our experimental evaluation is conducted using the CityLearn environment, an open-source, Gym-compatible simulation framework designed for developing intelligent agents that coordi￾nate building-level ener…
Figure 10
Figure 10. Figure 10: Cluster Evaluation: Silhouette Scores vs. Inconsistency Scores for Di [PITH_FULL_IMAGE:figures/full_fig_p031_10.png]
Figure 11
Figure 11. Figure 11: Hierarchical cluster construction dendogram. Each cluster are sequentially merged during the process maxi [PITH_FULL_IMAGE:figures/full_fig_p032_11.png]
Figure 12
Figure 12. Figure 12: w=3 group clustering of a subset of buildings using the clustering method applied on a) 1 year (m=8760) and b) 1 week (m=168) time series. Our 3 policy subtasks are defined based on those 3 different consumption patterns of non shiftable loads in kWh. 34 [PITH_FULL_I…
Figure 13
Figure 13. Figure 13: Convergence of the two algorithms compared on a given group of buildings. [PITH_FULL_IMAGE:figures/full_fig_p036_13.png]
Figure 14
Figure 14. Figure 14: Example of pricing time-series generated for evaluation. Each series represents an example of the pricing of [PITH_FULL_IMAGE:figures/full_fig_p037_14.png]
Figure 15
Figure 15. Figure 15: Agent’s behavior on an unseen building. Yellow spans represent high Time-of-Use tari [PITH_FULL_IMAGE:figures/full_fig_p038_15.png]

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

119 extracted references · 76 canonical work pages

  1. [1]

    I. E. A. (IEA), World electricity final consumption by sector, 1974-2019, Tech. rep., International Energy Agency (IEA), accessed: November 20, 2023 (2022). URLhttps://www.iea.org/data-and-statistics/charts/world- electricity-final-consumption-by-sector-1974-2019

    2022

  2. [2]

    Y . E. García Vera, R. Dufo-López, J. L. Bernal-Agustín, Energy management in micro- grids with renewable energy sources: A literature review, Applied Sciences (2019)

    2019

  3. [3]

    X. Chen, G. Qu, Y . Tang, S. Low, N. Li, Reinforcement learning for selective key appli- cations in power systems: Recent advances and future challenges, IEEE Transactions on Smart Grid (2022)

    2022

  4. [4]

    Delmastro, Buildings sectorial overview, Tech

    C. Delmastro, Buildings sectorial overview, Tech. rep., International Energy Agency (IEA), accessed: November 20, 2023 (2022). URLhttps://www.iea.org/reports/buildings

    2022

  5. [5]

    Paoli, Electric vehicles, tracking report, Tech

    L. Paoli, Electric vehicles, tracking report, Tech. rep., International Energy Agency (IEA), accessed: November 20, 2023 (2022). URLhttps://www.iea.org/reports/electric-vehicles 41

    2022

  6. [6]

    I. E. A. (IEA), Approximately 100 million households rely on rooftop solar pv by 2030, Tech. rep., International Energy Agency (IEA), accessed: November 20, 2023 (2022). URLhttps://www.iea.org/reports/approximately-100-million- households-rely-on-rooftop-solar-pv-by-2030

    2022

  7. [7]

    Heidary, M

    J. Heidary, M. Gheisarnejad, M. H. Khooban, Stability enhancement and energy man- agement of ac-dc microgrid based on active disturbance rejection control, Electric Power Systems Research 217 (2023) 109105

    2023

  8. [8]

    Chaudhary, J

    G. Chaudhary, J. J. Lamb, O. S. Burheim, B. Austbø, Review of energy storage and energy management system control strategies in microgrids, Energies 14 (16) (2021) 4929

    2021

Show all 119 references
  1. [9]

    Olatomiwa, S

    L. Olatomiwa, S. Mekhilef, M. S. Ismail, M. Moghavvemi, Energy management strate- gies in hybrid renewable energy systems: A review, Renewable and Sustainable Energy Reviews 62 (2016) 821–835

    2016

  2. [10]

    L. O. P. Vásquez, J. L. Redondo, J. D. Á. Hervás, V . M. Ramírez, J. L. Torres, Balancing co2 emissions and economic cost in a microgrid through an energy management system using mpc and multi-objective optimization, Applied Energy 347 (2023) 120998

    2023

  3. [11]

    Farrokhi, P

    E. Farrokhi, P. Safari, H. Ghoreishy, A rule-based energy management strategy with cur- rent estimation for controlling grid connected hybrid energy storage system, in: 2023 14th Power Electronics, Drive Systems, and Technologies Conference (PEDSTC), 2023, pp. 1–7.doi:10.1109/...

    2023

  4. [12]

    Rodriguez, D

    M. Rodriguez, D. Arcos-Aviles, F. Guinjoan, Simple fuzzy logic-based energy manage- ment for power exchange in isolated multi-microgrid systems: A case study in a remote community in the amazon region of ecuador, Applied Energy 357 (2024) 122522

    2024

  5. [13]

    Faria, C

    J. Faria, C. Marques, J. Pombo, S. Mariano, M. d. R. Calado, Optimal sizing of renew- able energy communities: A multiple swarms multi-objective particle swarm optimization approach, Energies 16 (21) (2023) 7227

    2023

  6. [14]

    A. H. Ganesh, B. Xu, A review of reinforcement learning based energy management sys- tems for electrified powertrains: Progress, challenge, and potential solution, Renewable and Sustainable Energy Reviews 154 (2022) 111833

    2022

  7. [15]

    Venkatasatish, C

    R. Venkatasatish, C. Dhanamjayulu, Reinforcement learning based energy management systems and hydrogen refuelling stations for fuel cell electric vehicles: An overview, In- ternational Journal of Hydrogen Energy 47 (64) (2022) 27646–27670

    2022

  8. [16]

    L. Yu, S. Qin, M. Zhang, C. Shen, T. Jiang, X. Guan, A review of deep reinforcement learning for smart building energy management, IEEE Internet Things J. 8 (15) (2021) 12046–12063

    2021

  9. [17]

    L. Yu, W. Xie, D. Xie, Y . Zou, D. Zhang, Z. Sun, L. Zhang, Y . Zhang, T. Jiang, Deep rein- forcement learning for smart home energy management, IEEE Internet of Things Journal 7 (4) (2019) 2751–2762

    2019

  10. [18]

    Lample, D

    G. Lample, D. S. Chaplot, Playing FPS games with deep reinforcement learning, in: S. Singh, S. Markovitch (Eds.), Proceedings of the Thirty-First AAAI Conference on Ar- tificial Intelligence, AAAI Press, 2017, pp. 2140–2146. 42

    2017

  11. [19]

    V . Mnih, K. Kavukcuoglu, D. Silver, A. Graves, I. Antonoglou, D. Wierstra, M. A. Riedmiller, Playing atari with deep reinforcement learning, CoRR abs/1312.5602 (2013). arXiv:1312.5602

    2013 arXiv

  12. [20]

    D. S. et al., Mastering chess and shogi by self-play with a general reinforcement learning algorithm, CoRR abs/1712.01815 (2017)

    2017 arXiv

  13. [21]

    R. S. Sutton, A. G. Barto, Reinforcement learning - an introduction, Adaptive computation and machine learning, MIT Press, 1998

    1998

  14. [22]

    Zhang, J

    J. Zhang, J. Kim, B. O’Donoghue, S. P. Boyd, Sample efficient reinforcement learning with REINFORCE, AAAI Press, 2021

    2021

  15. [23]

    Cobbe, O

    K. Cobbe, O. Klimov, C. Hesse, T. Kim, J. Schulman, Quantifying generalization in re- inforcement learning, in: Proceedings of the 36th International Conference on Machine Learning, V ol. 97 of Proceedings of Machine Learning Research, 2019, pp. 1282–1289

    2019

  16. [24]

    C. Deb, F. Zhang, J. Yang, S. E. Lee, K. W. Shah, A review on time series forecasting techniques for building energy consumption, Renewable and Sustainable Energy Reviews 74 (2017) 902–924

    2017

  17. [25]

    Manfren, P

    M. Manfren, P. A. James, L. Tronchin, Data-driven building energy modelling–an analysis of the potential for generalisation through interpretable machine learning, Renewable and Sustainable Energy Reviews 167 (2022) 112686

    2022

  18. [26]

    Hussain, P

    A. Hussain, P. Musilek, Energy management of buildings with energy storage and solar photovoltaic: A diversity in experience approach for deep reinforcement learning agents, Energy and AI 15 (2024) 100313

    2024

  19. [27]

    Beaudin, H

    M. Beaudin, H. Zareipour, Home energy management systems: A review of modelling and complexity, Renewable and Sustainable Energy Reviews (2015)

    2015

  20. [28]

    J. R. Vázquez-Canteli, J. Kämpf, G. Henze, Z. Nagy, Citylearn v1.0: An openai gym environment for demand response with deep reinforcement learning, in: Proceedings of the 6th ACM International Conference on Systems for Energy-Efficient Buildings, Cities, and Transportation, 20...

    2019

  21. [29]

    Zhang, S

    H. Zhang, S. Seal, D. Wu, B. Boulet, F. Bouffard, G. Joos, Data-driven model predictive and reinforcement learning based control for building energy management: A survey, arXiv preprint arXiv:2106.14450 (2021)

    2021 arXiv

  22. [30]

    Mariano-Hernández, L

    D. Mariano-Hernández, L. Hernández-Callejo, A. Zorita-Lamadrid, O. Duque-Pérez, F. S. García, A review of strategies for building energy management system: Model predictive control, demand side management, optimization, and fault detect & diagnosis, Journal of Building Enginee...

    2021

  23. [31]

    G. Zubi, R. Dufo-López, M. Carvalho, G. Pasaoglu, The lithium-ion battery: State of the art and future perspectives, Renewable and Sustainable Energy Reviews 89 (2018) 292–308. 43

    2018

  24. [32]

    Gonzalez, Smart grids infrastructure deep dive, Tech

    P. Gonzalez, Smart grids infrastructure deep dive, Tech. rep., International Energy Agency (IEA), accessed: November 20, 2023 (2022). URLhttps://www.iea.org/reports/smart-grids

    2022

  25. [33]

    Y . Cao, L. W. Mihardjo, T. Parikhani, Thermal performance, parametric analysis, and multi-objective optimization of a direct-expansion solar-assisted heat pump water heater using nsga-ii and decision makings, Applied Thermal Engineering 181 (2020) 115892

    2020

  26. [34]

    Z. Wei, F. Ren, B. Yue, Y . Ding, C. Zheng, B. Li, X. Zhai, R. Wang, Data-driven ap- plication on the optimization of a heat pump system for district heating load supply: A validation based on onsite test, Energy Conversion and Management 266 (2022) 115851

    2022

  27. [35]

    Sierla, H

    S. Sierla, H. Ihasalo, V . Vyatkin, A review of reinforcement learning applications to con- trol of heating, ventilation and air conditioning systems, Energies 15 (2022)

    2022

  28. [36]

    Byrne, Research summary and literature review on modelling and simulation of heat pumps for simultaneous heating and cooling for buildings, Energies 15 (10) (2022) 3529

    P. Byrne, Research summary and literature review on modelling and simulation of heat pumps for simultaneous heating and cooling for buildings, Energies 15 (10) (2022) 3529

    2022

  29. [37]

    Mohanraj, Y

    M. Mohanraj, Y . Belyayev, S. Jayaraj, A. Kaltayev, Research and developments on solar assisted compression heat pump systems–a comprehensive review (part a: Modeling and modifications), Renewable and sustainable energy reviews 83 (2018) 90–123

    2018

  30. [38]

    D. Cao, W. Hu, J. Zhao, Q. Huang, Z. Chen, F. Blaabjerg, A multi-agent deep reinforce- ment learning based voltage regulation using coordinated pv inverters, IEEE Transactions on Power Systems 35 (5) (2020) 4120–4123

    2020

  31. [39]

    Godina, E

    R. Godina, E. M. Rodrigues, E. Pouresmaeil, J. P. Catalão, Optimal residential model predictive control energy management performance with pv microgeneration, Computers & Operations Research 96 (2018) 143–156

    2018

  32. [40]

    J. F. Gaviria, G. Narváez, C. Guillen, L. F. Giraldo, M. Bressan, Machine learning in photovoltaic systems: A review, Renewable Energy 196 (2022) 298–318

    2022

  33. [41]

    Hasan, S

    M. Hasan, S. Parida, An overview of solar photovoltaic panel modeling based on analytical and experimental viewpoint, Renewable and Sustainable Energy Reviews 60 (2016) 75– 83

    2016

  34. [42]

    A. F. Abdul Kadir, T. Khatib, W. Elmenreich, Integrating photovoltaic systems in power system: Power quality impacts and optimal planning challenges, International Journal of Photoenergy 2014 (2014) 1–7

    2014

  35. [43]

    G. K. Singh, Solar power generation by pv (photovoltaic) technology: A review, Energy 53 (2013) 1–13

    2013

  36. [44]

    Hussein, I

    K. Hussein, I. Muta, T. Hoshino, M. Osakada, Maximum photovoltaic power tracking: an algorithm for rapidly changing atmospheric conditions, IEE Proceedings-Generation, Transmission and Distribution 142 (1) (1995) 59–64

    1995

  37. [45]

    Vachtsevanos, K

    G. Vachtsevanos, K. Kalaitzakis, A hybrid photovoltaic simulator for utility interactive studies, IEEE Transactions on Energy Conversion (2) (1987) 227–231. 44

    1987

  38. [46]

    C. Hua, J. Lin, C. Shen, Implementation of a dsp-controlled photovoltaic system with peak power tracking, IEEE transactions on industrial electronics 45 (1) (1998) 99–107

    1998

  39. [47]

    Altas, A

    I. Altas, A. Sharaf, A photovoltaic array simulation model for matlab-simulink gui envi- ronment, in: 2007 International Conference on Clean Electrical Power, IEEE, 2007, pp. 341–345

    2007

  40. [48]

    Bojek, Wind programs, Tech

    P. Bojek, Wind programs, Tech. rep., International Energy Agency (IEA), accessed: November 20, 2023 (2023). URLhttps://www.iea.org/energy-system/renewables/wind#programmes

    2023

  41. [49]

    Emami, P

    A. Emami, P. Noghreh, New approach on optimization in placement of wind turbines within wind farm by genetic algorithms, Renewable Energy 35 (7) (2010) 1559–1564

    2010

  42. [50]

    Fernandez-Gauna, M

    B. Fernandez-Gauna, M. Graña, J.-L. Osa-Amilibia, X. Larrucea, Actor-critic continu- ous state reinforcement learning for wind-turbine control robust optimization, Information Sciences 591 (2022) 365–380

    2022

  43. [51]

    Chehouri, R

    A. Chehouri, R. Younes, A. Ilinca, J. Perron, Review of performance optimization tech- niques applied to wind turbines, Applied Energy 142 (2015) 361–388

    2015

  44. [52]

    Petru, T

    T. Petru, T. Thiringer, Modeling of wind turbines for power system studies, IEEE transac- tions on Power Systems 17 (4) (2002) 1132–1139

    2002

  45. [53]

    Neammanee, S

    B. Neammanee, S. Sirisumrannukul, S. Chatratana, Development of a wind turbine simu- lator for wind generator testing, International Energy Journal 8 (1) (2007)

    2007

  46. [54]

    Santoso, H

    S. Santoso, H. T. Le, Fundamental time–domain wind turbine models for wind power studies, Renewable energy 32 (14) (2007) 2436–2452

    2007

  47. [55]

    J. F. Manwell, J. G. McGowan, A. L. Rogers, Wind energy explained: theory, design and application, John Wiley & Sons, 2010

    2010

  48. [56]

    Carrillo, A

    C. Carrillo, A. O. Montaño, J. Cidrás, E. Díaz-Dorado, Review of power curve modelling for wind turbines, Renewable and Sustainable Energy Reviews 21 (2013) 572–581

    2013

  49. [57]

    Sohoni, S

    V . Sohoni, S. Gupta, R. Nema, et al., A critical review on wind turbine power curve modelling techniques and their applications in wind based energy systems, Journal of Energy 2016 (2016)

    2016

  50. [58]

    Mathew, Wind energy: fundamentals, resource analysis and economics, V ol

    S. Mathew, Wind energy: fundamentals, resource analysis and economics, V ol. 1, Springer, 2006

    2006

  51. [59]

    S. Liu, C. Zhou, H. Guo, Q. Shi, T. E. Song, I. Schomer, Y . Liu, Operational optimization of a building-level integrated energy system considering additional potential benefits of energy storage, Protection and Control of Modern Power Systems 6 (2021) 1–10

    2021

  52. [60]

    Antoniadou-Plytaria, D

    K. Antoniadou-Plytaria, D. Steen, O. Carlson, M. A. F. Ghazvini, et al., Market-based energy management model of a building microgrid considering battery degradation, IEEE transactions on smart grid 12 (2) (2020) 1794–1804. 45

    2020

  53. [61]

    Y . Gao, Y . Matsunami, S. Miyata, Y . Akashi, Operational optimization for off-grid re- newable building energy system using deep reinforcement learning, Applied Energy 325 (2022) 119783

    2022

  54. [62]

    M. A. Hannan, S. Wali, P. J. Ker, M. S. Abd Rahman, M. Mansor, V . Ramachandara- murthy, K. Muttaqi, T. M. I. Mahlia, Z. Y . Dong, Battery energy-storage system: A review of technologies, optimization objectives, constraints, approaches, and outstanding issues, Journal of Ener...

    2021

  55. [63]

    K. W. E. Cheng, B. Divakar, H. Wu, K. Ding, H. F. Ho, Battery-management system (bms) and soc development for electrical vehicles, IEEE transactions on vehicular technology 60 (1) (2010) 76–88

    2010

  56. [64]

    Affanni, A

    A. Affanni, A. Bellini, G. Franceschini, P. Guglielmi, C. Tassoni, Battery choice and man- agement for new-generation electric vehicles, IEEE transactions on industrial electronics 52 (5) (2005) 1343–1349

    2005

  57. [65]

    Lavigne, J

    L. Lavigne, J. Sabatier, J. M. Francisco, F. Guillemard, A. Noury, Lithium-ion open circuit voltage (ocv) curve modelling and its ageing adjustment, Journal of Power Sources 324 (2016) 694–703

    2016

  58. [66]

    J. P. Rivera-Barrera, N. Muñoz-Galeano, H. O. Sarmiento-Maldonado, Soc estimation for lithium-ion batteries: Review and future challenges, Electronics 6 (4) (2017) 102

    2017

  59. [67]

    C. R. Lashway, O. A. Mohammed, Adaptive battery management and parameter estima- tion through physics-based modeling and experimental verification, IEEE Transactions on Transportation Electrification 2 (4) (2016) 454–464

    2016

  60. [68]

    W. Waag, C. Fleischer, D. U. Sauer, Adaptive on-line prediction of the available power of lithium-ion batteries, Journal of Power Sources 242 (2013) 548–559

    2013

  61. [69]

    Y . Han, R. Wang, Y . Dai, Thermal stratification within the water tank, Renewable and Sustainable Energy Reviews 13 (5) (2009) 1014–1026

    2009

  62. [70]

    Hasnain, Review on sustainable thermal energy storage technologies, part i: heat stor- age materials and techniques, Energy conversion and management 39 (11) (1998) 1127– 1138

    S. Hasnain, Review on sustainable thermal energy storage technologies, part i: heat stor- age materials and techniques, Energy conversion and management 39 (11) (1998) 1127– 1138

    1998

  63. [71]

    Hasnain, Review on sustainable thermal energy storage technologies, part ii: cool ther- mal storage, Energy conversion and management 39 (11) (1998) 1139–1153

    S. Hasnain, Review on sustainable thermal energy storage technologies, part ii: cool ther- mal storage, Energy conversion and management 39 (11) (1998) 1139–1153

    1998

  64. [72]

    K. J. Kircher, K. M. Zhang, Model predictive control of thermal storage for demand re- sponse, in: 2015 American control conference (ACC), IEEE, 2015, pp. 956–961

    2015

  65. [73]

    Fernández-Seara, F

    J. Fernández-Seara, F. J. Uhía, J. Sieres, Experimental analysis of a domestic electric hot water storage tank. part i: Static mode of operation, Applied Thermal Engineering 27 (1) (2007) 129–136

    2007

  66. [74]

    Fernandez-Seara, F

    J. Fernandez-Seara, F. J. Uhı, J. Sieres, et al., Experimental analysis of a domestic electric hot water storage tank. part ii: dynamic mode of operation, Applied thermal engineering 27 (1) (2007) 137–144. 46

    2007

  67. [75]

    Khavari, A

    F. Khavari, A. Badri, A. Zangeneh, Energy management in multi-microgrids considering point of common coupling constraint, International Journal of Electrical Power & Energy Systems 115 (2020) 105465

    2020

  68. [76]

    Khavari, A

    F. Khavari, A. Badri, A. Zangeneh, M. Shafiekhani, A comparison of centralized and decentralized energy-management models of multi-microgrid systems, in: 2017 Smart Grid Conference (SGC), 2017, pp. 1–6

    2017

  69. [77]

    Vuddanti, S

    S. Vuddanti, S. R. Salkuti, Review of energy management system approaches in micro- grids, Energies 14 (17) (2021) 5459

    2021

  70. [78]

    Taheri, P

    S. Taheri, P. Hosseini, A. Razban, Model predictive control of heating, ventilation, and air conditioning (hvac) systems: A state-of-the-art review, Journal of Building Engineering (2022) 105067

    2022

  71. [79]

    Teleke, M

    S. Teleke, M. E. Baran, S. Bhattacharya, A. Q. Huang, Rule-based control of battery energy storage for dispatching intermittent renewable sources, IEEE Transactions on Sus- tainable Energy 1 (3) (2010) 117–124

    2010

  72. [80]

    Salpakari, P

    J. Salpakari, P. Lund, Optimal and rule-based control strategies for energy flexibility in buildings with pv, Applied Energy 161 (2016) 425–436

    2016

  73. [81]

    Saloux, J

    E. Saloux, J. A. Candanedo, Optimal rule-based control for the management of thermal energy storage in a canadian solar district heating system, Solar Energy 207 (2020) 1191– 1201

    2020

  74. [82]

    M. S. Mahmoud, N. M. Alyazidi, M. I. Abouheaf, Adaptive intelligent techniques for microgrid control systems: A survey, International journal of electrical power & energy systems 90 (2017) 292–305

    2017

  75. [83]

    Jahedi, M

    G. Jahedi, M. Ardehali, Genetic algorithm-based fuzzy-pid control methodologies for en- hancement of energy efficiency of a dynamic energy system, Energy Conversion and Man- agement 52 (1) (2011) 725–732

    2011

  76. [84]

    A. M. Yousef, M. Ebeed, F. K. Abo-Elyousr, A. Elnozohy, M. Mohamed, S. M. Abdel- wahab, Optimization of pid controller for hybrid renewable energy system using adaptive sine cosine algorithm, International Journal of Renewable Energy Research-IJRER (2020) 670–677

    2020

  77. [85]

    R. P. Borase, D. Maghade, S. Sondkar, S. Pawar, A review of pid control, tuning methods and applications, International Journal of Dynamics and Control 9 (2021) 818–827

    2021

  78. [86]

    A. E.-D. Mady, G. Provan, C. Ryan, K. Brown, Stochastic model predictive controller for the integration of building use and temperature regulation, in: Proceedings of the AAAI conference on artificial intelligence, V ol. 25, 2011, pp. 1371–1376

    2011

  79. [87]

    C. Ju, P. Wang, L. Goel, Y . Xu, A two-layer energy management system for microgrids with hybrid energy storage considering degradation costs, IEEE Transactions on Smart Grid 9 (6) (2017) 6047–6057. 47

    2017

  80. [88]

    Aguilera-Gonzalez, I

    A. Aguilera-Gonzalez, I. Vechiu, R. H. L. Rodriguez, S. Bacha, Mpc energy management system for a grid-connected renewable energy/battery hybrid power plant, in: 2018 7th International Conference on Renewable Energy Research and Applications (ICRERA), IEEE, 2018, pp. 738–743

    2018

  81. [89]

    H.-S. Ko, J. Jatskevich, Power quality control of wind-hybrid power generation system using fuzzy-lqr controller, IEEE Transactions on energy conversion 22 (2) (2007) 516– 527

    2007

  82. [90]

    Markovic, Z

    U. Markovic, Z. Chu, P. Aristidou, G. Hug, Lqr-based adaptive virtual synchronous ma- chine for power systems with high inverter penetration, IEEE Transactions on Sustainable Energy 10 (3) (2018) 1501–1512

    2018

  83. [91]

    M. A. Abdullah, A. Q. Al-Shetwi, M. Mansor, M. Hannan, C. W. Tan, A. Yatim, Linear quadratic regulator controllers for regulation of the dc-bus voltage in a hybrid energy system: Modeling, design and experimental validation, Sustainable Energy Technologies and Assessments 50 (...

    2022

  84. [92]

    Y . Lu, S. Wang, Y . Sun, C. Yan, Optimal scheduling of buildings with energy generation and thermal energy storage under dynamic electricity pricing using mixed-integer nonlin- ear programming, Applied Energy 147 (2015) 49–58

    2015

  85. [93]

    Lauinger, P

    D. Lauinger, P. Caliandro, D. Kuhn, et al., A linear programming approach to the opti- mization of residential energy systems, Journal of Energy Storage 7 (2016) 24–37

    2016

  86. [94]

    Chen, Y .-C

    Y .-K. Chen, Y .-C. Wu, C.-C. Song, Y .-S. Chen, Design and implementation of energy management system with fuzzy control for dc microgrid systems, IEEE Transactions on power electronics 28 (4) (2012) 1563–1570

    2012

  87. [95]

    A. M. Eltamaly, H. M. Farh, Maximum power extraction from wind energy system based on fuzzy logic control, Electric Power Systems Research 97 (2013) 144–150

    2013

  88. [96]

    Khalid, N

    R. Khalid, N. Javaid, M. H. Rahim, S. Aslam, A. Sher, Fuzzy energy management con- troller and scheduler for smart homes, Sustainable Computing: Informatics and Systems 21 (2019) 103–118

    2019

  89. [97]

    M. Amer, A. Namaane, N. M’sirdi, Optimization of hybrid renewable energy systems (hres) using pso for cost reduction, Energy Procedia 42 (2013) 318–327

    2013

  90. [98]

    Sharafi, T

    M. Sharafi, T. Y . ELMekkawy, Multi-objective optimal design of hybrid renewable energy systems using pso-simulation based approach, Renewable energy 68 (2014) 67–79

    2014

  91. [99]

    M. A. Hossain, H. R. Pota, S. Squartini, A. F. Abdou, Modified pso algorithm for real- time energy management in grid-connected microgrids, Renewable Energy 136 (2019) 746–757

    2019

  92. [100]

    B. Chen, Z. Cai, M. Bergés, Gnu-rl: A precocial reinforcement learning solution for build- ing hvac control using a differentiable mpc policy, in: Proceedings of the 6th ACM inter- national conference on systems for energy-efficient buildings, cities, and transportation, 2019,...

    2019

  93. [101]

    Y . Wang, K. Velswamy, B. Huang, A long-short term memory recurrent neural network based reinforcement learning controller for office heating ventilation and air conditioning systems, Processes 5 (3) (2017) 46

    2017

  94. [102]

    Marantos, C

    C. Marantos, C. P. Lamprakos, V . Tsoutsouras, K. Siozios, D. Soudris, Towards plug&play smart thermostats inspired by reinforcement learning, in: Proceedings of the Workshop on INTelligent Embedded Systems Architectures and Applications, 2018, pp. 39–44

    2018

  95. [103]

    Zhang, A

    Z. Zhang, A. Chong, Y . Pan, C. Zhang, S. Lu, K. P. Lam, A deep reinforcement learn- ing approach to using whole building energy model for hvac optimal control, in: 2018 Building Performance Analysis Conference and SimBuild, V ol. 3, 2018, pp. 22–23

    2018

  96. [104]

    C. Wei, Z. Zhang, W. Qiao, L. Qu, Reinforcement-learning-based intelligent maximum power point tracking control for wind energy conversion systems, IEEE Transactions on Industrial Electronics 62 (10) (2015) 6360–6370

    2015

  97. [105]

    C. Wei, Z. Zhang, W. Qiao, L. Qu, An adaptive network-based reinforcement learning method for mppt control of pmsg wind energy conversion systems, IEEE Transactions on Power Electronics 31 (11) (2016) 7837–7848

    2016

  98. [106]

    R. C. Hsu, C.-T. Liu, W.-Y . Chen, H.-I. Hsieh, H.-L. Wang, et al., A reinforcement learning-based maximum power point tracking method for photovoltaic array, Interna- tional Journal of Photoenergy 2015 (2015)

    2015

  99. [107]

    J. Cao, D. Harrold, Z. Fan, T. Morstyn, D. Healey, K. Li, Deep reinforcement learning- based energy storage arbitrage with accurate lithium-ion battery degradation model, IEEE Transactions on Smart Grid 11 (5) (2020) 4513–4521

    2020

  100. [108]

    F. S. Gorostiza, F. M. Gonzalez-Longatt, Deep reinforcement learning-based controller for soc management of multi-electrical energy storage system, IEEE Transactions on Smart Grid 11 (6) (2020) 5039–5050

    2020

  101. [109]

    H. S. Qazi, N. Liu, T. Wang, Coordinated energy and reserve sharing of isolated microgrid cluster using deep reinforcement learning, in: 2020 5th Asia Conference on Power and Electrical Engineering (ACPEE), IEEE, 2020, pp. 81–86

    2020

  102. [110]

    T. A. Nakabi, P. Toivanen, Deep reinforcement learning for energy management in a mi- crogrid with flexible demand, Sustainable Energy, Grids and Networks 25 (2021) 100413

    2021

  103. [111]

    Łuczak, Hierarchical clustering of time series data with parametric derivative dynamic time warping, Expert Systems with Applications 62 (2016) 116–130

    M. Łuczak, Hierarchical clustering of time series data with parametric derivative dynamic time warping, Expert Systems with Applications 62 (2016) 116–130

    2016

  104. [112]

    I. N. Sneddon, Fourier transforms, Courier Corporation, 1995

    1995

  105. [113]

    Müller, Dynamic time warping, Information retrieval for music and motion (2007) 69–84

    M. Müller, Dynamic time warping, Information retrieval for music and motion (2007) 69–84

    2007

  106. [114]

    Senin, Dynamic time warping algorithm review, Information and Computer Science Department University of Hawaii at Manoa Honolulu, USA 855 (1-23) (2008) 40

    P. Senin, Dynamic time warping algorithm review, Information and Computer Science Department University of Hawaii at Manoa Honolulu, USA 855 (1-23) (2008) 40. 49

    2008

  107. [115]

    Zangato, A

    T. Zangato, A. Osmani, P. Alizadeh, Enhancing decision-making in energy management systems through action-independent dynamics learning, in: ECAI 2024, IOS Press, 2024, pp. 4571–4578

    2024

  108. [116]

    Huang, S

    S. Huang, S. Ontañón, A closer look at invalid action masking in policy gradient algo- rithms, CoRR abs/2006.14171 (2020)

    2020 arXiv

  109. [117]

    Schulman, F

    J. Schulman, F. Wolski, P. Dhariwal, A. Radford, O. Klimov, Proximal policy optimization algorithms, CoRR abs/1707.06347 (2017)

    2017 arXiv

  110. [118]

    Schulman, S

    J. Schulman, S. Levine, P. Abbeel, M. Jordan, P. Moritz, Trust region policy optimization, in: International conference on machine learning, PMLR, 2015, pp. 1889–1897

    2015

  111. [119]

    Haarnoja, A

    T. Haarnoja, A. Zhou, P. Abbeel, S. Levine, Soft actor-critic: Off-policy maximum en- tropy deep reinforcement learning with a stochastic actor, in: International conference on machine learning, PMLR, 2018, pp. 1861–1870. 50

    2018

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.