Learning to Modulate, Not to Cycle: Soft Actor---Critic Recovers Inverter-Style Heat-Pump Control

Aniket Dixit; Faizan Ahmed; James Brusey

REVIEW 3 major objections 5 minor 15 references

Learning to Modulate, Not to Cycle: Soft Actor---Critic Recovers Inverter-Style Heat-Pump Control

T0 review · 3 major / 5 minor · reviewed 2026-08-11 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read Adding a per-start compressor-wear penalty to the control reward lets Soft Actor–Critic learn a continuously modulating, zero-cycling heat-pump policy, whereas PPO on the same reward collapses to bang–bang control that cycles more than…

desk verdict A real SAC/PPO smoothness contrast with a zero-starts claim that only holds under a physically permissive duty threshold; worth reviewing with revisions. read the letter →

arxiv 2608.09453 v1 pith:V2HBSGUW submitted 2026-08-10 cs.LG cs.AI

Faizan Ahmed , Aniket Dixit , James Brusey This is my paper · ORCID

classification cs.LGcs.AI

keywords heatpumpscompressorcyclingreinforcementlearningSoftActor-CriticPPOBOPTESTinverterpumprewarddesign

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper asks whether reinforcement-learning controllers for heat pumps can be made to care about compressor wear, not just energy and comfort. It adds a levelised euro cost per compressor start to the reward and trains two algorithms, SAC and PPO, on the identical Markov decision process for the BOPTEST bestest hydronic heat-pump case. The central finding is an algorithm-class effect: SAC discovers continuous modulation that keeps the compressor permanently engaged, achieving zero starts per day in inverter-style operation, while PPO collapses to bang–bang switching that cycles more than the baseline. On the BOPTEST emulator, the SAC policy cuts thermal discomfort by up to 90.7% for an 11.5% cost increase while eliminating all baseline cycling. A sympathetic reader should care because compressor start-ups are the main driver of mechanical wear, and the result suggests that the choice of RL algorithm, not just the reward, determines whether learned control is hardware-friendly.

What carries the argument

The load-bearing object is the levelised compressor-wear reward term $J^{\text{wear}}_t = c_{\text{cyc}} \cdot \max(d_t - d_{t-1}, 0)$, where $d_t = \mathbb{1}[u_{HP,t} > \epsilon]$ is the binary duty signal with $\epsilon = 10^{-3}$ and $c_{\text{cyc}} = 0.0133$ euros per start, obtained by amortizing a representative compressor replacement (about 2000 euros) over its rated 150,000 starts, plus a short-cycle surcharge for starts separated by fewer than four steps. This term monetizes switching in the same currency as electricity cost, making the number of starts a first-class objective and creating the two zero-cost optima of never-on or always-on. The algorithm choice then decides which optimum is reached: SAC's maximum-entropy stochastic policy and replay buffer favour interior actions, while PPO's on-policy clipped updates favour the saturated rails.

What would settle it

Re-run the SAC evaluation on the BOPTEST emulator and count off→on transitions using duty thresholds of 0.10 and 0.20 instead of 0.001; if any rising edges appear, the zero-starts-per-day claim is false. Alternatively, record the heat pump's measured power trace and identify every start as a rise from below minimum modulation to above it.

Watch

Extended reading notes

Core claim

With the environment, reward, and observations held fixed, the qualitative form of the learned heat-pump controller is determined by the RL algorithm. The wear reward has two zero-cost optima: never run the compressor, or run it continuously so no rising edge ever occurs; the discomfort term rules out the first. Soft Actor–Critic's maximum-entropy, off-policy objective keeps probability mass inside the action interval and lands in the always-on optimum, producing a command signal that stays in [0.007, 0.93] on the peak day and [0.001, 0.85] on the typical day, with 0.0% of steps below the duty threshold. PPO, by contrast, spends 60.6% (peak) and 82.3% (typical) of steps fully off and saturates at the upper rail, generating 4.29 and 2.64 starts per day, more than the baseline's 1.07 and 1.50. The result is that SAC's policy is a learned inverter controller, while PPO's is a learned thermostat.

Load-bearing premise

The zero-start claim depends on defining 'off' as any command below 0.001; if the threshold were set at a real inverter's 10–20% minimum stable modulation, the SAC policy might show start-ups.

Editorial extensions

If this is right

Pricing each start endogenously produces inverter-style smoothness without hand-tuned timers or hysteresis, with the wear weight as a single knob along the smoothness–cost frontier.
The SAC policy's 11.5% cost increase buys extra delivered heat, not tariff mistiming, since source energy rises by almost the same fraction (11.0%).
Over the 14-day peak scenario, the 15 avoided baseline starts are worth 0.20 euros of levelised wear, exceeding the 0.10 euro operating premium, so the comfort-and-smoothness improvement roughly pays for itself.
PPO's lower cost and emissions are an artefact of under-delivery: it leaves the zone cold and cycles heavily, so cost-only metrics would misjudge it.
Sweeping the wear weight should trace a switches-versus-(cost+discomfort) Pareto frontier, benchmarking the smoothness–cost trade-off explicitly.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

If the duty threshold were raised to a real inverter's minimum stable modulation (10–20% of rated capacity), SAC's 'always on' claim could weaken; a command of 0.007 counts as on only under the paper's permissive $\epsilon = 10^{-3}$.
The algorithm-class effect likely generalises beyond heat pumps: entropy-regularised off-policy methods may be structurally aligned with any hardware that penalises switching, such as valves, refrigeration compressors, or battery cycling, a hypothesis the paper does not test.
A direct head-to-head against an MPC baseline with the same wear-aware objective would clarify whether the learned smoothness is competitive with an engineered controller; the paper lists this as future work.

Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

Key takeaway: real finding, soft headline. The paper shows SAC and PPO produce opposite qualitative control under the same MDP: SAC stays near a low positive capacity and avoids off/on transitions, while PPO slams the rails. That is a genuinely useful result for building-control RL, and the authors did the right thing by evaluating on the external BOPTEST emulator after training on a surrogate. The writing is clear, the reward is economically interpretable, and the Limitations section is honest.

The problem is the zero-starts claim. Zero starts/day is defined with duty threshold epsilon=1e-3, so commands at 0.7-0.1% capacity count as on. Real inverter compressors have minimum stable modulation around 10-20%. On the typical day the SAC command's mean is 0.098 with values down to 0.001; at epsilon=0.1 a large fraction of steps would be relabeled off, and the trace would show multiple rises. So zero starts and eliminating all baseline cycling are likely artifacts of the measurement definition. The authors flag this in Limitations, but the abstract and title don't.

That said, the qualitative SAC/PPO contrast doesn't depend on the exact threshold: SAC's trace is visibly smoother and spends essentially no time saturated at zero or full, while PPO is bang-bang. What changes is the strong claim about eliminating cycling.

Other soft spots are secondary: Tables 1-2 have no repeated seeds or error bars; the surrogate is uncalibrated and taken on trust from the authors' earlier work; no code or data is provided. These affect quantitative claims, not the qualitative comparison.

Who gets value: people doing RL for heat-pump or HVAC control, reward shaping for smoothness, and anyone comparing off-policy vs on-policy algorithms. It deserves a serious referee, but the revision needs to recalibrate the duty threshold to a physically meaningful value, report seed statistics, and temper the abstract. I would accept it for review with major revision expected.

Referee Report

3 major / 5 minor

Summary. The paper studies reinforcement learning (RL) control of a residential hydronic heat pump, adding a levelised per-start compressor-wear cost to the standard comfort-plus-energy reward. Training SAC and PPO on an identical MDP with a SINDy surrogate plant and then evaluating on the BOPTEST bestest hydronic case, the authors report that SAC learns a continuous modulation policy with zero starts/day, whereas PPO collapses to bang-bang control that cycles more than the baseline. The SAC policy is reported to reduce thermal discomfort by up to 90.7% on the peak scenario for an 11.5% cost increase. The central claim is that the choice of RL algorithm, not the reward alone, determines whether the learned controller behaves like an inverter-driven heat pump or an on-off thermostat.

Significance. If the findings are robust, the paper makes a useful contribution: it demonstrates that a monetised switching penalty can be incorporated directly into the reward, and it identifies an algorithm-class effect on control smoothness that goes beyond the usual cost-comfort trade-off. The BOPTEST evaluation is a strength, as is the explicit reporting of duty signals, starts per day, and total variation. The paper also honestly states several limitations in Section 6. However, the headline quantitative claim of 'zero starts' is tied to a permissive duty threshold, and the empirical comparisons rest on single runs without error bars. These issues need to be resolved before the main conclusion can be considered fully supported.

major comments (3)

[Eq. (2), Section 5.2, Section 6] The zero-start claim is load-bearing and depends on the binary duty threshold ε = 10^-3. Section 5.2 reports that on the typical day SAC commands u_HP as low as 0.001 with a mean of 0.098. Real inverter compressors typically have a minimum stable modulation of 10–20% of rated capacity, so under a physically meaningful threshold of ε = 0.1, the vast majority of steps on the typical day would be labelled 'off', and the trace would cross the threshold repeatedly, producing nonzero starts/day. The Limitations section acknowledges that ε is permissive, but the abstract and Section 5.1 present 'zero starts' and 'eliminating all baseline cycling' without this caveat. To support the inverter-style interpretation, the authors should either recompute the cycling metrics with a threshold at the minimum stable modulation or explicitly report the number of starts under a range of thresholds.
[Tables 1 and 2] The central algorithm-class comparison is based on a single training run per algorithm, with no repeated seeds, confidence intervals, or significance tests reported. RL training is stochastic, and the claim that SAC reliably converges to an always-on modulation policy while PPO reliably converges to bang-bang control requires a distribution over runs. I ask the authors to provide results for at least five independent seeds per algorithm, reporting the range or interquartile interval of starts/day, switches/day, discomfort, and cost, and to state whether the SAC-vs-PPO differences are consistent across seeds.
[Section 4 and Section 6] The surrogate plant is adopted as-is from previous work and is described as uncalibrated, with identification 'outside the present scope'. Since policies are trained entirely on this surrogate, the fidelity of the surrogate directly influences which behaviors are learned. The out-of-sample BOPTEST evaluation partially mitigates this, but the paper would be much stronger if the surrogate were validated against BOPTEST trajectories (e.g., open-loop prediction error over the evaluation horizon) or if at least a sensitivity analysis showed that the qualitative SAC/PPO difference is robust to surrogate errors. As written, the reader cannot tell whether the learned 'zero starts' behavior transfers because it is a genuine property of the reward and algorithm or because it exploits surrogate artifacts.

minor comments (5)

[Related Work, Section 2] The sentence 'Karlsson and Fahlén [3] quantifies how variable-capacity control reduces start-ups' contains a subject-verb agreement error; 'quantifies' should be 'quantify'.
[Section 5.1 heading] The heading 'PPO W orsens It' contains an obvious typo; it should read 'PPO Worsens It'.
[Tables 1 and 2] The tables would be easier to read if the columns were separated by units (e.g., 'Wear (€/d)', 'TV (p.u.)') and if the scenario names were aligned. Also, the meaning of 'TV' is given in the caption of Table 1 but not repeated in Table 2, where it does not appear; consider removing it from the caption or adding a note.
[Section 6] The discussion of the trade-off states that the SAC premium is recovered 'roughly twofold' over 14 days, but this calculation assumes the levelised cost c_cyc is a real economic cost rather than a lifetime proxy. Since the paper itself describes c_cyc as a lifetime proxy, the economic interpretation should be softened or clearly labelled as an illustrative example.
[Throughout] The notation for the action 'u_HP' is introduced in Section 3.1 but the subscript formatting is inconsistent (e.g., 'u HP' in text and 'u_HP' in equations). Please unify the notation.

Circularity Check

0 steps flagged · score 1.0 of 10

No significant circularity: the SAC/PPO contrast is an empirical optimization outcome evaluated on an external emulator, not an identity or a fitted prediction.

full rationale

The paper's central claims are self-contained empirical results. The wear reward of Eq. (2) defines a start as a rising edge of the binary duty signal d_t = 1[u_HP,t > ε] with ε = 10^-3 and charges a levelised cost c_cyc; the same duty signal is used to report starts/day. This consistency is standard RL objective evaluation and does not by construction force the algorithm-specific outcome: SAC and PPO optimize the identical reward, yet PPO produces many starts and SAC produces none, so the zero-start result is an emergent policy difference rather than an identity. The discomfort and cost KPIs are measured on the genuine BOPTEST emulator, not on the training surrogate, so the headline 90.7% discomfort reduction is externally grounded. The only self-citation is [14] for the SINDy surrogate; it is adopted as-is, but all reported evaluation is transferred to BOPTEST, so the citation is not load-bearing. The acknowledged sensitivity of the duty threshold ε (Limitations) is a physical-interpretation caveat about what counts as 'on', not a circular derivation; it does not reduce the algorithm comparison to a tautology. No circular step is present.

Assumptions & free parameters 3 free parameters · 4 assumptions · 0 invented entities

The only new object is a cost term in the reward, not an entity. The paper introduces no new particles, forces, dimensions, or conserved quantities.

free parameters (3)

c_cyc (per-start wear cost) = 0.0133 EUR
Amortizes about 2000 EUR compressor replacement over about 150,000 rated starts (Section 3.2). It is an assumed economic constant, not measured or fitted to the control outcome.
Duty threshold epsilon = 1e-3
Defines the on state in Eq. (2). Hand-chosen and permissive; the zero-start result depends on this threshold, as the limitations section admits.
Reward weights = w_disc = w_cost = w_wear = 1
Set equal by hand. Different weights would trace a different point on the smoothness-cost-comfort frontier.

assumptions (4)

domain assumption The SINDy surrogate accurately reproduces the BOPTEST plant dynamics for training
Section 4 adopts the surrogate from prior work [14] and calls it uncalibrated. If surrogate dynamics differ from the emulator, the trained policies may not transfer.
domain assumption The BOPTEST emulator is a valid ground truth for cost, comfort, and cycling metrics
All final claims in Section 5 use BOPTEST KPIs; this is a standard simulation assumption.
ad hoc to paper Compressor wear is monetized by a constant per-start cost independent of capacity and conditions
Section 3.2 introduces the levelised cost as a lifetime proxy. Starts at high load or low temperature may wear more, which is not modeled.
ad hoc to paper A near-zero capacity command counts as the compressor being on
Section 5.2 and the Limitations state that u_HP = 0.007 counts as on, which a real inverter may not support.

how reviews work

0 comments

Cite this review

Pith. "Pith review of Learning to Modulate, Not to Cycle: Soft Actor---Critic Recovers Inverter-Style Heat-Pump Control." pith.science (2026). https://pith.science/paper/V2HBSGUW

@misc{pith2026260809453,
  author       = {Pith},
  title        = {Pith review of: Learning to Modulate, Not to Cycle: Soft Actor---Critic Recovers Inverter-Style Heat-Pump Control},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/V2HBSGUW}},
  note         = {Machine review of arXiv:2608.09453}
}

read the original abstract

On--off cycling is the main cause of compressor wear in residential heat pumps, yet reinforcement learning (RL) controllers for buildings typically optimise only energy cost and thermal comfort, ignoring how much the learned policy cycles. We add a levelised compressor-wear term to the control reward and study how the resulting behaviour depends on the RL algorithm. Training Soft Actor---Critic (SAC) and Proximal Policy Optimisation (PPO) on an identical Markov decision process for the BOPTEST bestest hydronic heat pump case, we find that SAC learns a continuous modulation policy that keeps the compressor permanently engaged---the operating principle of an inverter-driven heat pump---achieving zero start-ups per day, whereas PPO collapses to bang-bang control that cycles more than the baseline. On the BOPTEST emulator the SAC policy cuts thermal discomfort by up to 90.7% for an 11.5% cost increase, while eliminating all baseline cycling.

Figures

Figures reproduced from arXiv: 2608.09453 by the authors.

**Figure 1.** Same reward, opposite controllers. Closed-loop traces on BOPTEST for the peak (top) and typical (bottom) heating days; each panel shows zone temperature against its comfort band (upper) and the commanded capacity uHP (lower). SAC (a, c) modulates smoothly and never lets the compressor restart—zero starts/day with comfort intact, learned inverter operation even on the milder day. PPO (b, d) slams uHP between off and… view at source ↗

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

15 extracted references · 10 canonical work pages

[1]

Capacity control in ground source heat pump systems: Part I: modeling and simulation

Hatef Madani, Joachim Claesson, and Per Lundqvist. “Capacity control in ground source heat pump systems: Part I: modeling and simulation”. In: International Journal of Refrigeration34.6 (2011), pp. 1338–1347.issn: 0140-7007.doi: https://doi.org/10.1016/j.ijrefrig.2011.05.007.url: https://www.sciencedirect.com/science/article/pii/S0140700711001137 (cit. on...

work page doi:10.1016/j.ijrefrig.2011.05.007.url: 2011
[2]

Capacity control in ground source heat pump systems part II: Comparative analysis between on/off controlled and variable capacity systems

Hatef Madani, Joachim Claesson, and Per Lundqvist. “Capacity control in ground source heat pump systems part II: Comparative analysis between on/off controlled and variable capacity systems”. In:International Journal of Refrigeration34.8 (2011), pp. 1934–1942.issn: 0140-7007.doi: https: //doi.org/10.1016/j.ijrefrig.2011.05.012.url: https://www.sciencedire...

work page doi:10.1016/j.ijrefrig.2011.05.012.url: 2011
[3]

Capacity-controlled ground source heat pumps in hydronic heating systems

Fredrik Karlsson and Per Fahl´ en. “Capacity-controlled ground source heat pumps in hydronic heating systems”. In:International Journal of Refrig- eration30.2 (2007), pp. 221–229.issn: 0140-7007.doi: https://doi.org/10. 1016/j.ijrefrig.2006.08.008.url: https://www.sciencedirect.com/science/ article/pii/S0140700706001654 (cit. on pp. 1–3, 7)

work page 2007
[4]

Performance rating of unitary air-conditioning & air- source heat pump equipment

AHRI Standard. “Performance rating of unitary air-conditioning & air- source heat pump equipment”. In:AHRI Standard210 (2008), p. 240 (cit. on pp. 1, 2)

work page 2008
[5]

Reinforcement learning for building con- trols: The opportunities and challenges

Zhe Wang and Tianzhen Hong. “Reinforcement learning for building con- trols: The opportunities and challenges”. In:Applied Energy269 (2020), p. 115036.issn: 0306-2619.doi: https://doi.org/10.1016/j.apenergy. 2020.115036.url: https://www.sciencedirect.com/science/article/pii/ S0306261920305481 (cit. on p. 2)

arXiv 2020
[6]

Reinforcement learning for demand response: A review of algorithms and modeling techniques

Jos´ e R. V´ azquez-Canteli and Zolt´ an Nagy. “Reinforcement learning for demand response: A review of algorithms and modeling techniques”. In: Applied Energy235 (2019), pp. 1072–1089.issn: 0306-2619.doi: https:// doi.org/10.1016/j.apenergy.2018.11.002.url: https://www.sciencedirect. com/science/article/pii/S0306261918317082 (cit. on p. 2). 9

work page doi:10.1016/j.apenergy.2018.11.002.url: 2019
[7]

Building optimization testing framework (BOPTEST) for simulation-based benchmarking of control strategies in buildings

David Blum et al. “Building optimization testing framework (BOPTEST) for simulation-based benchmarking of control strategies in buildings”. In: Journal of Building Performance Simulation14.5 (2021), pp. 586–610. doi: 10.1080/19401493.2021.1986574. eprint: https://doi.org/10.1080/ 19401493.2021.1986574.url: https://doi.org/10.1080/19401493.2021. 1986574 (c...

arXiv 2021
[8]

Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actor

Tuomas Haarnoja et al. “Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actor”. In:Proceedings of the 35th International Conference on Machine Learning. Ed. by Jennifer Dy and Andreas Krause. Vol. 80. Proceedings of Machine Learning Re- search. PMLR, July 2018, pp. 1861–1870.url: https://proceedings.mlr. press/v...

work page 2018

Show all 15 references

[9]

John Schulman et al.Proximal Policy Optimization Algorithms. 2017. arXiv: 1707 . 06347[cs.LG].url: https : / / arxiv . org / abs / 1707 . 06347 (cit. on pp. 2, 4)

2017
[10]

All you need to know about model predictive control for buildings

J´ an Drgoˇ na et al. “All you need to know about model predictive control for buildings”. In:Annual Reviews in Control50 (2020), pp. 190–232.issn: 1367-5788.doi: https://doi.org/10.1016/j.arcontrol.2020.09.001.url: https://www.sciencedirect.com/science/article/pii/S1367578820...

2020 doi
[11]

Reinforced model predictive control (RL-MPC) for building energy management

Javier Arroyo et al. “Reinforced model predictive control (RL-MPC) for building energy management”. In:Applied Energy309 (2022), p. 118346. issn: 0306-2619.doi: https://doi.org/10.1016/j.apenergy.2021.118346. url: https://www.sciencedirect.com/science/article/pii/S030626192101...

2022
[12]

Regularizing Action Policies for Smooth Control with Reinforcement Learning

Siddharth Mysore et al. “Regularizing Action Policies for Smooth Control with Reinforcement Learning”. In:2021 IEEE International Conference on Robotics and Automation (ICRA). 2021, pp. 1810–1816.doi: 10.1109/ ICRA48506.2021.9561138 (cit. on p. 2)

2021
[13]

Discovering governing equations from data: Sparse identification of nonlinear dynami- cal systems

Steven L Brunton, Joshua L Proctor, and J Nathan Kutz. “Discovering governing equations from data: Sparse identification of nonlinear dynami- cal systems”. In:arXiv preprint arXiv:1509.03580(2015) (cit. on p. 4)

2015 arXiv
[14]

Aniket Dixit et al.Learning from Less: SINDy Surrogates in RL. 2025. arXiv: 2504.18113[cs.LG].url: https://arxiv.org/abs/2504.18113 (cit. on p. 4)

2025 arXiv
[15]

Stable-Baselines3: Reliable Reinforcement Learn- ing Implementations

Antonin Raffin et al. “Stable-Baselines3: Reliable Reinforcement Learn- ing Implementations”. In:Journal of Machine Learning Research22.268 (2021), pp. 1–8.url: http://jmlr.org/papers/v22/20-1364.html (cit. on p. 4)

2021

Pith tools

Reviewed August 11, 2026 · model on record in the stance chip above.

[1] [1]

Capacity control in ground source heat pump systems: Part I: modeling and simulation

Hatef Madani, Joachim Claesson, and Per Lundqvist. “Capacity control in ground source heat pump systems: Part I: modeling and simulation”. In: International Journal of Refrigeration34.6 (2011), pp. 1338–1347.issn: 0140-7007.doi: https://doi.org/10.1016/j.ijrefrig.2011.05.007.url: https://www.sciencedirect.com/science/article/pii/S0140700711001137 (cit. on...

work page doi:10.1016/j.ijrefrig.2011.05.007.url: 2011

[2] [2]

Capacity control in ground source heat pump systems part II: Comparative analysis between on/off controlled and variable capacity systems

Hatef Madani, Joachim Claesson, and Per Lundqvist. “Capacity control in ground source heat pump systems part II: Comparative analysis between on/off controlled and variable capacity systems”. In:International Journal of Refrigeration34.8 (2011), pp. 1934–1942.issn: 0140-7007.doi: https: //doi.org/10.1016/j.ijrefrig.2011.05.012.url: https://www.sciencedire...

work page doi:10.1016/j.ijrefrig.2011.05.012.url: 2011

[3] [3]

Capacity-controlled ground source heat pumps in hydronic heating systems

Fredrik Karlsson and Per Fahl´ en. “Capacity-controlled ground source heat pumps in hydronic heating systems”. In:International Journal of Refrig- eration30.2 (2007), pp. 221–229.issn: 0140-7007.doi: https://doi.org/10. 1016/j.ijrefrig.2006.08.008.url: https://www.sciencedirect.com/science/ article/pii/S0140700706001654 (cit. on pp. 1–3, 7)

work page 2007

[4] [4]

Performance rating of unitary air-conditioning & air- source heat pump equipment

AHRI Standard. “Performance rating of unitary air-conditioning & air- source heat pump equipment”. In:AHRI Standard210 (2008), p. 240 (cit. on pp. 1, 2)

work page 2008

[5] [5]

Reinforcement learning for building con- trols: The opportunities and challenges

Zhe Wang and Tianzhen Hong. “Reinforcement learning for building con- trols: The opportunities and challenges”. In:Applied Energy269 (2020), p. 115036.issn: 0306-2619.doi: https://doi.org/10.1016/j.apenergy. 2020.115036.url: https://www.sciencedirect.com/science/article/pii/ S0306261920305481 (cit. on p. 2)

arXiv 2020

[6] [6]

Reinforcement learning for demand response: A review of algorithms and modeling techniques

Jos´ e R. V´ azquez-Canteli and Zolt´ an Nagy. “Reinforcement learning for demand response: A review of algorithms and modeling techniques”. In: Applied Energy235 (2019), pp. 1072–1089.issn: 0306-2619.doi: https:// doi.org/10.1016/j.apenergy.2018.11.002.url: https://www.sciencedirect. com/science/article/pii/S0306261918317082 (cit. on p. 2). 9

work page doi:10.1016/j.apenergy.2018.11.002.url: 2019

[7] [7]

Building optimization testing framework (BOPTEST) for simulation-based benchmarking of control strategies in buildings

David Blum et al. “Building optimization testing framework (BOPTEST) for simulation-based benchmarking of control strategies in buildings”. In: Journal of Building Performance Simulation14.5 (2021), pp. 586–610. doi: 10.1080/19401493.2021.1986574. eprint: https://doi.org/10.1080/ 19401493.2021.1986574.url: https://doi.org/10.1080/19401493.2021. 1986574 (c...

arXiv 2021

[8] [8]

Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actor

Tuomas Haarnoja et al. “Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actor”. In:Proceedings of the 35th International Conference on Machine Learning. Ed. by Jennifer Dy and Andreas Krause. Vol. 80. Proceedings of Machine Learning Re- search. PMLR, July 2018, pp. 1861–1870.url: https://proceedings.mlr. press/v...

work page 2018