Pith. sign in

REVIEW 4 major objections 6 minor 1 cited by

In an occupied home, a model-based RL controller matched MPC energy savings (about 22% vs 20%) while cutting per-home setup time from roughly five days to two—but kept the house cooler and trailed MPC on comfort-adjusted efficiency.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Model-based RL delivered energy savings comparable to MPC in an occupied residence (~22% vs ~20% vs PID), with lower recurring engineering effort, but MPC had better comfort-normalized efficiency and comfort.

T0 review reviewed 2026-08-04 challenge →

load-bearing objection First same-house month-long RL vs MPC HVAC field test; useful despite non-overlapping baselines and an abstract/body number mismatch. the 4 major comments →

arxiv 2510.01475 v2 pith:6OANX2YW submitted 2025-10-01 eess.SY cs.LGcs.SY

Comparative Field Deployment of Reinforcement Learning and Model Predictive Control for Residential HVAC

Ozan Baris Mulayim , Elias N. Pergantis , Levi D. Reyes Premer , Bingqing Chen , Guannan Qu , Kevin J. Kircher , Mario Berg\'es This is my paper
classification eess.SY cs.LGcs.SY
keywords reinforcement learningmodel predictive controlresidential HVACheat pumpfield deploymentoccupant comfortenergy savingsdifferentiable MPC
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper reports the first head-to-head field comparison of a model predictive control (MPC) and a model-based reinforcement learning (RL) controller for residential heat-pump heating, each run for a month in the same occupied house. Its central claim is that RL can match MPC's weather-normalized energy savings (~22% vs ~20%) with considerably less engineering effort (~2 vs >5 days per new home), because the RL agent learns the same physics-informed thermal model automatically. The catch: RL achieved lower comfort-normalized efficiency (~7.3% vs ~12.7% improvement over the baseline), caused three episodes of occupant discomfort during an initial adaptation period, and was more fragile to real-world operational failures. The finding matters because it separates the scalability promise of RL from its practical readiness, and points to safe initialization and model accuracy as the binding constraints.

Core claim

Both controllers share a 2R1C resistor-capacitor thermal model of the house and both set thermostat setpoints hourly using a 24-hour lookahead; the difference is how the model is obtained. MPC's parameters are hand-fitted through a multi-step manual procedure plus a support-vector machine for solar and internal gains. The RL controller learns the same model structure end-to-end from historical data, then fine-tunes both dynamics and cost weights online using a differentiable model-predictive policy. Over 23 comparable days each, RL used about 22% less weather-normalized energy than the baseline PID controller, MPC about 20%; once comfort is priced in, MPC's efficiency improvement (12.7%) nea

What carries the argument

The load-bearing object is a shared 2R1C thermal network—two resistances and one capacitance connecting indoor air, thermal mass, and outdoors—that both controllers use as their dynamics model, with the heat pump's coefficient of performance folded in. For RL, this same structure is embedded in a differentiable MPC policy whose parameters are learned end-to-end (imitation pre-training plus online gradient updates on state and reward losses). The comparison itself rests on energy-signature curves, E_e = (β0 + β1 T_out)/COP(T_out), fitted to daily energy use versus outdoor temperature, whose areas under a common temperature interval define savings.

Load-bearing premise

The headline savings comparison assumes that energy-signature curves fitted to different calendar periods (PID 2022–23, MPC 2023, RL 2025) can be extrapolated over a common outdoor-temperature range without unmeasured changes in occupancy, envelope, or equipment; if that comparability fails, the reported savings and the RL-vs-MPC ordering are not identifiable from the data.

What would settle it

Refit the energy-signature curves using only days where outdoor temperature lies within each controller's actually observed range, with occupancy logs as covariates; or rerun RL for a month after a two-week no-learning warm-up. If the 22%-vs-20% savings ordering or the 7.3%-vs-12.7% efficiency ordering does not reproduce, the central comparison collapses.

Watch this falsifier. Get emailed when new claim-graph text bears on it.

If this is right

  • In a cold-climate air-to-air heat-pump home, an RL thermostat supervisor can save as much energy as a carefully tuned MPC (about 22% vs 20%) without a simulator.
  • Per-home engineering effort drops from about 5 days for MPC to about 2 days for RL, because model fitting is automated.
  • The comfort cost is real: RL kept the house cooler during early adaptation, drew three discomfort reports, and showed higher PPD variance; longer or safer initialization would be needed in practice.
  • RL's learned cost parameters automatically penalized backup resistance heat and favored the heat pump, a behavior that emerged without explicit encoding.
  • Operational reliability—API failures, actuator mismatch, corrupted online-learning data—becomes the main risk for adaptive controllers, not the learning algorithm itself.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • If the ~2-day RL setup holds up in other homes, the economic case for RL flips: even one engineer-day of onboarding can exceed several years of energy savings in a typical residence, so the automation advantage is the decisive metric at scale.
  • The comfort-normalized efficiency gap suggests a testable hypothesis: an RL agent initialized with a non-quadratic reward and a short safe warm-up period would close the 5-point efficiency gap while keeping the labor advantage.
  • The shared 2R1C structure makes the comparison unusually clean; future deployments could strengthen it by randomizing controller order or running each controller twice to separate learning transients from steady-state performance.
  • The reported 'RL keeps house cooler' strategy may actually be a feature for demand response or time-of-use pricing, where deliberate setpoint modulation is valuable—an extension the paper does not quantify.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

4 major / 6 minor

Summary. The paper reports a comparative field deployment of one model predictive control (MPC) implementation and one model-based reinforcement learning (RL) implementation, each run for roughly one month in an occupied all-electric townhouse with a heat pump and staged electric backup. Both controllers set thermostat setpoints through a shared indirect actuation scheme, and both use a similar 2R1C thermal-network model: MPC fits it through a multi-stage manual identification procedure, while the RL agent (Ibex-RL) learns the same model structure and a quadratic cost function end-to-end via imitation learning followed by online reward-based tuning. The central claim is that RL achieved comparable weather-normalized energy savings relative to the PID baseline (approximately 22% versus 20% for MPC) at substantially lower recurring deployment effort (about 2 versus more than 5 engineer-days), but that MPC was superior on comfort-normalized efficiency (about 12.7% versus 7.3% improvement) and on occupant comfort. The paper also documents practical difficulties of online RL, including unsafe initialization, actuation mismatch, and sensitivity to data/API failures.

Significance. If the quantitative results are correct, this is one of the very few month-long residential RL deployments and the first reported model-based RL controller in an occupied residence, and it provides a rare head-to-head comparison with a well-documented MPC baseline in the same building. The paper is valuable for its empirical honesty: it reports three occupant discomfort incidents, degradation of learned dynamics parameters (e.g., negative solar aperture), API failures, and a transparent accounting of deployment labor. It also provides Monte Carlo uncertainty estimates for the savings and efficiency metrics. These features distinguish it from purely simulation-based comparisons. However, the strength of the empirical contribution depends on the comparability of non-overlapping evaluation periods and on the consistency of the reporting; both currently have issues that must be addressed before the headline numbers can be considered reliable.

major comments (4)
  1. [Abstract; §6.3.1; Fig. 9; §7.6; Conclusions] The reported savings numbers are internally inconsistent across versions of the paper. The full-text abstract and §7.6/Conclusions state RL ~22% and MPC ~20% savings. The Section 6.3.1 area calculations imply RL 522.19 vs PID 669.36 (~22.0%) and MPC 535.88 vs 669.36 (~19.9%). The Monte Carlo means in Fig. 9 are 21.95% for RL and 19.91% for MPC. The arXiv abstract states 18.1% for MPC and 20.9% for RL, with wide confidence intervals (4.4–30.9 and 2.6–38.3). These are not just cosmetic differences: the central claim is the RL-vs-MPC ordering and the magnitude of savings. Please state which numbers are the final ones, reconcile the confidence intervals with the Monte Carlo output, and ensure all sections use the same numbers.
  2. [§5.2.2; §6.3.1; Eq. (15); Fig. 8] The savings comparison is identified only under a strong stationarity assumption. PID data are from Dec 11, 2022–Apr 4, 2023; MPC from Feb 1–Mar 30, 2023; RL from Jan 23–Feb 23, 2025. Energy-signature curves are fitted to these non-overlapping periods and then extrapolated over a common outdoor-temperature interval (−7 °C to 5 °C). This assumes no material change in occupancy, envelope, equipment performance, thermostat behavior, or measurement setup across the two-year gap. The paper provides no evidence on these points. Because the PID baseline is not contemporaneous with the RL deployment, any drift — e.g., occupancy changes, air-sealing, heat-pump degradation — directly biases the estimated savings and could reverse the RL-vs-MPC ordering. Please provide sensitivity analyses: for example, restrict PID to the same calendar period as MPC (Feb–Mar 2023), use a contemporaneous baseline p
  3. [§6.3.1; Eq. (15); Fig. 5; §6.3.2; Eq. (16)] The headline 'comparable energy savings' is not a like-for-like comparison of control quality. Eq. (15) assumes a constant indoor setpoint, but Fig. 5 shows RL deliberately maintained daytime setpoints about 1 °C below user preferences (19 °C vs 20 °C), and §6.3.3 documents three occupant discomfort reports. Thus part of RL's apparent savings in Eq. (15) reflects lower delivered thermal comfort, not higher efficiency. The paper does acknowledge this and provides Eq. (16) as a comfort-normalized efficiency metric, where MPC is indeed better (12.7% vs 7.3%). However, §7.6 and the Conclusions still lead with the 'comparable energy savings' framing. Please rephrase the headline to make clear that the savings are unnormalized for comfort, and consider reporting the comfort-normalized efficiency as the primary measure of control performance.
  4. [§4.2; §4.1; §6.3; §7.6] The comparison is between two implementations that share crucial design elements: both controllers use the same 2R1C thermal model, and the RL reward signal is −J_t, i.e., the same MPC objective with a fixed discomfort price w_c instead of the adaptive PPD-based w_c used by MPC. This is a strength for isolating control-design choices, but it also means the paper does not compare RL against MPC as general paradigms. The paper is appropriately cautious in §7.6, yet the abstract and conclusions phrase the result as 'RL vs MPC' without this caveat. We recommend making this design-dependence explicit in the abstract and conclusions: the findings pertain to a physics-informed, model-based RL agent that already embeds an MPC-like model and objective.
minor comments (6)
  1. [Table 1] The entry for 'This study' states '14-30% energy savings with minimal discomfort vs. measured baseline', but the paper later reports three discomfort reports and a mean RL PPD of 14.05% (return-air). Please align the table wording with the body's nuanced comfort findings.
  2. [§6.3.4; §7.5] The deployment-labor accounting mixes non-recurring and recurring costs: the MPC initial deployment is given as ~190 student-days, while the recurring per-home estimate is ~5 engineer-days. Please clarify what exactly is counted as 'initial development' vs 'commissioning' and how the two estimates relate, otherwise readers may conflate one-time research infrastructure costs with per-home deployment costs.
  3. [§4.1; Eq (9)] The discrete-time conversion via zero-order hold is presented compactly. For reproducibility, please clarify the sampling interval Δt (stated as 1 hour in §5.1) and define the dimensions of B_u(T_out,t), especially because B_u is time-varying through COP(T_out).
  4. [§6.1; §4.2] The text says the RL agent's θ_cost includes {O_t, R_hp, R_bh}, but Eq. (6) and the surrounding notation use R_t for the control-cost matrix. Please unify the notation to avoid confusion between R_t (cost weight) and R_m/R_out (thermal resistances).
  5. [General] The paper would benefit from a table summarizing deployment periods, effective number of days, and key weather ranges for PID, MPC, and RL. Currently these are scattered across §5.2.2 and §6.3, making it harder to assess the comparability of the evaluation windows.
  6. [Data availability] The statement 'Data and code will be made available upon publishing' is vague. Since the paper's contribution is empirical, please provide a concrete repository or a detailed data description to allow independent verification of the energy-signature fits and Monte Carlo analyses.

Circularity Check

0 steps flagged

No significant circularity; the reported savings/efficiency are empirical fits and field measurements, not derivations that reduce to their inputs.

full rationale

The paper's central quantitative claims (Section 6.3.1-6.3.2, Figures 8-11) are obtained by fitting energy-signature regressions (Eqs. 15-16) to each controller's measured daily energy use, then comparing areas under the fitted curves. These are empirical estimates with Monte Carlo uncertainty bands, not first-principles predictions, and no fitted parameter is renamed as a predicted outcome. The RL controller does share design elements with the MPC benchmark: both use the 2R1C thermal model, and Section 4.2 states that the RL reward uses R_t = -J_t, the same objective as MPC. This is a disclosed design overlap that makes the comparison partly 'MPC vs. an RL trained to approximate MPC's objective,' but it does not make the measured savings or efficiency ordering true by construction: the RL's learned dynamics and cost parameters, its fixed w_c, and the online adaptation could have produced worse or different outcomes, and indeed produced lower comfort-normalized efficiency. The self-citations to the authors' own Ibex-RL [22] and MPC [23] describe the specific controllers under test; the algorithms are specified in-text (Algorithms 1-2, Section 4), so the citations are not load-bearing. No uniqueness theorem or prior ansatz is imported to force the comparison. The non-overlapping deployment periods create a legitimate stationarity/identifiability risk, but that is a correctness concern, not circularity.

Axiom & Free-Parameter Ledger

6 free parameters · 6 axioms · 0 invented entities

The headline savings are computed from fitted energy-signature models, not from a first-principles derivation; the main added assumptions are the affine weather-normalization model, the COP conversion, the indirect setpoint actuation, and the comparability of different deployment periods. No new physical entities are introduced.

free parameters (6)
  • β0, β1 (savings energy-signature fits) = PID: 150.84, -9.92; MPC: 121.64, -7.32; RL: 117.26, -8.03 (Fig. 8)
    Fitted to daily energy vs outdoor temperature; areas under these curves are the basis for the 20-22% savings claims.
  • β2, β3 (efficiency fits) = PID: -46.31, -9.94; MPC: -24.35, -7.65; RL: -46.39, -8.90 (Fig. 10)
    Fitted to daily energy vs indoor-outdoor temperature difference; basis for comfort-normalized efficiency claims.
  • 2R1C thermal model params (MPC) = R_m=1.06 °C/kW, R_out=2.04 °C/kW, C=2.34e7 J/°C, T_m=20.6 °C
    Identified from historical data; used by MPC to plan setpoints; not directly used in headline savings but part of controller comparison.
  • 2R1C + solar params (RL) = R_m=1.07, R_out=1.07, C=1.79e7, T_m=26.25, A_eff learned (converged negative), η assumed
    Learned end-to-end; A_eff negative indicates unrealistic fit; these parameters drive RL's control decisions.
  • Cost weights w_d, w_e, w_c = w_d=$0.8/kW, w_e=$0.15/kWh, w_c=3/(°C·h) (RL fixed; MPC adaptive)
    Selected by authors; directly set the energy/comfort trade-off, affecting both energy use and PPD.
  • Imitation learning hyperparameters α, λ = α=0.05, λ=1000 (selected by validation)
    Chosen to minimize validation action loss; affect the RL initial policy and safety.
axioms (6)
  • domain assumption Daily heating energy is affine in mean outdoor temperature: Q_day = β0 + β1 T_out (Eq. 14), with E_e ≈ Q_day / COP(T_out) (Eq. 15).
    Adopted from [60,47]; used to weather-normalize savings; assumes constant indoor setpoint and minor backup heat.
  • domain assumption The 2R1C RC network adequately represents the house thermal dynamics for both MPC and RL (Eq. 7-9).
    Basis for both controllers' internal model; if false, the setpoint translation and learned parameters are unreliable.
  • domain assumption Setting the thermostat setpoint to the predicted optimal next state x*_{t+1} yields energy use comparable to the optimal power u*_t (Section 5.1).
    Load-bearing for both controllers; authors note the mismatch between return-air sensor and thermostat sensor.
  • domain assumption The COP(T_out) curve and PPD comfort model are accurate for this equipment and occupants (Eq. 8; Table 2).
    COP is used to convert electrical energy to heating load; PPD is used for comfort conclusions.
  • domain assumption Comparability of non-simultaneous deployment periods after weather normalization.
    Underpins the headline savings comparison; not tested.
  • domain assumption The state is perfectly observable (Section 3).
    The control formulation ignores partial observability, yet the deployed system had a sensor mismatch.

reviewed 2026-08-04 · how reviews work

0 comments
Cite this review

Pith. "Pith review of Comparative Field Deployment of Reinforcement Learning and Model Predictive Control for Residential HVAC." pith.science (2026). https://pith.science/paper/6OANX2YW

@misc{pith2026251001475,
  author       = {Pith},
  title        = {Pith review of: Comparative Field Deployment of Reinforcement Learning and Model Predictive Control for Residential HVAC},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/6OANX2YW}},
  note         = {Machine review of arXiv:2510.01475}
}
Share X Bluesky LinkedIn Reddit HN
read the original abstract

Model Predictive Control (MPC) has demonstrated significant performance improvements over today's control methods for residential Heating, Ventilation, and Air Conditioning (HVAC), but deploying MPC often requires substantial engineering effort. Reinforcement Learning (RL) may offer comparable performance with easier deployment, but its practical application for residential HVAC remains largely undemonstrated, leaving open questions related to occupant comfort and data requirements. To investigate these issues, we deployed one MPC variant and one model-based RL variant for one month each in an occupied house in a cold climate. The controllers adjusted an air-to-air heat pump's thermostat temperature setpoint based on measurements of the indoor temperature and the electric power used for heating. Relative to constant-setpoint operation, MPC saved 18.1\% (95\% confidence interval: 4.4 to 30.9\%) of weather-normalized heat pump energy and RL saved 20.9\% (2.6 to 38.3\%). MPC maintained acceptable occupant comfort. RL kept the house cooler, particularly during an initial adaptation phase, leading to three reports of occupant discomfort. The two algorithms had similar data requirements. We estimate that for a fresh deployment in another house, RL would take about one-third less engineering effort than MPC. While RL reduces deployment effort, it faces difficulties related to safe controller initialization and to mismatches between the modeled and true state and action spaces.

Figures

Figures reproduced from arXiv: 2510.01475 by Bingqing Chen, Elias N. Pergantis, Guannan Qu, Kevin J. Kircher, Levi D. Reyes Premer, Mario Berg\'es, Ozan Baris Mulayim.

Figure 1
Figure 1. Figure 1: Overview of the RL and MPC controllers month-long field deployment, including adaptation performance, and a comparative analysis of the RL controller against MPC an existing benchmark con￾troller regarding savings, efficiency, and comfort. Section 7 reflects on the lessons learned from the deployment and discusses practical improvements for future RL and MPC implementations. Finally, the paper concludes in… view at source ↗
Figure 2
Figure 2. Figure 2: Thermal circuit model of the testbed. The general continuous-time equation for this 2R1C model is: C dT dt = Tm − T Rm + Tout − T Rout + Q˙ c + Q˙ e (7) Here, T represents the indoor air temperature (measured from the return duct), Tout is the out￾door air temperature, and Tm denotes the ther￾mal mass temperature. The key parameters in￾clude the thermal resistances between the indoor air and the thermal ma… view at source ↗
Figure 4
Figure 4. Figure 4: State, Action and Imitation Losses coming from [PITH_FULL_IMAGE:figures/full_fig_p013_4.png] view at source ↗
Figure 6
Figure 6. Figure 6: Warm day comparison: RL demonstrates proac [PITH_FULL_IMAGE:figures/full_fig_p015_6.png] view at source ↗
Figure 5
Figure 5. Figure 5: Cold weather comparison: RL demonstrates an [PITH_FULL_IMAGE:figures/full_fig_p015_5.png] view at source ↗
Figure 7
Figure 7. Figure 7: Mild day comparison: RL demonstrates proac [PITH_FULL_IMAGE:figures/full_fig_p016_7.png] view at source ↗
Figure 8
Figure 8. Figure 8: Fits of daily energy use vs. outdoor temperature. [PITH_FULL_IMAGE:figures/full_fig_p017_8.png] view at source ↗
Figure 9
Figure 9. Figure 9: Distribution of estimated energy savings (%) rela [PITH_FULL_IMAGE:figures/full_fig_p017_9.png] view at source ↗
Figure 10
Figure 10. Figure 10: Fits of daily energy use vs. outdoor-indoor tem [PITH_FULL_IMAGE:figures/full_fig_p017_10.png] view at source ↗
Figure 11
Figure 11. Figure 11: Distribution of estimated efficiency improvements [PITH_FULL_IMAGE:figures/full_fig_p018_11.png] view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score.

  1. IOGRUCloud: A Scalable AI-Driven IoT Platform for Climate Control in Controlled Environment Agriculture

    eess.SY 2026-04 unverdicted novelty 4.0

    IOGRUCloud is a scalable three-tier IoT platform with GRU-enhanced PID control for VPD that achieved 23% energy reduction and 31% climate stability gains in a 14-greenhouse deployment.

Reference graph

Works this paper leans on

69 extracted references · 2 canonical work pages · cited by 1 Pith paper

  1. [1]

    X. Li, J. Wen, Review of building energy mod- eling for control and operation, Renewable and Sustainable Energy Reviews 37 (2014) 517– 537

    2014

  2. [2]

    Bengea, A

    S. Bengea, A. Kelman, F. Borrelli, R. Tay- lor, S. Narayanan, Model predictive control for mid-size commercial building HVAC: Imple- mentation, results and energy savings, in: Sec- ond international conference on building en- ergy and environment, 2012, pp. 979–986

    2012

  3. [3]

    Finck, R

    C. Finck, R. Li, W. Zeiler, Optimal con- trol of demand flexibility under real-time pric- ing for heating systems in buildings: A real- life demonstration, Applied energy 263 (2020) 114671

    2020

  4. [4]

    Bünning, B

    F. Bünning, B. Huber, A. Schalbetter, A. Aboudonia, M. H. de Badyn, P. Heer, R. S. Smith, J. Lygeros, Physics-informed linear re- gressioniscompetitivewithtwomachinelearn- ing methods in residential building MPC, Ap- plied Energy 310 (2022) 118491

    2022

  5. [5]

    Brown, I

    S. Brown, I. Beausoleil-Morrison, Long-term implementation of a model predictive con- troller for a hydronic floor heating and cooling systeminahighlyglazedhouseinCanada, Ap- plied Energy 349 (2023) 121677

    2023

  6. [6]

    Thorsteinsson, A

    S. Thorsteinsson, A. A. S. Kalaee, P. Vogler- Finck, H. L. Stærmose, I. Katic, J. D. Bendt- sen, Long-term experimental study of price re- sponsive predictive control in a real occupied single-family house with heat pump, Applied Energy 347 (2023) 121398

    2023

  7. [8]

    Drgoňa, J

    J. Drgoňa, J. Arroyo, I. C. Figueroa, D. Blum, K. Arendt, D. Kim, E. P. Ollé, J. Oravec, M. Wetter, D. L. Vrabie, et al., All you need to know about model predictive control for build- ings, Annual Reviews in Control 50 (2020) 190–232. 24 Table 3: Mathematical Notation (Part 1: Physical System and Parameters) Symbol (Units) Meaning x, xt (◦C) System state...

    2020

  8. [9]

    Killian, M

    M. Killian, M. Kozek, Ten questions concern- ing model predictive control for energy effi- cient buildings, Building and Environment 105 (2016) 403–412

    2016

  9. [10]

    Chinde, Y

    V. Chinde, Y. Lin, M. J. Ellis, Data-enabled predictive control for building HVAC systems, Journal of Dynamic Systems, Measurement, and Control 144 (8) (2022) 081001

    2022

  10. [11]

    Drgoňa, K

    J. Drgoňa, K. Kiš, A. Tuor, D. Vrabie, M. Klaučo, Differentiable predictive control: Deep learning alternative to explicit model predictive control for unknown nonlinear sys- tems, Journal of Process Control 116 (2022) 80–92

    2022

  11. [12]

    O. B. Mulayim, E. Severnini, M. Bergés, Unmasking the role of remote sensors in comfort, energy, and demand response, Data-Centric Engineering 5 (2024) e28. doi:10.1017/dce.2024.25

    work page doi:10.1017/dce.2024.25 2024

  12. [13]

    R. S. Sutton, A. G. Barto, et al., Reinforce- ment learning: An introduction, Vol. 1, MIT press Cambridge, 1998

    1998

  13. [14]

    B. Chen, Z. Cai, M. Bergés, Gnu-RL: A Precocial Reinforcement Learning Solution for Building HVAC Control Using a Differen- tiable MPC Policy, in: Proceedings of the 6th ACM International Conference on Systems for Energy-Efficient Buildings, Cities, and Trans- portation, ACM,NewYorkNYUSA,2019, pp. 316–325. doi:10.1145/3360322.3360849

    arXiv 2019

  14. [15]

    Dulac-Arnold, N

    G. Dulac-Arnold, N. Levine, D. J. Mankowitz, J. Li, C. Paduraru, S. Gowal, T. Hester, Chal- lenges of real-world reinforcement learning: definitions, benchmarks and analysis, Machine Learning 110 (9) (2021) 2419–2468

    2021

  15. [16]

    Leurs, B

    T. Leurs, B. J. Claessens, F. Ruelens, S. Weckx, G. Deconinck, Beyond theory: Ex- perimental results of a self-learning air con- ditioning unit, in: 2016 IEEE International Energy Conference (ENERGYCON), IEEE, 2016, pp. 1–6

    2016

  16. [17]

    Kurte, J

    K. Kurte, J. Munk, O. Kotevska, K. Amasyali, R. Smith, E. McKee, Y. Du, B. Cui, T. Kuru- ganti, H. Zandi, Evaluating the adaptability of reinforcement learning based HVAC control for residential houses, Sustainability 12 (18) (2020) 7727

    2020

  17. [18]

    Svetozarevic, C

    B. Svetozarevic, C. Baumann, S. Muntwiler, L. Di Natale, M. N. Zeilinger, P. Heer, Data- driven control of room temperature and bidi- rectional ev charging using deep reinforcement learning: Simulations and experiments, Ap- plied Energy 307 (2022) 118127

    2022

  18. [19]

    Montazeri, C

    M. Montazeri, C. Remlinger, B. B. Haro, P. Heer, Fully data-driven and modular build- ing thermal control with physically consistent modeling, Applied Energy 390 (2025) 125770

    2025

  19. [20]

    D. Wang, W. Zheng, Z. Wang, Y. Wang, X. Pang, W. Wang, Comparison of reinforce- ment learning and model predictive control for building energy system optimization, Applied Thermal Engineering 228 (2023) 120430

    2023

  20. [21]

    Z. Nagy, G. Henze, S. Dey, J. Arroyo, L. Helsen, X. Zhang, B. Chen, K. Amasyali, K. Kurte, A. Zamzam, et al., Ten questions concerning reinforcement learning for build- ing energy management, Building and Envi- ronment 241 (2023) 110435

    2023

  21. [22]

    O. B. Mulayim, M. Bergés, Ibex-rl: In- terpretable and scalable control via physics- informed reinforcement learning, in: Proceed- ings of the 12th ACM International Confer- ence on Systems for Energy-Efficient Build- ings, Cities, and Transportation (Accepted), 2025

    2025

  22. [23]

    E. N. Pergantis, Priyadarshan, N. A. Theeb, P. Dhillon, J. P. Ore, D. Ziviani, E. A. Groll, K. J. Kircher, Field demonstration of predic- tive heating control for an all-electric house in a cold climate, Applied Energy 360 (2024) 122820. doi:10.1016/j.apenergy.2024.122820

    arXiv 2024

  23. [24]

    X. Wang, B. Dong, Long-term experimental evaluation and comparison of advanced con- trols for HVAC systems, Applied Energy 371 (2024) 123706

    2024

  24. [25]

    Lymperopoulos, P

    G. Lymperopoulos, P. Ioannou, Building tem- perature regulation in a multi-zone HVAC sys- tem using distributed adaptive control, Energy and Buildings 215 (2020) 109825

    2020

  25. [26]

    Levine, A

    S. Levine, A. Kumar, G. Tucker, J. Fu, Of- fline Reinforcement Learning: Tutorial, Re- view, and Perspectives on Open Problems, arXiv:2005.01643 (Nov. 2020). URLhttp://arxiv.org/abs/2005.01643 26

    Pith/arXiv arXiv 2005

  26. [27]

    H.-Y. Liu, B. Balaji, R. Gupta, D. Hong, Adaptive policy regularization for offline-to- online reinforcement learning in HVAC con- trol, in: Proceedings of the 11th ACM Inter- national Conference on Systems for Energy- Efficient Buildings, Cities, and Transporta- tion, 2024, pp. 1–10

    2024

  27. [28]

    Nakamoto, S

    M. Nakamoto, S. Zhai, A. Singh, M. Sobol Mark, Y. Ma, C. Finn, A. Ku- mar, S. Levine, Cal-ql: Calibrated offline RL pre-training for efficient online fine-tuning, Advances in Neural Information Processing Systems 36 (2024)

    2024

  28. [29]

    A.Kumar, A.Zhou, G.Tucker, S.Levine, Con- servative Q-learning for offline reinforcement learning, Advances in Neural Information Pro- cessing Systems 33 (2020) 1179–1191

    2020

  29. [30]

    Zhang, A

    Z. Zhang, A. Chong, Y. Pan, C. Zhang, S. Lu, K. P. Lam, A deep reinforcement learning ap- proach to using whole building energy model for HVAC optimal control, in: 2018 Build- ing Performance Analysis Conference and Sim- Build, Vol. 3, 2018, pp. 22–23

    2018

  30. [31]

    Y. Li, Y. Wen, D. Tao, K. Guan, Transform- ing cooling optimization for green data center viadeepreinforcementlearning, IEEEtransac- tions on cybernetics 50 (5) (2019) 2002–2013

    2019

  31. [32]

    R. Jia, M. Jin, K. Sun, T. Hong, C. Spanos, Advanced building control via deep reinforce- ment learning, Energy Procedia 158 (2019) 6158–6163

    2019

  32. [33]

    S. Xu, Y. Fu, Y. Wang, Z. Yang, C. Huang, Z. O’Neill, Z. Wang, Q. Zhu, Efficient and assured reinforcement learning-based build- ing HVAC control with heterogeneous expert- guided training, Scientific reports 15 (1) (2025) 7677

    2025

  33. [34]

    Zhang, S

    C. Zhang, S. R. Kuppannagari, V. K. Prasanna, Safe building HVAC control via batch reinforcement learning, IEEE Transac- tions on Sustainable Computing 7 (4) (2022) 923–934

    2022

  34. [35]

    X. Ding, W. Du, A. E. Cerpa, Mb2c: Model- based deep reinforcement learning for multi- zone building control, in: Proceedings of the 7th ACM International Conference on Systems for Energy-Efficient Buildings, Cities, and Transportation, BuildSys ’20, Association for Computing Machinery, New York, NY, USA, 2020, p. 50–59. doi:10.1145/3408308.3427986

    arXiv 2020

  35. [36]

    Z. An, X. Ding, A. Rathee, W. Du, CLUE: Safe Model-Based RL HVAC Control Using Epistemic Uncertainty Estimation, in: Pro- ceedings of the 10th ACM International Con- ference on Systems for Energy-Efficient Build- ings, Cities, and Transportation, BuildSys ’23, Association for Computing Machinery, New York, NY, USA, 2023, pp. 149–158. doi:10.1145/3600100.3623742

    arXiv 2023

  36. [37]

    B. Amos, I. Jimenez, J. Sacks, B. Boots, J. Z. Kolter, Differentiable MPC for end-to- end planning and control, Advances in neural information processing systems 31 (2018)

    2018

  37. [38]

    A. J. Khabbazi, E. N. Pergantis, L. D. Reyes Premer, P. Papageorgiou, A. H. Lee, J. E. Braun, G. P. Henze, K. J. Kircher, Lessons learned from field demonstrations of model predictive control and reinforcement learning for residential and commercial HVAC: A review, Applied Energy 399 (2025) 126459. doi:https://doi.org/10.1016/j.apenergy.2025.126459

    arXiv 2025

  38. [39]

    S. Liu, G. P. Henze, Experimental analysis of simulated reinforcement learning control for active and passive building thermal storage in- ventory: Part 2: Results and analysis, Energy and buildings 38 (2) (2006) 148–161

    2006

  39. [40]

    Zhang, A

    Z. Zhang, A. Chong, Y. Pan, C. Zhang, K. P. Lam, Whole building energy model for HVAC optimal control: A practical framework based on deep reinforcement learning, Energy and Buildings 199 (2019) 472–490

    2019

  40. [41]

    A. Naug, M. Quinones-Grueiro, G. Biswas, Re- inforcement learning-based HVAC supervisory control of a multi-zone building-A real case study, in: 2022 IEEE conference on control technology and applications (CCTA), IEEE, 2022, pp. 1172–1177

    2022

  41. [42]

    J. Luo, C. Paduraru, O. Voicu, Y. Chervonyi, S. Munns, J. Li, C. Qian, P. Dutta, J. Q. Davis, N. Wu, et al., Controlling commercial cooling systems using reinforcement learning, arXiv preprint arXiv:2211.07357 (2022). 27

    Pith/arXiv arXiv 2022

  42. [43]

    Silvestri, D

    A. Silvestri, D. Coraci, S. Brandi, A. Capoz- zoli, A. Schlueter, Practical deployment of re- inforcement learning for building controls us- ing an imitation learning approach, Energy and Buildings 335 (2025) 115511

    2025

  43. [44]

    Kouvaritakis, M

    B. Kouvaritakis, M. Cannon, Model predictive control, Switzerland: Springer International Publishing 38 (13-56) (2016) 7

    2016

  44. [45]

    B. Dong, K. P. Lam, A real-time model pre- dictive control for building heating and cool- ing systems based on the occupancy behavior pattern detection and local weather forecast- ing, in: Building Simulation, Vol. 7, Springer, 2014, pp. 89–106

    2014

  45. [46]

    Afram, F

    A. Afram, F. Janabi-Sharifi, Supervisory model predictive controller (MPC) for residen- tial HVAC systems: Implementation and ex- perimentation on archetype sustainable house in toronto, Energy and Buildings 154 (2017) 268–282

    2017

  46. [47]

    Lindelöf, H

    D. Lindelöf, H. Afshari, M. Alisafaee, J. Biswas, M. Caban, X. Mocellin, J. Viaene, Field tests of an adaptive, model-predictive heating controller for residential buildings, En- ergy and Buildings 99 (2015) 292–302

    2015

  47. [48]

    E. N. Pergantis, L. D. Reyes Premer, A. H. Lee, Priyadarshan, H. Liu, E. A. Groll, D. Zi- viani, K. J. Kircher, Protecting residential electrical panels and service through model predictive control: A field study, Applied En- ergy 386 (2025) 125528

    2025

  48. [49]

    D. Kim, J. Braun, J. Cai, D. Fugate, Devel- opment and experimental demonstration of a plug-and-play multiple RTU coordination con- trol algorithm for small/medium commercial buildings, Energy and Buildings 107 (2015) 279–293

    2015

  49. [50]

    S. C. Bengea, A. D. Kelman, F. Borrelli, R. Taylor, S. Narayanan, Implementation of model predictive control for an HVAC system in a mid-size commercial building, HVAC&R Research 20 (1) (2014) 121–135

    2014

  50. [51]

    Sturzenegger, D

    D. Sturzenegger, D. Gyalistras, M. Morari, R. S. Smith, Model predictive climate control of a swiss office building: Implementation, re- sults, and cost–benefit analysis, IEEE Trans- actions on Control Systems Technology 24 (1) (2015) 1–12

    2015

  51. [52]

    Y. Ma, F. Borrelli, B. Hencey, B. Coffey, S. Bengea, P. Haves, Model predictive control for the operation of building cooling systems, IEEE Transactions on control systems technol- ogy 20 (3) (2011) 796–803

    2011

  52. [53]

    Maasoumy, C

    M. Maasoumy, C. Rosenberg, A. Sangiovanni- Vincentelli, D. S. Callaway, Model predic- tive control approach to online computation of demand-sideflexibilityofcommercialbuildings HVAC systems for supply following, in: 2014 American control conference, IEEE, 2014, pp. 1082–1089

    2014

  53. [54]

    S. Zhan, Y. Lei, A. Chong, Comparing model predictive control and reinforcement learn- ing for the optimal operation of building-PV- battery systems, in: E3S Web of Conferences, Vol. 396, EDP Sciences, 2023, p. 04018

    2023

  54. [55]

    Arroyo, F

    J. Arroyo, F. Spiessens, L. Helsen, Compari- son of optimal control techniques for building energy management, Frontiers in Built Envi- ronment 8 (2022) 849754

    2022

  55. [56]

    Stoffel, L

    P. Stoffel, L. Maier, A. Kümpel, T. Schreiber, D. Müller, Evaluation of advanced control strategies for building energy systems, Energy and Buildings 280 (2023) 112709

    2023

  56. [57]

    Tassa, T

    Y. Tassa, T. Erez, E. Todorov, Synthe- sis and stabilization of complex behaviors through online trajectory optimization, in: 2012 IEEE/RSJ International Conference on Intelligent Robots and Systems, IEEE, 2012, pp. 4906–4913

    2012

  57. [58]

    S. Liu, G. P. Henze, Experimental analysis of simulated reinforcement learning control for active and passive building thermal storage inventory: Part 1. theoretical foundation, Energy and Buildings 38 (2) (2006) 142–147. doi:https://doi.org/10.1016/j.enbuild.2005.06.002

    work page doi:10.1016/j.enbuild.2005.06.002 2006

  58. [59]

    E. Atam, L. Helsen, Control-oriented thermal modeling of multizone buildings: Methods and issues: Intelligent control of a building system, IEEE Control systems magazine 36 (3) (2016) 86–111

    2016

  59. [60]

    Ghiaus, Experimental estimation of build- ing energy performance by robust regression, Energy and buildings 38 (6) (2006) 582–587

    C. Ghiaus, Experimental estimation of build- ing energy performance by robust regression, Energy and buildings 38 (6) (2006) 582–587. 28

    2006

  60. [61]

    Hammarsten, A critical appraisal of energy- signature models, Applied Energy 26 (2) (1987) 97–110

    S. Hammarsten, A critical appraisal of energy- signature models, Applied Energy 26 (2) (1987) 97–110

    1987

  61. [62]

    Enescu, A review of thermal comfort mod- els and indicators for indoor environments, Re- newable and Sustainable Energy Reviews 79 (2017) 1353–1379

    D. Enescu, A review of thermal comfort mod- els and indicators for indoor environments, Re- newable and Sustainable Energy Reviews 79 (2017) 1353–1379

    2017

  62. [63]

    Arendt, M

    K. Arendt, M. Jradi, H. R. Shaker, C. Veje, Comparative analysis of white-, gray-and black-box models for thermal simulation of indoor environment: Teaching building case study, in: Building Performance Analysis Conference and SimBuild: Co-organized by ASHRAE and IBPSA-USA, ASHRAE, 2018, pp. 173–180

    2018

  63. [64]

    Vallianos, J

    C. Vallianos, J. Candanedo, A. Athienitis, Ap- plication of a large smart thermostat dataset for model calibration and model predictive control implementation in the residential sec- tor, Energy 278 (2023) 127839

    2023

  64. [65]

    Arroyo, C

    J. Arroyo, C. Manna, F. Spiessens, L. Helsen, An Open-AI gym environment for the Building OptimizationTesting(BOPTEST)framework,

  65. [66]

    E. N. Pergantis, P. Dhillon, L. D. R. Premer, A. H. Lee, D. Ziviani, K. J. Kircher, Humidity- aware model predictive control for residential air conditioning: A field study, Building and Environment 266 (2024) 112093

    2024

  66. [67]

    De Coninck, F

    R. De Coninck, F. Magnusson, J. Åkesson, L. Helsen, Toolbox for development and val- idation of grey-box building models for fore- casting and control, Journal of building per- formance simulation 9 (3) (2016) 288–303

    2016

  67. [68]

    Drgona, A

    J. Drgona, A. R. Tuor, J. V. Koch, M. R. Shapiro, E. King, D. L. Vrabie, Do- main aware deep-learning algorithms inte- grated with scientific-computing technologies (dadaist), Tech. rep., Pacific Northwest Na- tional Laboratory (PNNL), Richland, WA (United States) (2023)

    2023

  68. [69]

    A. K. Prakash, F. De Andrade Pereira, M.Bergés, M.Pritoni, B.Akinci, Ontologiesat work: Analyzing information requirements for model predictive control in buildings, in: Pro- ceedings of the 11th ACM International Con- ference on Systems for Energy-Efficient Build- ings, Cities, and Transportation, 2024, pp. 214–218. 29

    2024

  69. [2021]

    doi:10.26868/25222708.2021.30380

    arXiv 2021

This paper was first reviewed by deepseek-v4-flash on August 4, 2026.