Pith. sign in

REVIEW 3 major objections 6 minor 20 references

HVAC-DPT: A Decision Pretrained Transformer for HVAC Control

T0 review · 3 major / 6 minor · reviewed 2026-08-12 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read The paper claims that a decision-pretrained transformer can control multi-zone HVAC systems in buildings it has never seen, with no parameter updates, using about 45% less energy than a fixed 50%-open damper baseline in simulation.

desk verdict First DPT-for-HVAC paper with a clean per-zone idea, but the headline 45% is a misread of its own table and the comfort-free reward makes the savings uninterpretable. read the letter →

arxiv 2411.19746 v1 pith:UHLVMP3M submitted 2024-11-29 cs.LG cs.AIcs.MA

Anaïs Berkes This is my paper · ORCID
classification cs.LGcs.AIcs.MA
keywords HVACcontrolin-contextreinforcementlearningdecision-pretrainedtransformermulti-zonebuildingsenergyefficiencyzero-shotgeneralisationmulti-agentbuildingsimulation
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper tries to establish that a single pretrained transformer can control the heating, ventilation, and air conditioning of a building it has never encountered, without retraining or collecting new data. The method, HVAC-DPT, reformulates HVAC control as next-action prediction: a causal transformer is trained on interaction histories collected from diverse reinforcement-learning agents in a small office building, and at deployment it conditions on the new building's own recent interactions to keep refining its policy. In a year-long EnergyPlus simulation of an unseen 15-zone medium office, HVAC-DPT consumed about 45% less HVAC energy than a 50%-open damper baseline and came within roughly 6% of a building-specific expert controller. A sympathetic reader would care because this applies in-context reinforcement learning to HVAC control for the first time, and it promises to remove the two main barriers to adoption: building-specific modelling and the long, comfort-degrading training phase of RL. The evidence is entirely simulated, and the paper defers real-world validation to future work.

What carries the argument

The load-bearing mechanism is the Decision-Pretrained Transformer (DPT): a causal GPT-2-style transformer trained by supervised learning to predict the action label $a^\star$ sampled from a diverse policy library, conditioned on a query state $s_{\text{query}}$ and an in-context dataset $D$ of transition tuples. Because actions are continuous, the training loss is the mean squared error between the predicted and labelled actions. The per-zone decomposition into identical action spaces is what lets one network serve buildings of different sizes, and the in-context dataset $D$ is what lets the policy adapt during deployment without any parameter updates.

What would settle it

Log the zone mean temperatures sampled by HVAC-DPT in the EnergyPlus medium-office test building at 15-minute intervals and compare them with the baseline controller's temperatures over the same weather year; if HVAC-DPT keeps occupied zones outside the ASHRAE 55 comfort band for a meaningful share of occupied hours, the energy savings are partly a comfort trade-off rather than pure efficiency. Retraining the policy library with a comfort-inclusive reward and checking whether the 45% reduction shrinks would settle the same question in one experiment.

Watch

Extended reading notes

Core claim

The central claim is that in-context reinforcement learning, realised as a decision-pretrained transformer, transfers across buildings whose state and action spaces differ. HVAC-DPT assigns one agent per variable-air-volume (VAV) zone, so the per-zone action is always a single damper position even when the number of zones, building size, or HVAC configuration changes. During pretraining the transformer learns to map a query state plus a history of state-action-reward tuples to the action that a trained PPO agent would have taken; at deployment the same network, with frozen parameters, consumes the stream of interactions collected online in the target building and thereby adapts its behaviour. The reported result is that this zero-update policy beats freshly trained single-agent and multi-agent RL controllers during the first year in an unseen building and approaches the hand-tuned expert controller.

Load-bearing premise

The reward used to train every agent measures only the VAV system's energy consumption, with no term for thermal comfort, humidity, or indoor air quality, so the reported savings could be achieved by letting occupants become uncomfortably hot or cold.

Editorial extensions

If this is right

  • Deploying HVAC-DPT in a new building reduces to initialising an empty context buffer and running forward passes; no per-building training or data collection is required.
  • Because each zone is an independent agent with the same action space, the same pretrained model can be attached to any number of zones, including buildings with more zones than any seen during training.
  • In the reported year-long simulation, HVAC-DPT's energy use is within about 6% of the building-specific expert controller and substantially below both freshly trained RL baselines.
  • If the result carries across climates and building types, in-context RL offers a concrete route to lowering the large share of global energy consumed by HVAC systems without waiting for per-building models.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Because the training reward is the negative energy consumption of the VAV system with no thermal comfort term, the 45% saving may partly come from holding dampers more closed than comfort allows; the paper reports no zone-temperature or comfort measurements, so the headline should be read as an energy-only claim.
  • A direct test: rerun the same protocol with a comfort penalty in the reward (e.g., a cap on zone temperature deviation or an ASHRAE 55 metric) and measure how much of the 45% reduction survives; the amount that survives is the genuinely useful efficiency gain.
  • The paper trains on one small Denver office and tests on one medium Denver office, so how much of the success is due to in-context adaptation versus shared climate is unclear; a multi-climate, multi-building evaluation would separate the two.
  • The paper itself defers real-world validation; EnergyPlus dynamics are not the same as live building physics, so the first deployment in a physical building should be treated as the decisive experiment rather than a formality.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 6 minor

Summary. The manuscript proposes HVAC-DPT, a decision-pretrained transformer for multi-zone HVAC control. The method trains a policy library of PPO agents on a source building, builds in-context datasets from their interactions, and trains a GPT-2-style transformer to predict actions for a query state conditioned on those histories. At deployment, the model is applied without weight updates to a new building. The main reported result is that HVAC-DPT reduces HVAC energy consumption by 45% compared to a fixed 50%-damper baseline and approaches the performance of a building-specific expert controller in a one-year EnergyPlus simulation.

Significance. If substantiated, the approach would be a valuable step toward generalizable HVAC controllers: it promises deployment in unseen buildings without retraining or site-specific data collection, and it uses in-context reinforcement learning in a practical application domain. The use of a standard building simulator, a comparison against a baseline and an expert controller, and the explicitly frozen-parameter deployment are strengths. However, the paper's own tables do not support the headline reduction, the reward used for all agents omits thermal comfort, and the evaluation covers only one target building, so the significance as currently evidenced is limited.

major comments (3)
  1. [Abstract, §4, Fig. 2, Table 3] The claim of a '45%' reduction compared to the Baseline is not supported by the paper's data. Summing the monthly values in Table 2, the Baseline consumes 730.76 MWh/year and HVAC-DPT consumes 501.84 MWh/year. That is a 31.3% reduction relative to the Baseline, not 45%. Table 3 reports '+45.62%' for the Baseline relative to HVAC-DPT, which is a different comparison base. Similarly, the conclusion's '70% ... compared to RL agents' misstates the relation: Table 3 shows MARL is 70.56% higher than HVAC-DPT, which corresponds to HVAC-DPT being 41.4% lower than MARL, not 70% lower. Please correct these claims and state the comparison base explicitly wherever percentages are given.
  2. [§2, §3, Appendix B] The reward for every agent is the negative VAV energy consumption with no thermal-comfort, humidity, or indoor-air-quality term, and the action is the VAV minimum damper position in [0,1]. Under this reward, simply closing dampers reduces measured VAV energy, regardless of zone temperatures or occupant comfort. The paper reports no zone-temperature, PMV/PPD, or comfort-violation statistics, and Appendix C contains only an energy table. Consequently, the reported energy savings cannot currently be attributed to intelligent control rather than to destroying comfort. Please add comfort metrics to the evaluation or state that the energy figures are conditional on a comfort constraint that is verified in the simulations.
  3. [§4, Appendix B] The evaluation is based on a single target building (BDenver), and the energy values are averages over 10 runs with no variance or confidence intervals. This makes it impossible to assess the statistical significance of the improvement or to support the claim that the method 'generalise[s] effectively across buildings.' Please report run-to-run dispersion and evaluate on at least one additional target building. Also clarify the number of training buildings N in Algorithm 1 and whether the 100-policy library was trained on Btrain alone.
minor comments (6)
  1. [Appendix C] Appendix C consists only of Table 3 with no accompanying text or analysis; it should be referenced and discussed in the main text.
  2. [References] Reference [1] contains a typographical artifact ('202ß'); the year should be corrected.
  3. [General] The term 'HVAC' is inconsistently typeset as 'HV AC' in many places, including the title and abstract.
  4. [§3, Algorithm 1] In line 14, the notation 'ˆpj(·)' uses an undefined index j; the per-element prediction over the in-context dataset should be clarified.
  5. [Appendix B] The episode length is inconsistent: the PPO batch size is 2,976 'corresponding to the length of one episode,' while the pretraining section states a horizon of 2,967 steps. Please reconcile these values.
  6. [Figures] Figures 1, 2, and 3 are referenced in the text but do not appear in the manuscript text; please ensure all figures are included and legible.

Circularity Check

0 steps flagged · score 0.0 of 10

No circular derivation: HVAC-DPT's energy saving is an external simulation measurement against fixed baselines on a held-out building with frozen weights.

full rationale

The derivation chain is self-contained. The pretraining objective (Eq. 1) is supervised action prediction on PPO interaction histories, and the deployment evaluation measures EnergyPlus energy totals on BDenver, a building not used in pretraining, with model parameters frozen. The reward in Section 2 (negative VAV energy) is the optimization target, not a fitted constant used to produce the reported energy figures; reporting energy reductions under that objective is the intended empirical claim, not a tautology. The 45% figure is arithmetically derived from Table 2/Table 3 relative to the DPT total (baseline is +45.6% relative to DPT, while DPT is 31.3% lower relative to baseline); this is a reporting/presentation issue, not a circular derivation. No load-bearing result is imported from the same authors' prior work: references [6], [12], and [18] are external, and the target building differs structurally from training buildings. The paper itself notes that real-world validation is future work, which is an external-validity caveat rather than a circularity issue. The main validity risk is reward misspecification (no thermal-comfort term), which is a correctness concern about whether energy savings come at the expense of comfort, not a circularity concern.

Assumptions & free parameters 6 free parameters · 4 assumptions · 0 invented entities

The central claim rests on a set of manually chosen hyperparameters, a single training building, and a reward that ignores comfort. No parameter is fitted to the test building, so the circularity burden is low, but the generalization claim is supported by only one held-out simulation.

free parameters (6)
  • Baseline damper position = 0.5 (50% open)
    The headline energy savings are computed relative to this fixed baseline; a different baseline would change the reported improvement.
  • Number of PPO policies in library = 100
    The pretraining dataset is generated from 100 diverse PPO policies; this number is chosen by hand and affects diversity and dataset size.
  • PPO training episodes = 1,000
    Each policy is trained for 1,000 month-long episodes, determining the quality of the action labels the transformer learns to imitate.
  • Transformer hyperparameters = 3 layers, 8 heads, embedding 128, dropout 0.0, weight decay 0.0001, lr 0.001, 118 epochs
    These pretraining hyperparameters are chosen by hand and influence the learned policy; no ablation or sensitivity analysis is provided.
  • Episode horizon = One month at 15-minute steps (2,976 steps; pretraining context 2,967)
    The context length for in-context learning and the definition of an episode are fixed choices that affect what the transformer can learn from history.
  • Training and testing weather = Training: January 1991; testing: year 2000
    Weather conditions are fixed and differ between training and test; the test weather year is chosen rather than varied.
assumptions (4)
  • domain assumption The reward for each zone is the negative VAV energy consumption only; thermal comfort, humidity, and indoor air quality are not part of the objective or evaluation.
    If damper closures reduce energy without a comfort penalty, savings may come at the cost of occupant comfort; no comfort metric is reported. This assumption is load-bearing for the claim that the energy reduction is beneficial.
  • domain assumption EnergyPlus is an accurate model of real building thermal dynamics and HVAC equipment.
    All claims are based on simulation; transfer to real buildings is not demonstrated.
  • domain assumption A policy library trained on one small office building with one weather month provides enough diversity for the transformer to generalize to a different building with a different number of zones.
    The entire generalization claim rests on this transfer being representative; only one target building is tested.
  • domain assumption Supervised pretraining on PPO action labels yields an in-context RL policy that improves with its own rollouts at deployment.
    The DPT method assumes that conditioning on the history of past transitions allows the transformer to refine its policy in context; this is inherited from Lee et al. and is not re-derived or verified in this paper.

how reviews work

0 comments
Cite this review

Pith. "Pith review of HVAC-DPT: A Decision Pretrained Transformer for HVAC Control." pith.science (2026). https://pith.science/paper/UHLVMP3M

@misc{pith2026241119746,
  author       = {Pith},
  title        = {Pith review of: HVAC-DPT: A Decision Pretrained Transformer for HVAC Control},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/UHLVMP3M}},
  note         = {Machine review of arXiv:2411.19746}
}
read the original abstract

Building operations consume approximately 40% of global energy, with Heating, Ventilation, and Air Conditioning (HVAC) systems responsible for up to 50% of this consumption. As HVAC energy demands are expected to rise, optimising system efficiency is crucial for reducing future energy use and mitigating climate change. Existing control strategies lack generalisation and require extensive training and data, limiting their rapid deployment across diverse buildings. This paper introduces HVAC-DPT, a Decision-Pretrained Transformer using in-context Reinforcement Learning (RL) for multi-zone HVAC control. HVAC-DPT frames HVAC control as a sequential prediction task, training a causal transformer on interaction histories generated by diverse RL agents. This approach enables HVAC-DPT to refine its policy in-context, without modifying network parameters, allowing for deployment across different buildings without the need for additional training or data collection. HVAC-DPT reduces energy consumption in unseen buildings by 45% compared to the baseline controller, offering a scalable and effective approach to mitigating the increasing environmental impact of HVAC systems.

Figures

Figures reproduced from arXiv: 2411.19746 by the authors.

Figure 1
Figure 1. Schematic overview of the proposed methodology. [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 2
Figure 2. HVAC energy consumption (MWh) of differ [PITH_FULL_IMAGE:figures/full_fig_p004_2.png] view at source ↗
Figure 3
Figure 3. Illustration of an air loop in a multi-zone building equipped with a forced-air heating and [PITH_FULL_IMAGE:figures/full_fig_p006_3.png] view at source ↗

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

20 extracted references · 13 canonical work pages

  1. [6]

    Mitigating an adoption barrier of reinforcement learning-based control strategies in buildings

    Aakash Krishna GS, Tianyu Zhang, Omid Ardakanian, and Matthew E Taylor. Mitigating an adoption barrier of reinforcement learning-based control strategies in buildings. Energy and Buildings, 285:112878, 2023

    work page 2023

  2. [1]

    In focus: Energy efficiency in buildings, 202ß

    European Commission. In focus: Energy efficiency in buildings, 202ß

    work page

  3. [2]

    A review on buildings energy consumption information

    Luis Pérez-Lombard, José Ortiz, and Christine Pout. A review on buildings energy consumption information. Energy and buildings, 40(3):394–398, 2008. 4

    work page 2008

  4. [3]

    Cooling the buildings–past, present and future

    Mat Santamouris. Cooling the buildings–past, present and future. Energy and Buildings , 128:617–638, 2016

    work page 2016

  5. [4]

    All you need to know about model predictive control for buildings

    Ján Drgoˇna, Javier Arroyo, Iago Cupeiro Figueroa, David Blum, Krzysztof Arendt, Donghun Kim, Enric Perarnau Ollé, Juraj Oravec, Michael Wetter, Draguna L Vrabie, et al. All you need to know about model predictive control for buildings. Annual Reviews in Control, 50:190–232, 2020

    work page 2020

  6. [5]

    Reinforcement learning for building controls: The opportunities and challenges

    Zhe Wang and Tianzhen Hong. Reinforcement learning for building controls: The opportunities and challenges. Applied Energy, 269:115036, 2020

    work page 2020

  7. [7]

    On the joint control of multiple building systems with reinforcement learning

    Tianyu Zhang, Gaby Baasch, Omid Ardakanian, and Ralph Evins. On the joint control of multiple building systems with reinforcement learning. In Proceedings of the Twelfth ACM International Conference on Future Energy Systems, pages 60–72, 2021

    work page 2021

  8. [8]

    Emergence of in-context reinforcement learning from noise distillation

    Ilya Zisman, Vladislav Kurenkov, Alexander Nikulin, Viacheslav Sinii, and Sergey Kolesnikov. Emergence of in-context reinforcement learning from noise distillation. arXiv preprint arXiv:2312.12275, 2023

    arXiv 2023

Show all 20 references
  1. [9]

    Diversity for transfer in learning-based control of buildings

    Tianyu Zhang, Mohammad Afshari, Petr Musilek, Matthew E Taylor, and Omid Ardakanian. Diversity for transfer in learning-based control of buildings. In Proceedings of the Thirteenth ACM International Conference on Future Energy Systems, pages 556–564, 2022

    2022

  2. [10]

    Attention is all you need

    A Vaswani. Attention is all you need. Advances in Neural Information Processing Systems, 2017

    2017

  3. [11]

    In-context reinforcement learning for variable action spaces

    Viacheslav Sinii, Alexander Nikulin, Vladislav Kurenkov, Ilya Zisman, and Sergey Kolesnikov. In-context reinforcement learning for variable action spaces. arXiv preprint arXiv:2312.13327, 2023

    2023 arXiv

  4. [12]

    Supervised pretraining can learn in-context reinforcement learning

    Jonathan Lee, Annie Xie, Aldo Pacchiano, Yash Chandak, Chelsea Finn, Ofir Nachum, and Emma Brunskill. Supervised pretraining can learn in-context reinforcement learning. Advances in Neural Information Processing Systems, 36, 2024

    2024

  5. [13]

    In-context reinforcement learning with algorithm distillation

    Michael Laskin, Luyu Wang, Junhyuk Oh, Emilio Parisotto, Stephen Spencer, Richie Steiger- wald, DJ Strouse, Steven Hansen, Angelos Filos, Ethan Brooks, et al. In-context reinforcement learning with algorithm distillation. arXiv preprint arXiv:2210.14215, 2022

    arXiv

  6. [14]

    Decision transformer as a foundation model for partially observable continuous control

    Xiangyuan Zhang, Weichao Mao, Haoran Qiu, and Tamer Ba¸ sar. Decision transformer as a foundation model for partially observable continuous control. arXiv preprint arXiv:2404.02407, 2024

    2024 arXiv

  7. [15]

    Pretraining decision transformers with reward prediction for in-context multi-task structured bandit learning

    Subhojyoti Mukherjee, Josiah P Hanna, Qiaomin Xie, and Robert Nowak. Pretraining decision transformers with reward prediction for in-context multi-task structured bandit learning. arXiv preprint arXiv:2406.05064, 2024

    2024

  8. [16]

    Transformers as decision makers: Provable in-context reinforcement learning via supervised pretraining

    Licong Lin, Yu Bai, and Song Mei. Transformers as decision makers: Provable in-context reinforcement learning via supervised pretraining. arXiv preprint arXiv:2310.08566, 2023

    2023 arXiv

  9. [17]

    Energyplus: creating a new-generation building energy simulation program

    Drury B Crawley, Linda K Lawrie, Frederick C Winkelmann, Walter F Buhl, Y Joe Huang, Curtis O Pedersen, Richard K Strand, Richard J Liesen, Daniel E Fisher, Michael J Witte, et al. Energyplus: creating a new-generation building energy simulation program. Energy and buildings, ...

    2001

  10. [18]

    Cobs: Comprehensive building simulator

    Tianyu Zhang and Omid Ardakanian. Cobs: Comprehensive building simulator. In Proceedings of the 7th ACM International Conference on Systems for Energy-Efficient Buildings, Cities, and Transportation, pages 314–315, 2020

    2020

  11. [19]

    Alternative approaches to hvac control of chat generative pre-trained transformer (chatgpt) for autonomous building system operations

    Ki Uhn Ahn, Deuk-Woo Kim, Hyun Mi Cho, and Chang-U Chae. Alternative approaches to hvac control of chat generative pre-trained transformer (chatgpt) for autonomous building system operations. Buildings, 13(11):2680, 2023

    2023

  12. [20]

    American Society of Heating, A.-C

    R. American Society of Heating, A.-C. Engineers. Energy Standard for Buildings Except Low- Rise Residential Buildings. ASHRAE Inc, Peachtree Corners, GA, USA, standard 90.1-2019 edition, 2019. 5 A Additional system details Figure 3: Illustration of an air loop in a multi-zone ...

    2019

Pith tools

Reviewed August 12, 2026 · model on record in the stance chip above.