{"id":"58c4f157-e41e-419a-9b17-fbf89988b304","arxiv_id":"2411.19746","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":6,"one_line_summary":"A decision-pretrained transformer that controls HVAC dampers in-context reduced simulated annual energy use by about 31% versus a fixed baseline in one unseen building.","lead":"This paper applies a decision-pretrained transformer to multi-zone HVAC control, training it on records of reinforcement-learning agents and deploying it in a new simulated building without retraining. In a one-year EnergyPlus simulation, it used about 31% less energy than a fixed 50% damper baseline and came close to an expert-designed controller.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Without comfort measurements, the reported energy savings could be damper-closing in disguise: the Section 2 reward excludes thermal comfort, so the efficiency claim is not yet established.","rationale":"I read the paper as a DPT-for-HVAC proof of concept with a clear pipeline and internally consistent simulation tables. The most load-bearing weakness is not the transformer methodology but the objective: Section 2 defines the reward as negative VAV energy with no comfort term, and the paper reports no comfort metrics in Section 4 or Table 2. Because the action directly controls damper opening, the cheapest policy under this reward is often to restrict airflow, and the reported energy reductions could therefore reflect reward misspecification rather than improved control. This is internal to the paper's setup, not a disagreement with external consensus. I also note the abstract's 45% figure is arithmetically inconsistent with Table 2: the actual reduction from Baseline is 31.3%, while the 45.62% in Table 3 is a percentage difference relative to HVAC-DPT, not relative to Baseline. That error matters for the headline's precision, but it is secondary to the comfort issue, because even a corrected 31.3% saving is only meaningful if comfort is held fixed. The proposed comfort-check experiment would settle the concern, so the appropriate stance remains conditional acceptance pending comfort-aware reporting.","tokens_in":5591,"tokens_out":6261,"duration_ms":59058,"concrete_test":"Re-run the year-long BDenver evaluation with HVAC-DPT, Baseline, and Expert in the same EnergyPlus model, logging each zone's dry-bulb temperature at every 15-minute step. Compute the fraction of occupied hours outside ASHRAE 55 comfort bounds (or a comparable comfort metric such as PMV/PPD) for each controller. If HVAC-DPT shows materially more comfort violations than Baseline or Expert, the energy savings are a comfort trade-off; if comfort performance is comparable, the concern is resolved.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim is that HVAC-DPT reduces energy in unseen buildings, but the reward used for training every agent is defined in Section 2 as negative VAV energy with no thermal-comfort, humidity, or indoor-air-quality term. The action space is the VAV minimum damper position (0 to 1). A policy maximizing this reward can simply drive dampers toward closed whenever doing so reduces fan or coil energy, regardless of resulting zone temperatures or occupant comfort. The same comfort-free reward is used for the PPO policy library (Appendix B), for the action labels in pretraining, and for the online in-context feedback, so nothing in the pipeline penalizes comfort degradation. EnergyPlus itself may regulate supply-air temperature and reheat coil power (Appendix A), but that does not guarantee zone comfort if dampers are closed, and the paper reports no zone temperatures, PMV/PPD, or comfort-violation statistics. Consequently, the reported savings (45% in the abstract; 31.3% if computed correctly from Table 2) cannot be attributed to intelligent control rather than to sacrificing occupant comfort. Since the claim is framed as an efficiency gain, this missing control makes the headline uninterpretable.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The manuscript proposes HVAC-DPT, a decision-pretrained transformer for multi-zone HVAC control. The method trains a policy library of PPO agents on a source building, builds in-context datasets from their interactions, and trains a GPT-2-style transformer to predict actions for a query state conditioned on those histories. At deployment, the model is applied without weight updates to a new building. The main reported result is that HVAC-DPT reduces HVAC energy consumption by 45% compared to a fixed 50%-damper baseline and approaches the performance of a building-specific expert controller in a one-year EnergyPlus simulation.","tokens_in":5831,"tokens_out":4635,"duration_ms":39370,"significance":"If substantiated, the approach would be a valuable step toward generalizable HVAC controllers: it promises deployment in unseen buildings without retraining or site-specific data collection, and it uses in-context reinforcement learning in a practical application domain. The use of a standard building simulator, a comparison against a baseline and an expert controller, and the explicitly frozen-parameter deployment are strengths. However, the paper's own tables do not support the headline reduction, the reward used for all agents omits thermal comfort, and the evaluation covers only one target building, so the significance as currently evidenced is limited.","major_comments":[{"comment":"The claim of a '45%' reduction compared to the Baseline is not supported by the paper's data. Summing the monthly values in Table 2, the Baseline consumes 730.76 MWh/year and HVAC-DPT consumes 501.84 MWh/year. That is a 31.3% reduction relative to the Baseline, not 45%. Table 3 reports '+45.62%' for the Baseline relative to HVAC-DPT, which is a different comparison base. Similarly, the conclusion's '70% ... compared to RL agents' misstates the relation: Table 3 shows MARL is 70.56% higher than HVAC-DPT, which corresponds to HVAC-DPT being 41.4% lower than MARL, not 70% lower. Please correct these claims and state the comparison base explicitly wherever percentages are given.","section":"Abstract, §4, Fig. 2, Table 3"},{"comment":"The reward for every agent is the negative VAV energy consumption with no thermal-comfort, humidity, or indoor-air-quality term, and the action is the VAV minimum damper position in [0,1]. Under this reward, simply closing dampers reduces measured VAV energy, regardless of zone temperatures or occupant comfort. The paper reports no zone-temperature, PMV/PPD, or comfort-violation statistics, and Appendix C contains only an energy table. Consequently, the reported energy savings cannot currently be attributed to intelligent control rather than to destroying comfort. Please add comfort metrics to the evaluation or state that the energy figures are conditional on a comfort constraint that is verified in the simulations.","section":"§2, §3, Appendix B"},{"comment":"The evaluation is based on a single target building (BDenver), and the energy values are averages over 10 runs with no variance or confidence intervals. This makes it impossible to assess the statistical significance of the improvement or to support the claim that the method 'generalise[s] effectively across buildings.' Please report run-to-run dispersion and evaluate on at least one additional target building. Also clarify the number of training buildings N in Algorithm 1 and whether the 100-policy library was trained on Btrain alone.","section":"§4, Appendix B"}],"minor_comments":[{"comment":"Appendix C consists only of Table 3 with no accompanying text or analysis; it should be referenced and discussed in the main text.","section":"Appendix C"},{"comment":"Reference [1] contains a typographical artifact ('202ß'); the year should be corrected.","section":"References"},{"comment":"The term 'HVAC' is inconsistently typeset as 'HV AC' in many places, including the title and abstract.","section":"General"},{"comment":"In line 14, the notation 'ˆpj(·)' uses an undefined index j; the per-element prediction over the in-context dataset should be clarified.","section":"§3, Algorithm 1"},{"comment":"The episode length is inconsistent: the PPO batch size is 2,976 'corresponding to the length of one episode,' while the pretraining section states a horizon of 2,967 steps. Please reconcile these values.","section":"Appendix B"},{"comment":"Figures 1, 2, and 3 are referenced in the text but do not appear in the manuscript text; please ensure all figures are included and legible.","section":"Figures"}],"recommendation":"major_revision","confidential_remarks":"The manuscript is more in the style of an extended abstract than a full journal paper. The central idea is timely, but the headline numbers need correction, the comfort-free reward raises a serious validity concern, and the evaluation is too thin for the strong generalization claims. These issues are addressable in a revision, but the claims as written are not currently supported."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Quick take: HVAC-DPT is the first application of decision-pretrained transformers to multi-zone HVAC control, with a clean per-zone formulation that lets the same pretrained model handle buildings with different zone counts. That part is real and worth knowing. But the headline number is wrong, the evaluation is thin, and the reward definition leaves the energy savings uninterpretable.\n\nWhat's new: the per-zone MARL-style framing of DPT, where each zone gets its own in-context history and the model outputs a damper position. The authors train a PPO policy library on a small office, pretrain a GPT-2 to mimic those labels, then deploy on a medium office in EnergyPlus without weight updates. That's a legitimate extension of Lee et al., and the EnergyPlus setup follows [6] faithfully.\n\nWhere it falls short: First, the abstract and Section 4 say 45% energy reduction, but Table 2/3 shows the baseline is 45.6% higher than HVAC-DPT, which is a 31.3% reduction. Just arithmetic, but it's the headline, so it needs fixing. Second, one target building, one weather file, 10 runs averaged with no variance. Generalization across 'unseen buildings' is not demonstrated. Third, and most important: the reward is negative VAV energy with no comfort term (Section 2). The action is the minimum damper position. A policy that closes dampers can look great on energy while starving zones of ventilation. The paper reports no zone temperatures, PMV, or complaint proxies. Appendix A's note that EnergyPlus regulates supply air temperature doesn't guarantee zone comfort when the agent controls dampers. So the efficiency claim is not yet established as efficiency rather than comfort trade-off.\n\nThe citation pattern is fine; the prior work is real and the building-simulation baselines are standard.\n\nWho it's for: someone working on in-context RL for building control. It's a plausible stepping stone, but as written it's a workshop-level result with a misleading abstract. If it comes to a journal/conference, send it to review — a referee can push for corrected math, multi-building evaluation, and comfort-aware reporting. It shouldn't be desk-rejected, but it needs major revision before the central claim can be trusted.","headline":"First DPT-for-HVAC paper with a clean per-zone idea, but the headline 45% is a misread of its own table and the comfort-free reward makes the savings uninterpretable.","tokens_in":6338,"tokens_out":3518,"would_cite":false,"duration_ms":28047,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"The paper claims that a decision-pretrained transformer can control multi-zone HVAC systems in buildings it has never seen, with no parameter updates, using about 45% less energy than a fixed 50%-open damper baseline in simulation.","keywords":["HVAC control","in-context reinforcement learning","decision-pretrained transformer","multi-zone buildings","energy efficiency","zero-shot generalisation","multi-agent reinforcement learning","building simulation"],"falsifier":"Log the zone mean temperatures sampled by HVAC-DPT in the EnergyPlus medium-office test building at 15-minute intervals and compare them with the baseline controller's temperatures over the same weather year; if HVAC-DPT keeps occupied zones outside the ASHRAE 55 comfort band for a meaningful share of occupied hours, the energy savings are partly a comfort trade-off rather than pure efficiency. Retraining the policy library with a comfort-inclusive reward and checking whether the 45% reduction shrinks would settle the same question in one experiment.","tokens_in":5378,"feed_emoji":"🏢","tokens_out":12395,"duration_ms":86606,"temperature":0.7,"pith_summary":"This paper tries to establish that a single pretrained transformer can control the heating, ventilation, and air conditioning of a building it has never encountered, without retraining or collecting new data. The method, HVAC-DPT, reformulates HVAC control as next-action prediction: a causal transformer is trained on interaction histories collected from diverse reinforcement-learning agents in a small office building, and at deployment it conditions on the new building's own recent interactions to keep refining its policy. In a year-long EnergyPlus simulation of an unseen 15-zone medium office, HVAC-DPT consumed about 45% less HVAC energy than a 50%-open damper baseline and came within roughly 6% of a building-specific expert controller. A sympathetic reader would care because this applies in-context reinforcement learning to HVAC control for the first time, and it promises to remove the two main barriers to adoption: building-specific modelling and the long, comfort-degrading training phase of RL. The evidence is entirely simulated, and the paper defers real-world validation to future work.","feed_headline":"Transformer cuts HVAC energy 45% in buildings it has never seen","feed_subtitle":"A decision-pretrained transformer adapts in context, so no retraining or building-specific data is needed.","key_machinery":"The load-bearing mechanism is the Decision-Pretrained Transformer (DPT): a causal GPT-2-style transformer trained by supervised learning to predict the action label $a^\\star$ sampled from a diverse policy library, conditioned on a query state $s_{\\text{query}}$ and an in-context dataset $D$ of transition tuples. Because actions are continuous, the training loss is the mean squared error between the predicted and labelled actions. The per-zone decomposition into identical action spaces is what lets one network serve buildings of different sizes, and the in-context dataset $D$ is what lets the policy adapt during deployment without any parameter updates.","core_discovery":"The central claim is that in-context reinforcement learning, realised as a decision-pretrained transformer, transfers across buildings whose state and action spaces differ. HVAC-DPT assigns one agent per variable-air-volume (VAV) zone, so the per-zone action is always a single damper position even when the number of zones, building size, or HVAC configuration changes. During pretraining the transformer learns to map a query state plus a history of state-action-reward tuples to the action that a trained PPO agent would have taken; at deployment the same network, with frozen parameters, consumes the stream of interactions collected online in the target building and thereby adapts its behaviour. The reported result is that this zero-update policy beats freshly trained single-agent and multi-agent RL controllers during the first year in an unseen building and approaches the hand-tuned expert controller.","pith_inferences":["Because the training reward is the negative energy consumption of the VAV system with no thermal comfort term, the 45% saving may partly come from holding dampers more closed than comfort allows; the paper reports no zone-temperature or comfort measurements, so the headline should be read as an energy-only claim.","A direct test: rerun the same protocol with a comfort penalty in the reward (e.g., a cap on zone temperature deviation or an ASHRAE 55 metric) and measure how much of the 45% reduction survives; the amount that survives is the genuinely useful efficiency gain.","The paper trains on one small Denver office and tests on one medium Denver office, so how much of the success is due to in-context adaptation versus shared climate is unclear; a multi-climate, multi-building evaluation would separate the two.","The paper itself defers real-world validation; EnergyPlus dynamics are not the same as live building physics, so the first deployment in a physical building should be treated as the decisive experiment rather than a formality."],"forward_implications":["Deploying HVAC-DPT in a new building reduces to initialising an empty context buffer and running forward passes; no per-building training or data collection is required.","Because each zone is an independent agent with the same action space, the same pretrained model can be attached to any number of zones, including buildings with more zones than any seen during training.","In the reported year-long simulation, HVAC-DPT's energy use is within about 6% of the building-specific expert controller and substantially below both freshly trained RL baselines.","If the result carries across climates and building types, in-context RL offers a concrete route to lowering the large share of global energy consumed by HVAC systems without waiting for per-building models."],"supporting_citations":[{"why":"Supplies the Decision-Pretrained Transformer supervised pretraining objective that HVAC-DPT adapts to HVAC control.","marker":"[12]"},{"why":"Provides the multi-agent RL formulation, the policy-library training procedure, and the RL baselines HVAC-DPT is compared against.","marker":"[6]"},{"why":"EnergyPlus is the physics simulator used both to train the policy library and to run the year-long evaluation.","marker":"[17]"},{"why":"COBS is the comprehensive building simulator wrapper used to train the diverse PPO policy library.","marker":"[18]"},{"why":"Defines the Baseline, Expert, SARL, and MARL controllers that serve as the comparison points for the reported savings.","marker":"[19]"},{"why":"Defines the ASHRAE prototype building models used for the training building and the unseen test building.","marker":"[20]"}],"fun_headline_variants":["Zero retraining: HVAC transformer adapts in context, cuts energy 45%","In-context RL lets HVAC control transfer with no extra training","Decision-pretrained transformer reduces HVAC energy 45% in new sites","HVAC-DPT: one pretrained policy, any building, 45% less energy"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The reward used to train every agent measures only the VAV system's energy consumption, with no term for thermal comfort, humidity, or indoor air quality, so the reported savings could be achieved by letting occupants become uncomfortably hot or cold.","fun_headline_variants_meta":{"raw":{"variants":["Zero retraining: HVAC transformer adapts in context, cuts energy 45%","In-context RL lets HVAC control transfer with no extra training","Decision-pretrained transformer reduces HVAC energy 45% in new sites","HVAC-DPT: one pretrained policy, any building, 45% less energy"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000612,"raw_usage":{"total_tokens":2819,"prompt_tokens":890,"completion_tokens":1929,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":506,"completion_tokens_details":{"reasoning_tokens":1846}},"tokens_in":506,"tokens_out":1929,"duration_ms":15852,"temperature":1.0,"reasoning_tokens":1846,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-12T05:52:53.721284+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Log the zone mean temperatures sampled by HVAC-DPT in the EnergyPlus medium-office test building at 15-minute intervals and compare them with the baseline controller's temperatures over the same weather year; if HVAC-DPT keeps occupied zones outside the ASHRAE 55 comfort band for a meaningful share of occupied hours, the energy savings are partly a comfort trade-off rather than pure efficiency. Retraining the policy library with a comfort-inclusive reward and checking whether the 45% reduction shrinks would settle the same question in one experiment.","supporting_citations":[{"cited_title":"Mitigating an adoption barrier of reinforcement learning-based control strategies in buildings","cited_arxiv_id":null,"evidence_quote":"Provides the multi-agent RL formulation, the policy-library training procedure, and the RL baselines HVAC-DPT is compared against."},{"cited_title":"Energyplus: creating a new-generation building energy simulation program","cited_arxiv_id":null,"evidence_quote":"EnergyPlus is the physics simulator used both to train the policy library and to run the year-long evaluation."},{"cited_title":"Cobs: Comprehensive building simulator","cited_arxiv_id":null,"evidence_quote":"COBS is the comprehensive building simulator wrapper used to train the diverse PPO policy library."},{"cited_title":"Alternative approaches to hvac control of chat generative pre-trained transformer (chatgpt) for autonomous building system operations","cited_arxiv_id":null,"evidence_quote":"Defines the Baseline, Expert, SARL, and MARL controllers that serve as the comparison points for the reported savings."},{"cited_title":"American Society of Heating, A.-C","cited_arxiv_id":null,"evidence_quote":"Defines the ASHRAE prototype building models used for the training building and the unseen test building."}],"review_version":1}