Pith. sign in

REVIEW 5 major objections 5 minor 74 references

Building2Building introduces a benchmark of 6,000 physically simulated HVAC buildings, with a morphology abstraction that lets a single policy transfer across different sensor and actuator layouts.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · deepseek-v4-flash

2026-08-01 20:40 UTC pith:NG7WWWN2

load-bearing objection A credible and useful HVAC RL benchmark with a strong morphology abstraction, but the 6,000-environment headline rests on an unvalidated generator and the paper contains internal contradictions. the 5 major comments →

arxiv 2607.16534 v1 pith:NG7WWWN2 submitted 2026-07-17 cs.LG cs.AI

Building2Building: A Large Scale Benchmark for Generalizable Real-World Reinforcement Learning

Vincent Taboga , Justin Veilleux , Doseok Jang , Anushree Rankawat , Pierre-Luc Bacon This is my paper
classification cs.LG cs.AI
keywords Building2BuildingHVAC controlreinforcement learninggeneralizationtransfer learningbuilding simulationmorphology abstractionbenchmark
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper introduces Building2Building (B2B), a benchmark suite of 6,000 HVAC control environments generated from parametric building models in a high-fidelity building energy simulator. Its central claim is that this scale and diversity—multiple building types, climate zones, and HVAC system configurations, with heterogeneous observation and action spaces—makes it possible to study generalization in continuous-control RL beyond the handful of buildings existing benchmarks provide. To handle that heterogeneity, B2B adds a structured representation: each building is a 'morphology,' a graph of zone and equipment nodes with type-specific observation and action spaces, and a single policy can be trained across morphologies. The paper demonstrates this with a transformer-based policy controlling retail, restaurant, small office, and medium office buildings, which tracks setpoints on unseen buildings and outperforms a scripted reactive controller on some of them. If true, researchers gain a reproducible, physically grounded testbed for transfer, meta-learning, and multi-task RL in an energy-relevant domain.

Core claim

The core discovery is a generative pipeline that turns parametric building models into thousands of RL environments compatible with the standard Python RL interface, combined with a 'morphological universe' abstraction that decomposes an environment into a common part and a per-building graph of typed nodes (thermal zones, HVAC equipment). With this abstraction, a single policy can be trained on environments with different observation and action spaces: the paper shows a transformer with type-specific encoders/decoders controlling four building types and, after 1M environment steps, generalizing to 20 unseen buildings, beating a scripted reactive baseline on several. The paper also defines f

What carries the argument

The parametric building generator, constrained to prescriptive building-energy code bounds for each climate zone, produces the 6,000 models; the morphological universe/morphology abstraction translates each environment into a common space plus a graph of typed nodes with local observation and action spaces; the transformer policy with type-specific encoders/decoders consumes this structure; and the reward normalization (energy divided by baseline mean power per building type and climate) makes a single weight meaningful across environments. Together these convert a fleet of heterogeneous simulators into a single structured learning problem.

Load-bearing premise

The 6,000 generated models are all physically valid and usable; the paper reports no validation statistics, and its own note about avoiding 'extreme HVAC autosizing artifacts' admits that some generated models can be pathological.

What would settle it

Run the full 6,000-model dataset through the building simulator and count models that fail to converge, violate equipment-sizing constraints, or produce physically implausible energy or comfort values; or retrain the cross-domain policy on the entire suite and check whether removing the questionable models changes the reported generalization results.

Watch this falsifier — get emailed when new claim-graph text bears on it.

If this is right

  • RL researchers can evaluate transfer and meta-learning algorithms on thousands of physically simulated buildings instead of a handful, all behind a standard RL interface.
  • A single policy can control buildings with different sensor and actuator layouts without per-building retraining, since the morphology abstraction absorbs the heterogeneity.
  • The four benchmark settings isolate distinct generalization challenges—goals, dynamics, action spaces, and cross-domain—so method comparisons reveal what varies.
  • Normalized rewards mean one energy/comfort trade-off weight can be used across the whole suite, avoiding per-environment reward tuning.
  • Because one multi-morphology policy already outperforms a scripted reactive baseline on some unseen buildings, the suite offers a practical target for generalizable HVAC controllers.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • The morphology abstraction is not specific to buildings; the same universe-plus-graph design could be applied to other fleets of heterogeneous continuous-control systems, such as robotic arms or industrial process units, though the paper only demonstrates buildings.
  • The 6,000-environment scale invites tests the paper does not run itself, such as in-context learning from B2B trajectories or pretraining a world model on the whole distribution, which could reveal whether cross-domain generalization scales with dataset size.
  • If the generated models are validated as physically sound, B2B could support 'train on a distribution, deploy on the real building' studies that reduce the need for per-building digital twins, as the paper gestures toward.
  • The climate-stratified test split makes it possible to probe whether failures concentrate in particular climate zones or envelope parameter ranges, which could guide targeted data augmentation—but the paper reports no such breakdown.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

5 major / 5 minor

Summary. The paper introduces B2B, a Gymnasium-compatible benchmark built on EnergyPlus for studying generalization in RL. It combines a parametric commercial-building generator with ASHRAE 90.1 climate-zone bounds and a Quebec residential generator to produce a claimed dataset of 6,000 environments across six building archetypes. It also defines a morphological-universe abstraction for heterogeneous observation/action spaces, four benchmark families (goal adaptation, dynamics adaptation, action-space transfer, cross-domain transfer), and validates the pipeline with ASHRAE-style reactive baselines, PPO specialists, multi-building PPO agents, and an Amorpheus-style transformer policy trained across four building types.

Significance. If the dataset is trustworthy, B2B fills a real gap: existing HVAC RL environments number only a few dozen models (Table 6), and generalization benchmarks are dominated by robotics and video games. The morphological abstraction in Section 3.4 is a clean, implementable way to make heterogeneous spaces addressable by graph/transformer policies, and the open-source release with detailed PPO and baseline hyperparameters (Table 8, Appendix D) supports reproducibility. The cross-domain experiment in Section 6.2, despite limitations, demonstrates the suite can support multi-morphology training. These strengths are substantial and should be credited.

major comments (5)
  1. [Section 4 / Appendix B] The 6,000-environment scale is the paper's core claim, but no generation or simulation validation is reported. Section B states that geometry scaling is 'narrowed to [0.7,1.5] to avoid extreme HVAC autosizing artifacts,' an explicit admission that the generator can produce pathological models. Yet no counts are given of EnergyPlus convergence failures, autosizing warnings, out-of-bound zone temperatures, or episodes that fail to complete through the Gymnasium wrapper. The experiments in Sections 5 and 6.1 use 8 buildings per type, and Section 6.2 uses 20 unseen buildings, so they do not certify the full pool. I request a validation table over all 6,000 models (generation success rate, simulation convergence rate, autosizing warnings, physical plausibility checks) and a statement of how many environments, if any, were discarded.
  2. [Section 6.1 / Table 3] Dynamics-adaptation Setting 1 is described as varying 'Climate, building envelope parameters,' but the same section states that 'For single-zone houses, which all belong to the same climate zone, 8 buildings are randomly sampled from the test set.' The house generator in Section 3.1 is a Quebec residential stock model, so all 1,000 houses share one climate. The dynamics-adaptation experiment therefore evaluates envelope/geometry variation only, not climate variation as claimed in Table 3. The text should correct this discrepancy or extend the generator to multiple ASHRAE climate zones.
  3. [Section 6.1 / Figure 2a] The text claims 'The gap between the non-specialist and parameterized models shows the benefit of conditioning on building parameters for dynamics adaptation.' In the figure as rendered, the non-specialist and parameterized bars appear nearly indistinguishable, and no numerical medians or confidence intervals are given for these two bars in the main text. The claim is load-bearing for the dynamics-adaptation conclusion. Please report the exact medians and per-seed standard errors (as described in Section F.2) or revise the claim if the difference is not significant.
  4. [Section F.1 / Section 5] The reactive baseline hyperparameters are 'tuned on the test buildings using Bayesian optimization' and one set is used per building type and climate zone. Since the normalized score in Section 5 divides by the reactive baseline's return, and Table 9's 'Beats' counts also compare to this baseline, tuning the baseline on the test set leaks test information into the evaluation and makes the baseline artificially strong. This affects the interpretation of every reported normalized score. Please tune the baseline on the training split, use fixed Guideline-36 defaults, or report sensitivity of the normalized scores to the baseline tuning.
  5. [Section E] The reward normalizer tau_E is defined as the mean per-step consumption of the reactive baseline policy for a given building type and climate zone, and w_E=0.5 is justified solely by observed non-degeneracy on a small test set. This makes the reward family and the benchmark tasks dependent on a single baseline and on informal test-set inspection. The claim that 'a single weight w_E may be used across environments' is therefore not validated beyond the small test set. Please provide a principled normalization protocol (e.g., based on ASHRAE loads) or a development-set procedure for selecting w_E, and report how sensitive the benchmark conclusions are to this choice.
minor comments (5)
  1. [Table 1] The table lists 'Meta-World (McLean et al., 2025)' but the cited work is Meta-World+; the original Meta-World (Yu et al., 2020) should be referenced as well.
  2. [Section 3.3 / Section E] Section 3.3 says tau_T and tau_E 'depend on the building type and the performances of a baseline policy,' but Section E fixes tau_T=1. The sentence should be corrected to say only tau_E depends on the baseline.
  3. [Section 6.2] The sentence 'Training data are collected in parallel on four buildings (one of each type) during winter. Buildings are resampled every 10 PPO updates...' is ambiguous: does the resampling occur over the full training set or only among the four running instances? Clarifying this is important for reproducing the multi-morphology experiment.
  4. [Section 3.1 / Abstract] The abstract and Section 4 emphasize the finite dataset of 6,000 environments, while Section 3.1 describes 'a virtually infinite number of environments.' The latter overstates the finite dataset and should be qualified.
  5. [Section C / Table 7] The text says 'An EnergyPlus timestep takes about 0.1ms,' but Table 7 reports means from 0.021 to 0.410 ms/step depending on building type. Citing the table range would be more accurate.

Circularity Check

0 steps flagged

No circular derivation found; the paper is a benchmark/dataset contribution whose experimental claims are empirical and not forced by construction.

full rationale

The paper is a benchmark/dataset contribution rather than a formal derivation. Its load-bearing claims — the 6000-environment suite, the task definitions, the morphology abstraction, and the held-out PPO generalization results — do not reduce by construction to their inputs. The reward normalization τ_E is defined as the baseline reactive policy's mean per-step consumption (Section E) and the normalized score divides by the reactive return, but this is a scale-normalization design choice: it sets the units and trade-off scale of the reward and does not determine the trained policies' outcomes. The choice of w_E = 0.5 based on observed non-degenerate behavior on the small test set is benchmark hyperparameter selection, not a fitted input called a prediction, and none of the reported performances are derived from that selection by construction. The cited building generators (Larochelle Martin et al., 2026; Shirzadi, 2025) are external works with no author overlap, so no load-bearing self-citation chain exists. Appendix B's statement that geometry scaling is narrowed to [0.7,1.5] to avoid extreme HVAC autosizing artifacts, together with the absence of validation statistics for the 6000 generated models, is a correctness/validation limitation of the dataset claim, not a circularity: it does not make any result equivalent to its assumptions. The dynamics-adaptation and cross-domain experiments compare held-out buildings to a reactive baseline; the measured gaps are empirical outcomes, not identities forced by the benchmark definition.

Axiom & Free-Parameter Ledger

4 free parameters · 6 axioms · 2 invented entities

The central claim — a usable, large-scale, diverse HVAC RL testbed — draws its legitimacy from EnergyPlus fidelity, ASHRAE-constrained generation, and two building generators; none are independently audited in the preprint. The benchmark's reward design adds fitted quantities (τ_E from the baseline's own consumption; w_E chosen on the test set) that the paper itself shows do not fully deliver. The morphology abstraction is the one genuinely new construct, and it carries its own evidence in Section 6.2.

free parameters (4)
  • τ_E (energy reward normalizer) = Mean per-step power consumption of the baseline reactive policy, per building type and climate zone
    Defined in Section 3.3 / Section E to normalize the energy reward term; fitted to baseline behavior, and the paper shows it does not equalize trade-offs across building types.
  • w_E (energy weight in reward) = 0.5
    Chosen because it 'yields no degenerate policies on every building and climate zones of the small test set' (Section E) — a benchmark hyperparameter selected using test buildings.
  • τ_T (comfort reward normalizer) = 1
    Fixed by hand: 'A difference of 1°C ... we thus fix τ_T = 1' (Section E).
  • Reactive-controller gains (K_p, K_i, Δ_trim, Δ_resp, deadband) = Tuned per building type and climate zone via Bayesian optimization on test buildings
    Section F.1: baseline hyperparameters are optimized on the test buildings for 25th-percentile performance; a baseline tuned on the test set is a weak comparison anchor.
axioms (6)
  • domain assumption EnergyPlus simulations are faithful enough proxies for real building thermal dynamics that policies trained in them transfer to physical buildings.
    Section 1, citing Zhang et al. (2019); the benchmark's real-world relevance rests on this.
  • domain assumption All 6,000 auto-generated building models simulate successfully and are physically valid (no autosizing failure, no numerical instability).
    Sections 3.1/B — the generator only narrows geometry to [0.7,1.5] 'to avoid extreme HVAC autosizing artifacts'; no failure-rate or validity statistics are reported.
  • domain assumption The residential building generator (Larochelle Martin et al., 2026) produces models representative of the Quebec housing stock.
    Section 3.1; the house archetype is taken wholesale from this external generator.
  • domain assumption ASHRAE 90.1-2022 climate-zone parameter bounds guarantee realistic envelope and HVAC sizing for sampled parameters.
    Section B; sampling bounds are derived from ASHRAE Tables 5.5-1 through 5.5-8.
  • domain assumption The reactive controller (ASHRAE Guideline 36-inspired PI + Trim-and-Respond) is a fair performance baseline.
    Sections 5/D; baselines were tuned on the test buildings via Bayesian optimization (Section F.1), which weakens the baseline's independence.
  • standard math Standard RL/ML background: PPO converges reliably with the given hyperparameters, and the transformer policy operates correctly on the morphology abstraction.
    Sections F.2/F.3; standard assumptions from cited literature (Stable-Baselines3, Amorpheus).
invented entities (2)
  • Morphological universe / morphology decomposition independent evidence
    purpose: Abstraction splitting each environment's observation and action spaces into a common part plus a typed graph, enabling one policy across heterogeneous observation/action spaces.
    Instantiated in the claimed B2B code and exercised by the cross-domain Amorpheus-variant policy in Section 6.2 / Figure 3, which is a falsifiable handle: the policy either controls unseen building types or it does not.
  • Parametric building generator independent evidence
    purpose: Samples 1,000 environments per building archetype from ASHRAE-constrained parameter ranges; the load-bearing tool behind the 6,000-environment claim.
    Produces concrete artifacts (6,000 models, Table 7 step times, baseline runs in Section F.1) that can be checked against EnergyPlus.

pith-pipeline@v1.3.0-alltime-deepseek · 17417 in / 17572 out tokens · 170951 ms · 2026-08-01T20:40:51.406464+00:00 · methodology

0 comments
read the original abstract

Reinforcement learning (RL) has achieved strong results in control, yet learned policies remain brittle to changes in dynamics, action spaces, observation spaces, or goals, a critical limitation for real-world deployment. Existing benchmarks offer limited diversity and complexity, making it difficult to rigorously study transfer, multi-task learning, and meta-learning in RL. We introduce Building2Building (B2B), a large-scale suite of realistic Heating, Ventilation, and Air Conditioning (HVAC) control environments built on EnergyPlus, a state-of-the-art building simulator. B2B is fully compatible with the Gymnasium interface and features a parametric building generator, enabling the systematic generation of diverse building configurations with heterogeneous observation and action spaces. Based on this suite, we define benchmark tasks targeting key open challenges in RL, including goal adaptation, dynamics adaptation, action-space shifts, and cross-domain transfer. By providing a large-scale, diverse, and physically grounded testbed with standardized evaluation protocols, B2B enables systematic investigation of generalization and transfer in continuous control. Beyond advancing research on generalization in RL, this new benchmark also carries significant societal implications by enabling improved HVAC control at scale, one of the most energy-intensive systems in buildings.

Figures

Figures reproduced from arXiv: 2607.16534 by Anushree Rankawat, Doseok Jang, Justin Veilleux, Pierre-Luc Bacon, Vincent Taboga.

Figure 1
Figure 1. Figure 1: Overview of the benchmark design. A large building pool is created using our building [PITH_FULL_IMAGE:figures/full_fig_p006_1.png] view at source ↗
Figure 2
Figure 2. Figure 2: Performances of three types of models for dynamics adaptation [PITH_FULL_IMAGE:figures/full_fig_p008_2.png] view at source ↗
Figure 3
Figure 3. Figure 3: Generalized policy test performance on 20 unseen buildings sampled from the test set. [PITH_FULL_IMAGE:figures/full_fig_p009_3.png] view at source ↗
Figure 4
Figure 4. Figure 4: Temperature distribution for different energy penalty weights [PITH_FULL_IMAGE:figures/full_fig_p019_4.png] view at source ↗
Figure 5
Figure 5. Figure 5: Energy consumption for different energy penalty weights [PITH_FULL_IMAGE:figures/full_fig_p020_5.png] view at source ↗
Figure 6
Figure 6. Figure 6: Per step reward distribution aggregated over the 8 buildings of the small test set for constant [PITH_FULL_IMAGE:figures/full_fig_p020_6.png] view at source ↗
Figure 7
Figure 7. Figure 7: Per step reward distribution aggregated over the 8 buildings of the small test set for occu [PITH_FULL_IMAGE:figures/full_fig_p021_7.png] view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

74 extracted references · 3 canonical work pages

  1. [1]

    Journal of Building Performance Simulation , volume =

    David Blum and Javier Arroyo and Sen Huang and Ján Drgoňa and Filip Jorissen and Harald Taxt Walnum and Yan Chen and Kyle Benne and Draguna Vrabie and Michael Wetter and Lieve Helsen , title =. Journal of Building Performance Simulation , volume =. 2021 , publisher =. doi:10.1080/19401493.2021.1986574 , URL =

    arXiv 2021

  2. [2]

    Energy and Buildings , volume =

    Sinergym – A virtual testbed for building energy optimization with Reinforcement Learning , author =. Energy and Buildings , volume =. 2025 , issn =. doi:10.1016/j.enbuild.2024.115075 , url =

    arXiv 2025

  3. [3]

    2025 , eprint=

    The Smart Buildings Control Suite: A Diverse Open Source Benchmark to Evaluate and Scale HVAC Control Policies for Sustainability , author=. 2025 , eprint=

    2025

  4. [4]

    Araújo , title =

    Shengyi Huang and Rousslan Fernand Julien Dossa and Chang Ye and Jeff Braga and Dipam Chakraborty and Kinal Mehta and João G.M. Araújo , title =. Journal of Machine Learning Research , year =

  5. [5]

    IEA World Energy Outlook report 2022 , howpublished =

    IEA , year =. IEA World Energy Outlook report 2022 , howpublished =

    2022

  6. [6]

    A practical deep reinforcement learning framework for multivariate occupant-centric control in buildings , journal =

    Yue Lei and Sicheng Zhan and Eikichi Ono and Yuzhen Peng and Zhiang Zhang and Takamasa Hasama and Adrian Chong , keywords =. A practical deep reinforcement learning framework for multivariate occupant-centric control in buildings , journal =. 2022 , issn =. doi:https://doi.org/10.1016/j.apenergy.2022.119742 , url =

    arXiv 2022

  7. [7]

    Reinforcement Learning Testbed for Power-Consumption Optimization

    Moriyama, Takao and De Magistris, Giovanni and Tatsubori, Michiaki and Pham, Tu-Hoa and Munawar, Asim and Tachibana, Ryuki. Reinforcement Learning Testbed for Power-Consumption Optimization. Methods and Applications for Modeling and Simulation of Complex Systems. 2018

    2018

  8. [8]

    and Adams, Mark and Bass, Brett and Berres, Anne and Clinton, Nicholas , title =

    New, Joshua R. and Adams, Mark and Bass, Brett and Berres, Anne and Clinton, Nicholas , title =. 2021 , publisher =. doi:10.13139/ORNLNCCS/1774134 , url =

    arXiv 2021

  9. [9]

    Journal of Building Performance Simulation , volume =

    Nweye, Kingsley and Kaspar, Kathryn and Buscemi, Giacomo and Fonseca, Tiago and Pinto, Giuseppe and Ghose, Dipanjan and Duddukuru, Satvik and Pratapa, Pavani and Li, Han and Mohammadi, Javad and Lino Ferreira, Luis and Hong, Tianzhen and Ouf, Mohamed and Capozzoli, Alfonso and Nagy, Zoltan , title =. Journal of Building Performance Simulation , volume =. ...

    arXiv 2024

  10. [10]

    Applied Sciences , VOLUME =

    Scharnhorst, Paul and Schubnel, Baptiste and Fernández Bandera, Carlos and Salom, Jaume and Taddeo, Paolo and Boegli, Max and Gorecki, Tomasz and Stauffer, Yves and Peppas, Antonis and Politi, Chrysa , TITLE =. Applied Sciences , VOLUME =. 2021 , NUMBER =

    2021

  11. [11]

    GitHub repository , howpublished =

    Yihao Sun , title =. GitHub repository , howpublished =. 2023 , publisher =

    2023

  12. [12]

    2024 , eprint=

    Gymnasium: A Standard Interface for Reinforcement Learning Environments , author=. 2024 , eprint=

    2024

  13. [13]

    Automatic building energy model development and debugging using large language models agentic workflow , journal =

    Liang Zhang and Vitaly Ford and Zhelun Chen and Jianli Chen , keywords =. Automatic building energy model development and debugging using large language models agentic workflow , journal =. 2025 , issn =. doi:https://doi.org/10.1016/j.enbuild.2024.115116 , url =

    arXiv 2025

  14. [14]

    Applied Energy , volume=

    Reinforcement learning in building energy management: A review , author=. Applied Energy , volume=. 2022 , publisher=

    2022

  15. [15]

    Energies , volume=

    Applications of reinforcement learning in HVAC control systems: A systematic review , author=. Energies , volume=. 2024 , publisher=

    2024

  16. [16]

    Proceedings of the International Conference on Autonomous Agents and Multiagent Systems (AAMAS) , year=

    Experimental evaluation of reinforcement learning algorithms for energy-efficient HVAC control in Sinergym , author=. Proceedings of the International Conference on Autonomous Agents and Multiagent Systems (AAMAS) , year=

  17. [17]

    Enhancing HVAC control systems through transfer learning with deep reinforcement learning agents , journal =

    Kevlyn Kadamala and Des Chambers and Enda Barrett , keywords =. Enhancing HVAC control systems through transfer learning with deep reinforcement learning agents , journal =. 2024 , issn =. doi:https://doi.org/10.1016/j.segy.2024.100131 , url =

    arXiv 2024

  18. [18]

    arXiv preprint arXiv:2008.03625 , year=

    One-for-many: Transfer learning for building HVAC control under varying configurations , author=. arXiv preprint arXiv:2008.03625 , year=

    Pith/arXiv arXiv 2008

  19. [19]

    Building Simulation , volume=

    Multi-source transfer learning for multi-zone building HVAC control with deep reinforcement learning , author=. Building Simulation , volume=. 2024 , publisher=

    2024

  20. [20]

    arXiv preprint arXiv:2503.19212 , year=

    Continual learning for HVAC control: A hypernetwork-based reinforcement learning approach , author=. arXiv preprint arXiv:2503.19212 , year=

    Pith/arXiv arXiv

  21. [21]

    arXiv preprint arXiv:2503.05022 , year=

    Methodological shortcomings in field demonstrations of advanced building control: A systematic review , author=. arXiv preprint arXiv:2503.05022 , year=

    arXiv

  22. [22]

    Thirty-seventh Conference on Neural Information Processing Systems , year=

    Supervised Pretraining Can Learn In-Context Reinforcement Learning , author=. Thirty-seventh Conference on Neural Information Processing Systems , year=

  23. [23]

    An Enhanced Adaptivity of Reinforcement Learning-Based Temperature Control in Buildings Using Generalized Training , year=

    Taboga, Vincent and Bellahsen, Amine and Dagdougui, Hanane , journal=. An Enhanced Adaptivity of Reinforcement Learning-Based Temperature Control in Buildings Using Generalized Training , year=

  24. [24]

    International Conference on Learning Representations , year=

    Cross-Domain Imitation Learning via Optimal Transport , author=. International Conference on Learning Representations , year=

  25. [25]

    2025 , url=

    In-Context Reinforcement Learning From Suboptimal Historical Data , author=. 2025 , url=

    2025

  26. [26]

    Successor Features for Transfer in Reinforcement Learning , url =

    Barreto, Andre and Dabney, Will and Munos, Remi and Hunt, Jonathan J and Schaul, Tom and van Hasselt, Hado P and Silver, David , booktitle =. Successor Features for Transfer in Reinforcement Learning , url =

  27. [27]

    2017 , month =

    NREL , title =. 2017 , month =

    2017

  28. [28]

    Whole building energy model for HVAC optimal control: A practical framework based on deep reinforcement learning , journal =

    Zhiang Zhang and Adrian Chong and Yuqi Pan and Chenlu Zhang and Khee Poh Lam , keywords =. Whole building energy model for HVAC optimal control: A practical framework based on deep reinforcement learning , journal =. 2019 , issn =. doi:https://doi.org/10.1016/j.enbuild.2019.07.029 , url =

    work page doi:10.1016/j.enbuild.2019.07.029 2019

  29. [29]

    Buildings , volume=

    Surrogate Modeling for Building Design: Energy and Cost Prediction Compared to Simulation-Based Methods , author=. Buildings , volume=. 2025 , publisher=. doi:10.3390/buildings15132361 , url=

    work page doi:10.3390/buildings15132361 2025

  30. [30]

    Proceedings of the 34th International Conference on Machine Learning , pages =

    Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networks , author =. Proceedings of the 34th International Conference on Machine Learning , pages =. 2017 , editor =

    2017

  31. [31]

    2016 , eprint=

    RL ^2 : Fast Reinforcement Learning via Slow Reinforcement Learning , author=. 2016 , eprint=

    2016

  32. [32]

    2022 , eprint=

    MetaMorph: Learning Universal Controllers with Transformers , author=. 2022 , eprint=

    2022

  33. [33]

    Proceedings of eSim 2026: 14th Conference of IBPSA-Canada , organization =

    Larochelle Martin, Gilbert and Delcroix, Benoit and Le Lostec, Brice and Hosseini, Seyedsaeid and Mbaye, Aziz and Coillot, Mike and Sansregret, Simon , title =. Proceedings of eSim 2026: 14th Conference of IBPSA-Canada , organization =

    2026

  34. [34]

    Scientific Reports , year =

    Xu, Shichao and Fu, Yangyang and Wang, Yixuan and Yang, Zhuoran and Huang, Chao and O'Neill, Zheng and Wang, Zhaoran and Zhu, Qi , title =. Scientific Reports , year =. doi:10.1038/s41598-025-91326-z , url =

    work page doi:10.1038/s41598-025-91326-z

  35. [35]

    2022 , editor =

    Trabucco, Brandon and Phielipp, Mariano and Berseth, Glen , booktitle =. 2022 , editor =

    2022

  36. [36]

    Gradient Surgery for Multi-Task Learning , url =

    Yu, Tianhe and Kumar, Saurabh and Gupta, Abhishek and Levine, Sergey and Hausman, Karol and Finn, Chelsea , booktitle =. Gradient Surgery for Multi-Task Learning , url =

  37. [37]

    The Thirty-ninth Annual Conference on Neural Information Processing Systems , year=

    Bigger, Regularized, Categorical: High-Capacity Value Functions are Efficient Multi-Task Learners , author=. The Thirty-ninth Annual Conference on Neural Information Processing Systems , year=

  38. [38]

    Bigger, Regularized, Optimistic: scaling for compute and sample efficient continuous control , url =

    Nauman, Michal and Ostaszewski, Mateusz and Jankowski, Krzysztof and Mi o\'. Bigger, Regularized, Optimistic: scaling for compute and sample efficient continuous control , url =. Advances in Neural Information Processing Systems , doi =

  39. [39]

    Embodied intelligence via learning and evolution , volume=

    Gupta, Agrim and Savarese, Silvio and Ganguli, Surya and Fei-Fei, Li , year=. Embodied intelligence via learning and evolution , volume=. Nature Communications , publisher=. doi:10.1038/s41467-021-25874-z , number=

    doi:10.1038/s41467-021-25874-z

  40. [40]

    The Twelfth International Conference on Learning Representations , year=

    Multi-Task Reinforcement Learning with Mixture of Orthogonal Experts , author=. The Twelfth International Conference on Learning Representations , year=

  41. [41]

    2018 , eprint=

    A Simple Neural Attentive Meta-Learner , author=. 2018 , eprint=

    2018

  42. [42]

    2019 , eprint=

    Efficient Off-Policy Meta-Reinforcement Learning via Probabilistic Context Variables , author=. 2019 , eprint=

    2019

  43. [43]

    2020 , eprint=

    VariBAD: A Very Good Method for Bayes-Adaptive Deep RL via Meta-Learning , author=. 2020 , eprint=

    2020

  44. [44]

    2017 , eprint=

    Distral: Robust Multitask Reinforcement Learning , author=. 2017 , eprint=

    2017

  45. [45]

    2018 , eprint=

    IMPALA: Scalable Distributed Deep-RL with Importance Weighted Actor-Learner Architectures , author=. 2018 , eprint=

    2018

  46. [46]

    2018 , eprint=

    Multi-task Deep Reinforcement Learning with PopArt , author=. 2018 , eprint=

    2018

  47. [47]

    2021 , eprint=

    Offline Reinforcement Learning as One Big Sequence Modeling Problem , author=. 2021 , eprint=

    2021

  48. [48]

    2022 , eprint=

    In-context Reinforcement Learning with Algorithm Distillation , author=. 2022 , eprint=

    2022

  49. [49]

    Improving Generalization for Temporal Difference Learning: The Successor Representation , year=

    Dayan, Peter , journal=. Improving Generalization for Temporal Difference Learning: The Successor Representation , year=

  50. [50]

    Proceedings of the 32nd International Conference on Machine Learning , pages =

    Universal Value Function Approximators , author =. Proceedings of the 32nd International Conference on Machine Learning , pages =. 2015 , editor =

    2015

  51. [51]

    2018 , eprint=

    Hindsight Experience Replay , author=. 2018 , eprint=

    2018

  52. [52]

    Are We Learning Yet? A Meta Review of Evaluation Failures Across Machine Learning , url =

    Liao, Thomas and Taori, Rohan and Raji, Deborah and Schmidt, Ludwig , booktitle =. Are We Learning Yet? A Meta Review of Evaluation Failures Across Machine Learning , url =

  53. [53]

    , journal=

    James, Stephen and Ma, Zicong and Arrojo, David Rovick and Davison, Andrew J. , journal=. RLBench: The Robot Learning Benchmark & Learning Environment , year=

  54. [54]

    Meta-World+: An Improved, Standardized,

    Reginald McLean and Evangelos Chatzaroulas and Luc McCutcheon and Frank R. Meta-World+: An Improved, Standardized,. Championing Open-source DEvelopment in ML Workshop @ ICML25 , year=

  55. [55]

    Software Impacts , volume =

    Saran Tunyasuvunakool and Alistair Muldal and Yotam Doron and Siqi Liu and Steven Bohez and Josh Merel and Tom Erez and Timothy Lillicrap and Nicolas Heess and Yuval Tassa , keywords =. Software Impacts , volume =. 2020 , issn =. doi:10.1016/j.simpa.2020.100022 , url =

    arXiv 2020

  56. [56]

    2019 , eprint=

    Challenges of Real-World Reinforcement Learning , author=. 2019 , eprint=

    2019

  57. [57]

    Journal of Artificial Intelligence Research , volume =

    A Survey of Multi-Objective Sequential Decision-Making , author =. Journal of Artificial Intelligence Research , volume =. 2013 , doi =

    2013

  58. [58]

    Reward Function Design in Reinforcement Learning for

    Togashi, Eisuke , journal =. Reward Function Design in Reinforcement Learning for. 2025 , doi =

    2025

  59. [59]

    Autonomous Agents and Multi-Agent Systems , volume =

    A Practical Guide to Multi-Objective Reinforcement Learning and Planning , author =. Autonomous Agents and Multi-Agent Systems , volume =. 2022 , doi =

    2022

  60. [60]

    Advances in Neural Information Processing Systems (NeurIPS) , volume =

    Pareto Multi-Task Learning , author =. Advances in Neural Information Processing Systems (NeurIPS) , volume =

  61. [61]

    Advances in Neural Information Processing Systems (NeurIPS) , volume =

    Learning Values Across Many Orders of Magnitude , author =. Advances in Neural Information Processing Systems (NeurIPS) , volume =

  62. [62]

    Multi-Task Deep Reinforcement Learning with

    Hessel, Matteo and Soyer, Hubert and Espeholt, Lasse and Czarnecki, Wojciech and Schmitt, Simon and van Hasselt, Hado , booktitle =. Multi-Task Deep Reinforcement Learning with. 2019 , doi =

    2019

  63. [63]

    2023 , eprint=

    ManiSkill2: A Unified Benchmark for Generalizable Manipulation Skills , author=. 2023 , eprint=

    2023

  64. [64]

    2020 , eprint=

    Leveraging Procedural Generation to Benchmark Reinforcement Learning , author=. 2020 , eprint=

    2020

  65. [65]

    arXiv preprint arXiv:2009.12293 , year=

    robosuite: A Modular Simulation Framework and Benchmark for Robot Learning , author=. arXiv preprint arXiv:2009.12293 , year=

    Pith/arXiv arXiv 2009

  66. [66]

    2019 , eprint=

    MineRL: A Large-Scale Dataset of Minecraft Demonstrations , author=. 2019 , eprint=

    2019

  67. [67]

    2022 , eprint=

    Habitat 2.0: Training Home Assistants to Rearrange their Habitat , author=. 2022 , eprint=

    2022

  68. [68]

    Journal of Machine Learning Research , year =

    Antonin Raffin and Ashley Hill and Adam Gleave and Anssi Kanervisto and Maximilian Ernestus and Noah Dormann , title =. Journal of Machine Learning Research , year =

  69. [69]

    VAEM: a Deep Generative Model for Heterogeneous Mixed Type Data , url =

    Ma, Chao and Tschiatschek, Sebastian and Turner, Richard and Hern\'. VAEM: a Deep Generative Model for Heterogeneous Mixed Type Data , url =. Advances in Neural Information Processing Systems , editor =

  70. [70]

    2021 , eprint=

    Decision Transformer: Reinforcement Learning via Sequence Modeling , author=. 2021 , eprint=

    2021

  71. [71]

    The Fourteenth International Conference on Learning Representations , year=

    Learning Massively Multitask World Models for Continuous Control , author=. The Fourteenth International Conference on Learning Representations , year=

  72. [72]

    2010.01856 , archivePrefix =

    Vitaly Kurin and Maximilian Igl and Tim Rocktäschel and Wendelin Boehmer and Shimon Whiteson , title =. 2010.01856 , archivePrefix =

    Pith/arXiv arXiv 2010

  73. [73]

    2408.01230 , archivePrefix =

    YiFan Hao and Yang Yang and Junru Song and Wei Peng and Weien Zhou and Tingsong Jiang and Wen Yao , title =. 2408.01230 , archivePrefix =

    Pith/arXiv arXiv

  74. [74]

    2015 , eprint=

    Contextual Markov Decision Processes , author=. 2015 , eprint=

    2015