Pith. sign in

REVIEW 3 major objections 8 minor 50 references

Upside-Down Reinforcement Learning for More Interpretable Optimal Control

T0 review · 3 major / 8 minor · reviewed 2026-08-12 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read The paper claims that UDRL's behavior function can be learned by tree ensembles, matching neural network performance on three control benchmarks while offering inherent interpretability.

desk verdict A useful first look at non-neural UDRL, but the headline comparison is confounded by per-model command selection and lacks statistical support. read the letter →

arxiv 2411.11457 v1 pith:YNXJ6W76 submitted 2024-11-18 cs.LG

Juan Cardenas-Cartagena , Massimiliano Falzari , Marco Zullich , Matthia Sabatelli This is my paper · ORCID
classification cs.LG
keywords Upside-DownReinforcementLearningRandomForestsExtremelyRandomizedTreesInterpretabilityFeatureImportanceOptimalControlSupervisedBehaviorFunction
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper asks whether Upside-Down Reinforcement Learning (UDRL) truly requires neural networks as its function approximator, or whether other supervised learners can serve as the behavior function. UDRL recasts control as supervised learning: the agent learns a mapping from a state, a desired reward, and a desired time horizon to an action. On CartPole, Acrobot, and Lunar Lander, the authors test five non-neural learners and report that random forests and extremely randomized trees achieve rewards comparable to a multilayer perceptron, with forests even besting the network on Lunar Lander. The interpretability benefit is that tree ensembles natively provide feature importance scores, so the learned policy can be inspected and audited. This matters because transparent policies are easier to trust in high-stakes control applications.

What carries the argument

The central object is the behavior function f(s, dr, dt) = a, a supervised map from the current state, a desired reward dr, and a desired time horizon dt to the action that supposedly achieves that reward within that horizon. Commands are sampled from recent successful episodes, transitions are stored in a replay buffer, and each learner fits this map from the collected data. The interpretability machinery is the mean impurity decrease feature importance computed from the tree ensembles, which ranks how much each state component and command contributes to the splitting decisions at inference time.

What would settle it

Run the same UDRL agents on Acrobot and Lunar Lander but query every behavior function with a single common (dr, dt) pair, say the values used for the neural network, and measure cumulative reward over 100 episodes. If the random forest and extra-trees scores fall well below the neural network score under this fixed query, the paper's central claim of on-par performance would be contradicted.

Watch

Extended reading notes

Core claim

The central claim is that the UDRL behavior function f(s, dr, dt) -> a does not need to be a neural network: tree-based supervised learners can learn it just as well. On CartPole, Acrobot, and Lunar Lander, the authors find that random forests and extremely randomized trees yield cumulative rewards comparable to a fully connected network after 500 training episodes, with forests performing best on Lunar Lander and XGBoost close behind. Because trees expose mean impurity decrease, the learned policy can be inspected: the paper reports which state variables dominate decisions at different phases of an episode, such as pole angular velocity for CartPole, angular velocities of both links for Acrobot, and the vertical position of the spaceship for Lunar Lander. The authors conclude that UDRL is not a neural-network-only paradigm and that forests offer an intrinsically interpretable alternative for optimal control.

Load-bearing premise

The comparison assumes that querying each behavior function with command values taken from its own recent training episodes — rather than with one fixed, neutral command — is a fair way to compare how well the models control the environment.

Editorial extensions

If this is right

  • UDRL agents can be built with transparent models, removing the black-box nature of neural policies on low-dimensional control tasks.
  • Feature importance scores give a direct explanation of which physical state variables drive a learned controller, without post-hoc explanation methods.
  • Boosting algorithms such as XGBoost are viable behavior functions in UDRL, expanding the toolset beyond tree ensembles.
  • The approach suggests a practical route to policy auditing: inspect the ranking of state features before deploying a controller in a high-stakes setting.
  • K-nearest neighbours performs poorly, indicating that instance-based methods lack the generalisation needed for UDRL behavior functions.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The reported importance scores hint that UDRL controllers discover physically meaningful variables — pole angular velocity for balancing, altitude for landing — which could be used to validate a policy against human intuition before deployment.
  • Extending the same tree-based UDRL recipe to image-based tasks would likely require random subwindows or convolutional ensembles, a direction the authors mention but do not test; a concrete next step is to measure whether such forests scale beyond the three low-dimensional benchmarks.
  • A fixed-query evaluation protocol, where all behavior functions are tested with identical desired-return and horizon commands, would provide a sharper comparison than the per-model commands used here, and would clarify whether tree methods truly match neural networks off-policy.
  • The finding suggests a broader design principle: behavior functions are supervised maps, so any inductive bias that suits tabular data — including gradient boosting — can be dropped into UDRL, potentially making the paradigm a test bed for interpretable control.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 8 minor

Summary. This paper investigates whether non-neural supervised learners can serve as the behavior function in Upside-Down Reinforcement Learning. The authors train six function approximators (an MLP, random forest, extra-trees, AdaBoost, XGBoost, and KNN) on CartPole, Acrobot, and Lunar Lander using a standard UDRL protocol with epsilon-greedy exploration and an experience replay buffer. They report training curves and inference results, and they use impurity-based feature importance to argue that tree-based behavior functions provide interpretable policies. The central claim is that random forests and extra-trees can perform as well as neural networks while being more interpretable.

Significance. The question is timely: UDRL is usually implemented with neural networks, and showing that tabular supervised learners work would extend the paradigm to settings where interpretability matters. The paper's strengths are its clear formulation of UDRL with non-neural regressors, the inclusion of five random seeds, the release of an interactive demo of trained behavior functions, and the feature-importance analyses for three environments. If the main comparison were adequately controlled, the result would be a useful practical finding. As it stands, however, the inference protocol confounds algorithm quality with command selection, and the statistical evidence is too weak to support the headline claim.

major comments (3)
  1. [Section 4, Tables 3 and 4] The central comparison is confounded by the inference-command protocol. For Acrobot and Lunar Lander, each behavior function is queried with the most common (dr, dt) from its own last 100 training episodes; the commands differ across algorithms (e.g., on Acrobot the NN uses f(., -63, 64) while XGBoost uses f(., -114, 120); on Lunar Lander the RF uses f(., 57, 102) while XGBoost uses f(., 136, 389)). Since the behavior function is explicitly conditional on dr and dt, the differences in Table 4 can reflect command difficulty rather than policy quality. The 'perform just as well' claim in the abstract and Section 6 depends on this comparison, so the authors should evaluate all behavior functions on a common set of commands (or a pre-specified fixed protocol) and report results per command.
  2. [Section 4, Table 4] No significance tests are provided, and the standard deviations are large relative to the reported differences. On Acrobot, NN (-75.00 ± 15.36) overlaps with RF (-100.05 ± 62.80) and ET (-100.00 ± 93.72); on Lunar Lander, RF (-54.74 ± 96.22), XGBoost (-76.96 ± 89.69), and NN (-157.04 ± 71.26) all have wide, overlapping intervals. The ranking of the 'best' behavior function and the assertion that tree methods 'perform just as well' therefore require paired statistical tests or confidence intervals over the five seeds; the current evidence is not sufficient.
  3. [Section 5] The interpretability claim is only illustrated, not validated. The feature-importance plots come from a trained RF or ET behavior function, but no stability analysis across seeds is shown, and there is no comparison with post-hoc explanations of the NN baseline. Since the paper's stated contribution is 'more interpretable optimal control,' the authors should at least show that the importances are stable across the five training runs and discuss how faithfully they reflect the policy's decisions.
minor comments (8)
  1. [Table 1] In the caption, 'd f' should be 'dt'.
  2. [Table 4] The caption says 'desired reward dt' but should say 'desired reward dr'.
  3. [Section 4] The text says 'avoid queer-ing the behavior function'; this should be 'avoid querying'.
  4. [Section 3.1] KNN is described as a 'clustering method'; it is an instance-based regression and classification method, not a clustering method.
  5. [Section 3.1] XGBoost is cited to Friedman (2001); the standard reference is Chen and Guestrin (2016).
  6. [Figure 2] Adding shaded confidence bands or per-seed traces would make the convergence claims checkable; the averaged curves alone do not convey run-to-run variance.
  7. [Appendix] The text refers to 'Appendix 6' but the appendix is unnumbered; adding a label would help readers locate the pseudo-code.
  8. [Section 6] For reproducibility, the source code for the UDRL training loop should be released alongside the trained behavior functions.

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity: the UDRL comparison rests on external Gym benchmarks and an independently trained NN baseline, not on a derivation from its own outputs.

full rationale

The paper's central claim—that tree-based regressors such as Random Forests and Extra-Trees can serve as UDRL behavior functions—is supported by training curves on external OpenAI Gym environments and by an NN baseline trained under the same protocol. The behavior function f(st, dr, dt) = at is a standard supervised mapping, and the experiments evaluate the resulting policies against the environments themselves; no step derives a result from the same quantity it claims to predict. The only soft spot is the Section 4 inference protocol, where for Acrobot and Lunar Lander the desired-return command dr and horizon dt are set to the most common values from each algorithm's own last 100 training episodes. This makes the Table 4 comparison a per-model self-consistency check rather than a common-command comparison, which is a validity or fairness concern but not a construction-level circularity: the achieved rewards are not forced by an equation to equal the chosen commands, and the central comparison is not statistically determined by the fitted commands alone. The paper contains no load-bearing self-citations (none of the present authors appear in the reference list) and no imported uniqueness theorem. The stated limitation about simple benchmarks is acknowledged explicitly and does not affect circularity.

Assumptions & free parameters 5 free parameters · 5 assumptions · 0 invented entities

The paper's central claim rests on an experimental comparison, not on a derivation, so the ledger is mostly about benchmark choices and hyperparameters. The most consequential free parameters are the inference command values (dr, dt), which differ per algorithm and are drawn from each model's own training history. There are no invented entities or new theoretical constructs, and the background assumptions are standard supervised learning and MDP assumptions.

free parameters (5)
  • Inference command (dr, dt) for Acrobot and Lunar Lander = Acrobot commands: NN (-63,64), RF (-79,82), ET (-75,77), AdaBoost (-77,81), XGBoost (-114,120), KNN (-129,132).
    Chosen per algorithm from the most common command values in the last 100 training episodes (Section 4, Table 3). This makes the evaluation query a fitted quantity rather than a fixed benchmark setting.
  • Exploration rate epsilon = 0.2 constant
    Hand-chosen in Section 3.3; it controls the mixture of random and behavior-function actions that generate the training data.
  • Replay buffer capacity = 700 transitions
    Hand-set in Section 3.3; the paper does not specify how buffer samples are selected for model updates.
  • Number of training episodes = 500
    Chosen because it is "known to be sufficient" for popular DRL algorithms (Section 3.3); it sets the amount of data available to each behavior function.
  • Baseline network architecture and optimizer = MLP with 3 hidden layers of 64 ReLU units, Adam learning rate 0.001
    Hand-picked in Section 3.3; the NN baseline performance depends on this configuration.
assumptions (5)
  • standard math The MDP formalism and UDRL behavior function f(st, dr, dt) = at are valid.
    The paper builds on Schmidhuber (2019) and the standard MDP framework, Section 2.1.
  • domain assumption OpenAI Gym CartPole, Acrobot, and Lunar Lander are meaningful optimal control benchmarks.
    All conclusions are drawn from these three environments; Section 6 acknowledges they are simple relative to DRL testbeds.
  • domain assumption Scikit-Learn default hyperparameters are appropriate for the non-NN baselines.
    The paper does not tune these models (Section 3.3), so defaults are implicitly assumed to be a fair comparison.
  • domain assumption Mean impurity decrease provides a faithful account of what the policy depends on.
    Section 5 uses Louppe et al. (2013) importances to interpret the behavior functions; the faithfulness of this measure is assumed.
  • ad hoc to paper Commands drawn from the last 100 training episodes are a valid way to query behavior functions at inference.
    Section 4 introduces this protocol; it is not justified externally and differs per algorithm, so the fairness of the comparison rests on it.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Upside-Down Reinforcement Learning for More Interpretable Optimal Control." pith.science (2026). https://pith.science/paper/YNXJ6W76

@misc{pith2026241111457,
  author       = {Pith},
  title        = {Pith review of: Upside-Down Reinforcement Learning for More Interpretable Optimal Control},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/YNXJ6W76}},
  note         = {Machine review of arXiv:2411.11457}
}
read the original abstract

Model-Free Reinforcement Learning (RL) algorithms either learn how to map states to expected rewards or search for policies that can maximize a certain performance function. Model-Based algorithms instead, aim to learn an approximation of the underlying model of the RL environment and then use it in combination with planning algorithms. Upside-Down Reinforcement Learning (UDRL) is a novel learning paradigm that aims to learn how to predict actions from states and desired commands. This task is formulated as a Supervised Learning problem and has successfully been tackled by Neural Networks (NNs). In this paper, we investigate whether function approximation algorithms other than NNs can also be used within a UDRL framework. Our experiments, performed over several popular optimal control benchmarks, show that tree-based methods like Random Forests and Extremely Randomized Trees can perform just as well as NNs with the significant benefit of resulting in policies that are inherently more interpretable than NNs, therefore paving the way for more transparent, safe, and robust RL.

Figures

Figures reproduced from arXiv: 2411.11457 by the authors.

Figure 1
Figure 1. A simple MDP whose behavior function f is sum￾marized in [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 2
Figure 2. Comparison of the performance of the six different tested behavior functions (NN, RF, ET, KNN, AdaBoost, and [PITH_FULL_IMAGE:figures/full_fig_p006_2.png] view at source ↗
Figure 3
Figure 3. Feature importance scores coming from a trained RF behavior function computed for three different states of the [PITH_FULL_IMAGE:figures/full_fig_p008_3.png] view at source ↗
Figures from the paper (2 more)
Figure 4
Figure 4. Figure 4: Feature importance scores coming from a trained ET behavior function computed for three different states of the [PITH_FULL_IMAGE:figures/full_fig_p008_4.png]
Figure 5
Figure 5. Figure 5: Feature importance scores coming from a trained RF behavior function computed for three different states of the [PITH_FULL_IMAGE:figures/full_fig_p009_5.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

50 extracted references · 32 canonical work pages

  1. [1]

    write newline

    " write newline "" before.all 'output.state := FUNCTION fin.entry add.period write newline FUNCTION new.block output.state before.all = 'skip after.block 'output.state := if FUNCTION new.sentence output.state after.block = 'skip output.state before.all = 'skip after.sentence 'output.state := if if FUNCTION not #0 #1 if FUNCTION and 'skip pop #0 if FUNCTIO...

  2. [2]

    and Mishra, S

    Agarwal, S. and Mishra, S. (2021). Responsible AI . Springer

    work page 2021

  3. [3]

    Arrighi, L., Pennella, L., Marques Tavares, G., and Barbon Junior, S. (2024). Decision predicate graphs: Enhancing interpretability in tree ensembles. In World Conference on Explainable Artificial Intelligence , pages 311--332. Springer

    work page 2024

  4. [4]

    R., Schmidhuber, J., and Srivastava, R

    Arulkumaran, K., Ashley, D. R., Schmidhuber, J., and Srivastava, R. K. (2022). All you need is supervised learning: From imitation learning to meta-rl with upside down rl. arXiv preprint arXiv:2202.11960

    arXiv 2022

  5. [5]

    Learning Relative Return Policies With Upside-Down Reinforcement Learning

    Ashley, D. R., Arulkumaran, K., Schmidhuber, J., and Srivastava, R. K. (2022). Learning relative return policies with upside-down reinforcement learning. arXiv preprint arXiv:2202.12742

    work page Pith review arXiv 2022

  6. [6]

    G., Sutton, R

    Barto, A. G., Sutton, R. S., and Anderson, C. W. (1983). Neuronlike adaptive elements that can solve difficult learning control problems. IEEE transactions on systems, man, and cybernetics , (5):834--846

    work page 1983

  7. [7]

    Breiman, L. (2001). Random forests. Machine learning , 45:5--32

    2001

  8. [8]

    Broniatowski, D. A. et al. (2021). Psychological foundations of explainability and interpretability in artificial intelligence. NIST, Tech. Rep

    work page 2021

Show all 50 references
  1. [9]

    Brukhim, N., Hazan, E., and Singh, K. (2022). A boosting approach to reinforcement learning. Advances in Neural Information Processing Systems , 35:33806--33817

    2022

  2. [10]

    Busoniu, L., Babuska, R., De Schutter, B., and Ernst, D. (2017). Reinforcement learning and dynamic programming using function approximators . CRC press

    2017

  3. [11]

    Chen, L., Lu, K., Rajeswaran, A., Lee, K., Grover, A., Laskin, M., Abbeel, P., Srinivas, A., and Mordatch, I. (2021). Decision transformer: Reinforcement learning via sequence modeling. Advances in neural information processing systems , 34:15084--15097

    2021

  4. [12]

    and Hart, P

    Cover, T. and Hart, P. (1967). Nearest neighbor pattern classification. IEEE transactions on information theory , 13(1):21--27

    1967

  5. [13]

    Degrave, J., Felici, F., Buchli, J., Neunert, M., Tracey, B., Carpanese, F., Ewalds, T., Hafner, R., Abdolmaleki, A., de Las Casas, D., et al. (2022). Magnetic control of tokamak plasmas through deep reinforcement learning. Nature , 602(7897):414--419

    2022

  6. [14]

    Ernst, D., Geurts, P., and Wehenkel, L. (2005). Tree-based batch mode reinforcement learning. Journal of Machine Learning Research , 6

    2005

  7. [15]

    Ernst, D., Mar \'e e, R., and Wehenkel, L. (2006a). Reinforcement learning with raw image pixels as input state. In Advances in Machine Vision, Image Processing, and Pattern Analysis: International Workshop on Intelligent Computing in Pattern Analysis/Synthesis, IWICPAS 2006 X...

    2006

  8. [16]

    Ernst, D., Stan, G.-B., Goncalves, J., and Wehenkel, L. (2006b). Clinical data based optimal sti strategies for hiv: a reinforcement learning approach. In Proceedings of the 45th IEEE Conference on Decision and Control , pages 667--672. IEEE

    2006

  9. [17]

    E., et al

    Freund, Y., Schapire, R. E., et al. (1996). Experiments with a new boosting algorithm. In icml , volume 96, pages 148--156. Citeseer

    1996

  10. [18]

    Friedman, J. H. (2001). Greedy function approximation: a gradient boosting machine. Annals of statistics , pages 1189--1232

    2001

  11. [19]

    Furuta, H., Matsuo, Y., and Gu, S. S. (2021). Generalized decision transformer for offline hindsight information matching. arXiv preprint arXiv:2111.10364

    2021 arXiv

  12. [20]

    Geurts, P., Ernst, D., and Wehenkel, L. (2006). Extremely randomized trees. Machine learning , 63:3--42

    2006

  13. [21]

    Glanois, C., Weng, P., Zimmer, M., Li, D., Yang, T., Hao, J., and Liu, W. (2024). A survey on interpretable reinforcement learning. Machine Learning , pages 1--44

    2024

  14. [22]

    HLEG, H.-L. E. G. o. A. (2019). Ethics guidelines for trustworthy ai

    2019

  15. [23]

    Ibrahim, M., Louie, M., Modarres, C., and Paisley, J. (2019). Global explanations of neural networks: Mapping the landscape of predictions. In Proceedings of the 2019 AAAI/ACM Conference on AI, Ethics, and Society , pages 279--287

    2019

  16. [24]

    James, G., Witten, D., Hastie, T., Tibshirani, R., et al. (2013). An introduction to statistical learning , volume 112. Springer

    2013

  17. [25]

    G., Pisane, J., Kolios, A., and Ernst, D

    Leroy, P., Morato, P. G., Pisane, J., Kolios, A., and Ernst, D. (2024). Imp-marl: a suite of environments for large-scale infrastructure management planning via marl. Advances in Neural Information Processing Systems , 36

    2024

  18. [26]

    Liu, M., Zhu, M., and Zhang, W. (2022). Goal-conditioned reinforcement learning: Problems and solutions. arXiv preprint arXiv:2201.08299

    2022 arXiv

  19. [27]

    Louppe, G., Wehenkel, L., Sutera, A., and Geurts, P. (2013). Understanding variable importances in forests of randomized trees. Advances in neural information processing systems , 26

    2013

  20. [28]

    Mar \'e e, R., Geurts, P., and Wehenkel, J. P. L. (2005). Random subwindows for robust image classification. In 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05) , volume 1, pages 34--40. IEEE

    2005

  21. [29]

    A., de Lope, J., and Maravall, D

    Mart \' n H, J. A., de Lope, J., and Maravall, D. (2009). The k nn-td reinforcement learning algorithm. In Methods and Models in Artificial and Natural Computation. A Homage to Professor Mira’s Scientific Legacy: Third International Work-Conference on the Interplay Between Nat...

    2009

  22. [30]

    and Elliott, L

    Min, J. and Elliott, L. T. (2022). Q-learning with online random forests. arXiv preprint arXiv:2204.03771

    2022 arXiv

  23. [31]

    Mirchevska, B., Blum, M., Louis, L., Boedecker, J., and Werling, M. (2017). Reinforcement learning for autonomous maneuvering in highway scenarios. In Workshop for Driving Assistance Systems and Autonomous Driving , pages 32--41

    2017

  24. [32]

    J., and Moyà-Alcover, G

    Miró-Nicolau, M., i Capó, A. J., and Moyà-Alcover, G. (2025). A comprehensive study on fidelity metrics for xai. Information Processing & Management , 62(1):103900

    2025

  25. [33]

    Muschalik, M., Baniecki, H., Fumagalli, F., Kolpaczki, P., Hammer, B., and H\" u llermeier, E. (2024). shapiq: Shapley interactions for machine learning. In The Thirty-eight Conference on Neural Information Processing Systems Datasets and Benchmarks Track

    2024

  26. [34]

    Nauta, M., Trienes, J., Pathak, S., Nguyen, E., Peters, M., Schmitt, Y., Schl \"o tterer, J., Van Keulen, M., and Seifert, C. (2023). From anecdotal evidence to quantitative evaluation methods: A systematic review on evaluating explainable ai. ACM Computing Surveys , 55(13s):1--42

    2023

  27. [35]

    Paszke, A., Gross, S., Chintala, S., Chanan, G., Yang, E., DeVito, Z., Lin, Z., Desmaison, A., Antiga, L., and Lerer, A. (2017). Automatic differentiation in pytorch

    2017

  28. [36]

    Pedregosa, F., Varoquaux, G., Gramfort, A., Michel, V., Thirion, B., Grisel, O., Blondel, M., Prettenhofer, P., Weiss, R., Dubourg, V., et al. (2011). Scikit-learn: Machine learning in python. the Journal of machine Learning research , 12:2825--2830

    2011

  29. [37]

    Prasad, N., Cheng, L.-F., Chivers, C., Draugelis, M., and Engelhardt, B. E. (2017). A reinforcement learning approach to weaning of mechanical ventilation in intensive care units. arXiv preprint arXiv:1704.06300

    2017 arXiv

  30. [38]

    Puterman, M. L. (2014). Markov decision processes: discrete stochastic dynamic programming . John Wiley & Sons

    2014

  31. [39]

    Rudin, C. (2019). Stop explaining black box machine learning models for high stakes decisions and use interpretable models instead. Nature machine intelligence , 1(5):206--215

    2019

  32. [40]

    E., Abdou, M., Perot, E., and Yogamani, S

    Sallab, A. E., Abdou, M., Perot, E., and Yogamani, S. (2017). Deep reinforcement learning framework for autonomous driving. arXiv preprint arXiv:1704.02532

    2017 arXiv

  33. [41]

    Schmidhuber, J. (2019). Reinforcement learning upside down: Don't predict rewards--just map them to actions. arXiv preprint arXiv:1912.02875

    2019 arXiv

  34. [42]

    and Xie, Q

    Shah, D. and Xie, Q. (2018). Q-learning with nearest neighbors. Advances in Neural Information Processing Systems , 31

    2018

  35. [43]

    and Armon, A

    Shwartz-Ziv, R. and Armon, A. (2022). Tabular data: Deep learning is not all you need. Information Fusion , 81:84--90

    2022

  36. [44]

    and Wang, L

    Song, Y. and Wang, L. (2024). Multiobjective tree-based reinforcement learning for estimating tolerant dynamic treatment regimes. Biometrics , 80(1):ujad017

    2024

  37. [45]

    K., Shyam, P., Mutz, F., Ja \'s kowski, W., and Schmidhuber, J

    Srivastava, R. K., Shyam, P., Mutz, F., Ja \'s kowski, W., and Schmidhuber, J. (2019). Training agents using upside-down reinforcement learning. arXiv preprint arXiv:1912.02877

    2019 arXiv

  38. [46]

    Sutton, R. S. (1995). Generalization in reinforcement learning: Successful examples using sparse coarse coding. Advances in neural information processing systems , 8

    1995

  39. [47]

    Wehenkel, L., Ernst, D., and Geurts, P. (2006). Ensembles of extremely randomized trees and some generic applications. In Robust methods for power system state estimation and load forecasting

    2006

  40. [48]

    Winter, E. (2002). The shapley value. Handbook of game theory with economic applications , 3:2025--2054

    2002

  41. [49]

    Yu, C., Liu, J., Nemati, S., and Yin, G. (2021). Reinforcement learning in healthcare: A survey. ACM Computing Surveys (CSUR) , 55(1):1--36

    2021

  42. [50]

    R., and Zeng, D

    Zhao, Y., Kosorok, M. R., and Zeng, D. (2009). Reinforcement learning design for cancer clinical trials. Statistics in medicine , 28(26):3294--3315

    2009

Pith tools

Reviewed August 12, 2026 · model on record in the stance chip above.