Pith. sign in

REVIEW 4 major objections 5 minor 24 references

BASIL: Best-Action Symbolic Interpretable Learning for Evolving Compact RL Policies

T0 review · 4 major / 5 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read BASIL claims that compact symbolic rule-list policies can be evolved from scratch, online, to match or beat deep Q-networks on classic control benchmarks while remaining fully human-readable.

desk verdict A plausible new combination of rule-list evolution and QD search, but the central consistency claim rests on one run per benchmark with a five-episode fitness estimate. read the letter →

arxiv 2506.00328 v3 pith:4RY7ODAK submitted 2025-05-31 cs.AI

Kourosh Shahnazari , Seyed Moein Ayyoubzadeh , Mohammadali Keshtparvar This is my paper
classification cs.AI
keywords interpretablereinforcementlearningsymbolicpolicygeneticalgorithmsquality-diversityarchiverule-basedagentssimplificationevolutionaryCartPole
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper introduces BASIL, a genetic algorithm that evolves policies in the form of ordered lists of if-then rules over thresholded state variables, with a fallback action. It argues that symbolic, human-readable controllers do not have to sacrifice performance: on CartPole-v1, MountainCar-v0, and Acrobot-v1, the evolved policies match or exceed a Deep Q-Network baseline while training in seconds rather than minutes. The authors report a one-rule, two-predicate CartPole policy that reaches the maximum reward of $500.0$ in five generations and under four seconds of CPU time. The contribution is a method and an empirical demonstration, not a formal guarantee.

What carries the argument

The central object is the rule-list policy: a policy is a sequence of rules, each a conjunction of predicates of the form $s[d] < th$ or $s[d] > th$, followed by an action, and a final fallback action used when no rule fires. Fitness is the mean return over $E$ episodes minus $\lambda$ times the total number of predicates, which biases the search toward short rules. A quality-diversity archive, indexed by the number of rules and the average absolute predicate threshold, stores the best policy per behavioral-structural niche and supplies the elite pool for selection, crossover, and mutation.

What would settle it

Run BASIL on a stochastic version of one benchmark (e.g., CartPole with random force perturbations) using the paper's settings, then evaluate the final archive policies over hundreds of episodes; if the top-ranked policies by the 5-episode fitness are not the top policies by long-run average reward, the selection signal is not reliable for that task.

Watch

Extended reading notes

Core claim

The central claim is that a complexity-aware fitness function plus a quality-diversity archive lets a simple genetic search find compact rule lists that are competitive with deep reinforcement learning. In the paper's strongest example, an evolved CartPole policy reads 'If $s[2] > -0.02$ and $s[3] > -0.30$, then action = 1; else action = 0', achieves a reward of $500.0$, and is found in five generations. Similar results are reported for MountainCar-v0 (5 rules, 7 predicates, mean reward $-88.7$) and Acrobot-v1 (3 rules, 7 predicates, mean reward $-75.1$), each outperforming the DQN baseline in reward while using far fewer parameters and no gradients.

Load-bearing premise

The method's success depends on treating the average reward over just five episodes as a reliable ranking signal; in a stochastic environment, that sample can misrank policies and feed noise into both selection and the archive.

Editorial extensions

If this is right

  • Users can pre-specify a maximum rule count, giving exact control over the interpretability-performance trade-off before training begins.
  • Because policies are plain rule lists, they can be read, audited, and hand-edited by a domain expert, and every action is traceable to a specific rule.
  • The same hyperparameters worked across three environments with different dynamics, suggesting the method needs little per-task tuning.
  • Training requires only CPU time and no neural network, making interpretable policy synthesis feasible in computationally constrained settings.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Going beyond the paper, the 5-episode fitness estimate is the load-bearing selection signal; if it is noisy on a given task, the archive rankings may not reflect true policy quality, and reporting variance across seeds would test this directly.
  • The paper's descriptor space (rule count and average threshold) hints at a natural user interface: a human could browse the archive and pick among strategies by complexity and behavior, which the authors mention but do not demonstrate.
  • The predicate language could be extended to continuous actions by replacing the discrete action with a parameterized action slot, preserving readability while widening applicability.
  • Rule-list policies are deterministic and finite, so they are natural inputs for formal verification; the paper gestures at this but does not verify a single policy.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. The manuscript introduces BASIL, an evolutionary method that synthesizes interpretable, rule-based RL policies as ordered lists of thresholded predicates over state variables, using a complexity-aware fitness and a Quality-Diversity archive. Experiments on CartPole-v1, MountainCar-v0, and Acrobot-v1 report compact policies with rewards of 500.0, -88.7, and -75.1, respectively, compared with a single DQN baseline, and the paper claims that BASIL 'consistently synthesizes' controllers comparable to deep RL baselines.

Significance. If the reported results replicate across seeds and evaluation protocols, the core idea is a useful contribution to interpretable RL: a fully symbolic, from-scratch, online evolutionary method that can produce human-readable policies on classical control benchmarks in seconds. The policy representation, the formal fitness definition in Eq. (10), and the QD archive update in Eq. (13) are clearly described, and the CartPole policy with one rule and two predicates is an appealing concrete outcome. However, the paper's central 'consistently' claim is not supported by the current evidence, which consists of single-run numbers and qualitative statements. The missing seed-level statistics, missing ablations, and narrow baseline comparison are substantial gaps that must be addressed before the performance and generalizability claims can be evaluated.

major comments (4)
  1. [Section 4.1, Table 1, Sections 4.2-4.4] The abstract and conclusion claim that BASIL 'consistently synthesizes' compact high-performing policies, and Section 4.1 states that each experiment was repeated with different random seeds, but no seed count, per-seed results, means, standard deviations, or confidence intervals are reported. Sections 4.2-4.4 explicitly describe 'a typical run' and 'a single trial run,' and Table 1 gives one number per environment. The central consistency claim therefore does not follow from the reported evidence; please report full seed-level statistics and a pre-registered final-evaluation protocol.
  2. [Eq. (8), Algorithm 1, Eq. (13), Section 4.1] Fitness is computed as the mean return over E=5 episodes, and this noisy estimate directly drives elite selection in Algorithm 1 and the QD archive update in Eq. (13). In CartPole-v1, MountainCar-v0, and Acrobot-v1, initial states and dynamics are stochastic and rewards are sparse or delayed, so a five-episode average is a very small sample that can misrank policies. No sensitivity analysis for E is provided, and no independent longer-horizon evaluation confirms that the reported best policies generalize. Please add a sensitivity study (e.g., E=5 vs. E=30) and report final evaluation over many episodes across seeds, including whether the reported policies remain the best under the larger evaluation.
  3. [Section 1 and Section 4] The introduction claims 'exhaustive ablation evaluations,' but Section 4 contains no ablation experiments. There are no studies isolating the effect of the complexity penalty (λ), the mutation and crossover probabilities, the QD archive descriptors, the archive itself, or the rule-count constraint. Without these ablations, the contribution of each algorithmic component and the claim that the QD archive is essential for the reported diversity and performance are unverified. Please add ablation results or explicitly remove the claim.
  4. [Table 1, Sections 4.2-4.4] The comparison against deep RL is limited to a single DQN baseline, and the paper does not specify the DQN architecture, hyperparameters, number of seeds, evaluation protocol, or training curves. No comparison is made with the interpretable or evolutionary RL methods cited in the related work, such as decision-tree evolution [14], QD decision trees [16], or programmatically interpretable RL [19]. The claim that BASIL's performance is 'comparable to deep reinforcement learning baselines' and 'state-of-the-art' needs a broader, well-documented baseline suite and a fair experimental protocol.
minor comments (5)
  1. [Eq. (7)] Equation (7) contains the corrupted token 'TeX' in the summation upper limit; this should be the terminal time step, such as T_e.
  2. [Section 3.1] Several mathematical definitions are missing their symbols, for example 'where represents' after Eq. (1) and Eq. (3), and Eq. (1) typesets 'Rf allback' instead of 'R_fallback'; please restore the notation.
  3. [Section 4.4] The best Acrobot policy contains a duplicate predicate in the second rule: 's[5] > 0.00 and s[4] < 0.50 and s[5] > 0.00'. This appears to be a transcription error and should be corrected so the reported policy is exactly the evaluated one.
  4. [Section 5] The conclusion states that BASIL is 'orders of magnitude faster' than DQN, but Table 1 shows speedups of roughly 16x, 5x, and 5.6x on the three tasks; these are not orders of magnitude, and the wording should be aligned with the reported numbers.
  5. [Section 4.2] The sentence 'This effect exists outside of the shaping or prior knowledge requirements' is unclear and appears to be a non-native or unfinished construction; please rewrite to state what was or was not provided to the algorithm.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: BASIL's derivation chain is self-contained and its reported policies are outputs of environment-based search.

full rationale

No load-bearing circular step is present in this paper. The policies are evaluated against environment returns (Eqs. 7 and 8), and the fitness function combines that measured performance with an explicit complexity penalty (Eq. 10); the compactness of the final policies is therefore an advertised objective of the method rather than an independent prediction smuggled in as a result. The QD archive update (Eq. 13) selects among candidate policies using fitness values that themselves derive from environment interaction, so no fitted parameter is renamed as a prediction. The paper cites external prior work for genetic algorithms, QD methods, and interpretable RL, but no load-bearing claim rests on a self-citation chain or on an imported uniqueness theorem. The reported policies, such as the CartPole rule with two predicates, are terminal outputs of the evolutionary search, not quantities used to define the fitness or to set hyperparameters. Concerns that five evaluation episodes per policy may give noisy fitness estimates, and that the results are reported for single typical runs, are statistical-evidence issues rather than circularity; they belong under correctness risk. Likewise, the claim of 'exhaustive ablation evaluations' in Section 1 is not supported by an ablation table in the paper, but this is a completeness issue, not a circular derivation.

Assumptions & free parameters 5 free parameters · 4 assumptions · 0 invented entities

The framework relies on several hand-chosen hyperparameters and unverified representational assumptions. No new physical or conceptual entities are introduced; the only novel object is the algorithmic combination itself.

free parameters (5)
  • Complexity penalty coefficient lambda = not reported
    Chosen by hand to balance reward and interpretability; exact value is not given in the paper (Eq. 10, Sec. 4.1).
  • Mutation probability p_mut = not reported
    Hyperparameter named in Algorithm 1; exact value is not listed despite the claim that the same hyperparameters were used across tasks.
  • Crossover probability p_cross = not reported
    Hyperparameter in Algorithm 1; value is not reported.
  • Evaluation episodes E = 5
    Reported in Sec. 4.1; fitness is the mean over five episodes, and no sensitivity analysis is provided.
  • Predefined threshold set = not fully specified
    Threshold values for predicates are drawn from a domain-relevant predefined set (Sec. 3.1), but the set is not enumerated.
assumptions (4)
  • domain assumption An ordered list of thresholded predicates with first-match semantics can represent near-optimal policies for CartPole-v1, MountainCar-v0, and Acrobot-v1.
    Used throughout Sec. 3.1 and Sec. 4; no expressiveness or completeness analysis is provided.
  • domain assumption Average return over E=5 episodes is a reliable fitness signal.
    Sec. 3.3 Eq. 8 and Sec. 4.1 define fitness on five episodes; no variance or seed analysis is reported.
  • domain assumption Descriptor vector [number of rules, mean absolute threshold] captures behaviorally meaningful diversity.
    Sec. 3.4 Eqs. 11-12; the paper does not validate that archive cells correspond to distinct behaviors.
  • domain assumption Selecting elites from the QD archive improves final policy quality.
    Algorithm 1 chooses elites from the archive; no evidence is given that archive diversity improves final performance over a plain genetic algorithm.

how reviews work

0 comments
Cite this review

Pith. "Pith review of BASIL: Best-Action Symbolic Interpretable Learning for Evolving Compact RL Policies." pith.science (2026). https://pith.science/paper/4RY7ODAK

@misc{pith2026250600328,
  author       = {Pith},
  title        = {Pith review of: BASIL: Best-Action Symbolic Interpretable Learning for Evolving Compact RL Policies},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/4RY7ODAK}},
  note         = {Machine review of arXiv:2506.00328}
}
read the original abstract

The quest for interpretable reinforcement learning is a grand challenge for the deployment of autonomous decision-making systems in safety-critical applications. Modern deep reinforcement learning approaches, while powerful, tend to produce opaque policies that compromise verification, reduce transparency, and impede human oversight. To address this, we introduce BASIL (Best-Action Symbolic Interpretable Learning), a systematic approach for generating symbolic, rule-based policies via online evolutionary search with quality-diversity (QD) optimization. BASIL represents policies as ordered lists of symbolic predicates over state variables, ensuring full interpretability and tractable policy complexity. By using a QD archive, the methodology in the proposed study encourages behavioral and structural diversity between top-performing solutions, while a complexity-aware fitness encourages the synthesis of compact representations. The evolutionary system supports the use of exact constraints for rule count and system adaptability for balancing transparency with expressiveness. Empirical comparisons with three benchmark tasks CartPole-v1, MountainCar-v0, and Acrobot-v1 show that BASIL consistently synthesizes interpretable controllers with compact representations comparable to deep reinforcement learning baselines. Herein, this article introduces a new interpretable policy synthesis method that combines symbolic expressiveness, evolutionary diversity, and online learning through a unifying framework.

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

24 extracted references · 9 canonical work pages

  1. [14]

    IEEE Access 8, 177437–177449 (2020) https://doi.org/10.1109/ACCESS.2020

    Custode, L.L., Iacca, G.: Evolutionary learning of interpretable decision trees. IEEE Access 8, 177437–177449 (2020) https://doi.org/10.1109/ACCESS.2020. 3027091

    doi:10.1109/access.2020 2020

  2. [16]

    In: Proceedings of the 38th ACM/SIGAPP Symposium on Applied Computing, pp

    Ferigo, A., Custode, L.L., Iacca, G.: Quality diversity evolutionary learning of decision trees. In: Proceedings of the 38th ACM/SIGAPP Symposium on Applied Computing, pp. 425–432 (2023). https://doi.org/10.1145/3555776.3577591

    work page arXiv 2023

  3. [19]

    In: Proceedings of the 35th International Conference on Machine Learning, pp

    Verma, A., Singh, R., Chaudhuri, S.: Programmatically interpretable reinforce- ment learning. In: Proceedings of the 35th International Conference on Machine Learning, pp. 5045–5054 (2018). https://doi.org/10.48550/arXiv.1804.02477

    arXiv doi:10.48550/arxiv.1804.02477 2018

  4. [1]

    MIT Press, Cambridge, MA (2018)

    Sutton, R.S., Barto, A.G.: Reinforcement Learning: An Introduction, 2nd edn. MIT Press, Cambridge, MA (2018)

    2018

  5. [2]

    Journal of Artificial Intelligence Research 4, 237–285 (1996)

    Kaelbling, L.P., Littman, M.L., Moore, A.W.: Reinforcement learning: A survey. Journal of Artificial Intelligence Research 4, 237–285 (1996)

    work page 1996

  6. [3]

    Nature 518, 529–533 (2015) https://doi.org/10.1038/nature14236

    Mnih, V., Kavukcuoglu, K., Silver, D., Rusu, A.A., Veness, J., Bellemare, M.G., Graves, A., Riedmiller, M., Fidjeland, A.K., Ostrovski, G., Petersen, S., Beattie, C., Sadik, A., Antonoglou, I., King, H., Kumaran, D., Wierstra, D., Legg, S., Hassabis, D.: Human-level control through deep reinforcement learning. Nature 518, 529–533 (2015) https://doi.org/10...

    doi:10.1038/nature14236 2015

  7. [4]

    In: arXiv Preprint arXiv:1707.06347 (2017)

    Schulman, J., Wolski, F., Dhariwal, P., Radford, A., Klimov, O.: Proximal policy optimization algorithms. In: arXiv Preprint arXiv:1707.06347 (2017). https://arxiv.org/abs/1707.06347

    arXiv 2017

  8. [5]

    In: Pro- ceedings of the 35th International Conference on Machine Learning (ICML) (2018)

    Haarnoja, T., Zhou, A., Abbeel, P., Levine, S.: Soft actor-critic: Off-policy maximum entropy deep reinforcement learning with a stochastic actor. In: Pro- ceedings of the 35th International Conference on Machine Learning (ICML) (2018). https://arxiv.org/abs/1801.01290

    arXiv 2018

Show all 24 references
  1. [6]

    Nature Machine Intelligence 1(5), 206–215 (2019) https://doi.org/10.1038/s42256-019-0048-x

    Rudin, C.: Stop explaining black box machine learning models for high stakes decisions and use interpretable models instead. Nature Machine Intelligence 1(5), 206–215 (2019) https://doi.org/10.1038/s42256-019-0048-x

    2019 doi

  2. [7]

    arXiv preprint arXiv:1702.08608 (2017)

    Doshi-Velez, F., Kim, B.: Towards a rigorous science of interpretable machine 20 learning. arXiv preprint arXiv:1702.08608 (2017)

    2017 arXiv

  3. [8]

    In: arXiv Preprint arXiv:1712.06567 (2017)

    Such, F.P., Madhavan, V., Conti, E., Lehman, J., Stanley, K.O., Clune, J.: Deep neuroevolution: Genetic algorithms are a competitive alternative for training deep neural networks for reinforcement learning. In: arXiv Preprint arXiv:1712.06567 (2017). https://arxiv.org/abs/1712.06567

    2017 arXiv

  4. [9]

    MIT Press, Cambridge, MA (1992)

    Koza, J.R.: Genetic Programming: On the Programming of Computers by Means of Natural Selection. MIT Press, Cambridge, MA (1992)

    1992

  5. [10]

    Journal of Artificial Evolution and Applications2009, 1–25 (2009) https://doi.org/10.1155/2009/736398

    Urbanowicz, R.J., Moore, J.H.: Learning classifier systems: A complete introduc- tion, review, and roadmap. Journal of Artificial Evolution and Applications2009, 1–25 (2009) https://doi.org/10.1155/2009/736398

    2009 doi

  6. [12]

    MIT Press, Cambridge, MA (1998)

    Mitchell, M.: An Introduction to Genetic Algorithms. MIT Press, Cambridge, MA (1998)

    1998

  7. [13]

    Frontiers in Robotics and AI 3, 40 (2016) https: //doi.org/10.3389/frobt.2016.00040

    Pugh, J.K., Soros, L.B., Stanley, K.O.: Quality diversity: A new frontier for evolutionary computation. Frontiers in Robotics and AI 3, 40 (2016) https: //doi.org/10.3389/frobt.2016.00040

    2016

  8. [15]

    Nature 521(7553), 503–507 (2015) https://doi.org/10.1038/nature14422

    Cully, A., Clune, J., Tarapore, D., Mouret, J.-B.: Robots that can adapt like ani- mals. Nature 521(7553), 503–507 (2015) https://doi.org/10.1038/nature14422

    2015 doi

  9. [17]

    In: 2021 IEEE Symposium Series on Computational Intelligence (SSCI), pp

    Custode, L.L., Iacca, G.: A co-evolutionary approach to interpretable reinforce- ment learning in environments with continuous action spaces. In: 2021 IEEE Symposium Series on Computational Intelligence (SSCI), pp. 1–8 (2021). https: //doi.org/10.1109/SSCI50451.2021.9659837

    2021

  10. [18]

    Complex & Intelligent Systems 6(3), 545–557 (2020) https://doi.org/10.1007/s40747-020-00156-6 21

    Zhang, H., Lin, X., Sun, Y., Jin, Y.: Interpretable policy derivation for reinforce- ment learning based on evolutionary feature synthesis. Complex & Intelligent Systems 6(3), 545–557 (2020) https://doi.org/10.1007/s40747-020-00156-6 21

    2020 doi

  11. [20]

    In: Proceedings of the 30th International Joint Conference on Artificial Intelligence (IJCAI), pp

    Ma, Y., Fan, Y., Wang, Y., Zhang, Z., Zhan, Y., Jin, Y.: Learning neural-symbolic policies with graph neural networks. In: Proceedings of the 30th International Joint Conference on Artificial Intelligence (IJCAI), pp. 2433–2439 (2021). https: //doi.org/10.24963/ijcai.2021/335

    2021 doi

  12. [21]

    In: Proceedings of the Genetic and Evolutionary Computation Conference Companion, pp

    Hein, D., Udluft, S., Runkler, T.: Generating interpretable fuzzy controllers using particle swarm optimization and genetic programming. In: Proceedings of the Genetic and Evolutionary Computation Conference Companion, pp. 139–140 (2018). https://doi.org/10.1145/3205651.3208253

    2018

  13. [22]

    In: Proceedings of the Genetic and Evolutionary Computation Conference, pp

    Babuˇ ska, R.: Genetic programming methods for reinforcement learning. In: Proceedings of the Genetic and Evolutionary Computation Conference, pp. 1389–1390 (2019). https://doi.org/10.1145/3319619.3326845

    2019

  14. [23]

    arXiv preprint arXiv:2101.09571 (2021)

    Liventsev, V., H¨ arm¨ a, A., Petkovi´ c, M.: Bf++: a language for general-purpose program synthesis. arXiv preprint arXiv:2101.09571 (2021)

    2021 arXiv

  15. [24]

    In: 2022 IEEE International Conference on Fuzzy Systems (FUZZ-IEEE), pp

    Gevaert, A., Davis, J., Saeys, Y.: Distilling deep rl models into interpretable neuro-fuzzy systems. In: 2022 IEEE International Conference on Fuzzy Systems (FUZZ-IEEE), pp. 1–8 (2022). https://doi.org/10.1109/FUZZ-IEEE52011.2022. 9857362

    2022

  16. [25]

    In: Advances in Neural Information Processing Systems (NeurIPS), pp

    Landajuela, M., Heide, F., Kolter, Z.: Discovering symbolic policies with deep reinforcement learning. In: Advances in Neural Information Processing Systems (NeurIPS), pp. 1–12 (2021). https://doi.org/10.48550/arXiv.2106.11299 22

    2021 doi arXiv

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.