{"id":"710b28f4-3e26-4db7-8fb2-df08f5612ef7","arxiv_id":"2607.07235","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"unknown","formal_verification":"none","parameter_count":8,"one_line_summary":"Oblique decision trees with local linear models can approximate deep RL policies with continuous actions using far fewer parameters than axis-aligned trees while retaining task performance.","lead":"ORCAID extracts interpretable rule-based policies from deep reinforcement learning agents that use continuous actions, by building oblique decision trees with linear models in each leaf. It matters because deploying RL in safety-critical domains requires transparent, verifiable policies that humans can inspect and trust.","discovery_kind":"unclear","skeptic_critique":{"model":"glm-5.2","headline":"Linear-in-leaf adequacy is the right concern, but the Half Cheetah gap points to a more specific bottleneck: per-region nonlinearity vs. insufficient partitioning depth.","rationale":"The reader's verdict of CONDITIONAL is appropriate. The central claim is well-supported by experiments across 9 environments, code is public, and the model size metric (Def. 7) appears fair across methods—ORCAID's shared tree structure across action dimensions is a genuine architectural advantage, not an artifact of the metric. The reader's four conditions for not granting ACCEPT are reasonable: (1) LLM-as-a-Judge is supplementary, not load-bearing for the main claim; (2) HC underperformance is acknowledged; (3) the 2-environment ablation is thin but covers one simple and one complex case; (4) unspecified hyperparameters (δ, pca_dim, differential evolution settings) are a reproducibility gap but the ablation suggests robustness. My concern aligns with the reader's (agreement on the linear model assumption) but I'd note it's more a scoping limitation than a correctness flaw. The method works as claimed for the tested environments; the open question is whether the linear-in-leaf assumption generalizes to higher-dimensional or more nonlinear policies. The proposed test would clarify whether HC's underperformance is a depth issue or a fundamental limitation, which would inform whether the method's scope should be explicitly bounded. I don't think this changes the verdict—CONDITIONAL with the reader's stated conditions is the right call.","tokens_in":43563,"tokens_out":7960,"duration_ms":470447,"concrete_test":"For the Half Cheetah environment, train ORCAID models at depths 6–12 (beyond the current max of 6) and report both reward ratio and per-region MSE. If reward ratio plateaus below 90% while per-region MSE remains high, the linear-in-leaf assumption is the bottleneck and nonlinear leaf models would be needed. If reward ratio continues climbing with depth, the issue is simply insufficient partitioning and the linear assumption holds at finer granularity. Also report the number of DAgger iterations needed for convergence in each environment to quantify how much performance depends on iterative refinement vs. the tree architecture itself.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The reader correctly identifies the linear-model-in-leaf assumption as load-bearing. I agree but want to sharpen it. The paper's worst result is Half Cheetah (17-D state, 6-D action): 78±2% reward ratio vs. Cubist's 85±1%. The confidence intervals barely overlap (ORCAID: [73.4, 82.6], Cubist: [82.7, 87.3]), making the paper's 'non-significant' characterization borderline. The central claim already excludes HC from the '≥90%' set, so this doesn't falsify the headline. However, it does reveal where the linear assumption strains: in a 6-D action space, a single oblique split partitions all action dimensions simultaneously, but each leaf still fits independent linear models per action dimension. If the policy's action surface is nonlinear within a region along one action dimension, the linear model introduces errors that DAgger cannot fully correct (DAgger adds data but still fits linear models). The ablation (Table 64) confirms DAgger is doing substantial work: removing it drops Swimmer from 90% to 72%. The concern is not that the method fails—it works well in 7/9 environments—but that the paper provides no diagnostic to distinguish 'linear models are adequate here' from 'DAgger is compensating for linear model inadequacy.' Without per-region MSE breakdowns, we cannot tell whether HC's underperformance stems from insufficient partitioning (fixable by deeper trees) or fundamental nonlinearity within regions (not fixable without nonlinear leaf models).","agreement_with_reader":"agree"},"referee_report":{"model":"glm-5.2","summary":"The paper introduces ORCAID, a method for extracting interpretable rule-based surrogate models from deep RL policies with continuous action spaces. The approach learns oblique decision trees (hyperplane-based splits) with local linear regression models in each leaf, then merges adjacent regions and prunes features to produce compact DNF rules. The method is evaluated on nine Gymnasium control tasks against decision trees (CART), Cubist, and RuleFit, with all methods wrapped in a DAgger loop. The paper also demonstrates that ORCAID rules can be mirrored to identify and exploit policy weaknesses (adapted LEGIBLE approach), and uses an LLM-as-a-Judge protocol to assess interpretability. The core idea—oblique splits plus per-leaf linear models for continuous-action policy distillation—is sound and the experimental coverage is broad. The main concerns center on the Half Cheetah underperformance, the absence of per-region diagnostics, and the validity of the LLM-based interpretability evaluation.","tokens_in":44496,"tokens_out":1644,"duration_ms":529098,"significance":"The paper addresses a genuine gap: most XRL work on policy extraction targets discrete actions, and oblique trees with linear leaf models for continuous actions are not well explored. The method is practical, the code is publicly available, and the evaluation across nine environments with multiple baselines is substantial. The policy-improvement application via rule mirroring is a creative and actionable contribution. The LLM-as-a-Judge protocol for interpretability is novel in this context, though its reliability is debatable. The work is a solid contribution to the XRL literature.","major_comments":[{"comment":"§5.1, Performance paragraph and Figure 3: The paper states that Cubist's advantage in Half Cheetah (HC) is 'non-significant, by the confidence intervals.' From Table 25, ORCAID achieves 78±2% and Cubist 85±1% reward ratio. Using the paper's own CI formula (t_{0.025,8} ≈ 2.3, N=9), ORCAID's CI is approximately [73.4, 82.6] and Cubist's is [82.7, 87.3]. These intervals do not overlap, contradicting the 'non-significant' characterization. This matters because HC is the highest-dimensional environment (17-D state, 6-D action) and is the primary case where the linear-in-leaf assumption appears to strain. The paper should either correct the significance claim or provide a proper statistical test (e.g., paired t-test or bootstrap) to support it.","section":null},{"comment":"§4.3 and §5.1: The paper's central methodological assumption is that local linear models in oblique-tree leaves adequately approximate the policy within each region. The Half Cheetah underperformance and the ablation results (Table 64: removing DAgger drops Swimmer from 90% to 72%) suggest that DAgger is doing substantial compensatory work, but the paper provides no per-region MSE breakdowns or diagnostics to distinguish 'linear models are adequate here' from 'DAgger is compensating for within-region nonlinearity.' Adding per-region MSE statistics (or at least a histogram of leaf-level MSEs for HC vs. a well-performing environment like Pendulum) would substantially strengthen the paper's claims and help readers understand when the method is expected to work.","section":null},{"comment":"§5.1, Table 2, and Appendix D: The LLM-as-a-Judge evaluation uses six free-tier LLMs (Appendix D.1.3) to rate interpretability on a 1–5 Likert scale across 15 questions. The axiom that LLMs can reliably assess interpretability of rule-based models is load-bearing for RQ4, yet the paper provides no validation against human judgments. The paper acknowledges this limitation in §6 ('establishing agreement with human judgments remains important future work'), but RQ4 is still answered affirmatively in the main text. At minimum, the paper should (a) report inter-LLM agreement (e.g., Krippendorff's alpha or ICC across the six judges), and (b) temper the RQ4 conclusion to reflect that this is an exploratory evaluation, not a validated measure of interpretability.","section":null}],"minor_comments":[{"comment":"Table 1, IP row: The entry '1720' for ORCAID's Rate (%) appears to be a formatting error (likely '17' or '20'). Please fix the table formatting.","section":null},{"comment":"Figure 3: The x-axis label 'Model Size (log)' is ambiguous—it is unclear whether the axis is log-scaled or whether the values themselves are logarithms. The tick labels (10^0 through 10^4) suggest log-scaling, but this should be stated explicitly.","section":null},{"comment":"§4.1, step 3 (Sampling): The noise standard deviation σ is set to 0.15 for most environments (Table 24) but 0.01–0.05 for others. The paper states 'small noise was enough to obtain good results' but does not explain why HC and Hopper require much smaller σ. A brief discussion of how σ was selected and why it varies would help reproducibility.","section":null},{"comment":"Definition 7 (ORCAID model size): The distinction between M_conds (non-zero coefficients in conditions) and M_Bools (count of conditions) is slightly confusing. If a condition has 3 non-zero coefficients, it contributes 3 to M_conds and 1 to M_Bools? Clarifying with a small worked example beyond Example 3 would help.","section":null},{"comment":"§4.4: The merging criteria reference τ = 2·max_mse, but max_mse is a per-environment hyperparameter (Table 24). This means the merging threshold varies across environments in a way that is not transparent from the main text. A note directing readers to Table 24 for the actual τ values would help.","section":null},{"comment":"Appendix D.1.3: The list of LLMs includes 'openai/gpt-oss-120b' which does not appear to be a standard model name. Please verify and correct.","section":null},{"comment":"§3, Example 1: The Mountain Car description says the car 'applies continuous left/right acceleration,' but the standard Gymnasium MountainCar has a discrete action space. The continuous version (MountainCarContinuous) should be specified.","section":null},{"comment":"References: The citation to Tappler et al. [43] (LEGIBLE) is described as 'adapted' for continuous environments. Since this adaptation is non-trivial and central to RQ3, a brief note on what changed beyond 'mirroring along axes' would help readers understand the contribution boundary.","section":null}],"recommendation":"minor_revision","confidential_remarks":"The stress-test concern about Half Cheetah is valid and well-targeted: the 'non-significant' claim is borderline-to-incorrect, and the absence of per-region diagnostics is a genuine gap. However, the central claim of the paper (ORCAID produces compact, performant surrogates in most tested environments) is supported by the data in 7/9 environments. The HC underperformance does not falsify the headline result but does warrant honest framing. The LLM-as-a-Judge evaluation is the weakest experimental component; I would not block acceptance on it but the authors should at least report inter-judge agreement and soften the RQ4 claims. The paper is a good fit for the journal's scope in explainable RL."},"author_rebuttal":null,"desk_editor":{"model":"glm-5.2","letter":"ORCAID extracts interpretable rule-based surrogates from deep RL policies with continuous actions by learning oblique decision trees with per-leaf linear regression models. The core idea is sound and the engineering is careful: the three-stage split search (random init via Aw=−1, differential evolution refinement, backward elimination) is well-motivated, the DNF-based region merging is a nice complexity-reduction step, and the DAgger loop is properly integrated. Code is on GitLab. Across 9 environments, ORCAID consistently produces smaller models than DT, Cubist, and RuleFit at comparable reward ratios, and the policy-improvement-via-rule-mirroring application is a genuine plus that shows the rules are actionable, not just descriptive. The reader's take is mostly right. The combination of oblique splits with per-leaf linear models for continuous-action RL policy extraction is a real, if incremental, methodological contribution. The experimental design (3 RL seeds × 3 surrogate seeds = 9 runs per condition, 1000 evaluation episodes) is adequate. The circularity concern doesn't land — fidelity is measured against the original policy's actions, performance against environment reward, both external. The stress-test note correctly sharpens the real weakness: Half Cheetah (17-D state, 6-D action) is where the linear-in-leaf assumption strains. ORCAID gets 78±2% reward ratio there vs. Cubist's 85±1%, and the confidence intervals barely overlap, making the paper's 'non-significant' characterization borderline. More importantly, the ablation (Table 64) shows DAgger is doing heavy lifting — removing it drops Swimmer from 90% to 72% — which raises the question the paper doesn't answer: is DAgger compensating for linear model inadequacy within regions, or just fixing distribution shift? Without per-region MSE breakdowns, we can't tell whether HC's underperformance is fixable with deeper trees or fundamental to the linear leaf model. The LLM-as-a-Judge evaluation (6 free-tier models, no human validation) is creative but shouldn't carry weight as evidence of interpretability. The ablation covering only 2 of 9 environments is thin. Several hyperparameters (pca_dim, differential evolution settings, backward elimination thresholds) are underspecified. None of these are fatal. The method works well in 7/9 environments, the code is available, and the contribution is clearly within scope for an XRL venue. A serious referee should push on the HC gap, demand per-region diagnostics, and ask whether the linear leaf model is a fundamental ceiling or a tunable trade-off. Recommend sending to review.","headline":"Solid engineering contribution to XRL with continuous actions; the linear-in-leaf assumption is the right thing to worry about, and Half Cheetah is where it shows.","tokens_in":44336,"tokens_out":1455,"would_cite":true,"duration_ms":102195,"reading_group":"no","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"glm-5.2","headline":"Oblique trees with local linear models distill deep RL policies","keywords":[],"falsifier":"Find a continuous-action RL environment where the optimal policy's action function is highly nonlinear within any convex region of the state space (e.g., a policy that requires sinusoidal or oscillatory action modulation within a single operating mode). If ORCAID's linear leaf models cannot approximate such functions without an impractical number of splits, the method would fail to produce a compact and faithful surrogate, and model drift would degrade task performance below acceptable thresholds.","tokens_in":43746,"feed_emoji":"🌳","tokens_out":937,"duration_ms":155279,"temperature":0.7,"pith_summary":"Deep reinforcement learning policies are powerful but opaque: they map sensory inputs to continuous control actions through neural networks that no human can inspect or verify. ORCAID is a method for converting such a policy into a small set of human-readable rules. Each rule says: when the system's state falls in a certain region (defined by a slanted hyperplane rather than a single feature threshold), apply a simple linear formula to compute the action. The method builds an oblique decision tree by searching for good hyperplane splits in three stages—random initialization, local optimization, and backward elimination of unnecessary features—then merges adjacent regions that can share the same linear model. The result is a compact surrogate that closely mimics the original neural network's behavior, retains strong task performance across nine control benchmarks, and uses far fewer parameters than standard axis-aligned decision trees or competing rule-learning methods. The paper also shows that the extracted rules can be mirrored along symmetry axes to identify and fix weaknesses in the original policy, yielding measurable performance improvements.","feed_headline":"Oblique trees distill deep RL policies into compact rules","feed_subtitle":"Hyperplane splits with local linear models match neural network performance using far fewer parameters across nine control tasks","key_machinery":"The load-bearing mechanism is the oblique split condition: a single inequality w · x ≤ b where w is a learned weight vector over all state features, not just one. This lets one split boundary separate states that an axis-aligned tree would need many nested thresholds to distinguish. Inside each leaf, ordinary least-squares regression maps state features to continuous action values, with backward elimination pruning irrelevant coefficients. The QM (Quine-McCluskey) method then simplifies the Boolean logic of merged regions into minimal disjunctive normal form. A DAgger loop iteratively collects states where the surrogate and original policy disagree, retraining to close the gap.","core_discovery":"The central technical claim is that partitioning the state space with oblique (linear-combination) hyperplanes and fitting a separate pruned linear regression in each leaf produces a faithful, compact surrogate of a continuous-action deep RL policy. The three-stage split search—random hyperplane initialization via solving Aw = -1, differential evolution refinement, and backward feature elimination—makes this tractable. Region merging via DNF simplification then reduces the rule count without sacrificing fidelity. Across nine environments, ORCAID achieves reward ratios at or near the original policy while using model sizes consistently smaller than axis-aligned decision trees, Cubist, and en.","pith_inferences":[],"forward_implications":["Safety-critical RL deployments (robotics, autonomous driving, industrial control) could replace opaque neural network policies with verifiable rule sets that regulators can inspect, satisfying transparency requirements like the EU AI Act's Article 13.3.","The rule-mirroring approach to policy improvement suggests a general methodology: extract interpretable structure from a learned policy, apply domain symmetries to generate counterfactual rules, and use enforcement testing to find regions where the original policy is suboptimal.","The oblique-tree training algorithm (random init + differential evolution + backward elimination) could be applied beyond RL policy distillation to any regression or classification problem where compact oblique partitions are preferred over deep axis-aligned trees.","The DAgger integration with oblique trees provides a template for iterative surrogate refinement that could extend to stochastic policies if leaf models are replaced with linear-Gaussian distributions, as the authors note."],"fun_headline_variants":["Oblique decision trees extract compact rule-based policies from deep RL","Linear hyperplane splits distill continuous-action RL into interpretable rules","Three-stage split search yields faithful surrogate policies from deep RL networks","Oblique tree surrogates match deep RL performance with fewer parameters"],"cache_read_input_tokens":0,"weakest_assumption_plain":"The method assumes that within each region carved out by hyperplane splits, the original neural network's action function is well-approximated by a single linear model. If the policy's action surface curves sharply within a region—not just across region boundaries—the linear fit introduces errors that accumulate as the surrogate executes, causing trajectories to diverge from the original agent's behavior. The DAgger loop mitigates this by collecting new data where the two dev","fun_headline_variants_meta":{"raw":{"variants":["Oblique decision trees extract compact rule-based policies from deep RL","Linear hyperplane splits distill continuous-action RL into interpretable rules","Three-stage split search yields faithful surrogate policies from deep RL networks","Oblique tree surrogates match deep RL performance with fewer parameters"]},"model":"glm-5.2","effort":"low","cost_usd":0.0,"raw_usage":{"total_tokens":558,"prompt_tokens":500,"completion_tokens":58,"prompt_tokens_details":null},"tokens_in":500,"tokens_out":58,"duration_ms":74718,"temperature":1.0,"reasoning_tokens":null,"cache_read_input_tokens":0,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-07-09T16:29:26.282859+00:00","model_set":{"reader":"glm-5.2"},"falsifier":"Find a continuous-action RL environment where the optimal policy's action function is highly nonlinear within any convex region of the state space (e.g., a policy that requires sinusoidal or oscillatory action modulation within a single operating mode). If ORCAID's linear leaf models cannot approximate such functions without an impractical number of splits, the method would fail to produce a compact and faithful surrogate, and model drift would degrade task performance below acceptable thresholds.","supporting_citations":[],"review_version":1}