{"id":"9e6b8957-3a8e-4789-b254-a14e417aa1a7","arxiv_id":"2607.27905","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":4.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":4,"one_line_summary":"Including the target model's predicted class in the RL state representation increases counterfactual validity and speeds up training compared with a class-blind state.","lead":"This paper tests whether adding the model's predicted class to the state used by a reinforcement-learning agent improves the counterfactual explanations the agent generates. Across seven tabular datasets, the class-aware agent produced more valid counterfactuals and trained faster than a class-blind version.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Missing dummy-feature control leaves the class-awareness effect confounded with extra input dimension.","rationale":"The reader's weakest_assumption correctly identifies the missing dummy-feature control. This is the most load-bearing issue because it directly questions the causal interpretation of the main result: the observed gains could be an artifact of the increased state dimension/policy capacity rather than the semantic content of the predicted class. The SHAP/LIME results do not resolve this, as a high-importance feature is expected whenever the agent learns to use it, regardless of whether an equally arbitrary extra feature would yield similar gains. The proposed permutation control is a feasible and decisive experiment. The paper's other limitations (qualitative convergence analysis, unreported code) are secondary. The conditional verdict remains appropriate: the paper is promising but the central claim is not yet causally identified without the control.","tokens_in":13794,"tokens_out":6460,"duration_ms":56457,"concrete_test":"Retrain the class-blind baseline with an additional input feature that is a random permutation of the predicted class values (preserving the state dimension and marginal distribution but destroying the relationship to the true class), keeping all hyperparameters identical. If this permuted-feature control achieves gains comparable to class-aware RL across the seven datasets, the improvement is due to the extra dimension/capacity, not class-awareness. If class-aware RL still significantly outperforms the control, the semantic content of the predicted class is supported. Additionally, report paired effect sizes and confidence intervals for validity to substantiate the significance claim.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The paper's central claim is that including the instance's predicted class in the RL state causally improves CFE generation. However, the class-aware state differs from the class-blind state along two axes: (1) an additional input dimension and correspondingly more parameters in the policy network's first layer, and (2) the semantic content of the predicted class. Because the predicted class is a deterministic function of the other features, it adds no information in an information-theoretic sense; any benefit could arise purely from increased model capacity or from having an extra (even random) input. The paper does not include a control with a dummy feature or a capacity-matched baseline. Therefore, the observed improvements in validity, reward, and episode length cannot be attributed to 'class-awareness' per se. The SHAP/LIME ranking of the class feature is also insufficient: it only shows that the agent uses the feature, not that the feature's semantic content is the causal driver. This confound is load-bearing because it undermines the paper's main conclusion.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The manuscript proposes adding the predicted class label of the input instance to the state representation of a PPO-based reinforcement learning agent that generates counterfactual explanations (CFEs) for a black-box classifier. The method is compared with a class-blind variant on seven tabular datasets. The authors report faster convergence, higher rewards, shorter episodes, and significantly higher CFE validity for the class-aware variant (p=0.0071), and use SHAP and LIME to show that the class feature is among the most important in the policy's action selection. The paper claims that class-awareness causally improves CFE search.","tokens_in":14039,"tokens_out":3664,"duration_ms":34987,"significance":"The proposed modification is simple and easy to adopt, and RL-based CFE is an active area; demonstrating a reliable benefit from including the predicted class would be a useful practical contribution. The paper's strengths include public datasets, provided code, and a significance test for the validity difference. However, the central causal claim is currently not established because the design confounds the semantic content of the class with an additional input dimension and policy capacity, and the convergence claims are not statistically tested.","major_comments":[{"comment":"The design misses a dummy-feature control. The class-aware state includes one extra input dimension, so the PPO policy has a larger first layer. Since the predicted class is a deterministic function of the other features (given the trained XGBoost model), it carries no additional information; the observed improvements in validity, reward, and episode length could be due to the extra capacity or to a generic beneficial effect of an extra input. To attribute the effect to 'class-awareness', the authors should add a control with a random/dummy feature of the same scale, or match capacity (e.g., an extra unused input in class-blind). Without this, the core conclusion is unidentified.","section":"Sections 3.1–3.3 and 5.2"},{"comment":"Convergence claims are not statistically supported. Figure 4 shows only mean curves from 10 runs; no confidence intervals, significance tests, or quantitative definitions of 'faster convergence' are provided. The statements that class-aware converges faster in some datasets and slower in Adult Income are based on visual inspection. The authors should report time-to-threshold or final reward/episode differences with confidence intervals and a paired test across seeds.","section":"Section 5.1"},{"comment":"The SHAP/LIME analysis is used as confirming evidence, but it cannot distinguish semantic benefit from the general use of a state feature. A feature that is always present and immutable will naturally be used by the policy, especially in the terminal reward condition. The fact that LIME does not rank 'Prediction' in the top 5 for two datasets also weakens the 'consistently among the top features' claim. This evidence does not address the confound raised in the previous comment.","section":"Section 5.3"}],"minor_comments":[{"comment":"A column for the proposed method would make the comparison easier; currently the class-aware results are in Table 2, so the reader must cross-reference.","section":"Table 3"},{"comment":"Typo in the heading: 'validaity' should be 'validity'.","section":"Table 2"},{"comment":"The text contains the encoding artifact \"instance?s\" (e.g., in the Introduction and Section 5.3). Please fix.","section":"Sections 1, 2, 5.3"},{"comment":"The sentence \"the predicted class-based feature was excluded when computing these measures\" is important for understanding the reported sparsity/proximity values; state this earlier and ensure the class-blind evaluation also excludes the same feature.","section":"Section 4"},{"comment":"Use consistent notation for the test set: D_ts vs \\bar{D}^{ts}.","section":"Section 3.4"}],"recommendation":"major_revision","confidential_remarks":"The core empirical idea is straightforward and potentially useful, but the missing dummy-feature/capacity control is a genuine confound that blocks the main causal claim. I would request that control and a more rigorous statistical treatment of the convergence analysis before considering acceptance. If the control shows the effect persists, the paper could be a solid contribution."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"The paper tests a simple idea: add the classifier's predicted class to the RL state when generating counterfactual explanations. That's genuinely new in this literature, and the comparison on seven datasets is more thorough than most. The strongest result is the validity improvement (p=0.0071), and the training curves are consistent with faster or better convergence for the class-aware agent. I'd call that a real empirical pattern.\n\nWhere it falls short is the interpretation. The predicted class is a deterministic function of the other features, so the class-aware state carries no new information—it just gives the policy network an extra input dimension and the first layer extra parameters. The paper never runs a dummy-feature or capacity-matched control. Without that, you can't attribute the gain to 'class-awareness' rather than to having a wider input. The SHAP/LIME analysis doesn't help: the policy obviously uses a feature that's in its state, and the feature's top ranking just says it correlates with actions. So the stress-test note is correct: this is a load-bearing confound for the paper's central claim, and it's fixable with one control experiment.\n\nOther soft spots are minor by comparison. The convergence-speed claims are visual; no statistical test or timing. Table 3 doesn't show an 'ours' column, which makes the benchmark comparison harder to read than it should be. The code is promised but no working link or verification. None of these are fatal.\n\nBottom line: the paper deserves a serious referee. It's a modest but useful contribution, and the validity result is worth knowing even if the mechanism is underspecified. What I'd ask the authors for in revision is the dummy-feature control, a capacity-matched baseline, and some error bars or tests on convergence. I'd bring it to a reading group as an example of a common RL-for-XAI pitfall, but I wouldn't treat the class-awareness effect as established until the control is done.","headline":"A genuinely useful empirical observation about RL state design for CFE, but the main claim needs a control for added input dimension before it's fully convincing.","tokens_in":14439,"tokens_out":2868,"would_cite":true,"duration_ms":26374,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"This paper argues that including the instance's predicted class in the reinforcement-learning state representation makes counterfactual explanation generation faster, higher-reward, and more often valid.","keywords":["counterfactual explanations","reinforcement learning","class-aware state","PPO","SHAP","LIME","tabular data","validity"],"falsifier":"Train the class-blind policy with an extra immutable state entry filled with a random or constant dummy value, keeping input dimension and network capacity the same. If this dummy-feature policy matches the class-aware policy's convergence and validity, the causal role of class-awareness is refuted; if it performs like the original class-blind policy, the class content matters.","tokens_in":13719,"feed_emoji":"🎯","tokens_out":6214,"duration_ms":60526,"temperature":0.7,"pith_summary":"Counterfactual explanations (CFEs) are 'what would need to change' instances that flip a black-box model's prediction, and this paper studies how to generate them with reinforcement learning. Its central claim is that adding the instance's predicted class to the RL state—alongside the usual feature values—improves the search. Across seven tabular datasets, the class-aware policy converged faster, reached higher terminal rewards, produced shorter episodes, and generated significantly more valid CFEs than a class-blind policy identical except for the missing class input. The paper also reports that the class feature ranks among the most influential inputs to the policy's action choices according to SHAP and LIME. If correct, this offers a simple, zero-cost change to many RL-based CFE systems: use a prediction that is already available.","feed_headline":"Predicted class in RL state boosts counterfactual validity","feed_subtitle":"A class-aware agent converged faster, earned higher rewards, and generated significantly more valid explanations on seven datasets.","key_machinery":"The load-bearing object is the MDP state used by a Proximal Policy Optimization agent (a standard clipped policy-gradient method). The state is the vector of transformed feature values plus one extra immutable entry: the predicted class of the current instance from the black-box classifier. Actions are single-feature increments or decrements chosen from a set of size twice the number of mutable features; transitions are deterministic and terminal rewards are computed only at episode end as +10 for a valid flip, minus penalties for sparsity and cosine-distance proximity, with a -1 timeout penalty. This state design is what carries the argument: it gives the policy a direct, always-up-to-date","core_discovery":"The discovery the paper aims to establish is that class-awareness is not just neutral but beneficial: a reinforcement-learning agent that observes the current predicted class of the instance it is editing becomes a better counterfactual generator. The authors set up counterfactual generation as an episodic Markov decision process in which an agent changes one feature at a time until the black-box model's prediction flips, with terminal rewards for validity, sparsity, and proximity. The proposed class-aware state representation appends the model's prediction for the current state to the other features; the class-blind baseline is the same formulation without that input. On seven tabular datas","pith_inferences":["Because the predicted class is a deterministic function of the other features, the class-aware state carries no new information in the information-theoretic sense; the benefit likely comes from making the goal condition directly visible and from the extra input dimension, so a dummy-feature control would separate these explanations.","A testable extension: train class-aware policies and then distill them into class-blind policies of the same architecture; if the distilled policies keep the validity gain, the class input matters only during learning and can be removed at deployment.","The same recipe—append the current model prediction to the state—could transfer to image or text counterfactual generation, where an agent editing pixels or tokens could similarly benefit from a direct 'am I done yet?' signal, though the paper does not test this."],"forward_implications":["RL-based CFE systems can incorporate the predicted class at zero additional data-collection cost, since the prediction is already computed for the instance.","Higher validity and shorter episodes mean faster, more reliable recourse generation for large tabular datasets and production black-box models.","The observed gains in both training convergence and test-time validity suggest class-awareness may help in other RL search tasks where a goal condition is cheaply computable.","Feature-importance analyses of RL policies should include the class input when it is present; otherwise they will miss a consistently top-ranked driver of action selection."],"fun_headline_variants":["Class-aware RL generates more valid counterfactual explanations","RL agents that know the predicted class make better counterfactuals","Class-aware state helps RL agents produce valid counterfactuals","Including predicted class boosts RL counterfactual validity"],"cache_read_input_tokens":2304,"weakest_assumption_plain":"The assumption that carries the paper is that the improvement comes from the class information itself; the experiments do not control for the extra input dimension, and because the class is a deterministic function of the other features, the added state entry provides no new information.","fun_headline_variants_meta":{"raw":{"variants":["Class-aware RL generates more valid counterfactual explanations","RL agents that know the predicted class make better counterfactuals","Class-aware state helps RL agents produce valid counterfactuals","Including predicted class boosts RL counterfactual validity"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000358,"raw_usage":{"total_tokens":1787,"prompt_tokens":768,"completion_tokens":1019,"prompt_tokens_details":{"cached_tokens":256},"prompt_cache_hit_tokens":256,"prompt_cache_miss_tokens":512,"completion_tokens_details":{"reasoning_tokens":953}},"tokens_in":512,"tokens_out":1019,"duration_ms":8014,"temperature":1.0,"reasoning_tokens":953,"cache_read_input_tokens":256,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-07-31T23:15:42.069978+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Train the class-blind policy with an extra immutable state entry filled with a random or constant dummy value, keeping input dimension and network capacity the same. If this dummy-feature policy matches the class-aware policy's convergence and validity, the causal role of class-awareness is refuted; if it performs like the original class-blind policy, the class content matters.","supporting_citations":[],"review_version":1}