{"id":"75cf0ddd-85ab-452d-b5c0-520055f00e4e","arxiv_id":"2502.09226","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":4.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":0,"one_line_summary":"CoGS generates causally compliant counterfactual explanations and intervention paths using s(CASP) and FOLD-SE rules, demonstrated on three UCI datasets.","lead":"This paper introduces CoGS, a framework that uses answer set programming and rule-based machine learning to generate counterfactual explanations, including step by step paths that respect causal links between features. Its goal is to make machine learning recourse recommendations realistic and actionable for consequential decisions such as loan approvals.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The 'realistic' claim rests on FOLD-SE learning true causal dependencies, but FOLD-SE learns association rules and the paper provides no validation against causal ground truth.","rationale":"The reader's weakest_assumption identifies the same load-bearing concern, and I agree with it. The paper's central contribution is conditional on the quality of the learned causal rules: if those rules are wrong, CoGS is only 'causally compliant' with an incorrect model, and the generated counterfactuals are not guaranteed to be realistic. The paper does not provide any systematic validation of the causal rules, no comparison against known causal structure, no interventional or semi-synthetic experiments, and no baseline showing that the learned rules outperform simple association rules. However, the planning framework itself is plausible and internally coherent: given correct causal constraints, the path generation via s(CASP) is a reasonable way to produce step-by-step counterfactual explanations. There is no internal inconsistency or obvious algorithmic error in the ASP encoding as far as the paper describes it. Therefore the appropriate verdict remains CONDITIONAL, as the reader concluded: the central idea is promising, but the central empirical premise—that FOLD-SE discovers genuine causal dependencies—must be tested before the 'realistic' claim can be accepted. The proposed synthetic benchmark would provide a direct, decisive test of that premise. I do not think the concern warrants rejection, because the paper is explicit that causal dependencies can also be user-defined, and the framework could be correct in settings where such dependencies are known. The missing piece is evidence that the learned rules are trustworthy.","tokens_in":6685,"tokens_out":3715,"duration_ms":36141,"concrete_test":"Construct a semi-synthetic benchmark from a known structural causal model, e.g., Debt -> CreditScore -> Approval, with Debt also directly affecting Approval, plus independent features like Age and Education. Generate observational data from this model, run FOLD-SE to learn causal rules, and feed those rules into CoGS. For each negative instance, compare CoGS's generated path with the path obtained by applying the true structural equations. Measure (i) whether the features intervened on match the true causal parents on the path, (ii) whether the path is executable under the true model, and (iii) whether following the path actually flips the outcome. If CoGS's success rate under the true model is materially lower than its success rate under its own learned rules, the causal learning step is the bottleneck and the headline 'realistic' claim should be weakened.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim—'CoGS computes paths that respect the causal constraints among features. Thus, the counterfactuals computed by CoGS are realistic'—rests on the premise that the constraints C, supplied to the ASP planner, are true causal dependencies. In Section 4 the paper says 'causal rules are also learnt using FOLD-SE and verified,' but no verification protocol, ground truth, or external test is provided. FOLD-SE is a rule-based classifier: it learns association rules from observational data. An association rule such as 'if debt obligations high then credit score low' can hold observationally even when the direction of causality is reversed, when both features are effects of a common cause, or when the dependency is definitional or selection-induced. Converting learned rules into the biconditional form (P => Q) and (not P => not Q) via s(CASP) completion (Section 2) then turns every learned association into a bidirectional causal constraint. If a learned rule is spurious, CoGS will either forbid an achievable intervention or force an unnecessary and unrealistic one, so the generated path does not reflect what would actually happen in the world. The experiments report only that counterfactuals can be computed from the learned rules; they do not measure realism against known causal structure or interventional data. Thus the 'realistic' part of the central claim is currently unsubstantiated.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes CoGS, a framework for generating counterfactual explanations from rule-based machine learning models using Answer Set Programming (specifically s(CASP)). CoGS takes decision rules D learned by FOLD-SE, learns or accepts causal dependency rules C among features, encodes both in ASP, and solves a planning problem whose goal state is a positive outcome consistent with C and inconsistent with D. The output is a step-by-step sequence of feature-value changes from a negative outcome to a positive one. The authors argue that because these paths respect causal constraints, the resulting counterfactuals are realistic and achievable. Preliminary results are reported on three UCI datasets (German, Adult, Car Evaluation) with one example path per dataset, plus counts of counterfactual sets that CoGS can generate. The paper is presented as early-stage research with planned extensions rather than a completed evaluation.","tokens_in":6910,"tokens_out":3370,"duration_ms":33699,"significance":"If the central claim were established, CoGS would be a useful contribution: it combines counterfactual explanation, causal modeling, and ASP-based planning in a way that addresses a known limitation of earlier counterfactual methods that assume feature independence. The paper's strengths include a concrete, open-source implementation, a clear motivating example (John's loan application), and a sensible high-level design that separates decision rules from causal constraints and allows user approval of learned dependencies. However, the evidence offered is anecdotal: the central claim that the counterfactuals are 'realistic' is not validated against causal ground truth, interventional data, or any systematic comparison with baselines. The evaluation currently consists of three hand-selected paths, and the causal learning step is both unverified and conceptually questionable because FOLD-SE is an association-rule learner, not a causal discovery method. These issues are load-bearing for the paper's main contribution.","major_comments":[{"comment":"The paper's central claim that CoGS computes 'realistic' counterfactuals is supported only by three hand-selected example paths in Tables 1 to 3. No systematic evaluation is provided: there are no metrics for realism, actionability, cost, or validity of the generated paths, no comparison to baseline counterfactual methods such as Wachter et al., and no error analysis over the 240, 112, and 78 counterfactual sets reported in Table 4. A convincing validation would define measurable criteria (e.g., whether the suggested intermediate actions are feasible, whether the final state satisfies the decision rules, and whether the path respects known causal dependencies) and report results over all generated counterfactuals.","section":"Section 4"},{"comment":"The statement that 'causal rules are also learnt using FOLD-SE and verified' is not backed by any verification protocol, ground-truth causal structure, or external check. FOLD-SE is a rule-based classifier that learns association rules from observational data; such rules can hold when the causal direction is reversed, when the two features are common effects of a hidden cause, or when the dependency is definitional or selection-induced. Converting every learned association into a biconditional causal constraint, as described in Section 2, then treats spurious associations as hard causal laws. The paper must provide evidence that the learned rules correspond to genuine causal dependencies, for example by using known causal benchmarks, expert validation, or interventional data, and must report how many learned rules were accepted, rejected, or found to be spurious.","section":"Section 4"},{"comment":"The paper equates causality with logical equivalence: 'P is the cause of Q, if (P ⇒ Q) ∧ (¬P ⇒ ¬Q)'. This is a strong assumption that is not generally valid for causal relations, which can be asymmetric and context-dependent. Program completion in s(CASP) turns every learned 'if' rule into an 'if and only if' rule, thereby imposing bidirectionality on associations that may not be causal. This is not merely a philosophical concern: if a learned rule is spurious, CoGS may forbid an achievable intervention or force an unnecessary and unrealistic one, undermining the paper's main claim. The authors should either justify the equivalence assumption for their application domain or use a causal discovery method that produces asymmetric, directed relations.","section":"Section 2"},{"comment":"The paper states that 'CoGS has the ability to find minimal paths by iteratively adjusting the path length', but no formal definition of minimality is given and no proof or experiment demonstrates that the generated paths are minimal in any meaningful sense (e.g., fewest interventions, lowest cost, or most plausible). Since the paper presents minimality as a contribution, this claim needs either a precise definition with a proof or an empirical study showing that the method returns optimal or near-optimal paths according to a stated criterion.","section":"Section 5"}],"minor_comments":[{"comment":"There is a typo in the first sentence: 'one of the limitations of of CoGS' should read 'one of the limitations of CoGS'.","section":"Section 6"},{"comment":"The caption contains a typo: 'counterfactuals produce b by' should read 'counterfactuals produced by'. Also, the dataset names are inconsistent: 'Cars' is used in Table 4 but 'Car Evaluation' is used in the text and Table 3.","section":"Table 4 caption"},{"comment":"The citation to Pearl for the definition of causality is imprecise; the formula (P ⇒ Q) ∧ (¬P ⇒ ¬Q) is more naturally a definition of logical equivalence or correlation than a Pearlian causal definition. A more careful treatment of the relationship between logical implication and causal dependence would improve the paper.","section":"Section 2"},{"comment":"The Adult dataset example changes 'Relationship' directly to 'husband' and then treats 'Marital Status' changing to 'married civ spouse' as a causal effect. This raises feasibility and ethical questions: a person cannot simply change their relationship status by direct action, and the path may be socially or legally constrained. The paper should discuss how such actionability issues are handled by the framework, for example through user approval of allowed interventions.","section":"Table 2"},{"comment":"The paper reports only successful example paths and gives no information about cases where CoGS fails to find a counterfactual, where it finds an infeasible path, or where runtime is excessive. Reporting such cases would give a more complete picture of the framework's behavior.","section":"Section 4"},{"comment":"The description of the goal state as 'consistent with the causal rules C and inconsistent with the decision rules D' is clear in outline, but the encoding details of the planning problem (how actions are represented, how feature domains are handled, and how numerical intervals are managed) are not given. A pseudocode listing or a small example program would help the reader understand the actual ASP implementation.","section":"Section 3"}],"recommendation":"major_revision","confidential_remarks":"This is a short paper with a plausible idea but the evaluation is currently at the level of a workshop demonstration. The main risk is that the 'realistic' claim is self-referential: the causal rules and the decision rules are both learned from the same data, and the counterfactuals are then validated only against those same learned rules. For a journal venue, the authors would need to validate against external causal knowledge or interventional data, and to compare against baseline counterfactual methods. If the authors can add such an evaluation, the paper could be a useful contribution; as it stands, the central claim is not yet supported."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Short version: this is a preliminary paper that combines FOLD-SE rule learning with s(CASP) goal-directed ASP to produce step-by-step counterfactual paths. The combination is new relative to the cited literature, and the paper makes its pipeline concrete. But the central claim—that the resulting counterfactuals are 'realistic'—rests on treating FOLD-SE's learned rules as causal, and the paper does not validate that.\n\nWhat it does well: CoGS is clearly described. The planning formulation is appropriate: states, goal, transitions constrained by causal rules. The use of s(CASP)'s program completion to turn 'if' into 'if and only if' is a specific modeling choice that captures a strong notion of 'P causes Q' (both necessity and sufficiency). The examples on German credit, Adult, and Car Evaluation are illustrative, and the GitHub code is a useful artifact for anyone who wants to see the mechanics.\n\nWhere it is soft: the load-bearing step is 'causal rules are also learned using FOLD-SE and verified.' FOLD-SE is a classification algorithm; it learns association rules, not causal mechanisms. The example they give—if Job is unemployed then Present Employment Since is unemployed/unskilled-non-resident—looks more like a logical consistency constraint than a cause-effect relationship. No ground truth or interventional data is used to verify the learned causal rules. If a learned rule is spurious or directionally wrong, the planner will either forbid real interventions or impose unrealistic ones, so the 'realistic' tag is unsupported. The empirical evaluation is anecdotal: three hand-picked paths, no baselines, no comparison against Karimi-like causal recourse methods, no error analysis. Table 4 gives counts of counterfactual sets but not how many are actually achievable or plausible. The paper also does not discuss how the biconditional completion could over-constrain the plan space.\n\nThat said, this is a short paper that labels itself as current status and preliminary results. It does not pretend to have a full evaluation. The framework is coherent and the idea of learning causal constraints from data and plugging them into a planner is worth pursuing. The missing piece is a serious validation of the causal learning step.\n\nFor whom: readers working on counterfactual explanations, algorithmic recourse, or applications of ASP to XAI. It's a quick read and the code is there. I would not cite it as evidence that causally compliant counterfactuals are achievable; it is evidence that the pipeline can be built.\n\nRecommendation: if this lands as a workshop paper, accept. If it is submitted as a full journal article, it needs a real causal validation (e.g., known causal graphs, sensitivity analysis, interventional data), baselines, and more careful claims about realism.","headline":"A useful preliminary framework that combines FOLD-SE rule learning with s(CASP) planning, but the 'realistic' claim rests on causal rules the paper never validates.","tokens_in":7447,"tokens_out":3802,"would_cite":false,"duration_ms":30707,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"CoGS generates counterfactual explanations as step-by-step intervention paths that respect causal dependencies among features, so a negative decision is flipped only through achievable changes.","keywords":["counterfactual explanations","causal dependencies","answer set programming","s(CASP)","rule-based machine learning","FOLD-SE","actionable recourse","planning problem"],"falsifier":"Run CoGS on a dataset whose true causal structure is known from external interventions, e.g., a loan simulator where clearing debt raises credit score but editing credit score does not change debt. If the FOLD-SE-learned rules omit the dependency or encode it backwards, CoGS's recommended interventions will fail to produce the predicted dependent-feature change or will fail to flip the decision, directly refuting the claim that the counterfactuals are realistically achievable.","tokens_in":6456,"feed_emoji":"🛤️","tokens_out":9465,"duration_ms":81625,"temperature":0.7,"pith_summary":"The paper is trying to establish that counterfactual explanations—the standard 'what would need to change to flip a no into a yes'—are only trustworthy when the suggested changes respect the causal links between features. CoGS models the decision with rule-based machine learning, models feature dependencies as causal rules, and then solves a planning problem that starts at the negative outcome, ends at a positive one, and changes one feature at a time under those constraints. The payoff is an explanation that tells a user what to do first: for example, clear debt obligations so that credit score rises, rather than being told to raise the credit score directly. The paper reports preliminary results on three benchmark datasets showing that such causally ordered paths can be computed, with a dependent feature changing only after its cause has been changed.","feed_headline":"Counterfactuals now come with a causally ordered action plan","feed_subtitle":"Instead of 'raise your credit score,' it tells you to clear your debts first—and why that works.","key_machinery":"The central mechanism is a causally constrained planning problem encoded as an answer set program and executed by s(CASP), a goal-directed answer set programming system that uses program completion to turn 'if' rules into 'if and only if' rules. States are sets of feature-value pairs; each transition changes one feature value; and a causal dependency between feature values $P$ and $Q$ is encoded as both $P \\Rightarrow Q$ and $\\neg P \\Rightarrow \\neg Q$, so directly changing $Q$ is blocked unless the appropriate cause $P$ holds. FOLD-SE supplies the decision rules and the candidate causal rules, which the user approves. This machinery carries the argument because it converts 'a counterfactual the model would accept' into 'a counterfactual reachable by an ordered sequence of realistic actions'.","core_discovery":"On its own terms, the central discovery is that generating realistic counterfactuals for rule-based decision models can be reformulated as a planning problem over causally constrained feature states. CoGS takes decision rules from a rule-based learner, takes causal rules expressing which feature values imply (and are implied by) other feature values, and encodes both in a goal-directed answer set programming system. The system searches for an intervention path from the initial negative state to a goal state that flips the decision and is consistent with the causal rules. A dependent feature such as credit score is never directly altered; an earlier step changes its cause, such as debt obligations, and the causal rule propagates the effect. This is the paper's reason for calling the counterfactuals realistic: every step is an achievable action, and dependent changes appear as consequences of earlier steps.","pith_inferences":["Beyond the paper, the causal constraints come from association rules learned by FOLD-SE and verified informally, so the 'realistic' claim is only as strong as those learned dependencies; testing on externally validated causal graphs would settle how often the paths are truly achievable.","Beyond the paper, the planning formulation invites extensions the author does not develop here, such as attaching costs to interventions, enforcing immutability of protected features, and searching for lowest-cost rather than shortest paths.","Beyond the paper, one could use CoGS to generate counterfactual training examples for imbalanced datasets, as the author lists as planned work; the testable prediction is that minority-class performance improves only when the learned rules faithfully capture the decision process.","Beyond the paper, comparing CoGS's paths with nearest-neighbor counterfactuals on a dataset with known causal structure would quantify the realism gain and show whether the extra constrained steps are worth the added complexity."],"forward_implications":["For rule-based decision systems, CoGS returns an ordered plan of interventions, not just a target feature vector, so a user knows which change to make first.","Causally dependent features are changed only as effects of changing their causes; direct edits to them are excluded, and this is what makes the counterfactual achievable.","CoGS can enumerate all possible paths to a counterfactual and can find minimal paths by iterating over path length, so the returned explanations are both causally consistent and minimal.","The framework is independent of the particular rule learner: any rule-based machine learning algorithm can supply the decision and causal rules, and statistical models can be included by first approximating them with rules.","Preliminary experiments on German, Adult, and Car Evaluation data produced counterfactuals and paths in times of about one to three seconds, indicating that the approach is computationally feasible on small structured problems."],"supporting_citations":[{"why":"Supplies the decision rules and the learned causal rules that CoGS encodes.","marker":"[16]"},{"why":"Provides the goal-directed ASP engine that executes the program without grounding and uses completion to model causality.","marker":"[1]"},{"why":"Gives the ASP formulation of planning with states, transitions, and constraints that CoGS adapts to non-independent interventions.","marker":"[8]"},{"why":"Defines the counterfactual explanation task CoGS targets and is the baseline whose causality-blind suggestions motivate the work.","marker":"[15]"},{"why":"Supplies the formal cause-effect criterion ($P \\Rightarrow Q$ and $\\neg P \\Rightarrow \\neg Q$) used to model feature dependencies.","marker":"[13]"},{"why":"Earlier work emphasizing causal rules for algorithmic recourse, which CoGS extends by adding the iff-completion property of s(CASP).","marker":"[11]"}],"fun_headline_variants":["Realistic counterfactuals: follow the causal chain","From no to yes: causally realistic action plans","CoGS: a plan to change outcomes realistically","Causal constraints shape counterfactual action plans","Rule-based counterfactuals: respect cause and effect"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that the rules FOLD-SE learns about feature dependencies are genuine causal laws; if they are only correlations, then the paths CoGS certifies as realistic may fail to produce the predicted changes or the promised outcome.","fun_headline_variants_meta":{"raw":{"variants":["Realistic counterfactuals: follow the causal chain","From no to yes: causally realistic action plans","CoGS: a plan to change outcomes realistically","Causal constraints shape counterfactual action plans","Rule-based counterfactuals: respect cause and effect"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000361,"raw_usage":{"total_tokens":1885,"prompt_tokens":817,"completion_tokens":1068,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":433,"completion_tokens_details":{"reasoning_tokens":994}},"tokens_in":433,"tokens_out":1068,"duration_ms":9282,"temperature":1.0,"reasoning_tokens":994,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-07T22:13:04.911970+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run CoGS on a dataset whose true causal structure is known from external interventions, e.g., a loan simulator where clearing debt raises credit score but editing credit score does not change debt. If the FOLD-SE-learned rules omit the dependency or encode it backwards, CoGS's recommended interventions will fail to produce the predicted dependent-feature change or will fail to flip the decision, directly refuting the claim that the counterfactuals are realistically achievable.","supporting_citations":[{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Supplies the decision rules and the learned causal rules that CoGS encodes."},{"cited_title":"Theory Pract","cited_arxiv_id":null,"evidence_quote":"Provides the goal-directed ASP engine that executes the program without grounding and uses completion to model causality."},{"cited_title":"In: Proc","cited_arxiv_id":null,"evidence_quote":"Earlier work emphasizing causal rules for algorithmic recourse, which CoGS extends by adding the iff-completion property of s(CASP)."}],"review_version":1}