{"id":"46d02516-37ae-40c5-9b9a-bb7ba4503ccb","arxiv_id":"1908.03840","paper_version":2,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":6,"one_line_summary":"LoRMIkA mines k-optimal class-association rules from a generated local neighbourhood to explain black-box predictions with supporting, contradicting, and counterfactual rules.","lead":"LoRMIkA explains a single prediction from any machine learning model by mining the best class-association rules from a neighborhood of similar data points, producing four kinds of rules including counterfactuals. It is a new local-interpretability tool for tabular data that outperforms or matches Anchor and LORE on coverage, confidence and lift in the reported experiments.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Quantitative evaluation is circular: all fidelity metrics are computed on the synthetic local dataset used to mine the rules, so the central claim of competitive local accuracy is not actually tested.","rationale":"The reader's weakest assumption identifies the synthetic neighbourhood generation as the fragile point; I agree that this is a serious risk, but I would sharpen it: the evaluation is circular even if the synthetic points were on-manifold. The reported coverage, confidence, and lift are computed on the same data used to mine the rules, so they quantify how well a rule fits the rule-mining training set, not whether the rule faithfully explains the black-box model's local decisions. This directly threatens the central claim that LoRMIkA is a reliable local explainer. The off-manifold concern compounds the circularity because the rule-mining distribution can be far from the true local data distribution, making the internal metrics even less informative. The perfect Jaccard stability in Table 5 is a concrete red flag that the experimental protocol may be comparing identical or degenerate rule sets across runs. These issues are addressable: a held-out local evaluation, a no-generation ablation, and a seed-controlled stability rerun would settle whether the method's apparent advantage is real. The paper does have independent support: it uses a well-known admissible search (OPUS), provides a qualitative worked example, makes code available, and states a clear algorithm. None of that, however, substitutes for the missing fidelity measurement. Since the concerns are empirical and fixable rather than formal contradictions, the reader's CONDITIONAL verdict remains appropriate; my read does not move it to accept or reject.","tokens_in":20511,"tokens_out":3885,"duration_ms":48002,"concrete_test":"Re-run the quantitative evaluation with a held-out local sample. For each explained instance, split the selected neighbourhood into a mining set and a test set (e.g., 80/20) before generating synthetic instances; mine the k-optimal rules on the mining set only, then measure rule-consequent agreement with the black-box model on the held-out local instances and on fresh synthetic points generated from a separate seed. Also re-run Tables 1–3 with NGenInst set to 0 (no synthetic generation); if the coverage/lift advantage disappears or the held-out fidelity is much lower than the reported confidence, the central claim is an artifact of the generator rather than of the mining method.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The load-bearing concern is that the paper never validates the mined rules against the black-box model on independent local data. The quantitative sections (Tables 1–3) report coverage, confidence, and lift computed on the same combined set SelecInst ∪ GenInst that was used to generate the rules in the first place. These are internal statistics of the rule on its own training distribution, not estimates of how well the rule predicts the black-box output on held-out local points. The qualitative claims and the abstract's reference to 'local accuracy' therefore rest on an evaluation that cannot detect a rule that is confident on synthetic crossover/mutation points but wrong about the model's actual behaviour near the explained instance. The generator makes this worse: mutation in Equation 9 with σ ∈ [0.5, 1] produces points outside the training data manifold, and Section 3.5 simply assumes that the black-box predictions on such off-manifold points are informative. If they are not, all four rule types describe artifacts of the generation procedure. The stability result in Table 5 is another symptom: a Jaccard coefficient of 1.00 with zero variance in every dataset and model, despite randomized neighbourhood subsampling in Algorithm 2, is only plausible if the comparison is somehow selecting identical rules by construction; as reported, it makes the stability claim vacuous. Finally, the counterfactual interpretation of ℜ−hβ rules is an overclaim: a mined rule with LHS false and RHS false is a correlational association rule, not a demonstrated minimal or actionable change that inverts the decision.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes LoRMIkA, a model-agnostic local explainer for tabular classifiers. The method selects a neighbourhood of the instance to be explained via Euclidean distance and an exponential similarity kernel, augments it with synthetic instances generated by crossover and mutation (Equations 8 and 9), obtains black-box predictions for the combined set, and mines k-optimal class association rules using the OPUS search algorithm under objectives such as confidence, lift, leverage, coverage, and support. The mined rules are classified into four types — current supporting, current contradicting, hypothetically supporting, and hypothetically contradicting — according to whether their left- and right-hand sides match the explained instance and its prediction. Experiments compare LoRMIkA with Anchor and LORE on COMPAS, Adult, German, and Covertype datasets using coverage, confidence, lift, running time, stability, and rule simplicity, supplemented by a qualitative COMPAS example.","tokens_in":20774,"tokens_out":7100,"duration_ms":70146,"significance":"If the empirical claims were supported, the contribution would be useful: the four-rule typology extends the outputs of Anchor and LORE, and the use of OPUS provides a principled way to optimize different rule-quality measures and to generate both predictive and interesting rules. The high-level algorithm is clearly presented, a GitHub implementation is referenced, and the qualitative COMPAS example is coherent and illustrative. The main value is conceptual and methodological. However, the quantitative evidence for the central claim of competitive local accuracy is currently missing, and the stability result as reported is not credible; these issues prevent the paper from being accepted in its present form.","major_comments":[{"comment":"The coverage, confidence, and lift values in Tables 1–3 are computed on the same combined neighbourhood CombInst = SelecInst ∪ GenInst that is used to mine the rules in MOGenRules (Algorithm 1, lines 5–7). These are therefore in-sample descriptive statistics of the rules on their own training distribution, not estimates of how well the rules match the black-box model on independent local points. The abstract's claim of 'competitive results in terms of local accuracy' is not tested by these tables. Please add a direct fidelity evaluation: hold out a random subset of the generated neighbourhood, or draw a fresh local sample, and report the precision and coverage of the top-k rules' predictions against the black-box labels on that held-out set.","section":"§4.6.1, Tables 1–3, Algorithm 1"},{"comment":"The reported Jaccard stability is 1.00 with standard deviation 0.00 for every dataset and every black-box model, including Covertype, despite the randomised neighbourhood sampling in Algorithm 2 (line 19) and the randomised instance generation in Algorithm 3 (α, σ, and random selection of x, y, z). As reported, this is only plausible if the compared runs are not actually independent or if the comparison selects identical rules by construction; otherwise the stability claim is vacuous. Please specify how the 10 runs differ (random seeds, resampling of the 50 instances), report the distribution of Jaccard values rather than only the mean, and state whether the comparison uses the single best rule per objective or the full rule set.","section":"§4.6.2, Table 5"},{"comment":"Algorithm 3 as written does not implement the parameter NGenInst: it draws one pair x, y for crossover and one triple x, y, z for mutation, and returns a single Icrossover and a single Imutation. There is no loop or repetition that would produce the NGenInst synthetic instances used in Algorithm 1, and the text's statement that '50% of the total instances' are generated by crossover is not reflected in the pseudocode. In the same vein, Section 4.4 does not report the values of k, NGenInst, M, or the number of discretization bins used in the experiments. These omissions make the experimental results unreproducible from the manuscript. Please correct the pseudocode and report the parameter values.","section":"§3.5, Algorithm 3"},{"comment":"The mutation operator Imutation = x + (y − z)·σ with σ ∈ [0.5, 1] can place generated points outside the convex hull of the selected neighbourhood and outside the training data manifold, since y and z are arbitrary neighbours. Section 3.5 simply assumes that black-box predictions on such off-manifold points are informative about the local decision logic of the instance to be explained ('we assume that the generated instance behaves similar...'). This assumption is load-bearing because every rule quality measure in Tables 1–3 is computed on predictions of these generated points. Please provide evidence for this assumption, for example by measuring the distance of generated points to the training data and showing that rules mined with and without mutation points are stable, or by replacing mutation with an in-manifold augmentation procedure.","section":"§3.5, Equations (8)–(9)"},{"comment":"The interpretation of hypothetically contradicting rules as counterfactuals is an overclaim. A rule with LHS=false for the explained instance and RHS=false is a statistical association in the neighbourhood; it does not imply that changing the instance's features to satisfy the LHS would invert the prediction, because (i) confidence is not an interventional estimate and (ii) satisfying the LHS may require changing features jointly in ways that violate feature dependencies and the black-box model's behaviour away from the neighbourhood. The qualitative example ('if this prisoner was older than 29 and priors count was less than 4 ... would be identified as Medium Low') should be rephrased as a description of similar instances that satisfy the LHS, or supported by a proper counterfactual evaluation (e.g., perturbation experiments on the black-box model).","section":"§3.7, ℜ−hβ rules"}],"minor_comments":[{"comment":"The abstract says 'three different datasets' and Section 4.1 says 'three real-world datasets', but the experiments actually use four datasets (Adult, COMPAS, German, and Covertype); please correct the inconsistent counts.","section":"Abstract and §4.1"},{"comment":"Equation (7) writes 'x−x′2' where a squared Euclidean norm ||x − x′||² is intended; the notation should be fixed.","section":"Equation (7)"},{"comment":"Section 4.4 says 'we use a common set of parameters in all three datasets', but four datasets are used; this appears to be a typo.","section":"§4.4"},{"comment":"Algorithm 2, line 19 uses 'eInstGtrSctGrp.choice(InstM)' without defining InstM or the choice operation; the intended uniform random subsample of size M should be stated explicitly.","section":"Algorithm 2, line 19"},{"comment":"References [40] and [41] are both Webb, 'Filtered-top-k association discovery' (2011); the duplicate should be removed and all citations reconciled.","section":"References"},{"comment":"Table 4 reports run-time only for Covertype, and the text says 'average running time over 50 instances'; please clarify whether the same evaluation was performed for the other datasets and why only Covertype is shown.","section":"Table 4"}],"recommendation":"major_revision","confidential_remarks":"The paper is within scope for an interpretability or machine-learning journal. The main concern is that the quantitative evaluation is in-sample, and the stability result as reported is not credible. If the authors add a held-out fidelity experiment, fix the stability reporting, and correct the pseudocode and parameter reporting, the contribution could become acceptable. I do not see a central algorithmic error that would justify rejection."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Short version: this is a worthwhile algorithmic contribution that needs a serious referee, but the experiments should not be taken at face value. The core idea—mine k-optimal class association rules from a local neighbourhood via OPUS and organize them into supporting, contradicting, hypothetical, and counterfactual rule types—is genuinely new relative to Anchor and LORE. The qualitative COMPAS walkthrough is instructive and shows what the four rule types can do. The authors also make a fair point that the most predictive rule is not always the best explanation, and OPUS lets them optimize for lift, leverage, coverage, and so on. The method section is clear enough to reproduce from the text, and the code is on GitHub.\n\nThe soft spots are in the evaluation. The abstract promises local accuracy, but Tables 1–3 report coverage, confidence, and lift computed on the same combined set (SelecInst ∪ GenInst) used to generate the rules. Those are in-sample descriptive statistics; they do not measure whether the rules match the black-box on independent local points. The stability result in Table 5 is a bigger red flag: a Jaccard coefficient of 1.00 with zero variance across all datasets and black-boxes, despite randomized neighbourhood subsampling, is not credible as reported. Either the comparison is selecting identical rules by construction, or the metric needs a better explanation. The values of k and NGenInst are also unreported, and there is no sensitivity analysis for them. The counterfactual interpretation of the ℜ−hβ rules is stronger than the evidence: a mined rule with LHS false and RHS false is a correlational association, not a demonstrated minimal actionable change that inverts the decision. The instance-generation mutation in Equation 9 can push points outside the training manifold, and the paper assumes the black-box’s predictions there are informative without validating that assumption.\n\nThese are all addressable, and the core method does not collapse because of them. The qualitative example shows real potential, and the four-rule taxonomy is useful to the community. But the competitive-accuracy claim is not supportable from the current quantitative sections. If this lands on my desk, I would send it to peer review with a request for major revision: it deserves referee time because the method is well specified and the rule taxonomy is a genuine contribution. I would not cite it in its current form, but I would want to see the revised version.","headline":"LoRMIkA's four-rule taxonomy and OPUS-based search are a real step forward for local rule-based explainability, but the reported metrics do not yet establish the accuracy claim.","tokens_in":21346,"tokens_out":1997,"would_cite":false,"duration_ms":22768,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Mining the k best association rules in a local neighbourhood explains a black-box prediction with four rule types.","keywords":["local interpretability","class association rules","k-optimal rules","OPUS search","counterfactual explanations","model-agnostic explainer","rule-based explanations","tabular classification"],"falsifier":"On a benchmark where the true local decision boundary is known (for example, a piecewise-linear classifier on synthetic data), build LoRMIkA's neighbourhood and compare each mined rule's confidence and lift against actual nearby training points of the same class; if the rules' confidence collapses when evaluated on genuine close points rather than synthetic ones, the neighbourhood-generation assumption fails.","tokens_in":20230,"feed_emoji":"🧩","tokens_out":6975,"duration_ms":68633,"temperature":0.7,"pith_summary":"LoRMIkA is a model-agnostic explainer for tabular classifiers: it builds a local neighbourhood around the instance to be explained, labels it with the black-box model's predictions, and mines class association rules from that local dataset. A class association rule is an if-then statement whose condition is a set of feature values and whose conclusion is the predicted class. The paper's central argument is that the most predictive rules are not always the best explanations, so the rule search is driven by a user-chosen objective — support, coverage, confidence, lift, or leverage — and returns the k best rules under that objective. The rules are organized into four types: conditions that support the current prediction, conditions that contradict it, hypothetical conditions that would strengthen it, and counterfactual conditions that would reverse it. On the Adult, COMPAS, German, and Covertype datasets, LoRMIkA reports coverage, confidence, lift, stability, and rule simplicity that are competitive with or better than two existing rule-based explainers, Anchor and LORE.","feed_headline":"Mine the k best rules to explain any black-box prediction","feed_subtitle":"Predictive rules are not always the best explanations; LoRMIkA mines four rule types, counterfactuals included.","key_machinery":"The central object is the k-optimal class association rule set, mined with the OPUS search algorithm. A class association rule is an if-then rule whose antecedent is a boolean condition on feature values and whose consequent is the predicted class; k-optimal means the top k rules under a user-chosen measure — support, coverage, confidence, lift, or leverage — returned without heuristic pruning. OPUS is an admissible unordered search that guarantees the top-k result and uses Fisher's exact test plus an m-estimate correction to avoid spurious low-coverage rules. The supporting machinery is the neighbourhood generator: an exponential similarity kernel over Euclidean distance selects close training points, and crossover (Equation 8) and mutation (Equation 9) synthesize additional points that inherit categorical values from the closest parent. The four-type contingency table then converts the mined rules into explanations for the specific instance.","core_discovery":"The paper claims that an admissible search called OPUS can find, in a locally generated neighbourhood, the k-optimal class association rules for any of the standard association-rule objectives, and that the resulting rule set gives a complete local explanation. The neighbourhood is formed by selecting the most similar training instances to the explained instance under an exponential similarity kernel, then generating synthetic neighbours by crossover and mutation; all of these instances are labelled by the global model, and rules are mined from the labelled set. Rules are then partitioned according to whether their antecedent matches the instance and whether their consequent matches the prediction, producing four types. The claim is that this yields both predictive rules (high confidence) and interesting rules (high lift or leverage), and that on the tested datasets LoRMIkA matches or beats Anchor and LORE on coverage, confidence, lift, stability, and simplicity, while additionally providing current-contradicting and hypothetical-supporting rules that the baselines do not provide.","pith_inferences":["Because the objective is a parameter, one could extend LoRMIkA to a multi-objective setting that requires both high coverage and high counterfactual actionability; the paper does not test such a combined objective.","The four-type taxonomy suggests a completeness metric that the paper does not define: every true nearby point of the predicted class should be covered by a current supporting rule, and every true nearby point of the opposite class should be covered by a counterfactual rule.","The fixed Jaccard stability of 1.00 likely reflects a deterministic neighbourhood selection given the training set; if so, it measures variance in the baselines' sampling rather than a difference in explanatory content."],"forward_implications":["Users can choose the search objective, so the same framework produces predictive explanations (optimized for confidence) or interesting ones (optimized for lift or leverage).","The four rule types give a practitioner both the reasons behind a prediction and concrete conditions that would reverse it, without a separate counterfactual search.","Because OPUS is admissible, the returned k rules are guaranteed optimal for the chosen objective rather than being a heuristic approximation.","In the reported experiments, LoRMIkA achieves higher or comparable coverage, confidence, and lift than Anchor and LORE, with perfect Jaccard stability of 1.00 for the current-supporting rule across runs.","LoRMIkA's explanation time stays roughly constant around 38 seconds across global models on Covertype, while the baselines vary more widely."],"supporting_citations":[{"why":"Supplies the OPUS admissible unordered search algorithm that finds the k-optimal rules.","marker":"[38]"},{"why":"Defines filtered top-k association discovery and the k-optimal rule objectives (support, coverage, confidence, lift, leverage).","marker":"[40]"},{"why":"Provides the local-neighbourhood sampling idea and the kernel width setting used in the similarity score.","marker":"[34]"},{"why":"Supplies the LORE baseline and the evaluation framework with datasets and experimental protocol.","marker":"[13]"},{"why":"Supplies the Anchor baseline that LoRMIkA is compared against for coverage, confidence, lift, and simplicity.","marker":"[35]"},{"why":"Supplies the differential-evolution mutation operator used to generate synthetic neighbourhood instances.","marker":"[37]"},{"why":"Provides the m-estimate correction that OPUS applies to confidence and lift to avoid overestimating low-coverage rules.","marker":"[11]"},{"why":"Grounds the paper's argument that the most predictive rules differ from the rules that explain best.","marker":"[27]"},{"why":"Surveys interestingness measures including confidence, lift, leverage, and coverage used as optimization objectives.","marker":"[12]"}],"fun_headline_variants":["k-optimal rule mining for local black-box explanations","LoRMIkA: local rules that are both predictive and interesting","Counterfactual rules and k-optimal associations for local explanations","Mine k-optimal rules, including counterfactuals, from any black-box","Explaining black-box predictions with k-optimal association rules"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The synthetic neighbours generated by crossover and mutation are assumed to carry the same decision logic as the instance being explained, even when they lie off the training-data manifold.","fun_headline_variants_meta":{"raw":{"variants":["k-optimal rule mining for local black-box explanations","LoRMIkA: local rules that are both predictive and interesting","Counterfactual rules and k-optimal associations for local explanations","Mine k-optimal rules, including counterfactuals, from any black-box","Explaining black-box predictions with k-optimal association rules"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000731,"raw_usage":{"total_tokens":3277,"prompt_tokens":953,"completion_tokens":2324,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":569,"completion_tokens_details":{"reasoning_tokens":2235}},"tokens_in":569,"tokens_out":2324,"duration_ms":18489,"temperature":1.0,"reasoning_tokens":2235,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-14T14:01:01.475034+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"On a benchmark where the true local decision boundary is known (for example, a piecewise-linear classifier on synthetic data), build LoRMIkA's neighbourhood and compare each mined rule's confidence and lift against actual nearby training points of the same class; if the rules' confidence collapses when evaluated on genuine close points rather than synthetic ones, the neighbourhood-generation assumption fails.","supporting_citations":[{"cited_title":"OPUS: An efficient admissible algorithm for unordered search","cited_arxiv_id":null,"evidence_quote":"Supplies the OPUS admissible unordered search algorithm that finds the k-optimal rules."},{"cited_title":"Filtered-top- k association discovery","cited_arxiv_id":null,"evidence_quote":"Defines filtered top-k association discovery and the k-optimal rule objectives (support, coverage, confidence, lift, leverage)."},{"cited_title":"Why should I trust you?: Explaining the Predictions of Any Classifier","cited_arxiv_id":null,"evidence_quote":"Provides the local-neighbourhood sampling idea and the kernel width setting used in the similarity score."},{"cited_title":"Anchors: High-Precision Model-Agnostic 25 explanations","cited_arxiv_id":null,"evidence_quote":"Supplies the Anchor baseline that LoRMIkA is compared against for coverage, confidence, lift, and simplicity."},{"cited_title":"Differential evolution – a simple and efficient heuristic for global optimization over continuous spaces","cited_arxiv_id":null,"evidence_quote":"Supplies the differential-evolution mutation operator used to generate synthetic neighbourhood instances."},{"cited_title":"Using the m-estimate in rule induction.Journal of computing and information technology , 1(1):37–46, March 1993","cited_arxiv_id":null,"evidence_quote":"Provides the m-estimate correction that OPUS applies to confidence and lift to avoid overestimating low-coverage rules."},{"cited_title":"Supervised descriptive rule discovery: A unifying survey of contrast set, emerging pattern and subgroup mining","cited_arxiv_id":null,"evidence_quote":"Grounds the paper's argument that the most predictive rules differ from the rules that explain best."},{"cited_title":"Interestingness measures for data mining: A survey","cited_arxiv_id":null,"evidence_quote":"Surveys interestingness measures including confidence, lift, leverage, and coverage used as optimization objectives."}],"review_version":1}