{"id":"9b807362-a84a-4eea-9dc0-2f9299e1dab5","arxiv_id":"2509.03169","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":2,"one_line_summary":"Equally accurate models trained on the same driving scenes often point to different features as the cause of an action, so the explanation you get depends on which model you happen to train.","lead":"This paper measures how much explanations differ among equally accurate AI models for predicting driver actions in real driving scenes. It finds that agreement on which features matter is low, especially for graph neural networks, casting doubt on the reliability of post-hoc explanations in autonomous driving.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Low Kendall's W may reflect attribution-method noise rather than Rashomon multiplicity; the paper is internally inconsistent about whether SHAP or integrated gradients was used.","rationale":"The reader's conditional verdict is appropriate. The load-bearing condition is that low explanation agreement reflects genuine model multiplicity rather than artifacts of the attribution method. I agree with the reader's weakest assumption: the paper never measures attribution noise or compares alternative attribution methods, and the text contradicts itself on whether integrated gradients or SHAP was used. This matters directly for the central claim because a low Kendall's W across models is exactly what one would expect from a high-variance attribution estimator applied to a single model. A repeated-attribution baseline on a fixed model is a cheap and decisive control. I do not see grounds to reject outright: the study is an empirical demonstration in a new domain, uses a real-world dataset, and provides a code/data link; the Limitations section also honestly acknowledges the narrow model-family scope. However, until the attribution-noise control is added, the 'inherent property' claim should remain conditional rather than being accepted as established.","tokens_in":11160,"tokens_out":4722,"duration_ms":58751,"concrete_test":"Using the released Zenodo models, take one fixed trained GNN and one input scene and compute feature attributions 20 times with integrated gradients using randomized baselines or algorithmic seeds. Compute Kendall's W across these repeated attributions of the same fixed model. If the single-model W is statistically indistinguishable from the reported graph-based W≈0.07, then attribution noise alone can reproduce the headline result and the Rashomon interpretation is unsupported. If W remains high (>0.8), the concern is refuted for this setting.","verdict_should_be":"UNCHANGED","load_bearing_attack":"To support the claim that explanation ambiguity is an inherent property of the problem, the paper must show that low agreement (W≈0.07 for GNNs, 0.32 for gradient boosting) is caused by the existence of multiple equally good models, not by variability or noise in the explanation estimator. The protocol has no no-model-change control: agreement is measured across models using one attribution pipeline, so a noisy attribution method would produce equally low W even for a single fixed model. The manuscript is internally inconsistent about the pipeline: the graph-based method section states integrated gradients '(used in the experiments)', Data Collection says SHAP, and Technical Setup attributes all explanations to Captum. Captum is a PyTorch library and does not provide the standard TreeSHAP implementation for LightGBM, so it is unclear which attribution quantities were actually ranked. Moreover, the Rashomon set is defined with a broad 5% validation band and no per-model validation losses are reported, so 'equally accurate' is not established. The observed disagreement is therefore also consistent with attribution instability or performance heterogeneity; the inherent-property conclusion overreaches the evidence.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper studies the Rashomon effect for explanation generation in automated driving scene understanding. It builds Qualitative Explainable Graphs (QXGs) from nuScenes/DriveLM and trains two model families: pair-based LightGBM gradient boosting models and graph-based GNNs. For each family, multiple models are trained with different random seeds, and validation Rashomon sets are selected using an epsilon of 0.05. Feature attributions are computed for the annotated relevant object and ranked; agreement across models is quantified by Fleiss' kappa for top-k feature sets and Kendall's W for full rankings. The main empirical finding is low-to-moderate explanation agreement: Kendall's W is 0.32 for pair-based models and 0.07 for graph-based models. The paper interprets this as evidence that explanation ambiguity is an inherent property of the problem, not merely a modeling artifact.","tokens_in":11393,"tokens_out":3587,"duration_ms":42500,"significance":"If correct, the finding would be a useful caution for post-hoc XAI in safety-critical domains, extending earlier Rashomon-XAI work from controlled settings to a real driving dataset. The use of two structurally different model classes and a public driving dataset is a concrete step, and the authors provide a reproducibility link. However, the strength of the conclusion currently exceeds what the experimental protocol can establish: the attribution pipeline is under-specified, no no-model-change control is included, and only one dataset/split/model representation is examined. The empirical observation is valuable, but the 'inherent property' framing is not yet supported.","major_comments":[{"comment":"The attribution method is described inconsistently. The graph-based section states integrated gradients are 'used in the experiments', Data Collection says SHAP, and Technical Setup attributes all explanations to Captum. For LightGBM, Captum does not natively provide TreeSHAP, so it is unclear what quantity was ranked. More importantly, the low Kendall's W values could reflect instability of the attribution estimator rather than genuine model multiplicity. The paper needs a fixed-model control: for a single trained model, compute attributions multiple times (e.g., different baselines or background samples) and report the resulting W. Without this, the central claim that disagreement is due to the Rashomon set is not established.","section":"Graph-based Action Explanation; Data Collection; Technical Setup"},{"comment":"The claim that explanation ambiguity is 'an inherent property of the problem, not just a modeling artifact' goes beyond the evidence. The study uses one dataset (nuScenes subset), one train/validation split, one scene representation (QXG), two model families, one epsilon value, and one attribution pipeline. Such a design cannot establish an inherent property. The conclusion should be reframed to 'observed in this setting' or supported by additional experiments (e.g., different splits, model families, attribution methods, epsilon sensitivity). This is not a wording issue: the overclaim is load-bearing for the paper's stated contribution.","section":"Abstract; Conclusion"},{"comment":"The operationalization of the Rashomon set is unclear. Definition 2 uses an additive epsilon on validation loss, but the text says 'at least 95% performance of the best model' and later mentions models are 'within 2% performance'. These are different selection criteria, and no per-model validation losses or accuracies are reported. Without showing the validation performance distribution, 'equally accurate' is not established, and a broad 5% band could include models with materially different behavior. Please report the validation performance of all selected models and clarify whether epsilon is additive in loss or relative to the best model's performance.","section":"Experimental Setup, Model Selection; Eq. (2)"}],"minor_comments":[{"comment":"The paper should clarify whether Fleiss' kappa is computed with fixed or free marginal probabilities, since the value depends on this choice. The cited Randolph reference suggests free-marginal kappa, but the text does not state it explicitly.","section":"Metrics, Eq. (3)"},{"comment":"The formula W = 12S / (m^2(n^3 - n)) omits a tie correction. Feature attributions often contain ties, so please state whether ties were handled and, if so, how.","section":"Metrics, Kendall's W"},{"comment":"The legend labels 'Model' with 'Pair-based' and 'Graph-based' is clear, but the shaded areas are not defined in the caption. Please specify whether they are standard deviations or confidence intervals.","section":"Figure 5"},{"comment":"Typo: 'it’s definition' should be 'its definition'. Also, the reference to 'M ¨uller' has inconsistent spacing in the bibliography.","section":"Background"},{"comment":"The footnote 'Online: https://doi.org/10.5281/zenodo.17045085' should describe exactly what is released (code, data, model weights) and the version used, so that the experiments are reproducible.","section":"Technical Setup"}],"recommendation":"major_revision","confidential_remarks":"The main blocker is the attribution-method inconsistency and the missing no-model-change control, which are fixable but require additional experiments. The 'first empirical quantification' claim should also be positioned against Müller et al. more precisely; the current abstract and conclusion risk overclaiming. If the authors add the control and temper the 'inherent' language, the paper could be a solid empirical contribution."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Quick take: the paper does something real. It shows that differently seeded gradient-boosting and GNN models, all within a validation epsilon-Rashomon set, produce quite different feature attributions for action prediction in real driving scenes. That is a useful extension of the Rashomon-for-XAI line to a practical domain, and the finding that GNNs have much lower explanation agreement (Kendall's W near 0.07) than gradient boosting (near 0.32) is genuinely interesting. The paper defines the validation Rashomon set clearly, uses appropriate agreement metrics, and is honest about several limitations (symmetries, overparameterization, model-class coverage). There is also a Zenodo DOI for replication, which deserves credit.\n\nThe soft spots are significant but fixable. The internal inconsistency about the attribution method is real: the graph-based section says integrated gradients was used in experiments, Data Collection says SHAP, and Technical Setup says Captum. Since Captum does not provide TreeSHAP for LightGBM, the reader cannot tell which attribution quantities were actually ranked. That must be cleaned up. More substantively, the paper never controls for attribution-method instability: it measures agreement across models with a single attribution pipeline but does not measure how stable each method is on a single model or across near-identical models. So the low W could reflect attribution sensitivity rather than genuine model multiplicity. That matters because the abstract and conclusion claim explanation ambiguity is an inherent property of the problem. The evidence (one dataset, one split, two model families, one epsilon) supports a more modest claim: in this setting, similarly performing models disagree. It does not yet justify 'inherent.'\n\nThe paper is still worth refereeing. The question is important for trust in post-hoc explanations, the empirical design is reasonable, and the flaws are addressable. I would send it out with a request for major revision: clarify the attribution pipeline, add a no-model-change control or attribution-stability analysis, report per-model validation performance, and soften the inherent-property conclusion. For readers in XAI or automated-driving safety, it is a relevant cautionary study; for cognitive science or theory, less so.","headline":"A useful empirical caution for driving XAI, but the 'inherent property' claim and the unresolved SHAP/integrated-gradients ambiguity need serious work before publication.","tokens_in":691,"tokens_out":720,"would_cite":false,"duration_ms":50534,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Equally good driving-scene models give sharply different explanations for the same action, indicating that explanation ambiguity is intrinsic to the task.","keywords":["Rashomon effect","explainable AI","feature attribution","scene understanding","autonomous driving","graph neural networks","gradient boosting","qualitative explainable graphs"],"falsifier":"Re-run the same GNN and boosting models while replacing SHAP with another attribution method (e.g., integrated gradients, as the method section names) or repeatedly sampling SHAP background data for a single fixed model; if the near-zero Kendall's W disappears or matches the within-model attribution noise, the claim that ambiguity is inherent to the problem would be undermined.","tokens_in":11082,"feed_emoji":"🚗","tokens_out":4926,"duration_ms":55145,"temperature":0.7,"pith_summary":"The paper asks whether the Rashomon effect—many equally accurate models trained on the same data—undermines explainable AI for autonomous driving. It trains Rashomon sets of gradient-boosting models and graph neural networks on Qualitative Explainable Graph (QXG) scene representations derived from nuScenes and DriveLM, then measures how much the models' feature-attribution explanations agree. For the boosted models, agreement on full feature rankings is moderate (Kendall's W ≈ 0.32); for the GNNs it is near zero (W ≈ 0.07), even though all models in each set perform almost identically. The paper interprets this as evidence that explanation ambiguity is an inherent property of the action-explanation problem, and argues that trustworthy XAI should present a set of possible rationales rather than a single 'ground-truth' explanation.","feed_headline":"Driving-scene explanations diverge even for equally good models","feed_subtitle":"GNN explanations agree barely above chance (W ≈ 0.07); boosted models show only moderate concordance (W ≈ 0.32).","key_machinery":"The central object is the Qualitative Explainable Graph (QXG), a symbolic spatio-temporal graph whose nodes are scene objects and whose edges carry qualitative relation labels from spatial and trajectory calculi. Around this, the paper builds a validation Rashomon set: models trained on the same data whose validation loss is within epsilon of the best model. Explanations are generated as feature-attribution rankings via SHAP, and agreement is measured by Fleiss' kappa (for top-k feature selection) and Kendall's W (for full-rank ordering). The comparison of explanations across a Rashomon set is the mechanism that carries the argument: near-identical performance with divergent attributions is","core_discovery":"This paper claims that explanation ambiguity is not a modeling artifact but a property of the scene-understanding problem itself. To support this, it constructs validation Rashomon sets—sets of models whose validation loss is within a small epsilon (0.05) of the best model—for two model classes on a real-world driving dataset: 100 gradient-boosting decision trees solving a pair-based action explanation task, and 116 graph neural networks solving a graph-based version, of which 32 met the Rashomon criterion. Across 2,131 ego-vehicle action scenes, the models' SHAP feature attributions were compared using Fleiss' kappa on top-k features and Kendall's W on full rankings. The GNNs showed very lo","pith_inferences":["The paper does not establish that the near-zero GNN agreement comes from model multiplicity rather than from the instability of SHAP itself; a control using a second attribution method or repeated SHAP sampling on a single fixed model would separate these sources.","The large gap between boosted trees (W ≈ 0.32) and GNNs (W ≈ 0.07) suggests that model class and capacity shape explanation ambiguity independently of the task; a testable prediction is that simpler models on the same QXG features will always exhibit higher explanation concordance than expressive GNNs.","Since the action-explanation task is deliberately simplified to single-cause labels, the measured ambiguity is likely a lower bound; labeling multi-causal scenes would probably reveal even larger explanation divergence."],"forward_implications":["If explanation ambiguity is inherent, a single post-hoc explanation for a driving decision cannot be treated as the model's true reasoning in safety audits.","Consensus explanations computed across a Rashomon set could replace single-model attribution as a more stable basis for human review.","Explanation variance across a Rashomon set becomes a measurable form of uncertainty, useful for deciding when a prediction needs extra scrutiny.","Training with human relevance labels as an auxiliary loss (or other explanation-regularizing objectives) may reduce but cannot eliminate the multiplicity of rationales."],"supporting_citations":[{"why":"Introduces the Rashomon effect as the multiplicity of equally good models, the theoretical basis of the study.","marker":"Breiman 2001"},{"why":"Formalizes the Rashomon set and model class reliance, motivating the measurement of explanation variation across models.","marker":"Fisher, Rudin, and Dominici 2019"},{"why":"The closest prior work empirically demonstrating the Rashomon effect for explainable ML; this paper extends that finding to real-world scene understanding.","marker":"Müller et al. 2023"},{"why":"Provides SHAP, the feature attribution method used to generate the explanations whose agreement is measured.","marker":"Lundberg and Lee 2017"},{"why":"The nuScenes dataset supplies the real-world driving scenes and object annotations used in all experiments.","marker":"Caesar et al. 2020"},{"why":"DriveLM provides the human-annotated relevance labels that define the correct action-causing object for each scene.","marker":"Sima et al. 2023"},{"why":"Defines the QXG representation and the pair-based action explanation approach that the interpretable model class builds on.","marker":"Belmecheri et al. 2024"},{"why":"Supplies the GNN architecture adapted here for graph-based action explanation and node-level attribution.","marker":"Belmecheri et al. 2025b"}],"fun_headline_variants":["Equal accuracy, divergent explanations: AI scene understanding is ambiguous","Driving AI explanations diverge even when models are equally accurate","Equally good driving models give different explanations","GNN explanations near chance (W=0.07) for equally accurate driving models","Rashomon effect confirmed in driving scenes: explanations are ambiguous"],"cache_read_input_tokens":2688,"weakest_assumption_plain":"The paper treats the divergence in SHAP rankings across equally well-performing models as evidence that the scene-explanation problem itself is ambiguous, rather than as noise from the SHAP attribution tool or from random variation in training.","fun_headline_variants_meta":{"raw":{"variants":["Equal accuracy, divergent explanations: AI scene understanding is ambiguous","Driving AI explanations diverge even when models are equally accurate","Equally good driving models give different explanations","GNN explanations near chance (W=0.07) for equally accurate driving models","Rashomon effect confirmed in driving scenes: explanations are ambiguous"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000785,"raw_usage":{"total_tokens":3269,"prompt_tokens":681,"completion_tokens":2588,"prompt_tokens_details":{"cached_tokens":256},"prompt_cache_hit_tokens":256,"prompt_cache_miss_tokens":425,"completion_tokens_details":{"reasoning_tokens":2502}},"tokens_in":425,"tokens_out":2588,"duration_ms":24092,"temperature":1.0,"reasoning_tokens":2502,"cache_read_input_tokens":256,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-05T11:05:24.263489+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Re-run the same GNN and boosting models while replacing SHAP with another attribution method (e.g., integrated gradients, as the method section names) or repeatedly sampling SHAP background data for a single fixed model; if the near-zero Kendall's W disappears or matches the within-model attribution noise, the claim that ambiguity is inherent to the problem would be undermined.","supporting_citations":[{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Introduces the Rashomon effect as the multiplicity of equally good models, the theoretical basis of the study."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Formalizes the Rashomon set and model class reliance, motivating the measurement of explanation variation across models."},{"cited_title":"M.; and Lee, S","cited_arxiv_id":null,"evidence_quote":"Provides SHAP, the feature attribution method used to generate the explanations whose agreement is measured."},{"cited_title":"H.; Vora, S.; Liong, V","cited_arxiv_id":null,"evidence_quote":"The nuScenes dataset supplies the real-world driving scenes and object annotations used in all experiments."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Defines the QXG representation and the pair-based action explanation approach that the interpretable model class builds on."}],"review_version":1}