REVIEW 4 major objections 4 minor 19 references
Explaining Autonomous Vehicles with Intention-aware Policy Graphs
T0 review · 4 major / 4 minor · reviewed 2026-08-15 · deepseek-v4-flash
Pith's one-line read A graph of inferred desires can explain 75.7% of autonomous vehicle behavior, with 92.0% certainty that attributed intentions are fulfilled, and it exposes dataset blind spots like a nighttime stop-sign gap and a map error.
desk verdict Applies the authors' own IPG method to AV driving with hand-crafted desires; the qualitative scene analysis is genuinely useful, but the headline metrics are largely self-referential, so the 'reliable explanations' claim needs tempering. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The central object is an Intention-aware Policy Graph: a directed graph whose nodes are semantic driving states and whose edges are labeled maneuvers with transition probabilities, augmented by a desire set and per-state intention values. The intention value $I_d(s)$ sums the probabilities of all paths from state $s$ that reach a state satisfying desire $d$ and execute a fulfilling action; this single quantity carries the explanation. Two global metrics summarize it: the attributed intention probability, the share of states where $I_d(s)$ exceeds the commitment threshold $C=0.5$, and the expected intention probability, the probability that an attributed intention is actually fulfilled. The graph also supports answering the telic questions 'what do you intend in state $s$?', 'why would you do action $a$ in $s$?', and 'how do you plan to fulfill intention $I$ from $s$?'.
What would settle it
In the scenes where the ego vehicle approaches a traffic light but does not slow or stop, read the actual light color from the camera images in those frames; if a substantial share are green, the red-light assumption is false and the compliance conclusions drawn from the 16.4% unfulfilled 'Approach Traffic Light' cases do not hold.
Extended reading notes
Core claim
The paper's central claim is that teleological explanation—explanation in terms of attributed goals rather than causes—can be made quantitative for autonomous vehicles. Observed trajectories are discretized into states described by natural-language predicates (speed, steering, lane position, nearby crosswalks, pedestrians, stop areas) and actions labelled as maneuvers; these populate a policy graph whose edges carry transition probabilities. Each hand-crafted desire $d$, safe or unsafe, specifies a set of relevant states and a set of fulfilling actions, and the intention value $I_d(s)$ is computed as the probability, summed over all paths from $s$, of reaching a state where $d$ is fulfilled. Across 830 scenes, the authors report that at threshold $C=0.5$ an intention can be attributed to 75.7% of states and that attributed intentions are fulfilled with 92.0% certainty. The same graph enables concrete failure diagnoses: at night the vehicle never manifests stop-sign intentions, and in scene 1084 the intended lane change appears later than the observed maneuver because the map omits a real lane divider.
Load-bearing premise
The compliance analysis assumes that every traffic light the vehicle approaches is red, because the dataset records no signal color; if some of those lights are green, the unfulfilled 'Approach Traffic Light' cases become lawful passes rather than violations, and the reported compliance distinction collapses.
Editorial extensions
If this is right
- Any autonomous driving system can be audited after the fact using only its observed trajectories and map data, without access to its planner, reward function, or training labels.
- Compliance can be assessed per desire: for example, stop-sign approach intentions are fulfilled 83.6% of the time when attributed, and the remaining cases can be inspected as rolling stops, violations, or dataset errors.
- The same policy graph doubles as a dataset-quality probe, as shown by the nighttime disappearance of stop-sign intentions and the map-divider omission in scene 1084.
- Local queries at any state produce plans and purpose attributions that support counterfactual reasoning, such as explaining why braking at a given state serves a weaker lane-change intention than the optimal accelerating turn.
Reading between the lines
- The headline 75.7% and 92.0% figures measure coverage of the hand-crafted desire vocabulary, not of all possible driving intentions; adding desires such as obstacle avoidance or emergency braking would change both numbers, a testable consequence of the framework.
- The unfulfilled 'Approach Traffic Light' cases (16.4%) conflate lawful green-light passes with violations because the color is assumed red; a dataset with signal annotations would let this distinction be resolved directly.
- The same intention metrics could be monitored online: streaming vehicle states could be flagged when an attributed intention is unlikely to be fulfilled, before the maneuver completes, turning explanation into a real-time safety signal.
- The scene-1084 diagnosis suggests the method can also audit the map itself; comparing computed intentions against observed maneuvers across all scenes might reveal further map errors beyond the single reported case.
Signed reviews
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper proposes a post-hoc, model-agnostic explainability method for autonomous vehicles, building on Intention-aware Policy Graphs (IPGs). It discretizes 830 nuScenes trajectories into interpretable states via eleven hand-crafted predicates and threshold-based action labels, defines safe and unsafe desires, and computes intention values as path probabilities to desire-fulfilling states. It reports a global attributed-intention probability of 75.7% and an expected-intention probability of 92.0%, analyzes how these metrics vary by day/night and rainy conditions, and demonstrates local explanations and question answering on scene-1084, including an identified map inconsistency.
Significance. If the quantitative claims were externally supported, this would be a useful contribution to XAI for autonomous vehicles: the method is model-agnostic, provides human-readable teleological explanations ('what/why/how'), and the scene-level analysis demonstrates a concrete ability to surface dataset and model anomalies. The nighttime stop-sign finding in Table 2 and the scene-1084 map inconsistency are promising qualitative results. However, the current evaluation does not establish the headline reliability claims: the state discretizer was explicitly optimized for the reported metrics, the 'expected intention probability' is definitionally tied to the intention value being thresholded, and the legal-compliance sub-claim rests on an unverified traffic-light assumption. The qualitative findings and the formal plan-based explanation mechanism should be preserved in a revision, but the quantitative claims need reframing and additional validation.
major comments (4)
- [Section 3.1 and Section 5.1] The central quantitative claim is evaluated in-sample in a way that is acknowledged in Section 3.1: the state discretizer was 'optimised for the metrics later presented', and Section 4.3 defines desires over those same predicates. Since the same 830 trajectories are used both to construct the policy graph and to compute the 75.7% and 92.0% headline metrics, these numbers reflect post-hoc feature selection rather than an independent assessment of explanation reliability. The paper should add a held-out evaluation, a pre-registered or fixed discretizer chosen without access to the evaluation scenes, and a sensitivity analysis over discretization thresholds and the commitment threshold C.
- [Section 3.2 and Section 5.1] The 'expected intention probability' is a within-graph statistic that is directly derived from the intention definition: Id(s) is the probability of reaching a desire-fulfilling state from s, and the metric averages Id(s) over states with Id(s)>C=0.5. High values such as 91.6% and 92.0% are therefore re-expressions of the same quantity used to define the intention, not independent evidence that the inferred intentions are reliable. The paper should relabel this quantity as internal consistency of the graph, or validate it against ground-truth labels such as human-annotated compliance judgments or an independently known set of traffic violations.
- [Section 4.3 and Section 5.1] The 'Approach Traffic Light' desire is defined under the assumption that a relevant traffic light is red whenever the vehicle is near it (Section 4.3), because nuScenes does not provide signal state. This makes the legal-compliance interpretation unverifiable: the reported 16.4% unfulfilled cases could be green-light passes, dataset annotation errors, or genuine violations, and the analysis as presented cannot distinguish these alternatives. The paper should either use a dataset with traffic-light phase information or explicitly restrict the claims to 'approach behavior' without labeling it as compliance or violation.
- [Section 4.2 and Section 5.1] The threshold-based action labelling heuristic and the breadth of the desire definitions substantially inflate the 'any desire' attribution metric. For example, Lane Keeping includes GoStraight, Gas, and Brake, so many ordinary states trivially support it; the 75.7% union metric over safe and unsafe desires is therefore not a strong interpretability result by itself. Reporting the distribution of attributed desires, an ablation with narrower Ad sets, or a comparison against a baseline with randomly paired desires would clarify how much information the attribution actually carries.
minor comments (4)
- [Section 5.1] In the sentence 'we assess the interpretability and reliability of the vehicle's behaviour using two metrics over a desired [6]', 'a desired' appears to be a typo for 'a desire d'.
- [Table 3] The state transition in Table 3 uses 'FrontObject(Yes)' and 'FrontObject(No)', but the predicate introduced in Table 1 is 'ObjectsNearby'; please use consistent feature names.
- [Section 5.3] In the sentence 'instead, it opts to break', 'break' should be 'brake'.
- [Figures 1 and 2] The captions say 'first metrics from the right' when referring to the global aggregate values; please label these values directly in the figures or in the caption to avoid ambiguity.
Circularity Check
Reliability metrics are circular: 'expected intention' restates the intention-value definition, and the state discretizer was optimized on the same metrics.
-
self definitional
[Section 3.2 (intention value) and Section 5.1 (expected intention probability)]
"The intention value Id(s) quantifies the vehicle's intention to achieve a desired starting from state s, and is computed as the sum of probabilities of all possible paths originating from s and reaching any state in Sd where the desire is achievable and fulfilled. ... the expected intention probability, the probability that once attributed, an intention of d is going to be fulfilled (reliability)."
Since Id(s) is defined as the probability of reaching Sd (fulfilment), the expected intention probability is just Pr(fulfilment | Id > C). Any state passing C=0.5 has, by construction, at least 0.5 probability of 'fulfilling' the desire; the reported 91.6%/92.0% certainty is therefore a property of the same graph transition probabilities used to define the intention, not an independent benchmark of explanation quality. The 'reliability' conclusion is already contained in the definition of the intention value, making the metric a tautological restatement rather than external evidence.
-
fitted input called prediction
[Section 3.1 (State-Action Discretisation)]
"we experimented with several state discretisers, this one was optimised for the metrics later presented, while keeping a manageable state space to allow the mapping of new observations to previously encountered states."
The headline numbers (75.7% attribution, 92.0% expected intention) are computed on the same 830 nuScenes trajectories used to build the PG, and the state discretizer was explicitly selected to maximize those very metrics. With no held-out scenes, pre-registered discretizer, or sensitivity analysis, the reported attribution/reliability values are in-sample maxima after feature selection rather than predictions. The central quantitative claim that the method reliably attributes intentions is therefore statistically forced by the optimization of the evaluation input.
1 more flagged steps
-
self definitional
[Section 3.1 (last paragraph) and Section 4.3 (Lane Keeping desire)]
"It is paramount that these features are defined to align with the hypothesised desires motivating the ego vehicle's actions. ... Lane Keeping : Sd includes all states where the vehicle is moving, oriented forward, and aligned with the direction of its current lane, with no intention to turn at the next intersection. Ad includes going straight, accelerating or braking."
The state predicates and the desires are co-designed: the features are chosen to align with the desires, and the Lane Keeping desire is defined over the most common nuScenes driving situation with a very permissive action set (straight, accelerate, brake). The paper itself notes that 'Except for Lane Keeping, the probability of manifesting intentions for other desires is generally low,' so the global 75.7% 'any desire' attribution is dominated by this deliberately broad desire. The interpretability result is thus a consequence of how the desire was defined rather than of discovered intentional structure in the vehicle's behaviour.
full rationale
The paper's central quantitative evaluation is not externally grounded. The 'expected intention probability' is the conditional probability of reaching the same target set Sd that defines the intention value Id(s), so the reported 92.0% certainty is a graph-derived restatement of the definition rather than independent evidence that the attributed desire is the vehicle's real goal. In addition, Section 3.1 states that the state discretizer was optimized for the metrics later presented, and the desires in Section 4.3 are defined over predicates chosen to align with them, using the same 830 trajectories for construction and evaluation. This makes the headline 75.7% attribution and 92.0% fulfilment figures in-sample and partly constructed. The case-study findings (nighttime stop sign gap, scene-1084 map inconsistency) are separate, non-circular empirical observations, and the method may still be useful for dataset auditing; however, these do not rescue the central reliability claim, which reduces to a definitional identity plus feature-selection optimization.
Assumptions & free parameters
free parameters (3)
- commitment threshold C =
0.5
- discretization thresholds
- desire definitions
assumptions (4)
- domain assumption The ego vehicle operates with the intent of achieving a set of hypothesised desires D
- domain assumption nuScenes annotations and map data are accurate for the main analysis
- ad hoc to paper Traffic light color is red when a relevant traffic light is near
- domain assumption The policy graph transition probabilities, estimated from 830 trajectories, are a faithful model of the vehicle's behavior
invented entities (1)
-
hypothesised desires (e.g., Lane Keeping, Peds at Crosswalk, Ignore Stop Sign)
Cite this review
Pith. "Pith review of Explaining Autonomous Vehicles with Intention-aware Policy Graphs." pith.science (2026). https://pith.science/paper/SHMWM6S3
@misc{pith2026250508404,
author = {Pith},
title = {Pith review of: Explaining Autonomous Vehicles with Intention-aware Policy Graphs},
year = {2026},
howpublished = {\url{https://pith.science/paper/SHMWM6S3}},
note = {Machine review of arXiv:2505.08404}
}
read the original abstract
The potential to improve road safety, reduce human driving error, and promote environmental sustainability have enabled the field of autonomous driving to progress rapidly over recent decades. The performance of autonomous vehicles has significantly improved thanks to advancements in Artificial Intelligence, particularly Deep Learning. Nevertheless, the opacity of their decision-making, rooted in the use of accurate yet complex AI models, has created barriers to their societal trust and regulatory acceptance, raising the need for explainability. We propose a post-hoc, model-agnostic solution to provide teleological explanations for the behaviour of an autonomous vehicle in urban environments. Building on Intention-aware Policy Graphs, our approach enables the extraction of interpretable and reliable explanations of vehicle behaviour in the nuScenes dataset from global and local perspectives. We demonstrate the potential of these explanations to assess whether the vehicle operates within acceptable legal boundaries and to identify possible vulnerabilities in autonomous driving datasets and models.
Figures
Figures from the paper (1 more)
Reference graph
Works this paper leans on
-
[6]
Intention-aware policy graphs: answering what, how, and why in opaque agents
Victor Gimenez-Abalos et al. Intention-aware policy graphs: answering what, how, and why in opaque agents. Sept. 2024.doi: 10.48550/arXiv. 2409.19038. url: http://arxiv.org/abs/2409.19038. 16 S. Montese et al
-
[1]
ShahinAtakishiyevetal.“ExplainableArtificialIntelligenceforAutonomous Driving: A Comprehensive Overview and Field Guide for Future Research Directions”. In:IEEE Access12 (2024), pp. 101603–101625.doi: 10.1109/ ACCESS.2024.3431437. url: https://ieeexplore.ieee.org/document/ 10604830/?arnumber=10604830
arXiv 2024
-
[2]
nuScenes: A Multimodal Dataset for Autonomous Driving
Holger Caesar et al. “nuScenes: A Multimodal Dataset for Autonomous Driving”. In:2020 IEEE/CVF Conference on Computer Vision and Pat- tern Recognition (CVPR). Seattle, WA, USA: IEEE, June 2020, pp. 11618– 11628. doi: 10.1109/CVPR42600.2020.01164. url: https://ieeexplore. ieee.org/document/9156412/
arXiv 2020
-
[3]
Antoni Climent, Dmitry Gnatyshak, and Sergio Alvarez-Napagao. “Apply- ing and Verifying an Explainability Method Based on Policy Graphs in the Context of Reinforcement Learning”. In:Frontiers in Artificial Intelligence and Applications. IOS Press, Oct. 2021.doi: 10.3233/FAIA210166. url: https://ebooks.iospress.nl/doi/10.3233/FAIA210166
-
[4]
An interpretation framework for autonomous vehicles decision-making via SHAP and RF
Zhihao Cui et al. “An interpretation framework for autonomous vehicles decision-making via SHAP and RF”. In:2022 6th CAA International Con- ference on Vehicular Control and Intelligence (CVCI). Oct. 2022, pp. 1–7. doi: 10.1109/CVCI56766.2022.9964561 . url: https://ieeexplore. ieee.org/document/9964561
arXiv 2022
-
[5]
Marc Domenech I Vila et al. “Explaining the Behaviour of Reinforcement Learning Agents in a Multi-Agent Cooperative Environment Using Pol- icy Graphs”. In: Electronics 13.3 (Jan. 2024), p. 573. doi: 10 . 3390 / electronics13030573. url: https://www.mdpi.com/2079-9292/13/3/ 573
work page 2024
-
[7]
Why Interpreting Intent Is Key for Trust- worthiness in the Age of Opaque Agents
Victor Gimenez-Abalos et al. “Why Interpreting Intent Is Key for Trust- worthiness in the Age of Opaque Agents”. In:Preprints (Feb. 2024). doi: 10 . 20944 / preprints202402 . 1446 . v1. url: https : / / doi . org / 10 . 20944/preprints202402.1446.v1
-
[8]
Balint Gyevnar et al. “A Human-Centric Method for Generating Causal Explanations in Natural Language for Autonomous Vehicle Motion Plan- ning”. In: Preprints (2022). doi: 10 . 48550 / arXiv . 2206 . 08783. url: http://arxiv.org/abs/2206.08783
Show all 19 references
-
[9]
CausalExplanationsforSequentialDecision-Making in Multi-Agent Systems
BalintGyevnaretal.“CausalExplanationsforSequentialDecision-Making in Multi-Agent Systems”. In:Proceedings of the 23rd International Confer- ence on Autonomous Agents and Multiagent Systems. AAMAS ’24. Rich- land, SC: International Foundation for Autonomous Agents and Multia- g...
2024
-
[10]
People Attribute Purpose to Autonomous Vehicles When Explaining Their Behavior: Insights from Cognitive Science for Ex- plainable AI
Balint Gyevnar et al. People Attribute Purpose to Autonomous Vehicles When Explaining Their Behavior: Insights from Cognitive Science for Ex- plainable AI. Feb. 2025. doi: 10 . 1145 / 3706598 . 3713509. url: http : //arxiv.org/abs/2403.08828
2025 arXiv
-
[11]
Improving Robot Controller Trans- parency Through Autonomous Policy Explanation
Bradley Hayes and Julie A. Shah. “Improving Robot Controller Trans- parency Through Autonomous Policy Explanation”. In:Proceedings of the 2017 ACM/IEEE International Conference on Human-Robot Interaction. Vienna Austria: ACM, Mar. 2017, pp. 303–312.doi: 10.1145/2909824. 302023...
2017
-
[12]
Explainable AI in Scene Understanding for Au- tonomous Vehicles in Unstructured Traffic Environments on Indian Roads Using the Inception U-Net Model with Grad-CAM Visualization
Suresh Kolekar et al. “Explainable AI in Scene Understanding for Au- tonomous Vehicles in Unstructured Traffic Environments on Indian Roads Using the Inception U-Net Model with Grad-CAM Visualization”. In:Sen- sors 22.24 (2022). doi: 10.3390/s22249677 . url: https://www.mdpi. ...
2022 doi
-
[13]
TowardsComputationalForeseeability
SaritKrausetal.“TowardsComputationalForeseeability”.In:(Sept.2024). Publisher: AAAI Conference on Artificial Intelligence.url: https://par. nsf.gov/biblio/10544017-towards-computational-foreseeability
2024
-
[14]
Explainable AI for Safe and Trustworthy Au- tonomous Driving: A Systematic Review
Anton Kuznietsov et al. “Explainable AI for Safe and Trustworthy Au- tonomous Driving: A Systematic Review”. In:IEEE Transactions on In- telligent Transportation Systems25.12 (Dec. 2024). arXiv:2402.10086 [cs], pp. 19342–19364. doi: 10 . 1109 / TITS . 2024 . 3474469. url: http...
2024 arXiv
-
[15]
Why Is the Current XAI Not Meeting the Expectations?
Alessio Malizia and Fabio Paternò. “Why Is the Current XAI Not Meeting the Expectations?” In: Communications of the ACM 66.12 (Dec. 2023), pp. 20–23. doi: 10.1145/3588313. url: https://dl.acm.org/doi/10. 1145/3588313
2023 doi
-
[16]
How the Mind Explains Behavior: Folk Explanation, Mean- ing and Social Interaction
Bertram Malle. How the Mind Explains Behavior: Folk Explanation, Mean- ing and Social Interaction. MIT Press, Jan. 2004
2004
-
[17]
OD-XAI: Explainable AI-Based Semantic Ob- ject Detection for Autonomous Vehicles
Harsh Mankodiya et al. “OD-XAI: Explainable AI-Based Semantic Ob- ject Detection for Autonomous Vehicles”. In:Applied Sciences12.11 (Jan. 2022). Number: 11 Publisher: Multidisciplinary Digital Publishing Insti- Explaining Autonomous Vehicles with Intention-aware Policy Graphs ...
2022 doi
-
[18]
Explanation in artificial intelligence: Insights from the social sciences
Tim Miller. “Explanation in artificial intelligence: Insights from the social sciences”. In: Artificial Intelligence 267 (Feb. 2019), pp. 1–38. doi: 10 . 1016/j.artint.2018.07.007 . url: https://linkinghub.elsevier. com/retrieve/pii/S0004370218305988
2019
-
[19]
Towards Accountability: Providing Intelligible Ex- planations in Autonomous Driving
Daniel Omeiza et al. “Towards Accountability: Providing Intelligible Ex- planations in Autonomous Driving”. In: 2021 IEEE Intelligent Vehicles Symposium (IV). Nagoya, Japan: IEEE, July 2021, pp. 231–237. doi: 10 . 1109 / IV48863 . 2021 . 9575917. url: https : / / ieeexplore . ...
2021
Reviewed August 15, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.