REVIEW 3 major objections 5 minor 41 references
Explainable Scene Understanding with Qualitative Representations and Graph Neural Networks
T0 review · 3 major / 5 minor · reviewed 2026-08-16 · deepseek-v4-flash
Pith's one-line read Graph that reads whole scene spots relevant objects at 86% AUC
desk verdict First GNN-on-QXG paper with a plausible but questionable headline comparison; the ego-star vs all-pairs evaluation asymmetry needs to be resolved before the main claim holds. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The Qualitative Explainable Graph (QXG), a symbolic graph whose edges carry qualitative spatial-temporal relations computed from detected bounding boxes using Qualitative Distance Calculus, Qualitative Trajectory Calculus, and Rectangle Algebra. The GNN architecture uses two graph-attention (GAT) layers with four heads each to propagate information along all edges, then extracts the ego-vehicle star graph and classifies each relation chain through a concatenated node-edge-node vector. A weighted sum of weighted binary cross-entropy and focal loss handles the few-positive-objects imbalance. The QXG supplies the interpretable intermediate representation; the GAT layers supply the whole-scene aggregation that the shallow baselines lack.
What would settle it
Ablate the graph context: train the same architecture on QXGs whose non-ego edges are randomly permuted across scenes, or removed entirely, and compare ROC-AUC against the reported 86.39; if the score does not drop materially, the claimed benefit of whole-scene structure is not what is carrying the result.
Extended reading notes
Core claim
The central claim is that a GNN processing an entire QXG can identify relevant objects in traffic scenes better than shallow models restricted to single relation chains. The paper casts relevant object identification as binary edge classification: each edge is an object relation chain whose label says whether one endpoint is relevant. After two graph-attention layers propagate information across the full graph, the model extracts the ego-centred star graph and classifies each ego-to-object chain from a concatenation of ego node, edge, and object-node embeddings. With a combined weighted cross-entropy and focal loss to counter class imbalance, the model outperforms the random forest and AdaBoost baselines on every reported metric. The authors read this as evidence that whole-graph context, expressed through qualitative spatial-temporal relations, improves explainable scene understanding.
Load-bearing premise
The whole approach assumes that the QXG built from detected bounding boxes and qualitative thresholds preserves the cues—relative distance, motion, and position—that make an object relevant, so anything the perception pipeline misses or mislabels is lost before the GNN sees it.
Editorial extensions
If this is right
- If the central claim holds, scene understanding modules can consume symbolic qualitative graphs directly with deep networks, keeping an interpretable intermediate representation while gaining whole-scene context.
- The GNN's ability to process variable graph sizes makes it applicable to scenes with arbitrary numbers and types of detected objects without fixed-size input vectors.
- The combined wBCE and focal loss result implies that standard cross-entropy is insufficient in the highly imbalanced regime of relevant object identification; class-sensitive losses are load-bearing.
- The ROC-AUC improvement over baselines suggests that contextual edges, not just the ego-pair chain, carry information about relevance, which motivates richer graph features and larger training sets.
- Because the approach is formulated as edge classification on relation chains, it can extend to other explanation tasks such as action explanation and prediction.
Reading between the lines
- A direct ablation the paper does not run: removing all non-ego edges from the QXG before the GAT layers should drop performance if whole-graph context is truly the cause; without that ablation, part of the gain could come from the richer feature embeddings alone.
- Since QXG relations depend on thresholds for qualitative distance, the method inherits a sensitivity: different thresholds change edge labels and could change GNN predictions; testing across thresholds could expose how much of the result rests on discretization choices.
- The star-graph extraction step means predictions for objects far from the ego are still mediated by ego-anchored chains; using global pooling over all nodes instead might change which objects are deemed relevant, a testable variant.
- DriveLM's annotation protocol may correlate relevance with proximity or motion; if so, the GNN's qualitative distance and trajectory features could be doing most of the work, and the graph-attention context may matter less than the loss function.
Signed reviews
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper proposes a graph neural network (GNN) architecture that operates on Qualitative Explainable Graphs (QXGs) for relevant object identification in automated driving. The QXG is a symbolic scene representation with qualitative spatial-temporal edge relations; the authors formulate the task as binary edge classification and introduce a combination of weighted binary cross-entropy and focal loss to handle class imbalance. The method is evaluated on 2465 nuScenes scenes with DriveLM relevance labels, using 10-fold cross-validation, and compared against random forest and AdaBoost baselines from prior QXG work. The authors report improved F1 and ROC-AUC for the GNN over the baselines, and an ablation study of loss functions. The paper concludes that deep learning can complement qualitative scene representations while retaining explainability.
Significance. If the central comparative claim were established, the paper would provide a useful bridge between symbolic qualitative scene representations and graph deep learning, addressing a recognized limitation of prior QXG methods that analyze isolated relation chains. The use of a real-world dataset with human relevance labels, the explicit treatment of class imbalance through a combined loss, and the report of a loss-function ablation are concrete strengths. However, the headline claim of 'superior performance compared to baseline methods' is not yet supported because the baselines appear to be evaluated on a different edge population than the GNN, and the text's claim of improvement 'across all metrics' is contradicted by the lower recall of the GNN in Table I. The significance of the contribution is therefore currently conditional on a corrected, apples-to-apples comparison.
major comments (3)
- [§IV-C and §V-A(b)] The main comparative claim rests on a likely mismatch between the evaluation sets of the GNN and the baselines. In §IV-C, the GNN extracts only star-graph edges centred on the ego vehicle, forming vObjRelChain = [x0||e0j||xj] for edges (ego, target). In §V-A(b), the baselines are trained 'over all object pairs in the dataset, independent of the scene and frame', and classify 'an object pair with the corresponding relations'. If the baseline test set includes non-ego edges that the GNN never scores, the F1 and ROC-AUC differences in Table I may reflect a different, larger, or differently balanced edge set rather than the value of graph context. The paper never states that the baseline edge list is restricted to the same ego-star edges. Please clarify the exact edge sets used for training and testing each model, or better, evaluate all models on the identical ego-star edge set; otherwise the headline 'superior performance compared to baseline methods' is not established.
- [Table I and §V-B(1)] The text in §V-B(1) states that the GNN shows 'substantial improvements over the baseline methods across all metrics', but Table I reports a GNN recall of 63.19, which is lower than the random forest recall of 71.90 and the AdaBoost recall of 74.69. Since recall is one of the listed metrics, the claim of improvement across all metrics is internally contradicted. Please either qualify the claim (e.g., improvements in F1 and ROC-AUC at the cost of recall) or provide additional evidence, such as per-fold error bars or significance tests, that explains the discrepancy. This is not a cosmetic issue; it affects how the reader interprets the trade-off between precision and recall in an imbalanced-task setting.
- [§V-A(b) and Table I] Even if the edge-population mismatch is resolved, the comparison as designed does not isolate the contribution of the GNN's graph context. The baselines are described as operating without any scene context, and they use the original pair-chain representation from prior work. To support the claim that 'considering the entire graph structure ... provides valuable context', a controlled baseline is needed that receives the same ego-star edge features as the GNN but does not perform message passing across the graph. Without such a baseline, the observed gains could be attributed to differences in feature encoding, the specific edge set, or the loss function rather than to the graph context. Please add such a controlled comparison or explicitly restrict the interpretation of the results.
minor comments (5)
- [§I] The Introduction contains two nearly identical consecutive paragraphs, both beginning 'However, current approaches to processing QXGs rely primarily on shallow machine learning models...'. One of these duplicated paragraphs should be removed.
- [§V-A(c)] The recall definition is given as 'fraction of correct negative classification', but the formula Recall = True Positives / (True Positives + False Negatives) defines recall as the fraction of correct positive classifications. Please correct the prose.
- [Table II and §V-B(2)] The loss-function ablation shows that FL-only achieves a higher F1 (28.54) than the combined wBCE+FL loss (27.28). The text explains the combination as a balance of precision and recall, which is reasonable, but the interpretation should acknowledge that FL alone yields a better F1 in this experiment.
- [§IV-C] The abstract and introduction emphasize processing 'complete spatial-temporal relationships between all objects', but the final readout only uses edges incident to the ego node. This is a sensible design choice, but the discrepancy between the full-graph claim and the ego-star readout should be clarified in the method description.
- [General] No code or data availability statement is provided. Given that the central comparison depends on exact edge-set construction, releasing the evaluation scripts would substantially improve reproducibility and would allow reviewers to verify the baseline edge lists.
Circularity Check
No circularity: the central comparison is an empirical evaluation on held-out DriveLM labels against independently fitted baselines.
full rationale
The paper's main claim is an empirical performance comparison for relevant object identification, evaluated with 10-fold cross-validation on 2465 nuScenes scenes labeled by DriveLM annotators. The GNN, random forest, and AdaBoost models are all trained on held-out folds and scored on test folds; no parameter is fitted to the target metric, and no result is derived by definition from its inputs. The QXG representation is introduced in the authors' prior work, but it is used here as an input representation rather than as a load-bearing theorem, and the comparison targets external human-annotated labels. The baselines are also from the authors' prior work, but a self-citation that supplies a comparison method is not circular: the baseline numbers are independently computed on the same dataset under the reported protocol. The skeptical concern that the baselines may have been evaluated on all object pairs while the GNN scores only ego-star edges is a potential comparability or correctness issue, not a circularity, because it does not make any reported prediction equivalent to a fitted input by construction. No self-definitional step, fitted-input-called-prediction step, uniqueness-imported-from-authors step, or ansatz-smuggled-in-via-citation step is present in the text. The derivation chain is therefore self-contained with respect to the circularity criteria.
Assumptions & free parameters
free parameters (6)
- distance discretization thresholds theta =
not reported
- focal loss alpha =
0.95
- focal loss gamma =
0.5
- loss combination weight w =
0.5
- class weights wp, wn for weighted BCE =
not reported
- GNN hidden dimensions and attention head sizes =
not reported
assumptions (3)
- domain assumption Qualitative relations computed from detected bounding boxes and thresholds theta faithfully represent the spatial-temporal context needed to judge relevance.
- domain assumption DriveLM human-annotated relevance labels are treated as ground truth for relevant object identification.
- domain assumption Edge classification over relation chains is an adequate proxy for identifying relevant objects.
Cite this review
Pith. "Pith review of Explainable Scene Understanding with Qualitative Representations and Graph Neural Networks." pith.science (2026). https://pith.science/paper/WLE4KE3J
@misc{pith2026250412817,
author = {Pith},
title = {Pith review of: Explainable Scene Understanding with Qualitative Representations and Graph Neural Networks},
year = {2026},
howpublished = {\url{https://pith.science/paper/WLE4KE3J}},
note = {Machine review of arXiv:2504.12817}
}
read the original abstract
This paper investigates the integration of graph neural networks (GNNs) with Qualitative Explainable Graphs (QXGs) for scene understanding in automated driving. Scene understanding is the basis for any further reactive or proactive decision-making. Scene understanding and related reasoning is inherently an explanation task: why is another traffic participant doing something, what or who caused their actions? While previous work demonstrated QXGs' effectiveness using shallow machine learning models, these approaches were limited to analysing single relation chains between object pairs, disregarding the broader scene context. We propose a novel GNN architecture that processes entire graph structures to identify relevant objects in traffic scenes. We evaluate our method on the nuScenes dataset enriched with DriveLM's human-annotated relevance labels. Experimental results show that our GNN-based approach achieves superior performance compared to baseline methods. The model effectively handles the inherent class imbalance in relevant object identification tasks while considering the complete spatial-temporal relationships between all objects in the scene. Our work demonstrates the potential of combining qualitative representations with deep learning approaches for explainable scene understanding in autonomous driving systems.
Figures
Reference graph
Works this paper leans on
-
[1]
Scene understanding in deep learning-based end-to-end controllers for autonomous vehicles,
S. Yang, W. Wang, C. Liu, and W. Deng, “Scene understanding in deep learning-based end-to-end controllers for autonomous vehicles,” IEEE Transactions on Systems, Man, and Cybernetics: Systems, vol. 49, no. 1, pp. 53–63, 2019
work page 2019
-
[2]
K. Muhammad, T. Hussain, H. Ullah, J. D. Ser, M. Rezaei, N. Kumar, M. Hijji, P. Bellavista, and V . H. C. de Albuquerque, “Vision-based semantic segmentation in scene understanding for autonomous driving: Recent achievements, challenges, and outlooks,” IEEE Trans. Intell. Transp. Syst., vol. 23, no. 12, pp. 22 694–22 715, 2022
work page 2022
-
[3]
Deep learning for safe autonomous driving: Current chal- lenges and future directions,
K. Muhammad, A. Ullah, J. Lloret, J. D. Ser, and V . H. C. de Al- buquerque, “Deep learning for safe autonomous driving: Current chal- lenges and future directions,” IEEE Trans. Intell. Transp. Syst. , vol. 22, no. 7, pp. 4316–4336, 2021
work page 2021
-
[4]
Acquiring qual- itative explainable graphs for automated driving scene interpretation,
N. Belmecheri, A. Gotlieb, N. Lazaar, and H. Spieker, “Acquiring qual- itative explainable graphs for automated driving scene interpretation,” CoRR, vol. abs/2308.12755, 2023
arXiv 2023
-
[5]
Trustworthy automated driving through qualitative scene under- standing and explanations,
——, “Trustworthy automated driving through qualitative scene under- standing and explanations,” in Transport Research Arena (TRA) , 2024
work page 2024
-
[6]
Toward trustworthy automated driving through qualitative scene understanding and explanations,
——, “Toward trustworthy automated driving through qualitative scene understanding and explanations,” SAE International Journal of Con- nected and Automated Vehicles , vol. 8, 2024
work page 2024
-
[7]
Relevant object identification from qualitative explainable graphs in automated driving,
——, “Relevant object identification from qualitative explainable graphs in automated driving,” NORA Annual Conference , 2024
work page 2024
-
[8]
Qualitative Spatial Reasoning Using Constraint Calculi,
J. Renz and B. Nebel, “Qualitative Spatial Reasoning Using Constraint Calculi,” in Handbook of Spatial Logics . Springer, 2007, pp. 161–215
work page 2007
Show all 41 references
-
[9]
A survey of qualitative spatial and tem- poral calculi: Algebraic and computational properties,
F. Dylla, J. H. Lee, T. Mossakowski, T. Schneider, A. van Delden, J. van de Ven, and D. Wolter, “A survey of qualitative spatial and tem- poral calculi: Algebraic and computational properties,” ACM Comput. Surv., vol. 50, no. 1, pp. 7:1–7:39, 2017
2017
-
[10]
On support relations and semantic scene graphs,
M. Y . Yang, W. Liao, H. Ackermann, and B. Rosenhahn, “On support relations and semantic scene graphs,” ISPRS journal of photogrammetry and remote sensing , vol. 131, pp. 15–25, 2017
2017
-
[11]
Visual graphs from motion (vgfm): Scene understanding with object geometry reasoning,
P. Gay, J. Stuart, and A. Del Bue, “Visual graphs from motion (vgfm): Scene understanding with object geometry reasoning,” in Computer Vision–ACCV 2018: 14th Asian Conference on Computer Vision, Perth, Australia, December 2–6, 2018, Revised Selected Papers, Part III 14 . Sprin...
2018
-
[12]
Representation learning on visual-symbolic graphs for video understanding,
E. Mavroudi, B. B. Haro, and R. Vidal, “Representation learning on visual-symbolic graphs for video understanding,” in European Confer- ence on Computer Vision . Springer, 2020, pp. 71–90
2020
-
[13]
Learning 3d semantic scene graphs from 3d indoor reconstructions,
J. Wald, H. Dhamo, N. Navab, and F. Tombari, “Learning 3d semantic scene graphs from 3d indoor reconstructions,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , 2020, pp. 3961–3970
2020
-
[14]
Explainable and explicit visual reasoning over scene graphs,
J. Shi, H. Zhang, and J. Li, “Explainable and explicit visual reasoning over scene graphs,” in Proceedings of the IEEE/CVF conference on computer vision and pattern recognition , 2019, pp. 8376–8384
2019
-
[15]
A semantic graph of traffic scenes for intelligent vehicle systems,
C. Guo and S. Mita, “A semantic graph of traffic scenes for intelligent vehicle systems,” IEEE intelligent systems , pp. 57–62, 2012
2012
-
[16]
Collaborative dynamic 3d scene graphs for automated driving,
E. Greve, M. B ¨uchner, N. V ¨odisch, W. Burgard, and A. Valada, “Collaborative dynamic 3d scene graphs for automated driving,” arXiv:2309.06635, 2023
2023 arXiv
-
[17]
Lane graph estimation for scene understanding in urban driving,
J. Z ¨urn, J. Vertens, and W. Burgard, “Lane graph estimation for scene understanding in urban driving,” IEEE Robotics and Automation Letters, vol. 6, no. 4, pp. 8615–8622, 2021
2021
-
[18]
Explainable action prediction through self-supervision on scene graphs,
P. Kochakarn, D. De Martini, D. Omeiza, and L. Kunze, “Explainable action prediction through self-supervision on scene graphs,” in 2023 IEEE International Conference on Robotics and Automation (ICRA) . IEEE, 2023, pp. 1479–1485
2023
-
[19]
A com- prehensive survey of scene graphs: Generation and application,
X. Chang, P. Ren, P. Xu, Z. Li, X. Chen, and A. Hauptmann, “A com- prehensive survey of scene graphs: Generation and application,” IEEE Transactions on Pattern Analysis and Machine Intelligence , vol. 45, no. 1, pp. 1–26, 2023
2023
-
[20]
Learning relational event models from video,
K. S. Dubba, A. G. Cohn, D. C. Hogg, M. Bhatt, and F. Dylla, “Learning relational event models from video,” Journal of Artificial Intelligence Research, vol. 53, pp. 41–90, 2015
2015
-
[21]
Explain- able video action reasoning via prior knowledge and state transitions,
T. Zhuo, Z. Cheng, P. Zhang, Y . Wong, and M. Kankanhalli, “Explain- able video action reasoning via prior knowledge and state transitions,” in Proceedings of the 27th acm international conference on multimedia , 2019, pp. 521–529
2019
-
[22]
Effect-abstraction based relaxation for linear numeric planning
D. Li, E. Scala, P. Haslum, and S. Bogomolov, “Effect-abstraction based relaxation for linear numeric planning.” in IJCAI, 2018, pp. 4787–4793
2018
-
[23]
Towards ex- plainable action recognition by salient qualitative spatial object relation chains,
H. Hua, D. Li, R. Li, P. Zhang, J. Renz, and A. Cohn, “Towards ex- plainable action recognition by salient qualitative spatial object relation chains,” in Proceedings of the AAAI Conference on Artificial Intelligence (AAAI-22), 2022
2022
-
[24]
Graph neural networks for intelligent transportation systems: A survey,
S. Rahmani, A. Baghbani, N. Bouguila, and Z. Patterson, “Graph neural networks for intelligent transportation systems: A survey,” IEEE Transactions on Intelligent Transportation Systems , vol. 24, no. 8, pp. 8846–8885, 2023
2023
-
[25]
Gsgformer: Generative social graph transformer for multimodal pedestrian trajectory prediction,
Z. Luo, M. Robin, and P. Vasishta, “Gsgformer: Generative social graph transformer for multimodal pedestrian trajectory prediction,”arXiv preprint arXiv:2312.04479, 2023
2023 arXiv
-
[26]
Graph neural networks for modelling traffic participant interaction,
F. Diehl, T. Brunner, M. T. Le, and A. Knoll, “Graph neural networks for modelling traffic participant interaction,” in 2019 IEEE Intelligent Vehicles Symposium (IV). IEEE, 2019, pp. 695–701
2019
-
[27]
Cooperative behavior planning for automated driving using graph neural networks,
M. Klimke, B. V ¨olz, and M. Buchholz, “Cooperative behavior planning for automated driving using graph neural networks,” in 2022 IEEE Intelligent Vehicles Symposium (IV) . IEEE, 2022, pp. 167–174
2022
-
[28]
Graph neural network based relation learning for abnormal perception information detection in self-driving scenarios,
K. Jin, H. Wang, C. Liu, Y . Zhai, and L. Tang, “Graph neural network based relation learning for abnormal perception information detection in self-driving scenarios,” in 2022 International Conference on Robotics and Automation (ICRA) . IEEE, 2022, pp. 8943–8949
2022
-
[29]
Scene: Reasoning about traffic scenes using heterogeneous graph neural networks,
T. Monninger, J. Schmidt, J. Rupprecht, D. Raba, J. Jordan, D. Frank, S. Staab, and K. Dietmayer, “Scene: Reasoning about traffic scenes using heterogeneous graph neural networks,” IEEE Robotics and Automation Letters, vol. 8, no. 3, pp. 1531–1538, 2023
2023
-
[30]
Driving scene understanding with traffic scene-assisted topology graph transformer,
F. Rong, W. Peng, M. Lan, Q. Zhang, and L. Zhang, “Driving scene understanding with traffic scene-assisted topology graph transformer,” in Proceedings of the 32nd ACM International Conference on Multimedia , 2024, pp. 10 075–10 084
2024
-
[31]
A gentle introduction to deep learning for graphs,
D. Bacciu, F. Errica, A. Micheli, and M. Podda, “A gentle introduction to deep learning for graphs,” Neural Networks, vol. 129, pp. 203–221, 2020
2020
-
[32]
Graph attention networks,
P. Velickovic, G. Cucurull, A. Casanova, A. Romero, P. Li `o, and Y . Bengio, “Graph attention networks,” in 6th International Conference on Learning Representations, ICLR . OpenReview.net, 2018. [Online]. Available: https://openreview.net/forum?id=rJXMpikCZ
2018
-
[33]
Neural machine translation by jointly learning to align and translate,
D. Bahdanau, K. Cho, and Y . Bengio, “Neural machine translation by jointly learning to align and translate,” in 3rd International Conference on Learning Representations, ICLR 2015, San Diego, CA, USA, May 7-9, 2015, Conference Track Proceedings , Y . Bengio and Y . LeCun, Eds...
2015 arXiv
-
[34]
Attention is all you need,
A. Vaswani, “Attention is all you need,” Advances in Neural Information Processing Systems, 2017
2017
-
[35]
Focal loss for dense object detection,
T. Lin, P. Goyal, R. B. Girshick, K. He, and P. Doll ´ar, “Focal loss for dense object detection,” IEEE Trans. Pattern Anal. Mach. Intell., vol. 42, no. 2, pp. 318–327, 2020. [Online]. Available: https://doi.org/10.1109/TPAMI.2018.2858826
2020
-
[36]
nuscenes: A multimodal dataset for autonomous driving,
H. Caesar, V . Bankiti, A. H. Lang, S. V ora, V . E. Liong, Q. Xu, A. Kr- ishnan, Y . Pan, G. Baldan, and O. Beijbom, “nuscenes: A multimodal dataset for autonomous driving,” in 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition, CVPR 2020, Seattle, WA, USA, J...
2020
-
[37]
Drivelm: Driving with graph visual question answering,
C. Sima, K. Renz, K. Chitta, L. Chen, H. Zhang, C. Xie, P. Luo, A. Geiger, and H. Li, “Drivelm: Driving with graph visual question answering,” CoRR, vol. abs/2312.14150, 2023
2023 arXiv
-
[38]
Random forests,
L. Breiman, “Random forests,” Machine learning , vol. 45, no. 1, pp. 5–32, 2001
2001
-
[39]
A decision-theoretic generalization of on-line learning and an application to boosting,
Y . Freund and R. E. Schapire, “A decision-theoretic generalization of on-line learning and an application to boosting,” Journal of computer and system sciences , vol. 55, no. 1, pp. 119–139, 1997
1997
-
[40]
Fast graph representation learning with PyTorch Geometric,
M. Fey and J. E. Lenssen, “Fast graph representation learning with PyTorch Geometric,” in ICLR Workshop on Representation Learning on Graphs and Manifolds , 2019
2019
-
[41]
Scikit-learn: Machine learning in python,
F. Pedregosa, G. Varoquaux, A. Gramfort, V . Michel, B. Thirion, O. Grisel, M. Blondel, P. Prettenhofer, R. Weiss, V . Dubourg, J. Vander- Plas, A. Passos, D. Cournapeau, M. Brucher, M. Perrot, and E. Duches- nay, “Scikit-learn: Machine learning in python,” J. Mach. Learn. Res...
2011
Reviewed August 16, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.