Pith. sign in

REVIEW 4 major objections 4 minor 37 references

A Shapley-value game over query atoms attributes each part of a complex knowledge-graph query to its effect on an answer's rank.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

CQD-SHAP uses Shapley values over query atoms to quantify how much neural (versus symbolic) execution of each atom contributes to a target answer's ranking in complex query answering.

T0 review reviewed 2026-08-04 challenge →

load-bearing objection A clean, workmanlike contribution: exact Shapley values for query atoms under a rank-based game, with honest limitations and fixable empirical gaps. the 4 major comments →

arxiv 2510.15623 v2 pith:SH53WQIN submitted 2025-10-17 cs.LG cs.AI

CQD-SHAP: Explainable Complex Query Answering via Shapley Values

classification cs.LG cs.AI
keywords Explainable Query AnsweringShapley ValueComplex Query AnsweringNeurosymbolic Query AnsweringKnowledge GraphRanking ExplanationNecessary and Sufficient Explanations
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper introduces CQD-SHAP, a framework that explains why a neurosymbolic query-answering model ranks a particular entity as an answer to a complex query. It defines a cooperative game whose players are the query's atoms: an atom can be executed either by the neural link predictor or by plain symbolic graph lookup, and the payoff of a coalition is the improvement in the target answer's filtered rank over purely symbolic execution. The Shapley value of each atom measures, averaged over all combinations of other atoms, how much switching that atom from symbolic to neural execution helps or hurts the answer's rank. The values satisfy efficiency: they sum exactly to the rank difference between fully neural and fully symbolic execution, giving a formal budget for how the neurosymbolic model changed the ranking. The authors evaluate the resulting explanations on two knowledge-graph benchmarks, showing that the most important atoms identified by CQD-SHAP are more often necessary and sufficient for the ranking than baseline atom-selection rules.

Core claim

Under CQD-SHAP, a query atom's explanation is its Shapley value in the game v(S)=r(Q_empty)-r(Q_S), where Q_S executes atoms in S neurally and the rest symbolically, and r is the target answer's filtered rank. Because the empty coalition's payoff is zero by construction, the four Shapley axioms hold; in particular, the efficiency axiom yields that the sum of the atoms' values equals the total rank shift from fully symbolic to fully neural execution. Individual values are exact (no approximation is needed for typical query sizes), and they cleanly distinguish atoms whose high ranking comes from knowledge already in the graph from atoms where the neural model's inferred links are doing the wor

What carries the argument

The central object is the Shapley game over query atoms: players are the atoms of a DNF query, a coalition S is the partial query that executes atoms in S with the neural link predictor and all others with symbolic graph lookups, and the payoff v(S)=r(Q_empty)-r(Q_S) is the improvement in the target answer's filtered rank relative to fully symbolic execution. The Shapley value formula (weighted average of marginal payoffs over all coalitions) then yields per-atom contributions that satisfy the efficiency, symmetry, linearity, and null-player axioms; the efficiency identity (sum of atom values equals r(Q_empty)-r(Q_all)) is what gives the explanations a formal, checkable meaning.

Load-bearing premise

The whole approach equates 'an atom's importance' with the change in the target answer's filtered rank when that atom is switched between neural and symbolic execution; if a user's notion of importance is better captured by score margins, set membership, or correctness, the Shapley values remain mathematically valid but may not match that notion.

What would settle it

Construct a query with a dummy atom that never affects the answer set (e.g., a relation that holds for no entities in either graph) and observe whether CQD-SHAP assigns it zero Shapley value; the null-player axiom demands zero, so a nonzero value would show the implementation violates the paper's axiomatic guarantees. Alternatively, on a small 2-atom query where all four coalition rankings can be computed by hand, verify that the reported Shapley values sum to r(Q_empty)-r(Q_neural) and that swapping the top-valued atom to symbolic actually produces the largest rank drop among all atoms—if a l

Watch this falsifier. Get emailed when new claim-graph text bears on it.

If this is right

  • For a given answer, each query atom receives a number that sums exactly to the total rank improvement of using the neurosymbolic model; this gives users a budget-like explanation of why the answer appeared.
  • Users can distinguish atoms that rely on facts already present in the knowledge graph from atoms where the neural predictor inferred missing links, enabling them to check whether the ranking is driven by a misleading correlation.
  • Because the method computes exact Shapley values (no sampling) and typical complex queries have few atoms, the explanation is fast enough for interactive inspection (tens to hundreds of milliseconds per query-answer pair).
  • The framework can also explain false positives: for an incorrect answer, the per-atom contributions show which query part the neural model wrongly over-trusted.
  • The necessary/sufficient evaluation protocol introduced here gives a concrete way to compare future explanation methods for complex query answering.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • The same 'switch each atom between neural and symbolic execution and measure rank change' recipe could be applied to other neurosymbolic CQA models (e.g., box-based or message-passing decoders) as long as the model decomposes queries into atoms; the explanation method is not intrinsically tied to CQD's fuzzy-logic aggregation.
  • Because Shapley values are averages over all coalitions, the paper's own caveat holds: greedily executing atoms with positive values neurally and negative ones symbolically does not guarantee an optimal ranking; a natural extension is to use the values as inputs to a planner that selects a subset of atoms for neural execution under a budget.
  • The value function could be replaced by alternative quantities of interest (score margins, set membership, correctness) to match different user notions of 'importance'; comparing those variants against human judgments would clarify what the rank-based explanation really means.
  • The dataset-side observation in the paper (some 'hard' answers are actually symbolically reachable) suggests that future benchmark construction should verify answer hardness per query atom, otherwise explanation evaluations on such data can be biased.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

4 major / 4 minor

Summary. The paper proposes CQD-SHAP, a Shapley-value framework for explaining, at the level of query atoms, why a neurosymbolic complex-query-answering model ranks a particular answer entity where it does. The players of the cooperative game are the atoms of a query; a coalition S is executed by running the atoms in S with the neural link predictor and the remaining atoms symbolically, and the payoff is the change in filtered rank of the target answer between fully symbolic execution and the partial execution Q_S. Exact Shapley values are computed without approximation, and efficiency gives Eq. (9): the sum of atom attributions equals the total rank change between fully neural and fully symbolic execution. The method is evaluated with necessary/sufficient explanation scenarios on FB15k-237 and NELL995, plus a small case study.

Significance. If validated, CQD-SHAP is a useful first step toward explaining neural/neurosymbolic CQA systems, and it has the appeal of being exact and axiomatically grounded rather than heuristic. The game definition is clean and the efficiency identity is explicit. The paper is also reproducible: code, data, and pretrained models are released, and the main computation is exact with no sampling. The significance is, however, conditional on the choice of the rank-based quantity of interest being semantically appropriate and on the empirical evaluation being statistically credible; the current validation is weaker than the abstract implies.

major comments (4)
  1. [Section 4, Eq. (7)] The choice of DeltaRank as the quantity of interest is load-bearing but unvalidated. The Shapley values are exact for this particular game, but the game encodes a specific definition of 'importance' as improvement in filtered rank. Rank is a global and highly non-linear statistic: it can change because of score fluctuations of many irrelevant entities, so the attributions may reflect global embedding-space behavior rather than the atom's direct logical or reasoning contribution. The paper cites Pliatsika et al. for QoI definitions but gives no evidence that filtered rank improvement matches user-intuitive importance or better alternatives such as score margins, top-k membership, or set membership. Please either validate DeltaRank against alternative QoIs (and ideally human judgments) or reframe the central claim as 'attribution under the DeltaRank game'.
  2. [Section 5.2, Table 2] The empirical validation reports means only, with no error bars, confidence intervals, or significance tests. Several comparisons with the score-based baseline are very close or even identical, for example FB15k-237 sufficient 2u (+0.348 vs. +0.348) and 2u1p (+0.205 vs. +0.205). The conclusion of 'consistent effectiveness' is therefore not statistically supported. Moreover, the necessary/sufficient evaluation is a consistency check on the same rank-based criterion used to define the Shapley game: selecting the atom with the highest Shapley value and then toggling that atom and measuring the rank change tests internal consistency, not external explanatory fidelity. Please report bootstrap confidence intervals and paired significance tests, and consider an independent ground truth or user study.
  3. [Section 4, Eqs. (5)-(6)] The notation in the filtered-rank definition is ambiguous and possibly inconsistent. If E_A is literally the 'answer set', then E_A \ (E_easy ∪ (E_hard \ {e_i})) leaves only {e_i}, making every rank equal to 1. The formula only makes sense if E_A is the full candidate entity set over which CQD scores are computed. In Section 3.1, however, E_A is defined as the answer set. Please define E_A, E_easy, and E_hard unambiguously and state explicitly that the filtered ranking is computed over the full entity set, with correct answers removed except for the target hard answer.
  4. [Section 4, Eq. (4)] The implementation of symbolic intermediate atoms is underspecified with respect to tie-breaking. A symbolic atom assigns score 1 to every entity connected via the relation in the observed graph and a score near 0 to all other entities. When more than k entities satisfy an intermediate symbolic atom, the top-k beam search must break ties arbitrarily, and this can make the computed Shapley values depend on entity order and on beam size k. Since the paper claims exactness and reproducibility, the tie-breaking rule should be stated and sensitivity to k should be reported.
minor comments (4)
  1. [Section 5, p. 7] The phrase 'quantitative results of a case study' appears before Section 5.3, but the case study is qualitative. Reword to 'a case study' or 'quantitative results and a case study'.
  2. [Section 4, after Eq. (9)] The paper states that the value function satisfies 'all the fundamental Shapley value axioms' but does not verify them in the text. Since efficiency is central to the contribution, add a short proof or a precise reference to the matching proof in Chowdhury et al. / Pliatsika et al. for DeltaRank.
  3. [Appendix B, Table 4] The use of the terms 'Nec.' and 'Suff.' in Table 4 is confusing because 'necessary' and 'sufficient' are also the names of the evaluation scenarios. Clarify that these columns are the numbers of hard answers that are ranked first by CQD (necessary) and not ranked first (sufficient).
  4. [Section 5.2] In the paragraph explaining structural differences, the statement 'this pattern is not completely observed for other baselines' is vague. Please name the specific baselines and query types for which the pattern does or does not hold.

Circularity Check

0 steps flagged

No circularity: exact Shapley values are derived from a self-contained game definition and external axioms; the evaluation checks an independent consequence rather than a fitted prediction.

full rationale

The paper's derivation chain is: define a cooperative game over query atoms where the payoff is v(S)=r_i(Q_empty)-r_i(Q_S) (Eq. 7), plug this into the standard Shapley formula (Eq. 8), and then note that the efficiency axiom gives Eq. 9, i.e., the sum of the Shapley values equals the rank difference between fully neural and fully symbolic execution. Equation 9 is a mathematical theorem of Shapley values, not an empirical prediction or a fitted outcome, so stating it as an identity is not circular. No parameters are fitted to the data and then renamed as predictions: CQD-SHAP computes exact Shapley values for a fixed, externally specified game, and the CQD link predictor is taken as a pretrained model. The load-bearing axiomatic and QoI results are cited from external works (Shapley 1953; Pliatsika et al. 2025; Chowdhury et al. 2025), not from the present authors' prior work, and no uniqueness theorem or ansatz is imported from self-citations. The necessary/sufficient evaluation checks whether the atom with the highest Shapley value, when its execution mode is flipped, changes the rank; because a Shapley value is an average over all coalitions, this behavior is not guaranteed by construction, and the paper reports varying effectiveness across query types, confirming the evaluation is not merely unpacking the definition. The concern that ΔRank is an unvalidated proxy for user-perceived importance is a validity question, not a circularity: the Shapley values are exactly what they are defined to be, and the paper does not claim to fit or derive that choice from the data. The paper is self-contained against external benchmarks and contains no reduction of a claimed prediction to its own inputs.

Axiom & Free-Parameter Ledger

2 free parameters · 6 axioms · 0 invented entities

This work does not fit any data; it defines a game and computes exact Shapley values. The only free choices are hyperparameters inherited from CQD (beam size k, product t-norm) plus the rank-based payoff. The axioms are mostly standard game theory and domain assumptions about the CQD pipeline; no new entities are introduced.

free parameters (2)
  • beam size k = 10
    Number of intermediate entities retained in partial-query execution; set to default 10 for all query types, not tuned in this paper, but affects ranks and hence Shapley values.
  • t-norm/t-conorm = product norm and probabilistic sum conorm
    Choice of fuzzy logic connectives for combining atom scores; inherited from CQD, but value function depends on it.
axioms (6)
  • standard math Shapley value formula and its axioms (efficiency, symmetry, linearity, null player)
    Used to compute per-atom attributions; assumed from game theory.
  • domain assumption CQD's neural link predictor scores rho reflect likelihood of atom truth
    Partial queries executed with CQD rely on pretrained scores; if scores are poorly calibrated, rank-based attributions inherit the error.
  • domain assumption The complete test graph G_test is available to define hard and easy answers
    Ground-truth hard answers are defined as those reachable in G_test but not G_valid; the paper uses benchmark splits from Arakelyan et al. [1].
  • domain assumption Beam search with top-k is a faithful approximation of symbolic/neural partial-query execution
    Intermediate atoms are truncated to top k entities; this may omit relevant entities and affect rank computations.
  • domain assumption Rank is a meaningful scalar payoff for explanation
    The value function is rank-based (Eq. 7); no user study or alternative validation.
  • domain assumption EPFO/DNF query formalism and fuzzy t-norm/t-conorm semantics
    Follows CQD; the game value function depends on this semantics.

reviewed 2026-08-04 · how reviews work

0 comments
Cite this review

Pith. "Pith review of CQD-SHAP: Explainable Complex Query Answering via Shapley Values." pith.science (2026). https://pith.science/paper/SH53WQIN

@misc{pith2026251015623,
  author       = {Pith},
  title        = {Pith review of: CQD-SHAP: Explainable Complex Query Answering via Shapley Values},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/SH53WQIN}},
  note         = {Machine review of arXiv:2510.15623}
}
Share X Bluesky LinkedIn Reddit HN
read the original abstract

Complex query answering (CQA) goes beyond the widely studied link prediction task by addressing more sophisticated queries that require multi-hop reasoning over incomplete knowledge graphs (KGs). Research on neural and neurosymbolic CQA methods is still an emerging field. Almost all of these methods can be regarded as black-box models, which may raise concerns about user trust. Although neurosymbolic approaches like CQD are slightly more interpretable, allowing intermediate results to be tracked, the importance of different parts of the query remains unexplained. In this paper, we propose CQD-SHAP, a novel framework that computes the contribution of each query part to the ranking of a specific answer. This contribution explains the value of leveraging a neural predictor that can infer new knowledge from an incomplete KG, rather than a symbolic approach relying solely on existing facts in the KG. CQD-SHAP is formulated based on Shapley values from cooperative game theory and satisfies all fundamental Shapley axioms. Automated evaluation of these explanations in terms of necessary and sufficient explanations, and comparisons with various baselines, show the consistent effectiveness of this approach across all studied datasets and query types.

Figures

Figures reproduced from arXiv: 2510.15623 by Parsa Abbasi, Stefan Heindorf.

Figure 1
Figure 1. Figure 1: Illustration of all possible partial queries [PITH_FULL_IMAGE:figures/full_fig_p004_1.png] view at source ↗
Figure 2
Figure 2. Figure 2: An example of a 2𝑖 query in the FB15k-237 test data. The query asks for piano instrumentalists who are also artists in the Rock music genre. This query has 22 hard answers (e.g., Serj Tankian, Jon Bon Jovi) and 113 easy answers. (Piano, /music/instrument/instrumentalists, Paul Weller) does not exist in the graph, so leveraging CQD enables the model to predict this answer at a better rank. If the query is e… view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

37 extracted references · 4 linked inside Pith

  1. [1]

    Erik Arakelyan, Daniel Daza, Pasquale Minervini, and Michael Cochez. 2021. Complex Query Answering with Neural Link Predictors. InICLR. OpenRe- view.net

  2. [2]

    Erik Arakelyan, Pasquale Minervini, Daniel Daza, Michael Cochez, and Isabelle Augenstein. 2023. Adapting Neural Link Predictors for Data-Efficient Complex Query Answering. InNeurIPS

  3. [3]

    Meghyn Bienvenu, Diego Figueira, and Pierre Lafourcade. 2024. Shapley Value Computation in Ontology-Mediated Query Answering. InKR

  4. [4]

    Bollacker, Colin Evans, Praveen K

    Kurt D. Bollacker, Colin Evans, Praveen K. Paritosh, Tim Sturge, and Jamie Taylor

  5. [5]

    Antoine Bordes, Nicolas Usunier, Alberto García-Durán, Jason Weston, and Oksana Yakhnenko. 2013. Translating Embeddings for Modeling Multi-relational Data. InNIPS. 2787–2795

  6. [6]

    Heng Chang, Jiangnan Ye, Alejo Lopez-Avila, Jinhua Du, and Jia Li. 2024. Path- based Explanation for Knowledge Graph Completion. InKDD. ACM, 231–242

  7. [7]

    Adriane Chapman and H. V. Jagadish. 2009. Why not?. InSIGMOD Conference. ACM, 523–534

  8. [8]

    Tanya Chowdhury, Yair Zick, and James Allan. 2025. RankSHAP: Shapley Value Based Feature Attributions for Learning to Rank. InICLR. OpenReview.net

  9. [9]

    Daniel Daza and Michael Cochez. 2020. Message Passing for Query Answering over Knowledge Graphs.CoRRabs/2002.02406 (2020)

  10. [10]

    Caglar Demir, Michel Wiebesiek, Renzhong Lu, Axel-Cyrille Ngonga Ngomo, and Stefan Heindorf. 2023. LitCQD: Multi-hop Reasoning in Incomplete Knowledge Graphs with Numeric Literals. InECML/PKDD. Springer, 617–633

  11. [11]

    Daniel Deutch, Nave Frost, Benny Kimelfeld, and Mikaël Monet. 2022. Computing the Shapley Value of Facts in Query Answering. InSIGMOD Conference. ACM, 1570–1583

  12. [12]

    Sneha Gathani, Peter Lim, and Leilani Battle. 2020. Debugging Database Queries: A Survey of Tools, Techniques, and Users. InCHI. ACM, 1–16

  13. [13]

    Cosimo Gregucci, Bo Xiong, Daniel Hernández, Lorenzo Loconte, Pasquale Min- ervini, Steffen Staab, and Antonio Vergari. 2024. Is Complex Query Answering Really Complex?CoRRabs/2410.12537 (2024)

  14. [14]

    Hamilton, Payal Bajaj, Marinka Zitnik, Dan Jurafsky, and Jure Leskovec

    William L. Hamilton, Payal Bajaj, Marinka Zitnik, Dan Jurafsky, and Jure Leskovec. 2018. Querying Complex Networks in Vector Space.CoRR abs/1806.01445 (2018)

  15. [15]

    Maria Heuss, Maarten de Rijke, and Avishek Anand. 2025. RankingSHAP - Faithful Listwise Feature Attribution Explanations for Ranking Models. InSIGIR. ACM, 381–391

  16. [16]

    Bertossi, Benny Kimelfeld, and Moshe Sebag

    Ester Livshits, Leopoldo E. Bertossi, Benny Kimelfeld, and Moshe Sebag. 2020. The Shapley Value of Tuples in Query Answering. InICDT (LIPIcs, Vol. 155). Schloss Dagstuhl - Leibniz-Zentrum für Informatik, 20:1–20:19

  17. [17]

    Lundberg and Su-In Lee

    Scott M. Lundberg and Su-In Lee. 2017. A Unified Approach to Interpreting Model Predictions. InNIPS. 4765–4774

  18. [18]

    Debasis Mohapatra, Satnam Singh, Rupsalin Pradhan, and Soumya Ranjan Behera

  19. [19]

    Simon Ott, Melisachew Wudage Chekol, Christian Meilicke, and Heiner Stuck- enschmidt. 2025. Training-Free Score Calibration for Complex Query Decompo- sition. InESWC, Vol. 15718. Springer, 188–207

  20. [20]

    Pouya Pezeshkpour, Yifan Tian, and Sameer Singh. 2019. Investigating Robust- ness and Interpretability of Link Prediction via Adversarial Modifications. In NAACL-HLT. ACL, 3336–3347

  21. [21]

    Venetia Pliatsika, João Fonseca, Kateryna Akhynko, Ivan Shevchenko, and Julia Stoyanovich. 2025. ShaRP: Explaining Rankings and Preferences with Shapley Values.Proc. VLDB Endow.18, 11 (2025), 4131–4143

  22. [22]

    Hongyu Ren, Mikhail Galkin, Michael Cochez, Zhaocheng Zhu, and Jure Leskovec. 2023. Neural Graph Reasoning: Complex Logical Query Answering Meets Graph Databases.CoRRabs/2303.14617 (2023)

  23. [23]

    Hongyu Ren, Weihua Hu, and Jure Leskovec. 2020. Query2box: Reasoning over Knowledge Graphs in Vector Space Using Box Embeddings. InICLR. OpenRe- view.net

  24. [24]

    Andrea Rossi, Denilson Barbosa, Donatella Firmani, Antonio Matinata, and Paolo Merialdo. 2021. Knowledge Graph Embedding for Link Prediction: A Comparative Analysis.ACM Trans. Knowl. Discov. Data15, 2 (2021), 14:1–14:49

  25. [25]

    Andrea Rossi, Donatella Firmani, Paolo Merialdo, and Tommaso Teofili. 2022. Explaining Link Prediction Systems based on Knowledge Graph Embeddings. In SIGMOD Conference. ACM, 2062–2075

  26. [26]

    Lloyd S. Shapley. 1953. A Value for n-Person Games. InContributions to the Theory of Games II. Princeton University Press, 307–317

  27. [27]

    Kristina Toutanova and Danqi Chen. 2015. Observed versus latent features for knowledge base and text inference. InCVSC. ACL, 57–66

  28. [28]

    Denny Vrandecic and Markus Krötzsch. 2014. Wikidata: a free collaborative knowledgebase.Commun. ACM57, 10 (2014), 78–85

  29. [29]

    Meihong Wang, Linling Qiu, and Xiaoli Wang. 2021. A Survey on Knowledge Graph Embeddings for Link Prediction.Symmetry13, 3 (2021), 485

  30. [30]

    Wong, and Simon See

    Zihao Wang, Yangqiu Song, Ginny Y. Wong, and Simon See. 2023. Logical Message Passing Networks with One-hop Inference on Atomic Formulas. In ICLR. OpenReview.net

  31. [31]

    Robert West, Evgeniy Gabrilovich, Kevin Murphy, Shaohua Sun, Rahul Gupta, and Dekang Lin. 2014. Knowledge base completion via search-based question answering. InWWW. ACM, 515–526

  32. [32]

    Eyal Winter. 2002. The Shapley value.Handbook of Game Theory with Economic Applications3 (2002), 2025–2054

  33. [33]

    Wenhan Xiong, Thien Hoang, and William Yang Wang. 2017. DeepPath: A Reinforcement Learning Method for Knowledge Graph Reasoning. InEMNLP. ACL, 564–573

  34. [34]

    Bohui Zhang, Filip Ilievski, and Pedro A. Szekely. 2022. Enriching Wikidata with Linked Open Data. InWikidata@ISWC (CEUR Workshop Proceedings, Vol. 3262). CEUR-WS.org

  35. [35]

    Shichang Zhang, Jiani Zhang, Xiang Song, Soji Adeshina, Da Zheng, Christos Faloutsos, and Yizhou Sun. 2023. PaGE-Link: Path-based Graph Neural Network Explanation for Heterogeneous Link Prediction. InWWW. ACM, 3784–3793. Parsa Abbasi and Stefan Heindorf A Runtime Table 3: Average runtime (in milliseconds) per query type for computing Shapley values of a g...

  36. [2008]

    InSIGMOD Conference

    Freebase: a collaboratively created graph database for structuring human knowledge. InSIGMOD Conference. ACM, 1247–1250

  37. [2023]

    Integrating structural features for link prediction in directed social network using supervised machine learning with SHapley Additive exPlanations. InOCIT. IEEE, 468–472

This paper was first reviewed by deepseek-v4-flash on August 4, 2026.