Causal Cartographer: From Mapping to Reasoning Over Counterfactual Worlds

Ga\"el Gendron; Gillian Dobbie; Jo\v{z}e M. Ro\v{z}anec; Michael Witbrock

REVIEW 4 major objections 6 minor 66 references

Causal Cartographer: From Mapping to Reasoning Over Counterfactual Worlds

T0 review · 4 major / 6 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read The paper proves real-world counterfactual 'what if' queries can be rewritten as ordinary observational queries by matching two observed worlds through a causal blanket.

desk verdict The CausalWorld resource and efficiency results are real, but the paper's central claim—that K-Matching provably yields real-world counterfactuals—rests on a theorem whose proof doesn't go through. read the letter →

arxiv 2505.14396 v1 pith:MFKLBMI3 submitted 2025-05-20 cs.AI cs.CLcs.LG

Gaël Gendron , Jože M. Rožanec , Michael Witbrock , Gillian Dobbie This is my paper · ORCID

classification cs.AIcs.CLcs.LG

keywords causalextractioncounterfactualreasoningworldmodelsgraphretrieval-augmentedgenerationblanketslargelanguageinferenceK-matching

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Large language models answer causal questions by pattern-matching memorized relationships rather than by causal inference, and real-world counterfactuals are hard to evaluate because only the factual world is ever observed. This paper addresses both problems with a two-part framework: an extraction agent that reads news articles and builds a causal network called CausalWorld (975 variables, 1337 causal relationships from 2020 oil-price coverage), and a reasoning agent that solves counterfactual queries by step-by-step inference from causal parents. The load-bearing claim is Theorem 1 (K-Matching Equivalence): if two observed worlds share a target variable and one world's causal blanket can be matched to the other's observations, the counterfactual query collapses to an ordinary observational query, $P(T \mid \mathrm{do}(B_c \setminus O_s), O_o \setminus O_s) = P(T \mid B_c)$. If the assumptions hold, counterfactual questions can be generated and answered from real-world text, and the constrained reasoning agent does so with accuracy comparable to chain-of-thought prompting while cutting inference cost by up to 70%.

What carries the argument

The causal blanket is the load-bearing object: a set of ancestors of a target $T$ that fully determines $T$ through a deterministic function, so that knowing the blanket renders $T$ conditionally independent of everything else. Unlike a Markov blanket, it can include non-parent ancestors (in the chain $A \to B \to C$, $A$ is a causal blanket for $C$). K-Matching is the pairing operation that makes the theorem usable: given two observed worlds $O_o$ and $O_c$ sharing observations $O_s$, and a blanket $B_c$ of the counterfactual world, it splits $B_c$ into $N-K$ matched observations and $K$ interventions. The theorem states the identity $P(T \mid \mathrm{do}(B_c \setminus O_s), O_o \setminus O_s) = P(T \mid B_c)$, and the proof carries the reduction by rewriting the counterfactual over shared exogenous variables and applying rule 2 of do-calculus to drop the intervening variables.

What would settle it

Build a synthetic structural causal model with known ground-truth equations, run the same extraction and K-matching pipeline on text describing its worlds, and compare the theorem's predicted value $P(T \mid B_c)$ against the true counterfactual computed from the model's equations; any systematic disagreement would trace to a violated blanket-completeness or identifiability assumption and would delimit the claim's reach.

Watch

Extended reading notes

Core claim

The paper's central claim is that real-world counterfactual inference becomes tractable and provably correct once causal knowledge is organized as a graph whose nodes carry multiple observed 'worlds.' The authors define a causal blanket — a set of ancestors that fully determines the target through a deterministic function — and a K-Matching procedure that finds two observed worlds that can serve as a factual/counterfactual pair. Theorem 1 (K-Matching Equivalence) asserts that, provided the unmatched factual observations themselves form causal blankets over each shared variable, the target's distribution under interventions on the unmatched blanket variables equals its plain conditional distribution: $P(T \mid \mathrm{do}(B_c \setminus O_s), O_o \setminus O_s) = P(T \mid B_c)$. In the paper's own terms, this proves that the CausalWorld graph gives provable access to counterfactual knowledge, so real-world counterfactuals can be estimated from natural-language data rather than only from synthetic simulators.

Load-bearing premise

The framework assumes the causal graph extracted from news is complete and correct enough that a causal blanket fully and deterministically fixes the target variable, and that the unmatched factual observations uniquely determine each shared variable; if either fails, the K-Matching equality and the provable-access claim collapse.

Editorial extensions

If this is right

Counterfactual evaluation need no longer be confined to synthetic datasets: matched real-world news worlds can serve as ground-truth factual and counterfactual pairs.
Restricting an LLM's context to causal parents and children preserves accuracy relative to chain-of-thought prompting while cutting inference cost by up to 70%.
The extraction pipeline can grow a causal repository from unstructured text, with the graph retrieval component preventing duplicate nodes so that knowledge accumulates incrementally.
Smaller models benefit most: an 8-billion-parameter model that timed out under the chain-of-thought baseline completed the tasks under causal constraints, using roughly 72% less context and 91% less output.
Because most nodes lie in one connected component linked by bridge nodes, a large share of the network's variables can participate in counterfactual computations.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

A direct stress test of the proof's reach would run the same extraction and K-matching pipeline on a corpus paired with a known ground-truth structural causal model, then measure how the equality degrades as the causal blanket is corrupted; the theorem predicts the error stays at zero as long as blankets remain identifiable.
The causal blanket concept sits between a Markov blanket and full structural-model determinism, effectively proposing a minimal sufficient statistic for counterfactual queries; if that reading is right, similar matching identities might be derivable for other query types, such as pure interventions in partially observed settings.
Because every document is recorded as its own world instantiation, the framework should transfer beyond news to any narrative corpus — clinical notes, incident reports, historical accounts — where overlapping descriptions of the same variables can be aligned.

Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

Read this one for the resource, not for the theorem. The CausalWorld network (975 nodes, 1337 edges, multi-world instantiations from 2020 news) and the NxGraphRAG retrieval pipeline are genuinely useful contributions; the CTG-Reason agent shows real efficiency gains—up to 70% cost reduction with comparable accuracy on the boolean/trend subsets—and the code and data are public. That is more than most papers ship.

The problem is the load-bearing claim. The abstract and Section 5 say the framework 'provably' estimates true real-world counterfactuals via K-Matching (Theorem 1). The proof in Appendix C does not hold together. It expands over exogenous variables U, then defines S as 'the variables instantiating Os' and sums over S in U—conflating observed endogenous variables with unobserved exogenous noise. The independence T ⊥⊥ U\S | Bc\S is asserted without proof. The step where Oo\Os forms a causal blanket 'over S' is undefined, and the point-mass reduction of P(S | Oo\Os) doesn't follow. The final application of do-calculus Rule 2 is also invalid: the rule requires T to be independent of the intervened variables given the other conditioning set in the graph with incoming edges to those variables removed, but Bc\Os are ancestors of T by Definition 1, so that d-separation fails in general.

Even if the proof were repaired, the hypotheses aren't met in practice. Definition 1 requires a deterministic function T=f(Bc); the graph stores text descriptions, not functional or distributional mechanisms. Section 5.1 simply assumes blankets can be constructed from CausalWorld, and Section 7 concedes full knowledge of the causal graph cannot be guaranteed. The evaluation doesn't rescue this: the CausalWorld-CR ground truth is generated by the same extraction and matching pipeline, so the reported accuracies measure internal consistency with the extractor, not validity against any external benchmark. There are also no significance tests on the accuracy comparisons, so 'similar results' is doing a lot of work.

None of this makes the paper worthless. The dataset and the cost-reduction results are real. But the central theoretical claim, as written, is not supported. I'd send this to peer review—it's exactly the kind of paper a good referee can sharpen—but I'd expect the counterfactual-equivalence claim to be either fixed with a valid theorem and verified hypotheses, or dropped in favor of a systems/dataset framing.

Referee Report

4 major / 6 minor

Summary. The paper proposes Causal Cartographer, a two-agent framework: CTG-Extract uses a graph retrieval-augmented generation agent to extract causal variables and relationships from 500 EventRegistry news events, building a CausalWorld graph with 975 nodes and 1,337 edges; CTG-Reason then answers counterfactual queries by stepwise inference using parent/child variables. Section 5 introduces causal blankets and K-Matching, states Theorem 1 claiming P(T | do(Bc \ Os), Oo \ Os) = P(T | Bc), and uses this to construct a 400-sample CausalWorld-CR evaluation dataset. The authors evaluate o3-mini, GPT-4.1, and LLaMA-3.1-8B against a CausalCoT baseline, reporting comparable accuracy for boolean/trend queries and large reductions in context window and output length. The paper claims that the method 'provably' accesses counterfactual knowledge and enables estimation of real-world counterfactuals.

Significance. If Theorem 1 were valid and its hypotheses satisfied, the paper would make a useful contribution by showing how extracted real-world causal graphs could support counterfactual evaluation without synthetic data. The engineering contributions are real: the graph-RAG extraction pipeline, the released code, and the documented inference-cost reduction (up to 72% input-token and 91% output-token reduction on LLaMA-3.1-8B) are concrete and reproducible. However, the central theoretical claim is not established: the proof of Theorem 1 in Appendix C is internally invalid, and the hypotheses of the theorem are not satisfied by the CausalWorld extraction. In addition, the evaluation ground truth is generated by the same extraction pipeline that creates the graph, so the reported accuracies measure internal consistency with the extraction rather than correctness of real-world counterfactuals. The system contribution is promising, but the 'provable counterfactual access' claim, which is load-bearing for the abstract and conclusion, is unsupported.

major comments (4)

[Appendix C, proof of Theorem 1] The proof conflates observed endogenous variables with exogenous variables. After expanding P(T | do(Bc \ Os), Oo \ Os) as a sum over exogenous U, it defines S as 'the variables instantiating Os' and then sums over S in U, although Os is by Definition 2 a subset of observed observations from V. The assertion T independent of U \ S given Bc \ S is introduced without proof and does not follow from the blanket condition, which only gives T independent of U given Bc. The point-mass claim that P(S | Oo \ Os) is concentrated on the observed values of Os is also unjustified: even if Oo \ Os were a causal blanket for each variable of Os in the sense of Definition 1, that would require a deterministic function, which the extracted graph does not provide. Finally, the do-calculus step misapplies Rule 2: Rule 2 requires T independent of Bc \ Os given Os in the graph with incoming edges to Bc \ Os removed, but by Definition 1 every variable in Bc is an ancestor of T, so this conditional independence generally fails. The equality in Theorem 1 is therefore not proven.
[Sections 5.1 and 7, hypotheses of Theorem 1] Theorem 1 presupposes that Bc is a causal blanket satisfying T = f(Bc) for a deterministic function f, and that Oo \ Os forms a causal blanket over each variable of Os. The CausalWorld graph cannot support these hypotheses: its edges carry textual descriptions, Figures 4 and 15 state explicitly that 'the strength and function related to the causal relationships are not shown,' and the relationship syntax in Appendix J.1.2 lists 'function' as optional. Section 5.1 only assumes that causal blankets can be constructed from the CausalWorld graph, and Section 7 concedes that full knowledge of the true causal graph cannot be guaranteed. Since neither the deterministic functional form nor the completeness of the graph is established for the constructed queries, the abstract's and conclusion's claim that the method 'provably' estimates real-world counterfactuals is unsupported.
[Section 5.2, evaluation validity] The ground truth for CausalWorld-CR is built by the same extraction and matching pipeline that produces the CausalWorld graph: observation ground truth comes from the source documents from which the worlds were extracted, and counterfactual ground truth is obtained by K-matching within that same graph. Accuracy on this dataset therefore measures whether the reasoning agent reproduces the conclusions implicit in the extraction process, not whether the answers are correct counterfactuals against an external standard. No confidence intervals or significance tests are reported for the comparisons in Figure 7, and the LLaMA-3.1-8B CausalCoT arm is largely missing due to timeouts, so the claim that CTG-Reason achieves 'competitive performance' while reducing cost is only weakly supported.
[Section 5.1, formal statement of K-Matching] Definition 2 states that Bc can be K-matched with Oo over T if one can 'build a new causal blanket for T with K interventions from Bc \ Os and N-K observations from Os.' The paper does not define what 'build a new causal blanket' means formally, how the interventions are assigned values, or why the resulting object is guaranteed to satisfy Definition 1. Since Theorem 1 depends exactly on this construction, the missing formalization is load-bearing rather than stylistic.

minor comments (6)

[Figure 7 caption] The caption says results are shown for 'GPT-4o' while the text and experiments use GPT-4.1; this should be corrected.
[Appendix I, Figure 23] Both panels of Figure 23 are labeled 'Observation set'; the second panel appears to show the counterfactual set and should be relabeled.
[Appendix C] In the proof, the sentence 'Bc = (Bc \ Os) ∪ Bc forms a causal blanket over T' contains a typo; the second term on the right should presumably be Os.
[Section 3.1] The name 'NxGaphRAG' appears once in Section 3.1 and should be 'NxGraphRAG'.
[Section 5.3] The phrase 'An complete overview' should read 'A complete overview'.
[Section 5.4] The phrase 'automatically scrapped real-world events' should read 'automatically scraped real-world events.'

Circularity Check

1 steps flagged · score 6.0 of 10

Evaluation of 'real-world counterfactuals' is circular: ground-truth labels are produced by the same CTG-Extract pipeline that builds the CausalWorld graph and blankets used in the queries.

fitted input called prediction [Section 5.2 (CausalWorld-CR) and Appendix B]
"Since the query is built from a single world, the ground-truth is provided by the source document from which the world is built. ... Then, ground-truth counterfactuals are sampled from the CausalWorld graph using matching: observations in the counterfactual world that match those in a factual world are removed and replaced with the factual world for abduction."

The counterfactual query P(T|do(Bc \ Os), Oo \ Os) is constructed from CausalWorld, and its ground-truth label is the value of T in the matched counterfactual world of the same CausalWorld graph. CausalWorld's variables, values, worlds, edges, and the causal blankets Bc are all produced by the CTG-Extract LLM pipeline (Section 3). Thus the 'real-world counterfactual' being predicted is defined by the same extraction process that defines the query's causal structure; the evaluation measures agreement between the reasoning agent and the extraction agent, not correspondence to any independent real-world counterfactual. The abstract's claim that the method 'allows us to construct a large network of real-world causal relationships ...

full rationale

The central derivation chain is: Theorem 1 -> K-Matching -> CausalWorld-CR queries -> claim of provable real-world counterfactual estimation. Theorem 1 is a conditional statement whose hypotheses (existence of deterministic causal blankets, with Oo \ Os blanketing each Os variable) are assumed rather than verified: Section 5.1 states 'We assume that we can construct causal blankets from the CausalWorld graph', and Section 7 concedes that full knowledge of the true causal graph cannot be guaranteed. That is a correctness/verification gap, not by itself circularity. The circularity is in the empirical claim: the counterfactual ground truth is sampled from CausalWorld (Appendix B), and CausalWorld is built by the same CTG-Extract pipeline that defines the variables, worlds, edges, and blankets used to construct each query (Sections 3 and 5.2). Therefore the evaluation can only demonstrate that CTG-Reason is consistent with CTG-Extract; it cannot demonstrate that real-world counterfactuals are estimated. The self-citation [11] for the reasoning-agent methodology is present but is not the load-bearing circular step. No external benchmark is used, so the 'real-world counterfactual' claim reduces to internal consistency of the extraction pipeline; score 6 reflects this partial but central circularity.

Assumptions & free parameters 4 free parameters · 5 assumptions · 1 invented entities

The central claim rests on strong assumptions: the extracted graph is complete and correct, causal blankets are deterministic and identifiable, Theorem 1's blanket condition on Oo \ Os holds, and the LLM extraction is accurate. The hand-chosen hyperparameters (retrieval K, traversal depth P, dataset size, maximum path count) shape the graph and the evaluation set. No new physical entities are postulated; the causal blanket is the main invented formal construct.

free parameters (4)

K (top retrieved nodes in NxGraphRAG) = 3
Chosen by hand in Appendix D to balance exhaustiveness and efficiency; affects which causal variables are matched during extraction and hence the graph structure and queries.
P (neighborhood traversal depth) = 2
Chosen by hand in Appendix D; controls how many neighboring nodes are added to the retrieval context and can influence variable merging and edge creation.
CausalWorld-CR dataset size = 400 samples
Balanced to 400 samples in Section 5.2 and Appendix G to mitigate degree imbalance; the size and balancing affect all reported accuracy and efficiency numbers.
Maximum causal paths per query = 50 paths
Queries with 50 or more causal paths are removed in Appendix G to keep problems tractable; this exclusion affects the composition of the evaluation set.

assumptions (5)

domain assumption The SCM framework with Markovian exogenous variables and deterministic endogenous functions applies to news-derived variables.
Invoked throughout Section 2 and used to write Equation 1; news text does not come with an SCM realization.
ad hoc to paper CausalWorld is a complete and correct causal graph, so causal blankets can be constructed from ancestor sets.
Section 5.1 says 'We assume that we can construct causal blankets from the CausalWorld graph'; Section 7 concedes real-world full-graph knowledge is not guaranteed.
ad hoc to paper Oo \ Os forms a causal blanket over each variable of Os (Theorem 1).
This condition is assumed in the statement of Theorem 1 and is not verified during query construction in Section 5.2; it is essential for the equality.
domain assumption LLM-extracted causal relationships and variable types/values are accurate and grounded in the source text.
Used throughout Sections 3 and 4; the paper provides no human precision/recall evaluation of extraction, and Section 7 acknowledges reliance on source accuracy.
ad hoc to paper The causal blanket is deterministic: T = f(B) for some function f.
Definition 1 requires a deterministic function; this is a strong assumption not established for real-world events, where effects are typically stochastic.

invented entities (1)

Causal blanket
purpose: A set of ancestors that fully determines a target variable; used to justify K-Matching equivalence.
Introduced in Definition 1. It is a formal construct with no falsifiable handle outside the paper; its identifiability is assumed rather than demonstrated.

how reviews work

0 comments

Cite this review

Pith. "Pith review of Causal Cartographer: From Mapping to Reasoning Over Counterfactual Worlds." pith.science (2026). https://pith.science/paper/MFKLBMI3

@misc{pith2026250514396,
  author       = {Pith},
  title        = {Pith review of: Causal Cartographer: From Mapping to Reasoning Over Counterfactual Worlds},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/MFKLBMI3}},
  note         = {Machine review of arXiv:2505.14396}
}

read the original abstract

Causal world models are systems that can answer counterfactual questions about an environment of interest, i.e. predict how it would have evolved if an arbitrary subset of events had been realized differently. It requires understanding the underlying causes behind chains of events and conducting causal inference for arbitrary unseen distributions. So far, this task eludes foundation models, notably large language models (LLMs), which do not have demonstrated causal reasoning capabilities beyond the memorization of existing causal relationships. Furthermore, evaluating counterfactuals in real-world applications is challenging since only the factual world is observed, limiting evaluation to synthetic datasets. We address these problems by explicitly extracting and modeling causal relationships and propose the Causal Cartographer framework. First, we introduce a graph retrieval-augmented generation agent tasked to retrieve causal relationships from data. This approach allows us to construct a large network of real-world causal relationships that can serve as a repository of causal knowledge and build real-world counterfactuals. In addition, we create a counterfactual reasoning agent constrained by causal relationships to perform reliable step-by-step causal inference. We show that our approach can extract causal knowledge and improve the robustness of LLMs for causal reasoning tasks while reducing inference costs and spurious correlations.

Figures

Figures reproduced from arXiv: 2505.14396 by the authors.

**Figure 1.** Counterfactual twin graph for three endogenous variables {X, Y ,Z} and two exogenous variables {U, V }, under an intervention over X. A counterfactual world is identical to the factual world, except for the intervention and its effects. Exogenous variables are shared by both worlds, connecting the two graphs. X is being intervened upon and so has no incoming edges. The counterfactual value of Y given by P(Y | do(X… view at source ↗

**Figure 2.** Overview of the Causal Cartographer Extraction agent (CTG-Extract). (a) Illustration of the extraction pipeline. The title, content, and metadata from a news source are provided to the agent context. It then generates causal variables and their relationships and adds them to the causal world model. A graphRAG system further compares article data and agent node suggestions and adds to the agent context the nodes and … view at source ↗

**Figure 3.** CausalWorld graph structure. Colors show the cluster in which the node belongs. Node [PITH_FULL_IMAGE:figures/full_fig_p005_3.png] view at source ↗

Figures from the paper (22 more)

**Figure 4.** Figure 4: Illustration of a direct causal path in the CausalWorld graph. Nodes are [PITH_FULL_IMAGE:figures/full_fig_p006_4.png]

**Figure 5.** Figure 5: Illustration of a cycle in the CausalWorld graph. The legend is the same as in Figure 4. [PITH_FULL_IMAGE:figures/full_fig_p006_5.png]

**Figure 6.** Figure 6: Overview of the Causal Cartographer Reasoning agent (CTG-Reason) and the evaluation pipeline. (a) A query is sampled from the CausalWorld graph using counterfactual matching: values in the counterfactual world that match those in a factual world are abducted and replac…

**Figure 7.** Figure 7: (a) Results on the boolean and trend subsets of CausalWorld-CR. Results are shown [PITH_FULL_IMAGE:figures/full_fig_p009_7.png]

**Figure 8.** Figure 8: Statistics on the model answers with CTG-Reason and CausalCoT. (left) The average [PITH_FULL_IMAGE:figures/full_fig_p009_8.png]

**Figure 9.** Figure 9: Overview of the complete Causal Cartographer pipeline with extraction and reasoning [PITH_FULL_IMAGE:figures/full_fig_p014_9.png]

**Figure 10.** Figure 10: Overview of the NxGraphRAG pipeline. The attributes of the graph database are embedded [PITH_FULL_IMAGE:figures/full_fig_p016_10.png]

**Figure 11.** Figure 11: Distribution of weakly and strongly connected components and node degrees in Causal [PITH_FULL_IMAGE:figures/full_fig_p017_11.png]

**Figure 12.** Figure 12: Distribution of cycles in CausalWorld. There are 149 cycles in the graph, with lengths [PITH_FULL_IMAGE:figures/full_fig_p017_12.png]

**Figure 13.** Figure 13: CausalWorld graph structure divided by Louvain communities. Colors correspond to the [PITH_FULL_IMAGE:figures/full_fig_p018_13.png]

**Figure 14.** Figure 14: Distribution of the nodes among the worlds extracted from the source documents. The [PITH_FULL_IMAGE:figures/full_fig_p019_14.png]

**Figure 15.** Figure 15: Illustration of the longest direct causal path in the CausalWorld graph. It contains 18 nodes, [PITH_FULL_IMAGE:figures/full_fig_p020_15.png]

**Figure 16.** Figure 16: Additional direct causal paths in CausalWorld graph. [PITH_FULL_IMAGE:figures/full_fig_p021_16.png]

**Figure 17.** Figure 17: Distribution of the query types and target domains in the CausalWorld-CR dataset. [PITH_FULL_IMAGE:figures/full_fig_p021_17.png]

**Figure 18.** Figure 18: Distribution of the number of nodes in the query graphs of the CausalWorld-CR dataset. [PITH_FULL_IMAGE:figures/full_fig_p021_18.png]

**Figure 19.** Figure 19: Distribution of query types and answer types from CTG-Reason. Types correspond to [PITH_FULL_IMAGE:figures/full_fig_p022_19.png]

**Figure 20.** Figure 20: Results on the counterfactual set for the boolean and trend queries of CausalWorld-CR. Results are shown for o3-mini, GPT-4o and LLaMA-3.1-8B, using CTG-Reason and CausalCoT. (*) The majority of queries with LLaMA-3.1-8B-CausalCoT returned with a timeout. 22 [PITH_FU…

**Figure 21.** Figure 21: Results on the observational set for the boolean and trend queries of CausalWorld-CR. Results are shown for o3-mini, GPT-4o and LLaMA-3.1-8B, using CTG-Reason and CausalCoT. (*) The majority of queries with LLaMA-3.1-8B-CausalCoT returned with a timeout. −0.5 0 0.5 1 …

**Figure 22.** Figure 22: Violin plots of the semantic (cosine) similarity between ground truth and model answers. [PITH_FULL_IMAGE:figures/full_fig_p023_22.png]

**Figure 23.** Figure 23: Violin plots of the relative error numerical between ground truth and prediction. The [PITH_FULL_IMAGE:figures/full_fig_p024_23.png]

**Figure 24.** Figure 24: Confusion matrices for boolean and trend queries of CausalWorld-CR for o3-mini using [PITH_FULL_IMAGE:figures/full_fig_p024_24.png]

**Figure 25.** Figure 25: Violin plots of the BLEU score for o3-mini and GPT-4.1 using CTG-Reason on coun [PITH_FULL_IMAGE:figures/full_fig_p024_25.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

66 extracted references · 42 canonical work pages

[1]

Correa, Duligur Ibeling, and Thomas Icard

Elias Bareinboim, Juan D. Correa, Duligur Ibeling, and Thomas Icard. On pearl’s hierarchy and the foundations of causal inference. In Hector Geffner, Rina Dechter, and Joseph Y . Halpern (eds.),Probabilistic and Causal Inference: The Works of Judea Pearl, volume 36 ofACM Books, pp. 507–556. ACM, 2022. doi: 10.1145/3501714.3501743. URL https: //doi.org/10....

arXiv 2022
[2]

a is b" fail to learn

Lukas Berglund, Meg Tong, Maximilian Kaufmann, Mikita Balesni, Asa Cooper Stickland, Tomasz Korbak, and Owain Evans. The reversal curse: Llms trained on "a is b" fail to learn "b is a". InThe Twelfth International Conference on Learning Representations, ICLR 2024, Vienna, Austria, May 7-11, 2024. OpenReview.net, 2024. URL https://openreview.net/ forum?id=...

2024
[3]

Fast unfolding of communities in large networks.Journal of statistical mechanics: theory and experiment, 2008(10):P10008, 2008

Vincent D Blondel, Jean-Loup Guillaume, Renaud Lambiotte, and Etienne Lefebvre. Fast unfolding of communities in large networks.Journal of statistical mechanics: theory and experiment, 2008(10):P10008, 2008

work page 2008
[4]

Markus J. Buehler. Agentic deep graph reasoning yields self-organizing knowledge networks. CoRR, abs/2502.13025, 2025. doi: 10.48550/ARXIV .2502.13025. URL https://doi.org/ 10.48550/arXiv.2502.13025

arXiv doi:10.48550/arxiv.2502.13025 2025
[5]

Sirui Chen, Mengying Xu, Kun Wang, Xingyu Zeng, Rui Zhao, Shengjie Zhao, and Chaochao Lu. CLEAR: can language models really understand causal graphs? In Yaser Al-Onaizan, Mohit Bansal, and Yun-Nung Chen (eds.),Findings of the Association for Computational Linguistics: EMNLP 2024, Miami, Florida, USA, November 12-16, 2024, pp. 6247–6265. Association for Co...

work page 2024
[6]

BERT: pre-training of deep bidirectional transformers for language understanding

Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. BERT: pre-training of deep bidirectional transformers for language understanding. In Jill Burstein, Christy Doran, and Thamar Solorio (eds.),Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, NAACL-...

work page 2019
[7]

From local to global: A graph RAG approach to query-focused summarization.CoRR, abs/2404.16130, 2024

Darren Edge, Ha Trinh, Newman Cheng, Joshua Bradley, Alex Chao, Apurva Mody, Steven Truitt, and Jonathan Larson. From local to global: A graph RAG approach to query-focused summarization.CoRR, abs/2404.16130, 2024. doi: 10.48550/ARXIV .2404.16130. URL https://doi.org/10.48550/arXiv.2404.16130

arXiv doi:10.48550/arxiv.2404.16130 2024
[8]

A survey of methods, challenges and perspectives in causality.CoRR, abs/2302.00293, 2023

Gaël Gendron, Michael Witbrock, and Gillian Dobbie. A survey of methods, challenges and perspectives in causality.CoRR, abs/2302.00293, 2023. doi: 10.48550/ARXIV .2302.00293. URLhttps://doi.org/10.48550/arXiv.2302.00293

arXiv doi:10.48550/arxiv.2302.00293 2023

Show all 66 references

[9]

Large language models are not strong abstract reasoners

Gaël Gendron, Qiming Bao, Michael Witbrock, and Gillian Dobbie. Large language models are not strong abstract reasoners. InProceedings of the Thirty-Third International Joint Conference on Artificial Intelligence, IJCAI 2024, Jeju, South Korea, August 3-9, 2024, pp. 6270–6278....

2024
[10]

Witbrock, and Gillian Dobbie

Gaël Gendron, Bao Trung Nguyen, Alex Yuxuan Peng, Michael J. Witbrock, and Gillian Dobbie. Can large language models learn independent causal mechanisms? In Yaser Al- Onaizan, Mohit Bansal, and Yun-Nung Chen (eds.),Proceedings of the 2024 Conference 10 on Empirical Methods in ...

2024
[11]

Rozanec, Michael Witbrock, and Gillian Dobbie

Gaël Gendron, Joze M. Rozanec, Michael Witbrock, and Gillian Dobbie. Counterfactual causal inference in natural language with large language models.CoRR, abs/2410.06392, 2024. doi: 10.48550/ARXIV .2410.06392. URLhttps://doi.org/10.48550/arXiv.2410.06392

2024 doi arXiv
[12]

Inductive biases for deep learning of higher-level cognition

Anirudh Goyal and Yoshua Bengio. Inductive biases for deep learning of higher-level cognition. CoRR, abs/2011.15091, 2020. URLhttps://arxiv.org/abs/2011.15091

2011 arXiv
[13]

Causenet: Towards a causality graph extracted from the web

Stefan Heindorf, Yan Scholten, Henning Wachsmuth, Axel-Cyrille Ngonga Ngomo, and Martin Potthast. Causenet: Towards a causality graph extracted from the web. In Mathieu d’Aquin, Stefan Dietze, Claudia Hauff, Edward Curry, and Philippe Cudré-Mauroux (eds.),CIKM ’20: The 29th AC...

2020
[14]

Statistics and causal inference.Journal of the American statistical Association, 81(396):945–960, 1986

Paul W Holland. Statistics and causal inference.Journal of the American statistical Association, 81(396):945–960, 1986

1986
[15]

Oil price, green innovation and institutional pressure: A china’s perspective.Resources Policy, 78:102788, 2022

Jinyan Hu, Kai-Hua Wang, Chi Wei Su, and Muhammad Umar. Oil price, green innovation and institutional pressure: A china’s perspective.Resources Policy, 78:102788, 2022

2022
[16]

Cladder: A benchmark to assess causal reasoning capabilities of language models

Zhijing Jin, Yuen Chen, Felix Leeb, Luigi Gresele, Ojasv Kamal, Zhiheng Lyu, Kevin Blin, Fernando Gonzalez Adauto, Max Kleiman-Weiner, Mrinmaya Sachan, and Bernhard Schölkopf. Cladder: A benchmark to assess causal reasoning capabilities of language models. In Al- ice Oh, Trist...

2023
[17]

Diab, and Bernhard Schölkopf

Zhijing Jin, Jiarui Liu, Zhiheng Lyu, Spencer Poff, Mrinmaya Sachan, Rada Mihalcea, Mona T. Diab, and Bernhard Schölkopf. Can large language models infer causation from correlation? InThe Twelfth International Conference on Learning Representations, ICLR 2024, Vienna, Austria,...

2024
[18]

Efficient causal graph discovery using large language models.CoRR, abs/2402.01207, 2024

Thomas Jiralerspong, Xiaoyin Chen, Yash More, Vedant Shah, and Yoshua Bengio. Efficient causal graph discovery using large language models.CoRR, abs/2402.01207, 2024. doi: 10.48550/ARXIV .2402.01207. URLhttps://doi.org/10.48550/arXiv.2402.01207

2024 doi arXiv
[19]

Llms are prone to fallacies in causal inference

Nitish Joshi, Abulhair Saparov, Yixin Wang, and He He. Llms are prone to fallacies in causal inference. In Yaser Al-Onaizan, Mohit Bansal, and Yun-Nung Chen (eds.),Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing, EMNLP 2024, Miami, FL, US...

2024
[20]

Ramnani, Mayuresh Anand, Shubhashis Sengupta, and Andrew E

Vivek Khetan, Roshni R. Ramnani, Mayuresh Anand, Shubhashis Sengupta, and Andrew E. Fano. Causal BERT: language models for causality detection between events expressed in text. In Kohei Arai (ed.),Intelligent Computing - Proceedings of the 2021 Computing Conference, Volume 1, ...

2021 doi
[21]

Event registry: learning about world events from news

Gregor Leban, Blaz Fortuna, Janez Brank, and Marko Grobelnik. Event registry: learning about world events from news. In Chin-Wan Chung, Andrei Z. Broder, Kyuseok Shim, and Torsten Suel (eds.),23rd International World Wide Web Conference, WWW ’14, Seoul, Republic of Korea, Apri...

2014
[22]

Retrieval-augmented generation for knowledge-intensive NLP tasks

Patrick Lewis, Ethan Perez, Aleksandra Piktus, Fabio Petroni, Vladimir Karpukhin, Na- man Goyal, Heinrich Küttler, Mike Lewis, Wen-tau Yih, Tim Rocktäschel, Sebastian Riedel, and Douwe Kiela. Retrieval-augmented generation for knowledge-intensive NLP tasks. In Hugo Larochelle,...

2020
[23]

DALK: dynamic co-augmentation of llms and KG to answer alzheimer’s disease questions with scientific literature

Dawei Li, Shu Yang, Zhen Tan, Jae Young Baik, Sukwon Yun, Joseph Lee, Aaron Chacko, Bojian Hou, Duy Duong-Tran, Ying Ding, Huan Liu, Li Shen, and Tianlong Chen. DALK: dynamic co-augmentation of llms and KG to answer alzheimer’s disease questions with scientific literature. In ...

2024
[24]

Causal transportability for visual recognition

Chengzhi Mao, Kevin Xia, James Wang, Hao Wang, Junfeng Yang, Elias Bareinboim, and Carl V ondrick. Causal transportability for visual recognition. InIEEE/CVF Conference on Computer Vision and Pattern Recognition, CVPR 2022, New Orleans, LA, USA, June 18-24, 2022, pp. 7511–7521...

2022
[25]

Introducing llama 3.1: Our most capable models to date, 2024

Meta. Introducing llama 3.1: Our most capable models to date, 2024. URL https://ai.meta. com/blog/meta-llama-3-1/. Accessed: 2025-05-15

2024
[26]

Gsm-symbolic: Understanding the limitations of mathematical reasoning in large language models.arXiv preprint arXiv:2410.05229, 2024

Iman Mirzadeh, Keivan Alizadeh, Hooman Shahrokhi, Oncel Tuzel, Samy Bengio, and Mehrdad Farajtabar. Gsm-symbolic: Understanding the limitations of mathematical reasoning in large language models.arXiv preprint arXiv:2410.05229, 2024

arXiv
[27]

Text and code embeddings by contrastive pre-training

Arvind Neelakantan, Tao Xu, Raul Puri, Alec Radford, Jesse Michael Han, Jerry Tworek, Qiming Yuan, Nikolas Tezak, Jong Wook Kim, Chris Hallacy, Johannes Heidecke, Pranav Shyam, Boris Power, Tyna Eloundou Nekoul, Girish Sastry, Gretchen Krueger, David Schnurr, Felipe Petroski S...

2022 arXiv
[28]

Openai o3-mini, 2025

OpenAI. Openai o3-mini, 2025. URL https://openai.com/index/openai-o3-mini/ . Accessed: 2025-05-05

2025
[29]

Introducing gpt-4.1 in the api - openai, 2025

OpenAI. Introducing gpt-4.1 in the api - openai, 2025. URL https://openai.com/index/ gpt-4-1/. Accessed: 2025-05-13

2025
[30]

Bleu: a method for automatic evaluation of machine translation

Kishore Papineni, Salim Roukos, Todd Ward, and Wei-Jing Zhu. Bleu: a method for automatic evaluation of machine translation. InProceedings of the 40th Annual Meeting of the Association for Computational Linguistics, July 6-12, 2002, Philadelphia, PA, USA, pp. 311–318. ACL,

2002
[31]

Cambridge university press, 2009

Judea Pearl.Causality. Cambridge university press, 2009

2009
[32]

The do-calculus revisited

Judea Pearl. The do-calculus revisited. In Nando de Freitas and Kevin P. Mur- phy (eds.),Proceedings of the Twenty-Eighth Conference on Uncertainty in Artificial Intelligence, Catalina Island, CA, USA, August 14-18, 2012, pp. 3–11. AUAI Press,

2012
[33]

The deductive approach to causal inference.Journal of Causal Inference, 2(2): 115–129, 2014

Judea Pearl. The deductive approach to causal inference.Journal of Causal Inference, 2(2): 115–129, 2014

2014
[34]

Elsevier, 2014

Judea Pearl.Probabilistic reasoning in intelligent systems: networks of plausible inference. Elsevier, 2014

2014
[35]

Graph retrieval-augmented generation: A survey.CoRR, abs/2408.08921,

Boci Peng, Yun Zhu, Yongchao Liu, Xiaohe Bo, Haizhou Shi, Chuntao Hong, Yan Zhang, and Siliang Tang. Graph retrieval-augmented generation: A survey.CoRR, abs/2408.08921,

arXiv
[36]

Robust agents learn causal world models

Jonathan Richens and Tom Everitt. Robust agents learn causal world models. InThe Twelfth International Conference on Learning Representations, ICLR 2024, Vienna, Austria, May 7-11,

2024
[37]

Silhouettes: a graphical aid to the interpretation and validation of cluster analysis.Journal of computational and applied mathematics, 20:53–65, 1987

Peter J Rousseeuw. Silhouettes: a graphical aid to the interpretation and validation of cluster analysis.Journal of computational and applied mathematics, 20:53–65, 1987. 12

1987
[38]

Building a causality graph for strategic foresight

Jože M Rožanec, Beno Šircelj, Peter Nemec, Gregor Leban, and Dunja Mladeni´c. Building a causality graph for strategic foresight. 2023

2023
[39]

Estimating causal effects of treatments in randomized and nonrandomized studies.Journal of educational Psychology, 66(5):688, 1974

Donald B Rubin. Estimating causal effects of treatments in randomized and nonrandomized studies.Journal of educational Psychology, 66(5):688, 1974

1974
[40]

URL https://doi.org/10.48550/arXiv.2408

doi: 10.48550/ARXIV .2408.08921. URL https://doi.org/10.48550/arXiv.2408. 08921

doi arXiv
[41]

sentence-transformers/all-mpnet-base-v2 · hugging face, 2024

Sentence Transformers. sentence-transformers/all-mpnet-base-v2 · hugging face, 2024. URL https://huggingface.co/sentence-transformers/all-mpnet-base-v2 . Accessed: 2025-05-05

2024
[42]

URLhttps://openreview.net/forum?id=pOoKI3ouv1

OpenReview.net, 2024. URLhttps://openreview.net/forum?id=pOoKI3ouv1

2024
[43]

Chi, Quoc V

Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Brian Ichter, Fei Xia, Ed H. Chi, Quoc V . Le, and Denny Zhou. Chain-of-thought prompting elicits reasoning in large language models. In Sanmi Koyejo, S. Mohamed, A. Agarwal, Danielle Belgrave, K. Cho, and A. Oh (eds.),Ad...

2022
[44]

Reasoning or reciting? exploring the capabilities and limitations of language models through counterfactual tasks

Zhaofeng Wu, Linlu Qiu, Alexis Ross, Ekin Akyürek, Boyuan Chen, Bailin Wang, Najoung Kim, Jacob Andreas, and Yoon Kim. Reasoning or reciting? exploring the capabilities and limitations of language models through counterfactual tasks. In Kevin Duh, Helena Gómez- Adorno, and Ste...

2024
[45]

The causal-neural con- nection: Expressiveness, learnability, and inference

Kevin Xia, Kai-Zhan Lee, Yoshua Bengio, and Elias Bareinboim. The causal-neural con- nection: Expressiveness, learnability, and inference. In Marc’Aurelio Ranzato, Alina Beygelzimer, Yann N. Dauphin, Percy Liang, and Jennifer Wortman Vaughan (eds.), Advances in Neural Informat...

2021
[46]

Toward causal representation learning.Proc

Bernhard Schölkopf, Francesco Locatello, Stefan Bauer, Nan Rosemary Ke, Nal Kalchbrenner, Anirudh Goyal, and Yoshua Bengio. Toward causal representation learning.Proc. IEEE, 109 (5):612–634, 2021. doi: 10.1109/JPROC.2021.3058954. URL https://doi.org/10.1109/ JPROC.2021.3058954

2021
[47]

Narasimhan, and Yuan Cao

Shunyu Yao, Jeffrey Zhao, Dian Yu, Nan Du, Izhak Shafran, Karthik R. Narasimhan, and Yuan Cao. React: Synergizing reasoning and acting in language models. InThe Eleventh International Conference on Learning Representations, ICLR 2023, Kigali, Rwanda, May 1-5,

2023
[48]

Renewable energy, oil prices, and economic activity: A granger-causality in quantiles analysis.Energy Economics, 70:440–452, 2018

Victor Troster, Muhammad Shahbaz, and Gazi Salah Uddin. Renewable energy, oil prices, and economic activity: A granger-causality in quantiles analysis.Energy Economics, 70:440–452, 2018

2018
[52]

A survey on extraction of causal rela- tions from natural language text.Knowl

Jie Yang, Soyeon Caren Han, and Josiah Poon. A survey on extraction of causal rela- tions from natural language text.Knowl. Inf. Syst., 64(5):1161–1186, 2022. doi: 10.1007/ S10115-022-01665-W. URLhttps://doi.org/10.1007/s10115-022-01665-w

2022 doi
[55]

Causal parrots: Large language models may talk causality but are not causal.Trans

Matej Zecevic, Moritz Willig, Devendra Singh Dhami, and Kristian Kersting. Causal parrots: Large language models may talk causality but are not causal.Trans. Mach. Learn. Res., 2023,

2023
[56]

name": <string>, # The name of the variable

URLhttps://openreview.net/forum?id=tv46tCzs83. 13 LLM Agent Sampling Evaluation Response Factual World Query Matched Counterfactual Ground-Truth Newspapers Extract Extract of a news article Extraction Graph-RAG LLM Agent Causal World Model Figure 9: Overview of the complete Ca...
[57]

Use the variables provided when possible or create new ones when no variable matches

Define the causal variables observed in the text. Use the variables provided when possible or create new ones when no variable matches
[58]

Define the confounders that are not observed in the text or for which a value is not given, and that affect one or several of the causal variables defined in step 1
[59]

Use the ‘{{ retrieval_tool_name}}‘ tool to assess if the variable is already in the database

Verify if the new variables have correspondance in the causal graph database. Use the ‘{{ retrieval_tool_name}}‘ tool to assess if the variable is already in the database. If it is, use it instead of creating a new one. It may have a different name in the database, the tool re...
[60]

Do not create causal relationships that already exist in the causal graph

Define the causal relationships between the variables, based on the text and common sense knowledge. Do not create causal relationships that already exist in the causal graph
[61]

Each step should be a separate ’Thought:’, ’Code:’, and ’Observation:’ sequence

Build the full causal graph as a networkx DiGraph object. Each step should be a separate ’Thought:’, ’Code:’, and ’Observation:’ sequence. The code MUST be executed in two code blocks minimum. After step 3, use <end_code> to indicate the end of the code block and retrieve the ...
[62]

Air Pollution - representing high levels of air pollution in cities
[63]

Temperature - higher temperatures during the summer months
[64]

Industrial Pollution - arising from intensified industrial activities during hotter months
[65]

Respiratory Issues - the rise in respiratory issues among the population
[66]

Increased air pollution leads to higher respiratory issues

Pre-existing Respiratory Conditions - representing vulnerable individuals (e.g., those with asthma) at greater risk. I will verify if these variables exist in our causal graph database using the tool graph_retriever for each variable by calling it with a query that includes th...

2009
[2002]

URLhttps://aclanthology.org/P02-1040/

doi: 10.3115/1073083.1073135. URLhttps://aclanthology.org/P02-1040/
[2012]

URL https://dslpitt.org/uai/displayArticleDetails.jsp?mmnu=1&smnu= 2&article_id=2330&proceeding_id=28
[2023]

URLhttps://openreview.net/forum?id=WE_vluYUL-X

OpenReview.net, 2023. URLhttps://openreview.net/forum?id=WE_vluYUL-X

2023
[2024]

URLhttps://aclanthology.org/2024.emnlp-main.590

2024
[4186]

doi: 10.18653/V1/N19-1423

Association for Computational Linguistics, 2019. doi: 10.18653/V1/N19-1423. URL https://doi.org/10.18653/v1/n19-1423

2019 doi

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.

[1] [1]

Correa, Duligur Ibeling, and Thomas Icard

Elias Bareinboim, Juan D. Correa, Duligur Ibeling, and Thomas Icard. On pearl’s hierarchy and the foundations of causal inference. In Hector Geffner, Rina Dechter, and Joseph Y . Halpern (eds.),Probabilistic and Causal Inference: The Works of Judea Pearl, volume 36 ofACM Books, pp. 507–556. ACM, 2022. doi: 10.1145/3501714.3501743. URL https: //doi.org/10....

arXiv 2022

[2] [2]

a is b" fail to learn

Lukas Berglund, Meg Tong, Maximilian Kaufmann, Mikita Balesni, Asa Cooper Stickland, Tomasz Korbak, and Owain Evans. The reversal curse: Llms trained on "a is b" fail to learn "b is a". InThe Twelfth International Conference on Learning Representations, ICLR 2024, Vienna, Austria, May 7-11, 2024. OpenReview.net, 2024. URL https://openreview.net/ forum?id=...

2024

[3] [3]

Fast unfolding of communities in large networks.Journal of statistical mechanics: theory and experiment, 2008(10):P10008, 2008

Vincent D Blondel, Jean-Loup Guillaume, Renaud Lambiotte, and Etienne Lefebvre. Fast unfolding of communities in large networks.Journal of statistical mechanics: theory and experiment, 2008(10):P10008, 2008

work page 2008

[4] [4]

Markus J. Buehler. Agentic deep graph reasoning yields self-organizing knowledge networks. CoRR, abs/2502.13025, 2025. doi: 10.48550/ARXIV .2502.13025. URL https://doi.org/ 10.48550/arXiv.2502.13025

arXiv doi:10.48550/arxiv.2502.13025 2025

[5] [5]

Sirui Chen, Mengying Xu, Kun Wang, Xingyu Zeng, Rui Zhao, Shengjie Zhao, and Chaochao Lu. CLEAR: can language models really understand causal graphs? In Yaser Al-Onaizan, Mohit Bansal, and Yun-Nung Chen (eds.),Findings of the Association for Computational Linguistics: EMNLP 2024, Miami, Florida, USA, November 12-16, 2024, pp. 6247–6265. Association for Co...

work page 2024

[6] [6]

BERT: pre-training of deep bidirectional transformers for language understanding

Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. BERT: pre-training of deep bidirectional transformers for language understanding. In Jill Burstein, Christy Doran, and Thamar Solorio (eds.),Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, NAACL-...

work page 2019

[7] [7]

From local to global: A graph RAG approach to query-focused summarization.CoRR, abs/2404.16130, 2024

Darren Edge, Ha Trinh, Newman Cheng, Joshua Bradley, Alex Chao, Apurva Mody, Steven Truitt, and Jonathan Larson. From local to global: A graph RAG approach to query-focused summarization.CoRR, abs/2404.16130, 2024. doi: 10.48550/ARXIV .2404.16130. URL https://doi.org/10.48550/arXiv.2404.16130

arXiv doi:10.48550/arxiv.2404.16130 2024

[8] [8]

A survey of methods, challenges and perspectives in causality.CoRR, abs/2302.00293, 2023

Gaël Gendron, Michael Witbrock, and Gillian Dobbie. A survey of methods, challenges and perspectives in causality.CoRR, abs/2302.00293, 2023. doi: 10.48550/ARXIV .2302.00293. URLhttps://doi.org/10.48550/arXiv.2302.00293

arXiv doi:10.48550/arxiv.2302.00293 2023