Commercial LLMs agree only modestly with expert confounder lists for the Coronary Drug Project and flip answers under trivial prompt changes, so they cannot yet act as reliable causal knowledge repositories.
Mining Causality: AI-Assisted Search for Instrumental Variables
1 Pith paper cite this work. Polarity classification is still indexing.
abstract
The instrumental variables (IVs) method is a leading empirical strategy for causal inference. Finding IVs is a heuristic and creative process, and justifying its validity -- especially exclusion restrictions -- is largely rhetorical. We propose using large language models (LLMs) to search for new IVs through narratives and counterfactual reasoning, similar to how a human researcher would. The stark difference, however, is that LLMs can dramatically accelerate this process and explore an extremely large search space. We demonstrate how to construct prompts to search for potentially valid IVs. We contend that multi-step and role-playing prompting strategies are effective for simulating the endogenous decision-making processes of economic agents and for navigating language models through the realm of real-world scenarios, rather than anchoring them within the narrow realm of academic discourses on IVs. We apply our method to three well-known examples in economics: returns to schooling, supply and demand, and peer effects. We then extend our strategy to finding (i) control variables in regression and difference-in-differences and (ii) running variables in regression discontinuity designs.
fields
econ.EM 1years
2024 1verdicts
CONDITIONAL 1representative citing papers
citing papers explorer
-
Do LLMs Act as Repositories of Causal Knowledge?
Commercial LLMs agree only modestly with expert confounder lists for the Coronary Drug Project and flip answers under trivial prompt changes, so they cannot yet act as reliable causal knowledge repositories.