Pith. sign in

REVIEW 4 major objections 5 minor 49 references

Eliciting Causal Abilities in Large Language Models for Reasoning Tasks

T0 review · 4 major / 5 minor · reviewed 2026-08-11 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read The paper claims that prompt wording can be treated as a causal treatment, and that LLM-generated observational data plus average treatment effect estimation yields rewritten instructions that improve reasoning accuracy at lower training…

desk verdict The causal inference story doesn't hold up, but the assembled prompt-rewriting pipeline is new, empirical, and worth a careful look as a heuristic. read the letter →

arxiv 2412.15314 v1 pith:3NTT7TYV submitted 2024-12-19 cs.CL cs.AI

Yajing Wang , Zongwei Luo , Jingzhe Wang , Zhanke Zhou , Yongqiang Chen , Bo Han This is my paper · ORCID
classification cs.CLcs.AI
keywords promptoptimizationcausalinferencelargelanguagemodelschain-of-thoughtaveragetreatmenteffectmeta-promptingreasoningtasksinstructionenhancement
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper proposes treating the wording of a prompt instruction as a treatment in a causal study, and using the LLM itself to supply the observational data. Its Self-Causal Instruction Enhancement (SCIE) method has the LLM generate many instruction variants, score them on named textual features such as clarity and directness, estimate the average effect of each feature on answer correctness, and then write an enhanced instruction that amplifies features with positive effects. The paper reports that this rewritten instruction improves accuracy over the base 'Let's think step by step' prompt on nine of ten reasoning tasks, and beats the LLM-as-optimizer baseline OPRO on GSM8K (78.6 versus 77.7), without gradient training and with a plain-language explanation. A companion Object-Relational (OR) module treats the learned causal pattern as an inheritable class, so related tasks can reuse it at low cost.

What carries the argument

The central mechanism is the proxy feature: a named, interpretable textual property of an instruction (e.g., 'Directness', 'Clarity', 'Specification of Detail Level') that the LLM enumerates and scores. The pipeline treats each proxy feature in turn as a binary treatment; generates counterfactual instruction variants that flip that feature; Cartesian-products those variants with sampled questions; labels each instruction-question pair with answer correctness; and estimates the average treatment effect of each feature using a T-Learner, implemented by an LLM with a code interpreter via in-context learning. The LLM then generates an enhanced instruction that raises positive-ATE features and lowers negative-ATE features, and the Object-Relational (OR) module packages the discovered causal pattern as an inheritable class for other tasks.

What would settle it

Generate many instruction pairs matched on all eight proxy features but differing only in surface wording; if accuracy varies substantially within matched pairs, the proxy features do not capture the full causal path and the ATE-based rewrite has no stable target.

Watch

Extended reading notes

Core claim

The central claim is that an LLM's own reasoning can be improved by first eliciting its causal estimates of how prompt features affect answer correctness, then having it rewrite the prompt to maximize the estimated overall average treatment effect. Under the potential-outcomes framework, the instruction is the treatment, answer correctness is the outcome, and proxy textual features are the treatment variables, each binarized in turn while the others serve as covariates. The paper asserts that the generated observational data satisfy ignorability, positivity, and consistency, and that LLM-estimated ATE values align closely with T-Learner estimates. On that basis, SCIE-enhanced instructions outperform the original prompts on most of the ten evaluated reasoning tasks and outperform OPRO on GSM8K, while the OR module transfers the learned causal relationships to new tasks.

Load-bearing premise

The pipeline works only if the instruction variants generated by the LLM differ in the measured textual feature and in nothing else that also affects answer correctness, so the estimated effects are not secretly driven by length, topic leakage, or other hidden wording differences.

Editorial extensions

If this is right

  • On GPT-3.5 Turbo, SCIE raises Zero-Shot CoT accuracy on nine of the ten reasoning tasks, with GSM8K moving from 75.5 to 77.3 and StrategyQA from 65.5 to 71.1.
  • SCIE also improves more elaborate base instructions: on GSM8K it beats Plan-and-Solve prompting and AgentInstruct while keeping the same student model.
  • Against OPRO, the LLM-as-optimizer baseline, SCIE reaches 78.6 versus 77.7 on GSM8K, and it produces an interpretable explanation of the rewrite.
  • The OR inheritance step transfers a causal class learned on GSM8K to MultiArith and to high-difficulty fresh-gaokao-math-2023 problems, improving accuracy without recomputing ATE from scratch.
  • Across tasks, the LLM's ATE estimates closely match T-Learner estimates, which the paper takes as evidence that the generated observational data supports causal effect estimation.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Editorial inference: the same data-generation and ATE pipeline is outcome-agnostic, so it could optimize prompts for answer confidence, response length, or refusal rates rather than only correctness.
  • Editorial inference: because the 'counterfactual instructions' are LLM paraphrases rather than controlled edits of a single feature, the ATE numbers should be read as correlations over generated wordings; a randomized swap of one feature would test whether the causal interpretation holds.
  • Editorial inference: the OR module implies a cheap transfer recipe—learn which prompt features matter on one task and reuse that class on related tasks—but the paper's evidence for transfer covers only a few dataset pairs.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. The paper proposes SCIE (Self-Causal Instruction Enhancement), a prompt-optimization method that treats LLM-generated instruction variants as observational data, extracts interpretable textual 'proxy features', estimates an Average Treatment Effect for each feature using a T-learner (implemented via an LLM with code interpreter), and then asks the LLM to compose an enhanced instruction that maximizes the estimated ATEs. An 'Object-Relational' (OR) module is added to reuse the discovered causal relationships across tasks. The authors claim improved accuracy on multiple reasoning benchmarks relative to Zero-Shot CoT, Plan-and-Solve, AgentInstruct, and OPRO, while offering interpretability and lower training cost. The evaluation covers ten datasets, three student models, and two base-instruction settings, with additional OR-reuse experiments.

Significance. If the causal interpretation were valid, the paper would offer a novel and interpretable alternative to gradient-based prompt optimization, with a plausible mechanism for transferring prompt improvements across tasks. The manuscript also has concrete strengths: it releases code, evaluates on a broad set of reasoning benchmarks, and includes an ablation study in Appendix E. However, the central causal claims are not established. The data-generation procedure does not satisfy the identification assumptions stated in Section 3.1, the theoretical justification for instruction optimization is flawed, and the empirical evaluation lacks error bars, confidence intervals, or significance tests. As a result, the paper currently reads as a heuristic instruction-rewriting method with a causal vocabulary, rather than a causally grounded optimization method. The significance of the claimed contribution is therefore not supported by the evidence as presented.

major comments (4)
  1. [3.2 (High-quality Observational Data Generation)] The paper asserts that ignorability is satisfied because the proxy features 'must be independent of each other (ignorability)'. This is incorrect: ignorability (Eq. 2) requires that all confounders of the treatment-outcome relationship are observed and controlled for, not that the treatment features are mutually independent. The 'counterfactual instructions' are LLM-generated rephrasings that vary one feature but inevitably alter many other dimensions (length, lexical specificity, topic overlap, formatting, tone), and these unmeasured dimensions plausibly affect the correctness outcome. Consequently, the conditional means in Eq. (6) are descriptive associations, not potential-outcome contrasts, and the downstream ATE-based instruction optimization and OR-reuse claims inherit this problem. This is load-bearing because the paper's stated contribution is causal prompt enhancement.
  2. [3.4 (Enhanced Instructions Generation)] The proof that ATEi > 0 justifies adjusting Ti from 0 to 1 for an individual is invalid. The law of large numbers only relates the average of individual treatment effects to the ATE; it does not imply that any given individual has a high probability of positive ITE, and the statement 'adjusting Ti from 0 to 1 will increase the value of ITE' is not a well-defined probabilistic consequence of a positive average. Even for binary outcomes, the argument conflates average with individual-level guarantees. Since this is the theoretical basis for generating instructions with optimized ATEoverall (Eq. 9), the causal optimization step is not supported.
  3. [4 (Experiments and Results), Tables 1-2, Figure 5, Appendices B and E] All reported accuracies are single numbers without error bars, confidence intervals, or significance tests. Many improvements are small (e.g., GSM8K 75.5 to 77.3, CommonsenseQA 71.7 to 72.0), and some results worsen under SCIE (Boolean Expressions on GPT-4o mini: 98.4 to 97.7; CommonsenseQA on Llama-3-70B: 88.6 to 85.4; Date Understanding decreases on multiple models). The comparison with OPRO in Table 2 rests on a 0.9-point difference (77.7 vs 78.6) with no indication of variance. Without repeated runs or statistical testing, the claim that SCIE 'effectively enhances instructions for most reasoning tasks' is not established.
  4. [4.3 (Evaluation of OR-SCIE) and Appendix E] The OR-module evaluation is anecdotal. The aggregation experiment uses a single random 70% subset of GSM8K and reports one accuracy (77.9%), which the authors themselves describe as possibly due to 'the randomly extracted test data'. The generalization experiment reports one accuracy on MultiArith and a qualitative 'several more questions correct' on a 30-problem set. Separately, Appendix E shows that removing the counterfactual module makes the ATE estimates 'not stable and robust' and degrades accuracy, which underscores how sensitive the method is to data-generation choices. The paper provides no evidence of stability across random seeds, question subsets, or instruction-generation runs, so the claimed reusability and robustness are not demonstrated.
minor comments (5)
  1. [3.1 (Preliminaries)] In Eq. (2), the expression 'T/upmodelsY (t)' is a rendering artifact and should read 'T _||_ Y(t) | X'. Also, the equality in Eq. (5) between causal estimands and observed conditional expectations silently assumes ignorability and positivity; these assumptions should be stated explicitly at that point.
  2. [3.2 (High-quality Observational Data Generation)] The description of Figure 3 is unclear about how 'probability density distributions' are obtained for ordinal or categorical proxy features after binarizing only the treatment feature. The figure caption and surrounding text should specify the estimation method and the sample size.
  3. [3.4 (Enhanced Instructions Generation)] The connection between the numerical ATE values and the LLM's instruction-generation prompt is not made precise. The paper says the LLM will 'generate instructions that optimize ATEoverall', but it does not give the exact prompt template or state whether the LLM receives the full ATE vector or only the recommended directions. Providing this material would improve reproducibility.
  4. [Appendix B] The Llama-3-70B table has a formatting error in the 'Other logical reasoning' row: the SCIE line contains six numbers instead of five, which obscures the comparison. The paper should also fix the typo 'Plan-and-Sove' in Appendix C.
  5. [Introduction] The first contribution claims this is 'the first work that enhances the prompts of LLMs for reasoning tasks from a causal perspective'. Given prior work on causal effects of linguistic properties (e.g., Pryzant et al. 2021, Feder et al. 2022) and the paper's own use of those methods, the novelty claim should be more carefully qualified.

Circularity Check

1 steps flagged · score 3.0 of 10

Main accuracy gains rest on independent held-out benchmarks, but the paper's validation that LLMs master T-Learner ATE estimation is circular because the T-Learner code was supplied as input and then compared with itself.

  1. self definitional [Estimating Causal Effect with LLMs / Evaluation of Estimating Causal Effect with LLMs]
    "we utilize the in-context learning strategy (Brown et al. 2020), providing the relevant code along with ⌈i/2⌉ ATE results as the demonstration to the LLMs and ask LLMs estimating the complete i ATE results. ... We can observe that the LLM has fully mastered the use of the T-Learner for ATE estimation, as all the results it generated, including the points with unknown values, aligning perfectly with those produced by the T-learner."

    The LLM's ATE estimates are produced by supplying the T-Learner code and some T-Learner outputs via Open Interpreter; the later 'perfect alignment' with T-Learner is therefore guaranteed by construction, not an independent confirmation of LLM causal ability. The evaluation treats execution of the provided algorithm as evidence that LLMs have mastered causal-effect estimation, so the validation claim reduces to the code that was given as input. This circularity is confined to the supporting ATE-validation experiment; the downstream instruction enhancement is separately measured on held-out accuracy.

full rationale

The central SCIE pipeline is not circular: ATE_i estimates are computed from LLM-generated instruction-question instances, the optimized instruction is produced heuristically from the estimated ATE_i, and the resulting accuracy is evaluated on held-out test questions against fixed baselines (Table 1, Table 2, Figure 5). These outcomes are externally observable and can fail (e.g., Date Understanding decreases), so they are not forced by construction. The identified circularity is limited to the paper's claim that the LLM 'has fully mastered the use of the T-Learner': because the LLM was handed the T-Learner code and example ATE values, perfect agreement with T-Learner is a tautology. Self-citations (e.g., Wang & Luo 2024 for the OR module) are not load-bearing here, as they are paired with independent references and the OR module is secondary. Non-circular validity concerns remain: the equation of ignorability with pairwise feature independence is not a sufficient condition for unconfoundedness, and the claim that ATE_i>0 raises the probability of a positive ITE is logically false; these are causal-identification and proof errors rather than circle-of-derivation errors.

Assumptions & free parameters 6 free parameters · 6 assumptions · 3 invented entities

The central claim rests on the LLM being a reliable generator of counterfactual instructions and a reliable estimator of its own treatment effects. No external benchmark constrains any of these steps; each is either asserted or chosen by hand, which is why the ledger is large relative to the contribution.

free parameters (6)
  • a (number of generated instructions) = 9 (5 for AgentInstruct)
    Chosen by hand for cost control; determines the size of the observational dataset via n(a+1).
  • b (number of sampled training questions) = 5
    Randomly selected from the training set; the small sample makes ATE estimates noisy and not statistically robust.
  • n (number of proxy features) = 8
    Automatically generated by GPT-4o mini; the list is not externally validated and varies per task.
  • proxy feature set = Directness, Clarity, Conciseness, Actionability, Tone, Structure, Specification of Detail Level, Emphasis on Process
    These textual features serve as treatments and covariates; they are elicited from the LLM, not grounded in an external taxonomy.
  • binarization threshold for treatment features = not specified
    The authors binarize a treatment for plotting but do not specify thresholds for all features; this affects ATE estimates.
  • in-context demonstration count = ceil(i/2) ATE results
    The number of ATE examples given to the LLM for estimation is selected by hand and affects estimation quality.
assumptions (6)
  • ad hoc to paper The LLM-generated observational data satisfy ignorability, positivity, and consistency.
    Section 'High-quality Observational Data Generation': authors assert proxy feature independence grounds ignorability and counterfactual generation grounds positivity; no evidence is given that all confounders are observed. Ignorability is misstated as feature independence.
  • ad hoc to paper Proxy features are independent of each other.
    Used to justify ATEoverall = (1/n) sum ATEi in Eq. 9; not tested, and interdependent textual features likely violate it.
  • ad hoc to paper A positive ATE implies a high probability of positive ITE for individual instructions.
    Used in 'Enhanced Instructions Generation' to justify adjusting treatments in the sign of ATE; mathematically invalid because one large positive effect can dominate many small negative effects.
  • ad hoc to paper Different proxy treatments affect potential outcomes independently (consistency).
    Introduced in the same section to support the adjustment strategy; this is a new assumption not present in the preliminaries.
  • domain assumption The teacher LLM (GPT-4o) can generate high-quality counterfactual instructions and reliable ATE estimates.
    The whole method relies on LLM self-report; no external validation of causal estimates is provided.
  • domain assumption Causal relationships extracted from one task can be inherited by another task via Object-Relational relationships.
    The OR module assumes causal effects transfer across tasks; only one small demonstration is given, with no theoretical or empirical validation.
invented entities (3)
  • Proxy features (Directness, Clarity, Conciseness, etc.)
    purpose: Serve as treatment variables and covariates for causal effect estimation on instructions.
    Features are generated by the LLM, not derived from an external validated taxonomy; no falsifiable handle outside the paper.
  • Causal relationship class in OR module
    purpose: A reusable meta-template of causal relationships inherited by new tasks.
    The class is extracted from LLM explanations; its transferability is asserted, not demonstrated with external benchmarks.
  • Counterfactual instructions
    purpose: Observational data points intended to satisfy positivity for causal estimation.
    These are not true counterfactuals in the Rubin sense; they are alternative phrasings generated by the LLM, with no guarantee that they represent interventions on a single feature.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Eliciting Causal Abilities in Large Language Models for Reasoning Tasks." pith.science (2026). https://pith.science/paper/3NTT7TYV

@misc{pith2026241215314,
  author       = {Pith},
  title        = {Pith review of: Eliciting Causal Abilities in Large Language Models for Reasoning Tasks},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/3NTT7TYV}},
  note         = {Machine review of arXiv:2412.15314}
}
read the original abstract

Prompt optimization automatically refines prompting expressions, unlocking the full potential of LLMs in downstream tasks. However, current prompt optimization methods are costly to train and lack sufficient interpretability. This paper proposes enhancing LLMs' reasoning performance by eliciting their causal inference ability from prompting instructions to correct answers. Specifically, we introduce the Self-Causal Instruction Enhancement (SCIE) method, which enables LLMs to generate high-quality, low-quantity observational data, then estimates the causal effect based on these data, and ultimately generates instructions with the optimized causal effect. In SCIE, the instructions are treated as the treatment, and textual features are used to process natural language, establishing causal relationships through treatments between instructions and downstream tasks. Additionally, we propose applying Object-Relational (OR) principles, where the uncovered causal relationships are treated as the inheritable class across task objects, ensuring low-cost reusability. Extensive experiments demonstrate that our method effectively generates instructions that enhance reasoning performance with reduced training cost of prompts, leveraging interpretable textual features to provide actionable insights.

Figures

Figures reproduced from arXiv: 2412.15314 by the authors.

Figure 1
Figure 1. Illustrative examples demonstrating the purpose [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. The overall process of (Object-Relational) Self-Causal Instruction Enhancement includes Data Generation, Causal Effect Estimation, Enhanced Instructions Generation, and optional OR Module. Theoretically, the counterfactual Y (t) can be defined for any treatment t, but it might be restricted to the represen￾tation of the text (Feder et al. 2022). Methods for handling high-dimensional text as treatment can be divided … view at source ↗
Figure 3
Figure 3. Probability density distributions of the proxy fea [PITH_FULL_IMAGE:figures/full_fig_p004_3.png] view at source ↗
Figures from the paper (2 more)
Figure 4
Figure 4. Figure 4: The causal effect estimation with the LLM (asterisk points), T-Learner (blue circular points), and S-Learner (red [PITH_FULL_IMAGE:figures/full_fig_p006_4.png]
Figure 5
Figure 5. Figure 5: (a) The reasoning accuracy with and without SCIE, [PITH_FULL_IMAGE:figures/full_fig_p007_5.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

49 extracted references · 26 canonical work pages

  1. [1]

    , " * write output.state after.block = add.period write newline

    ENTRY address archivePrefix author booktitle chapter edition editor eid eprint howpublished institution isbn journal key month note number organization pages publisher school series title type volume year label extra.label sort.label short.list INTEGERS output.state before.all mid.sentence after.sentence after.block FUNCTION init.state.consts #0 'before.a...

  2. [2]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 global.max substring 't := if while FUNCTION word.in bbl.in capitalize " " * FUNCT...

  3. [3]

    L.; Almeida, D.; Altenschmidt, J.; Altman, S.; Anadkat, S.; et al

    Achiam, J.; Adler, S.; Agarwal, S.; Ahmad, L.; Akkaya, I.; Aleman, F. L.; Almeida, D.; Altenschmidt, J.; Altman, S.; Anadkat, S.; et al. 2023. Gpt-4 technical report. arXiv preprint arXiv:2303.08774

    arXiv 2023

  4. [4]

    Bengio, Y.; et al. 2019. From system 1 deep learning to system 2 deep learning. In Neural Information Processing Systems

    work page 2019

  5. [5]

    D.; Dhariwal, P.; Neelakantan, A.; Shyam, P.; Sastry, G.; Askell, A.; et al

    Brown, T.; Mann, B.; Ryder, N.; Subbiah, M.; Kaplan, J. D.; Dhariwal, P.; Neelakantan, A.; Shyam, P.; Sastry, G.; Askell, A.; et al. 2020. Language models are few-shot learners. Advances in neural information processing systems, 33: 1877--1901

    2020

  6. [6]

    Cao, C.; Zhong, Z.; Zhou, Z.; Liu, Y.; Liu, T.; and Han, B. 2024. Envisioning Outlier Exposure by Large Language Models for Out-of-Distribution Detection. In ICML

    work page 2024

  7. [7]

    J.; DeWitt, D

    Carey, M. J.; DeWitt, D. J.; Naughton, J. F.; Asgarian, M.; Brown, P.; Gehrke, J. E.; and Shah, D. N. 1997. The BUCKY object-relational benchmark. In Proceedings of the 1997 ACM SIGMOD international conference on Management of data, 135--146

    work page 1997

  8. [8]

    Chang, K.; Xu, S.; Wang, C.; Luo, Y.; Xiao, T.; and Zhu, J. 2024. Efficient Prompting Methods for Large Language Models: A Survey. arXiv preprint arXiv:2404.01077

    arXiv 2024

Show all 49 references
  1. [9]

    Cobbe, K.; Kosaraju, V.; Bavarian, M.; Chen, M.; Jun, H.; Kaiser, L.; Plappert, M.; Tworek, J.; Hilton, J.; Nakano, R.; et al. 2021. Training verifiers to solve math word problems. arXiv preprint arXiv:2110.14168

    2021 arXiv

  2. [10]

    Crispino, N.; Montgomery, K.; Zeng, F.; Song, D.; and Wang, C. 2024. Agent Instructs Large Language Models to be General Zero-Shot Reasoners. In Forty-first International Conference on Machine Learning

    2024

  3. [11]

    Dhawan, N.; Cotta, L.; Ullrich, K.; Krishnan, R.; and Maddison, C. J. 2024. End-To-End Causal Effect Estimation from Unstructured Natural Language Data. In ICML 2024 Workshop on Foundation Models in the Wild

    2024

  4. [12]

    Dong, L.; Yang, N.; Wang, W.; Wei, F.; Liu, X.; Wang, Y.; Gao, J.; Zhou, M.; and Hon, H.-W. 2019. Unified language model pre-training for natural language understanding and generation. Advances in neural information processing systems, 32

    2019

  5. [13]

    Dubey, A.; Jauhri, A.; Pandey, A.; Kadian, A.; Al-Dahle, A.; Letman, A.; Mathur, A.; Schelten, A.; Yang, A.; Fan, A.; et al. 2024. The llama 3 herd of models. arXiv preprint arXiv:2407.21783

    2024 arXiv

  6. [14]

    L.; Jiang, L.; Lin, B

    Dziri, N.; Lu, X.; Sclar, M.; Li, X. L.; Jiang, L.; Lin, B. Y.; Welleck, S.; West, P.; Bhagavatula, C.; Le Bras, R.; et al. 2024. Faith and fate: Limits of transformers on compositionality. Advances in Neural Information Processing Systems, 36

    2024

  7. [15]

    A.; Manzoor, E.; Pryzant, R.; Sridhar, D.; Wood-Doughty, Z.; Eisenstein, J.; Grimmer, J.; Reichart, R.; Roberts, M

    Feder, A.; Keith, K. A.; Manzoor, E.; Pryzant, R.; Sridhar, D.; Wood-Doughty, Z.; Eisenstein, J.; Grimmer, J.; Reichart, R.; Roberts, M. E.; et al. 2022. Causal inference in natural language processing: Estimation, prediction, interpretation and beyond. Transactions of the Ass...

    2022

  8. [16]

    Geva, M.; Khashabi, D.; Segal, E.; Khot, T.; Roth, D.; and Berant, J. 2021. Did aristotle use a laptop? a question answering benchmark with implicit reasoning strategies. Transactions of the Association for Computational Linguistics, 9: 346--361

    2021

  9. [17]

    Holland, P. W. 1986. Statistics and causal inference. Journal of the American statistical Association, 81(396): 945--960

    1986

  10. [18]

    Howard, J.; and Ruder, S. 2018. Universal Language Model Fine-tuning for Text Classification. In Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), 328--339

    2018

  11. [19]

    Jensen, D. D. 2021. Improving causal inference by increasing model expressiveness. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 35, 15053--15057

    2021

  12. [20]

    K c man, E.; Ness, R.; Sharma, A.; and Tan, C. 2023. Causal reasoning and large language models: Opening a new frontier for causality. arXiv preprint arXiv:2305.00050

    2023 arXiv

  13. [21]

    S.; Reid, M.; Matsuo, Y.; and Iwasawa, Y

    Kojima, T.; Gu, S. S.; Reid, M.; Matsuo, Y.; and Iwasawa, Y. 2022. Large language models are zero-shot reasoners. Advances in neural information processing systems, 35: 22199--22213

    2022

  14. [22]

    R.; Sekhon, J

    Kunzel, S. R.; Sekhon, J. S.; Bickel, P. J.; and Yu, B. 2019. Metalearners for estimating heterogeneous treatment effects using machine learning. Proceedings of the national academy of sciences, 116(10): 4156--4165

    2019

  15. [23]

    Lee, Y.; and Ogburn, E. L. 2021. Network dependence can lead to spurious associations and invalid inference. Journal of the American Statistical Association, 116(535): 1060--1074

    2021

  16. [24]

    Lester, B.; Al-Rfou, R.; and Constant, N. 2021. The Power of Scale for Parameter-Efficient Prompt Tuning. In Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, 3045--3059

    2021

  17. [25]

    Lewis, M.; Liu, Y.; Goyal, N.; Ghazvininejad, M.; Mohamed, A.; Levy, O.; Stoyanov, V.; and Zettlemoyer, L. 2019. Bart: Denoising sequence-to-sequence pre-training for natural language generation, translation, and comprehension. arXiv preprint arXiv:1910.13461

    2019 arXiv

  18. [26]

    Li, X.; Zhou, Z.; Zhu, J.; Yao, J.; Liu, T.; and Han, B. 2023. Deepinception: Hypnotize large language model to be jailbreaker. arXiv preprint arXiv:2311.03191

    2023 arXiv

  19. [27]

    Liu, C.; Chen, Y.; Liu, T.; Gong, M.; Cheng, J.; Han, B.; and Zhang, K. 2024. Discovery of the Hidden World with Large Language Models. In Proceedings of the Thirty-eighth Annual Conference on Neural Information Processing Systems

    2024

  20. [28]

    Liu, P.; Yuan, W.; Fu, J.; Jiang, Z.; Hayashi, H.; and Neubig, G. 2023. Pre-train, prompt, and predict: A systematic survey of prompting methods in natural language processing. ACM Computing Surveys, 55(9): 1--35

    2023

  21. [29]

    Maiya, A. S. 2021. CausalNLP: A practical toolkit for causal inference with text. arXiv preprint arXiv:2106.08043

    2021 arXiv

  22. [30]

    Open Interpreter . 2024. open-interpreter: A natural language interface for computers. https://github.com/OpenInterpreter. Accessed: 2024-08-03

    2024

  23. [31]

    OpenAI . 2022. GPT-3.5 Turbo fine-tuning and API updates. https://openai.com/index/gpt-3-5-turbo-fine-tuning-and-api-updates/. Accessed: 2024-12-16

    2022

  24. [32]

    Prasad, A.; Hase, P.; Zhou, X.; and Bansal, M. 2023. GrIPS: Gradient-free, Edit-based Instruction Search for Prompting Large Language Models. In Proceedings of the 17th Conference of the European Chapter of the Association for Computational Linguistics, 3845--3864

    2023

  25. [33]

    Pryzant, R.; Card, D.; Jurafsky, D.; Veitch, V.; and Sridhar, D. 2021. Causal Effects of Linguistic Properties. In Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, 4095--4109

    2021

  26. [34]

    Gradient Descent

    Pryzant, R.; Iter, D.; Li, J.; Lee, Y. T.; Zhu, C.; and Zeng, M. 2023. Automatic Prompt Optimization with" Gradient Descent" and Beam Search. In The 2023 Conference on Empirical Methods in Natural Language Processing

    2023

  27. [35]

    Pryzant, R.; Shen, K.; Jurafsky, D.; and Wagner, S. 2018. Deconfounded lexicon induction for interpretable social science. In Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume ...

    2018

  28. [36]

    R.; and Rubin, D

    Rosenbaum, P. R.; and Rubin, D. B. 1983. The central role of the propensity score in observational studies for causal effects. Biometrika, 70(1): 41--55

    1983

  29. [37]

    Roy, S.; and Roth, D. 2015. Solving General Arithmetic Word Problems. In Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing. Association for Computational Linguistics

    2015

  30. [38]

    Rubin, D. B. 1974. Estimating causal effects of treatments in randomized and nonrandomized studies. Journal of educational Psychology, 66(5): 688

    1974

  31. [39]

    W.; Chowdhery, A.; Le, Q.; Chi, E.; Zhou, D.; et al

    Suzgun, M.; Scales, N.; Sch \"a rli, N.; Gehrmann, S.; Tay, Y.; Chung, H. W.; Chowdhery, A.; Le, Q.; Chi, E.; Zhou, D.; et al. 2023. Challenging BIG-Bench Tasks and Whether Chain-of-Thought Can Solve Them. In Findings of the Association for Computational Linguistics: ACL 2023,...

    2023

  32. [40]

    Talmor, A.; Herzig, J.; Lourie, N.; and Berant, J. 2019. CommonsenseQA: A question answering challenge targeting commonsense knowledge. In Proceedings of the 2019 Conference of the North, 4149. Association for Computational Linguistics

    2019

  33. [41]

    Tang, Z.; Zhang, X.; Wang, B.; and Wei, F. 2024. MathScale: Scaling Instruction Tuning for Mathematical Reasoning. In Forty-first International Conference on Machine Learning

    2024

  34. [42]

    K.-W.; and Lim, E.-P

    Wang, L.; Xu, W.; Lan, Y.; Hu, Z.; Lan, Y.; Lee, R. K.-W.; and Lim, E.-P. 2023. Plan-and-Solve Prompting: Improving Zero-Shot Chain-of-Thought Reasoning by Large Language Models. In Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume...

    2023

  35. [43]

    Wang, Y.; and Luo, Z. 2024. Exploring latent discrimination through an Object-Relational Causal Inference method. Knowledge-Based Systems, 112148

    2024

  36. [44]

    V.; Zhou, D.; et al

    Wei, J.; Wang, X.; Schuurmans, D.; Bosma, M.; Xia, F.; Chi, E.; Le, Q. V.; Zhou, D.; et al. 2022. Chain-of-thought prompting elicits reasoning in large language models. Advances in neural information processing systems, 35: 24824--24837

    2022

  37. [45]

    Wood-Doughty, Z.; Shpitser, I.; and Dredze, M. 2018. Challenges of using text classifiers for causal inference. In Proceedings of the Conference on Empirical Methods in Natural Language Processing. Conference on Empirical Methods in Natural Language Processing, volume 2018, 45...

    2018

  38. [46]

    Xu, H.; Chen, Y.; Du, Y.; Shao, N.; Yanggang, W.; Li, H.; and Yang, Z. 2022. GPS: Genetic Prompt Search for Efficient Few-Shot Learning. In Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing, 8162--8171

    2022

  39. [47]

    V.; Zhou, D.; and Chen, X

    Yang, C.; Wang, X.; Lu, Y.; Liu, H.; Le, Q. V.; Zhou, D.; and Chen, X. 2023. Large Language Models as Optimizers. ArXiv, abs/2309.03409

    2023 arXiv

  40. [48]

    I.; Han, Z.; Paster, K.; Pitis, S.; Chan, H.; and Ba, J

    Zhou, Y.; Muresanu, A. I.; Han, Z.; Paster, K.; Pitis, S.; Chan, H.; and Ba, J. 2022. Large Language Models are Human-Level Prompt Engineers. In The Eleventh International Conference on Learning Representations

    2022

  41. [49]

    Zhou, Z.; Tao, R.; Zhu, J.; Luo, Y.; Wang, Z.; and Han, B. 2024. Can Language Models Perform Robust Reasoning in Chain-of-thought Prompting with Noisy Rationales? In The Thirty-eighth Annual Conference on Neural Information Processing Systems

    2024

Pith tools

Reviewed August 11, 2026 · model on record in the stance chip above.