Pith. sign in

REVIEW 3 major objections 5 minor 44 references

The Confounder Trap: Treatment-Encoding Representations in Causal Inference with Text

T0 review · 3 major / 5 minor · reviewed 2026-08-01 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read Masking treatment-defining words before representation learning prevents the overlap collapse that makes text-based causal adjustment fail.

desk verdict A solid, honestly-scoped method paper: the formal overlap result for deletion masking is correct, the LLM extension is a useful heuristic with a disclosed but non-estimable sensitivity bound. read the letter →

arxiv 2607.26309 v1 pith:PGSZC4K3 submitted 2026-07-28 stat.ME cs.CLstat.APstat.ML

Marie Neubrander , Graham Tierney , Alexander Volfovsky This is my paper
classification stat.MEcs.CLstat.APstat.ML MSC 62D2068T50
keywords causalinferencefromtextoverlapassumptiontreatment-encodingrepresentationsmaskinglatenttreatmentspropensityscoretopicmodelslargelanguage
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper's claim is that in text-as-treatment settings where treatment is encoded by identifiable words—sentiment terms, partisan framing, hedging—the standard practice of learning adjustment representations from full text can break causal inference not by missing confounders but by encoding treatment directly, making treated and control documents separable and destroying overlap. It proposes masking: delete or replace treatment-defining words before representation learning. For bag-of-words/topic models, deletion masking gives a formal guarantee—any representation based on the masked text has propensity scores bounded away from 0 and 1 whenever the true confounders capture remaining treatment-related variation. For large language models, replacement masking is a deliberate relaxation, paired with an adaptive penalty that keeps estimated propensities inside an overlap band. In simulations the masking pipeline keeps 87–98% of propensity scores in a safe range where full-text representations put nearly all scores outside it, and it reduces error relative to both unmasked and concept-erased baselines.

What carries the argument

The carrying mechanism is the treatment-removing transformation of the text performed before representation learning. Deletion masking removes treatment-defining tokens so the representation is a function only of remaining words; the proof shows the induced propensity score is a conditional expectation of the true propensity e⋆(Z) given the representation, inheriting the [α⋆,1−α⋆] bounds. Replacement masking for language models replaces the tokens with [MASK] and adds a training-time penalty that activates when estimated propensities leave [α,1−α]; its sensitivity bound (Theorem 6) says overlap degrades smoothly with residual leakage rather than collapsing. The encompassing device is the com

What would settle it

In a simulation where the true confounders Z are known, compute the Bayes error R⋆ of T from Z and the Bayes error R_b of T from the masked representation. Theorem 2 and Theorem 4 jointly imply R_b ≥ R⋆ whenever masking preserves overlap. A single configuration—for example, a corpus where product-category words are correlated with sentiment and survive deletion—where a classifier on the masked bag-of-words achieves R_b < R⋆ would show the independence premise fails and the guarantee breaks.

Watch

Extended reading notes

Core claim

The central result is Theorem 4: if W− is a treatment-removing transformation with T⊥W−|Z, then any representation b−(W)=h(W−) induces a propensity score P(T=1|b−(W)) inside [α⋆,1−α⋆], the same overlap margin assumed for the true confounders Z. In words, once the lexical shortcut is removed, any remaining treatment predictability in the representation is attributable to confounders and is legitimate for adjustment. Deletion of lexicon words realizes this for bag-of-words/topic models. Replacement masking—substituting [MASK] tokens—does not satisfy the strict independence, so the paper instead provides a sensitivity bound: the fraction of propensities leaving a widened band is at most a treat

Load-bearing premise

The entire overlap guarantee rests on the existence of a transformation with T⊥W−|Z—for deletion masking, that after removing the defining words, no remaining text predicts treatment except through the true confounders; the paper concedes replacement masking does not satisfy this and falls back on a sensitivity bound.

Editorial extensions

If this is right

  • Any full-text representation in a lexicon-defined text-treatment study should be checked for overlap collapse; Theorem 2 shows the fraction of documents with propensities inside [α,1−α] is at most Bayes error divided by α, and Corollary 3 gives exact collapse when treatment is recoverable.
  • Analysts already using topic-model adjustments can adopt deletion masking with minimal change and obtain a provable overlap guarantee, with simulated gains in IPW particularly large.
  • Concept-erasure approaches that strip all treatment signal restore overlap but discard confounder information; in the paper's simulations they perform no better than the naive difference in means.
  • For large language models, replacement masking plus an adaptive overlap penalty improves overlap diagnostics and lowers bias relative to unmasked training, while full-text evaluation of the mask-trained encoder is better than masked evaluation.
  • Replacement masking carries no formal guarantee; the honest reading is that it converts overlap collapse into a smooth degradation controlled by a non-estimable surplus, so practitioners should report the fraction of propensities in the overlap band.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Inference: The core mechanism—removing the named textual cue before representation learning—should transfer to any treatment with a discoverable cue, such as politeness markers or sentence length, even without a fixed lexicon; the paper tests only lexicon-defined treatments.
  • Inference: The adaptive overlap penalty alone, without any masking, may be a general-purpose safeguard for text-as-treatment settings where no lexicon exists; the paper's ablation suggests it works but relegates it to an appendix.
  • Inference: A direct test of the mechanism would compare the empirical Bayes error of masked-text representations to the Bayes error of the true confounders in a simulation; if masked representations ever predict treatment better than the oracle confounders, the assumed independence fails and the guarantee does not apply.
  • Inference: The non-estimability of the sensitivity bound suggests a practical design rule the paper does not state: in real applications, tune the overlap penalty to hold out a validation set and stop when the in-band fraction stabilizes, since the bound itself cannot be checked.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. The paper studies causal inference when the treatment is a latent linguistic property encoded in the text itself. It argues that representations learned from full text can encode treatment so well that overlap fails, even when the underlying causal model satisfies overlap, and proposes masking treatment-defining lexical tokens before representation learning: deletion masking for bag-of-words/topic models and replacement masking for large language models. It formalizes representation-induced overlap failure (Theorem 2), proves that deletion masking preserves overlap under a conditional-independence assumption (Theorem 4), and gives a sensitivity bound for replacement masking (Theorem 6). Simulations with LDA and DistilBERT on semi-synthetic Amazon reviews, plus a CFPB application, show improved MSE, bias, and overlap statistics relative to unmasked baselines.

Significance. If taken at face value, the paper identifies a genuinely important failure mode and offers a simple, interpretable remedy for lexicon-based text treatments. The formal results are modest but correctly proved under explicit assumptions; the LDA simulation is thorough, with topic-misspecification and parameter-sensitivity checks, and the LLM section includes valuable ablations that separate the contributions of masking and the overlap penalty. The central conceptual contribution—separating treatment-defining lexical signal from adjustment-relevant text—is useful. However, the strongest advertised guarantee applies only to deletion masking when a strong, untestable conditional-independence condition holds. The LLM variant has no formal overlap guarantee: its sensitivity bound depends on an unestimable quantity, and the reported in-band propensity fractions are partly produced by an adaptively tuned penalty that mechanically pushes propensity scores into the band. These caveats are acknowledged in places but not reflected in the abstract and several result presentations.

major comments (3)
  1. [Section 3.2, Theorem 4] The theorem's conclusion is entirely conditional on T⊥W⁻|Z. The sentence after the theorem states that this 'holds whenever Z captures any remaining treatment-correlated variation in non-lexicon text,' which is close to restating the assumption and is untestable in practice. If out-of-lexicon words carry treatment information beyond Z, e_b⁻(W) can fall outside [α⋆,1−α⋆]. The LDA simulation enforces the condition by construction—treatment/control topic words are exactly the deleted lexicon—so it cannot validate the general claim in the abstract that deletion masking preserves overlap for bag-of-words/topic-model representations. Please either develop an empirical check for residual dependence or explicitly restrict the claim to settings in which the conditional independence is known or plausibly holds.
  2. [Section 5 and Appendix A.5] Replacement masking is presented as a relaxation with no formal overlap guarantee. Theorem 6 bounds the fraction of propensity scores outside [α⋆−η,1−α⋆+η] by εM/η, but εM is defined via an unobservable conditional expectation eedel_M and is explicitly 'not directly estimable'; α⋆ is also unknown. Thus the bound is structural only and cannot underwrite the reported overlap improvements. Moreover, the 87–98% in-band fractions in Table 2 are partly produced by Lpenalty, whose weight is adaptively updated to target a 90% validation overlap rate (Section 5.1 and Appendix D.1). The ablation in D.6.1 shows masking without the penalty has only 16–27% in-band, so the headline overlap diagnostic largely measures the penalty, not the masking transformation. The authors acknowledge parts of this in Section 7, but the abstract and Section 5.2 still attribute the overlap improvement to masking. Pleas
  3. [Section 5.1 and Appendix D.8] The adaptive penalty introduces a tuning target (90% validation overlap) and an initial weight λP. Appendix D.8 shows that MSE is non-monotonic in λP^(0): for binary high confounding, the default λP^(0)=100 produces a spike in MSE, while neighboring values perform better. The authors report this sensitivity, but the absence of a principled criterion for choosing the overlap target or the initial penalty means the LLM method's performance depends on a hyperparameter whose default is not derived from the formal theory. This reinforces the gap between the claimed overlap guarantee and the actual procedure.
minor comments (5)
  1. [Abstract, Section 4.1] The abstract says deletion masking 'prove[s] that deletion masking preserves overlap for bag-of-words/topic-model representations' without the conditional-independence qualifier. Please add the assumption to the abstract or to the contribution list.
  2. [Table 1] The table formatting for the grouped columns (full/masked/leace under IPW, OR, AIPW) is hard to read; the column headers appear to be misaligned with the numeric entries. Please use a clearer layout with explicit subcolumn labels.
  3. [Section 3.1] The Bayes error R_b is unobservable, and the paper notes that any classifier gives an upper bound. This is fine, but the subsequent diagnostic uses estimated propensity scores, not the true e_b. Please clarify in the text that the empirical overlap fraction is an estimate of the quantity in Theorem 2 and is subject to estimation error.
  4. [References [11] and [12]] References [11] and [12] appear to be the same paper (Gui and Veitch, 'Causal estimation for text data with (apparent) overlap violations') in preprint and published form. Duplicate citation is confusing; retain one or clearly label as preprint/version.
  5. [Section 5.1] The auto-calibration of λQ says the outcome and cross-entropy losses 'contribute equally to the gradient,' but the total loss also contains Lpenalty and LU. Please clarify that equalization is only between Lout and LCE, not all terms.

Circularity Check

1 steps flagged · score 3.0 of 10

LLM overlap diagnostic is partly a training target; formal masking theorem is self-contained.

  1. fitted input called prediction [Section 5.1, multi-task loss paragraph; Appendix D.1 eq. (2); Table 2 PS column]
    "The second term is a soft overlap penalty that activates only when the estimated propensity leaves a target overlap band [α,1−α]: Lpenalty =E[(α−eg)2I{eg < α}+ (eg−(1−α))2I{eg >1−α}]. ... The penalty weight λP is tuned adaptively during training, targeting a 90% validation overlap rate."

    The reported overlap improvement for the LLM pipeline is the median fraction of held-out test propensity scores in [0.1,0.9] (Table 2 PS column). This is the same band that Lpenalty penalizes and the same 90% in-band rate that λP is adaptively updated to achieve (target violation q⋆=0.1 in Appendix D.1). Thus the headline 'masking improves overlap diagnostics' is partly a report of the training objective, not an independent test of the theory. The use of a held-out test split and the authors' no-penalty ablation (16-27% in band) show masking itself has real effect, so the circularity is partial rather than total.

full rationale

The formal derivation chain is not circular. Theorem 4 is proved from the conditional-independence assumption T⊥W−|Z by a tower-property argument: e_{b−}(W)=E[e*(Z)|b−(W)] ∈ [α*,1−α*]. This is a direct implication of the assumption, not a restatement of the conclusion. Theorem 2 and Theorem 6 are elementary Markov/tower-property inequalities with no fitted inputs. No load-bearing self-citation is present: citations to overlapping-author work [31] are used for framing and outcome interpretation, not to supply the core proof. The only place where an output reduces to an input is the LLM overlap diagnostic: λP is adaptively tuned to achieve 90% validation in-band propensities, and the PS column reports test fractions in the same [0.1,0.9] band that Lpenalty explicitly penalizes. The paper discloses this with ablations, and the causal MSE/bias improvements are not forced by the penalty, so the circularity is partial. The heavy reliance of Theorem 4 on untestable T⊥W−|Z and the non-estimable εM in Theorem 6 are limitations, not circularity.

Assumptions & free parameters 3 free parameters · 5 assumptions · 1 invented entities

The theoretical core is standard probability; the load-bearing assumptions are domain assumptions about the lexicon and conditional independence. The LLM empirical results depend on tuned hyperparameters λP and α, which are fitted to the overlap diagnostic. No new physical entities are introduced; ε_M is an unobservable theoretical construct.

free parameters (3)
  • λP (overlap penalty weight) = Adaptive; initialized 100, multiplicatively updated to target a 90% validation overlap rate (range [1,10^5])
    Controls how strongly LLM propensity scores are pulled into the overlap band; Appendix D.8 shows MSE and overlap vary with λP(0), so reported performance depends on this tuning.
  • Overlap band width α = 0.1 (diagnostic/winsorization band [0.1, 0.9])
    Chosen by analogy to Stürmer trimming (footnote 1); used in the soft penalty, the winsorization, and the reported PS diagnostic. Different α changes reported overlap rates.
  • Number of topics K in LDA = K=18 in main simulation (true K+2=20); sensitivity tested K+2 ∈ {10,...,30}
    Analyst-selected dimensionality; appendix shows masking still helps under misspecification, but errors depend on K.
assumptions (5)
  • standard math Standard causal assumptions A1-A4 (SUTVA, latent mean ignorability, outcome sufficiency, representation-level overlap)
    Section 2; Theorem 1 identification relies on these.
  • domain assumption D = T: the observed label matches the latent perceived treatment
    Section 2 'we work under the simplifying alignment Di = Ti'; required for lexicon labels to give treatment.
  • domain assumption T ⊥ W− | Z for the treatment-removing transformation
    Section 3 'We assume that the transformed texts carry no direct treatment information beyond the latent adjustment attributes'; Theorem 4's guarantee is conditional on this.
  • domain assumption Treatment has a known lexicon and is determined by T = g(W∩L)
    Section 3.2 'Suppose treatment is determined by a known lexicon L'; masking requires enumerating treatment-defining words.
  • ad hoc to paper Simulation DGP: documents generated from K+2 topics with treatment/control topic words and confounding topics
    Section 4.2/Appendix C; simulation evidence is valid for this generative process, not for arbitrary text.
invented entities (1)
  • Treatment-information surplus ε_M
    purpose: Measures residual treatment leakage in the mask-trained representation relative to deletion masking; used in Theorem 6
    Defined in Appendix A.5 conditional on an unobservable benchmark; the paper states it is not directly estimable, so it is a conceptual device.

how reviews work

0 comments
Cite this review

Pith. "Pith review of The Confounder Trap: Treatment-Encoding Representations in Causal Inference with Text." pith.science (2026). https://pith.science/paper/PGSZC4K3

@misc{pith2026260726309,
  author       = {Pith},
  title        = {Pith review of: The Confounder Trap: Treatment-Encoding Representations in Causal Inference with Text},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/PGSZC4K3}},
  note         = {Machine review of arXiv:2607.26309}
}
read the original abstract

Estimating causal effects of linguistic properties from observational text is difficult because the same document can contain both the treatment of interest and the non-treatment textual attributes needed for adjustment. Existing approaches often learn representations from the full text to capture latent confounding, but when treatment status is itself encoded by words in the text, these representations can directly encode treatment. This creates a confounder trap: richer representations can make treated and control documents separable, inducing overlap violations even when the underlying causal problem satisfies overlap. We study latent text treatments that are encoded through lexicons or other treatment-defining lexical information, and propose masking-based adjustment representations that remove this lexical treatment signal before representation learning. We formalize representation-induced overlap failure, prove that deletion masking preserves overlap for bag-of-words/topic-model representations, and characterize replacement masking as a natural relaxation for large language models that hides treatment-defining tokens while preserving word order and context. Across simulations, masking improves overlap diagnostics, stabilizes treatment effect estimates, and reduces bias relative to adjustment methods that learn from the unmasked text.

Figures

Figures reproduced from arXiv: 2607.26309 by the authors.

Figure 1
Figure 1. The confounder trap. (A) A document contains both treatment-defining words and [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 2
Figure 2. DAG. Grey nodes: observed outcome Y , labels D, and words W. White nodes: unobserved latent treat￾ment T and confounders Z. We work under the simplifying alignment Di = Ti : the label is assumed to match the latent perceived treatment. For treatments whose defining signal is observable in the text, the assumption is natural: treatment status is determined by the presence of specific words, and it is reasonable to as… view at source ↗
Figure 3
Figure 3. LDA plate diagram for assumed document structure. [PITH_FULL_IMAGE:figures/full_fig_p016_3.png] view at source ↗
Figures from the paper (7 more)
Figure 4
Figure 4. Figure 4: For each topic (y axis), the distributions of their true proportions across 2500 generated [PITH_FULL_IMAGE:figures/full_fig_p017_4.png]
Figure 5
Figure 5. Figure 5: compares the squared error values for outcome regression, IPW, and AIPW estimators computed using the full K + 2 topics (x-axis) and masked K topics (y-axis). Points that fall below the x = y line have a higher squared error for the *_full estimators. In outcome regres…
Figure 6
Figure 6. Figure 6: Mispecification plots for topics 10, 15, 18, 22, and 25. [PITH_FULL_IMAGE:figures/full_fig_p019_6.png]
Figure 7
Figure 7. Figure 7: Error comparison plots for varying Pα and β. topic word distributions of treatment and control topics. These topics have each have probabilities distributed across 100 non-overlapping words. For remaining topics, we take ηt ∈ {0.1, 0.5} as the Dirichlet parameter for t…
Figure 8
Figure 8. Figure 8: Estimated propensity score distributions for the masking-only estimator (no penalty; left [PITH_FULL_IMAGE:figures/full_fig_p025_8.png]
Figure 9
Figure 9. Figure 9: Exploratory; single simulation on binary [PITH_FULL_IMAGE:figures/full_fig_p027_9.png]
Figure 10
Figure 10. Figure 10: MSE versus median overlap rate (fraction of propensity scores in [PITH_FULL_IMAGE:figures/full_fig_p030_10.png]

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

44 extracted references · 9 canonical work pages

  1. [1]

    Bayesian topic regres- sion for causal inference

    Maximilian Ahrens, Julian Ashwin, Jan-Peter Calliess, and Vu Nguyen. Bayesian topic regres- sion for causal inference. In Marie-Francine Moens, Xuanjing Huang, Lucia Specia, and Scott Wen-tau Yih, editors,Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, pages 8162–8188, Online and Punta Cana, Dominican Republic, Nove...

    work page doi:10.18653/v1/2021.emnlp-main.644 2021

  2. [2]

    Leace: Perfect linear concept erasure in closed form, 2025

    Nora Belrose, David Schneider-Joseph, Shauli Ravfogel, Ryan Cotterell, Edward Raff, and Stella Biderman. Leace: Perfect linear concept erasure in closed form, 2025. URL https: //arxiv.org/abs/2306.03819

    arXiv 2025

  3. [3]

    Blei and John D

    David M. Blei and John D. Lafferty. Topic models. In Ashok N. Srivastava and Mehran Sahami, editors,Text Mining: Classification, Clustering and Applications, pages 71–93. Chapman & Hall/CRC, 2009. ISBN 978-1-4200-5940-3. doi: 10.1201/9781420059458-12. URL https://doi.org/10.1201/9781420059458-12

    work page doi:10.1201/9781420059458-12 2009

  4. [4]

    Blei, Andrew Y

    David M. Blei, Andrew Y . Ng, and Michael I. Jordan. Latent dirichlet allocation.J. Mach. Learn. Res., 3(null):993–1022, March 2003. ISSN 1532-4435

    2003

  5. [5]

    Choosing the number of topics in lda models — a monte carlo comparison of selection criteria.Journal of Machine Learning Research, 25(79):1–30, 2024

    Victor Bystrov, Viktoriia Naboka-Krell, Anna Staszewska-Bystrova, and Peter Winker. Choosing the number of topics in lda models — a monte carlo comparison of selection criteria.Journal of Machine Learning Research, 25(79):1–30, 2024. URL http://jmlr.org/papers/ v25/23-0188.html

    2024

  6. [6]

    Dealing with limited overlap in estimation of average treatment effects.Biometrika, 96(1):187–199, 2009

    Richard K Crump, V Joseph Hotz, Guido W Imbens, and Oscar A Mitnik. Dealing with limited overlap in estimation of average treatment effects.Biometrika, 96(1):187–199, 2009

    2009

  7. [7]

    Fong, Justin Grimmer, Margaret E

    Naoki Egami, Christian J. Fong, Justin Grimmer, Margaret E. Roberts, and Brandon M. Stewart. How to make causal inferences using texts.Science Advances, 8(42):eabg2652,

  8. [8]

    Discovery of treatments from text corpora

    Christian Fong and Justin Grimmer. Discovery of treatments from text corpora. In Katrin Erk and Noah A. Smith, editors,Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 1600–1609, Berlin, Germany, August 2016. Association for Computational Linguistics. doi: 10.18653/v1/P16-1151. URL http...

    work page doi:10.18653/v1/p16-1151 2016

Show all 44 references
  1. [9]

    Causal inference with latent treatments.American Journal of Political Science, 67(2):374–389, 2023

    Christian Fong and Justin Grimmer. Causal inference with latent treatments.American Journal of Political Science, 67(2):374–389, 2023

    2023

  2. [10]

    topicmodels: An r package for fitting topic models.Journal of Statistical Software, 40(13):1–30, 2011

    Bettina Grün and Kurt Hornik. topicmodels: An r package for fitting topic models.Journal of Statistical Software, 40(13):1–30, 2011. doi: 10.18637/jss.v040.i13. URL https://www. jstatsoft.org/index.php/jss/article/view/v040i13

    2011 doi

  3. [12]

    Causal estimation for text data with (apparent) overlap violations,

    Lin Gui and Victor Veitch. Causal estimation for text data with (apparent) overlap violations,

  4. [13]

    John Harker

    W. John Harker. Literary communication: The author, the reader, the text.Journal of Aesthetic Education, 22(2):5–14, 1988. ISSN 00218510, 15437809. URL http://www.jstor.org/ stable/3333119

    1988

  5. [14]

    Bridging language and items for retrieval and recommendation.arXiv preprint arXiv:2403.03952, 2024

    Yupeng Hou, Jiacheng Li, Zhankui He, An Yan, Xiusi Chen, and Julian McAuley. Bridging language and items for retrieval and recommendation.arXiv preprint arXiv:2403.03952, 2024

    2024 arXiv

  6. [15]

    an essay on the logical foundations of survey sampling, part one

    Jaroslav Hájek. Comment on "an essay on the logical foundations of survey sampling, part one". In V . P. Godambe and D. A. Sprott, editors,Foundations of Statistical Inference. Holt, Rinehart and Winston, Toronto, 1971

    1971

  7. [16]

    Johns Hopkins University Press, Baltimore, 1978

    Wolfgang Iser.The Act of Reading: A Theory of Aesthetic Response. Johns Hopkins University Press, Baltimore, 1978

    1978

  8. [17]

    Using text-based causal inference to disentangle factors influencing online review ratings

    Linsen Li, Aron Culotta, and Nicholas Mattei. Using text-based causal inference to disentangle factors influencing online review ratings. In Luis Chiruzzo, Alan Ritter, and Lu Wang, editors, Proceedings of the 2025 Conference of the Nations of the Americas Chapter of the Assoc...

    2025

  9. [18]

    Sentiment analysis and subjectivity

    Bing Liu. Sentiment analysis and subjectivity. InHandbook of Natural Language Processing, volume 2, pages 627–666. 2010

    2010

  10. [19]

    Estimating the effect of exercising on users’ online behavior.Proceedings of the International AAAI Con- ference on Web and Social Media, 11(1):734–738, May 2017

    Seyed Mirlohi Falavarjani, Hawre Hosseini, Zeinab Noorian, and Ebrahim Bagheri. Estimating the effect of exercising on users’ online behavior.Proceedings of the International AAAI Con- ference on Web and Social Media, 11(1):734–738, May 2017. doi: 10.1609/icwsm.v11i1.14975. UR...

    2017 doi

  11. [20]

    Matching with text data: An experimental evaluation of methods for matching documents and of measuring match quality.Political Analysis, 28(4):445–468, 2020

    Reagan Mozer, Luke Miratrix, Aaron Russell Kaufman, and L Jason Anastasopoulos. Matching with text data: An experimental evaluation of methods for matching documents and of measuring match quality.Political Analysis, 28(4):445–468, 2020

    2020

  12. [21]

    Kaufman, Leo A

    Reagan Mozer, Aaron R. Kaufman, Leo A. Celi, and Luke Miratrix. Leveraging text data for causal inference using electronic health records, 2024. URL https://arxiv.org/abs/ 2307.03687

    2024 arXiv

  13. [22]

    Yang, Zach DeVito, Martin Raison, Alykhan Tejani, Sasank Chilamkurthy, Benoit Steiner, Lu Fang, Junjie Bai, and Soumith Chintala

    Adam Paszke, Sam Gross, Francisco Massa, Adam Lerer, James Bradbury, Gregory Chanan, Trevor Killeen, Zeming Lin, Natalia Gimelshein, Luca Antiga, Alban Desmaison, Andreas Köpf, Edward Z. Yang, Zach DeVito, Martin Raison, Alykhan Tejani, Sasank Chilamkurthy, Benoit Steiner, Lu ...

    1912 arXiv

  14. [23]

    Pedregosa, G

    F. Pedregosa, G. Varoquaux, A. Gramfort, V . Michel, B. Thirion, O. Grisel, M. Blondel, P. Prettenhofer, R. Weiss, V . Dubourg, J. Vanderplas, A. Passos, D. Cournapeau, M. Brucher, M. Perrot, and E. Duchesnay. Scikit-learn: Machine learning in Python.Journal of Machine Learnin...

    2011

  15. [24]

    Causal effects of linguistic properties

    Reid Pryzant, Dallas Card, Dan Jurafsky, Victor Veitch, and Dhanya Sridhar. Causal effects of linguistic properties. In Kristina Toutanova, Anna Rumshisky, Luke Zettlemoyer, Dilek Hakkani- Tur, Iz Beltagy, Steven Bethard, Ryan Cotterell, Tanmoy Chakraborty, and Yichao Zhou, ed...

    2021

  16. [25]

    Roberts, Brandon M

    Margaret E. Roberts, Brandon M. Stewart, and Richard A. Nielsen. Adjusting for confounding with text matching.American Journal of Political Science, 64(4):887–903, 2020. doi: https:// doi.org/10.1111/ajps.12526. URL https://onlinelibrary.wiley.com/doi/abs/ 10.1111/ajps.12526. 11

    2020 doi

  17. [26]

    Robins, Andrea Rotnitzky, and Lue Ping Zhao

    James M. Robins, Andrea Rotnitzky, and Lue Ping Zhao. Analysis of semiparametric regression models for repeated outcomes in the presence of missing data.Journal of the American Statistical Association, 90(429):106–121, 1995. ISSN 01621459, 1537274X. URL http: //www.jstor.org/s...

    1995

  18. [27]

    Rosenbaum and Donald B

    Paul R. Rosenbaum and Donald B. Rubin. The central role of the propensity score in observa- tional studies for causal effects.Biometrika, 70(1):41–55, 1983. ISSN 00063444, 14643510. URLhttp://www.jstor.org/stable/2335942

    1983

  19. [28]

    Distilbert, a distilled version of BERT: smaller, faster, cheaper and lighter.CoRR, abs/1910.01108, 2019

    Victor Sanh, Lysandre Debut, Julien Chaumond, and Thomas Wolf. Distilbert, a distilled version of BERT: smaller, faster, cheaper and lighter.CoRR, abs/1910.01108, 2019. URL http://arxiv.org/abs/1910.01108

    1910 arXiv

  20. [29]

    Estimating causal effects of tone in online debates

    Dhanya Sridhar and Lise Getoor. Estimating causal effects of tone in online debates. In Proceedings of the Twenty-Eighth International Joint Conference on Artificial Intelligence, IJCAI-19, pages 1872–1878. International Joint Conferences on Artificial Intelligence Organiza- t...

    2019 doi

  21. [30]

    Til Stürmer, Michael Webster-Clark, Jennifer L Lund, Richard Wyss, Alan R Ellis, Mark Lunt, Kenneth J Rothman, and Robert J Glynn. Propensity score weighting and trimming strategies for reducing variance and bias of treatment effect estimates: A simulation study.American Journ...

    2021 doi

  22. [31]

    A design-based solution for causal inference with text: Can a language model be too large?,

    Graham Tierney, Srikar Katta, Christopher Bail, Sunshine Hillygus, and Alexander V olfovsky. A design-based solution for causal inference with text: Can a language model be too large?,

  23. [32]

    Adapting text embeddings for causal inference

    Victor Veitch, Dhanya Sridhar, and David Blei. Adapting text embeddings for causal inference. InConference on Uncertainty in Artificial Intelligence, pages 919–928. PMLR, 2020

    2020

  24. [33]

    Huggingface’s transformers: State-of-the-art natural language processing.CoRR, abs/1910.03771, 2019

    Thomas Wolf, Lysandre Debut, Victor Sanh, Julien Chaumond, Clement Delangue, Anthony Moi, Pierric Cistac, Tim Rault, Rémi Louf, Morgan Funtowicz, and Jamie Brew. Huggingface’s transformers: State-of-the-art natural language processing.CoRR, abs/1910.03771, 2019. URL http://arx...

    1910 arXiv

  25. [34]

    Detecting politeness in natural language.The R Journal, 10(2):489–502, 2018

    Michael Yeomans, Alejandro Kantor, and Dustin Tingley. Detecting politeness in natural language.The R Journal, 10(2):489–502, 2018. A Proof of Theorems A.1 Theorem 1 Proof.By the tower property and Assumption A1-A3, EB E(Y|T=t, B) =E B E(Y(t)|T=t, B) (Assumption A1) =E B E(Y(t...

    2018

  26. [39]

    The original code computes the ATE as the difference between the control and treated outcomes; we flip the direction to be the standard treated minus control outcomes in line with most literature

  27. [40]

    We apply the same data split as in our masking approach, training on the train data and testing on the test data

    The original application uses all data for both training and testing. We apply the same data split as in our masking approach, training on the train data and testing on the test data. This provides comparable estimates and avoids inducing post-selection bias. Furthermore, the ...

  28. [41]

    Again, we use the same training and test split our masking procedure employs

    The propensity score estimators fit the propensity scores on the same data used to estimate treatment effects. Again, we use the same training and test split our masking procedure employs

  29. [42]

    We fit propensity scores using the Random Forest Classifier in scitkit-learn

  30. [43]

    Many estimated propensity scores are degenerate (exactly 0 or 1); not doing so leads to unstable andNaNATE estimates

    As discussed in the main body, propensity scores are trimmed and winsorized. Many estimated propensity scores are degenerate (exactly 0 or 1); not doing so leads to unstable andNaNATE estimates. D.3 Amazon simulations: Confounders Table 5: Confounding Topic Descriptions Topic ...

  31. [44]

    Lexicon available:Apply masking and the overlap penalty jointly for the strongest and most theoretically grounded performance

  32. [45]

    D.7 Evaluation on Masked Text Section 3.2 introduced the design choice to train on masked text but evaluate on the original document

    No lexicon available:The overlap penalty alone provides a lexicon-free alternative that still substantially reduces bias and improves overlap relative to existing text-as-treatment methods. D.7 Evaluation on Masked Text Section 3.2 introduced the design choice to train on mask...

    2020

  33. [2021]

    doi: 10.18653/v1/2021.naacl-main.323

    Association for Computational Linguistics. doi: 10.18653/v1/2021.naacl-main.323. URL https://aclanthology.org/2021.naacl-main.323/

    2021 doi

  34. [2022]

    URL https://www.science.org/doi/abs/10

    doi: 10.1126/sciadv.abg2652. URL https://www.science.org/doi/abs/10. 1126/sciadv.abg2652

    doi

  35. [2023]

    URLhttps://arxiv.org/abs/2210.00079. 10

    arXiv

  36. [2025]

    URLhttps://arxiv.org/abs/2510.08758

Pith tools

Reviewed August 1, 2026 · model on record in the stance chip above.