Pith. sign in

REVIEW 2 major objections 2 minor 38 references

What's the Magic Formula Instrument?

T0 review · 2 major / 2 minor · reviewed 2026-06-26 · grok-4.3

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read Formula instrument estimates can take many signs and sizes when the shock distribution assumption is changed slightly.

desk verdict This paper gives a practical sensitivity tool for formula instruments and shows the Borusyak-Hull applications recover estimates of different signs with modest changes to the shock distribution. read the letter →

arxiv 2606.21569 v1 pith:5LRXEWAZ submitted 2026-06-19 econ.EM stat.ME

Peizan Sheng (University of Chicago , Harris School of Public Policy) , Alexander Torgovitsky (University of Chicago , Kenneth C. Griffin Department of Economics) This is my paper · ORCID
classification econ.EMstat.ME
keywords formulainstrumentinstrumentalvariablessensitivityanalysisshockdistributionparametricassumptionseconometricmethodscausalinference
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper develops a method to test how much formula instrument estimates depend on the specific parametric distribution chosen for the unobserved shocks. This assumption is required to implement the adjustment proposed in recent work on linear instrumental variables with confounding covariates. Applying the method to two existing applications shows that different but nearby distributions produce estimates that differ in both sign and magnitude. A reader would care because the approach is meant to deliver more credible causal estimates, yet the results appear fragile to a modeling choice that is often made without much justification.

What carries the argument

The sensitivity evaluation method that varies only the parametric distribution of unobserved shocks while keeping other modeling choices fixed.

What would settle it

Re-running the two applications with the sensitivity method and finding that estimates remain stable in sign and magnitude across a wide but plausible range of shock distributions would show the reported sensitivity does not hold.

Watch

Extended reading notes

Core claim

The authors introduce a systematic sensitivity method, implemented in an accompanying R package, that isolates the effect of the parametric shock distribution assumption in the formula instrument estimator. When this method is applied to the applications in Borusyak and Hull (2023) and (2026), a range of estimates with varying signs and magnitudes can be recovered by making only small changes to the assumed distribution of the shocks.

Load-bearing premise

The sensitivity method isolates the effect of the shock distribution assumption without introducing its own unexamined modeling choices.

Editorial extensions

If this is right

  • Formula instrument estimates in the examined applications are not robust to reasonable changes in the shock distribution.
  • Slight alterations to the assumed distribution can reverse the sign of the estimated effect.
  • Researchers using the formula instrument approach need to check sensitivity to the shock distribution choice.
  • The adjustment procedure can support multiple conflicting conclusions depending on the distributional assumption.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Similar sensitivity checks may be useful for other instrumental variable adjustments that rely on parametric distributional assumptions.
  • The findings suggest value in exploring non-parametric or distribution-free versions of the formula instrument adjustment.
  • Applied researchers may want to report ranges of estimates rather than single numbers when using this method.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

2 major / 2 minor

Summary. The paper develops a sensitivity analysis method for the 'formula instrument' IV estimators proposed by Borusyak and Hull, which require a parametric assumption on the distribution of unobserved shocks. Implemented in the companion R package formulaiv, the method is applied to reanalyze the two Borusyak-Hull applications; the central finding is that modest changes to the shock distribution recover estimates of varying signs and magnitudes.

Significance. If the method correctly isolates variation in the shock-distribution assumption while holding all other modeling choices fixed, the paper supplies a practical robustness tool for formula-instrument applications and indicates that the original point estimates may be fragile. The open-source package supports reproducibility and direct use by applied researchers.

major comments (2)
  1. [Method (sensitivity procedure)] The manuscript must demonstrate, with explicit steps or pseudocode, that the sensitivity procedure varies only the parametric shock distribution while leaving the rest of the formula-instrument estimator (including covariate adjustment and weighting) unchanged; without this isolation the reanalysis results cannot be interpreted as evidence of sensitivity to the distribution assumption alone.
  2. [Empirical applications] In the reanalysis sections for both Borusyak-Hull applications, report the exact parametric families and parameter values used for each sensitivity draw, together with the full set of resulting point estimates and standard errors, so that readers can verify the claim that estimates of different signs and magnitudes are recovered.
minor comments (2)
  1. [Abstract and §1] Clarify in the abstract and introduction whether the sensitivity method introduces any auxiliary parametric choices (e.g., discretization of the support or numerical integration rules) beyond the shock distribution itself.
  2. [Monte Carlo validation] Add a short simulation exercise in which the true shock distribution is known and the sensitivity procedure is shown to recover the correct range of estimates.

Simulated Author's Rebuttal

2 responses · 0 unresolved

We thank the referee for the constructive comments and positive recommendation. We address each major comment below and will revise the manuscript accordingly.

read point-by-point responses

  1. Referee: [Method (sensitivity procedure)] The manuscript must demonstrate, with explicit steps or pseudocode, that the sensitivity procedure varies only the parametric shock distribution while leaving the rest of the formula-instrument estimator (including covariate adjustment and weighting) unchanged; without this isolation the reanalysis results cannot be interpreted as evidence of sensitivity to the distribution assumption alone.

    Authors: We agree that explicit isolation of the shock-distribution assumption is necessary for clear interpretation. In the revised manuscript we will add a dedicated subsection (with numbered steps and pseudocode) that shows the sensitivity procedure modifies only the parametric family and parameters of the unobserved shocks while holding fixed the formula-instrument construction, covariate adjustment, weighting, and all other estimator components. This addition will directly address the concern. revision: yes

  2. Referee: [Empirical applications] In the reanalysis sections for both Borusyak-Hull applications, report the exact parametric families and parameter values used for each sensitivity draw, together with the full set of resulting point estimates and standard errors, so that readers can verify the claim that estimates of different signs and magnitudes are recovered.

    Authors: We will expand both reanalysis sections to include tables that list, for every sensitivity draw: (i) the exact parametric family (e.g., normal, Student-t), (ii) the specific parameter values, and (iii) the resulting point estimate and standard error. These tables will replace the current summary statements and will allow readers to verify the range of signs and magnitudes obtained. revision: yes

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity identified

full rationale

The paper develops an independent sensitivity analysis method (implemented in formulaiv) to evaluate robustness of formula-instrument IV estimates to parametric shock distribution assumptions. It applies this tool to reanalyze two existing Borusyak-Hull applications and reports that varying the shock distribution recovers estimates of differing signs and magnitudes. No derivation chain is claimed that reduces a prediction or result to its own fitted inputs by construction, nor does the central claim rest on self-citation load-bearing steps, uniqueness theorems imported from the authors, or ansatzes smuggled via citation. The method is presented as a standalone tool for isolating the effect of one modeling choice, making the paper self-contained against external benchmarks with no load-bearing circular reductions.

Assumptions & free parameters 0 free parameters · 0 assumptions · 0 invented entities

Abstract-only; no explicit free parameters, axioms, or invented entities are described beyond the general parametric shock distribution assumption already present in the cited work.

how reviews work

0 comments
Cite this review

Pith. "Pith review of What's the Magic Formula Instrument?." pith.science (2026). https://pith.science/paper/5LRXEWAZ

@misc{pith2026260621569,
  author       = {Pith},
  title        = {Pith review of: What's the Magic Formula Instrument?},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/5LRXEWAZ}},
  note         = {Machine review of arXiv:2606.21569}
}
read the original abstract

Two recent papers by Borusyak and Hull (2023, 2026) propose using known formulas to adjust linear instrumental variable estimators for confounding covariates. Implementing this "formula instrument" approach requires making a parametric assumption on the distribution of the unobserved shocks that generated the instrument. We develop a method for systematically evaluating the sensitivity of formula instrument estimates to this parametric assumption. The method is straightforward to implement using our companion R package formulaiv. We use our method to reanalyze the applications in both Borusyak and Hull (2023) and Borusyak and Hull (2026). In both applications, we find that a variety of estimates of different signs and magnitudes can be recovered by slightly changing the shock distribution.

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

38 extracted references

  1. [1]

    Andresen, M. E. (2018): Exploring Marginal Treatment Effects : Flexible Estimation Using Stata , The Stata Journal: Promoting communications on statistics and Stata, 18, 118--158

    2018

  2. [2]

    Makany, G

    Bagues, M., M. Makany, G. Vattuone, and N. Zinovyeva (2026): Female Promotions and the Academic Pipeline: Evidence from a Natural Experiment,

    2026

  3. [3]

    Bartik, T. J. (1991): Who Benefits from State and Local Economic Development Policies ? , W.E. Upjohn Institute

    1991

  4. [4]

    Blanchard, O. J. and L. F. Katz (1992): Regional Evolutions , Brookings Papers on Economic Activity, 1992, 1

    1992

  5. [5]

    Bonhomme, S. and M. Weidner (2022): Minimizing Sensitivity to Model Misspecification, Quantitative Economics, 13, 907--954

    2022

  6. [6]

    Borusyak, K. and P. Hull (2021): Non-Random Exposure to Exogenous Shocks: Theory and Applications, Tech. rep., Mimeo

    2021

  7. [7]

    --- -.1pt --- -.1pt --- (2023): Nonrandom Exposure to Exogenous Shocks , Econometrica, 91, 2155--2185

    2023

  8. [8]

    --- -.1pt --- -.1pt --- (2026): Optimal Formula Instruments, Econometrica

    2026

Show all 38 references
  1. [9]

    Bosshart, L. and M. Weigand (2025): Crisis, State Capacity, and the Rise of Autocracy a Design-Based Analysis of the Thirty Years' War,

    2025

  2. [10]

    Boyd, S. and L. Vandenberghe (2004): Convex Optimization, Cambridge university press

    2004

  3. [11]

    Buhler, M. and A. Dickens (2025): National Content and Local Political Consequences: Evidence from Public and Private Television,

    2025

  4. [12]

    Lin, and A

    Cai, J., W. Lin, and A. Szeidl (2026): Firm-to-Firm Access in Production Networks,

    2026

  5. [13]

    Carneiro, P., J. J. Heckman, and E. J. Vytlacil (2011): Estimating Marginal Returns to Education , American Economic Review, 101, 2754--81

    2011

  6. [14]

    Charnes, A. and W. W. Cooper (1962): Programming with Linear Fractional Functionals, Naval Research Logistics Quarterly, 9, 181--186

    1962

  7. [15]

    Nayyar, S

    Chaurey, R., G. Nayyar, S. Sharma, and E. Verhoogen (2025): Social Learning among Urban Manufacturing Firms : Energy-Efficient Motors in Bangladesh , Tech. Rep. w34296, National Bureau of Economic Research, Cambridge, MA

    2025

  8. [16]

    Tamer, and A

    Chen, X., E. Tamer, and A. Torgovitsky (2011): Sensitivity Analysis in Semiparametric Likelihood Models , Cowles Foundation Discussion Paper 1836

    2011

  9. [17]

    Chetverikov, M

    Chernozhukov, V., D. Chetverikov, M. Demirer, E. Duflo, C. Hansen, W. Newey, and J. Robins (2018): Double/Debiased Machine Learning for Treatment and Structural Parameters, The Econometrics Journal, 21, C1--C68

    2018

  10. [18]

    Christensen, T. and B. Connault (2023): Counterfactual Sensitivity and Robustness , Econometrica, 91, 263--298

    2023

  11. [19]

    (1999): GMM estimation with cross sectional dependence, Journal of Econometrics, 92, 1--45

    Conley, T. (1999): GMM estimation with cross sectional dependence, Journal of Econometrics, 92, 1--45

    1999

  12. [20]

    Conley, T. G., C. B. Hansen, and P. E. Rossi (2012): Plausibly Exogenous, Review of Economics and Statistics, 94, 260--272

    2012

  13. [21]

    Currie, J. and J. Gruber (1996): Health Insurance Eligibility , Utilization of Medical Care , and Child Health , The Quarterly Journal of Economics, 111, 431--466

    1996

  14. [22]

    Dell, M. and B. A. Olken (2020): The Development Effects of the Extractive Colonial Economy : The Dutch Cultivation System in Java , The Review of Economic Studies, 87, 164--203

    2020

  15. [23]

    Ellingsen, and G

    Do, Q.-A., S. Ellingsen, and G. Lim (2025): Communities of Commerce: The Legacy of Chinese Immigration on Java,

    2025

  16. [24]

    Gruber, and B

    Frean, M., J. Gruber, and B. D. Sommers (2017): Premium subsidies, the mandate, and Medicaid expansion: Coverage effects of the Affordable Care Act, Journal of Health Economics, 53, 72--86

    2017

  17. [25]

    (2026): Endogenous Interference in Randomized Experiments,

    Gao, M. (2026): Endogenous Interference in Randomized Experiments,

    2026

  18. [26]

    Sorkin, and H

    Goldsmith-Pinkham , P., I. Sorkin, and H. Swift (2020): Bartik Instruments : What , When , Why , and How , American Economic Review, 110, 2586--2624

    2020

  19. [27]

    Gruber, J. and E. Saez (2002): The Elasticity of Taxable Income: Evidence and Implications, Journal of Public Economics, 84, 1--32

    2002

  20. [28]

    Gu, J. and T. M. Russell (2024): Wasserstein-Robust Counterfactuals,

    2024

  21. [29]

    Hristakeva, and K

    Hollenbeck, B., S. Hristakeva, and K. Uetake (2025): Retailer Price Competition and Assortment Differentiation: Evidence from Entry Lotteries,

    2025

  22. [30]

    Kumar, and K

    Jensen, A., N. Kumar, and K. Poensgen (2026): Tax Knowledge in Equilibrium: Evidence from an Education Program in Ghana,

    2026

  23. [31]

    Kline, P. and A. Santos (2013): Sensitivity to missing data assumptions: Theory and an evaluation of the U.S. wage structure: Sensitivity to missing data assumptions, Quantitative Economics, 4, 231--267

    2013

  24. [32]

    (1991): Sensitivity in Bayesian Statistics: The Prior and the Likelihood, Journal of the American Statistical Association, 86, 396--399

    Lavine, M. (1991): Sensitivity in Bayesian Statistics: The Prior and the Likelihood, Journal of the American Statistical Association, 86, 396--399

    1991

  25. [33]

    Masten, M. A. and A. Poirier (2025): Assessing Sensitivity to Identifying Assumptions , Working Paper

    2025

  26. [34]

    Nicoletti, K

    Moroni, G., C. Nicoletti, K. G. Salvanes, and E. Tominey (2025): Gender Equality through Marriage, IZA Discussion Paper 18288, Institute of Labor Economics (IZA), Bonn

    2025

  27. [35]

    Nevo, A. and A. M. Rosen (2012): Identification With Imperfect Instruments, Review of Economics and Statistics, 94, 659--671

    2012

  28. [36]

    Robinson, P. M. (1988): Root- N-Consistent Semiparametric Regression , Econometrica, 56, 931--954

    1988

  29. [37]

    (1976): Principles of Mathematical Analysis, New York: McGraw-Hill

    Rudin, W. (1976): Principles of Mathematical Analysis, New York: McGraw-Hill

    1976

  30. [38]

    Wasserman, L. and J. B. Kadane (1992): Computing Bounds on Expectations, Journal of the American Statistical Association, 87, 516--522

    1992

Pith tools

Reviewed June 26, 2026 · model on record in the stance chip above.