{"id":"22ba9345-3a2d-470d-a20a-d7065efc6a24","arxiv_id":"2606.21569","paper_version":1,"verdict":"UNVERDICTED","confidence":"LOW","novelty_score":6.0,"correctness_risk":"unknown","formal_verification":"none","parameter_count":0,"one_line_summary":"Authors develop a sensitivity method for formula instrument estimators and show that reanalyses of Borusyak-Hull applications produce estimates of varying signs and magnitudes under different shock distributions.","lead":"The paper introduces a sensitivity analysis method for formula instrument IV estimators that depend on parametric assumptions about unobserved shocks. Smart generalists might read it to see how robust recent causal inference techniques are when those assumptions vary slightly.","discovery_kind":"new_method","skeptic_critique":{"model":"grok-4.3","headline":"No significant objection identified","rationale":"The reader's weakest assumption correctly flags the isolation question as the point where the argument is least secured given the information available. Because the query supplies the full manuscript yet no concrete internal flaw (e.g., an unstated auxiliary assumption in a displayed equation or algorithm) is detectable, the honest non-finding follows. The claim remains internally consistent on its face; external disagreement with Borusyak-Hull is not, by itself, a load-bearing concern here.","tokens_in":1580,"tokens_out":322,"duration_ms":15007,"concrete_test":"Re-run the two applications using the formulaiv package while logging every auxiliary modeling choice (e.g., support of the distribution, moments matched, numerical integration settings) and confirm that only the shock-distribution parameter is altered between runs; if any other choice covaries, recompute the sensitivity surface after freezing it.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim is that a sensitivity method (implemented in formulaiv) shows formula-instrument estimates are fragile to modest changes in the parametric shock distribution, recovering estimates of varying sign and magnitude in the two Borusyak-Hull applications. For this to be load-bearing, the method must vary only the shock-distribution assumption while holding all other modeling choices fixed. The abstract provides no implementation details that would allow assessment of whether that isolation holds, but the full manuscript (per the query) would be required to locate any internal inconsistency. No such inconsistency is apparent from the stated claim or the reader's weakest assumption.","agreement_with_reader":"agree"},"referee_report":{"model":"grok-4.3","summary":"The paper develops a sensitivity analysis method for the 'formula instrument' IV estimators proposed by Borusyak and Hull, which require a parametric assumption on the distribution of unobserved shocks. Implemented in the companion R package formulaiv, the method is applied to reanalyze the two Borusyak-Hull applications; the central finding is that modest changes to the shock distribution recover estimates of varying signs and magnitudes.","tokens_in":1674,"tokens_out":437,"duration_ms":21765,"significance":"If the method correctly isolates variation in the shock-distribution assumption while holding all other modeling choices fixed, the paper supplies a practical robustness tool for formula-instrument applications and indicates that the original point estimates may be fragile. The open-source package supports reproducibility and direct use by applied researchers.","major_comments":[{"comment":"The manuscript must demonstrate, with explicit steps or pseudocode, that the sensitivity procedure varies only the parametric shock distribution while leaving the rest of the formula-instrument estimator (including covariate adjustment and weighting) unchanged; without this isolation the reanalysis results cannot be interpreted as evidence of sensitivity to the distribution assumption alone.","section":"Method (sensitivity procedure)"},{"comment":"In the reanalysis sections for both Borusyak-Hull applications, report the exact parametric families and parameter values used for each sensitivity draw, together with the full set of resulting point estimates and standard errors, so that readers can verify the claim that estimates of different signs and magnitudes are recovered.","section":"Empirical applications"}],"minor_comments":[{"comment":"Clarify in the abstract and introduction whether the sensitivity method introduces any auxiliary parametric choices (e.g., discretization of the support or numerical integration rules) beyond the shock distribution itself.","section":"Abstract and §1"},{"comment":"Add a short simulation exercise in which the true shock distribution is known and the sensitivity procedure is shown to recover the correct range of estimates.","section":"Monte Carlo validation"}],"recommendation":"minor_revision","confidential_remarks":null},"author_rebuttal":{"model":"grok-4.3","summary":"We thank the referee for the constructive comments and positive recommendation. We address each major comment below and will revise the manuscript accordingly.","responses":[{"response":"We agree that explicit isolation of the shock-distribution assumption is necessary for clear interpretation. In the revised manuscript we will add a dedicated subsection (with numbered steps and pseudocode) that shows the sensitivity procedure modifies only the parametric family and parameters of the unobserved shocks while holding fixed the formula-instrument construction, covariate adjustment, weighting, and all other estimator components. This addition will directly address the concern.","revision_made":"yes","referee_comment":"[Method (sensitivity procedure)] The manuscript must demonstrate, with explicit steps or pseudocode, that the sensitivity procedure varies only the parametric shock distribution while leaving the rest of the formula-instrument estimator (including covariate adjustment and weighting) unchanged; without this isolation the reanalysis results cannot be interpreted as evidence of sensitivity to the distribution assumption alone."},{"response":"We will expand both reanalysis sections to include tables that list, for every sensitivity draw: (i) the exact parametric family (e.g., normal, Student-t), (ii) the specific parameter values, and (iii) the resulting point estimate and standard error. These tables will replace the current summary statements and will allow readers to verify the range of signs and magnitudes obtained.","revision_made":"yes","referee_comment":"[Empirical applications] In the reanalysis sections for both Borusyak-Hull applications, report the exact parametric families and parameter values used for each sensitivity draw, together with the full set of resulting point estimates and standard errors, so that readers can verify the claim that estimates of different signs and magnitudes are recovered."}],"tokens_in":1218,"tokens_out":379,"duration_ms":11358,"standing_objections":[]},"desk_editor":{"model":"grok-4.3","letter":"The main point is a new sensitivity procedure for checking how formula-instrument IV estimates respond to the parametric assumption on unobserved shocks, packaged in an R tool called formulaiv, plus a reanalysis of the two Borusyak-Hull papers that finds the estimates can shift sign and size under small tweaks to that distribution.\n\nThe sensitivity method and the package are the actual new pieces. The reanalysis applies the tool directly to the existing applications and produces a clear cautionary result without needing new data.\n\nThe work is straightforward on the implementation side and lowers the cost for others to run the same checks. That is useful for anyone already working with the formula-instrument adjustment.\n\nThe soft spot is whether the procedure truly varies only the shock distribution while holding every other modeling choice fixed. The abstract does not spell out the exact steps, so the full paper needs to show that isolation explicitly; otherwise the fragility result could partly reflect other decisions. If the manuscript does that cleanly, the finding stands on firmer ground.\n\nThis is for applied econometricians who use or review shift-share style instruments and the Borusyak-Hull adjustment. Readers focused on robustness checks in IV settings will get direct value from the package and the two examples. It is narrow in scope but fills a specific gap.\n\nIt deserves a serious referee. The contribution is concrete, the applications are real, and the package makes the method testable. Referees can check the implementation details and decide how much weight to give the sensitivity results.","headline":"This paper gives a practical sensitivity tool for formula instruments and shows the Borusyak-Hull applications recover estimates of different signs with modest changes to the shock distribution.","tokens_in":2128,"tokens_out":384,"would_cite":false,"duration_ms":19279,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"grok-4.3","headline":"Formula instrument estimates can take many signs and sizes when the shock distribution assumption is changed slightly.","keywords":["formula instrument","instrumental variables","sensitivity analysis","shock distribution","parametric assumptions","econometric methods","causal inference"],"falsifier":"Re-running the two applications with the sensitivity method and finding that estimates remain stable in sign and magnitude across a wide but plausible range of shock distributions would show the reported sensitivity does not hold.","tokens_in":2509,"feed_emoji":"","tokens_out":578,"duration_ms":21459,"temperature":0.7,"pith_summary":"The paper develops a method to test how much formula instrument estimates depend on the specific parametric distribution chosen for the unobserved shocks. This assumption is required to implement the adjustment proposed in recent work on linear instrumental variables with confounding covariates. Applying the method to two existing applications shows that different but nearby distributions produce estimates that differ in both sign and magnitude. A reader would care because the approach is meant to deliver more credible causal estimates, yet the results appear fragile to a modeling choice that is often made without much justification.","feed_headline":"Formula instrument estimates flip with small distribution changes","feed_subtitle":"Reanalysis of two applications recovers estimates of different signs and magnitudes from minor tweaks to the shock distribution assumption","key_machinery":"The sensitivity evaluation method that varies only the parametric distribution of unobserved shocks while keeping other modeling choices fixed.","core_discovery":"The authors introduce a systematic sensitivity method, implemented in an accompanying R package, that isolates the effect of the parametric shock distribution assumption in the formula instrument estimator. When this method is applied to the applications in Borusyak and Hull (2023) and (2026), a range of estimates with varying signs and magnitudes can be recovered by making only small changes to the assumed distribution of the shocks.","pith_inferences":["Similar sensitivity checks may be useful for other instrumental variable adjustments that rely on parametric distributional assumptions.","The findings suggest value in exploring non-parametric or distribution-free versions of the formula instrument adjustment.","Applied researchers may want to report ranges of estimates rather than single numbers when using this method."],"forward_implications":["Formula instrument estimates in the examined applications are not robust to reasonable changes in the shock distribution.","Slight alterations to the assumed distribution can reverse the sign of the estimated effect.","Researchers using the formula instrument approach need to check sensitivity to the shock distribution choice.","The adjustment procedure can support multiple conflicting conclusions depending on the distributional assumption."],"fun_headline_variants":["Formula estimates flip with small shock distribution changes","Sensitivity analysis reveals formula instrument estimate variation","Formula instruments yield different signs under minor assumption tweaks","Shock distribution tweaks alter formula IV estimate signs","Reanalysis shows formula estimates vary by shock assumptions"],"cache_read_input_tokens":64,"weakest_assumption_plain":"The sensitivity method isolates the effect of the shock distribution assumption without introducing its own unexamined modeling choices.","fun_headline_variants_meta":{"raw":{"variants":["Formula estimates flip with small shock distribution changes","Sensitivity analysis reveals formula instrument estimate variation","Formula instruments yield different signs under minor assumption tweaks","Shock distribution tweaks alter formula IV estimate signs","Reanalysis shows formula estimates vary by shock assumptions"]},"model":"grok-4.3","cost_usd":0.004064,"raw_usage":{"total_tokens":2012,"prompt_tokens":558,"num_sources_used":0,"completion_tokens":65,"cost_in_usd_ticks":40637000,"prompt_tokens_details":{"text_tokens":558,"audio_tokens":0,"image_tokens":0,"cached_tokens":256},"completion_tokens_details":{"audio_tokens":0,"reasoning_tokens":1389,"accepted_prediction_tokens":0,"rejected_prediction_tokens":0}},"tokens_in":558,"tokens_out":65,"duration_ms":11169,"temperature":1.0,"reasoning_tokens":1389,"cache_read_input_tokens":256,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-06-26T12:23:59.346330+00:00","model_set":{"reader":"grok-4.3"},"falsifier":"Re-running the two applications with the sensitivity method and finding that estimates remain stable in sign and magnitude across a wide but plausible range of shock distributions would show the reported sensitivity does not hold.","supporting_citations":[],"review_version":1}