{"id":"3ffa1048-ae04-414a-b9dd-3d25612880d7","arxiv_id":"2504.16244","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":4.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":1,"one_line_summary":"ASCM-SC stratifies donor units by neighbor-treatment status to estimate direct, total, and spillover effects under interference; in the Chilean COVID-19 application, most confidence intervals include zero.","lead":"The paper extends the augmented synthetic control method to settings where treatment in one area can spill over into neighboring areas, using separate control pools for directly and indirectly affected units. The method is applied to COVID-19 lockdowns in Chilean municipalities, where most estimated effects come with confidence intervals that include zero.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The binary neighbor-summary q(i)t is the load-bearing assumption; if spillover magnitude depends on how many neighbors are treated, ASCM-SC's donor-pool exchangeability fails and the Section 4.1 bias-reduction claim is untested.","rationale":"The reader's weakest-assumption diagnosis is the same one that appears most load-bearing to me: the binary summary of neighbor treatment. The paper's own application covariates encode a continuous measure of neighbor lockdown exposure, so the assumption that q = 1 adequately captures spillover is doubtful in exactly the empirical setting the method is meant to serve. If the assumption fails, the S01 donor pool mixes units with materially different counterfactual outcomes, and the claimed bias reduction of ASCM-SC over naive ASCM is not established. I agree with the reader that the paper should be conditional rather than accepted outright: the method may still be salvageable by extending q to ordinal or continuous summaries or by matching on dose, and a targeted simulation would determine whether the current formulation supports the headline claim. I do not see a more fundamental flaw; the stratified-control idea is clearly described, and the simulation evidence, though incomplete without Web Appendices and code, is consistent with the stated claim when the binary-q assumption holds. Therefore the reader's CONDITIONAL verdict remains appropriate.","tokens_in":13774,"tokens_out":4298,"duration_ms":44440,"concrete_test":"Simulate the Section 4.1 DGPs with a dose-dependent spillover term: Yit(0, q) = μit + β qit + εit, where qit is the proportion of neighbors under lockdown, and keep own treatment binary as in the paper. Apply ASCM-SC exactly as specified (q = 1 if any neighbor treated) and naive ASCM across β in {0, 0.1, 0.3, 0.5}, using the same number of units, pre-periods, and covariates as the reported simulation. If ASCM-SC's direct-effect bias is not uniformly smaller than naive ASCM's, or if its bias grows with β, the binary neighbor-summary is the load-bearing misspecification and the Section 4.1 claim needs qualification.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central bias-reduction claim for ASCM-SC rests on exchangeability between the stratified donor pools and the treated unit's counterfactual. Section 3.2.1 collapses the entire neighbor treatment vector into a binary q(i)t = 1 if any neighbor is treated and 0 otherwise. Under SNTVA with this summary, Yit(0,1) is treated as a single potential outcome value for every untreated unit with at least one treated neighbor, regardless of which or how many neighbors are treated. The direct-effect estimator (6) then treats all j in S01 \\ N(i) as valid donors for Yit(0,1). If spillover intensity depends on the number or proportion of treated neighbors, S01 units with one lightly affected neighbor and S01 units with several heavily affected neighbors have different Y(0,1) values; no single synthetic control can represent the target counterfactual, and the direct-effect estimate is biased. This is not a remote possibility: the Chile application includes 'proportion of the population in neighboring municipalities under lockdown' as a covariate (Section 5), indicating the authors themselves model spillover as dose-dependent. The manuscript text does not report simulations in which the spillover magnitude varies with the number or proportion of treated neighbors, so the setting that stresses this assumption appears untested. Section 3.3.3's claim that stratified controls preserve the error bounds is also asserted rather than proved, but the binary-q assumption is more fundamental because it threatens the estimator's validity directly, not just the stated error bound.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper extends the Augmented Synthetic Control Method (ASCM) to settings with interference by proposing Ridge ASCM with stratified controls (ASCM-SC). Under the Stable Neighbor Treatment Value Assumption with a binary summary of whether any neighbor is treated, the authors define direct, total, and spillover effects and estimate them by using stratified donor pools: untreated units with at least one treated neighbor (S01) for the direct effect, and untreated units with no treated neighbors (S00) for the total effect. The method is evaluated in simulations calibrated to a COVID-19 lockdown application in Chile, comparing ASCM-SC to naive ASCM, iSCM, and a modification from Li et al. (2022), and is then applied to estimate lockdown effects on the instantaneous reproduction number in seven Chilean comunas.","tokens_in":14020,"tokens_out":5375,"duration_ms":53745,"significance":"If the proposed method works as claimed, it offers a practical extension of synthetic control methods to a common policy setting where multiple spatially clustered units are treated and spillover between treated and untreated units is plausible. The paper is clearly written, the estimators are defined precisely, and the simulation results support the bias-reduction claim under the stated data-generating processes. The authors also provide an empirical application that illustrates the method's use. However, the central contribution depends heavily on the binary neighbor-treatment summary, which may be unrealistic for the motivating application. The claim that the stratified design preserves the validity of existing error bounds is not substantiated. These issues limit the generalizability of the method and the strength of the empirical conclusions.","major_comments":[{"comment":"The binary neighbor-summary assumption q(i)t = 1 if any neighbor is treated is load-bearing for the direct-effect estimator. Under this assumption, all units in S01 are treated as exchangeable for Y(0,1), regardless of how many or which neighbors are treated. In the motivating application, the covariate 'proportion of the population in neighboring municipalities under lockdown' (Section 5) indicates that spillover intensity is likely dose-dependent. If spillover magnitude varies with the number or proportion of treated neighbors, then S01 donors with different neighbor-treatment configurations have different underlying Y(0,1) values, and the synthetic control formed from them cannot represent the treated unit's counterfactual. The paper does not report simulations in which the spillover effect depends on the number or proportion of treated neighbors, so the central bias-reduction claim is not tested under this realistic violation. Please either relax the binary summary (e.g., by stratifying on the proportion of treated neighbors) or explicitly characterize the conditions under which ASCM-SC remains approximately unbiased.","section":"Section 3.2.1, Eq. (6)"},{"comment":"The claim that stratifying controls 'preserves the independence assumption' and therefore 'ensures that the bias bounds described above remain valid' is asserted without proof. The error bounds in Ben-Michael et al. (2021) assume independent errors across units and time. Interference generally induces dependence through shared latent factors, spatial correlation, or the treatment status of neighbors; stratifying the donor pool does not by itself restore independence. Even under the binary q(i)t assumption, outcomes of the treated unit and the S01 donors may be correlated. Provide a formal argument for why the existing bounds apply, or soften the claim to a heuristic statement. This is important because the conformal inference procedure in Section 3.2.4 also relies on exchangeability of residuals, which is not established in the presence of interference.","section":"Section 3.3.3"},{"comment":"The simulation study design should be described in the main text, at least to the extent of clarifying whether the DGP generates spillover effects that are binary (any neighbor treated) or dose-dependent (varying with the proportion or number of treated neighbors). As written, the phrase 'spillover effect due to some portion of neighbors being treated' together with the binary q(i)t notation creates ambiguity. If all simulations use binary spillover, the claim 'in general, the bias of the direct effect estimate is smaller when using ASCM-SC compared to naive ASCM' is established only for that special case. The paper would be strengthened by including a DGP where spillover intensity depends on the proportion of treated neighbors, which is the setting suggested by the Chile application.","section":"Section 4.1"}],"minor_comments":[{"comment":"The author name 'T aylor' should be 'Taylor' and the municipality '˜Nu˜ noa' should be 'Ñuñoa'.","section":"Title page / author line"},{"comment":"The sentence 'Throughout, P indicates PN i=2' is malformed and should be rewritten, e.g., 'Throughout, summations are over i = 2,...,N unless noted otherwise.'","section":"Section 3.1.1"},{"comment":"The definition of q(i)t is introduced as 'a binary function' but is later described with 'some proportion of N(i)' in the text and in the estimand definitions. Since q(i)t is binary, please consistently clarify that q(i)t = 1 if at least one neighbor is treated, and avoid the word 'proportion' in the definition of the effects.","section":"Section 3.2.1"},{"comment":"The paper states that Web Appendices A and B are 'available upon request.' For a journal submission, these materials should be included as supplementary files for the review process.","section":"Supplementary materials"},{"comment":"The pre-intervention period is described as 'the week prior to the first wave of lockdowns.' Please state the exact number of pre-treatment time points T0 used in the analysis, as this affects the reliability of the ASCM fit and the conformal inference procedure.","section":"Section 5"},{"comment":"The captions for Figures 1–5 should specify what each panel displays in more detail, particularly the units of the outcome (log(Rt)) and the meaning of 'bias' and 'coverage' in the simulation figures.","section":"Figure captions"}],"recommendation":"major_revision","confidential_remarks":"The paper addresses a timely and practical problem, and the ASCM-SC idea is sensible under the stated assumptions. However, the binary neighbor-summary assumption is a severe restriction that is likely violated in the application, and the paper does not test the method under dose-dependent spillover. The error-bound claim in Section 3.3.3 is also not substantiated. These issues are fixable with additional simulations, a sensitivity analysis, and a more careful discussion of limitations, so I recommend major revision rather than rejection."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Quick take: this is a useful, clean extension of Ridge ASCM to settings with interference, and the main weakness is exactly the binary neighbor summary the authors use. I think it deserves peer review, but the authors need to either defend or relax that assumption.\n\nWhat's new: they stratify the donor pool by the unit's own treatment and whether any neighbor is treated (S11/S01/S00), then run Ridge ASCM within strata to estimate direct, total, and spillover effects. That's a simple and practical idea, and I haven't seen it in exactly this form. The simulation comparisons against naive ASCM and iSCM are appropriate for the claim they make, and the bias reduction for direct effects in the settings they try is plausible. The Chile application is reported honestly, with most confidence intervals including zero and the authors saying so plainly.\n\nThe soft spots are real but not fatal. First, the binary summary q(i)t collapses the entire neighbor treatment vector into \"any neighbor treated or not.\" If spillover magnitude depends on how many or which neighbors are treated, then S01 units are not exchangeable with the counterfactual Y(0,1) for a specific treated unit. Their own analysis uses \"proportion of neighboring population under lockdown\" as a covariate, which suggests dose-dependence. The simulations apparently only include a spillover term that depends on binary q, so the bias-reduction claim is untested in exactly the setting where it would fail. Second, Section 3.3.3 asserts that stratified controls preserve the error bounds from Ben-Michael et al. That is plausible but not proved; interference can still create dependence across units within strata. This is a moderate concern, not fatal, because the main estimator doesn't depend on that bound in an obvious way. Third, no code or web appendix is provided, so the simulation results can't be independently checked.\n\nOverall: this is a solid incremental contribution. It should be sent to a serious referee, and the revision should address the dose-dependence issue, either by generalizing q or by adding simulations where spillover intensity varies with the proportion of treated neighbors.","headline":"A clean and useful extension of Ridge ASCM to interference via stratified donor pools, but the binary neighbor summary is a genuine soft spot that needs addressing before publication.","tokens_in":14621,"tokens_out":2202,"would_cite":false,"duration_ms":21543,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Stratifying synthetic-control donor pools by neighbor treatment status separates the direct, total, and spillover effects of localized lockdowns when neighboring regions are also treated.","keywords":["causal inference","interference","spillover effects","synthetic control","augmented synthetic control","Ridge ASCM","COVID-19 lockdowns","conformal inference"],"falsifier":"Simulate data in which the true potential outcome is $Y_{it}(a, q(i)t)$ with $q(i)t$ equal to the proportion of treated neighbors, then apply ASCM-SC's binary summary and measure the bias of the direct-effect estimate as the number of treated neighbors varies. If bias rises with neighbor count while the binary version stays low, the estimator's performance is an artifact of the all-or-nothing spillover assumption.","tokens_in":13489,"feed_emoji":"📉","tokens_out":9192,"duration_ms":79558,"temperature":0.7,"pith_summary":"This paper asks how to estimate the effect of a localized lockdown when lockdowns in neighboring regions also affect transmission. It extends the augmented synthetic control method by splitting control units into two strata: untreated units with no treated neighbor, and untreated units with at least one treated neighbor. Separate synthetic controls are built from each stratum, so that the direct effect of lockdown, the total effect of lockdown plus neighbor lockdown, and the spillover effect from neighbors are estimated without blending one into another. A reader should care because this is exactly the situation of many real policy rollouts: several nearby jurisdictions are treated at once, so the usual \"no interference\" donor pool is contaminated.","feed_headline":"Stratified controls curb spillover bias in synthetic control estimates","feed_subtitle":"Splitting donor pools by neighbor exposure separates direct, total, and spillover lockdown effects.","key_machinery":"The engine is the Stable Neighbor Treatment Value Assumption, written $Y_{it}(a,q)$, which says unit $i$'s outcome depends on its own treatment $a$ and a binary summary $q(i)t$ of whether any neighbor is treated. Under that assumption, ASCM-SC partitions post-treatment units into treated-with-treated-neighbor ($S_{11}$), untreated-with-treated-neighbor ($S_{01}$), and untreated-with-no-treated-neighbor ($S_{00}$). For each treated unit, a Ridge-augmented synthetic control is fitted to units in $S_{01} \\setminus N(i)$ to estimate the counterfactual $Y(0,1)$ and hence the direct effect, and to units in $S_{00}$ to estimate $Y(0,0)$ and hence the total effect; subtracting the two estimates gives the spillover effect. The Ridge augmentation corrects pre-treatment imbalance between the treated unit and its donor pool, and conformal inference converts post-treatment residuals into pointwise confidence intervals.","core_discovery":"The paper's central claim is that under interference, the standard synthetic-control donor pool is contaminated: control units that are untreated but have a treated neighbor are experiencing the spillover outcome $Y(0,1)$, not the pure no-treatment outcome $Y(0,0)$, so using them as controls for a treated unit conflates direct and spillover effects. ASCM-SC removes that contamination by stratifying donors: for each treated unit with a treated neighbor, the direct effect is estimated from synthetic controls built on units in $S_{01} \\setminus N(i)$ (untreated, with a treated neighbor, outside the unit's own neighborhood), and the total effect from synthetic controls built on units in $S_{00}$ (untreated, no treated neighbor); the difference estimates the spillover effect. Simulations show the direct-effect estimator generally has smaller bias than naive Ridge ASCM except when no spillover is present, and smaller bias than an existing inclusive synthetic control competitor. In the Chile application, estimated direct and total effects of the first-wave lockdowns on $\\log(R_t)$ vary across the seven treated comunas, with most conformal confidence intervals containing zero.","pith_inferences":["If the true spillover depends on how many or which neighbors are treated, the binary $q(i)t$ summary mis-specifies the counterfactual; a natural test is to re-estimate with $q(i)t$ equal to the proportion of neighboring population under lockdown, a covariate the Chile dataset already contains.","Because ASCM-SC needs enough units in each stratum, its reliability in sparse spatial panels is an open practical question; pooling donors across treated units or stronger ridge regularization might be needed.","The stratified-donor idea is not tied to ASCM specifically and could be grafted onto other panel causal estimators whenever interference is monotone in neighbor exposure, giving a family of spillover-aware estimators."],"forward_implications":["In multi-unit rollouts with spatial clustering, estimates from a naive synthetic control blend the treated unit's own effect with the effect of neighbors' treatment; ASCM-SC keeps these separate by construction.","The direct-effect estimator's bias advantage over naive Ridge ASCM grows as the spillover effect grows, and disappears only when there is no spillover.","Total-effect estimates provide the policy-relevant quantity of going under lockdown together with neighbors, while the difference between total and direct gives the spillover contribution.","Conformal inference intervals used with ASCM-SC maintain at least nominal coverage in the simulations, usually overcovering rather than undercovering.","Applied to the seven Chilean comunas that locked down in late March 2020, the method finds estimated reductions in $R_t$ from direct and neighbor lockdowns that vary across comunas, but most confidence intervals include zero."],"supporting_citations":[{"why":"Introduces Ridge ASCM, its augmented weights, and the error bounds that ASCM-SC inherits for each stratified donor pool.","marker":"Ben-Michael et al. (2021)"},{"why":"Provides the Stable Neighbor Treatment Value Assumption that lets outcomes depend on a unit's own treatment and a summary of neighbor treatment.","marker":"Agarwal et al. (2023)"},{"why":"Supplies the Chile comuna COVID-19 dataset and a previous spillover-aware ASCM variant used as a comparator in simulations.","marker":"Li et al. (2022)"},{"why":"Proposes the inclusive synthetic control method for multiple treated units under interference, used as the main simulation comparator.","marker":"Di Stefano and Mellace (2024)"},{"why":"Establishes the synthetic control estimator and placebo-based logic that ASCM and ASCM-SC build on.","marker":"Abadie et al. (2010)"},{"why":"Provides the conformal inference procedure used to construct confidence intervals for direct and total effects.","marker":"Chernozhukov et al. (2019)"},{"why":"Gives the method for estimating the instantaneous reproduction number used as the outcome in the Chile analysis.","marker":"Cori et al. (2013)"}],"fun_headline_variants":["Stratified synthetic controls parse direct and spillover lockdown effects","Neighbor-aware donor pools separate spillover from direct effects in ASCM","Augmented synthetic control with stratified donors tackles spillover bias","Spillover-aware synthetic control estimates Chile lockdown direct and total effects","Stratified controls in synthetic control method isolate spillover from direct impact"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing assumption is that spillover is all-or-nothing: a unit's outcome responds to whether any neighbor is treated, not to how many or which neighbors are treated, so units in the \"untreated but neighboring a treated unit\" group are interchangeable as controls.","fun_headline_variants_meta":{"raw":{"variants":["Stratified synthetic controls parse direct and spillover lockdown effects","Neighbor-aware donor pools separate spillover from direct effects in ASCM","Augmented synthetic control with stratified donors tackles spillover bias","Spillover-aware synthetic control estimates Chile lockdown direct and total effects","Stratified controls in synthetic control method isolate spillover from direct impact"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000518,"raw_usage":{"total_tokens":2530,"prompt_tokens":984,"completion_tokens":1546,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":600,"completion_tokens_details":{"reasoning_tokens":1456}},"tokens_in":600,"tokens_out":1546,"duration_ms":12382,"temperature":1.0,"reasoning_tokens":1456,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-16T11:08:34.369705+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Simulate data in which the true potential outcome is $Y_{it}(a, q(i)t)$ with $q(i)t$ equal to the proportion of treated neighbors, then apply ASCM-SC's binary summary and measure the bias of the direct-effect estimate as the number of treated neighbors varies. If bias rises with neighbor count while the binary version stays low, the estimator's performance is an artifact of the all-or-nothing spillover assumption.","supporting_citations":[{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Introduces Ridge ASCM, its augmented weights, and the error bounds that ASCM-SC inherits for each stratified donor pool."},{"cited_title":"H., Shah, D., and Yu, C","cited_arxiv_id":null,"evidence_quote":"Provides the Stable Neighbor Treatment Value Assumption that lets outcomes depend on a unit's own treatment and a summary of neighbor treatment."},{"cited_title":"and Mellace, G","cited_arxiv_id":null,"evidence_quote":"Proposes the inclusive synthetic control method for multiple treated units under interference, used as the main simulation comparator."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Establishes the synthetic control estimator and placebo-based logic that ASCM and ASCM-SC build on."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Provides the conformal inference procedure used to construct confidence intervals for direct and total effects."},{"cited_title":"M., Fraser, C., and Cauchemez, S","cited_arxiv_id":null,"evidence_quote":"Gives the method for estimating the instantaneous reproduction number used as the outcome in the Chile analysis."}],"review_version":1}