{"id":"93c2e13a-5bdd-48cf-82e3-9beeaba95f91","arxiv_id":"2606.03012","paper_version":1,"verdict":"UNVERDICTED","confidence":"LOW","novelty_score":7.0,"correctness_risk":"unknown","formal_verification":"none","parameter_count":0,"one_line_summary":"Derives closed-form multi-level asymptotic variance for individual-level OLS in switchback experiments, showing macro shocks create a power floor penalized by cluster size imbalance.","lead":"The paper derives a closed-form asymptotic variance formula for the OLS estimator in switchback experiments and identifies a power floor caused by macro shocks scaled by cluster imbalance. A smart generalist might read it to understand limits on experiment power in platform settings where cluster-time assignments are common.","discovery_kind":"new_method","skeptic_critique":{"model":"grok-4.3","headline":"No significant objection identified","rationale":"Reader's weakest assumption matches the only potential soft spot (accuracy of the multi-level approximation under the paper's error and assignment model). Because the manuscript claims direct verification via derivation and simulation, and no counter-evidence appears, the UNVERDICTED status is not altered by this pass.","tokens_in":1654,"tokens_out":232,"duration_ms":15336,"concrete_test":"Recompute the Monte Carlo variance estimates in the simulation section using the exact parameter grid from the analytical derivation; if the closed-form expression deviates by more than sampling error from the simulated variance for any interior point, the approximation claim weakens.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim is a closed-form multi-level asymptotic variance for the individual-level OLS estimator in switchback designs, with an explicit structural floor on power from macro shocks. The paper states that analytical derivations plus Monte Carlo simulations establish exactness under typical parameters and conservative bounding in extremes. No internal inconsistency, hidden assumption, or regime where the stated conditions fail is visible in the argument structure.","agreement_with_reader":"agree"},"referee_report":{"model":"grok-4.3","summary":"The paper derives a closed-form multi-level asymptotic variance approximation for the individual-level OLS estimator in switchback experiments (treatment assigned at cluster-time level). It identifies a structural floor on statistical power arising because macro-level shocks are multiplicatively penalized by cluster-size imbalance while idiosyncratic noise vanishes with observation density. The formula is claimed to be exact under typical parameters and a conservative upper bound in boundary regimes, with confirmation via analytical derivations and Monte Carlo simulations. Three applications are developed: stratification only partially mitigates the imbalance penalty; variance-reduction methods targeting macro shocks yield larger efficiency gains; and finite-sample power trade-offs between individual-level and cell-level estimators are formalized.","tokens_in":1729,"tokens_out":375,"duration_ms":17592,"significance":"If the multi-level asymptotic derivation holds, the closed-form variance expression directly addresses a documented gap in power calculations for switchback designs common in marketplace settings. Explicit credit is due for the parameter-free structural insight on the macro-shock floor, the Monte Carlo confirmation of exactness under typical regimes, and the three concrete methodological applications that translate the formula into design recommendations. These elements would make the result useful for practitioners budgeting experiments under clustered temporal assignment.","major_comments":[],"minor_comments":[{"comment":"The abstract and introduction should explicitly state the precise error-component model (e.g., the decomposition into macro shock, cluster-time interaction, and idiosyncratic terms) used to obtain the multiplicative penalty factor, so readers can verify the scope of the closed-form result without consulting the full derivation.","section":null},{"comment":"Monte Carlo results would benefit from an additional table or figure panel reporting coverage of the analytic variance estimator (or its conservative bound) across the boundary regimes mentioned in the abstract, rather than only point estimates of power.","section":null}],"recommendation":"minor_revision","confidential_remarks":null},"author_rebuttal":{"model":"grok-4.3","summary":"We thank the referee for their positive evaluation of the paper and the recommendation for minor revision. The report provides no specific major comments to address.","responses":[],"tokens_in":1221,"tokens_out":47,"duration_ms":9550,"standing_objections":[]},"desk_editor":{"model":"grok-4.3","letter":"The main thing here is a usable closed-form variance formula for the individual-level OLS estimator under switchback assignment. That is new relative to prior work on these designs, which apparently lacked an analytic power expression.\n\nThe derivation looks solid on its face. It separates idiosyncratic noise (which averages out with density) from macro shocks (which get hit by cluster imbalance), and the Monte Carlo checks confirm the formula is exact in typical regimes and conservative at the edges. The three applications follow directly: stratification only partially offsets the imbalance penalty, macro-targeted variance reduction beats residual-noise fixes, and the cell-level vs individual-level estimator trade-off gets quantified.\n\nThe soft spots are limited. The multi-level asymptotic approximation rests on the paper's error structure and assignment model; if real platforms have different dependence patterns the numbers could shift, though the authors flag the boundary behavior. No load-bearing gaps appear in the argument as presented.\n\nThis is for people running marketplace or platform experiments who need to set sample sizes and choose designs. It is worth a serious referee because the result is new, the checks are there, and the practical payoff is immediate.","headline":"The paper gives the first closed-form multi-level asymptotic variance for individual OLS in switchback designs and shows the power floor from macro shocks.","tokens_in":2155,"tokens_out":301,"would_cite":true,"duration_ms":14834,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"grok-4.3","headline":"A closed-form variance formula for switchback experiments shows macro shocks create a power floor penalized by cluster imbalance.","keywords":["switchback experiments","power analysis","asymptotic variance","cluster size imbalance","OLS estimator","marketplace experiments","experimental design","variance approximation"],"falsifier":"Monte Carlo simulations or empirical variance calculations under the paper's typical parameter regimes that deviate substantially from the closed-form predictions.","tokens_in":2563,"feed_emoji":"","tokens_out":593,"duration_ms":17192,"temperature":0.7,"pith_summary":"The paper derives a closed-form multi-level asymptotic variance approximation for the individual-level OLS estimator in switchback experiments, where treatment is assigned at the cluster-by-time level. This approximation separates the vanishing contribution of idiosyncratic noise as observation density rises from the persistent multiplicative penalty that macro-level shocks incur when cluster sizes are imbalanced. A sympathetic reader would care because the formula supplies a practical tool for power budgeting in marketplace and platform settings and identifies why simply adding more data cannot overcome certain design limits. The work further shows that stratification only partially mitigates the imbalance penalty and that variance-reduction efforts aimed at macro shocks deliver larger gains than those aimed at residual noise.","feed_headline":"Switchback power hits a floor set by cluster imbalance","feed_subtitle":"Closed-form variance formula shows macro shocks are penalized by uneven clusters even as individual noise disappears with denser data.","key_machinery":"The closed-form multi-level asymptotic variance approximation for the individual-level OLS estimator, which decomposes variance by level and isolates the multiplicative interaction between macro shocks and cluster size imbalance.","core_discovery":"The central claim is that the closed-form multi-level asymptotic variance approximation for the individual-level OLS estimator is exact across typical parameter ranges and serves as a mathematically conservative upper bound in boundary regimes, revealing that idiosyncratic noise vanishes with observation density while macro-level shocks are multiplicatively penalized by cluster size imbalance.","pith_inferences":["Experimenters may need to balance cluster sizes directly rather than rely on post-design adjustments to reach high power.","The formula could guide choices between individual-level and cell-level estimation when both power and bias are considered.","Similar structural power floors may appear in other settings that combine cluster and time dimensions."],"forward_implications":["Stratification and similar advanced assignment designs only partially eliminate the power penalty from cluster size imbalance.","Variance reduction techniques that target macro-level shocks produce larger efficiency gains than those targeting residual idiosyncratic noise.","Finite-sample power trade-offs exist between the individual-level OLS estimator and the cell-level estimator."],"fun_headline_variants":["Cluster imbalance floors switchback power","Macro shocks bound switchback experiment power","New variance formula reveals switchback power floor","Imbalance penalizes switchback power despite denser data","Switchback power faces multiplicative imbalance penalty"],"cache_read_input_tokens":64,"weakest_assumption_plain":"The approximation accurately reflects how macro-level shocks interact with cluster size imbalance under the paper's model of cluster-time assignment and error structure.","fun_headline_variants_meta":{"raw":{"variants":["Cluster imbalance floors switchback power","Macro shocks bound switchback experiment power","New variance formula reveals switchback power floor","Imbalance penalizes switchback power despite denser data","Switchback power faces multiplicative imbalance penalty"]},"model":"grok-4.3","cost_usd":0.003457,"raw_usage":{"total_tokens":1789,"prompt_tokens":597,"num_sources_used":0,"completion_tokens":62,"cost_in_usd_ticks":34574500,"prompt_tokens_details":{"text_tokens":597,"audio_tokens":0,"image_tokens":0,"cached_tokens":256},"completion_tokens_details":{"audio_tokens":0,"reasoning_tokens":1130,"accepted_prediction_tokens":0,"rejected_prediction_tokens":0}},"tokens_in":597,"tokens_out":62,"duration_ms":8750,"temperature":1.0,"reasoning_tokens":1130,"cache_read_input_tokens":256,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-06-28T09:21:50.983074+00:00","model_set":{"reader":"grok-4.3"},"falsifier":"Monte Carlo simulations or empirical variance calculations under the paper's typical parameter regimes that deviate substantially from the closed-form predictions.","supporting_citations":[],"review_version":1}