{"id":"78b3f9d3-b718-4d41-b3bc-47e52087b97a","arxiv_id":"2505.00925","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":7.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":0,"one_line_summary":"In 2-period cluster randomized crossover trials with informative sizes, independence estimating equations always target the intended average treatment effect estimands, while nested exchangeable mixed model estimators do not.","lead":"This paper maps which common statistical estimators in two-period cluster randomized crossover trials are consistent for four different causal estimands when treatment effects depend on cluster, period, or cluster-period sizes. It gives trial analysts a practical guide to avoiding biased estimates when 'informative sizes' are suspected.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"No significant objection identified","rationale":"The reader's verdict of CONDITIONAL is reasonable: the appendix algebra is not machine-checked, and the abstract contains some overstatement. My stress-test pass did not find a flaw in the central consistency mapping itself. The weakest assumption identified by the reader, the iid superpopulation framework, is indeed a boundary condition on all probability-limit statements, but it is explicitly stated in Section 2 and is standard for this literature; it does not undermine the central claim as scoped. The remaining issues, including the delta_2 = 0.6 versus 0.5 discrepancy in the simulation text and the data-availability statement, are reporting problems that do not affect the theoretical estimator-to-estimand results. The NEME variance-component caveat is real but the qualitative conclusion survives because the limiting weights remain data-dependent under any fixed probability limits of the ICC estimators. I therefore recommend no change to the reader's conditional verdict; the paper should be accepted after the noted reporting corrections are made.","tokens_in":60405,"tokens_out":26152,"duration_ms":269813,"concrete_test":"Independently re-derive Equation (17) from Appendix C.1 without skipping the Slutsky/LLN steps, and verify that after imposing S_i independent of Omega the probability limit simplifies to E[sum_{j,k}(Y(1)-Y(0))]/E[sum_j K_ij]. If it does, the central claim about the unweighted IEE estimator is confirmed.","verdict_should_be":"UNCHANGED","load_bearing_attack":"I could not identify a load-bearing concern with the central estimator-to-estimand mapping. The consistency results for the unweighted and weighted IEE estimators, and the data-dependent limits for the NEME estimators, follow from the explicitly stated iid superpopulation model with the sequence indicator independent of potential outcomes and sizes (Section 2; Section 4.1). Under that framework, the probability limits in Equations (17), (19), (22), (24), and (26)-(28) are internally coherent. The finite-population estimands in Table 1 are not the targets of these asymptotic limits unless the superpopulation model is adopted, but this limitation is stated in the manuscript; it is a scope condition rather than a defect in the central argument. The NEME limits are stated for GLS weights, and in practice variance components are estimated; the paper acknowledges that the limit weights depend on the probability limits of the model-based ICC estimators. This is a technical caveat about exact limits under misspecification, not a reason to doubt the qualitative conclusion that NEME is typically inconsistent for the four named estimands. The simulation-text discrepancy about delta_2 appears to be a typographical error in the DGP description rather than a substantive flaw in the theoretical claims.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"This paper studies four target estimands for 2-period cross-sectional cluster randomized crossover (CRXO) trials with continuous outcomes: the individual-average (iATE), cluster-period-average (cpATE), cluster-average (cATE), and period-average (pATE) treatment effects. It formalizes informative cluster, period, and cluster-period sizes, and derives the probability limits of unweighted and inverse-size-weighted versions of the independence estimating equation (IEE), fixed effects (FE), exchangeable mixed effects (EME), and nested exchangeable mixed effects (NEME) estimators under an iid superpopulation model in which sequence assignment is independent of potential outcomes and cluster-period sizes. The central results are: the unweighted IEE is always consistent for the iATE, and its inverse cluster-period, inverse cluster, and inverse period size weighted counterparts are always consistent for the cpATE, cATE, and pATE; EME and weighted EME are consistent for iATE, cpATE, and cATE when Ki1=Ki2; NEME and its weighted versions generally converge to weighted estimands with data-dependent weights and are not consistent for any of the four estimands; and FE results require lambda_i=lambda for all i and often target the pATE or cpATE instead of the iATE or cATE. A simulation study and a reanalysis of the PEPTIC CRXO trial illustrate and apply these results.","tokens_in":60622,"tokens_out":7162,"duration_ms":78572,"significance":"If the results hold, the paper provides a practical and theory-grounded map from common estimators to well-defined causal estimands in CRXO trials, and it sharpens warnings about the use of NEME in the presence of informative sizes. The algebra in Appendices C through F is detailed, and the simulation results are consistent with the stated theory in the specified DGP. The finite-population versus superpopulation distinction is explicitly acknowledged as a scope condition, and the derivations rely on clearly stated assumptions. I could not identify a load-bearing error in the central consistency mapping. The main caveat is that the exact NEME limits are derived for GLS weights with fixed variance components; the manuscript acknowledges the dependence on probability limits of estimated ICCs, which is sufficient for the qualitative conclusion but should be stated more precisely.","major_comments":[],"minor_comments":[{"comment":"The DGP description states delta_2 = 0.6, but the formulas and numerical values immediately afterward use delta_2 = 0.5 (for example, the iATE formula yielding 0.53 uses delta_2 = 0.6, while the following parenthetical says delta_2 = 0.5); please reconcile the two values, since the reported bias results correspond to only one specification.","section":"Section 5.2 and Appendix H"},{"comment":"The probability limit is derived for GLS weights with fixed variance components, while in practice the NEME weights use estimated ICCs; please state explicitly that Eq. (26) is the limit under variance components fixed at their probability limits and that the feasible GLS estimator is assumed to share this limit.","section":"Section 4.2.1, Eq. (26)"},{"comment":"The proof of Proposition 1.2 is described as a proof by induction, but it proceeds by constructing an example; please correct the wording to 'by example' or 'by construction.'","section":"Appendix B"},{"comment":"The statement 'no new data were created or analyzed' conflicts with the reanalysis of the PEPTIC trial dataset in Section 6; please amend the statement to reflect that existing trial data were reanalyzed.","section":"Data Availability Statement"},{"comment":"The NEME covariance matrix block is defined only under Ki1 = Ki2, and this restriction is later used for the NEME results; please flag the equal-within-cluster-period-size assumption at the first definition to avoid confusion for readers who encounter the restriction only in Section 4.","section":"Section 3.3"}],"recommendation":"minor_revision","confidential_remarks":null},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"You should know this paper for its consistency table: it tells you which common estimator in a 2-period cluster randomized crossover trial actually targets which estimand when cluster, period, or cluster-period sizes are informative. The headline result is that unweighted IEE is always consistent for the iATE, its inverse-size weighted counterparts always hit cpATE, cATE, and pATE, while NEME estimators converge to data-dependent weighted estimands that are generally not any of the four named targets. The FE results are the least obvious part: under a constant period-size ratio across clusters, FE targets pATE, not iATE.\n\nThe paper does solid work. The estimand framework is a natural extension of prior P-CRT and PB-CRT work, and the appendix derivations are thorough enough that I could follow the main steps without finding a load-bearing gap. The simulation study is correctly aligned with the theory, and the PEPTIC reanalysis makes the practical stakes concrete. The self-citation is fine; it builds on prior work rather than relitigating it.\n\nSoft spots are proportionate and mostly at the boundaries. The convergence results live in a superpopulation framework with iid clusters and sequence independent of potential outcomes and sizes. That is stated clearly, but it means the finite-population design-based reading of the probability limits is not covered. The NEME limits also assume known variance components; with estimated weights, the exact limits under misspecification are a caveat, though not one that overturns the qualitative conclusion. Minor reporting issues: the data availability statement says no new data were created even though they reanalyze an existing trial, the abstract slightly overstates the generality of the IEE consistency by not highlighting the superpopulation assumption, and there is a typo in the simulation DGP description for delta_2. These are fixable.\n\nWho is this for? Anyone analyzing or planning a 2-period CRXO with variable cluster-period sizes and heterogeneous treatment effects. It will be a useful reference and should go to a serious methods journal rather than being desk rejected. I would cite it and likely bring it to our reading group.","headline":"A careful, useful mapping of CRXO estimators to causal estimands under informative sizes; the consistency table is the real contribution, and the paper deserves serious peer review.","tokens_in":61162,"tokens_out":1086,"would_cite":true,"duration_ms":13579,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":["62F12","62K10"],"pacs":[],"model":"deepseek-v4-flash","headline":"This paper asks what common estimators in two-period cluster randomized crossover (CRXO) trials actually estimate when cluster, period, or cluster-period sizes carry information about treatment effects, and derives which estimators…","keywords":["cluster randomized crossover trials","informative sizes","estimands","independence estimating equation","mixed effects models","fixed effects","consistency","treatment effect heterogeneity"],"falsifier":"Under the paper's own assumptions, simulate a 2-period CRXO with treatment effects that depend on cluster-period size, e.g. $Y_{ijk}(1)-Y_{ijk}(0)=a+bK_{ij}$, and compute the unweighted IEE estimate as $I$ grows; for the central claim to hold, it must converge to $\\mathbb{E}[\\sum_{j,k}(a+bK_{ij})]/\\mathbb{E}[\\sum_j K_{ij}]$, the individual-average limit, not to the equally weighted cluster-period average. A data-generating process satisfying cluster i.i.d. sampling and independent randomization that produces a different probability limit would refute the always-consistency result.","tokens_in":60245,"feed_emoji":"🎯","tokens_out":9779,"duration_ms":87630,"temperature":0.7,"pith_summary":"This paper asks what common estimators in two-period cluster randomized crossover (CRXO) trials actually estimate when cluster, period, or cluster-period sizes carry information about treatment effects. It defines four interpretable estimands—the individual-average, cluster-period-average, cluster-average, and period-average treatment effects—as special cases of a weighted average treatment effect. Its central result is that the unweighted independence estimating equation (IEE) estimator is always consistent for the individual-average estimand, and its inverse cluster-period, inverse cluster, and inverse period size weighted versions are always consistent for the other three estimands. The unweighted and weighted nested exchangeable mixed effects estimators instead converge to data-dependent weighted averages that are generally not any of the four estimands. This gives trial analysts a map for choosing an estimator when informative sizes are plausible, and a warning that some widely used mixed models can answer no clear causal question.","feed_headline":"Unweighted IEE always hits individual-average effect in CRXO trials","feed_subtitle":"In crossover trials with informative sizes, weighted IEE hits its target; nested mixed models miss.","key_machinery":"The central object is the family of weighted average treatment effect estimands, $wATE=\\sum_{i,j,k}w_{ijk}(Y_{ijk}(1)-Y_{ijk}(0))/\\sum_{i,j,k}w_{ijk}$, with weights $1$, $1/K_{ij}$, $1/\\sum_j K_{ij}$, and $1/\\sum_i K_{ij}$ selecting the iATE, cpATE, cATE, and pATE. The consistency proofs expand each estimator's probability limit under the i.i.d. cluster superpopulation assumption and check which limit equals one of these weighted forms. For the nested exchangeable mixed model, the limiting per-cluster weight is proportional to $1/[1+(K_{i-}-1)\\rho_{wp}-K_{i-}\\rho_{bp}]$, so whenever this factor varies with cluster-period size the limit is a data-dependent weighted average outside the named estimand family.","core_discovery":"The paper claims that, in a 2-period, 2-sequence cross-sectional CRXO trial with continuous outcomes and clusters drawn as i.i.d. superpopulation units, the unweighted IEE estimator is always consistent for the iATE, $\\mathbb{E}[\\sum_{j=1}^2\\sum_{k=1}^{K_{ij}}(Y_{ijk}(1)-Y_{ijk}(0))]/\\mathbb{E}[\\sum_{j=1}^2 K_{ij}]$, and the IEEcpw, IEEcw, and IEEpw estimators are always consistent for the cpATE, cATE, and pATE, respectively, regardless of informative sizes. The same is true of the FEcpw estimator for cpATE. The exchangeable mixed effects model is consistent for iATE (and its inverse-size weighted forms for cpATE/cATE) only when cluster-period sizes are equal within clusters, $K_{i1}=K_{i2}$ for all $i$; the fixed effects model generally converges to pATE rather than iATE unless a constant period-size ratio $\\lambda_i=\\lambda$ holds. The nested exchangeable mixed effects model, unweighted or weighted, converges to limits like $\\mathbb{E}\\{ [1+(K_{i-}-1)\\rho_{wp}-K_{i-}\\rho_{bp}]^{-1} \\sum_{j,k}(Y_{ijk}(1)-Y_{ijk}(0))\\}/\\mathbb{E}\\{2K_{i-}[1+(K_{i-}-1)\\rho_{wp}-K_{i-}\\rho_{bp}]^{-1}\\}$, which depends on cluster sizes and on estimated within- and between-period ICCs and is generally not iATE, cpATE, cATE, or pATE. Simulations confirm the derived relative biases, and a reanalysis of a 49-cluster CRXO trial shows NEME estimates drifting from the IEE-based estimates.","pith_inferences":["A natural testable extension is a sensitivity diagnostic that reports IEE, IEEcw, IEEpw, and IEEcpw estimates side by side; under the paper's consistency map, material differences among them indicate which sizes are informative, although the paper explicitly notes it does not provide a formal statistical test.","The FE result suggests that protocols pre-specifying fixed-effects analyses in CRXO trials should also pre-specify the target as pATE (or verify a constant period-size ratio), since the FE estimator's estimand depends on the balance of period sizes rather than on the analyst's intent.","One can quantify how far a NEME estimate is from any named estimand by plugging estimated ICCs and observed cluster sizes into the limiting weight $1/[1+(K_{i-}-1)\\hat{\\rho}_{wp}-K_{i-}\\hat{\\rho}_{bp}]$ and reporting the implied weighted average as a sensitivity target."],"forward_implications":["When informative sizes are plausible, the unweighted IEE estimator gives a defensible estimate of the individual-average treatment effect without requiring a no-informative-sizes assumption.","Weighted IEE estimators (IEEcpw, IEEcw, IEEpw) give defensible estimates of cluster-period, cluster, and period average effects, so a pre-specified analysis can use them to answer each level of hypothesis.","NEME-based estimates should not be interpreted as targeting any of the four estimands if treatment effects vary with cluster, period, or cluster-period size; a notable gap between NEME and IEE estimates may itself signal informative sizes.","Fixed effects and exchangeable mixed models are usable for their intended estimands only under restrictions such as $K_{i1}=K_{i2}$ (or a constant period-size ratio for fixed effects); otherwise an FE analysis may be estimating the period-average rather than the individual-average effect."],"supporting_citations":[{"why":"Shows in parallel cluster randomized trials that unweighted and inverse cluster-size weighted IEE estimates are consistent for iATE and cATE, the result this paper extends to the crossover setting.","marker":"[17]"},{"why":"Establishes the corresponding estimand and estimator results for parallel cluster randomized trials with a baseline period, including the NEME inconsistency that is compared here.","marker":"[18]"},{"why":"Defines the weighted average treatment effect family from which the paper's four estimands are drawn.","marker":"[10]"},{"why":"Provides the inverse-size weighting scheme used to construct the weighted estimators for informative cluster sizes.","marker":"[14]"},{"why":"Introduces the individual-average and cluster-average estimand framework and motivates defining estimands before choosing estimators.","marker":"[11]"},{"why":"Supplies the model-assisted perspective linking model-based point estimators to potential-outcome estimands.","marker":"[9]"},{"why":"Provides the comparison of CRXO analysis methods whose model family the paper examines.","marker":"[6]"},{"why":"Supplies the 49-cluster CRXO trial data used in the reanalysis of hospital length of stay.","marker":"[37]"}],"fun_headline_variants":["Unweighted IEE always consistent for iATE in CRXO","IEE wins: unweighted hits iATE; weighted hits others","Nested mixed models biased under informative sizes","CRXO estimands: which estimators stay honest?"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"All stated probability limits assume clusters are independent, identically distributed draws from an infinite superpopulation and that the treatment sequence is independent of potential outcomes and all cluster-period sizes; the GLS limits also assume the variance components converge to fixed values rather than being re-estimated in a way that shifts the weights.","fun_headline_variants_meta":{"raw":{"variants":["Unweighted IEE always consistent for iATE in CRXO","IEE wins: unweighted hits iATE; weighted hits others","Nested mixed models biased under informative sizes","CRXO estimands: which estimators stay honest?"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000251,"raw_usage":{"total_tokens":1712,"prompt_tokens":1257,"completion_tokens":455,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":873,"completion_tokens_details":{"reasoning_tokens":388}},"tokens_in":873,"tokens_out":455,"duration_ms":4721,"temperature":1.0,"reasoning_tokens":388,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-16T04:31:49.809414+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Under the paper's own assumptions, simulate a 2-period CRXO with treatment effects that depend on cluster-period size, e.g. $Y_{ijk}(1)-Y_{ijk}(0)=a+bK_{ij}$, and compute the unweighted IEE estimate as $I$ grows; for the central claim to hold, it must converge to $\\mathbb{E}[\\sum_{j,k}(a+bK_{ij})]/\\mathbb{E}[\\sum_j K_{ij}]$, the individual-average limit, not to the equally weighted cluster-period average. A data-generating process satisfying cluster i.i.d. sampling and independent randomization that produces a different probability limit would refute the always-consistency result.","supporting_citations":[{"cited_title":"Model-assisted analysis of covariance estimators for stepped wedge clus- ter randomized experiments.Scandinavian Journal of Statistics.2025;52(1):416–446","cited_arxiv_id":null,"evidence_quote":"Defines the weighted average treatment effect family from which the paper's four estimands are drawn."},{"cited_title":"Demystifying estimands in cluster-randomised trials","cited_arxiv_id":"2303.13960","evidence_quote":"Introduces the individual-average and cluster-average estimand framework and motivates defining estimands before choosing estimators."},{"cited_title":"Analysis of cluster randomized cross-over trial data: a comparison of methods.Statistics in Medicine.2007;26(2):274–289","cited_arxiv_id":null,"evidence_quote":"Provides the comparison of CRXO analysis methods whose model family the paper examines."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Supplies the 49-cluster CRXO trial data used in the reanalysis of hospital length of stay."}],"review_version":1}