pith. sign in

arxiv: 2606.04250 · v1 · pith:M2BHEKBXnew · submitted 2026-06-02 · 📊 stat.ME

Locally Equivalent Weights for Multilevel Regression and Poststratification

Ryan Giordano , Alice Cima , Jared Murray , Erin Hartman , Avi Feller This is my paper

Pith reviewed 2026-06-28 08:28 UTC · model grok-4.3

classification 📊 stat.ME
keywords multilevel regression and poststratificationMrPlocally equivalent weightsinfinitesimal jackknifevariance estimationcovariate balanceexponential familyMCMC computation
0
0 comments X

The pith

MrP can be expressed using locally equivalent weights that support standard weighting diagnostics and match the infinitesimal jackknife variance estimator.

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper develops MrP locally equivalent weights (MrPlew) as a way to represent multilevel regression and poststratification estimators in a weighting form that is locally equivalent to the model near the observed data. This allows users of MrP to apply diagnostics like variance estimation, covariate balance checks, and subgroup analysis that are standard for weighting methods, even for nonlinear models such as logistic regression. The authors show that the variance estimator from these weights is asymptotically equivalent to the infinitesimal jackknife for exponential family models. MrPlew is easy to compute from existing MCMC output, making it practical to add to standard MrP workflows.

Core claim

We develop a natural generalization, MrP locally equivalent weights (MrPlew), which represent MrP as a weighting-style estimator that is locally equivalent to calibration weights near the observed responses. This enables a suite of standard weighting diagnostics. We prove the MrPlew-based variance estimator is asymptotically equivalent to the infinitesimal jackknife for common exponential family models, and we introduce a novel class of model checks based on invariance to data perturbations that generalize covariate balance and subgroup contribution to nonlinear models. MrPlew can be computed easily using existing MCMC samples.

What carries the argument

MrP locally equivalent weights (MrPlew): a weighting representation of the MrP estimator that matches it locally near the observed responses, allowing transfer of weighting diagnostics to model-based MrP.

If this is right

  • MrPlew-based variance matches the infinitesimal jackknife asymptotically for exponential family models.
  • Standard weighting diagnostics like covariate balance can be applied to MrP models including logistic regression.
  • Model checks based on invariance to data perturbations are enabled for nonlinear models.
  • MrPlew is computable directly from existing MCMC samples without additional computation.
  • Implied covariate balance can be compared between MrP and raking in applied studies.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • Adopting MrPlew could standardize how practitioners interrogate MrP models beyond just the point estimate.
  • Extensions might allow similar local equivalence for other model classes beyond exponential families.
  • The approach bridges model-based and design-based survey estimation more closely in practice.
  • Software implementations could integrate MrPlew calculation into popular MrP packages.

Load-bearing premise

The local equivalence between the MrP estimator and its weighting representation holds in a neighborhood of the observed responses for the exponential-family models considered.

What would settle it

A numerical check showing that the MrPlew variance estimator differs from the infinitesimal jackknife variance by more than sampling error in a logistic MrP model with known population quantities.

Figures

Figures reproduced from arXiv: 2606.04250 by Alice Cima, Avi Feller, Erin Hartman, Jared Murray, Ryan Giordano.

Figure 1
Figure 1. Figure 1: Balance [PITH_FULL_IMAGE:figures/full_fig_p003_1.png] view at source ↗
Figure 2
Figure 2. Figure 2: Preview of MrP Diagnostics made possible by MrPlew for the Name Change analysis [PITH_FULL_IMAGE:figures/full_fig_p004_2.png] view at source ↗
Figure 3
Figure 3. Figure 3: Comparison of weights [PITH_FULL_IMAGE:figures/full_fig_p023_3.png] view at source ↗
Figure 4
Figure 4. Figure 4: Estimates of the frequentist standard deviation of [PITH_FULL_IMAGE:figures/full_fig_p024_4.png] view at source ↗
Figure 5
Figure 5. Figure 5: Implied covariate balance on raking marginals for the Election Forecasting and Same-Sex Marriage [PITH_FULL_IMAGE:figures/full_fig_p026_5.png] view at source ↗
Figure 7
Figure 7. Figure 7: Subgroup contribution for the Same-Sex Marriage application, with California and Missouri as [PITH_FULL_IMAGE:figures/full_fig_p027_7.png] view at source ↗
Figure 6
Figure 6. Figure 6: Subgroup contribution for Name Change 27 [PITH_FULL_IMAGE:figures/full_fig_p027_6.png] view at source ↗
Figure 8
Figure 8. Figure 8: The effect of adding random and fixed effects for an imbalanced interaction in the Name Change [PITH_FULL_IMAGE:figures/full_fig_p029_8.png] view at source ↗
Figure 9
Figure 9. Figure 9: Balance [PITH_FULL_IMAGE:figures/full_fig_p057_9.png] view at source ↗
Figure 10
Figure 10. Figure 10: Balance 57 [PITH_FULL_IMAGE:figures/full_fig_p057_10.png] view at source ↗
Figure 11
Figure 11. Figure 11: Estimates of the frequentist standard deviation of [PITH_FULL_IMAGE:figures/full_fig_p058_11.png] view at source ↗
Figure 12
Figure 12. Figure 12: Different variability in per-state subgroup contribution weights for Same-Sex Marriage. Each [PITH_FULL_IMAGE:figures/full_fig_p058_12.png] view at source ↗
Figure 13
Figure 13. Figure 13: Refit [PITH_FULL_IMAGE:figures/full_fig_p059_13.png] view at source ↗
Figure 14
Figure 14. Figure 14: Refit E.3 Nonlinearity in Same-Sex Marriage analysis In this section we briefly demonstrate that posterior is in fact locally linear in fig. 14, but that even the least￾perturbed binary vectors leave the domain of linearity quickly due to a large degree of posterior curvature. 59 [PITH_FULL_IMAGE:figures/full_fig_p059_14.png] view at source ↗
Figure 15
Figure 15. Figure 15: Local behavior of the Same-Sex Marriage perturbation. We only consider [PITH_FULL_IMAGE:figures/full_fig_p060_15.png] view at source ↗
read the original abstract

Multilevel regression and poststratification (MrP) has become a workhorse method for estimating population quantities from non-probability surveys, and is the primary model-based alternative to traditional survey calibration weighting methods, such as raking. For simple linear regression models, MrP methods admit ``equivalent weights'', allowing for direct comparisons between MrP and traditional calibration weighting. Such weights, however, have been unavailable for the most widely used MrP models, such as logistic regression. In this paper, we develop a natural generalization, ``MrP locally equivalent weights'' (MrPlew), which represent MrP as a weighting-style estimator that is locally equivalent to calibration weights near the observed responses. This enables a suite of standard weighting diagnostics, including frequentist sampling variability, covariate balance, and subgroup contribution. We formally justify the use of MrPlew in these cases: we prove the MrPlew-based variance estimator is asymptotically equivalent to the infinitesimal jackknife for common exponential family models, and we introduce a novel class of model checks based on invariance to data perturbations that generalize covariate balance and subgroup contribution to nonlinear models. We further show that MrPlew can be computed easily using existing MCMC samples and provide open-source software to compute MrPlew using the output of standard software. We illustrate our approach for several canonical studies that use MrP, including via a logistic regression outcome model, showing that implied covariate balance can sometimes be worse for MrP than for raking. Given the ease of computing, we recommend making MrPlew a standard part of the MrP model interrogation workflow.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit. Tearing a paper down is the easy half of reading it; the pith above is the substance, this is the friction.

Referee Report

3 major / 3 minor

Summary. The paper introduces 'MrP locally equivalent weights' (MrPlew) as a generalization of equivalent weights for multilevel regression and poststratification (MrP) to nonlinear models such as logistic regression. It claims that MrPlew represent the MrP estimator as a locally equivalent weighting estimator near observed responses, enabling standard weighting diagnostics (variance, balance, subgroup contribution). The central technical claims are an asymptotic equivalence between the MrPlew variance estimator and the infinitesimal jackknife for common exponential-family models, invariance-based model checks that generalize covariate balance, and straightforward computation of MrPlew from existing MCMC samples, supported by open-source software and empirical illustrations on canonical MrP studies.

Significance. If the local-equivalence construction and asymptotic equivalence hold, the work provides a practical bridge between model-based MrP and traditional calibration weighting, allowing routine application of weighting diagnostics to nonlinear MrP without refitting. Explicit credit is due for the MCMC-based computation procedure (which reuses standard posterior samples) and the release of open-source software; these lower the barrier to adoption and support reproducibility.

major comments (3)
  1. [§3.2] §3.2, definition of MrPlew via local linearization: the neighborhood in which local equivalence is asserted is stated only qualitatively ('near the observed responses'); a concrete radius or rate condition on the perturbation size would be needed to make the subsequent asymptotic equivalence claim (to the infinitesimal jackknife) fully rigorous for finite samples.
  2. [Theorem 1] Theorem 1 (asymptotic equivalence): the proof sketch relies on a first-order Taylor expansion of the MrP estimator around the observed data; it is not shown whether the remainder term vanishes at the same rate under the multilevel random-effects structure when the number of poststratification cells grows with sample size, which is the regime of primary interest for MrP applications.
  3. [§4.3] §4.3, invariance-based model checks: the claim that these checks 'generalize covariate balance' to nonlinear models is load-bearing for the diagnostic utility argument, yet the paper provides no simulation or theoretical result quantifying power against misspecification relative to standard posterior predictive checks.
minor comments (3)
  1. Notation: the symbol for the poststratification weights W is overloaded between the classical calibration weights and the MrPlew; a distinct symbol would improve readability.
  2. Figure 2 caption: the legend does not indicate whether the 'raking' comparator uses the same poststratification cells as the MrP model.
  3. Software section: the vignette should include an explicit example of extracting MrPlew from brms or rstanarm output rather than only from the authors' wrapper.

Simulated Author's Rebuttal

3 responses · 0 unresolved

We thank the referee for the constructive comments and positive recommendation. We address each major point below and will revise the manuscript accordingly.

read point-by-point responses

  1. Referee: [§3.2] §3.2, definition of MrPlew via local linearization: the neighborhood in which local equivalence is asserted is stated only qualitatively ('near the observed responses'); a concrete radius or rate condition on the perturbation size would be needed to make the subsequent asymptotic equivalence claim (to the infinitesimal jackknife) fully rigorous for finite samples.

    Authors: We agree that a more precise characterization of the local neighborhood strengthens the claim. In revision we will replace the qualitative description with an explicit rate condition: perturbations of size o_p(n^{-1/2}) in the response scale, which is the regime in which the first-order Taylor expansion underlying both the local equivalence and the infinitesimal jackknife equivalence holds. This will be stated formally in §3.2 and referenced in the proof of Theorem 1. revision: yes

  2. Referee: [Theorem 1] Theorem 1 (asymptotic equivalence): the proof sketch relies on a first-order Taylor expansion of the MrP estimator around the observed data; it is not shown whether the remainder term vanishes at the same rate under the multilevel random-effects structure when the number of poststratification cells grows with sample size, which is the regime of primary interest for MrP applications.

    Authors: The observation is correct: the current proof sketch assumes a fixed number of poststratification cells to control the remainder uniformly. We will revise Theorem 1 to state the result explicitly under the fixed-cell regime (common in many MrP applications) and add a remark noting that extensions to growing cells require additional uniformity arguments over the cell-specific random effects that are left for future work. This clarifies the scope without overstating the current proof. revision: partial

  3. Referee: [§4.3] §4.3, invariance-based model checks: the claim that these checks 'generalize covariate balance' to nonlinear models is load-bearing for the diagnostic utility argument, yet the paper provides no simulation or theoretical result quantifying power against misspecification relative to standard posterior predictive checks.

    Authors: We acknowledge the absence of power comparisons. In the revision we will add a brief simulation study in §4.3 (or an appendix) that compares the rejection rates of the invariance-based checks against posterior predictive checks under controlled misspecification (e.g., omitted group-level interactions) for logistic MrP. This will quantify the diagnostic power while keeping the addition modest given the minor-revision recommendation. revision: yes

Circularity Check

0 steps flagged

No significant circularity detected

full rationale

The paper defines MrPlew as a local equivalence construction to MrP near observed responses for exponential-family models, then derives a proof that the resulting variance estimator is asymptotically equivalent to the external infinitesimal jackknife. This is a mathematical result under stated assumptions rather than a reduction by construction. No self-citation chains, fitted inputs renamed as predictions, or ansatzes smuggled via prior work appear in the load-bearing steps. The model-dependence of the weights is explicit by design and does not force the equivalence claim. The derivation is self-contained against external benchmarks.

Axiom & Free-Parameter Ledger

0 free parameters · 1 axioms · 1 invented entities

The central contribution rests on the definition of local equivalence for exponential-family models and the asymptotic equivalence result; no free parameters or invented physical entities are described.

axioms (1)
  • domain assumption The outcome model belongs to the exponential family
    Invoked for the infinitesimal-jackknife equivalence proof.
invented entities (1)
  • MrP locally equivalent weights (MrPlew) no independent evidence
    purpose: Represent MrP as a weighting-style estimator locally equivalent to calibration weights
    Newly defined construct introduced to enable diagnostics for nonlinear models.

pith-pipeline@v0.9.1-grok · 5821 in / 1193 out tokens · 20579 ms · 2026-06-28T08:28:02.321102+00:00 · methodology

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

17 extracted references · 6 canonical work pages

  1. [1]

    Local posterior robustness with parametric priors: Maximum and average sensitivity

    Alexander, M. (2019).Analyzing name changes after marriage using a non-representative survey.url: https://www.monicaalexander.com/posts/2019-08-07-mrp/. Alexander, R. (2023).Telling Stories with Data: With Applications in R. Chapman and Hall/CRC. Basu, S., S. Rao Jammalamadaka, and W. Liu (1996). “Local posterior robustness with parametric priors: Maximum...

    2019

  2. [2]

    Multilevel calibration weighting for survey data

    Hoboken, New Jersey: John Wiley & Sons.isbn: 978-0-471-69693-3. Ben-Michael, Eli, Avi Feller, and Erin Hartman (2024). “Multilevel calibration weighting for survey data”. In:Political Analysis32.1, pp. 65–83. Ben-Michael, Eli, Avi Feller, David A. Hirshberg, et al. (2021). “The Balancing Act in Causal Inference”. arXiv:2110.14831.url:http://arxiv.org/abs/...

    work page doi:10.18637/jss.v080.i01 2024

  3. [3]

    1995.10476572

    — (1986). “Assessment of local influence”. In:Journal of the Royal Statistical Society. Series B (Method- ological), pp. 133–169. DeBell, M. and J. Krosnick (2009). “Computing weights for American national election study survey data”. In:nes012427. Ann Arbor, MI, Palo Alto, CA: ANES Technical Report Series. Deming, W Edwards and Frederick F Stephan (1940)...

    work page doi:10.1080/01621459 1986

  4. [4]

    Struggles with Survey Weighting and Regression Modeling

    Gelman, Andrew (2007). “Struggles with Survey Weighting and Regression Modeling”. In:Statistical Science 22.2, pp. 153–164.doi:10.1214/088342306000000691. Gelman, Andrew, Ginger L. Chew, and Michael Shnaidman (June 2004). “Bayesian Analysis of Serial Dilution Assays”. en. In:Biometrics60.2, pp. 407–417.issn: 0006-341X, 1541-0420.doi:10 . 1111 / j . 0006 -...

    work page doi:10.1214/088342306000000691 2007

  5. [5]

    Local robustness in Bayesian analysis

    — (2000). “Local robustness in Bayesian analysis”. In:Robust Bayesian Analysis. Ed. by D. R. Insua and F. Ruggeri. Vol

    2000

  6. [6]

    Construction of Weights in Surveys: A Review

    Springer Science & Business Media. Haziza, David and Jean-Fran¸ cois Beaumont (2017). “Construction of Weights in Surveys: A Review”. In: Statistical Science32.2, pp. 206–226.doi:10.1214/16-STS608.url:https://doi.org/10.1214/16- STS608. Henderson, H. and S. Searle (1981). “On deriving the inverse of a sum of matrices”. In:SIAM review23.1, pp. 53–60. Huggi...

    work page doi:10.1214/16-sts608.url:https://doi.org/10.1214/16- 2017

  7. [7]

    Estimating state public opinion with multi-level regression and poststratification using R

    Kastellec, J., J. Lax, and J. Phillips (2010). “Estimating state public opinion with multi-level regression and poststratification using R”. In:Unpublished manuscript, Princeton University29.3. Kennedy, Lauren, Aki Vehtari, and Andrew Gelman (2023). “Model validation for aggregate inferences in out-of-sample prediction”. In:arXiv preprint arXiv:2312.06334...

    arXiv 2010

  8. [8]

    Using leave-one-out cross validation (LOO) in a multilevel regression and post- stratification (MRP) workflow: A cautionary tale

    Springer. Kuh, Swen et al. (2024). “Using leave-one-out cross validation (LOO) in a multilevel regression and post- stratification (MRP) workflow: A cautionary tale”. In:Statistics in Medicine43.5, pp. 953–982. Lax, J. and J. Phillips (2009). “Same-sex rights in the states: Public opinion and policy responsiveness”. In: American Political Science Review10...

    2024

  9. [9]

    Calibrated Bayes: An alternative inferential paradigm for official statistics

    — (2012). “Calibrated Bayes: An alternative inferential paradigm for official statistics”. In:Journal of Of- ficial Statistics28.3, pp. 309–372. 35 Lopez-Martin, J., J. Phillips, and A. Gelman (2026).Multilevel regression and poststratification case studies. url:https://juanlopezmartin.github.io(visited on 03/26/2026). Lumley, T. (2024).survey: Analysis o...

    work page doi:10.1111/ajps.12361 2012

  10. [10]

    Minimal dispersion approximately balancing weights: Asymp- totic properties and practical considerations

    Cambridge university press. Wang, Yixin and Jose R Zubizarreta (2020). “Minimal dispersion approximately balancing weights: Asymp- totic properties and practical considerations”. In:Biometrika107.1, pp. 93–105.issn: 14643510.doi: 10.1093/biomet/asz050. arXiv:1705.00998. Wang, Zhenhua, Jae Kwang Kim, and Shu Yang (2018). “An approximate Bayesian inference ...

    work page doi:10.1093/biomet/asz050 2020

  11. [11]

    importance ratio,

    shows that, under mild regularity conditions, 1 NS X i∈[NS] N ψi −(N ψ) 2 →V. Since ˆVof Theorem 4.1 converges, in general, to a different limit than that of the IJ estimator, it cannot be consistent. □ A.2 Details for Example 3.1 It will be convenient to define the following quantities: ˆMxx := 1 NS X ⊺X fMxx := ˆMxx + σ2 NS Σ−1 ˆMxy := 1 NS X ⊺Y. 37 For...

    1981

  12. [12]

    Define ∆a i :=a i(ˆβ)−E P(β|Y) [ai(β)] ∆b :=b( ˆβ)−E P(β|Y) [b(β)] ai(β) :=a i(β)−a i(ˆβ) b(β) :=b(β)−b( ˆβ) abi(β) := ai(β)b(β). We can then rewrite the covariance as CovP(β|Y) (ai(β), b(β)) =E P(β|Y) (ai(β) + ∆a i )(b(β) + ∆b) =E P(β|Y) abi(β) + ∆a i ∆b+ EP(β|Y) [ai(β)] ∆ b + ∆aEP(β|Y) b(β) =E P(β|Y) abi(β) −∆ a i ∆b. By Theorem 1 of Giordano and Broder...

    2024

  13. [13]

    Combining gives NSCovP(β|Y) (ai(β), b(β)) = 1 2 ∇βai(ˆβ)⊺ˆI −1∇βb(ˆβ) + ˜O N −1 S E ab i − E a i E b

    Then, noting that∇ βabi(ˆβ) =0, another application of Giordano and Broderick (2024) Theorem 1 also gives that NSEP(β|Y) abi(β) = 1 2 ∇βai(ˆβ)⊺ˆI −1∇βb(ˆβ) + ˜O N −1 S E ab i , where againE ab i is finitely square-summable with probability approaching one. Combining gives NSCovP(β|Y) (ai(β), b(β)) = 1 2 ∇βai(ˆβ)⊺ˆI −1∇βb(ˆβ) + ˜O N −1 S E ab i − E a i E b...

    2024

  14. [14]

    Define∇ βg(β) the derviative ofg(β). By Theorem 1 of Giordano and Broderick (2024) NSwMrP i εi −ψ i =N SCovP(β|Y) (g(β), A(β ⊺xi)−ˆyiβ⊺xi) By Lemma B.4, we can write NSCovP(β|Y) (g(β), A(β ⊺xi)−ˆyiβ⊺xi) = ∇βg(ˆβ)⊺ ∇1 ηA(ˆβ⊺xi)xi −ˆyixi + ˜Op N −1 E cov i . Additionally, by Theorem 1 of Giordano and Broderick (2024), ˆyi =E P(β|Y) [m(β⊺xj)] =m( ˆβ⊺xj) + ˜O...

    2024

  15. [15]

    Then Giordano and Broderick (2024) Theorem 1 holds uniformly overt∈Ω t in the following sense

    Definition 2). Then Giordano and Broderick (2024) Theorem 1 holds uniformly overt∈Ω t in the following sense. For any target probability0< ρ <1, there existsC ∗ andN ∗ not depending ontfor whichN S > N ∗ implies that sup t∈Ωt EP(β|Y,t) [ϕ(β)]−ϕ( ˆβt)− N −1 S 1 2 ∇2 βϕ(ˆβt)ˆI −1 t + 1 6 ∇βϕ(ˆβt)∇3 β ˆLt(ˆβt) ˆM ≤N −2 S C ∗ (37) with probability at leastρ. ...

    2024

  16. [16]

    sup β∈B∆ f(β) # = EPS(x,y)

    Definition 2). Then, for any probability0< ρ <1there exists anN ∗ andC ∗ such thatN S > N ∗ implies that N2 S sup t∈Ωt KP(β|Y,t) (a(β), b(β), c(β)) ≤C ∗ with probability at leastρ. Proof.First, for each ofa,b, andc, define ∆a :=a( ˆβt)−E P(β|Y,t) [a(β)] a(β) :=a(β)−a( ˆβt), and so on. Note thatE P(β|Y)t [a] =−∆ a. Then KP(β|Y,t) (a(β), b(β), c(β)) = EP(β|...

    2024

  17. [17]

    imbalanced

    A small modification of the proof of Lemma B.2 gives that, for sufficiently largeN S, there exists aγ >0 such that sup δ,r∈(0,δ+)×R ˆℓNS(β∗ δr;δr)−sup β∈∂B∆(δr) ˆℓNS(β;δ, r) ! > γ >0, where here∂B ∆(δr) ={β:∥β−β ∗ δr∥2 ≤∆}. This follows by a uniform law of large numbers applied to bothβand toδ, r. The remainder of the proof of Lemma B.2 is unchanged, givi...

    1994