pith. sign in

arxiv: 2604.05286 · v1 · submitted 2026-04-07 · 💰 econ.EM

Estimating Long Run Welfare Outcome in Rotating Panel with Grouped Fixed Effects: Application to Poverty Dynamics in Peru

Hongdi Zhao , Seungmin Lee This is my paper

Pith reviewed 2026-05-10 19:32 UTC · model grok-4.3

classification 💰 econ.EM
keywords poverty dynamicsrotating panelgrouped fixed effectsPeruwelfare mobilitysynthetic panelENAHOlong-run outcomes
0
0 comments X

The pith

Grouped fixed effects applied to rotating panels produce poverty transition estimates closer to observed data than synthetic panel methods.

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

Long-term panel data on household welfare is rare, so researchers rely on repeated cross-sections or synthetic panels to track poverty over time. This paper demonstrates that grouped fixed effects can be applied directly to rotating panel surveys to exploit within-household changes while accounting for unobserved heterogeneity through a small set of latent groups. Using Peru's ENAHO survey, the resulting transition probabilities closely match actual observed movements in and out of poverty. The estimates also outperform synthetic panel benchmarks on average and yield an interpretable group structure that distinguishes different patterns of persistence and mobility.

Core claim

The grouped fixed effects estimator, applied to the rotating panel structure of Peru's ENAHO survey, generates poverty transition measures that align closely with observed transitions in the data. One-step-ahead validation exercises, which withhold each household's final observed year, show that predicted transition shares remain near realized shares. Compared with synthetic panel point estimates, the GFE results are closer to the observed transitions on average while also supplying a grouping structure that supports detailed descriptions of long-run poverty persistence and mobility.

What carries the argument

Grouped fixed effects (GFE), which assigns each household to one of a modest number of latent groups with time-invariant membership to absorb unobserved heterogeneity in a rotating panel without requiring long continuous observations on the same units.

If this is right

  • GFE transition shares closely track the realized poverty entry and exit rates visible in the rotating panel.
  • The approach yields smaller average deviations from observed transitions than synthetic panel estimates do.
  • The latent groups produce an interpretable partition that describes distinct patterns of poverty persistence and mobility.
  • One-step-ahead predictions from the fitted model remain accurate for held-out final observations.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • The same GFE setup could be applied to other national rotating panel surveys to obtain comparable long-run welfare measures where true long panels do not exist.
  • Policy analysis could use the estimated group-specific transition matrices to target interventions at subgroups with high persistence.
  • Direct comparisons between GFE and other panel estimators on the same rotating data would clarify when the fixed-membership grouping assumption is most reliable.

Load-bearing premise

A small number of latent groups with fixed membership over time can capture the unobserved heterogeneity that drives poverty dynamics without the grouping process itself biasing the transition estimates.

What would settle it

Re-running the estimator with a substantially different number of groups or with time-varying group membership and finding that the new transition probabilities deviate markedly from the actually observed transitions in the same data.

Figures

Figures reproduced from arXiv: 2604.05286 by Hongdi Zhao, Seungmin Lee.

Figure 1
Figure 1. Figure 1: GFE Estimation Flowchart n_starts itermax neighmax local Start Step 1: Initialization θ⁰, μ⁰ ← pooled OLS; α⁰ ← year means γ_init ← assign each i to best group Set G, n_starts, neighmax, itermax, max_local_iter Step 2: Set n = 1 Initialize neighborhood size Step 3: Neighborhood jump (shake) Randomly reassign n units → γ'; OLS update Step 4: Iterative refinement Repeat Assign → OLS update for s = 1, …, max_… view at source ↗
Figure 2
Figure 2. Figure 2: The trend of poverty status by year Source: Authors’ calculations using ENAHO 2007–2019, restricted to households observed in at least three survey years and to household heads aged 25–55 in all observed years. Note: We applied our sample restriction: keep households that stayed in the survey for at three two years, and restricted the sample to household heads between 25 and 55 years old in any given surve… view at source ↗
Figure 3
Figure 3. Figure 3: Model fit (BIC and RMSE) by different sets of covariates ( [PITH_FULL_IMAGE:figures/full_fig_p024_3.png] view at source ↗
Figure 4
Figure 4. Figure 4: Model fit (BIC and RMSE) by different sets of covariates ( [PITH_FULL_IMAGE:figures/full_fig_p026_4.png] view at source ↗
Figure 5
Figure 5. Figure 5: shows the estimated group-year intercepts 𝑔𝑡 ̂𝛼 for our selected model with 𝐺 = 4, overlaying specification 1 (solid lines) and specification 2 (dotted lines). These intercept paths summarize the evolution of latent welfare components after netting out the contribution of observed covariates and the province fixed effects, so difference in 𝑔𝑡 ̂𝛼 reflect time-varying unobserved heterogeneity that is common … view at source ↗
Figure 6
Figure 6. Figure 6: Poverty rate trends by GFE group (G=4): Specification 1 vs. Specification 2. [PITH_FULL_IMAGE:figures/full_fig_p029_6.png] view at source ↗
Figure 7
Figure 7. Figure 7: Actual vs. predicted poverty rate over time in the test data (G=4) [PITH_FULL_IMAGE:figures/full_fig_p029_7.png] view at source ↗
Figure 8
Figure 8. Figure 8: One-step-ahead poverty transition shares into held-out final observations: actual vs. [PITH_FULL_IMAGE:figures/full_fig_p031_8.png] view at source ↗
Figure 10
Figure 10. Figure 10: Prediction error in poverty transition shares: GFE one-step versus synthetic panel. [PITH_FULL_IMAGE:figures/full_fig_p034_10.png] view at source ↗
Figure 11
Figure 11. Figure 11: Mean completed welfare by GFE group over time ( [PITH_FULL_IMAGE:figures/full_fig_p036_11.png] view at source ↗
Figure 12
Figure 12. Figure 12: BIC vs. RMSE by different sets of covariates (no age restriction) [PITH_FULL_IMAGE:figures/full_fig_p041_12.png] view at source ↗
Figure 13
Figure 13. Figure 13: BIC vs. RMSE by different sets of covariates for selected G (no age restriction) [PITH_FULL_IMAGE:figures/full_fig_p042_13.png] view at source ↗
Figure 14
Figure 14. Figure 14: Estimated group–year intercept paths 𝑔𝑡 ̂𝛼 for 𝐺 = 4 (no age restriction) Data: Estimated group-year intercept paths for G=4 using training data of sample without age restriction. Note: Colors indicate groups. Solid lines are specification 1 and dotted lines are specification 2. ̂𝛼𝑔𝑡 is the estimated group×year intercept from the GFE model (net of covariates). Interpretation should focuses on relative dif… view at source ↗
Figure 15
Figure 15. Figure 15: BIC vs. RMSE by different sets of covariates for selected G (households surveyed [PITH_FULL_IMAGE:figures/full_fig_p047_15.png] view at source ↗
Figure 16
Figure 16. Figure 16: Estimated group–year intercept paths 𝑔𝑡 ̂𝛼 for 𝐺 = 4 (households surveyed at least five years) Data: Estimated group-year intercept paths for G=4 using training data of households stayed in the survey for at least five years. Note: Colors indicate groups. Solid lines are specification 1 and dotted lines are specification 2. ̂𝛼𝑔𝑡 is the estimated group×year intercept from the GFE model (net of covariates).… view at source ↗
read the original abstract

Household welfare dynamics are often difficult to investigate due to lack of long-term panel data. Existing methods, such as pseudo-panel and synthetic panel, offer widely used solutions based on repeated cross-section designs, but they do not exploit within-household variation in rotating panel designs, which provide very useful information for estimating long-run dynamics. This paper applies grouped fixed effects (GFE) to estimate poverty mobility and persistence in a rotating panel setting, using National Household Survey on Living Conditions and Poverty (ENAHO) in Peru. Using observed transitions, we show that GFE-implied poverty transitions closely track the data. In a one-step-ahead validation that holds out each household's final observed year, predicted transition shares remain close to realized transition shares, indicating that the method captures short-run entry and exit dynamics out of sample. When benchmarked against synthetic panel point estimates, the GFE approach delivers transition measures that are closer to observed transitions on average, while also providing an interpretable grouping structure that supports richer descriptions of poverty persistence and mobility.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit. Tearing a paper down is the easy half of reading it; the pith above is the substance, this is the friction.

Referee Report

2 major / 2 minor

Summary. The paper claims that grouped fixed effects (GFE) applied to rotating panel data can estimate long-run poverty dynamics more effectively than synthetic or pseudo-panel methods by exploiting within-household variation. Using Peru's ENAHO survey, it reports that GFE-implied poverty transitions closely track observed transitions, perform well in one-step-ahead hold-out validation of final-year transitions, deliver lower average deviation from observed transitions than synthetic panel benchmarks, and yield an interpretable grouping structure that supports richer descriptions of persistence and mobility.

Significance. If the central results hold after addressing the grouping assumptions, the approach would strengthen empirical work on welfare dynamics in rotating-panel settings common in developing-country surveys, by providing a middle ground between cross-section synthetic panels and true long panels while adding interpretable latent groups. The one-step-ahead validation and direct benchmarking against synthetic panels are explicit strengths that supply some external grounding for the short-run performance claims.

major comments (2)
  1. [GFE estimation procedure] The central claim that GFE produces reliable long-run transition measures rests on the assumption that a modest number of time-invariant groups fully absorb persistent unobserved heterogeneity without the grouping step being contaminated by the binary poverty outcome itself. The manuscript does not report sensitivity of the transition estimates to the number of groups or results from a grouping step that excludes the outcome variable; in rotating panels where most households contribute only 2–4 observations, this leaves open the possibility that short-run autocorrelation is mechanically reproduced while long-run persistence and mobility projections are biased.
  2. [Validation exercise] The one-step-ahead validation (holding out each household's final observed year) shows predicted transition shares close to realized ones, but this tests only short-run entry/exit and does not directly validate the long-run projections that are the paper's primary target; given the rotating design, additional checks such as multi-period hold-outs or comparison of implied long-run stationary distributions against any available longer panels would be needed to support the long-run welfare claims.
minor comments (2)
  1. [Abstract] The abstract states that GFE transitions are 'closer to observed transitions on average' than synthetic panel estimates, but does not report the magnitude of the improvement, standard errors, or the exact metric used for 'on average'; adding these would strengthen the benchmarking claim.
  2. [Results] The description of the grouping structure as 'interpretable' and supportive of 'richer descriptions' would benefit from an explicit example in the results section showing how group membership correlates with observable covariates or transition patterns.

Simulated Author's Rebuttal

2 responses · 1 unresolved

We thank the referee for the constructive and insightful comments, which help strengthen the paper's claims about using grouped fixed effects to estimate long-run poverty dynamics in rotating panels. We address each major comment below and have revised the manuscript to incorporate additional checks where feasible.

read point-by-point responses

  1. Referee: The central claim that GFE produces reliable long-run transition measures rests on the assumption that a modest number of time-invariant groups fully absorb persistent unobserved heterogeneity without the grouping step being contaminated by the binary poverty outcome itself. The manuscript does not report sensitivity of the transition estimates to the number of groups or results from a grouping step that excludes the outcome variable; in rotating panels where most households contribute only 2–4 observations, this leaves open the possibility that short-run autocorrelation is mechanically reproduced while long-run persistence and mobility projections are biased.

    Authors: We agree that robustness to the grouping procedure is essential for supporting the long-run claims. In the revised manuscript we add an appendix reporting transition estimates for 3, 5, and 7 groups; the poverty entry, exit, and persistence probabilities remain qualitatively unchanged. We also implement an alternative grouping step that excludes the poverty outcome and relies only on covariates and initial conditions; the resulting transition matrix is close to the baseline, indicating that short-run autocorrelation is not mechanically driving the groups. These additions reduce the concern that long-run projections are biased. revision: yes

  2. Referee: The one-step-ahead validation (holding out each household's final observed year) shows predicted transition shares close to realized ones, but this tests only short-run entry/exit and does not directly validate the long-run projections that are the paper's primary target; given the rotating design, additional checks such as multi-period hold-outs or comparison of implied long-run stationary distributions against any available longer panels would be needed to support the long-run welfare claims.

    Authors: We acknowledge that the one-step-ahead exercise primarily confirms short-run performance. Because long-run transitions are obtained by iterating the estimated one-period matrix, the short-run validation provides supporting evidence, but we agree additional checks are valuable. The revision now includes multi-period hold-out validation for households observed at least four times (predicting the final two years). We also report the implied stationary distribution and note its consistency with observed cross-sectional poverty rates. Direct comparison to longer panels is not feasible with the ENAHO rotating design. revision: partial

standing simulated objections not resolved
  • Direct comparison of implied long-run stationary distributions against observed longer panels, because the ENAHO rotating panel does not contain households observed over many consecutive years.

Circularity Check

0 steps flagged

No significant circularity detected in derivation or validation chain

full rationale

The paper's claims rest on applying grouped fixed effects to rotating panel data, showing that implied transitions track observed data, and validating via one-step-ahead hold-out of each household's final year (with predicted shares compared to realized) plus benchmarking against synthetic panel estimates. These steps use independent out-of-sample checks and external method comparisons rather than reducing any prediction to a fitted input by construction. No self-definitional equations, load-bearing self-citations, uniqueness theorems from the authors, or ansatz smuggling are present in the provided text. The grouping structure and transition estimates are estimated from the data but subjected to hold-out validation that supplies independent grounding, keeping the chain self-contained.

Axiom & Free-Parameter Ledger

0 free parameters · 1 axioms · 0 invented entities

Review based solely on abstract; full manuscript text unavailable. Likely relies on standard GFE assumptions (fixed group membership, conditional independence given groups) plus survey sampling assumptions for ENAHO. No explicit free parameters or invented entities named in abstract.

axioms (1)
  • domain assumption Grouped fixed effects with time-invariant group membership adequately capture unobserved heterogeneity in poverty dynamics
    Central to applying GFE to rotating panels; invoked implicitly when claiming long-run estimates from short observed spells.

pith-pipeline@v0.9.0 · 5480 in / 1350 out tokens · 44369 ms · 2026-05-10T19:32:41.518911+00:00 · methodology

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

25 extracted references · 25 canonical work pages

  1. [1]

    and Ripollés, J

    Aparicio-Pérez, D. and Ripollés, J. (2025). Disentangling the heterogeneous effect of natural resources on economic growth. Economic Modelling , 142:106927

    work page 2025

  2. [2]

    and Székely, M

    Attanasio, O. and Székely, M. (2001). Going beyond income: Redefining poverty in latin america. In Portrait of the Poor: An Assets-Based Approach , pages 1--43. Inter-American Development Bank

    work page 2001

  3. [3]

    Bai, J. (2009). Panel data models with interactive fixed effects. Econometrica , 77(4):1229--1279

    work page 2009

  4. [4]

    Bailar, B. A. (1975). The effects of rotation group bias on estimates from panel surveys. Journal of the American Statistical Association , 70(349):23--30

    work page 1975

  5. [5]

    and Hoddinott, J

    Baulch, B. and Hoddinott, J. (2000). Economic mobility and poverty dynamics in developing countries. Journal of Development Studies , 36(6):1--24

    work page 2000

  6. [6]

    Bonhomme, S., Lamadon, T., and Manresa, E. (2022). Discretizing unobserved heterogeneity. Econometrica , 90(2):625--643

    work page 2022

  7. [7]

    and Manresa, E

    Bonhomme, S. and Manresa, E. (2015). Grouped patterns of heterogeneity in panel data. Econometrica , 83(3):1147--1184

    work page 2015

  8. [8]

    Carter, M. R. and Barrett, C. B. (2006). The economics of poverty traps and persistent poverty: An asset-based approach. Journal of Development Studies , 42(2):178--199

    work page 2006

  9. [9]

    Cruces, G., Lanjouw, P., Lucchetti, L., Perova, E., Vakis, R., and Viollaz, M. (2015). Estimating poverty transitions using repeated cross-sections: a three-country validation exercise. The Journal of Economic Inequality , 13(2):161--179

    work page 2015

  10. [10]

    and Lanjouw, P

    Dang, H.-A. and Lanjouw, P. (2023). Measuring poverty dynamics with synthetic panels based on repeated cross sections. Oxford Bulletin of Economics and Statistics , 85(3):599--622

    work page 2023

  11. [11]

    Dang, H.-A., Lanjouw, P., Luoto, J., and McKenzie , D. (2014). Using repeated cross-sections to explore movements into and out of poverty. Journal of Development Economics , 107:112--128

    work page 2014

  12. [12]

    Deaton, A. (1985). Panel data from time series of cross-sections. Journal of Econometrics , 30(1):109--126

    work page 1985

  13. [13]

    D’Alberto, R., De Nicolò, S., and Gardini, A. (2025). Estimating poverty transitions from repeated cross-sections: A statistical perspective. Social Indicators Research , 179(3):1143--1164

    work page 2025

  14. [14]

    D., Davis, A

    Hill, T. D., Davis, A. P., Roos, J. M., and French, M. T. (2020). Limitations of fixed-effects models for panel data. Sociological Perspectives , 63(3):357--369

    work page 2020

  15. [15]

    and Jenkins, S

    Hérault, N. and Jenkins, S. P. (2019). How valid are synthetic panel estimates of poverty dynamics? The Journal of Economic Inequality , 17(1):51--76

    work page 2019

  16. [16]

    Peru final version of OECD review of official statistics and statistical system

    INEI (2017). Peru final version of OECD review of official statistics and statistical system

    work page 2017

  17. [17]

    and Siflinger, B

    Janys, L. and Siflinger, B. (2024). Mental health and abortions among young women: time-varying unobserved heterogeneity, health behaviors, and risky decisions. Journal of Econometrics , 238(1):105580

    work page 2024

  18. [18]

    Lucchetti, L., Corral, P., Ham, A., and Garriga, S. (2025). An application of LASSO and multiple imputation techniques to income dynamics with cross-sectional data. Review of Income and Wealth , 71(1):e12693

    work page 2025

  19. [19]

    Millimet, D. L. and Bellemare, M. F. (2025). On the (mis) use of the fixed effects estimator. Oxford Bulletin of Economics and Statistics

    work page 2025

  20. [20]

    Mundlak, Y. (1978). On the pooling of time series and cross section data. Econometrica , 46(1):69--85

    work page 1978

  21. [21]

    and Sardon, S

    Murillo, D. and Sardon, S. (2024). Commodity booms, local state capacity, and development

    work page 2024

  22. [22]

    Oberlander, L., Disdier, A.-C., and Etilé, F. (2017). Globalisation and national trends in nutrition and health: A grouped fixed-effects approach to intercountry heterogeneity. Health Economics , 26(9):1146--1161

    work page 2017

  23. [23]

    Oh, D. H. and Patton, A. J. (2023). Dynamic factor copula models with estimated cluster assignments. Journal of Econometrics , 237(2):105374

    work page 2023

  24. [24]

    Pigini, C., Pionati, A., and Valentini, F. (2025). Grouped fixed effects regularization for binary choice models. arXiv , ( arXiv :2502.06446)

    work page arXiv 2025

  25. [25]

    Van den Brakel, J. A. and Krieg, S. (2015). Dealing with small sample sizes, rotation group bias and discontinuities in a rotating panel design. Survey Methodology , 41(2):267--297

    work page 2015