REVIEW 3 major objections 6 minor 34 references
Exploring the heterogeneous impacts of Indonesia's conditional cash transfer scheme (PKH) on maternal health care utilisation using instrumental causal forests
T0 review · 3 major / 6 minor · reviewed 2026-08-10 · deepseek-v4-flash
Pith's one-line read Instrumental causal forests show that Indonesia's PKH cash-transfer effects on maternal care vary by village health supply and household poverty, with the largest swings in 2013 post-natal visits.
desk verdict Useful applied IV-forest study of PKH, but the headline negative 2013 post-natal findings rest on an unstated monotonicity assumption; needs a major revision before I'd trust the causal claim. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The central mechanism is the instrumental causal forest, a tree-based estimator from the generalized random-forest family that recursively partitions the covariate space into leaves where the conditional 2SLS estimand $\mathrm{Cov}[Y,Z\mid X]/\mathrm{Cov}[D,Z\mid X]$ is approximately constant, using honest sample splitting and out-of-bag prediction. Here $Y$ is the utilisation outcome, $D$ is actual PKH enrolment, and $Z$ is the randomised offer to live in a treatment subdistrict. The forest returns CLATE estimates for every mother; doubly robust scores aggregate them into an overall LATE, and supplementary analyses (best linear predictors, classification analyses, policy trees) turn the forest output into interpretable statements about which covariates drive heterogeneity: the supply of health workers, household amenities, and survey wave.
What would settle it
Check whether the randomisation predicts outcomes among mothers who never enrolled (never-takers); if it does, the exclusion restriction fails. Also compute the compliance difference P[enrolled|treatment subdistrict, X] − P[enrolled|control subdistrict, X] within finely defined covariate cells; a negative difference in any cell would reveal defiers and invalidate the CLATE interpretation.
Extended reading notes
Core claim
On the paper's own terms, the central discovery is that the effect of PKH enrolment on maternal health-care utilisation is a conditional local average treatment effect (CLATE) that varies substantially with observable covariates, not a single average. Using the randomised subdistrict assignment as an instrument, the authors report that for complier mothers PKH raises the probability of good assisted delivery by about 15–16 percentage points in both 2009 and 2013, but has no significant average effect on facility delivery in either year, and increases the probability of meeting the pre-natal and post-natal visit thresholds only in 2009. Beyond averages, the estimated CLATE distributions span negative and positive values for every outcome, and the 2013 post-natal-visit distribution is the widest, ranging from roughly −0.5 to 0.7. The drivers of this heterogeneity include the per-capita supply of health workers, household poverty (lack of electricity, clean water, septic tank), and survey wave; best-linear-predictor and classification analyses point to supply-side readiness as a key moderator, and policy trees choose health-worker supply variables as splitting criteria in three of four 2013 outcomes.
Load-bearing premise
The estimates are complier causal effects only if the randomisation offer changes health-care use solely through actual enrolment, no mother does the opposite of her assignment, and the 2007 randomisation remains valid through 2013 despite programme expansion and a halved transfer value; the paper asserts relevance and exclusion but does not test monotonicity or spillovers.
Editorial extensions
If this is right
- If the estimates are correct, the average LATE hides a wide distribution: in 2013 some complier mothers are predicted to lower their post-natal-visit attendance, so describing the programme as 'positive on average' would be misleading for those women.
- Supply-side readiness is a first-order moderator: expanding PKH in villages with richer health-worker supply would raise average assisted-delivery gains, while in low-supply villages the cash alone is unlikely to convert into better maternal care.
- The timing of measurement matters: effects on pre- and post-natal visits present in 2009 disappear by 2013, consistent with the transfer shrinking from 14% to 7% of household consumption, so any evaluation window must be stated along with the estimate.
- Policy trees suggest that simple, interpretable rules—such as enrolling only households in areas with above-median health-worker supply—could capture much of the benefit, and these allocation rules are directly testable.
Reading between the lines
- A natural extension is to estimate separate CLATEs by the type of birth attendant available, since the paper's supply variables lump doctors, nurses, midwives, and traditional birth attendants together; splitting them could reveal whether the negative 2013 post-natal effects come from substitution toward traditional attendants.
- The results imply that cost-effectiveness calculations should use the joint distribution of CLATEs, not the LATE, because targeting the most-affected quartile could multiply health benefits per rupiah transferred; the paper stops short of drawing this implication.
- If supply-side readiness is the true mechanism, a replication in a region with uniformly low supply should find near-zero average effects on assisted delivery; that is a concrete test of the paper's interpretation.
- The 2013 policy trees' reliance on health-worker supply suggests demand-side cash transfers and supply-side investment are complements, so a combined intervention bundling PKH with a midwife-incentive payment could be evaluated against PKH alone.
Signed reviews
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper estimates heterogeneous effects of Indonesia's conditional cash transfer programme (PKH) on four maternal health care utilisation outcomes, using random assignment to PKH subdistricts as an instrument for actual enrolment. The authors apply instrumental causal forests to obtain conditional local average treatment effects (CLATEs) for the 2009 and 2013 waves, and then summarise heterogeneity with best linear predictors (BLP), classification analysis (CLAN), and depth-two policy trees. They report positive average effects on good assisted delivery in both years, and on pre-natal and post-natal visit thresholds in 2009 only. The CLATE histograms show wide dispersion, especially for post-natal visits in 2013, where some estimated effects are negative. Supply-side variables and household poverty indicators are claimed to predict the heterogeneity. The appendix provides continuous-outcome robustness checks.
Significance. If the estimates are valid, the paper makes a useful empirical contribution to the CCT literature by moving beyond average effects and by demonstrating an instrumental-forest workflow for a large-scale randomised policy evaluation. The use of a randomised instrument, out-of-bag prediction, cluster-robust standard errors at the subdistrict level, and three complementary characterisations of heterogeneity are clear strengths. The policy trees and CLAN results also connect the statistical findings to actionable targeting questions. However, the causal interpretation of the CLATEs rests on the instrumental-variables assumptions, and the manuscript is currently silent on one of them (monotonicity). Since the headline finding of negative post-natal effects in 2013 is driven by CLATEs whose interpretation requires that assumption, the central claim is not yet fully secured. The paper does not provide code or data, so computational reproducibility cannot be verified, but the methods are standard and externally established.
major comments (3)
- [Section 3.1, Eq. (1)] The identification statement for the CLATE is incomplete. Equation (1) identifies a conditional local average treatment effect only if the instrument satisfies relevance, independence, exclusion, and monotonicity (no defiers). The paper explicitly states relevance and exclusion but never discusses monotonicity. This is not a minor omission: Table 1 shows substantial two-sided noncompliance, with 10% of control-arm mothers enrolled in 2009 and 14% in 2013, while 51-52% of treated-arm mothers were not enrolled. Given the programme expansion after 2007 and the decline in transfer value, the assumption that D_i(1) >= D_i(0) for every mother is not guaranteed. Without monotonicity, Eq. (1) is a weighted average of per-group effects with possibly negative weights, so the negative CLATEs in Figure 2 for 2013 post-natal visits could be an artefact of unstable IV weights rather than evidence that some complier mothers were made less likely to attend post-natal check-ups. The authors should either defend monotonicity in this setting or provide sensitivity analyses, such as bounds, alternative principal-strata assumptions, or a discussion of the direction of possible bias.
- [Section 3.1 and Table 2] The treatment of supply-side variables in the nuisance functions is unclear and potentially consequential. The text in the analytical steps says that the propensity score e(x) is estimated without supply-side variables, while Table 2 labels its last column as 'Used in m(x)' and marks all supply-side variables as 'No'. It is therefore not clear which nuisance functions (m, e, g) include which covariates. If supply-side variables are excluded from e(x) and g(x), and if they are correlated with the instrument or with enrolment conditional on covariates, the residualised treatment and instrument may still be confounded, which would bias the CLATE estimates. Because supply-side variables are central to the paper's heterogeneity claims, this issue is load-bearing. The authors should clarify the exact specification of each nuisance function and, ideally, show robustness to including the full covariate vector in all nuisance functions.
- [Section 4.2 and Figure 3] The BLP and CLAN results are based on many individual significance tests: roughly 30 covariates across four outcomes and two years, with 95% confidence intervals, and no adjustment for multiple testing. Without such adjustment, some of the reported 'significant' predictors of heterogeneity are likely to be false positives. The qualitative agreement across BLP, CLAN, and policy trees is reassuring, but it is not a formal correction. The authors should either report multiple-testing-corrected confidence sets, pre-specify a smaller set of effect modifiers, or explicitly frame the individual coefficient tests as exploratory and highlight only the patterns that replicate across the complementary analyses.
minor comments (6)
- [Section 3.1, Eq. (1)] The notation Cov[Y, Z | X_i = x] is informal; each term should be written as a conditional covariance given X_i = x, not a covariance of conditional objects.
- [Figure 2 caption] The caption refers to 'ATE point estimates' from the AIPTW estimator, but the estimates are LATEs because the analysis uses an instrument. Please use the correct estimand label.
- [Table 2 and Figure 3 notes] The coding of terciles is described inconsistently: Table 2 says 'q1 = highest quantity' while Figure 3 says 'q1 = largest quantity', and Figure B.7 uses q1-q4 with q4 as the reference. Please define the direction of the tercile indicators once and use consistent wording.
- [Section 2.3] Please clarify whether the supply-side covariates are measured at baseline (2007) or at the follow-up waves (2009/2013). If they are measured after treatment, they could be affected by PKH and would be bad controls rather than pre-treatment effect modifiers.
- [Section 4.4] The policy trees are interpreted qualitatively and no uncertainty quantification is attached to the tree structure. Please state explicitly that the trees are descriptive summaries and do not carry confidence statements, or add an appropriate inferential procedure.
- [Throughout] There are several typographical and grammatical errors, for example 'hetreatment effects', 'facilites', and 'painting of pictures of enrolled mothers being typically of worth socioeconomic status'. A careful proofread is needed.
Circularity Check
No significant circularity: the CLATE estimates and heterogeneity findings are data-driven from externally established instrumental forest methodology; the only self-citation (Kreif et al. 2022) is not load-bearing.
full rationale
This is an empirical estimation paper, not a derivation, and I find no step where a 'prediction' reduces to its inputs by construction. The central estimand, the conditional local average treatment effect, is identified through the standard instrumental variable ratio in Eq. (1), following Athey et al. (2019), which is external, machine-checkable literature rather than a self-citation. The instrumental forest is trained on the data, and the reported heterogeneity summaries (histograms, best linear predictors, classification analysis, and policy trees) are explicitly described as assessments of the heterogeneity 'captured by the forest outputs' (Section 3.2, step 4); they are descriptive summaries of the estimated function rather than out-of-sample predictions that could be forced by construction. The one self-citation, Kreif et al. (2022), appears in a list of prior applications of causal forests and is not used to justify the method, the identification strategy, or the interpretation of the results; it is therefore not load-bearing. The skeptic's concern about the unstated monotonicity assumption is a substantive identification or correctness risk: if monotonicity fails, the ratio in Eq. (1) is not the CLATE and the negative post-natal estimates in 2013 could reflect unstable IV weights rather than genuine complier harm. However, that is an omitted assumption, not a circular reduction: the paper does not define the estimand in terms of a fitted parameter, and no equation is equivalent to its own inputs by construction.
Assumptions & free parameters
assumptions (5)
- domain assumption Randomization assignment Z is a valid instrument: relevant, independent of potential outcomes, and affects Y only through enrollment D conditional on X.
- domain assumption Monotonicity (no defiers) and SUTVA/no spillovers across subdistricts hold.
- domain assumption Complete-case sample is representative and missingness is ignorable.
- domain assumption Survey-based outcomes and covariates are measured without systematic error.
- standard math Generalized random forest consistency and honest tree asymptotic theory hold as in Athey et al. (2019).
Cite this review
Pith. "Pith review of Exploring the heterogeneous impacts of Indonesia's conditional cash transfer scheme (PKH) on maternal health care utilisation using instrumental causal forests." pith.science (2026). https://pith.science/paper/7ALM6ZS4
@misc{pith2026250112803,
author = {Pith},
title = {Pith review of: Exploring the heterogeneous impacts of Indonesia's conditional cash transfer scheme (PKH) on maternal health care utilisation using instrumental causal forests},
year = {2026},
howpublished = {\url{https://pith.science/paper/7ALM6ZS4}},
note = {Machine review of arXiv:2501.12803}
}
read the original abstract
This paper uses instrumental causal forests, a novel machine learning method, to explore the treatment effect heterogeneity of Indonesia's conditional cash transfer scheme on maternal health care utilisation. Using randomised programme assignment as an instrument for enrollment in the scheme, we estimate conditional local average treatment effects for four key outcomes: good assisted delivery, delivery in a health care facility, pre-natal visits, and post-natal visits. We find significant treatment effect heterogeneity by supply-side characteristics, even though supply-side readiness was taken into account during programme development. Mothers in areas with more doctors, nurses, and delivery assistants were more likely to benefit from the programme, in terms of increased rates of good assisted delivery outcome. We also find large differences in benefits according to indicators of household poverty and survey wave, reflecting the possible impact of changes in programme design in its later years. The impact on post-natal visits in 2013 displayed the largest heterogeneity among all outcomes, with some women less likely to attend post-natal check ups after receiving the cash transfer in the long term.
Figures
Reference graph
Works this paper leans on
-
[1]
Alatas, V. (2011). Program keluarga harapan: Impact evaluation of indonesia's pilot household conditional cash transfer program. Technical report, The World Bank
work page 2011
-
[2]
Athey, S., Tibshirani, J., and Wager, S. (2019). Generalized random forests. Annals of Statistics , 47(2):1148--1178
2019
-
[3]
Athey, S. and Wager, S. (2019). Estimating treatment effects with causal forests: an application. Observational Studies , 5(2):37--51
work page 2019
-
[4]
and Wager, S
Athey, S. and Wager, S. (2021). Policy learning with observational data. Econometrica , 89(1):133--161
2021
-
[5]
Barber, S. L. and Gertler, P. J. (2009). Empowering women to obtain high quality care: evidence from an evaluation of mexico's conditional cash transfer programme. Health policy and planning , 24(1):18--25
work page 2009
-
[6]
Bastagli, F., Hagen-Zanker, J., Harman, L., Barca, V., Sturge, G., and Schmidt, T. (2019). The impact of cash transfers: A review of the evidence from low- and middle-income countries. Journal of social policy , 48(3):569--594
work page 2019
-
[7]
Bertrand , Cr \'e pon, M., Marguerie, B., Premand, A., and Patrick (2017). Contemporaneous and post-program impacts of a public works program: evidence from C \^o te d' I voire. World Bank
work page 2017
-
[8]
Cahyadi, N., Hanna, R., Olken, B. A., Prima, R. A., Satriawan, E., and Syamsulhakim, E. (2020). Cumulative impacts of conditional cash transfer programs: Experimental evidence from indonesia. American Economic Journal: Economic Policy , 12(4):88--110
work page 2020
Show all 34 references
-
[9]
Chernozhukov, V., Chetverikov, D., Demirer, M., Duflo, E., Hansen, C., Newey, W., and Robins, J. (2018a). Double/debiased machine learning for treatment and structural parameters. The Econometrics Journal , 21(1):C1--C68
2018
-
[10]
Chernozhukov, V., Demirer, M., Duflo, E., and Fern \'a ndez-Val, I. (2018b). Generic machine learning inference on heterogeneous treatment effects in randomized experiments, with an application to immunization in I ndia. Technical Report 24678, National Bureau of Economic Research
2018
-
[11]
E., Benmarhnia, T., Koski, A., and King, N
Cooper, J. E., Benmarhnia, T., Koski, A., and King, N. B. (2020). Cash transfer programs have differential effects on health: A review of the literature from low and middle-income countries. Social science & medicine , 247:112806
2020
-
[12]
Davis, J. M. V. and Heller, S. B. (2017). Using causal forests to predict treatment heterogeneity: an application to summer jobs. American Economic Review , 107(5):546--550
2017
-
[13]
and Schady, N
Fiszbein, A. and Schady, N. R. (2009). Conditional Cash Transfers: Reducing Present and Future Poverty . World Bank Publications
2009
-
[14]
M., Glassman, A., and Todd, J
Gaarder, M. M., Glassman, A., and Todd, J. E. (2010). Conditional cash transfers and health: unpacking the causal chain. Journal of Development Effectiveness , 2(1):6--50
2010
-
[15]
and Saavedra, J
Garc \' a, S. and Saavedra, J. E. (2017). Educational impacts and Cost-Effectiveness of conditional cash transfer programs in developing countries: A Meta-Analysis . Review of educational research , 87(5):921--965
2017
-
[16]
Glassman, A., Duran, D., Fleisher, L., Singer, D., Sturke, R., Angeles, G., Charles, J., Emrey, B., Gleason, J., Mwebsa, W., Saldana, K., Yarrow, K., and Koblinsky, M. (2013). Impact of conditional cash transfers on maternal and newborn health. Journal of health, population, a...
2013
-
[17]
E., and Gaarder, M
Glassman, A., Todd, J. E., and Gaarder, M. (2007). Performance-Based Incentives for Health: Conditional Cash Transfer Programs in Latin America and the Caribbean . Center for Global Development
2007
-
[18]
and Mast, E
Hoffman, I. and Mast, E. (2019). Heterogeneity in the effect of federal spending on local crime: Evidence from causal forests. Regional science and urban economics , 78:103463
2019
-
[19]
and Waddington, H
Kabeer, N. and Waddington, H. (2015). Economic impacts of conditional cash transfer programmes: a systematic review and meta-analysis. Journal of Development Effectiveness , 7(3):290--303
2015
-
[20]
Kennedy, E. H. (2020). Optimal doubly robust estimation of heterogeneous causal effects. https://arxiv.org/abs/2004.14497. Accessed: 2023-2-5
2020 arXiv
-
[21]
C., Lechner, M., and Strittmatter, A
Knaus, M. C., Lechner, M., and Strittmatter, A. (2021). Machine learning estimation of heterogeneous causal effects: empirical monte carlo evidence. The Econometrics Journal , 24(1):134--161
2021
-
[22]
Kreif, N., DiazOrdaz, K., Moreno-Serra, R., Mirelman, A., Hidayat, T., and Suhrcke, M. (2022). Estimating heterogeneous policy impacts using causal machine learning: a case study of health insurance reform in I ndonesia. Health Services & Outcomes Research Methodology , 22(2):192--227
2022
-
[23]
Kusuma, D., Cohen, J., McConnell, M., and Berman, P. (2016). Can cash transfers improve determinants of maternal mortality? evidence from the household and community programs in indonesia. Social science & medicine , 163:10--20
2016
-
[24]
Lagarde, M., Haines, A., and Palmer, N. (2007). Conditional cash transfers for improving uptake of health interventions in low- and middle-income countries: a systematic review. JAMA: the journal of the American Medical Association , 298(16):1900--1910
2007
-
[25]
M., Barham, T., Macours, K., Maluccio, J
Mill \'a n, T. M., Barham, T., Macours, K., Maluccio, J. A., and Stampini, M. (2019). Long-Term impacts of conditional cash transfers: Review of the evidence. The World Bank research observer , 34(1):119--159
2019
-
[26]
S., Flores, R., Olinto, P., and Medina, J
Morris, S. S., Flores, R., Olinto, P., and Medina, J. M. (2004). Monetary incentives in primary health care and effects on use and coverage of preventive health care interventions in rural honduras: cluster randomised trial. The Lancet , 364(9450):2030--2037
2004
-
[27]
O'Neill and Weeks, M. (2018). Causal tree estimation of heterogeneous household response to time-of-use electricity pricing schemes. https://arxiv.org/abs/1810.09179. Accessed: 2023-2-5
2018 arXiv
-
[28]
Owusu-Addo, E., Renzaho, A. M. N., and Smith, B. J. (2018). The impact of cash transfers on social determinants of health and health inequalities in sub-saharan africa: a systematic review. Health policy and planning , 33(5):675--696
2018
-
[29]
Parker, S. W. and Todd, P. E. (2017). Conditional cash transfers: The case of progresa/oportunidades . Journal of economic literature , 55(3):866--915
2017
-
[30]
and Lagarde, M
Ranganathan, M. and Lagarde, M. (2012). Promoting healthy behaviours and improving health outcomes in low and middle income countries: a review of the impact of conditional cash transfer programmes. Preventive medicine , 55 Suppl:S95--S105
2012
-
[31]
Ravallion, M. (2005). Evaluating anti-poverty programs . The World Bank
2005
-
[32]
Robinson, P. M. (1988). Semiparametric econometrics: a survey. Journal of Applied Econometrics , 3(1):35--51
1988
-
[33]
and Chernozhukov, V
Semenova, V. and Chernozhukov, V. (2021). Debiased machine learning of conditional average treatment effects and other causal functions. The Econometrics Journal , 24(2):264--289
2021
-
[34]
BLT temporary unconditional cash transfer: Social assistance program & public expenditure review 2
World Bank (2012). BLT temporary unconditional cash transfer: Social assistance program & public expenditure review 2
2012
Reviewed August 10, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.