Pith. sign in

REVIEW 4 major objections 5 minor 20 references

A novel two-stage parameter estimation framework integrating Approximate Bayesian Computation and Machine Learning: The ABC-RF-rejection algorithm

T0 review · 4 major / 5 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read The paper claims that ABC-RF-rejection matches standard ABC rejection accuracy while using far fewer simulations, by training a Random Forest on a first ABC pass to pre-screen candidate parameter sets.

desk verdict Two-stage ABC-RF hybrid with an interesting CBSD application, but the efficiency numbers are internally inconsistent and the posterior is biased by RF truncation. read the letter →

arxiv 2507.02072 v1 pith:JHXSRMPZ submitted 2025-07-02 stat.ME

Renata Retkute , Christopher A. Gilligan This is my paper · ORCID
classification stat.ME MSC 62F1562M3062P10
keywords ApproximateBayesianComputationRandomForestparameterestimationepidemicmodellingSIRmodelstochasticspatialcassavabrownstreakdiseasesimulation-basedinference
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper introduces ABC-RF-rejection, a two-stage approach that first runs ABC rejection to label particles and trains a Random Forest classifier on those labels, then uses the classifier to pre-screen a much larger pool of candidates before simulating them. The authors test it on a deterministic SIR model and a stochastic spatial epidemic model, reporting substantial reductions in the number of simulations needed while recovering true parameters and producing posterior distributions comparable to standard ABC rejection. They also apply the method to estimate parameters for cassava brown streak disease spread in a district of Uganda, where only sparse annual surveillance data are available.

What carries the argument

The central object is the ABC-RF-rejection pipeline: a two-stage algorithm in which (1) a standard ABC rejection sampler produces a labelled training set of particles (accepted/rejected according to a summary-statistic threshold), (2) a Random Forest classifier learns the mapping from model parameters to acceptance probability, and (3) a second, larger sample of prior draws is filtered by the classifier's predicted probability threshold, with only the retained particles being simulated and then filtered again by the true ABC criterion. The Random Forest functions as a learned surrogate for the simulation cost, concentrating computational effort in regions of parameter space that the first stage indicates are promising.

What would settle it

Run standard ABC rejection on the SIR model with a very low threshold and large simulation budget to obtain a near-exact posterior, and compare its mean and width with the ABC-RF-rejection posterior; if the two differ by more than Monte Carlo error, the claim of comparable accuracy is falsified.

Watch

Extended reading notes

Core claim

The central claim is that a Random Forest classifier trained on the output of an initial ABC rejection stage can act as a cheap surrogate for the simulation-based acceptance test, allowing a second stage to screen a large number of prior draws and simulate only those with high predicted acceptance probability. The paper reports that this two-stage ABC-RF-rejection scheme recovers true parameters in a synthetic spatial epidemic and yields efficiency—the ratio of posterior particles to total simulations—of 0.19, 0.06, and 0.05 across the three case studies, compared with 0.0035 for standard ABC rejection on the deterministic SIR model and 0.006 for a stochastic spatial model in earlier work. For the CBSD case study, the method produces posterior distributions for the transmission parameters from annual presence/absence survey data in Nakasongola district.

Load-bearing premise

The load-bearing premise is that particles retained by a threshold on the Random Forest's predicted acceptance probability can be treated as an approximate ABC posterior without correcting for the selection bias introduced by the classifier; if the classifier's probability estimates are even slightly biased, the resulting sample is not the ABC posterior and the claimed 'comparable accuracy' is not guaranteed.

Editorial extensions

If this is right

  • In the SIR case study, the algorithm needs about 2,368 simulated posterior particles from roughly 10^5 first-stage plus 10^6 screened second-stage candidates, whereas standard ABC rejection requires 666,972 simulations to obtain the same number of accepted particles, an efficiency gain from 0.0035 to 0.19.
  • The method recovers the true parameters in the synthetic stochastic spatial epidemic, with the true values falling within the marginal posterior distributions.
  • The framework accommodates summary statistics tailored to sparse surveillance data, such as annual radial spread and relative infection intensity, rather than requiring full spatial coverage.
  • For the CBSD case study in Nakasongola district, the algorithm yields posterior distributions for the primary and secondary transmission rates and dispersal scale from annual presence/absence surveys.
  • Raising the predicted-probability threshold in the second stage further improves efficiency at the cost of fewer retained particles, so the threshold offers a tunable trade-off.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Because the second stage selects particles by classifier probability without a bias correction, the reported 'comparable accuracy' holds only if the Random Forest probabilities are well calibrated; an importance-weighting correction would make the posterior approximation exact even with a biased classifier (editorial inference).
  • The efficiency metric—posterior particles divided by total simulations—does not count the cost of training the classifier or of any rejected second-stage simulations, so the practical speed-up may be lower than the reported ratios for models with expensive simulations (editorial inference).
  • The same pre-screening idea could be applied to other simulation-based inference schemes, such as sequential Monte Carlo ABC or likelihood-free MCMC, by replacing the Random Forest with any probabilistic surrogate (editorial inference).
  • A direct empirical check of classifier calibration—comparing the actual acceptance rate among particles with predicted probability above a threshold—would provide a straightforward test of whether the retained sample is a faithful ABC posterior (editorial inference).
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. The paper proposes ABC-RF-rejection, a two-stage algorithm for parameter inference in simulation-based models. In Stage 1, standard ABC rejection generates a labelled dataset of accepted/rejected particles, which is used to train a Random Forest classifier. In Stage 2, a much larger set of particles is drawn from the prior, and only those with high predicted acceptance probability are retained for simulation and treated as an approximate posterior. The method is evaluated on a deterministic SIR epidemic model, a stochastic spatial epidemic model, and applied to cassava brown streak disease data from Nakasongola district, Uganda. The authors claim substantial gains in computational efficiency while maintaining inference accuracy comparable to standard ABC rejection.

Significance. The idea of using a supervised classifier to focus simulation effort in ABC is potentially valuable for expensive epidemiological models, and the COVID-19/CBSD application is practically relevant. However, as presented, the evidence does not support the central claims. The reported efficiency numbers are internally inconsistent with the simulation counts, and the second-stage sample is not a valid ABC posterior because the RF filter truncates the parameter space without any bias correction. The paper also lacks any reference comparison or calibration check for the posterior approximations. If the methodological validity gap were addressed and the efficiency figures corrected, the approach could be a useful contribution; in its current form, the contribution is not established.

major comments (4)
  1. [Section 4 and Section 3.2] The efficiency definition in the Discussion states the ratio of posterior-contributing samples to total simulations, but the reported values (0.19, 0.06, 0.05) do not match the counts in the case studies. For Case 1, with 100,000 Stage-1 and 1,000,000 Stage-2 simulations and a posterior of 2,368 particles, the efficiency is approximately 0.0022, not 0.19. Case 2 gives 867/10,100,000 ≈ 0.000086, and Case 3 gives 562/11,000,000 ≈ 0.000051. The central efficiency claim is therefore unsupported by the reported numbers.
  2. [Section 2 and Sections 3.2–3.4] The retained particles after Stage 2 constitute a sample from p(theta)·I(p_hat(theta) ≥ t)·I(ss < eps) (or from p(theta)·I(p_hat(theta) ≥ t) if the ABC filter is not applied), not from the ABC posterior p(theta)·I(ss < eps). The RF classifier restricts the support of the proposal to high predicted acceptance regions, and no importance weighting or bias correction is applied. The 'notably narrower posterior distributions' in Figures 1H–I are consistent with truncation bias rather than improved precision. The paper provides no theoretical or empirical justification that the filtered sample is a valid approximation of the ABC posterior, so the claim of 'comparable accuracy' is not supported.
  3. [Section 3.2] The algorithm description is ambiguous about whether the final posterior sample consists of all 2,368 retained particles or only the subset with ss < 2000. If the former, the posterior includes particles violating the ABC tolerance; if the latter, the distribution is an ABC-filtered version of the RF-truncated proposal. In either case, no reference posterior from standard ABC is shown for comparison, and no coverage or calibration analysis is reported. The accuracy claim rests entirely on descriptive comparisons of posterior means.
  4. [Sections 3.3 and 3.4] In Case 2, only 867 of 4,473 particles selected by the RF pass the summary statistic criteria; in Case 3, only 562 of 1,205 pass. These low pass rates indicate that the RF classifier's predicted acceptance probabilities are poorly calibrated, which further undermines the claim that the method focuses computation on high-probability regions. The efficiency gains over standard ABC are marginal or negative once the low pass rates are accounted for.
minor comments (5)
  1. [Section 3.3.2, Eq. (8)] The first summary statistic is defined in Eq. (8) but labeled ss2; it should be ss1. The second summary statistic is described in the text but never given an explicit equation.
  2. [Section 3.4.2] The prior uses the text 'epsilon' while Eq. (6) uses the symbol ϵ; use consistent notation throughout.
  3. [Figure 2] The text in Section 3.3.2 says 'Marginal posterior distributions are shown in Figure 2F', but the figure caption lists 2F as 'Summary statistics of particles accepted after the stage 2' and 2G as the marginal posteriors. The figure numbering and/or callouts need correction.
  4. [Introduction and Discussion] The paper would benefit from citing and comparing against existing machine-learning-based ABC methods, such as Raynal et al. (2019) 'ABC random forests', to properly position the novelty of the proposed approach.
  5. [Section 3.2 and Section 3.4.2] The acceptance probability threshold (0.75 in Case 1, 0.5 in Cases 2 and 3) is described as subjective; a sensitivity analysis across a range of thresholds would help justify the choice and show robustness of the results.

Circularity Check

2 steps flagged · score 5.0 of 10

The second-stage 'posterior' is defined by the RF's own fitted acceptance scores rather than by the ABC criterion, and no correction is applied; the efficiency and precision comparisons are therefore partially self-referential.

  1. fitted input called prediction [Section 2 (Overview) and Section 3.2 (Case study 1)]
    "The Random Forest model is trained using the labelled dataset generated by the ABC rejection step. ... The trained Random Forest model is then used to predict the probability of acceptance for each new particle. Only those particles with high predicted acceptance probabilities are used for further simulation and posterior approximation. ... In the second stage, we sampled 10 6 new particles, predicted acceptance probabilities using the trained model (Fig. 1C), and retained particles with predicted probability ≥ 0.75 as approximate posterior samples."

    The RF's 'predicted probability of acceptance' is a fitted classifier score derived from the first-stage ABC acceptance labels, not an independent ABC calculation. Because this score is then used as the selection rule that defines the 'approximate posterior samples,' the resulting posterior is a function of the fitted classifier rather than of the ABC acceptance criterion. No importance reweighting or coverage correction is applied, so comparing the output with standard ABC and attributing 'greater precision' to the method attributes a property of the RF truncation filter to the inference. The prediction step is therefore the fitted decision boundary renamed as a posterior approximation.

  2. self definitional [Section 3.2 (Case study 1) and Section 4 (Discussion)]
    "Of the 2,368 selected particles, over 96% had ss <2000 (Fig. 1E), demonstrating high efficiency ... Defining efficiency as the ratio of parameter samples contributing to the posterior distribution relative to the total number of simulations required across both stages, the algorithm achieved efficiencies of 0.19, 0.06, and 0.05 across the three case studies."

    The efficiency numerator counts all RF-selected particles as 'parameter samples contributing to the posterior distribution,' even though the paper itself reports that over 96% (not 100%) of them satisfy the ABC threshold ss<2000 in Case 1. The posterior is thus defined by the RF probability threshold, and the efficiency comparison counts particles that would be rejected by standard ABC. The claimed gain is therefore not measured against the ABC posterior but is built into the definition of which samples are counted, making the efficiency comparison circular by construction.

full rationale

There is no load-bearing self-citation: the stochastic model from Retkute and Gilligan (2025) is used as a testbed, not as an argument for the algorithm's validity. The core problem is that the second-stage selection by RF-predicted acceptance probability is never corrected by importance weighting or coverage analysis, so the final 'posterior' is the RF-truncated proposal rather than the ABC posterior. The accuracy and efficiency claims therefore contain a self-referential element, because the output distribution is defined by the fitted classifier scores and because some retained particles fail the ABC criterion. However, the paper does run real simulations and compares output with standard ABC, so the central claim is not simply equivalent to its input; the issue is partial circularity and selection bias rather than a pure restatement of the input.

Assumptions & free parameters 4 free parameters · 3 assumptions · 0 invented entities

The central efficiency and accuracy claims rest on subjective thresholds and on an unproven equivalence between RF-filtered samples and ABC posterior samples. No new physical or statistical entities are invented.

free parameters (4)
  • RF acceptance probability threshold = 0.75 in case 1, 0.5 in cases 2 and 3
    Chosen by hand; controls which particles are retained for posterior estimation, directly affecting the final set and the efficiency calculation.
  • Summary statistic tolerance thresholds = 2000 (SIR), 100 (case 2), 0.1 (case 3)
    User-defined tolerances for acceptance; they define the labels in stage 1 and therefore shape the classifier and the posterior.
  • RF hyperparameters (ntree, mtry) = not specified
    No details are given for the randomForest package settings, leaving the classifier behavior underspecified.
  • Target infection proportion for case 3 = 0.15 plus/minus 0.1
    Set by hand from surveillance data; defines the accepted region for the second summary statistic.
assumptions (3)
  • domain assumption The random forest classifier, trained on a small number of accepted particles, accurately predicts acceptance probability over the entire prior distribution.
    Invoked in stage 2 of all case studies; without this, the filtered samples do not represent the ABC posterior.
  • ad hoc to paper Retaining particles with high predicted acceptance probability yields an unbiased sample from the ABC posterior.
    Never proven; the paper simply labels these samples as the empirical posterior in Sections 3.2, 3.3.2, and 3.4.2.
  • standard math Standard ABC rejection with the chosen threshold yields a valid approximation to the posterior.
    Standard assumption in ABC; used to define labels in stage 1.

how reviews work

0 comments
Cite this review

Pith. "Pith review of A novel two-stage parameter estimation framework integrating Approximate Bayesian Computation and Machine Learning: The ABC-RF-rejection algorithm." pith.science (2026). https://pith.science/paper/JHXSRMPZ

@misc{pith2026250702072,
  author       = {Pith},
  title        = {Pith review of: A novel two-stage parameter estimation framework integrating Approximate Bayesian Computation and Machine Learning: The ABC-RF-rejection algorithm},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/JHXSRMPZ}},
  note         = {Machine review of arXiv:2507.02072}
}
read the original abstract

We introduce a novel two-stage parameter estimation framework designed to improve computational efficiency in settings involving complex, stochastic, or analytically intractable dynamic models. The proposed method, termed \textit{ABC-RF-rejection}, integrates Approximate Bayesian Computation (ABC) rejection sampling with Random Forest (RF) classification to efficiently screen parameter sets that produce simulations consistent with observed data. We evaluate the performance of this approach using both a deterministic Susceptible-Infected-Removed (SIR) epidemic model and a spatially explicit stochastic epidemic model. Results indicate that ABC-RF-rejection achieves substantial gains in computational efficiency while maintaining parameter inference accuracy comparable with standard ABC rejection methods. Finally, we apply the algorithm to estimate parameters governing the spatial spread of cassava brown streak disease (CBSD) in Nakasongola district, Uganda.

Figures

Figures reproduced from arXiv: 2507.02072 by the authors.

Figure 1
Figure 1. (A) Simulated epidemic based on SIR model (black curve) and observed data (red dots). [PITH_FULL_IMAGE:figures/full_fig_p012_1.png] view at source ↗
Figure 2
Figure 2. (A) Spatial distribution of the simulated outbreak. (B) Temporal changes in number [PITH_FULL_IMAGE:figures/full_fig_p013_2.png] view at source ↗
Figure 3
Figure 3. (A) Progression of the fraction of infected hosts over time within cell: data from (Katono [PITH_FULL_IMAGE:figures/full_fig_p014_3.png] view at source ↗

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

20 extracted references · 16 canonical work pages

  1. [6]

    Beaumont, M.A.,

    URL: http://dx.doi.org/10.1038/s41597-019-0334-9, doi:10.1038/s41597-019-0334-9. Beaumont, M.A.,

    work page doi:10.1038/s41597-019-0334-9

  2. [7]

    11 Figure 1: (A) Simulated epidemic based on SIR model (black curve) and observed data (red dots)

    URL: http://dx.doi.org/10.1038/s41597-020-0501-z, doi:10.1038/s41597-020-0501-z. 11 Figure 1: (A) Simulated epidemic based on SIR model (black curve) and observed data (red dots). (B) Sampled 10 5 particles after Stage 1, colours indicate log10 transformed values of summary statistics. True parameter values shown as a red star. (C) Sampled 10 6 particles ...

    work page doi:10.1038/s41597-020-0501-z

  3. [13]

    Katono, K., Alicai, T., Baguma, Y., Edema, R., Bua, A., Omongo, C.A.,

    URL: http://dx.doi.org/10.1038/s41598-023-38819-x, doi:10.1038/s41598-023- 38819-x. Katono, K., Alicai, T., Baguma, Y., Edema, R., Bua, A., Omongo, C.A.,

    work page doi:10.1038/s41598-023-38819-x

  4. [15]

    Szyniszewska, A.M.,

    URL: http://dx.doi.org/10.1214/21-AOAS1486, doi:10.1214/21-aoas1486. Szyniszewska, A.M.,

    work page doi:10.1214/21-aoas1486

  5. [16]

    Epidemics 29, 100368

    Approximate bayesian computation for infectious disease mod- elling. Epidemics 29, 100368. URL: http://dx.doi.org/10.1016/j.epidem.2019.100368, doi:10.1016/j.epidem.2019.100368. Retkute, R., Gilligan, C.A.,

    work page arXiv 2019

  6. [17]

    Retkute, R., Touloupou, P., Bas´ a˜ nez, M.G., Hollingsworth, T.D., Spencer, S.E.F.,

    URL: http://dx.doi.org/10.3389/fpls.2025.1521620, doi:10.3389/fpls.2025.1521620. Retkute, R., Touloupou, P., Bas´ a˜ nez, M.G., Hollingsworth, T.D., Spencer, S.E.F.,

    work page arXiv 2025

  7. [20]

    (B) The model host landscape, representing the fraction of 1 km2 grid cell occupied by cassava, derived from Cas- savaMap (Szyniszewska, 2020)

    (blue and green points); and fitted logistic equation (red line). (B) The model host landscape, representing the fraction of 1 km2 grid cell occupied by cassava, derived from Cas- savaMap (Szyniszewska, 2020). (C) Surveillance data from 2008 to 2017 for Nakasongola district (Alicai et al., 2019). (D) Summary statistics of particles accepted after stage

    work page 2020

  8. [1976]

    Journal of Computational Physics 22, 403–434

    A general method for numerically simulating the stochastic time evolu- tion of coupled chemical reactions. Journal of Computational Physics 22, 403–434. URL: http://dx.doi.org/10.1016/0021-9991(76)90041-3, doi:10.1016/0021-9991(76)90041-3. Gillespie, D.T.,

    doi:10.1016/0021-9991(76)90041-3

Show all 20 references
  1. [1977]

    The Jour- nal of Physical Chemistry 81, 2340–2361

    Exact stochastic simulation of coupled chemical reactions. The Jour- nal of Physical Chemistry 81, 2340–2361. URL: http://dx.doi.org/10.1021/j100540a008, doi:10.1021/j100540a008. Godding, D., Stutt, R.O.J.H., Alicai, T., Abidrabo, P., Okao-Okuja, G., Gilligan, C.A.,

    doi

  2. [2003]

    Proceedings of the National Academy of Sciences 100, 15324–15328

    Markov chain monte carlo with- out likelihoods. Proceedings of the National Academy of Sciences 100, 15324–15328. URL: http://dx.doi.org/10.1073/pnas.0306899100, doi:10.1073/pnas.0306899100. Maruthi, M.N., Hillocks, R.J., Mtunda, K., Raya, M.D., Muhanna, M., Kiozia, H., Rekha,...

    doi

  3. [2005]

    Journal of Phytopathology 153, 307–312

    Transmission of cassava brown streak virus by bemisia tabaci (gennadius). Journal of Phytopathology 153, 307–312. URL: http://dx.doi.org/10.1111/j.1439-0434.2005.00974.x, doi:10.1111/j.1439- 0434.2005.00974.x. 10 McQuaid, C.F., van den Bosch, F., Szyniszewska, A., Alicai, T., ...

    2005

  4. [2007]

    Plant Disease 91, 24–29

    Re-emergence of cassava brown streak disease in uganda. Plant Disease 91, 24–29. URL: http://dx.doi.org/10.1094/PD-91-0024, doi:10.1094/pd-91-0024. Alicai, T., Szyniszewska, A.M., Omongo, C.A., Abidrabo, P., Okao-Okuja, G., Baguma, Y., Og- wok, E., Kawuki, R., Esuma, W., Tairo...

    doi

  5. [2010]

    Annual Review of Ecology, Evolution, and Systematics 41, 379–406

    Approximate bayesian computation in evolution and ecol- ogy. Annual Review of Ecology, Evolution, and Systematics 41, 379–406. URL: http://dx.doi.org/10.1146/annurev-ecolsys-102209-144621, doi:10.1146/annurev- ecolsys-102209-144621. 9 Bjørnstad, O.N., Shea, K., Krzywinski, M.,...

    doi

  6. [2015]

    Plant Pathology 65, 299–309

    Cassava brown streak disease and the sustainability of a clean seed system. Plant Pathology 65, 299–309. URL: http://dx.doi.org/10.1111/ppa.12453, doi:10.1111/ppa.12453. Minter, A., Retkute, R.,

    doi

  7. [2016]

    Systematic Biology , syw077URL: http://dx.doi.org/10.1093/sysbio/syw077, doi:10.1093/sysbio/syw077

    Fundamentals and recent developments in approximate bayesian computation. Systematic Biology , syw077URL: http://dx.doi.org/10.1093/sysbio/syw077, doi:10.1093/sysbio/syw077. Marjoram, P., Molitor, J., Plagnol, V., Tavar´ e, S.,

    doi

  8. [2017]

    PLOS Computational Biology 13, e1005654

    Spatial dynamics and control of a crop pathogen with mixed-mode transmission. PLOS Computational Biology 13, e1005654. URL: http://dx.doi.org/10.1371/journal.pcbi.1005654, doi:10.1371/journal.pcbi.1005654. McQuaid, C.F., Sseruwagi, P., Pariyo, A., van den Bosch, F.,

    doi

  9. [2019]

    Expansion of the cassava brown streak pandemic in uganda revealed by annual field survey data for 2004 to

    2004

  10. [2020]

    Nature Methods 17, 455–456

    Modeling infectious epi- demics. Nature Methods 17, 455–456. URL: http://dx.doi.org/10.1038/s41592-020-0822-z, doi:10.1038/s41592-020-0822-z. Breiman, L.,

    doi

  11. [2023]

    Heliyon 9, e19939

    Cassava production in africa: A panel analysis of the drivers and trends. Heliyon 9, e19939. URL: http://dx.doi.org/10.1016/j.heliyon.2023.e19939, doi:10.1016/j.heliyon.2023.e19939. Alicai, T., Omongo, C.A., Maruthi, M.N., Hillocks, R.J., Baguma, Y., Kawuki, R., Bua, A., Otim-...

    2023 doi

  12. [2025]

    arXiv preprint arXiv:2504.19698

    Advances in approximate bayesian inference for models in epidemiology. arXiv preprint arXiv:2504.19698 . Lintusaari, J., Gutmann, M.U., Dutta, R., Kaski, S., Corander, J.,

    arXiv

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.