{"id":"e6f3db5e-3717-495d-9b71-0316b9123348","arxiv_id":"2507.02072","paper_version":1,"verdict":"REJECT","confidence":"MODERATE","novelty_score":3.0,"correctness_risk":"high","formal_verification":"none","parameter_count":4,"one_line_summary":"A two-stage ABC-rejection-plus-random-forest algorithm is proposed for faster parameter inference, but its efficiency statistics do not match the stated definition and its posterior validity is unsupported.","lead":"This paper introduces ABC-RF-rejection, a two-stage method that trains a random forest classifier to pre-screen parameter samples before running costly simulations in Approximate Bayesian Computation. It claims major simulation-efficiency gains for epidemic models, but the efficiency numbers are internally inconsistent and the method's posterior approximation is not justified.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The RF-filtered sample is treated as an ABC posterior without correcting for selection bias or missing mass; the 'comparable accuracy' claim is therefore unsupported.","rationale":"The reader's weakest assumption is the selection bias in the second stage, and that is exactly the load-bearing issue. The paper's novelty claim and efficiency accounting also have problems, but those could be revised without changing the method. The posterior validity problem is fundamental: without reweighting or a demonstration that the RF-filtered sample has the same support and distribution as the ABC target, the central claim of 'comparable accuracy' is unsupported. The reported narrower posteriors are a red flag rather than evidence of improvement, since truncation of the prior by an RF threshold would produce the same symptom. The proposed coverage test would settle the matter empirically: if the posterior intervals under ABC-RF-rejection do not achieve nominal coverage, the method does not maintain accuracy comparable to standard ABC rejection. My read therefore does not change the reader's verdict of REJECT; the manuscript would need a theoretical analysis of the bias or a rigorous empirical validation (such as coverage across repeated simulations) before the accuracy claim can be accepted.","tokens_in":8162,"tokens_out":6984,"duration_ms":80931,"concrete_test":"Using the SIR model of Section 3.1, simulate a new observed dataset with known true parameters and run the ABC-RF-rejection algorithm exactly as specified (Stage 1: 10^5 draws, threshold 2000; Stage 2: 10^6 candidates, RF from randomForest, retain p_hat >= 0.75). Construct 95% credible intervals from the reported posterior (all retained particles, or the subset with ss < 2000). Repeat for 500 simulated datasets and compute the empirical coverage of the true (beta, gamma). If coverage departs from 95% by more than simulation error, the posterior is biased. Compare with standard ABC rejection using the same simulation budget; if ABC-RF-rejection coverage is substantially lower, the 'comparable accuracy' claim fails.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central accuracy claim requires the second-stage retained particles to approximate the ABC posterior. In the algorithm, Stage 2 samples particles from the prior but retains only those with RF-predicted acceptance probability above a threshold (Sections 3.2, 3.3, 3.4). The RF score is a function of the parameters, so the proposal distribution is p(theta) * I(p_hat(theta) >= t), not the prior p(theta). The final sample is then either (i) all retained particles, including the roughly 4% in Case 1 with ss >= 2000 that fail the ABC tolerance, or (ii) the subset with ss < 2000, which is the ABC-filtered version of the RF-proposal. In case (ii), the accepted particles are drawn from p(theta) * I(p_hat >= t) * I(ss < eps), whereas the ABC target is p(theta) * I(ss < eps). The RF truncation can exclude parts of the true acceptance region, so the support of the empirical posterior is restricted to high-predicted-probability parameters; importance weighting cannot restore missing mass if the acceptance region is not contained in the selected set. The paper provides no theoretical correction, no coverage analysis, and no comparison to a reference posterior. The reported 'notably narrower posterior distributions' (Figs. 1H-I) are consistent with truncation bias, not with improved precision. The accuracy claim is therefore unsubstantiated.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes ABC-RF-rejection, a two-stage algorithm for parameter inference in simulation-based models. In Stage 1, standard ABC rejection generates a labelled dataset of accepted/rejected particles, which is used to train a Random Forest classifier. In Stage 2, a much larger set of particles is drawn from the prior, and only those with high predicted acceptance probability are retained for simulation and treated as an approximate posterior. The method is evaluated on a deterministic SIR epidemic model, a stochastic spatial epidemic model, and applied to cassava brown streak disease data from Nakasongola district, Uganda. The authors claim substantial gains in computational efficiency while maintaining inference accuracy comparable to standard ABC rejection.","tokens_in":8442,"tokens_out":5308,"duration_ms":56689,"significance":"The idea of using a supervised classifier to focus simulation effort in ABC is potentially valuable for expensive epidemiological models, and the COVID-19/CBSD application is practically relevant. However, as presented, the evidence does not support the central claims. The reported efficiency numbers are internally inconsistent with the simulation counts, and the second-stage sample is not a valid ABC posterior because the RF filter truncates the parameter space without any bias correction. The paper also lacks any reference comparison or calibration check for the posterior approximations. If the methodological validity gap were addressed and the efficiency figures corrected, the approach could be a useful contribution; in its current form, the contribution is not established.","major_comments":[{"comment":"The efficiency definition in the Discussion states the ratio of posterior-contributing samples to total simulations, but the reported values (0.19, 0.06, 0.05) do not match the counts in the case studies. For Case 1, with 100,000 Stage-1 and 1,000,000 Stage-2 simulations and a posterior of 2,368 particles, the efficiency is approximately 0.0022, not 0.19. Case 2 gives 867/10,100,000 ≈ 0.000086, and Case 3 gives 562/11,000,000 ≈ 0.000051. The central efficiency claim is therefore unsupported by the reported numbers.","section":"Section 4 and Section 3.2"},{"comment":"The retained particles after Stage 2 constitute a sample from p(theta)·I(p_hat(theta) ≥ t)·I(ss < eps) (or from p(theta)·I(p_hat(theta) ≥ t) if the ABC filter is not applied), not from the ABC posterior p(theta)·I(ss < eps). The RF classifier restricts the support of the proposal to high predicted acceptance regions, and no importance weighting or bias correction is applied. The 'notably narrower posterior distributions' in Figures 1H–I are consistent with truncation bias rather than improved precision. The paper provides no theoretical or empirical justification that the filtered sample is a valid approximation of the ABC posterior, so the claim of 'comparable accuracy' is not supported.","section":"Section 2 and Sections 3.2–3.4"},{"comment":"The algorithm description is ambiguous about whether the final posterior sample consists of all 2,368 retained particles or only the subset with ss < 2000. If the former, the posterior includes particles violating the ABC tolerance; if the latter, the distribution is an ABC-filtered version of the RF-truncated proposal. In either case, no reference posterior from standard ABC is shown for comparison, and no coverage or calibration analysis is reported. The accuracy claim rests entirely on descriptive comparisons of posterior means.","section":"Section 3.2"},{"comment":"In Case 2, only 867 of 4,473 particles selected by the RF pass the summary statistic criteria; in Case 3, only 562 of 1,205 pass. These low pass rates indicate that the RF classifier's predicted acceptance probabilities are poorly calibrated, which further undermines the claim that the method focuses computation on high-probability regions. The efficiency gains over standard ABC are marginal or negative once the low pass rates are accounted for.","section":"Sections 3.3 and 3.4"}],"minor_comments":[{"comment":"The first summary statistic is defined in Eq. (8) but labeled ss2; it should be ss1. The second summary statistic is described in the text but never given an explicit equation.","section":"Section 3.3.2, Eq. (8)"},{"comment":"The prior uses the text 'epsilon' while Eq. (6) uses the symbol ϵ; use consistent notation throughout.","section":"Section 3.4.2"},{"comment":"The text in Section 3.3.2 says 'Marginal posterior distributions are shown in Figure 2F', but the figure caption lists 2F as 'Summary statistics of particles accepted after the stage 2' and 2G as the marginal posteriors. The figure numbering and/or callouts need correction.","section":"Figure 2"},{"comment":"The paper would benefit from citing and comparing against existing machine-learning-based ABC methods, such as Raynal et al. (2019) 'ABC random forests', to properly position the novelty of the proposed approach.","section":"Introduction and Discussion"},{"comment":"The acceptance probability threshold (0.75 in Case 1, 0.5 in Cases 2 and 3) is described as subjective; a sensitivity analysis across a range of thresholds would help justify the choice and show robustness of the results.","section":"Section 3.2 and Section 3.4.2"}],"recommendation":"reject","confidential_remarks":"The manuscript's central claims are not supported by the evidence: the efficiency figures are arithmetically inconsistent, and the algorithm as described does not produce a valid ABC posterior. The methodological flaw is fundamental and would require a major redesign or substantial additional theoretical and empirical work to fix. The paper appears not yet ready for publication. I see no issue with the relevance of the topic to the journal's scope, but the technical deficiencies are decisive."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"The core idea here is not new: Pudlo et al. (2016) and Raynal et al. (2019) already use random forests in ABC, and the paper's framing of ABC-RF-rejection as novel is misleading. What is new is the specific application to epidemic models, including a real CBSD dataset from Uganda. The summary statistics designed for sparse surveillance data are thoughtful, and the three case studies are clearly described. The CBSD application is a genuine attempt to infer parameters from limited field data, which is worthwhile.\n\nThe paper has two load-bearing problems. First, the efficiency numbers are internally inconsistent. Case 1 reports an efficiency of 0.19, but the paper's own counts (100,000 stage-1 simulations plus ~2,368 stage-2 simulations, with ~2,273 passing the threshold) give an efficiency of about 0.02. Similarly, case 2 reports 0.06 versus about 0.008, and case 3 reports 0.05 versus about 0.0006. The abstract's claim of substantial computational gains is not supported by the numbers as reported. Second, the posterior approximation is never justified. The final sample is drawn from the prior truncated by the RF prediction and then by the ABC tolerance, which is not the ABC posterior. If the RF's predicted acceptance probability excludes parts of the true acceptance region, the missing mass cannot be recovered by any weighting scheme. The paper's observation of 'notably narrower posterior distributions' is consistent with truncation bias, not improved precision. The paper also misses the key ABC-RF references, which matters for the novelty claim.\n\nThe CBSD application is interesting, and a revision that corrects the efficiency accounting, addresses the selection bias, and positions the work against the existing ABC-RF literature could be worth considering. As it stands, the central claims do not hold up. I would not cite this paper, but I would send it to peer review because the flaws are substantive and correctable, and the application domain is relevant. It could also serve as a useful reading-group example of how to check an efficiency claim.","headline":"Two-stage ABC-RF hybrid with an interesting CBSD application, but the efficiency numbers are internally inconsistent and the posterior is biased by RF truncation.","tokens_in":8968,"tokens_out":2803,"would_cite":false,"duration_ms":31837,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":["62F15","62M30","62P10"],"pacs":[],"model":"deepseek-v4-flash","headline":"The paper claims that ABC-RF-rejection matches standard ABC rejection accuracy while using far fewer simulations, by training a Random Forest on a first ABC pass to pre-screen candidate parameter sets.","keywords":["Approximate Bayesian Computation","Random Forest","parameter estimation","epidemic modelling","SIR model","stochastic spatial model","cassava brown streak disease","simulation-based inference"],"falsifier":"Run standard ABC rejection on the SIR model with a very low threshold and large simulation budget to obtain a near-exact posterior, and compare its mean and width with the ABC-RF-rejection posterior; if the two differ by more than Monte Carlo error, the claim of comparable accuracy is falsified.","tokens_in":7934,"feed_emoji":"🦠","tokens_out":9003,"duration_ms":84367,"temperature":0.7,"pith_summary":"The paper introduces ABC-RF-rejection, a two-stage approach that first runs ABC rejection to label particles and trains a Random Forest classifier on those labels, then uses the classifier to pre-screen a much larger pool of candidates before simulating them. The authors test it on a deterministic SIR model and a stochastic spatial epidemic model, reporting substantial reductions in the number of simulations needed while recovering true parameters and producing posterior distributions comparable to standard ABC rejection. They also apply the method to estimate parameters for cassava brown streak disease spread in a district of Uganda, where only sparse annual surveillance data are available.","feed_headline":"Two-stage ABC-RF method slashes simulations in epidemic models","feed_subtitle":"A classifier trained on a first ABC pass pre-filters parameters, cutting simulation costs while keeping accuracy","key_machinery":"The central object is the ABC-RF-rejection pipeline: a two-stage algorithm in which (1) a standard ABC rejection sampler produces a labelled training set of particles (accepted/rejected according to a summary-statistic threshold), (2) a Random Forest classifier learns the mapping from model parameters to acceptance probability, and (3) a second, larger sample of prior draws is filtered by the classifier's predicted probability threshold, with only the retained particles being simulated and then filtered again by the true ABC criterion. The Random Forest functions as a learned surrogate for the simulation cost, concentrating computational effort in regions of parameter space that the first stage indicates are promising.","core_discovery":"The central claim is that a Random Forest classifier trained on the output of an initial ABC rejection stage can act as a cheap surrogate for the simulation-based acceptance test, allowing a second stage to screen a large number of prior draws and simulate only those with high predicted acceptance probability. The paper reports that this two-stage ABC-RF-rejection scheme recovers true parameters in a synthetic spatial epidemic and yields efficiency—the ratio of posterior particles to total simulations—of 0.19, 0.06, and 0.05 across the three case studies, compared with 0.0035 for standard ABC rejection on the deterministic SIR model and 0.006 for a stochastic spatial model in earlier work. For the CBSD case study, the method produces posterior distributions for the transmission parameters from annual presence/absence survey data in Nakasongola district.","pith_inferences":["Because the second stage selects particles by classifier probability without a bias correction, the reported 'comparable accuracy' holds only if the Random Forest probabilities are well calibrated; an importance-weighting correction would make the posterior approximation exact even with a biased classifier (editorial inference).","The efficiency metric—posterior particles divided by total simulations—does not count the cost of training the classifier or of any rejected second-stage simulations, so the practical speed-up may be lower than the reported ratios for models with expensive simulations (editorial inference).","The same pre-screening idea could be applied to other simulation-based inference schemes, such as sequential Monte Carlo ABC or likelihood-free MCMC, by replacing the Random Forest with any probabilistic surrogate (editorial inference).","A direct empirical check of classifier calibration—comparing the actual acceptance rate among particles with predicted probability above a threshold—would provide a straightforward test of whether the retained sample is a faithful ABC posterior (editorial inference)."],"forward_implications":["In the SIR case study, the algorithm needs about 2,368 simulated posterior particles from roughly 10^5 first-stage plus 10^6 screened second-stage candidates, whereas standard ABC rejection requires 666,972 simulations to obtain the same number of accepted particles, an efficiency gain from 0.0035 to 0.19.","The method recovers the true parameters in the synthetic stochastic spatial epidemic, with the true values falling within the marginal posterior distributions.","The framework accommodates summary statistics tailored to sparse surveillance data, such as annual radial spread and relative infection intensity, rather than requiring full spatial coverage.","For the CBSD case study in Nakasongola district, the algorithm yields posterior distributions for the primary and secondary transmission rates and dispersal scale from annual presence/absence surveys.","Raising the predicted-probability threshold in the second stage further improves efficiency at the cost of fewer retained particles, so the threshold offers a tunable trade-off."],"supporting_citations":[{"why":"Provides the Random Forest ensemble method used as the classifier for predicting acceptance probabilities.","marker":"Breiman, 2001"},{"why":"Establishes the ABC rejection sampling framework for infectious disease modelling that the first stage is built on.","marker":"Minter and Retkute, 2019"},{"why":"Supplies the stochastic simulation algorithm used to generate epidemic trajectories for the spatial model.","marker":"Gillespie, 1976"},{"why":"Supplies the exact stochastic simulation algorithm for coupled events used in the spatial model.","marker":"Gillespie, 1977"},{"why":"Defines the spatially explicit stochastic epidemic model used in case studies 2 and 3.","marker":"Retkute and Gilligan, 2025"},{"why":"Provides the baseline ABC efficiency of 0.006 for a stochastic spatial model of CBSD spread, used for comparison.","marker":"Godding et al., 2023"},{"why":"Supplies the annual CBSD surveillance data for Nakasongola district used in case study 3.","marker":"Alicai et al., 2019"},{"why":"Provides experimental field-trial data used to fit the within-cell logistic growth parameters for CBSV.","marker":"Katono et al., 2015"}],"fun_headline_variants":["ABC-RF-rejection: two-stage screening cuts simulation costs","Random Forest speeds up ABC parameter estimation","Two-stage ABC with RF surrogate: faster inference, same accuracy","RF classifier pre-filters ABC, slashing epidemic model runs","ABC-RF-rejection: efficient parameter estimation for epidemics"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that particles retained by a threshold on the Random Forest's predicted acceptance probability can be treated as an approximate ABC posterior without correcting for the selection bias introduced by the classifier; if the classifier's probability estimates are even slightly biased, the resulting sample is not the ABC posterior and the claimed 'comparable accuracy' is not guaranteed.","fun_headline_variants_meta":{"raw":{"variants":["ABC-RF-rejection: two-stage screening cuts simulation costs","Random Forest speeds up ABC parameter estimation","Two-stage ABC with RF surrogate: faster inference, same accuracy","RF classifier pre-filters ABC, slashing epidemic model runs","ABC-RF-rejection: efficient parameter estimation for epidemics"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000602,"raw_usage":{"total_tokens":2766,"prompt_tokens":855,"completion_tokens":1911,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":471,"completion_tokens_details":{"reasoning_tokens":1831}},"tokens_in":471,"tokens_out":1911,"duration_ms":14308,"temperature":1.0,"reasoning_tokens":1831,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-06T20:38:48.792398+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run standard ABC rejection on the SIR model with a very low threshold and large simulation budget to obtain a near-exact posterior, and compare its mean and width with the ABC-RF-rejection posterior; if the two differ by more than Monte Carlo error, the claim of comparable accuracy is falsified.","supporting_citations":[],"review_version":1}