Pith. sign in

REVIEW 2 major objections 5 minor 33 references

Dice, but don't slice: Optimizing the efficiency of ONEAudit

T0 review · 2 major / 5 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read Tuning ONEAudit's betting tests cuts expected election-audit workloads by 70–85%, while stratified sampling makes them worse.

desk verdict Worth refereeing: the workload savings are real and well demonstrated, but the headline claim needs an error-sensitivity boundary. read the letter →

arxiv 2507.22179 v2 pith:DLBBEK3V submitted 2025-07-29 stat.ME

Jacob V Spertus , Amanda K Glazer , Philip B Stark This is my paper · ORCID
classification stat.ME MSC 62L1062D0562F03
keywords risk-limitingauditsONEAudittestingbybettingKellyuniversalportfoliosstratifiedsamplingelectionauditingoverstatementassorters
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

ONEAudit is a risk-limiting audit method for elections in which voting systems cannot supply a cast-vote record linked to each ballot card; it verifies reported winners by comparing hand-read cards with reference values built from batch totals and whatever records exist. This paper claims that the workload of ONEAudit can be cut sharply by changing the betting rules inside its statistical tests: bet as if the reported batch tallies were true (a priori Kelly betting), or spread bets across many possibilities (a discrete universal portfolio). In simulations spanning very close and lopsided contests, these tuned tests reduce the expected number of cards that must be hand-checked by 70–85% relative to the current implementation. The paper also finds that stratified sampling does not help: it raises expected workloads by about 25% even when using the sharpest stratified methods. If these results hold, routine audits of elections with mixed voting technology become substantially cheaper and faster, and the improved bets shrink the workload for the 2024 San Francisco mayoral race from about 200 cards to about 100.

What carries the argument

The machinery is testing-by-betting with test supermartingales (TSMs), nonnegative supermartingales that start at 1 when the null hypothesis that a reported winner did not actually beat a reported loser is true. ONEAudit maps each card to an overstatement assorter value whose population mean is at most 1/2 exactly when the reported outcome is wrong, so checking the reported winner reduces to rejecting that null. The new bets are the a priori Kelly bet, computed by numerically solving the first-order condition for maximal expected log growth against the population of overstatements built from reported batch tallies, and the discrete universal portfolio, which mixes TSMs over a fixed grid of $D = 100$ bet sizes and approximates the integral form of a universal portfolio by a Riemann sum. The load-bearing identity is that the mixture portfolio's wealth equals an average of individual TSMs, so the mixture is dominated by whichever component bet grows fastest; this is why the portfolio nearly matches the oracle Kelly bet without knowing the true population.

What would settle it

Run the same simulated audit comparison on elections whose reported outcome is correct but whose reported batch totals are wrong by more than half the reported margin, with errors concentrated in a few large batches; if the a priori Kelly bet then requires more hand-checked cards than the universal portfolio, the paper's efficiency claim does not extend to that regime.

Watch

Extended reading notes

Core claim

The paper's central discovery is that ONEAudit's efficiency is governed by the betting strategy used in its test supermartingales, not by whether the sample is stratified. A bet optimized for the population of overstatements implied by the reported batch tallies, called the a priori Kelly bet, produces expected sample sizes almost as small as those of an oracle Kelly bet that knows the true population; a discrete universal portfolio, formed by averaging test supermartingales over a uniform grid of bet sizes, is the next-most efficient and does not rely on the reported tallies being right. Across simulated plurality elections with reported assorter means between 0.505 and 0.60 and varied batch heterogeneity, these two strategies cut expected workloads by 70–85% compared with the default truncated-shrinkage and comparison-optimal bets, and the comparison-optimal bet is often the worst. Stratifying cards into a CVR stratum and a batch stratum and testing with a union-of-intersections sequence never beats the unstratified betting test in the simulations, with expected sample sizes roughly 20% larger under stratification. For the 2024 San Francisco mayoral race, an instant-runoff contest, the 80th percentile of simulated audit workloads falls from about 200 cards with current defaults to about 100 cards with the new bets.

Load-bearing premise

The workload reductions from a priori Kelly betting assume the reported batch tallies are accurate enough that the overstatement population built from them resembles the true population; the San Francisco case study also assumes the cast-vote records faithfully match human readings of the cards.

Editorial extensions

If this is right

  • Jurisdictions using ONEAudit with accurate reported tallies can expect to hand-check roughly a quarter to a third as many cards as under current defaults, cutting both cost and time-to-completion.
  • Because the discrete universal portfolio nearly matches the a priori Kelly bet without trusting the reported tallies, auditors can obtain most of the efficiency gain even when they are unsure of batch-level accuracy.
  • Stratification should be used for legal or logistical reasons only; using it to reduce workload is counterproductive, since it raises expected sample sizes by about 20%.
  • The lower workload makes risk-limiting audits feasible in elections with mixed voting technology, including instant-runoff contests, without requiring a linked cast-vote record for every card.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • If reported batch tallies are badly wrong while the outcome is still correct, the a priori Kelly bet could lose its advantage; a mixture that starts with high weight on a priori Kelly and shifts toward the universal portfolio as sampling proceeds would be a natural robustness fix.
  • The same betting machinery should transfer to other social choice functions, since any assertion expressible as an overstatement assorter population can be tested with these bets; the gains are likely similar whenever reported totals are accurate.
  • A simulation study that varies the size, location, and correlation of tally errors could identify the threshold at which universal-portfolio betting overtakes a priori Kelly betting, an open question the paper explicitly leaves unresolved.
  • Real-world adoption could translate the workload reductions into audits that finish within the post-election canvass period, which may be the practical barrier that determines whether smaller jurisdictions conduct any risk-limiting audit at all.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

2 major / 5 minor

Summary. The manuscript studies how to reduce the expected workload of ONEAudit risk-limiting audits by changing the statistical tests (betting strategies) and by stratified sampling. The authors propose an a priori Kelly bet based on reported batch-level tallies, an approximation to the universal portfolio of Cover, and AGRAPA, and compare them against existing COBRA and truncated-shrinkage betting in simulated elections with varying margins and heterogeneity. They also compare unstratified betting TSMs with stratified UI-TSs. The main quantitative claims are that the new betting strategies reduce expected workloads by 70–85% relative to current ONEAudit implementations, that stratified sampling increases workloads by about 25% on average, and that applying the improved tests to the 2024 San Francisco mayoral race reduces the 80th-percentile sample size from about 200 to about 100 cards.

Significance. If the claims hold, the paper has practical value for election auditing: reducing the number of cards that must be hand-checked lowers the cost and may make risk-limiting audits more widely adopted. The paper is transparent about its simulation design, provides code, compares against external benchmark methods, and explicitly acknowledges its main limitation. The universal-portfolio betting strategy is a credible robustness-improving fallback, and the suggestion to mix a priori Kelly with the universal portfolio is sensible. The central quantitative claims, however, currently depend on an unquantified assumption about the accuracy of reported tallies, and the stratification claim is broader than the evidence presented.

major comments (2)
  1. [Section 5.2, Table 2, Section 7] The headline workload reduction of 70–85% is driven primarily by the a priori Kelly bet, which is constructed from reported batch-level tallies. The only tally-error scenario simulated is one in which the true margin is half the reported margin (\bar A_m halfway between \bar A_c and \eta). The paper states in Section 7: 'We did not determine how much tally error would be necessary to make a priori Kelly betting worse than the universal portfolio or some other strategy that does not rely on the reported results.' This is load-bearing because the setting in which RLAs are most relevant is exactly one where the reported outcome is correct but the tallies may contain nontrivial error. As written, the abstract's reduction claim is not conditioned on a validated error regime. I recommend adding a targeted experiment that varies the magnitude of tally error (and, if feasible, CVR error) and reports the crossing point where AP Kelly's expected workload exceeds that of the universal portfolio, and that the abstract either attribute the reduction claim to the robust universal portfolio or explicitly condition it on a demonstrated error bound.
  2. [Section 5.1, Table 1, Section 7] The conclusion that stratified sampling does not help and increases workloads by about 25% on average is established only for a single stratification scheme (CVR cards versus batch cards) under exactly accurate reported tallies. Section 7 explicitly concedes that stratification by error rate was not examined. Because the abstract makes an unqualified claim that 'stratified sampling does not help,' the scope of the conclusion is overstated. Please either add simulations with error-rate-based strata or qualify the claim to the studied design and data-generating process.
minor comments (5)
  1. [Section 6, Table 3] The San Francisco case study reports 80th-percentile sample sizes estimated from only 100 simulations and treats CVRs as accurate manual-vote records. Please include Monte Carlo standard errors or a brief sensitivity check for the CVR-accuracy assumption, since the 'about 100 cards' summary rests on that assumption.
  2. [Section 5.2, Table 2] The geometric-mean workload ratios used to support the 70–85% claim are described in the text but not displayed in Table 2; adding a column or appendix table with the pairwise ratios and their Monte Carlo uncertainty would make the headline claim directly auditable.
  3. [Section 3.1, Eq. (2)] The a priori population \{\tilde{x}_i\} is defined in words as derived from reported tallies, but the mapping from reported tallies to this population is not given as a formula; a short formal definition would aid reproducibility.
  4. [Abstract and Section 5.1] The statement that stratification increases workloads by about 25% on average is not immediately visible in Table 1, which reports the ratio of unstratified to stratified sample sizes; please state explicitly that the 25% figure is the geometric mean of the inverse ratios.
  5. [Section 2] There is a typographical error in Section 2: '¯r1/2' should read '\bar r > 1/2'.

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity; workload reductions are empirical comparisons against external benchmarks, with the undetermined tally-error threshold flagged as a robustness limitation.

full rationale

The paper's derivation chain is transparent and non-circular. ONEAudit reference values are constructed from reported batch tallies, and the a priori Kelly bet is optimized for an assumed population derived from those same tallies, but the workload claims are evaluated by simulation on true populations that are not identical to the assumed population in the error rows (e.g., true assorter means halfway between reported and null, Section 5.2). The universal portfolio and AGRAPA, which do not rely on reported tallies for betting, also achieve large workload reductions, so the central efficiency claim does not reduce to a fitted input. The comparisons are against named external benchmark bets (COBRA, truncated shrinkage) with stated parameter assumptions; no fitted parameter is renamed as a prediction. The San Francisco case study explicitly conditions on treating CVRs as accurate manual-vote records, a stated assumption rather than a hidden circular step. Self-citations to ONEAudit, UI-TS, and COBRA are either the method under study, the state-of-the-art benchmark, or a comparison method; none is invoked as an external uniqueness theorem to force the paper's conclusions. Section 7's admission that the paper did not determine how much tally error would make a priori Kelly worse than the universal portfolio is a robustness gap affecting external validity, not evidence that the workload reduction is equivalent to its inputs by construction. The stratified-sampling finding is also an empirical simulation result using oracle Kelly bets for both designs, not a definitional artifact.

Assumptions & free parameters 4 free parameters · 4 assumptions · 0 invented entities

The proposed methods introduce no new entities or forces. They rely on standard sequential-testing mathematics (Ville, Kelly, Cover) and on domain assumptions about election data. The user-chosen constants (c, D, G) are method settings, not free parameters fitted to the central claim.

free parameters (4)
  • AGRAPA bet cap c = 0.99
    User-chosen constant in Equation (3) controlling how aggressive the AGRAPA bet can be; set to 0.99 in all simulations. Not fitted to the data but affects AGRAPA performance.
  • Universal portfolio grid size D = 100
    Number of grid points in the discrete mixture approximation to the universal portfolio (Section 3.3). The paper argues D=100 is adequate for audit sample sizes; larger D increases compute.
  • UI-TS number of bands G = 100
    Number of bands used to approximate the Kelly-optimal union-of-intersections test sequence (Section 5.1). Affects the stratified baseline only.
  • COBRA assumed overstatement rates = 0 one-vote, 0.1% two-vote
    Fixed assumptions for the COBRA baseline bet (Section 5.2); this is the old method being compared, not a new method parameter.
assumptions (4)
  • standard math Ville's inequality: for a nonnegative supermartingale M_t starting at 1, P(inf_t 1/M_t <= p) <= p.
    Basis for sequentially valid p-values and TSM validity (Section 3).
  • domain assumption SHANGRLA reduction: checking an assertion that an assorter mean > 1/2 suffices to verify a reported election outcome for most social choice functions.
    Used throughout to reduce audits to testing means (Section 2).
  • domain assumption Reference values satisfy mean > 1/2 (reported outcome is not wrong), so the overstatement assorter population has null mean 1/2.
    Assumed in the setup; if the reported outcome is wrong, the audit should escalate to a full hand count (Section 2).
  • ad hoc to paper In simulations, reported batch totals are either exactly accurate or have errors that halve the margin but preserve the correct winner.
    Defines the simulation envelope (Section 5.2); the paper does not simulate CVR errors or larger tally errors.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Dice, but don't slice: Optimizing the efficiency of ONEAudit." pith.science (2026). https://pith.science/paper/DLBBEK3V

@misc{pith2026250722179,
  author       = {Pith},
  title        = {Pith review of: Dice, but don't slice: Optimizing the efficiency of ONEAudit},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/DLBBEK3V}},
  note         = {Machine review of arXiv:2507.22179}
}
read the original abstract

ONEAudit provides more efficient risk-limiting audits than other extant methods when the voting system cannot report a cast-vote record linked to each cast card. It obviates the need for re-scanning; it is simpler and more efficient than 'hybrid' audits; and it is far more efficient than batch-level comparison audits. There may be room to improve the efficiency of ONEAudit further by tuning the statistical tests it uses and by using stratified sampling. We show that tuning the tests by optimizing for the reported batch-level tallies or integrating over a distribution reduces expected workloads by 70-85% compared to the current ONEAudit implementation across a range of simulated elections. The improved tests reduce the expected workload to audit the 2024 Mayoral race in San Francisco, California, by half -- from about 200 cards to about 100 cards. In contrast, stratified sampling does not help: it increases workloads by about 25% on average.

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

33 extracted references · 32 canonical work pages

  1. [1]

    Election Law Journal, Rules, Politics, and Policy (2020)

    Appel, A., DeMillo, R., Stark, P.: Ballot-marking devices cannot assure the will of the voters. Election Law Journal, Rules, Politics, and Policy (2020)

    work page 2020

  2. [2]

    Georgetown Law Technology Review4.2, 523–541 (2020)

    Appel, A., Stark, P.: Evidence-based elections: Create a meaningful paper trail, then audit. Georgetown Law Technology Review4.2, 523–541 (2020)

    work page 2020

  3. [3]

    In: Financial Cryptography and Data Security

    Blom, M., Stark, P., Stuckey, P., Teague, V., Vukcevic, D.: Auditing Hamiltonian elections. In: Financial Cryptography and Data Security. FC 2021 International Workshops. pp. 235–250. Springer, Berlin, Heidelberg (2021) Dice, but don’t slice: Optimizing the efficiency of ONEAudit 15

    work page 2021

  4. [4]

    In: Krimmer, R

    Blom, M., Stuckey, P., Teague, V.: Ballot-polling risk limiting audits for IRV elections. In: Krimmer, R. et al. (ed.) Electronic Voting. pp. 17–34. Springer, Cham (2018)

    work page 2018

  5. [5]

    John Wiley & Sons, Inc., New York, 3rd edn

    Cochran, W.: Sampling Techniques. John Wiley & Sons, Inc., New York, 3rd edn. (1977)

    work page 1977

  6. [6]

    IEEE Transactions on Information Theory42(2), 348–363 (March 1996)

    Cover, T.M., Ordentlich, E.: Universal portfolios with side information. IEEE Transactions on Information Theory42(2), 348–363 (March 1996)

    work page 1996

  7. [7]

    Mathematical Finance1, 1–29 (1991)

    Cover, T.: Universal portfolios. Mathematical Finance1, 1–29 (1991)

    work page 1991

  8. [8]

    In: Volkamer, M

    Ek, A., Stark, P., Stuckey, P., Vukcevic, D.: Adaptively weighted audits of instant- runoff voting elections: AWAIRE. In: Volkamer, M. et al. (ed.) Electronic Voting. pp. 35–51. Springer Nature, Cham (2023)

    work page 2023

Show all 33 references
  1. [9]

    Ek, A., Stark, P., Stuckey, P., Vukcevic, D.: Efficient Weighting Schemes for Auditing Instant-Runoff Voting Elections, pp. 18–32. Springer Nature (2024)

    2024

  2. [10]

    In: 2023 IEEE Symposium on Security and Privacy (SP)

    Fuller, B., Harrison, A., Russell, A.: Adaptive risk-limiting comparison audits. In: 2023 IEEE Symposium on Security and Privacy (SP). pp. 3314–3331 (2023)

    2023

  3. [11]

    Digital Threats: Research and Practice (DTRAP) 2, 1–15 (2021)

    Glazer, A., Spertus, J., Stark, P.: More style, less work: card-style data decrease risk-limiting audit sample sizes. Digital Threats: Research and Practice (DTRAP) 2, 1–15 (2021)

    2021

  4. [12]

    In: Pro- ceedings of E-Vote-ID 2023

    Glazer, A., Spertus, J., Stark, P.: Stylish risk-limiting audits in practice. In: Pro- ceedings of E-Vote-ID 2023. Lecture Notes in Informatics (LNI), Gesellschaft für Informatik, Bonn (To appear)

    2023

  5. [13]

    Journal of Machine Learning Research3(Nov), 423–440 (2002)

    Kalai, A., Vempala, S.: Efficient algorithms for universal portfolios. Journal of Machine Learning Research3(Nov), 423–440 (2002)

    2002

  6. [14]

    The American Statistician41, 157–158 (1987)

    Kaplan, H.: A method of one-sided nonparametric inference for the mean of a nonnegative population. The American Statistician41, 157–158 (1987)

    1987

  7. [15]

    In: Proceedings of the 2011 Electronic Voting Technology Workshop / Workshop on Trustworthy Elections (EVT/WOTE ’11)

    Lindeman, M., Stark, P., Yates, V.: BRAVO: Ballot-polling risk-limiting audits to verify outcomes. In: Proceedings of the 2011 Electronic Voting Technology Workshop / Workshop on Trustworthy Elections (EVT/WOTE ’11). USENIX (2012)

    2012

  8. [16]

    In: Electronic Voting

    Ottoboni, K., Stark, P., Lindeman, M., McBurnett, N.: Risk-limiting audits by stratified union-intersection tests of elections (SUITE). In: Electronic Voting. E- Vote-ID 2018. Lecture Notes in Computer Science. Springer (2018)

    2018

  9. [17]

    The Annals of Statistics2(3), 415 – 436 (1974)

    Robbins, H., Siegmund, D.: The Expected Sample Size of Some Tests of Power One. The Annals of Statistics2(3), 415 – 436 (1974)

    1974

  10. [18]

    In: International Conference on Financial Cryptography and Data Security

    Spertus, J.: COBRA: Comparison-Optimal Betting for Risk-limiting Audits. In: International Conference on Financial Cryptography and Data Security. pp. 95–109. Springer, Cham (2023)

    2023

  11. [19]

    arXiv preprint arXiv:2409.06680 (2024)

    Spertus, J., Sridhar, M., Stark, P.: Sequential stratified inference for the mean. arXiv preprint arXiv:2409.06680 (2024)

    2024

  12. [20]

    In: Krimmer, R

    Spertus, J., Stark, P.: Sweeter than SUITE: Supermartingale stratified union- intersection tests of elections. In: Krimmer, R. et al (ed.) Electronic Voting. pp. 106–121. Springer, Cham (2022)

    2022

  13. [21]

    IEEE Transactions on Information Forensics and Security4, 1005–1014 (2009)

    Stark, P.: Risk-limiting post-election audits:P-values from common probability inequalities. IEEE Transactions on Information Forensics and Security4, 1005–1014 (2009)

    2009

  14. [22]

    In: Proceed- ings of the 2010 Electronic Voting Technology Workshop / Workshop on Trustworthy Elections (EVT/WOTE ’10)

    Stark, P.: Super-simple simultaneous single-ballot risk-limiting audits. In: Proceed- ings of the 2010 Electronic Voting Technology Workshop / Workshop on Trustworthy Elections (EVT/WOTE ’10). USENIX (2010)

    2010

  15. [23]

    Financial Cryptography and Data Security, Lecture Notes in Computer Science 12063 (2020) 16 J

    Stark, P.: Sets of half-average nulls generate risk-limiting audits: SHANGRLA. Financial Cryptography and Data Security, Lecture Notes in Computer Science 12063 (2020) 16 J. Spertus et al

    2020

  16. [24]

    The Annals of Applied Statistics17(1), 641–679 (2023)

    Stark, P.: ALPHA: Audit that learns from previously hand-audited ballots. The Annals of Applied Statistics17(1), 641–679 (2023)

    2023

  17. [25]

    IEEE Security and Privacy10, 33–41 (2012)

    Stark, P., Wagner, D.: Evidence-based elections. IEEE Security and Privacy10, 33–41 (2012)

    2012

  18. [26]

    In: K., Sokratis et al

    Stark, P.: Non(c)esuch ballot-level comparison risk-limiting audits. In: K., Sokratis et al. (ed.) Computer Security. ESORICS 2022 International Workshops. pp. 541–554. Springer International Publishing, Cham (2023)

    2023

  19. [27]

    In: Interna- tional Conference on Financial Cryptography and Data Security

    Stark, P.: Overstatement-net-equivalent risk-limiting audit: Oneaudit. In: Interna- tional Conference on Financial Cryptography and Data Security. pp. 63–78. Springer (2023)

    2023

  20. [28]

    Gauthier-Villars, Paris (1939)

    Ville, J.: Étude critique de la notion de collectif. Gauthier-Villars, Paris (1939)

    1939

  21. [29]

    The Annals of Mathematical Statistics 16, 117–186 (1945)

    Wald, A.: Sequential tests of statistical hypotheses. The Annals of Mathematical Statistics 16, 117–186 (1945)

    1945

  22. [30]

    Dover Publications, New York (1947)

    Wald, A.: Sequential Analysis. Dover Publications, New York (1947)

    1947

  23. [31]

    Journal of the Royal Statistical Society Series B: Statistical Methodology 86 (2024)

    Waudby-Smith, I., Ramdas, A.: Estimating means of bounded random variables by betting. Journal of the Royal Statistical Society Series B: Statistical Methodology 86 (2024)

    2024

  24. [32]

    arXiv:2504.02818 (2025)

    Waudby-Smith, I., Sandoval, R., Jordan, M.: Universal log-optimality for general classes of e-processes and sequential hypothesis tests. arXiv:2504.02818 (2025)

    2025 arXiv

  25. [33]

    Waudby-Smith, I., Stark, P., Ramdas, A.: RiLACS: Risk Limiting Audits via Confidence Sequences. In: R. Krimmer et al. (ed.) Electronic Voting. pp. 124–139. Springer International Publishing, Cham (2021)

    2021

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.