pith. sign in

arxiv: 2606.31022 · v1 · pith:RRIL6EIRnew · submitted 2026-06-30 · 📊 stat.AP · stat.ME

Bayesian Uncertainty Quantification for Ranked Choice Voting Polls

Evan T. R. Rosenman , Jason Liang This is my paper

Pith reviewed 2026-07-01 03:22 UTC · model grok-4.3

classification 📊 stat.AP stat.ME
keywords ranked choice votingBayesian inferenceuncertainty quantificationpoll analysiswin probabilityDirichlet multinomialconjugacy
0
0 comments X

The pith

A Bayesian model estimates each leading candidate's win probability in ranked choice voting polls by conditioning directly on the observed ballot counts.

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper sets out a Bayesian method for turning RCV poll results into probabilities that each top candidate ultimately wins. It models voter preferences as draws from a common distribution and uses a conjugacy property so that the poll counts update the win probabilities without extra simulation steps. This matters because RCV outcomes depend on the order of eliminations and reallocations, which standard margins of error do not capture. The authors apply the method to the 2021 New York City mayoral primary and the 2022 Alaska House election to illustrate how it handles path-dependent results. They also contrast it with frequentist approaches on real cast-vote records.

Core claim

The paper claims that a Dirichlet-multinomial model for ranked ballots yields a conjugacy relationship that lets poll counts be plugged in directly to compute the posterior probability that any given candidate wins the RCV tabulation, thereby supplying a single-number uncertainty measure for each leading contender.

What carries the argument

The conjugacy relationship between the Dirichlet prior and the multinomial likelihood on preference vectors, which updates win probabilities conditional on the poll sample.

If this is right

  • Polls can report the probability that each candidate wins rather than a margin around a point estimate.
  • Uncertainty statements automatically incorporate the full sequence of possible eliminations and vote transfers.
  • Direct comparison of the two real elections shows where frequentist intervals fail to reflect path dependence.
  • The same conjugacy lets analysts update the probabilities as new poll waves arrive without refitting.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • Campaigns could use the output probabilities to decide which voters to target for second-preference persuasion.
  • The framework could be extended to include demographic weighting of the poll sample while retaining the closed-form update.
  • Repeated application across many RCV races would let analysts test whether the model probabilities are well calibrated.

Load-bearing premise

The poll respondents are drawn from the same underlying preference distribution that will determine the actual election outcome.

What would settle it

A pre-election RCV poll analyzed with the method produces win probabilities that can be checked directly against the observed winner and elimination sequence in the subsequent election.

Figures

Figures reproduced from arXiv: 2606.31022 by Evan T. R. Rosenman, Jason Liang.

Figure 1
Figure 1. Figure 1: The left panel plots the probability that each candidate wins, in an unbiased sample from the cast [PITH_FULL_IMAGE:figures/full_fig_p007_1.png] view at source ↗
Figure 2
Figure 2. Figure 2: Order in which each candidate is eliminated in simulated instant runoffs in random samples of [PITH_FULL_IMAGE:figures/full_fig_p010_2.png] view at source ↗
Figure 3
Figure 3. Figure 3: Accuracy of candidate-set pruning under repeated sampling from the cast vote record of the 2021 [PITH_FULL_IMAGE:figures/full_fig_p015_3.png] view at source ↗
Figure 4
Figure 4. Figure 4: Bayesian Simulation Method Pipeline 5.1 Reanalyzing the Data for Progress NYC Poll We begin with the final Data for Progress poll of the 2021 New York City Democratic mayoral primary. As discussed in Section 3.2, this poll identified Eric Adams, Kathryn Garcia, and Maya Wiley as the three leading candidates. In the penultimate round, Adams led the poll (38% support) and Garcia and Wiley were nearly tied (3… view at source ↗
Figure 5
Figure 5. Figure 5: Posterior probability of victory as a function of sample size under the Bayesian polling framework. [PITH_FULL_IMAGE:figures/full_fig_p020_5.png] view at source ↗
read the original abstract

Ranked choice voting (RCV) is a popular alternative voting method in which voters are asked to list their favored candidates in preference order, rather than vote for a single candidate. When these ballots are tabulated, candidates are successively eliminated, and their votes are reallocated to each voter's next-preferred choice. The process continues until a candidate commands a majority of the active ballots and is declared the winner. As RCV gains wider adoption, the method poses novel challenges for pollsters. Unlike plurality elections, the event that a candidate wins cannot be expressed in terms of a single population parameter. Hence, the basic concept of a margin-of-error is not straightforward to define. Moreover, a candidate's ability to win may depend on both their support across the ballot and the order in which other candidates are eliminated. Existing measures of sampling uncertainty for polls of RCV elections do not clearly quantify these path-dependent outcomes. Here, we propose a simple, Bayesian framework to quantify uncertainty in polls of RCV elections. We cast the problem as one of estimating win probabilities for each leading candidate, and leverage a simple conjugacy relationship to estimate these probabilities conditional on the poll results. We include applied analyses involving two prominent ranked choice voting elections: the 2021 New York City Democratic mayoral primary, in which Eric Adams narrowly defeated Kathryn Garcia in the final round; and the 2022 special election to Alaska's U.S. House seat, in which Mary Peltola was elected despite not being a Condorcet winner. Using the cast vote records from both elections, we demonstrate some challenges of traditional frequentist uncertainty quantification in RCV polls. We also demonstrate the utility of our approach using a poll of the NYC primary obtained from the polling firm Data for Progress.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit. Tearing a paper down is the easy half of reading it; the pith above is the substance, this is the friction.

Referee Report

2 major / 2 minor

Summary. The paper proposes a Bayesian framework for uncertainty quantification in ranked choice voting (RCV) polls. It models voter preference distributions via a Dirichlet prior updated with poll counts through Multinomial conjugacy, then induces posterior distributions on candidate win probabilities by simulating the RCV elimination process on draws from the posterior. The approach is demonstrated on cast vote records (CVRs) from the 2021 NYC Democratic mayoral primary and 2022 Alaska U.S. House special election, plus a real poll from Data for Progress, to contrast with frequentist methods that struggle with path-dependent outcomes.

Significance. If the exchangeability assumption holds, the method supplies a computationally convenient route to credible intervals for RCV win probabilities that fully incorporate the sequential elimination mechanics, addressing a gap in existing poll uncertainty measures. The use of actual CVRs to illustrate frequentist difficulties and the application to a live poll are concrete strengths; the conjugacy yields an exact posterior without requiring MCMC for the preference parameters themselves.

major comments (2)
  1. [§3] §3 (Bayesian model and conjugacy): the posterior win probabilities are obtained by conditioning the RCV outcome function directly on the observed poll counts via Dirichlet-Multinomial updating; this step is valid only under the assumption that poll ballots are i.i.d. draws from the identical preference distribution that will generate election-day ballots, yet no sensitivity analysis to differential non-response, turnout bias, or weighting is reported.
  2. [§4.2 and §4.3] §4.2 (Alaska application) and §4.3 (NYC poll application): the reported credible intervals for Peltola and Adams are presented without any diagnostic comparing the posterior predictive distribution of rankings to the observed CVR marginals or to external benchmarks for non-response bias, leaving the practical reliability of the intervals under real polling conditions unquantified.
minor comments (2)
  1. [§2] Notation for the preference vector θ and the RCV win indicator W_c(θ) could be introduced earlier and used consistently when describing the induced posterior p(W_c | data).
  2. [§3] The manuscript would benefit from a short table listing the Dirichlet hyperparameters chosen for each election and the effective sample size after updating.

Simulated Author's Rebuttal

2 responses · 0 unresolved

We thank the referee for their constructive comments. We respond to each major comment below.

read point-by-point responses

  1. Referee: [§3] §3 (Bayesian model and conjugacy): the posterior win probabilities are obtained by conditioning the RCV outcome function directly on the observed poll counts via Dirichlet-Multinomial updating; this step is valid only under the assumption that poll ballots are i.i.d. draws from the identical preference distribution that will generate election-day ballots, yet no sensitivity analysis to differential non-response, turnout bias, or weighting is reported.

    Authors: The Dirichlet-Multinomial conjugacy does rest on the i.i.d. sampling assumption, which is the same modeling choice made by standard frequentist poll margins of error. The manuscript's contribution is the exact posterior simulation of the full RCV elimination process under this assumption. When poll weights or non-response adjustments are available they can be folded into the effective counts before the Dirichlet update; the framework is modular in that respect. Because the applications use either full CVRs or an unweighted poll, we did not perform sensitivity analyses. We will revise §3 to state the assumption explicitly and flag sensitivity to turnout bias and weighting as an important direction for applied follow-up work. revision: partial

  2. Referee: [§4.2 and §4.3] §4.2 (Alaska application) and §4.3 (NYC poll application): the reported credible intervals for Peltola and Adams are presented without any diagnostic comparing the posterior predictive distribution of rankings to the observed CVR marginals or to external benchmarks for non-response bias, leaving the practical reliability of the intervals under real polling conditions unquantified.

    Authors: The CVR illustrations in §4.2 and the first part of §4.3 treat the full election records as the population; the exercise is to show how frequentist intervals behave when the RCV outcome is path-dependent, not to validate a sampling model. For the Data for Progress poll in §4.3 we will add, in revision, a short posterior-predictive check that compares the marginal distribution of simulated rankings to the observed poll counts. This will provide a direct diagnostic of model fit for the polling application. revision: yes

Circularity Check

0 steps flagged

No circularity: standard conjugate updating applied without self-referential definitions or fitted predictions.

full rationale

The derivation relies on Dirichlet-Multinomial conjugacy to obtain posterior win probabilities directly from observed poll counts under an exchangeability assumption. This is a standard Bayesian update with no evidence that win probabilities are defined in terms of the fitted quantities, that a parameter is fitted to data and then renamed as a prediction, or that load-bearing steps reduce to self-citations. The method is self-contained against external benchmarks once the representativeness assumption is granted; no reduction by construction is exhibited in the provided text.

Axiom & Free-Parameter Ledger

0 free parameters · 0 axioms · 0 invented entities

Abstract-only view yields no identifiable free parameters, axioms, or invented entities; the conjugacy step is described as simple and standard.

pith-pipeline@v0.9.1-grok · 5848 in / 1048 out tokens · 36794 ms · 2026-07-01T03:22:13.032169+00:00 · methodology

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

46 extracted references · 46 canonical work pages

  1. [1]

    Arrow, K. J. (1951). Social Choice and Individual Values . Yale University Press

    work page 1951

  2. [2]

    Atkinson, N., Foley, E., & Ganz, S. C. (2024). Beyond the spoiler effect: Can ranked-choice voting solve the problem of political polarization? University of Illinois Law Review , (pp.\ 1655)

    work page 2024

  3. [3]

    J., & Teague, V

    Blom, M., Conway, A., Stuckey, P. J., & Teague, V. J. (2020). Did that lost ballot box cost me a seat? C omputing manipulations of STV elections. In Proceedings of the AAAI Conference on Artificial Intelligence , volume 34 (pp.\ 13235--13240)

    work page 2020

  4. [4]

    J., & Teague, V

    Blom, M., Stuckey, P. J., & Teague, V. J. (2019). Toward computing the margin of victory in single transferable vote elections. INFORMS Journal on Computing , 31(4), 636--653

    work page 2019

  5. [5]

    Bolstad, W. M. (2007). Introduction to Bayesian Statistics . Hoboken, NJ: Wiley-Interscience, 2. ed edition

    work page 2007

  6. [6]

    Bridges, K. (2025). Siena College 2025 AARP New York Mayoral Race Poll . Technical report, AARP Research, Washington, DC

    work page 2025

  7. [7]

    Burnett, C. M. & Kogan, V. (2014). Ballot (and voter) ``exhaustion'': A failure of ranked-choice voting. Electoral Studies , 37, 41--51

    work page 2014

  8. [8]

    Cary, D. (2011). Estimating the margin of victory for Instant-Runoff voting. In 2011 Electronic Voting Technology Workshop/Workshop on Trustworthy Elections (EVT/WOTE 11). San Francisco, CA: USENIX Association

    work page 2011

  9. [9]

    After heated second debate and SuperPAC attacks, Mamdani and Cuomo race to statistical tie

    Center for Strategic Politics (2025). After heated second debate and SuperPAC attacks, Mamdani and Cuomo race to statistical tie. https://stratpolitics.org/2025/06/after-heated-second-debate-and-superpac-attacks-mamdani-and-cuomo-race-to-statistical-tie/

    work page 2025

  10. [10]

    New York City mayor's race new Democratic primary poll

    Change Research (2021). New York City mayor's race new Democratic primary poll. https://changeresearch.com/new-york-city-mayoral-democratic-primary-poll/

    work page 2021

  11. [11]

    Coll, J. A. (2021). Demographic disparities using ranked-choice voting? R anking difficulty, under-voting, and the 2020 D emocratic primary. Politics and Governance , 9(2), 293--305

    work page 2021

  12. [12]

    Cormack, L. (2026). More chances, fewer problems? R evisiting ranked choice voting errors in New York City . American Politics Research , 54(3), 340--346

    work page 2026

  13. [13]

    New Y ork C ity mayoral and comptroller Democratic primary crosstabs

    Data for Progress (2021). New Y ork C ity mayoral and comptroller Democratic primary crosstabs. https://www.filesforprogress.org/datasets/2021/6/dfp_nyc_pre_election_mayoral_comptroller_crosstabs.pdf

    work page 2021

  14. [14]

    Mary P eltola would be front-runner for A laska governor

    Data for Progress (2025). Mary P eltola would be front-runner for A laska governor. https://www.dataforprogress.org/blog/2025/8/8/mary-peltola-would-be-front-runner-for-alaska-governor

    work page 2025

  15. [15]

    Ranked-choice voting simulation for the 2021 New York City Democratic primary

    Data for Progress Tech Team (2021). Ranked-choice voting simulation for the 2021 New York City Democratic primary. https://www.dropbox.com/scl/fi/fiuhzhchvs3ci8hum8jsr/dfp_tech_team_memo_nyc_rcv_2021_06_21.pdf. Technical memorandum

    work page 2021

  16. [16]

    De Condorcet, N. et al. (2014). Essai sur l'application de l'analyse \`a la probabilit \'e des d \'e cisions rendues \`a la pluralit \'e des voix . Cambridge University Press

    work page 2014

  17. [17]

    Deshpande, S., Garg, N., & Jacobson, S. (2026). Simpler than you think: The practical dynamics of ranked choice voting. Journal of Computational Social Science , 9(2), 40

    work page 2026

  18. [18]

    Dowling, E., Tolbert, C., Micatka, N., & Donovan, T. (2024). Does ranked choice voting increase voter turnout and mobilization? Electoral Studies , 90, 102816

    work page 2024

  19. [19]

    & Strano, M

    Drutman, L. & Strano, M. (2021). What we know about ranked-choice voting . New America Washington, DC

    work page 2021

  20. [20]

    NYC mayor 2021: Adams stays ahead but rank choice voting closes gap for Wiley and Garcia

    Emerson College Polling (2021). NYC mayor 2021: Adams stays ahead but rank choice voting closes gap for Wiley and Garcia . https://emersonpolling.reportablenews.com/pr/nyc-mayor-2021-adams-stays-ahead-but-rank-choice-voting-closes-gap-for-wiley-and-garcia

    work page 2021

  21. [21]

    New York City mayoral poll: Mamdani catches Cuomo in rank choice voting

    Emerson College Polling (2025). New York City mayoral poll: Mamdani catches Cuomo in rank choice voting. https://emersoncollegepolling.com/new-york-city-mayoral-poll-june/

    work page 2025

  22. [22]

    Fix the city final pre-primary poll: Cuomo maintains ‘comfortable’ lead over M amdani

    Empire Report New York (2025). Fix the city final pre-primary poll: Cuomo maintains ‘comfortable’ lead over M amdani. https://empirereportnewyork.com/fix-the-city-final-pre-primary-poll-cuomo-maintains-comfortable-lead-over-mamdani/

    work page 2025

  23. [23]

    June 2021 FairVote / Citizen data poll of New York City likely Democratic primary voters

    FairVote (2021). June 2021 FairVote / Citizen data poll of New York City likely Democratic primary voters. https://fairvote.app.box.com/s/uwqzhjcmu6pvjuz5qve33ioff0u7zndp

    work page 2021

  24. [24]

    & McCune, D

    Graham-Squire, A. & McCune, D. (2022). A mathematical analysis of the 2022 Alaska special election for us house. arXiv preprint arXiv:2209.04764

    work page arXiv 2022

  25. [25]

    & McCune, D

    Graham-Squire, A. & McCune, D. (2025). An examination of ranked-choice voting in the United States , 2004--2022. Representation , 61(1), 1--19

    work page 2025

  26. [26]

    In the race to be the next mayor of New York, Adams leads but ranked-choice voting means the race is still wide open

    Ipsos (2021). In the race to be the next mayor of New York, Adams leads but ranked-choice voting means the race is still wide open. https://www.ipsos.com/en-us/news-polls/Adams-leads-but-ranked-choice-voting

    work page 2021

  27. [27]

    Jeffreys, H. (1946). An invariant form for the prior probability in estimation problems. Proceedings of the Royal Society of London. A. Mathematical and Physical Sciences , 186(1007), 453--461

    work page 1946

  28. [28]

    Kish, L. (1995). Methods for design effects. Journal of Official Statistics , 11(1), 55

    work page 1995

  29. [29]

    Kropf, M. (2021). Using campaign communications to analyze civility in ranked choice voting elections. Politics and Governance , 9(2), 280--292

    work page 2021

  30. [30]

    R., Mutlu, C., Samarth, T., Acevedo Jetter, K

    Kuriwaki, S., Reece, M., Baltz, S., Conevska, A., Loffredo, J. R., Mutlu, C., Samarth, T., Acevedo Jetter, K. E., Djanogly Garai, Z., Murray, K., Hirano, S., Lewis, J. B., Snyder, J. M., & Stewart, C. (2024). Cast vote records: A database of ballots from the 2020 U.S. election. Scientific Data , 11(1), 1304

    work page 2024

  31. [31]

    Lehmann, E. L. & Romano, J. P. (2005). Testing Statistical Hypotheses . Springer

    work page 2005

  32. [32]

    Maloy, J. S. & Ward, M. (2021). The impact of input rules and ballot options on voting error: An experimental analysis. Politics and Governance , 9(2), 306--318

    work page 2021

  33. [33]

    2025 NYC mayoral poll: C uomo holds his lead one week before primary day, as voters express unease with city’s direction

    Manhattan Institute (2025). 2025 NYC mayoral poll: C uomo holds his lead one week before primary day, as voters express unease with city’s direction. https://manhattan.institute/article/2025-nyc-mayoral-poll

    work page 2025

  34. [34]

    New York City Democratic mayoralty primary: Adams leads with G arcia and W iley in final round top tier

    Marist Poll (2021). New York City Democratic mayoralty primary: Adams leads with G arcia and W iley in final round top tier. https://maristpoll.marist.edu/polls/wnbc-telemundo-47-politico-marist-poll-nyc-democratic-primary-for-mayor/

    work page 2021

  35. [35]

    NYC mayoralty, J une 2025: Cuomo breaks 50\ https://maristpoll.marist.edu/polls/nyc-mayoralty-june-2025/

    Marist Poll (2025). NYC mayoralty, J une 2025: Cuomo breaks 50\ https://maristpoll.marist.edu/polls/nyc-mayoralty-june-2025/

    work page 2025

  36. [36]

    & Wilson, J

    McCune, D. & Wilson, J. (2023). Ranked-choice voting and the spoiler effect. Public Choice , 196(1), 19--50

    work page 2023

  37. [37]

    McDaniel, J. A. (2016). Writing the rules to rank the candidates: Examining the impact of instant-runoff voting on racial group turnout in S an F rancisco mayoral elections. Journal of Urban Affairs , 38(3), 387--408

    work page 2016

  38. [38]

    Mercer, A. (2016). 5 key things to know about the margin of error in election polls. Pew Research Center. https://www.pewresearch.org/short-reads/2016/09/08/understanding-the-margin-of-error-in-election-polls/

    work page 2016

  39. [39]

    Mercer, A., Lau, A., & Kennedy, C. (2018). For weighting online opt-in samples, what matters most? Pew Research Center. https://www.pewresearch.org/methods/2018/01/26/for-weighting-online-opt-in-samples-what-matters-most/

    work page 2018

  40. [40]

    Pettigrew, S., Wolff, M., Wallsten, K., & Prillaman, S. (2023). Overvotes, overranks, and skips: Mismarked and rejected votes in ranked choice voting. Election Law Journal , 22(3), 181--195

    work page 2023

  41. [41]

    Santucci, J. (2021). Variants of ranked-choice voting from a strategic perspective. Politics and Governance , 9(2), 344--353

    work page 2021

  42. [42]

    Shirani-Mehr, H., Rothschild, D., Goel, S., & Gelman, A. (2018). Disentangling bias and variance in election polls. Journal of the American Statistical Association , 113(522), 607--614

    work page 2018

  43. [43]

    Silvapulle, M. J. & Sen, P. K. (2011). Constrained Statistical Inference: Order, Inequality, and Shape Constraints . John Wiley & Sons

    work page 2011

  44. [44]

    Singh, M., Donnini, Z., Beck, S., Cortes, K., Richardson, L., Quintos, M., & Wang, D. (2025). We polled New York City . here’s what we found. Yale Youth Poll. https://yalepolling.substack.com/p/new-yoll-city

    work page 2025

  45. [45]

    Van der Vaart, A. W. (2000). Asymptotic Statistics , volume 3. Cambridge university press

    work page 2000

  46. [46]

    Vishwanath, A. (2025). The effects of ranked choice voting on substantive representation. Quarterly Journal of Political Science , 20(3), 409--437

    work page 2025