REVIEW 4 major objections 5 minor 27 references
Risk-Limiting Tallies
T0 review · 4 major / 5 minor · reviewed 2026-08-14 · deepseek-v4-flash
Pith's one-line read Risk-limiting tallies reveal only enough ballots to call the race.
desk verdict A genuinely new idea for coercion-resistant tallying whose statistical engine is currently unverifiable—it deserves a serious referee, but only if the companion proof becomes available. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The engine is Eq. (1), an integral over a mixing parameter $\gamma$ of products of terms built from the sampled labels; reference [11] proves that under the null $\mu = t$ this sequence is a nonnegative closed martingale with expectation 1. A martingale maximal inequality then gives $\Pr(\max_{1\le j\le J} Y_j(t) > 1/p) \le p$, turning the integral into a sequential p-value. For RLT, ballots are relabeled per candidate pair as 1, 0, or $1/2$, making '$w$ beats $\ell$' equivalent to the population mean exceeding $1/2$; testing all $C(C-1)$ pairwise hypotheses yields the winner set.
What would settle it
Compute Eq. (1) exactly for a tiny population, say $N=3$ with two 0s and one 1, over all sampling orders; if the expected value is not 1 under the null mean $1/2$, or if the maximal exceedance probability exceeds the claimed bound, the risk limit is false.
Extended reading notes
Core claim
For plurality contests, a risk-limiting tally (RLT) recasts each pairwise contest between candidates $w$ and $\ell$ as a test of whether the population mean of relabeled ballots exceeds $1/2$: a vote for $w$ counts as 1, a vote for $\ell$ as 0, and any other ballot as $1/2$. Sampling without replacement and using the martingale-based test of reference [11], the protocol rejects the null hypothesis $\mu_{w\ell} \le 1/2$ at level $\alpha$. It stops when one set of $k$ candidates has beaten every outsider, and Proposition 1 bounds the chance of misidentifying the winners by $k(C-k)\alpha$. The paper reports minimum sample sizes for unanimity: at $\alpha = 10^{-9}$, 35 ballots suffice for two candidates and 38 for ten.
Load-bearing premise
The risk guarantee depends on the theorem in reference [11] that the integral in Eq. (1) is a nonnegative closed martingale under the null hypothesis, and on the bulletin board containing the correct, complete set of cast votes; if either fails, the stated bounds do not follow.
Editorial extensions
If this is right
- If the election is lopsided, very few ballots need be opened: with unanimous support, 35 ballots give error probability below $10^{-9}$ in a two-candidate race.
- The risk of misidentifying winners grows with the number of candidates through the factor $k(C-k)$; the paper's table shows the multiplicity penalty adds only a few ballots under near-unanimity.
- For close races the sample may consume all ballots, so the paper proposes fallback strategies: plaintext equivalence tests on unrevealed ballots, tally hiding by multiparty computation, or a secretly computed tally to guide sampling.
- Applying the same idea to Selene, revealing only a random subset of vote trackers lets a coerced voter claim her tracker was never published.
Reading between the lines
- If the bound $k(C-k)\alpha$ is conservative, as the paper hints, the true risk may be lower than $\alpha$ for many margins; a sharper analysis might allow stopping even earlier for close races, but this is not established here.
- The RLT idea transfers naturally to any setting where a population mean must be certified from sequential samples, such as audit or quality-control tests; the paper does not develop these applications.
- A practical deployment would need to settle the source of public randomness and the trust model for the random sampling authority; the paper leaves those choices to stakeholders, so the realized guarantee depends on that external setup.
- The security improvement is about plausible deniability rather than cryptographic coercion resistance: a coercer who watches the revealed ballots still knows unrevealed ballots exist, so the voter's claim is only as strong as the fraction of shrouded ballots.
Signed reviews
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper proposes Risk-Limiting Tallies (RLT), a mechanism for partially tallying encrypted ballots in end-to-end verifiable voting systems: ballots are revealed in a random order until a sequential statistical test achieves a pre-specified risk limit α for the set of winners, leaving the remaining ballots unrevealed. The authors argue that this enhances coercion resistance and mitigates Italian-style attacks by allowing coerced voters to claim that the demanded ballot is among the unrevealed ones. They also propose Risk-Limiting Verification (RLV) for Selene, in which only a random subset of trackers is revealed. The statistical engine is a new ballot-polling sequential test based on a nonnegative martingale for the mean of a nonnegative population (Eq. (1)), whose validity is delegated to an unpublished companion paper. The paper states a Bonferroni-based bound k(C−k)α on the probability of misidentifying the winners, tabulates minimum sample sizes for two-candidate and ten-candidate plurality contests, and sketches protocol and security considerations.
Significance. If the underlying sequential test is valid, the RLT idea is a genuinely useful bridge between risk-limiting audits and tally-hiding cryptography: it provides a practical way to balance statistical confidence against information leakage, and it gives voters a concrete form of plausible deniability in extreme-tally scenarios. The paper also introduces a new type of statistical test whose operating characteristics do not depend on a reported outcome, which is of independent interest. However, the central statistical result is not self-contained and the sample-size claims are unsubstantiated; these points are load-bearing for every downstream guarantee (Proposition 1, Table 1, and the stopping rule). The security analysis is largely informal. For these reasons, the contribution is promising but not yet established.
major comments (4)
- [Section 4.1, Eq. (1)] The entire RLT risk guarantee rests on the claim, stated in Section 4.1, that the process (Y_j) defined by Eq. (1) is a nonnegative closed martingale with expectation 1 when μ = t. This claim is cited to reference [11] (Evans and Stark, in preparation), which is co-authored by one of the present authors, and the recursive algorithm for computing the integral is also delegated to [11]. As printed, the formula is ambiguous: the typesetting does not make clear whether the denominator in the integrand is (\tilde{j} t − \tilde{S}_{j−1}) or some other grouping, and no convention is given for sample paths on which the denominator is zero or negative. Because this martingale property is the engine for Proposition 1, Table 1, and the sequential stopping guarantee, the paper is not self-contained at its most critical point. The authors should either include a proof of the martingale property and the recursive algorithm in the paper, or make [11] available and restate Eq. (1) unambiguously.
- [Section 4.2] The test described in Section 4.1 is presented as a test of the point null hypothesis μ = t, but the RLT procedure uses it to test the composite null hypothesis μ_{wℓ} ≤ 1/2 for each pair (w, ℓ). The paper does not prove that the sequential test controls the type I error uniformly over the composite null. A level-α test of a point null does not automatically yield a level-α test of a one-sided composite null unless monotonicity of the rejection probability in μ is established (for example, via stochastic dominance of the test statistic under larger μ). This monotonicity is asserted only obliquely in the proof sketch of Proposition 2 and is not referenced to a specific theorem. Since Proposition 1's error bound k(C−k)α presupposes that each individual hypothesis is tested at level α under the composite null, this gap must be closed.
- [Section 4.3, Table 1] The minimum sample sizes in Table 1 are asserted without derivation. It is not explained how the numbers 5, 9, 13, ... for α = 10^{-1}, 10^{-2}, ... are computed from Eq. (1), nor what assumptions are made about the label distribution beyond unanimity. The accompanying claims that the sample size is stochastically larger when more than one candidate receives votes, and that an invalid-ballot fraction u multiplies the expected sample size by 1/(1−u), are stated without proof. These sample-size statements are presented as concrete guarantees and are experimentally falsifiable; they need either a derivation, a reference to a public implementation, or a citation to a result that provides them.
- [Sections 5.1 and 6] The paper's central motivation is enhanced coercion resistance via plausible deniability, but Section 5.1 concedes that in close contests the RLT may reveal all or almost all ballots, and the fallback strategies (PET testing, tally hiding, secret tally computation) are only sketched. The security analysis in Section 6 is informal: it does not state a precise adversary model, does not define the coercion-resistance property being claimed, and does not connect the statistical risk limit to the security definition. As a result, the paper does not fully substantiate the contribution item 'strategies to ensure plausible deniability whatever the vote distribution'. The authors should either provide a more rigorous treatment of the fallback mechanisms and the security reduction, or explicitly narrow the claim to scenarios where the RLT terminates with a bounded proportion of unrevealed ballots.
minor comments (5)
- [Introduction] There is a typo: 'undemocractic' should be 'undemocratic'.
- [Section 4.1] The notation mixes raw labels X_j with normalized quantities: X_j is not divided by N, while \tilde{S}_{j−1} = S_{j−1}/N and \tilde{j} = 1 − (j−1)/N. The units in the denominator of Eq. (1) are therefore unclear; the authors should state explicitly that X_j is also normalized, or introduce a normalized label.
- [Section 4.3] The sentence 'the protection a RLT offers is greatest when the risk is greatest' is confusing; presumably it means when the margin is largest (so the risk of coercion is smallest, but the number of unrevealed ballots is largest). Please rephrase.
- [Section 5] The protocol description does not fully pin down the generation of the certified random sampling order. The text lists options (stock-market prices, NIST beacon, etc.) but does not specify one as part of the protocol; this is acceptable for a high-level paper, but a concrete instantiation or a reference to a standard RLA randomness-generation procedure would strengthen the exposition.
- [Section 7.1] The phrase 'some of the approaches to linearising the JCJ-style checks' is vague; please give a specific example or reference.
Circularity Check
The RLT risk guarantee is powered by Eq. (1), but the martingale property and integral recursion are delegated to an unpublished, co-authored companion paper [11], making the central statistical engine a load-bearing self-citation.
-
self citation load bearing
[Section 4.1, Eq. (1) and subsequent paragraph; relied on by Sections 4.2-4.3 (Proposition 1 and Table 1); reference [11]]
"It has been shown in [11] that if µ =t (i.e., if the null hypothesis is true), then (Yj )N j=1 is a nonnegative closed martingale with expected value 1. ... We also note that [11] provides a recursive algorithm for computing the integral (1). [11] S.N. Evans and P.B. Stark. Confidence bounds for the mean of a non-negative population, 2019. in prep."
The RLT stopping guarantee, Proposition 1's error bound k(C-k)α, and Table 1's sample sizes all depend on the claim that Eq. (1) defines a nonnegative martingale with expectation 1 when μ=t, and that the integral can be computed recursively. Neither property is proved or fully specified in this paper; both are imported solely from [11], an unpublished, in-preparation manuscript co-authored by one of the present authors (Stark). This is not an independent, checkable proof: the central statistical engine of the paper reduces to that self-citation. If [11] is unavailable or its theorem does not hold as stated, the RLT risk limit does not follow. The RLT protocol and RLV design have independent content, so the circularity is partial rather than total.
full rationale
The paper's own derivation is mostly compositional: RLT recasts winner determination as C(C-1) pairwise hypotheses μ_wℓ ≤ 1/2, rejects each with a sequential test, and Proposition 1 applies a Bonferroni/union bound to get error ≤ k(C-k)α; Table 1 follows from the test's operating characteristics. Those steps are not circular in the forbidden sense: no parameters are fitted, no prediction is forced by construction, and no known result is merely renamed. The vulnerability is upstream: the level-α test itself is Eq. (1), whose martingale property and recursive computation are asserted only by reference to [11], a companion paper 'in prep.' co-authored by Stark. That makes the central statistical premise a load-bearing self-citation that cannot currently be audited. One further gap—using the point-null martingale at t=1/2 to test the composite null μ_wℓ ≤ 1/2—is asserted without proof; this is a correctness risk rather than a circularity and is not scored here. The coercion-resistance and RLV claims are independent contributions. Score 4: some self-citation with the central statistical guarantee resting on an unverified companion paper, while the protocol-level results still have independent content.
Assumptions & free parameters
assumptions (5)
- domain assumption Evans-Stark martingale inequality: if µ=t, Y_n in Eq. (1) is a nonnegative closed martingale with expected value 1, and a recursive algorithm computes the integral.
- domain assumption The bulletin board contains the correct, complete set of voter-verified encrypted ballots.
- domain assumption The revealed sampling order is uniformly random and independent of vote choices.
- domain assumption The trust assumptions of the underlying voting scheme hold, including at least one honest mix node, threshold tellers, and Selene's trapdoor commitment and tracker properties.
- standard math Kolmogorov's maximal inequality applies to the martingale to bound the sequential error rate.
Cite this review
Pith. "Pith review of Risk-Limiting Tallies." pith.science (2026). https://pith.science/paper/HKMP4OV3
@misc{pith2026190804947,
author = {Pith},
title = {Pith review of: Risk-Limiting Tallies},
year = {2026},
howpublished = {\url{https://pith.science/paper/HKMP4OV3}},
note = {Machine review of arXiv:1908.04947}
}
read the original abstract
Many voter-verifiable, coercion-resistant schemes have been proposed, but even the most carefully designed systems necessarily leak information via the announced result. In corner cases, this may be problematic. For example, if all the votes go to one candidate then all vote privacy evaporates. The mere possibility of candidates getting no or few votes could have implications for security in practice: if a coercer demands that a voter cast a vote for such an unpopular candidate, then the voter may feel obliged to obey, even if she is confident that the voting system satisfies the standard coercion resistance definitions. With complex ballots, there may also be a danger of "Italian" style (aka "signature") attacks: the coercer demands the voter cast a ballot with a specific, identifying pattern. Here we propose an approach to tallying end-to-end verifiable schemes that avoids revealing all the votes but still achieves whatever confidence level in the announced result is desired. Now a coerced voter can claim that the required vote must be amongst those that remained shrouded. Our approach is based on the well-established notion of Risk-Limiting Audits, but here applied to the tally rather than to the audit. We show that this approach counters coercion threats arising in extreme tallies and "Italian" attacks. We illustrate our approach by applying it to the Selene scheme, and we extend the approach to Risk-Limiting Verification, where not all vote trackers are revealed, thereby enhancing the coercion mitigation properties of Selene.
Reference graph
Works this paper leans on
-
[11]
S.N. Evans and P.B. Stark. Confidence bounds for the mean o f a non-negative population, 2019. in prep
work page 2019
- [1]
-
[2]
B. Adida and C. A. Neff. Ballot casting assurance. In Proceedings of the USENIX/Accurate Electronic Voting Technology Workshop 20 06 on Electronic Voting Technology Workshop, EVT’06, pages 7–7, 2006. 16 W. Jamroga, P. B. Roenne, and P. Y. A. Ryan, and P. B. Stark
work page 2006
- [3]
-
[4]
J. Benaloh and D. Tuinstra. Receipt-free secret-ballot e lections. In Proceedings of the twenty-sixth annual ACM symposium on Theory of Computin g, pages 544–553. ACM, 1994
work page 1994
-
[5]
J. Benaloh. Simple verifiable elections. In Proceedings of the USENIX/Accurate Electronic Voting Technology Workshop 2006 on Electronic V oting Technology Workshop, EVT’06, pages 5–5, 2006
work page 2006
- [6]
-
[7]
D. Chaum. Random-sample voting. http://rsvoting.org/whitepaper/white_paper.pdf
Show all 27 references
-
[8]
Chaum, P.Y.A
D. Chaum, P.Y.A. Ryan, and S.A. Schneider. A Practical Vot er-Verifiable Elec- tion Scheme. In Proceedings of ESORICS , LNCS, volume 3679, pages 118–139. Springer-Verlag, 2005
2005
-
[9]
J. Cohen. Improving privacy in cryptographic elections. Technical report, 1986
1986
-
[10]
Delaune, S
S. Delaune, S. Kremer, and M. Ryan. Towards trustworthy e lections. chapter Verifying Privacy-type Properties of Electronic Voting Pr otocols: A Taster, pages 289–309. Springer-Verlag, 2010
2010
-
[12]
Juels, D
A. Juels, D. Catalano, and M. Jakobsson. Coercion-resis tant electronic elections. In Proceedings of the 2005 ACM workshop on Privacy in the electr onic society , pages 61–70. ACM, 2005
2005
-
[13]
Kiayias, T
A. Kiayias, T. Zacharias, and B. Zhang. DEMOS-2: scalabl e E2E verifiable elec- tions without random oracles. In Proceedings of CCS, pages 352–363, 2015
2015
-
[14]
K¨ usters, T
R. K¨ usters, T. Truderung, and A. Vogt. A game-based defin ition of coercion- resistance and its applications. In Proceedings of IEEE Computer Security Foun- dations Symposium (CSF) , pages 122–136, 2010
2010
-
[15]
Lindeman, P.B
M. Lindeman, P.B. Stark, and V. Yates. BRA VO: Ballot-pol ling risk-limiting audits to verify outcomes. Proceedings of EVT/WOTE ’11, 2012
2012
-
[16]
Lindeman and P.B
M. Lindeman and P.B. Stark. A gentle introduction to risk -limiting audits. IEEE Security and Privacy , 10:42–49, 2012
2012
-
[17]
S. Micali. ALGORAND: the efficient and democratic ledger. CoRR, abs/1607.01341, 2016
2016 arXiv
-
[18]
Ottoboni, P.B
K. Ottoboni, P.B. Stark, M. Lindeman, and N. McBurnett. R isk-limiting audits by stratified union-intersection tests of elections (suite ). In Electronic Voting. E- Vote-ID 2018. Lecture Notes in Computer Science , 2018
2018
-
[19]
R.L. Rivest. The ThreeBallot Voting System. https://people.csail.mit.edu/rivest/Rivest-TheThreeBallotVotingSystem.pdf
-
[20]
Ryan, P.B
P.Y.A. Ryan, P.B. Rønne, and V. Iovino. Selene: Voting wi th transparent veri- fiability and coercion-mitigation. In Financial Cryptography and Data Security: Workshops, pages 176–192, 2016
2016
-
[21]
Ryan, S.A
P.Y.A. Ryan, S.A. Schneider, and V. Teague. End-to-end v erifiability in voting systems, from theory to practice. IEEE Security & Privacy , 13(3):59–62, 2015
2015
-
[22]
Ryan and V
P.Y.A. Ryan and V. Teague. Pretty good democracy. In WORKSHOP ON SE- CURITY PROTOCOLS , 2009
2009
-
[23]
Schneider and A
S. Schneider and A. Sidiropoulos. CSP and anonymity. In Proceedings of ESORICS, pages 198–218, 1996. Risk-Limiting Tallies 17
1996
-
[24]
P.B. Stark. Conservative statistical post-election au dits. Ann. Appl. Stat. , 2:550– 581, 2008
2008
-
[25]
Stark and D.A
P.B. Stark and D.A. Wagner. Evidence-based elections. IEEE Security and Pri- vacy, 10:33–41, 2012
2012
-
[26]
Szepieniec and B
A. Szepieniec and B. Preneel. New techniques for electro nic voting. USENIX J. of Election Technology and Systems (JETS) , 3(2):46–69, 2015
2015
-
[27]
Teague, K
V. Teague, K. Ramchen, and L. Naish. Coercion-resistant tallying for STV voting. In 2008 USENIX/ACCURATE Electronic Voting Workshop, EVT 2008 , Proceed- ings, 2008
2008
Reviewed August 14, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.