Pith. sign in

REVIEW 6 major objections 5 minor 19 references

Evaluating Voting Design Vulnerabilities for Retroactive Funding

T0 review · 6 major / 5 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read The paper argues that quadratic voting, not mean or median aggregation, is the manipulation-resistant rule for Optimism's RetroPGF, proving attack bounds and confirming them in 10,000 simulated rounds at real-round scale.

desk verdict The theory is honest, but the simulation equations that drive the headline result are infeasible as written, so the comparative claim needs a rework. read the letter →

arxiv 2505.16068 v1 pith:VW62MN7D submitted 2025-05-21 cs.GT

Jay Yu , Austin Bennett , Billy Gao , Rebecca Joseph This is my paper · ORCID
classification cs.GT MSC 91B1291B14
keywords RetroactivepublicgoodsfundingQuadraticvotingMeanMedianPhantomvoteattackPairwiseManipulationScoreSocialchoicetheoryDAOgovernance
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Can Optimism's RetroPGF safely keep its current voting rule? The paper argues no: among the three mechanisms the program has used, quadratic voting (Round 1) is markedly more resistant to manipulation than mean voting (Round 2) or median voting (Rounds 3–4), and it should be the design basis for future rounds. The paper proves a concrete exploit for each mechanism: two colluding voters can multiply quadratic voting power by $\sqrt{2}$; $k$ phantom voters casting near-zero allocations can shrink a project's mean by a factor $n/(n+k)$; and the same strategy can push a project's median down by about half the number of attackers in the ordered allocation list. In 10,000 simulated rounds at Optimism's scale (133 voters, 374 projects), mean and median project-attack manipulation scores concentrate around 9,000 on the paper's pairwise scale, whereas quadratic voter-attack scores sit below 0.02. This is why the paper's bottom line is that the ideal RetroPGF rule is quadratic, with voter rotation and minimum allocation thresholds to blunt the collusion exploit.

What carries the argument

The load-bearing object is the allocation matrix $A \in \mathbb{R}^{N \times P}$ (voter $i$'s tokens to project $j$) together with the three aggregation rules $\mathrm{mean}_j = \frac{1}{n}\sum_i A_{ij}$, $\mathrm{median}_j$, and $\mathrm{quad}_j=\sum_i \sqrt{A_{ij}}$. The attack mechanism that does the work is the phantom vote: an adversary allocates an arbitrarily small $\varepsilon>0$ to a project they want to harm, exploiting the convention that only non-zero wallets enter $n$; for median voting the same $\varepsilon$ is placed below the current minimum allocation so it shifts the order statistics. The collusion identity that carries the quadratic result is the split-budget optimum $p_1=p_2=P/2$, which yields the $\sqrt{2}$ power gain. The evaluation instrument is the Pairwise Manipulation Score (PMS), a percentage-normalized, baseline-relative squared difference between pre- and post-attack allocation distributions, with 0 meaning identical distributions.

What would settle it

Take the official RetroPGF Round 2 and Round 4 ballot data and recompute each project's allocation two ways: with all badgeholders in the denominator (zero allocations included) and with only non-zero wallets counted. If the two recomputations give the same project rankings, then the phantom-vote premise is absent from the deployed mechanism, and the reported vulnerability gap would not reproduce under the deployed aggregation rule.

Watch

Extended reading notes

Core claim

On the paper's own terms, the central claim is that the three aggregation rules have fundamentally different response surfaces to a minimal attack. For quadratic voting, a colluding pair that each controls $T$ tokens can raise each member's effective contribution from $\sqrt{T}$ to $\sqrt{2T}$ by splitting every budget evenly across both members' favorite projects; the paper stresses that this only pays off if the same delegates can coordinate repeatedly. For mean voting, if the measure denominator counts only the $n$ non-zero allocations to a project, $k$ attackers each casting a vanishing $\varepsilon$ produce a post-attack mean of $(\sum_i a_{ij})/(n+k)$, an $n/(n+k)$ reduction; were all $N$ voters counted, mean voting would be a linear mechanism. For median voting, the same $\varepsilon$ votes occupy the bottom of the ordered list and move the median down by about $k/2$ order statistics, with the post-attack median bounded above by $a_j^{(m-\lceil k/2\rceil+1)}$, so a large gap between ordered allocations can be exploited for a much larger drop than under mean voting. The simulations then show project-level coordination driving mean and median manipulation scores into the 9,000s while quadratic scores remain below 0.02 under voter attacks, which is the empirical pillar of the paper's ordering.

Load-bearing premise

The paper's attack results assume that when a badgeholder does not allocate to a project, that zero is not counted in the denominator, so abstentions are invisible; if the real Optimism aggregation counts every badgeholder, the mean attack collapses and the median result changes.

Editorial extensions

If this is right

  • If the paper is right, future RetroPGF rounds should default to quadratic voting; mean and median should not be used without allocation floors or caps because phantom-vote attack scores sit near 9,000.
  • Voter rotation periods close most of the quadratic collusion hole, since the $\sqrt{2}$ gain depends on the same pair of delegates coordinating repeatedly rather than in one-shot rounds.
  • The Round 3 quorum-cap and Round 4 min/max-threshold variants of median voting are not clean fixes: quorum caps can be bypassed by calibrating $\varepsilon > q/n$, and the caps may collapse genuine preference variation.
  • The simulation's rankings are conditional on its preference model (Pareto-distributed attention with $\alpha=2.5$); the paper's dashboard is the intended route for testing other distributions and real non-aggregated ballot data.
  • Mean and median vulnerabilities are systematic rather than random: the manipulation scores form tight normal distributions, so the attack works predictably, not probabilistically.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Beyond the paper: the entire mean-attack result depends on whether the deployed system excludes zero ballots from the denominator; if Optimism's actual aggregation counts every badgeholder, that attack collapses to a constant rescaling, so the production threat level hinges on a data detail the authors did not verify.
  • Beyond the paper: the phantom-vote idea generalizes to any denominator-flexible aggregation rule, so the same $\varepsilon$-strategy should be tested against trimmed means, winner-take-all thresholds, and quorum-gated schemes.
  • Beyond the paper: PMS measures relative percentage distortion; an absolute-dollar-shift metric would reveal whether these attacks also change the total diverted funds, which is the quantity an attacker ultimately cares about.
  • Beyond the paper: a direct welfare test—comparing each mechanism's allocation to the truthful utility-maximizing allocation on the same preference draws—would show whether resistance to manipulation comes at the cost of allocative accuracy.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

6 major / 5 minor

Summary. The paper analyzes three voting mechanisms used in Optimism's RetroPGF rounds—quadratic, mean, and median voting—from a mechanism-design perspective. It provides formal 'Remark' proofs of vulnerabilities: a collusion attack on quadratic voting that amplifies voting power by a factor of sqrt(2), a phantom-vote attack on mean voting that dilutes a project's mean score when zero allocations are excluded, and an analogous but potentially more severe phantom-vote attack on median voting. The paper then simulates these mechanisms with 133 voters, 374 projects, and Pareto(2.5)-distributed preferences, introducing a Pairwise Manipulation Score (PMS) to compare baseline and attacked allocations across voter and project attack scenarios. Based on the simulation results, the authors conclude that quadratic voting provides 'superior protection across all attack vectors, with manipulation scores consistently an order of magnitude lower than alternative mechanisms,' and they recommend quadratic voting for future RetroPGF rounds, with floors/caps as a fallback for mean and median. The paper also releases an open-source dashboard for replication and extension.

Significance. If the central claims are correct, the paper would offer concrete, actionable guidance for a high-stakes real-world mechanism design problem—Optimism's RetroPGF distributes substantial funds—and would contribute a comparative, proof-plus-simulation framework that is rare in the DAO governance literature. The explicit formal statements of vulnerabilities (even if simple) and the open-source simulation tool are genuine strengths, as is the paper's clear delineation of the conditions under which the phantom-vote attacks operate. However, the significance is currently limited by three gaps: the simulation equations for the mean and median attacks are infeasible as written, the simulated attacks do not actually match the theoretical phantom-vote attacks proved earlier, and the key assumption about exclusion of zero-value allocations is not verified against Optimism's real implementations. These issues bear directly on the empirical ordering that motivates the paper's central recommendation.

major comments (6)
  1. [Section 5.3.3, Eq. (10)] With w_v = 1 (as stated in Section 5.2) and ε = 0.01, |P| = 374, the target allocation w_v − (|P|−1)ε = 1 − 3.73 = −2.73 is negative, violating the non-negativity constraint A ≥ 0 from Section 3.2. If, on the other hand, the weights are normalized to Σ w_v = 1000, then w_v ≈ 7.52 and the target allocation is about 3.79, which is roughly half of the voter's budget and not a 'minimum viable' attack; the quadratic attack in Eq. (7), by contrast, only splits the voter's top two preferences. Either reading makes the comparison unfair or infeasible, so the reported manipulation scores in Section 6.2 do not support the ordering 'quadratic ≫ mean ≈ median'.
  2. [Section 5.3.2, Eqs. (7)-(8)] The quadratic attack formulas are inconsistent with the quadratic baseline in Eq. (6). For non-colluding voters, the formulas use M_{v,p}·w_v instead of sqrt(M_{v,p}·w_v), which is the quadratic transformation defined earlier. If this is a typo, the equations should be corrected; if it is intentional, the authors must justify why non-colluding votes are counted linearly in an otherwise quadratic mechanism, since this would bias the comparison in favor of quadratic voting.
  3. [Section 4.2, Remarks 2-3 and Section 7] The phantom-vote attacks rely on the assumption that zero-value allocations are excluded from the aggregation (Remark 3: 'n is the number of non-zero wallets'). The paper provides no evidence that Optimism's actual Round 2 mean voting or Rounds 3-4 median variants exclude zero ballots; Remark 2 concedes that the mean attack collapses to linear scaling when zeros are included, and Section 7 concedes that the real mechanisms used quorum caps and min/max thresholds. Without verification of this assumption against the real system, the external validity of the central recommendation is not established.
  4. [Sections 4.2-4.3 and 5.3.3-5.3.4] The simulation attacks are not the phantom-vote attacks proved in Section 4. The proofs involve adding small ε allocations from previously zero-allocating voters to the target project, while the simulations have existing voters reallocate their entire budgets, giving ε to all non-target projects and the remainder to the target. This is a preference-concentration attack, not a phantom-vote attack, so the simulation results do not empirically validate the theoretical vulnerabilities, and the claim that the simulations 'substantiate the theoretical vulnerabilities' (Section 6.2) is unsupported.
  5. [Section 4.1, Quadratic Collusion Attack] The proof's token accounting is inconsistent. The text states that 'two colluding voters have a total of T tokens,' but then sets P = Q = T, which requires each voter to have T tokens (i.e., 2T total). The intended result—that collusion yields a sqrt(2) amplification—is correct, but the proof should be rewritten with explicit per-voter budgets (e.g., each voter has one token) so that the constraint p1 + q1 = 1 and p2 + q2 = 1 is made clear.
  6. [Sections 3.3 and 4.2-4.3] The proofs in Section 4 analyze changes to raw mean and median scores, but the final allocation is normalized by the sum of all project scores (Section 3.3, e.g., Allocation = T · P_j / Σ P_j). An attack on one project also changes the denominator, so the derived ratios (e.g., n/(n+k) in Remark 3) do not directly translate to final allocation shares. The authors should state explicitly that the bounds apply to raw scores and discuss how normalization affects the magnitude of the vulnerability, or provide bounds on the actual funding allocation.
minor comments (5)
  1. [Section 5.2] The description of voter weights is contradictory: '∀v(wv = 1)' is followed by 'Then, voter weights are normalized such that Σ v w_v = c.' Since the sum of 133 unit weights is 133, not c = 1000, the authors should clarify the pre- and post-normalization weights actually used in the simulations.
  2. [Section 6.1, Observation 1] The numerical example contains a typo: 'If an adversary contributes 0 to both projects' appears to mean 'to this project,' and the resulting vector should have 9 entries, not 8. The example should be corrected to make the demonstration clear.
  3. [Section 5 and Section 6.2] There are several typos, including 'Mechnanism Simulations' in the Section 5 header and 'Obversation' in Section 6.2. The text also has inconsistent spacing (e.g., 'V oters') and should be proofread.
  4. [Section 5.4, Eq. (15)] In the definition of PMS, the paper should explicitly state which distribution is the baseline (p1) and which is the attack (p2) to avoid ambiguity in interpreting the score.
  5. [Throughout] The results labeled 'Remark' are effectively propositions or lemmas; consider using a standard numbered theorem environment to improve readability and formality. Also, the reference to Lalley and Weyl [15] is formatted inconsistently with the other references.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the proofs are direct algebra from the stated mechanism definitions, and the central claims rest on simulated attacks rather than on fitted parameters or self-citations.

full rationale

The paper's derivation chain is self-contained with respect to its inputs. Section 4's vulnerability results are conditional algebraic statements from the definitions in Section 3.3: Remark 2 shows mean voting with all wallets included is proportional to linear voting; Remark 3 computes the mean phantom-vote ratio n/(n+k) by direct summation; Remark 4 bounds median shifts by order-statistic arguments. No step equates a purported prediction with a fitted parameter, and no load-bearing claim is justified by a self-citation. The simulation framework uses an explicitly stated Pareto(2.5) preference model, w_v=1, and epsilon=0.01; these are assumptions, not parameters fitted to make quadratic voting win. The PMS metric is presented as a modification of MSE, not as a newly unificatory quantity. The main weaknesses are correctness and external-validity issues rather than circularity: with w_v=1, |P|=374, and epsilon=0.01, Eqs. (10)-(13) assign an attacking voter a target allocation w_v - (|P|-1)epsilon = -2.73, violating the Section 3.2 constraint A>=0; the simulated 'voter attacks' are fixed-voter concentration attacks rather than the phantom-voter attacks proved in Section 4; and Section 7 concedes that actual Rounds 3-4 used capped/quorum median variants not modeled in the headline comparison. These concerns undermine the quantitative cross-mechanism ordering in Section 6.3, but they do not make the derivation circular. The paper also candidly flags that testing modified median mechanisms is future work, which further supports a non-circular reading.

Assumptions & free parameters 3 free parameters · 6 assumptions · 0 invented entities

The central claims rest on a few modelling assumptions: rational utility-maximizing voters, a fixed voter set with no vote cost, and the choice of Pareto(2.5) for preference generation. The phantom-vote attacks additionally assume zero allocations are excluded from the aggregate, which is a domain assumption not verified for Optimism's actual rounds. No entity is invented; no constant is fitted to the reported manipulation scores.

free parameters (3)
  • Pareto shape alpha = 2.5
    Chosen to model moderate preference concentration and finite variance, described as matching empirical observations (Section 5.2); not fitted to the target result but substantially determines the simulated preference distribution.
  • epsilon = 0.01
    The minimum allocation used in simulated phantom attacks (Sections 5.3.3-5.3.4); its value affects the magnitude of the reported attack scores.
  • normalization constant c = 1000
    Chosen large enough for floating point stability when normalizing voter weights (Section 5.2); this is an implementation detail rather than a substantive modelling parameter.
assumptions (6)
  • domain assumption Voters are rational expected-utility maximizers.
    Stated in Section 3.1, assumption 5.
  • domain assumption A voter cannot split their funds into multiple voting wallets.
    Stated in Section 3.1, assumption 2; matches Optimism's badged-identity model.
  • domain assumption No costs are associated with casting a vote.
    Section 3.2: 'We assume that there are no costs associated with casting a vote.'
  • ad hoc to paper Zero-value allocations are excluded from mean and median aggregation in the phantom-vote attacks.
    Remarks 3 and 4 require n to count only non-zero wallets; the paper does not verify this for Optimism's actual rounds.
  • ad hoc to paper Voter preferences follow a Pareto distribution with shape 2.5.
    Section 5.2, Equation (2); chosen to model preference concentration, without direct empirical validation in this paper.
  • standard math Standard social-choice background results (Arrow, Gibbard-Satterthwaite, median voter theorem).
    Used in Section 2 to frame the analysis; not re-derived in the paper.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Evaluating Voting Design Vulnerabilities for Retroactive Funding." pith.science (2026). https://pith.science/paper/VW62MN7D

@misc{pith2026250516068,
  author       = {Pith},
  title        = {Pith review of: Evaluating Voting Design Vulnerabilities for Retroactive Funding},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/VW62MN7D}},
  note         = {Machine review of arXiv:2505.16068}
}
read the original abstract

Retroactive Public Goods Funding (RetroPGF) rewards blockchain projects based on proven impact rather than future promises. This paper reviews voting mechanisms for Optimism's RetroPGF, where "badgeholders" allocate rewards to valuable projects. We explore Optimism's previous schemes for RetroPGF voting, including quadratic, mean, and median voting. We present a proof-based formal analysis for vulnerabilities in these voting schemes, empirically validate these vulnerabilities using voting simulations, and offer assessments and practical recommendations for future iterations of Optimism's system based on our findings.

Figures

Figures reproduced from arXiv: 2505.16068 by the authors.

Figure 1
Figure 1. Distribution of manipulation scores across voting mechanisms. Each row shows a different [PITH_FULL_IMAGE:figures/full_fig_p013_1.png] view at source ↗

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

19 extracted references · 17 canonical work pages

  1. [1]

    Kenneth J. Arrow. Social Choice and Individual Values. Yale University Press, New Haven, 1951

    work page 1951

  2. [2]

    RPGF Recipients

    Optimism Atlas. RPGF Recipients. https://atlas.optimism.io/round/results. Ac- cessed on: 2025-01-31

    work page 2025

  3. [3]

    DAO Decentralization: V oting-Bloc Entropy, Bribery, and Dark DAOs.arXiv preprint, 2023

    James Austgen, Andrés Fábrega, Sarah Allen, Kushal Babel, Mahimna Kelkar, and Ari Juels. DAO Decentralization: V oting-Bloc Entropy, Bribery, and Dark DAOs.arXiv preprint, 2023. https://arxiv.org/abs/2311.03530

    arXiv 2023

  4. [4]

    On the rationale of group decision-making

    Duncan Black. On the rationale of group decision-making. Journal of Political Economy, 56(1):23–34, 1948

    work page 1948

  5. [5]

    Condorcet: foundations of social choice and political theory

    Jean-Antoine-Nicolas de Caritat Condorcet, Iain McLean, and Fiona Hewitt. Condorcet: foundations of social choice and political theory . E. Elgar, Aldershot, Hants, England ; Brookfield, Vt, 1994

    work page 1994

  6. [6]

    DeepDAO Organizations

    DeepDAO. DeepDAO Organizations. https://deepdao.io/organizations. Accessed 2024-11-02

    work page 2024

  7. [7]

    Governance

    Compound Docs. Governance. https://docs.compound.finance/v2/governance/. Ac- cessed 2024-10-22

    work page 2024

  8. [8]

    How retro funding works

    Optimism Citizen House Docs. How retro funding works. https://community.optimism. io/citizens-house/how-retro-funding-works . Accessed 2024-11-02

    work page 2024

Show all 19 references
  1. [9]

    V ox populi

    Francis Galton. V ox populi. Nature, 75:450–451, 1907

    1907

  2. [10]

    Manipulation of voting schemes: A general result

    Allan Gibbard. Manipulation of voting schemes: A general result. Econometrica, 41(4):587– 601, 1973

    1973

  3. [11]

    Optimism Retro Funding Round 3

    Optimism Governance. Optimism Retro Funding Round 3. https://community.optimism. io/citizens-house/rounds/retropgf-3 . Accessed on: 2025-02-03

    2025

  4. [12]

    Optimism Retro Funding Round 4

    Optimism Governance. Optimism Retro Funding Round 4. https://community.optimism. io/citizens-house/rounds/retropgf-4 . Accessed on: 2025-02-03

    2025

  5. [13]

    Hall and Eliza R

    Andrew B. Hall and Eliza R. Oak. What Kinds of Incentives Encourage Participation in Democracy? Evidence from a Massive Online Governance Experiment. 2023. https:// andrewbenjaminhall.com/Hall_Oak_Airdrop_Effects_on_Participation.pdf

    2023

  6. [14]

    Decentralized autonomous organization

    Samer Hassan and Primavera De Filippi. Decentralized autonomous organization. Internet Policy Review, 10(2):1–10, 2021

    2021

  7. [15]

    Glen Weyl

    Steven Lalley and E. Glen Weyl. Quadratic voting: How mechanism design can radicalize democracy. American Economic Association Papers and Proceedings, 1(1):1–5, 2018

    2018

  8. [16]

    Social Choice Theory

    Christian List. Social Choice Theory. In Edward N. Zalta and Uri Nodelman, editors, The Stanford Encyclopedia of Philosophy. Metaphysics Research Lab, Stanford University, Win- ter 2022 edition, 2022. https://plato.stanford.edu/archives/win2022/entries/ social-choice/

    2022

  9. [17]

    Posner and E

    Eric A. Posner and E. Glen Weyl. Quadratic voting and the public good, 2017. Accessed on: 2024-12-05

    2017

  10. [18]

    Unpacking how decentralized autonomous organizations (daos) work in practice

    Tanusree Sharma, Yujin Kwon, Kornrapat Pongmala, Henry Wang, Andrew Miller, Dawn Song, and Yang Wang. Unpacking how decentralized autonomous organizations (daos) work in practice. ArXiv Preprint, 2023. https://arxiv.org/abs/2304.09822

    2023 arXiv

  11. [19]

    The Wisdom of Crowds

    James Surowiecki. The Wisdom of Crowds. Knopf Doubleday Publishing Group, Westminster, 2005. 16

    2005

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.