Pith. sign in

REVIEW 3 major objections 4 minor 54 references

Private Rank Aggregation under Local Differential Privacy

T0 review · 3 major / 4 minor · reviewed 2026-08-14 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read This paper claims rank aggregation can be made locally differentially private without a trusted curator, with randomized pairwise answers and a provable error bound.

desk verdict First local-DP rank aggregation paper, but Algorithm 1 as printed never uses the agent's true answer, so the main utility theorem doesn't apply to it; fixable but load-bearing. read the letter →

arxiv 1908.04486 v3 pith:HPATTTAZ submitted 2019-08-13 cs.DS cs.CRcs.MA

Ziqi Yan , Gang Li , Jiqiang Liu This is my paper · ORCID
classification cs.DScs.CRcs.MA
keywords localdifferentialprivacyrankaggregationKwikSortrandomizedresponseLaplacemechanismMallowsmodelpairwisecomparisonsKendalltaudistance
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Rank aggregation over crowdsourced preferences usually requires a trusted curator who sees everyone's full rankings. This paper claims that the curator can be untrusted: each agent can randomly perturb answers to pairwise comparison queries, and the server can still recover an aggregate ranking with acceptable accuracy. The proposed protocol, LDP-KwikSort, comes in two variants, one built on randomized response and one on the Laplace mechanism, and the randomized-response variant satisfies epsilon-local differential privacy while the Laplace variant satisfies epsilon-local individual differential privacy. The authors prove an error bound under a Mallows model of ranking preferences and show empirically that the randomized-response variant is close to the central-model private baseline while protecting each agent's pairwise preferences.

What carries the argument

The machinery is KwikSort, an approximate rank aggregation algorithm that only needs the aggregate pairwise comparison profile $\mathrm{cmp}(L) = (C_{a_j a_l}(L) - C_{a_l a_j}(L))$ for each pair; the sign of each entry decides the relative order, and KwikSort sorts with a randomized pivot to get an $11/7$-approximation to the Kemeny optimal ranking. LDP-KwikSort perturbs each agent's pairwise answer with either Randomized Response or Laplace noise, then the curator estimates $\widehat{\mathrm{cmp}}(L)$ from the noisy counters and feeds it to KwikSort. The Mallows model provides the generative assumption under which the sign of $\widehat{\mathrm{cmp}}(L)$ matches $\mathrm{cmp}(L)$ with high probability, and Hoeffding and Chernoff bounds turn that per-pair probability into the global error bound.

What would settle it

Run LDP-KwikSort:RR on Mallows-generated rankings with $m=45$ alternatives, $n=5000$ agents, $\epsilon=2$, and $K=1$, and record the fraction of pairs $(a_j, a_l)$ with zero respondents and the empirical sign-error rate of $\widehat{\mathrm{cmp}}(L)$. If the fraction of unqueried pairs is substantial while the Theorem 2 bound predicts error below $6\mu$ with probability $1-2^{-6\mu}$, then the bound and protocol disagree in a way that would falsify the utility guarantee as stated.

Watch

Extended reading notes

Core claim

The paper's central claim is that locally private rank aggregation is feasible: instead of trusting the curator with full rankings, each agent answers K randomly chosen pairwise comparison queries with a locally randomized response. Theorem 1 states that LDP-KwikSort:RR satisfies epsilon-LDP and LDP-KwikSort:Lap satisfies epsilon-LiDP. Theorem 2 states that when rankings come from the Mallows model, the estimate of the aggregate pairwise comparison profile has error below $6\mu$ with probability at least $1 - 2^{-6\mu}$, where $\mu = 2\binom{m}{2} \exp\left(-\frac{\epsilon^2 K}{(\epsilon+2K)^2} \cdot \frac{{\theta^*}^2 n}{m(m-1)}\right)$ and $\theta^* = \frac{\theta}{2-\theta}$. The Laplace variant has an analogous bound with a different exponent. The paper further claims that the error is minimized near $K = \epsilon/2$, and that experiments on real and synthetic data show LDP-KwikSort:RR keeps aggregate ranking utility close to DP-KwikSort and non-private KwikSort.

Load-bearing premise

The load-bearing premise is that every pair of alternatives receives exactly $n^* = nK/\binom{m}{2}$ independent noisy answers, each with the same probability of being truthful; the actual protocol instead has each agent answer only $K$ random pairs, leaving many pairs unanswered and linking answers within an agent, so the proven error bound does not cover the protocol as run.

Editorial extensions

If this is right

  • If the protocol works as claimed, crowdsourcing platforms can collect ranking preferences from agents who do not trust the platform, with each agent's pairwise comparisons protected by local differential privacy.
  • The recommended operating point $K \approx \epsilon/2$ follows directly from the analysis, giving practitioners a concrete rule for setting the number of queries per agent.
  • The randomized-response variant is the better of the two solutions in the experiments, so a practical deployment need not rely on Laplace noise to reach acceptable utility.
  • Utility improves as the number of agents or the privacy budget grows, and degrades as the number of alternatives grows, matching the intuition that more information outweighs noise.
  • Because KwikSort only needs pairwise comparisons, the protocol inherits a low per-agent cost of $O(K)$ computation and $O(\log K)$ communication.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The paper's Theorem 2 assumes every pair is answered by exactly $n^* = nK/\binom{m}{2}$ independent agents, but Algorithm 1 has each agent answer only $K$ randomly chosen pairs; when $K$ is small relative to $\binom{m}{2}$, many pairs receive zero answers and the comparison is effectively a coin flip, so the bound may be optimistic in exactly the regime the paper recommends for large $m$ or small
  • A direct extension would replace uniform random pair sampling with a covering design or repeated sampling so that every pair is queried at least once, which would make the theorem's assumption match the protocol and likely improve practical accuracy.
  • The analysis could be tightened by accounting for the correlation among the $K$ answers from the same agent; treating all $nK$ answers as independent likely overstates the effective sample size.
  • If the missing-data gap is closed, a similar protocol could be applied to other pairwise aggregation problems, such as tournament-style preference elicitation or peer grading, where an untrusted aggregator is the norm.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 4 minor

Summary. The paper introduces LDP-KwikSort, a protocol for rank aggregation under local differential privacy. The curator asks each agent K pairwise comparison queries; the agent perturbs each answer with randomized response (RR) or Laplace noise, and the curator estimates the aggregate pairwise comparison profile and runs the KwikSort algorithm to produce a ranking. The main theoretical claims are Theorem 1 (ε-LDP for the RR instantiation and ε-LiDP for the Laplace instantiation), Theorem 2 (a utility bound for LDP-KwikSort:RR under the Mallows model, with error below 6μ with high probability, where μ is minimized near K = ε/2), and Theorem 3 (an analogous bound for LDP-KwikSort:Lap). The paper also reports experiments on three real-world datasets and Mallows-generated synthetic data, comparing against non-private KwikSort and the central-model DP-KwikSort.

Significance. If the central claims were fully supported, the paper would make a useful contribution: it is, to my knowledge, the first to formulate rank aggregation under the local model of differential privacy, and the idea of leveraging KwikSort to reduce the number of pairwise queries per agent is natural and relevant. The privacy guarantee for the Laplace variant is standard and sound, and the experiments on real crowdsourcing datasets are a strength. However, the utility analysis as written does not apply to the RR algorithm actually specified, because that algorithm ignores the agent's true answer; the utility theorem also relies on assumptions about response counts and independence that the protocol does not satisfy. These issues are load-bearing rather than cosmetic.

major comments (3)
  1. [Section 4.3, Algorithm 1; Theorem 2 and Eq. (3)] Algorithm 1 as printed does not implement randomized response. In lines 2–8, the agent draws g uniformly from [0,1] and sets the reported bit to 1 if g ≤ p_rr and to 0 otherwise; the true answer o_ik never appears. Thus the reported bit is Bernoulli(p_rr) independent of the agent's preference. The proof of Theorem 2, however, assumes that each agent reports the true answer with probability p_rr and the false answer with probability q_rr, leading to p = p_M p_rr + q_M q_rr for agreement with the ground truth. Under the literal algorithm, p = p_rr, so p − q = 0, the exponent in Eq. (3) is zero, P_ι ≈ 1/2, and the sign of the estimated comparison is no better than random for every pair. Consequently, the utility guarantee of Theorem 2 does not hold for the described protocol, and K ≈ ε/2 as an optimal query count is unsupported by that argument. The privacy claim remains true because a data-independent output is private, but the central utility claim fails even if every pair were answered by exactly n* = nK/C(m,2) agents.
  2. [Section 5.2.1, Theorem 2 proof and Algorithm 2] The utility proof assumes that for each pair (a_j, a_l) there are exactly n* = nK/C(m,2) respondents and that the noisy per-pair counts are independent Bernoulli variables. The protocol in Section 4.2 and Algorithm 2 has each agent answer K randomly selected pairs, so the number of respondents per pair is random and can be zero; pairs with no respondents have ~C_{a_j a_l} = ~C_{a_l a_j} = 0, which corresponds to an effectively random sign. Moreover, if one agent answers several pairs, the resulting pairwise error indicators X_ι are correlated, so the Chernoff bound in Lemma 2, which requires independent Poisson trials, cannot be applied as written. These are first-order effects, not approximation artifacts: for large m or small n, a substantial fraction of pairs will receive no answers. The theorem must either analyze the actual random assignment of pairs to agents or explicitly state and justify a design in which every pair receives n* responses that are independent across pairs.
  3. [Section 5.2.1, Mallows model definition] The paper defines p_M as a single constant such that for every pair (a_j, a_l), the relative order of the pair agrees with the ground truth with probability p_M and disagrees with probability q_M = 1 − p_M, with θ = 1 − q_M/p_M. This is not a property of the standard Mallows model cited as [35]. In the Mallows model with Kendall-tau distance, the marginal probability that a given item precedes another item in the generated ranking is not constant across pairs; for example, for m = 3 and θ = 0.5 the probability that item 1 precedes item 2 differs from the probability that item 1 precedes item 3. Consequently, the expression for P_ι in Eq. (3) is not a valid per-pair error probability for all pairs under the Mallows model, and the subsequent bound μ = Σ P_ι ≤ 2(m choose 2) exp(...) requires an argument that either uses pair-specific probabilities and takes a maximum or justifies a different model. This issue affects the formal content of Theorem 2 and, through it, the theoretical rationale for the recommended choice K ≈ ε/2.
minor comments (4)
  1. [Section 5.1, Theorem 1 proof] The displayed ratio for the RR mechanism is written as 2 p_rr / (1 − p_rr) = e^{ε_k}; for standard randomized response the correct ratio is p_rr / (1 − p_rr) = e^{ε_k}. The theorem statement is unaffected, but the proof as written is algebraically incorrect.
  2. [Algorithm 2] The pseudocode does not record which pair each ~o_ik refers to; the classification loop increments ~C_{a_j a_l}(L) or ~C_{a_l a_j}(L) without specifying a mechanism for mapping the query index k to the pair (a_j, a_l). This is an implementation detail, but it should be made explicit because the protocol uses different random pairs for different agents.
  3. [Section 6.3] The text says DP-KwikSort is “based on the local model of DP,” but Section 6.1.1 and the central-model definition state that DP-KwikSort is a central-model algorithm. This appears to be a typographical error and should be corrected.
  4. [Lemma 2 usage] The Chernoff bound is stated for R ≥ 6μ, but the theorem applies it with R = 6μ; if R must be an integer, this requires a short justification or a ceiling on R. This is minor relative to the issues above but should be cleaned up.

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity; the derivation is self-contained, and the noted Algorithm 1 defect is a correctness gap rather than a circular reduction.

full rationale

The paper's claimed derivation chain is self-contained against external benchmarks. KwikSort, randomized response, the Laplace mechanism, and the Mallows model are all prior external results, and the paper's contribution is a composition of them. The K = epsilon/2 recommendation comes from maximizing the analytic function g(K) = epsilon^2 K/(epsilon+2K)^2 in Theorem 2, not from fitting to experimental data; the experiments test the predicted trend, which is the opposite of circularity. The only relevant self-citation, reference [52] on weighted voting, appears in related work and is not load-bearing for the LDP-KwikSort privacy or utility theorems. The reviewer-noted defect in Algorithm 1 (the reported bit is generated independently of the true answer) is a correctness gap between the pseudocode and the randomized-response model assumed in Theorem 2's proof, not a circular reduction: it makes the utility bound inapplicable to the printed algorithm rather than making a conclusion equal to an input by construction. No equation or fitted parameter is reused as the predicted output, so no circularity is present.

Assumptions & free parameters 0 free parameters · 5 assumptions · 0 invented entities

The protocol introduces no new entities or fitted constants. The load-bearing assumptions are the Mallows data model, the deterministic per-pair response count, and independence of noisy counts; all three enter the utility analysis rather than the privacy proof. The privacy guarantee itself rests on standard composition and does not require these assumptions.

assumptions (5)
  • domain assumption All agent rankings are generated by the Mallows model with a common dispersion parameter theta and a single ground-truth ranking.
    Theorem 2 and Theorem 3 derive error bounds only under this model; Section 5.2.1 states 'based on the assumption that the agents' ranking preference profiles are generated by the Mallows model.' Real datasets need not satisfy it.
  • ad hoc to paper For every pair of alternatives, exactly n* = nK/C(m,2) agents respond to that pair.
    The proof of Theorem 2 says 'there will be n* agents involved' for a certain pairwise comparison query. In Algorithm 1 the curator samples K random pairs per agent, so the number of respondents per pair is binomial and can be zero, which the proof ignores.
  • ad hoc to paper The noisy per-pair counts are independent Bernoulli variables and the pairwise error indicators X_iota are independent across pairs.
    Hoeffding and Chernoff bounds in Theorem 2 require independence; a single agent answers K pairs, so answers within an agent are correlated, and the X_iota are not independent as used in Lemma 2.
  • standard math Sequential composition of K local randomizers each with budget epsilon/K yields the stated epsilon-LDP guarantee.
    Definition 3 and Theorem 1 rely on the standard composition property of differential privacy, which is background in the field.
  • standard math KwikSort achieves an 11/7 approximation on the true aggregate pairwise comparison profile.
    The paper cites Ailon et al. [1,2] for this guarantee; it is invoked in Section 5.2.1 to justify focusing on estimation error of cmp(L).

how reviews work

0 comments
Cite this review

Pith. "Pith review of Private Rank Aggregation under Local Differential Privacy." pith.science (2026). https://pith.science/paper/HPATTTAZ

@misc{pith2026190804486,
  author       = {Pith},
  title        = {Pith review of: Private Rank Aggregation under Local Differential Privacy},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/HPATTTAZ}},
  note         = {Machine review of arXiv:1908.04486}
}
read the original abstract

As a method for answer aggregation in crowdsourced data management, rank aggregation aims to combine different agents' answers or preferences over the given alternatives into an aggregate ranking which agrees the most with the preferences. However, since the aggregation procedure relies on a data curator, the privacy within the agents' preference data could be compromised when the curator is untrusted. Existing works that guarantee differential privacy in rank aggregation all assume that the data curator is trusted. In this paper, we formulate and address the problem of locally differentially private rank aggregation, in which the agents have no trust in the data curator. By leveraging the approximate rank aggregation algorithm KwikSort, the Randomized Response mechanism, and the Laplace mechanism, we propose an effective and efficient protocol LDP-KwikSort. Theoretical and empirical results show that the solution LDP-KwikSort:RR can achieve the acceptable trade-off between the utility of aggregate ranking and the privacy protection of agents' pairwise preferences.

Figures

Figures reproduced from arXiv: 1908.04486 by the authors.

Figure 1
Figure 1. Comparison of two DP models cmp(L) in the LDP-KwikSort protocol, we show that the utility can achieve the approximate maximum value around K =  2 , which is then further verified by extensive experiments. 3. For performance evaluation, we conduct experiments on three real-world datasets (TurkDots, TurkPuzzle and SUSHI) and several synthetic datasets generated from the Mallows model. By observing the error rate and … view at source ↗
Figure 2
Figure 2. Rationale for LDP-KwikSort:RR Solution 4.3 Local Perturbation As shown in Algorithm 1, when receiving the K queries, an agent i ∈ N adopts the standard RR mechanism as the local randomizer MR, and reports the curator with the perturbed answer oei . Here, we adopt the transformation matrix Mrr introduced in Section 2.3 to include the coin flipping probabilities poeik,oik of a true answer oik being transformed into th… view at source ↗
Figure 3
Figure 3. Performance of LDP-KwikSort:RR in terms of error rate and average Kendall tau distance on the real-world and synthetic datasets, across varying the number of queries K. 18 [PITH_FULL_IMAGE:figures/full_fig_p018_3.png] view at source ↗
Figures from the paper (3 more)
Figure 4
Figure 4. Figure 4: Performance of LDP-KwikSort:Lap in terms of error rate and average Kendall tau distance on the real-world and synthetic datasets, across varying the number of queries K. 19 [PITH_FULL_IMAGE:figures/full_fig_p019_4.png]
Figure 5
Figure 5. Figure 5: Comparison of protocols in terms of error rate and average Kendall tau distance on the [PITH_FULL_IMAGE:figures/full_fig_p020_5.png]
Figure 6
Figure 6. Figure 6: Comparison of protocols in terms of average Kendall tau distance on the real-world and [PITH_FULL_IMAGE:figures/full_fig_p021_6.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

54 extracted references · 51 canonical work pages

  1. [35]

    Non-null ranking models

    Colin L Mallows. Non-null ranking models. Biometrika, 44(1/2):114–130, 1957

    work page 1957

  2. [1]

    Aggregating inconsistent information: ranking and clustering

    Nir Ailon, Moses Charikar, and Alantha Newman. Aggregating inconsistent information: ranking and clustering. In STOC, pages 684–693. ACM, 2005

    work page 2005

  3. [2]

    Aggregating inconsistent information: Ranking and clustering

    Nir Ailon, Moses Charikar, and Alantha Newman. Aggregating inconsistent information: Ranking and clustering. J. ACM, 55(5):23:1–23:27, 2008

    2008

  4. [3]

    Crowdsourcing as a forensic psychology research tool

    Melissa A Baker, Paul Fox, and T Wingrove. Crowdsourcing as a forensic psychology research tool. American Journal of Forensic Psychology , 34(1):37–50, 2016

    work page 2016

  5. [4]

    Practical locally private heavy hitters

    Raef Bassily, Kobbi Nissim, Uri Stemmer, and Abhradeep Guha Thakurta. Practical locally private heavy hitters. In NIPS, pages 2285–2293, 2017

    work page 2017

  6. [5]

    Conducting psychology student research via the mechanical turk crowdsourcing service

    John A Bates and Brian A Lanza. Conducting psychology student research via the mechanical turk crowdsourcing service. North American Journal of Psychology , 15(2), 2013

    work page 2013

  7. [6]

    The viability of crowdsourcing for survey research

    Tara S Behrend, David J Sharek, Adam W Meade, and Eric N Wiebe. The viability of crowdsourcing for survey research. Behavior Research Methods, 43(3):800, 2011

    work page 2011

  8. [7]

    A Practical Guide to Averaging Functions, volume 329 of Studies in Fuzziness and Soft Computing

    Gleb Beliakov, Humberto Bustince Sola, and Tomasa Calvo. A Practical Guide to Averaging Functions, volume 329 of Studies in Fuzziness and Soft Computing . Springer, 2016

    work page 2016

Show all 54 references
  1. [8]

    Rank aggregation with ties: Experiments and analysis

    Bryan Brancotte, Bo Yang, Guillaume Blin, Sarah Cohen Boulakia, Alain Denise, and Sylvie Hamel. Rank aggregation with ties: Experiments and analysis. PVLDB, 8(11):1202–1213, 2015

    2015

  2. [9]

    Procaccia, editors

    Felix Brandt, Vincent Conitzer, Ulle Endriss, J´ erˆ ome Lang, and Ariel D. Procaccia, editors. Handbook of Computational Social Choice . Cambridge University Press, 2016

    2016

  3. [10]

    Randomized Response and Indirect Questioning Techniques in Surveys

    Arijit Chaudhuri. Randomized Response and Indirect Questioning Techniques in Surveys . Chapman and Hall/CRC, 2016

    2016

  4. [11]

    Kash, Tal Moran, and Salil P

    Yiling Chen, Stephen Chong, Ian A. Kash, Tal Moran, and Salil P. Vadhan. Truthful mechanisms for agents that value privacy. In EC, pages 215–232. ACM, 2013

    2013

  5. [12]

    Kash, Tal Moran, and Salil P

    Yiling Chen, Stephen Chong, Ian A. Kash, Tal Moran, and Salil P. Vadhan. Truthful mechanisms for agents that value privacy. ACM Trans. Economics and Comput. , 4(3):13:1–13:30, 2016

    2016

  6. [13]

    Ordering lists based on aggregated pairwise preferences, 2017

    David Cortes. Ordering lists based on aggregated pairwise preferences, 2017. https://pypi.org/project/pwlistorder/

    2017

  7. [14]

    Duchi, Michael I

    John C. Duchi, Michael I. Jordan, and Martin J. Wainwright. Privacy aware learning. In NIPS, pages 1439–1447, 2012

    2012

  8. [15]

    Duchi, Michael I

    John C. Duchi, Michael I. Jordan, and Martin J. Wainwright. Local privacy and statistical minimax rates. In FOCS, pages 429–438. IEEE Computer Society, 2013

    2013

  9. [16]

    Differential privacy

    Cynthia Dwork. Differential privacy. In ICALP (2), volume 4052 of Lecture Notes in Computer Science, pages 1–12. Springer, 2006

    2006

  10. [17]

    The algorithmic foundations of differential privacy.Foundations and Trends in Theoretical Computer Science , 9(3-4):211–407, 2014

    Cynthia Dwork and Aaron Roth. The algorithmic foundations of differential privacy.Foundations and Trends in Theoretical Computer Science , 9(3-4):211–407, 2014. 24

    2014

  11. [18]

    Instability vs anonymization in e pluribus hugo, 2015

    David Eppstein. Instability vs anonymization in e pluribus hugo, 2015. https://11011110. github.io/blog/2015/09/09/instability-vs-anonymization.html

    2015

  12. [19]

    Greenberg, Abdel-Latif A

    Bernard G. Greenberg, Abdel-Latif A. Abul-Ela, Walt R. Simmons, and Daniel G. Horvitz. The unrelated question randomized response model: Theoretical framework. Journal of the American Statistical Association, 64(326):520–539, 1969

    1969

  13. [20]

    Groat, Benjamin Edwards, James Horey, Wenbo He, and Stephanie Forrest

    Michael M. Groat, Benjamin Edwards, James Horey, Wenbo He, and Stephanie Forrest. Appli- cation and analysis of multidimensional negative surveys in participatory sensing applications. Pervasive and Mobile Computing , 9(3):372–391, 2013

    2013

  14. [21]

    Differentially private rank aggregation

    Michael Hay, Liudmila Elagina, and Gerome Miklau. Differentially private rank aggregation. In SDM, pages 669–677. SIAM, 2017

    2017

  15. [22]

    Leith, and Oliver Mason

    Naoise Holohan, Douglas J. Leith, and Oliver Mason. Optimal differentially private mechanisms for randomised response. IEEE Trans. Information Forensics and Security , 12(11):2726–2735, 2017

    2017

  16. [23]

    Optrr: Optimizing randomized response schemes for privacy- preserving data mining

    Zhengli Huang and Wenliang Du. Optrr: Optimizing randomized response schemes for privacy- preserving data mining. In ICDE, pages 705–714. IEEE Computer Society, 2008

    2008

  17. [24]

    Ekhine Irurozki, Borja Calvo, and Jose A. Lozano. Permallows: An r package for mallows and generalized mallows models. Journal of Statistical Software , 71(i12), 2016

    2016

  18. [25]

    Nantonac collaborative filtering: recommendation based on order responses

    Toshihiro Kamishima. Nantonac collaborative filtering: recommendation based on order responses. In KDD, pages 583–588. ACM, 2003

    2003

  19. [26]

    Ruogu Kang, Stephanie Brown, Laura Dabbish, and Sara B. Kiesler. Privacy attitudes of mechanical turk workers and the U.S. public. In SOUPS, pages 37–49. USENIX Association, 2014

    2014

  20. [27]

    Lee, Kobbi Nissim, Sofya Raskhodnikova, and Adam D

    Shiva Prasad Kasiviswanathan, Homin K. Lee, Kobbi Nissim, Sofya Raskhodnikova, and Adam D. Smith. What can we learn privately? In FOCS, pages 531–540. IEEE Computer Society, 2008

    2008

  21. [28]

    Mechanical turk is not anonymous

    Matthew Lease, Jessica Hullman, Jeffrey Bigham, Michael Bernstein, Juho Kim, Walter Lasecki, Saeideh Bakhshi, Tanushree Mitra, and Robert Miller. Mechanical turk is not anonymous. Available at SSRN 2228728 , 2013

    2013

  22. [29]

    Efficient, private, and eps-strategyproof elicitation of tournament voting rules

    David Timothy Lee. Efficient, private, and eps-strategyproof elicitation of tournament voting rules. In IJCAI, pages 2026–2032. AAAI Press, 2015

    2026

  23. [30]

    Franklin

    Guoliang Li, Jiannan Wang, Yudian Zheng, and Michael J. Franklin. Crowdsourced data management: A survey. IEEE Trans. Knowl. Data Eng. , 28(9):2296–2319, 2016

    2016

  24. [31]

    Differential Privacy: From Theory to Practice

    Ninghui Li, Min Lyu, Dong Su, and Weining Yang. Differential Privacy: From Theory to Practice. Synthesis Lectures on Information Security, Privacy, & Trust. Morgan & Claypool Publishers, 2016. 25

    2016

  25. [32]

    An efficient two-layer mechanism for privacy-preserving truth discovery

    Yaliang Li, Chenglin Miao, Lu Su, Jing Gao, Qi Li, Bolin Ding, Zhan Qin, and Kui Ren. An efficient two-layer mechanism for privacy-preserving truth discovery. In KDD, pages 1705–1714. ACM, 2018

    2018

  26. [33]

    Towards differentially private truth discovery for crowd sensing systems.CoRR, abs/1810.04760, 2018

    Yaliang Li, Houping Xiao, Zhan Qin, Chenglin Miao, Lu Su, Jing Gao, Kui Ren, and Bolin Ding. Towards differentially private truth discovery for crowd sensing systems.CoRR, abs/1810.04760, 2018

    2018 arXiv

  27. [34]

    How private is your voting? A framework for comparing the privacy of voting mechanisms

    Ao Liu, Yun Lu, Lirong Xia, and Vassilis Zikas. How private is your voting? A framework for comparing the privacy of voting mechanisms. CoRR, abs/1805.05750, 2018

    2018 arXiv

  28. [36]

    Procaccia, and Yiling Chen

    Andrew Mao, Ariel D. Procaccia, and Yiling Chen. Better human computation through principled voting. In AAAI. AAAI Press, 2013

    2013

  29. [37]

    Using online, crowdsourcing platforms for data collection in personality disorder research: The example of amazons mechanical turk

    Joshua D Miller, Michael Crowe, Brandon Weiss, Jessica L Maples-Keller, and Donald R Lynam. Using online, crowdsourcing platforms for data collection in personality disorder research: The example of amazons mechanical turk. Personality Disorders: Theory, Research, and Treatmen...

    2017

  30. [38]

    Kobbi Nissim, Sofya Raskhodnikova, and Adam D. Smith. Smooth sensitivity and sampling in private data analysis. In STOC, pages 75–84. ACM, 2007

    2007

  31. [39]

    Beyond the turk: Alternative platforms for crowdsourcing behavioral research

    Eyal Peer, Laura Brandimarte, Sonam Samat, and Alessandro Acquisti. Beyond the turk: Alternative platforms for crowdsourcing behavioral research. Journal of Experimental Social Psychology, 70:153–163, 2017

    2017

  32. [40]

    Amazon mechanical turk crowdsourcing: A research tool in mobile internet era

    Kaiping Peng, Shiqun Liu, and Shiguang Ni. Amazon mechanical turk crowdsourcing: A research tool in mobile internet era. Journal of Northwest Normal University (Social Sciences) , v.55No.259(3):115–125, 2018

    2018

  33. [41]

    ”it was a shady hit”: Navigating work-related privacy concerns on mturk

    Shruti Sannon and Dan Cosley. ”it was a shady hit”: Navigating work-related privacy concerns on mturk. In CHI Extended Abstracts. ACM, 2018

    2018

  34. [42]

    Differential private data collection and analysis based on randomized multiple dummies for untrusted mobile crowdsensing

    Yuichi Sei and Akihiko Ohsuga. Differential private data collection and analysis based on randomized multiple dummies for untrusted mobile crowdsensing. IEEE Trans. Information Forensics and Security, 12(4):926–939, 2017

    2017

  35. [43]

    Kulkarni

    Shang Shang, Tiance Wang, Paul Cuff, and Sanjeev R. Kulkarni. The application of differential privacy for rank aggregation: Privacy and accuracy. In FUSION, pages 1–7. IEEE, 2014

    2014

  36. [44]

    Using crowdsourcing websites for sociological research: The case of amazon mechanical turk

    Daniel B Shank. Using crowdsourcing websites for sociological research: The case of amazon mechanical turk. The American Sociologist, 47(1):47–55, 2016

    2016

  37. [45]

    Individual differential privacy: A utility-preserving formulation of differential privacy guarantees

    Jordi Soria-Comas, Josep Domingo-Ferrer, David S´ anchez, and David Meg´ ıas. Individual differential privacy: A utility-preserving formulation of differential privacy guarantees. IEEE Trans. Information Forensics and Security, 12(6):1418–1429, 2017. 26

    2017

  38. [46]

    Aggregating votes with local differential privacy: Usefulness, soundness vs

    Shaowei Wang, Jiachun Du, Wei Yang, Xinrong Diao, Zichun Liu, Yiwen Nie, Liusheng Huang, and Hongli Xu. Aggregating votes with local differential privacy: Usefulness, soundness vs. indistinguishability. CoRR, abs/1908.04920, 2019

    1908 arXiv

  39. [47]

    Locally differentially private protocols for frequency estimation

    Tianhao Wang, Jeremiah Blocki, Ninghui Li, and Somesh Jha. Locally differentially private protocols for frequency estimation. In USENIX Security Symposium, pages 729–745. USENIX Association, 2017

    2017

  40. [48]

    Using randomized response for differential privacy preserving data collection

    Yue Wang, Xintao Wu, and Donghui Hu. Using randomized response for differential privacy preserving data collection. In EDBT/ICDT Workshops , volume 1558 of CEUR Workshop Proceedings. CEUR-WS.org, 2016

    2016

  41. [49]

    Stanley L. Warner. Randomized response: A survey technique for eliminating evasive answer bias. Journal of the American Statistical Association , 60(309):63–69, 1965

    1965

  42. [50]

    Hoeffding’s inequality, 2019

    Wikipedia. Hoeffding’s inequality, 2019. https://en.wikipedia.org/wiki/Hoeffding’s˙inequality

    2019

  43. [51]

    ”our privacy needs to be protected at all costs”: Crowd workers’ privacy experiences on amazon mechanical turk

    Huichuan Xia, Yang Wang, Yun Huang, and Anuj Shah. ”our privacy needs to be protected at all costs”: Crowd workers’ privacy experiences on amazon mechanical turk. PACMHCI, 1 (CSCW):113:1–113:22, 2017

    2017

  44. [52]

    Dpwevote: Differentially private weighted voting protocol for cloud-based decision-making

    Ziqi Yan, Jiqiang Liu, and Shaowu Liu. Dpwevote: Differentially private weighted voting protocol for cloud-based decision-making. Enterprise IS, 13(2):236–256, 2019

    2019

  45. [53]

    Tianqing Zhu, Gang Li, Wanlei Zhou, and Philip S. Yu. Differentially private data publishing and analysis: A survey. IEEE Trans. Knowl. Data Eng. , 29(8):1619–1638, 2017

    2017

  46. [54]

    Tianqing Zhu, Gang Li, Wanlei Zhou, and Philip S. Yu. Differential Privacy and Applications , volume 69 of Advances in Information Security . Springer, 2017. 27

    2017

Pith tools

Reviewed August 14, 2026 · model on record in the stance chip above.