Pith. sign in

REVIEW 4 major objections 5 minor 24 references

LDP$^3$: An Extensible and Multi-Threaded Toolkit for Local Differential Privacy Protocols and Post-Processing Methods

T0 review · 4 major / 5 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read LDP3 is an open-source toolkit claiming that selecting a protocol and post-processing method through its benchmarking pipeline substantially improves frequency-estimation utility over a bad or random choice.

desk verdict A genuinely useful LDP benchmarking toolkit, but the reported utility tables contain an unexplained Norm-column artifact that must be fixed before the experimental claims can be trusted. read the letter →

arxiv 2507.05872 v1 pith:QEBD4N6C submitted 2025-07-08 cs.CR

Berkay Kemal Balioglu , Alireza Khodaie , Mehmet Emre Gursoy This is my paper · ORCID
classification cs.CR
keywords localdifferentialprivacyLDP3post-processingmethodsfrequencyestimationtoolkitmulti-threadingutilitybenchmarkingopen-sourcesoftware
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper presents LDP3, an open-source toolkit that bundles six local differential privacy (LDP) protocols, seven post-processing methods, and several utility metrics in a modular, multi-threaded design. In LDP, each user randomizes their own data on-device before sharing it, and the server estimates population frequencies; LDP3 is built to find which protocol and post-processing correction gives the most accurate estimates. The paper's central claim is that using LDP3 to choose a combination substantially improves utility compared with a bad or random choice, and that its multi-threaded execution cuts running times by splitting the dataset across threads and averaging the per-thread estimates. A sympathetic reader would take the contribution to be a practical benchmarking resource: a standard place to compare new protocols against existing ones and to pick a configuration for a real deployment.

What carries the argument

The central object is the modular pipeline made of five modules: Protocol, Post-Processing, Utility Measurement, Multi-Threading, and Execution. The Protocol Module standardizes each LDP method as two functions, user-side perturbation and server-side estimation, covering GRR, RAPPOR, OUE, BLH, OLH, and SS. The Post-Processing Module applies correction formulas such as Base-Pos, Norm, Norm-Cut, Norm-Sub, Norm-Mul, Power, and PowerNS to the raw estimates $\hat{f}(v)$ to produce $\tilde{f}(v)$. The Multi-Threading Module splits the user population into $t$ equal chunks, runs a protocol on each chunk in parallel with its own random seed, and averages the per-chunk estimates before post-processing. This pipeline is what lets LDP3 benchmark many combinations and repeat experiments enough times to tame the randomness of LDP.

What would settle it

Take a synthetic dataset whose raw LDP frequency estimates are known not to sum to 1, run LDP3 with the Norm method, and compare the post-processed output with a direct computation of Eq. 17; if the Norm output equals the raw estimates, the implementation is not adding the constant $\sigma$ required to make the sum equal 1, and the post-processing comparisons in Tables I-III would need to be redone.

Watch

Extended reading notes

Core claim

The paper's central claim is that LDP3 lets a user experimentally identify a high-utility combination of LDP protocol and post-processing method, and that this matters because no protocol and no post-processing method dominates across datasets. Across the Kosarak, BMS-POS, and Porto datasets at $\varepsilon = 1$, the reported $\ell^1$ errors show post-processing consistently reducing error relative to no post-processing, while the best method varies by protocol and dataset; for instance, Norm-Mul is best under GRR while PowerNS is best under SS. The paper further reports that increasing from one to eight threads roughly halves or thirds execution times for many protocols, with non-linear scaling because dataset splitting and result combination remain serial. The intended consequence is that practitioners can benchmark their own data through LDP3's command-line interface and deploy the best configuration, and researchers can add new protocols and post-processing methods through its standardized interfaces.

Load-bearing premise

The benchmark's conclusion that post-processing consistently improves utility rests on the assumption that every post-processing implementation in LDP3 matches the published algorithm, and the paper's own tables make this assumption doubtful: for every protocol and dataset, the Norm column is identical to the no-post-processing column, which is inconsistent with Eq. 17 unless the raw estimates already sum to 1.

Editorial extensions

If this is right

  • A practitioner with a surrogate dataset, a privacy budget, and a utility metric can use LDP3 to find the protocol and post-processing method with the lowest error for deployment.
  • A researcher adding a new protocol can benchmark it against GRR, RAPPOR, OUE, BLH, OLH, and SS, and can search which post-processing method works best with it.
  • Because the best post-processing method changes across protocols and datasets, a fixed or random choice can be substantially worse than the best combination, which is the paper's main argument for experimenting with LDP3.
  • Multi-threading reduces wall-clock time for large experiments, with the largest gains for protocols such as OLH and BLH whose per-user cost is high.
  • The reported experiments find no universally best post-processing method, so the paper's utility claim is about selection rather than a single winning method.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • A direct extension the paper leaves untested is how the best combination changes with the privacy budget; the reported utility tables fix $\varepsilon = 1$, so the selection guidance is not shown at smaller or larger budgets.
  • The same modular split between protocol, post-processing, and utility metric could apply to downstream tasks such as heavy hitter identification or multidimensional frequency estimation, because the toolkit's interfaces are written independently of the metric.
  • A reader relying on the paper's utility conclusions should first verify that each post-processing implementation matches its published formula, since a silent mismatch in one method would change the rankings of all the others.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. The paper presents LDP3, an open-source, modular, multi-threaded Python toolkit for local differential privacy (LDP) protocols and post-processing (PP) methods. It implements six LDP protocols (GRR, RAPPOR, OUE, BLH, OLH, SS), seven PP methods (Base-Pos, Norm, Norm-Cut, Norm-Sub, Norm-Mul, Power, PowerNS), utility metrics, and a command-line interface. The experimental evaluation reports ℓ1-distance-based utility for protocol-PP combinations on three real-world datasets (Kosarak, BMS-POS, Porto) at ε=1, and execution-time speedups from multithreading. The central claims are that post-processing consistently improves utility, that LDP3 enables users to identify good protocol-PP combinations, and that multithreading substantially reduces execution times.

Significance. If the implemented formulas match the published algorithms, LDP3 is a potentially valuable benchmarking resource for the LDP community: it is open-source, modular, extensible, and combines protocols, PP methods, and utility metrics under one interface. The protocol formulas in Section III-A are standard and appear correct, and the multi-threaded execution design is a practical engineering contribution. The authors also make the code publicly available, which aids reproducibility. However, the experimental evidence contains an internal inconsistency in the reported Norm column and an overstated claim that post-processing consistently helps; these issues currently limit the strength of the empirical conclusions.

major comments (4)
  1. [Section IV-B, Tables I-III] The Norm column in Tables I-III is identical to the w/o PP column for every protocol and dataset (e.g., OLH/Kosarak both 1.56, OUE/BMS-POS both 2.17). This is inconsistent with Eq. (17), which defines Norm as adding a constant σ to every estimated frequency so that the frequencies sum to 1. For GRR the raw estimates already sum to 1, so Norm being a no-op is mathematically expected, but for OLH, OUE, BLH, RAPPOR, and SS the raw estimates do not sum to 1. For example, for OUE at ε=1 and |D|=128, the sum of the raw estimates from Eq. (8) is approximately 2(e^ε+1−d)/(e^ε−1) ≈ −144.6, so Norm would need to add roughly 1.14 to every frequency, which would substantially change the ℓ1 distance. The reported identical columns therefore indicate either that the Norm implementation does not implement Eq. (17) or that the table values were copied incorrectly. This undermines the claim in Section IV-B that the PP implementations are faithfully benchmarked, and because the Avg. w/ PP column includes Norm, the averages are also affected. The authors should correct the tables or the implementation, or explain why Norm is a no-op for all protocols.
  2. [Section IV-B, Tables I-III] The text states that 'Post-processing methods consistently help in reducing errors.' This is contradicted by the reported results for Power and PowerNS. For example, in Table I, Power yields higher ℓ1 distance than w/o PP for OLH (1.59 > 1.56), OUE (1.50 > 1.47), RAPPOR (1.71 > 1.65), and BLH (1.84 > 1.83). In Tables II and III, Power is worse than w/o PP for most protocols as well (e.g., Table II OLH 2.52 > 2.14, BLH 2.91 > 2.45, OUE 2.53 > 2.17; Table III OLH 1.27 > 1.17, BLH 1.45 > 1.34). The claim of consistency is therefore too strong; it should be restricted to the average over all methods or to specific methods that actually show improvement. This overstatement affects the interpretation of the utility benefits of post-processing as a general phenomenon.
  3. [Section III-B, Eq. (21)] The description of the Power method is not sufficiently detailed to be reproducible. Eq. (21) states a generic minimization over a distribution P, but the paper does not specify the family of distributions (despite the name suggesting a power-law family), the parameter estimation method, the initialization, or the numerical optimization procedure. The PowerNS method inherits this ambiguity. Since the experimental results for Power and PowerNS are part of the benchmark, the current description prevents readers from verifying the implementation or reproducing the results. The authors should either provide the full algorithm or cite a precise source and describe how the toolkit implements it.
  4. [Section IV-A, IV-B] All utility comparisons are averages over 10 repetitions with no standard deviations, confidence intervals, or significance tests. Many reported differences are small (e.g., Table I: Norm-Sub 1.27 vs. Norm-Mul 1.39 for OLH; Table III: Norm-Mul 0.94 vs. Base-Pos 0.93 for SS), and without error bars the claim that one PP method is 'best' for a given protocol and dataset is not statistically supported. Adding variance estimates or paired significance tests would materially strengthen the benchmarking conclusions.
minor comments (5)
  1. [Section III-B, Eq. (18)] The Norm-Cut method sets a threshold θ with the condition that the post-processed frequencies sum to 1, but the choice of θ is ambiguous when multiple thresholds produce the same sum. The paper should specify the exact selection rule (e.g., the largest θ such that all remaining frequencies are positive and sum to 1).
  2. [Section III-B] The Power method is named 'Power' but the distribution in Eq. (21) is generic; consider clarifying the name or the intended distribution family.
  3. [Section IV-C, Figures 3-5] The text says execution times decrease as threads increase from 1 to 8, but the figures show that for some protocols the decrease saturates or even reverses at 8 threads (e.g., OUE in Figure 4); this behavior should be discussed.
  4. [Section I] The abstract and introduction refer to 'LDP$^3$' with superscript formatting while the body uses 'LDP3' throughout; unify the notation for clarity.
  5. [Section IV-B, Tables I-III] The table captions note that all values are ×10^{-3}, but the running text sometimes refers to values without recalling the scaling (e.g., '1.57' vs. '> 2.0'); consistent use of the scaling would improve readability.

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity found: LDP3 implements published protocol and post-processing formulas and evaluates them empirically; its self-citations are not load-bearing and no derivation reduces to its own inputs.

full rationale

The paper is a systems/benchmarking paper, not a derivation of a result from fitted inputs. Its central claims are that LDP3 implements several published LDP protocols and PP methods and that using the toolkit to select protocol/PP combinations improves utility and multi-threading improves speed. The protocol equations (Eqs. 2-15) and PP equations (Eqs. 16-20) are stated as standard published formulas with no parameter fitted to the experimental outcomes. The utility metrics (Eqs. 22-25) are independent definitions. The experimental conclusion that post-processing helps is an empirical measurement over simulated estimates, not a quantity that is equivalent to an input by construction. The paper's self-citations ([7], [21], by co-author M.E. Gursoy) appear only in related-work comparisons and do not carry any load-bearing uniqueness or correctness argument. The reader's observation that the Norm column equals the w/o PP column in every table row, while Eq. 17 would require adding a nonzero constant for most protocols, is a potentially serious correctness or reporting issue, but it is not circularity: a no-op Norm implementation would be a bug, not a derivation that assumes its conclusion. The paper makes no claim that could be classified as self-definitional, fitted-input-as-prediction, self-citation load-bearing, imported uniqueness, ansatz-smuggling, or renaming a known result.

Assumptions & free parameters 1 free parameters · 3 assumptions · 0 invented entities

The toolkit itself introduces no new free parameters; protocol defaults (OLH g, SS k) are inherited from cited work. The only loose end is the Power post-processing method, whose fitted distribution is unspecified. The main axioms are that the cited protocol formulas are correct and that the single-categorical-value data model applies to the preprocessed datasets.

free parameters (1)
  • Power method distribution parameters = unspecified
    Eq. 21 minimizes expected square error but does not specify the distribution family or fitting algorithm; this internal fitting is part of the Power PP method (Section III-B).
assumptions (3)
  • domain assumption Cited protocol equations (Eq. 2-15) correctly implement their published LDP mechanisms.
    The paper does not re-derive these; Section III-A relies on prior literature [6]-[8].
  • domain assumption Each user's data can be represented as a single categorical value.
    The toolkit processes one value per user; Section IV-A preprocesses all three datasets to fit this model.
  • standard math Post-processing does not weaken the LDP guarantee.
    This post-processing property is standard and implicitly relied on in Section III-B.

how reviews work

0 comments
Cite this review

Pith. "Pith review of LDP$^3$: An Extensible and Multi-Threaded Toolkit for Local Differential Privacy Protocols and Post-Processing Methods." pith.science (2026). https://pith.science/paper/QEBD4N6C

@misc{pith2026250705872,
  author       = {Pith},
  title        = {Pith review of: LDP$^3$: An Extensible and Multi-Threaded Toolkit for Local Differential Privacy Protocols and Post-Processing Methods},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/QEBD4N6C}},
  note         = {Machine review of arXiv:2507.05872}
}
abstract

Local differential privacy (LDP) has become a prominent notion for privacy-preserving data collection. While numerous LDP protocols and post-processing (PP) methods have been developed, selecting an optimal combination under different privacy budgets and datasets remains a challenge. Moreover, the lack of a comprehensive and extensible LDP benchmarking toolkit raises difficulties in evaluating new protocols and PP methods. To address these concerns, this paper presents LDP$^3$ (pronounced LDP-Cube), an open-source, extensible, and multi-threaded toolkit for LDP researchers and practitioners. LDP$^3$ contains implementations of several LDP protocols, PP methods, and utility metrics in a modular and extensible design. Its modular design enables developers to conveniently integrate new protocols and PP methods. Furthermore, its multi-threaded nature enables significant reductions in execution times via parallelization. Experimental evaluations demonstrate that: (i) using LDP$^3$ to select a good protocol and post-processing method substantially improves utility compared to a bad or random choice, and (ii) the multi-threaded design of LDP$^3$ brings substantial benefits in terms of efficiency.

Figures

Figures reproduced from arXiv: 2507.05872 by the authors.

Figure 1
Figure 1. Multi-threaded execution in LDP3 square error. For example, with ˜f(v) ∼ P where P is a power law distribution, the goal is to minimize: min P E "X v∈D  ˆf(v) − ˜f(v) 2 # (21) PowerNS: PowerNS first applies Power and then uses Norm-Sub on Power’s outputs to obtain the final result. C. Utility Measurement Module LDP3 adopts several utility metrics to calculate the errors between frequencies. We exemplify four utili… view at source ↗
Figure 2
Figure 2. Command structure to run LDP3 with various options and parameters. Github repository3 , and Porto is from the ECML-PKDD Taxi Service Prediction Challenge. • Kosarak contains click-stream data of a Hungarian online news portal. Due to many URLs having very few occur￾rences (e.g., one or two), we pre-processed the dataset by identifying the top-128 most visited URLs and removed the rest, i.e., |D| = 128. For users who… view at source ↗
Figure 3
Figure 3. Execution time vs number of threads (Kosarak). [PITH_FULL_IMAGE:figures/full_fig_p008_3.png] view at source ↗
Figures from the paper (2 more)
Figure 4
Figure 4. Figure 4: Execution time vs number of threads (BMS-POS). [PITH_FULL_IMAGE:figures/full_fig_p008_4.png]
Figure 5
Figure 5. Figure 5: Execution time vs number of threads (Porto). [PITH_FULL_IMAGE:figures/full_fig_p008_5.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

24 extracted references · 23 canonical work pages

  1. [1]

    Privacy at scale: Local differential privacy in practice,

    G. Cormode, S. Jha, T. Kulkarni, N. Li, D. Srivastava, and T. Wang, “Privacy at scale: Local differential privacy in practice,” in Proceedings of the 2018 International Conference on Management of Data , 2018, pp. 1655–1658

    work page 2018

  2. [2]

    Local differential privacy and its applications: A comprehensive survey,

    M. Yang, T. Guo, T. Zhu, I. Tjuawinata, J. Zhao, and K.-Y . Lam, “Local differential privacy and its applications: A comprehensive survey,” Computer Standards & Interfaces , p. 103827, 2023

    work page 2023

  3. [3]

    Emoji frequency detection and deep link frequency,

    A. G. Thakurta, A. H. Vyrros, U. S. Vaishampayan, G. Kapoor, J. Freudinger, V . V . Prakash, A. Legendre, and S. Duplinsky, “Emoji frequency detection and deep link frequency,” Dec. 14 2017, uS Patent App. 15/640,266

    work page 2017

  4. [4]

    Collecting telemetry data privately,

    B. Ding, J. Kulkarni, and S. Yekhanin, “Collecting telemetry data privately,”Advances in Neural Information Processing Systems , vol. 30, 2017

    work page 2017

  5. [5]

    Rappor: Randomized aggre- gatable privacy-preserving ordinal response,

    ´U. Erlingsson, V . Pihur, and A. Korolova, “Rappor: Randomized aggre- gatable privacy-preserving ordinal response,” in Proceedings of the 2014 ACM SIGSAC Conference on Computer and Communications Security , 2014, pp. 1054–1067

    work page 2014

  6. [6]

    Locally differentially private pro- tocols for frequency estimation,

    T. Wang, J. Blocki, N. Li, and S. Jha, “Locally differentially private pro- tocols for frequency estimation,” in 26th USENIX Security Symposium , 2017, pp. 729–745

    work page 2017

  7. [7]

    An adversarial approach to protocol analysis and selection in local differential privacy,

    M. E. Gursoy, L. Liu, K.-H. Chow, S. Truex, and W. Wei, “An adversarial approach to protocol analysis and selection in local differential privacy,” IEEE Transactions on Information Forensics and Security , vol. 17, pp. 1785–1799, 2022

    work page 2022

  8. [8]

    Frequency estimation under local differential privacy,

    G. Cormode, S. Maddock, and C. Maple, “Frequency estimation under local differential privacy,”Proceedings of the VLDB Endowment, vol. 14, no. 11, pp. 2046–2058, 2021

    work page 2021

Show all 24 references
  1. [9]

    Calibrate: Frequency estimation and heavy hitter identification with local differential privacy via incorporating prior knowledge,

    J. Jia and N. Z. Gong, “Calibrate: Frequency estimation and heavy hitter identification with local differential privacy via incorporating prior knowledge,” in IEEE Conference on Computer Communications (INFOCOM). IEEE, 2019, pp. 2008–2016

    2019

  2. [10]

    Locally differentially private frequency estimation with consistency,

    T. Wang, M. Lopuhaa-Zwakenberg, Z. Li, B. Skoric, and N. Li, “Locally differentially private frequency estimation with consistency,” in Proceedings of the Network and Distributed System Security (NDSS) Symposium, 2020

    2020

  3. [11]

    Felip: A local differentially private approach to frequency estimation on multidimensional datasets

    J. S. da Costa Filho and J. C. Machado, “Felip: A local differentially private approach to frequency estimation on multidimensional datasets.” in EDBT, 2023, pp. 671–683

    2023

  4. [12]

    Random sampling plus fake data: Multidimensional frequency estimates with local differential privacy,

    H. H. Arcolezi, J.-F. Couchot, B. Al Bouna, and X. Xiao, “Random sampling plus fake data: Multidimensional frequency estimates with local differential privacy,” in Proceedings of the 30th ACM International Conference on Information & Knowledge Management , 2021, pp. 47– 57

    2021

  5. [13]

    Calm: Consistent adaptive local marginal for marginal release under local differential privacy,

    Z. Zhang, T. Wang, N. Li, S. He, and J. Chen, “Calm: Consistent adaptive local marginal for marginal release under local differential privacy,” in Proceedings of the 2018 ACM SIGSAC Conference on Computer and Communications Security , 2018, pp. 212–229

    2018

  6. [14]

    Heavy hitter identifica- tion over large-domain set-valued data with local differential privacy,

    Y . Zhu, Y . Cao, Q. Xue, Q. Wu, and Y . Zhang, “Heavy hitter identifica- tion over large-domain set-valued data with local differential privacy,” IEEE Transactions on Information Forensics and Security , 2023

    2023

  7. [15]

    Locally differentially private heavy hitter identification,

    T. Wang, N. Li, and S. Jha, “Locally differentially private heavy hitter identification,” IEEE Transactions on Dependable and Secure Computing, vol. 18, no. 02, pp. 982–993, 2021

    2021

  8. [16]

    Locally private set-valued data analyses: Distribution and heavy hitters estimation,

    S. Wang, Y . Li, Y . Zhong, K. Chen, X. Wang, Z. Zhou, F. Peng, Y . Qian, J. Du, and W. Yang, “Locally private set-valued data analyses: Distribution and heavy hitters estimation,” IEEE Transactions on Mobile Computing, 2023

    2023

  9. [17]

    Joint distribution analysis for set-valued data with local differential privacy,

    Y . Huang, K. Xue, B. Zhu, D. S. Wei, Q. Sun, and J. Lu, “Joint distribution analysis for set-valued data with local differential privacy,” IEEE Transactions on Information Forensics and Security , 2024

    2024

  10. [18]

    Collecting geospatial data under local differential privacy with improving frequency estimation,

    D. Hong, W. Jung, and K. Shim, “Collecting geospatial data under local differential privacy with improving frequency estimation,” IEEE Transactions on Knowledge and Data Engineering , 2022

    2022

  11. [19]

    Ldp- trace: Locally differentially private trajectory synthesis,

    Y . Du, Y . Hu, Z. Zhang, Z. Fang, L. Chen, B. Zheng, and Y . Gao, “Ldp- trace: Locally differentially private trajectory synthesis,” Proceedings of the VLDB Endowment , vol. 16, no. 8, pp. 1897–1909, 2023

    1909

  12. [20]

    Ppefl: Privacy-preserving edge federated learning with local differential privacy,

    B. Wang, Y . Chen, H. Jiang, and Z. Zhao, “Ppefl: Privacy-preserving edge federated learning with local differential privacy,” IEEE Internet of Things Journal , vol. 10, no. 17, pp. 15 488–15 500, 2023

    2023

  13. [21]

    Ldp-fed: Federated learning with local differential privacy,

    S. Truex, L. Liu, K.-H. Chow, M. E. Gursoy, and W. Wei, “Ldp-fed: Federated learning with local differential privacy,” in Proceedings of the Third ACM International Workshop on Edge Systems, Analytics and Networking, 2020, pp. 61–66

    2020

  14. [22]

    Multi-freq-LDPy: Multiple frequency estimation under local differential privacy in python,

    H. H. Arcolezi, J.-F. Couchot, S. Gambs, C. Palamidessi, and M. Zolfaghari, “Multi-freq-LDPy: Multiple frequency estimation under local differential privacy in python,” in Computer Security – ESORICS

  15. [23]

    Improving the utility of locally differentially private protocols for longitudinal and multidimensional frequency estimates,

    H. H. Arcolezi, J.-F. Couchot, B. Al Bouna, and X. Xiao, “Improving the utility of locally differentially private protocols for longitudinal and multidimensional frequency estimates,” Digital Communications and Networks, 2022

    2022

  16. [2022]

    Springer Nature Switzerland, 2022, pp. 770–775. [Online]. Available: https://doi.org/10.1007/978-3-031-17143-7 40

    2022 doi

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.