REVIEW 3 major objections 5 minor 26 references
AVEC claims that adaptive per-query budgeting and entity-level randomized response give a session-level differential privacy guarantee for delegated LLM queries.
Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →
AVEC is a proposed framework for per-query differential privacy budgeting, entity-level randomized response, and hash-based verification when delegating LLM queries to a remote model.
T0 review reviewed 2026-08-04 challenge →
load-bearing objection An honest position paper whose central end-to-end privacy guarantee does not hold as stated: the released budget makes the randomized response's epsilon data-dependent, which breaks the composition argument. the 3 major comments →
AVEC: Bootstrapping Privacy for Local LLMs
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
Core claim
The paper's central claim is Theorem 5: a full session of k delegated queries, each processed by adaptive per-query budgeting, entity-level randomized response, and randomized delegation gating, satisfies (epsilon_tot, delta_tot)-differential privacy as a whole. The total privacy loss is obtained by summing the Rényi differential privacy costs of the individual mechanisms, converting the sum to (epsilon, delta)-DP, and adding a delta term for per-entity composition. If correct, this means an external observer watching the traffic or a semi-honest remote model receiving transformed queries cannot learn much about any specific sensitive entity in the transcript. The paper pairs this with suppo
What carries the argument
The load-bearing machinery is the adaptive edge privacy budgeting function f_AEPB, which maps query sensitivity, local confidence, and session position to a proposed epsilon per query, then adds Laplace noise before releasing the budget. The privacy transformation is entity-level k-ary randomized response: each sensitive token is replaced by the true token with probability e^eps/(e^eps+k-1) and by any other token with uniform probability, giving (eps,0)-DP per entity. The delegation decision is likewise privatized with binary randomized response at eps_gate. The accounting layer uses Rényi differential privacy with privacy odometers: per-query RDP costs add linearly, and the sum is converted
Load-bearing premise
The guarantee depends on knowing how much the budget-setting calculation can change when one sensitive token changes, and on the usual rules for adding privacy costs; the paper assumes both rather than proving them.
What would settle it
Run the budget-setting calculation on pairs of queries that differ in one sensitive entity and measure the largest possible change in the proposed budget; if it exceeds 0.3, the reported privacy numbers are too optimistic. On a small vocabulary, brute-force the composition when each query's privacy parameter is chosen from the previous privatized outputs, and check whether the RDP sum still bounds the true privacy loss.
If this is right
- Every delegated query in a session carries a concrete, auditable (epsilon, delta) privacy budget against external observers and semi-honest providers, not just an ad hoc heuristic.
- Any privacy-preserving delegation scheme must randomize its decision to delegate: deterministic gating is incompatible with entity-level differential privacy.
- Hash-based proofs of transformation certify consistency of declared parameters, but can never by themselves certify that a transformation is differentially private; stronger attested execution or cryptographic proofs are required.
- There is a hard accuracy ceiling for entity recovery under this mechanism: no estimator can beat e^eps/(e^eps+k-1), so downstream tasks that need exact entities will pay a predictable utility cost.
- In simulation, adaptive budgeting delegated about two-thirds of queries instead of all of them, lowering cost and latency while all verification checks passed; this is a simulation result, not a deployment claim.
Where Pith is reading between the lines
- The paper does not derive the sensitivity of its budget function; a natural fix is to clip the function's output to a public range and precompute the sensitivity of the clipped version, which would make the privacy lemma unconditional.
- If data-dependent epsilon breaks odometer composition in practice, the same architecture could instead pre-commit to a fixed sequence of budgets per session, trading adaptivity for a provable guarantee; this is a testable fallback.
- The session budget introduces an equity dimension: users who spend their budget early receive more privatized answers, so default caps and budget rollover policies could be studied as a fairness parameter.
- The verifiability layer could be upgraded from hash consistency to lightweight zero-knowledge proofs of the randomized-response computation, closing the malicious-translation-agent gap the paper explicitly leaves open.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper presents AVEC, a framework for inference-time privacy in local LLMs. The proposed pipeline consists of an adaptive per-query budget allocator (AEPB), entity-level randomized response for sensitive tokens, randomized delegation gating, and hash-based proofs of transformation. The paper claims formal guarantees: Lemma 1 gives DP for the budget release, Theorem 5 gives a session-level end-to-end (ε,δ)-DP bound via RDP odometer composition, and Theorems 1–4 give utility ceilings, delegation leakage bounds, and impossibility results for deterministic gating and hash-only certification. The evaluation is a 30-trial simulation with 100 users per trial across 25 domains, comparing AVEC against fixed-epsilon and always-delegate/always-local baselines on cost, latency, delegation rate, and verification success. The paper explicitly identifies itself as a position paper and disclaims deployment readiness.
Significance. If the end-to-end privacy claim (Theorem 5) were rigorously established, AVEC would make a useful contribution to inference-time privacy for LLM delegation: adaptive budget allocation with explicit verifiability is a timely and practical direction. The impossibility results for deterministic delegation and hash-only certification are clear and relevant, and the simulation is reproducible by design, with honest limitations stated. The paper's central weakness is that its main theorem rests on two unproven technical premises: the sensitivity of the budget function and the validity of standard RDP composition when the released budget becomes the privacy parameter of the subsequent mechanism. The simulation does not test privacy leakage; it only measures delegation/cost behavior and verification consistency, so it cannot rescue the proof gap.
major comments (3)
- [§5.2, Lemma 1] Lemma 1 states that releasing Δε+η satisfies (εη,0)-DP with εη=Δf/b, but no bound on the L1 sensitivity Δf of f_AEPB is derived. The function includes S_qi, C_local, and F_seq, each of which can vary with the input; their combined sensitivity is not analyzed. Appendix A simply sets Δf=0.3. Since the budget release is part of the mechanism M_i in Theorem 5, this unproven sensitivity is load-bearing. The authors must either prove a concrete bound from the definitions of S_qi, C_local, and F_seq or restate the guarantee with Δf as a formal parameter rather than an assigned constant.
- [§5.6, Theorem 5] The proof sketch claims that RDP costs add linearly under adaptive choices by the odometer framework. However, the effective ε_i used for entity-level randomized response is the noisy budget released by the AEPB mechanism for the same query. Thus ε_i is a data-dependent random variable correlated with the query, and the joint distribution of (budget release, randomized response with that budget) is not the independent composition of a Laplace release and a fixed-ε mechanism. Rogers et al.'s odometer addresses adaptive mechanism selection based on previous outputs, not a mechanism whose privacy parameter is a stochastic function of the current input. No conditioning or ex-post DP argument is provided, so the claimed end-to-end guarantee is unsupported. This is the central claim of the paper and needs a complete proof or a substantial revision.
- [§5.6, Theorem 5 (δ_tot)] The formula δ_tot = k·δ_ent + δ* introduces δ_ent without definition. The entity-level randomized response is stated to satisfy (ε,0)-DP, so the source of a per-entity δ term is unclear. If this δ arises from some approximate-DP composition or from a different mechanism, it must be specified. This is a clarity issue that affects the exact statement of the end-to-end guarantee.
minor comments (5)
- [§2] The reference 'Alizadeh et al.' appears in the text without a year or a full bibliographic entry. Please complete the reference.
- [§3.1] The formula for Δεi is typeset awkwardly: '∆εi = (εbase · S qi ·(1−C local)) · F seq + η'. Use consistent notation and clarify the precedence of operations.
- [§3.5] The sentence 'Because only a fraction of queries are delegated, privacy amplification by subsampling yields tighter bounds' is asserted without specifying what is subsampled or how the amplification bound is derived. It is not used in the formal theorems, but it should either be removed or formally stated.
- [§5.3] The proof of Theorem 1 is terse: 'By symmetry of randomized response, this is the true token with probability ...' A one-line derivation of the Bayes optimal estimator and the success probability would make the argument self-contained.
- [Throughout] The paper alternates between 'A VEC' and 'AVEC'; please standardize. Also, several sentences in §7 contain grammatical errors ('The experiments indicates', 'the verifiability mechanism ndicates').
Circularity Check
Theorem 5 treats the data-dependent noisy budget as a fixed per-query ε; end-to-end DP is asserted via an inapplicable odometer composition.
specific steps
-
other
[Section 5.6, Theorem 5 proof sketch; see also Sections 3.1 and 3.5]
"Each delegated query contributes RDP cost from gating and entity privatization. Costs add linearly under adaptive choices by the privacy odometer framework Rogers et al. (2016)."
The per-query entity randomized response is parameterized by ε_i, but ε_i is not an external fixed choice: it is the capped value of Δε_i = (ε_base · S_qi · (1 − C_local)) · F_seq + η (Sec. 3.1), and Lemma 1 treats the release of Δε+η as itself a Laplace DP mechanism. Thus the RR parameter is a random output of the same query's budget-release mechanism, is correlated with the current input, and is part of the observed transcript. The RDP odometer theorem permits mechanism choice to depend on previous outputs, not on a current mechanism whose ε is a stochastic function of the current input. Summing ε_RDP_Mi(α) as fixed per-query costs assumes the very independence that the released budget violates, so the end-to-end DP guarantee is assumed rather than derived.
full rationale
The paper is mostly a synthesis of standard, externally grounded DP machinery: Laplace mechanism, entity-level randomized response, RDP conversion, post-processing, and elementary impossibility arguments. There are no load-bearing self-citations; the cited odometer and RDP works are independent external results. The simulation is explicitly self-contained, does not fit parameters to external data, and makes no deployment prediction, so the empirical section is not a fitted-input-called-prediction circularity. The one genuinely circular step is in Theorem 5: ε_i entering the entity RR is the random output of the AEPB budget-release mechanism for the same query, yet the proof composes RDP costs as if ε_i were fixed or chosen solely from previous outputs. The sentence 'costs add linearly under adaptive choices' therefore assumes the exact composition property needed for the conclusion. This makes the central end-to-end guarantee partially circular, while most components retain independent theoretical content.
Axiom & Free-Parameter Ledger
free parameters (6)
- epsilon_base =
0.05 (high privacy), 0.10 (medium privacy)
- Delta_f =
0.3
- eps_eta =
0.01
- kappa =
5
- C_local threshold =
0.8
- vocabulary size k (entity RR) =
between 8 and 64
axioms (5)
- standard math Standard DP/RDP definitions, Laplace mechanism, post-processing invariance, and privacy odometer composition hold as cited (Dwork et al., Mironov et al., Rogers et al.).
- ad hoc to paper The budget function f_AEPB has bounded L1 sensitivity Delta_f with respect to entity adjacency.
- domain assumption Entity-level randomized response over tokens, composed per entity and per query, is the correct and sufficient privacy model for LLM queries (entity adjacency).
- domain assumption The remote agent and external observers can be modeled as DP adversaries with post-processing; the translation agent is semi-trusted and emits verifiable proofs.
- ad hoc to paper Standard RDP conversion and advanced composition are applicable to the composed two-stage mechanism where the released budget is used as the epsilon of the subsequent randomized response.
Cite this review
Pith. "Pith review of AVEC: Bootstrapping Privacy for Local LLMs." pith.science (2026). https://pith.science/paper/PL65Y4GK
@misc{pith2026250910561,
author = {Pith},
title = {Pith review of: AVEC: Bootstrapping Privacy for Local LLMs},
year = {2026},
howpublished = {\url{https://pith.science/paper/PL65Y4GK}},
note = {Machine review of arXiv:2509.10561}
}
read the original abstract
This position paper presents AVEC (Adaptive Verifiable Edge Control), a framework for bootstrapping privacy for local language models by enforcing privacy at the edge with explicit verifiability for delegated queries. AVEC introduces an adaptive budgeting algorithm that allocates per-query differential privacy parameters based on sensitivity, local confidence, and historical usage, and uses verifiable transformation with on-device integrity checks. We formalize guarantees using R\'enyi differential privacy with odometer-based accounting, and establish utility ceilings, delegation-leakage bounds, and impossibility results for deterministic gating and hash-only certification. Our evaluation is simulation-based by design to study mechanism behavior and accounting; we do not claim deployment readiness or task-level utility with live LLMs. The contribution is a conceptual architecture and theoretical foundation that chart a pathway for empirical follow-up on privately bootstrapping local LLMs.
Reference graph
Works this paper leans on
-
[1]
write newline
" write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 global.max substring 't := if while FUNCTION format.date year duplicate empty "emp...
-
[2]
Deep learning with differential privacy
Martin Abadi, Andy Chu, Ian Goodfellow, H Brendan McMahan, Ilya Mironov, Kunal Talwar, and Li Zhang. Deep learning with differential privacy. In Proceedings of the 2016 ACM SIGSAC conference on computer and communications security, pp.\ 308--318, 2016
work page 2016
-
[3]
Llm in a flash: Efficient large language model inference with limited memory (2024)
Keivan Alizadeh, Iman Mirzadeh, Dmitry Belenko, Karen Khatamifard, Minsik Cho, CCD Mundo, Mohammad Rastegari, and Mehrdad Farajtabar. Llm in a flash: Efficient large language model inference with limited memory (2024). URL https://arxiv.org/abs/2312.11514
Pith/arXiv arXiv 2024
-
[4]
Fu Bang. Gptcache: An open-source semantic cache for llm applications enabling faster answers and cost savings. In Proceedings of the 3rd Workshop for Natural Language Processing Open Source Software (NLP-OSS 2023), pp.\ 212--218, 2023
work page 2023
-
[5]
Bulletproofs: Short proofs for confidential transactions and more
Benedikt B \"u nz, Jonathan Bootle, Dan Boneh, Andrew Poelstra, Pieter Wuille, and Greg Maxwell. Bulletproofs: Short proofs for confidential transactions and more. In 2018 IEEE symposium on security and privacy (SP), pp.\ 315--334. IEEE, 2018
work page 2018
-
[6]
The algorithmic foundations of differential privacy
Cynthia Dwork, Aaron Roth, et al. The algorithmic foundations of differential privacy. Foundations and trends in theoretical computer science , 9 0 (3--4): 0 211--407, 2014
2014
-
[7]
Non-interactive verifiable computing: Outsourcing computation to untrusted workers
Rosario Gennaro, Craig Gentry, and Bryan Parno. Non-interactive verifiable computing: Outsourcing computation to untrusted workers. In Annual Cryptology Conference, pp.\ 465--482. Springer, 2010
work page 2010
-
[8]
Explaining and harnessing adversarial examples
Ian J Goodfellow, Jonathon Shlens, and Christian Szegedy. Explaining and harnessing adversarial examples. arXiv preprint arXiv:1412.6572, 2014
Pith/arXiv arXiv 2014
-
[9]
N-sanitization: A semantic privacy-preserving framework for unstructured medical datasets
Celestine Iwendi, Syed Atif Moqurrab, Adeel Anjum, Sangeen Khan, Senthilkumar Mohan, and Gautam Srivastava. N-sanitization: A semantic privacy-preserving framework for unstructured medical datasets. Computer Communications, 161: 0 160--171, 2020
work page 2020
-
[10]
Bin Jiang, Jianqiang Li, Huihui Wang, and Houbing Song. Privacy-preserving federated learning for industrial edge computing via hybrid differential privacy and adaptive compression. IEEE Transactions on Industrial Informatics, 19 0 (2): 0 1136--1144, 2021
work page 2021
-
[11]
Scalm: Towards semantic caching for automated chat services with large language models
Jiaxing Li, Chi Xu, Feng Wang, Isaac M von Riedemann, Cong Zhang, and Jiangchuan Liu. Scalm: Towards semantic caching for automated chat services with large language models. In 2024 IEEE/ACM 32nd International Symposium on Quality of Service (IWQoS), pp.\ 1--10. IEEE, 2024
work page 2024
-
[12]
t-closeness: Privacy beyond k-anonymity and l-diversity
Ninghui Li, Tiancheng Li, and Suresh Venkatasubramanian. t-closeness: Privacy beyond k-anonymity and l-diversity. In 2007 IEEE 23rd international conference on data engineering, pp.\ 106--115. IEEE, 2006
work page 2007
-
[13]
l-diversity: Privacy beyond k-anonymity
Ashwin Machanavajjhala, Daniel Kifer, Johannes Gehrke, and Muthuramakrishnan Venkitasubramaniam. l-diversity: Privacy beyond k-anonymity. Acm transactions on knowledge discovery from data (tkdd), 1 0 (1): 0 3--es, 2007
work page 2007
-
[14]
Anonymization techniques for privacy preserving data publishing: A comprehensive survey
Abdul Majeed and Sungchang Lee. Anonymization techniques for privacy preserving data publishing: A comprehensive survey. IEEE access, 9: 0 8512--8545, 2020
work page 2020
-
[15]
Communication-efficient learning of deep networks from decentralized data
Brendan McMahan, Eider Moore, Daniel Ramage, Seth Hampson, and Blaise Aguera y Arcas. Communication-efficient learning of deep networks from decentralized data. In Artificial intelligence and statistics, pp.\ 1273--1282. PMLR, 2017
2017
-
[16]
R 'enyi differential privacy of the sampled gaussian mechanism
Ilya Mironov, Kunal Talwar, and Li Zhang. R 'enyi differential privacy of the sampled gaussian mechanism. arXiv preprint arXiv:1908.10530, 2019
Pith/arXiv arXiv 1908
-
[17]
Pinocchio: Nearly practical verifiable computation
Bryan Parno, Jon Howell, Craig Gentry, and Mariana Raykova. Pinocchio: Nearly practical verifiable computation. Communications of the ACM, 59 0 (2): 0 103--112, 2016
work page 2016
-
[18]
A survey of zero-knowledge proof based verifiable machine learning
Zhizhi Peng, Taotao Wang, Chonghe Zhao, Guofu Liao, Zibin Lin, Yifeng Liu, Bin Cao, Long Shi, Qing Yang, and Shengli Zhang. A survey of zero-knowledge proof based verifiable machine learning. arXiv preprint arXiv:2502.18535, 2025
Pith/arXiv arXiv 2025
-
[19]
Privacy odometers and filters: Pay-as-you-go composition
Ryan M Rogers, Aaron Roth, Jonathan Ullman, and Salil Vadhan. Privacy odometers and filters: Pay-as-you-go composition. Advances in Neural Information Processing Systems, 29, 2016
work page 2016
-
[20]
k-anonymity: A model for protecting privacy
Latanya Sweeney. k-anonymity: A model for protecting privacy. International journal of uncertainty, fuzziness and knowledge-based systems, 10 0 (05): 0 557--570, 2002
work page 2002
-
[21]
Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, ukasz Kaiser, and Illia Polosukhin. Attention is all you need. In Advances in Neural Information Processing Systems, volume 30, 2017
work page 2017
-
[22]
Voyager: An open-ended embodied agent with large language models
Guanzhi Wang, Yuqi Xie, Yunfan Jiang, Ajay Mandlekar, Chaowei Xiao, Yuke Zhu, Linxi Fan, and Anima Anandkumar. Voyager: An open-ended embodied agent with large language models. arXiv preprint arXiv:2305.16291, 2023
Pith/arXiv arXiv 2023
-
[23]
A survey on large language model based autonomous agents
Lei Wang, Chen Ma, Xueyang Feng, Zeyu Zhang, Hao Yang, Jingsen Zhang, Zhiyuan Chen, Jiakai Tang, Xu Chen, Yankai Lin, et al. A survey on large language model based autonomous agents. Frontiers of Computer Science, 18 0 (6): 0 186345, 2024
2024
-
[24]
On-device language models: A comprehensive review
Jiajun Xu, Zhiyuan Li, Wei Chen, Qun Wang, Xin Gao, Qi Cai, and Ziyuan Ling. On-device language models: A comprehensive review. arXiv preprint arXiv:2409.00088, 2024
Pith/arXiv arXiv 2024
-
[25]
React: Synergizing reasoning and acting in language models
Shunyu Yao, Jeffrey Zhao, Dian Yu, Nan Du, Izhak Shafran, Karthik Narasimhan, and Yuan Cao. React: Synergizing reasoning and acting in language models. In International Conference on Learning Representations (ICLR), 2023
2023
-
[26]
A survey on large language model based human-agent systems
Henry Peng Zou, Wei-Chieh Huang, Yaozu Wu, Yankai Chen, Chunyu Miao, Hoang Nguyen, Yue Zhou, Weizhi Zhang, Liancheng Fang, Langzhou He, et al. A survey on large language model based human-agent systems. arXiv preprint arXiv:2505.00753, 2025
Pith/arXiv arXiv 2025
This paper was first reviewed by deepseek-v4-flash on August 4, 2026.
discussion (0)
Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.