{"id":"bc2a3737-dab3-4d82-b304-7f3218731272","arxiv_id":"2608.09140","paper_version":1,"verdict":"REJECT","confidence":"HIGH","novelty_score":4.0,"correctness_risk":"high","formal_verification":"none","parameter_count":5,"one_line_summary":"Optimizing a privacy-preserving LLM delegation pipeline with an LLM-based k-anonymity objective improves quality for most models, but the objective itself contributed little variation and privacy gains are not robustly established.","lead":"This paper adds a probabilistic k-anonymity score, estimated by a language model, to the objective of a privacy-preserving query delegation pipeline, and tests it on a small new dataset of self-disclosing user queries. It reports quality gains for most local models, but the privacy gains are inconsistent and the k-anonymity estimates are not validated against real population data.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"No control condition isolates the added k-anonymity objective; reported gains may stem from SIMBA optimizing quality alone, so the central claim that k-anonymity augmentation helps is unsupported.","rationale":"In good faith, the paper is honest about its limitations and the dataset and pipeline may be useful building blocks, but the experimental design cannot support the causal statement implied by the title and abstract. The missing control is the most load-bearing weakness because it targets the mechanism the paper claims to introduce: the probabilistic k-anonymity objective. The reader's weakest assumption concerns the uncalibrated BRANCH estimator, which is also serious. However, even if the estimator were perfectly calibrated, the paper still would not show that optimizing with k_Anon(q') improves anything over optimizing quality alone. The internal admission in Section 5.2 that the objective is 'driven largely by the quality metric' strengthens this reading. The concrete test is a standard ablation and would settle the point. Since this concern reinforces the REJECT verdict rather than changing it, I recommend UNCHANGED.","tokens_in":10932,"tokens_out":3183,"duration_ms":28663,"concrete_test":"Run SIMBA with the identical setup (same local models, GPT-4o-mini as M_REMOTE, same PUPA-SD training and PUPA-TNB evaluation) but set the optimization metric to MetricPAPILLON only (or MetricPAPILLON plus a constant k-anonymity reward) for at least Llama-3.2-3B and Qwen-2.5-1.5B. Compare the resulting quality, PII leakage, and k-anonymity scores on PUPA-TNB against Table 2. If the no-k-anonymity control matches or exceeds the reported results, the k-anonymity objective is not load-bearing for the central claim.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The paper's central claim is that adding an LLM-estimated k-anonymity objective to PAPILLON improves delegation quality and privacy-utility balance (abstract; Section 5.2). The experiments compare only 'before optimization' against 'after optimization' using the composite metric MetricPAPILLON + k_Anon(q'). There is no ablation that optimizes with MetricPAPILLON alone, nor a control with a constant or random k-anonymity term. In fact, Section 5.2 reports that k-anonymity scores 'remain largely stable across all models and conditions,' and the authors explicitly note that with the k-anonymity term contributing little variation and PII leakage excluded from the metric, 'the objective is driven largely by the quality metric.' Thus, the observed quality gains on PUPA-TNB (e.g., Llama-3.2-3B +13.2, Qwen-2.5-1.5B +17.3) are entirely consistent with SIMBA's generic prompt optimization for response quality; they do not demonstrate that the k-anonymity term plays any causal role. The privacy-utility balance claim additionally depends on PII leakage improvements that were not optimized for and could be incidental. Without a control optimization that omits k_Anon, the central contribution of this paper is not identified.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes PCD-k, a variant of Privacy-Conscious Delegation that augments the PAPILLON prompt-optimization objective with an LLM-estimated k-anonymity term. The authors introduce PUPA-SD, a small dataset of 166 user queries with self-disclosure, and implement a BRANCH-based k-anonymity estimator. They report before/after optimization results on the held-out PUPA-TNB set across six local models, claiming that quality improves on unseen conversations and that Llama-3.2-3B achieves the best privacy-utility balance. The central contribution is the proposal to use probabilistic k-anonymity as an auxiliary metric and optimization target for PCD.","tokens_in":11190,"tokens_out":4481,"duration_ms":42698,"significance":"If the central claim were established, this would be a useful step beyond PII-only redaction for privacy-conscious delegation, and the paper's emphasis on quasi-identifiers and self-disclosure is well motivated. The paper also contributes a new dataset and a cheaper BRANCH-style estimator, and it is commendably explicit about several limitations. However, the experiments as designed do not isolate the effect of the k-anonymity objective, and the k-anonymity estimator is not validated against any ground truth. Since the same uncalibrated estimator is used both as the optimization target and as the privacy outcome measure, the reported privacy-utility balance does not support the paper's claim. The contribution is therefore not established at the level required for publication.","major_comments":[{"comment":"The experiments compare only 'before optimization' with 'after optimization' under the composite objective MetricPAPILLON + k_Anon(q'). There is no ablation that optimizes with MetricPAPILLON alone, and no control condition with a constant or random k-anonymity term. Since k-anonymity scores 'remain largely stable across all models and conditions' and the paper itself states that 'the objective is driven largely by the quality metric,' the observed quality gains (e.g., Llama-3.2-3B +13.2, Qwen-2.5-1.5B +17.3) are entirely consistent with SIMBA optimizing response quality alone. The abstract's claim that adding a k-anonymity objective improves delegation quality is therefore not identified by the reported experiments.","section":"Section 5.2, Table 2"},{"comment":"The load-bearing measurement instrument is the BRANCH-based k-anonymity estimator, which relies on GPT-5-mini's pretraining knowledge, a fallback population of 400M, and a simplified cumulative-conditioning structure. The Limitations section explicitly states that 'our estimators are not calibrated against or grounded in real population statistics.' Because the same kind of estimator is used to compute the optimization objective and to report the privacy outcome, the reported k-anonymity values cannot be interpreted as evidence about real re-identification risk. The sanity check in Section 5.1 (Spearman rho = -0.4045) only confirms monotonic behavior by construction, as the paper acknowledges. Without external validation against ground-truth population statistics or a concrete re-identification attack, the privacy-utility balance claim is unsupported.","section":"Section 4.2 and Limitations"},{"comment":"The optimization metric is MetricPAPILLON + k_Anon(q'), with PII leakage deliberately excluded. The k-anonymity term is normalized as log2(k)/log2(400M), and the paper does not report sensitivity to this normalization or to the weights within MetricPAPILLON. Since k_Anon contributes little variation, the effective objective is quality-driven; any privacy improvements in Table 2 (e.g., Llama-3.2-3B leakage 29.1 to 11.0) are not attributable to the proposed mechanism, and Qwen models show increased leakage after optimization (e.g., Qwen-2.5-1.5B 54.40 to 62.95). The claim of 'best privacy-utility balance' for Llama-3.2-3B is therefore not tied to the proposed k-anonymity objective.","section":"Section 4.3, optimization metric"}],"minor_comments":[{"comment":"'An GPT-5-based LLM judge' should be 'A GPT-5-based LLM judge.'","section":"Section 3.1"},{"comment":"The acronym 'BRANCH' is never expanded; please define it or state that it is the system name from Zheng et al. (2025).","section":"Section 2.2"},{"comment":"The composite metric is described as 'normalized,' but the normalization procedure is not specified; please state how MetricPAPILLON and k_Anon(q') are combined and normalized.","section":"Section 5.2"},{"comment":"The axes of Figure 2 are not labeled; the distribution of PII counts would be clearer with explicit x- and y-axis titles.","section":"Figure 2"},{"comment":"Several references use 'and 1 others' or truncated author lists (e.g., Carlini et al. 2021, Zheng et al. 2023); please provide complete author lists or use the journal's abbreviation style consistently.","section":"References"}],"recommendation":"reject","confidential_remarks":"The authors' own Limitations section honestly acknowledges the calibration problem, but the missing control condition is not listed as a limitation, and it is the most serious issue: without optimizing with the k-anonymity term removed, the paper cannot attribute any observed gains to its proposed contribution. The claims are also 'preliminary' with N=166, which reinforces the sense that this is not yet at the standard of a full paper. A resubmission that adds a quality-only ablation, a random/constant k-anonymity control, and external calibration of the estimator would be needed before the contribution can be evaluated."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Quick take: the new thing here is PUPA-SD, a small hand-inspected set of self-disclosing queries, plus an integration of BRANCH-style k-anonymity estimation into PAPILLON's prompt-optimization objective. That integration is a natural next step, and the paper is refreshingly honest about what it does and does not show. But the headline claim—that adding a k-anonymity objective improves delegation quality and privacy-utility balance—is not supported by the reported experiments.\n\nWhat is genuinely useful: the dataset (166 queries from WildChat and LMSYS-Chat-1M, with self-disclosures extracted and manually checked), the full prompts in the appendix, and a simplified BRANCH estimator that is cheap enough to use inside optimization. The sanity check showing negative correlation between PII count and estimated k-anonymity is correctly labeled as a monotonicity check rather than validation. The authors do not oversell the estimator's calibration; they flag it as a limitation.\n\nThe soft spots are load-bearing. The experiments compare before vs. after optimization using a composite metric that includes k-anonymity, but there is no ablation that optimizes with the quality metric alone. Since k-anonymity scores remain largely flat across all models and conditions, and since PII leakage is explicitly excluded from the optimization metric, the observed quality gains are entirely consistent with SIMBA simply optimizing response quality. The authors essentially acknowledge this in Section 5.2 when they say the objective is driven largely by the quality metric. The reported leakage improvement for Llama-3.2-3B is not optimized-for and could be incidental. The estimator is uncalibrated, the fallback population and log normalization introduce free parameters, and PUPA-SD is only 166 instances, so the quantitative results are pilot-level. None of these problems are hidden, but they are not cosmetic; they directly undermine the central claim.\n\nWho gets value: researchers working on privacy-preserving delegation or LLM-based privacy risk estimation may want PUPA-SD and the simplified estimator as building blocks. The paper is a reasonable pilot, not a demonstration.\n\nMy recommendation: I would not desk-reject this—it is a serious attempt with useful artifacts—but I would send it to peer review only with the expectation of major revision. The authors should add an ablation without the k-anonymity term, report variance, and either calibrate the estimator or reframe the paper as proposing a metric rather than showing that the metric helps optimization.","headline":"This is an honest pilot study with a useful new dataset, but the experiments never isolate the k-anonymity term, so the paper's central claim is unsupported as written.","tokens_in":11727,"tokens_out":2365,"would_cite":false,"duration_ms":24309,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Adding k-anonymity to prompt optimization improves privacy-utility balance in LLM query delegation.","keywords":["privacy-conscious delegation","k-anonymity","self-disclosure","prompt optimization","LLM privacy","quasi-identifiers","probabilistic risk estimation","PUPA-SD"],"falsifier":"Take a set of disclosures with known population counts (for example, from census microdata or voter records), compute the true k-anonymity, and compare it with the estimator's output on the same text. If the estimated values are uncorrelated or systematically wrong, or if optimizing on the estimated k leaves true re-identification risk unchanged, the central argument for using the metric fails.","tokens_in":10723,"feed_emoji":"🔐","tokens_out":4067,"duration_ms":33891,"temperature":0.7,"pith_summary":"The paper argues that privacy in LLM delegation cannot be secured by removing explicit identifiers alone, because quasi-identifying traits can still single users out. It augments the Privacy-Conscious Delegation (PCD) pipeline with an LLM-driven probabilistic estimate of k-anonymity, treating a larger anonymity set as a second objective alongside response quality. To train and test this objective, the authors build PUPA-SD, a small dataset of 166 naturalistic user queries containing self-disclosures, and optimize PAPILLON-style prompts on it. Their preliminary results show that the optimized prompts improve response quality on held-out conversations for most local models, with Llama-3.2-3B achieving the strongest privacy-utility balance.","feed_headline":"Adding k-anonymity to prompt optimization improves LLM privacy-utility balance","feed_subtitle":"A probabilistic privacy metric and new dataset help local models redact quasi-identifiers, with Llama-3.2-3B gaining most.","key_machinery":"The load-bearing mechanism is an efficient, LLM-based estimator of k-anonymity adapted from BRANCH. Given a rewritten query, it extracts estimable disclosures, orders them into cumulative conditioning groups, and estimates each conditional probability from the LLM's pretraining knowledge of population statistics, with a fallback population of 400 million English speakers when information is insufficient. Multiplying the population by the estimated percentages yields the expected number of people matching the disclosed traits. This scalar is inserted into the PAPILLON optimization objective as the k-anonymity reward, so prompt optimization can trade off utility against the size of the anonymity set.","core_discovery":"The central claim is that k-anonymity, estimated probabilistically from text by an LLM, is a useful auxiliary metric for privacy-conscious delegation. On the task of rewriting a user query before sending it to a remote model, the paper shows that adding a BRANCH-based k-anonymity term to the optimization objective, measured as log2(k)/log2(400M), transfers to unseen conversations: after optimizing on PUPA-SD and evaluating on PUPA-TNB, most models improve quality, and Llama-3.2-3B reduces PII leakage from 29.1 to 11.0 while raising quality from 54.2 to 67.4. The authors also find that smaller models struggle to jointly optimize quality and privacy, and that k-anonymity scores stay largely stable, suggesting rewritten queries often fall back to the full population estimate.","pith_inferences":["If the estimator is uncalibrated, optimizing against it may lower a meaningless number; a natural extension is to ground estimates in real census or registry data via retrieval.","The near-stable k-anonymity values suggest the metric may be easy to satisfy by removing all estimable disclosures, which could push rewrites toward vagueness and hurt utility on tasks needing specific detail.","The method could be tested as a defense against actual linkability: measure whether optimized rewrites reduce the success of an external re-identification attack, not just the estimated k-anonymity.","The PUPA-SD dataset is tiny (166 instances); scaling extraction could turn the objective into a training signal for smaller models that currently fail to balance privacy and utility."],"forward_implications":["Optimizing PAPILLON prompts on PUPA-SD improves response quality on held-out PUPA-TNB conversations for most local models tested.","Llama-3.2-3B-Instruct achieves the best privacy-utility balance, with quality rising from 54.2 to 67.4 and PII leakage falling from 29.1 to 11.0.","Smaller models such as Qwen-2.5-0.5B and Qwen-2.5-1.5B struggle to optimize quality and privacy together, with leakage increasing after optimization.","Because k-anonymity scores remain largely stable across conditions, the objective's variation may come mainly from the quality term; future work should test optimizing on both k-anonymity and PII leakage."],"supporting_citations":[{"why":"Defines k-anonymity, the formal risk notion the paper adopts as an auxiliary privacy objective.","marker":"Sweeney, 2002"},{"why":"Supplies the BRANCH probabilistic reasoning framework that the paper simplifies into an efficient LLM-driven k-anonymity estimator.","marker":"Zheng et al., 2025"},{"why":"Introduces PAPILLON and PUPA, the PCD baseline, evaluation judges, and dataset structure that this work extends.","marker":"Siyan et al., 2025a"},{"why":"Provides WildChat, one of the two real-user conversation corpora from which PUPA-SD queries are extracted.","marker":"Zhao et al., 2024"},{"why":"Provides LMSYS-Chat-1M, the other real-user conversation corpus used to build PUPA-SD.","marker":"Zheng et al., 2023"},{"why":"Supplies the MIPROv2 prompt optimizer used to tune the self-disclosure extraction module.","marker":"Opsahl-Ong et al., 2024"}],"fun_headline_variants":["Probabilistic k-anonymity boosts LLM privacy-utility balance","k-anonymity metric for LLM delegation reduces PII leaks","Probabilistic privacy risk estimation improves LLM query delegation","For Llama-3.2-3B, k-anonymity yields best privacy-utility balance","New dataset and k-anonymity objective improve query privacy"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The whole approach depends on the LLM's k-anonymity estimates approximating real re-identification risk, but the paper states plainly that the estimators are not calibrated against or grounded in real population statistics.","fun_headline_variants_meta":{"raw":{"variants":["Probabilistic k-anonymity boosts LLM privacy-utility balance","k-anonymity metric for LLM delegation reduces PII leaks","Probabilistic privacy risk estimation improves LLM query delegation","For Llama-3.2-3B, k-anonymity yields best privacy-utility balance","New dataset and k-anonymity objective improve query privacy"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.001134,"raw_usage":{"total_tokens":4700,"prompt_tokens":922,"completion_tokens":3778,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":538,"completion_tokens_details":{"reasoning_tokens":3682}},"tokens_in":538,"tokens_out":3778,"duration_ms":22501,"temperature":1.0,"reasoning_tokens":3682,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-11T22:44:29.195183+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Take a set of disclosures with known population counts (for example, from census microdata or voter records), compute the true k-anonymity, and compare it with the estimator's output on the same text. If the estimated values are uncorrelated or systematically wrong, or if optimizing on the estimated k leaves true re-identification risk unchanged, the central argument for using the metric fails.","supporting_citations":[],"review_version":1}