Pith. sign in

REVIEW 1 major objections 5 minor 62 references

When Absence Is Evidence: Evaluating Completeness-Sensitive Negative Reasoning in Large Language Models

T0 review · 1 major / 5 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read LLMs treat missing items as confirmed absences even when the evidence is partial

desk verdict A well-designed paired benchmark that cleanly isolates query-relative coverage and finds a consistent over-closure failure, but the load-bearing implicit-partial labels lack a human baseline. read the letter →

arxiv 2608.04591 v1 pith:O3X63NWG submitted 2026-08-05 cs.CL cs.AI

Byoungjae Min , Kennedy Edemacu , Sae-Hong Cho , Yoonhyuk Choi , Beakcheol Jang , Jong Wook Kim This is my paper
classification cs.CLcs.AI
keywords completeness-sensitivenegativereasoningcertifiedquery-relativecoverageover-closureabstentionregimesCROWN-QAretrieval-augmentedgeneration
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper argues that a 'no' answer to a question about a record, list, or retrieved context is justified only when the evidence completely covers the question's scope; otherwise the correct answer is 'unknown.' It introduces CROWN-QA, a controlled paired benchmark that fixes the question and the observed facts while varying only query-relative coverage, to test whether large language models make this distinction. Across three model families, the benchmark finds substantial over-closure: models answer 'certified negative' on evidence that is actually partial, and the most persistent failure is asymmetric, concentrated on implicitly partial source-type framings such as 'activity feed' and 'highlights digest' rather than on the matched implicitly complete framings. The paper claims this failure matters because prompting interventions redistribute the errors between over- and under-closure instead of repairing the underlying judgment, and because the same partial-coverage gap appears on real-document content.

What carries the argument

The load-bearing object is CROWN-QA itself, built from two controlled components. CROWN-Synth is a paired core of 5,000 examples (2,500 pairs) in which the same question and identical observed factual content are matched across two members that differ only in whether the evidence closes the query scope, across four coverage-expression regimes: L1 explicit ('this is the complete list'), L2 paraphrased ('the official registry of all X'), L3 implicit (source-type phrases such as 'master index' versus 'highlights digest' with no completeness language), and L4 adversarial (a saliently mentioned complete source that does not license closure), plus Scope-Mismatch cases where evidence is complete only for a narrower scope. CROWN-Real is a 533-set contrast evaluation on ACL Anthology proceedings and DailyMed drug labels, where a fixed question and target phrase are paired with variant A (query-covering complete), variant B (complete for a narrower scope), and variant C (non-exhaustive). The formal criterion is the containment condition $c = 1$ if and only if the evidence establishes complete coverage of its asserted scope and $S(q) \subseteq S(E)$, and the diagnostic instrument is a structured certificate $(\hat{S}_q, \hat{S}_E, \hat{c})$ that separates query-scope extraction, evidence-coverage characterization, and the final Boolean judgment.

What would settle it

A human-subject study on the L3 partial items would settle the central claim: present competent readers with the same question and listed titles under the four partial source-type phrases and ask whether the list could still fail to contain a relevant item. If a large share of readers treat 'highlights digest' or 'activity feed' as potentially exhaustive in context, the gold Unknown label is contested and the asymmetry would be partly a labeling artifact; conversely, if readers reliably read these phrases as partial, the model failure is confirmed.

Watch

Extended reading notes

Core claim

CROWN-QA defines completeness-sensitive negative reasoning as an absence-conditioned judgment: given that the queried fact is unsupported, the label is Certified-Negative only when the evidence establishes complete coverage $c = \mathrm{Comp}(E, S(q)) = 1$, meaning the evidence's scope $S(E)$ contains the query scope $S(q)$; otherwise the answer is Unknown. The central empirical discovery is an asymmetric closure failure on the CROWN-Synth L3 regime, where coverage is conveyed implicitly through source-type phrases: models answer the implicitly complete member ('the master index lists these titles') correctly 82.4-100.0% of the time, yet answer the matched implicitly partial member ('the highlights digest lists these titles') as Certified-Negative instead of Unknown, with partial-member correct rates of only 0.0-27.9%. The asymmetry holds across all four complete and four partial source-type families, and the same ordering -- partial evidence at least as error-prone as scope-mismatched complete evidence -- transfers to the real-document Proceedings contrast sets, where the partial-versus-narrower-complete gap is nonnegative in all 15 model-condition cells and excludes zero in 12. Prompting interventions (definition-aware rules, chain of thought, abstention, self-check, certificate elicitation) shift errors between over-closure and under-closure without a consistent repair, and structured certificates trace the earliest error most often to a mischaracterized evidence-coverage field rather than to the final judgment.

Load-bearing premise

The load-bearing premise is that the L3 implicit-partial gold labels are valid, meaning source-type phrases such as 'activity feed,' 'update bulletin,' 'highlights digest,' and 'briefing summary' convey partial, non-exhaustive coverage to competent readers, so the gold label Unknown is correct independent of the model, yet the paper validates these labels only by manual inspection plus an LLM judge, with no human baseline or agreement rate reported.

Editorial extensions

If this is right

  • Retrieval-augmented systems that answer 'no' from whatever context was retrieved would produce unjustified negatives whenever retrieval is partial; the results imply that absent support alone never licenses negation.
  • High-stakes settings -- contraindication lists, exclusion lists, official indexes -- should treat 'not found' as 'unknown' unless the evidence shown is complete for the full query scope.
  • No tested prompting style (rules, chain of thought, abstention bias, self-check, structured certificates) consistently repairs partial-evidence over-closure across models, so prompt-level gains are not a reliable fix.
  • Because the earliest certificate error is most often a mischaracterized evidence-coverage field (45-72% of incorrect outputs), the paper points to coverage inference, not the final judgment, as the bottleneck to target.
  • Since partial evidence is at least as error-prone as narrower-scope complete evidence on real documents, completeness auditing of retrieved context should treat explicit partiality markers, not just scope mismatches, as a trigger for uncertainty.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The asymmetry suggests models may be running a heuristic in which certain source phrases ('official registry,' 'master index') mean exhaustive while digest-style phrases do not; a direct follow-up would swap the source-type phrases across framings and check whether closure rates follow the phrases rather than the semantics.
  • A human baseline on the L3 partial items would settle whether the gold 'Unknown' labels match competent-reader intuitions; without one, part of the headline asymmetry could be a labeling artifact rather than a reasoning failure.
  • The certificate design hints at a training intervention the paper does not pursue: supervise the evidence-coverage field directly, since that is the field where errors first appear.
  • The real-document ordering -- partial lists more error-prone than narrower-but-complete lists -- suggests models have partially learned scope containment but not partiality inference, a distinction that could transfer beyond question answering to tool use and planning.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

1 major / 5 minor

Summary. The paper formalizes completeness-sensitive negative reasoning: a negative answer ('Certified-Negative') is justified only when the evidence completely covers the query scope; otherwise the correct answer is 'Unknown.' It introduces CROWN-Synth, a controlled paired benchmark with explicit, paraphrased, implicit, and adversarial coverage regimes plus scope-mismatch pairs, and CROWN-Real, an A/B/C contrast-set evaluation grounded in ACL Anthology proceedings and DailyMed drug labels. Three LLMs are evaluated under seven prompting conditions, with directional error rates, paired completeness sensitivity, structured certificate elicitation, and bootstrap confidence intervals. The principal finding is an asymmetric over-closure pattern: models often answer correctly on implicitly complete evidence but over-close on implicitly partial evidence, and prompting mostly redistributes the two error types rather than resolving them.

Significance. The task formulation is clean, practically motivated, and well positioned against prior abstention, sufficiency, and open-world work. The controlled paired design—same question, same observed facts, varying only query-relative coverage—is a genuine methodological advance, and the paper ships exact prompts, datasets, raw outputs, inference and scoring code, and bootstrap analyses. The main empirical claims are presented with confidence intervals and item-level decompositions, and the CROWN-Real Proceedings result is a useful transfer check. However, the headline asymmetry is only as strong as the L3 implicit-partial gold labels, which currently rest on the authors' stipulation about source-type semantics rather than on human judgments; this is the main risk to the paper's central contribution.

major comments (1)
  1. [CROWN-Synth coverage regimes; Quality Control; Tables 5 and 14] The headline asymmetry—that models recognize implicitly complete evidence but treat implicitly partial evidence as query-covering—is quantified in the L3 regime, where the gold label Unknown for the partial member is assigned by stipulating that the source-type families 'activity feed,' 'update bulletin,' 'highlights digest,' and 'briefing summary' convey partial, non-exhaustive coverage. The Quality Control section reports only manual inspection of a stratified sample and an LLM judge as a secondary check; no human baseline, agreement rate, or independent norm is provided, and an LLM judge could share the very over-closure tendency under study. Because Table 5 and Table 14 compute the complete-versus-partial asymmetry against these labels (e.g., Haiku Def.+CoT: complete 92.9% vs. partial 22.1%; Gemma partial 0.0% in several cells), a mis-calibrated label would make the asymmetry partly a construction artifact rather than a model reasoning failure. I ask for a human annotation study of the L3 items, or at minimum of the eight source-type families, reporting the proportion of competent readers who infer 'partial/non-exhaustive' for each family and reporting agreement; the inspected/rejected item counts should also be reported. If the labels are confirmed, the asymmetry stands; if not, the corresponding claims must be reframed as applying to the stipulated semantics only.
minor comments (5)
  1. [Models and Evaluation Conditions; Appendix B] The Naive condition still instructs the model to 'classify whether the observed absence licenses a negative answer using only the provided evidence,' so it is a minimally framed baseline rather than a truly default zero-shot response; consider renaming it or adding an even less directive condition to support claims about default behavior.
  2. [Diagnostic Decomposition; Table 7] The 'earliest erroneous field' assignment compares model-reported fields against the constructed gold metadata, whose L3 evidence-coverage labels are the same stipulation discussed in the major comment; the text should state explicitly that the gold evidence scope is the authors' metadata rather than an independently validated annotation, and the RQ4 percentages should be interpreted conditionally on that metadata.
  3. [Conclusion; Appendix D Table 16] The conclusion states that 'partial evidence is at least as error-prone as narrower-scope complete evidence across every model and condition' for CROWN-Real Proceedings, but the main-text Table 8 shows only four conditions per model; the full 15-cell support first appears in Appendix D, Table 16. Please point to the appendix table in the main text when making this claim.
  4. [Overall Closure Profiles; Table 3] The claim that 'explicit task rules improve Acc and CS for all three models' is visually supported by Table 3, but the statement could be sharpened by noting that the bootstrap intervals in Appendix D, Table 17 confirm the aggregate improvements while the directional OCR/UCR changes differ by model; the current wording already conveys this, so only a small clarification is needed.
  5. [Prompting Effects; Table 6] Table 6 reports item-level accuracy changes for two transitions (Definition-aware to Definition-aware+CoT and Certificate to Certificate+CoT) without bootstrap intervals for those item-group contrasts; the appendix provides intervals for aggregate metrics, but it would be useful to state that the item-group changes are descriptive and may be less stable than the aggregate contrasts.

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity: CROWN-QA is an evaluation benchmark with externally stated gold-label rules; the findings are empirical measurements, not derivations from the benchmark inputs.

full rationale

The paper's claims are behavioral measurements: models are scored against a deterministic gold-label rule c = Comp(E, S(q)) defined from controlled metadata, with the question and observed facts held fixed and only coverage varied. No parameter is fitted to a subset of data and then reported as a prediction; no result is cited from the authors' prior work as load-bearing evidence; no uniqueness theorem is invoked; and no empirical pattern is merely renamed. The L3 implicit-partial labels are stipulated by construction from source-type families rather than validated by a human annotation study; this is a labeling-validity question, not a circularity, because the gold labels are inputs to the evaluation rather than outputs derived from the models. Similarly, the certificate analysis compares model-reported scope fields to the benchmark metadata to attribute errors to the first inconsistent field; this is a diagnostic measurement against the same gold standard, not a derivation that assumes its conclusion. Prompting, CoT, and certificate conditions are all evaluated on the same frozen items, and the paper's main asymmetry finding is a measured outcome of those evaluations. Therefore there is no circular step that reduces a claimed result to its own input.

Assumptions & free parameters 0 free parameters · 3 assumptions · 1 invented entities

No numeric free parameters are fitted; the benchmark labels are controlled by a formal definitional rule. The central claims rest on domain assumptions about label validity, especially the L3 implicit-partial items, and on the adequacy of exact-match absence verification for CROWN-Real. The only invented entity is the benchmark suite itself, which lacks independent external validation in this preprint.

assumptions (3)
  • domain assumption Gold labels are fully determined by the formal rule c = Comp(E, S(q)) applied to controlled metadata, and this rule matches the intended semantics of natural-language completeness statements.
    The benchmark assigns Certified-Negative iff evidence is complete for the query scope, and Unknown otherwise. This rule is used to generate labels without human annotation (Task Definition; CROWN-Synth Quality Control).
  • ad hoc to paper L3 partial source-type families (activity feed, update bulletin, highlights digest, briefing summary) convey partial, non-exhaustive coverage to competent readers.
    The main asymmetry finding depends on these implicit-partial items being genuinely partial. Validation is only manual inspection plus an LLM judge, with no human baseline or agreement rate reported (CROWN-Synth coverage regimes; Quality Control).
  • domain assumption Normalized exact string matching suffices to confirm the target phrase is absent in CROWN-Real variants.
    Used to exclude positive-support items from A/B/C contrast sets. This is appropriate for exact-phrase queries, but assumes no displayed variant contains the phrase under the normalization used (CROWN-Real; Quality Control).
invented entities (1)
  • CROWN-QA suite (CROWN-Synth and CROWN-Real)
    purpose: To measure completeness-sensitive negative reasoning by fixing question and observed facts while varying only query-relative coverage.
    The suite is introduced in this paper; no external validation set or public release link is given in the preprint, so its validity rests on the paper's own construction and labeling.

how reviews work

0 comments
Cite this review

Pith. "Pith review of When Absence Is Evidence: Evaluating Completeness-Sensitive Negative Reasoning in Large Language Models." pith.science (2026). https://pith.science/paper/O3X63NWG

@misc{pith2026260804591,
  author       = {Pith},
  title        = {Pith review of: When Absence Is Evidence: Evaluating Completeness-Sensitive Negative Reasoning in Large Language Models},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/O3X63NWG}},
  note         = {Machine review of arXiv:2608.04591}
}
read the original abstract

Large language models (LLMs) are often asked whether something is absent from a record, list, or retrieved context. Yet non-observation licenses a negative answer only when evidence completely covers the query scope; otherwise, the answer should remain unknown. We call this completeness-sensitive negative reasoning. We introduce CROWN-QA, comprising CROWN-Synth, a controlled paired core that fixes the question and observed facts while varying only query-relative coverage, and CROWN-Real, a real-document contrast-set evaluation with controlled coverage variants. Across three LLM families, models show unstable closure judgments and substantial over-closure, failing to reliably distinguish a justified negative answer (Certified-Negative) from insufficient evidence (Unknown). The dominant CROWN-Synth failure is asymmetric: models often recognize implicitly complete evidence yet treat implicitly partial evidence as query-covering. Prompting redistributes errors between over- and under-closure rather than consistently resolving them. Structured certificate elicitation traces many errors to evidence-coverage mischaracterization. CROWN-Real shows that the core partial-coverage asymmetry persists on real-document content, while its strength and the balance between over- and under-closure vary by model, prompt, and source.

Figures

Figures reproduced from arXiv: 2608.04591 by the authors.

Figure 1
Figure 1. Matched variants with identical titles. Only the [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. Conceptual outcome space. Positive-support cases [PITH_FULL_IMAGE:figures/full_fig_p002_2.png] view at source ↗

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

62 extracted references · 37 canonical work pages

  1. [1]

    Proceedings of the International Conference on Neural Information Processing Systems , pages =

    Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks , author =. Proceedings of the International Conference on Neural Information Processing Systems , pages =

    work page

  2. [2]

    Proceedings of the International Conference on Learning Representations , year =

    Sufficient Context: A New Lens on Retrieval Augmented Generation Systems , author =. Proceedings of the International Conference on Learning Representations , year =

    work page

  3. [3]

    Proceedings of the Annual Meeting of the Association for Computational Linguistics , pages =

    Unanswerability Evaluation for Retrieval Augmented Generation , author =. Proceedings of the Annual Meeting of the Association for Computational Linguistics , pages =

    work page

  4. [4]

    arXiv preprint arXiv:2506.09038 , year =

    AbstentionBench: Reasoning LLMs Fail on Unanswerable Questions , author =. arXiv preprint arXiv:2506.09038 , year =

    arXiv

  5. [5]

    arXiv preprint arXiv:2506.11440 , year =

    AbsenceBench: Language Models Can't Tell What's Missing , author =. arXiv preprint arXiv:2506.11440 , year =

    arXiv

  6. [6]

    Proceedings of the Conference on Empirical Methods in Natural Language Processing , pages =

    This is not a Dataset: A Large Negation Benchmark to Challenge Large Language Models , author =. Proceedings of the Conference on Empirical Methods in Natural Language Processing , pages =

    work page

  7. [7]

    ACM Computing Surveys , volume =

    Completeness, Recall, and Negation in Open-world Knowledge Bases: A Survey , author =. ACM Computing Surveys , volume =

    work page

  8. [8]

    Proceedings of the International Semantic Web Conference , pages =

    Completeness Statements about RDF Data Sources and Their Use for Query Answering , author =. Proceedings of the International Semantic Web Conference , pages =

    work page

Show all 62 references
  1. [9]

    Proceedings of the ACM SIGMOD International Conference on Management of Data , pages =

    Identifying the Extent of Completeness of Query Answers over Partially Complete Databases , author =. Proceedings of the ACM SIGMOD International Conference on Management of Data , pages =

  2. [10]

    Journal of Machine Learning Research , volume =

    On the Foundations of Noise-Free Selective Classification , author =. Journal of Machine Learning Research , volume =

  3. [11]

    Proceedings of the International Conference on Neural Information Processing Systems , pages =

    Selective Classification for Deep Neural Networks , author =. Proceedings of the International Conference on Neural Information Processing Systems , pages =

  4. [12]

    arXiv preprint arXiv:2207.05221 , year =

    Language Models (Mostly) Know What They Know , author =. arXiv preprint arXiv:2207.05221 , year =

    arXiv

  5. [13]

    Findings of the Association for Computational Linguistics: ACL 2023 , pages =

    Do Large Language Models Know What They Don't Know? , author =. Findings of the Association for Computational Linguistics: ACL 2023 , pages =

    2023

  6. [14]

    Proceedings of the Annual Meeting of the Association for Computational Linguistics , pages =

    Know What You Don't Know: Unanswerable Questions for SQuAD , author =. Proceedings of the Annual Meeting of the Association for Computational Linguistics , pages =

  7. [15]

    Proceedings of the Conference on Empirical Methods in Natural Language Processing , pages =

    Selectively Answering Ambiguous Questions , author =. Proceedings of the Conference on Empirical Methods in Natural Language Processing , pages =

  8. [16]

    Proceedings of the International Conference on Learning Representations , year =

    Uncertainty Sets for Image Classifiers using Conformal Prediction , author =. Proceedings of the International Conference on Learning Representations , year =

  9. [17]

    Saad-Falcon, Jon and Khattab, Omar and Potts, Christopher and Zaharia, Matei , booktitle =

  10. [18]

    Niu, Cheng and Wu, Yuanhao and Zhu, Juno and Xu, Siliang and Shum, KaShun and Zhong, Randy and Song, Juntong and Zhang, Tong , booktitle =

  11. [19]

    Proceedings of the Conference on Empirical Methods in Natural Language Processing , pages =

    Chain-of-Note: Enhancing Robustness in Retrieval-Augmented Language Models , author =. Proceedings of the Conference on Empirical Methods in Natural Language Processing , pages =

  12. [20]

    Proceedings of the Conference on Empirical Methods in Natural Language Processing , pages =

    Do You Know What You Are Talking About? Characterizing Query-Knowledge Relevance for Reliable Retrieval Augmented Generation , author =. Proceedings of the Conference on Empirical Methods in Natural Language Processing , pages =

  13. [21]

    Glockner, Max and Jiang, Xiang and Ribeiro, Leonardo F. R. and Gurevych, Iryna and Dreyer, Markus , booktitle =

  14. [22]

    Madhusudhan, Nishanth and Madhusudhan, Sathwik Tejaswi and Yadav, Vikas and Hashemi, Masoud , booktitle =. Do

  15. [23]

    Proceedings of the 3rd Workshop on Towards Knowledgeable Foundation Models , year =

    Teaching Large Language Models to Express Knowledge Boundary from Their Own Signals , author =. Proceedings of the 3rd Workshop on Towards Knowledgeable Foundation Models , year =

  16. [24]

    Zhang, Tong and others , booktitle =

  17. [25]

    Fatahi Bayat, Farima and Zhang, Lechen and Munir, Sheza and Wang, Lu , booktitle =

  18. [26]

    Shafiei, Mohammadamin and Saffari, Hamidreza and Moosavi, Nafise Sadat , booktitle =

  19. [27]

    2026 , howpublished =

    2026

  20. [28]

    Es, Shahul and James, Jithin and Espinosa-Anke, Luis and Schockaert, Steven , booktitle=

  21. [29]

    Proceedings of the AAAI Conference on Artificial Intelligence , pages=

    Benchmarking Large Language Models in Retrieval-Augmented Generation , author=. Proceedings of the AAAI Conference on Artificial Intelligence , pages=

  22. [30]

    Yang, Xiao and Sun, Kai and Xin, Hao and Sun, Yushi and others , booktitle=

  23. [31]

    Communication, Simulation, and Intelligent Agents: Implications of Personal Intelligent Machines for Medical Education

    Clancey, William J. Communication, Simulation, and Intelligent Agents: Implications of Personal Intelligent Machines for Medical Education. Proceedings of the Eighth International Joint Conference on Artificial Intelligence (IJCAI-83)

  24. [32]

    Classification Problem Solving

    Clancey, William J. Classification Problem Solving. Proceedings of the Fourth National Conference on Artificial Intelligence

  25. [33]

    , title =

    Robinson, Arthur L. , title =. 1980 , doi =. https://science.sciencemag.org/content/208/4447/1019.full.pdf , journal =

    1980

  26. [34]

    New Ways to Make Microcircuits Smaller---Duplicate Entry

    Robinson, Arthur L. New Ways to Make Microcircuits Smaller---Duplicate Entry. Science

  27. [35]

    Clancey and Glenn Rennels , abstract =

    Diane Warner Hasling and William J. Clancey and Glenn Rennels , abstract =. Strategic explanations for a diagnostic consultation system , journal =. 1984 , issn =. doi:https://doi.org/10.1016/S0020-7373(84)80003-6 , url =

    1984 doi

  28. [36]

    and Rennels, Glenn R

    Hasling, Diane Warner and Clancey, William J. and Rennels, Glenn R. and Test, Thomas. Strategic Explanations in Consultation---Duplicate. The International Journal of Man-Machine Studies

  29. [37]

    Poligon: A System for Parallel Problem Solving

    Rice, James. Poligon: A System for Parallel Problem Solving

  30. [38]

    Transfer of Rule-Based Expertise through a Tutorial Dialogue

    Clancey, William J. Transfer of Rule-Based Expertise through a Tutorial Dialogue

  31. [39]

    The Engineering of Qualitative Models

    Clancey, William J. The Engineering of Qualitative Models

  32. [40]

    2023 , eprint=

    Attention Is All You Need , author=. 2023 , eprint=

    2023

  33. [41]

    Pluto: The 'Other' Red Planet

    NASA. Pluto: The 'Other' Red Planet

  34. [42]

    Evaluating Very Long-Term Conversational Memory of

    Maharana, Adyasha and Lee, Dong-Ho and Tulyakov, Sergey and Bansal, Mohit and Barbieri, Francesco and Fang, Yuwei , booktitle=. Evaluating Very Long-Term Conversational Memory of. 2024 , address=

    2024

  35. [43]

    arXiv preprint arXiv:2507.05257 , year=

    Evaluating Memory in LLM Agents via Incremental Multi-Turn Interactions , author=. arXiv preprint arXiv:2507.05257 , year=

    arXiv

  36. [44]

    Unveiling Privacy Risks in

    Wang, Bo and He, Weiyi and Zeng, Shenglai and Xiang, Zhen and Xing, Yue and Tang, Jiliang and He, Pengfei , booktitle=. Unveiling Privacy Risks in. 2025 , address=. doi:10.18653/v1/2025.acl-long.1227 , url=

    2025 doi

  37. [45]

    Lyu, Xingyu and He, Jianfeng and Wang, Ning and Hu, Yidan and Li, Tao and Chen, Danjue and Li, Shixiong and Chen, Yimin , journal=

  38. [46]

    arXiv preprint arXiv:2605.09530 , year=

    MemPrivacy: Privacy-Preserving Personalized Memory Management for Edge-Cloud Agents , author=. arXiv preprint arXiv:2605.09530 , year=

    arXiv

  39. [47]

    Agent-Memory Protocol: A Privacy-Focused Protocol for

    Wu, Junde and Hu, Minhao and Zhu, Jiayuan and Wang, Jiaye and Jin, Yueming , booktitle=. Agent-Memory Protocol: A Privacy-Focused Protocol for. 2026 , volume=

    2026

  40. [48]

    2024 , publisher=

    Bagdasarian, Eugene and Yi, Ren and Ghalebikesabi, Sahra and Kairouz, Peter and Gruteser, Marco and Oh, Sewoong and Balle, Borja and Ramage, Daniel , booktitle=. 2024 , publisher=

    2024

  41. [49]

    Mireshghallah, Niloofar and Kim, Hyunwoo and Zhou, Xuhui and Tsvetkov, Yulia and Sap, Maarten and Shokri, Reza and Choi, Yejin , booktitle=. Can

  42. [50]

    2025 , howpublished=

    Introducing Apps in. 2025 , howpublished=

    2025

  43. [51]

    2026 , howpublished=

    Personal Intelligence: Connecting. 2026 , howpublished=

    2026

  44. [52]

    Findings of the Association for Computational Linguistics: EMNLP 2025 , pages=

    Safeguarding Privacy of Retrieval Data against Membership Inference Attacks: Is This Query Too Close to Home? , author=. Findings of the Association for Computational Linguistics: EMNLP 2025 , pages=. 2025 , address=. doi:10.18653/v1/2025.findings-emnlp.438 , url=

    2025 doi

  45. [53]

    arXiv preprint arXiv:2412.04697 , year=

    Privacy-Preserving Retrieval Augmented Generation with Differential Privacy , author=. arXiv preprint arXiv:2412.04697 , year=

  46. [54]

    arXiv preprint arXiv:2602.14374 , year=

    Differentially Private Retrieval-Augmented Generation , author=. arXiv preprint arXiv:2602.14374 , year=

  47. [55]

    48th Annual IEEE Symposium on Foundations of Computer Science , pages=

    Mechanism Design via Differential Privacy , author=. 48th Annual IEEE Symposium on Foundations of Computer Science , pages=. 2007 , organization=

    2007

  48. [56]

    2014 , publisher=

    The Algorithmic Foundations of Differential Privacy , author=. 2014 , publisher=

    2014

  49. [57]

    Proceedings of the 37th International Conference on Machine Learning (ICML) , series=

    Optimal Differential Privacy Composition for Exponential Mechanisms , author=. Proceedings of the 37th International Conference on Machine Learning (ICML) , series=. 2020 , publisher=

    2020

  50. [58]

    Individual Privacy Accounting via a R

    Feldman, Vitaly and Zrnic, Tijana , booktitle=. Individual Privacy Accounting via a R

  51. [59]

    Mironov, Ilya , booktitle=. R. 2017 , organization=

    2017

  52. [60]

    Theory of Cryptography Conference (TCC) , pages=

    Concentrated Differential Privacy: Simplifications, Extensions, and Lower Bounds , author=. Theory of Cryptography Conference (TCC) , pages=. 2016 , publisher=

    2016

  53. [61]

    Foundations of Software Science and Computational Structures , pages=

    On the Foundations of Quantitative Information Flow , author=. Foundations of Software Science and Computational Structures , pages=. 2009 , publisher=

    2009

  54. [62]

    2012 IEEE 25th Computer Security Foundations Symposium , pages=

    Measuring Information Leakage Using Generalized Gain Functions , author=. 2012 IEEE 25th Computer Security Foundations Symposium , pages=. 2012 , organization=

    2012

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.