Ghost Vectors: Soft-Deleted Embeddings Remain Reconstructible in HNSW Vector Databases
Pith reviewed 2026-06-26 23:49 UTC · model grok-4.3
The pith
Soft-deleted embeddings in HNSW vector databases remain physically recoverable from raw index files.
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
Core claim
Analysis of three HNSW implementations shows that deleted vectors stay recoverable by bypassing API access to read raw index files. Vec2Text inversion recovers 25.5% exact names and 46.4% locations on Wikipedia data, 100% age and gender on medical data, 100% tissue classification on images, and 99% identity on faces. Epoch Key Rotation encrypts vectors and discards the key at deletion time, achieving 0% PII recovery in 2.5 ms for 500 vectors with signed proofs.
What carries the argument
Soft-delete mechanism in HNSW indexes that marks records deleted without altering stored embeddings, enabling reconstruction via embedding inversion models like Vec2Text.
If this is right
- Vector database systems must implement physical deletion or encryption to meet data erasure requirements.
- Privacy risks in RAG pipelines increase when deletions rely solely on marking records.
- Epoch key rotation offers a low-overhead method to enforce deletion with cryptographic audit trails.
- Recovery of sensitive information from deleted embeddings is feasible across text and image modalities.
Where Pith is reading between the lines
- Similar vulnerabilities could exist in other approximate nearest neighbor indexes if they use soft deletes.
- Organizations may need to review storage access controls in addition to API-level deletion policies.
- Testing the approach on additional inversion models or fine-tuned variants could reveal higher recovery rates.
Load-bearing premise
Raw index files are accessible to an attacker and the Vec2Text model successfully inverts the stored embeddings at the reported rates without domain-specific adaptation.
What would settle it
Demonstrating that deleted embeddings cannot be recovered from raw HNSW index files or that the key rotation method fails to prevent reconstruction in practice.
Figures
read the original abstract
Retrieval-augmented generation (RAG) allows large language models to access external and private corpora for factual, domain-specific responses. Modern RAG pipelines use hierarchical navigable small world (HNSW) vector databases for efficient similarity search. When a user requests data deletion, the systems typically only mark the record as deleted, leaving the embedding on disk physically unchanged. This soft-delete operation raises compliance concerns under data-erasure and retention requirements such as GDPR Article 17 and HIPAA. Analysis on three HNSW implementations confirms that deleted vectors remain physically recoverable by accessing the raw index files at the storage layer, bypassing API access. Using the Vec2Text inversion model without domain-specific fine-tuning, we show this vulnerability on multiple real-world datasets and data modalities. On Wikipedia biographical living persons dataset (BLP), we successfully recover 25.5% of exact person names and 46.4% of geographic locations (ROUGE-L 0.185). Recovery reaches 100% for both patient age and gender markers (ROUGE-L 0.290) on highly structured, sensitive data (NIH Synthea dataset). On soft-deleted image embeddings, we show 100% tissue classification on histopathology patches (p=1.02e-07) and top-1 identity recovery reaches 99% on facial embeddings (p<0.01). This work introduces Epoch Key Rotation, which encrypts vectors and discards the key upon deletion. Epoch key rotation reduces observed PII recovery to 0% and completes in 2.5 ms for 500 deleted vectors (approximately 0.005 ms/record). Additionally, it generates an ECDSA-signed cryptographic proof as an auditable record of the deletion event.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper claims that soft-deleted embeddings in HNSW vector databases remain physically present in raw index files and can be extracted to bypass API-level deletion. Using the off-the-shelf Vec2Text inversion model on three HNSW implementations, it reports concrete recovery rates on real-world datasets: 25.5% exact person names and 46.4% geographic locations (ROUGE-L 0.185) on BLP; 100% recovery of age and gender (ROUGE-L 0.290) on Synthea; and 100% tissue classification (p=1.02e-07) plus 99% top-1 identity recovery (p<0.01) on image embeddings. It introduces Epoch Key Rotation as a mitigation that encrypts vectors, discards the key on deletion, reduces PII recovery to 0%, runs in ~0.005 ms/record, and emits an ECDSA-signed proof.
Significance. If the empirical inversion results hold after full protocol disclosure, the work is significant for highlighting a storage-layer privacy gap in RAG pipelines that affects GDPR/HIPAA compliance. The multi-implementation, multi-modality evaluation and the concrete performance numbers for the proposed Epoch Key Rotation countermeasure provide actionable evidence for practitioners.
major comments (3)
- [Abstract and results sections] Abstract and results sections: the reported inversion rates (25.5% exact names on BLP, 100% on Synthea attributes, 99% identity on faces) rest on the unstated claim that vectors read from raw HNSW index files can be fed directly to Vec2Text. No extraction routine, handling of per-implementation quantization/serialization, normalization steps, exact Vec2Text checkpoint, or prompt template is described. This is load-bearing because any mismatch between stored vectors and Vec2Text's training distribution would invalidate the percentages.
- [Statistical claims] Statistical claims: p-values (p=1.02e-07, p<0.01) are given without error bars, exclusion criteria, sample sizes per condition, or the full experimental protocol. This prevents assessment of whether post-hoc choices affect the central recovery numbers on BLP, Synthea, and image modalities.
- [Mitigation section] Mitigation section: Epoch Key Rotation is presented as reducing recovery to 0% with 2.5 ms overhead for 500 vectors and an ECDSA proof, yet the manuscript supplies no details on key generation, storage-layer integration with HNSW, deletion API changes, or proof verification procedure. These omissions are load-bearing for evaluating whether the countermeasure is practical and complete.
minor comments (1)
- [Dataset descriptions] Dataset descriptions: add explicit citation or version numbers for the BLP, Synthea, and histopathology/face embedding sources to improve reproducibility.
Simulated Author's Rebuttal
We thank the referee for their constructive feedback highlighting areas where additional methodological detail would strengthen the manuscript. We address each major comment below and will incorporate clarifications and expansions in the revised version to improve reproducibility.
read point-by-point responses
-
Referee: [Abstract and results sections] Abstract and results sections: the reported inversion rates (25.5% exact names on BLP, 100% on Synthea attributes, 99% identity on faces) rest on the unstated claim that vectors read from raw HNSW index files can be fed directly to Vec2Text. No extraction routine, handling of per-implementation quantization/serialization, normalization steps, exact Vec2Text checkpoint, or prompt template is described. This is load-bearing because any mismatch between stored vectors and Vec2Text's training distribution would invalidate the percentages.
Authors: We agree that explicit documentation of the vector extraction and preprocessing pipeline is necessary for reproducibility. Raw vectors are obtained by parsing the on-disk HNSW index files (after standard deserialization and dequantization for each library), followed by L2 normalization to align with Vec2Text input expectations. The publicly released Vec2Text checkpoint was used without modification or fine-tuning. In the revision we will add a dedicated methods subsection plus an appendix containing the extraction pseudocode, per-implementation handling details, normalization steps, exact checkpoint identifier, and inversion prompt template. These additions will allow direct verification that the reported recovery rates are not artifacts of distribution mismatch. revision: yes
-
Referee: [Statistical claims] Statistical claims: p-values (p=1.02e-07, p<0.01) are given without error bars, exclusion criteria, sample sizes per condition, or the full experimental protocol. This prevents assessment of whether post-hoc choices affect the central recovery numbers on BLP, Synthea, and image modalities.
Authors: The p-values derive from two-tailed t-tests against a random-guess baseline on the observed recovery rates. Dataset sizes are 2000 records (BLP), 500 records (Synthea), and 1000 embeddings (images). In the revised manuscript we will report standard-error bars on all metrics, state that no records were excluded beyond dataset availability, and include a reproducibility appendix with exact sample sizes per condition, random seeds, and statistical test implementations. These changes will demonstrate that the significance results are robust and not driven by post-hoc decisions. revision: yes
-
Referee: [Mitigation section] Mitigation section: Epoch Key Rotation is presented as reducing recovery to 0% with 2.5 ms overhead for 500 vectors and an ECDSA proof, yet the manuscript supplies no details on key generation, storage-layer integration with HNSW, deletion API changes, or proof verification procedure. These omissions are load-bearing for evaluating whether the countermeasure is practical and complete.
Authors: We concur that implementation specifics are required to evaluate practicality. Keys are 256-bit AES keys generated per epoch from a cryptographically secure RNG; vectors are stored encrypted at the HNSW storage layer. Deletion triggers key discard, epoch increment, and generation of an ECDSA signature over the deletion metadata. The revised section will include pseudocode for key rotation and proof generation, concrete integration steps for the three evaluated HNSW libraries, the modified deletion API surface, and the public-key verification procedure. Expanded timing measurements on commodity hardware will also be provided. revision: yes
Circularity Check
No circularity; purely empirical measurements with no derivations or self-referential steps
full rationale
The paper reports direct experimental results: extraction of soft-deleted vectors from raw HNSW index files across three implementations, followed by inversion via an off-the-shelf Vec2Text model on BLP, Synthea, and image datasets, yielding reported recovery rates. No equations, first-principles derivations, fitted parameters renamed as predictions, or load-bearing self-citations appear in the provided text. The Epoch Key Rotation proposal is a mitigation technique evaluated empirically (2.5 ms for 500 vectors, 0% recovery). All claims rest on measurements rather than any reduction to inputs by construction, satisfying the default expectation of no circularity for empirical work.
Axiom & Free-Parameter Ledger
axioms (1)
- domain assumption Vec2Text inversion model recovers original data from HNSW-stored embeddings without domain-specific fine-tuning
invented entities (1)
-
Epoch Key Rotation
no independent evidence
Reference graph
Works this paper leans on
-
[1]
Regulation (eu) 2016/679 of the european parliament and of the council (general data protection regulation),
“Regulation (eu) 2016/679 of the european parliament and of the council (general data protection regulation),” 2016. Article 17: Right to erasure
2016
-
[2]
Guidelines 05/2019 on the right to erasure,
E. D. P. Board, “Guidelines 05/2019 on the right to erasure,” 2019
2019
-
[3]
Humans forget, machines remember: Artificial intelligence and the right to be forgotten,
E. M. Villaronga, P. Kieseberg, and T. Peyr, “Humans forget, machines remember: Artificial intelligence and the right to be forgotten,”Com- puter Law & Security Review, vol. 34, no. 2, pp. 304–313, 2018
2018
-
[4]
Forgetting personal data and revoking consent under the gdpr: Challenges and proposed solutions,
E. Politou, E. Alepis, and C. Patsakis, “Forgetting personal data and revoking consent under the gdpr: Challenges and proposed solutions,” Journal of Cybersecurity, vol. 4, no. 1, 2018
2018
-
[5]
J. X. Morris, W. Zhao, J. T. Chiu, V . Shmatikov, and A. M. Rush, “Transformers can invert their inputs: Joint embeddings and inverse transformers for text reconstruction,”arXiv preprint arXiv:2305.12556, 2023
-
[6]
Y . A. Malkov and D. A. Yashunin, “Efficient and robust approxi- mate nearest neighbor search using hierarchical navigable small world graphs,”IEEE Transactions on Pattern Analysis and Machine Intelli- gence, vol. 42, no. 4, pp. 824–836, 2018. arXiv:1603.09320
work page internal anchor Pith review Pith/arXiv arXiv 2018
-
[7]
Efficient indexing of billion-scale datasets of deep descriptors,
A. Babenko and V . Lempitsky, “Efficient indexing of billion-scale datasets of deep descriptors,” inIEEE Conference on Computer Vision and Pattern Recognition, pp. 2055–2063, 2016
2055
-
[8]
Chromadb,
“Chromadb,” 2023. Accessed: 2026-02-21
2023
-
[9]
Billion-scale similarity search with gpus,
J. Johnson, M. Douze, and H. J ´egou, “Billion-scale similarity search with gpus,”IEEE Transactions on Big Data, vol. 7, no. 3, pp. 535–547, 2019
2019
-
[10]
M. Douze, A. Guzhva, C. Deng, J. Johnson, G. Szilvasy, P. Mazar ´e, M. Lomeli, L. Hosseini, and H. J´egou, “The faiss library,”arXiv preprint arXiv:2401.08281, 2024
work page internal anchor Pith review Pith/arXiv arXiv 2024
-
[11]
Weaviate,
“Weaviate,” 2023. Accessed: 2026-02-21
2023
-
[12]
Pinecone vector database,
“Pinecone vector database,” 2023. Accessed: 2026-02-21
2023
-
[13]
Unsupervised Dense Information Retrieval with Contrastive Learning
J. Ni, C. Li, and J. McAuley, “Large dual encoders are generalizable retrievers,”arXiv preprint arXiv:2112.09118, 2022
work page internal anchor Pith review Pith/arXiv arXiv 2022
-
[14]
Sentence-bert: Sentence embeddings using siamese bert-networks,
N. Reimers and I. Gurevych, “Sentence-bert: Sentence embeddings using siamese bert-networks,” pp. 3982–3992, 2019
2019
-
[15]
A deep learning-driven framework for detecting anomalous data breaches in distributed cloud storage infrastructures,
S. Potluri, “A deep learning-driven framework for detecting anomalous data breaches in distributed cloud storage infrastructures,”International Journal of Artificial Intelligence, Data Science, and Machine Learning, vol. 5, no. 3, pp. 80–87, 2024
2024
-
[16]
Guidance regarding methods for de-identification of protected health information in accordance with the HIPAA privacy rule,
U.S. Department of Health & Human Services, Office for Civil Rights, “Guidance regarding methods for de-identification of protected health information in accordance with the HIPAA privacy rule,” tech. rep., U.S. Department of Health & Human Services, Nov. 2012
2012
-
[17]
Relations among notions of security for public-key encryption schemes,
M. Bellare, A. Desai, D. Pointcheval, and P. Rogaway, “Relations among notions of security for public-key encryption schemes,” inAnnual International Cryptology Conference, pp. 26–45, Springer, 1998
1998
-
[18]
Fips pub 197: Advanced encryption standard (aes),
N. I. of Standards and Technology, “Fips pub 197: Advanced encryption standard (aes),” tech. rep., U.S. Department of Commerce, 2001
2001
-
[19]
Authenticated encryption in the public-key setting: Security notions and analyses,
M. Bellare and P. Rogaway, “Authenticated encryption in the public-key setting: Security notions and analyses,” inCRYPTO, 2000
2000
-
[20]
Generating classes of test data using fake data generation and static data manipulation,
F. S. Hameed, P. Fatima, M. P. Venkataprasad, J. Subramanian, and S. Ganesan, “Generating classes of test data using fake data generation and static data manipulation,” inAIP Conference Proceedings, vol. 3279, p. 020088, AIP Publishing LLC, 2025
2025
-
[21]
Mimic-iii, a freely accessible critical care database,
A. E. Johnson, T. J. Pollard, L. Shen, L.-w. H. Lehman, M. Feng, M. Ghassemi, B. Moody, P. Szolovits, L. A. Celi, and R. G. Mark, “Mimic-iii, a freely accessible critical care database,”Nature Scientific Data, vol. 3, no. 1, pp. 1–9, 2016
2016
-
[22]
Labeled faces in the wild: A database for studying face recognition in uncon- strained environments,
G. B. Huang, M. Ramesh, T. Berg, and E. Learned-Miller, “Labeled faces in the wild: A database for studying face recognition in uncon- strained environments,” Tech. Rep. 07-49, University of Massachusetts, Amherst, Oct. 2007
2007
-
[23]
Medmnist v2: A large-scale lightweight benchmark for 2d and 3d biomedical image classification,
J. Yang, R. Shi, D. Wei, Z. Liu, L. Zhao, B. Ke, H. Pfister, and B. Ni, “Medmnist v2: A large-scale lightweight benchmark for 2d and 3d biomedical image classification,”Scientific Data, vol. 10, no. 1, p. 41, 2023
2023
-
[24]
Rouge: A package for automatic evaluation of summaries,
C.-Y . Lin, “Rouge: A package for automatic evaluation of summaries,” inText Summarization Branches Out, pp. 74–81, 2004
2004
-
[25]
Bertscore: Evaluating text generation with bert,
T. Zhang, V . Kishore, F. Wu, K. Q. Weinberger, and Y . Artzi, “Bertscore: Evaluating text generation with bert,” inInternational Conference on Learning Representations, 2020
2020
-
[26]
Zero2text: Zero-training embedding inversion via la- tent space optimization,
Anonymous, “Zero2text: Zero-training embedding inversion via la- tent space optimization,”arXiv preprint, 2025. Concurrent work on zero-training inversion; higher cross-architecture transferability than Vec2Text
2025
-
[27]
Zsinvert: Zero-shot text reconstruction from embeddings,
Anonymous, “Zsinvert: Zero-shot text reconstruction from embeddings,” arXiv preprint, 2025. Concurrent work on zero-shot inversion; validates threat model assumption
2025
-
[28]
Understanding and mitigating the threat of vec2text to dense retrieval systems,
S. Zhuang, B. Koopman, X. Chu, and G. Zuccon, “Understanding and mitigating the threat of vec2text to dense retrieval systems,” inPro- ceedings of the 47th International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR ’24), pp. 1234–1243, ACM, 2024
2024
-
[29]
Overview of imageclefmedical 2025–medical concept detection and interpretable caption generation,
H. Damm, T. M. Pakull, H. Becker, B. Bracke, B. Eryilmaz, L. Bloch, R. Br ¨ungel, C. S. Schmidt, J. R ¨uckert, O. Pelka,et al., “Overview of imageclefmedical 2025–medical concept detection and interpretable caption generation,”Proceedings of the CLEF , Madrid, Spain, pp. 9–12, 2025
2025
-
[30]
Labeled faces in the wild: A database for studying face recognition in uncon- strained environments,
G. B. Huang, M. Ramesh, T. Berg, and E. Learned-Miller, “Labeled faces in the wild: A database for studying face recognition in uncon- strained environments,” Tech. Rep. 07-49, University of Massachusetts, Amherst, 2007. Publicly available face benchmark for unconstrained recognition
2007
-
[31]
Medmnist v2 – a large-scale lightweight benchmark for 2d and 3d biomedical image classification,
J. Yang, R. Shi, D. Wei, Z. Liu, L. Zhao, B. Ke, H. Pfister, and B. Ni, “Medmnist v2 – a large-scale lightweight benchmark for 2d and 3d biomedical image classification,”Scientific Data, vol. 10, no. 1, p. 41, 2023
2023
-
[32]
GaussDB-Vector: A large-scale persistent real-time vector database for LLM applications,
J. Sun, G. Li, J. Pan, J. Wang, Y . Xie, R. Liu,et al., “GaussDB-Vector: A large-scale persistent real-time vector database for LLM applications,” inProceedings of the 2025 International Conference on Very Large Data Bases (VLDB), 2025
2025
-
[34]
Chakraborttii,Improving performance of solid state drives using machine learning
C. Chakraborttii,Improving performance of solid state drives using machine learning. PhD thesis, University of California, Santa Cruz, 2021
2021
-
[35]
Information leakage in embedding models,
C. Song and A. Raghunathan, “Information leakage in embedding models,” inProceedings of the 2020 ACM SIGSAC conference on computer and communications security, pp. 377–390, 2020
2020
-
[36]
Understanding and mitigating the threat of vec2text to dense retrieval,
J. X. Morris, W. Zhao, J. T. Chiu, and V . Shmatikov, “Understanding and mitigating the threat of vec2text to dense retrieval,”arXiv preprint arXiv:2402.12784, 2024. Vec2Text attack on live dense retrieval sys- tems; query-side embedding transformation defense
-
[37]
Hnsw index deletion evaluation: Persistent storage ar- tifacts,
e. a. Wang, “Hnsw index deletion evaluation: Persistent storage ar- tifacts,”arXiv preprint arXiv:2512.06200, 2025. Evaluates HNSW deletion behavior across implementations
-
[38]
Memtrust: A zero- trust architecture for unified ai memory system,
X. Zhou, D. Ustiugov, H. Shang, and K. Lin, “Memtrust: A zero- trust architecture for unified ai memory system,”arXiv preprint arXiv:2601.07004, 2026
-
[39]
S. Alqithami, “Forgetful but faithful: A cognitive memory architecture and benchmark for privacy-aware generative agents,”arXiv preprint arXiv:2512.12856, 2025
-
[40]
Towards making systems forget with machine unlearning,
Y . Cao and J. Yang, “Towards making systems forget with machine unlearning,” in2015 IEEE Symposium on Security and Privacy, pp. 463– 480, 2015
2015
-
[41]
Making ai forget you: Data deletion in machine learning,
A. Ginart, M. Y . Guan, G. Valiant, and J. Zou, “Making ai forget you: Data deletion in machine learning,” inAdvances in Neural Information Processing Systems 32 (NeurIPS 2019), 2019
2019
-
[42]
Beyond data erasure: The eu’s convoluted approach to model deletion,
W. Li and J. Zhao, “Beyond data erasure: The eu’s convoluted approach to model deletion,”Available at SSRN, 2025
2025
-
[43]
Membership inference attacks against machine learning models,
R. Shokri, M. Stronati, C. Song, and V . Shmatikov, “Membership inference attacks against machine learning models,” inIEEE Symposium on Security and Privacy, pp. 3–18, 2017
2017
-
[44]
Vanish: Increasing data privacy with self-destructing data,
R. Geambasu, T. Kohno, A. Levy, and H. M. Levy, “Vanish: Increasing data privacy with self-destructing data,” inUSENIX Security Symposium, 2009
2009
-
[45]
The ephemerizer: Making data disappear,
R. Perlman, “The ephemerizer: Making data disappear,”Sun Labs Technical Report, 2005
2005
-
[46]
Secure deletion of data from magnetic and solid-state memory,
P. Gutmann, “Secure deletion of data from magnetic and solid-state memory,” in6th USENIX Security Symposium, pp. 77–90, 1996
1996
-
[47]
Not what you’ve signed up for: Compromising real-world llm-integrated applications with indirect prompt injection,
K. Greshake, S. Abdelnabi, S. Mishra, C. Endres, T. Holz, and M. Fritz, “Not what you’ve signed up for: Compromising real-world llm-integrated applications with indirect prompt injection,” inAISec Workshop at ACM CCS, 2023
2023
-
[48]
Automated systems for the de-identification of longitudinal clinical narratives: Overview of 2014 i2b2/uthealth shared task track 1,
A. Stubbs, C. Kotfila, and ¨O. Uzuner, “Automated systems for the de-identification of longitudinal clinical narratives: Overview of 2014 i2b2/uthealth shared task track 1,”Journal of Biomedical Informatics, vol. 58, pp. S11–S19, 2015
2014
-
[49]
Identification of protected health information in clinical notes using neural networks,
F. Dernoncourt, J. Y . Lee, O. Uzuner, and P. Szolovits, “Identification of protected health information in clinical notes using neural networks,” Journal of the American Medical Informatics Association, vol. 24, no. e1, pp. e114–e118, 2017
2017
-
[50]
Cloaked ai: Encryption sdk for protecting ai embeddings in vector databases
I. Labs, “Cloaked ai: Encryption sdk for protecting ai embeddings in vector databases.” urlhttps://ironcorelabs.com/products/cloaked-ai/, 2023. Client-side en- cryption for embeddings before storage; protects live data at rest but does not address post-deletion cryptographic erasure or proof of erasure
2023
discussion (0)
Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.