REVIEW 3 major objections 5 minor 11 references
The paper claims that a vocabulary-scale bijection plus API-only fine-tuning hides prompt and response text from LLM APIs while preserving over 81% of plaintext-oracle performance.
Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →
T0 review · deepseek-v4-flash
2026-08-03 06:27 UTC pith:J5QTF5HL
load-bearing objection A novel and well-executed empirical study, but the losslessness guarantee is broken by subword tokenizer non-compositionality, and the provider-side known-plaintext attack is not tested, so the privacy claims as stated do not hold. the 3 major comments →
AlienLM: Alienization of Language for API-Boundary Privacy in Black-Box LLMs
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
Core claim
On its own terms, AlienLM claims to be a deployable, API-only exposure-reduction layer. The central construction is an alien language: a bijection f over non-special token IDs that maps each token's surface string to a very different-looking string while keeping its embedding neighborhood similar. The client uses f to encode prompts and decode responses, so the provider sees only alien text; the seed of f is the client-held secret. The paper shows that models fine-tuned on alienized data recover, on average, 81%+ of the unmodified model's task performance, and that three escalating observer scenarios—passive observation, bounded plaintext–alien leakage, and access to the adapted model weight
What carries the argument
The load-bearing object is the alienization bijection f and its induced translator pair (E, D). E(x)=τ⁻¹(f(τ(x))) and D reverses it, so round-trip is lossless if re-tokenization behaves. The bijection is constructed by a greedy k-nearest-neighbor pairing in embedding space: for each token, among the k closest embeddings from a public proxy model, pick the candidate with largest normalized edit distance minus a trade-off weight times embedding distance; this simultaneously maximizes human opacity and model learnability. The adaptation step, Alien Adaptation Training, is standard causal language-model fine-tuning on alienized input–output pairs through the API. The bijection therefore does dou
Load-bearing premise
The decoder is lossless only if the API tokenizer re-tokenizes alienized text into exactly the permuted token-ID sequence; subword tokenizers can merge adjacent tokens and break that identity.
What would settle it
Run the encoder on a large diverse corpus, re-tokenize each alien string with the same tokenizer, and compare the resulting token-ID sequence to the permuted original; any merge of two mapped tokens into a different vocabulary token is a counterexample that makes D(E(x)) ≠ x.
If this is right
- Users of commercial LLM APIs could move sensitive prompts and responses outside plaintext observability without switching to special-purpose cryptographic infrastructure.
- The lossless decoder means authorized clients can reconstruct exact original responses as long as tokenizer re-tokenization matches; no model internals are needed.
- The alienization ratio ρ gives operators a dial: alienizing only a subset of sensitive tokens (e.g., ρ=0.3) is reported to preserve about 93% of oracle performance, so deployments can tune opacity versus utility.
- Because the bijection is seed-derived and pairwise overlap across seeds is low, per-tenant keys and key rotation are feasible, with retraining taking only hours at reported costs.
- The reported token-recovery rates below 0.22% under weight-access and known-plaintext attacks suggest that, if replicated, token-level mapping recovery is hard even when the model itself leaks.
Where Pith is reading between the lines
- A concrete test of the losslessness assumption: apply the encoder to a large corpus, re-tokenize with the same tokenizer, and check whether the ID sequence equals the permuted original; any token-boundary merge would break D(E(x))=x and require boundary markers or byte-level encoding.
- If such merges occur only occasionally, the 81% performance numbers could still hold while losslessness fails; the paper does not report this check.
- The privacy claim is inherently comparative: a stronger adversary with a larger aligned corpus or a model trained on the plaintext domain could push recovery beyond the tested limits, so deployments should set leakage budgets accordingly.
- A natural follow-up is to replace the proxy embeddings with the target's own coarse statistics from public API outputs, removing even the open-model assumption; this is not explored.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper proposes AlienLM, a client-side, text-level privacy layer for black-box LLM APIs. It defines an 'Alien Language' by a bijective permutation over the token-ID vocabulary, a client-side encoder/decoder pair, and Alien Adaptation Training (AAT), which fine-tunes the target model through standard APIs on alienized inputs. The authors claim that, across four backbones and seven benchmarks, AlienLM retains over 81% of plaintext-oracle performance while recovery attacks infer fewer than 0.22% of alienized tokens. The paper also introduces an alienization ratio ρ for selective protection and studies key diversity and multi-tenant adaptation.
Significance. If the claims held, AlienLM would be a practical, low-overhead mechanism for reducing plaintext exposure at the API boundary, with an extensive empirical study across models and benchmarks, a publicly released codebase, and ablations isolating the effects of adaptation, tokenizer type, and bijection optimization. The paper's framing as an 'exposure-reduction layer' rather than a formal cryptographic guarantee is appropriate. However, the central losslessness proof is mathematically unsupported, and the security evaluation omits a natural attack that the threat model itself enables. These issues affect the core claims of the paper, not just presentation.
major comments (3)
- [Appendix B.3, Eqs. (2)–(3)] The proof that D(E(x)) = x rests on the claim that τ_tgt(E(x)) = f(z) 'because the transformation is defined purely in the ID space.' This is false for subword tokenizers: τ_tgt(τ_tgt^{-1}(f(z))) is not guaranteed to equal f(z), since concatenating the surface strings of mapped tokens can merge across boundaries into different vocabulary tokens (e.g., 'a' and 'bc' may re-tokenize as 'ab' + 'c'). Thus the token-ID sequence delivered to the API is not necessarily the permuted sequence f(z), and client-side decoding via f^{-1} is not lossless. The paper provides no empirical verification that retokenization preserves token boundaries, and the losslessness guarantee is load-bearing for the API-compatibility and AAT arguments.
- [Sections 3.4 and 4.1; Appendix D.7] AAT uses 300K instruction examples and 150K reasoning examples from the publicly available Magpie dataset. The fine-tuning provider, who in the threat model observes all transmitted text, can download the same public dataset and align it with the uploaded alienized training pairs, obtaining hundreds of thousands of known plaintext–alien pairs. This is a known-plaintext attack far beyond the bounded 1K-pair O2 scenario evaluated in Table 3. The paper does not consider this channel, and it may trivially compromise the mapping for tokens appearing in the training data, undermining the central privacy claim.
- [Section 4.5, O3 weight-based mapping] The reported O3 attack is designed to fail: the bijection optimization (Eq. 6) explicitly enforces embedding similarity between mapped tokens, so a nearest-neighbor matching attack finds many plausible candidates. The paper interprets this ambiguity as evidence of robustness, but it does not evaluate a stronger adversary who uses the adapted weights plus the publicly known training data, or who trains a classifier on the LM head with a small amount of aligned data. The claim that weight access yields 'fewer than 0.22%' recovery is therefore an artifact of the chosen attack, not a general robustness result.
minor comments (5)
- [Figure 1] The figure labels the fine-tuning component 'EAT'; this appears to be a typo for 'AAT'.
- [Appendix C.6] 'Paged Adaw (8-bit)' should be 'Paged AdamW (8-bit)'.
- [Section 3.3 / Appendix C] The text references 'Appendix 2' and 'Appendix 4' without a clear mapping to the appendix labels; use the actual appendix letters.
- [General] Some table headers (e.g., Table 12) are visually cluttered; separating methods and per-seed results would improve readability.
- [Section 4.2] The statement that 'AlienLM preserves the model’s familiar subword structure by relabeling token IDs directly' is in tension with the losslessness issue raised above; the tokenizer may not preserve the relabeled boundaries.
Circularity Check
Lossless round-trip D(E(x))=x is assumed via an unproven re-tokenization identity; empirical utility and attack results are otherwise independent.
specific steps
-
self definitional
[Appendix B.3, Eqs. (2)-(3) and proof sketch; also Section 3.2]
"Re-tokenizing gives τ_tgt(E(x)) = f(z) because the transformation is defined purely in the ID space under the same tokenizer and vocabulary. Applying Eq. 3 yields τ^{-1}_tgt(f^{-1}(f(z))) = τ^{-1}_tgt(z) = x."
The proof of D(E(x))=x assumes τ_tgt(τ^{-1}_tgt(f(z))) = f(z), i.e., that the server re-tokenizes the alien text to exactly the permuted ID sequence. That is precisely the losslessness property to be established. The translator is defined as an ID-space inverse pair (Eqs. 2-3), so the text-level guarantee holds only if this re-tokenization identity holds; for subword tokenizers it does not in general (token strings can merge across boundaries). The claim 'by construction' therefore reduces to the very assumption the proof needs, making the lossless round-trip a definitional assumption rather than a derived result.
full rationale
The main empirical contributions—utility retention across four backbones (Table 1) and recovery robustness (Table 3)—are genuine measurements against external baselines and do not reduce to the paper's own construction. The optimized bijection is tested against random-bijection and character-level baselines, and the proxy-embedding claim is supported by an ablation (Table 8). No load-bearing self-citation appears; the representation-alignment citations (Kornblith et al., Bansal et al.) are external and independently supported by the paper's own ablation. The O3 weight-based attack result is explained by the design objective (mapped pairs are intentionally embedding-similar), which is a design property rather than a circular derivation, though it is a limitation of the specific attack tested. The one genuinely circular step is the lossless round-trip proof: D(E(x))=x is asserted 'by construction' while the proof assumes the re-tokenization identity τ_tgt(E(x)) = f(z), which is exactly the property needed for text-level losslessness. For subword tokenizers this identity is not guaranteed, so the losslessness guarantee is not derived but assumed. This affects the client-side decoding guarantee and the premise that AAT sees exactly the permuted token-ID sequence; the empirical utility numbers could still stand, but the core losslessness claim is circular/unsupported as stated.
Axiom & Free-Parameter Ledger
free parameters (3)
- trade-off weight mu =
1 (default)
- kNN neighbor count k =
100 (default)
- alienization ratio rho =
0-1; default 1
axioms (5)
- ad hoc to paper Retokenization identity: tau(tau^{-1}(f(tau(x)))) = f(tau(x))
- domain assumption Cross-model representation alignment
- domain assumption Honest-but-curious provider
- domain assumption Trusted client environment
- domain assumption Bounded plaintext-alien leakage (O2)
invented entities (1)
-
Alien Language
independent evidence
read the original abstract
Modern LLMs are increasingly accessed via black-box APIs, requiring users to transmit sensitive prompts, outputs, and fine-tuning data to external providers, creating a critical privacy risk at the API boundary. We introduce AlienLM, a deployable API-only \cradd{exposure-reduction layer that reduces plaintext exposure} by translating text into an Alien Language via a vocabulary-scale bijection, enabling lossless recovery on the client side. Using only standard fine-tuning APIs, Alien Adaptation Training (AAT) adapts target models to operate directly on alienized inputs. Across four LLM backbones and seven benchmarks, AlienLM retains over 81\% of plaintext-oracle performance on average, substantially outperforming random-bijection and character-level baselines. Under adversaries with access to model weights, corpus statistics, and learning-based inverse translation, recovery attacks reconstruct fewer than 0.22\% of alienized tokens. Our results demonstrate a practical pathway for \cradd{privacy-aware} LLM deployment under API-only access, substantially reducing plaintext exposure while maintaining task performance. Code and data are available at https://github.com/KimJaehee0725/AlienLM.
Figures
Reference graph
Works this paper leans on
-
[3]
Gilad-Bachrach, R., Dowlin, N., Laine, K., Lauter, K., Naehrig, M., and Wernsing, J
URLhttps://arxiv.org/abs/2407.21783. Gilad-Bachrach, R., Dowlin, N., Laine, K., Lauter, K., Naehrig, M., and Wernsing, J. Cryptonets: Applying neural networks to encrypted data with high throughput and accuracy. InProceedings of the 33rd International Conference on Machine Learning (ICML), pp. 201–210,
-
[5]
URLhttps://arxiv.org/abs/2412.15115. Yao, Y ., Zhang, J., Wu, J., Huang, C., Xia, Y ., Yu, T., Zhang, R., Kim, S., Rossi, R., Li, A., Yao, L., McAuley, J., Chen, Y ., and Joe-Wong, C. Federated large language models: Current progress and future directions.arXiv preprint arXiv:2409.15723, 2024. doi: 10.48550/arXiv.2409.1572
Pith/arXiv arXiv 2024
-
[10]
URLhttps://arxiv.org/abs/2409.15723. Survey. Ye, R., Ge, R., Zhu, X., Chai, J., Du, Y ., Liu, Y ., Wang, Y ., and Chen, S. Fedllm-bench: Realistic benchmarks for federated learning of large language models.arXiv preprint arXiv:2406.04845, 2024. doi: 10.48550/arXiv.2 406.04845. URLhttps://arxiv.org/abs/2406 .04845. Benchmark. Zellers, R., Holtzman, A., Bis...
Pith/arXiv arXiv 2024
-
[100]
d ¨uzenlenen(t] 211 right left? ... 23
-
[229]
acl-long.229/
URLhttps://aclanthology.org/2022. acl-long.229/. Lin, S., Hua, W., Wang, Z., Jin, M., Fan, L., and Zhang, Y . EmojiPrompt: Generative prompt obfusca- tion for privacy-preserving communication with cloud- based LLMs. In Chiruzzo, L., Ritter, A., and Wang, L. (eds.),Proceedings of the 2025 Conference of the Nations of the Americas Chapter of the Association...
2022
-
[2016]
Google Cloud
URLhttps://proceedings.mlr.press/ v48/gilad-bachrach16.html. Google Cloud. Vertex ai and zero data retention, 2024. URL https://cloud.google.com/vertex-ai/ge nerative-ai/docs/data-governance. Ac- cessed 2026-01-29. Hendrycks, D., Burns, C., Basart, S., Zou, A., Mazeika, M., Song, D., and Steinhardt, J. Measuring massive multitask language understanding.Pr...
2024
-
[2018]
URLhttps://arxiv.org/abs/1803.054 57. Cobbe, K., Kosaraju, V ., Bavarian, M., Chen, M., Jun, H., Kaiser, L., Plappert, M., Tworek, J., Hilton, J., Nakano, R., Hesse, C., and Schulman, J. Training verifiers to solve math word problems, 2021. URLhttps://arxiv. org/abs/2110.14168. Cox, S. R., Jacobsen, R. M., and van Berkel, N. The impact of a chatbot’s ephe...
Pith/arXiv arXiv 2021
-
[2021]
ISSN 0001-0782. doi: 10.1145/3474381. URL https://doi.org/10.1145/3474381. Team, G., Riviere, M., Pathak, S., Sessa, P. G., Hardin, C., Bhupatiraju, S., Hussenot, L., Mesnard, T., Shahri- ari, B., Ram´e, A., Ferret, J., Liu, P., Tafti, P., Friesen, A., Casbon, M., Ramos, S., Kumar, R., Lan, C. L., Jerome, S., Tsitsulin, A., Vieillard, N., Stanczyk, P., Gi...
Pith/arXiv arXiv 2024
-
[2023]
Lin, S., Hilton, J., and Evans, O
URLhttps://https://huggingface.co /Open-Orca/SlimOrca. Lin, S., Hilton, J., and Evans, O. TruthfulQA: Measuring how models mimic human falsehoods. In Muresan, S., Nakov, P., and Villavicencio, A. (eds.),Proceedings of the 60th Annual Meeting of the Association for Compu- tational Linguistics (Volume 1: Long Papers), pp. 3214– 3252, Dublin, Ireland, May 20...
-
[2025]
URLhttps: //doi.org/10.56553/popets-2025-0160
doi: 10.56553/popets-2025-0160. URLhttps: //doi.org/10.56553/popets-2025-0160. Mishra, A., Li, M., and Deo, S. Sentinellms: en- crypted input adaptation and fine-tuning of language models for private and secure inference. InProceed- ings of the AAAI Conference on Artificial Intelligence, AAAI’24/IAAI’24/EAAI’24. AAAI Press, 2024. ISBN 978-1-57735-887-9. d...
-
[2026]
URLhttps://privacy.claude.com/en/ articles/7996866-how-long-do-you-sto re-my-organization-s-data. Accessed 2026- 01-29. Bansal, Y . S. et al. Revisiting model stitching to compare neural representations. InNeurIPS, 2021. Chen, Y ., Marchisio, K., Raileanu, R., Adelani, D. I., Stene- torp, P., Riedel, S., and Artetxe, M. Improving language plasticity via p...
arXiv 2026
discussion (0)
Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.