Pith. sign in

REVIEW 3 major objections 5 minor 11 references

The paper claims that a vocabulary-scale bijection plus API-only fine-tuning hides prompt and response text from LLM APIs while preserving over 81% of plaintext-oracle performance.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

T0 review · deepseek-v4-flash

2026-08-03 06:27 UTC pith:J5QTF5HL

load-bearing objection A novel and well-executed empirical study, but the losslessness guarantee is broken by subword tokenizer non-compositionality, and the provider-side known-plaintext attack is not tested, so the privacy claims as stated do not hold. the 3 major comments →

arxiv 2601.22710 v2 pith:J5QTF5HL submitted 2026-01-30 cs.CR cs.CL

AlienLM: Alienization of Language for API-Boundary Privacy in Black-Box LLMs

classification cs.CR cs.CL
keywords AlienLMAPI-boundary privacytoken-level bijectionalien languageblack-box LLMfine-tuningobfuscationexposure reduction
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

AlienLM tackles the privacy problem of using commercial LLM APIs: prompts, outputs, and fine-tuning data cross the boundary as readable text. The paper's move is to translate that text into a secret 'alien language' defined by a bijective permutation over the model's token vocabulary, so the API sees only unreadable strings. The authors argue that with fine-tuning through ordinary APIs (Alien Adaptation Training) the model can learn to work with alienized text, and their experiments report over 81% of the original plaintext-oracle average performance across four backbones and seven benchmarks, with recovery attempts under strong observer models reconstructing fewer than 0.22% of tokens. Whether the scheme is fully lossless in practice rests on an identity about how subword tokenizers re-tokenize alien strings.

Core claim

On its own terms, AlienLM claims to be a deployable, API-only exposure-reduction layer. The central construction is an alien language: a bijection f over non-special token IDs that maps each token's surface string to a very different-looking string while keeping its embedding neighborhood similar. The client uses f to encode prompts and decode responses, so the provider sees only alien text; the seed of f is the client-held secret. The paper shows that models fine-tuned on alienized data recover, on average, 81%+ of the unmodified model's task performance, and that three escalating observer scenarios—passive observation, bounded plaintext–alien leakage, and access to the adapted model weight

What carries the argument

The load-bearing object is the alienization bijection f and its induced translator pair (E, D). E(x)=τ⁻¹(f(τ(x))) and D reverses it, so round-trip is lossless if re-tokenization behaves. The bijection is constructed by a greedy k-nearest-neighbor pairing in embedding space: for each token, among the k closest embeddings from a public proxy model, pick the candidate with largest normalized edit distance minus a trade-off weight times embedding distance; this simultaneously maximizes human opacity and model learnability. The adaptation step, Alien Adaptation Training, is standard causal language-model fine-tuning on alienized input–output pairs through the API. The bijection therefore does dou

Load-bearing premise

The decoder is lossless only if the API tokenizer re-tokenizes alienized text into exactly the permuted token-ID sequence; subword tokenizers can merge adjacent tokens and break that identity.

What would settle it

Run the encoder on a large diverse corpus, re-tokenize each alien string with the same tokenizer, and compare the resulting token-ID sequence to the permuted original; any merge of two mapped tokens into a different vocabulary token is a counterexample that makes D(E(x)) ≠ x.

Watch this falsifier — get emailed when new claim-graph text bears on it.

If this is right

  • Users of commercial LLM APIs could move sensitive prompts and responses outside plaintext observability without switching to special-purpose cryptographic infrastructure.
  • The lossless decoder means authorized clients can reconstruct exact original responses as long as tokenizer re-tokenization matches; no model internals are needed.
  • The alienization ratio ρ gives operators a dial: alienizing only a subset of sensitive tokens (e.g., ρ=0.3) is reported to preserve about 93% of oracle performance, so deployments can tune opacity versus utility.
  • Because the bijection is seed-derived and pairwise overlap across seeds is low, per-tenant keys and key rotation are feasible, with retraining taking only hours at reported costs.
  • The reported token-recovery rates below 0.22% under weight-access and known-plaintext attacks suggest that, if replicated, token-level mapping recovery is hard even when the model itself leaks.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • A concrete test of the losslessness assumption: apply the encoder to a large corpus, re-tokenize with the same tokenizer, and check whether the ID sequence equals the permuted original; any token-boundary merge would break D(E(x))=x and require boundary markers or byte-level encoding.
  • If such merges occur only occasionally, the 81% performance numbers could still hold while losslessness fails; the paper does not report this check.
  • The privacy claim is inherently comparative: a stronger adversary with a larger aligned corpus or a model trained on the plaintext domain could push recovery beyond the tested limits, so deployments should set leakage budgets accordingly.
  • A natural follow-up is to replace the proxy embeddings with the target's own coarse statistics from public API outputs, removing even the open-model assumption; this is not explored.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. The paper proposes AlienLM, a client-side, text-level privacy layer for black-box LLM APIs. It defines an 'Alien Language' by a bijective permutation over the token-ID vocabulary, a client-side encoder/decoder pair, and Alien Adaptation Training (AAT), which fine-tunes the target model through standard APIs on alienized inputs. The authors claim that, across four backbones and seven benchmarks, AlienLM retains over 81% of plaintext-oracle performance while recovery attacks infer fewer than 0.22% of alienized tokens. The paper also introduces an alienization ratio ρ for selective protection and studies key diversity and multi-tenant adaptation.

Significance. If the claims held, AlienLM would be a practical, low-overhead mechanism for reducing plaintext exposure at the API boundary, with an extensive empirical study across models and benchmarks, a publicly released codebase, and ablations isolating the effects of adaptation, tokenizer type, and bijection optimization. The paper's framing as an 'exposure-reduction layer' rather than a formal cryptographic guarantee is appropriate. However, the central losslessness proof is mathematically unsupported, and the security evaluation omits a natural attack that the threat model itself enables. These issues affect the core claims of the paper, not just presentation.

major comments (3)
  1. [Appendix B.3, Eqs. (2)–(3)] The proof that D(E(x)) = x rests on the claim that τ_tgt(E(x)) = f(z) 'because the transformation is defined purely in the ID space.' This is false for subword tokenizers: τ_tgt(τ_tgt^{-1}(f(z))) is not guaranteed to equal f(z), since concatenating the surface strings of mapped tokens can merge across boundaries into different vocabulary tokens (e.g., 'a' and 'bc' may re-tokenize as 'ab' + 'c'). Thus the token-ID sequence delivered to the API is not necessarily the permuted sequence f(z), and client-side decoding via f^{-1} is not lossless. The paper provides no empirical verification that retokenization preserves token boundaries, and the losslessness guarantee is load-bearing for the API-compatibility and AAT arguments.
  2. [Sections 3.4 and 4.1; Appendix D.7] AAT uses 300K instruction examples and 150K reasoning examples from the publicly available Magpie dataset. The fine-tuning provider, who in the threat model observes all transmitted text, can download the same public dataset and align it with the uploaded alienized training pairs, obtaining hundreds of thousands of known plaintext–alien pairs. This is a known-plaintext attack far beyond the bounded 1K-pair O2 scenario evaluated in Table 3. The paper does not consider this channel, and it may trivially compromise the mapping for tokens appearing in the training data, undermining the central privacy claim.
  3. [Section 4.5, O3 weight-based mapping] The reported O3 attack is designed to fail: the bijection optimization (Eq. 6) explicitly enforces embedding similarity between mapped tokens, so a nearest-neighbor matching attack finds many plausible candidates. The paper interprets this ambiguity as evidence of robustness, but it does not evaluate a stronger adversary who uses the adapted weights plus the publicly known training data, or who trains a classifier on the LM head with a small amount of aligned data. The claim that weight access yields 'fewer than 0.22%' recovery is therefore an artifact of the chosen attack, not a general robustness result.
minor comments (5)
  1. [Figure 1] The figure labels the fine-tuning component 'EAT'; this appears to be a typo for 'AAT'.
  2. [Appendix C.6] 'Paged Adaw (8-bit)' should be 'Paged AdamW (8-bit)'.
  3. [Section 3.3 / Appendix C] The text references 'Appendix 2' and 'Appendix 4' without a clear mapping to the appendix labels; use the actual appendix letters.
  4. [General] Some table headers (e.g., Table 12) are visually cluttered; separating methods and per-seed results would improve readability.
  5. [Section 4.2] The statement that 'AlienLM preserves the model’s familiar subword structure by relabeling token IDs directly' is in tension with the losslessness issue raised above; the tokenizer may not preserve the relabeled boundaries.

Circularity Check

1 steps flagged

Lossless round-trip D(E(x))=x is assumed via an unproven re-tokenization identity; empirical utility and attack results are otherwise independent.

specific steps
  1. self definitional [Appendix B.3, Eqs. (2)-(3) and proof sketch; also Section 3.2]
    "Re-tokenizing gives τ_tgt(E(x)) = f(z) because the transformation is defined purely in the ID space under the same tokenizer and vocabulary. Applying Eq. 3 yields τ^{-1}_tgt(f^{-1}(f(z))) = τ^{-1}_tgt(z) = x."

    The proof of D(E(x))=x assumes τ_tgt(τ^{-1}_tgt(f(z))) = f(z), i.e., that the server re-tokenizes the alien text to exactly the permuted ID sequence. That is precisely the losslessness property to be established. The translator is defined as an ID-space inverse pair (Eqs. 2-3), so the text-level guarantee holds only if this re-tokenization identity holds; for subword tokenizers it does not in general (token strings can merge across boundaries). The claim 'by construction' therefore reduces to the very assumption the proof needs, making the lossless round-trip a definitional assumption rather than a derived result.

full rationale

The main empirical contributions—utility retention across four backbones (Table 1) and recovery robustness (Table 3)—are genuine measurements against external baselines and do not reduce to the paper's own construction. The optimized bijection is tested against random-bijection and character-level baselines, and the proxy-embedding claim is supported by an ablation (Table 8). No load-bearing self-citation appears; the representation-alignment citations (Kornblith et al., Bansal et al.) are external and independently supported by the paper's own ablation. The O3 weight-based attack result is explained by the design objective (mapped pairs are intentionally embedding-similar), which is a design property rather than a circular derivation, though it is a limitation of the specific attack tested. The one genuinely circular step is the lossless round-trip proof: D(E(x))=x is asserted 'by construction' while the proof assumes the re-tokenization identity τ_tgt(E(x)) = f(z), which is exactly the property needed for text-level losslessness. For subword tokenizers this identity is not guaranteed, so the losslessness guarantee is not derived but assumed. This affects the client-side decoding guarantee and the premise that AAT sees exactly the permuted token-ID sequence; the empirical utility numbers could still stand, but the core losslessness claim is circular/unsupported as stated.

Axiom & Free-Parameter Ledger

3 free parameters · 5 axioms · 1 invented entities

The central claim rests on a small number of hand-chosen constants (mu, k, rho) and several strong assumptions: that the tokenizer re-tokenizes alien text to the intended ID sequence, that proxy embeddings transfer across models, that the provider is honest-but-curious, and that leakage stays bounded. The retokenization axiom is the most fragile because it is asserted rather than derived and is false for common subword tokenizers.

free parameters (3)
  • trade-off weight mu = 1 (default)
    In the bijection objective (Eq. 6), mu balances edit-distance opacity against embedding-similarity learnability. It is chosen by hand without a sensitivity analysis.
  • kNN neighbor count k = 100 (default)
    Algorithm 2 uses the top-k nearest neighbors as bijection candidates; this strongly shapes which token pairs are possible and is not swept.
  • alienization ratio rho = 0-1; default 1
    Controls the fraction of vocabulary permuted and directly drives the opacity-utility trade-off. Empirically swept in Figure 2, but for a given deployment it is a free choice, not a fitted constant.
axioms (5)
  • ad hoc to paper Retokenization identity: tau(tau^{-1}(f(tau(x)))) = f(tau(x))
    Assumed in Appendix B.3 for the lossless round-trip proof. Subword tokenizers can merge concatenated token strings across boundaries, so this property is not generally true.
  • domain assumption Cross-model representation alignment
    Proxy embeddings from an open-source model stand in for the target model's embeddings (Section 3.3, Appendix D.1). If relative token neighborhood structure does not transfer, bijection quality and AAT utility degrade.
  • domain assumption Honest-but-curious provider
    Appendix A.1 assumes the provider executes inference and fine-tuning faithfully and only observes/logs; active tampering with outputs is out of scope.
  • domain assumption Trusted client environment
    Appendix A.3 states that compromise of the client translator or bijection seed trivially breaks the scheme. This is standard for key-based schemes but is a load-bearing premise.
  • domain assumption Bounded plaintext-alien leakage (O2)
    Section 3.1 and Appendix E evaluate leakage up to 1,000 aligned pairs, but the fine-tuning provider receives the entire alienized training set, which can be far larger and may be alignable to public data.
invented entities (1)
  • Alien Language independent evidence
    purpose: A constructed token-level 'language' produced by a secret permutation over token IDs, used to obscure plaintext at the API boundary.
    The alien language is directly implemented in the translator and shown in concrete examples; its existence is not in question. Its security, however, is an empirical claim and is not independently evidenced beyond the paper's chosen attack scenarios.

pith-pipeline@v1.3.0-alltime-deepseek · 27477 in / 15975 out tokens · 164583 ms · 2026-08-03T06:27:02.012898+00:00 · methodology

0 comments
read the original abstract

Modern LLMs are increasingly accessed via black-box APIs, requiring users to transmit sensitive prompts, outputs, and fine-tuning data to external providers, creating a critical privacy risk at the API boundary. We introduce AlienLM, a deployable API-only \cradd{exposure-reduction layer that reduces plaintext exposure} by translating text into an Alien Language via a vocabulary-scale bijection, enabling lossless recovery on the client side. Using only standard fine-tuning APIs, Alien Adaptation Training (AAT) adapts target models to operate directly on alienized inputs. Across four LLM backbones and seven benchmarks, AlienLM retains over 81\% of plaintext-oracle performance on average, substantially outperforming random-bijection and character-level baselines. Under adversaries with access to model weights, corpus statistics, and learning-based inverse translation, recovery attacks reconstruct fewer than 0.22\% of alienized tokens. Our results demonstrate a practical pathway for \cradd{privacy-aware} LLM deployment under API-only access, substantially reducing plaintext exposure while maintaining task performance. Code and data are available at https://github.com/KimJaehee0725/AlienLM.

Figures

Figures reproduced from arXiv: 2601.22710 by Jaehee Kim, Pilsung Kang.

Figure 1
Figure 1. Figure 1: Overview of AlienLM: client-side translation via a vocabulary-level bijection, server-side processing on alien text, and lossless decoding back to plaintext using shared token IDs under a permuted mapping. task competence can be decoupled, and that shifting sur￾face form need not erase task-relevant representations (Chen et al., 2023; Deng et al., 2025; Huben et al., 2024). Together with observed cross-mod… view at source ↗
Figure 2
Figure 2. Figure 2: Figure 2 [PITH_FULL_IMAGE:figures/full_fig_p006_2.png] view at source ↗
Figure 3
Figure 3. Figure 3: Key diversity across seeds. Pairwise token overlap between seed-specific bijections remains low, supporting per-tenant keys and practical key rotation. cabulary), n-gram extrapolation achieves 0% bijection re￾covery because observed mappings provide no information about unseen tokens under a random-looking permutation. Weight-based mapping (O3). The strongest attack as￾sumes access to adapted model weights… view at source ↗
Figure 4
Figure 4. Figure 4: Examples of alienized text across alienization ratios (ρ). Green spans indicate tokens that change surface form under the alien tokenizer; ⟨U⟩ denotes non-ASCII/unicode markers. datasets cover coding, math, and general Q&A and are used consistently across backbones. E. Recovery Robustness: Settings and Extended Results This appendix provides extended details for Section 4.5. We evaluate whether an observer… view at source ↗
Figure 5
Figure 5. Figure 5: Pairwise token-overlap heatmap between seeds (overlap ratio, %; lower is better). E.2. O1: Passive Observation (Frequency Matching) Attack. The observer collects token-frequency statistics from alien text and attempts to match them to public cor￾pora token frequencies under the same tokenizer. This is the natural analogue of classical frequency analysis, but applied to subword token IDs rather than charact… view at source ↗
Figure 7
Figure 7. Figure 7: Qualitative example from MBPP (code genera￾tion). Variable names are consistently mapped (e.g., s list → productList), while byte-level tokens produce unreadable identifiers. natural Q: Write a Python function to reverse only the vow￾els of a given string. A: def reverse vowels(s): vowels = ’aeiouAEIOU’ s list = list(s) left, right = 0, len(s) − 1 while left < right: if s list[left] not in vowels: left += … view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

11 extracted references · 6 linked inside Pith

  1. [3]

    Gilad-Bachrach, R., Dowlin, N., Laine, K., Lauter, K., Naehrig, M., and Wernsing, J

    URLhttps://arxiv.org/abs/2407.21783. Gilad-Bachrach, R., Dowlin, N., Laine, K., Lauter, K., Naehrig, M., and Wernsing, J. Cryptonets: Applying neural networks to encrypted data with high throughput and accuracy. InProceedings of the 33rd International Conference on Machine Learning (ICML), pp. 201–210,

  2. [5]

    Yao, Y ., Zhang, J., Wu, J., Huang, C., Xia, Y ., Yu, T., Zhang, R., Kim, S., Rossi, R., Li, A., Yao, L., McAuley, J., Chen, Y ., and Joe-Wong, C

    URLhttps://arxiv.org/abs/2412.15115. Yao, Y ., Zhang, J., Wu, J., Huang, C., Xia, Y ., Yu, T., Zhang, R., Kim, S., Rossi, R., Li, A., Yao, L., McAuley, J., Chen, Y ., and Joe-Wong, C. Federated large language models: Current progress and future directions.arXiv preprint arXiv:2409.15723, 2024. doi: 10.48550/arXiv.2409.1572

  3. [10]

    some leakage happens

    URLhttps://arxiv.org/abs/2409.15723. Survey. Ye, R., Ge, R., Zhu, X., Chai, J., Du, Y ., Liu, Y ., Wang, Y ., and Chen, S. Fedllm-bench: Realistic benchmarks for federated learning of large language models.arXiv preprint arXiv:2406.04845, 2024. doi: 10.48550/arXiv.2 406.04845. URLhttps://arxiv.org/abs/2406 .04845. Benchmark. Zellers, R., Holtzman, A., Bis...

  4. [100]

    d ¨uzenlenen(t] 211 right left? ... 23

  5. [229]

    acl-long.229/

    URLhttps://aclanthology.org/2022. acl-long.229/. Lin, S., Hua, W., Wang, Z., Jin, M., Fan, L., and Zhang, Y . EmojiPrompt: Generative prompt obfusca- tion for privacy-preserving communication with cloud- based LLMs. In Chiruzzo, L., Ritter, A., and Wang, L. (eds.),Proceedings of the 2025 Conference of the Nations of the Americas Chapter of the Association...

  6. [2016]

    Google Cloud

    URLhttps://proceedings.mlr.press/ v48/gilad-bachrach16.html. Google Cloud. Vertex ai and zero data retention, 2024. URL https://cloud.google.com/vertex-ai/ge nerative-ai/docs/data-governance. Ac- cessed 2026-01-29. Hendrycks, D., Burns, C., Basart, S., Zou, A., Mazeika, M., Song, D., and Steinhardt, J. Measuring massive multitask language understanding.Pr...

  7. [2018]

    Cobbe, K., Kosaraju, V ., Bavarian, M., Chen, M., Jun, H., Kaiser, L., Plappert, M., Tworek, J., Hilton, J., Nakano, R., Hesse, C., and Schulman, J

    URLhttps://arxiv.org/abs/1803.054 57. Cobbe, K., Kosaraju, V ., Bavarian, M., Chen, M., Jun, H., Kaiser, L., Plappert, M., Tworek, J., Hilton, J., Nakano, R., Hesse, C., and Schulman, J. Training verifiers to solve math word problems, 2021. URLhttps://arxiv. org/abs/2110.14168. Cox, S. R., Jacobsen, R. M., and van Berkel, N. The impact of a chatbot’s ephe...

  8. [2021]

    doi: 10.1145/3474381

    ISSN 0001-0782. doi: 10.1145/3474381. URL https://doi.org/10.1145/3474381. Team, G., Riviere, M., Pathak, S., Sessa, P. G., Hardin, C., Bhupatiraju, S., Hussenot, L., Mesnard, T., Shahri- ari, B., Ram´e, A., Ferret, J., Liu, P., Tafti, P., Friesen, A., Casbon, M., Ramos, S., Kumar, R., Lan, C. L., Jerome, S., Tsitsulin, A., Vieillard, N., Stanczyk, P., Gi...

  9. [2023]

    Lin, S., Hilton, J., and Evans, O

    URLhttps://https://huggingface.co /Open-Orca/SlimOrca. Lin, S., Hilton, J., and Evans, O. TruthfulQA: Measuring how models mimic human falsehoods. In Muresan, S., Nakov, P., and Villavicencio, A. (eds.),Proceedings of the 60th Annual Meeting of the Association for Compu- tational Linguistics (Volume 1: Long Papers), pp. 3214– 3252, Dublin, Ireland, May 20...

  10. [2025]

    URLhttps: //doi.org/10.56553/popets-2025-0160

    doi: 10.56553/popets-2025-0160. URLhttps: //doi.org/10.56553/popets-2025-0160. Mishra, A., Li, M., and Deo, S. Sentinellms: en- crypted input adaptation and fine-tuning of language models for private and secure inference. InProceed- ings of the AAAI Conference on Artificial Intelligence, AAAI’24/IAAI’24/EAAI’24. AAAI Press, 2024. ISBN 978-1-57735-887-9. d...

  11. [2026]

    Accessed 2026- 01-29

    URLhttps://privacy.claude.com/en/ articles/7996866-how-long-do-you-sto re-my-organization-s-data. Accessed 2026- 01-29. Bansal, Y . S. et al. Revisiting model stitching to compare neural representations. InNeurIPS, 2021. Chen, Y ., Marchisio, K., Raileanu, R., Adelani, D. I., Stene- torp, P., Riedel, S., and Artetxe, M. Improving language plasticity via p...