Pith. sign in

REVIEW 3 major objections 8 minor 26 references

Simple linear maps between final hidden states of independently trained language models largely preserve classification and can sometimes generate coherent text across models.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · grok-4.5

2026-07-13 22:19 UTC pith:5WXQS22R

load-bearing objection Solid empirical map of when linear stitching works for LLMs, including the first clear demos of multi-step generation for compatible pairs, plus a practical HE application; generation novelty is real but narrower than the abstract implies. the 3 major comments →

arxiv 2603.18908 v4 pith:5WXQS22R submitted 2026-03-19 cs.AI

Characterizing Linear Alignment Across Language Models

Matt Gorbett , Suman Jana This is my paper
classification cs.AI
keywords linear alignmentmodel stitchingrepresentational similaritycross-model generationhomomorphic encryptionprivacy-preserving inferenceCKAlanguage models
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

This paper asks how far independently trained language models have converged in their output representations, and whether that convergence is practical enough to use. The authors learn affine maps that send one model’s final (or penultimate) hidden states into another model’s feature space, then run the target’s frozen linear head or language-model head on the mapped vectors. Across embedding APIs and open models they show that classification accuracy and out-of-distribution detection remain largely intact, and—for the first time—that the same linear stitch can produce coherent zero-shot text generation on some model pairs. Success for generation is strongly predicted by tokenizer compatibility (exact token-match rate r = 0.898) and by source-model scale (pairs with sources below roughly 4B parameters fail). Building on that compatibility they introduce HELIX, a privacy protocol that fits the map on a shared public corpus and encrypts only the linear classification step under homomorphic encryption, yielding sub-second cross-silo inference without sharing private data or full model weights.

Core claim

A learned affine transformation between the final hidden states of independently trained language models is often sufficient to transfer a fixed linear head for classification and out-of-distribution detection with little loss, and in favorable cases is also sufficient to drive coherent autoregressive text generation through another model’s frozen language-model head. Tokenizer exact-match rate and source-model scale are the primary predictors of generation success; the linear correspondence itself is concentrated at the top of the residual stream.

What carries the argument

Affine alignment of final (or penultimate) hidden states: given source embeddings Z_B and target embeddings Z_A on shared inputs, solve W* = (Z_B^T Z_B + λI)^(-1) Z_B^T Z_A (ridge regression) and decode with the target’s frozen head. HELIX encrypts only this linear map and the linear classifier under CKKS, keeping the client’s query private while achieving sub-second latency.

Load-bearing premise

A shared public dataset is distributionally good enough to learn a linear map that carries the target model’s decision boundaries and next-token structure without private data or matching architectures.

What would settle it

Train the same affine map on a deliberately mismatched public corpus (different domain and length distribution) and measure whether classification accuracy, OOD AUROC, and Alpaca generation quality collapse relative to an in-distribution map; a large drop would show the public-data assumption is load-bearing.

Watch this falsifier — get emailed when new claim-graph text bears on it.

If this is right

  • Cross-model classification and OOD detection can be performed by shipping only a small affine map plus a frozen linear head, without exchanging private data or full model weights.
  • When tokenizer overlap is high and the source model is large enough (≥4B), zero-shot hybrid generation becomes possible: source embeddings decoded by a different model’s language-model head.
  • Privacy-preserving cross-silo inference can encrypt only the final linear step, reducing latency from tens of seconds to sub-second under a semi-honest threat model.
  • Layer sweeps imply that late-stage residual-stream geometry, not intermediate layers, is where cross-model linear interchangeability concentrates.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • If late-layer linear interchangeability is a general consequence of large-scale pretraining, modular multi-vendor pipelines could treat model heads and bodies as separately swappable components.
  • Tokenizer-agnostic alignment (or shared subword spaces) would be the highest-leverage extension for making generation transfer routine rather than pair-specific.
  • The same public-map + HE-linear-head pattern could apply to any modality where final representations already show high CKA, not only language embeddings.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

3 major / 8 minor

Summary. The paper empirically characterizes how well affine maps between final (or penultimate) hidden states of independently trained LLMs preserve downstream behavior. It shows that a fixed target linear head largely retains classification accuracy and Energy-based OOD AUROC when applied to source embeddings after a learned map (Tables 1–2, 6–7). For instruction-tuned models it further reports that the same style of map, trained by ridge regression on character-offset-aligned token pairs, can drive the frozen target LM head to produce coherent text for some pairs (Figs. 1, 3–5; Tables 3–4), with success strongly predicted by tokenizer exact-match rate (r=0.898, n=23) and source scale (≥4B). Building on late-layer compatibility, it proposes HELIX: a semi-honest two-party protocol that fits the map on public (optionally few-shot ID) data and encrypts only the linear classification step under CKKS, claiming sub-second latency and <1MB communication.

Significance. If the generation results hold under the stated conditions, this is a useful empirical extension of model stitching and linear identifiability to open instruction-tuned LLMs, with a clear practical takeaway (tokenizer overlap and source scale as predictors) and multi-signal evaluation (embedding similarity, dual LLM judges, human ratings on 200 samples, layer-depth ablation). The classification/OOD transfer results strengthen the case that vendor and open embedding spaces share usable linear structure. HELIX is a sensible systems application of that observation: encrypting only the linear head is a real efficiency win relative to full-transformer HE/MPC baselines. Strengths include breadth of model pairs, explicit tokenizer analysis, Appendix E’s depth sweep localizing alignability to late residual-stream states, and a concrete privacy protocol with threat-model discussion. The work is primarily empirical rather than theoretical; its value is in careful measurement and a deployable sketch, not a new theorem.

major comments (3)
  1. Abstract and §3.2 claim that linear alignment “sometimes enables text generation” and is evaluated as zero-shot cross-model decoding. The procedure is standard stitching (map h_B→h_A, freeze f_A), not leakage of the target head weights into W*, but training still uses 4k MMLU/Alpaca pairs with character-offset token matching (j=arg min_k {e_k^B | e_k^B ≥ e_i^A}). Combined with the load-bearing finding that exact token match rate predicts judge scores (r=0.898, Fig. 5; high-quality pairs ≥0.67 match), the result is better stated as: late residual streams are linearly interchangeable for multi-step decoding primarily when tokenizers already largely agree and the source is large enough. Please elevate that scope condition into the abstract and intro so the novelty claim is not read as unrestricted residual-stream interchangeability.
  2. HELIX’s pure public-data setting (§4–5, Eq. 1, Table 2) is central to the privacy story, yet Public Only often collapses relative to Full Data (e.g., TREC Gemini→OpenAI 57.6% vs 95.4%; several MNLI rows near or below useful baselines). Public+ID (64–128) recovers much of the gap but, as the paper notes, weakens zero-shot confidentiality. The abstract’s “shared public dataset” framing and the sub-second latency claim should be paired with a clear, prominent statement of when public-only transfer is adequate versus when few-shot ID exposure is required; otherwise the application overstates utility under the strongest privacy constraint.
  3. §3.2.1 / left of Fig. 3: asymmetric MMLU transfer (strong→weak preserves accuracy; weak→strong degrades sharply) is important for interpreting what is being transferred. The text attributes this to source capacity, but without controls that hold tokenizer fixed while varying scale (or vice versa) the confound with tokenizer family and training recipe remains. A short controlled comparison (same-family scale pairs already partially present, e.g. Llama-3.2-1B→Llama-3-8B) should be discussed explicitly as evidence for the capacity claim rather than left as a qualitative pattern.
minor comments (8)
  1. Fig. 1 caption and hybrid-model description are clear; ensure the main text states unambiguously that only the affine map is learned and both transformer stacks remain frozen.
  2. §3.2: “penultimate layer” vs “final hidden states” / post-final-norm is clarified in Appendix E; a one-sentence pointer in the main generation setup would help readers who skip the appendix.
  3. Table 1 OOD columns sometimes improve over baseline (e.g., TREC OpenAI→Qwen 0.738→0.916). Briefly discuss whether this is calibration recovery, distribution shift of Energy scores, or noise from small OOD sets.
  4. Related work: distinguish more sharply from Chen et al. (2025) (linear feature transfer for efficiency) and Jiang & Li (2024) (GPT–BERT stitching) so the generation-under-frozen-LM-head claim is the explicit delta.
  5. Typo/fragment in §3.2 Experimental Setup: “give me this in latex: We extract hidden states…” appears to be an author note left in the manuscript; remove before camera-ready.
  6. Appendix algorithms: Algorithm 2’s input list says alignment parameters are held by PARTYA, which contradicts the main text (PARTYB retains W*). Align the pseudocode with §4.3 / H.6–H.7.
  7. Human evaluation (§B.2) uses a single evaluator on 200 samples; state that limitation and the blinding procedure more clearly in the main text.
  8. Report ridge λ=10^{-4} and 4k training size sensitivity (Fig. 12) is useful; consider a one-line main-text note that results are stable near that operating point.

Circularity Check

0 steps flagged

Empirical OLS/ridge maps fit on training pairs and scored on held-out external metrics; no result is forced by construction or self-citation chain.

full rationale

The paper's core claims rest on ordinary least-squares (or ridge) affine maps learned from paired final/penultimate hidden states on a training split (public D_pub, MMLU/Alpaca 4k examples, or few-shot ID), then evaluated on completely held-out test prompts/samples via independent metrics: classification accuracy, Energy-score OOD AUROC, greedy generation accuracy on MMLU letters, embedding cosine to native outputs, dual LLM-as-judge scores, human Likert ratings, and tokenizer-match correlations. None of these quantities is algebraically identical to the fitted residual or to a parameter that was optimized against the same target. The closed-form W* = (Z_B^T Z_B + λI)^{-1} Z_B^T Z_A is the standard ridge solution; downstream heads remain frozen and are never re-fit. Token-offset alignment is a preprocessing step that produces training pairs, not a definition of success. HELIX encrypts only the already-learned linear head; latency and accuracy numbers are measured, not derived from the encryption scheme. Related-work citations (CKA, stitching, Platonic hypothesis, CKKS) are to external literature; no uniqueness theorem or ansatz is imported from the present authors to forbid alternatives. The single minor self-reference is the forthcoming code release, which is not load-bearing. Consequently the derivation chain is self-contained against external benchmarks and exhibits no circular reduction.

Axiom & Free-Parameter Ledger

3 free parameters · 4 axioms · 1 invented entities

The central empirical claims rest on standard linear algebra, the assumption that final-layer representations of modern LLMs share enough linear structure to be usefully mapped, the availability of a public alignment corpus, and the semi-honest CKKS threat model. HELIX itself is an engineered protocol rather than a new physical entity. Free parameters are ordinary regularization and data-size choices, not ad-hoc constants invented to force a theoretical fit.

free parameters (3)
  • ridge_lambda = 1e-4
    Regularization coefficient λ=10^{-4} used for all OLS maps; chosen by hand, not cross-validated in the main text.
  • alignment_training_size = 4000
    Number of examples (4,000) or token pairs used to fit W*; selected after a small loss-vs-size curve (Fig. 12).
  • public_plus_ID_shots = 64/128
    Optional 64 or 128 in-distribution samples mixed into D_pub for HELIX; chosen to illustrate privacy-utility trade-off.
axioms (4)
  • domain assumption Final (or penultimate) hidden states of independently trained LLMs share sufficient linear structure that an affine map can transfer linear-head behavior.
    Invoked throughout §3 and justified by CKA/SVCCA measurements and the Platonic Representation Hypothesis / Roeder et al. linear identifiability; not proved for mismatched architectures or data.
  • domain assumption A shared non-sensitive public dataset D_pub is available and distributionally adequate for fitting W*.
    Stated explicitly in §4.3 and used for all HELIX experiments; without it the privacy protocol collapses.
  • domain assumption CKKS semantic security under the semi-honest (honest-but-curious) threat model.
    Security claims of HELIX (§4.2, Appendix H) rest on this standard cryptographic assumption; malicious adversaries are out of scope.
  • standard math Ordinary least-squares / ridge regression yields the optimal linear map in Frobenius norm.
    Closed-form solution Eq. (1) is classical; used without modification.
invented entities (1)
  • HELIX protocol no independent evidence
    purpose: Two-party privacy-preserving cross-silo inference that learns an affine map on public data and encrypts only the linear classification step.
    Named framework introduced in §4; it is an engineering composition of existing HE and linear-alignment primitives rather than a new physical or mathematical object. Independent evidence is the latency and accuracy tables, not an external prediction.

pith-pipeline@v1.1.0-grok45 · 39521 in / 3183 out tokens · 33467 ms · 2026-07-13T22:19:03.867990+00:00 · methodology

0 comments
read the original abstract

Language models increasingly appear to learn similar representations, despite differences in training objectives, architectures, and data modalities. This emerging compatibility between independently trained models introduces new opportunities for cross-model alignment to downstream objectives. Moreover, this capability unlocks new potential application domains, such as settings where security, privacy, or competitive constraints prohibit direct data or model sharing. In this work, we investigate the extent to which representational convergence enables practical linear alignment between large language models. Specifically, we learn affine transformations between the final hidden states of independent models and empirically evaluate these mappings across text generation, embedding classification, and out-of-distribution detection. We find that performance is largely preserved across model pairs, and show for the first time that linear alignment sometimes enables text generation across independently trained models. We further highlight a potential application of linear alignment for privacy-preserving cross-silo inference. The framework learns an affine transformation over a shared public dataset and uses homomorphic encryption to protect client queries. By encrypting only the linear classification operation, the method achieves sub-second inference latency.

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

26 extracted references · 2 linked inside Pith

  1. [1]

    URL https://www.usenix.org/conference/usenixsecuri ty21/presentation/carlini-extracting

    ISBN 978-1-939133-24-3. URL https://www.usenix.org/conference/usenixsecuri ty21/presentation/carlini-extracting. Nicholas Carlini, Daniel Paleka, Krishnamurthy Dj Dvijotham, Thomas Steinke, Jonathan Hayase, A. Feder Cooper, Katherine Lee, Matthew Jagielski, Milad Nasr, Arthur Conmy, Itay Yona, Eric Wallace, David Rolnick, and Florian Tram`er. Stealing par...

    Pith/arXiv arXiv 2024

  2. [2]

    Feng Jiang and Jing Li

    URLhttps://arxiv.org/abs/2405.07987. Feng Jiang and Jing Li. Stitching gpt to bert for look-ahead language understanding. In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (ACL), 2024. Nikola Jovanovic, Marc Fischer, Samuel Steffen, and Martin Vechev. Private and reliable neural network inference. InProceedings of ...

    Pith/arXiv arXiv 2024

  3. [3]

    Relevance: Does the response address the prompt appropriately?

  4. [4]

    Quality: Is the response well-written, coherent, and accurate?

  5. [5]

    score": <1-10>,

    Helpfulness: Is the response useful and informative? Provide your evaluation in JSON format: {"score": <1-10>, "reasoning": "<explanation>", "relevance": <1-10>, "quality": <1-10>, "helpfulness": <1-10>} The aggregate score is the mean of the three dimensions. Both judges operate at temperature 0.0 with JSON-formatted outputs. B.2 Human Evaluation To vali...

    2000

  6. [6]

    In fact, a dog’s sense of smell is up to 1,000 to 10,000 more sensitive than a human’s

    ”c”, ”b”, ”a” Tell me one fascinating fact about dogs One fascinating fact about dogs is that they have a unique, highly developed sense of smell. In fact, a dog’s sense of smell is up to 1,000 to 10,000 more sensitive than a human’s. This is due to the large number of specialized, highly specialized, and highly specialized, and highly specialized, and hi...

  7. [7]

    To build the bridge we need construction equipment, including drills and jackhammers

    **Clean-up and Maintenance: - Organize a community clean-up event to collect and dispose of any litter, debris, and hazardous materials. - Establish a regular maintenance schedule for the park’s landscaping, including regular watering, pruning, and planting of new plants and trees. - Implement a recycling program for waste and encourage the use of reusabl...

  8. [8]

    Improved Mental Health: The presence of a pet has been shown to reduce stress, anxiety, and depression, and can improve overall mood and well- being

  9. [9]

    Increased Physical Activity: C a t s and other pets can encourage their owners to exercise and be more active, which is beneficial for overall health

  10. [10]

    LM-head-ready

    Responsibility and Responsibility: C a re for a pet can help individuals develop a sense of responsibility Create a 1-2 sentence summary of this research paper: Input: The paper, titled ”Exploring Voice-Based Human-Robot Interaction in Non-Technical Domains” describes the development of an interactive AI system that enables natural language-based human-ro...

    2016

  11. [11]

    Client encryption and transmission.PARTYB encrypts its embedding matrix under pkand sendsEnc pk(ZB)to PARTYA

  12. [12]

    This can be imple- mented as homomorphic linear aggregation over samples: Enc(Z ⊤ A ZB) = N ∑ k=1 ZA[k, :]⊤ ·Enc(Z B[k, :])

    Encrypted cross-covariance computation.PARTYA computes the encrypted cross- covariance Enc(Z ⊤ A ZB) using plaintext ZA and encrypted ZB. This can be imple- mented as homomorphic linear aggregation over samples: Enc(Z ⊤ A ZB) = N ∑ k=1 ZA[k, :]⊤ ·Enc(Z B[k, :]). PARTYA returnsEnc(Z ⊤ A ZB)to PARTYB

  13. [13]

    32 Deployment of W∗.Unlike traditional outsourced training schemes, PARTYB retains the learned map (W∗, b∗) and uses it locally during inference

    Decryption and solving.PARTYB decrypts to obtain Z⊤ A ZB in plaintext and trans- poses it to formZ ⊤ B ZA, then computes W∗ = (Z ⊤ B ZB +λI) −1Z⊤ B ZA locally using their plaintextZ B. 32 Deployment of W∗.Unlike traditional outsourced training schemes, PARTYB retains the learned map (W∗, b∗) and uses it locally during inference. PARTYA never obtains the a...

  14. [14]

    Local alignment at the client.PARTYB computes zB =g B(x) and applies the affine map locally: ˆzA =z BW∗ +b ∗

  15. [15]

    Encrypt aligned embedding.PARTYB encrypts ˆzA and sends Encpk I ( ˆzA) to PARTY A

  16. [16]

    Homomorphic classification.PARTYA evaluates the classifier on encrypted inputs: Enc( ˆy) =Enc( ˆzA)·V+c

  17. [17]

    Argmax-only outputs.To reduce leakage about (V, c) through black-box queries, the protocol may return only a predicted class label via encrypted argmax rather than full logits

    Return and decrypt.PARTYA returns the encrypted prediction, which PARTYB decrypts: ˆy=Dec sk I (Enc( ˆy)). Argmax-only outputs.To reduce leakage about (V, c) through black-box queries, the protocol may return only a predicted class label via encrypted argmax rather than full logits. This limits per-query information leakage and provides practical defense ...

    2016

  18. [18]

    Public embedding extraction; PARTYA:Z A ←g A(Dpub)∈R N×d A; PARTYB:Z B ←g B(Dpub)∈R N×d B

  19. [19]

    Encryption of client representations; PARTYB:Enc(Z B)←Enc pk(ZB); sendEnc(Z B)to PARTYA

  20. [20]

    Secure cross-covariance computation (encrypted); PARTYA: compute bC=Enc(Z ⊤ A ZB)∈ C dA×dB via homomorphic linear aggregation:; fori∈[d A],j∈[d B]:; bCij ←LN k=1 (ZA)ki ⊗Enc(Z B)kj ; PARTYA: send bCto PARTYB

  21. [21]

    Output:Predictionyrevealed to PARTYB

    Decryption and local solve (plaintext); PARTYB:C←Dec sk(bC);//C=Z ⊤ A ZB PARTYB:Σ B ←Z ⊤ B ZB +λI dB; PARTYB:W ∗ ←Σ −1 B C⊤ ;//W ∗ = (Z ⊤ B ZB +λI) −1Z⊤ B ZA PARTYB: optionally computeb ∗ (e.g., via mean-centering statistics); Algorithm 2:Privacy-Preserving Inference via Encrypted Alignment and Linear Head Input:Queryxheld by PARTYB;; representation funct...

  22. [22]

    Inference key setup; PARTYB: generate fresh HE keys(pk I,sk I ); sendpk I to PARTYA

  23. [23]

    Local encoding and encryption; PARTYB:z B ←g B(x)∈R dB; PARTYB: ˆzB ←Enc pk I (zB); send ˆzB to PARTYA

  24. [24]

    Homomorphic alignment (encrypted); PARTYA: ˆzA ← ˆzB ⊗W ∗; PARTYA: if bias is used, ˆzA ← ˆzA ⊕b ∗

  25. [25]

    Homomorphic prediction (encrypted); PARTYA: ˆy← ˆzA ⊗V⊕c; send ˆyto PARTYB

  26. [26]

    35 I Extended Related Works In this section we include additional research related to security and machine learning

    Decryption; PARTYB:y←Dec sk I ( ˆy); Remark.Guarantees are per-execution and do not preclude statistical inference under unbounded adaptive querying, a limitation shared with other ML-as-a-service and HE-based systems. 35 I Extended Related Works In this section we include additional research related to security and machine learning. Representational Simi...

    2021