REVIEW 4 major objections 5 minor 21 references
Superposition in Transformers: A Novel Way of Building Mixture of Experts
T0 review · 4 major / 5 minor · reviewed 2026-08-10 · deepseek-v4-flash
Pith's one-line read Merging two GPT-2 checkpoints by B-spline-blending hidden states and reconstructing them with autoencoders preserves both languages better than linear interpolation or task arithmetic.
desk verdict New merging idea, but the headline comparison is measured on training perplexity, so the main claim is unsupported as written. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
Per layer $l$, the blended hidden state is $h_l = (1-\alpha(l))h^{\text{base}}_l + \alpha(l)h^{\text{fine}}_l$, with $\alpha(l) = \operatorname{clamp}\left(\sum_{i=1}^N c_i B_{i,k}(l) + b_l, 0, 1\right)$, where $B_{i,k}$ are B-spline basis functions, $c_i$ are trainable control points, and $b_l$ is a layer bias. At selected layers an autoencoder compresses the blend to a bottleneck $z_l = \operatorname{Encoder}(h_l)$ and reconstructs $\hat{h}_l = \operatorname{Decoder}(z_l)$, trained to match either $h^{\text{base}}_l$ or $h^{\text{fine}}_l$ while a language-modeling loss keeps the output fluent. The B-spline makes $\alpha(l)$ vary smoothly across layers, the bottleneck creates polysemantic pressure, and joint training lets $\alpha(l)$ shift toward the expert state the autoencoder can reconstruct most faithfully for the current input.
What would settle it
Train the merging procedure with domain labels withheld and compare the unlabeled merged model's perplexity against linear interpolation on both languages; or feed the trained model a mixed English–French sequence and compare its reconstructed layer-4 states with the base and fine-tuned target states. If unlabeled training no longer beats the interpolation perplexity, or if the reconstructed states do not align with the correct expert model on mixed inputs, the adaptive label-free switching the method depends on is not actually happening.
Extended reading notes
Core claim
The paper's central claim is that a transformer merged by B-spline-blending hidden states plus autoencoder reconstruction outperforms both linear interpolation and task arithmetic for combining a base and a fine-tuned model. Concretely, it reports overall perplexity 47.01 and next-token accuracy 0.3270 for the autoencoder-merged model, versus 60.29 and 0.3039 for linear interpolation and 61.30 and 0.2957 for task arithmetic. The authors interpret this as evidence that the autoencoders act as a learned gate over the representation space: for English inputs the reconstructed states align with the base model, for French inputs they align with the fine-tuned model, and the bottleneck forces the two expert states to share neurons, increasing the fraction of polysemantic neurons. The base and fine-tuned weights stay frozen throughout, so the only trained parameters are B-spline control points, layer biases, and autoencoders; the paper calls the result an 'in-model' superposition.
Load-bearing premise
The load-bearing premise is that the autoencoder, at inference time, will infer from the blended hidden state alone whether the input belongs to the base or fine-tuned domain and reconstruct the right expert state—training uses domain labels (Section 3.5.4), no experiment isolates the label-free case, and the Limitations section concedes that switching within a single input is not supported.
Editorial extensions
If this is right
- A base model and a fine-tuned model can be collapsed into one set of merged parameters plus small autoencoders, so one checkpoint can serve two specializations without storing both original models separately.
- Because the base and fine-tuned weights stay frozen and only the B-spline control points, biases, and autoencoders are trained, the cost of merging is small relative to fine-tuning or to building a mixture of experts with separate expert modules.
- The merged model's lower perplexity and higher next-token accuracy than linear interpolation and task arithmetic imply that smooth layer-wise blending with reconstruction is a stronger parameter-efficient merging baseline than simple weight averaging.
- The reported rise in polysemantic neurons under the bottleneck suggests the method trades some per-domain specialization for shared, reusable features, making bottleneck size a direct control on the specialization-versus-generality balance.
Reading between the lines
- Extension: bottleneck width should trace a measurable polysemanticity-versus-domain-accuracy frontier; a sweep of bottleneck sizes on the same two models could map that frontier without retraining either expert.
- Extension: because the learned blending coefficient depends only on layer index, the $\alpha(l)$ profile is interpretable as a layer-wise routing policy, and comparing that profile across language pairs could reveal which layers are consistently more transferable.
- Extension: the same autoencoder-gated blending could in principle merge more than two models by extending the reconstruction target to several expert states, although the paper explicitly leaves multi-model scaling unresolved.
- Extension: if the domain labels used in training are truly optional, the method points toward a fully self-supervised model-merging objective in which the autoencoder's reconstruction error alone selects the expert state; an experiment that trains without labels would settle this.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper proposes a model-merging method in which a base and a fine-tuned transformer are combined by interpolating their per-layer hidden states with B-spline-generated coefficients, with autoencoders inserted at selected layers to reconstruct either the base or fine-tuned hidden state. The training objective combines language modeling, reconstruction, and optional alpha-regularization losses. Experiments on English/French GPT-2 models compare the merged model's perplexity and accuracy with linear interpolation and task arithmetic, and analyze neuron sparsity, polysemanticity, and hidden-state geometry. The central quantitative claim is that the autoencoder-merged model reaches lower perplexity (47.01) than linear interpolation (60.29) or task arithmetic (61.30) at the last training epoch.
Significance. The idea of merging two models through a low-cost learned interpolation with autoencoder reconstruction is interesting and potentially practical, and the paper is honest in listing limitations, including the absence of intra-input switching. However, the significance of the reported results is not yet established: the headline comparison is computed on training data, the label-free inference mechanism is unvalidated, and the paper vacillates between hidden-state blending and parameter-space merging. If the authors supply held-out evaluations with a clear protocol and resolve the method inconsistency, the contribution could be a modest but useful addition to the model-merging literature.
major comments (4)
- [Section 4.2.2, Figure 2, Section 4.1] The central comparison is reported as perplexity “during the last epoch of training,” not on held-out data. The validation split described in Section 4.1 is not used for this comparison, no seeds or error bars are given, and the protocol for the static interpolation and task-arithmetic baselines across epochs is unclear. Since the merged method trains roughly 1.7 million extra parameters per autoencoder, lower training perplexity may reflect memorization rather than generalization. Please report held-out perplexity and accuracy on the validation or test set for all methods under an identical protocol, ideally with multiple seeds and error bars.
- [Section 3.5.4 and Sections 3.3.2, 4.3.1] The autoencoder is trained with domain labels to decide whether to reconstruct h_base or h_fine, but labels are not available at inference. The paper assumes the autoencoder will infer the correct expert state from the blended hidden state alone; no experiment isolates this mechanism. Please add an experiment that measures reconstruction error of the merged model's output relative to h_base versus h_fine for English and French inputs without using labels, or compare with an oracle-label upper bound.
- [Section 3.2.2, Figure 1, Appendix Algorithm 1] The method is defined inconsistently. Section 3.2.2 and Figure 1 blend hidden states h_base_l and h_fine_l, while Algorithm 1 first merges parameters theta_l = (1-alpha)theta_base + alpha theta_fine and then runs the autoencoder on the activations of the merged parameter set. These are different operations and should lead to different results. Please specify which forward pass was actually trained and evaluated.
- [Abstract, Section 3.3.1, Section 5] The abstract claims the method “supports dynamic switching between model states during inference,” but Section 5 states that the approach “does not support dynamic state switching within a single input.” This is a direct contradiction and should be resolved by removing the overclaim or by demonstrating the dynamic-switching scenario.
minor comments (5)
- [Section 2] The heading contains a typo: “learing” should be “learning,” and “F orgetting” has a stray space.
- [Section 4.2.2] The task arithmetic perplexity is labeled “I-PPL=61.30”; this appears to be a typo for “T-PPL.”
- [Table 1] The table does not state whether the perplexities are on the validation split, the training split, or a separate test set, and no confidence intervals are given.
- [Figure 2] The figure lacks a description of how many epochs the static baselines were run for; linear interpolation and task arithmetic do not train, so it is unclear what “perplexity evolution” means for them.
- [Section 3.5.3 and Algorithm 1] Section 3.5.3 says backpropagation updates “alpha and the autoencoder parameters,” but Algorithm 1 updates only the control points and autoencoder parameters; the set of trainable variables should be stated consistently.
Circularity Check
The claimed superiority over linear interpolation and task arithmetic rests on last-epoch training perplexity, which is the exact objective the merged model was trained to minimize; held-out Table 1 results give the paper only partial independent content.
-
fitted input called prediction
[Section 4.2.2 (Overall Performance), Eq. (8) in Section 3.5.2, Algorithm 1 lines 26-29]
"The perplexity of the autoencoder-merged model (M-PPL=47.01) was markedly lower than that of the linearly interpolated model (I-PPL=60.29) and the task arithmetic model ( I-PPL=61.30) during the last epoch of training, suggesting a higher confidence in predicting the next token."
M-PPL is the exponential of the language-modeling loss LLM that is explicitly optimized in the total loss L1D = λReconLRecon + λLMLLM (Eq. 8; Algorithm 1 lines 26-29). The B-spline control points, biases, and autoencoder parameters are trained to minimize this loss on the combined corpus, and the reported number is taken during the last epoch of training, i.e., at convergence of that optimization. The linear-interpolation and task-arithmetic baselines are static merges with no parameters fit to the corpus, so the comparison reports an optimized training objective against unoptimized baselines. The lower M-PPL is therefore statistically forced by the fitting procedure, not an independent held-out prediction or demonstration of generalization.
full rationale
The paper's headline empirical claim—that Superposition Merging outperforms linear interpolation and task arithmetic—is supported in Section 4.2.2 only by perplexity measured during the last epoch of training. Perplexity is exp(LLM), and LLM is a component of the total loss L1D that the trainable control points, biases, and autoencoders are explicitly optimized to minimize on the same combined corpus. Reporting the converged training objective as the metric of advantage over static, unoptimized baselines makes that particular comparison circular in the fitted-input-called-prediction sense. I did not score this higher because the paper also reports per-language perplexity in Table 1 on an evaluation split (10% held out per Section 4.1), which independently shows the merged model bridging base and fine-tuned performance, and because there are no load-bearing self-citations: all cited works are external and no prior work of the authors is invoked. The polysemantic-neuron analysis has a self-confirmatory flavor, but the paper itself flags the fixed-threshold limitation and the bottleneck pressure is an explicit design choice rather than a hidden imported assumption, so I do not count it as a separate circular step under the hard rules.
Assumptions & free parameters
free parameters (8)
- B-spline control points {c_i}
- Layer biases {b_l}
- Autoencoder weights and biases
- Loss weights lambda_LM, lambda_Recon, lambda_Alpha
- Bottleneck size (1D model) =
576
- B-spline degree k and control point count N
- Polysemantic neuron threshold =
0.05
- Variance bias target sigma^2_target
assumptions (4)
- domain assumption Base and fine-tuned models have identical architectures and their hidden states and parameters are meaningfully alignable by layer.
- ad hoc to paper A single autoencoder can reconstruct each expert's state from an interpolation that depends only on the input's content, without explicit domain labels at inference.
- ad hoc to paper The alpha blending coefficients and autoencoder jointly converge to a useful solution with the given loss weights and B-spline settings.
- domain assumption The English subset (from GPT-2 training data) and French Wikipedia articles are representative evaluation sets for the two domains.
Cite this review
Pith. "Pith review of Superposition in Transformers: A Novel Way of Building Mixture of Experts." pith.science (2026). https://pith.science/paper/HIBJPX2A
@misc{pith2026250100530,
author = {Pith},
title = {Pith review of: Superposition in Transformers: A Novel Way of Building Mixture of Experts},
year = {2026},
howpublished = {\url{https://pith.science/paper/HIBJPX2A}},
note = {Machine review of arXiv:2501.00530}
}
read the original abstract
Catastrophic forgetting remains a major challenge when adapting large language models (LLMs) to new tasks or domains. Conventional fine-tuning often overwrites existing knowledge, causing performance degradation on original tasks. We introduce Superposition in Transformers, a novel architecture that leverages autoencoders to superimpose the hidden representations of a base model and a fine-tuned model within a shared parameter space. By using B-spline-based blending coefficients and autoencoders that adaptively reconstruct hidden states based on the input data distribution, our method effectively mitigates catastrophic forgetting and enables a new paradigm of "in-model" superposition. This approach preserves original model capabilities while allowing compact domain-specific expertise to be added, and it supports dynamic switching between model states during inference.
Figures
Figures from the paper (5 more)
Reference graph
Works this paper leans on
-
[1]
Tom B. Brown, Benjamin Mann, Nick Ry- der, Melanie Subbiah, Jared Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, Sandhini Agar- wal, Ariel Herbert-Voss, Gretchen Krueger, Tom Henighan, Rewon Child, Aditya Ramesh, Daniel M. Ziegler, Jeffrey Wu, Clemens Win- ter, Christopher Hesse, Mark Chen, Eric Sigler, Mateusz ...
arXiv 2005
-
[2]
OpenAI. GPT-4 Technical Report. arXiv preprint arXiv:2303.08774, 2023
arXiv 2023
-
[3]
M. McCloskey and N. J. Cohen. Catastrophic interference in connectionist networks: The se- quential learning problem. Psychology of Learn- ing and Motivation, vol. 24, pp. 109–165, 1989
work page 1989
-
[4]
Disentan- gling Neuron Representations with Concept Vec- tors.arXiv preprint arXiv:2304.09707, 2023
Laura O’Mahony, Vincent Andrearczyk, Hen- ning Muller and Mara Graziani. Disentan- gling Neuron Representations with Concept Vec- tors.arXiv preprint arXiv:2304.09707, 2023
arXiv 2023
-
[5]
Andrei A. Rusu, Neil C. Rabinowitz, Guil- laume Desjardins, Hubert Soyer, James Kirk- patrick, Koray Kavukcuoglu, Razvan Pascanu and Raia Hadsell. Progressive neural networks. arXiv preprint arXiv:1606.04671, 2016
arXiv 2016
-
[6]
James Kirkpatrick, Razvan Pascanu, Neil Rabi- nowitz, Joel Veness, Guillaume Desjardins, An- drei A. Rusu, Kieran Milan, John Quan, Tiago Ramalho, Agnieszka Grabska-Barwinska, Demis Hassabis, Claudia Clopath, Dharshan Kumaran and Raia Hadsell. Overcoming catastrophic for- getting in neural networks. Proceedings of the National Academy of Sciences, vol. 11...
work page 2017
-
[7]
Robert A. Jacobs, Michael I. Jordan, Steven J. Nowlan and Geoffrey E. Hinton. Adaptive mixtures of local experts. Neural Computation, vol. 3, no. 1, pp. 79–87, 1991
work page 1991
-
[8]
Outrageously large neural net- works: The sparsely-gated mixture-of-experts layer
Noam Shazeer, Azalia Mirhoseini, Krzysztof Maziarz, Andy Davis, Quoc Le, Geoffrey Hinton, and Jeff Dean. Outrageously large neural net- works: The sparsely-gated mixture-of-experts layer. International Conference on Learning Rep- resentations, 2017
work page 2017
Show all 21 references
-
[9]
Toward Inference-optimal Mixture-of-Expert Large Language Models
Longfei Yun, Yonghao Zhuang, Yao Fu, Eric P Xing and Hao Zhang. Toward Inference-optimal Mixture-of-Expert Large Language Models. arXiv preprint arXiv:2404.02852, 2024
2024 arXiv
-
[10]
Parameter-efficient transfer learn- ing for NLP
Neil Houlsby, Andrei Giurgiu, Stanislaw Jas- trzebski, Bruna Morrone, Quentin de Larous- silhe, Andrea Gesmundo, Mona Attariyan, and Sylvain Gelly. Parameter-efficient transfer learn- ing for NLP. International Conference on Ma- chine Learning, 2019
2019
-
[11]
Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, and Weizhu Chen
Edward J. Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, and Weizhu Chen. LoRA: Low-rank adaptation of large language models. International Confer- ence on Learning Representations, 2021
2021
-
[12]
AdaMix: Mixture-of-Adaptations for Parameter-efficient Model Tuning
Yaqing Wang, Sahaj Agarwal, Subhabrata Mukherjee, Xiaodong Liu, Jing Gao, Ahmed Hassan Awadallah and Jianfeng Gao. AdaMix: Mixture-of-Adaptations for Parameter-efficient Model Tuning. arXiv preprint arXiv:2205.12410, 2022
2022 arXiv
-
[13]
Pushing mixture of experts to the limit: Ex- tremely parameter efficient moe for instruction tuning
Ted Zadouri, Ahmet ¨Ust¨ un, Arash Ahmadian, Beyza Ermi¸ s, Acyr Locatelli and Sara Hooker. Pushing mixture of experts to the limit: Ex- tremely parameter efficient moe for instruction tuning. arXiv preprint arXiv:2309.05444, 2023
2023 arXiv
-
[14]
Omni-SMoLA: Boosting Generalist Multimodal Models with Soft Mix- ture of Low-rank Experts
Jialin Wu, Xia Hu, Yaqing Wang, Bo Pang and Radu Soricut. Omni-SMoLA: Boosting Generalist Multimodal Models with Soft Mix- ture of Low-rank Experts. arXiv preprint arXiv:2312.00968, 2023
2023 arXiv
-
[15]
Toy Models of Superposition
Nelson Elhage, Tristan Hume, Catherine Ols- son, Nicholas Schiefer, Tom Henighan, Shauna Kravec, Zac Hatfield-Dodds, Robert Lasenby, Dawn Drain, Carol Chen, Roger Grosse, Sam McCandlish, Jared Kaplan, Dario Amodei, Martin Wattenberg and Christopher Olah. Toy Models of Superpos...
2022 arXiv
-
[16]
Zoom In: An Introduction to Circuits
Chris Olah, Nick Cammarata, Ludwig Schubert, Gabriel Goh, Michael Petrov, and Shan Carter. Zoom In: An Introduction to Circuits. Distill, 2020
2020
-
[17]
Sparse autoencoders find highly interpretable features in language models
Robert Huben, Hoagy Cunningham, Logan Riggs, Aidan Ewart, and Lee Sharkey. Sparse autoencoders find highly interpretable features in language models. International Conference on Learning Representations, 2024. 9
2024
-
[18]
Towards Monosemanticity: Decomposing Language Mod- els With Dictionary Learning
Trenton Bricken, Adly Templeton, Joshua Bat- son, Brian Chen, Adam Jermyn, Tom Con- erly, Nick Turner, Cem Anil, Carson Deni- son, Amanda Askell, Robert Lasenby, Yifan Wu, Shauna Kravec, Nicholas Schiefer, Tim Maxwell, Nicholas Joseph, Zac Hatfield-Dodds, Alex Tamkin, Karina N...
2023
-
[19]
Train big, then compress: Rethinking model size for efficient training and inference of transformers
Tao Li, Lei Deng, Sheng Lin, Meng Li, Dong Huang, and Yuan Xie. Train big, then compress: Rethinking model size for efficient training and inference of transformers. International Confer- ence on Machine Learning, 2020
2020
-
[20]
Merging mod- els with fisher-weighted averaging
William Matena and Colin Raffel. Merging mod- els with fisher-weighted averaging. International Conference on Learning Representations, 2022
2022
-
[21]
Language Models are Unsupervised Multitask Learners
Alec Radford, Jeffrey Wu, Rewon Child, David Luan, Dario Amodei and Ilya Sutskever. Language Models are Unsupervised Multitask Learners. OpenAI, 2019. 10 Appendix A Training Algorithms A.1 Loss Functions The losses involved in the different architectures combines: Reconstructi...
2019
Reviewed August 10, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.