Assembly of Experts: Linear-time construction of the Chimera LLM variants with emergent and adaptable behaviors

Benjamin Merkel; Daniel Klingmann; Dan Zecha; David A. Reiss; Fabian Klemm; Henrik Klagges; Robert Dahlke

REVIEW 4 major objections 6 minor 44 references

Assembly of Experts: Linear-time construction of the Chimera LLM variants with emergent and adaptable behaviors

T0 review · 4 major / 6 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read A 671B reasoning model can be assembled by transplanting only the routed-expert tensors of DeepSeek-R1 into DeepSeek-V3-0324, with no gradient training, preserving R1-level reasoning while emitting roughly 40% fewer output tokens.

desk verdict A clean, reproducible 671B merge experiment whose headline numbers need a held-out evaluation before you trust them. read the letter →

arxiv 2506.14794 v1 pith:XD6WMHWW submitted 2025-05-31 cs.LG cs.AIcs.CL

Henrik Klagges , Robert Dahlke , Fabian Klemm , Benjamin Merkel , Daniel Klingmann , David A. Reiss , Dan Zecha This is my paper

classification cs.LGcs.AIcs.CL

keywords modelmergingmixtureofexpertsroutedlosslandscapeinterpolationreasoningefficiencyDeepSeek-R1emergentbehavior671Blanguage

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper proposes Assembly-of-Experts, a method that builds new Mixture-of-Experts LLMs in linear time by linearly interpolating individual weight tensors from parent models. Applying it to the 671B DeepSeek-V3 family, the authors construct DeepSeek-R1T-Chimera by taking only R1's routed-expert tensors and keeping V3-0324's attention, shared experts, and gates. The child scores near R1 on reasoning benchmarks while generating about 40% fewer output tokens. The authors also report that all interpolated models remain functional, with some traits changing gradually and the reasoning behavior turning on sharply near an R1 fraction of about 0.5. If this holds, it means capable specialized models can be assembled without expensive fine-tuning or distillation.

What carries the argument

The central object is the Assembly-of-Experts construction defined by Eq. (1): for each tensor index l, the child tensor is either a convex combination W_l^(*) = Σ_i λ_i W_l^(i) or the base tensor, depending on whether the normalized Frobenius norm of the difference from the base exceeds a threshold δ and whether l belongs to the selected subset S. The mechanism that carries the argument is the expert-merging subset S = {routed-experts tensors only}, where routed experts are the per-token-selected feed-forward subnetworks in a Mixture-of-Experts transformer. This subset allows the authors to transplant R1's routed experts into V3-0324 without touching attention or shared experts. The per-tensor thresholding borrows the trimming idea from TIES-merging and filters out tensors that barely differ between parents.

What would settle it

Repeat the routed-expert-only transplant on another pair of same-architecture MoE checkpoints that are not in the same loss valley, for example two models fine-tuned on contradictory tasks; if the result is not a functional, capable model, the same-basin premise is doing the work rather than the tensor transplant itself.

Watch

Extended reading notes

Core claim

The central discovery is that in a 671B Mixture-of-Experts model, reasoning ability is largely carried by the routed-expert tensors: replacing only those tensors in DeepSeek-V3-0324 with the corresponding tensors from DeepSeek-R1, while leaving attention, shared experts, and router unchanged, yields a model with R1-level intelligence at roughly 40% lower output-token cost. The authors interpret this as evidence that V3 and its fine-tunes share a loss valley, so per-tensor convex combinations remain coherent. They further observe a sharp behavioral transition in the interpolation path: models with an R1 weight fraction of about 0.504 or more consistently emit <think> reasoning traces, while those below generally do not.

Load-bearing premise

The load-bearing premise is that DeepSeek-V3-0324 and DeepSeek-R1 lie in the same basin of the loss landscape, so replacing one block of tensors while keeping the rest produces a coherent model rather than a broken one.

Editorial extensions

If this is right

Building a capable variant of an existing 671B model can take minutes of tensor copying rather than the 10^13-10^15 FLOPs of pretraining or fine-tuning.
The routed experts are the main carriers of R1's reasoning ability within this architecture; attention, shared experts, and the router from V3-0324 suffice to coordinate them.
Because every interpolated model the authors tried was functional, the space of merged models is effectively searchable by scanning the mixing coefficient λ instead of retraining.
The 40% token reduction at roughly R1-level intelligence directly lowers inference cost and latency for reasoning-heavy workloads.
The same thresholded per-tensor merging recipe can be applied to future fine-tunes of the same MoE family to produce other hybrid behaviors.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

The same recipe should transfer to other MoE families whose fine-tunes share a base checkpoint; a cheap check is to replicate the routed-expert swap on a smaller model pair and compare the performance and token curves with this paper's Figures 5-7.
The sharp <think>-tag transition at a mixing coefficient near 0.5 points to a discrete behavioral phase change in weight space; other traits such as tool-use, formatting, or refusal might show similar critical ratios and could be mapped the same way.
Because every interpolation the authors tried stayed functional, model construction becomes a one-dimensional scan over λ; this could be used as a cheap knob for continuously trading off verbosity against accuracy between any two fine-tunes.

Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

The real news here is scale and simplicity. This is the first merge of 671B-parameter MoE models that works, and the recipe is almost embarrassingly easy: take V3-0324, swap in R1's routed expert tensors, keep everything else, and you get a model that looks roughly as smart as R1 while emitting about 40% fewer tokens. No training, no distillation, a few lines of tensor copying. That is a practical result worth knowing, and the sharp transition in reasoning behavior around a 50/50 R1 mix is genuinely interesting even if it is one empirical pattern rather than a law. The authors are also honest: they call the loss-valley story a hypothesis, they admit the benchmark set is not comprehensive, and they shipped the model on Hugging Face. The deterministic construction is reproducible in principle, and the released weights let others verify the claims directly.

Now the soft spots, and they are real but not disqualifying. The headline numbers rest on three benchmarks: AIME-2024 (30 problems), MT-Bench (LLM-judged, 80 questions), and BigCodeBench. No error bars, no repeated sampling, no per-task breakdown in the text. Worse, the same benchmarks were used to look at the merge curves and pick the configuration. That is a mild selection loop. I would not call it circular in the equation-fitting sense, because the final model is not a finely tuned point in parameter space—it is the natural extreme of expert-only merging, lambda=(0,1)—but the reported performance is still optimistically biased. The stress-test note is too harsh when it implies the central claim is unverified; that claim is plausible and partially supported by the model's real-world use (5B tokens/day on Chutes is anecdotal but not nothing). But the note is right that a fair test requires held-out tasks. The loss-valley hypothesis is also only evidenced by these same benchmark scores, so it is not independently confirmed.

Bottom line: this paper deserves a serious referee, but the referee should push for more benchmarks with variance, a held-out evaluation split, per-task numbers, and release of the merge and evaluation scripts. If those are added, the central claim will either hold or fail cleanly. It is a useful contribution either way, and I would cite it if I worked on model merging. For a reading group, it would spark a good discussion about evaluation selection bias, but it is not a paradigm shift.

Referee Report

4 major / 6 minor

Summary. This paper introduces Assembly-of-Experts (AoE), a deterministic method for constructing new Mixture-of-Experts language models by interpolating weight tensors of same-architecture parent models. The authors define a weighted, thresholded tensor merge rule (Eq. 1) and apply it to DeepSeek-V3-0324 and DeepSeek-R1, exploring full-model and routed-expert-only merges with varying mixing coefficients. They report that most merged models remain functional, that some capabilities change gradually while the <think> reasoning behavior appears sharply near an R1 fraction of about 0.5, and that the final R1T-Chimera model—using V3-0324 as the base with only R1's routed-expert tensors—achieves near-R1 benchmark performance with roughly 40% fewer output tokens. The central scientific claims are that this behavior supports a shared-loss-valley hypothesis for the two DeepSeek checkpoints and that expert-only merging provides a linear-time, training-free way to obtain a cost-efficient reasoning model.

Significance. If the quantitative performance claims hold, the contribution is significant: it demonstrates a practical, gradient-free construction of a 671B-parameter reasoning model from existing checkpoints, with an explicit linear-time recipe and a public model release. The paper is transparent about the merge rule, and the thresholding and tensor-group difference analysis is a useful extension of model-merging methods to large MoE architectures. The strongest assets are the reproducibility of the construction from public weights and the existence of a deployed open-weights model that can be checked independently. However, the paper's own evaluation is too thin to support the headline numbers; the configuration-selection loop and the missing uncertainty quantification are the main concerns.

major comments (4)

[§4.4, Fig. 8] The abstract's central claims—R1-level intelligence and about 40% fewer output tokens—are supported by scores on AIME-2024, MT-Bench, and BigCodeBench, but the R1T-Chimera configuration was motivated by the same AIME/MT-Bench curves in Figures 4, 6, and 7. Because the merge recipe was selected from these data, the reported final point estimates are vulnerable to selection bias. Please separate configuration-selection benchmarks from a held-out evaluation set, and report per-task numerical scores for all models.
[§3.3, Figs. 4–8] No confidence intervals, standard errors, or repeated-sampling results are reported for any benchmark or token-count figure. AIME-2024 is a 30-item test and MT-Bench is an 80-question LLM-judged set, so single-run differences of a few points are not statistically meaningful; the 40% token reduction is presented as an average over an unspecified prompt set. Please add uncertainty estimates and per-benchmark/per-question token counts.
[§3.3] The evaluation protocol does not state decoding hyperparameters (temperature, top-p, max tokens, number of samples) or the exact MT-Bench judging setup. These choices materially affect both benchmark scores and output-token counts, so the central quantitative claims are not reproducible from the paper alone. Please provide a complete protocol, including whether reported token counts include the <think>...</think> reasoning trace.
[§4.4, Hypotheses 1–2] The shared-loss-valley and routed-expert-centrality hypotheses are inferred from the same benchmark curves that motivated the construction, so they are not independently confirmed. A direct test—for example, evaluating merged models on held-out tasks, or measuring the child's loss or calibration against both parents—would substantiate the explanatory claim and the 'emergent behavior' narrative.

minor comments (6)

[Abstract/§4.4] The model name is rendered inconsistently as 'R1T-Chimera' and 'DeepSeek-R1T-Chimera'; please standardize.
[Fig. 8] Figure 8 has no numerical labels; please add a table with exact benchmark values for Chimera and both parents.
[§3.1.1] The heading contains a typo: 'W eighted-Average Merging' should be 'Weighted-Average Merging'.
[§4.1] The claim that 'no parameter sets yielded poorly performing or broken models' is based on a limited set of merge paths; please temper it or quantify the coverage of the search.
[§2.2] A direct comparison with standard task-vector or TIES-style merging at this scale would help position the expert-only merge against existing baselines.
[Figs. 2–3] The captions of Figures 2 and 3 do not define the color scale or the normalization used for the Frobenius norms; please add this information.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: Chimera is a constructed object evaluated on external benchmarks; the selection loop is a validation concern, not an equation-level circularity.

full rationale

The paper's construction (Eq. 1) takes source tensors, coefficients lambda, and a threshold delta as inputs; it contains no fitted parameters and no target quantity inside the construction. The central claim about Chimera is a reported measurement, not a derived prediction. MT-Bench, AIME-2024, and BigCodeBench are external benchmarks; the paper's use of the same benchmarks that motivated the expert-only lambda=(0,1) choice creates a model-selection bias (Sections 4.2-4.4), as the paper itself says: 'The observations of Section 4.3 in combination with those of Section 4.2 motivate the following hypotheses.' That is a statistical validation risk, not circularity: there is no equation-level identity or fitted parameter renamed as a prediction. The self-citation [11] is only a model-card link and is not load-bearing. The loss-valley hypothesis (Sections 1 and 4.4) is post-hoc, but it is not used to derive the benchmark numbers. Therefore, no circular step reaches the bar of quoting a specific reduction, and the honest finding is no significant circularity.

Assumptions & free parameters 4 free parameters · 4 assumptions · 0 invented entities

The construction uses only existing weights; no new physical or mathematical entities are postulated. The main free parameters are the merge recipe choices (lambda, delta, subset) and the ad hoc intelligence metric. The load-bearing assumptions are the shared-loss-valley premise and the router-expert compatibility premise, both stated as hypotheses rather than proven.

free parameters (4)

merge coefficients lambda_1, lambda_2 = lambda=(0,1) for Chimera
Merge coefficients for V3-0324 and R1; chosen by hand. The paper explores intermediate values and selects the extreme for Chimera.
threshold delta = delta around 0 to 3 in experiments
Threshold on normalized Frobenius norm of tensor differences; chosen based on the distribution in Figure 3, and varied to control which tensors merge.
tensor subset S = routed-experts only for Chimera
Choice of which tensor groups to merge (full vs expert-merging subset) is a free design decision; expert-merging excludes gating tensors.
intelligence score weights = 0.5 * AIME-2024 + 0.5 * MT-Bench
Ad hoc composite metric defined in Section 4.1; no justification for equal weighting or for treating this average as 'intelligence'.

assumptions (4)

domain assumption Shared loss valley: linear interpolation of V3-0324 and R1 weights stays in a functional region.
Invoked in the Introduction ('shared loss valley') and Section 5. All merge results depend on this; if false, merged models would be incoherent.
domain assumption Tensor index correspondence is semantically meaningful across the two checkpoints.
Section 3 assumes corresponding weight tensors W(i)_l exist for the same index l. This is standard in model merging but requires that no permutation or reparameterization differs between the two models.
domain assumption V3-0324's router can effectively route tokens to R1's routed expert tensors.
Section 4.4, Hypotheses 1 and 2. Chimera keeps V3-0324's gating while using R1 experts; the paper provides no mechanistic analysis of router-expert compatibility.
domain assumption AIME-2024 and MT-Bench are valid proxies for general intelligence.
Section 4.1 defines the 'intelligence score' as the average of these two benchmarks. The paper itself admits the benchmark set is not fully comprehensive.

how reviews work

0 comments

Cite this review

Pith. "Pith review of Assembly of Experts: Linear-time construction of the Chimera LLM variants with emergent and adaptable behaviors." pith.science (2026). https://pith.science/paper/XD6WMHWW

@misc{pith2026250614794,
  author       = {Pith},
  title        = {Pith review of: Assembly of Experts: Linear-time construction of the Chimera LLM variants with emergent and adaptable behaviors},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/XD6WMHWW}},
  note         = {Machine review of arXiv:2506.14794}
}

abstract

Requiring $10^{13}$-$10^{15}$ FLOPs to calculate one 8 bit weight in an LLM during pretraining is extremely expensive and seems inefficient. To better leverage the huge investments made into pretrained models, we develop the new "Assembly-of-Experts" (AoE) construction method to create capable child variants of existing Mixture-of-Experts parent models in linear time. Model weight tensors get interpolated individually, allowing to enhance or suppress semantic features of the parents. Varying the proportion of weights taken from the parent models, we observe some properties of the AoE child model changing gradually, while other behavioral traits emerge with a sharp transition. Surprisingly, nearly every generated model is functional and capable, which makes searching the model space straightforward. We construct the DeepSeek R1T "Chimera", a 671B open-weights hybrid model combining DeepSeek's V3-0324 and R1 model variants. The child inherits only the routed expert tensors of R1, but still achieves about R1-level intelligence. At the same time, it uses about 40\% fewer output tokens, close to V3 speed. Constructed without any fine-tuning or distillation, the Chimera exhibits surprisingly compact, orderly reasoning compared to its parent models.

Figures

Figures reproduced from arXiv: 2506.14794 by the authors.

**Figure 2.** Illustration of the tensor differences between DeepSeek-R1 and DeepSeek-V3-0324 categorized [PITH_FULL_IMAGE:figures/full_fig_p006_2.png] view at source ↗

**Figure 3.** Distribution of normalized tensor differences, restricted to tensors with a normalized difference [PITH_FULL_IMAGE:figures/full_fig_p007_3.png] view at source ↗

Figures from the paper (5 more)

**Figure 4.** Figure 4: Intelligence score against the resulting inference cost by means of the average DeepSeek-R1 output [PITH_FULL_IMAGE:figures/full_fig_p008_4.png]

**Figure 5.** Figure 5: Relative inference cost measured in the number of output tokens relative to the ones produces [PITH_FULL_IMAGE:figures/full_fig_p009_5.png]

**Figure 6.** Figure 6: Frequency of a </think>tag in the model response, which can be regarded a proxy for the behavioral change to a reasoning model on the path from V3-0324 to R1. We noticed an almost clear cut around a R1 fraction of 0.504. Compared to [PITH_FULL_IMAGE:figures/full_fig_p…

**Figure 7.** Figure 7: Results for several threshold values δ in eq. (1) for the case of full merging with equal V3-0324 and R1 ratio. 10 [PITH_FULL_IMAGE:figures/full_fig_p010_7.png]

**Figure 8.** Figure 8: Benchmark results for DeepSeek-R1T-Chimera. [PITH_FULL_IMAGE:figures/full_fig_p011_8.png]

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

44 extracted references · 26 canonical work pages

[1]

Model soups: Averaging weights of multiple fine-tuned models improves accuracy without increasing inference time

Mitchell Wortsman et al. “Model soups: Averaging weights of multiple fine-tuned models improves accuracy without increasing inference time”. In: Proceedings of the 39th International Conference on Machine Learning. Vol. 162. PMLR. 2022, pp. 23965–23998

work page 2022
[2]

TIES-merging: Resolving interference when merging models

Prateek Yadav et al. “TIES-merging: Resolving interference when merging models”. In: Advances in Neural Information Processing Systems 36 (2023), pp. 7093–7115

work page 2023
[3]

Animating rotation with quaternion curves

Ken Shoemake. “Animating rotation with quaternion curves”. In: Proceedings of the 12th annual con- ference on Computer graphics and interactive techniques . 1985, pp. 245–254

work page 1985
[4]

Model breadcrumbs: Scaling multi-task model merg- ing with sparse masks

MohammadReza Davari and Eugene Belilovsky. “Model breadcrumbs: Scaling multi-task model merg- ing with sparse masks”. In: European Conference on Computer Vision . Springer. 2024, pp. 270–287

work page 2024
[5]

Language models are Super Mario: Absorbing abilities from homologous models as a free lunch

Le Yu et al. “Language models are Super Mario: Absorbing abilities from homologous models as a free lunch”. In: 41st International Conference on Machine Learning . 2024

work page 2024
[6]

Model fusion via optimal transport

Sidak P Singh and Martin Jaggi. “Model fusion via optimal transport”. In: Advances in Neural Infor- mation Processing Systems 33 (2020), pp. 22045–22055

work page 2020
[7]

Transformer fusion with optimal transport

Moritz Imfeld et al. Transformer fusion with optimal transport . 2023. arXiv: 2310.05719

arXiv 2023
[8]

Evolutionary optimization of model merging recipes

Takuya Akiba et al. “Evolutionary optimization of model merging recipes”. In: Nature Machine Intel- ligence (2025), pp. 1–10

work page 2025

Show all 44 references

[9]

Command A: An Enterprise-Ready Large Language Model

Team Cohere. Command A: An Enterprise-Ready Large Language Model . 2025. arXiv: 2504.00698 [cs.CL]. url: https://arxiv.org/abs/2504.00698

2025 arXiv
[10]

Unlocking efficient long-to-short LLM reasoning with model merging

Han Wu et al. Unlocking efficient long-to-short LLM reasoning with model merging . 2025. arXiv: 2503. 20641

2025
[11]

DeepSeek R1T Chimera

TNG Technology Consulting GmbH. DeepSeek R1T Chimera . https://huggingface.co/tngtech/ DeepSeek-R1T-Chimera. Hugging Face Model Card. Apr. 2025

2025
[12]

Rayon Labs. Chutes. url: https://chutes.ai (visited on 05/27/2025)

2025
[13]

Learning to reason with LLMs

OpenAI. Learning to reason with LLMs . url: https://openai.com/index/learning-to-reason- with-llms/ (visited on 05/27/2025)

2025
[14]

DeepSeek-R1: Incentivizing reasoning capability in LLMs via reinforcement learning

DeepSeek-AI. DeepSeek-R1: Incentivizing reasoning capability in LLMs via reinforcement learning
[15]

Gemini 2.5: Our most intelligent AI model

Koray Kavukcuoglu. Gemini 2.5: Our most intelligent AI model . url: https : / / blog . google / technology/google- deepmind/gemini- model- thinking- updates- march- 2025/#gemini- 2- 5- thinking (visited on 05/27/2025)

2025
[16]

Qwen3 technical report

Qwen Team. Qwen3 technical report. 2025. arXiv: 2505.09388

2025 arXiv
[17]

Do NOT think that much for 2+ 3=? On the overthinking of o1-like LLMs

Xingyu Chen et al. Do NOT think that much for 2+ 3=? On the overthinking of o1-like LLMs . 2024. arXiv: 2412.21187

2024 arXiv
[18]

Stop overthinking: A survey on efficient reasoning for large language models

Yang Sui et al. Stop overthinking: A survey on efficient reasoning for large language models . 2025. arXiv: 2503.16419

2025 arXiv
[19]

Token-budget-aware LLM reasoning

Tingxu Han et al. Token-budget-aware LLM reasoning. 2024. arXiv: 2412.18547

2024 arXiv
[20]

Reward-guided speculative decoding for efficient LLM reasoning

Baohao Liao et al. Reward-guided speculative decoding for efficient LLM reasoning. 2025. arXiv: 2501. 19324

2025
[21]

Training language models to reason efficiently

Daman Arora and Andrea Zanette. Training language models to reason efficiently . 2025. arXiv: 2502. 04463

2025
[22]

C3oT: Generating shorter Chain-of-Thought without compromising effectiveness

Yu Kang et al. “C3oT: Generating shorter Chain-of-Thought without compromising effectiveness”. In: Proceedings of the AAAI Conference on Artificial Intelligence . Vol. 39. 23. 2025, pp. 24312–24320

2025
[23]

Can language models learn to skip steps? 2024

Tengxiao Liu et al. Can language models learn to skip steps? 2024. arXiv: 2411.01855

2024 arXiv
[24]

Kimi k1.5: Scaling reinforcement learning with LLMs

Kimi Team. Kimi k1.5: Scaling reinforcement learning with LLMs . 2025. arXiv: 2501.12599

2025 arXiv
[25]

Merge to mix: Mixing datasets via model merging

Zhixu Silvia Tao et al. Merge to mix: Mixing datasets via model merging . 2025. arXiv: 2505.16066. 14

2025 arXiv
[26]

Editing models with task arithmetic

Gabriel Ilharco et al. Editing models with task arithmetic . 2022. arXiv: 2212.04089

2022 arXiv
[27]

DeepSeekMath: Pushing the limits of mathematical reasoning in open language models

Zhihong Shao et al. DeepSeekMath: Pushing the limits of mathematical reasoning in open language models. 2024. arXiv: 2402.03300

2024 arXiv
[28]

DeepSeek-Coder: When the large language model meets programming–the rise of code intelligence

Daya Guo et al. DeepSeek-Coder: When the large language model meets programming–the rise of code intelligence. 2024. arXiv: 2401.14196

2024 arXiv
[29]

Unconstrained model merging for enhanced LLM reasoning

Yiming Zhang et al. Unconstrained model merging for enhanced LLM reasoning . 2024. arXiv: 2410. 13699

2024
[30]

Deep model fusion: A survey

Weishi Li et al. Deep model fusion: A survey . 2023. arXiv: 2309.15698

2023 arXiv
[31]

Mixtral of experts

Albert Q Jiang et al. Mixtral of experts . 2024. arXiv: 2401.04088

2024 arXiv
[32]

DeepSeek-V3 technical report

DeepSeek-AI. DeepSeek-V3 technical report. 2024. arXiv: 2412.19437

2024 arXiv
[33]

Adaptive mixtures of local experts

Robert A Jacobs et al. “Adaptive mixtures of local experts”. In: Neural computation 3.1 (1991), pp. 79– 87

1991
[34]

DeepSeekMoE: Towards ultimate expert specialization in Mixture-of-Experts language models

Damai Dai et al. DeepSeekMoE: Towards ultimate expert specialization in Mixture-of-Experts language models. 2024. arXiv: 2401.06066

2024 arXiv
[35]

DeepSeek-V2: A strong, economical, and efficient Mixture-of-Experts language model

DeepSeek-AI. DeepSeek-V2: A strong, economical, and efficient Mixture-of-Experts language model
[36]

DeepSeek-V3-0324

DeepSeek-AI. DeepSeek-V3-0324. https : / / huggingface . co / deepseek - ai / DeepSeek - V3 - 0324. Hugging Face Model Card. (Visited on 05/27/2025)

2025
[37]

DeepSeek-Prover-V2: Advancing formal mathematical reasoning via reinforcement learn- ing for subgoal decomposition

ZZ Ren et al. DeepSeek-Prover-V2: Advancing formal mathematical reasoning via reinforcement learn- ing for subgoal decomposition. 2025. arXiv: 2504.21801

2025 arXiv
[38]

The Lean 4 theorem prover and programming language

Leonardo de Moura and Sebastian Ullrich. “The Lean 4 theorem prover and programming language”. In: Automated Deduction–CADE 28: 28th International Conference on Automated Deduction, Virtual Event, July 12–15, 2021, Proceedings 28 . Springer. 2021, pp. 625–635

2021
[39]

deepseek-ai/DeepSeek-R1-0528

DeepSeek-AI. deepseek-ai/DeepSeek-R1-0528. https://huggingface.co/deepseek- ai/DeepSeek- R1-0528. Hugging Face Model Card. (Visited on 05/29/2025)

2025
[40]

PyTorch 2: Faster Machine Learning Through Dynamic Python Bytecode Trans- formation and Graph Compilation

Jason Ansel et al. “PyTorch 2: Faster Machine Learning Through Dynamic Python Bytecode Trans- formation and Graph Compilation”. In: Proceedings of the 29th ACM International Conference on Architectural Support for Programming Languages and Operating Systems, Volume 2. ASPLOS ’...

2024
[41]

Efficient Memory Management for Large Language Model Serving with Page- dAttention

Woosuk Kwon et al. “Efficient Memory Management for Large Language Model Serving with Page- dAttention”. In: Proceedings of the ACM SIGOPS 29th Symposium on Operating Systems Principles . 2023

2023
[42]

Judging LLM-as-a-Judge with MT-Bench and Chatbot Arena

Lianmin Zheng et al. “Judging LLM-as-a-Judge with MT-Bench and Chatbot Arena”. In: Advances in Neural Information Processing Systems 36 (2023), pp. 46595–46623

2023
[43]

American Invitational Mathematics Examination - AIME 2024

Mathematical Association of America (MAA). American Invitational Mathematics Examination - AIME 2024

2024
[44]

BigCodeBench: Benchmarking code generation with diverse function calls and complex instructions

Terry Yue Zhuo et al. BigCodeBench: Benchmarking code generation with diverse function calls and complex instructions. 2025. arXiv: 2406.15877. 15

2025 arXiv

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.

[1] [1]

Model soups: Averaging weights of multiple fine-tuned models improves accuracy without increasing inference time

Mitchell Wortsman et al. “Model soups: Averaging weights of multiple fine-tuned models improves accuracy without increasing inference time”. In: Proceedings of the 39th International Conference on Machine Learning. Vol. 162. PMLR. 2022, pp. 23965–23998

work page 2022

[2] [2]

TIES-merging: Resolving interference when merging models

Prateek Yadav et al. “TIES-merging: Resolving interference when merging models”. In: Advances in Neural Information Processing Systems 36 (2023), pp. 7093–7115

work page 2023

[3] [3]

Animating rotation with quaternion curves

Ken Shoemake. “Animating rotation with quaternion curves”. In: Proceedings of the 12th annual con- ference on Computer graphics and interactive techniques . 1985, pp. 245–254

work page 1985

[4] [4]

Model breadcrumbs: Scaling multi-task model merg- ing with sparse masks

MohammadReza Davari and Eugene Belilovsky. “Model breadcrumbs: Scaling multi-task model merg- ing with sparse masks”. In: European Conference on Computer Vision . Springer. 2024, pp. 270–287

work page 2024

[5] [5]

Language models are Super Mario: Absorbing abilities from homologous models as a free lunch

Le Yu et al. “Language models are Super Mario: Absorbing abilities from homologous models as a free lunch”. In: 41st International Conference on Machine Learning . 2024

work page 2024

[6] [6]

Model fusion via optimal transport

Sidak P Singh and Martin Jaggi. “Model fusion via optimal transport”. In: Advances in Neural Infor- mation Processing Systems 33 (2020), pp. 22045–22055

work page 2020

[7] [7]

Transformer fusion with optimal transport

Moritz Imfeld et al. Transformer fusion with optimal transport . 2023. arXiv: 2310.05719

arXiv 2023

[8] [8]

Evolutionary optimization of model merging recipes

Takuya Akiba et al. “Evolutionary optimization of model merging recipes”. In: Nature Machine Intel- ligence (2025), pp. 1–10

work page 2025