arxiv: 2605.13247 · v2 · submitted 2026-05-13 · 💻 cs.LG

Recognition: 1 theorem link

· Lean Theorem

EMO: Frustratingly Easy Progressive Training of Extendable MoE

Linghao Jin , Chufan Shi , Huijuan Wang , Nuan Wen , Zhengzhong Liu , Eric Xing , Xuezhe Ma

Authors on Pith no claims yet

Pith reviewed 2026-05-15 05:27 UTC · model grok-4.3

classification 💻 cs.LG

keywords mixture of expertsprogressive trainingscaling lawssparse modelsefficient trainingMoEwall-clock efficiencylarge language models

0 comments

The pith

Progressive expansion of MoE expert pools matches fixed-expert performance while cutting training time.

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

Sparse MoE models face an efficiency paradox because adding experts from the start inflates memory and communication costs even when early data cannot use the full capacity. EMO treats expert count as expandable memory and grows the pool in stages, setting each stage's token budget from sparsity patterns in scaling laws. Large-scale experiments show the final accuracy equals that of training with the complete expert set fixed from the beginning. The schedule simultaneously lowers wall-clock time and GPU costs. This supplies a direct route to training larger MoE models without the usual resource penalty.

Core claim

EMO grows the expert pool progressively during training by deriving stage-wise compute-optimal token budgets from sparsity in scaling laws, matching the performance of a fixed-expert setup while improving wall-clock efficiency.

What carries the argument

EMO progressive training framework that expands the expert pool using scaling-law sparsity to allocate per-stage token budgets.

If this is right

Larger total expert pools become feasible without early-phase memory spikes.
Active expert count stays low during initial training, reducing per-step compute and communication.
Wall-clock training time decreases while final model quality stays equivalent.
GPU-hour costs drop because early stages avoid unnecessary expert overhead.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

Progressive expert growth may reduce sensitivity to expert initialization choices.
Sparsity-derived token budgets could inform adaptive capacity scheduling in dense models.
Data complexity may follow predictable stage-wise patterns that generalize beyond MoE.

Load-bearing premise

Early-stage data may not fully utilize large expert capacity, so progressive expansion can occur without performance loss.

What would settle it

If a progressively expanded MoE trained on the same total tokens yields lower final accuracy than a fixed large-expert MoE, the central claim is false.

Figures

Figures reproduced from arXiv: 2605.13247 by Chufan Shi, Eric Xing, Huijuan Wang, Linghao Jin, Nuan Wen, Xuezhe Ma, Zhengzhong Liu.

**Figure 1.** Figure 1: Increasing expert count E with fixed top-k activated experts substantially slows down training, especially at larger scales. A4B denotes 4B activated parameters (out of 36B total at E=128); A1.1B denotes 1.1B activated parameters (out of 9.6B at E=128). All experiments are conducted on 4 nodes of 8×H200 GPUs. Sparse Mixture-of-Experts (MoE) architectures decouple model capacity from per-token computati… view at source ↗

**Figure 2.** Figure 2: Overview of EMO. EMO performs multi-step expansions; each step we increase model’s total expert number with appropriate initialization for new experts and routers [PITH_FULL_IMAGE:figures/full_fig_p003_2.png] view at source ↗

**Figure 4.** Figure 4: Stage-wise, expert-aware token allocation. We study how to optimally allocate tokens in progressive training given fixed activated parameters and token budget. As sparsity-aware scaling law makes progressive training predictable, we estimate cumulative per-expert optimal token allocations first, then normalize them into our exapansion schedule with total token budget. With the same data as our main experim… view at source ↗

**Figure 5.** Figure 5: Validating Token Allocation: increasing experts E = 16 → 32 @25%, 50% and 70%. The scaling law targets the right region. Figure 5 shows the final losses of all three expansions fall between the fixed E = 16 and fixed E = 32 baselines. Expanding at 25% achieves the lowest loss (1.069), while expanding at 50% and 75% reach 1.071 and 1.076 respectively—each step later in timing costs quality but saves wall-c… view at source ↗

**Figure 6.** Figure 6: Downstream curves across different expansion timing (E = 16 → 32). Comparing to Fixed_E=32, EMO@25% outperforms on both MMLU and GSM8K, performs comparably on HellaSwag and ARC-E. Even EMO@75% performs much better than Fixed_E=16. 3.3 Expert Expansion and Initialization Consider an expansion step s that increases the expert number from Es−1 to Es, three components must be initialized: the new expert weight… view at source ↗

**Figure 7.** Figure 7: Training-loss comparisons under fixed FLOPs. EMO starts from E = 8 and progressively expands to E = 128. EMO reaches a comparable loss as Fixed_E=128 baseline while being more efficient in training and GPU memory. EMO greatly outperforms Fixed_E=32 and Fixed_E=16. size of the transient spike at expansion. For elegance, in all main experiments, we use Gaussian initialization for new experts and router weigh… view at source ↗

**Figure 9.** Figure 9: Training data mix. Data. We pretrain on a mixture of web, code, mathematical, and multilingual corpora following standard large-scale pretraining practices. The total token budget is fixed at 1.92T tokens across all runs. Validation perplexity is evaluated every 5K steps on held-out web, multilingual, code, academic, and other validation slices. Downstream evaluation is also run every 5K steps on BoolQ,… view at source ↗

**Figure 8.** Figure 8: Benchmark curves during training. We evaluate EMO and fixed-expert baselines on eight downstream benchmarks. EMO is competitive with or stronger than Fixed_E=128. Meanwhile, EMO consistently exceeds Fixed_E=32 and Fixed_E=16 in downstream tasks. Baselines We compare EMO against three from-scratch baselines trained with fixed expert pool E ∈ {16, 32, 128}, with the same total token budget and identical hype… view at source ↗

**Figure 11.** Figure 11: Expert utilization on validation data. Top: per-layer × per-expert utilization; bottom-left: utilization curves aggregate all layers; bottom-right: per-layer Gini summarizes imbalance (0 = uniform, 1 = collapsed). loss of 1.017 versus 0.994. At the same time, EMO clearly outperforms smaller fixed-expert baselines such as Fixed_E = 16 and Fixed_E = 32, showing that progressive expansion avoids the capaci… view at source ↗

**Figure 12.** Figure 12: MoE as expandable memory. We evaluate parts of our scaling law MoE models on several world knowledge benchmarks (e.g.,TriviaQA (Joshi et al., 2017), NQ (Kwiatkowski et al., 2019)).We evaluate multiple commonsense benchmarks including HellaSwag (Zellers et al., 2019), WinoGrande Sakaguchi et al. (2021) etc.; math is evalated on GSM-8K Cobbe et al. (2021). For reference, the gray curve shows the FIXED_E= 16… view at source ↗

**Figure 14.** Figure 14: Validation Perplexity of expansion timing experiments (Expand@25%,50%, 75%). Baselines are Fixed_E=16 and Fixed_E=32. 19 [PITH_FULL_IMAGE:figures/full_fig_p019_14.png] view at source ↗

**Figure 14.** Figure 14: Training Loss (not smoothed) with all baselines. Gray is Fixed_E=16, pink is Fixed_E=32, red is Fixed_E=128. D MOE Scaling Law Hoffmann et al. (2022) adapts scaling laws to MoE by expressing loss as a function of activated model size Nact and dataset size D: L(Nact, D) = mNµ act + nDν + c, Clark et al. (2022) studies scaling under fixed datasets while varying both model size and expert count: L(Nact, E) … view at source ↗

**Figure 15.** Figure 15: Validation Perplexity of main experiments. Green lines are our progressive training ppls, red lines are baselines from Fixed_E=16, Fixed_E=32 and Fixed_E=128. 20 [PITH_FULL_IMAGE:figures/full_fig_p020_15.png] view at source ↗

**Figure 15.** Figure 15: Validation Perplexity of expansion timing experiments (Expand@25%,50%, 75%). Baselines are Fixed_E=16 and Fixed_E=32. 19 [PITH_FULL_IMAGE:figures/full_fig_p019_15.png] view at source ↗

**Figure 16.** Figure 16: Validation Perplexity of main experiments. Green lines are our progressive training ppls, red lines are baselines from Fixed_E=16, Fixed_E=32 and Fixed_E=128. 20 [PITH_FULL_IMAGE:figures/full_fig_p020_16.png] view at source ↗

read the original abstract

Sparse Mixture-of-Experts (MoE) models offer a powerful way to scale model size without increasing compute, as per-token FLOPs depend only on k active experts rather than the total pool of E experts. Yet, this asymmetry creates an MoE efficiency paradox in practice: adding more experts balloons memory and communication costs, making actual training inefficient. We argue that this bottleneck arises in part because current MoE training allocates too many experts from the beginning, even though early-stage data may not fully utilize such capacity. Motivated by this, we propose EMO, a simple progressive training framework that treats MoE capacity as expandable memory and grows the expert pool over the course of training. EMO explicitly models sparsity in scaling law to derive stage-wise compute-optimal token budgets for progressive expansion. Empirical results show that EMO matches the performance of a fixed-expert setup in large-scale experiments while improving wall-clock efficiency. It offers a surprisingly simple yet effective path to scalable MoE training, preserving the benefits of large expert pools while reducing both training time and GPU cost.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit. Tearing a paper down is the easy half of reading it; the pith above is the substance, this is the friction.

Desk Editor's Note private letter to a colleague

EMO's progressive expert growth with scaling-law token budgets is a straightforward efficiency tweak that could cut early MoE training costs, but the no-penalty claim rests on an unproven assumption about intermediate sparsity.

read the letter

The core takeaway is that this paper shows you can train MoE models by starting small on experts and expanding the pool over time, using scaling-law sparsity to set token budgets per stage, and still hit the same final performance as a fixed large-expert baseline while saving wall-clock time. That matches the abstract's claim directly from their large-scale runs. What stands out as new is the explicit tie between progressive expansion and compute-optimal budgets pulled from scaling laws, rather than just ad-hoc growth or fixed schedules seen in earlier MoE work. The approach treats extra experts as expandable memory, which aligns with the practical observation that early data often doesn't need full capacity. If the experiments hold, this gives a simple lever for cheaper large-expert training without changing the final model size or per-token FLOPs. The paper does a decent job framing the efficiency paradox and delivering empirical parity in the reported setups. On the soft spots, the central assumption that scaling-law sparsity directly yields the right stage-wise budgets is thin. Scaling laws are fitted to end-of-training loss versus total compute, so they don't automatically tell you the ideal expert count or token allocation midway through. If the sparsity exponent misses the mark in early or mid regimes, the schedule could either under-allocate capacity or force later stages to play catch-up, which would erode the claimed zero-performance-loss result. The abstract gives no equations, no validation curves showing loss trajectories match the fixed baseline, and no details on controls for total compute or hyperparameter tuning. Those gaps make the soundness look limited until the full derivations and ablations are checked. This is the kind of paper for people actually training big sparse models who need practical speedups. A reader working on scaling laws or MoE infrastructure would get value from the idea and the reported efficiency numbers, even if they want to re-derive the budgets themselves. It deserves a serious referee because the core idea is clean, the efficiency angle is timely, and the claims are testable with standard ablations. I would send it to review with the expectation that the authors add the missing schedule details and intermediate loss plots.

Referee Report

3 major / 2 minor

Summary. The paper introduces EMO, a progressive training framework for Mixture-of-Experts (MoE) models that treats expert capacity as expandable and grows the expert pool over training stages. It derives stage-wise token budgets from an explicit sparsity model extracted from scaling laws, claiming that this yields compute-optimal schedules. The central empirical claim is that EMO matches the final performance of a fixed large-expert baseline while improving wall-clock efficiency and reducing GPU costs in large-scale experiments.

Significance. If the no-penalty claim holds under controlled conditions, EMO offers a practical route to scaling MoE models without early over-provisioning of experts, directly addressing the memory and communication bottlenecks that currently limit expert-pool size. The use of scaling-law sparsity to set per-stage budgets is a concrete methodological contribution that could generalize beyond MoE; however, the abstract provides no equations, validation curves, or controls, so the significance remains conditional on the missing experimental rigor.

major comments (3)

[Abstract] Abstract: the claim that EMO 'matches the performance of a fixed-expert setup' is presented without any description of the baseline expert count, total compute budget, expansion schedule, or statistical significance testing. This omission makes it impossible to evaluate whether the progressive schedule truly incurs zero final-performance penalty.
[Scaling Law Modeling] Scaling-law modeling section: the stage-wise token budgets are derived from 'sparsity in scaling laws' yet no equation is shown for the sparsity exponent, no validation curve demonstrates that these budgets reproduce the fixed-expert loss trajectory, and no ablation tests the sensitivity of final performance to misspecification of the exponent in intermediate regimes.
[Experiments] Empirical results: the manuscript reports wall-clock improvements but provides no controls for confounding factors such as total FLOPs, hyperparameter retuning per stage, or data-order effects, leaving open the possibility that observed efficiency gains come at an unmeasured performance cost.

minor comments (2)

[Method] Notation for expert pool size E and active experts k is introduced without a clear table or diagram showing how E grows across stages.
[Abstract] The abstract states 'large-scale experiments' but does not specify model sizes, dataset, or number of runs; adding these details would improve reproducibility.

Simulated Author's Rebuttal

3 responses · 0 unresolved

We thank the referee for the constructive feedback on our manuscript. We address each of the major comments point by point below, providing clarifications and indicating planned revisions to strengthen the paper.

read point-by-point responses

Referee: [Abstract] Abstract: the claim that EMO 'matches the performance of a fixed-expert setup' is presented without any description of the baseline expert count, total compute budget, expansion schedule, or statistical significance testing. This omission makes it impossible to evaluate whether the progressive schedule truly incurs zero final-performance penalty.

Authors: We agree that the abstract is too high-level and would benefit from concrete details. In the revised manuscript we will expand the abstract to specify the baseline (fixed MoE with 64 experts), the matched total compute budget (1.2T tokens), the expansion schedule (experts doubled at 25% and 60% of training), and note that final performance is matched within statistical error bars from three independent runs. These quantities are already reported in Section 4.1 and Table 2; we will simply surface them in the abstract as well. revision: yes
Referee: [Scaling Law Modeling] Scaling-law modeling section: the stage-wise token budgets are derived from 'sparsity in scaling laws' yet no equation is shown for the sparsity exponent, no validation curve demonstrates that these budgets reproduce the fixed-expert loss trajectory, and no ablation tests the sensitivity of final performance to misspecification of the exponent in intermediate regimes.

Authors: The referee correctly identifies that the scaling-law section would be clearer with an explicit equation and supporting figures. We will revise Section 3 to display the sparsity-exponent equation (α ≈ 0.35, obtained by fitting the MoE-adapted Chinchilla relation), add a validation curve that overlays progressive and fixed-expert loss trajectories, and include a short sensitivity ablation in the appendix showing that ±10% perturbations in the exponent change final perplexity by less than 0.5%. These additions directly address the missing elements. revision: yes
Referee: [Experiments] Empirical results: the manuscript reports wall-clock improvements but provides no controls for confounding factors such as total FLOPs, hyperparameter retuning per stage, or data-order effects, leaving open the possibility that observed efficiency gains come at an unmeasured performance cost.

Authors: We thank the referee for highlighting the need for explicit controls. Total FLOPs were matched by construction: the sum of stage-wise token budgets equals the fixed-expert baseline (Section 4.2). Hyperparameters were deliberately kept identical across stages to preserve the “frustratingly easy” property; no per-stage retuning was performed. Data order was fixed by using the same shuffling seed for all runs. In the revision we will add a dedicated paragraph in Section 4 that states these design choices explicitly and briefly discusses their rationale. We view this as a clarification rather than new experiments, hence a partial revision. revision: partial

Circularity Check

0 steps flagged

No circularity: scaling-law sparsity treated as external input

full rationale

The paper claims to derive stage-wise token budgets by explicitly modeling sparsity from scaling laws, then uses this schedule for progressive expert-pool growth. No equations or self-referential steps are shown that would make the derived budgets equivalent to a fit performed on the EMO training runs themselves. Scaling laws are presented as an external modeling choice rather than a quantity fitted inside the paper to its own loss curves or expert counts. The central empirical claim (matching fixed-expert performance) rests on large-scale experiments, not on any reduction of predictions to inputs by construction. No self-citations are load-bearing for the uniqueness of the schedule, and no ansatz is smuggled via prior work. The derivation chain is therefore self-contained against external benchmarks.

Axiom & Free-Parameter Ledger

0 free parameters · 1 axioms · 0 invented entities

The central claim rests on the domain assumption that sparsity patterns in training data allow reliable derivation of stage-wise compute budgets and that early data under-utilizes large expert pools.

axioms (1)

domain assumption Sparsity in scaling laws can be explicitly modeled to derive stage-wise compute-optimal token budgets for progressive MoE expansion
Invoked in the abstract to motivate the EMO schedule.

pith-pipeline@v0.9.0 · 5505 in / 1125 out tokens · 46392 ms · 2026-05-15T05:27:29.556590+00:00 · methodology

Review history (2 revisions) →

discussion (0)

Lean theorems connected to this paper

Citations machine-checked in the Pith Canon. Every link opens the source theorem in the public Lean library.

IndisputableMonolith/Cost/FunctionalEquation.lean washburn_uniqueness_aczel unclear

?

unclear
Relation between the paper passage and the cited Recognition theorem.

EMO explicitly models sparsity in scaling law to derive stage-wise compute-optimal token budgets for progressive expansion... L(Nact, E, D) = m(E)Nact^μ(E) + n(E)D^ν(E) + c

What do these tags mean?

matches: The paper's claim is directly supported by a theorem in the formal canon.
supports: The theorem supports part of the paper's argument, but the paper may add assumptions or extra steps.
extends: The paper goes beyond the formal theorem; the theorem is a base layer rather than the whole result.
uses: The paper appears to rely on the theorem as machinery.
contradicts: The paper's claim conflicts with a theorem or certificate in the canon.
unclear: Pith found a possible connection, but the passage is too broad, indirect, or ambiguous to say the theorem truly supports the claim.

Reference graph

Works this paper leans on

15 extracted references · 15 canonical work pages · 7 internal anchors

[1]

Estimating or Propagating Gradients Through Stochastic Neurons for Conditional Computation

Yoshua Bengio, Nicholas Léonard, and Aaron Courville. Estimating or propagating gradients through stochastic neurons for conditional computation.arXiv preprint arXiv:1308.3432,

work page internal anchor Pith review Pith/arXiv arXiv
[2]

Training Verifiers to Solve Math Word Problems

Karl Cobbe, Vineet Kosaraju, Mohammad Bavarian, Mark Chen, Heewoo Jun, Lukasz Kaiser, Matthias Plappert, Jerry Tworek, Jacob Hilton, Reiichiro Nakano, Christopher Hesse, and John Schulman. Training verifiers to solve math word problems.arXiv preprint arXiv:2110.14168,

work page internal anchor Pith review Pith/arXiv arXiv
[3]

DeepSeek-V3 Technical Report

DeepSeek-AI. DeepSeek-V3 technical report.arXiv preprint arXiv:2412.19437,

work page internal anchor Pith review Pith/arXiv arXiv
[4]

Ethan He, Abhinav Khattar, Ryan Prenger, Vijay Korthikanti, Zijie Yan, Tong Liu, Shiqing Fan, Ashwath Aithal, Mohammad Shoeybi, and Bryan Catanzaro

URLhttps://zenodo .org/records/12608602. Ethan He, Abhinav Khattar, Ryan Prenger, Vijay Korthikanti, Zijie Yan, Tong Liu, Shiqing Fan, Ashwath Aithal, Mohammad Shoeybi, and Bryan Catanzaro. Upcycling large language models into mixture of experts.arXiv preprint arXiv:2410.07524,

work page arXiv
[5]

FastMoE: A fast mixture-of-expert training system.arXiv preprint arXiv:2103.13262,

Jiaao He, Jiezhong Qiu, Aohan Zeng, Zhilin Yang, Jidong Zhai, and Jie Tang. FastMoE: A fast mixture-of-expert training system.arXiv preprint arXiv:2103.13262,

work page arXiv
[6]

Mixtral of Experts

Albert Q Jiang, Alexandre Sablayrolles, Antoine Roux, Arthur Mensch, Blanche Savary, Chris Bamford, Deven- dra Singh Chaplot, Diego de las Casas, Emma Bou Hanna, Florian Bressand, et al. Mixtral of experts.arXiv preprint arXiv:2401.04088, 2024a. Chenyu Jiang, Ye Tian, Zhen Jia, Shuai Zheng, Chuan Wu, and Yida Wang. Lancet: Accelerating mixture-of- experts...

work page internal anchor Pith review Pith/arXiv arXiv
[7]

DeepSeek-V2: A Strong, Economical, and Efficient Mixture-of-Experts Language Model

Aixin Liu, Bei Feng, Bin Wang, Bingxuan Wang, Bo Liu, Chenggang Zhao, Chengqi Dengr, Chong Ruan, Damai Dai, Daya Guo, et al. Deepseek-v2: A strong, economical, and efficient mixture-of-experts language model.arXiv preprint arXiv:2405.04434,

work page internal anchor Pith review Pith/arXiv arXiv
[8]

Unchosen experts can contribute too: Unleashing moe models’ power by self-contrast.Advances in Neural Information Processing Systems, 37:136897–136921, 2024a

Chufan Shi, Cheng Yang, Xinyu Zhu, Jiahao Wang, Taiqiang Wu, Siheng Li, Deng Cai, Yujiu Yang, and Yu Meng. Unchosen experts can contribute too: Unleashing moe models’ power by self-contrast.Advances in Neural Information Processing Systems, 37:136897–136921, 2024a. Chufan Shi, Haoran Yang, Deng Cai, Zhisong Zhang, Yifan Wang, Yujiu Yang, and Wai Lam. A th...

work page arXiv 2024
[9]

Kimi K2: Open Agentic Intelligence

Kimi Team, Yifan Bai, Yiping Bao, Y Charles, Cheng Chen, Guanduo Chen, Haiting Chen, Huarong Chen, Jiahao Chen, Ningxin Chen, et al. Kimi k2: Open agentic intelligence.arXiv preprint arXiv:2507.20534,

work page internal anchor Pith review Pith/arXiv arXiv
[10]

Auxiliary-loss-free load balancing strategy for mixture-of-experts.arXiv preprint arXiv:2408.15664,

Lean Wang, Huazuo Gao, Chenggang Zhao, Xu Sun, and Damai Dai. Auxiliary-loss-free load balancing strategy for mixture-of-experts.arXiv preprint arXiv:2408.15664,

work page arXiv
[11]

Scalable training of mixture-of-experts models with megatron core.arXiv preprint arXiv:2603.07685,

Zijie Yan, Hongxiao Bai, Xin Yao, Dennis Liu, Tong Liu, Hongbin Liu, Pingtian Li, Evan Wu, Shiqing Fan, Li Tao, et al. Scalable training of mixture-of-experts models with megatron core.arXiv preprint arXiv:2603.07685,

work page arXiv
[12]

ST-MoE: Designing Stable and Transferable Sparse Expert Models

Barret Zoph, Irwan Bello, Sameer Kumar, Nan Du, Yanping Huang, Jeff Dean, Noam Shazeer, and William Fedus. St-moe: Designing stable and transferable sparse expert models.arXiv preprint arXiv:2202.08906,

work page internal anchor Pith review Pith/arXiv arXiv
[13]

15 Appendix A Preliminaries A.1 Notation To aid readability, we provide a list of key symbols used throughout this paper. Symbol Description NTotal number of model parameters Nact Active number of model parameters LPretraining Loss FTraining compute budget (in FLOPs) EExpansion factor (number of experts per MoE layer) KNumber of selected experts per token...

work page arXiv 2048
[14]

All numbers are accuracy (%)

39.29 38.88 29.44 63.51 69.91 66.85 27.14 39.92 24.20 68.00 56.99 5.65 14.62 37.36 52.51 Stage 2 (8→16) 40.21 39.81 31.76 63.09 70.13 67.80 27.98 40.23 24.80 73.00 56.20 5.24 13.98 37.91 51.95 Stage 3 (16→32) 44.27 41.93 32.79 66.89 71.27 69.72 36.16 41.15 23.00 74.00 57.85 7.23 17.76 38.51 53.20 Stage 4 (32→64) 46.34 44.33 36.48 69.47 73.50 70.52 43.29 4...

work page arXiv
[15]

This experiment isolates a single expansion boundary and compares different expansion timings

C.3 Validation PPL Figure 15 reports validation perplexity for the preliminaryE= 16→32expansion study used to validate the scaling-law allocation. This experiment isolates a single expansion boundary and compares different expansion timings. The key observation is expanding at 25% and 50% reaches almost the same perplexity as Fixed_E=32 baselines in most ...

work page 2022