{"id":"2143325e-b799-40f3-9ffb-1410ccc437cd","arxiv_id":"2604.00421","paper_version":2,"verdict":"CONDITIONAL","confidence":"LOW","novelty_score":6.5,"correctness_risk":"medium","formal_verification":"none","parameter_count":2,"one_line_summary":"A designated subspace of the token hidden state can replace a learned MoE router and stay competitive on language modeling and ImageNet while improving routing balance.","lead":"Self-Routing drops the learned MoE router and uses a slice of each token’s hidden state as expert scores. If it holds up, MoE models can keep capacity gains with fewer parameters and more balanced expert use.","discovery_kind":"new_method","skeptic_critique":{"model":"grok-4.5","headline":"Abstract-only review leaves the core premise (that a fixed hidden-state subspace can serve as expert logits without fatally harming representation quality) uncheckable; no load-bearing flaw can be confirmed or refuted from available text.","rationale":"The Reader correctly identified the weakest assumption and assigned CONDITIONAL / LOW confidence precisely because the abstract cannot substantiate it. No additional internal contradiction is visible from the abstract alone, and inventing one would violate the good-faith rule. The concrete test above is the minimal experiment that would convert the premise from an uncheckable claim into a falsifiable one; until that (or equivalent full-text evidence) appears, the verdict should stay CONDITIONAL. Agreement with the Reader is therefore complete on both the concern and the recommended disposition.","tokens_in":2021,"tokens_out":506,"duration_ms":5478,"concrete_test":"Once the full paper or code is released, re-train the language-modeling MoE under three controlled variants: (1) Self-Routing with the authors' exact subspace rule, (2) Self-Routing after randomly permuting which dimensions are designated as logits each step (destroying any learned specialization), and (3) a learned-router baseline with identical expert count and capacity. If (1) remains competitive with (3) while (2) collapses toward random-routing performance, the premise holds; if (1) and (2) are statistically indistinguishable, the claimed emergence is illusory.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim rests on the premise that a designated subspace of the token hidden state already (or after joint training) carries sufficient expert-discriminative signal to act as logits, and that reserving those dimensions for routing does not degrade the residual representation enough to erase the capacity gains of MoE. Because only the abstract is available, neither the subspace selection rule (fixed indices? learned mask? first-k dims?), the training dynamics that would allow the subspace to specialize, nor any ablation that isolates representation damage can be inspected. The reported competitiveness and ~17% entropy gain are therefore uninterpretable: they could reflect a genuine free lunch or an artifact of an unstated design choice that effectively reintroduces a soft router. This is the same soft spot the Reader flagged; with no full text it remains the single load-bearing uncertainty rather than a demonstrated inconsistency.","agreement_with_reader":"agree"},"referee_report":{"model":"grok-4.5","summary":"The manuscript proposes Self-Routing, a parameter-free Mixture-of-Experts routing scheme that treats a designated subspace of each token’s hidden state as expert logits, thereby removing the learned router projection while leaving the rest of the MoE layer unchanged. On language-modeling tasks across expert counts and model scales, and on ImageNet-1K with DeiT-S/16, the abstract reports that Self-Routing remains competitive with a standard learned-router baseline, slightly improves ImageNet accuracy relative to that baseline, and yields roughly 17% higher average normalized routing entropy without an explicit load-balancing loss. The central claim is that effective expert assignment can emerge from the hidden representation itself, rendering a dedicated router unnecessary.","tokens_in":2240,"tokens_out":776,"duration_ms":16354,"significance":"If the empirical claims hold under full scrutiny, Self-Routing would simplify MoE design by eliminating all dedicated routing parameters and potentially the need for auxiliary load-balancing losses, while remaining competitive with learned routers. A parameter-free construction that still produces more balanced expert utilization is a concrete engineering and scientific contribution; the ImageNet result, if real, further suggests the idea is not language-specific. These strengths are currently asserted only at abstract level and therefore remain provisional.","major_comments":[{"comment":"The abstract’s central claim—that a designated hidden subspace can serve as expert logits without a learned projection—rests on an unstated subspace-selection rule (fixed indices, first-k dimensions, learned mask, etc.). Without that rule, and without any ablation isolating representation damage from reserving those dimensions, the reported competitiveness and the parameter-free claim cannot be assessed; this is load-bearing for every result advertised.","section":"Abstract"},{"comment":"The ~17% higher average normalized routing entropy is presented as evidence of balanced utilization without a load-balancing loss. The abstract supplies no definition of the entropy metric, no comparison protocol against the learned-router baseline under matched capacity and training budget, and no check that training dynamics do not reintroduce soft routing through the shared representation. Until those details appear, the entropy gain is uninterpretable.","section":"Abstract"},{"comment":"Competitiveness on language modeling “across expert counts and model scales” and the slight DeiT-S/16 ImageNet gain are stated without numbers, error bars, training details, or dense/random baselines quantified. These comparisons are the sole empirical support for the claim that Self-Routing does not fatally harm residual representation quality; they must be fully specified before the claim can be accepted or rejected.","section":"Abstract"}],"minor_comments":[{"comment":"The abstract uses “designated subspace” without even a parenthetical indication of dimensionality relative to hidden size or expert count; a one-sentence clarification would help readers judge capacity trade-offs.","section":"Abstract"},{"comment":"“About 17 % higher average normalized routing entropy” should state the baseline value and the aggregation (mean over layers/tokens/runs) so the figure is reproducible from the text alone.","section":"Abstract"}],"recommendation":"uncertain","confidential_remarks":"Only the abstract was available for this review; the full manuscript could not be inspected. I therefore cannot verify soundness, novelty relative to prior router-free or representation-based routing work, or experimental rigor. Recommendation is uncertain pending the complete paper. If the full text is supplied, the three major comments above are the first items I would re-check against sections, equations, and tables."},"author_rebuttal":null,"desk_editor":{"model":"grok-4.5","letter":"The one thing to know: this is a simple, concrete proposal—use a designated subspace of the token hidden state as expert logits and drop the learned router entirely—and the abstract claims it stays competitive with a standard router while improving load balance (~17% higher normalized routing entropy) without a balancing loss. That is a useful within-MoE simplification if the numbers hold.\n\nWhat is new is the design choice itself. Most MoE work treats the router as a necessary learned projection; Self-Routing removes those parameters by construction and leaves the rest of the layer alone. The baselines they name (learned router, random, dense) are the right ones, and they report results across expert counts and scales on LM plus a small ImageNet gain with DeiT-S/16. Circularity burden is low: they are not fitting a router to the metric they then celebrate.\n\nThe soft spot is exactly the one the stress-test flags, and it is load-bearing rather than minor. Everything rests on the premise that a fixed designated subspace already (or after joint training) carries enough expert-discriminative signal and that reserving those dimensions does not wreck the residual representation. With only the abstract we cannot see how the subspace is chosen (fixed indices? first-k? something else?), whether any ablation isolates representation damage, or the training dynamics that would let the subspace specialize. So the competitiveness and entropy claims are currently uninterpretable—they could be a free lunch or an artifact of an unstated design that soft-reintroduces routing capacity. Free parameters are the subspace designation and the usual expert/top-k choices; nothing invented.\n\nThis is for people who build or ablate sparse transformers and MoE layers. A serious referee should see the full paper: if the methods section specifies the subspace rule cleanly and the tables match the abstract, it deserves a careful review and possible acceptance after revision. If the subspace choice is hand-wavy or the gains vanish under ablation, it is a short note. I would not desk-reject it on the abstract alone. Bring it to reading group only once the PDF is up; until then it is a clean idea with an open empirical check.","headline":"Clean parameter-free MoE idea that is worth a look if the full paper backs the abstract; currently uncheckable on the key premise.","tokens_in":2838,"tokens_out":533,"would_cite":false,"duration_ms":4465,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"grok-4.5","headline":"Self-Routing turns a fixed subspace of each token’s hidden state into expert logits, matching learned MoE routers with zero routing parameters.","keywords":["Mixture-of-Experts","Self-Routing","parameter-free routing","expert utilization","hidden states","language modeling","ImageNet classification","load balancing"],"falsifier":"Train matched Self-Routing and learned-router MoE models at the same scale, expert count, and compute; if Self-Routing shows clearly worse language-model perplexity or ImageNet top-1 accuracy, or collapses into substantially more unbalanced expert loads than the baseline, the central claim fails.","tokens_in":2894,"feed_emoji":"🔀","tokens_out":806,"duration_ms":15557,"temperature":0.7,"pith_summary":"Mixture-of-Experts layers usually need a learned router that maps hidden states to expert choices. This paper asks whether that router is necessary at all. Self-Routing designates a fixed subspace of the token hidden state and treats those dimensions directly as expert logits, removing the router projection and all of its parameters while leaving the rest of the MoE layer unchanged. Across language-modeling scales and expert counts, and on ImageNet-1K with DeiT-S/16, the method stays competitive with a standard learned router, slightly beats it on the vision task, and yields more balanced expert use—about 17% higher average normalized routing entropy—without any explicit load-balancing loss. The claim is that effective expert selection can already be read out of the representation itself.","feed_headline":"MoE routing without a router: hidden states pick the experts","feed_subtitle":"A fixed subspace as logits matches learned routers and balances experts better, with zero routing parameters.","key_machinery":"Self-Routing: a parameter-free mechanism that reuses a fixed designated subspace of the token hidden state as the vector of expert logits, eliminating the router projection while leaving expert computation and residual pathways unchanged.","core_discovery":"Effective MoE routing can emerge from the hidden representation: a designated subspace of each token’s hidden state used directly as expert logits remains competitive with a learned-router baseline, removes all dedicated routing parameters, and produces more balanced expert utilization without a load-balancing loss.","pith_inferences":["If a fixed subspace already carries expert signal, specialization may be better studied as a property of the residual stream than of the router weights.","Subspace size becomes a free dial: larger subspaces may improve routing fidelity at the cost of representation capacity, a tradeoff that can be ablated directly.","The same readout idea may apply to other sparse conditional modules (e.g., selecting attention heads or memory banks) where a state subspace is reused as a selection score.","Removing the router matmul could reduce memory bandwidth and latency in large-scale MoE serving."],"forward_implications":["MoE layers can drop every dedicated routing parameter while remaining competitive with learned routers.","Expert utilization becomes more balanced (roughly 17% higher average normalized routing entropy) without an explicit load-balancing loss.","On ImageNet-1K with DeiT-S/16, Self-Routing slightly improves over the corresponding learned-router MoE.","Routing can be treated as a readout from the residual stream rather than a separately parameterized map."],"fun_headline_variants":["Hidden states route MoE experts with zero parameters","Self-Routing: subspace logits replace learned routers","No router needed: token states select MoE experts","Parameter-free MoE routing from hidden subspaces","Experts picked by hidden-state subspace, no router"],"cache_read_input_tokens":128,"weakest_assumption_plain":"A fixed designated subspace of the token hidden state already carries enough expert-discriminative signal to serve as logits, and using those dimensions for routing does not fatally harm the other representational work those dimensions would otherwise do.","fun_headline_variants_meta":{"raw":{"variants":["Hidden states route MoE experts with zero parameters","Self-Routing: subspace logits replace learned routers","No router needed: token states select MoE experts","Parameter-free MoE routing from hidden subspaces","Experts picked by hidden-state subspace, no router"]},"model":"grok-4.5","effort":"low","cost_usd":0.003348,"raw_usage":{"total_tokens":1118,"prompt_tokens":744,"num_sources_used":0,"completion_tokens":57,"cost_in_usd_ticks":33480000,"prompt_tokens_details":{"text_tokens":744,"audio_tokens":0,"image_tokens":0,"cached_tokens":256},"completion_tokens_details":{"audio_tokens":0,"reasoning_tokens":317,"accepted_prediction_tokens":0,"rejected_prediction_tokens":0}},"tokens_in":744,"tokens_out":57,"duration_ms":73105,"temperature":1.0,"reasoning_tokens":317,"cache_read_input_tokens":256,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-07-13T15:08:50.692517+00:00","model_set":{"reader":"grok-4.5"},"falsifier":"Train matched Self-Routing and learned-router MoE models at the same scale, expert count, and compute; if Self-Routing shows clearly worse language-model perplexity or ImageNet top-1 accuracy, or collapses into substantially more unbalanced expert loads than the baseline, the central claim fails.","supporting_citations":[],"review_version":1}