{"id":"b0c34af8-0691-4944-ba26-8805a2b95b5b","arxiv_id":"2607.29463","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":6,"one_line_summary":"A hierarchical mixture-of-experts transformer that classifies images from the weights of implicit neural representations achieves state-of-the-art accuracy in weight-space learning.","lead":"The authors propose a mixture-of-experts transformer that classifies images by reading the weights of the neural networks that encode them, rather than the pixels themselves. It sets new accuracy records for weight-space classification on MNIST, CIFAR-10 and ImageNet-1K, and adds a tool to visualize which parts of those weight networks matter for each decision.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Large-scale SOTA claims rest on an unreplicated ImageNet run and an untabulated 20-block CIFAR-10 result; without error bars or configuration detail these numbers are not yet established.","rationale":"The reader's weakest_assumption focused on the lambda*(phi-theta) input scaling and the class-discriminative content of the delta. I think the delta assumption is empirically validated by the generalization results themselves; the classifier could not reach 90%+ on MNIST/Fashion-MNIST if the delta carried no class signal. The fixed lambda=500 is unlikely to be load-bearing because a standard Transformer includes LayerNorm, which normalizes each token across features; a global scalar scaling of all token features is largely absorbed by the normalization. The reader's rationale also flagged the single-run ImageNet result, which I agree is the more serious gap. I therefore choose the unreplicated large-scale numbers as the load-bearing concern, giving partial agreement with the reader. The architectural thesis (hierarchical MoE aligned with INR layer structure) is independently supported by the ablation in Table 5 and the parameter-matched comparison in Table 3, so I do not see a reason to move the verdict beyond CONDITIONAL. Adding multi-seed results for ImageNet and tabulating the 20-block CIFAR-10 configuration would be sufficient to lift the condition.","tokens_in":13945,"tokens_out":11289,"duration_ms":98511,"concrete_test":"Run HMoE-MWT-L on ImageNet-1K with the reported protocol for at least 3 random seeds (varying meta-initialization and data order) and report mean±std. If the lower confidence bound does not exceed MWT-L's 24.11%, the ImageNet SOTA claim is not supported. Separately, reproduce the 20-block + augmentation CIFAR-10 configuration and add a row to Table 1 with error bars; if the mean is not significantly above the 10-block model's 65.01%, retract or qualify the 69.11% claim.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim is that HMoE-MWT achieves state-of-the-art accuracy among weight-space classifiers across benchmarks. The two most scalable results in this claim are the ImageNet-1K accuracy of 26.73% (Table 4) and the 20-block CIFAR-10 accuracy of 69.11% mentioned only in Section 4.1. Table 4 reports a single run for each method, with no standard deviation, for a protocol using 1% coordinate subsampling and a reconstruction PSNR of only 22.06 dB. At this subsampling rate the INR is heavily underfit; the classification signal may be sensitive to the exact random seed and meta-initialization. A 2.6-point gap over MWT-L (24.11%) could be within run-to-run variance, especially since MWT-L itself is reported without error bars. The 69.11% CIFAR-10 number is not present in any table; the full configuration (exact augmentations, whether the 40-epoch schedule and loss weights are identical to the 10-block model) is not specified, so the gap over the 10-block model (65.01%) cannot be audited. The parameter-matched Imagenette results (Table 3) are more robust evidence for the architectural advantage, but the headline 'state-of-the-art across low- to high-resolution datasets' leans disproportionately on these two unverified numbers.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes a hierarchical Mixture-of-Experts (HMoE) Transformer for classifying implicit neural representations (INRs) of images. It extends the Meta Weight Transformer (MWT) framework by replacing the standard feed-forward blocks with a two-stage sparse MoE: a layer-wise stage that routes all tokens originating from a given INR layer to a selected expert, and a token-wise stage that performs fine-grained routing conditioned on the layer-level gating. The classifier operates on λ(ϕ−θ), the scaled difference between the per-image adapted INR parameters and the shared meta-initialization. The authors report state-of-the-art accuracy among weight-space classifiers on MNIST, Fashion-MNIST, CIFAR-10, Imagenette, and ImageNet-1K, including parameter-matched comparisons with MWT on Imagenette. They also introduce Grad-CAG, a gradient-weighted attribution score for INR weights, and use it for structured pruning and for generating spatial attribution maps, with a pointing-game evaluation. An ablation study on CIFAR-10 indicates that the combined hierarchical MoE outperforms either the layer-wise or token-wise stage alone.","tokens_in":14233,"tokens_out":7680,"duration_ms":65525,"significance":"If the empirical claims hold, this is a meaningful advance in weight-space learning: the hierarchical MoE is a natural inductive bias for the layer-structured weight space of INRs, and the paper provides credible evidence of architectural advantage via parameter-matched Imagenette comparisons against a strong dense baseline. The release of code, the reporting of means and standard deviations on multiple benchmarks, and the detailed ablations are strengths. The attribution and pruning analyses, including the pointing-game evaluation, are a useful first step toward interpretability in weight-space classification. However, the broad 'state-of-the-art across low- to high-resolution datasets' claim is stronger than the evidence: the ImageNet-1K result is a single unreplicated run, and the 20-block CIFAR-10 SOTA number is not tabulated or fully specified. The fixed input scaling λ=500 and its use in baselines also needs clarification for a fair comparison.","major_comments":[{"comment":"The ImageNet-1K result is reported as a single run (Table 4: HMoE-MWT-L 26.73% vs MWT-L 24.11%) with no standard deviation, at 0.01 coordinate subsampling and a reconstruction PSNR of 22.06 dB. Given the heavy subsampling and the 2.6-point gap, this number could be within run-to-run variance; the claim of state-of-the-art performance on high-resolution ImageNet-1K should be supported by multiple seeds or explicitly labeled as a preliminary single run.","section":"§4.1, Table 4"},{"comment":"The claimed 69.11% CIFAR-10 accuracy for the 20-block HMoE-MWT with augmentations is not tabulated, and its configuration (exact augmentations, loss weights, training schedule, number of runs) is not specified. Since this number is used to claim a new SOTA and to argue that architectural design beats model scaling, it must be reported with full settings and variance in a table, or removed from the headline claims.","section":"§4.1, paragraph following Table 1"},{"comment":"The classifier input is λ(ϕ−θ) with λ fixed at 500, but no ablation or sensitivity analysis is provided, and the paper does not state whether the MWT baselines also use this scaling. If MWT does not, the architecture comparisons in Tables 1–3 may partly reflect a difference in input normalization rather than the MoE design; please ablate λ and run a matched baseline with identical scaling.","section":"§3.2 and Supplementary 'Additional training details'"}],"minor_comments":[{"comment":"The default-configuration row appears to have concatenated numbers ('1 4 1 465.0131.71 11M'), making the token-expert count and the accuracy/PSNR values ambiguous; please reformat so each column is distinct.","section":"§4.3, Table 5"},{"comment":"The text says the first three rows of Table 5 correspond to replacing the layer-wise MoE with a standard token-wise MoE, but the first row has the layer-wise MoE active and token-wise disabled; the row grouping and the wording should be reconciled.","section":"§4.3"},{"comment":"The main-text ablation (Table 5) does not state that some ablations are trained on 80% of CIFAR-10; the supplementary caption says this. Please state the data split in the main text, since the default accuracy in Table 5 (65.01%) differs from the 80%-split default (63.14%) in the supplementary.","section":"§4.3 and Supplementary Table 1"},{"comment":"Table 1 compares HMoE-WT (11M classifier) with WT (1.1M classifier) without parameter matching; the claim that HMoE-WT 'substantially outperforms' WT should point to the parameter-matched results in Table 3 to avoid overstating the comparison.","section":"§4.1, Table 1"}],"recommendation":"major_revision","confidential_remarks":"The two issues that drive my major_revision are verifiable and fixable: replicate the ImageNet run (or label it preliminary) and put the 20-block CIFAR-10 result in a table with full configuration. The λ scaling question is also important for a fair comparison with MWT, since it could affect the interpretation of the architectural advantage."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Worth a look if you follow weight-space learning. The core idea is a two-stage hierarchical Mixture-of-Experts transformer that routes INR weight tokens first by layer, then by token, trained end-to-end with a meta-learned SIREN. It works: on MNIST, Fashion-MNIST, CIFAR-10 and Imagenette it beats the previous MWT baselines, and the parameter-matched comparison on Imagenette (61.47 vs 56.78 at 1.1M; 62.52 vs 59.92 at 11M) shows the architecture itself buys the gains, not just extra parameters. The ablation on CIFAR-10 also supports the layer-wise routing component. The paper also introduces Grad-CAG, a weight-space attribution method, and shows it can localize objects better than magnitude or random pruning. That is honest, useful work.\n\nSoft spots are mostly about the headline claims at scale. The ImageNet-1K number (26.73%) is a single run with no error bar, at 1% coordinate subsampling with a PSNR of only 22.06 dB; the 2.6-point gap over MWT-L could easily be seed noise. Similarly, the 69.11% CIFAR-10 result with 20 blocks and augmentations appears only in a sentence, with no table and no configuration details, so it cannot be audited. The input scaling lambda=500 is never ablated, which matters because the classifier sees lambda(phi-theta). The code link is given but no commit hash or released data, so reproducibility is not yet verifiable. The ablation table has a garbled row (the default HMoE-MWT row appears to be mangled), and the ablations use an 80/20 split of CIFAR-10, so they are not directly comparable to the main numbers. The pointing-game score of 0.35 is modest, though better than the baselines.\n\nNone of these are fatal. The core contribution is the hierarchical MoE, and the evidence for that is solid on the mid-scale benchmarks. The fix is straightforward: add error bars or at least repeated runs for ImageNet, put the 20-block CIFAR-10 result in a table with full config, ablate lambda, and clean up the code release. This deserves serious peer review with those revisions.","headline":"Solid empirical advance in INR weight-space classification with a well-motivated hierarchical MoE, but the large-scale SOTA claims rest on a couple of unreplicated results.","tokens_in":14781,"tokens_out":3255,"would_cite":true,"duration_ms":26993,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Mixture-of-experts transformer reads labels hidden in network weights","keywords":["implicit neural representations","weight-space learning","mixture-of-experts","meta-learning","image classification","SIREN","explainability","network pruning"],"falsifier":"Train the full pipeline but set the classification weight in the meta-learning objective to zero, so the INR weight space is not shaped by the classifier; if HMoE accuracy stays at the reported level, the claim that meta-shaping the weight distribution is load-bearing would be falsified. A related check is to sweep the scaling factor \\(\\$\\lambda$\\) over orders of magnitude and see whether accuracy collapses away from 500.","tokens_in":13706,"feed_emoji":"🎯","tokens_out":8027,"duration_ms":70698,"temperature":0.7,"pith_summary":"Implicit neural representations (INRs) turn each image into a small coordinate-based network, so image classification can be done on network weights instead of pixels. This paper argues that the right way to read those weights is with a hierarchical mixture-of-experts transformer: first route whole INR layers to specialists, then route individual neurons to specialists conditioned on the layer choice. Trained jointly with a meta-learned SIREN initializer, the scheme reaches reported accuracies of 99.06% on MNIST, 65.01% on CIFAR-10, 62.96% on Imagenette, and 26.73% on ImageNet-1K, ahead of earlier weight-space classifiers. The same architecture comes with a gradient-based attribution method, Grad-CAG, and a pruning analysis intended to show that class-relevant structure concentrates in INR subcircuits. A sympathetic reader would take the paper's core contribution to be the demonstration that conditional, layer-aware computation rather than a larger dense transformer is what makes INR weight-space classification scale.","feed_headline":"Mixture-of-experts transformer reads labels hidden in network weights","feed_subtitle":"Meta-learned neural weight deltas plus sparse experts beat previous weight-space classifiers and reveal where class signal lives.","key_machinery":"The load-bearing object is the HMoE block: a two-stage sparse mixture of experts that replaces the ordinary feed-forward network in each Transformer layer. Stage one pools the tokens belonging to each INR layer, produces a layer-level routing vector via a softmax over expert scores, and processes the whole layer's tokens with the selected top-k experts; stage two routes each individual token using an attention-like score that combines the token's own projection with a learned projection of the stage-one layer routing vector. Around this sits an input transform the paper fixes at \\(\\$\\lambda$(\\phi-\\$\\theta$)\\) with \\(\\$\\lambda$=500\\), and a meta-learning loop (an inner reconstruction fit, an outer objective mixing reconstruction, classification, and expert-balancing losses) that updates the shared initializer \\(\\$\\theta$\\) and the per-step learning-rate schedule. Grad-CAG, computed as \\(|\\phi_j \\, \\partial c_i / \\partial \\phi_j|\\) per weight, provides the attribution scores used for pruning and spatial explanation.","core_discovery":"On its own terms, the paper claims that the learned weight deltas \\(\\$\\lambda$(\\phi-\\$\\theta$)\\) produced by a short inner-loop fit of a shared SIREN contain enough class-discriminative structure for a Transformer classifier, and that a hierarchical Mixture-of-Experts block extracts that structure better than a standard block. The block splits the weight token stream by INR layer, sends each whole layer to a top-1 expert, then lets individual tokens choose experts using the layer-level routing as a conditioning signal. Because the cross-entropy loss backpropagates through the classifier into the meta-initialization and per-step learning rates, the INR weight space itself is shaped so that class differences become readable. Evidence in its support includes gains over both dense and parameter-matched baselines, an ablation in which removing the layer-wise stage lowers accuracy, and routing statistics showing stable layer-expert specialization without collapse.","pith_inferences":["Because the layer-wise experts specialize by INR layer, a dense transformer with layer-specific positional embeddings may recover part of the HMoE gain, which would isolate the contribution of routing from the contribution of layer information.","The fixed \\(\\lambda=500\\) scaling of \\(\\phi-\\theta\\) is an untested hyperparameter; a calibration sweep across a range of meta-initialization scales could improve robustness at ImageNet-1K resolution.","Grad-CAG pruning that preserves coarse object identity could be repurposed as a no-pixel-access audit tool for detecting dataset biases, since the tench-fishing-net error appears as a spatial attribution.","Applying the same HMoE tokenization to other INR families such as audio wavefields or 3D NeRFs is stated as future work, but the layer-frequency correspondence motivating the design suggests the gain should transfer if those INRs also show layer specialization."],"forward_implications":["With 10 blocks and sparse top-1 routing, HMoE-MWT reports higher accuracy than the earlier 20-block dense model on CIFAR-10, so architectural structure rather than model width is the primary lever.","HMoE-MWT beats the parameter-matched dense baseline: at 1.1M classifier parameters it reports 61.47% on Imagenette versus 56.78%, and at 11M parameters it reports 62.52% versus 59.92%, indicating the gain is not just from extra capacity.","Replacing the layer-wise stage with a token-wise MoE drops accuracy, so grouping tokens by INR layer is a functional part of the method.","Grad-CAG attribution maps score 0.35 on a 200-mask pointing-game test, versus 0.22 for magnitude pruning and 0.15 for random, indicating class-relevant weights have a spatial signature that can be localized.","The framework extends to high resolution with 26.73% top-1 on ImageNet-1K, suggesting weight-space learning can operate beyond toy-scale domains."],"supporting_citations":[{"why":"Defines the end-to-end meta-learning plus weight Transformer framework that this paper extends, and supplies the MWT and MWT-L baselines used throughout.","marker":"[14]"},{"why":"Supplies the MAML/MetaSGD inner-loop and per-parameter learning-rate schedule used to produce the adapted weights phi.","marker":"[9]"},{"why":"SIREN is the sinusoidal implicit network whose weights become the classifier tokens.","marker":"[40]"},{"why":"Provides the sparse top-k mixture-of-experts routing and load-balancing ideas the paper adapts to weight space.","marker":"[37]"},{"why":"Grad-CAM is the formula that Grad-CAG generalizes to weight-space attribution.","marker":"[36]"},{"why":"ScaleGMN is a symmetry-aware weight-space baseline this method compares against on the low-resolution benchmarks.","marker":"[21]"},{"why":"Permutation-equivariant neural functionals provide a representative equivariant weight-space classifier that the proposed method aims to outperform.","marker":"[45]"}],"fun_headline_variants":["Hierarchical MoE transformer decodes class info from INR weights","Meta-learned weight deltas enable MoE classification of neural fields","Layer-wise expert routing reveals where class info lives in INRs","Weight-space MoE beats dense baselines on INR classification","Sparse experts extract discriminative signals from INR weight space"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that the per-image change in fitted weights from the shared initialization, after one fixed scaling of 500, reliably separates classes and does not discard the information the classifier needs.","fun_headline_variants_meta":{"raw":{"variants":["Hierarchical MoE transformer decodes class info from INR weights","Meta-learned weight deltas enable MoE classification of neural fields","Layer-wise expert routing reveals where class info lives in INRs","Weight-space MoE beats dense baselines on INR classification","Sparse experts extract discriminative signals from INR weight space"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000338,"raw_usage":{"total_tokens":1846,"prompt_tokens":902,"completion_tokens":944,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":518,"completion_tokens_details":{"reasoning_tokens":859}},"tokens_in":518,"tokens_out":944,"duration_ms":8741,"temperature":1.0,"reasoning_tokens":859,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-15T15:21:47.802401+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Train the full pipeline but set the classification weight in the meta-learning objective to zero, so the INR weight space is not shaped by the classifier; if HMoE accuracy stays at the reported level, the claim that meta-shaping the weight distribution is load-bearing would be falsified. A related check is to sweep the scaling factor \\(\\$\\lambda$\\) over orders of magnitude and see whether accuracy collapses away from 500.","supporting_citations":[{"cited_title":"In: Proceedings of the Computer Vision and Pattern Recognition Confer- ence","cited_arxiv_id":null,"evidence_quote":"Defines the end-to-end meta-learning plus weight Transformer framework that this paper extends, and supplies the MWT and MWT-L baselines used throughout."},{"cited_title":"In: International conference on machine learning","cited_arxiv_id":null,"evidence_quote":"Supplies the MAML/MetaSGD inner-loop and per-parameter learning-rate schedule used to produce the adapted weights phi."},{"cited_title":"Advances in neural information processing systems36(2024)","cited_arxiv_id":null,"evidence_quote":"Permutation-equivariant neural functionals provide a representative equivariant weight-space classifier that the proposed method aims to outperform."}],"review_version":2}