{"id":"49fce562-63ed-46d7-a3e8-8aa313594137","arxiv_id":"2509.07373","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":3,"one_line_summary":"SBS improves NeRN weight compression by unidirectional kernel smoothing and RFF bandwidth that shrinks for larger networks, cutting required MLP parameters by roughly 2 to 3 times.","lead":"Researchers propose SBS, a method that makes it cheaper to store a trained neural network by describing its weights with a smaller helper network. SBS reorders the big network's kernels to be smooth and tunes the helper's input encoding, reporting 2 to 3 times better compression than the prior NeRN approach on CIFAR and ImageNet.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"UOS-RFF bandwidth rule is confounded: Table 5 shows ResNet20 optimum at σ=1000 while main experiments fix σ=400, so the rule is not validated and CIFAR gains are at a suboptimal setting.","rationale":"The empirical ImageNet result is plausible and UOS alone is supported by Table 4, so I am not disputing the core efficiency direction. The load-bearing weakness is the UOS-RFF design rule, which the paper presents as a generalizable principle and uses to select the bandwidth for every main experiment. The reader identified this as the weakest assumption; I agree and sharpen it with an internal inconsistency: Table 5 shows ResNet20 accuracy still improving at σ=1000, yet the CIFAR tables use σ=400. This means the paper's own ablation does not support the bandwidth used in the headline CIFAR results, and the inverse-bandwidth rule is inferred from a two-point comparison that is fully confounded. A fixed-dataset, architecture-width sweep would settle whether the rule is real or whether the observed behavior is per-dataset tuning. If the rule fails, the SBS contribution is weakened but not necessarily falsified: UOS and RFF separately still appear to help. Therefore the reader's CONDITIONAL verdict remains appropriate; no verdict change is needed.","tokens_in":12157,"tokens_out":8805,"duration_ms":96098,"concrete_test":"Run a single controlled experiment on CIFAR-10: train UOS+RFF for ResNet20 (width 16) and a same-depth wider variant (e.g., ResNet20 with doubled stage widths) using identical training budgets, sweeping σ ∈ {50,100,200,400,800,1600,3200}. Record accuracy at the best σ and at σ=400 for both. If the optimal σ for the wider net is not strictly smaller (or if either optimum falls at the grid edge), the UOS-RFF 'reduce σ with width' rule is not supported. This isolates width from the dataset/architecture confound in Table 5's ResNet20/CIFAR-10 vs ResNet18/ImageNet comparison. Also report whether σ=1000 beats σ=400 for ResNet20, resolving the Table 5 contradiction.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central parameter-efficiency claim (Sec. 5.3: 'reaches the performance of NeRN at 46% parameter usage with only about 15% of the MLP parameters') depends on SBS performing well with a chosen RFF bandwidth. The UOS-RFF design rule in Sec. 4.3 — 'for wider CNNs, where layer-wise parameter counts grow, you should reduce RFF bandwidth (σ2 down)' — is the stated basis for setting σ=400 on CIFAR and σ=100 on ImageNet. But the only evidence for this rule is Fig. 4(c)/Table 5, a two-point comparison that changes architecture and dataset simultaneously (ResNet20/CIFAR-10 vs ResNet18/ImageNet). The inverse relationship is therefore confounded by dataset difficulty, input resolution, and training distribution; it is not established as a causal layer-count law. Moreover, Table 5 is internally inconsistent with the paper's own CIFAR-10 configuration: for ResNet20, accuracy improves monotonically from 91.05 at σ=10 to 91.44 at σ=1000, with no peak reached, yet the main CIFAR experiments fix σ=400. On the paper's own grid, σ=400 is not optimal, so the reported CIFAR-10 and CIFAR-100 efficiency gains are not at SBS's best configuration. If optimal bandwidth is governed by dataset/architecture rather than layer-wise parameter counts, the second contribution reduces to per-dataset RFF tuning, and the claimed UOS-RFF advantage is not validated.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes SBS, a parameter-efficiency enhancement to NeRN (neural representation for neural networks). SBS augments a coordinate-based ReLU-MLP that generates CNN weights with two mechanisms: (1) Unidirectional Ordering-based Smoothing (UOS), a greedy permutation that reorders CNN kernels along one direction using Euclidean distance to increase low-frequency energy in the output space; and (2) UOS-RFF, a Random Fourier Feature input encoding whose bandwidth σ² is reduced for wider CNNs, based on the authors' observation that larger models exhibit stronger low-frequency bias under UOS. The method is evaluated on ResNet20/ResNet56 for CIFAR-10, ResNet56 for CIFAR-100, and ResNet18 for ImageNet, comparing against NeRN and a stronger-teacher variant. The central claim is a 2–3× improvement in parameter efficiency, e.g., reaching NeRN's 46%-parameter-usage accuracy with about 15% of the MLP parameters on ImageNet.","tokens_in":12519,"tokens_out":6007,"duration_ms":70342,"significance":"If the empirical claims hold, the paper makes a useful practical contribution: it improves NeRN's parameter efficiency without external supervision, with consistent gains across CIFAR-10, CIFAR-100, and ImageNet. The experimental tables are the main strength: SBS consistently outperforms NeRN at the same MLP hidden sizes, the ImageNet result (Table 3) is competitive even against a stronger-teacher variant, and the UOS ablation (Table 4) supports the ordering strategy. However, the paper's theoretical scaffolding is substantially weaker than claimed. Equations (5) and (7) are asserted without derivations, the UOS-RFF bandwidth rule rests on a confounded two-point comparison, and the claim of a proof for RFF's advantage is unsupported. These gaps do not invalidate the empirical comparison, but they prevent the paper from supporting its advertised principled design rules.","major_comments":[{"comment":"The UOS-RFF design rule ('for wider CNNs, reduce RFF bandwidth σ²') is supported only by two configurations that change architecture, dataset, and input resolution simultaneously (ResNet20/CIFAR-10 vs ResNet18/ImageNet), so the inverse relationship is confounded. Moreover, Table 5 is internally inconsistent with the main CIFAR setup: for ResNet20, accuracy increases monotonically from 91.05 at σ=10 to 91.44 at σ=1000, with no peak reached, yet the main CIFAR experiments fix σ=400. On the paper's own grid, σ=400 is suboptimal. The reported CIFAR gains are therefore not at SBS's best configuration, and the claimed adaptivity of UOS-RFF is not validated as a causal layer-count law. Controlled experiments (e.g., same dataset/architecture with varying layer widths) are needed.","section":"§4.3, Table 5, Fig. 4(c)"},{"comment":"The bound in Eq. (5) is stated without derivation and with several undefined quantities. What exactly are Δf_i, n_k, δ, and ε, and how does the first inequality follow from the linear-region argument? The step from this bound to the regularization term S(w) in Eq. (6) is the theoretical basis for UOS, but as written Eq. (5) is not checkable. Please provide a complete derivation or explicitly label the statement as heuristic motivation.","section":"§4.1, Eq. (5)"},{"comment":"The claim that unidirectional smoothing yields faster convergence than multi-directional smoothing is asserted as Eq. (7) without derivation. The coefficients c1, c2, c3 are not defined, and the approximation ‖ft(X)-Y‖² ≈ |c1| e^{-ηλ1t} does not follow from Eqs. (2) and (4) as written: replacing the full eigenvalue sum by a single term requires an error bound. Since UOS is one of the paper's two main contributions, this step needs either a rigorous justification or an explicit statement that it is an empirical observation.","section":"§4.2, Eq. (7)"},{"comment":"The RFF kernel approximation contains a factor error. With B∼N(0,σ²I) and φ(x)=[cos(πBx), sin(πBx)], the expected inner product is exp(-π²σ²‖x-y‖²/2), not exp(-σ²‖x-y‖²/2) as written in Eq. (12). Additionally, the sentence in §4.3 claiming RFF's advantage is 'theoretically proven' is not backed by any theorem or proof in the paper; please correct the formula and soften or substantiate the claim.","section":"§4.3, Eqs. (10)–(12)"}],"minor_comments":[{"comment":"Typos: 'Achitecture' and 'Smothness' in the table headers. Also, the asterisk in Table 3 is not explained in the caption.","section":"Tables 1–2"},{"comment":"The phrase 'reaches the performance of NeRN at 46% parameter usage with only about 15% of the MLP parameters' is imprecise: in Table 3, SBS at 15% gives 68.57 while NeRN at 46% gives 68.74, a 0.17 accuracy gap. State the tolerance or compare within variance.","section":"§5.3"},{"comment":"Training details are missing: values of α and β in the loss, learning-rate schedule, number of epochs, RFF feature dimension, number of random seeds, and standard deviations. These are needed to assess whether reported differences of 0.1–0.3% are significant.","section":"§5"},{"comment":"The caption of Fig. 4 appears garbled in the reviewed version (e.g., '/uni00000013/...' tokens). Please ensure the final figure is legible and the caption renders correctly.","section":"Fig. 4"}],"recommendation":"major_revision","confidential_remarks":"The empirical core of the paper is defensible and likely of interest to the journal's audience; the SBS-vs-NeRN comparison is not circular, and the parameter-efficiency gains are consistently in the same direction. However, the theoretical sections need substantial work: Eq. (5), Eq. (7), and the RFF kernel formula are not correct/complete as written, and the UOS-RFF bandwidth rule is confounded. I would not reject the paper, but the authors should either supply rigorous derivations or recast the claims as heuristics, and they should add controlled experiments for the bandwidth rule. The current version overstates the theoretical support."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"The empirical core is believable; the theory around it is not. SBS combines a unidirectional greedy permutation of CNN kernels (UOS, Euclidean distance) with an RFF input encoding whose bandwidth is scaled down for wider layers. That specific combination is new relative to NeRN, Choi et al., and Tancik et al., and the tables consistently show SBS beating NeRN at the same MLP size and matching NeRN's accuracy at roughly half to a third of the MLP parameters. I think the 2–3x parameter-efficiency claim is numerically supported for the tested ResNets.\n\nWhere it gets soft is the explanatory apparatus. Eq. (5) is garbled and not derivable as written; Eq. (7) is asserted, not derived; and the loss weights alpha/beta in Fig. 1 never appear in the experiments. The paper reads as a heuristic that works, wrapped in NTK language it doesn't fully earn. That doesn't kill the empirical contribution, but it needs cleanup.\n\nThe stress-test about sigma is worth taking seriously but is subtler than it looks. Table 5 does show ResNet20 accuracy still increasing at sigma=1000 (91.44 vs 91.25 at sigma=100), and the main CIFAR experiments fix sigma=400. That means the headline CIFAR numbers are not at SBS's best configuration; if anything, the gains would be slightly larger with sigma=1000. So the main efficiency claim is not inflated. What is weakened is the UOS-RFF design rule: the evidence for 'reduce sigma as layer count grows' is a two-point comparison that changes dataset and architecture at once, and the ResNet20 side of the grid never reaches a peak. That rule should be treated as a per-dataset tuning heuristic, not a validated law. Also, there is no code, no error bars, and the alpha/beta omissions make the training objective under-specified.\n\nWho gets value: researchers working on INR-based weight compression or weight-generating hardware. For them the empirical comparison is useful even without the theory. It deserves a serious referee — ideally one who will push for derivations for Eqs. (5) and (7), a proper sigma ablation that includes the CIFAR-10 optimum, and code release. Desk rejection would be premature.","headline":"A plausible incremental win over NeRN with a garbled theory section and a sigma-selection rule that is weaker than the authors claim.","tokens_in":13014,"tokens_out":3165,"would_cite":false,"duration_ms":34813,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"A small MLP can regenerate a pretrained CNN's weights with 2–3x better parameter efficiency if its spectral bias is suppressed in both output and input space.","keywords":["Implicit Neural Representation","Spectral Bias","Weight Generation","Parameter Efficiency","Random Fourier Features","Neural Network Compression","ResNet"],"falsifier":"Take a CNN with larger layers than ResNet18 (for instance, ResNet50) on ImageNet, fix UOS and the SBS training objective, and sweep the RFF bandwidth sigma. The paper's rule predicts the optimal sigma should drop below the value used for ResNet18 (100); finding that the best sigma is set by dataset or architecture in a way uncorrelated with layer count would undercut the UOS-RFF contribution.","tokens_in":12039,"feed_emoji":"🧠","tokens_out":9724,"duration_ms":97123,"temperature":0.7,"pith_summary":"This paper claims that the standard recipe for compressing a pretrained CNN — train a small coordinate-based MLP to output its weights — is held back by spectral bias: ReLU networks learn low-frequency structure early and smear high-frequency detail. The authors propose two complementary fixes: reorder each layer's kernels along one direction so the weight signal becomes smoother and lower-frequency, and encode the MLP's input coordinates with random Fourier features whose frequency bandwidth shrinks for wider layers. On CIFAR-10, CIFAR-100, and ImageNet with ResNet backbones, the combined method reconstructs higher-accuracy networks from smaller MLPs than the NeRN baseline, and on ImageNet it matches NeRN's accuracy at about a third of NeRN's MLP parameters while using no external teacher. If correct, this makes weight-generation compression noticeably more practical for memory-limited deployment.","feed_headline":"Spectral-bias fix cuts MLP size for CNN reconstruction by 2–3x","feed_subtitle":"No extra teacher: reordering kernels and tuning input frequency lets a small MLP match a 3x larger MLP's reconstruction.","key_machinery":"The two load-bearing mechanisms are the UOS permutation and the UOS-RFF encoding. UOS is a greedy reordering of CNN kernels within each layer along a single axis, the filter index, minimizing Euclidean distance between neighboring kernels; this reshapes the target weight signal so its energy sits in the low-frequency eigenmodes that a ReLU-MLP learns fastest, as measured by the neural tangent kernel (NTK), the Gram matrix governing frequency-wise convergence under gradient descent. UOS-RFF is a random Fourier feature input encoding whose Gaussian bandwidth sigma is reduced as layer-wise parameter count grows, preventing the input coordinates from collapsing into too narrow a frequency band a","core_discovery":"The paper's claim is that the limiting factor in NeRN-style MLPs that generate CNN weights is spectral bias: the ReLU-MLP acts as a low-pass filter, so it reconstructs smooth, low-frequency weight arrangements well but loses high-frequency kernel detail. SBS attacks this on both sides of the MLP. In output space, UOS greedily reorders kernels within each layer along one direction (filter-wise) using Euclidean distance, concentrating energy at low frequencies so the target signal is easier for the MLP to fit. In input space, UOS-RFF maps coordinates through random Fourier features and sets the bandwidth sigma inversely to layer-wise parameter count, rebalancing the frequency content the MLP s","pith_inferences":["A natural extension is per-layer or per-block bandwidth selection instead of one global sigma; the paper's inverse-relationship rule suggests such adaptivity could remove the remaining hyperparameter tuning.","Because UOS concentrates kernel energy at low frequencies, the reconstructed weights may also compress better under standard quantization or sparsification downstream — the paper does not test this.","The unidirectional-over-multidirectional insight may transfer to other INR tasks where the target signal has a natural axis ordering (e.g., video frames along time, volumetric slices), not just CNN kernels.","The paper's comparison against a distillation-augmented baseline suggests a fairer protocol for weight-space INRs: measure gains over unaugmented baselines before importing external supervision."],"forward_implications":["NeRN-style weight reconstruction can reach a given accuracy with roughly one-third to one-half of the MLP parameters previously needed, making weight-representation compression more practical for mobile and edge deployment.","The same ReLU-MLP backbone no longer needs a stronger teacher or extra distillation signal: at equal MLP size, SBS outperforms the stronger-teacher variant while using no external supervision.","The bandwidth rule makes the method transfer across CNN widths: wider layers automatically receive smaller RFF bandwidth, reducing per-dataset frequency hyperparameter search.","The gains grow as the MLP shrinks, so the method is most effective exactly in the low-parameter regime where compression matters."],"supporting_citations":[{"why":"Defines the NeRN task and the coordinate-based ReLU-MLP baseline, and supplies the original permutation smoothing that SBS extends and compares against.","marker":"[2]"},{"why":"Provides the spectral-bias/low-pass characterization of standard neural networks that motivates both the output-space and input-space fixes.","marker":"[28]"},{"why":"Together they supply the frequency-dependent convergence-rate analysis of MLPs used to derive the low-pass behavior in the NTK picture.","marker":"[4,1]"},{"why":"Establishes that random Fourier features beat fixed positional encodings for learning high-frequency functions, the basis for UOS-RFF.","marker":"[32]"},{"why":"Provides the stronger-teacher distillation baseline whose ImageNet performance SBS matches or exceeds without a teacher.","marker":"[9]"},{"why":"Supplies the ResNet architectures used as the CNN weight-representation targets in all experiments.","marker":"[15]"}],"fun_headline_variants":["Suppress spectral bias to shrink MLP for CNN weights","Reordering kernels boosts MLP efficiency for CNN generation","Smaller MLPs for CNN weights via spectral-bias suppression","SBS: fewer parameters, better CNN weight reconstruction","Frequency-aware encoding lets MLPs mimic bigger ones"],"cache_read_input_tokens":2688,"weakest_assumption_plain":"The method's central premise is that the ideal frequency bandwidth of the input encoding depends mainly on how many parameters the CNN layer has, so the rule 'wider layers need smaller bandwidth' carries over to new networks rather than being a per-dataset hyperparameter choice.","fun_headline_variants_meta":{"raw":{"variants":["Suppress spectral bias to shrink MLP for CNN weights","Reordering kernels boosts MLP efficiency for CNN generation","Smaller MLPs for CNN weights via spectral-bias suppression","SBS: fewer parameters, better CNN weight reconstruction","Frequency-aware encoding lets MLPs mimic bigger ones"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000155,"raw_usage":{"total_tokens":1028,"prompt_tokens":697,"completion_tokens":331,"prompt_tokens_details":{"cached_tokens":256},"prompt_cache_hit_tokens":256,"prompt_cache_miss_tokens":441,"completion_tokens_details":{"reasoning_tokens":253}},"tokens_in":441,"tokens_out":331,"duration_ms":4090,"temperature":1.0,"reasoning_tokens":253,"cache_read_input_tokens":256,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-04T22:17:44.507477+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Take a CNN with larger layers than ResNet18 (for instance, ResNet50) on ImageNet, fix UOS and the SBS training objective, and sweep the RFF bandwidth sigma. The paper's rule predicts the optimal sigma should drop below the value used for ResNet18 (100); finding that the best sigma is set by dataset or architecture in a way uncorrelated with layer count would undercut the UOS-RFF contribution.","supporting_citations":[{"cited_title":"In: The Eleventh International Conference on Learning Representations (2023) 14 Q","cited_arxiv_id":null,"evidence_quote":"Defines the NeRN task and the coordinate-based ReLU-MLP baseline, and supplies the original permutation smoothing that SBS extends and compares against."},{"cited_title":"In: Proceedings of the 36th International Conference on Machine Learning","cited_arxiv_id":null,"evidence_quote":"Provides the spectral-bias/low-pass characterization of standard neural networks that motivates both the output-space and input-space fixes."},{"cited_title":"Advances in Neural Infor- mation Processing Systems33(2020)","cited_arxiv_id":null,"evidence_quote":"Establishes that random Fourier features beat fixed positional encodings for learning high-frequency functions, the basis for UOS-RFF."},{"cited_title":"Enhancing Accuracy and Parameter-Efficiency of Neural Representations for Network Parameterization","cited_arxiv_id":"2407.00356","evidence_quote":"Provides the stronger-teacher distillation baseline whose ImageNet performance SBS matches or exceeds without a teacher."}],"review_version":1}