{"id":"8406923c-ae6e-4750-84ff-bc2cbfc7aeec","arxiv_id":"1908.08932","paper_version":2,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":4,"one_line_summary":"A split-wise filter basis decomposition unifies earlier low-rank approximations, derives an optimal split point, and compresses VGG, ResNet, DenseNet, SRResNet, and EDSR with modest accuracy loss.","lead":"This paper compresses deep convolutional networks by learning a small set of shared basis filters and representing each filter split as a linear combination of that basis. The method is tested on image classification and super-resolution networks and is claimed to outperform earlier filter-decomposition compression schemes.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The SOTA claim rests on parameter counts measured per residual block, not per full network; the reported compression ratios and EDSR-to-SRResNet size comparison do not reflect actual model sizes.","rationale":"I read the paper's central claim as an empirical state-of-the-art claim: the method is said to 'outperform state-of-the-art compression method gracefully with lower classification error and fewer parameters' and to bring 'model size from EDSR level to SRResNet level.' The theoretical derivation in Eq. (12)-(13) supports the parameter-count algebra for a fixed basis size m, and the reader's concern that m may not be independent of the split width p is a legitimate secondary issue. However, the most load-bearing weakness is that the experimental tables measure only one residual block while the surrounding text makes network-level claims. The paper does disclose the per-block measurement in one sentence, so the flaw is presentation rather than hidden manipulation, but it directly undermines the numerical magnitude of the central claim. If full-model recomputation still shows a favorable comparison to Factor and Group, the conditional verdict would remain appropriate; if the ratios shift substantially, the state-of-the-art claim needs qualification. The concrete test above would settle this. I therefore do not change the reader's CONDITIONAL verdict, though I differ from the reader's stated weakest assumption about Eq. (13) and instead point to the network-level parameter accounting as the primary load-bearing concern.","tokens_in":15828,"tokens_out":8223,"duration_ms":79912,"concrete_test":"Recompute full-network parameter counts for EDSR, EDSR-8-128, and SRResNet from the released code and configuration tables (Tables 7-9), including every uncompressed layer (head, tail, residual blocks, upsampling). Re-express each reported parameter count and compression ratio in Tables 1-3 as a fraction of the full baseline model. If any reported ratio changes by more than a factor of two, the headline compression claims must be restated at network level.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central empirical claim is that the method compresses full networks to state-of-the-art parameter counts. Section 5.1 states 'we report the number of parameters and compression ratio for one residual block since all of the other blocks has the same parameter. And we keep to this setting in Table 2 and Table 3.' Thus Tables 1-3 report per-residual-block parameters: the 1180k baseline in Tables 1 and 3 is one EDSR residual block, and the 74k/295k baselines in Table 2 are one SRResNet/EDSR-8-128 residual block. Yet the text uses these numbers to claim network-level effects: 'this operating point brings the model size from EDSR level to SRResNet level.' A compressed residual block may be small, but EDSR also contains many residual blocks plus uncompressed head/tail/upsampling layers and other convolutions. For example, at the most aggressive point, if one residual block drops from 1180k to 17k, 32 residual blocks contribute roughly 0.54M, but the full EDSR baseline is about 43M; after adding the unchanged layers, the full compressed model is far from SRResNet's 1.5M. Therefore the reported 'Comp. (%)' values of 1.5% and 13.9% are residual-block compression ratios, not network-level compression ratios, and the central 'fewer parameters' / 'model size from EDSR level to SRResNet level' claim is not established as stated.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes a filter-basis decomposition method for convolutional neural network compression. It splits the 3D filters of a convolution layer along the input channel dimension and represents the resulting p×w×h filter splits as linear combinations of a learned basis set. The authors derive compression-rate formulas for whole-filter, channel-wise, and split-wise decompositions, obtain a closed-form split choice that minimizes the parameter count for a fixed basis size, and show that the decomposition can be implemented as a shared-weight convolution followed by a 1×1 convolution. Experiments are reported on CIFAR-10 image classification (VGG-16, ResNet-56, DenseNet-12-40) and on image super-resolution (SRResNet, EDSR-8-128, EDSR), with comparisons to several filter-decomposition baselines. The paper claims state-of-the-art compression performance in terms of parameter reduction and accuracy preservation.","tokens_in":16095,"tokens_out":16474,"duration_ms":140163,"significance":"The split-wise formulation is a natural generalization that interpolates between whole-filter and channel-wise filter decomposition, and the convolution-equivalence derivation in Section 3.3 is sound. If the experimental claims are confirmed, the method would be a flexible and practical compression tool, particularly for architectures with many 1×1 convolutions or narrow layers. Strengths of the paper include the transparent derivation of the parameter-count formulas (Eqs. 7–13), the practical implementation via shared-weight convolutions, and validation on external benchmarks with publicly released code. However, the central claim of network-level state-of-the-art compression currently rests on per-residual-block parameter accounting, which is not sufficient to establish the advertised model-size reductions. The significance as presented is therefore not yet fully established.","major_comments":[{"comment":"The paper states that 'we report the number of parameters and compression ratio for one residual block since all of the other blocks has the same parameter,' but the surrounding text uses these numbers for network-level conclusions: 'this operating point brings the model size from EDSR level to SRResNet level' and the abstract claims fewer parameters than prior compression baselines. Full-network parameter counts are not reported. EDSR has 32 residual blocks plus uncompressed head, tail, and upsampling convolutions; a per-block count of 17k at the most aggressive setting gives roughly 0.54M parameters for the blocks alone, and the unchanged layers still contribute several million. The reported 'Comp. (%)' values (e.g., 1.5%, 7.6%, 13.9%) are therefore residual-block compression ratios, not whole-network ratios. The 'fewer parameters' and 'EDSR-level to SRResNet-level' claims are not established as stated.","section":"Section 5.1, Tables 1-3"},{"comment":"The parameter counts in Table 1 appear inconsistent with a per-residual-block interpretation. For EDSR with c=n=256, w=h=3 and two splits, Eq. (12) gives 53,248 parameters for a single 3×3 convolution; a residual block of two such convolutions would have 106,496 parameters. Table 1 reports 53k for m=32 without sharing, which is the per-convolution number. The 'Basis Share' entry of 35k is close to the value obtained by amortizing a network-wide shared basis over 32 residual blocks, rather than by sharing only the two convolutions within a residual block as stated in the text. Please clarify the exact parameter accounting and report full-network parameter totals for the compressed models.","section":"Tables 1 and 3, Eq. (12)"},{"comment":"The optimization in Eq. (13) minimizes the parameter count for a fixed basis size m. The closed-form split s* is therefore a parameter-count optimum conditional on m, not an accuracy-parameter optimum. The paper does not analyze or measure how the required basis size m varies with the split width p; the experiments tune both m and p per architecture (Tables 7-9). Thus the claim that Eq. (13) identifies the optimal split configuration for compression is stronger than what is demonstrated. This is a limitation rather than an internal error, but it should be acknowledged and ideally tested with a controlled experiment varying s and m independently.","section":"Section 3.2, Eq. (13)"}],"minor_comments":[{"comment":"The caption says 'The number of splits p for one convolution is 4,' but in the main text s denotes the number of splits and p the split width; please harmonize the notation.","section":"Table 1 caption"},{"comment":"The sentence 'when m = 64 and basis sharing is used, the compressed model only accounts for 9% of the parameters' should read 'without basis sharing' to match the 9.0% entry; the basis-sharing entry for m=64 is 5.9%.","section":"Section 5.1, Table 1"},{"comment":"Equations (9) and (10) repeat Eqs. (7) and (8) verbatim; please remove the duplicate lines.","section":"Section 3.2, Eqs. (7)-(10)"},{"comment":"The ResNet-56 baseline error for the proposed method is listed as 7.05 in Table 6, while Table 5 uses 6.28 for the same network; the discrepancy should be explained.","section":"Table 6"},{"comment":"The statement 'the optimal group s* ≈ sqrt(w×h)' assumes c≈n and ignores the integrality constraints on s and p; a sentence explaining how non-divisible cases are handled in practice would be helpful.","section":"Section 3.2"}],"recommendation":"major_revision","confidential_remarks":"The parameter-accounting issues in Tables 1-3 are serious because the paper's headline claims are stated in network-level terms. I recommend asking the authors to provide full-network parameter counts, a precise description of what is included in each reported number, and a comparison to baselines at the same granularity. The code release should make this verification straightforward. If the corrected numbers still show a clear advantage, the paper could be a solid contribution."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Colleague,\n\nThe core idea here is worth your attention: split the 3D filters along the input channel dimension, learn a shared basis for the splits, and you get a decomposition that interpolates between whole-filter and channel-wise schemes. The compression-rate formula and the optimal split s* = sqrt(cwh/n) are derived cleanly, and the equivalence to a basis convolution followed by a 1x1 convolution is correct. The paper also generalizes to 1x1 convolutions and narrow networks, which earlier filter decomposition methods struggled with. That is a real step forward, and they ship code.\n\nThe experiments are where I part ways with the authors' framing. For the super-resolution tables, the parameter counts and compression ratios are measured per residual block, not for the full network. The text says as much in Section 5.1, yet the conclusion that the most aggressive operating point 'brings the model size from EDSR level to SRResNet level' only concerns the residual blocks. The rest of EDSR—head, tail, upsampling layers—is not compressed and not counted. So the headline model-size claim is not established. This is fixable reporting, but it means the SR SOTA claims should be taken with a grain of salt until full-network numbers appear.\n\nThe classification experiments look more solid: VGG, DenseNet, and ResNet results report full network parameters, and the method holds its own against KSE, Factor, and Group. There is a minor inconsistency in Table 5 where KSE's ResNet baseline differs from the others, so cross-method comparisons there are apples-to-oranges, but the authors' own operating points are clear.\n\nOne theoretical caveat: the optimal split derivation treats the basis size m as independent of the split width p. If the number of basis filters needed to hold accuracy grows as splits get narrower, the parameter-optimal point may not be the accuracy-optimal point. They tune m and p per network without testing this, so the optimality claim is conditional. Not fatal, but it deserves a sensitivity analysis.\n\nOverall: the method is sound and useful. The math is correct, the unification is real, and the classification evidence is credible. The SR claims need full-network reporting before I'd trust the SOTA headline. This paper deserves peer review; a referee should push for corrected tables and clearer claims.\n\nRecommendation: engage with it, but be aware of the residual-block reporting issue before quoting any SR compression numbers.","headline":"Split-wise filter basis is a real contribution, but the SR compression numbers only count residual blocks—so the 'EDSR to SRResNet' claim doesn't hold as stated.","tokens_in":16661,"tokens_out":3845,"would_cite":true,"duration_ms":33136,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Splitting filters along channels beats whole-filter CNN compression","keywords":["CNN compression","filter basis learning","split-wise filter decomposition","low-rank approximation","basis sharing","network compression","image super-resolution","image classification"],"falsifier":"Take a trained convolutional layer with fixed $c$, $n$, $w$, $h$; keep $m$ fixed, sweep $s$ over divisors of $c$, train each split-wise decomposition to convergence, and measure the resulting reconstruction error or end-to-end accuracy. If the $s$ that minimizes $\\Gamma_{\\text{split}}$ does not also give the best accuracy at a fixed parameter budget, or if narrow splits require a noticeably larger $m$ to match the error of $s=1$, then Eq. (13) selects parameters rather than accuracy and the paper's trade-off claim would fail.","tokens_in":15595,"feed_emoji":"🧩","tokens_out":6692,"duration_ms":67296,"temperature":0.7,"pith_summary":"This paper proposes a filter-basis method for compressing convolutional neural networks. Instead of treating each 3D filter as one indivisible object or each 2D channel filter separately, it splits every filter into $s$ groups along the input-channel dimension and represents all $n \\cdot s$ splits as linear combinations of a small shared basis. The paper derives the split count that minimizes the parameter count, $s^* = \\sqrt{cwh/n}$, and shows that whole-filter and channel-wise decompositions are only the two extremes of the same formula. On VGG, ResNet, and DenseNet for CIFAR-10 classification and on SRResNet and EDSR for super-resolution, it reports lower error or better PSNR than prior decomposition methods at equal or smaller model size. The point of the approach is that the best compression operates in between the two 'hard' decomposition strategies rather than at either boundary.","feed_headline":"Splitting filters along channels beats whole-filter CNN compression","feed_subtitle":"The in-between split point cuts parameters on VGG, ResNet, DenseNet, and EDSR while keeping accuracy close to baseline.","key_machinery":"The central object is split-wise filter-basis decomposition: a $c\\times w\\times h$ filter tensor is cut into $s$ slices of size $p\\times w\\times h$ with $c = s \\cdot p$, and the resulting $n \\cdot s$ slices are written as linear combinations of $m$ shared basis filters. The identity that carries the argument is the compression-rate formula $\\Gamma_{\\text{split}} = \\frac{m}{n s} + \\frac{m}{p w h}$, which balances basis storage against coefficient storage; minimizing it yields $s^* = \\sqrt{cwh/n}$, the in-between operating point. The second mechanism is convolutional re-implementation: the reconstructed filter convolution equals a convolution of each input split with the basis followed by a $1\\times 1$ convolution of the coding coefficients, so no special hardware or custom operator is needed. Basis sharing across layers, residual blocks, or entire networks is a further knob that reduces the $m$ term in the compression ratio.","core_discovery":"The central claim is that every convolutional layer can be approximated as a small learned basis $B$ shared across channel splits plus a $1\\times 1$ convolution of coding coefficients $A$, and that the parameter-optimal degree of splitting is neither whole-filter ($s=1$) nor per-channel ($s=c$) but the middle point given by $s^* = \\sqrt{cwh/n}$. With $c$ input channels, $n$ output channels, and kernel size $w\\times h$, the compression ratio is $\\Gamma_{\\text{split}} = \\frac{m}{n s} + \\frac{m}{p w h}$; minimizing it over $s$ with $c = s p$ gives the optimal split. Because the basis has only $p$ input channels, the method can reduce input channels, making narrow networks like DenseNet and $1\\times 1$ convolutions compressible, and because convolution is linear, the decomposed operation is implemented exactly as a basis convolution followed by a $1\\times 1$ convolution. The paper validates the claim with classification experiments on VGG-16, ResNet-56, and DenseNet-12-40 and super-resolution experiments on SRResNet and EDSR, reporting accuracy and parameter trade-offs favorable against the Factor, Group, K-means, and KSE baselines.","pith_inferences":["Since Eq. (13) reduces to $s^* \\approx \\sqrt{w \\cdot h}$ when $c \\approx n$, a no-search rule of thumb follows: try 3-way splits for $3\\times 3$ kernels and 5-way splits for $5\\times 5$ kernels; the paper's tables are consistent with that region but do not state it as a rule.","The decomposition is a structured low-rank factorization in which input-channel splits share one basis, which suggests the same ratio-balancing argument could be ported to tensorized architectures or to convolution-like operators in other domains where $w \\cdot h$ is replaced by the effective kernel dimension.","Because $m$ and $s$ are selected manually per network in the experiments, an immediate extension is to treat them as learnable per-layer hyperparameters, using Eq. (13) as initialization or as a regularizer toward the parameter-optimal point.","The compression is orthogonal to pruning and quantization, so stacking all three on one network should give roughly multiplicative savings; the paper does not test this combination."],"forward_implications":["Every standard convolution can be replaced by a basis convolution followed by a $1\\times 1$ convolution, so the compressed model runs on ordinary deep-learning libraries without custom layers.","Layers with many more input channels than output channels, the narrow-network case such as DenseNet, become compressible because splitting reduces the per-basis input channel count.","$1\\times 1$ convolutions fall under the same formula, so bottleneck and transition layers can be compressed instead of being left untouched.","The compression-rate identity unifies prior decomposition work: $s=1$ is whole-filter decomposition and $s=c$ is channel-wise decomposition, making them boundary cases of one knob.","Basis sharing within residual blocks, groups, or whole networks multiplies the savings of the split-wise scheme with little additional accuracy loss, as shown in the EDSR and DenseNet results."],"supporting_citations":[{"why":"Supplies the 2D separable-filter basis decomposition that the split-wise method generalizes.","marker":"[21]"},{"why":"Defines the whole-3D-filter low-rank decomposition that is the $s=1$ boundary case.","marker":"[53]"},{"why":"The Factorized CNN baseline that the method outperforms on super-resolution and classification.","marker":"[47]"},{"why":"The k-means kernel clustering baseline compared on VGG, DenseNet, and ResNet.","marker":"[46]"},{"why":"The group-filter approximation baseline compared on classification.","marker":"[41]"},{"why":"The kernel-sparsity and entropy baseline compared on DenseNet and ResNet.","marker":"[32]"},{"why":"Defines EDSR, the large super-resolution network used for the main SR compression tests.","marker":"[34]"},{"why":"Defines DenseNet, the narrow-network architecture that motivates input-channel splitting.","marker":"[19]"},{"why":"Defines ResNet, one of the classification architectures used to validate the method.","marker":"[16]"}],"fun_headline_variants":["Optimal channel split compresses CNNs most","Learn filter basis, split channels to slim CNNs","Basis filters and 1x1 conv shrink CNN size","Middle split point beats whole-filter CNN compression","Split at the sweet spot to compress CNNs accurately"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The optimal split count is derived from parameter counts alone: it assumes that a basis of fixed size $m$ represents narrow splits about as accurately as wide ones, so the cheapest split is also the accuracy-preserving one, but the experiments choose $m$ and $p$ per network rather than testing that invariance.","fun_headline_variants_meta":{"raw":{"variants":["Optimal channel split compresses CNNs most","Learn filter basis, split channels to slim CNNs","Basis filters and 1x1 conv shrink CNN size","Middle split point beats whole-filter CNN compression","Split at the sweet spot to compress CNNs accurately"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000719,"raw_usage":{"total_tokens":3236,"prompt_tokens":958,"completion_tokens":2278,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":574,"completion_tokens_details":{"reasoning_tokens":2203}},"tokens_in":574,"tokens_out":2278,"duration_ms":17688,"temperature":1.0,"reasoning_tokens":2203,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-14T11:25:33.107373+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Take a trained convolutional layer with fixed $c$, $n$, $w$, $h$; keep $m$ fixed, sweep $s$ over divisors of $c$, train each split-wise decomposition to convergence, and measure the resulting reconstruction error or end-to-end accuracy. If the $s$ that minimizes $\\Gamma_{\\text{split}}$ does not also give the best accuracy at a fixed parameter budget, or if narrow splits require a noticeably larger $m$ to match the error of $s=1$, then Eq. (13) selects parameters rather than accuracy and the paper's trade-off claim would fail.","supporting_citations":[{"cited_title":"Speeding up convolutional neural networks with low rank expansions","cited_arxiv_id":null,"evidence_quote":"Supplies the 2D separable-filter basis decomposition that the split-wise method generalizes."},{"cited_title":"Accelerating very deep convolutional networks for classi- fication and detection","cited_arxiv_id":null,"evidence_quote":"Defines the whole-3D-filter low-rank decomposition that is the $s=1$ boundary case."},{"cited_title":"Factorized convolutional neural networks","cited_arxiv_id":null,"evidence_quote":"The Factorized CNN baseline that the method outperforms on super-resolution and classification."},{"cited_title":"Cluster- ing convolutional kernels to compress deep neural networks","cited_arxiv_id":null,"evidence_quote":"The k-means kernel clustering baseline compared on VGG, DenseNet, and ResNet."},{"cited_title":"Extreme network compression via filter group approximation","cited_arxiv_id":null,"evidence_quote":"The group-filter approximation baseline compared on classification."},{"cited_title":"Exploiting kernel sparsity and entropy for inter- pretable CNN compression","cited_arxiv_id":null,"evidence_quote":"The kernel-sparsity and entropy baseline compared on DenseNet and ResNet."},{"cited_title":"Enhanced deep residual networks for single image super-resolution","cited_arxiv_id":null,"evidence_quote":"Defines EDSR, the large super-resolution network used for the main SR compression tests."},{"cited_title":"Densely connected convolutional net- works","cited_arxiv_id":null,"evidence_quote":"Defines DenseNet, the narrow-network architecture that motivates input-channel splitting."},{"cited_title":"Deep residual learning for image recognition","cited_arxiv_id":null,"evidence_quote":"Defines ResNet, one of the classification architectures used to validate the method."}],"review_version":1}