{"id":"5f3b5329-0bcb-4aa2-a9df-5d35d021e390","arxiv_id":"2506.00992","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":1,"one_line_summary":"A quotient network that multiplies old features by a learned ratio instead of adding a difference achieves small, consistent accuracy gains over ResNet on CIFAR10, CIFAR100, and SVHN.","lead":"A new neural network block replaces ResNet's additive residual with multiplication by a learned quotient, so each layer scales the previous features rather than adding to them. On CIFAR and SVHN image benchmarks the authors report small but consistent accuracy gains over matching ResNets with no extra parameters.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Reported gains over ResNet are confounded: quotient networks also replace ReLU with the Eq. 1 activation in the head, shortcut projections, and block outputs; with no control that keeps the activation fixed, the improvements cannot be attributed to the quotient operation.","rationale":"I agree with the reader's weakest-assumption analysis. The strongest claim (abstract: 'stably achieve considerable improvements over ResNet... without adding new parameters') is an empirical causal claim about the quotient operation. For it to hold, the quotient network must be compared with a ResNet that is identical except for the arithmetic operation. The paper instead changes the activation function in three places that are known to affect training dynamics. Appendix Table 8 shows that activation placement changes 20-layer quotient accuracy by 1–2 points (89.15 to 91.72), which is larger than most reported ResNet-vs-quotient gaps (0.17–0.68 points, with overlapping std ranges). This makes the confound quantitatively plausible, not merely theoretical. A single additive-twin experiment would settle it. I also note the per-depth tuning of α (Section 4.3) adds a hyperparameter to the quotient side, and no statistical significance test is reported, but those are secondary. The paper's qualitative feature-map motivation and the absence of easily accessible code are not the main problem. The verdict should remain CONDITIONAL, conditioned on the additive-twin ablation and code release.","tokens_in":16445,"tokens_out":4616,"duration_ms":46602,"concrete_test":"For each setting in Tables 1–3, build the additive twin of the quotient network: identical architecture, depths, widths, α values, and the Eq. 1 activation in the head, shortcut projections, and block outputs, but replace H=F∗x with H=F+x. Train with the exact protocol of §4.2 and report mean±std from the same number of runs. If the additive twin matches the quotient network's accuracy (difference within 1 std), the quotient operation is not the cause of the reported improvements; if the quotient network beats it consistently and by more than the error bars, the confound is resolved and the paper's interpretation is supported.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The paper's central claim is that learning a quotient, H(x)=F(x)*x, is what improves over ResNet's H(x)=F(x)+x. But the comparison architectures differ in two changes at once: (1) multiplication vs addition, and (2) replacement of ReLU by the bounded positive activation of Eq. 1, sigmoid(x−ln(α−1))·α, in the network head (§3.3.3), in channel-changing shortcut convolutions (§3.3.2), and in the final convolution of every quotient block (Figure 1). The experiments in Tables 1–3 therefore do not isolate the quotient mechanism. The appendix ablation (Table 8) varies only the placement of Eq. 1 inside quotient networks; it never tests a ResNet whose activations have been swapped to Eq. 1 while keeping addition. If the activation swap alone yields the same accuracy gains, the 'quotient' contribution is unsupported. This is load-bearing because the abstract and conclusion attribute the improvements specifically to quotient learning, and the small margins (e.g., 93.10±0.15 vs 92.84±0.18 on CIFAR10-56; overlapping ±std ranges in several rows) leave no room for a confound.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes a 'quotient network' in which a residual block computes H(x) = F(x) * x instead of ResNet's H(x) = F(x) + x, with F(x) learned as a quotient. To make multiplication-based blocks trainable, the authors introduce design rules: a bounded, positive, globally differentiable activation function (Eq. 1) that passes through (0,1), use of this activation in the network head and in channel-increasing shortcut convolutions, and 3x3 convolutional projections for channel changes. Experiments on CIFAR-10, CIFAR-100, and SVHN with 44/56/110-layer variants report small consistent accuracy gains over ResNet with the same parameter count, plus visualizations of intermediate feature maps. The paper also provides an appendix ablation of the design-rule choices and an analysis of the increased computational cost.","tokens_in":16630,"tokens_out":5503,"duration_ms":53435,"significance":"The idea of learning a quotient rather than a difference is conceptually interesting and, if the gains are real and attributable to the quotient mechanism, it would be a meaningful architectural contribution. The paper includes useful elements: an explicit design-rule ablation in Appendix A, an honest account of the added computation in Section 3.5, and a genuine out-of-sample transfer of the alpha hyperparameter from CIFAR-10 to CIFAR-100 and SVHN. However, the central empirical claim that quotient learning itself yields the improvements is not yet supported because the comparison changes more than the residual/quotient operation.","major_comments":[{"comment":"The comparison of the quotient network to ResNet changes two factors simultaneously: the combine operation (addition vs multiplication) and the activation function. The quotient network uses Eq. 1 not only in the final convolution of each block (Figure 1), but also in the network head (Figure 2) and in channel-increasing shortcut convolutions (Figure 3), whereas the ResNet baseline uses ReLU. No experiment holds the activation fixed and varies only the combine operation. The appendix ablation in Table 8 shows that placement of Eq. 1 inside quotient networks changes accuracy by about one percentage point (e.g., sigmoid 90.67 with no placement vs 91.72 with head+shortcut placement), so the activation change is a plausible alternative explanation for the gains in Tables 1-3. I request an ablation that applies Eq. 1 to a ResNet while keeping addition (i.e., swap only the activation in the head, shortcuts, and block output), or a quotient network that uses the same activation as the corresponding ResNet, to isolate the contribution of multiplication. As written, the abstract's claim that improvements come from quotient learning is not supported.","section":"Sections 4.3 and Tables 1-3"},{"comment":"The claimed 'stable' and 'considerable' improvements are not backed by sufficient statistical evidence. The paper reports mean ± std but never states the number of runs, whether the variation is over random seeds, data splits, or both, and it does not report any significance test or confidence interval. Several key comparisons have overlapping standard-deviation ranges, for example CIFAR-10-44 (92.78±0.25 vs 92.61±0.33) and CIFAR-100-44 (73.25±0.27 vs 72.66±1.24). Given the small margins, the authors should state the number of runs and either report paired significance tests (e.g., matched-seed t-tests) or show that the improvement holds for every run. Otherwise the wording 'prove that this network can stably achieve considerable improvements' overstates the evidence.","section":"Sections 4.3 and Tables 1-3"},{"comment":"The text describing the shortcut projection is ambiguous and potentially misleading. The sentence 'Unlike the original paper shortcuts, which add new channels with all zeros when the number of channels increases, we use a 3x3 convolution with stride 2 to increase the number of channels' appears to describe the base network used for both ResNet and the quotient network, but it could be read as a change introduced specifically for the quotient network. If the ResNet baseline uses the same 3x3 convolutional projections, please say so explicitly; if it does not, then the comparison in Tables 1-3 is confounded by a shortcut-type difference and the parameter-count claim needs reconciliation.","section":"Section 3.4"}],"minor_comments":[{"comment":"The abstract and introduction use the word 'perfectly solves' and 'prove'; given the experimental margins and the confounds discussed above, these claims should be softened to 'addresses' and 'suggests' or 'indicates'.","section":"Abstract and Section 1"},{"comment":"Please replace 'Formula 1' with 'Equation (1)' and consistently number equations; the manuscript currently uses only a single unnumbered equation.","section":"Throughout"},{"comment":"Please state explicitly how many independent runs produced the mean ± std values in Tables 1-3, and whether the reported value is the standard deviation or the standard error of the mean.","section":"Section 4.3"},{"comment":"The ablation in Table 8 is informative, but the 'null' condition should be described more precisely: does it mean ReLU is used in the head and shortcuts, or that no activation is applied after the final convolution? Clarify to make the baseline unambiguous.","section":"Appendix Table 8"},{"comment":"The claim that 'useful feature representations are often positive numbers' is stated without support; a brief citation or a short justification would help.","section":"Section 3.3.1"}],"recommendation":"major_revision","confidential_remarks":"The confound identified in the major comments is the key issue: the paper's central claim requires an ablation that isolates multiplication from the activation change. The paper also suffers from under-specified statistics. If the authors add the requested controls and clarify the shortcut-projection description, the contribution could become publishable; as it stands, the evidence does not yet support the abstract's strong attribution to quotient learning."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Two things you should know about arXiv:2506.00992. First, the core idea is genuinely a different parameterization: learn a quotient H(x)/x and multiply, instead of adding a residual. The authors think carefully about what the multiplier needs to look like—positive, bounded, passing through (0,1)—and turn that into a concrete activation function and design rules. That is a real contribution, and the paper is clear about it.\n\nSecond, the experiments do not establish that the quotient operation is what helps. The quotient network differs from the control ResNet in two ways: multiplication vs addition, and the activation function in the head, in the channel-changing shortcuts, and before the final multiplication. The appendix ablations only move the new activation around inside quotient networks; they never take a standard ResNet and swap ReLU for the same activation while keeping addition. So the comparisons in Tables 1–3 are confounded. Given the gains are small (roughly 0.1–0.5 points, with overlapping ±std in several rows) and no significance tests are reported, the abstract's claim that the gains 'prove' quotient learning is too strong.\n\nThe cross-dataset transfer is a bright spot: they tune α per depth on CIFAR10 and use the same values on CIFAR100 and SVHN, which is a genuine out-of-sample check. Reporting mean±std and being upfront about the extra compute in Section 3.5 also counts in their favor.\n\nThe soft spots are the confound and the effect size. The confound is load-bearing because the abstract and conclusion attribute improvements specifically to quotient learning. There is also a smaller issue: the NeurIPS checklist says code is in the supplementary material, but no code or URL appears in the arXiv v1 we read.\n\nWho gets value from this? Someone designing multiplicative feature-update blocks might find the design rules useful as a starting point. But I would not cite it as evidence that quotient learning beats residual learning without an isolating ablation.\n\nRecommendation: send it to peer review, not desk reject. The idea deserves referee time, but the first round should ask for the missing control—a ResNet with the same activation and addition instead of multiplication—and for either significance tests or more runs. That will settle whether the quotient is doing the work.","headline":"A genuinely different block parameterization, but the headline claim is undercut by a confound: the gains could come from the activation swap rather than the quotient.","tokens_in":17217,"tokens_out":2672,"would_cite":false,"duration_ms":27553,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Learning ratios instead of differences improves on ResNet","keywords":["quotient network","residual learning","multiplicative gating","activation function design","CIFAR-10","CIFAR-100","SVHN","deep convolutional networks"],"falsifier":"Train two otherwise identical networks on CIFAR10, one residual with $H=F(x)+x$ and one quotient with $H=F(x)\\times x$, using the same sigmoid-style activation of Equation 1 in the head and shortcuts for both. If the residual network with the new activation matches or beats the quotient network's accuracy, the paper's attribution of gains to quotient learning is falsified; if the quotient network still wins, the claim survives.","tokens_in":16179,"feed_emoji":"➗","tokens_out":4335,"duration_ms":42516,"temperature":0.7,"pith_summary":"This paper argues that residual learning in ResNet addresses the wrong quantity: the absolute difference $H(x)-x$ between new and old features is sensitive to feature scale and does not carry clear semantic meaning. The proposed quotient network instead learns the ratio $H(x)/x$ and reconstructs features by multiplication, $H(x)=F(x)\\times x$. The authors give design rules, including a positive, globally differentiable activation that passes through $(0,1)$ and placement of that activation in the network head and channel-changing shortcuts, and report that on CIFAR10, CIFAR100, and SVHN the quotient network stably outperforms ResNet with the same layer count and parameter count. The point of the claim is that relative change is a more natural learning target than absolute change.","feed_headline":"Learning ratios instead of differences lifts image classification","feed_subtitle":"Quotient blocks H=F(x)×x beat residual blocks H=F(x)+x on CIFAR and SVHN with no extra parameters.","key_machinery":"The quotient module replaces the residual addition $H=F+x$ with the multiplication $H=F(x)\\times x$, and it uses the activation $\\mathrm{activate}(x)=\\operatorname{sigmoid}(x-\\ln(\\alpha-1))\\cdot\\alpha$ before that multiplication. This function is positive, bounded on $(0,\\alpha)$, globally differentiable, and passes through the point $(0,1)$, which lets the layer approximate an identity mapping when weights are small and prevents multiplicative explosion of feature magnitudes. The same activation is also placed in the first convolution of the network and in shortcut convolutions that change channel counts, where zero-padding and ReLU would otherwise corrupt multiplicative learning.","core_discovery":"The central claim is that a network block can learn the quotient of its target and current features instead of their difference, and that doing so makes deep networks easier to train and more accurate. Formally, where ResNet computes $H(x)=F(x)+x$ with $F(x)$ learned as a residual, the quotient network computes $H(x)=F(x)\\times x$ with $F(x)$ learned as a quotient, using the activation of Equation 1 as the final nonlinearity so the multiplier stays positive, bounded, and close to 1 when the pre-activation is near 0. On CIFAR10, CIFAR100, and SVHN, with the scale parameter $\\alpha$ tuned per depth ($1.8$, $1.7$, and $1.5$ for 44-, 56-, and 110-layer networks), the quotient versions of ResNet are reported to be consistently more accurate than the corresponding ResNets without adding any parameters. The paper also reports that the learned quotient feature maps are visually clearer than residual feature maps, supporting the motivation that quotients carry independent meaning.","pith_inferences":["The experiments do not isolate the quotient operation from the activation change, because the quotient network also swaps ReLU for the sigmoid-style activation in the head and shortcut paths; a fair test would hold the activation fixed and vary only addition versus multiplication.","If the quotient mechanism is what drives the gains, the same transformation could apply to other residual-style architectures, including transformer blocks where $H = X + \\mathrm{Attention}(X)$ could become a multiplicative update with a learned positive gate; this is not tested in the paper.","The optimal $\\alpha$ decreasing with depth hints at a scaling rule worth testing: deeper layers may need a smaller multiplier range to counteract growing feature magnitudes, and $\\alpha$ might be scheduled rather than tuned per architecture."],"forward_implications":["On CIFAR10, the 44-layer quotient network reaches 92.78% versus 92.61% for ResNet-44, and the 56-layer quotient network reaches 93.1%, which is above the 93.02% of ResNet-110.","The pattern holds on CIFAR100 and SVHN, where the 44-layer quotient network already outperforms all tested ResNet depths, e.g., 73.25% versus 72.66% for ResNet-44 on CIFAR100.","The gains come without increasing parameter count; the extra cost is only pointwise multiplication and the sigmoid-style activation, adding about three percent to training time for a 56-layer CIFAR10 model.","The design rules are stated as a general recipe: any ResNet can be converted into a quotient network by replacing residual blocks and adjusting activation placement."],"supporting_citations":[{"why":"Supplies the ResNet residual formulation and the CIFAR-style training setup that the quotient network modifies and compares against.","marker":"[10]"},{"why":"Provides the CIFAR10 and CIFAR100 datasets used for the main classification experiments.","marker":"[12]"},{"why":"Provides the SVHN dataset used as the third benchmark in the experiments.","marker":"[13]"},{"why":"Defines the ReLU activation that the quotient network replaces in the head, shortcuts, and final layer of the quotient module.","marker":"[42]"}],"fun_headline_variants":["Replace differences with quotients for better image classification","Quotient blocks beat residual blocks on CIFAR and SVHN","Learn quotients, not differences: deeper nets, better scores","Divider not subtractor: new network improves on ResNet","Quotient network: multiply instead of add for deep learning"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that the accuracy gains come specifically from learning quotients rather than from the accompanying switch in activation function and shortcut design; no experiment isolates the division operation while keeping the activation identical.","fun_headline_variants_meta":{"raw":{"variants":["Replace differences with quotients for better image classification","Quotient blocks beat residual blocks on CIFAR and SVHN","Learn quotients, not differences: deeper nets, better scores","Divider not subtractor: new network improves on ResNet","Quotient network: multiply instead of add for deep learning"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000374,"raw_usage":{"total_tokens":2007,"prompt_tokens":968,"completion_tokens":1039,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":584,"completion_tokens_details":{"reasoning_tokens":954}},"tokens_in":584,"tokens_out":1039,"duration_ms":7576,"temperature":1.0,"reasoning_tokens":954,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-07T11:52:40.633070+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Train two otherwise identical networks on CIFAR10, one residual with $H=F(x)+x$ and one quotient with $H=F(x)\\times x$, using the same sigmoid-style activation of Equation 1 in the head and shortcuts for both. If the residual network with the new activation matches or beats the quotient network's accuracy, the paper's attribution of gains to quotient learning is falsified; if the quotient network still wins, the claim survives.","supporting_citations":[{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Provides the SVHN dataset used as the third benchmark in the experiments."}],"review_version":1}