{"id":"c0bc27f6-ec55-4eea-afda-33122c594e85","arxiv_id":"2501.01239","paper_version":5,"verdict":"REJECT","confidence":"MODERATE","novelty_score":4.0,"correctness_risk":"high","formal_verification":"none","parameter_count":0,"one_line_summary":"A tensor-based reformulation of sparse CNNs and backpropagation is derived, but the key gradient formula is incorrect.","lead":"This paper rewrites convolutional neural networks and backpropagation using high-order tensor products and inner products, aiming for a generic mathematical framework for sparse CNNs. The central filter-gradient formula contains an index error, and no experiments or code are provided.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The filter gradient in Eq. (65) rests on Eq. (67), which misstates the derivative of the compounded filter function: the derivative depends on the sliding-window index difference, not on the output index alone.","rationale":"The paper's central claim is the closed-form filter gradient (65) that feeds Algorithm 1. Tracing that derivation shows it depends on Eq. (67), which is asserted to follow from the Kronecker delta contraction. That assertion is not correct: the compounded filter function in Eq. (33)/(47) is linear in X, but its derivative is not a constant tensor with entries δ_{i t}. Instead, the derivative encodes the sliding-window relation t = b-a+1, as the paper's own example (20) demonstrates. This is not an issue of convention or notation; Eq. (67) is nonzero only when the output index equals the filter index, which is false for any convolution with k>1 and s=1 except at the first tap. Once Eq. (67) fails, Eq. (68) is not the derivative; moreover Eq. (68) has a leg indexed by j+s i, which can exceed the filter dimension, so it cannot be a gradient with respect to a filter of size k. The error propagates into the batch-mode formulation and Algorithm 1, so the filter update is not a valid gradient step. This is a correctness failure in the central construction, not a disagreement with current consensus or a missing experiment. The reader's weakest-assumption analysis identifies the same equation and the same index error, so I agree with the REJECT verdict; no adjustment is needed.","tokens_in":135,"tokens_out":7242,"duration_ms":78645,"concrete_test":"Implement a scalar 1D version of the claimed identity: choose q=1, k=2, n=5, s=1, so F(X) is the 4×5 Toeplitz matrix from Eq. (21). Compute ∂[F(X)]_{a,b}/∂X_1 and ∂[F(X)]_{a,b}/∂X_2 directly from Eq. (19) for all (a,b), and compare with Eq. (67). Then compute the right-hand side of Eq. (65) using Eq. (68) with random Z and δ, and compare it to the standard filter gradient ∂L/∂F_t = Σ_{i=0}^{3} δ_{i+1} Z_{i+t}; if the two differ, the central formula is invalid.","verdict_should_be":"UNCHANGED","load_bearing_attack":"Equations (67)-(68) are load-bearing: they define the alleged constant tensor ∇F^(r) that turns Eq. (65) into a filter update. The asserted derivative is incorrect. From the definition in Eq. (33)/(47), the coordinate [F^(r)(X)]_{a_1...a_q,b_1...b_q} equals X_{b_1-a_1+1,...,b_q-a_q+1} when a_l = 1+s_l i_l and b_l = j_l+s_l i_l for valid i_l,j_l, and is zero otherwise. Therefore ∂[F^(r)(X)]_{a,b}/∂X_t is nonzero only when t_l = b_l-a_l+1 for every l, not when t_l = a_l as Eq. (67) asserts. For the 1D example in Eq. (20) with k=2, n=5, s=1, row a=1, column b=2 has derivative 1 with respect to t=2, while Eq. (67) gives δ_{1,2}=0. This error propagates into Eq. (68): its third leg, ^z^r_{j+s i}, ranges over coordinates that can exceed the filter size k, whereas a filter gradient must be indexed by j=1..k. Consequently the gradient ∇Xψ^(r) in Eq. (65) is not a tensor in the filter space, and the gradient descent update in Eqs. (69)-(70) does not compute a true gradient. The central backpropagation result and Algorithm 1 therefore fail.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper develops a tensor-calculus formalism for convolution and uses it to derive a purported generic backpropagation algorithm for high-order sparse convolutional neural networks. Convolution is defined through a compounded filter function F^(r) (Eq. 47), and the central result is a batch-loss gradient with respect to the filter, ∇_X ψ^(r) = (1/p) Σ_i Z^(r-1)_[i] ⊙_q ∇F^(r) ⊙_q δ^(r), where ∇F^(r) is claimed to be the constant tensor given by Eqs. (67)-(68). These gradients feed Algorithm 1. The paper contains no numerical experiments.","tokens_in":11345,"tokens_out":10194,"duration_ms":105625,"significance":"If the derivation were correct, the paper would provide a compact and fully tensorized derivation of backpropagation for convolutional layers with arbitrary-order tensors, strides, and padding, and the compounded-filter-function notation is a useful idea. However, the central gradient formula is false as stated. The error is load-bearing: it invalidates the filter-gradient expression in Eq. (65), the constant-tensor construction in Eq. (68), and the gradient-descent update in Eq. (70). The paper therefore does not currently establish its claimed result.","major_comments":[{"comment":"The derivative of the compounded filter function is misstated. From Eq. (47), a nonzero coordinate [F^(r)(X)]_{a_1...a_q,b_1...b_q} exists only when a_l = 1 + s_l i_l and b_l = j_l + s_l i_l for a common sliding index i_l, so b_l - a_l = j_l - 1. Hence the partial derivative with respect to the filter coordinate X_{t_1...t_q} is nonzero precisely when t_l = j_l = b_l - a_l + 1 for every l, subject to the valid sliding-window ranges, and not when t_l = a_l as Eq. (67) asserts. The paper's own example in Eq. (20) is a concrete counterexample: for k=2, n=5, s=1, the coordinate [F(X)]_{1,2} equals X_2, so ∂[F(X)]_{1,2}/∂X_2 = 1, whereas Eq. (67) evaluates this derivative to δ_{1,2}=0.","section":"§2.1, Eq. (67)"},{"comment":"Because Eq. (67) is wrong, the tensor ∇F^(r) defined in Eq. (68) is not the derivative of F^(r) with respect to the filter. Moreover, the first leg of each term in Eq. (68) is indexed by ^z^(r-1)_{j_l+s_l i_l}, whose coordinate ranges over the input-space positions and can exceed the filter size k_l; a filter gradient must instead be indexed by j_l ∈ {1,...,k_l}. Eq. (68) therefore does not even have the correct tensor order for a derivative with respect to filter coordinates. Consequently, ∇_X ψ^(r) in Eq. (65) is not a tensor indexed by filter coordinates, and the update F^(r)_{t+1} = F^(r)_t + γ∇_X ψ^(r) in Eq. (70) is dimensionally inconsistent. Algorithm 1 as stated does not perform gradient descent on the filter variables.","section":"§2.1, Eqs. (65), (68), (70)"},{"comment":"The factorization ∇_X (F^(r)(X) ⊙_q Z^(r-1)_[i] + Y) = Z^(r-1)_[i] ⊙_q ∇F^(r)(X) is asserted without a proof of the contraction structure, and it inherits the index error of Eq. (67). A correct derivation would need to specify the exact contracted indices in the r-order inner product and verify by finite differences or direct coordinate computation that the resulting expression is a tensor in the filter index space. As it stands, the chain-rule step is not established.","section":"§2.1, Eq. (65)"}],"minor_comments":[{"comment":"The multilayer network structure is imported from the author's prior preprint [2] without derivation or a statement of the assumptions needed for the later calculus; the paper should be self-contained or clearly mark which results are taken from [2].","section":"§2, Eqs. (36)-(37)"},{"comment":"The list of loss functions jumps from item ii to item iv; the missing item iii should be restored or renumbered.","section":"§2, Eqs. (41)-(45)"},{"comment":"The symbols i_l and j_l are used both as summation indices in the definition of F^(r) and as coordinate labels in the derivative expression; this notational collision makes Eq. (67) especially unclear and should be disambiguated.","section":"§2.1, Eq. (47) and Eq. (67)"},{"comment":"No numerical verification is reported. Even a simple finite-difference check of Eq. (65) on a one-layer, one-dimensional convolution would have exposed the index error and would be a useful addition if the derivation is revised.","section":"§2.1, Algorithm 1"}],"recommendation":"reject","confidential_remarks":"The stress-test concern lands directly: Eq. (67) contradicts the definition of F^(r) and the example in Eq. (20), and the error propagates into the main algorithm. This is not a local typo; the central backpropagation derivation and the resulting filter update are not valid. A corrected derivation would be a substantially new manuscript, so I would not invite a major revision on this version."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Quick take: the paper's main contribution—a high-order tensor version of backpropagation for CNNs—rests on Eq. (67), and that equation is wrong. The convolution definitions in Eqs. (19)–(33) are sound and the notation is reasonably clean; the problem is the derivative. From the definition of the compounded filter function, the coordinate [F(X)]_{a,b} depends on X_{b-a+1} (for stride 1); it does not depend on the output coordinate a alone. Eq. (67) asserts the derivative is nonzero only when i = t, which is not what the definition gives. The stress-test note is correct: for the k=2, n=5 example, F_2 contributes to output coordinate 1, so ∂[F(X)]_{1,2}/∂X_2 = 1, while Eq. (67) would give zero. This error propagates into Eq. (68), where the free leg is indexed by j+s i rather than the filter coordinate j, so the proposed ∇F tensor does not even have the right shape to be contracted into a filter-space gradient. Consequently Eq. (65) is not a genuine gradient, and the gradient descent update in Eqs. (69)–(70) does not compute what it claims.\n\nWhat is actually new: the r-order inner product and the compounded filter function are a tidy notation for expressing stride, padding, and multi-dimensional convolution in a single formula. That is textbook convolution re-expressed, but it is done clearly. The generalization to high-order tensors in Eq. (33) is formally consistent. Credit where due: the definitions are precise enough that a reader can check the derivation, which is exactly how the error is caught. That is a real virtue.\n\nSoft spots: besides the load-bearing error, the paper cites no prior work on tensorized CNNs or tensor networks, so the claim of a 'generic' or 'significantly different' approach is unsubstantiated. There are no experiments or complexity measurements, so the asserted computational savings are a statement, not a result. The ANN layer structure is imported from the author's own previous paper [2] without a self-contained derivation. These are secondary, but they compound the problem.\n\nWho this is for: someone interested in a clean tensor notation for convolution might skim Section 1 and borrow the notation. Anyone planning to use the backpropagation result should not, because it is wrong.\n\nRecommendation: a serious referee should see this. Do not desk-reject; the error is concrete, clearly checkable, and the framework might be repairable by correcting the derivative index and reshaping ∇F. Expect rejection in the current form, but a referee report that pinpoints Eq. (67) would genuinely help the author revise.","headline":"The backpropagation derivation has a concrete index error in Eq. (67) that breaks the claimed gradient update; the convolution notation is clean, but the central result is not correct.","tokens_in":11904,"tokens_out":6303,"would_cite":false,"duration_ms":58440,"reading_group":"no","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":["68T07","15A69"],"pacs":[],"model":"deepseek-v4-flash","headline":"A high-order tensor identity rewrites CNN backpropagation as a single contraction.","keywords":["tensor regression","convolutional neural networks","backpropagation","high-order tensors","sparse networks","compounded filter function","r-order inner product","batch gradient"],"falsifier":"On a small CNN with a single filter of size 2×2, stride 1, and a few random inputs, compute the filter gradient from Eq. (65) and compare it with numerical central differences of the batch loss. A mismatch larger than floating-point tolerance would refute Eq. (67) and therefore the proposed backpropagation algorithm.","tokens_in":10820,"feed_emoji":"🧠","tokens_out":7357,"duration_ms":62675,"temperature":0.7,"pith_summary":"This paper builds a tensor calculus for convolution in sparse convolutional neural networks and claims that backpropagation, for any layer and any tensor order, reduces to one contraction formula. The formula says the gradient of the batch loss with respect to a filter is the previous layer's activations contracted with a constant tensor built from that filter and with the error signal propagating from the next layer. If this is correct, the implementation of CNN training no longer needs separate cases for different filter sizes, strides, paddings, or input dimensionalities; the same expression covers them all. The paper presents the result as a direct extension of a tensor-based regression theory to convolutional architectures.","feed_headline":"CNN backprop is one tensor contraction","feed_subtitle":"One contraction gives filter gradients for any stride, padding, or tensor order.","key_machinery":"The two objects that carry the argument are the $r$-order inner product (Eq. 10), which contracts $r$ matching tensor orders, and the compounded filter function $\\mathcal{F}^{(r)}(X)$ (Eq. 33), which maps a filter array into a sparse weight tensor by placing each filter entry at all positions shifted by the stride. Their composition defines convolution for any tensor order. The load-bearing identity is Eq. (67): the derivative of a coordinate of $\\mathcal{F}^{(r)}(X)$ with respect to a filter entry is a Kronecker-delta product on the output index. This makes the gradient tensor $\\nabla\\mathcal{F}^{(r)}$ constant, so the average filter gradient becomes a fixed contraction of activations, error signal, and a precomputable tensor.","core_discovery":"The paper's central claim is that a CNN layer is $f^{(l)}(X) = \\varphi^{(l)}(F^{(l)} \\circledast X + B^{(l)})$, where the convolution $F^{(l)} \\circledast X$ is defined as the $q$-order inner product $\\mathcal{F}^{(l)}(F^{(l)}) \\odot_q X$ with $\\mathcal{F}^{(l)}$ the compounded filter function of Eq. (33). Under that definition, the filter gradient is $\\nabla_X \\psi^{(r)} = \\frac{1}{p} \\sum_{i=1}^p Z^{(r-1)}_{[i]} \\odot_q \\nabla\\mathcal{F}^{(r)} \\odot_q \\delta^{(r)}$, and the tensor $\\nabla\\mathcal{F}^{(r)}$ is constant, with coordinates given by Eq. (68). This constancy is traced to Eq. (67), where the derivative of a coordinate of the compounded filter output with respect to a filter entry is a product of Kronecker deltas. The paper then rewrites the classic backpropagation algorithm, Algorithm 1, on top of this identity, with no need for the usual notions of local receptive field or weight sharing.","pith_inferences":["A straightforward check would be to compare the analytic filter gradient from Eq. (65) with finite differences on a small one-layer CNN; disagreement would isolate Eq. (67) as the point of failure.","The same machinery might extend to transposed convolutions, dilated convolutions, or graph-structured filters by choosing different compounded filter functions, though the paper does not develop these cases.","The paper does not give an algorithm for assembling the constant tensor $\\nabla\\mathcal{F}^{(r)}$ in code; an efficient construction routine would be needed before the identity can be used in practice.","A consequence left implicit is that the framework suggests convolution is one member of a larger family of structured sparse linear maps whose gradients are constant contractions, which could unify several network architectures under one tensor language."],"forward_implications":["The same filter-update formula applies to every layer, for any filter size, stride, padding, and tensor order; only the constant tensor $\\nabla\\mathcal{F}^{(r)}$ changes.","Since $\\nabla\\mathcal{F}^{(r)}$ does not depend on the data, it can be computed once per layer and reused throughout training.","Batch processing is absorbed by adding one tensor order for the $p$ samples, so the averaged gradient is produced by the same contraction in a single pass.","The formulation makes concepts such as local receptive field and weight sharing superfluous, because the connectivity pattern is encoded entirely in the compounded filter function."],"supporting_citations":[{"why":"Provides the tensor-based regression and backpropagation framework for low-order tensors that this paper extends to high-order convolution; the notation and chain-rule setup in Eqs. (35)–(66) follow its approach.","marker":"[2]"}],"fun_headline_variants":["Backprop as a single tensor contraction","Tensor regression unifies sparse CNN backprop","One contraction: generic CNN backprop","High-order tensor view simplifies CNN training"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that Eq. (67) gives the correct derivative of the compounded filter function: changing one filter entry affects only the output coordinate at the matching shifted position, and not any other, so the derivative is a one-or-zero delta; if that is wrong, the constant gradient tensor and every filter update in Algorithm 1 are wrong.","fun_headline_variants_meta":{"raw":{"variants":["Backprop as a single tensor contraction","Tensor regression unifies sparse CNN backprop","One contraction: generic CNN backprop","High-order tensor view simplifies CNN training"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000145,"raw_usage":{"total_tokens":1139,"prompt_tokens":868,"completion_tokens":271,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":484,"completion_tokens_details":{"reasoning_tokens":219}},"tokens_in":484,"tokens_out":271,"duration_ms":3009,"temperature":1.0,"reasoning_tokens":219,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-10T22:33:27.119388+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"On a small CNN with a single filter of size 2×2, stride 1, and a few random inputs, compute the filter gradient from Eq. (65) and compare it with numerical central differences of the batch loss. A mismatch larger than floating-point tolerance would refute Eq. (67) and therefore the proposed backpropagation algorithm.","supporting_citations":[{"cited_title":"Tensor-Based Foundations of Ordinary Least Squares and Neural Network Regression Models","cited_arxiv_id":"2411.12873","evidence_quote":"Provides the tensor-based regression and backpropagation framework for low-order tensors that this paper extends to high-order convolution; the notation and chain-rule setup in Eqs. (35)–(66) follow its approach."}],"review_version":1}