{"id":"6b241fd9-df53-4443-8fdf-b61db30fa137","arxiv_id":"2411.15659","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":0,"one_line_summary":"SMM-Conv computes convolutions by accumulating kernel-scaled shifted sub-matrices into the output, reusing a single buffer, and reports speedups over im2col+GEMM and MEC on CPU.","lead":"SMM-Conv replaces im2col plus matrix multiplication with scalar-matrix multiply-accumulate over shifted input slices, reporting 2.0 to 3.4 times faster CPU inference than im2col+GEMM on AlexNet, VGG, and YOLO. A generalist might care because it suggests embedded devices could run standard CNNs faster and with less temporary memory.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Stride and padding are never defined: Sec. 3.2 and Algorithm 1 only compute stride-1, zero-padding valid convolution, so the Table 2 speedups on AlexNet/VGG/YOLO rest on an unspecified and unverified extension.","rationale":"The central claim is empirical: SMM-Conv is faster than im2col+GEMM and MEC on AlexNet, VGG, and YOLO. That claim only has force if the measured operation is a correct convolution for those networks' layer configurations. The paper's formal development stops at the stride-1, no-padding case, while all benchmark networks contain padded and/or strided layers. This is not a minor implementation detail: the shifting step relies on output rows being consecutive rows of the input, which is exactly what stride > 1 breaks. A decimation-based fix multiplies the MAC count by s^2 and thus cannot support the paper's 'same number of multiplications' framing. Algorithm 2's channel indexing is also internally inconsistent as written, reinforcing that the described code is not a reliable basis for Table 2. The reader's weakest assumption, the undocumented handling of padding and stride, is the same load-bearing concern I identify, so the conditional verdict stands. The concrete test is to demand the code and verify output equivalence on exactly the layers whose speedups are reported.","tokens_in":7698,"tokens_out":14251,"duration_ms":135110,"concrete_test":"Ask the authors to provide the exact C++/OpenMP implementation and run, for every conv layer of AlexNet, VGG-16, and YOLOv3 reported in Table 2, an output-equivalence comparison against PyTorch conv2d with a float64 reference and tolerances atol=rtol=1e-4, explicitly including AlexNet conv1 (stride=4, pad=2) and the first stride-2 conv of YOLO. If any layer's max-abs error exceeds tolerance, the speedup table cannot be attributed to the described SMM-Conv algorithm. This single test settles whether the undocumented stride/padding extension is correct.","verdict_should_be":"UNCHANGED","load_bearing_attack":"Section 3.2 defines SMM-Conv for the case where T^c_j = I[c, 1:h, j:j+w'-1] and ShiftedMat = SlicedMat[k:h'+k,:], which is exactly valid convolution with stride 1 and no padding. The paper never states how h' and w' are derived when padding p > 0 or stride s > 1, and never describes zero-boundary handling or output decimation. Yet the benchmarked networks include AlexNet conv1 (11x11, stride 4, pad 2) and YOLO stride-2 convolutions; VGG uses pad=1 throughout. For s > 1, the consecutive-row shift of Algorithm 1 no longer produces the strided output: output row r needs input rows r*s+1 ... r*s+kh, which are not a contiguous h'xw' slice of the hxw buffer, so the central buffer-reuse claim does not carry over. If instead one computes the dense stride-1 convolution and then subsamples, the MAC count grows by s^2, contradicting Sec. 4.2's statement that the compared methods share the same number of multiplications and accumulations. No code is released, so Table 2 cannot be checked. Even for the stride-1 case, Algorithm 2 line 17 writes O[lambda*#n,:,:] (and line 16 indexes K with lambda*#n); with d > 1 threads this would produce overlapping or missing output channels as written. The network-level speedups therefore rest on an unstated and possibly different implementation.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes SMM-Conv, a CPU convolution method for channels-first tensors that replaces im2col+GEMM with repeated scalar-matrix multiplications on shifted h×w' slices, reusing one memory buffer per thread. It claims to reduce memory overhead and to achieve end-to-end speedups of 3.42x, 2.11x, and 2.00x on AlexNet, VGG, and YOLO, respectively, compared with im2col+GEMM, and to run faster than MEC in all three networks. The core derivation for stride-1, no-padding valid convolution is straightforward, but the printed algorithms contain an output-channel indexing error, and the paper never specifies how padding and stride are handled in the benchmarked networks.","tokens_in":7817,"tokens_out":5677,"duration_ms":52437,"significance":"If the claims hold, SMM-Conv would be an attractive simple alternative for CPU inference: it avoids im2col's memory duplication, has no fitted parameters, and is derived directly from the definition of convolution. The memory-reduction argument around Eq. (1) is sound for the described valid-convolution setting. However, the significance is conditional because the central empirical claim depends on an undocumented extension to non-unit strides and padding, and because the pseudocode as printed is not correct as written. The paper would be strengthened substantially by releasing the implementation and by fixing the algorithm specification.","major_comments":[{"comment":"The accumulation target is O[c,:,:] while c is the input-channel loop variable; the correct target is O[m,:,:], where m is the output-channel loop variable. As printed, the single-thread algorithm accumulates all output-channel contributions into the same input-channel-indexed slice, producing an incorrect output tensor or an out-of-bounds access when co < ci. This is a load-bearing error in the algorithm that defines the method.","section":"Algorithm 1, line 11"},{"comment":"The shifting step ShiftedMat = SlicedMat[k:h'+k,:] computes valid convolution with stride 1 and no padding only. The paper never gives the output-size formulas h'=(h+2p-kh)/s+1 and w'=(w+2p-kw)/s+1, nor does it explain how boundary zeros or output decimation are handled for stride s>1. Yet the benchmarked networks contain non-unit strides and padding: AlexNet conv1 uses stride 4 and pad 2, YOLO contains stride-2 convolutions, and VGG uses pad 1 throughout. Consequently, the end-to-end speedups in Table 2 are not supported by the described algorithm. If the implementation computes a dense stride-1 convolution and then subsamples, the MAC count grows by s^2, which contradicts the statement in Section 4.2 that the compared methods share the same number of multiplications and accumulations.","section":"Section 3.2 / Algorithms 1-2 / Table 2"},{"comment":"The parallel indexing lambda*#n is not a valid assignment of output channels. With d threads, #n takes values 0,...,d-1, so K and O are accessed at indices 0, lambda, 2*lambda, ... for different lambda values. This causes overlapping writes for some channels and leaves other channels unwritten, so the parallel algorithm is incorrect as written. The intended indexing is presumably lambda + #n*(co/d) (or an equivalent block partition). Without a correct parallel description, the OpenMP-based speedups in Section 4 cannot be reproduced from the pseudocode.","section":"Algorithm 2, lines 15-17"},{"comment":"No code is released, and the paper does not state how padding and stride were treated in the implementations that produced Table 2 and Figure 3. Given that the printed algorithm covers only stride-1, no-padding convolution, the experimental section must be accompanied either by the implementation or by detailed per-layer configurations (padding, stride, input/output channel counts, kernel sizes, and how each is mapped to the slicing scheme). Without this, the claimed network-level speedups cannot be checked.","section":"Section 4.1 / 4.2"}],"minor_comments":[{"comment":"The notation T^1_j is inconsistent with Table 1, which defines T^c_j; please use the latter consistently. Additionally, define h' and w' by explicit formulas that include padding and stride.","section":"Section 3.2.1"},{"comment":"The phrase 'zero packing' is never defined. If it refers to zero-initializing the output buffer, that is not the same as zero-padding the input; please clarify the terminology.","section":"Title / Abstract / Section 3.2"},{"comment":"Please report compiler flags, OpenMP thread count, CPU frequency/power settings, and whether timings are averaged over multiple runs. These details are important for interpreting the speedup numbers.","section":"Section 4.1"},{"comment":"The sentence 'reduces the total temporary memory by ci*Kh*kw' is ambiguous; Eq. (1) shows a reduction by a factor of approximately ci*kh*kw, so the text should say 'by a factor of approximately ...'.","section":"Section 3.4 / Eq. (1)"},{"comment":"The cited reference [6] is YOLO-Lite, not YOLOv3; please cite the original YOLOv3 paper (or clarify if YOLO-Lite is indeed the network used).","section":"Section 4.2 / Reference [6]"},{"comment":"The figure should identify which curve corresponds to AlexNet, VGG, and YOLO, either directly in the plot or in the caption.","section":"Figure 3"}],"recommendation":"major_revision","confidential_remarks":"The core idea is simple and potentially useful, and the stride-1 valid-convolution derivation is correct. However, the output-channel indexing error in Algorithm 1, the undefined stride/padding handling in the network-level experiments, and the incorrect parallel indexing in Algorithm 2 are serious technical issues. I would require a corrected algorithm description, explicit handling of padding and stride, and release of the implementation before further consideration."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Short version: this is a decent engineering paper about replacing im2col+GEMM with scalar-matrix multiplies over a single reused h×w′ buffer for channels-first CPU inference. The identity is just the definition of convolution, but the packaging is sensible and the memory reduction is real. What is new is the specific recipe: extract kw horizontal slices, shift vertically kh times, multiply by kernel scalars, accumulate into one buffer. For stride-1 valid convolution the algorithm is correct and the memory analysis (Eq. 1) checks out.\n\nThe soft spots are where the empirical claims live. The paper only defines the stride-1, no-padding case. It never says how stride and padding are folded in, yet AlexNet and YOLO have stride-2 and stride-4 layers, and VGG uses padding throughout. Your stress-test note is right: for stride s>1, the consecutive-row shift no longer samples the correct input rows without either strided access or a dense-then-subsample approach that multiplies MACs by s². So Table 2's speedups cannot be reproduced or even understood from the paper. The parallel pseudocode also has an indexing bug (O[λ*#n] does not allocate each thread a contiguous co/d block), and no code is released. On baselines, beating PyTorch's im2col+GEMM on a 4-core i7 is a modest bar; oneDNN or a tuned direct conv would be the real comparison for CPU inference.\n\nNone of this kills the core idea. If the authors scope the claims to stride-1 valid convolution (or describe their actual stride/padding mechanism) and release code, the recipe is a useful data point for embedded CPU kernels. The paper also correctly credits MEC and the channels-last direct-conv work; the citation pattern is clean.\n\nWho gets value: practitioners building CPU inference loops on NCHW tensors and researchers working on memory-efficient convolution. I would not cite it in my own work, but I would send it to a serious referee. It deserves a round of revision, not a desk rejection.","headline":"A clean stride-1 convolution recipe with a reused buffer, but the claimed network speedups rest on an undocumented stride/padding extension and thin baselines.","tokens_in":8519,"tokens_out":3411,"would_cite":false,"duration_ms":31050,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"SMM-Conv replaces im2col+GEMM with scalar-matrix multiplication into one reused buffer per thread, and reports end-to-end CPU speedups of 3.42x on AlexNet, 2.11x on VGG, and 2.00x on YOLO.","keywords":["convolution acceleration","CPU inference","im2col","scalar-matrix multiplication","memory-efficient convolution","channels-first layout","deep neural networks"],"falsifier":"Take the first convolutional layer of AlexNet (11x11 kernel, stride 4, padding 2) or a VGG 3x3 stride-1 padding-1 layer, run SMM-Conv against a reference convolution implementation on random input, and compare outputs exactly; a mismatch would show the published algorithm does not cover the configurations claimed in Table 2.","tokens_in":7284,"feed_emoji":"⚡","tokens_out":5323,"duration_ms":44642,"temperature":0.7,"pith_summary":"This paper tries to establish that a convolution can be computed on CPU without packing the image into a large matrix. The proposed method, SMM-Conv, slices each input channel into overlapping sub-matrices, shifts them down the kernel height, and accumulates scalar-weight products into a single reused buffer per thread. If the measurements are right, this simple recipe outperforms the standard im2col-plus-GEMM pipeline and a memory-efficient convolution baseline on AlexNet, VGG, and YOLO while using far less temporary memory. The practical payoff would be faster inference on mobile and low-power devices with existing, already-trained channels-first models.","feed_headline":"One reused buffer makes CPU convolution up to 3.4x faster","feed_subtitle":"SMM-Conv replaces im2col+GEMM with scalar-matrix multiplication, cutting memory and speeding AlexNet, VGG, and YOLO.","key_machinery":"The load-bearing mechanism is scalar-matrix multiplication with zero packing, driven by a 'shifting' operation. For a kernel of height $k_h$ and width $k_w$, the input is sliced into $k_w$ sub-matrices of size $h \\times w'$ (all rows, $w'$ consecutive columns); each slice is shifted down $k_h$ times so the rows align with the $k_h$ kernel rows, and each shifted $h' \\times w'$ window is multiplied by a single scalar kernel weight and accumulated into the output. Because all scalar-matrix products read from one contiguous $h \\times w'$ buffer that is reused across channels and offsets, the method needs only $d$ such buffers for $d$ threads and avoids the $c_i \\cdot k_h \\cdot k_w \\cdot h' \\cdot w'$ temporary matrix of im2col. The kernel is stored in $c_i \\times k_w \\times k_h \\times c_o$ layout so the scalar weights are accessed in the same order as the shifts.","core_discovery":"On the paper's own terms, the central discovery is that the standard im2col+GEMM pipeline is not the best way to compute a channels-first convolution on a CPU. Instead of copying every kernel-sized image block into a column of a large matrix and then calling a matrix-matrix product, SMM-Conv keeps the input in place: for each input channel it extracts $k_w$ horizontal slices of width $w'$, shifts each slice down $k_h$ times, multiplies the shifted $h' \\times w'$ windows by the corresponding scalar kernel weights, and accumulates into the output. The same $h \\times w'$ buffer is overwritten for every slice, so temporary memory is about one output-sized matrix per thread. The authors report that this approach runs the convolutional layers of AlexNet, VGG, and YOLO in 0.1348 s, 1.3535 s, and 0.2889 s respectively, corresponding to 3.42x, 2.11x, and 2.00x speedups over im2col+GEMM and faster than the memory-efficient convolution baseline.","pith_inferences":["Editorial extension: the published description covers stride-1 valid convolution only; real networks use stride and padding, so either the implementation contains undocumented preprocessing or the Table 2 speedups apply only to the subset of layers that match the described setting.","Editorial extension: the speedup mechanism implies the largest wins occur when im2col duplication is worst, namely small kernels, many input channels, and spatial sizes where $h' \\approx h$; the scalability plots support this, and a user should expect smaller gains for large-stride or heavily padded layers.","Editorial extension: the same buffer-reuse idea could be tested on depthwise convolutions and on transposed convolutions, where im2col-style packing is also memory-heavy; the paper does not address these."],"forward_implications":["The convolution is exact: the number of multiply-accumulate operations is the same as direct convolution, so the reported speedups come without accuracy loss or retraining.","Temporary memory falls from $c_i k_h k_w h' w'$ to $h w'$ per thread, roughly a factor of $c_i k_h k_w$ when $h' \\approx h$.","Memory use becomes independent of the number of input channels, which matters for low-power devices with tight memory budgets.","The method extends naturally to multi-threading: $d$ threads, each owning one $h \\times w'$ buffer and $c_o/d$ output maps, run in parallel with a synchronization point per slice."],"supporting_citations":[{"why":"Defines the memory-efficient convolution baseline that SMM-Conv must beat and supplies its published implementation.","marker":"[4]"},{"why":"Defines the im2col packing operation whose memory overhead and access pattern SMM-Conv replaces.","marker":"[23]"},{"why":"Supplies the direct-convolution and FMA arguments used to justify scalar-matrix multiplication and cache reuse.","marker":"[24]"},{"why":"Provides the deep-learning framework's C++ API used to build the im2col+GEMM baseline in the experiments.","marker":"[15]"},{"why":"Provides the optimized GEMM library that the im2col+GEMM baseline calls.","marker":"[7]"},{"why":"Supplies AlexNet, the first benchmark network for the end-to-end timing comparison.","marker":"[10]"},{"why":"Supplies VGG, the second benchmark network for the end-to-end timing comparison.","marker":"[20]"},{"why":"Supplies the YOLO-based benchmark network used in the end-to-end timing comparison.","marker":"[6]"}],"fun_headline_variants":["SMM-Conv: zero packing, scalar-matrix math, up to 3.4x faster","No im2col buffer: scalar-matrix convolution speeds CPUs 3.4x","In-place scalar-matrix multiplication makes convolutions 3.4x faster","SMM-Conv: reuse one buffer, skip im2col, speed AlexNet VGG YOLO","Scalar-matrix convolution cuts memory and yields 3.4x speedup"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The network-level speedups assume the undocumented extension of the stride-1, no-padding algorithm to the padding and stride settings of real network layers; if that extension is wrong, the end-to-end numbers do not hold.","fun_headline_variants_meta":{"raw":{"variants":["SMM-Conv: zero packing, scalar-matrix math, up to 3.4x faster","No im2col buffer: scalar-matrix convolution speeds CPUs 3.4x","In-place scalar-matrix multiplication makes convolutions 3.4x faster","SMM-Conv: reuse one buffer, skip im2col, speed AlexNet VGG YOLO","Scalar-matrix convolution cuts memory and yields 3.4x speedup"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000657,"raw_usage":{"total_tokens":2986,"prompt_tokens":901,"completion_tokens":2085,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":517,"completion_tokens_details":{"reasoning_tokens":1968}},"tokens_in":517,"tokens_out":2085,"duration_ms":12183,"temperature":1.0,"reasoning_tokens":1968,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-12T14:03:54.615931+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Take the first convolutional layer of AlexNet (11x11 kernel, stride 4, padding 2) or a VGG 3x3 stride-1 padding-1 layer, run SMM-Conv against a reference convolution implementation on random input, and compare outputs exactly; a mismatch would show the published algorithm does not cover the configurations claimed in Table 2.","supporting_citations":[{"cited_title":"Mec: memory-efficient con- volution for deep neural network","cited_arxiv_id":null,"evidence_quote":"Defines the memory-efficient convolution baseline that SMM-Conv must beat and supplies its published implementation."},{"cited_title":"High performance zero-memory overhead direct convolutions","cited_arxiv_id":null,"evidence_quote":"Supplies the direct-convolution and FMA arguments used to justify scalar-matrix multiplication and cache reuse."},{"cited_title":"Pytorch: An im- perative style, high-performance deep learning library","cited_arxiv_id":null,"evidence_quote":"Provides the deep-learning framework's C++ API used to build the im2col+GEMM baseline in the experiments."},{"cited_title":"Math kernel library https://software.intel.com/en- us/intel-mkl, 2015","cited_arxiv_id":null,"evidence_quote":"Provides the optimized GEMM library that the im2col+GEMM baseline calls."},{"cited_title":"Imagenet classification with deep convolutional neural net- works","cited_arxiv_id":null,"evidence_quote":"Supplies AlexNet, the first benchmark network for the end-to-end timing comparison."},{"cited_title":"Yolo- lite: a real-time object detection algorithm optimized for non-gpu computers","cited_arxiv_id":null,"evidence_quote":"Supplies the YOLO-based benchmark network used in the end-to-end timing comparison."}],"review_version":1}