{"id":"ac34ae7a-8fb8-4971-90fb-b3eb34a36905","arxiv_id":"2507.19723","paper_version":1,"verdict":"REJECT","confidence":"HIGH","novelty_score":2.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":0,"one_line_summary":"A consumer-laptop benchmark shows CUDA matrix multiplication outpacing a naive sequential C++ baseline by up to 593x, but the claimed shared-memory optimization is absent from the supplied code.","lead":"This paper benchmarks naive C++ matrix multiplication against an OpenMP-parallel CPU version and a CUDA GPU version on a consumer laptop, reporting a 593x GPU speedup over the sequential baseline at 4096x4096. It illustrates the well-known scaling advantage of GPU parallelism, though the GPU kernel described as shared-memory optimized does not match the code supplied in the appendix.","discovery_kind":"replication","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Appendix A kernel contradicts the paper's shared-memory claim: the printed CUDA kernel (Listing 1, lines 58-69) has no `__shared__` memory, yet the paper says shared-memory tiling produced all results.","rationale":"The reader's weakest assumption is correct and is the single most load-bearing issue. The paper's reproducibility claim (Appendix A) is directly falsified by the printed kernel: no shared memory is declared or used. This is not a style issue; it means the experimental result cannot be traced to a concrete implementation. A secondary support is that the GPU time of 663 ms for the printed naive kernel is implausibly fast on a GTX 1650 Mobile given the memory traffic, though I would not rely on this alone. The CPU baseline being naive is a further weakness, but it is secondary to the artifact contradiction. Therefore the REJECT verdict stands; no adjustment is needed. I propose a concrete check that would settle the concern: run the printed code and compare.","tokens_in":10621,"tokens_out":5103,"duration_ms":56536,"concrete_test":"Compile and run the exact Appendix A code on the stated Lenovo IdeaPad Gaming 3 with Ubuntu 24.04, CUDA 12.2, and -O3, and compare the measured GPU times with Table 1. Also grep the kernel for `__shared__` or `extern __shared__`; the printed kernel contains neither. If the 4096 GPU time exceeds ~663 ms by a large margin, or if the source is accepted as the benchmark code, then the shared-memory claims and reported speedups are contradicted by the supplied artifact.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central empirical result is not reproducible from the provided artifact. Section 3.1.3 and the abstract state that the CUDA implementation uses shared memory tiling to reduce global memory traffic. Section 5.3 explicitly says the kernel 'effectively uses shared memory.' However, Appendix A's 'complete source code used to generate all performance data' contains a naive kernel that allocates no shared memory, has no `__syncthreads()`, and simply loops over k in global memory for each output element. If this is the code that was run, the methodology does not match the described algorithm; if the benchmark used a different, tiled kernel, the printed code cannot reproduce Table 1. A second inconsistency reinforces this: the paper claims the GPU timing includes memory allocation on the device, but the code places `cudaMalloc` before `start_gpu` (lines 105-111), so allocation is excluded. Either way, the identity between the described implementation, the printed source, and the measured timings is broken, so the headline 593x/45x speedups are unverifiable. The CPU baseline is also only a naive OpenMP loop, not an optimized BLAS-level implementation, so calling it 'optimized' overstates the comparison; however, the primary load-bearing defect is the kernel/code mismatch.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"This manuscript reports an empirical benchmark of three square matrix multiplication implementations—a sequential C++ loop, an OpenMP-parallel version using collapse(2), and a CUDA kernel—on a Lenovo IdeaPad Gaming 3 with an 8-core Ryzen 7 5800H and a GTX 1650 Mobile. Execution times for N = 128 to 4096 are tabulated, and speedups are computed as ratios of wall-clock times. The headline results are approximately 593x GPU speedup over the sequential baseline and 45x over the parallel CPU at N=4096. The paper interprets these as evidence of GPU dominance for data-parallel workloads on consumer hardware and states that the CUDA kernel uses shared-memory tiling.","tokens_in":10744,"tokens_out":6994,"duration_ms":81198,"significance":"The contribution is purely empirical and would be a modest but useful data point if internally consistent: it uses a clear speedup metric, supplies a benchmark harness, and reports a hardware/software configuration that is easy to replicate. The paper also connects its CPU-vs-GPU comparison to the earlier caveat by Lee et al. about CPU-side optimization. However, the manuscript's central claim is not independently verifiable because the printed source code contradicts the described algorithm. Since the entire conclusion rests on the measured timings, this inconsistency is disqualifying for the submitted version.","major_comments":[{"comment":"The paper claims the CUDA implementation 'makes extensive use of on-chip shared memory' and that the measured GPU time corresponds to this optimized kernel. The complete source in Appendix A contains no __shared__ declarations, no __syncthreads(), and each thread computes one output element by looping over k with global accesses A[row*size+k] and B[k*size+col]. Because the appendix says this is 'the complete C++/CUDA source code used to generate all performance data,' the reported Table 1 cannot be reproduced from the described tiled kernel. If the actual benchmark used a different tiled kernel, the appendix is not the code that was run; if it used this kernel, the shared-memory claims in the abstract and §5.3 are false. Either way the central empirical result is unverifiable.","section":"Appendix A, Listing 1 (lines 58-69); §3.1.3; Abstract"},{"comment":"The methodology states that the GPU time is comprehensive and includes device memory allocation. In the supplied code, the three cudaMalloc calls appear before 'auto start_gpu', and the timed region covers only the two host-device transfers and the kernel launch. The timing convention therefore excludes allocation, contrary to the text. The reported GPU wall-clock times should be recomputed with a uniform convention (including or explicitly excluding allocation) so that the speedup ratios have a well-defined meaning.","section":"§3.2 vs Appendix A lines 105-119"},{"comment":"The abstract and §5.1 describe the OpenMP version as an 'optimized parallel CPU' implementation, but the printed source is the same naive triple loop with a collapse(2) pragma. It does not use SIMD vectorization, cache blocking, or a vendor-tuned BLAS routine. Consequently, the 45.73x 'GPU vs. optimized parallel CPU' speedup at N=4096 is a comparison against a naive OpenMP loop, and the wording overstates what the experiment demonstrates. The paper should either replace the CPU baseline with a genuinely optimized implementation (e.g., an optimized BLAS library) or reword the claims to say naive OpenMP.","section":"Abstract; §3.1.2; §5.1"}],"minor_comments":[{"comment":"The table reports single timings without repetitions, standard deviations, or a statement that values are medians. On a laptop with thermal and frequency variation, at least three to five runs per size should be reported to support the speedup ratios.","section":"§3.2 / Table 1"},{"comment":"The listing appears corrupted in the preprint (identifiers such as 'c u d a _ r u n t i m e .h', 'co ll aps e (2)', and 'bl ock Id x' contain spurious spaces). If this is not simply a text-extraction artifact, the actual source code in the final version must be verbatim compilable.","section":"Appendix A"},{"comment":"The comparison with Lee et al. would be more informative if the manuscript also measured a vendor-optimized CPU BLAS (e.g., OpenBLAS or Intel MKL) and possibly cuBLAS, since these are the standard reference points for 'optimized' performance.","section":"§5.2"}],"recommendation":"reject","confidential_remarks":"The contradiction between the stated algorithm and the supplied code is not a stylistic issue; it undermines the reproducibility of the headline numbers. If the authors had provided the actual tiled kernel and rerun the benchmarks, a revision could be considered, but as submitted I do not see a path to acceptance without new experimental work."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Short version: this is a clear, honest-in-intent benchmark of matrix multiplication on a consumer laptop, confirming the well-established result that a discrete GPU beats a multi-core CPU on dense matmul and that the gap grows with size. The numbers in Table 1 are internally plausible for a naive CUDA kernel. The paper even handles the small-matrix case well, noting the OpenMP overhead at 128x128.\n\nWhat the paper does well: it frames itself as a contemporary validation of Lee et al. (2010) rather than claiming a new phenomenon. The writing is straightforward, the experimental setup is described in enough detail to identify the hardware and toolchain, and the reported scaling trends match what the literature would predict.\n\nThe soft spots are not minor. The abstract, Section 3.1.3, and Section 5.3 all say the CUDA kernel uses shared-memory tiling. Appendix A, explicitly labeled 'complete source code used to generate all performance data,' contains a plain element-per-thread kernel with no __shared__ memory, no __syncthreads(), and a simple k-loop over global memory. If that code was run, the described algorithm was not benchmarked. If a different, tiled kernel was run, the printed code cannot reproduce the timings. Either way the central empirical claim is unverifiable from the paper as submitted. This is not a stylistic quibble; it is a contradiction between the method description and the artifact.\n\nTwo more issues amplify this. The benchmarking protocol says GPU time includes device memory allocation, but the code calls cudaMalloc before start_gpu, so allocation is excluded. That is a smaller error, but it shows the timings need scrutiny. And the 'optimized parallel CPU' baseline is a naive OpenMP loop with no tiling, no SIMD, and no BLAS-level optimization, so the headline 45x GPU-vs-CPU figure is against a weak baseline. There are also no repeated trials or error bars, so the single-run wall-clock numbers carry unknown variance.\n\nBottom line: the field conclusion the paper reaches is almost certainly correct, but this manuscript cannot support it because the evidence is not reproducible. The fixes are straightforward — supply the actual kernel, correct the timing description, strengthen the CPU baseline, and run repeated trials — but they are not minor edits. As submitted, I would not send this to a referee; I would desk reject with an invitation to resubmit after major revision. A corrected version would be a modest but usable data point for practitioners comparing consumer hardware, though it would still be low-novelty.","headline":"A readable replication of a well-known GPU-vs-CPU result whose headline numbers are not reproducible from the printed source; the shared-memory kernel described in the text is not the kernel in Appendix A.","tokens_in":11430,"tokens_out":3323,"would_cite":false,"duration_ms":37417,"reading_group":"no","serious_thinker":"no","would_accept_peer_review":false},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":["68W10","65Y05","68M20"],"pacs":[],"model":"deepseek-v4-flash","headline":"A mobile GPU multiplies 4096x4096 matrices 45x faster than a 16-thread CPU.","keywords":["matrix multiplication","GPU acceleration","CUDA","OpenMP","heterogeneous computing","performance benchmarking","high-performance computing","speedup measurement"],"falsifier":"Recompile and run the printed code on the same laptop (Ryzen 7 5800H, GTX 1650 Mobile, CUDA 12.2, -O3) and compare the measured 4096x4096 GPU time to the reported 663.24 ms; also inspect whether the kernel allocates any `__shared__` memory. A zero-shared-memory kernel or a large time difference would indicate that the printed code is not the code that was benchmarked, or that the shared-memory description is inaccurate.","tokens_in":10328,"feed_emoji":"⚡","tokens_out":5905,"duration_ms":59962,"temperature":0.7,"pith_summary":"This paper tries to establish, by direct measurement on a consumer laptop, that a discrete GPU executes dense matrix multiplication far faster than either a single CPU core or a 16-thread multi-core CPU, and that the advantage grows with matrix size. It reports that for a 4096x4096 matrix of 32-bit floats, a CUDA implementation on a GTX 1650 Mobile took about 663 ms, versus about 393 s for a sequential C++ loop and about 30 s for an OpenMP loop, giving speedups of roughly 593x and 45x, respectively. The paper also documents a small-matrix regime where OpenMP overhead makes the parallel CPU slower than the sequential baseline. If the measurements are right, they quantify how much of the parallel-computing gap is accessible to ordinary hardware for a foundational scientific kernel.","feed_headline":"Laptop GPU beats 16-thread CPU by 45x on matrix math","feed_subtitle":"On a 4096x4096 matrix, CUDA takes 663 ms versus 30 s for OpenMP on the same laptop.","key_machinery":"The mechanism is the choice of parallel execution model. The sequential baseline is a plain nested loop; the CPU parallel version adds `#pragma omp parallel for collapse(2)` to fuse the two outer loops across 16 threads; the GPU version is a CUDA kernel that the text describes as cooperatively loading tiles of A and B into shared memory to cut global-memory traffic, though the code listing in Appendix A contains no shared memory and directly reads global memory in the inner loop. The performance metric that carries the argument is the speedup ratio S = T_seq / T_parallel, and the reported scaling of this ratio with matrix size is what the paper uses to conclude that GPU advantage grows with workload.","core_discovery":"The central claim is that on the tested heterogeneous system, the GPU's many-core throughput dominates as problem size grows. The authors implement three versions of the same triple-nested loop and measure wall-clock time including GPU memory transfers. Their data show the GPU speedup over the sequential CPU climbing monotonically from about 8x at 128x128 to about 593x at 4096x4096, while the OpenMP speedup plateaus near 12-14x. They interpret the 45x GPU-over-OpenMP figure as the fairer comparison between optimized implementations, and read it as consistent with earlier results that narrowed the '100x GPU vs CPU' gap when both sides are tuned.","pith_inferences":["Editorial: If the printed kernel is authoritative, the reported GPU timings were produced by a straightforward global-memory kernel, not the shared-memory tiled kernel described in the methodology; a properly tiled kernel would likely be faster, so the paper may understate the GTX 1650's ceiling.","Editorial: The sequential baseline's 393 s for 4096x4096 suggests the C++ loop did not exploit cache blocking or SIMD; a tuned BLAS library on the same CPU would shrink the denominator and could reduce the headline speedup considerably.","Editorial: The claimed monotonic scaling of GPU speedup implies that for even larger matrices (8192x8192 and beyond, within VRAM limits) the GTX 1650 would continue to widen its lead, a prediction the paper does not test.","Editorial: A reader wanting to extend the work could rerun the benchmark with cublasSgemm as the GPU baseline to see how close the hand-written kernel comes to the vendor-optimized library."],"forward_implications":["For large dense matrix multiplication on consumer hardware, a GPU can deliver hundreds-fold speedup over a single-threaded CPU and tens-fold over a 16-thread CPU, making GPU offload worthwhile even on a laptop.","The GPU advantage is workload-dependent: it grows super-linearly with matrix size, so small kernels may run faster on the CPU despite the presence of a GPU.","End-to-end GPU time, including host-device copies, remains far below CPU time at scale, so the transfer overhead is not a disqualifier for large matrices.","Using an optimized parallel CPU as the baseline, rather than a sequential loop, changes the headline from roughly 593x to roughly 45x, a more realistic framing for the architectural comparison.","The same measurement protocol can be applied to other basic kernels to map which computations benefit from GPU offload on commodity systems."],"supporting_citations":[{"why":"Supplies the shared-memory and coalescing optimization principles the CUDA kernel is claimed to apply.","marker":"[5]"},{"why":"Defines the CUDA execution and memory model on which the GPU implementation is built.","marker":"[8]"},{"why":"Provides the '100x GPU vs CPU' debunking baseline that the paper uses to frame its 45x optimized-to-optimized comparison.","marker":"[9]"},{"why":"Defines the OpenMP API and the collapse clause used for the parallel CPU version.","marker":"[7]"},{"why":"Prior GPU matrix multiplication analysis whose finding of growing GPU advantage with matrix size this work confirms.","marker":"[2]"}],"fun_headline_variants":["GPU 593x faster than single-core CPU on 4096x4096 multiply","Laptop GPU beats optimized CPU by 45x in matrix multiplication","Matrix math: GPU dominates CPU, 45x at 4096x4096","Consumer GPU achieves 593x speedup over sequential CPU","GPU outruns multi-core CPU 45x on large dense matrices"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The entire experimental result rests on the assumption that the source code printed in Appendix A is exactly the code that produced Table 1, since the printed CUDA kernel does not use shared memory despite the paper's description of a shared-memory tiled kernel.","fun_headline_variants_meta":{"raw":{"variants":["GPU 593x faster than single-core CPU on 4096x4096 multiply","Laptop GPU beats optimized CPU by 45x in matrix multiplication","Matrix math: GPU dominates CPU, 45x at 4096x4096","Consumer GPU achieves 593x speedup over sequential CPU","GPU outruns multi-core CPU 45x on large dense matrices"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000281,"raw_usage":{"total_tokens":1655,"prompt_tokens":927,"completion_tokens":728,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":543,"completion_tokens_details":{"reasoning_tokens":644}},"tokens_in":543,"tokens_out":728,"duration_ms":6831,"temperature":1.0,"reasoning_tokens":644,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-06T14:05:09.629437+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Recompile and run the printed code on the same laptop (Ryzen 7 5800H, GTX 1650 Mobile, CUDA 12.2, -O3) and compare the measured 4096x4096 GPU time to the reported 663.24 ms; also inspect whether the kernel allocates any `__shared__` memory. A zero-shared-memory kernel or a large time difference would indicate that the printed code is not the code that was benchmarked, or that the shared-memory description is inaccurate.","supporting_citations":[{"cited_title":"Optimization principles and application performance evaluation of a multithreaded gpu using cuda,","cited_arxiv_id":null,"evidence_quote":"Supplies the shared-memory and coalescing optimization principles the CUDA kernel is claimed to apply."},{"cited_title":"CUDA C Programming Guide,","cited_arxiv_id":null,"evidence_quote":"Defines the CUDA execution and memory model on which the GPU implementation is built."},{"cited_title":"Debunking the 100x gpu vs. cpu myth: an evaluation of throughput computing on cpu and gpu,","cited_arxiv_id":null,"evidence_quote":"Provides the '100x GPU vs CPU' debunking baseline that the paper uses to frame its 45x optimized-to-optimized comparison."},{"cited_title":"Openmp: an industry standard api for shared-memory programming,","cited_arxiv_id":null,"evidence_quote":"Defines the OpenMP API and the collapse clause used for the parallel CPU version."},{"cited_title":"GPU computing performance analysis on matrix multiplication,","cited_arxiv_id":null,"evidence_quote":"Prior GPU matrix multiplication analysis whose finding of growing GPU advantage with matrix size this work confirms."}],"review_version":1}