{"id":"820b6d0f-23ed-4dc5-8a51-a78499723b60","arxiv_id":"2506.02233","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":0,"one_line_summary":"Current LLVM and MLIR auto-vectorizers produce slow Streaming SVE code because their cost models ignore streaming-mode overheads and memory hazards.","lead":"This paper shows that current compilers generate much slower code for Arm Streaming SVE than for standard NEON on most benchmark loops, despite four times wider SIMD units. It proposes cost-model and transform changes that could make automatic offload to these wider units profitable.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Single-platform measurements, plus lack of error bars and artifacts, make the central empirical claim fragile; a second SME implementation or named hardware is needed before the 0.32x-over-NEON result can generalize.","rationale":"The reader's weakest_assumption identifies exactly the same load-bearing concern: the generalization from one unnamed processor. The paper is honest and well-scoped as a position/call-to-action, and the proposed compiler extensions are plausible, but the empirical core must carry the argument that SSVE auto-vectorization is broadly unprofitable. Since that core is a single point measurement with no error analysis or second platform, the paper should not be accepted unconditionally. Conditional acceptance is appropriate: the authors should name the hardware, release data and scripts, and ideally add a second microarchitecture or an artifacts appendix. I do not find an internal inconsistency that would justify rejection; the strongest claim is plausible but not yet demonstrated beyond one platform. The concrete test would settle whether the concern lands.","tokens_in":6618,"tokens_out":1601,"duration_ms":14892,"concrete_test":"Obtain a second SME implementation with a different SVL (e.g., 256-bit or 1024-bit) or at least the same hardware with the processor named, rerun the TSVC suite under the same Clang 19.1.4 flags and iteration/size parameters as in Table 1, and compare SSVE-over-NEON geomeans. If the geomean remains near 0.3x and the five highlighted loops (s115, s132, s2233, s1281, s443) still show similar slowdowns, the central claim is supported on a second platform. If the geomean changes substantially or the slowdowns disappear, the paper's generalization fails. Additionally, the authors should publish raw data and scripts to make the single-platform result auditable.","verdict_should_be":"CONDITIONAL","load_bearing_attack":"The paper's central claim is that current compilers cannot auto-vectorize for SSVE profitably, backed by TSVC results showing SSVE at a geomean of 0.32x over NEON. All quantitative evidence comes from one unnamed 'ArchProcessor' with 128-bit NEON and 512-bit SVL (Table 1). The paper's own proposed cost model in Sec. 3.3 identifies platform-specific costs: mode-switch overhead, LSRT hazards, stack hazard paddings, prefetcher behavior, and synchronization penalties—none of which are portable without additional data. The 0.32x figure is therefore a single-microarchitecture observation, not a broad result. Internal consistency is not the problem: the 0.52x-over-scalar and 1.62x-over-scalar figures are arithmetically consistent (0.52/1.62 ≈ 0.32). The problem is external validity: no error bars, no repeated runs, no processor name, no artifacts release, and no second implementation. Since the paper's motivation and call-to-action rest on this slowdown being representative of SME-class hardware, the empirical foundation is the load-bearing weak point. If another SME implementation shows different SSVE/NEON ratios—e.g., due to different LSRT or synchronization behavior—the 0.32x geomean could be an artifact of this specific machine.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper examines whether current LLVM and MLIR compiler ecosystems can automatically generate profitable Arm Streaming SVE (SSVE) code for C/C++ workloads. Using a TSVC-based benchmark, a Mandelbrot kernel, and one SPEC 2017 benchmark, it reports that SSVE auto-vectorization produces a geomean of 0.32x the performance of NEON (0.52x over scalar), with 35% of TSVC loops auto-vectorized to SSVE and only 22 loops outperforming NEON. The paper attributes these results to inadequate compiler cost models and heuristics for streaming-mode execution, arguing that microarchitectural factors such as mode-switch costs, GPR/FPR and predicate synchronization, LSRT hazards, and prefetcher behavior must be incorporated. It then proposes design-level extensions to LLVM's Loop Vectorizer and VPlan cost modeling and to MLIR's Polygeist, SuperVectorizer, ArmSME/ArmSVE dialects, and VectorLegalization/EnableArmStreaming transforms. The proposals are not implemented, and all quantitative evidence comes from a single unnamed processor with 128-bit NEON and 512-bit SVL.","tokens_in":6840,"tokens_out":3821,"duration_ms":37568,"significance":"If the measured 0.32x SSVE-over-NEON geomean is representative of SME-class hardware, the paper would document an important negative result: current auto-vectorizers not only fail to exploit wider SSVE vectors but actively generate code that is far slower than NEON, motivating substantial cost-model and transform development. The paper has clear strengths: it uses the standard TSVC suite, carefully distinguishes compiler flags for SSVE/NEON/scalar modes, reports arithmetic consistency between geomeans, and grounds its proposed cost-model additions in concrete microarchitectural mechanisms (LSRT, stack-hazard padding, synchronization). It also avoids fitted parameters and is candid that its extensions are a 'path' rather than a finished implementation. However, the empirical foundation is a single unnamed platform with no error bars or artifact, and the proposed compiler enhancements are untested; as a result the paper currently reads more as a position/motivation study than as a complete systems evaluation.","major_comments":[{"comment":"All central quantitative claims (the 0.52x/0.32x geomeans, the 153x/268x maximum slowdowns, the 35% SSVE auto-vectorization rate, and the 'only 22 loops' observation) rest on measurements from a single unnamed 'ArchProcessor' with 128-bit NEON and 512-bit SVL. The paper provides no processor name, no information on whether the measurements came from silicon or a simulator, no repeated-run statistics or error bars, and no artifact. Because the paper's Sec. 1 and Sec. 4 conclusions generalize to SSVE/SME auto-vectorization as a whole, this single-platform evidence is load-bearing. The 0.32x geomean could plausibly be an artifact of this one implementation's LSRT and synchronization behavior, which the paper itself identifies as highly implementation-dependent. Please name the processor (or at least give a detailed microarchitectural description), report variance across runs, and ideally add a second SME implementation or a sensitivity study across SVL/configurations before claiming broad compiler inadequacy.","section":"Section 2, Table 1"},{"comment":"The proposed LLVM and MLIR enhancements—per-VPlan static streaming costs, SuperVectorizer scalable-vector support, ArmSME SSVE dialect operations, and VectorLegalization/EnableArmStreaming extensions—are described at design level only. There is no implementation, no prototype, and no experimental validation of any of these ideas. The paper is explicit that this is a 'path,' but Sec. 4 nonetheless concludes that 'compiler auto-vectorizers need updating' and presents the proposed techniques as the route to automatic code generation. Without at least a minimal prototype or measurements demonstrating that, e.g., adding a static streaming-mode cost to VPlan changes code generation decisions in the intended direction, the constructive contribution remains speculative. I recommend either implementing one of the proposed enhancements (even for a small set of loops) or explicitly recasting Sec. 3 as a research agenda and softening the corresponding conclusions.","section":"Section 3"},{"comment":"The stall-penalty figures reported for GPR/FPR synchronization—'17% for a single streaming loop, 61% for 100 concatenated loops'—are given without any measurement methodology: no benchmark description, no number of trials, no variance, and no comparison baseline. These numbers are used to justify the need for stack-hazard padding and, indirectly, the proposed cost-model changes, so they carry weight in the paper's argument. They should be presented with the same rigor as the TSVC results (e.g., name the microbenchmark, the processor, and the run-to-run spread), or the claims should be downgraded to qualitative observations.","section":"Section 3.3"}],"minor_comments":[{"comment":"The suite name is written as 'TSCV_2' in the header and 'TSCV_2 suite' in the text; the correct name is TSVC (Torrance/San Diego Vectorizing Compiler test suite). Please fix this typo throughout.","section":"Section 2"},{"comment":"The phrase 'we observe that the slowdown decreases linearly with increasing input sizes' is imprecise: the slowdown ratio appears to approach 1 (i.e., the overhead becomes relatively amortized) rather than decreasing linearly in the ratio. Please rephrase to describe the actual trend, e.g., 'the geomean slowdown decreases toward 1 as input size grows.'","section":"Section 2"},{"comment":"There is a typo in 'NEON shM ows' which should read 'NEON shows.'","section":"Section 2"},{"comment":"The sentence describing the SME unit's 'private L1 cache' and its relationship to the core's cache hierarchy is ambiguous: it is unclear whether the SME unit's L1 is coherent with the core's L1, and how this interacts with the later claim that the closest common cache is a shared cache. Please clarify the memory hierarchy in Fig. 1 or the accompanying text, since this bears on the LSRT and stack-hazard discussions in Sec. 3.3.","section":"Section 1"},{"comment":"Reference [4] is cited for the behavior of 'smstart sm'/'smstop sm' instructions; consider citing the Arm Architecture Reference Manual or the LLVM source directly for that instruction behavior, as a blog post may not be the most authoritative source for a journal paper.","section":"References"}],"recommendation":"major_revision","confidential_remarks":"The paper is honest and clearly written, but for a journal venue the evidential base is thin: one unnamed processor, no error bars, no artifact, and no implementation of the proposals. The central empirical claim about 0.32x geomean needs at least a named platform, run-to-run statistics, and ideally a second SME implementation or simulation study. The paper might be better suited to a workshop or position-track venue in its current form; for this journal, I would ask for substantial strengthening of the experimental methodology and a demonstration that at least one proposed cost-model change has the intended effect."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"This is a useful, honest position paper with a small empirical core. What's new is the data: TSVC and SPEC measurements of LLVM's streaming-mode auto-vectorization on real SME hardware, showing SSVE at a geomean of 0.32x over NEON and only 35% of TSVC loops auto-vectorized to SSVE. That's a concrete, reproducible-sounding negative result for a real problem—compilers today cannot profitably target streaming SVE, and the paper lists the missing cost-model features (mode-switch overhead, LSRT hazards, stack-hazard padding, prefetcher behavior) with enough specificity that someone could actually implement them.\n\nCredit where due: the arithmetic is internally consistent (0.52/1.62 ≈ 0.32), the paper is explicit that this is rudimentary compilation and the proposed enhancements are not implemented, and the architectural discussion (predicate sync tens of cycles, LSRT 1KB granularity, QPSK kernel improving with hazard avoidance) reads like real experience. The citation pattern is clean.\n\nThe soft spot is exactly what the stress-test note says: all quantitative evidence comes from one unnamed processor with 512-bit SVL. No error bars, no repeated runs, no artifact, no second implementation. Given the paper's own microbenchmarks show stall penalties ranging from 17% to 61% depending on loop count, it's very plausible that the 0.32x geomean is partly a property of this one microarchitecture—different LSRT behavior or sync overhead could move the ratio significantly. So the generalization 'SSVE auto-vectorization is broadly unprofitable' is really 'on this machine, with LLVM 19.1.4, it is unprofitable.' That is still worth saying, but it needs a second data point or at least named hardware and scripts.\n\nThe proposed compiler extensions are natural but completely unvalidated. No prototype, no measurement of whether adding streaming overhead to the VPlan cost model would actually change codegen decisions. The QPSK LSRT anecdote is suggestive, not systematic.\n\nWho's it for: compiler and HPC researchers working on SME/SSVE, and to a lesser extent anyone designing cost models for core-adjacent accelerators. It would be a solid workshop paper and a decent position paper for a main track, but only if the authors release artifacts and name the hardware, or ideally add a second implementation. I'd send it to review, with the expectation of major revisions focusing on external validity.","headline":"Useful, honest position paper on why streaming SVE auto-vectorization is currently poor, but the single-platform empirical basis and lack of any implementation of the proposed fixes keep it from being more than a motivated agenda.","tokens_in":7378,"tokens_out":2057,"would_cite":true,"duration_ms":18196,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Current auto-vectorizers are not equipped to offload to Arm's Streaming SVE mode.","keywords":["Auto-vectorization","Arm SME","Streaming SVE","LLVM","MLIR","Polygeist","Cost model","TSVC"],"falsifier":"Run the same clang SSVE and NEON builds from Table 1 on an SME platform with a different SVL, such as 256-bit or 2048-bit, or with a different shared-cache and LSRT design: if the TSVC SSVE-versus-NEON geomean reaches at least 1.0, or if more than half of the 146 loops beat NEON, the paper's claim that current auto-vectorizers cannot profitably target SSVE would fail on that hardware.","tokens_in":6425,"feed_emoji":"⚙️","tokens_out":9611,"duration_ms":79952,"temperature":0.7,"pith_summary":"The paper tries to establish that current LLVM and MLIR-based compilers are not capable of profitable auto-vectorization for Arm's Streaming SVE (SSVE) mode, the streaming-compatible subset of SVE2 that runs on the Scalable Matrix Extension (SME) unit. Forcing automatic vectorization to emit SSVE code, the authors measure a geomean of 0.32x relative to NEON across the 146-loop TSVC suite, with only 22 loops faster and worst-case 268x slowdowns. They attribute this to cost models that ignore streaming-mode overheads such as mode-switch costs, stack hazards, predicate synchronization, and load-store region table (LSRT) stalls. The paper then proposes concrete compiler changes—new block-level or plan-level costs in LLVM's loop vectorizer and new SSVE operations and legalization in MLIR—so that SSVE code is generated only when profitable.","feed_headline":"SSVE auto-vectorization runs 3x slower than NEON","feed_subtitle":"Compiler cost models ignore streaming-mode switch and sync overheads; TSVC loops show a 0.32x geomean.","key_machinery":"The load-bearing machinery is the loop vectorizer's cost model operating on scalable vector types. LLVM represents an SVE vector as $\\langle vscale \\times N \\times Ty \\rangle$, where $vscale$ is an unknown positive compile-time multiple, and its vectorization factor (VF) is $\\langle vscale \\times N \\rangle$; the cost model assumes speedup is monotone in vector size and checks only the smallest legal $vscale$. Against that, the paper places the SSVE-specific costs that the current model cannot see: the smstart/smstop mode switch, GPR/FPR stack-hazard stalls (mitigated by -aarch64-stack-hazard-size padding), predicate-register synchronization latency, LSRT hazard stalls at 1KB granularity, and the performance knee below which offload is unprofitable. In the MLIR path, the corresponding machinery is the SuperVectorizer transform plus the Vector, ArmSVE, and ArmSME dialects, where the proposed extension is to represent SSVE operations in the ArmSME dialect and let EnableArmStreaming and VectorLegalization lower them with a cost model, rather than the current fixed-width NEON-only lowering.","core_discovery":"The central discovery, on the paper's own terms, is that the compiler's profitability machinery for scalable vectorization is built around an assumption that does not hold for SSVE: that speedup is constant or strictly increasing as vector length grows. The LLVM loop vectorizer therefore performs a single cost analysis for the smallest scalable vector and has no representation of the costs that appear when SVE code executes on a disaggregated SME unit in streaming mode. Empirically, clang 19.1.4 forced into streaming auto-vectorization produces SSVE code that runs at a geomean of 0.52x over scalar and 0.32x over NEON on TSVC, and a 1.9x slowdown on SPEC 2017's mcf. The same cost-model gap is identified in the MLIR/Polygeist flow, whose SuperVectorizer and ArmSME legalization passes cannot currently generate scalable vectors or account for streaming overheads. The paper concludes that adding streaming-mode costs to the cost model, extending VPlan with a static streaming cost, and routing MLIR through ArmSME SSVE operations are the necessary updates.","pith_inferences":["Left implicit in the paper is that all quantitative results come from one unnamed processor, so the 0.32x geomean is a measurement of that chip's streaming-mode overheads, not a universal property of Arm SME hardware.","If the cost-model extensions are implemented, a natural next experiment is to compile TSVC with an added flat streaming cost per loop and measure how the SSVE-versus-NEON geomean moves toward 1.0; the paper's data imply such a tunable constant can be calibrated from the observed knee.","The same pattern seen historically with SIMD extensions—hand-written intrinsics precede compiler auto-vectorization—may repeat for SME: the paper's intrinsics-based Mandelbrot still trails NEON, suggesting offload overhead, not code quality, is what a compiler cost model must first recover.","The LSRT hazard guidance suggests a testable compiler optimization: place stack objects accessed by both the core and the SME unit in 1KB-aligned regions, then measure whether the QPSK kernel's 6x slowdown disappears."],"forward_implications":["LLVM's loop vectorizer must treat streaming overheads as block-level or plan-level constants in VPlan instead of per-instruction lookup costs, because the overheads are incurred once per streaming region.","A conservative default of NEON for gather/scatter loops, small trip counts, data-dependent addresses, and loops below the performance knee would avoid most of the 268x worst-case slowdowns measured on TSVC.","The MLIR/Polygeist flow can be extended to generate scalable vectors by teaching SuperVectorizer to emit scalable vector types and by lowering them through ArmSME SSVE operations rather than fixed-width NEON.","On the measured platform, only about 15% of TSVC loops (22 of 146) were profitable in SSVE, so even a corrected cost model would offload a minority of loops; the rest should stay in NEON or scalar mode."],"supporting_citations":[{"why":"Supplies the TSVC loop suite whose 146 loops are the paper's primary evidence that SSVE auto-vectorization underperforms NEON.","marker":"[9]"},{"why":"Provides the architectural characterization of SME with SVL as a power of two from 128 to 2048 bits that motivates the wider-vector expectation.","marker":"[12]"},{"why":"Defines SSVE mode and PSTATE.SM, the execution mode the paper targets.","marker":"[3]"},{"why":"Provides the Mandelbrot benchmark showing SSVE intrinsics at 0.35x over NEON and the linear amortization of offload cost with input size.","marker":"[6]"},{"why":"Supplies the SPEC 2017 mcf benchmark used to show a 1.9x SSVE slowdown over NEON on a real workload.","marker":"[5]"},{"why":"Documents the scalable vector type and vscale representation whose monotonic-speedup assumption the paper identifies as the core cost-model flaw.","marker":"[11]"},{"why":"Describes Polygeist, the MLIR-based C/C++ flow that the paper proposes to extend for SSVE generation.","marker":"[10]"},{"why":"Provides the MLIR infrastructure and its dialects (Vector, ArmSVE, ArmSME) on which the proposed SSVE lowering pipeline is built.","marker":"[7]"},{"why":"Underlies the GPR/FPR stack-hazard discussion and the -aarch64-stack-hazard-size padding heuristic for streaming-mode synchronization stalls.","marker":"[8]"},{"why":"Supplies the QPSK kernel from Arm RAL that shows a 6x SSVE slowdown over NEON and demonstrates the LSRT hazard issue.","marker":"[1]"}],"fun_headline_variants":["SSVE auto-vectorization 3x slower than NEON due to cost model gaps","Compiler cost models ignore streaming overhead, making SSVE 3x slower","Cost model blind spot makes SSVE auto-vectorization 3x slower than NEON","Streaming SVE auto-vectorization: 3x slowdown from missing sync costs","Why SSVE auto-vectorization is 3x slower: compiler cost model flaws"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The conclusion rests on measurements from a single unnamed processor with 128-bit NEON and 512-bit SVL, so it assumes that that chip's streaming-mode costs—mode switch, cache-coherence and stack-access stalls, and address-translation synchronization—are typical of Arm SME implementations.","fun_headline_variants_meta":{"raw":{"variants":["SSVE auto-vectorization 3x slower than NEON due to cost model gaps","Compiler cost models ignore streaming overhead, making SSVE 3x slower","Cost model blind spot makes SSVE auto-vectorization 3x slower than NEON","Streaming SVE auto-vectorization: 3x slowdown from missing sync costs","Why SSVE auto-vectorization is 3x slower: compiler cost model flaws"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000636,"raw_usage":{"total_tokens":2934,"prompt_tokens":947,"completion_tokens":1987,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":563,"completion_tokens_details":{"reasoning_tokens":1877}},"tokens_in":563,"tokens_out":1987,"duration_ms":12989,"temperature":1.0,"reasoning_tokens":1877,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-07T11:26:52.580985+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run the same clang SSVE and NEON builds from Table 1 on an SME platform with a different SVL, such as 256-bit or 2048-bit, or with a different shared-cache and LSRT design: if the TSVC SSVE-versus-NEON geomean reaches at least 1.0, or if more than half of the 146 loops beat NEON, the paper's claim that current auto-vectorizers cannot profitably target SSVE would fail on that hardware.","supporting_citations":[{"cited_title":"In: 2011 International Conference on Parallel Architectures and Compilation Techniques","cited_arxiv_id":null,"evidence_quote":"Supplies the TSVC loop suite whose 146 loops are the paper's primary evidence that SSVE auto-vectorization underperforms NEON."},{"cited_title":"https://developer.arm.com/documentation/109246/ 0100/SME-Overview/Streaming-SVE-mode","cited_arxiv_id":null,"evidence_quote":"Defines SSVE mode and PSTATE.SM, the execution mode the paper targets."},{"cited_title":"https://github.com/skeeto/ mandel-simd","cited_arxiv_id":null,"evidence_quote":"Provides the Mandelbrot benchmark showing SSVE intrinsics at 0.35x over NEON and the linear amortization of offload cost with input size."},{"cited_title":"https://llvm.org/devmtg/2021-11/slides/2021- OptimizingCodeForScalableVectorArchitectures.pdf","cited_arxiv_id":null,"evidence_quote":"Documents the scalable vector type and vscale representation whose monotonic-speedup assumption the paper identifies as the core cost-model flaw."},{"cited_title":"In: Proceedings of the ACM International Conference on Parallel Ar- chitectures and Compilation Techniques","cited_arxiv_id":null,"evidence_quote":"Describes Polygeist, the MLIR-based C/C++ flow that the paper proposes to extend for SSVE generation."},{"cited_title":"https://github.com/llvm/llvm-project/blob/ llvmorg-20.1.5/llvm/lib/Target/AArch64/AArch64FrameLowering.cpp, Stack hazards for GPR and FPR","cited_arxiv_id":null,"evidence_quote":"Underlies the GPR/FPR stack-hazard discussion and the -aarch64-stack-hazard-size padding heuristic for streaming-mode synchronization stalls."},{"cited_title":"https://learn.arm.com/ learning-paths/servers-and-cloud-computing/ran/, RAL library for various vector processing technologies","cited_arxiv_id":null,"evidence_quote":"Supplies the QPSK kernel from Arm RAL that shows a 6x SSVE slowdown over NEON and demonstrates the LSRT hazard issue."}],"review_version":1}