{"id":"9a6f1cb4-0fd7-4cad-a654-048ccb347eec","arxiv_id":"2607.25493","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":2,"one_line_summary":"A pluggable WebAssembly serverless framework keeps a fixed application API while letting operators swap runtimes, datastores, and caching policies to tune edge-cloud latency and memory.","lead":"WASP is a configurable serverless framework that lets operators swap WebAssembly runtimes and datastores while keeping application code unchanged, and the authors show it runs stateful functions on both cloud servers and a Raspberry Pi. The paper matters because it targets the one-size-fits-all problem of WASM FaaS at the edge, offering measurable latency and memory trade-offs.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"WASP's 'negligible overhead' claim is not yet supportable: its cold-start baseline is the standalone CLI (process-cold) while WASP is module-cold, and no ablation separates the adapter/HTTP/worker-pool cost.","rationale":"Good-faith reading: WASP is a legitimate modular-systems contribution; the architecture is clear, the integration-cost numbers are concrete, and RQ1/RQ4 do show different profiles across runtimes/datastores and portability to a Pi. The load-bearing point is the overhead claim in the abstract/conclusion. The reader's weakest assumption identifies the CLI-baseline problem, and I agree. The issue is not that WASP is slow; it may well be efficient. It is that the only evidence for 'negligible' is a comparison that cannot isolate WASP's own cost. This is an addressable experimental-design gap rather than an internal contradiction; CONDITIONAL remains the right verdict. Re-running with a native-adapter ablation and process-cold measurement would either support the claim or bound the overhead honestly. No code/data artifact is provided, which makes the baseline check more important, not less.","tokens_in":9797,"tokens_out":6207,"duration_ms":70477,"concrete_test":"Run the RQ2 workloads on a variant of WASP in which the WASM Runtime Adapter is replaced by a native-execution adapter: same fasthttp front-end, same 32-worker pool, same LRU caches and Storage Adapter, but the Fibonacci/Hash/Stateful logic is compiled natively and invoked through the adapter interface. Compare WASP-Native against WASP-Wasmtime-AOT, WASP-Wazero-AOT, and WASP-WasmEdge-Interp on Hash and Stateful/Redis warm executions, and also restart the WASP process before each 'cold' measurement so cold includes process boot. If WASP-WASM exceeds WASP-Native by more than the documented standalone-WASM-vs-native baseline on the same workload, the extra gap is WASP's framework overhead; if cold-start numbers flip when process boot is included, the current baseline is unfair.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim — 'negligible runtime overhead' (Abstract; §4.2 RQ2; §5) — rests on Table 1 comparisons against standalone runtimes invoked via their native CLI. Those baselines differ from WASP in at least two confounded ways. First, WASP's 'Cold' is the first request to an already-running service (engine initialized, worker pool live, caches empty); the CLI's 'Cold' includes OS process startup and one-shot engine initialization. The paper even credits WASP's lower cold-start times to 'amortizes runtime initialization across invocations' (§4.2), which confirms the quantities are not comparable as an overhead measurement. Second, the CLI has no HTTP front-end, no executor pool, and no cache; WASP has all three. Seeing WASP-Wasmtime-AOT Hash at 180.59ms vs Standalone-AOT at 187.73ms therefore cannot support 'no measurable cost'; it compares a differently structured end-to-end stack. The stateful case is worse: no standalone stateful baseline is reported at all, so get/put adapter overhead is unmeasured. Without an ablation of WASP's own adapter/front-end layer, the abstract's central overhead claim is under-determined.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper introduces WASP, a stateful WebAssembly serverless framework built from decoupled, pluggable components: a Function Manager with an executor pool and multi-level cache, a WASM Runtime Adapter, and a Storage Adapter exposing a uniform get/put state API. The authors argue that, by keeping these components interchangeable, WASP lets administrators swap WASM runtimes and datastores without changing application code, and that this configurability comes at negligible runtime overhead. The evaluation covers three runtimes (Wasmtime, Wazero, WasmEdge), two datastores (Redis, PostgreSQL), several execution modes, a comparison with Sledge, and a Raspberry Pi deployment. The central claims are that WASP's abstraction layer adds negligible overhead and that different configurations expose radically different latency/memory profiles, confirming adaptability across the edge-cloud continuum.","tokens_in":10054,"tokens_out":3564,"duration_ms":43593,"significance":"If the claims are sustained, WASP would make a useful contribution: it targets a real gap in today's WASM FaaS systems, which are typically monolithic and either stateless or tied to one runtime/backend. The paper's strengths include a genuinely modular architecture, adapter cost reported in lines of code (Wazero 245, Wasmtime 263, WasmEdge 317; Redis 53, PostgreSQL 76), concrete integration of three substantially different runtime execution models, and an edge deployment on a Raspberry Pi. The comparison against Sledge on lightweight workloads is informative and not circular. The main gap is that the 'negligible overhead' claim is not yet adequately supported by the evaluation methodology, so the paper's headline result requires stronger experimental evidence or a substantially qualified wording.","major_comments":[{"comment":"The central claim that WASP introduces 'negligible runtime overhead' is not supported by the chosen baselines. WASP's 'Cold' is the first request served by an already-running service (engine initialized, worker pool alive, caches empty), while the standalone CLI's 'Cold' includes OS process startup and one-shot engine initialization. The paper itself attributes WASP's lower cold-start time to amortizing runtime initialization across invocations, which confirms the two quantities are not measuring the same thing. Moreover, no standalone warm-start measurement is reported at all, and the stateful workload has no standalone stateful baseline, so the get/put adapter cost is unmeasured. A proper overhead assessment needs an ablation that isolates WASP's framework layer, e.g., WASP with and without adapters under identical process-warm/module-cold conditions, or a same-process baseline without","section":"§4.2, RQ2, Table 1"},{"comment":"All quantitative conclusions are based on point estimates without variance. For example, the paper states that 'on Hash, the two converge within 4% (180.6 vs. 187.7ms)' and concludes 'no measurable cost.' Without confidence intervals, standard deviations, or at least per-repetition data, a 4% difference cannot be distinguished from noise, especially with only 10 averaged runs for Hash. The same issue affects the warm-start speedup claims and the memory comparisons. Since RQ1 and RQ2 are the backbone of the paper's central claims, the tables need error bars or statistical significance measures.","section":"§4.2, RQ2, Table 1"},{"comment":"The 'negligible overhead' claim is unqualified with respect to memory. The evaluation reports that WASP's peak RSS exceeds the standalone runtime by 10–50 MB, and for the stateful AOT configurations the gap is much larger (e.g., WasmEdge AOT stateful at 180 MB vs. 41 MB for the interpreted configuration). On memory-constrained edge devices, this is not obviously 'negligible.' The paper should either restrict the overhead claim to execution time, or add a qualified statement that the memory overhead is an explicit cost of the framework's caching and worker-pool design, and discuss its impact on the edge deployment scenarios claimed in the introduction.","section":"§4.2, RQ2 and §5"}],"minor_comments":[{"comment":"The column headers are ambiguous. The table appears to contain two sets of Cold/Warm/RSS columns for Fibonacci and Hash, and later for Redis/PostgreSQL stateful runs, but the header row does not explicitly label which pair belongs to which workload or backend. This makes the table hard to read without inferring from the text.","section":"Table 1"},{"comment":"Figure 2 reports throughput and latency without confidence intervals or statistical significance annotations. Since the comparison with Sledge is a key RQ3 result, at least marking the spread across the 10 repetitions would help assess whether the observed differences are stable.","section":"§4.1, Figure 2"},{"comment":"The paper uses 'Cold' and 'Warm' without formal definitions in the main text. Specifically, it should state explicitly that WASP 'Cold' is first-request-without-cached-module while the process is already running, and that 'Standalone' has no warm variant. Adding a sentence or a footnote would prevent the misinterpretation that the paper is comparing the same start condition.","section":"§3.2, §4.2"},{"comment":"The Sledge comparison covers only CPU-bound stateless workloads. Since the paper's contribution is stateful execution, it would be useful to at least discuss why a stateful comparison is not possible (e.g., Sledge's lack of a state API) and whether the throughput results could be affected by state operations in a more realistic mixed workload.","section":"§4.2, RQ3"},{"comment":"Minor typos and formatting inconsistencies exist, such as the title being rendered as 'W ASP' in the header and the lack of a space in 'W ASM' in the abstract. The reference list also contains a few inconsistent name abbreviations; a final proofread would be helpful.","section":"General"}],"recommendation":"major_revision","confidential_remarks":"The paper's architecture and experimental breadth are promising, but the headline 'negligible runtime overhead' claim rests on a comparison between a process-warm WASP and a process-cold standalone CLI. This is a correctness concern for the central claim, not just a presentation issue. I would encourage the editor to require an ablation or an explicitly qualified statement before the paper is accepted; the current evidence is insufficient to support the abstract's wording."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Plainly: the \"negligible overhead\" claim in the abstract and §4.2 is not yet supportable. The cold-start baseline is the standalone runtime's native CLI, which includes OS process startup and one-shot engine initialization; WASP's \"cold\" is the first request to an already-running service with worker pool and caches up. That comparison measures different things. The paper even credits WASP's faster cold starts to amortizing runtime initialization across invocations, which confirms the mismatch. There is no ablation of WASP's own HTTP front-end, worker pool, or adapter layer, so the framework's actual cost is not isolated. For stateful workloads there is no standalone stateful baseline at all, so get/put adapter overhead is unmeasured. That weakens the central \"negligible\" claim.\n\nThat said, the paper has real merit. The contribution is a concrete, working system: a modular adapter architecture with a fixed state API (get/put), a two-level cache, and an executor pool, demonstrated across three WASM runtimes (Wasmtime, Wazero, WasmEdge) and two datastores (Redis, PostgreSQL). The evaluation does show radically different latency and memory profiles across configurations, and the warm-start speedups are large. The integration cost numbers (245–317 lines per runtime adapter, 53–76 lines per storage adapter) are useful. The comparison against Sledge is informative and honestly reports the inversion on CPU-heavy workloads. No circularity: the results compare against external baselines and an independent system.\n\nBeyond the baseline issue, results are reported without error bars, and only one server plus one Pi are used. The stateful overhead claim is especially thin. These are addressable rather than fatal. Also no artifacts or code are made available, which limits reproducibility.\n\nThis deserves a serious referee. I would send it to peer review, but with a request for an ablation that separates framework overhead from runtime performance, and ideally error bars and an artifact. The architecture and evaluation are solid enough for a systems venue after revision.","headline":"A useful modular stateful WASM FaaS framework with an honest configurability evaluation, but the 'negligible overhead' claim rests on a mismatched baseline and needs an ablation.","tokens_in":10559,"tokens_out":1828,"would_cite":true,"duration_ms":19930,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"WASP shows that stateful WebAssembly serverless applications can be made portable and configurable—runtimes and datastores swap without touching application code.","keywords":["WebAssembly","serverless computing","stateful FaaS","edge-cloud continuum","pluggable runtime","storage adapter","function caching","configurability"],"falsifier":"Run the same workload three ways: full WASP with a stub storage adapter, WASP with the HTTP layer and worker pool stripped out but adapters retained, and the standalone runtime CLI behind the same HTTP front-end. If full-WASP end-to-end latency exceeds the stripped version by more than the paper's reported overhead on a given benchmark, the 'negligible overhead' claim is falsified.","tokens_in":9701,"feed_emoji":"⚙️","tokens_out":4484,"duration_ms":45810,"temperature":0.7,"pith_summary":"This paper tries to establish that stateful serverless computing based on WebAssembly can be portable across the edge-cloud continuum and configurable to different hardware and workload constraints. It presents WASP, a framework that exposes a fixed get/put state API to applications while routing execution through a pluggable WASM runtime adapter and storage adapter. The framework's design lets system administrators swap the execution engine and the datastore with no application code changes. The paper's evaluation claims this abstraction adds negligible runtime overhead while exposing wide latency and memory profiles, including on a Raspberry Pi. A reader would care because existing WASM serverless systems are monolithic, stateless-oriented, and tied to a single execution engine.","feed_headline":"Swap runtimes and datastores without rewriting serverless apps","feed_subtitle":"WASP's decoupled adapters keep application code fixed while memory and latency profiles shift by up to orders of magnitude.","key_machinery":"The key mechanism is the adapter pattern in front of each pluggable component. The WASM Runtime Adapter exposes a uniform WASM Execution API that wraps compilation, instantiation, invocation, and memory-boundary traversal for runtimes as different as C/Rust JIT engines and pure-Go AOT runtimes, parameterizing the executor pool's concurrency model. The Storage Adapter exposes a uniform state management API (get/put) with a linear-memory sharing protocol for host functions, so the same .wasm binary can talk to Redis or PostgreSQL. The Function Manager's two-level cache—a raw-binary artifact cache with singleflight and an LRU compiled-module cache—makes warm starts nearly as fast as native exec","core_discovery":"The central discovery is an architecture, not a single algorithm: by strictly decoupling the Function Manager (with its executor pool and two-level cache) from the WASM runtime and storage backends through adapter interfaces, WASP makes the execution environment a deployment-time configuration. Applications compiled to WASM call get/put through a uniform state management API; the runtime adapter handles engine-specific memory and concurrency, and the storage adapter maps the API onto the chosen datastore. The paper reports that swapping these components yields radically different performance and memory profiles—for example, interpreted versus ahead-of-time compilation on the same runtime cha","pith_inferences":["If the overhead result survives an ablation that separates the HTTP front-end and worker pool from the adapter layer, the same architecture could support dynamic runtime selection per invocation, letting a scheduler move functions between interpreter and AOT engines based on observed load.","The singleflight artifact cache suggests a straightforward extension to multi-node deployments: a shared registry plus local caches could keep cross-node invocation stampede-free, something the paper does not test.","The uniform get/put state API could be extended with multi-key transactions or versioned reads without changing the portability argument, since the adapter would simply map richer semantics onto the backend.","The paper's comparison with the state-of-the-art framework leaves open how WASP behaves under skewed or bursty workloads; a test with non-uniform request arrival would locate the pragmatic ceiling of the FIFO scheduler."],"forward_implications":["A single application binary can be deployed unchanged from server-class machines to low-memory edge devices; the paper demonstrates this on a Raspberry Pi.","Swapping execution modes changes cold-start latency by orders of magnitude, letting operators choose the right CPU-to-memory trade-off per workload.","Warm execution via the module cache lands within a small constant factor of native Go (0.08ms for Fibonacci on Wasmtime-AOT), so per-request overhead is dominated by the runtime, not WASP.","Datastore choice becomes a configuration decision with measurable effect on warm stateful calls (Redis ~0.19ms vs PostgreSQL ~1.49ms on Wasmtime-AOT).","Adding a new runtime or datastore costs only the adapter code (245–317 lines for runtimes, 53–76 for stores), so the platform can track the WASM ecosystem cheaply."],"fun_headline_variants":["Swap runtimes and datastores per deployment, no code rewrite","WASP: pluggable runtime and storage for stateful serverless","Configurable stateful serverless: swap components at deploy time","Edge-cloud serverless: change runtime and store without app edits","Stateful WASM serverless with swappable runtimes and stores"],"cache_read_input_tokens":2304,"weakest_assumption_plain":"The 'negligible overhead' conclusion assumes the standalone runtime CLI is a fair baseline that isolates WASP's cost, but that CLI has no HTTP front-end, worker pool, or cache, so the comparison may understate WASP's true overhead.","fun_headline_variants_meta":{"raw":{"variants":["Swap runtimes and datastores per deployment, no code rewrite","WASP: pluggable runtime and storage for stateful serverless","Configurable stateful serverless: swap components at deploy time","Edge-cloud serverless: change runtime and store without app edits","Stateful WASM serverless with swappable runtimes and stores"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000466,"raw_usage":{"total_tokens":2133,"prompt_tokens":690,"completion_tokens":1443,"prompt_tokens_details":{"cached_tokens":256},"prompt_cache_hit_tokens":256,"prompt_cache_miss_tokens":434,"completion_tokens_details":{"reasoning_tokens":1352}},"tokens_in":434,"tokens_out":1443,"duration_ms":11461,"temperature":1.0,"reasoning_tokens":1352,"cache_read_input_tokens":256,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-01T02:14:26.744603+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run the same workload three ways: full WASP with a stub storage adapter, WASP with the HTTP layer and worker pool stripped out but adapters retained, and the standalone runtime CLI behind the same HTTP front-end. If full-WASP end-to-end latency exceeds the stripped version by more than the paper's reported overhead on a given benchmark, the 'negligible overhead' claim is falsified.","supporting_citations":[],"review_version":1}