{"id":"ba8a42ac-3449-4235-a9db-3fa16a3c85d2","arxiv_id":"2509.01966","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":1,"one_line_summary":"OASIS offloads filters, aggregates, sorts, and array-aware expressions onto a two-level object storage stack, reducing SQL query times in HPC analytics by up to 32.7% versus an emulated COS baseline.","lead":"This paper presents OASIS, a storage system that runs parts of SQL queries inside storage to cut data movement in scientific analysis. It reports up to 32.7% speedups over a single-layer offloading setup and up to 70.6% over Spark without offloading.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"32.7% speedup claim is measured against an emulated FE-only 'COS' configuration with assumed full operator support, not against any actual COS product; the central comparison is therefore unverified.","rationale":"The reader's weakest_assumption focuses on SODA's coefficient/histogram estimates. That is a real risk for the SODA contribution, but it is not the most direct threat to the abstract's headline number: the 32.7% improvement is reported for Q2, a filter+project query that §V-F says 'does not undergo plan splitting.' For Q2, SODA's split-point choice is not the deciding factor; the deciding factor is that OASIS executes at OASIS-A while the 'COS' comparison executes at OASIS-FE. Therefore the load-bearing question is whether the 'COS' configuration actually represents existing COS systems. The paper defines it as an emulation and explicitly assumes full operator support, so the 32.7% figure is a comparison to a hypothetical idealized COS, not to MinIO/Ceph/SkyhookDM. The paper's own I/O measurements (OASIS slower than MinIO for GET at 1GB) show the emulation cannot be assumed to bracket real COS performance. I am not claiming the system does not work; I am claiming the headline comparative result is unverified. The reader already marked the paper CONDITIONAL and flagged the emulation caveat in the rationale, so I do not propose moving the verdict.","tokens_in":20674,"tokens_out":8656,"duration_ms":94513,"concrete_test":"Run Q2 from Table IV on the same 13GB DeepWater-Impact dataset, same Spark 4.0.0 cluster, same network, with the comparison target being an actual existing COS: MinIO Select (or Ceph RGW S3 Select) serving the same Parquet objects from the same OASIS storage node, and Spark reading the SELECT result via its normal connector. Measure end-to-end query time for OASIS and for the real COS. If the OASIS speedup over the real COS is materially below 32.7% (or negative), the headline claim as stated is not established; repeating with Ceph/SkyhookDM would confirm the sensitivity.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central performance claim ('up to a 32.7% performance improvement over Spark configured with existing COS-based storage systems') is not yet supported by a measurement against any existing COS. The comparison configuration in §V-A is defined as: 'COS: Emulates how OASIS would operate under the computation model of COSs by executing all operators at the OASIS-FE.' This is an FE-only variant of OASIS itself, not MinIO Select, Ceph S3 Select, or SkyhookDM. §V-C1 then adds: 'In scalar-based query evaluation, we assume that COS supports all candidate operators, as operator support may vary across systems.' Real COS systems do not have uniform operator support; MinIO/Ceph S3 Select lack aggregate/sort and array expressions, while SkyhookDM lacks aggregate/sort and computed projections. Because the 32.7% figure is from Q2, a filter+project query that actual COS systems can execute, this assumption is not harmless. The emulation reuses OASIS's DuckDB executor and storage stack, so it isolates the hierarchical-execution effect but cannot validate the abstract's comparative claim. The direction of bias is also unclear: §V-B shows OASIS GET throughput is 605.6 MB/s vs MinIO's 1080 MB/s at 1GB objects, so an actual COS may perform better or worse than the emulation. Without running the query against a real COS product, the headline speedup over 'existing COS-based storage systems' is unverified.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes OASIS, a computation-enabled object storage system that offloads SQL operators to storage arrays and a frontend node, using a cost-based plan decomposition algorithm (SODA) to split query execution between the OASIS-A and OASIS-FE. The authors build a prototype with SPDK and DuckDB, integrate it with Spark via Substrait IR, and evaluate on three scientific workloads (Laghos, DeepWater Impact, CMS Open Data). They report up to 70.59% speedup over a traditional Spark baseline and up to 32.7% speedup over a configuration labeled 'COS,' which emulates single-layer execution by running all operators at the OASIS-FE.","tokens_in":21029,"tokens_out":3498,"duration_ms":38333,"significance":"If the claims were fully supported, the paper would make a useful contribution: it demonstrates a working hierarchical execution path for storage-side query offloading, with support for array expressions and Arrow output. The prototype is real; the queries come from public HPC benchmarks; SODA's overhead is measured (126 ms estimation, 1.81 s decomposition); and the selectivity experiments provide a clear characterization of when offloading wins. The main weakness is that the headline comparison against existing COS systems is not actually carried out against any existing COS product. This is a central, load-bearing issue that must be addressed before the comparative claim can be accepted.","major_comments":[{"comment":"The abstract claims 'up to a 32.7% performance improvement over Spark configured with existing COS-based storage systems,' but the 'COS' configuration is defined in §V-A as an emulation in which all operators execute at the OASIS-FE. This is not MinIO Select, Ceph S3 Select, or SkyhookDM. The scalar-query evaluation assumes COS supports all candidate operators, yet real COS systems have limited operator support (MinIO/Ceph S3 Select lack aggregate/sort and array expressions; SkyhookDM lacks aggregate/sort and computed projections). Q2, which yields the 32.7% figure, is a filter+project query that real COS systems can execute, so the assumption is not harmless. Moreover, §V-B shows OASIS GET throughput is 605.6 MB/s vs MinIO's 1,080 MB/s at 1 GB objects, so an actual COS could perform better or worse than the emulation. Without a measurement against a real COS product, the headline compar","section":"§V-A, §V-C1, Abstract"},{"comment":"SODA is the core contribution, but its effectiveness is evaluated on a single query (Q1). The five static split configurations in Figure 10(b) are all internal configurations of OASIS, and SODA's selection of cfg4 is compared only within the same prototype. No query with array-based predicates (Q4) is used to validate the SAP strategy, and no sensitivity analysis is provided for the cost-model parameters (e.g., histogram sampling ratio 0.5–5%, per-operator coefficients, or the one-way transfer assumption). Given that the paper's central claim is that SODA 'correctly selects the optimal split point,' the evidence is narrow. At minimum, the paper should broaden the SODA evaluation to include array-based queries and report how sensitive the split choice is to the estimated coefficients.","section":"§V-F, Figure 10"},{"comment":"The cost model that drives SODA assumes (a) one-way data transfer from OASIS-A to OASIS-FE without return traffic, and (b) that histogram-based selectivity estimates from 0.5–5% sampled histograms accurately predict output sizes. These assumptions are load-bearing: if a selectivity estimate is wrong, the chosen split point could be suboptimal, and the reported speedups could be artifacts of the chosen dataset. The paper does not validate the one-way transfer assumption or report the error in the histograms' selectivity estimates. A sensitivity analysis over sampling ratios and a validation of the coefficient model against actual intermediate sizes is needed to support the optimality claim.","section":"§IV-G2, §V"}],"minor_comments":[{"comment":"Typo: 'Mote that' should be 'Note that.'","section":"§V-C1"},{"comment":"Section heading contains 'Exisiting' — should be 'Existing.'","section":"§III-B"},{"comment":"The grammar in 'equipped 1 NVMe SSD' is awkward; revise to 'equipped with one NVMe SSD.'","section":"§V-A"},{"comment":"The hatch patterns in Figure 10(b) are hard to distinguish in grayscale; consider adding value labels or using distinct fills.","section":"Figure 10"},{"comment":"The decrease in execution time from 1,000,000 to 10,000,000 records is explained by an increased partition count; this should be stated before the figure is discussed, not after.","section":"§V-D"},{"comment":"Some references lack consistent metadata (e.g., [34] is a URL-only entry; [41] is a presentation). Please complete the bibliographic details.","section":"References"}],"recommendation":"major_revision","confidential_remarks":"The paper's central comparative claim (32.7% over existing COS) is not supported by the experiments, because the 'COS' baseline is an internal emulation with an idealized operator-support assumption. However, the paper is transparent about this emulation, and the underlying OASIS architecture and prototype are real. The issue is fixable within the paper's scope: either add a real COS comparison or rephrase the claim as an ablation of hierarchical vs. single-layer execution within OASIS. I lean toward major_revision rather than reject because the technical contribution is plausible and the missing comparison is identifiable and addressable."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"The key thing to know about this paper is that the engineering is real and the hierarchical execution idea is sensible, but the 32.7% speedup claim over existing COS systems is not backed by a measurement against any actual COS product. That number comes from comparing OASIS to a configuration labeled 'COS' that is just OASIS itself with all operators forced to run on the frontend. The paper admits this ('we assume that COS supports all candidate operators'), but it's a load-bearing assumption, not a minor footnote.\n\nWhat's genuinely new: two-layer execution with Substrait-based plan decomposition (SODA), array-aware expressions via DuckDB, and Arrow intermediate/final output. The analysis of real HPC queries (CFD, HEP, PIC) is useful, and the observation that joins are absent is a nice empirical grounding. The Q1 SODA split-point comparison is the most informative experiment: offloading read+filter+aggregate+project and keeping sort at the frontend gives ~45% improvement over frontend-only, and that result doesn't depend on the COS emulation.\n\nThe biggest soft spot is the COS baseline. MinIO/Ceph S3 Select don't support aggregate/sort or array expressions, and SkyhookDM doesn't support computed projections. The emulation gives COS capabilities it doesn't have, so the 32.7% figure is an upper bound at best. The paper's own I/O numbers show OASIS GET throughput is about half of MinIO's at 1GB objects, so an actual comparison could plausibly go either way. Also, no error bars are reported, no artifacts are released, and the SODA validation is on a single query (Q1). The cost model relies on histograms sampling 0.5–5% of the object and one-way transfer; that's reasonable but unvalidated. These are real weaknesses, but they are fixable, and they don't sink the central architectural argument.\n\nFor a reader working on computational storage or storage-side query processing, this is worth reading for the design and the workload analysis, but treat the 32.7% as provisional. I'd send it to peer review — the prototype is substantial and the ideas merit scrutiny — but I'd expect a major revision that either adds a real COS baseline or substantially recalibrates the claims.","headline":"Real prototype and a sensible hierarchical-execution design, but the headline speedup over existing COS systems is measured against an emulated baseline, not an actual product.","tokens_in":21533,"tokens_out":2178,"would_cite":false,"duration_ms":24028,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"OASIS claims storage-side SQL planning can cut Spark query time by up to 32.7% over existing COS storage.","keywords":["object storage","SQL query offloading","in-storage compute","query plan decomposition","scientific tabular workloads","data movement","columnar format","Substrait IR"],"falsifier":"Run Q1–Q4 on the same hardware while mechanically varying the deployment point: force the split at every operator boundary (filter-only, filter+aggregate, filter+aggregate+project, full offload) and measure execution time. If any configuration other than SODA's chosen split consistently wins across selectivities from 0.0001% to 75%, or if increasing the histogram sample size from 0.5% to 5% changes SODA's split decision and materially changes runtime, the cost model is wrong.","tokens_in":20615,"feed_emoji":"🗄️","tokens_out":4870,"duration_ms":51498,"temperature":0.7,"pith_summary":"This paper argues that the next step for computation-enabled object storage is not just pushing filters into storage, but deciding, operator by operator, which part of a SQL query should run on the storage array and which should run at the storage frontend. The authors build OASIS, a two-tier object-storage system whose Local Optimizer decomposes a query plan expressed in the Substrait intermediate representation and places each operator according to estimated data-reduction ratios. The load-bearing result is that this decomposition is what delivers the gains: on a representative aggregate-then-sort query, the SODA algorithm's chosen split performs 45% faster than the frontend-only configuration that models existing COS systems, and overall OASIS reports up to 32.7% speedup over existing COS-based setups on real scientific queries. A reader should care because scientific post-hoc analysis moves terabytes across slow I/O for queries that return tiny regions of interest; if storage can genuinely reduce data before it leaves the array, both turnaround and infrastructure cost drop.","feed_headline":"Storage-side query splitting cuts Spark analytics time 32.7%","feed_subtitle":"OASIS picks which SQL operators run on storage arrays versus the frontend, moving only what's needed upward.","key_machinery":"SODA (Storage-side Query Plan Offloading and Decomposition Algorithm): a plan-splitting algorithm that classifies Substrait operators by input-output coefficient, chains those coefficients through the operator tree to estimate intermediate sizes, and picks the split point between OASIS-A and OASIS-FE that minimizes data movement. For array-level conditions where histogram-based coefficients cannot be estimated, the companion Structure-Aware Placement (SAP) strategy evaluates those predicates at the data-resident array and lazily forwards results only when they fit within internal transfer limits.","core_discovery":"OASIS claims to be a COS design that treats the internal storage hierarchy as a query execution tier: both the frontend (OASIS-FE) and array nodes (OASIS-A) run embedded SQL executors, and a Local Optimizer partitions the Substrait IR plan between them. The partition is chosen by SODA, which classifies operators by input-output coefficient (1:1 for read/sort, 1:x<=1 for filter/project/aggregate), estimates coefficients from 0.5–5% sampled histograms built at ingestion, and selects the split point that minimizes one-way data movement from array to frontend. For array-typed predicates whose selectivity cannot be estimated, SODA switches to SAP, always evaluating those predicates at the array a","pith_inferences":["The 0.5–5% histogram sampling is an unverified assumption: one could test whether increasing the sample size changes SODA's chosen split point and by how much, since the whole speedup argument depends on those coefficient estimates.","Because SODA's cost model ignores per-operator compute cost, the optimal split likely shifts toward the frontend on CPU-starved arrays even when data reduction is high; the paper's own closing remark points in this direction.","The paper only evaluates queries drawn from the Op-1 and Op-2 operator classes; a natural extension is to see whether coefficient chaining can be extended to join-like operators when such queries appear in other scientific communities.","The Arrow-versus-CSV loading test suggests OASIS could make Arrow the default output and relegate CSV to a compatibility fallback, an implication the paper sketches but does not fully pursue."],"forward_implications":["If SODA's cost model is reliable, the same decomposition idea transfers to other disaggregated storage settings, such as DPU-based flash arrays, where compute exists at multiple tiers.","Storage-side execution can support aggregate and sort, not just filter and project, so more real-world HPC queries can be offloaded without transferring whole files.","Arrow as the intermediate format between tiers removes serialization overhead, making early data reduction cheap enough to benefit client-side Spark ingestion.","The selectivity experiments imply that offloading decisions should be dynamic: when selectivity exceeds about 25% and a heavy sort follows filtering, the baseline can beat storage-side offloading, so the optimal split depends on per-query selectivity and operator mix.","Using Substrait IR as the plan representation lets OASIS swap in other query engines beyond Spark, since the decomposition operates on a language-agnostic plan rather than SQL text."],"supporting_citations":[{"why":"MinIO is the existing object-interface COS baseline that OASIS is measured against in I/O and query experiments.","marker":"[18]"},{"why":"SkyhookDM is the representative embedded COS baseline whose operator and array-semantics limitations OASIS targets.","marker":"[21]"},{"why":"Ceph S3 Select exemplifies the interface-level COS approach with fixed single-layer execution that SODA is designed to improve.","marker":"[23]"},{"why":"DuckDB is the embedded SQL engine used as the in-storage Query Executor on both OASIS tiers, providing aggregate, sort, and array expressions.","marker":"[27]"},{"why":"Substrait is the intermediate representation that enables operator-level plan decomposition and reconstruction across OASIS-FE and OASIS-A.","marker":"[57]"},{"why":"The Laghos 3D Mesh Dataset supplies Q1 and the low-selectivity analysis that motivates early data reduction.","marker":"[40]"},{"why":"The Deep Water Impact dataset supplies Q2 and Q3 used to compare hierarchical execution against baseline and COS configurations.","marker":"[50]"},{"why":"CMS Open Data supplies Q4, the array-based muon-pair query used to test array-aware offloading.","marker":"[64]"},{"why":"The HEP benchmark is the origin of Q4's dimuon query pattern, grounding the array-based evaluation in a recognized workload.","marker":"[65]"}],"fun_headline_variants":["OASIS runs SQL inside storage, cuts analytics time 32.7%","Storage-tier SQL optimizer speeds Spark queries by 32.7%","Smart storage executes SQL filters, boosts Spark 32.7%","Split SQL between storage and frontend for 32.7% gains","Cost-based storage SQL offloading cuts Spark runtime 32.7%"],"cache_read_input_tokens":2688,"weakest_assumption_plain":"The speedup claims rest on SODA's cost model accurately estimating intermediate data sizes from tiny sampled histograms (0.5–5%) and on assuming one-way data transfer from array to frontend; if those estimates are off, the chosen split point may be suboptimal and the 32.7% figure would not hold.","fun_headline_variants_meta":{"raw":{"variants":["OASIS runs SQL inside storage, cuts analytics time 32.7%","Storage-tier SQL optimizer speeds Spark queries by 32.7%","Smart storage executes SQL filters, boosts Spark 32.7%","Split SQL between storage and frontend for 32.7% gains","Cost-based storage SQL offloading cuts Spark runtime 32.7%"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000405,"raw_usage":{"total_tokens":1995,"prompt_tokens":847,"completion_tokens":1148,"prompt_tokens_details":{"cached_tokens":256},"prompt_cache_hit_tokens":256,"prompt_cache_miss_tokens":591,"completion_tokens_details":{"reasoning_tokens":1052}},"tokens_in":591,"tokens_out":1148,"duration_ms":9407,"temperature":1.0,"reasoning_tokens":1052,"cache_read_input_tokens":256,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-05T11:59:51.378904+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run Q1–Q4 on the same hardware while mechanically varying the deployment point: force the split at every operator boundary (filter-only, filter+aggregate, filter+aggregate+project, full offload) and measure execution time. If any configuration other than SODA's chosen split consistently wins across selectivities from 0.0001% to 75%, or if increasing the histogram sample size from 0.5% to 5% changes SODA's split decision and materially changes runtime, the cost model is wrong.","supporting_citations":[{"cited_title":"Minio: High performance object storage","cited_arxiv_id":null,"evidence_quote":"MinIO is the existing object-interface COS baseline that OASIS is measured against in I/O and query experiments."},{"cited_title":"Skyhookdm: Data processing in ceph with programmable storage,","cited_arxiv_id":null,"evidence_quote":"SkyhookDM is the representative embedded COS baseline whose operator and array-semantics limitations OASIS targets."},{"cited_title":"S3 select support in rados gateway","cited_arxiv_id":null,"evidence_quote":"Ceph S3 Select exemplifies the interface-level COS approach with fixed single-layer execution that SODA is designed to improve."},{"cited_title":"Foundation, “DuckDB.” https://duckdb.org/, 2024","cited_arxiv_id":null,"evidence_quote":"DuckDB is the embedded SQL engine used as the in-storage Query Executor on both OASIS tiers, providing aggregate, sort, and array expressions."},{"cited_title":"Substrait","cited_arxiv_id":null,"evidence_quote":"Substrait is the intermediate representation that enables operator-level plan decomposition and reconstruction across OASIS-FE and OASIS-A."},{"cited_title":"Laghos sample dataset,","cited_arxiv_id":null,"evidence_quote":"The Laghos 3D Mesh Dataset supplies Q1 and the low-selectivity analysis that motivates early data reduction."},{"cited_title":"Deep water impact dataset (la-ur-17-21595),","cited_arxiv_id":null,"evidence_quote":"The Deep Water Impact dataset supplies Q2 and Q3 used to compare hierarchical execution against baseline and COS configurations."},{"cited_title":"Versity Gateway","cited_arxiv_id":null,"evidence_quote":"CMS Open Data supplies Q4, the array-based muon-pair query used to test array-aware offloading."},{"cited_title":"Evaluating query languages and systems for high-energy physics data,","cited_arxiv_id":null,"evidence_quote":"The HEP benchmark is the origin of Q4's dimuon query pattern, grounding the array-based evaluation in a recognized workload."}],"review_version":1}