Pith. sign in

REVIEW 3 major objections 6 minor 67 references

OASIS: Object-based Analytics Storage for Intelligent SQL Query Offloading in Scientific Tabular Workloads

T0 review · 3 major / 6 minor · reviewed 2026-08-05 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read OASIS claims storage-side SQL planning can cut Spark query time by up to 32.7% over existing COS storage.

desk verdict Real prototype and a sensible hierarchical-execution design, but the headline speedup over existing COS systems is measured against an emulated baseline, not an actual product. read the letter →

arxiv 2509.01966 v1 pith:BBFO5R6B submitted 2025-09-02 cs.DB cs.DC

Soon Hwang , Junhyeok Park , Junghyun Ryu , Seonghoon Ahn , Jeoungahn Park , Jeongjin Lee , Soonyeal Yang , Jungki Noh
show 3 more authors
Woosuk Chung Hoshik Kim Youngjae Kim
This is my paper · ORCID
classification cs.DBcs.DC
keywords objectstorageSQLqueryoffloadingin-storagecomputeplandecompositionscientifictabularworkloadsdatamovementcolumnarformatSubstraitIR
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper argues that the next step for computation-enabled object storage is not just pushing filters into storage, but deciding, operator by operator, which part of a SQL query should run on the storage array and which should run at the storage frontend. The authors build OASIS, a two-tier object-storage system whose Local Optimizer decomposes a query plan expressed in the Substrait intermediate representation and places each operator according to estimated data-reduction ratios. The load-bearing result is that this decomposition is what delivers the gains: on a representative aggregate-then-sort query, the SODA algorithm's chosen split performs 45% faster than the frontend-only configuration that models existing COS systems, and overall OASIS reports up to 32.7% speedup over existing COS-based setups on real scientific queries. A reader should care because scientific post-hoc analysis moves terabytes across slow I/O for queries that return tiny regions of interest; if storage can genuinely reduce data before it leaves the array, both turnaround and infrastructure cost drop.

What carries the argument

SODA (Storage-side Query Plan Offloading and Decomposition Algorithm): a plan-splitting algorithm that classifies Substrait operators by input-output coefficient, chains those coefficients through the operator tree to estimate intermediate sizes, and picks the split point between OASIS-A and OASIS-FE that minimizes data movement. For array-level conditions where histogram-based coefficients cannot be estimated, the companion Structure-Aware Placement (SAP) strategy evaluates those predicates at the data-resident array and lazily forwards results only when they fit within internal transfer limits.

What would settle it

Run Q1–Q4 on the same hardware while mechanically varying the deployment point: force the split at every operator boundary (filter-only, filter+aggregate, filter+aggregate+project, full offload) and measure execution time. If any configuration other than SODA's chosen split consistently wins across selectivities from 0.0001% to 75%, or if increasing the histogram sample size from 0.5% to 5% changes SODA's split decision and materially changes runtime, the cost model is wrong.

Watch

Extended reading notes

Core claim

OASIS claims to be a COS design that treats the internal storage hierarchy as a query execution tier: both the frontend (OASIS-FE) and array nodes (OASIS-A) run embedded SQL executors, and a Local Optimizer partitions the Substrait IR plan between them. The partition is chosen by SODA, which classifies operators by input-output coefficient (1:1 for read/sort, 1:x<=1 for filter/project/aggregate), estimates coefficients from 0.5–5% sampled histograms built at ingestion, and selects the split point that minimizes one-way data movement from array to frontend. For array-typed predicates whose selectivity cannot be estimated, SODA switches to SAP, always evaluating those predicates at the array a

Load-bearing premise

The speedup claims rest on SODA's cost model accurately estimating intermediate data sizes from tiny sampled histograms (0.5–5%) and on assuming one-way data transfer from array to frontend; if those estimates are off, the chosen split point may be suboptimal and the 32.7% figure would not hold.

Editorial extensions

If this is right

  • If SODA's cost model is reliable, the same decomposition idea transfers to other disaggregated storage settings, such as DPU-based flash arrays, where compute exists at multiple tiers.
  • Storage-side execution can support aggregate and sort, not just filter and project, so more real-world HPC queries can be offloaded without transferring whole files.
  • Arrow as the intermediate format between tiers removes serialization overhead, making early data reduction cheap enough to benefit client-side Spark ingestion.
  • The selectivity experiments imply that offloading decisions should be dynamic: when selectivity exceeds about 25% and a heavy sort follows filtering, the baseline can beat storage-side offloading, so the optimal split depends on per-query selectivity and operator mix.
  • Using Substrait IR as the plan representation lets OASIS swap in other query engines beyond Spark, since the decomposition operates on a language-agnostic plan rather than SQL text.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The 0.5–5% histogram sampling is an unverified assumption: one could test whether increasing the sample size changes SODA's chosen split point and by how much, since the whole speedup argument depends on those coefficient estimates.
  • Because SODA's cost model ignores per-operator compute cost, the optimal split likely shifts toward the frontend on CPU-starved arrays even when data reduction is high; the paper's own closing remark points in this direction.
  • The paper only evaluates queries drawn from the Op-1 and Op-2 operator classes; a natural extension is to see whether coefficient chaining can be extended to join-like operators when such queries appear in other scientific communities.
  • The Arrow-versus-CSV loading test suggests OASIS could make Arrow the default output and relegate CSV to a compatibility fallback, an implication the paper sketches but does not fully pursue.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 6 minor

Summary. The paper proposes OASIS, a computation-enabled object storage system that offloads SQL operators to storage arrays and a frontend node, using a cost-based plan decomposition algorithm (SODA) to split query execution between the OASIS-A and OASIS-FE. The authors build a prototype with SPDK and DuckDB, integrate it with Spark via Substrait IR, and evaluate on three scientific workloads (Laghos, DeepWater Impact, CMS Open Data). They report up to 70.59% speedup over a traditional Spark baseline and up to 32.7% speedup over a configuration labeled 'COS,' which emulates single-layer execution by running all operators at the OASIS-FE.

Significance. If the claims were fully supported, the paper would make a useful contribution: it demonstrates a working hierarchical execution path for storage-side query offloading, with support for array expressions and Arrow output. The prototype is real; the queries come from public HPC benchmarks; SODA's overhead is measured (126 ms estimation, 1.81 s decomposition); and the selectivity experiments provide a clear characterization of when offloading wins. The main weakness is that the headline comparison against existing COS systems is not actually carried out against any existing COS product. This is a central, load-bearing issue that must be addressed before the comparative claim can be accepted.

major comments (3)
  1. [§V-A, §V-C1, Abstract] The abstract claims 'up to a 32.7% performance improvement over Spark configured with existing COS-based storage systems,' but the 'COS' configuration is defined in §V-A as an emulation in which all operators execute at the OASIS-FE. This is not MinIO Select, Ceph S3 Select, or SkyhookDM. The scalar-query evaluation assumes COS supports all candidate operators, yet real COS systems have limited operator support (MinIO/Ceph S3 Select lack aggregate/sort and array expressions; SkyhookDM lacks aggregate/sort and computed projections). Q2, which yields the 32.7% figure, is a filter+project query that real COS systems can execute, so the assumption is not harmless. Moreover, §V-B shows OASIS GET throughput is 605.6 MB/s vs MinIO's 1,080 MB/s at 1 GB objects, so an actual COS could perform better or worse than the emulation. Without a measurement against a real COS product, the headline compar
  2. [§V-F, Figure 10] SODA is the core contribution, but its effectiveness is evaluated on a single query (Q1). The five static split configurations in Figure 10(b) are all internal configurations of OASIS, and SODA's selection of cfg4 is compared only within the same prototype. No query with array-based predicates (Q4) is used to validate the SAP strategy, and no sensitivity analysis is provided for the cost-model parameters (e.g., histogram sampling ratio 0.5–5%, per-operator coefficients, or the one-way transfer assumption). Given that the paper's central claim is that SODA 'correctly selects the optimal split point,' the evidence is narrow. At minimum, the paper should broaden the SODA evaluation to include array-based queries and report how sensitive the split choice is to the estimated coefficients.
  3. [§IV-G2, §V] The cost model that drives SODA assumes (a) one-way data transfer from OASIS-A to OASIS-FE without return traffic, and (b) that histogram-based selectivity estimates from 0.5–5% sampled histograms accurately predict output sizes. These assumptions are load-bearing: if a selectivity estimate is wrong, the chosen split point could be suboptimal, and the reported speedups could be artifacts of the chosen dataset. The paper does not validate the one-way transfer assumption or report the error in the histograms' selectivity estimates. A sensitivity analysis over sampling ratios and a validation of the coefficient model against actual intermediate sizes is needed to support the optimality claim.
minor comments (6)
  1. [§V-C1] Typo: 'Mote that' should be 'Note that.'
  2. [§III-B] Section heading contains 'Exisiting' — should be 'Existing.'
  3. [§V-A] The grammar in 'equipped 1 NVMe SSD' is awkward; revise to 'equipped with one NVMe SSD.'
  4. [Figure 10] The hatch patterns in Figure 10(b) are hard to distinguish in grayscale; consider adding value labels or using distinct fills.
  5. [§V-D] The decrease in execution time from 1,000,000 to 10,000,000 records is explained by an increased partition count; this should be stated before the figure is discussed, not after.
  6. [References] Some references lack consistent metadata (e.g., [34] is a URL-only entry; [41] is a presentation). Please complete the bibliographic details.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the speedup claims are measured against defined baselines, and SODA's split choice is empirically validated rather than derived from its own inputs.

full rationale

The paper's central claims are empirical measurements, not derivations from self-defined constants. SODA's decomposition uses histogram-derived coefficients to choose a split point, but the evaluation in §V-F compares the chosen configuration against five static alternatives and reports measured runtimes (cfg4: 76s vs ~83s and ~145s), so the 'optimal' claim is validated by external runtime data, not equated to the cost model's own estimate. The CAD/SAP distinction is a stated heuristic with explicit assumptions (e.g., one-way data transfer in §IV-G2), not a hidden equivalence. The only notable concern is the 'COS' baseline in §V-A: it is an emulation of the COS computation model by executing all operators at OASIS-FE, and §V-C1 explicitly assumes COS supports all candidate operators. This means the abstract's 'over existing COS-based storage systems' overstates what was measured; but this is a benchmark-validity/external-comparison issue, not circularity, because the result is not equivalent by construction to an input of the derivation. Self-citations [24], [25], [42] appear only as related-work background on computational storage and are not load-bearing for OASIS's design or its performance claims. No uniqueness theorem, no ansatz smuggled via citation, and no renaming of a known result as a new contribution were found. Therefore, no significant circularity.

Assumptions & free parameters 1 free parameters · 4 assumptions · 0 invented entities

OASIS introduces architectural layers and a decomposition algorithm, but no new physical or formal entities whose existence requires independent evidence. The free parameters are confined to implementation choices such as histogram sampling ratio and the underspecified SAP transfer threshold.

free parameters (1)
  • Histogram sampling ratio = 0.5-5% of object
    The Metadata Manager builds histograms from a sampled subset of each object; the sampling ratio is chosen by the designers and directly affects selectivity estimates used by CAD.
assumptions (4)
  • domain assumption HPC tabular queries in this setting contain no join operators
    Observation 2 (Section III-A) derives from collected queries, and SODA excludes Op-3 and Op-4 operators (join, set) from coefficient-based cost estimation. If joins appear, the decomposition strategy is not covered.
  • domain assumption One-way data transfer from OASIS-A to OASIS-FE
    CAD assumes one-way transfer without return traffic (Section IV-G2). Iterative or round-trip data movement would break the cost model.
  • domain assumption Sampled histograms accurately represent column distributions
    CAD estimates filter selectivity and projection output sizes from histograms built from 0.5-5% samples. Poor sampling leads to wrong split points.
  • domain assumption DuckDB can execute on storage servers with constrained resources without becoming a bottleneck
    The in-storage executor is DuckDB running on OASIS-A with 16 cores. The Q3 result shows the compute capability gap narrows the benefit of offloading, so this assumption is load-bearing.

how reviews work

0 comments
Cite this review

Pith. "Pith review of OASIS: Object-based Analytics Storage for Intelligent SQL Query Offloading in Scientific Tabular Workloads." pith.science (2026). https://pith.science/paper/BBFO5R6B

@misc{pith2026250901966,
  author       = {Pith},
  title        = {Pith review of: OASIS: Object-based Analytics Storage for Intelligent SQL Query Offloading in Scientific Tabular Workloads},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/BBFO5R6B}},
  note         = {Machine review of arXiv:2509.01966}
}
read the original abstract

Computation-Enabled Object Storage (COS) systems, such as MinIO and Ceph, have recently emerged as promising storage solutions for post hoc, SQL-based analysis on large-scale datasets in High-Performance Computing (HPC) environments. By supporting object-granular layouts, COS facilitates column-oriented access and supports in-storage execution of data reduction operators, such as filters, close to where the data resides. Despite growing interest and adoption, existing COS systems exhibit several fundamental limitations that hinder their effectiveness. First, they impose rigid constraints on output data formats, limiting flexibility and interoperability. Second, they support offloading for only a narrow set of operators and expressions, restricting their applicability to more complex analytical tasks. Third--and perhaps most critically--they fail to incorporate design strategies that enable compute offloading optimized for the characteristics of deep storage hierarchies. To address these challenges, this paper proposes OASIS, a novel COS system that features: (i) flexible and interoperable output delivery through diverse formats, including columnar layouts such as Arrow; (ii) broad support for complex operators (e.g., aggregate, sort) and array-aware expressions, including element-wise predicates over array structures; and (iii) dynamic selection of optimal execution paths across internal storage layers, guided by operator characteristics and data movement costs. We implemented a prototype of OASIS and integrated it into the Spark analytics framework. Through extensive evaluation using real-world scientific queries from HPC workflows, OASIS achieves up to a 32.7% performance improvement over Spark configured with existing COS-based storage systems.

Figures

Figures reproduced from arXiv: 2509.01966 by the authors.

Figure 1
Figure 1. Comparison of traditional storage systems, existing [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 2
Figure 2. A scientific analytics workflow, illustrating a post-hoc analysis pipeline ( [PITH_FULL_IMAGE:figures/full_fig_p003_2.png] view at source ↗
Figure 3
Figure 3. Heatmaps showing distribution rates (‰) of filtered [PITH_FULL_IMAGE:figures/full_fig_p003_3.png] view at source ↗
Figures from the paper (6 more)
Figure 4
Figure 4. Figure 4: illustrates the end-to-end query offloading process in the OASIS system. 1 The process begins when a client-side query engine submits an SQL query. 2 The query is translated into an Intermediate Representation (IR), Substrait [57], which describes the operator-level ex…
Figure 5
Figure 5. Figure 5: Illustration of the mechanism by which the IR plan is [PITH_FULL_IMAGE:figures/full_fig_p008_5.png]
Figure 6
Figure 6. Figure 6: Throughput comparison of OASIS and MinIO. [PITH_FULL_IMAGE:figures/full_fig_p010_6.png]
Figure 8
Figure 8. Figure 8: Comparison of input parsing times of client-side Spark [PITH_FULL_IMAGE:figures/full_fig_p011_8.png]
Figure 9
Figure 9. Figure 9: Execution time comparison between the baseline and O [PITH_FULL_IMAGE:figures/full_fig_p012_9.png]
Figure 10
Figure 10. Figure 10: (a) Description of decomposition configurations and [PITH_FULL_IMAGE:figures/full_fig_p012_10.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

67 extracted references · 67 canonical work pages

  1. [1]

    A quantitative review of data formats for hep analyses,

    J. Blomer, “A quantitative review of data formats for hep analyses,” in Journal of Physics: Conference Series , vol. 1085, p. 032020, IOP Publishing, 2018

    work page 2018

  2. [2]

    Scientific data management in the coming decade,

    J. Gray, D. T. Liu, M. Nieto-Santisteban, A. Szalay, D. J. DeWitt, and G. Heber, “Scientific data management in the coming decade,” Acm Sigmod Record, vol. 34, no. 4, pp. 34–41, 2005

    work page 2005

  3. [3]

    Processing particle data flows with smartnics,

    J. Liu, C. Maltzahn, M. L. Curry, and C. Ulmer, “Processing particle data flows with smartnics,” in 2022 IEEE High Performance Extreme Computing Conference (HPEC) , pp. 1–8, IEEE, 2022

    work page 2022

  4. [4]

    A framework for constrained control allo- cation using cfd-based tabular data,

    A. Da Ronch, M. Ghoreyshi, D. Vallespin, K. Badcock, Z. Mengmeng, J. Opplestrup, and A. Rizzi, “A framework for constrained control allo- cation using cfd-based tabular data,” in 49th AIAA Aerospace Sciences Meeting including the New Horizons Forum and Aerospace Exposition , p. 925, 2011

    work page 2011

  5. [5]

    In-memory query system for scientific dataseis,

    H.-T. Chiu, J. Chou, V . Vishwanath, and K. Wu, “In-memory query system for scientific dataseis,” in 2015 IEEE 21st International Con- ference on Parallel and Distributed Systems (ICPADS) , pp. 362–371, IEEE, 2015

    work page 2015

  6. [6]

    Apache Spark: Unified engine for large-scale data analytics,

    Apache Foundation, “Apache Spark: Unified engine for large-scale data analytics,” 2024

    work page 2024

  7. [7]

    Parallel data analysis directly on scientific file formats,

    S. Blanas, K. Wu, S. Byna, B. Dong, and A. Shoshani, “Parallel data analysis directly on scientific file formats,” in 2014 ACM SIGMOD International Conference on Management of Data , SIGMOD ’14, (New York, NY , USA), p. 385–396, Association for Computing Machinery, 2014

    work page 2014

  8. [8]

    Querying large scientific data sets with adaptable IO system ADIOS,

    J. Gu, S. Klasky, N. Podhorszki, J. Qiang, and K. Wu, “Querying large scientific data sets with adaptable IO system ADIOS,” in Supercomput- ing Frontiers (R. Yokota and W. Wu, eds.), (Cham), pp. 51–69, Springer International Publishing, 2018

    work page 2018

Show all 67 references
  1. [9]

    LANL’s Journey Toward Compu- tational Storage

    Gary Grider, “LANL’s Journey Toward Compu- tational Storage.” https://snia.org/educational-library/ lanls-journey-toward-computational-storage-2024, 2024. Presented at the SNIA Storage Developer Conference (SDC), 2024

    2024

  2. [10]

    ORNL Frontier

    O. R. N. Laboratory, “ORNL Frontier.” https://www.olcf.ornl.gov/ frontier/, 2022

    2022

  3. [11]

    High-luminosity large hadron collider (hl-lhc)

    CERN, “High-luminosity large hadron collider (hl-lhc).” https:// hilumilhc.web.cern.ch/, 2025. Accessed: 2025-05-12

    2025

  4. [12]

    Arraybridge: Interweaving declarative array processing in scidb with imperative hdf5-based programs,

    H. Xing, S. Floratos, S. Blanas, S. Byna, M. Prabhat, K. Wu, and P. Brown, “Arraybridge: Interweaving declarative array processing in scidb with imperative hdf5-based programs,” in 2018 IEEE 34th Inter- national Conference on Data Engineering (ICDE) , pp. 977–988, IEEE, 2018

    2018

  5. [13]

    Understanding data movement patterns in hpc: A nersc case study,

    A. Giannakou, D. Hazen, B. Enders, L. Ramakrishnan, and N. J. Wright, “Understanding data movement patterns in hpc: A nersc case study,” in SC24: International Conference for High Performance Computing, Networking, Storage and Analysis , pp. 1–17, 2024

    2024

  6. [14]

    A survey on spatio-temporal big data analytics ecosystem: Resource management, processing platform, and applications,

    H. Liang, Z. Zhang, C. Hu, Y . Gong, and D. Cheng, “A survey on spatio-temporal big data analytics ecosystem: Resource management, processing platform, and applications,” IEEE Transactions on Big Data, 2023

    2023

  7. [15]

    Cicotti, S

    P. Cicotti, S. Oral, G. Kestor, R. Gioiosa, S. Strande, M. Taufer, J. H. Rogers, H. Abbasi, J. Hill, and L. Carrington, Data Movement in Data- Intensive High Performance Computing , pp. 31–59. Cham: Springer International Publishing, 2016

    2016

  8. [16]

    Status and planning of high energy physics data storage system

    H. Li, “Status and planning of high energy physics data storage system.” Presentation at the 2023 International Workshop on the High Energy Circular Electron Positron Collider (CEPC), October 2023. Institute of High Energy Physics (IHEP), Chinese Academy of Sciences

    2023

  9. [17]

    A ceph s3 object data store for hep,

    N. Smith, B. Jayatilaka, D. Mason, O. Gutsche, A. Peisker, R. Illing- worth, and C. Jones, “A ceph s3 object data store for hep,” 2023

    2023

  10. [18]

    Minio: High performance object storage

    MinIO, Inc., “Minio: High performance object storage.” https://min.io/

  11. [19]

    Ceph: A scalable, high-performance distributed file system,

    S. A. Weil, S. A. Brandt, E. L. Miller, D. D. E. Long, and C. Maltzahn, “Ceph: A scalable, high-performance distributed file system,” in 7th Symposium on Operating Systems Design and Implementation , OSDI ’06, (USA), p. 307–320, USENIX Association, 2006

    2006

  12. [20]

    S3 Select and Glacier Select – Retrieving Subsets of Objects,

    Randall Hunt, “S3 Select and Glacier Select – Retrieving Subsets of Objects,” Nov. 29, 2017

    2017

  13. [21]

    Skyhookdm: Data processing in ceph with programmable storage,

    J. LeFevre and C. Maltzahn, “Skyhookdm: Data processing in ceph with programmable storage,” USENIX login;, vol. 45, no. 2, 2020

    2020

  14. [22]

    mc sql command — run sql queries on object data

    MinIO, Inc., “mc sql command — run sql queries on object data.” https: //min.io/docs/minio/linux/reference/minio-mc/mc-sql.html. Accessed: 2025-05-12

    2025

  15. [23]

    S3 select support in rados gateway

    Ceph Project, “S3 select support in rados gateway.” https://docs.ceph. com/en/latest/radosgw/s3select/. Accessed: 2025-05-12

    2025

  16. [24]

    AnalyzeThis: an analysis workflow-aware storage system,

    H. Sim, Y . Kim, S. S. Vazhkudai, D. Tiwari, A. Anwar, A. R. Butt, and L. Ramakrishnan, “AnalyzeThis: an analysis workflow-aware storage system,” in International Conference for High Performance Computing, Networking, Storage and Analysis , pp. 1–12, 2015

    2015

  17. [25]

    Tagit: an integrated indexing and search service for file systems,

    H. Sim, Y . Kim, S. S. Vazhkudai, G. R. Vall ´ee, S.-H. Lim, and A. R. Butt, “Tagit: an integrated indexing and search service for file systems,” in International Conference for High Performance Computing, Networking, Storage and Analysis , pp. 1–12, 2017

    2017

  18. [26]

    Apache Arrow: A cross-language development platform for in-memory analytics

    Apache, “Apache Arrow: A cross-language development platform for in-memory analytics.” https://arrow.apache.org/, 2024

    2024

  19. [27]

    Foundation, “DuckDB.” https://duckdb.org/, 2024

    D. Foundation, “DuckDB.” https://duckdb.org/, 2024

    2024

  20. [28]

    OCS Laghos Sample Dataset

    L. A. N. Laboratory, “OCS Laghos Sample Dataset.” https://github.com/ lanl-ocs/laghos-sample-dataset, 2024

    2024

  21. [29]

    Spark and hpc for high energy physics data analyses,

    S. Sehrish, J. Kowalkowski, and M. Paterno, “Spark and hpc for high energy physics data analyses,” in 2017 IEEE International Parallel and Distributed Processing Symposium Workshops (IPDPSW) , pp. 1048– 1057, IEEE, 2017

    2017

  22. [30]

    H5spark: bridging the i/o gap between spark and scientific data formats on hpc systems,

    J. Liu, E. Racah, Q. Koziol, R. S. Canon, A. Gittens, L. Gerhardt, S. Byna, and M. F. Ringenburg, “H5spark: bridging the i/o gap between spark and scientific data formats on hpc systems,” Cray user group , 2016

    2016

  23. [31]

    Parquet

    Apache, “Parquet.” https://parquet.apache.org/, 2024

    2024

  24. [32]

    High-performance data format for scientific data storage and analysis,

    G. Gavalian, “High-performance data format for scientific data storage and analysis,” arXiv preprint arXiv:2501.07666 , 2025

    2025 arXiv

  25. [33]

    Accelerating data analysis in simulation neuroscience with big data technologies,

    J. Planas, F. Delalondre, and F. Sch ¨urmann, “Accelerating data analysis in simulation neuroscience with big data technologies,” in Computa- tional Science–ICCS 2018: 18th International Conference, Wuxi, China, June 11–13, 2018, Proceedings, Part I 18, pp. 363–377, Springer, 2018

    2018

  26. [34]

    Apache spark for high energy physics,

    L. Canali, “Apache spark for high energy physics,” 2024. Benchmark notebooks demonstrating the use of Apache Spark for High Energy Physics data analysis

    2024

  27. [35]

    Parallel query evaluation as a scien- tific data service,

    B. Dong, S. Byna, and K. Wu, “Parallel query evaluation as a scien- tific data service,” in 2014 IEEE International Conference on Cluster Computing (CLUSTER), pp. 194–202, IEEE, 2014

    2014

  28. [36]

    A comparative survey of the hpc and big data paradigms: Analysis and experiments,

    H. Asaadi, D. Khaldi, and B. Chapman, “A comparative survey of the hpc and big data paradigms: Analysis and experiments,” in 2016 IEEE International Conference on Cluster Computing (CLUSTER) , pp. 423– 432, IEEE, 2016

    2016

  29. [37]

    Toward open object-based computational storage for analysis query pushdown

    Q. Zheng, “Toward open object-based computational storage for analysis query pushdown.” PDSW Work-in-Progress, 2023

    2023

  30. [38]

    Improving storage systems for simulation science with computational storage

    D. Manno, “Improving storage systems for simulation science with computational storage.” Compute+Memory+Storage Summit, 2023

    2023

  31. [39]

    Kinetic campaign: Speeding scientific data analytics with computational storage drives

    Q. Zheng, “Kinetic campaign: Speeding scientific data analytics with computational storage drives.” Presented at SDC, 2022

    2022

  32. [40]

    Laghos sample dataset,

    L. A. N. L. O. C. S. (LANL-OCS), “Laghos sample dataset,” 2024. Sample dataset generated by the Laghos simulation application for system prototyping and benchmarking

    2024

  33. [41]

    Leveraging Computational Storage for Simulation Science Storage System Design

    Gary Grider, “Leveraging Computational Storage for Simulation Science Storage System Design.” Presented at the SNIA Storage Developer Conference (SDC), 2023, 2023

    2023

  34. [42]

    Kv-csd: A hardware-accelerated key- value store for data-intensive applications,

    I. Park, Q. Zheng, D. Manno, S. Yang, J. Lee, D. Bonnie, B. Settlemyer, Y . Kim, W. Chung, and G. Grider, “Kv-csd: A hardware-accelerated key- value store for data-intensive applications,” in 2023 IEEE International Conference on Cluster Computing (CLUSTER) , pp. 132–144, 2023

    2023

  35. [43]

    Using ceph’s bluestore as object storage in hpc storage framework,

    K. Duwe and M. Kuhn, “Using ceph’s bluestore as object storage in hpc storage framework,” CHEOPS ’21, (New York, NY , USA), Association for Computing Machinery, 2021

    2021

  36. [44]

    Amazon S3, Object storage built to retrieve any amount of data from anywhere

    Amazon Web Services, “Amazon S3, Object storage built to retrieve any amount of data from anywhere.” https://aws.amazon.com/s3/, 2024

    2024

  37. [45]

    Fugaku aws s3 service guide,

    R. C. for Computational Science, “Fugaku aws s3 service guide,” 2024. Documentation for utilizing AWS S3-compatible storage on the Fugaku supercomputer

    2024

  38. [46]

    Ibm storage ceph s3 object deep dive,

    IBM Corporation, “Ibm storage ceph s3 object deep dive,” May 2024. Technical white paper detailing IBM Storage Ceph’s S3 object storage features and deployment strategies

    2024

  39. [47]

    S3 and analytics: Taming your storage costs,

    NetApp, “S3 and analytics: Taming your storage costs,” November 2018. White Paper WP-7289

    2018

  40. [48]

    A moveable beast: Partitioning data and compute for computational storage,

    A. Montana, Y . Xue, J. LeFevre, C. Maltzahn, J. Stuart, P. Kufeldt, and P. Alvaro, “A moveable beast: Partitioning data and compute for computational storage,” 2023

    2023

  41. [49]

    Pushdowndb: Accelerating a dbms using s3 computation,

    X. Yu, M. Youill, M. Woicik, A. Ghanem, M. Serafini, A. Aboulnaga, and M. Stonebraker, “Pushdowndb: Accelerating a dbms using s3 computation,” in 2020 IEEE 36th International Conference on Data Engineering (ICDE), pp. 1802–1805, 2020

    2020

  42. [50]

    Deep water impact dataset (la-ur-17-21595),

    L. A. N. Laboratory, “Deep water impact dataset (la-ur-17-21595),”

  43. [51]

    C2 vpic sample dataset,

    L. A. N. Laboratory, “C2 vpic sample dataset,” 2022. Sample dataset generated by running the open-source VPIC particle simulation code for local C2 development and testing

    2022

  44. [52]

    openpmd-api: C++ & python api for scientific i/o with openpmd,

    A. Huebl, F. Poeschel, F. Koller, J. Gu, M. Bussmann, J.-L. Vay, and K. Wu, “openpmd-api: C++ & python api for scientific i/o with openpmd,” 2018. Version 0.17.0-dev

    2018

  45. [53]

    Presto: SQL on Everything,

    R. Sethi, M. Traverso, D. Sundstrom, D. Phillips, W. Xie, Y . Sun, N. Yegitbasi, H. Jin, E. Hwang, N. Shingte, and C. Berner, “Presto: SQL on Everything,” in 2019 IEEE 35th International Conference on Data Engineering (ICDE) , 2019

    2019

  46. [54]

    Querying data in place with amazon s3 select,

    Amazon Web Services, “Querying data in place with amazon s3 select,”

  47. [55]

    Supermicro’s Petascale All-Flash GEN 5 JBOF Storage Solution with NVIDIA BlueField-3 DPU

    Supermicro, “Supermicro’s Petascale All-Flash GEN 5 JBOF Storage Solution with NVIDIA BlueField-3 DPU.” https://www.supermicro.com/ en/products/jbof, 2024

    2024

  48. [56]

    Amazon Simple Storage Service (S3) User Guide

  49. [57]

    Substrait

    S. Project, “Substrait.” https://substrait.io/, 2024

    2024

  50. [58]

    DDN Appliance Combines AI Storage and NVIDIA BlueField-3 DPUs for Enhanced Full-Stack Data Center and Cloud Efficiency

    DDN, “DDN Appliance Combines AI Storage and NVIDIA BlueField-3 DPUs for Enhanced Full-Stack Data Center and Cloud Efficiency.” https://www.ddn.com/press-releases/ ddn-appliance-combines-ai-storage-and-nvidia-bluefield-bf-3-dpus/, 2024

    2024

  51. [59]

    DuckDB ADBC – Zero-Copy Data Transfer via Arrow Database Connectivity,

    P. Holanda, “DuckDB ADBC – Zero-Copy Data Transfer via Arrow Database Connectivity,” Aug. 2023. Accessed June 12, 2025

    2023

  52. [60]

    NVM Express Specification

    NVM Express Inc., “NVM Express Specification.” https://nvmexpress. org/developers/nvme-specification/, 2011

    2011

  53. [61]

    SPDK: A development kit to build high performance storage applications,

    Z. Yang, J. R. Harris, B. Walker, D. Verkamp, C.-p. Liu, C. Chang, G. Cao, J. Stern, V . Verma, and L. E. Paul, “SPDK: A development kit to build high performance storage applications,” in IEEE International Conference on Cloud Computing Technology and Science (CloudCom) , 2017

    2017

  54. [62]

    Velox: meta’s unified execution engine,

    P. Pedreira, O. Erling, M. Basmanova, K. Wilfong, L. Sakka, K. Pai, W. He, and B. Chattopadhyay, “Velox: meta’s unified execution engine,” VLDB Endowment, vol. 15, p. 3372–3384, Aug 2022

    2022

  55. [63]

    Visualization and analysis of threats from asteroid ocean impacts,

    J. M. P. F. J. S. K. C. T. G. R. G. D. H. R. G. D. A. T. L. Turton, “Visualization and analysis of threats from asteroid ocean impacts,” Technical report, 2016

    2016

  56. [64]

    Versity Gateway

    Versity, “Versity Gateway.” https://www.versity.com/products/ versitygw/, 2024

    2024

  57. [65]

    Evaluating query languages and systems for high-energy physics data,

    D. Graur, I. M ¨uller, M. Proffitt, G. Fourny, G. T. Watts, and G. Alonso, “Evaluating query languages and systems for high-energy physics data,” Proc. VLDB Endow., vol. 15, p. 154–168, Oct. 2021

    2021

  58. [66]

    SingleMu primary dataset in AOD format from Run 2012B (22 Jan 2013 re-reconstruction)

    CMS Collaboration, “SingleMu primary dataset in AOD format from Run 2012B (22 Jan 2013 re-reconstruction).” CERN Open Data Portal,

    2013

  59. [2024]

    Sample dataset generated by converting an existing LANL deep water impact simulation dataset for system prototyping and benchmark- ing

Pith tools

Reviewed August 5, 2026 · model on record in the stance chip above.