Pith. sign in

REVIEW 4 major objections 6 minor 1 cited by

High-Performance Data Format for Scientific Data Storage and Analysis

T0 review · 4 major / 6 minor · reviewed 2026-08-10 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read The paper claims that HiPO, a data format built for the CLAS12 experiment, reads and fills histograms from a subset of columns about three times faster than ROOT RNTuple and Parquet on the same benchmark.

desk verdict Useful format description, but the headline performance claim outruns a single warm-cache laptop benchmark. read the letter →

arxiv 2501.07666 v3 pith:PIIGULOO submitted 2025-01-13 physics.data-an

Gagik Gavalian This is my paper · ORCID
classification physics.data-an
keywords HiPOcolumnardatastorageeventtaggingformatbenchmarkingnuclearphysicsanalysisROOTRNTupleApacheParquethistogramfilling
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper presents the HiPO data format, developed for the CLAS12 nuclear physics experiment, and argues that the same format can serve the full data lifecycle, from raw detector output to columnar physics analysis. The paper claims that HiPO's columnar mode, which stores each table column in separately tagged records, lets analysis programs read only the columns they need and fill histograms without deserializing the rest of the file. On a synthetic 24-column, 50-million-row file, the reported read-and-fill times are about 0.5 seconds for HiPO, about 1.6 seconds for ROOT RNTuple, and about 1.8 seconds for Parquet when only four columns are read. A sympathetic reader would take this as evidence that a format originally built for one experiment's event data can compete with and outperform general-purpose columnar formats in common analysis workloads.

What carries the argument

The load-bearing mechanism is the record-tagging scheme. In columnar mode, each column is assigned a unique tag and serialized into its own record, or bucket; the file footer maps every record's position, size, and tag. When a user declares branches to read, the reader opens only records whose tags match, deserializes those columns, and synchronizes rows across columns. A second piece of machinery is a purpose-built histogramming routine used in the benchmark, which the paper says avoids overhead in ROOT's TH1D.Fill() and contributes to the measured gap.

What would settle it

Run the same four-column read-and-fill benchmark on a multi-user cluster filesystem using real reconstructed physics data with non-uniform column distributions and many small files, and check whether HiPO's recorded lead over ROOT RNTuple and Parquet persists; a finding that ROOT or Parquet matches or beats HiPO under those conditions would falsify the paper's general performance claim.

Watch

Extended reading notes

Core claim

The central claim is that HiPO's columnar storage design is faster than established formats for selective column reads. The mechanism is record tagging: each column is written into its own record with a unique tag, and the file footer records each record's position and tag, so a reader can jump directly to requested columns. The paper reports that in a loop-based histogram fill over four of 24 columns, HiPO takes 0.523 seconds versus 1.589 seconds for ROOT RNTuple after both use the same simplified histogramming code, and that HiPO Java DataFrame bulk filling reaches 0.452 seconds, compared with 1.812 seconds for Parquet. The paper also states that interactive use in JShell does not degrade HiPO performance, while RNTuple slows by a factor of about two in interactive mode.

Load-bearing premise

The performance claim rests on the assumption that a synthetic 50-million-row file of uniform random numbers read from a laptop SSD represents the I/O patterns and data characteristics of real experimental analysis on cluster file systems.

Editorial extensions

If this is right

  • A single HiPO file can carry raw, reconstructed, and analysis-ready columnar data, removing format-conversion steps across the data lifecycle.
  • Analysis programs that read a few columns from wide tables can expect substantially lower I/O and CPU time with HiPO than with ROOT TTree, RNTuple, or Parquet in similar single-machine benchmarks.
  • The DataFrame-style bulk histogram fill is reported to be about 50% faster than the loop-based HiPO read, suggesting a cheap speedup for histogram-heavy workflows.
  • Interactive exploratory environments such as JShell can keep HiPO's read speed, while ROOT RNTuple's interactive read is reported to be roughly twice as slow as its batch read.
  • Because the columnar implementation is still labeled experimental, the performance numbers are presented as preliminary rather than as a final product.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • If the record-tagging scheme generalizes, it could be adopted by other event-based formats as a lightweight columnar-read layer without restructuring their event model.
  • The benchmark's synthetic uniform random columns are unusually compressible and carry no selection selectivity; real physics columns with sparse values and correlated branches may shift the relative performance, so the threefold claim should be tested on real experimental data.
  • The reported advantage is for reading from a local SSD on a single laptop; on shared cluster file systems with many concurrent readers, metadata lookups and random seeks could change the ranking.
  • A testable extension is whether the tagging scheme helps when reading only a small fraction of rows by tag (event selection) as well as a subset of columns, since the format already supports tag-filtered event reads.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 6 minor

Summary. The paper presents HiPO, a data format developed at Jefferson Laboratory for the CLAS12 experiment, designed to cover the full data lifecycle from raw detector data to columnar analysis. It describes the file structure (header, records, footer), event and bank/tuple APIs, an event-tagging mechanism for selective access to records, and an experimental columnar storage mode. The principal quantitative claim is in Section 5.2: a benchmark on 50M rows of synthetic data reports that HiPO reads four columns and fills histograms roughly three times faster than ROOT RNTuple in a simplified test, and faster than TTree and Parquet in other comparisons. This leads to the Abstract's claim that HiPO "surpasses established data formats in performance" in columnar data analysis and can be applied to other scientific fields.

Significance. If the performance claims were rigorously established, the paper would be relevant to data-intensive nuclear and particle physics analyses, where ROOT and Parquet dominate. The HiPO format has the strength of having been in production for CLAS12 for several years, with bindings to C++, Java, Python, Julia, and FORTRAN, and the paper gives a clear description of the file layout and APIs. The benchmark, despite its limitations, is a concrete attempt at quantitative comparison and the authors have been transparent about some of the setup. However, the evidence for the headline claim is currently a single warm-cache, single-machine, synthetic-data benchmark with no error bars, no public C++ implementation, and no released dataset, which is insufficient for the broad "surpasses established data formats" and "other scientific fields" conclusions.

major comments (4)
  1. [§5.2, Table 1] The benchmark is not reproducible: the C++ source for HiPO is not yet public (the paper states it "will be made public after thorough debugging"), the synthetic dataset is not released, and the tests were run on a single M1 MacBook with the file in the OS page cache (ten consecutive reads, last four averaged). No error bars or standard deviations are reported. This narrow setup cannot support the general claim in the Abstract and Section 6 that HiPO "surpasses established data formats" for columnar data analysis; the paper should either provide a full reproducibility package or restrict the claim to the specific tested configuration.
  2. [§5.2, Table 1, Listing 8] The comparison is inconsistent across formats: Parquet is tested only via a DataFrame path, while HiPO is also tested via a native C++ loop; the simplified histogram test (Listing 8) that yields the "three times faster" figure compares only HiPO with RNTuple, not with TTree or Parquet. As a result, the data do not establish the broad claim that HiPO outperforms all established formats in columnar analysis. A consistent set of comparisons (same API level, same operations, same file sizes) across all formats is needed, or the conclusions must be limited to the specific implementations compared.
  3. [§5.2, ROOT version choice] The 3x speedup claim depends on the choice of ROOT 6.34 as the comparison baseline, but the paper acknowledges that ROOT 6.32 gave RNTuple times of 1.87 seconds versus 2.62 seconds in 6.34, and justifies 6.34 only by stating that "version 6.34 incorporates format changes that will serve as the foundation for future developments." This is not a neutral basis for selecting the comparison version. The authors need to justify why ROOT 6.34 is the fair baseline (e.g., by describing the relevant format changes and their impact) or report results for both versions; otherwise the reported speedup may overstate the improvement.
  4. [§5.2 and Section 6] The benchmark exercises only full sequential scans of four float columns with histogram filling. It does not test predicate selections, varying column selectivity, integer/string/variable-length data, write performance, or parallel/cluster file systems. The paper itself states, in Section 5.2, that "more comprehensive performance tests are beyond the scope of this article," which is an explicit acknowledgment of limitation. Given this, the Abstract's assertion that HiPO "can be effectively applied to data analysis in other scientific fields" is not supported by the evidence presented; the claims should be tempered or the tests expanded.
minor comments (6)
  1. [Abstract and Table 1] "Parquete" should be "Parquet" in the Abstract, Section 5.2 heading, and Table 1.
  2. [Listings 5 and 6] The comment "Writing arrays into an Event" appears above both Listing 5 and Listing 6, but the code in these listings demonstrates tagged file reading and writing, not writing arrays into an event.
  3. [Listing 4] The printf statement in Listing 4 uses the variable `row`, but `row` is not declared in the snippet; this makes the example incomplete and potentially confusing.
  4. [Listing 1] HiPO files are given the extension ".h5" in Listing 1 and elsewhere, which may be confused with HDF5 files; a distinct extension would avoid ambiguity.
  5. [Table 1] The row "Tuple HiPO Julia DataFrame" is described in the text as a "preliminary benchmark" and a "port," but the table gives no such qualification; the table should note the status of the Julia implementation.
  6. [§5.2] The paper should include a data-availability statement for the synthetic dataset and the benchmark scripts, particularly since the C++ code is not yet public.

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity: the HiPO paper is a descriptive format and benchmark study whose performance claims rest on measured comparisons against external tools, not on fitted inputs or self-citation chains.

full rationale

The paper contains no derivation chain that reduces to its own inputs. Section 3 describes the HiPO file layout and API, Section 5.2 reports measured read-and-fill times for HiPO, ROOT TTree/RNTuple, and Parquet on a synthetic 24-column, 50M-row dataset, and the performance comparison is empirical rather than a prediction derived from fitted parameters or from the format definition itself. The only self-reference is citation [7], the public HiPO repository, which points to the artifact under discussion and is not used as evidence for the benchmark results; the timing numbers are presented as direct measurements, with the test conditions stated (M1 MacBook, 1 TB SSD, JDK 21, ROOT 6.34.08, ten reads with the last four averaged). The paper even discloses that ROOT 6.32 gave faster RNTuple times than 6.34 and that the C++ columnar source will be made public later, which cuts against any suggestion that unfavorable data was suppressed. The main concerns—one synthetic dataset, warm-cache timing, single-machine hardware, and the same author developing both the format and the benchmark—are validity and generalizability limitations, not circularity. Under the hard rules, worry about benchmark realism belongs in correctness risk, not in the circularity score. No equation, definition, or cited theorem is equivalent to the paper's central claim, so the honest finding is no significant circularity.

Assumptions & free parameters 0 free parameters · 3 assumptions · 0 invented entities

The HiPO format is a real software artifact, not a postulated entity, so no invented entities are listed. The axioms are the implicit assumptions that make the benchmark results meaningful as a general performance claim.

assumptions (3)
  • domain assumption A synthetic dataset of 50M rows of uniform random numbers in [0,1] is representative of experimental columnar analysis workloads.
    The benchmark and the conclusion that HiPO surpasses established formats depend on this representativeness. Stated in Section 5.2.
  • domain assumption Performance measured on a single M1 MacBook laptop with a 1 TB SSD generalizes to production cluster environments.
    The paper generalizes from this single machine to broad claims in the Abstract and Section 6. The benchmark hardware is described in Section 5.2.
  • ad hoc to paper ROOT 6.34 is the appropriate version for comparison because it incorporates future format changes.
    The paper notes ROOT 6.32 was faster (1.87s versus 2.62s) but chooses 6.34 as the moving-forward version, which is a judgment call that affects the comparison. Stated in Section 5.2.

how reviews work

0 comments
Cite this review

Pith. "Pith review of High-Performance Data Format for Scientific Data Storage and Analysis." pith.science (2026). https://pith.science/paper/PIIGULOO

@misc{pith2026250107666,
  author       = {Pith},
  title        = {Pith review of: High-Performance Data Format for Scientific Data Storage and Analysis},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/PIIGULOO}},
  note         = {Machine review of arXiv:2501.07666}
}
read the original abstract

In this article, we present the High-Performance Output (HiPO) data format developed at Jefferson Laboratory for storing and analyzing data from Nuclear Physics experiments. The format was designed to efficiently store large amounts of experimental data, utilizing modern fast compression algorithms. The purpose of this development was to provide organized data in the output, facilitating access to relevant information within the large data files. The HiPO data format has features that are suited for storing raw detector data, reconstruction data, and the final physics analysis data efficiently, eliminating the need to do data conversions through the lifecycle of experimental data. The HiPO data format is implemented in C++ and JAVA, and provides bindings to FORTRAN, Python, and Julia, providing users with the choice of data analysis frameworks to use. In this paper, we will present the general design and functionalities of the HiPO library and compare the performance of the library with more established data formats used in data analysis in High Energy and Nuclear Physics (such as ROOT and Parquete).

Figures

Figures reproduced from arXiv: 2501.07666 by the authors.

Figure 1
Figure 1. Schematic view of the HiPO file structure. The figure shows the conceptual design of the file [PITH_FULL_IMAGE:figures/full_fig_p004_1.png] view at source ↗
Figure 2
Figure 2. Schematic view of the file output for tagged and untagged events. When using the tagged mode [PITH_FULL_IMAGE:figures/full_fig_p008_2.png] view at source ↗
Figure 3
Figure 3. Schematic view of arranging each event in the file from columnar table. [PITH_FULL_IMAGE:figures/full_fig_p010_3.png] view at source ↗
Figures from the paper (1 more)
Figure 4
Figure 4. Figure 4: Reading benchmark for different file formats reading the columnar data with 24 columns and 50 M [PITH_FULL_IMAGE:figures/full_fig_p011_4.png]

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. OASIS: Object-based Analytics Storage for Intelligent SQL Query Offloading in Scientific Tabular Workloads

    cs.DB 2025-09 conditional novelty 6.0 of 10

    OASIS offloads filters, aggregates, sorts, and array-aware expressions onto a two-level object storage stack, reducing SQL query times in HPC analytics by up to 32.7% versus an emulated COS baseline.

Reference graph

Works this paper leans on

7 extracted references · 5 canonical work pages · cited by 1 Pith paper

  1. [1]

    The CEBAF Large Acceptance Spectrometer (CLAS),

    B. A. Mecking et al. [CLAS], “The CEBAF Large Acceptance Spectrometer (CLAS),” Nucl. Instrum. Meth. A 503 (2003), 513-553 doi:10.1016/S0168-9002(03)01001-5

    doi:10.1016/s0168-9002(03)01001-5 2003

  2. [2]

    and others, The CLAS12 Spectrometer at Jefferson Laboratory, Nucl

    Burkert, V.D. and others, The CLAS12 Spectrometer at Jefferson Laboratory, Nucl. Instrum. Meth. A 959,163419 (2020)

    work page 2020

  3. [3]

    ROOT: An object oriented data analysis framework,

    R. Brun and F. Rademakers, “ROOT: An object oriented data analysis framework,” Nucl. Instrum. Meth. A 389 (1997), 81-86 doi:10.1016/S0168-9002(97)00048-X

    doi:10.1016/s0168-9002(97)00048-x 1997

  4. [4]

    LCIO: A Per- sistency Framework and Event Data Model for HEP,

    S. Aplin, J. Engels, F. Gaede, N. A. Graf, T. Johnson and J. McCormick, “LCIO: A Per- sistency Framework and Event Data Model for HEP,” doi:10.1109/NSSMIC.2012.6551478

    work page arXiv 2012

  5. [5]

    The HDF Group, http://www.hdfgroup.org/HDF5/

    work page

  6. [6]

    Apache Parquet, https://parquet.apache.org

    work page

  7. [7]

    The HiPO Group, https://github.com/gavalian/hipo 13

    work page

Pith tools

Reviewed August 10, 2026 · model on record in the stance chip above.