Pith. sign in

REVIEW 4 major objections 5 minor 16 references

CkIO: Parallel File Input for Over-Decomposed Task-Based Systems

T0 review · 4 major / 5 minor · reviewed 2026-08-12 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read Splitting file-reading tasks from data-consuming tasks keeps parallel input fast no matter how finely the application is decomposed, cutting ChaNGa's input time by over half.

desk verdict CkIO is a useful, well-motivated two-phase input library for Charm++-style overdecomposition, but the paper overstates the scaling claim beyond the demonstrated 64 clients/PE envelope. read the letter →

arxiv 2411.18593 v2 pith:PYYEO47L submitted 2024-11-27 cs.DC

Mathew Jacob , Maya Taylor , Laxmikant Kale This is my paper · ORCID
classification cs.DC
keywords parallelI/Ooverdecompositiontask-basedruntimetwo-phaseasynchronouscallbacksCharm++ChaNGafileinput
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

In task-based systems that overdecompose work into many more tasks than cores, letting every task read directly from a shared file congests the file system and makes input time depend on task count. This paper argues that input decomposition should be independent of application decomposition: a small configurable set of file-reader tasks should fetch large contiguous chunks from disk, and the consumers of that data should receive it over the fast node interconnect rather than from the file system. The resulting library, CkIO, reads asynchronously through callback continuations, so computation can proceed while input is in flight, and tasks can migrate between nodes while a read session is open. The payoff claimed is stable throughput regardless of overdecomposition factor, and a more-than-2x speedup in the ChaNGa cosmological simulation compared with its hand-optimized input path.

What carries the argument

The load-bearing object is the buffer chare array, a configurable set of reader tasks inserted between the file system and the application's client tasks (chares are the task objects of the Charm++ runtime). Each buffer chare owns a disjoint byte range of the file and, when a read session starts, spawns a helper pthread to read that range into memory asynchronously; client read requests are then fulfilled as zero-copy transfers from the buffer chare to the requesting processor's ReadAssembler, which assembles the pieces and fires a split-phase callback to the waiting client. Following two-phase I/O, this separate decomposition replaces many small uncoordinated file-system calls with a few large ones, and the callback-based API is what allows the scheduler to overlap I/O with unrelated computation and lets a migrated client keep reading through a virtual proxy.

What would settle it

On a cluster where a local NVMe or burst-buffer file system delivers data faster than the interconnect can transfer it, run CkIO's disjoint-read benchmark with one buffer chare per node and 64 clients per PE: if the CkIO path is slower than naive direct reads at the same client count, or if end-to-end throughput drops as client count increases, the claim of decomposition-independent I/O performance fails.

Watch

Extended reading notes

Core claim

The central claim is that the performance of parallel file input in an overdecomposed task-based system can be decoupled from the application's own choice of decomposition by inserting an intermediary layer between the file system and the client tasks. CkIO implements this with a buffer chare array: each buffer chare is responsible for a disjoint section of the file, begins reading its section greedily when a read session starts, and later serves client read requests from memory over the interconnect. Requests from clients on a processor are funneled through a per-processor ReadAssembler, which gathers the pieces and invokes a user callback when the read completes, keeping the read non-blocking and splittable. Because the number and placement of buffer chares can be tuned independently of the number of clients, the authors claim that the file system sees an optimal number of large reads no matter how many application tasks exist. They report that CkIO matches or beats MPI-I/O on disjoint reads, maintains over 75% overlap of input with background work up to 64 clients per PE, supports migration of clients holding active sessions, and gives ChaNGa over a 2x speedup over the hand-optimized input code.

Load-bearing premise

The design assumes that moving data over the node interconnect is much faster than reading it from the file system, so the extra buffer-chare-to-client transfer is always a net win.

Editorial extensions

If this is right

  • An application can choose its overdecomposition factor purely for computational efficiency; input throughput no longer forces a particular client count.
  • File input need not stall computation: because reads are split-phase callbacks, the runtime can schedule unrelated tasks while data is being fetched.
  • Persistent tasks can migrate between nodes mid-read-session without breaking input, enabling load balancing and data-locality optimizations such as moving clients to the node holding their data.
  • On systems where the network is much faster than the file system, a few buffer chares reading large contiguous chunks can match or beat MPI-I/O's collective reads.
  • For an existing production code such as ChaNGa, replacing a hand-optimized application-level collective input scheme with CkIO improves input time by more than 2x while simplifying the code.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The core assumption—network cheaper than disk—is explicitly a machine-dependent ratio; on clusters with very fast local storage or a slow interconnect, the extra buffer-chare-to-client hop could make CkIO slower than direct reads, so the claimed independence from decomposition would not hold.
  • The buffer-chare count is a manual tuning knob; an automatic selector based on file size, node count, and file-system characteristics would make the performance-independence claim turnkey, and prior aggregator-selection work for MPI-I/O suggests such a selector is feasible.
  • The splintered-I/O idea the paper lists as future work—reading buffer-chare chunks in smaller pieces so small requests are served without waiting for a whole large block—could reduce latency for sparse access patterns like those in graph algorithms.
  • The same two-phase, callback-based structure should extend beyond raw sequential files to hierarchical formats such as HDF5, since the clients only consume already-materialized byte ranges; the paper notes this possibility.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. The paper presents CkIO, a two-phase input library for Charm++ and other overdecomposed task-based systems. CkIO introduces a configurable array of buffer chares that read file data asynchronously and serve client chares' requests through an intermediate ReadAssembler, decoupling the file-read decomposition from the application's overdecomposition. The authors claim that this decoupling lets applications scale the number of consumers without affecting I/O performance, supports computation/I/O overlap, and enables object migration during active file sessions. The evaluation consists of microbenchmarks (disjoint reads vs MPI-IO, computation overlap, migration) and an integration into ChaNGa, reporting over 2x speedup over the hand-optimized input path.

Significance. If the central claims hold, CkIO would fill a real gap: overdecomposed task-based runtimes lack a principled parallel input layer that preserves migration and nonblocking overlap. The paper provides several independent benchmarks and a real application integration, which is creditworthy. The proposed abstraction is simple and generally applicable, and the comparisons to MPI-IO and to a hand-optimized production code are appropriate. However, as detailed below, the scaling claim is overbroad relative to the data, the ChaNGa speedup metric is not robust, and the printed architecture description is internally inconsistent. These are correctable issues rather than fatal design flaws.

major comments (4)
  1. [Abstract, Section I, Section III-C, Figure 6a, Figure 9] The paper's central claim is not supported beyond the tested range and is at odds with the broadcast design. The abstract and Section I state that separating consumer decomposition from file-reader tasks 'enables applications to scale the number of consumers of data without impacting I/O behavior or performance.' However, Section III-C and Figure 6a describe that every client read request is broadcast by the manager to all buffer chares, so the message volume is O(#clients x #buffer chares). Figure 9 shows that with 8 buffer chares on 8 PEs, increasing clients from 1 to 1024 raises the read-phase time from about 0.05 s to 0.30 s and reduces the background-work fraction from 92% to 78%, and the text attributes this to buffer chares serially handling requests. Figure 4, the main evidence for the scaling claim, only reaches 2^15 clients on 512 PEs (64 clients/PE), and Section V-C's statement that the curve is stable 'even up to 256 clients per PE' is not supported by that figure. The abstract and conclusion should be rewritten to state a qualified scaling range, or the design should be changed to avoid the broadcast.
  2. [Section IV-B, Figure 13b] The ChaNGa speedup is computed using a biased summary statistic. Figure 13b's caption explains that while plotted runtimes are means, the speedup is the ratio of the minimum (best) iteration of each implementation. With filesystem variability, which the paper itself acknowledges, min-to-min ratios can substantially overstate the typical speedup, especially when the baseline's best run is unusually fast or CkIO's best run is unusually slow. No error bars are shown in Figure 13a or 13b, and the number of repetitions is not stated. The paper should report the mean of per-pair speedups (or median with confidence intervals) and should state the number of runs.
  3. [Section III-C.3, Figure 6a] There is a direct inconsistency between the architectural description and the communication diagram. Section III-C.3 says the ReadAssembler 'issues the request to the required buffer chares,' implying only buffer chares holding relevant data are contacted. But Figure 6a and its caption state that 'the manager broadcasts the request to all Buffer Chares.' These are different message patterns with different complexity. If the broadcast is what is implemented, the O(clients x buffer chares) growth explains the degradation in Figure 9 and contradicts the scaling claim even more directly; if the targeted request is what is implemented, Figure 6a must be corrected. The authors should clarify which behavior is implemented and, if it is the targeted one, explain how the assembler knows which buffer chares to contact.
  4. [Section I (contributions), Section IV] The paper presents CkIO as an 'open source implementation' but gives no repository, version, DOI, or artifact identifier. Without access to the code, or at least a detailed specification of the benchmarking harness, the experiments—especially the ChaNGa integration—cannot be reproduced or independently verified. The authors should provide a persistent link to the code and data (or a clear explanation of why this is not possible).
minor comments (5)
  1. [Figure 9 and accompanying text] The text says 'up to1024 clients (64 clients per PE)' on 4 nodes with 2 cores per node, i.e., 8 PEs total; 1024/8 = 128 clients per PE, so the parenthetical is incorrect.
  2. [Section V-C, Figure 4] Section V-C claims the runtime 'remains relatively stable, even up to 256 clients per PE,' but Figure 4's largest point is 2^15 clients on 512 PEs, which is 64 clients per PE, not 256.
  3. [Figures 7, 8, 9, 12, 13] Several figures plot means without error bars or stated numbers of runs, despite the paper's own observation of filesystem variability. Adding error bars or at least reporting the number of repetitions for each configuration is necessary to assess whether observed differences are meaningful.
  4. [Section IV-A.3, Figure 12] The migration experiment appears to have only two client chares and two buffer chares on two nodes. The paper should state how many runs were averaged and whether the reported read time (the maximum of the two clients) is stable across runs.
  5. [General] There are several typos and minor language issues, including 'Party due to' (Section I), 'Futhermore' (Section I), 'encapsultes' (Section II-B.1), and 'the file input itself is asynchronous' (Section III-D, which reads awkwardly). A copyedit pass is needed.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: CkIO is an empirical systems paper with no fitted-input-as-prediction chain and its benchmarks are measured against external baselines.

full rationale

The paper contains no mathematical derivation, fitted parameter, or uniqueness theorem whose conclusion is equivalent to its inputs. The load-bearing design premise, that network transfer is faster than file-system reads, is supported by an independent measurement in Figure 2 rather than by definition or by a self-citation. Self-references to Charm++ [11] and to the prior CkIO output work [13] are infrastructure citations: they describe the programming model and an existing output API, but they do not pre-encode or assert the benchmark outcomes claimed here. The central performance comparisons are against external baselines (MPI-IO collective input, naive direct reads, and ChaNGa's hand-optimized input code), so the claimed speedups are externally falsifiable rather than circular. The paper also candidly reports the regime where the central scaling claim weakens (Figure 9, with degradation beyond 64 clients per PE) and lists it as a limitation, which further shows the evaluation is not constructed to force the advertised conclusion. Overall, no step in the paper reduces by construction to its own input, and any remaining concerns about the strength of the scaling claim are empirical-correctness questions, not circularity.

Assumptions & free parameters 1 free parameters · 4 assumptions · 0 invented entities

No mathematical derivation is attempted; the central claims are empirical. The load-bearing assumptions are the measured interconnect/disk speed gap, the sweet-spot reader-count behavior, sequential file layout, and the availability of Charm++ runtime primitives. The main hand-chosen parameter is the buffer chare count, which the paper acknowledges is not auto-tuned.

free parameters (1)
  • numReaders (number of buffer chares) = Hand-set per experiment: 32 or 64 per node (Figure 7), 2^9 total (Figure 4), 8 (Figures 8 and 9), 2 (Figure 12)
    This knob controls file system concurrency in CkIO and is chosen manually for each benchmark. The paper does not provide a default selection policy, and Section VI-A lists auto-tuning as future work.
assumptions (4)
  • domain assumption The node interconnect is much faster than the file system for moving the same data, so the extra buffer-chare-to-client transfer is a net win.
    Section I and Figure 2 measure a >6x gap on Bridges2 and generalize it to 'most supercomputing clusters'; the design does not address systems where the ratio is reversed.
  • domain assumption File input throughput has a non-monotonic dependence on the number of readers: too few readers underuse disk parallelism and too many readers congest the file system.
    Figure 1 motivates the need for a tunable reader layer; CkIO's buffer chare count is the manual mechanism for hitting the sweet spot, and no automatic policy is provided.
  • domain assumption Input files have a sequential organization; structured formats such as HDF5 are out of scope.
    Section II-C states this assumption explicitly as a limitation of the current design.
  • domain assumption Charm++ runtime provides the required primitives: chare groups, virtual proxies, zero-copy transfers, non-preemptive message-driven scheduling.
    Sections II-B and III-C build directly on these mechanisms; results may not transfer to other AMT runtimes lacking them.

how reviews work

0 comments
Cite this review

Pith. "Pith review of CkIO: Parallel File Input for Over-Decomposed Task-Based Systems." pith.science (2026). https://pith.science/paper/PYYEO47L

@misc{pith2026241118593,
  author       = {Pith},
  title        = {Pith review of: CkIO: Parallel File Input for Over-Decomposed Task-Based Systems},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/PYYEO47L}},
  note         = {Machine review of arXiv:2411.18593}
}
read the original abstract

Parallel input performance issues are often neglected in large scale parallel applications in Computational Science and Engineering. Traditionally, there has been less focus on input performance because either input sizes are small (as in biomolecular simulations) or the time doing input is insignificant compared with the simulation with many timesteps. But newer applications, such as graph algorithms add a premium to file input performance. Additionally, over-decomposed systems, such as Charm++/AMPI, present new challenges in this context in comparison to MPI applications. In the over-decomposition model, naive parallel I/O in which every task makes its own I/O request is impractical. Furthermore, load balancing supported by models such as Charm++/AMPI precludes assumption of data contiguity on individual nodes. We develop a new I/O abstraction to address these issues by separating the decomposition of consumers of input data from that of file-reader tasks that interact with the file system. This enables applications to scale the number of consumers of data without impacting I/O behavior or performance. These ideas are implemented in a new input library, CkIO, that is built on Charm++, which is a well-known task-based and overdecomposed-partitions system. CkIO is configurable via multiple parameters (such as the number of file readers and/or their placement) that can be tuned depending on characteristics of the application, such as file size and number of application objects. Additionally, CkIO input allows for capabilities such as effective overlap of input and application-level computation, as well as load balancing and migration. We describe the relevant challenges in understanding file system behavior and architecture, the design alternatives being explored, and preliminary performance data.

Figures

Figures reproduced from arXiv: 2411.18593 by the authors.

Figure 1
Figure 1. Naive overdecomposed input in Charm++. Results were [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 4
Figure 4. Performance of naive parallel input (where each client [PITH_FULL_IMAGE:figures/full_fig_p005_4.png] view at source ↗
Figure 3
Figure 3. Schematics of (a) naive parallel input vs (b) input with [PITH_FULL_IMAGE:figures/full_fig_p005_3.png] view at source ↗
Figures from the paper (8 more)
Figure 5
Figure 5. Figure 5: Diagram of the CkIO system architecture. Note that the Buffer Chares begin reading on session instantiation, without [PITH_FULL_IMAGE:figures/full_fig_p006_5.png]
Figure 6
Figure 6. Figure 6: Outline of the communication involved when CkIO receives a request from a Client. [PITH_FULL_IMAGE:figures/full_fig_p007_6.png]
Figure 8
Figure 8. Figure 8: Runtime comparison between CkIO input and naive [PITH_FULL_IMAGE:figures/full_fig_p008_8.png]
Figure 9
Figure 9. Figure 9: Execution time and percentage of time spent on [PITH_FULL_IMAGE:figures/full_fig_p008_9.png]
Figure 10
Figure 10. Figure 10: Diagram of the start of the experiment. The arrows [PITH_FULL_IMAGE:figures/full_fig_p009_10.png]
Figure 11
Figure 11. Figure 11: Diagram of the migration experiment after a migration [PITH_FULL_IMAGE:figures/full_fig_p009_11.png]
Figure 12
Figure 12. Figure 12: The performance difference in read times before vs [PITH_FULL_IMAGE:figures/full_fig_p010_12.png]
Figure 13
Figure 13. Figure 13: Runtime comparisons (a) and speedup (b) of the file input involved in a ChaNGa test code under three IO [PITH_FULL_IMAGE:figures/full_fig_p011_13.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

16 extracted references · 16 canonical work pages

  1. [1]

    Power, reliability, and performance: One system to rule them all

    Bilge Acun, Akhil Langer, Esteban Meneses, Harshitha Menon, Osman Sarood, Ehsan Totoni, and Laxmikant Kal ´e. Power, reliability, and performance: One system to rule them all. Computer, 49:30–37, 10 2016

    work page 2016

  2. [2]

    Starpu: A unified platform for task scheduling on heterogeneous multicore architectures

    C ´edric Augonnet, Samuel Thibault, Raymond Namyst, and Pierre- Andr´e Wacrenier. Starpu: A unified platform for task scheduling on heterogeneous multicore architectures. volume 23, 08 2009

    work page 2009

  3. [3]

    Legion: Expressing locality and independence with logical regions

    Michael Bauer, Sean Treichler, Elliott Slaughter, and Alex Aiken. Legion: Expressing locality and independence with logical regions. pages 1–11, 11 2012

    work page 2012

  4. [4]

    Optimizing i/o performance of hpc applications with autotuning

    Babak Behzad, Suren Byna, Mr Prabhat, and Marc Snir. Optimizing i/o performance of hpc applications with autotuning. ACM Transactions on Parallel Computing, 5:1–27, 03 2019

    work page 2019

  5. [5]

    Bridges-2: A platform for rapidly- evolving and data intensive research

    Shawn Brown, Paola Buitrago, Edward Hanna, Sergiu Sanielevici, Robin Scibek, and Nicholas Nystrom. Bridges-2: A platform for rapidly- evolving and data intensive research. pages 1–4, 07 2021

    work page 2021

  6. [6]

    Automatically selecting the number of aggregators for collective i/o operations

    Mohamad Chaarawi and Edgar Gabriel. Automatically selecting the number of aggregators for collective i/o operations. pages 428–437, 09 2011

    work page 2011

  7. [7]

    Im- proved parallel i/o via a two-phase run-time access strategy

    Juan Miguel del Rosario, Rajesh Bordawekar, and Alok Choudhary. Im- proved parallel i/o via a two-phase run-time access strategy. SIGARCH Comput. Archit. News , 21(5):31–38, dec 1993

    work page 1993

  8. [8]

    Massively parallel cosmological simulations with changa

    Pritish Jetley, Filippo Gioachin, Celso Mendes, Laxmikant Kal ´e, and Thomas Quinn. Massively parallel cosmological simulations with changa. pages 1–12, 04 2008

    work page 2008

Show all 16 references
  1. [9]

    Inte- grating external resources with a task-based programming model

    Zhihao Jia, Sean Treichler, Galen Shipman, Michael Bauer, Noah Watkins, Carlos Maltzahn, Patrick McCormick, and Alex Aiken. Inte- grating external resources with a task-based programming model. pages 307–316, 12 2017

    2017

  2. [10]

    Hpx - the c++ standard library for parallelism and concurrency

    Hartmut Kaiser, Patrick Diehl, Adrian Lemoine, Bryce Lelbach, Parsa Amini, Agust´ın Berge, John Biddiscombe, Steven Brandt, Nikunj Gupta, Thomas Heller, Kevin Huck, Zahra Khatami, Alireza Kheirkhanan, Auriane Reverdell, Shahrzad Shirzad, Mikael Simberg, Bibek Wagle, Weile Wei,...

    2020

  3. [11]

    Charm++: A portable concurrent object oriented system based on c++

    Laxmikant Kale and Sanjeev Krishnan. Charm++: A portable concurrent object oriented system based on c++. ACM Sigplan Notes, 28, 10 1995

    1995

  4. [12]

    Hierarchical Data Format 5 : HDF5, pages 191–200

    Sandeep Koranne. Hierarchical Data Format 5 : HDF5, pages 191–200. Springer US, Boston, MA, 2011

    2011

  5. [13]

    Asynchronous collective output with non-dedicated cores

    Phil Miller, Shen Li, and Chao Mei. Asynchronous collective output with non-dedicated cores. In Workshop on Interfaces and Architectures for Scientific Data Storage , September 2011

    2011

  6. [14]

    Tipsy: Code for display and analysis of n-body simula- tions

    N-Body Shop. Tipsy: Code for display and analysis of n-body simula- tions. Astrophysics Source Code Library , pages 11015–, 11 2011

    2011

  7. [15]

    Tapi- oca: An i/o library for optimized topology-aware data aggregation on large-scale supercomputers

    Franc ¸ois Tessier, Venkatram Vishwanath, and Emmanuel Jeannot. Tapi- oca: An i/o library for optimized topology-aware data aggregation on large-scale supercomputers. In 2017 IEEE International Conference on Cluster Computing (CLUSTER) , pages 70–80, 2017

    2017

  8. [16]

    Data sieving and collective i/o in romio

    Rajeev Thakur, William Gropp, and Ewing Lusk. Data sieving and collective i/o in romio. pages 182–189, 03 1999

    1999

Pith tools

Reviewed August 12, 2026 · model on record in the stance chip above.