REVIEW 3 major objections 5 minor 40 references
Accelerating Loading WebGraphs in ParaGrapher
T0 review · 3 major / 5 minor · reviewed 2026-08-06 · deepseek-v4-flash
Pith's one-line read Two tweaks speed compressed-graph loading by up to 21.8x
desk verdict Honest, modest performance engineering with PG-Fuse speedups that need cache-controlled reruns before I'd trust the numbers. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The load-bearing pieces are two mechanisms. PG-Fuse is a FUSE filesystem that partitions each graph file into 32 MiB blocks, tracks each block with an atomic status (unloaded, being loaded, accessible, under revocation), and answers concurrent reads from an in-memory cache, revoking least-recently-used blocks. CompBin is a minimal-byte CSR: the offsets array remains standard, while the neighbors array packs vertex IDs into $b=\lceil(\log_2|V|)/8\rceil$ bytes, and the $n$-th neighbor of vertex $v$ is decoded as $\sum_{i=0}^{b-1} \mathrm{neighbors}[(\mathrm{offsets}[v]+n)b+i] \ll 8i$. This preserves direct indexing while shrinking the array and making decompression a few shift-add operations.
What would settle it
Re-run the Section V loading benchmarks twice per configuration, once after dropping the OS page cache and once with the graph files warmed in memory, and compare PG-Fuse's speedup in the two cases; a large gap would show that the reported 7.6x depends on page-cache state rather than on 32 MiB block reads.
Extended reading notes
Core claim
The paper's central claim is that the dominant cost in loading compressed WebGraphs is not decompression per se but the interaction of small storage requests and per-vertex decode overhead, and that both are addressable. PG-Fuse replaces the Java side's ~128 kB read pattern with 32 MiB block reads cached in user space, so high-bandwidth storage is used efficiently and repeated block accesses never touch the filesystem again. CompBin stores the neighbors array with $b=\lceil(\log_2|V|)/8\rceil$ bytes per vertex ID, so a neighbor is recovered by combining $b$ bytes with shifts and adds; when $|V| \ge 2^{24}$ this is exactly the standard 4-byte binary CSR. On 12 graphs up to 128.7 billion edges, PG-Fuse gives 0.9–7.6x speedups and CompBin gives up to 21.8x, with CompBin winning on small graphs and PG-Fuse winning once the storage-size gap to WebGraph exceeds roughly 100 GiB.
Load-bearing premise
The measurements assume the shared Lustre filesystem and OS page cache behave as a cold, high-bandwidth device: every dataset fits in the machine's 2 TB RAM and the paper does not state whether caches were flushed between runs, so part of the speedup could come from data already being in memory.
Editorial extensions
If this is right
- Users of ParaGrapher can enable PG-Fuse without modifying WebGraph, so future WebGraph versions remain usable.
- CompBin is equivalent to binary CSR once $|V| \ge 2^{24}$; the small-graph speedups therefore extend to any 4-byte CSR loader facing decompression-bound loads.
- For graphs whose storage size is within roughly 50 GiB of the WebGraph version, CompBin/binary CSR loads faster; beyond roughly 100 GiB, PG-Fuse wins on this system.
- The thresholds separating the two regimes are hardware-dependent: faster storage or slower CPUs will shift the crossover.
Reading between the lines
- A hybrid loader that measures graph size and picks PG-Fuse or CompBin automatically would capture both regimes; the paper lists hybrid policies as future work.
- PG-Fuse's block cache suggests an obvious extension: track thread access order and prefetch the next block before it is needed, converting latency into bandwidth.
- CompBin's fixed-width packing applies to any CSR/CSC consumer, not just ParaGrapher; libraries that mmap binary CSR could adopt minimal bytes with small decoder changes.
- Because all datasets fit in 2 TB RAM and cache clearing is not reported, part of the measured speedup could reflect page-cache state; a cold-cache rerun would tell how much.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. This paper presents two optimizations to the ParaGrapher graph-loading library. PG-Fuse is a FUSE-based filesystem that reads and caches large (default 32 MiB) blocks from the underlying storage, aiming to improve bandwidth utilization on high-performance parallel filesystems. CompBin is a compact binary CSR representation that stores vertex IDs in the minimum number of bytes (3 bytes for graphs with fewer than 16.7 million vertices, 4 bytes otherwise) and decompresses with a few shift/add operations. The evaluation on 12 real-world and synthetic graphs reports PG-Fuse speedups of 0.9x--7.6x and a CompBin speedup of up to 21.8x compared with ParaGrapher without these optimizations.
Significance. If the results hold, PG-Fuse is a useful, WebGraph-version-agnostic mechanism for improving read granularity on high-bandwidth storage, and CompBin demonstrates a simple, direct-access format that can be much faster than WebGraph decompression for small graphs. The paper is honest about performance losses, the equivalence of CompBin to ordinary binary CSR for most datasets, and the hardware dependence of the reported thresholds. The source code is publicly available, which is a concrete reproducibility strength. The main reservation is that the headline speedup figures rest on an uncontrolled page-cache measurement environment, so the engineering claims are plausible but not yet fully verified.
major comments (3)
- [Section V-A; Figures 2 and 3] The evaluation does not state whether OS page caches were dropped between runs, how many repetitions were performed, or whether the reported times are means, medians, or single runs. All datasets fit in the machine's 2 TB RAM (the largest CompBin file is 506.1 GiB), so reads from the second and later runs can be served from page cache rather than from Lustre. Because PG-Fuse's mechanism is precisely to alter storage-read granularity, the measured 0.9x--7.6x speedup range is directly confounded by the cache state. The authors should add cache-isolated, repeated measurements with variance reported.
- [Section III; Section V-B] The PG-Fuse block size is a free parameter with a 32 MiB default, and Section V-B states that reducing the block size should improve performance for small graphs, attributing the twitter-2010 slowdown to this effect. The evaluation does not say which block size was used for each dataset or provide a sensitivity analysis. Without this information, the reported speedup range is not reproducible and the central PG-Fuse claim is incomplete.
- [Section V-C; abstract] The 21.8x CompBin speedup is supported by exactly one dataset (enwiki-2023, |V| = 6.6M). For the other 11 datasets, CompBin is equivalent to ordinary 4-byte binary CSR, so those measurements compare WebGraph against binary CSR, not against a new CompBin-specific mechanism. The paper does acknowledge this in the text, but the abstract's unqualified 'CompBin achieves up to 21.8 times speedup' overstates the evidence. The authors should either add more small-graph datasets below the 2^24-vertex threshold or explicitly scope the claim in the abstract.
minor comments (5)
- [Section IV, Eq. (1)] The summation notation is garbled in the provided text; it should be rendered as a proper sum over i from 0 to b-1 of neighbors[(offset[v]+n)*b+i] shifted left by 8i.
- [Section IV] The condition 'for 2^24 <= |V| < 2^32, the CompBin representation is equivalent to the binary CSR format' is off by one: for |V| = 2^24 exactly, b = ceil(24/8) = 3 bytes, not 4. The correct condition for equivalence to 4-byte CSR is 2^24 < |V| <= 2^32. This does not affect the reported results because enwiki-2023 has fewer than 2^24 vertices and all other datasets exceed it, but the mathematical statement should be corrected.
- [Section V-B] The phrase 'speedup of 0.9--7.6 times' conflates speedups and slowdowns; values below 1 indicate a slowdown. Consider reporting this as 'relative loading time of 0.9x--7.6x' or explicitly noting that values below 1 are slowdowns.
- [Section V-D, Figure 4] The axes in Figure 4 are not fully explained: the X-axis is described as 'difference in graph sizes' without units, and the annotated point '[0.2, 0.1]' is ambiguous. Please add units and clarify what the point represents.
- [Table I] For enwiki-2023, the CompBin file (0.5 GiB) is larger than the WebGraph file (0.3 GiB), which may initially seem inconsistent with the 'preventing storage usage for unused bytes' description. The paper should state explicitly that the storage-size comparison is in per-neighbor bytes and that the speedup comes from avoiding decompression overhead, not from smaller storage footprint in this case.
Circularity Check
No significant circularity: the reported speedups are empirical measurements against a legitimate baseline, and no derivation reduces to its own inputs.
full rationale
The paper presents two engineering optimizations, PG-Fuse and CompBin, and evaluates them empirically. The central claims are measured loading-time speedups (0.9–7.6x and up to 21.8x) obtained by comparing ParaGrapher with and without the new optimizations. This is a legitimate experimental comparison rather than a derivation. CompBin's decoding formula (Eq. 1) is a definitional description of the byte layout, not a result derived from the claimed speedup; the claim that CompBin is equivalent to binary CSR for |V| in [2^24, 2^32) follows directly from the byte-width formula b = ceil(log2|V|/8) and is not circular. PG-Fuse's 32 MiB block size is a design parameter, not fitted to the measured outcomes, and the paper does not present the speedups as predictions from a fitted model. Self-citations to the prior ParaGrapher paper and MS-BioGraphs provide background and dataset provenance, but no load-bearing argument in this paper depends on an unverified self-citation. The evaluation's page-cache and shared-filesystem confounds are real correctness/measurement risks, but they are not circularity: they concern whether the measurements reflect the intended I/O behavior, not whether the claims reduce to their inputs by construction.
Assumptions & free parameters
free parameters (1)
- PG-Fuse block size =
32 MiB (default)
assumptions (3)
- domain assumption The Java WebGraph implementation exhibits frequent small (128 kB) storage accesses, and this is the primary bottleneck on high-bandwidth storage.
- domain assumption FUSE-based user-space reads of 32 MiB blocks have lower end-to-end overhead than direct 128 kB reads on the evaluated Lustre SSD pool.
- domain assumption Filesystem page cache state does not dominate the measurements; all datasets fit in the machine's 2 TB RAM.
Cite this review
Pith. "Pith review of Accelerating Loading WebGraphs in ParaGrapher." pith.science (2026). https://pith.science/paper/XIX5KLEK
@misc{pith2026250700716,
author = {Pith},
title = {Pith review of: Accelerating Loading WebGraphs in ParaGrapher},
year = {2026},
howpublished = {\url{https://pith.science/paper/XIX5KLEK}},
note = {Machine review of arXiv:2507.00716}
}
read the original abstract
ParaGrapher is a graph loading API and library that enables graph processing frameworks to load large-scale compressed graphs with minimal overhead. This capability accelerates the design and implementation of new high-performance graph algorithms and their evaluation on a wide range of graphs and across different frameworks. However, our previous study identified two major limitations in ParaGrapher: inefficient utilization of high-bandwidth storage and reduced decompression bandwidth due to increased compression ratios. To address these limitations, we present two optimizations for ParaGrapher in this paper. To improve storage utilization, particularly for high-bandwidth storage, we introduce ParaGrapher-FUSE (PG-Fuse) a filesystem based on the FUSE (Filesystem in User Space). PG-Fuse optimizes storage access by increasing the size of requested blocks, reducing the number of calls to the underlying filesystem, and caching the received blocks in memory for future calls. To improve the decompression bandwidth, we introduce CompBin, a compact binary representation of the CSR format. CompBin facilitates direct accesses to neighbors while preventing storage usage for unused bytes. Our evaluation on 12 real-world and synthetic graphs with up to 128 billion edges shows that PG-Fuse and CompBin achieve up to 7.6 and 21.8 times speedup, respectively.
Figures
Reference graph
Works this paper leans on
-
[1]
Selective parallel loading of large-scale compressed graphs with ParaGrapher,
M. Koohi Esfahani, M. D’Antonio, S. I. Tauhidi, T. S. Mai, and H. V andierendonck, “Selective parallel loading of large-scale compressed graphs with ParaGrapher,” CoRR, 2024. [Online]. Available: https://arxiv.org/abs/ 2404.19735
arXiv 2024
-
[2]
PIGO: A parallel graph input/output library,
K. Gabert and U. V . C ¸ ataly¨urek, “PIGO: A parallel graph input/output library,” in 2021 IEEE International Par- allel and Distributed Processing Symposium Workshops (IPDPSW). IEEE, 2021, pp. 276–279
work page 2021
-
[3]
GVEL: Fast graph loading in edgelist and compressed sparse row (csr) formats,
S. Sahu, “GVEL: Fast graph loading in edgelist and compressed sparse row (csr) formats,” 2023
work page 2023
-
[4]
R-mat: A recursive model for graph mining
D. Chakrabarti, Y . Zhan, and C. Faloutsos, “R-mat: A recursive model for graph mining.” in SDM. SIAM, 2004, pp. 442–446
work page 2004
-
[5]
Gtgraph: A synthetic graph generator suite,
D. A. Bader and K. Madduri, “Gtgraph: A synthetic graph generator suite,” Atlanta, GA , vol. 38, 2006
work page 2006
-
[6]
Trilliong: A trillion-scale syn- thetic graph generator using a recursive vector model,
H. Park and M.-S. Kim, “Trilliong: A trillion-scale syn- thetic graph generator using a recursive vector model,” in Proceedings of the 2017 ACM International Conference on Management of Data , ser. SIGMOD ’17. ACM, 2017, p. 913–928
work page 2017
-
[7]
Smooth kronecker: Solving the combing problem in kronecker graphs,
V . Anand, P . Mehrotra, D. Margo, and M. Seltzer, “Smooth kronecker: Solving the combing problem in kronecker graphs,” ser. GRADES-NDA ’20. ACM, 2020
work page 2020
-
[8]
The webgraph framework i: Compression techniques,
P . Boldi and S. Vigna, “The webgraph framework i: Compression techniques,” in Proceedings of the 13th International Conference on World Wide Web, ser. WWW ’04. ACM, 2004, p. 595–602
work page 2004
Show all 40 references
-
[9]
Sparskit: a basic tool kit for sparse matrix computations - version 2,
Y . Saad, “Sparskit: a basic tool kit for sparse matrix computations - version 2,” 1994. [Online]. Available: https://citeseerx.ist.psu.edu/viewdoc/ summary?doi=10.1.1.41.3853
1994
-
[10]
Ubi- crawler: A scalable fully distributed web crawler,
P . Boldi, B. Codenotti, M. Santini, and S. Vigna, “Ubi- crawler: A scalable fully distributed web crawler,” Softw. Pract. Exper ., vol. 34, no. 8, p. 711–726, Jul. 2004
2004
-
[11]
Bubing: Massive crawling for the masses,
P . Boldi, A. Marino, M. Santini, and S. Vigna, “Bubing: Massive crawling for the masses,” ACM Trans. Web , vol. 12, no. 2, Jun. 2018
2018
-
[12]
Ultra- large-scale repository analysis via graph compression,
P . Boldi, A. Pietri, S. Vigna, and S. Zacchiroli, “Ultra- large-scale repository analysis via graph compression,” in SANER. IEEE, 2020, pp. 184–194
2020
-
[13]
The software heritage graph dataset: Public software development un- der one roof,
A. Pietri, D. Spinellis, and S. Zacchiroli, “The software heritage graph dataset: Public software development un- der one roof,” in 2019 IEEE/ACM 16th International Conference on Mining Software Repositories (MSR) , 2019
2019
- [14]
-
[15]
Webgraph: The next generation (is in rust),
T. Fontana, S. Vigna, and S. Zacchiroli, “Webgraph: The next generation (is in rust),” in ACM Web Conference 2024, 2024
2024
-
[16]
Clique partitions, graph com- pression and speeding-up algorithms,
T. Feder and R. Motwani, “Clique partitions, graph com- pression and speeding-up algorithms,” in Proceedings of the twenty-third annual ACM symposium on Theory of computing, 1991, pp. 123–133
1991
-
[17]
Graphzip: a clique-based sparse graph compression method,
R. A. Rossi and R. Zhou, “Graphzip: a clique-based sparse graph compression method,” Journal of Big Data , vol. 5, no. 1, p. 10, 2018
2018
-
[18]
Qclique: Optimizing performance and acuracy in maximum weighted clique,
Q. Abbas, M. Koohi Esfahani, I. Overton, and H. V andierendonck, “Qclique: Optimizing performance and acuracy in maximum weighted clique,” in Euro-Par 2024
2024
-
[19]
Graph summarization with bounded error,
S. Navlakha, R. Rastogi, and N. Shrivastava, “Graph summarization with bounded error,” in Proceedings of the 2008 ACM SIGMOD international conference on Management of data , 2008, pp. 419–432
2008
-
[20]
Graph compression,
F. Zhou, “Graph compression,” Department of Computer Science and Helsinki Institute for Information Technol- ogy HIIT , pp. 1–12, 2015
2015
-
[21]
Compressgraph: Ef fi cient parallel graph analytics with rule-based compression,
Z. Chen, F. Zhang, J. Guan, J. Zhai, X. Shen, H. Zhang, W. Shu, and X. Du, “Compressgraph: Ef fi cient parallel graph analytics with rule-based compression,” Proceed- ings of the ACM on Management of Data , vol. 1, no. 1, pp. 1–31, 2023
2023
-
[22]
Estimating and sampling graphs with multidimensional random walks,
B. Ribeiro and D. Towsley, “Estimating and sampling graphs with multidimensional random walks,” in Pro- ceedings of the 10th ACM SIGCOMM conference on Internet measurement , 2010, pp. 390–403
2010
-
[23]
Query preserving graph compression,
W. Fan, J. Li, X. Wang, and Y . Wu, “Query preserving graph compression,” in Proceedings of the 2012 ACM SIGMOD international conference on management of data, 2012, pp. 157–168
2012
-
[24]
Compressing network graphs,
A. C. Gilbert and K. Levchenko, “Compressing network graphs,” in Proceedings of the LinkKDD workshop at the 10th ACM Conference on KDD , vol. 124, 2004
2004
-
[25]
A survey on methods and systems for graph compression,
S. Maneth and F. Peternek, “A survey on methods and systems for graph compression,” arXiv preprint arXiv:1504.00616, 2015
2015 arXiv
-
[26]
Summa- rizing semantic graphs: a survey,
ˇS. ˇCebiri´c, F. Goasdou ´e, H. Kondylakis, D. Kotzinos, I. Manolescu, G. Troullinou, and M. Zneika, “Summa- rizing semantic graphs: a survey,” The VLDB journal , vol. 28, pp. 295–327, 2019
2019
-
[27]
Summarizing static and dynamic big graphs,
A. Khan, S. S. Bhowmick, and F. Bonchi, “Summarizing static and dynamic big graphs,” 2017
2017
-
[28]
Endgraph: An ef fi cient distributed graph preprocessing system,
T. Liu and D. Li, “Endgraph: An ef fi cient distributed graph preprocessing system,” in 2022 IEEE 42nd Inter- national Conference on Distributed Computing Systems (ICDCS), 2022, pp. 111–121
2022
-
[29]
Grapu: Ac- celerate streaming graph analysis through preprocessing buffered updates,
F. Sheng, Q. Cao, H. Cai, J. Y ao, and C. Xie, “Grapu: Ac- celerate streaming graph analysis through preprocessing buffered updates,” in Proceedings of the ACM Symposium on Cloud Computing . ACM, 2018
2018
-
[30]
Evaluation of parallel graph loading techniques,
M. Then, M. Kaufmann, A. Kemper, and T. Neumann, “Evaluation of parallel graph loading techniques,” ser. GRADES ’16. ACM, 2016
2016
-
[31]
A survey of graph pre-processing methods: From algorithmic to hardware perspectives,
Z. Lv, M. Y an, X. Liu, M. Dong, X. Y e, D. Fan, and N. Sun, “A survey of graph pre-processing methods: From algorithmic to hardware perspectives,” 2023
2023
-
[32]
The lustre storage architecture,
P . Braam, “The lustre storage architecture,” 2019. [Online]. Available: https://arxiv.org/abs/1903.01955
2019 arXiv
-
[33]
Lustre unveiled: Evolution, design, advancements, and current trends,
A. George, A. Dilger, M. J. Brim, R. Mohr, A. She- hata, J. Y . Choi, A. M. Karimi, J. Hanley, J. Simmons, D. Manno, V . M. V ergara, S. Oral, and C. Zimmer, “Lustre unveiled: Evolution, design, advancements, and current trends,” ACM Trans. Storage , 2025
2025
-
[34]
Toward greener matrix operations by lossless compressed formats,
F. Tosoni, P . Bille, V . Brunacci, A. d. Angelis, P . Ferrag- ina, and G. Manzini, “Toward greener matrix operations by lossless compressed formats,” IEEE Access , vol. 13, pp. 56 756–56 779, 2025
2025
-
[35]
The graph structure in the web – analyzed on different aggregation levels,
R. Meusel, S. Vigna, O. Lehmberg, and C. Bizer, “The graph structure in the web – analyzed on different aggregation levels,” The Journal of Web Science , vol. 1, no. 1, pp. 33–47, 2015
2015
-
[36]
Graph structure in the web: Aggregated by pay-level domain,
O. Lehmberg, R. Meusel, and C. Bizer, “Graph structure in the web: Aggregated by pay-level domain,” in Pro- ceedings of the 2014 ACM Conference on Web Science , ser. WebSci ’14. ACM, 2014, p. 119–128
2014
-
[37]
Graph structure in the web — revisited: A trick of the heavy tail,
R. Meusel, S. Vigna, O. Lehmberg, and C. Bizer, “Graph structure in the web — revisited: A trick of the heavy tail,” ser. WWW ’14 Companion. ACM, 2014
2014
-
[38]
A large time-aware graph,
P . Boldi, M. Santini, and S. Vigna, “A large time-aware graph,” SIGIR F orum, vol. 42, no. 2, pp. 33–38, 2008
2008
-
[39]
What is twitter, a social network or a news media?
H. Kwak, C. Lee, H. Park, and S. Moon, “What is twitter, a social network or a news media?” in Proceedings of the 19th International Conference on World Wide Web , ser. WWW ’10. ACM, 2010, p. 591–600
2010
-
[40]
Introducing the graph 500,
R. C. Murphy, K. B. Wheeler, B. W. Barrett, and J. A. Ang, “Introducing the graph 500,” Cray Users Group , 2010
2010
Reviewed August 6, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.