{"id":"13a9573b-b406-4bc0-89b2-82f0519679ac","arxiv_id":"2506.19175","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":0,"one_line_summary":"Binsparse defines a portable binary format for sparse matrices and tensors that is smaller and faster than ASCII formats, with reference parsers in five languages.","lead":"Binsparse is a new open specification that stores sparse matrices and tensors as a small JSON header plus binary arrays inside standard containers such as HDF5, Zarr, and NPZ, replacing text-based formats. It reports 1.7 to 7.5x smaller files and up to 26.5x faster reads than Matrix Market, targeting the I/O bottleneck that often dominates sparse computation.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Compressed-format benchmarks compare against uncompressed .mtx, not the common .mtx.gz alternative; against gzipped text the headline 7.5x size and 2.6x/1.4x speedups likely shrink materially.","rationale":"The reader's weakest assumption correctly identifies the missing gzipped-text baseline, and I find this to be the single most load-bearing issue. The paper's headline numbers for compressed storage (7.5x size reduction, 2.6x read, 1.4x write) all derive from a comparison against raw .mtx files, even though gzipped Matrix Market is a routine real-world alternative and the paper never measures it. The concern is concrete and falsifiable: gzip compresses ASCII text well, so the compressed-size advantage will narrow substantially, and the compressed read/write speedups—already modest—may disappear or become regressions when the baseline itself requires decompression plus ASCII parsing. This does not invalidate the specification: the format is sensible, implementations are provided, and the uncompressed CSR benefits (2.4x size, 26.5x warm-cache read) remain significant. But the abstract's 'large practical wins' claim is materially weaker for the compressed mode that the paper emphasizes. The conditional verdict already allows for this kind of qualification; my read does not move it to accept or reject, so I recommend UNCHANGED with the condition that the authors re-benchmark against .mtx.gz and revise the compressed claims accordingly.","tokens_in":18275,"tokens_out":6109,"duration_ms":60629,"concrete_test":"Re-run the Section 5.1/5.4/5.6 experiments with gzipped Matrix Market as the compressed-text baseline: for each SuiteSparse matrix with >1M entries, create file.mtx.gz using gzip -1 (matching HDF5's fastest level), and measure (a) file size ratio .csr.bsp.gz vs .mtx.gz; (b) single-threaded warm-cache read time for .mtx.gz via a gzip-aware path (e.g., fast_matrix_market with a gzip stream, or scipy.io.mmread on the .gz) vs the Binsparse compressed read; (c) flushed write time to .mtx.gz vs .csr.bsp.gz. If median size reduction drops below ~2x and read speedup below ~1.2x, the abstract's compressed claims require revision.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The most load-bearing concern is the choice of baseline for the compressed-format claims. Section 5.1/Table 3 compute file size reductions for .coo.bsp.gz and .csr.bsp.gz against raw .mtx files, and the abstract's 7.5x size reduction and 2.6x/1.4x compressed read/write speedups inherit that baseline. But Matrix Market text is highly compressible: SuiteSparse matrices are commonly distributed as .mtx.gz, so a practitioner's real compressed alternative is gzipped text. ASCII digits and whitespace compress well, while HDF5-gzipped binary floats contain high-entropy mantissas, so the size advantage of compressed Binsparse over .mtx.gz will be far smaller than 7.5x (likely ~2x or less based on typical gzip ratios of 3-5x on text). Similarly, reading a .mtx.gz file requires decompression plus text parsing, so the warm-cache read speedup of 2.6x for compressed Binsparse may shrink toward 1x or reverse; write speedups are already marginal at 1.4x and could become slowdowns. The uncompressed CSR numbers (2.4x, 26.5x read) are against raw text and remain valid, so the specification's core value is not destroyed, but the abstract's compressed-storage headline benefits are overstated. Because the paper's central claim is that Binsparse offers 'large practical wins,' this baseline omission is load-bearing for the size/performance contributions.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper introduces Binsparse, a specification for storing sparse matrices and tensors as a JSON descriptor plus binary arrays, embeddable in containers such as HDF5, Zarr, and NPZ, and supporting native in-memory formats including COO, CSR, CSC, DCSR, and DCSC, as well as custom hierarchical tensor formats. The authors report reference implementations across several languages and frameworks, convert the SuiteSparse Matrix Collection and a subset of FROSTT tensors to Binsparse, and benchmark file sizes and read/write times against fast_matrix_market and splatt. The headline results are large average file size reductions (e.g., 2.4x for uncompressed CSR, 7.5x for compressed CSR) and warm-cache read speedups (e.g., 26.5x for uncompressed CSR), with smaller speedups for compressed formats.","tokens_in":18556,"tokens_out":4707,"duration_ms":48919,"significance":"If the compressed-format claims are placed on a proper baseline, this paper makes a valuable contribution: it provides a concrete, embeddable, cross-platform binary specification for sparse data, with multiple reference implementations and a large-scale conversion and validation effort. The uncompressed-format benefits (2.4x size reduction, 26.5x read speedup, 31x write speedup for CSR) appear robust and are not undermined by the baseline concern raised below. The release of parsers in multiple languages and containers, plus bitwise-identical validation on the SuiteSparse collection, are notable strengths. The main reservation is that the compressed-format comparisons use an uncompressed text baseline, which overstates the practical advantage of compressed Binsparse for users whose current alternative is gzipped Matrix Market.","major_comments":[{"comment":"The compressed-size claims are computed against uncompressed .mtx files rather than gzipped Matrix Market (.mtx.gz). Table 3 reports average reductions of 7.2x and 7.5x for .coo.bsp.gz and .csr.bsp.gz over raw .mtx, and the abstract inherits the 7.5x number. Since ASCII text compresses well and .mtx.gz is the common compressed-text alternative for SuiteSparse data, the true size advantage of compressed Binsparse over compressed text is likely much smaller. The same baseline issue affects the compressed read/write speedups in Tables 5 and 8 (2.6x/1.4x average). The authors should either add .mtx.gz file-size and read/write measurements as a baseline or explicitly scope all compressed-format claims to comparisons against uncompressed text.","section":"Section 5.1 / Table 3 / Abstract"},{"comment":"The abstract says the format is evaluated 'on every matrix in the SuiteSparse Matrix Collection' and then reports average reductions of 2.4x and 7.5x, but those averages (Table 3) are taken only over matrices with at least one million entries, as stated in the table caption and Section 5.1. A reader cannot tell from the abstract that the headline numbers exclude the majority of the collection. Please state the subset explicitly in the abstract and conclusions, or report full-collection statistics separately.","section":"Abstract / Section 5.1"},{"comment":"The contribution bullet claims 'parallel reads on average about 2x faster than sequential reads,' but the body reports roughly 1.4x improvement for uncompressed cold reads, no improvement for warm uncompressed reads, and about 2.4x for compressed COO. This is inconsistent unless 'average' is defined more precisely. Please either reconcile the claim with the per-format numbers or remove the aggregate statement.","section":"Section 5.5"}],"minor_comments":[{"comment":"The abstract claims 4 binary containers, but the text lists HDF5, Zarr, and NPZ as the container libraries and describes in-memory interchange separately. Please clarify whether 'In-Memory' is counted as a container in Table 1, and align the abstract, contributions, and table.","section":"Section 4 / Table 1 / Abstract"},{"comment":"The y-axis label and the title both read 'File Size (Bytes)'; consider removing the duplicate label or making one of them the axis unit only.","section":"Figure 1"},{"comment":"The sentence 'the average speedups are 1.4x and 1.4x' should identify the format as .csr.bsp.gz explicitly, since multiple compressed formats appear in Table 8.","section":"Section 5.6"},{"comment":"A brief statement in the related work section that gzip-compressed Matrix Market is a common distribution format, and roughly what compression ratios it achieves on SuiteSparse matrices, would help readers contextualize the compressed-format comparisons.","section":"Section 2"}],"recommendation":"major_revision","confidential_remarks":"The compressed-baseline issue is the main load-bearing weakness, but it is fixable within the manuscript's scope by adding .mtx.gz comparisons or by explicitly re-scoping the compressed claims. The specification itself and the uncompressed-format results are strong; this is a revise, not a reject."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Binsparse is a genuine artifact: a cross-platform binary format for sparse matrices and tensors, defined by a JSON header plus arrays inside HDF5/Zarr/NPZ, with implementations in several languages and a broad benchmark over SuiteSparse and FROSTT. I think it deserves a serious referee. The spec is well thought out—using existing containers for portability, preserving native in-memory layouts, and supporting custom fibertree formats is the right design.\n\nThe uncompressed results are strong and credible. CSR via HDF5 gives a 2.4x average size reduction and a 26.5x warm-cache read speedup over fast_matrix_market because you skip ASCII parsing entirely. The validation step, showing bitwise-identical in-memory representations after conversion, is the kind of evidence that makes me trust the pipeline.\n\nThe soft spot is the compressed-format comparison. The 7.5x size reduction and the 2.6x/1.4x compressed speedups are computed against raw .mtx files, not the .mtx.gz files that most practitioners actually use. Matrix Market text compresses very well, so the size advantage over gzipped text will be much smaller, and the speed numbers are simply unknown until measured. This is a load-bearing omission for the compressed-storage claims, not for the format itself. The paper should add .mtx.gz as a baseline or clearly limit the claims to uncompressed text.\n\nThere are also smaller issues. The abstract says “every matrix in the SuiteSparse Matrix Collection” but the benchmark uses the subset with more than one million entries; the write speedup is reported unflushed (31x) when the flushed number is 8.5x; and there is no link to benchmark artifacts or the converter code. None of these threaten the central idea.\n\nTake the headline performance numbers with a grain of salt, but take the format seriously. It fills a real gap for sparse library maintainers and practitioners who move data between frameworks. If I were an editor, I would send it to review and ask for the .mtx.gz comparison before acceptance.","headline":"Binsparse is a real and useful specification with strong uncompressed results, but the compressed-storage headline numbers are measured against raw .mtx rather than the .mtx.gz baseline practitioners actually use.","tokens_in":19106,"tokens_out":3718,"would_cite":true,"duration_ms":37185,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Binsparse proposes the first cross-platform binary standard for sparse matrices and tensors, with large measured size and read-time gains over ASCII formats.","keywords":["Binsparse","sparse matrix storage","sparse tensor storage","binary file format","HDF5","Matrix Market","FROSTT","CSR"],"falsifier":"Take any matrix in the >1M-entry SuiteSparse set, store it as gzipped Matrix Market (.mtx.gz) and as Binsparse HDF5 CSR with gzip, and time full reads including decompression on the same hardware; if .mtx.gz reads as fast as or faster than .csr.bsp.gz while staying comparable in size, the paper's compressed-format claims fail.","tokens_in":18081,"feed_emoji":"💾","tokens_out":4566,"duration_ms":45967,"temperature":0.7,"pith_summary":"The paper aims to establish Binsparse as the first cross-platform standard for storing sparse matrices and tensors directly in binary form. Its central claim is that a small JSON descriptor plus standard binary arrays, embedded in existing containers such as HDF5, can replace ASCII-based Matrix Market and FROSTT files with roughly 2.4x smaller files for CSR (7.5x with compression) and read times about 26.5x faster in warm-cache tests. If this holds, the standard would give ordinary users the speed of hand-rolled binary formats without the portability cost, and would let libraries read and write their native in-memory formats to disk without format conversion.","feed_headline":"Binary standard shrinks sparse matrices 7.5x, reads 26x faster","feed_subtitle":"Binsparse stores matrices as a JSON header plus binary arrays, beating ASCII Matrix Market on size and parse time.","key_machinery":"The central object is the JSON descriptor plus a set of named binary arrays: indices_0, indices_1, pointers_to_1, and values, whose meanings are fixed per format. The descriptor alone determines which arrays exist and what they mean, so the same binary arrays work in HDF5, Zarr, NPZ, or DLPack without changing the data. For tensors, a hierarchical level description based on the fibertree abstraction lets any array-of-arrays layout be described. This machinery is what makes native binary storage possible without inventing a new container format.","core_discovery":"Binsparse proposes that any common sparse matrix or tensor format—COO, CSR, CSC, doubly compressed variants, and hierarchical fiber formats—can be described by a short JSON header that names the format, shape, number of stored values, structure, and data types, with the actual data stored as plain binary arrays inside a general-purpose binary container. By separating metadata from arrays and letting the container handle endianness, compression, and chunking, the format avoids inventing a new container format. Measured on every SuiteSparse Matrix Collection matrix with at least a million entries, the HDF5 CSR form averages 2.4x smaller than Matrix Market text without compression and 7.5x smaller with gzip; the reference parser averages 26.5x faster warm-cache reads and 31x faster unflushed writes than fast_matrix_market for uncompressed CSR, with similar advantages for tensors against FROSTT parsers. The specification also defines a level-based custom tensor language so formats native to tensor compilers can be stored directly.","pith_inferences":["If gzipped Matrix Market text, rather than plain text, is the realistic alternative for practitioners, the compressed-file size and speed advantages likely shrink; a direct .mtx.gz versus .bsp.gz comparison would settle how large the real-world win is.","If Binsparse becomes a shared interchange contract, it could also serve as the in-memory exchange layer between sparse libraries, making custom shim code unnecessary.","Since the JSON header is human-readable and external metadata can be attached, the format could double as a self-describing archival standard, allowing large matrix collections to be distributed as a single HDF5 file per matrix set."],"forward_implications":["Reading a sparse matrix becomes a near memcpy-style load into the CSR or COO structure instead of token-by-token text parsing, so bandwidth-bound routines spend less time in file I/O.","Because the same named arrays slot into multiple established containers, files written once can be read by C, C++, Python, Julia, and other frameworks through existing container libraries.","The level-based tensor language means in-memory formats used by tensor compilers can be serialized directly, eliminating conversion to coordinate form before saving.","Parallel reading that splits each dataset across processes gives large compressed files roughly 2x additional speedup over sequential reads in the paper's measurements."],"supporting_citations":[{"why":"Defines the Matrix Market ASCII format that Binsparse aims to replace as the dominant sparse matrix storage standard.","marker":"[12]"},{"why":"Provides the state-of-the-art Matrix Market parser used as the speed baseline in all read and write comparisons.","marker":"[27]"},{"why":"Supplies the SuiteSparse Matrix Collection, the corpus for all matrix file-size and parsing experiments.","marker":"[17]"},{"why":"Defines the FROSTT tensor format and repository, the baseline and corpus for tensor experiments.","marker":"[33]"},{"why":"Provides the SPLATT parser used as the baseline for FROSTT tensor read comparisons.","marker":"[34]"},{"why":"Describes HDF5, the binary container that gives the reference implementation portability and built-in gzip compression.","marker":"[26]"},{"why":"Introduces the fibertree abstraction that underlies Binsparse's level-based custom sparse tensor formats.","marker":"[35]"},{"why":"Defines DLPack, the in-memory zero-copy interchange format used for Binsparse-based exchange between Python sparse libraries.","marker":"[5]"}],"fun_headline_variants":["Binsparse: binary sparse storage, 7.5x smaller, 26x faster","Cross-platform binary format for sparse matrices shrinks 7.5x","New spec: binary sparse tensors, 7.5x smaller, 26x faster reads","Binsparse spec cuts sparse matrix size by 7.5x, reads 26x faster","Binary standard for sparse data: 2.4x smaller, 26x faster reads"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The evaluation assumes the relevant alternative is uncompressed Matrix Market text; if users would actually store gzipped Matrix Market files, the reported size and read/write speed advantages shrink because ASCII text compresses well.","fun_headline_variants_meta":{"raw":{"variants":["Binsparse: binary sparse storage, 7.5x smaller, 26x faster","Cross-platform binary format for sparse matrices shrinks 7.5x","New spec: binary sparse tensors, 7.5x smaller, 26x faster reads","Binsparse spec cuts sparse matrix size by 7.5x, reads 26x faster","Binary standard for sparse data: 2.4x smaller, 26x faster reads"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.00044,"raw_usage":{"total_tokens":2326,"prompt_tokens":1135,"completion_tokens":1191,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":751,"completion_tokens_details":{"reasoning_tokens":1074}},"tokens_in":751,"tokens_out":1191,"duration_ms":10202,"temperature":1.0,"reasoning_tokens":1074,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-06T23:07:18.549409+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Take any matrix in the >1M-entry SuiteSparse set, store it as gzipped Matrix Market (.mtx.gz) and as Binsparse HDF5 CSR with gzip, and time full reads including decompression on the same hardware; if .mtx.gz reads as fast as or faster than .csr.bsp.gz while staying comparable in size, the paper's compressed-format claims fail.","supporting_citations":[{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Defines the Matrix Market ASCII format that Binsparse aims to replace as the dominant sparse matrix storage standard."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Provides the state-of-the-art Matrix Market parser used as the speed baseline in all read and write comparisons."},{"cited_title":"Choi, Jiajia Li, Richard Vuduc, Jongsoo Park, Xing Liu, and George Karypis","cited_arxiv_id":null,"evidence_quote":"Defines the FROSTT tensor format and repository, the baseline and corpus for tensor experiments."},{"cited_title":"Sidiropoulos, and George Karypis","cited_arxiv_id":null,"evidence_quote":"Provides the SPLATT parser used as the baseline for FROSTT tensor read comparisons."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Describes HDF5, the binary container that gives the reference implementation portability and built-in gzip compression."},{"cited_title":"DLPack: Open In Memory Tensor Structure","cited_arxiv_id":null,"evidence_quote":"Defines DLPack, the in-memory zero-copy interchange format used for Binsparse-based exchange between Python sparse libraries."}],"review_version":1}