REVIEW 2 major objections 5 minor 28 references
Specification for the Siril HEALpixel Catalog Format
T0 review · 2 major / 5 minor · reviewed 2026-08-10 · deepseek-v4-flash
Pith's one-line read This paper specifies a binary format that turns sky-region queries into a few file seeks.
desk verdict A clear, implementable binary catalog spec for Siril with real data products behind it; the performance claims are unmeasured and there are a few numerical typos, but it deserves a serious referee if the venue publishes specifications. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The load-bearing object is the nested HEALpixel numbering combined with a cumulative-count index. HEALpix divides the sky into equal-area pixels whose numbering is hierarchical, so pixels near each other on the sky tend to have close numbers; the format indexes a catalog at a fixed level N and stores, for each pixel in numerical order, a uint32_t cumulative count of sources up to and including that pixel. Because records are fixed-size and sorted by pixel, the difference between two cumulative counts gives the number of records in the pixel range, and the byte offset of the first record is the header size plus index size plus the count of preceding sources times the record size. This turns a geometric query into a small set of contiguous reads. A second ingredient is the fixed-length record with scaled coordinates: RA and Dec share the scale factor $360/(2^{31}-1)$, giving a constant angular resolution around 0.60 mas, and the astrometric record deliberately omits the Gaia source_id to keep records at 16 bytes.
What would settle it
Generate a set of random cone searches at the catalog's indexing level, compute the touched HEALpixels, and merge adjacent numbers into ranges: if a typical one-degree cone at level 8 yields hundreds of disjoint ranges instead of a handful, the claimed seek-based retrieval loses its advantage. A second check is to implement the format and time the same queries against a SQLite table of the same records; the claim would be weakened if the seek-based reader were not faster at modest catalog sizes.
Extended reading notes
Core claim
The central claim is that a catalog built from this specification can combine storage economy with fast spatial lookup. The file is a fixed 128-byte header followed by an index array and then data records; because every source record has a fixed size and records are grouped by HEALpixel in numerical pixel order, the cumulative count in each index cell pins down the exact byte offset of every pixel's records. A cone search therefore reduces to computing which level-N HEALpixels touch the region, merging adjacent pixel numbers into ranges, and reading the corresponding record blocks directly into memory. The paper argues this is sufficient for Gaia DR3-scale data, gives two packed record formats, and describes chunking for large or remote catalogs.
Load-bearing premise
The format's speed advantage rests on the unmeasured assumption that the HEALpixels touched by a typical sky query collapse into just a few consecutive ranges, so that a handful of seeks and block reads beats scanning the file or using a database index.
Editorial extensions
If this is right
- A cone search at the catalog's indexing level needs only the header, the index, and the record blocks for the merged pixel ranges; no per-source scan of unrelated sky regions is required.
- The index at level 8 is a fixed $786,432 \times 4 = 3,145,728$ bytes, a small constant overhead that can be read once and reused for many queries.
- Chunked catalogs can be queried online with HTTP range requests, and a sparse local file can cache previously fetched chunks for later offline use.
- Additional catalog types can be added without breaking existing files by assigning new values in the catalog-type enum and defining fixed-size record layouts externally.
- Omitting the 8-byte Gaia source_id from the astrometric record keeps each source at 16 bytes, roughly halving the per-source footprint of the coordinate-plus-motion core.
Reading between the lines
- The paper leaves the efficiency claim unbenchmarked; a direct test would be to count, for random cone radii at level 8, how many disjoint HEALpixel ranges a search actually touches, since the seek-based strategy wins only when that number stays small.
- Because the ordering within each pixel is declared irrelevant, a producer can append records per pixel in any batch order and then sort groups numerically, an implementation freedom the spec permits but does not spell out.
- The same cumulative-count scheme should transfer to any hierarchical, locality-preserving spatial index, not only HEALPix; other pixelizations with the same nesting property could reuse the file layout almost unchanged.
- The 127-sources-per-pixel cap described for the initial extracts means the published Siril catalogs are bounded subsets of Gaia DR3; holding the full data release would require finer chunking or a larger index/record budget.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. This manuscript specifies the Siril Catalog Format, a binary file format for storing and querying nested HEALpixel-based astronomical catalogs such as Gaia DR3. The specification defines a 128-byte header, a cumulative-count index over HEALpixels, fixed-size data records for astrometric and spectrophotometric extracts, chunking conventions, and recommended search strategies for monolithic, chunked, and online/hybrid access. The document is self-contained and implementation-oriented, with examples, naming conventions, licensing, and links to a reference implementation. The central design claim is that the format provides efficient storage and fast spatial querying via seek-based retrieval over contiguous HEALpixel ranges, but this claim is not supported by measurements or formal bounds. The manuscript contains a few numerical typos in the index-size expression and the example index values, which are local but relevant to a document whose purpose is precision.
Significance. If the efficiency claim is borne out, this format could be a useful building block for astronomy software such as Siril, enabling local and hybrid online/local access to Gaia catalogs with lower overhead than raw CSV or FITS-based extracts. The specification is concrete and implementable: it gives exact byte layouts, endianness, scaling factors, and a reference implementation link. It is also falsifiable in the sense that the search strategy in Section 7 yields measurable performance. However, the paper is a format proposal rather than a validated system; the headline efficiency advantage is unexplored, and the presence of typos in core arithmetic and example values weakens the document's authority as a normative specification. The contribution is moderate in novelty but potentially useful for interoperability in the open-source astronomy toolchain.
major comments (2)
- [Section 7 (Search Strategy)] The central efficiency claim is not demonstrated. The recommended strategy assumes without measurement or worst-case analysis that (a) a typical conesearch reduces to a small number of contiguous HEALpixel ranges and (b) seek-and-read per range is faster than alternatives such as scanning or database queries. The number of ranges can be large for arbitrary query shapes or high indexing levels, and the specification does not bound it. No benchmark, complexity estimate, or comparison to KStars or other binary formats is provided. The sentence 'Efficient search strategies make use of the property of the nested HEALpixel scheme' is presented as fact, but it is a testable assumption. The manuscript should either add measured performance on representative queries or explicitly state that efficiency is a design goal pending validation, rather than a demonstrated property of the specification.
- [Sections 3 and 6 (Index size and example index)] Two concrete numerical errors undermine the precision of the specification. In Section 3, the index size example states '28 × 28 × 12 = 786,432'; however, 28 × 28 × 12 = 9,408, not 786,432. The correct expression is 2^8 × 2^8 × 12 (or 256 × 256 × 12) for level 8. In Section 6, the cumulative index for first five pixels with source counts 10, 20, 0, 15, and 30 is listed as [0x0A, 0x1E, 0x1E, 0x33, 0x51], but the correct cumulative values are 10, 30, 30, 45, 75, i.e., [0x0A, 0x1E, 0x1E, 0x2D, 0x4B]. These are local typos, but in a normative specification document, arithmetic errors in the central index definition can lead to implementation mistakes and should be corrected before publication.
minor comments (5)
- [Section 6 (Example Astrometric Extract Data Record)] The example G_mean_mag value of -20, with a scaling factor of 10^3, represents -0.02 mag, which is unphysical for Gaia G-band magnitudes; the intended value is likely 20000 (i.e., 20.000 mag) or the scaling description should include an offset. Similarly, Dec_scaled = -987654321 corresponds to roughly -165.5 degrees, which is outside the valid Declination range of [-90, 90]; the example would be more useful if it used physically realizable coordinates.
- [Section 3 (Header layout)] The example header states '77 bytes of zero padding', but this space actually contains the chunked-catalog fields (chunked flag, chunk level, chunk HEALpixel, first and last HEALpixel numbers) plus the 63 reserved bytes. Labeling these fields explicitly in the example would prevent implementers from misreading the padding as purely reserved.
- [Section 4 (Supported Data Types)] The index uses uint32_t, which limits a monolithic catalog to 2^32 - 1 total records; the specification does not state what should happen if a catalog exceeds this limit, for example at higher HEALpixel levels or with denser source extraction. A one-line error-handling note would be helpful.
- [Section 11 (Implementation Guidance)] The reference implementation URL is truncated as 'https://gitlab.com/free-astro/siril/-/blob'; the full path (including branch and file name) should be provided so readers can locate the code.
- [Section 4 (Encoding Details)] The specification uses float16 for the spectrophotometric flux values but does not state the rounding mode or the handling of NaN/Infinity in half-precision conversion; citing the IEEE 754 section for these details would remove ambiguity.
Circularity Check
No circularity: the paper is a self-contained binary format specification with no fitted parameters, predictions, or derivation chain that reduces to its inputs.
full rationale
This document is a format specification, not a derivation. It defines a fixed 128-byte header, a cumulative-count HEALpix index, fixed-size data records, and search strategies; none of these elements is derived from a quantity that the paper itself claims to predict. The stated efficiency goals are supported by design choices (fixed-size records, cumulative counts enabling seek-based reads, HEALpix nested ordering) but are presented as design rationale rather than as results fitted to or validated by the same document. Self-citations appear only as provenance for the Siril software, the published data extracts, and an offline database; they are not invoked to justify a load-bearing mathematical claim or to forbid alternative formats. The only potentially unverified element is the implicit performance assumption of the search strategy in Section 7, but an unbenchmarked assumption is a correctness or evidence gap, not circularity. Therefore the appropriate finding is no significant circularity, score 0.
Assumptions & free parameters
assumptions (2)
- domain assumption HEALPix nested numbering yields numerically close pixel numbers for similar sky regions.
- domain assumption float16 representation of xp_sampled flux with an int8 shared exponent preserves sufficient precision for Siril's photometric calibration.
Cite this review
Pith. "Pith review of Specification for the Siril HEALpixel Catalog Format." pith.science (2026). https://pith.science/paper/AMA34HHG
@misc{pith2026250112466,
author = {Pith},
title = {Pith review of: Specification for the Siril HEALpixel Catalog Format},
year = {2026},
howpublished = {\url{https://pith.science/paper/AMA34HHG}},
note = {Machine review of arXiv:2501.12466}
}
read the original abstract
This document specifies the structure of the Siril Catalog Format, designed to support efficient storage and querying of nested HEALpixel based astronomical catalogs such as Gaia DR3. The format includes a fixed length header, an index for rapid look-up, and structured data records optimized for space and speed. The document also outlines recommended search strategies for utilizing the format effectively.
Reference graph
Works this paper leans on
-
[1]
Introduction Acknowledgment This work has made use of data from the European Space Agency ( ESA) mission Gaia 1, processed by the Gaia Data Processing and Analysis Consortium 2 (DPAC). Funding for the DPAC has been provided by national institutions, in particular the insti tutions participating in the Gaia Multilateral Agreement [8, 9]. Purpose This docum...
-
[2]
• Structured for compatibility with Gaia data releases
Data Format Overview Design Principles • Efficient for storage and search. • Structured for compatibility with Gaia data releases. • Extensible to allow future updates without breaking compa tibility. • Supports customizing catalog content to match the use case . For example, an annota- tions catalog may opt for encyclopaedic coverage down to a sp ecified l...
-
[3]
Siril Gaia DR3 astrometric extract
File Structure HEADER The header is a fixed 128 bytes as follows: • 48 bytes: Catalog title (e.g., "Siril Gaia DR3 astrometric extract" ). • 1 byte: Gaia version designator as an integer enum with values defin ed as follows: – DR1 = 0 – DR2 = 1 – eDR3 = 2 – DR3 = 3 – DR4 = 4 – DR5 = 5 • 1 uint8_t HEALpixel indexing level, denoted as N. For Gaia catalogs, th...
-
[4]
• int16_t dRA : Proper motion in Right Ascension (mas/yr), rounded to the n earest inte- ger
is applied. • int16_t dRA : Proper motion in Right Ascension (mas/yr), rounded to the n earest inte- ger. • int16_t dDec : Proper motion in Declination (mas/yr), rounded to the near est integer. • uint16_t Teff : Effective temperature in Kelvin (or 0 if not available). • int16_t G_mean_mag : Scaled mean magnitude in the Gaia G band. A scaling factor of 1 ...
-
[5]
• int16_t dRA : Proper motion in Right Ascension (mas/yr), rounded to the n earest inte- ger
is applied. • int16_t dRA : Proper motion in Right Ascension (mas/yr), rounded to the n earest inte- ger. • int16_t dDec : Proper motion in Declination (mas/yr), rounded to the near est integer. • int16_t G_mean_mag : Scaled mean magnitude in the Gaia G band. A scaling factor of 1 × 103 is applied. • int8_t fexpo : Exponent for use with the following flux ...
-
[6]
• uint8_t: Unsigned 8-bit integer
Data Types and Encoding Supported Data Types • int8_t: Signed 8-bit integer. • uint8_t: Unsigned 8-bit integer. • int16_t: Signed 16-bit integer. • uint16_t: Unsigned 16-bit integer. • int32_t: Signed 32-bit integer. • uint32_t: Unsigned 32-bit integer. • int64_t: Signed 64-bit integer. • uint64_t: Unsigned 64-bit integer. • float16: Half precision 16-bit...
-
[7]
• Gaia Data Release: Enum specifying the Gaia data release
Metadata Specifications The following metadata fields are mandatory in the header (se e Section 3): • Catalog Title: Human-readable identifier, ASCII encoded. • Gaia Data Release: Enum specifying the Gaia data release. • Healpix Indexing Level (N): Integer defining the spatial re solution. • Catalog Type: Indicator of astrometric or photometric dat a. • Reser...
-
[8]
Siril Gaia DR3 astrometric extract
Examples Example Header "Siril Gaia DR3 astrometric extract" (48 bytes) 0x03 (Gaia DR3) 0x08 (Healpix Level 8) 0x01 (Astrometric catalogue) 77 bytes of zero padding Example Index For Healpix Level 8 with the first five pixels containing 10, 20 , 0, 15, and 30 sources respectively: [0x0000000A, 0x0000001E, 0x0000001E, 0x00000033, 0x0000 0051, ...] Example As...
Show all 28 references
-
[9]
5 Monolithic Catalogs • Conduct a conesearch that returns a list of HEALpixels (at t he same HEALpixel level as the catalog is indexed) touching the search area
Search Strategy Efficient search strategies make use of the property of the ne sted HEALpixel scheme used by the Gaia catalog, which means that the HEALpix numbers of HEALpi xels in a similar part of the sky will be close together. 5 Monolithic Catalogs • Conduct a conesearch t...
-
[10]
• Extensions are appended without altering existing struct ure
Compatibility and Extensions Backward Compatibility Future updates will ensure that: • Unused header bytes retain compatibility. • Extensions are appended without altering existing struct ure. Custom Fields In order to create a catalog with different fields, the catalo g should...
-
[11]
Here, <chunk_level> and <N> may be omitted for monolithic catalogs; <type> can currently be ” astro”, ” xpsamp” or ” xpcts”
Best Practices Siril catalog files should follow the convention: siril_cat<chunk_level>_healpix<level>_<type>_<N>.dat. Here, <chunk_level> and <N> may be omitted for monolithic catalogs; <type> can currently be ” astro”, ” xpsamp” or ” xpcts”. Other <type> strings may be locall...
-
[12]
The xp sampled catalog is split into chunks each covering one level 1 HEALpixel
Data Integrity The initial Siril catalogs were compiled by importing the gaia_source table (and, in the case of the xp_sampled catalog, the xp_sampled datalink product) into a local SQL database and using python to extract the top 127 records sorted by G_mean_mag from each lev...
-
[13]
Within Siril a subset of libhealpix_cxx [ 3] is used
Implementation Guidance Libraries/Code • Use of these catalogs requires a library that support effici ent HEALpixel-based searching. Within Siril a subset of libhealpix_cxx [ 3] is used. • The reference implementation can be found at https://gitlab.com/free-astro/siril/-/blob
-
[14]
• Citation: Cite this document using its DOI and version numb er: • The DOI 10.5281/zenodo.14697485 will always refer to the l atest version of the document
Licensing and Citation • License: CC-BY -4.0 (Creative Commons Attribution). • Citation: Cite this document using its DOI and version numb er: • The DOI 10.5281/zenodo.14697485 will always refer to the l atest version of the document. • The DOI 10.5281/zenodo.14697486 refers t...
-
[15]
• RA/Dec: Right Ascension and Declination, celestial co-or dinates
Glossary • HEALpix: Hierarchical Equal Area IsoLatitude Pixelizati on. • RA/Dec: Right Ascension and Declination, celestial co-or dinates. • DR: Data Release (of Gaia data). 7
-
[16]
https://gea.esac.esa.int/archive
Gaia Data Release documentation. https://gea.esac.esa.int/archive
-
[17]
https://api.kde.org/kstars/html/Stars.html#BinaryForma
Kstars catalogue binary format. https://api.kde.org/kstars/html/Stars.html#BinaryForma
-
[18]
https://healpix.sourceforge.io/
libhealpix cxx. https://healpix.sourceforge.io/
-
[19]
https://github.com/IanCassTwo/Siril-Offline-Databas e
Siril Offline Database. https://github.com/IanCassTwo/Siril-Offline-Databas e
-
[20]
doi:10.1109/IEEESTD.2019.8766229
IEEE Standard for Floating-Point Arithmetic, 2019. doi:10.1109/IEEESTD.2019.8766229
2019
-
[21]
Siril Astrometric Extract from Gaia DR3, 2025
Ian Cass, Adrian Knagg-Baugh, C´ ecile Melis, and Cyril R ichard. Siril Astrometric Extract from Gaia DR3, 2025. doi:10.5281/zenodo.14692304
2025 doi
-
[22]
Siril Spectrophotometric Extract from Gaia DR3, 2025
Ian Cass, Adrian Knagg-Baugh, C´ ecile Melis, and Cyril Richard. Siril Spectrophotometric Extract from Gaia DR3, 2025. doi:10.5281/zenodo.14697693
2025 doi
-
[23]
The Gaia Mission
Gaia Collaboration et al. The Gaia Mission. Astronomy & Astrophysics , 595:A1, November 2016. URL: http://dx.doi.org/10.1051/0004-6361/201629272, doi:10.1051/0004-6361/201629272
2016 doi
-
[24]
Gaia Data Release 3: Summary of the con- tent and survey properties
Gaia Collaboration et al. Gaia Data Release 3: Summary of the con- tent and survey properties. Astronomy & Astrophysics , 674:A1, June
-
[26]
Analysis issues for large cmb data sets
Krzysztof M G´ orski, Eric Hivon, and Benjamin D Wandelt . Analysis issues for large cmb data sets. 1998. arXiv:astro-ph/9812350
1998 arXiv
-
[27]
The healpix primer
Krzysztof M Gorski, Benjamin D Wandelt, Frode K Hansen, Eric Hivon, and Anthony J Banday. The healpix primer. 1999. arXiv:astro-ph/9905275
1999 arXiv
-
[28]
Siril: An advanced tool for astronomical image processing
Cyril Richard, Vincent Hourdin, C´ ecile Melis, and Adrian Knagg-Baugh. Siril: An advanced tool for astronomical image processing. Journal of Open Source Software , 9(102):7242, 2024. doi:10.21105/joss.07242. 8
2024 doi
-
[2023]
URL: http://dx.doi.org/10.1051/0004-6361/202243940, doi:10.1051/0004-6361/202243940
Reviewed August 10, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.