REVIEW 3 major objections 5 minor 4 cited by
Africanus II. QuartiCal: calibrating radio interferometer data at scale using Numba and Dask
T0 review · 3 major / 5 minor · reviewed 2026-08-11 · deepseek-v4-flash
Pith's one-line read QuartiCal claims that radio-interferometer calibration can be reduced to per-element update rules for arbitrary Jones chains and scaled from a laptop to a cloud cluster via Dask.
desk verdict A solid engineering paper that delivers a working, scalable calibration package; the math is not new, the software is, and the main caveat is the unvalidated AllJones approximation in the multi-direction regime. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The load-bearing object is the diagonal-approximated, per-element Gauss-Newton update for a term $n$ in a Jones chain, written as Eq. 34. Here the model visibility for baseline $pq$ in direction $d$ is factored as left-chain times the target term times right-chain (Eq. 11), so differentiating with respect to that term's gain produces a Kronecker product of the left chain and the conjugated or transposed right chain. Under AllJones, the full normal matrix $J^{\dagger}WJ$ is replaced by its diagonal blocks, which is equivalent to assuming no covariance between any two parameters, including a gain and its conjugate. This turns a large matrix inversion into a set of independent scalar or small-block updates, and it is what makes the problem embarrassingly parallel: each solution interval can be solved without communication. The implementation then compiles the hot loops in Numba and expresses the whole calibration as a Dask task graph, with scheduler plugins to keep data on the worker that owns it.
What would settle it
Take a simulated observation with two overlapping direction-dependent gain terms where the true gains are known, calibrate with QuartiCal under AllJones, and compare the recovered gains or residual dynamic range against the same problem solved with the full Gauss-Newton normal matrix. If the AllJones solution stops improving or differs systematically at fixed iteration count, the diagonal approximation is the cause; if the two match, the approximation is doing its job.
Extended reading notes
Core claim
The paper's central claim is that the full radio interferometer measurement equation can be calibrated with per-element update rules valid for a chain of any length, with any term parameterized by real parameters, under the AllJones approximation. Working from the Wirtinger complex-derivative formalism, the authors re-express the model visibility so the gain of interest sits in a 'vec trick' factorization, then derive closed forms for the diagonal of $J^{\dagger}WJ$ and the vector $J^{\dagger}W\breve{r}$ (Eqs. 30 and 32). These combine into Eq. 34, a Gauss-Newton step for each gain element that needs no full Jacobian, no matrix inverse, and no visibility-sized intermediates. The same framework gives three concrete parameterized scenarios: a scalar parameter per gain element, parameters at coarser time or frequency resolution than the gains, and direction-independent parameters that generate direction-dependent gains. The paper demonstrates the result on MeerKAT observations of PSR J2009-2026 through first-generation, second-generation, and a direction-dependent peel, and reports roughly an order-of-magnitude lower memory use than CubiCal with better wall-clock scaling.
Load-bearing premise
The load-bearing assumption is that dropping every off-diagonal entry of the Gauss-Newton normal matrix, treating each gain and its conjugate as statistically independent, still yields updates that converge to the correct gains, especially when direction-dependent terms are coupled through the sky model.
Editorial extensions
If this is right
- A single code can carry 1GC, 2GC, and 3GC, because arbitrary Jones chains can mix direction-independent and direction-dependent terms.
- Because each solution interval is an independent subproblem, the calibration task graph can scale nearly linearly with Dask workers across cloud nodes, as measured up to 64 workers on a distributed deployment.
- Avoiding visibility-sized intermediates keeps memory use low: the paper reports approximately an order-of-magnitude reduction in footprint compared with CubiCal.
- The Zarr/XArray gain format lets solutions be reused, interpolated across fields, and consumed by an imaging pipeline without writing corrected visibility columns.
Reading between the lines
- The derivation implies that the AllJones approximation is the single point controlling accuracy on coupled direction-dependent gains; the paper demonstrates its adequacy only qualitatively, so a quantitative comparison against a full normal-matrix solve would be the natural test.
- The same per-element update structure is a natural fit for GPU execution, since each element's update is small and independent; the authors list GPU solvers as a future direction rather than claiming them here.
- The Zarr-backed Measurement Set path suggests a fully object-store-based reduction, where data is read from storage, gains are written back as Zarr, and an imaging suite applies them without ever materializing corrected-data columns; this workflow is sketched in the paper but not demonstrated.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper introduces QuartiCal, a Python package for radio interferometric calibration that generalizes the CubiCal framework to Jones chains of arbitrary length, including parameterized gain terms. Section 2 derives per-element complex Gauss-Newton update rules under the AllJones diagonal approximation, using Wirtinger calculus and the RIME. Section 4 describes the Dask/Numba implementation, including data ingestion, graph construction, solvers, and scheduler plugins. Section 5 presents qualitative MeerKAT results (1GC through a single-direction, diagonal 3GC term) and benchmarks comparing QuartiCal with CubiCal, CTDS-backed versus Zarr-backed Measurement Sets, and cloud-based distributed runs on AWS. The central claims are that QuartiCal correctly calibrates from 1GC through 3GC, outperforms CubiCal in memory and wall time, and scales from a laptop to distributed clusters.
Significance. If the claims are fully supported, QuartiCal would be a valuable community resource, addressing a real need for scalable, distributed calibration software with flexible Jones-chain support and low memory overhead. The mathematical derivation in Section 2 is a clean formalization of per-element updates for parameterized chains, and the implementation choices (Numba kernels, Dask task graphs, Zarr-backed storage, scheduler plugins) are sensible and clearly described. The reported order-of-magnitude memory improvement over CubiCal is plausible and well aligned with the stated design goal of avoiding visibility-sized intermediates. The paper is honest about several limitations, including the lack of quantitative cross-package comparison and the qualitative nature of the imaging demonstration. However, the evidence for the 3GC and wall-time claims needs strengthening before the stated conclusions can be accepted as established.
major comments (3)
- [§2.3, §5.1] The AllJones approximation is load-bearing for the 3GC claim, but it is only demonstrated for a diagonal, single-direction dE term. The approximation explicitly discards all off-diagonal entries of J†WJ, which the text describes as assuming no covariance between any parameters, including gains and their conjugates. For a multi-direction solve, Eq. (12) sums over directions, so the per-element update in Eq. (34) ignores covariance between gains in different directions; whether those off-diagonal blocks are small is not tested. The paper should add a quantitative test of this regime, ideally a simulation with known direction-dependent gains, and report gain error or residual metrics as a function of SNR, or explicitly restrict the 3GC claim to the demonstrated case.
- [§5.2] The wall-time comparison with CubiCal is not controlled. The text states that CubiCal was run with fewer solution epochs and, in the K+dE experiment, with a second diagonal complex term instead of the K term. Since one of the paper's central performance claims is that QuartiCal outperforms CubiCal in wall time, these differences undermine that specific conclusion. The memory-footprint comparison is less affected by this asymmetry, but the wall-time curves and projected lines should either be produced under matched iteration counts and term types, or the claim should be downgraded to per-iteration or per-unit-work performance.
- [§5.1] The calibration demonstration is qualitative only. The paper states that no quantitative comparison with competitors is included, but a quantitative validation of gain correctness is needed even without a competitor comparison. The conclusion that QuartiCal is 'successful in performing calibration from 1GC through to 3GC' rests on visual inspection of images. Please report at least residual RMS or dynamic-range improvement before and after each calibration stage, and, for the direction-dependent case, gain error against a known model would be much more convincing.
minor comments (5)
- [§6] In the Conclusions, 'there are still many ways in which in could be improved' should read 'in which it could be improved'.
- [§5.2, §5.4] The 'projected' curves in Figures 4, 6, and 7 appear to be linear extrapolations from the four-core or single-worker point; the figure captions or text should state this more explicitly so readers do not mistake them for measurements or fits.
- [§5.4] The GKB cloud experiment is limited to eight workers because there are eight data partitions. The statement that QuartiCal 'scales nearly linearly' should be qualified as 'over the tested range of one to eight workers', since this is a weak-scaling demonstration with a very small number of points.
- [§2.3] When introducing AllJones as the 'most extreme' approximation from Smirnov and Tasse (2015), a brief reminder of the less extreme alternatives would help readers understand the design choice.
- [§4.7] The phrase 'on the fly' throughout the paper is generally used correctly, but in §4.7 it appears as 'on the fly degridding' without hyphens; 'on-the-fly degridding' would be more standard.
Circularity Check
No significant circularity: the Gauss-Newton update rules are derived from the stated model, and the performance claims rest on external benchmarks rather than on fitted inputs or self-citations.
full rationale
No circular step could be exhibited. Section 2 derives the per-element update rules (Eqs. 30, 32, 34) from the RIME model (Eq. 12), the definition of the residual (Eq. 13), and standard Wirtinger calculus with the vec trick; the derivation is self-contained and does not take its conclusion as an input. The AllJones approximation (Section 2.3) is explicitly introduced as an assumption—'we can discard all off-diagonal entries from J†WJ. This is equivalent to assuming that there is no covariance between any of the parameters'—rather than being presented as a derived prediction, so any concern about its accuracy is a correctness/validation risk, not a circularity. The performance results in Sections 5.2-5.4 are measured wall times and memory footprints comparing QuartiCal with CubiCal and across cloud configurations; they are not quantities manufactured by fitting the model. The projected scaling curves are labeled as projections from the measured 4-core point, not as validated predictions, and this labeling is explicit. Self-citations to Paper I, Paper IV, Smirnov & Tasse 2015, and Kenyon et al. 2018 are used for background, implementation dependencies, and acknowledgement of prior mathematical results; where prior mathematics is invoked, the paper re-derives the needed expressions in Section 2 rather than relying on the citation as the sole source of the result. The paper's own stated limitations—absence of a quantitative comparison with competitors, the qualitative nature of the 3GC demonstration, the single diagonal direction-dependent term used, and the acknowledged lack of I/O/compute interleaving—are honest validation gaps and do not reduce any central claim to its own inputs. Therefore no circularity is found.
Assumptions & free parameters
assumptions (4)
- domain assumption The radio interferometer measurement equation (RIME) correctly models observed visibilities as a sum over directions of Jones chains.
- domain assumption The AllJones approximation, discarding all off-diagonal entries of J†WJ, yields sufficiently accurate updates.
- domain assumption Weights W are diagonal.
- domain assumption Gains are constant over each solution interval and independent between intervals.
Cite this review
Pith. "Pith review of Africanus II. QuartiCal: calibrating radio interferometer data at scale using Numba and Dask." pith.science (2026). https://pith.science/paper/QFV3CM7J
@misc{pith2026241210072,
author = {Pith},
title = {Pith review of: Africanus II. QuartiCal: calibrating radio interferometer data at scale using Numba and Dask},
year = {2026},
howpublished = {\url{https://pith.science/paper/QFV3CM7J}},
note = {Machine review of arXiv:2412.10072}
}
read the original abstract
Calibration of radio interferometer data ought to be a solved problem; it has been an integral part of data reduction for some time. However, as larger, more sensitive radio interferometers are conceived and built, the calibration problem grows in both size and difficulty. The increasing size can be attributed to the fact that the data volume scales quadratically with the number of antennas in an array. Additionally, new instruments may have up to two orders of magnitude more channels than their predecessors. Simultaneously, increasing sensitivity is making calibration more challenging: low-level RFI and calibration artefacts (in the resulting images) which would previously have been subsumed by the noise may now limit dynamic range and, ultimately, the derived science. It is against this backdrop that we introduce QuartiCal: a new Python package implementing radio interferometric calibration routines. QuartiCal improves upon its predecessor, CubiCal, in terms of both flexibility and performance. Whilst the same mathematical framework - complex optimization using Wirtinger derivatives - is in use, the approach has been refined to support arbitrary length chains of parameterized gain terms. QuartiCal utilizes Dask, a library for parallel computing in Python, to express calibration as an embarrassingly parallel task graph. These task graphs can (with some constraints) be mapped onto a number of different hardware configurations, allowing QuartiCal to scale from running locally on consumer hardware to a distributed, cloud-based cluster. QuartiCal's qualitative behaviour is demonstrated using MeerKAT observations of PSR J2009-2026. These qualitative results are followed by an analysis of QuartiCal's performance in terms of wall time and memory footprint for a number of calibration scenarios and hardware configurations.
Figures
Figures from the paper (4 more)
Forward citations
Cited by 4 Pith papers
-
The Peculiar Mini-Halo in the Shapley Supercluster Member Abell 3558
The diffuse radio emission in A3558 is a peculiar mini-halo with a new northern extension, a steep spectrum of 1.18, and a sublinear radio-X-ray correlation that steepens with frequency, consistent with sloshing-induc...
-
Africanus I. Scalable, distributed and efficient radio data processing with Dask-MS and Codex Africanus
Dask-MS and Codex Africanus let radio astronomers run interferometry data reduction on Dask clusters, with demonstrated strong and weak scaling for a DFT model predict on AWS.
-
Africanus IV. The Stimela2 framework: scalable and reproducible workflows, from local to cloud compute
Stimela2 introduces a YAML-based workflow framework that combines readable linear recipes with containerization and Kubernetes or Slurm backends for reproducible, scalable radio astronomy data reduction.
-
Africanus III. pfb-imaging -- a flexible radio interferometric imaging suite
A flexible imaging library, pfb-imaging, implements a sparsity-based SARA algorithm with interpretable parameters and demonstrates it on terabyte-scale MeerKAT data.
Reference graph
Works this paper leans on
-
[5]
doi: 10.5334/jors.148. J´ozsa, G.I.G., White, S.V ., Thorat, K., Smirnov, O.M., Serra, P., Ramatsoku, M., Ramaila, A.J.T., Perkins, S.J., Maccagni, F.M., Makhathini, S., Moln´ar, D.C., Kamphuis, P., Kleiner, D., Hugo, B.V ., de Blok, W.J.G., Andati, L.A.L.,
-
[10]
MeerKAT observations of starburst galaxies and AGNs within the core of XMMXCS J2215.9-1738 at z = 1.46. MNRAS 532, 2842–2859. doi: 10.1093/mnras/stae1640, arXiv:2407.03667. Kreutz-Delgado, K.,
-
[11]
ArXiv e-prints arXiv:0906.4835
The Complex Gradient Operator and the CR- Calculus. ArXiv e-prints arXiv:0906.4835. Lam, S.K., Pitrou, A., Seibert, S.,
-
[16]
Robust distributed calibration of radio interferometers with direction dependent distortions
Robust distributed calibration of radio interferometers with direction dependent dis- tortions. Signal Processing 153, 348–354. doi:10.1016/j.sigpro.2018. 07.024, arXiv:1807.11738. Parekh, V ., Kincaid, R., Hugo, B., Ramaila, A., Oozeer, N.,
work page Pith review arXiv 2018
-
[18]
Scalable, distributed and e fficient radio data pro- cessing with Dask-MS and Codex Africanus
Africanus I. Scalable, distributed and e fficient radio data pro- cessing with Dask-MS and Codex Africanus. Astronomy and Computing submitted. arXiv:2412.12052. Roberts, D.A., Crutcher, R., Young, W., Kemball, A.J.,
-
[19]
The LOFAR Two-metre Sky Survey. II. First data release. A&A 622, A1. doi:10.1051/0004-6361/201833559, arXiv:1811.07926. Smirnov, O.M.,
-
[21]
The Stimela2 framework: scalable and reproducible workflows, from local to cloud compute
Africanus IV . The Stimela2 framework: scalable and reproducible workflows, from local to cloud compute. Astron- omy and Computing submitted. arXiv:2412.10080. Smirnov, O.M., Stappers, B.W., Tasse, C., Bester, H.L., Bignall, H., Walker, M.A., Caleb, M., Rajwade, K.M., Buchner, S., Woudt, P., Ivchenko, M., Roth, L., Noordam, J.E., Camilo, F.,
-
[22]
Radio interferometric gain calibration as a complex optimization problem. MNRAS 449, 2668–2684. doi: 10.1093/ mnras/stv418, arXiv:1502.06974. Sob, U.M., Bester, H.L., Smirnov, O.M., Kenyon, J.S., Grobler, T.L.,
Show all 31 references
-
[25]
A&A 611, A87
Faceting for direction-dependent spectral deconvolution. A&A 611, A87. doi:10.1051/ 0004-6361/201731474, arXiv:1712.02078. THE CASA TEAM, Bean, B., Bhatnagar, S., Castro, S., Donovan Meyer, J., Emonts, B., Garcia, E., Garwood, R., Golap, K., Gonzalez Villalba, J., Har- ris, P....
-
[27]
Astronomy and Computing 12, 174–180
Casacore Table Data System and its use in the Mea- surementSet. Astronomy and Computing 12, 174–180. doi: 10.1016/j. ascom.2015.06.002. van Haarlem, M.P., et al.,
2015 doi
-
[30]
MNRAS 449, 4506–4514
Distributed radio interferometric calibration. MNRAS 449, 4506–4514. doi: 10.1093/mnras/stv596, arXiv:1502.00858. Yatawatta, S.,
-
[31]
MNRAS 493, 6071–6078
Stochastic calibration of radio interferometers. MNRAS 493, 6071–6078. doi: 10.1093/mnras/staa648, arXiv:2003.00986. 22
2003 arXiv
-
[90]
3390/galaxies9040090, arXiv:2111.01858
doi:10. 3390/galaxies9040090, arXiv:2111.01858. Perkins, S.J., Kenyon, J.S., Andati, L.A.L., Bester, H.L., Smirnov, O.M., Hugo, B.V .,
-
[167]
doi: 10.1145/2806777. 2809955. Castro, S., Gonzalez, J., Taylor, J., Bhatnagar, S., Caillat, M., Ford, P., Go- lap, K., Jakobs, J., Kawasaki, W., Kern, J., Kuniyoshi, M., Loveland, S., Mehringer, D., Miel, R., Moellenbrock, G., Nakazato, T., Petry, D., Poko- rny, M., Rao, U., ...
-
[255]
Hamaker, J.P., Bregman, J.D., Sault, R.J.,
arXiv:1907.07648. Hamaker, J.P., Bregman, J.D., Sault, R.J.,
1907 arXiv
- [635]
-
[2009]
IEEE Proceedings 97, 1482–1496
The Square Kilometre Array. IEEE Proceedings 97, 1482–1496. doi:10.1109/JPROC. 2009.2021005. Frostig, R., Johnson, M.J., Leary, C.,
2009
-
[2010]
A&A 524, A61
The MeqTrees software system and its use for third-generation calibration of radio interferometers. A&A 524, A61. doi:10.1051/0004-6361/201015013, arXiv:1101.1745. Offringa, A.R., McKinley, B., Hurley-Walker, N., Briggs, F.H., Wayth, R.B., Kaplan, D.L., Bell, M.E., Feng, L., N...
-
[2011]
Revisiting the radio interferometer measurement equa- tion. I. A full-sky Jones formalism. A&A 527, A106. doi: 10.1051/ 0004-6361/201016082, arXiv:1101.1764. Smirnov, O.M., Makhathini, S., Kenyon, J.S., Bester, H.L., Perkins, S.J., Ra- maila, A.J.T., Hugo, B.V .,
-
[2012]
Uncon- strained optimization of real functions in complex variables. SIAM J. Optim. 22, 879–898. doi: 10.1137/110832124, arXiv:https://doi.org/10.1137/110832124. Tasse, C., Hugo, B., Mirmont, M., Smirnov, O., Atemkeng, M., Bester, L., Hardcastle, M.J., Lakhoo, R., Perkins, S.,...
-
[2013]
A&A 556, A2
LOFAR: The LOw-Frequency ARray. A&A 556, A2. doi: 10.1051/0004-6361/201220873, arXiv:1305.3550. Villars, R.L., Olofson, C.W., Eastwood, M.,
-
[2014]
MNRAS 444, 606–619
WSCLEAN: an implementation of a fast, generic wide-field imager for radio astronomy. MNRAS 444, 606–619. doi:10.1093/mnras/stu1368, arXiv:1407.1943. Ollier, V ., El Korso, M.N., Ferrari, A., Boyer, R., Larzabal, P.,
1943 arXiv
-
[2015]
(Ed.), LLVM ’15: Proceedings of the Second Workshop on the LLVM Compiler Infrastructure in HPC, Association for Computing Machinery, New York
Numba: A llvm-based python jit com- piler, in: Finkel, H. (Ed.), LLVM ’15: Proceedings of the Second Workshop on the LLVM Compiler Infrastructure in HPC, Association for Computing Machinery, New York. doi:10.1145/2833157.2833162. Madsen, K., Nielsen, H.B., Tingleff, O.,
-
[2017]
arXiv e-prints , arXiv:1712.05889arXiv:1712.05889
Ray: A Distributed Framework for Emerging AI Applications. arXiv e-prints , arXiv:1712.05889arXiv:1712.05889. Noordam, J.E., Smirnov, O.M.,
-
[2018]
MNRAS 478, 2399–2415
CUBI- CAL - fast radio interferometric calibration suite exploiting complex op- timization. MNRAS 478, 2399–2415. doi: 10.1093/mnras/sty1221, arXiv:1805.03410. Klutse, D.Y ., Hilton, M., Heywood, I., Smail, I., Swinbank, A.M., Knowles, K., Sikhosana, S.P.,
-
[2019]
The DSA-2000 — A Radio Survey Camera, in: Bulletin of the American Astronomical Society, p
2000
-
[2020]
MNRAS 491, 1026–1042
Radio interferometric calibration using a complex Student’s t-distribution and Wirtinger derivatives. MNRAS 491, 1026–1042. doi:10.1093/mnras/ stz3037. Sorber, L., van Barel, M., de Lathauwer, L.,
-
[2021]
Computing in Science and Engineering 23, 47–54
mpi4py: Status Update After 12 Years of Devel- opment. Computing in Science and Engineering 23, 47–54. doi: 10.1109/ MCSE.2021.3083216. Dewdney, P.E., Hall, P.J., Schilizzi, R.T., Lazio, T.J.L.W.,
2021
-
[2022]
arXiv e-prints , arXiv:2210.02276arXiv:2210.02276
CASA, the Common Astronomy Software Applications for Radio Astronomy. arXiv e-prints , arXiv:2210.02276arXiv:2210.02276. Thompson, A.R., Moran, J.M., Swenson, George W., J.,
-
[2023]
AAP 673, A113
The MeerKAT Absorption Line Survey: Homogeneous continuum catalogues towards a measurement of the cosmic radio dipole. AAP 673, A113. doi: 10.1051/0004-6361/202245477, arXiv:2302.10696. Wirtinger, W.,
-
[2024]
pfb-imaging – a flexible radio interferometric imaging suite
Africanus III. pfb-imaging – a flexible radio interferometric imaging suite. Astronomy and Computing submitted. arXiv:2412.10073. Brewer, E.A.,
Reviewed August 11, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.