REVIEW 4 major objections 7 minor 31 references
SONNET: Enhancing Time Delay Estimation by Leveraging Simulated Audio
T0 review · 4 major / 7 minor · reviewed 2026-08-12 · deepseek-v4-flash
Pith's one-line read SONNET, a neural network trained entirely on simulated room audio, estimates inter-microphone time differences more accurately than the standard GCC-PHAT method on novel real recordings, and the better estimates let a downstream…
desk verdict A credible sim-to-real TDE result, but the 'novel real data' claim is softened by model selection on the single real test set. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The central object is the SONNET model: two recordings are transformed with a fast Fourier transform, frequencies above 4800 Hz are discarded, and the real and imaginary parts are fed as separate channels through three 1D convolutional layers, followed by stacked residual blocks with skip connections, ending in a linear layer over 1000 TDOA classes. The model is trained with cross-entropy loss and label smoothing on 12 million pairs of simulated recordings, taken from 50 choose 2 pairs in each of 10,000 rooms. The simulation machinery matters as much as the network: moving sources are approximated by splitting each source signal into parts and simulating each part as a stationary source at successive points along a path, and directional microphones use a subcardioid sensitivity pattern. This combination is what the paper argues closes the gap between synthetic training data and real audio.
What would settle it
Take SONNET without retraining and evaluate it on a second real-world TDOA dataset with ground truth, recorded in different rooms, with different microphone hardware and different sound content; if its inlier ratio at a 10 cm threshold does not beat GCC-PHAT, the generalization claim is falsified. A cheaper check is to retrain the model with the movement or directionality augmentations removed and observe whether the real-data performance gap the paper reports disappears.
Extended reading notes
Core claim
On the paper's own terms, the discovery is that the sim-to-real gap for audio time-delay estimation is closable by scaling up and enriching the simulation rather than by adding more network complexity. The training set is generated with an image-source room simulator: rectangular rooms with side lengths sampled uniformly from 1 to 10 meters, wall reflection coefficients from 0.05 to 0.99, 50 microphones per room, and sources that are either stationary or moving along smooth quadratic paths at speeds up to 5 m/s, modeled as discretized jumps. Each source is given a subcardioid directional sensitivity pattern. The trained model outputs logits over 1000 classes, each corresponding to a one-sample TDOA bin at 16 kHz, matching GCC-PHAT's resolution. On real data, SONNET achieves a higher inlier ratio at every tested threshold, and the ablation studies show that source movement contributes more than directionality and that larger training sets generalize better.
Load-bearing premise
The load-bearing premise is that the simulated rooms, with randomly sampled sizes, reflection coefficients, moving sources, and directional microphones, faithfully represent the acoustic conditions of real recordings such as the tdoa_20201016 dataset, and that this one dataset stands for novel real-world data generally.
Editorial extensions
If this is right
- A ready-to-use, real-time TDE model exists for ad-hoc microphone pairs: SONNET runs in about 0.94 ms per pair on a CPU and 0.022 ms on a GPU, roughly four times slower than GCC-PHAT but still real-time.
- Self-calibration of microphone arrays becomes far more reliable: with SONNET's TDOA estimates the calibration system converges on all tested recordings, whereas with GCC-PHAT several experiments fail or give errors above 1 m.
- The synthetic-data recipe can be reused to train estimators for problems where ground-truth TDOA is hard to collect, such as multiple simultaneous sources, multipath components, or more than two microphones.
- The model's one-sample-bin output at 16 kHz means the accuracy gain is not achieved by coarser quantization than the baseline.
Reading between the lines
- If the sim-to-real transfer holds beyond this one dataset, the same large-scale simulation strategy could be applied to other signal modalities such as radio or ultrasound, where ground-truth TDOA labels are equally scarce.
- A natural next test is a second independent real-world corpus with different rooms and hardware; the current evidence rests on a single dataset, so that experiment would directly measure how representative that dataset is.
- Because the network outputs a full distribution over delay classes, it could be used to flag uncertain or outlier estimates before feeding downstream solvers, an extension the paper does not explore.
- The ablation result that movement matters more than directionality suggests that simulating dynamic sources is the key ingredient; testing even richer motion models, such as continuous Doppler shifts, is a concrete next step.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper proposes SONNET, a learned model for time-delay estimation (TDE) from pairs of audio recordings. The model is trained exclusively on a large simulated dataset generated with Pyroomacoustics, using random rectangular rooms, wall reflection coefficients in [0.05, 0.99], subcardioid directivity, and sound sources that are either stationary or follow discretized quadratic Bézier paths. The network uses a Fourier front-end, a convolutional backbone, and a ResNet with regression-via-classification over 1000 one-sample TDOA bins. The authors claim that SONNET, trained only on synthetic data, significantly outperforms GCC-PHAT on novel real-world recordings, and they demonstrate improved downstream self-calibration using SONNET-based TDOA estimates. The experiments include inference speed measurements, simulated noise/reverberation sensitivity tests, an evaluation on the real tdoa_20201016 dataset with ground truth, self-calibration results, and ablations over the simulation augmentations and dataset size.
Significance. If the central claim holds, the paper offers a practical and immediately usable TDE model that improves over a classical baseline without retraining on real data. The strengths are the release of a pretrained model, the use of an external real-world dataset with ground truth, the downstream self-calibration evaluation, and the ablation studies that identify which simulation choices matter. However, the generality of the result is currently supported by only one real-world dataset, and the simulation design choices were, in part, selected using that same dataset. These issues do not invalidate the pipeline, but they do require additional evidence or more cautious claims before the paper can be accepted.
major comments (4)
- [Section 6.5, Fig. 7a] The ablation study that motivates the two central simulation augmentations (source movement and directionality) is evaluated on tdoa_20201016, the same dataset used for the headline real-data results in Section 6.3 and Fig. 4. The full SONNET configuration was therefore retained, at least partly, because it improves accuracy on this exact dataset. As a result, the dataset cannot be considered 'novel' with respect to all design choices, and the reported margin over GCC-PHAT may partly reflect selection on the test distribution. Please provide an additional real-world evaluation on a dataset that played no role in any ablation or design decision, or use a nested holdout procedure in which the final model is evaluated only on data not used for any model selection.
- [Section 6.3 and abstract] The real-world evidence rests entirely on a single dataset, tdoa_20201016, which appears to involve one recording environment and one microphone configuration. The abstract's claim of 'novel real world data' and Section 1.3's claim of generalization to real data extrapolate from this single dataset. Please add at least one independent real-world TDE dataset with ground truth, or explicitly limit the claims to the tested dataset and describe what properties of the dataset are needed for the result to transfer.
- [Figures 3, 4, 7 and Table 2] The paper reports inlier-ratio curves and RMS errors without error bars, confidence intervals, or significance tests. Since the abstract says SONNET 'significantly outperforms' GCC-PHAT, the word 'significantly' needs statistical support. Furthermore, the 384,648 window pairs come from a small number of rooms and recordings, so pairs are not independent; any confidence intervals or tests should account for clustering by room or recording, or the effective sample size and the significance claim will be unclear.
- [Section 1.3 and Section 6] The contribution statement says SONNET 'outperforms state of the art methods for TDE', but the experiments compare only with GCC-PHAT. The related work lists several learned TDE methods (refs. 9-11, 25), so a comparison with at least one learned baseline trained on similar simulated data is needed to support the 'state of the art' claim. Alternatively, the claim should be rephrased to say that SONNET outperforms GCC-PHAT, which is what the experiments actually show.
minor comments (7)
- [Section 3.1, Eq. (4)] The notation hi(t, j/k) in Eq. (4) is ambiguous; it should be hi(t, t_j) with t_j = jT/k. In addition, the surrounding sentence about 'simulating part x(t), i-1 k T < t < i k T' is garbled by the formatting and should be rewritten with clear parentheses.
- [Section 6.2] The simulated evaluation in Fig. 3 uses audio from tdoa_20201016 played through simulated rooms. This is a reasonable stress test, but it should be described as testing robustness to those source types in synthetic reverberation, not as a measure of generalization to novel real-world reverberation.
- [Fig. 3 caption] The caption states that 'GCC-PHAT is very robust against white noise' without explaining that this refers to the flatness of the curve; please clarify the intended meaning.
- [Fig. 7a caption] The legend entries 'SONNET (d)', 'SONNET (m)', and 'SONNET (d+m)' are not defined in the caption. Please define the abbreviations in the caption or refer explicitly to the text where they are introduced.
- [Table 1] Table 1 would benefit from stating the CPU model and whether the reported times include preprocessing or only the network forward pass; the text gives the CPU model but the caption does not.
- [Section 5] The 12 million training pairs are generated from only 10,000 rooms, so pairs from the same room share the room impulse response and are not independent. This should be acknowledged when discussing dataset scale and generalization.
- [Section 6.3] Please provide a brief description of the tdoa_20201016 recording setup (room size, microphone layout, source types) or a more specific reference to where it is described, since the paper's main evaluation depends on this dataset.
Circularity Check
Simulation recipe is ablated on the same real dataset later used to claim 'novel real world data'; the core synthetic training is still independent.
-
fitted input called prediction
[Sec 3 (simulation motivation); Sec 6.3 (real-data evaluation); Sec 6.5 (ablation, Fig 7)]
"with the goal of making the simulation better reflect reality, we both simulate a moving sound source and also microphones and sound sources which are not omnidirectional. How and why will be explained in more detail in the following sections and motivated by our ablation study in section 6.5. ... For each of the four configurations we trained a separate model in the same way as the full SONNET model. Each of the models were then evaluated on tdoa_20201016 in the same way as in section 6.3, and the results are shown in Fig 7a."
The simulation recipe (movement, directionality, dataset scale) is selected by measuring performance on tdoa_20201016 in the ablation study, and the headline 'novel real world data' result is then reported on the same tdoa_20201016 dataset. The evaluation target is therefore not independent of the model-selection decisions: the reported margin over GCC-PHAT is partly a chosen optimum on that dataset, not an out-of-the-box prediction on unseen real data. The core training loop is not circular because SONNET never trains on real TDOA labels, so this is a mild evaluation circularity/selection bias rather than a definitional equivalence.
full rationale
The model is trained entirely on simulated audio with ground-truth TDOA labels generated from room geometry; no real labels enter training. The real-data benchmark (tdoa_20201016) is an external measurement, and even the ablated baseline without movement/directionality outperforms GCC-PHAT, so the central sim-to-real transfer is not tautological. The main circularity concern is that the simulation hyperparameters were selected via ablation on the same real dataset used for the headline claim, so 'novel real world data' is overstated. No self-citation is load-bearing: the cited prior work [2,17,30] shares authors but supplies architecture conventions and downstream machinery rather than the claimed result. The paper would be strengthened by an independent real dataset held out from all model-selection decisions.
Assumptions & free parameters
free parameters (6)
- Simulation room geometry range =
1-10 m per dimension
- Wall reflection coefficient range =
[0.05, 0.99]
- Maximum source velocity =
5 m/s
- Directionality pattern =
subcardioid
- Dataset scale =
10,000 rooms / 12 million pairs
- Model output classes and bin width =
1000 classes, 1 sample bins
assumptions (5)
- domain assumption Pyroomacoustics image-source method produces impulse responses that are realistic enough for training to transfer to real rooms.
- domain assumption Subcardioid directivity is a sufficient model of microphone and loudspeaker directionality for the test hardware.
- domain assumption The tdoa_20201016 dataset provides accurate ground truth TDOA values and represents novel real-world audio.
- domain assumption Audio content in MUSAN is representative of real-world signals encountered in TDE applications.
- domain assumption FFT truncation at 4800 Hz retains enough information for accurate TDOA estimation.
Cite this review
Pith. "Pith review of SONNET: Enhancing Time Delay Estimation by Leveraging Simulated Audio." pith.science (2026). https://pith.science/paper/3O5IYCO6
@misc{pith2026241113179,
author = {Pith},
title = {Pith review of: SONNET: Enhancing Time Delay Estimation by Leveraging Simulated Audio},
year = {2026},
howpublished = {\url{https://pith.science/paper/3O5IYCO6}},
note = {Machine review of arXiv:2411.13179}
}
read the original abstract
Time delay estimation or Time-Difference-Of-Arrival estimates is a critical component for multiple localization applications such as multilateration, direction of arrival, and self-calibration. The task is to estimate the time difference between a signal arriving at two different sensors. For the audio sensor modality, most current systems are based on classical methods such as the Generalized Cross-Correlation Phase Transform (GCC-PHAT) method. In this paper we demonstrate that learning based methods can, even based on synthetic data, significantly outperform GCC-PHAT on novel real world data. To overcome the lack of data with ground truth for the task, we train our model on a simulated dataset which is sufficiently large and varied, and that captures the relevant characteristics of the real world problem. We provide our trained model, SONNET (Simulation Optimized Neural Network Estimator of Timeshifts), which is runnable in real-time and works on novel data out of the box for many real data applications, i.e. without re-training. We further demonstrate greatly improved performance on the downstream task of self-calibration when using our model compared to classical methods.
Figures
Figures from the paper (4 more)
Reference graph
Works this paper leans on
-
[1]
The Journal of the Acoustical Society of America65(4), 943–950 (1979)
Allen, J.B., Berkley, D.A.: Image method for efficiently simulating small-room acoustics. The Journal of the Acoustical Society of America65(4), 943–950 (1979)
work page 1979
-
[2]
Berg, A., O’Connor, M., Åström, K., Oskarsson, M.: Extending GCC-PHAT us- ing Shift Equivariant Neural Networks. In: Proc. Interspeech 2022. pp. 1791–1795 (2022). https://doi.org/10.21437/Interspeech.2022-524
-
[3]
In: 2011 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)
De Sena, E., Hacıhabiboğlu, H., Cvetković, Z.: A generalized design method for directivity patterns of spherical microphone arrays. In: 2011 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP). pp. 125–128. IEEE (2011)
work page 2011
-
[4]
IEEE/ACM Transactions on Audio, Speech, and Language Processing29, 300–311 (2020)
Diaz-Guerra, D., Miguel, A., Beltran, J.R.: Robust sound source tracking using srp- phat and 3d convolutional neural networks. IEEE/ACM Transactions on Audio, Speech, and Language Processing29, 300–311 (2020)
work page 2020
-
[5]
In: 2014 22nd European Signal Processing Conference (EUSIPCO)
Dokmanić, I., Daudet, L., Vetterli, M.: How to localize ten microphones in one finger snap. In: 2014 22nd European Signal Processing Conference (EUSIPCO). pp. 2275–2279. IEEE (2014)
work page 2014
-
[6]
Feng, L., Gong, Y., Zhang, X.L.: Soft label coding for end-to-end sound source localization with ad-hoc microphone arrays. In: ICASSP 2023-2023 IEEE Interna- tional Conference on Acoustics, Speech and Signal Processing (ICASSP). pp. 1–5. IEEE (2023)
work page 2023
-
[7]
Flood, G., Heyden, A., Åström, K.: Stochastic analysis of time-difference and doppler estimates for audio signals. In: Pattern Recognition Applications and Methods: 7th International Conference, ICPRAM 2018, Funchal, Madeira, Portu- gal, January 16-18, 2018, Revised Selected Papers 7. pp. 116–138. Springer (2019)
work page 2019
-
[8]
Gong, Y., Liu, S., Zhang, X.L.: End-to-end two-dimensional sound source localiza- tion with ad-hoc microphone arrays. In: 2022 Asia-Pacific Signal and Information SONNET: Enhancing Time Delay Estimation 15 Processing Association Annual Summit and Conference (APSIPA ASC). pp. 1944–
work page 2022
Show all 31 references
-
[9]
In: ICASSP 2023-2023 IEEE In- ternational Conference on Acoustics, Speech and Signal Processing (ICASSP)
Grinstein, E., Brookes, M., Naylor, P.A.: Graph neural networks for sound source localization on distributed microphone networks. In: ICASSP 2023-2023 IEEE In- ternational Conference on Acoustics, Speech and Signal Processing (ICASSP). pp. 1–5. IEEE (2023)
2023
-
[10]
EURASIP Journal on Audio, Speech, and Music Pro- cessing 2023(1), 32 (2023)
Grinstein, E., Neo, V.W., Naylor, P.A.: Dual input neural networks for positional sound source localization. EURASIP Journal on Audio, Speech, and Music Pro- cessing 2023(1), 32 (2023)
2023
-
[11]
The Journal of the Acoustical Society of America 152(1), 107–151 (2022)
Grumiaux, P.A., Kitić, S., Girin, L., Guérin, A.: A survey of sound source localiza- tion with deep learning methods. The Journal of the Acoustical Society of America 152(1), 107–151 (2022)
2022
-
[12]
IEEE Trans
Gustafsson, T., Rao, B.D., Trivedi, M.: Source localization in reverberant envi- ronments: modeling and statistical analysis. IEEE Trans. Speech Audio Process. 11(6), 791–803 (Nov 2003).https://doi.org/10.1109/TSA.2003.818027
2003
-
[13]
In: Interspeech
He, Y., Markham, A.: Sounddoa: Learn sound source direction of arrival and se- mantics from sound raw waveforms. In: Interspeech. pp. 2408–2412 (2022)
2022
-
[14]
arXiv preprint arXiv:1606.08415 (2016)
Hendrycks, D., Gimpel, K.: Gaussian error linear units (gelus). arXiv preprint arXiv:1606.08415 (2016)
2016 arXiv
-
[15]
Acoustics, Speech and Signal Processing, IEEE Transactions on24(4), 320 – 327 (aug 1976).https://doi.org/10.1109/TASSP.1976.1162830, https://www
Knapp, C., Carter, G.: The generalized correlation method for estimation of time delay. Acoustics, Speech and Signal Processing, IEEE Transactions on24(4), 320 – 327 (aug 1976).https://doi.org/10.1109/TASSP.1976.1162830, https://www. ee.iitb.ac.in/course/~sachinnayak/finalpaper2.pdf
1976
-
[16]
In: ICASSP 2019 - 2019 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)
Larsson, M., Larsson, V., Åström, K., Oskarsson, M.: Optimal Trilateration Is an Eigenvalue Problem. In: ICASSP 2019 - 2019 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP). pp. 5586–5590 (May 2019). https://doi.org/10.1109/ICASSP.2019.8683355,...
2019
-
[17]
In: IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) (2021)
Larsson, M., Flood, G., Oskarsson, M., Åström, K.: Fast and robust stratified self- calibration using time-difference-of-arrival measurements. In: IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) (2021)
2021
-
[18]
arXiv preprint arXiv:1711.05101 (2017)
Loshchilov, I., Hutter, F.: Decoupled weight decay regularization. arXiv preprint arXiv:1711.05101 (2017)
2017 arXiv
-
[19]
In: Wallach, H., Larochelle, H., Beygelzimer, A., d’Alché Buc, F., Fox, E., Garnett, R
Paszke, A., Gross, S., Massa, F., Lerer, A., Bradbury, J., Chanan, G., Killeen, T., Lin, Z., Gimelshein, N., Antiga, L., Desmaison, A., Kopf, A., Yang, E., DeVito, Z., Raison, M., Tejani, A., Chilamkurthy, S., Steiner, B., Fang, L., Bai, J., Chintala, S.: PyTorch: An Imperativ...
2019
-
[20]
IEEE Signal Processing Magazine33(4), 14–29 (2016)
Plinge, A., Jacob, F., Haeb-Umbach, R., Fink, G.A.: Acoustic microphone ge- ometry calibration: An overview and experimental evaluation of state-of-the-art algorithms. IEEE Signal Processing Magazine33(4), 14–29 (2016)
2016
-
[21]
In: International conference on machine learning
Rahaman, N., Baratin, A., Arpit, D., Draxler, F., Lin, M., Hamprecht, F., Ben- gio, Y., Courville, A.: On the spectral bias of neural networks. In: International conference on machine learning. pp. 5301–5310. PMLR (2019)
2019
-
[22]
In: 2018 IEEE interna- tional conference on acoustics, speech and signal processing (ICASSP)
Scheibler, R., Bezzam, E., Dokmanić, I.: Pyroomacoustics: A python package for audio room simulation and array processing algorithms. In: 2018 IEEE interna- tional conference on acoustics, speech and signal processing (ICASSP). pp. 351–
2018
-
[23]
Snyder, D., Chen, G., Povey, D.: MUSAN: A Music, Speech, and Noise Corpus (2015), arXiv:1510.08484v1
2015 arXiv
-
[24]
In: 30th European Signal Processing Con- ference, EUSIPCO 2022 - Proceedings
Tegler, E., Larsson, M., Oskarsson, M., Åström, K.: Sensor node calibration in presence of a dominant reflective plane. In: 30th European Signal Processing Con- ference, EUSIPCO 2022 - Proceedings. pp. 1941–1945. European Signal Processing Conference, European Signal Processin...
2022
-
[25]
Sensors 18(10), 3418 (2018)
Vera-Diaz, J.M., Pizarro, D., Macias-Guarasa, J.: Towards end-to-end acoustic localization using deep learning: From audio signals to source position coordinates. Sensors 18(10), 3418 (2018)
2018
-
[26]
arXiv preprint arXiv:2305.19610 (2023)
Wang, Y., Yang, B., Li, X.: Fn-ssl: Full-band and narrow-band fusion for sound source localization. arXiv preprint arXiv:2305.19610 (2023)
2023 arXiv
-
[27]
In: ICASSP 2023-2023 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)
Zetterqvist, G., Gustafsson, F., Hendeby, G.: Using received power in microphone arrays to estimate direction of arrival. In: ICASSP 2023-2023 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP). pp. 1–5. IEEE (2023)
2023
-
[28]
In: 2014 22nd European Signal Processing Conference (EUSIPCO)
Zhayida, S., Andersson, F., Kuang, Y., Åström, K.: An automatic system for mi- crophone self-localization using ambient sound. In: 2014 22nd European Signal Processing Conference (EUSIPCO). IEEE (2014)
2014
-
[29]
arXiv preprint arXiv:1610.02392 (2016)
Zhayida, S., Rex, S.S., Kuang, Y., Andersson, F., Åström, K.: An automatic sys- tem for acoustic microphone geometry calibration based on minimal solvers. arXiv preprint arXiv:1610.02392 (2016)
2016 arXiv
-
[30]
In: 29th Eu- ropean Signal Processing Conference (EUSIPCO) (2021)
Åström, K., Larsson, M., Flood, G., Oskarsson, M.: Extension of Time-Difference- of-Arrival Self Calibration Solutions Using Robust Multilateration. In: 29th Eu- ropean Signal Processing Conference (EUSIPCO) (2021). https://doi.org/10. 23919/EUSIPCO54536.2021.9616051
2021
-
[355]
IEEE (2018) 16 Tegler et al
2018
Reviewed August 12, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.