pith. sign in

arxiv: 2606.22399 · v1 · pith:AUHZBP5Snew · submitted 2026-06-21 · 💻 cs.SD

ATCCaps: A Call-Sign-Aware Speech Dataset for Air Traffic Control Recognition

Dongdong Li , Jianwei Song , Jianwei Wang , Zhe Wang This is my paper

Pith reviewed 2026-06-26 09:59 UTC · model grok-4.3

classification 💻 cs.SD
keywords air traffic controlspeech datasetcall signsautomatic speech recognitionaudio-text supervisionradiotelephony recordingsdataset construction pipeline
0
0 comments X

The pith

ATCCaps supplies 202.94 hours of ATC audio paired with normalized call-sign labels and captions.

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper presents ATCCaps as a dataset built from real ATC radiotelephony recordings that adds call-sign supervision to audio-text pairs. A pipeline of transcript parsing, ADS-B metadata integration, normalization, rule-based filtering, and LLM caption generation produces the annotations. The resulting collection supports ASR evaluation, call-sign matching, and audio-text retrieval tasks. Characterization includes split statistics, coverage of 922 unique call signs, seen versus unseen analysis, and caption quality checks. Evaluation draws from the human-annotated ATCO2-test-set, showing the dataset enables scalable supervision while underscoring the need to verify call-sign and numeric accuracy.

Core claim

ATCCaps is a call-sign-aware ATC speech dataset containing 202.94 hours of curated audio, 170,385 utterances, and 922 unique normalized call signs. Each sample is paired with transcript descriptions, call-sign descriptions, and ATC-style captions generated through a pipeline that combines confidence-aware transcript parsing, ADS-B-derived metadata, normalization, rule-based filtering, and LLM assistance. This structure supports ASR evaluation, call-sign matching, and call-sign-aware audio-text retrieval, with the evaluation subset drawn from the human-annotated ATCO2-test-set.

What carries the argument

The multi-step construction pipeline that merges ADS-B-derived call-sign metadata with transcript parsing, normalization, filtering, and LLM-assisted caption generation to create labeled audio samples.

If this is right

  • Enables reference ASR evaluation using the human-annotated ATCO2-test-set.
  • Supports call-sign matching and call-sign-aware audio-text retrieval experiments.
  • Provides data for analyzing seen versus unseen call-sign generalization.
  • Highlights the requirement for explicit validation of call-sign and numeric fidelity in LLM-generated captions.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • The dataset could serve as a template for adding entity-specific labels to speech corpora in other safety-critical domains.
  • Caption quality audits may need expansion to include numeric sequences beyond call signs.
  • The scale of 170k utterances suggests utility for pre-training retrieval models if label noise remains low.
  • Seen/unseen call-sign splits could be used to test robustness of future recognition systems.

Load-bearing premise

The multi-step pipeline produces accurate call-sign labels and captions without substantial undetected errors.

What would settle it

A manual review of a random sample of generated captions and assigned call signs that finds frequent mismatches with the spoken content or ADS-B metadata.

Figures

Figures reproduced from arXiv: 2606.22399 by Dongdong Li, Jianwei Song, Jianwei Wang, Zhe Wang.

Figure 1
Figure 1. Figure 1: Overview of the ATCCaps construction pipeline. The ATCO2-PL branch parses machine-transcription files into speaker-turn utterances and uses [PITH_FULL_IMAGE:figures/full_fig_p004_1.png] view at source ↗
read the original abstract

Call signs are safety-critical entities in air traffic control (ATC) communications because they identify the target aircraft of each spoken instruction. This paper presents ATCCaps, a call-sign-aware ATC speech dataset with caption-level audio-text supervision. Built from real ATC radiotelephony recordings, ATCCaps contains 202.94 hours of curated audio, 170,385 utterances, and 922 unique normalized call signs. The construction pipeline combines confidence-aware transcript parsing, ADS-B-derived call-sign metadata, call-sign normalization, rule-based quality filtering, and LLM-assisted caption generation. Each retained sample is paired with transcript descriptions, call-sign descriptions, and ATC-style captions, supporting ASR evaluation, call-sign matching, and call-sign-aware audio-text retrieval. We further characterize ATCCaps through split statistics, call-sign coverage, seen/unseen call-sign analysis, filtering audits, and caption quality evaluation. The evaluation subset is derived from the human-annotated ATCO2-test-set, enabling reference evaluation with manual transcripts. Results show that ATCCaps provides scalable audio-grounded call-sign supervision, while caption analysis highlights the need to explicitly validate call-sign and numeric fidelity. Reference ASR and CLAP-based baselines demonstrate the usability of ATCCaps for call-sign-aware ATC speech modeling.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit. Tearing a paper down is the easy half of reading it; the pith above is the substance, this is the friction.

Referee Report

2 major / 1 minor

Summary. The paper presents ATCCaps, a call-sign-aware ATC speech dataset containing 202.94 hours of audio across 170,385 utterances and 922 unique normalized call signs. It is constructed from real radiotelephony recordings via a pipeline of confidence-aware transcript parsing, ADS-B-derived call-sign metadata alignment, normalization, rule-based filtering, and LLM-assisted caption generation. Each sample includes transcript descriptions, call-sign descriptions, and ATC-style captions. The work characterizes the dataset via split statistics, call-sign coverage, seen/unseen analysis, filtering audits, and caption quality evaluation; an evaluation subset is drawn from the human-annotated ATCO2-test-set. Reference ASR and CLAP-based baselines are provided to demonstrate usability for call-sign-aware modeling.

Significance. If the pipeline yields accurate call-sign and numeric labels, ATCCaps would constitute a substantial resource for safety-critical ATC speech tasks by supplying large-scale audio-grounded supervision at the call-sign level. The explicit inclusion of seen/unseen call-sign splits and the use of ADS-B as ground truth for metadata are strengths. The paper itself notes the need for explicit validation of call-sign and numeric fidelity, which correctly identifies a key requirement for downstream adoption.

major comments (2)
  1. [Abstract and Results] Abstract and Results section: The central claim that ATCCaps supplies 'scalable audio-grounded call-sign supervision' depends on low undetected error in the multi-step pipeline. Only aggregate filtering statistics and audits are reported; no per-sample fidelity metrics, manual validation error rates, or quantitative call-sign accuracy assessment on the 170k utterances are provided. This is load-bearing for the dataset's claimed utility.
  2. [Pipeline description] Pipeline description (construction section): The rule-based filtering and LLM-assisted caption generation steps can introduce misalignment or hallucination on numeric and call-sign content even when ADS-B metadata is reliable. No ablation or sensitivity analysis quantifies the contribution of each filtering stage to final label accuracy.
minor comments (1)
  1. Consider adding a dedicated limitations subsection that explicitly discusses the absence of per-sample validation and the reliance on aggregate statistics.

Simulated Author's Rebuttal

2 responses · 0 unresolved

We thank the referee for the thoughtful and constructive review. The comments correctly identify that pipeline fidelity is central to the dataset's value for safety-critical ATC tasks. We address each major comment below and outline planned revisions.

read point-by-point responses

  1. Referee: [Abstract and Results] Abstract and Results section: The central claim that ATCCaps supplies 'scalable audio-grounded call-sign supervision' depends on low undetected error in the multi-step pipeline. Only aggregate filtering statistics and audits are reported; no per-sample fidelity metrics, manual validation error rates, or quantitative call-sign accuracy assessment on the 170k utterances are provided. This is load-bearing for the dataset's claimed utility.

    Authors: We agree that undetected errors in call-sign and numeric labels would undermine the claimed utility. The manuscript already states in the abstract and conclusion that 'caption analysis highlights the need to explicitly validate call-sign and numeric fidelity.' The provided aggregate statistics, filtering audits, and seen/unseen splits represent the initial characterization feasible at dataset-release scale. Manual per-sample validation across 170k utterances was not performed due to cost; we will add an expanded Limitations section that quantifies this gap, reports error rates on the human-annotated ATCO2-derived evaluation subset, and outlines a protocol for future large-scale validation. revision: partial

  2. Referee: [Pipeline description] Pipeline description (construction section): The rule-based filtering and LLM-assisted caption generation steps can introduce misalignment or hallucination on numeric and call-sign content even when ADS-B metadata is reliable. No ablation or sensitivity analysis quantifies the contribution of each filtering stage to final label accuracy.

    Authors: We concur that rule-based filters and LLM captioning can introduce errors even with reliable ADS-B metadata. The construction section details each stage and supplies aggregate audit results, but no stage-wise ablation or sensitivity study was included. We will add a short sensitivity analysis in the revised manuscript that measures the incremental effect of each filtering rule on call-sign retention and accuracy using the ATCO2-test-set evaluation subset, thereby quantifying the contribution of individual stages. revision: yes

Circularity Check

0 steps flagged

No circularity: dataset construction paper with no derivations or fitted predictions

full rationale

The manuscript is a data curation and characterization effort. It describes a multi-stage pipeline (transcript parsing, ADS-B alignment, normalization, filtering, LLM captioning) and reports aggregate statistics, coverage metrics, and baseline evaluations. No equations, parameter fitting, uniqueness theorems, or self-citation chains appear. The central claim—that the resulting dataset supplies scalable call-sign supervision—is an empirical assertion about the output of the described process, not a derivation that reduces to its own inputs by construction. The reader's assessment of zero circularity is confirmed by direct inspection of the text.

Axiom & Free-Parameter Ledger

0 free parameters · 0 axioms · 0 invented entities

Dataset construction paper with no mathematical model, free parameters, axioms, or postulated entities.

pith-pipeline@v0.9.1-grok · 5756 in / 1134 out tokens · 29443 ms · 2026-06-26T09:59:27.970967+00:00 · methodology

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

40 extracted references · 8 canonical work pages · 2 internal anchors

  1. [1]

    Spoken instruction understanding in air traffic control: Chal- lenge, technique, and application,

    Y . Lin, “Spoken instruction understanding in air traffic control: Chal- lenge, technique, and application,”Aerospace, vol. 8, no. 3, p. 65, 2021

    2021

  2. [2]

    Automatic speech recognition for air traffic control communications,

    S. Badrinath and H. Balakrishnan, “Automatic speech recognition for air traffic control communications,”Transportation Research Record, vol. 2676, no. 1, pp. 798–810, 2022

    2022

  3. [3]

    Speech recognition for air traffic control via feature learning and end-to-end training,

    P. Fan, X. Hua, Y . Lin, B. Yang, J. Zhang, W. Ge, and D. Guo, “Speech recognition for air traffic control via feature learning and end-to-end training,”IEICE Transactions on Information and Systems, vol. 106, no. 4, pp. 538–544, 2023

    2023

  4. [4]

    Adapting automatic speech recognition for accented air traffic control communications,

    M. Y . Z. Wee, J. J. H. Wong, L. Lim, J. Y . W. Tan, P. Gupta, D. Lim, E. H. Tew, A. K. S. Han, and Y . Z. Lim, “Adapting automatic speech recognition for accented air traffic control communications,”arXiv preprint arXiv:2502.20311, 2025

    work page arXiv 2025

  5. [5]

    wav2vec 2.0: A framework for self-supervised learning of speech representations,

    A. Baevski, Y . Zhou, A. Mohamed, and M. Auli, “wav2vec 2.0: A framework for self-supervised learning of speech representations,” Advances in neural information processing systems, vol. 33, pp. 12 449– 12 460, 2020

    2020

  6. [6]

    Hubert: Self-supervised speech representation learning by masked prediction of hidden units,

    W.-N. Hsu, B. Bolte, Y .-H. H. Tsai, K. Lakhotia, R. Salakhutdinov, and A. Mohamed, “Hubert: Self-supervised speech representation learning by masked prediction of hidden units,”IEEE/ACM Transactions on Audio, Speech, and Language Processing, vol. 29, pp. 3451–3460, 2021

    2021

  7. [7]

    Robust speech recognition via large-scale weak super- vision,

    A. Radford, J. W. Kim, T. Xu, G. Brockman, C. McLeavey, and I. Sutskever, “Robust speech recognition via large-scale weak super- vision,” inInternational Conference on Machine Learning. PMLR, 2023, pp. 28 492–28 518

    2023

  8. [8]

    Fused acoustic and text encoding for multimodal bilingual pretraining and speech translation,

    R. Zheng, J. Chen, M. Ma, and L. Huang, “Fused acoustic and text encoding for multimodal bilingual pretraining and speech translation,” inInternational Conference on Machine Learning. PMLR, 2021, pp. 12 736–12 746

    2021

  9. [9]

    Speechlm: Enhanced speech pre-training with unpaired textual data,

    Z. Zhang, S. Chen, L. Zhou, Y . Wu, S. Ren, S. Liu, Z. Yao, X. Gong, L. Dai, J. Liet al., “Speechlm: Enhanced speech pre-training with unpaired textual data,”IEEE/ACM Transactions on Audio, Speech, and Language Processing, 2024

    2024

  10. [10]

    Large-scale contrastive language-audio pretraining with feature fusion and keyword-to-caption augmentation,

    Y . Wu, K. Chen, T. Zhang, Y . Hui, T. Berg-Kirkpatrick, and S. Dubnov, “Large-scale contrastive language-audio pretraining with feature fusion and keyword-to-caption augmentation,” inICASSP 2023-2023 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP). IEEE, 2023, pp. 1–5

    2023

  11. [11]

    The airbus air traffic control speech recognition 2018 challenge: Towards atc automatic transcription and call sign detection,

    T. Pellegrini, J. Farinas, E. Delpech, and F. Lancelot, “The airbus air traffic control speech recognition 2018 challenge: Towards atc automatic transcription and call sign detection,”arXiv preprint arXiv:1810.12614, 2018

    work page arXiv 2018

  12. [12]

    Automatic call sign detection: Matching air surveillance data with air traffic spoken communications,

    J. Zuluaga-Gomez, K. Vesel `y, A. Blatt, P. Motlicek, D. Klakow, A. Tart, I. Sz ¨oke, A. Prasad, S. Sarfjoo, P. Kol ˇc´areket al., “Automatic call sign detection: Matching air surveillance data with air traffic spoken communications,” inProceedings, vol. 59, no. 1. MDPI, 2020, p. 14

    2020

  13. [13]

    Real- time integration of dynamic context information for improving automatic speech recognition,

    Y . Oualil, M. Schulder, H. Helmke, A. Schmidt, and D. Klakow, “Real- time integration of dynamic context information for improving automatic speech recognition,” inInterspeech 2015, 2015, pp. 2107–2111

    2015

  14. [14]

    Call- sign recognition and understanding for noisy air-traffic transcripts using surveillance information,

    A. Blatt, M. Kocour, K. Vesel `y, I. Sz ¨oke, and D. Klakow, “Call- sign recognition and understanding for noisy air-traffic transcripts using surveillance information,” inICASSP 2022-2022 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP). IEEE, 2022, pp. 8357–8361

    2022

  15. [15]

    Im- proving callsign recognition with air-surveillance data in air-traffic communication,

    I. Nigmatulina, R. Braun, J. Zuluaga-Gomez, and P. Motlicek, “Im- proving callsign recognition with air-surveillance data in air-traffic communication,”arXiv preprint arXiv:2108.12156, 2021

    work page arXiv 2021

  16. [16]

    Toward effective aircraft call sign detection using fuzzy string-matching between asr and ads-b data,

    M. S. Kasttet, A. Lyhyaoui, D. Zbakh, A. Aramja, and A. Kachkari, “Toward effective aircraft call sign detection using fuzzy string-matching between asr and ads-b data,”Aerospace, vol. 11, no. 1, p. 32, 2023

    2023

  17. [17]

    The atcosim corpus of non- prompted clean air traffic control speech,

    K. Hofbauer, S. Petrik, and H. Hering, “The atcosim corpus of non- prompted clean air traffic control speech,” inLREC, vol. 3, no. 5. Citeseer, 2008, p. 8. IEEE TRANSACTIONS ON INTELLIGENT VEHICLES 10

    2008

  18. [18]

    Atcspeech: A multilingual pilot-controller speech corpus from real air traffic control environment,

    B. Yang, X. Tan, Z. Chen, B. Wang, D. Li, Z. Yang, X. Wu, and Y . Lin, “Atcspeech: A multilingual pilot-controller speech corpus from real air traffic control environment,”arXiv preprint arXiv:1911.11365, 2019

    work page arXiv 1911

  19. [19]

    Atco2 corpus: A large-scale dataset for research on automatic speech recognition and natural language understanding of air traffic control communications,

    J. Zuluaga-Gomez, K. Vesel `y, I. Sz¨oke, A. Blatt, P. Motlicek, M. Kocour, M. Rigault, K. Choukri, A. Prasad, S. S. Sarfjooet al., “Atco2 corpus: A large-scale dataset for research on automatic speech recognition and natural language understanding of air traffic control communications,” arXiv preprint arXiv:2211.04054, 2022

    work page arXiv 2022

  20. [20]

    Atco2 corpus: A large-scale dataset for research on automatic speech recognition and natural language understanding of air traffic control communications,

    J. P. Z. Gomez, K. Vesel `y, I. Sz ¨oke, A. Blatt, P. Motlicek, M. Kocour, K. Choukri, I. Nigmatulina, C. Cevenini, A. Tartet al., “Atco2 corpus: A large-scale dataset for research on automatic speech recognition and natural language understanding of air traffic control communications,” Journal of Data-centric Machine Learning Research, 2024

    2024

  21. [21]

    M2ats: A real-world multimodal air traffic situation benchmark dataset and beyond,

    D. Guo, Y . Lin, X. You, Z. Yang, J. Zhou, B. Yang, J. Zhang, H. Shi, S. Hu, and Z. Zhang, “M2ats: A real-world multimodal air traffic situation benchmark dataset and beyond,” inProceedings of the 31st ACM International Conference on Multimedia. ACM, 2023, pp. 213– 221

    2023

  22. [22]

    Atsiu: A large-scale dataset for spoken instruction understanding in air traffic control,

    M. Zhang, Y . Yang, S. Qian, Q. Deng, J. Fang, and K. Cai, “Atsiu: A large-scale dataset for spoken instruction understanding in air traffic control,”Advanced Engineering Informatics, p. 103170, 2025

    2025

  23. [23]

    End-to-end asr-free keyword search from speech,

    K. Audhkhasi, A. Rosenberg, A. Sethy, B. Ramabhadran, and B. Kings- bury, “End-to-end asr-free keyword search from speech,”IEEE Journal of Selected Topics in Signal Processing, vol. 11, no. 8, pp. 1351–1359, 2017

    2017

  24. [24]

    Perfect match: Self- supervised embeddings for cross-modal retrieval,

    S.-W. Chung, J. S. Chung, and H.-G. Kang, “Perfect match: Self- supervised embeddings for cross-modal retrieval,”IEEE Journal of Selected Topics in Signal Processing, vol. 14, no. 3, pp. 568–576, 2020

    2020

  25. [25]

    Audio-text retrieval in context,

    S. Lou, X. Xu, M. Wu, and K. Yu, “Audio-text retrieval in context,” in ICASSP 2022-2022 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP). IEEE, 2022, pp. 4793–4797

    2022

  26. [26]

    Contrastive representation learning: A framework and review,

    P. H. Le-Khac, G. Healy, and A. F. Smeaton, “Contrastive representation learning: A framework and review,”IEEE Access, vol. 8, pp. 193 907– 193 934, 2020

    2020

  27. [27]

    Contrastive data and learning for natural language processing,

    R. Zhang, Y . Ji, Y . Zhang, and R. J. Passonneau, “Contrastive data and learning for natural language processing,” inProceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies: Tutorial Abstracts, 2022, pp. 39–47

    2022

  28. [28]

    A comprehensive survey on contrastive learning,

    H. Hu, X. Wang, Y . Zhang, Q. Chen, and Q. Guan, “A comprehensive survey on contrastive learning,”Neurocomputing, p. 128645, 2024

    2024

  29. [29]

    Low-resource automatic speech recognition domain adaptation: A case-study in aviation maintenance,

    N. A. M. Amin, “Low-resource automatic speech recognition domain adaptation: A case-study in aviation maintenance,” Ph.D. dissertation, Purdue University Graduate School, 2023

    2023

  30. [30]

    Air-ground datalink technologies (ldacs and current cpdlc) in flight centric atc: Analysis of a fast-time simulation and outlook on a complementary real-time simulation,

    M. Weber, A. P. Martins, C. Edinger, and C. S. Kluenker, “Air-ground datalink technologies (ldacs and current cpdlc) in flight centric atc: Analysis of a fast-time simulation and outlook on a complementary real-time simulation,” in2024 AIAA DATC/IEEE 43rd Digital Avionics Systems Conference (DASC). IEEE, 2024, pp. 1–8

    2024

  31. [31]

    Effects of language ontology on transatlantic automatic speech understanding research collaboration in the air traffic manage- ment domain,

    S. Chen, H. Helmke, R. M. Tarakan, O. Ohneiser, H. Kopald, and M. Kleinert, “Effects of language ontology on transatlantic automatic speech understanding research collaboration in the air traffic manage- ment domain,”Aerospace, vol. 10, no. 6, p. 526, 2023

    2023

  32. [32]

    Recent advances in end-to-end automatic speech recog- nition,

    J. Liet al., “Recent advances in end-to-end automatic speech recog- nition,”APSIPA Transactions on Signal and Information Processing, vol. 11, no. 1, 2022

    2022

  33. [33]

    Air traffic control communication (atcc) speech corpora and their use for asr and tts development,

    L. ˇSm´ıdl, J. ˇSvec, D. Tihelka, J. Matou ˇsek, J. Romportl, and P. Ircing, “Air traffic control communication (atcc) speech corpora and their use for asr and tts development,”Language Resources and Evaluation, vol. 53, no. 3, pp. 449–464, 2019

    2019

  34. [34]

    Attention is all you need,

    A. Vaswani, N. Shazeer, N. Parmar, J. Uszkoreit, L. Jones, A. N. Gomez, Ł. Kaiser, and I. Polosukhin, “Attention is all you need,” inAdvances in Neural Information Processing Systems, 2017, pp. 5998–6008

    2017

  35. [35]

    Bert: Pre-training of deep bidirectional transformers for language understanding,

    J. Devlin, M.-W. Chang, K. Lee, and K. Toutanova, “Bert: Pre-training of deep bidirectional transformers for language understanding,” inPro- ceedings of the 2019 conference of the North American chapter of the association for computational linguistics: human language technologies, volume 1 (long and short papers), 2019, pp. 4171–4186

    2019

  36. [36]

    DeepSeek-V3 Technical Report

    A. Liu, B. Feng, B. Xue, B. Wang, B. Wu, C. Lu, C. Zhao, C. Deng, C. Zhang, C. Ruanet al., “Deepseek-v3 technical report,”arXiv preprint arXiv:2412.19437, 2024

    work page internal anchor Pith review Pith/arXiv arXiv 2024

  37. [37]

    Clap learning audio concepts from natural language supervision,

    B. Elizalde, S. Deshmukh, M. Al Ismail, and H. Wang, “Clap learning audio concepts from natural language supervision,” inICASSP 2023- 2023 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP). IEEE, 2023, pp. 1–5

    2023

  38. [38]

    VSE++: Improving Visual-Semantic Embeddings with Hard Negatives

    F. Faghri, D. J. Fleet, J. R. Kiros, and S. V . Fidler, “Improv- ing visual-semantic embeddings with hard negatives,”arXiv preprint arXiv:1707.05612, pp. 7161–7170, 2017

    work page internal anchor Pith review Pith/arXiv arXiv 2017

  39. [39]

    Attentional feature fusion,

    Y . Dai, F. Gieseke, S. Oehmcke, Y . Wu, and K. Barnard, “Attentional feature fusion,” inProceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, 2021, pp. 3560–3569

    2021

  40. [40]

    Cross-modal contrastive learning for speech translation,

    R. Ye, M. Wang, and L. Li, “Cross-modal contrastive learning for speech translation,”arXiv preprint arXiv:2205.02444, 2022

    work page arXiv 2022