Pith. sign in

REVIEW 5 major objections 4 minor 37 references

WAVE-DETR Multi-Modal Visible and Acoustic Real-Life Drone Detector

T0 review · 5 major / 4 minor · reviewed 2026-08-04 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read Adding audio to a vision drone detector improves small-drone detection substantially, the paper claims.

desk verdict A useful new dataset and a plausible fusion architecture, but the headline audio-gain claim is confounded by unmatched training budgets. read the letter →

arxiv 2509.09859 v1 pith:UMLS4MHH submitted 2025-09-11 cs.CV cs.LG

Razvan Stefanescu , Ethan Oh , Ruben Vazquez , Chris Mesterharm , Constantin Serban , Ritu Chadha This is my paper
classification cs.CVcs.LG
keywords dronedetectionmulti-modalfusionDeformableDETRWav2Vec2acousticfeaturessmallobjectARDronedatasetgated
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Drone detection from a camera alone struggles when the drone is far away and occupies only a handful of pixels. This paper claims that adding the drone's sound, captured by a single microphone, helps a transformer-based detector find those small drones. WAVE-DETR fuses audio embeddings from Wav2Vec2 with the multi-scale visual feature maps of Deformable DETR before the transformer encoder, and compares four fusion layers. The best is a gated mechanism that learns a per-location weight between RGB and audio features; it improves small-drone mean average precision by 11.1% to 15.3% over the vision-only baseline on both in-distribution and out-of-distribution splits. The paper also contributes the ARDrone dataset, with more than 7,500 synchronized RGB-audio pairs gathered from a real drone in varied weather, backgrounds, and distances.

What carries the argument

The central mechanism is the gated fusion layer placed before the Deformable DETR encoder. Audio embeddings from Wav2Vec2 are reshaped and linearly interpolated to match each RGB feature map, concatenated with the visual features, passed through a linear layer, and then a sigmoid generates per-location weights that blend the RGB and audio feature maps. This lets the model learn when to trust the sound over the picture. The supporting resource is the ARDrone dataset: more than 7,500 synchronized pairs of one-second audio and center-aligned RGB frames, with bounding boxes, distance metadata, and deliberate variation in weather, background, and viewing angle.

What would settle it

Train the same gated fusion with audio segments randomly swapped across videos, so the audio does not correspond to the drone in the frame; if the small-drone mAP gain over RGB-only persists, the gain is not from acoustic content. Separately, manually verify a random sample of small-drone ground-truth boxes in ARDrone to check whether auto-label bias explains part of the improvement.

Watch

Extended reading notes

Core claim

WAVE-DETR extends Deformable DETR, a transformer detector with sparse attention over multi-scale feature maps, by injecting Wav2Vec2 audio embeddings before the encoder. Among four fusion designs, a gated mechanism—sigmoid-weighted blending of RGB and audio features—works best. On the new ARDrone dataset, gated fusion raises small-drone mAP from 0.486 to 0.540 in-distribution and from 0.443 to 0.511 out-of-distribution, relative gains of 11.1% and 15.3% across IoU 0.5-0.9. Medium and large drones improve too, with overall gains of 3.27% to 5.84%. The authors credit the drone's clear acoustic signature, which stays informative when the drone is only a few pixels wide.

Load-bearing premise

The ground-truth bounding boxes in the ARDrone dataset were generated automatically by a Deformable DETR detector trained on Drone-vs-Bird, and the paper does not report manual verification of those labels; if those auto boxes are biased for small drones, both training and evaluation could be biased and the audio gain may partly be fitting to label noise.

Editorial extensions

If this is right

  • Adding a gated audio channel improves small-drone detection by 11.1% to 15.3% relative mAP over a vision-only baseline at IoU 0.5-0.9.
  • The gated fusion is the best of the four tested fusions (linear, MLP, cross-attention) for all drone sizes on both in-distribution and out-of-distribution splits.
  • Initializing the RGB backbone from a locally fine-tuned detector, rather than COCO, yields consistently larger fusion gains.
  • The ARDrone dataset, with over 7,500 synchronized RGB-audio pairs and varied conditions, can support further multimodal drone detection research.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The stated gains could partly reflect the auto-generated labels: because the ground truth comes from a Deformable DETR trained on Drone-vs-Bird, small-drone boxes may be systematically biased; a manual-verification study on small boxes would clarify how much of the audio benefit is real.
  • A direct test of whether the gate actually uses acoustic content would be to train the same architecture with shuffled, misaligned audio; if the mAP gain persists, the improvement is not from the drone's sound.
  • The gated fusion design is not specific to drones; the same 'gate audio into the encoder' recipe could be tried for other small-object tasks, such as bird vs. drone discrimination or detecting distant vehicles from microphone-equipped cameras.
  • The single-microphone setup limits spatial localization; using a microphone array could turn the same acoustic features into direction cues and potentially improve the gate's decisions.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

5 major / 4 minor

Summary. The paper proposes WAVE-DETR, a multi-modal object detector that fuses wav2vec2 audio embeddings with Deformable DETR multi-scale RGB feature maps through four fusion layers: linear, MLP, gating, and cross-attention. The authors also introduce the ARDrone dataset of ~7,500 synchronized RGB/audio pairs. They report that gated fusion improves the mAP of a Deformable DETR baseline by 11.1% on in-distribution small drones and by 15.3% on out-of-distribution small drones, with smaller gains for medium and large drones. The central claim is that acoustic information materially improves small-drone detection.

Significance. If the claim holds, the paper offers a useful application of audio-visual fusion to a practical small-drone detection problem and contributes a new multimodal dataset. The four-way fusion-layer ablation is a reasonable exploration, and the idea that acoustic cues help exactly where RGB is weakest (small objects) is plausible. However, the experimental design currently does not isolate the contribution of audio. The Local-Init fusion models are warm-started from the RGB-only baseline checkpoint and receive 60 additional training epochs, with no RGB-only control retrained under the same conditions. Headline numbers mix initialization and dropout settings, and all results are single runs without error bars. The ground-truth boxes are auto-generated without reported manual verification. These issues prevent the results from supporting the paper's central causal claim in their present form.

major comments (5)
  1. [Section V.D, Tables I–II] The central comparison is confounded. Local-Init fusion models are initialized from the fine-tuned COCO-Init RGB checkpoint and trained for 60 additional epochs, whereas the baseline COCO-Init RGB is the checkpoint before this additional training. No RGB-only model is retrained from the same local initialization for the same 60 epochs. The consistent pattern in Tables I–IV—Local-Init fusion variants always beat COCO-Init variants, which in turn sometimes fall below baseline—is exactly what one would expect from extra training rather than from acoustic information. Add an RGB-only control initialized from the locally fine-tuned checkpoint and trained for 60 epochs with identical hyperparameters. Without this control, the 11.1% in-distribution small-drone gain (Table II, Local-Init Gated Fusion, dropout 0.1) is not causally attributable to audio.
  2. [Tables II and VI] The headline 11.1%–15.3% range mixes Local-Init (in-distribution) and COCO-Init (out-of-distribution) results and selects the best dropout rate post hoc. The out-of-distribution 15.3% figure is COCO-Init Gated Fusion at dropout 0.2 reaching 0.511 vs baseline 0.443, but the same architecture at dropout 0 scores 0.422, below baseline. This instability, combined with single runs and no error bars, means the improvement is not established as a stable property of the fusion. Report results over multiple random seeds with error bars, and pre-specify how dropout/initialization configurations are selected before stating headline numbers.
  3. [Section III] Ground-truth bounding boxes in the ARDrone dataset are produced automatically by a Deformable DETR detector trained on Drone-vs-Bird, and the paper does not report manual verification or any label-quality statistics. If the auto-labeler is systematically biased for small drones, both training and evaluation are biased, and the audio benefit could reflect fitting to label noise. Provide a quantitative evaluation of the auto-labeler against manually verified boxes (e.g., on a random sample stratified by drone size), and report the agreement.
  4. [Section V.D] The number of training epochs for the RGB-only baseline is not explicitly stated. The text says 'The architectures were trained using 60 epochs' after describing both unimodal and multimodal training, but the baseline section for COCO-Init RGB does not state its epoch count. This must be clarified to verify that COCO-Init fusion models are matched in training budget to the baseline. If the baseline was trained for 60 epochs, state that explicitly; otherwise, the comparison is not controlled.
  5. [Section V.E] The naming-convention paragraph is self-contradictory: it defines 'Local-Init Gated Fusion' as a Wav2Vec2+RGB gated fusion 'with the RGB weights initialized from COCO dataset.' This contradicts the tables, where Local-Init is used for locally initialized models. The mismatch makes it difficult to map descriptions to table entries and should be corrected.
minor comments (4)
  1. [Throughout] Numerous typographical errors and inconsistencies: 'binarry' (Section V.D), 'syncronization' (Section III), 'backround' (Appendix), 'Wav2Vec2 binarry classifier' (Section V.D), and inconsistent use of 'mAP' vs 'MAP'. A careful proofreading pass is needed.
  2. [Table V and VI] Several entries are missing for COCO-Init Cross Attention Fusion at dropout 0.2 and 0.3 (shown as '- -'). The paper should explain why these runs are absent (e.g., training divergence, resource limits), as missing data without explanation is a reproducibility concern.
  3. [Section V.F] The text says 'The Local-Init Linear Fusion method priorities the detection of small and medium drones'; the word 'priorities' should be 'prioritizes.' Also, some percentage gains are computed relative to baseline but the basis (relative vs absolute) is not always clear; state explicitly.
  4. [Appendix A] The audio classifier fine-tuning uses the same ARDrone dataset splits as the detection experiments. This is acceptable if the splits are strictly held out, but the paper should state that the test split is not used for fine-tuning the Wav2Vec2 backbone used in the fusion model.

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity found: empirical comparisons on held-out splits; no claim reduces to its inputs by construction.

full rationale

The paper makes no first-principles derivation claim; its contribution is an empirical comparison of Deformable DETR variants with and without Wav2Vec2 acoustic fusion on the ARDrone dataset. The mAP results are measured on held-out in-distribution and out-of-distribution test splits (Sec. V.B, V.F) against fixed annotations. No quantity is defined in terms of another predicted quantity: the fusion output is not used to define the baseline, and no fitted parameter is renamed as a prediction. The main caveats—pseudo-labels generated by a Deformable DETR trained on Drone-vs-Bird (Sec. III) and the absence of a matched RGB-only control with the same extra fine-tuning for Local-Init fusion (Sec. V.D)—are validity and attribution concerns, not circularity: the label generator was trained on a different dataset, and the extra-epochs confound does not make the compared metric equal to the training signal by construction. The only self-citation ([24], a Kalman/EnKF assimilation reference in related work) is not load-bearing. Therefore no circular step is identifiable under the quoted-evidence standard.

Assumptions & free parameters 2 free parameters · 4 assumptions · 0 invented entities

The central result depends on the quality of the auto-generated ARDrone labels, the synchronization assumption, and hyperparameters selected on validation/test results. No new physical or theoretical entities are introduced.

free parameters (2)
  • Dropout rate in fusion layers = Selected per fusion type; best small-drone results use 0.1 (in-distribution) and 0.2 (out-of-distribution) for gated fus
    Tables I-VIII report mAP across {0, 0.1, 0.2, 0.3}; the headline improvement range picks specific dropout rates rather than one fixed setting.
  • Backbone learning rate = 1e-5
    Section V-D: among tested {0, 1e-4, 1e-5, 1e-6}, 1e-5 gave the best mAP for all multimodal architectures; chosen after evaluation.
assumptions (4)
  • domain assumption The automated Deformable DETR labeler produces accurate bounding boxes for the ARDrone dataset
    Section III states annotation was automated with a Deformable DETR trained on Drone-vs-Bird; no manual verification is reported.
  • domain assumption The smartphone hardware maintains exact synchronization between RGB frames and audio segments
    Section III: 'the smartphone hardware already solved the time synchronization'; pairs are formed by selecting the frame at the midpoint of each 1s audio segment.
  • domain assumption Every annotated drone is audible in the corresponding 1-second audio segment
    The dataset protocol records drone flights with a built-in phone microphone; the paper assumes the acoustic signature is present for visible drones, and spectral analysis shows separability on test splits.
  • domain assumption Fine-tuned Wav2Vec2 embeddings learned on the ARDrone dataset transfer to the fusion detector
    The same dataset is used to fine-tune both the audio model and the fusion detector; no external audio benchmark is used.

how reviews work

0 comments
Cite this review

Pith. "Pith review of WAVE-DETR Multi-Modal Visible and Acoustic Real-Life Drone Detector." pith.science (2026). https://pith.science/paper/UMLS4MHH

@misc{pith2026250909859,
  author       = {Pith},
  title        = {Pith review of: WAVE-DETR Multi-Modal Visible and Acoustic Real-Life Drone Detector},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/UMLS4MHH}},
  note         = {Machine review of arXiv:2509.09859}
}
read the original abstract

We introduce a multi-modal WAVE-DETR drone detector combining visible RGB and acoustic signals for robust real-life UAV object detection. Our approach fuses visual and acoustic features in a unified object detector model relying on the Deformable DETR and Wav2Vec2 architectures, achieving strong performance under challenging environmental conditions. Our work leverage the existing Drone-vs-Bird dataset and the newly generated ARDrone dataset containing more than 7,500 synchronized images and audio segments. We show how the acoustic information is used to improve the performance of the Deformable DETR object detector on the real ARDrone dataset. We developed, trained and tested four different fusion configurations based on a gated mechanism, linear layer, MLP and cross attention. The Wav2Vec2 acoustic embeddings are fused with the multi resolution feature mappings of the Deformable DETR and enhance the object detection performance over all drones dimensions. The best performer is the gated fusion approach, which improves the mAP of the Deformable DETR object detector on our in-distribution and out-of-distribution ARDrone datasets by 11.1% to 15.3% for small drones across all IoU thresholds between 0.5 and 0.9. The mAP scores for medium and large drones are also enhanced, with overall gains across all drone sizes ranging from 3.27% to 5.84%.

Figures

Figures reproduced from arXiv: 2509.09859 by the authors.

Figure 2
Figure 2. Sample frames extracted from the Peraton dataset videos showing the [PITH_FULL_IMAGE:figures/full_fig_p003_2.png] view at source ↗
Figure 3
Figure 3. Distribution of drone sizes across the ground truth annotations in the [PITH_FULL_IMAGE:figures/full_fig_p003_3.png] view at source ↗
Figure 6
Figure 6. Self Supervised Wav2vec2+ Deformable DETR architecture fusing [PITH_FULL_IMAGE:figures/full_fig_p004_6.png] view at source ↗
Figures from the paper (6 more)
Figure 5
Figure 5. Figure 5: Self Supervised wav2vec2 architecture. Image is reproduced from [PITH_FULL_IMAGE:figures/full_fig_p004_5.png]
Figure 7
Figure 7. Figure 7: Gate Fusion combining linear and audio feature maps. [PITH_FULL_IMAGE:figures/full_fig_p005_7.png]
Figure 8
Figure 8. Figure 8: Average spectral analysis of in-distribution test dataset samples with [PITH_FULL_IMAGE:figures/full_fig_p010_8.png]
Figure 9
Figure 9. Figure 9: Average spectral analysis of in-distribution test dataset samples with [PITH_FULL_IMAGE:figures/full_fig_p010_9.png]
Figure 10
Figure 10. Figure 10: Average spectral analysis of out-of-distribution test dataset samples [PITH_FULL_IMAGE:figures/full_fig_p011_10.png]
Figure 11
Figure 11. Figure 11: Average spectral analysis of out-of-distribution test dataset samples [PITH_FULL_IMAGE:figures/full_fig_p011_11.png]

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

37 extracted references · 4 linked inside Pith

  1. [1]

    A review of yolo algorithm developments,

    P. Jiang, D. Ergu, F. Liu, Y . Cai, and B. Ma, “A review of yolo algorithm developments,”Procedia computer science, vol. 199, pp. 1066–1073, 2022

    2022

  2. [2]

    Faster r-cnn: Towards real-time object detection with region proposal networks,

    S. Ren, K. He, R. Girshick, and J. Sun, “Faster r-cnn: Towards real-time object detection with region proposal networks,”Advances in neural information processing systems, vol. 28, 2015

    2015

  3. [3]

    End-to-end object detection with transformers,

    N. Carion, F. Massa, G. Synnaeve, N. Usunier, A. Kirillov, and S. Zagoruyko, “End-to-end object detection with transformers,” in European conference on computer vision. Springer, 2020, pp. 213– 229

    2020

  4. [4]

    Deformable detr: Deformable transformers for end-to-end object detection,

    X. Zhu, W. Su, L. Lu, B. Li, X. Wang, and J. Dai, “Deformable detr: Deformable transformers for end-to-end object detection,”arXiv preprint arXiv:2010.04159, 2020

    arXiv 2010

  5. [5]

    nuscenes: A multimodal dataset for autonomous driving,

    H. Caesar, V . Bankiti, A. H. Lang, S. V ora, V . E. Liong, Q. Xu, A. Krishnan, Y . Pan, G. Baldan, and O. Beijbom, “nuscenes: A multimodal dataset for autonomous driving,” inProceedings of the IEEE/CVF conference on computer vision and pattern recognition, 2020, pp. 11 621–11 631

    2020

  6. [6]

    Seeing through fog without seeing fog: Deep multi- modal sensor fusion in unseen adverse weather,

    M. Bijelic, T. Gruber, F. Mannan, F. Kraus, W. Ritter, K. Dietmayer, and F. Heide, “Seeing through fog without seeing fog: Deep multi- modal sensor fusion in unseen adverse weather,” inProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2020, pp. 11 682–11 692

    2020

  7. [7]

    St-p3: End-to-end vision-based autonomous driving via spatial-temporal feature learning,

    S. Hu, L. Chen, P. Wu, H. Li, J. Yan, and D. Tao, “St-p3: End-to-end vision-based autonomous driving via spatial-temporal feature learning,” inEuropean Conference on Computer Vision. Springer, 2022, pp. 533– 549

    2022

  8. [8]

    Robust target recognition and tracking of self-driving cars with radar and camera information fusion under severe weather conditions,

    Z. Liu, Y . Cai, H. Wang, L. Chen, H. Gao, Y . Jia, and Y . Li, “Robust target recognition and tracking of self-driving cars with radar and camera information fusion under severe weather conditions,”IEEE Transactions on Intelligent Transportation Systems, vol. 23, no. 7, pp. 6640–6653, 2021

    2021

Show all 37 references
  1. [9]

    A survey of motion planning and control techniques for self-driving urban vehicles,

    B. Paden, M. ˇC´ap, S. Z. Yong, D. Yershov, and E. Frazzoli, “A survey of motion planning and control techniques for self-driving urban vehicles,” IEEE Transactions on intelligent vehicles, vol. 1, no. 1, pp. 33–55, 2016

    2016

  2. [10]

    A planning and control system for self-driving racing vehicles,

    D. Caporale, A. Fagiolini, L. Pallottino, A. Settimi, A. Biondo, F. Amerotti, F. Massa, S. De Caro, A. Corti, and L. Venturini, “A planning and control system for self-driving racing vehicles,” in2018 IEEE 4th international forum on research and technology for society and indu...

    2018

  3. [11]

    Futr3d: A unified sensor fusion framework for 3d detection,

    X. Chen, T. Zhang, Y . Wang, Y . Wang, and H. Zhao, “Futr3d: A unified sensor fusion framework for 3d detection,” inproceedings of the IEEE/CVF conference on computer vision and pattern recognition, 2023, pp. 172–181

    2023

  4. [12]

    Mt-detr: Robust end-to-end multimodal detection with confidence fusion,

    S.-Y . Chu and M.-S. Lee, “Mt-detr: Robust end-to-end multimodal detection with confidence fusion,” inProceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, 2023, pp. 5252– 5261

    2023

  5. [13]

    Drone detection and tracking with yolo and a rule-based method,

    P. Bhattacharya and P. Nowak, “Drone detection and tracking with yolo and a rule-based method,”arXiv preprint arXiv:2502.05292, 2025

    2025 arXiv

  6. [14]

    Yolo-drone: an optimized yolov8 network for tiny uav object detection,

    X. Zhai, Z. Huang, T. Li, H. Liu, and S. Wang, “Yolo-drone: an optimized yolov8 network for tiny uav object detection,”Electronics, vol. 12, no. 17, p. 3664, 2023

    2023

  7. [15]

    Dronet: Efficient convolutional neural network detector for real-time uav applications,

    C. Kyrkou, G. Plastiras, T. Theocharides, S. I. Venieris, and C.-S. Bouganis, “Dronet: Efficient convolutional neural network detector for real-time uav applications,” in2018 Design, Automation & Test in Europe Conference & Exhibition (DATE). IEEE, 2018, pp. 967–972

    2018

  8. [16]

    The unmanned aerial vehicle benchmark: Object detection and tracking,

    D. Du, Y . Qi, H. Yu, Y . Yang, K. Duan, G. Li, W. Zhang, Q. Huang, and Q. Tian, “The unmanned aerial vehicle benchmark: Object detection and tracking,” inProceedings of the European conference on computer vision (ECCV), 2018, pp. 370–386

    2018

  9. [17]

    Yolc: You only look clusters for tiny object detection in aerial images,

    C. Liu, G. Gao, Z. Huang, Z. Hu, Q. Liu, and Y . Wang, “Yolc: You only look clusters for tiny object detection in aerial images,”IEEE transactions on intelligent transportation systems, vol. 25, no. 10, pp. 13 863–13 875, 2024

    2024

  10. [18]

    The drone-vs-bird detection grand challenge at icassp 2023: A review of methods and results,

    A. Coluccia, A. Fascista, L. Sommer, A. Schumann, A. Dimou, and D. Zarpalas, “The drone-vs-bird detection grand challenge at icassp 2023: A review of methods and results,”IEEE Open Journal of Signal Processing, 2024

    2023

  11. [19]

    Deep residual learning for image recognition,

    K. He, X. Zhang, S. Ren, and J. Sun, “Deep residual learning for image recognition,” inProceedings of the IEEE conference on computer vision and pattern recognition, 2016, pp. 770–778

    2016

  12. [20]

    wav2vec 2.0: A framework for self-supervised learning of speech representations,

    A. Baevski, Y . Zhou, A. Mohamed, and M. Auli, “wav2vec 2.0: A framework for self-supervised learning of speech representations,” Advances in neural information processing systems, vol. 33, pp. 12 449– 12 460, 2020

    2020

  13. [21]

    Microsoft coco: Common objects in context,

    S. B. L. B. R. G. J. H. P. P. D. R. C. L. Z. P. D. Tsung-Yi Lin, Michael Maire, “Microsoft coco: Common objects in context,” pp. 740– 755, 2014

    2014

  14. [22]

    Real-time drone detection and tracking with visible, thermal and acoustic sensors,

    F. Svanstr ¨om, C. Englund, and F. Alonso-Fernandez, “Real-time drone detection and tracking with visible, thermal and acoustic sensors,” in 2020 25th International Conference on Pattern Recognition (ICPR). IEEE, 2021, pp. 7265–7272

    2020

  15. [23]

    A new approach to linear filtering and prediction problems,

    R. E. Kalman, “A new approach to linear filtering and prediction problems,”Transactions of the ASME–Journal of Basic Engineering, vol. 82, no. Series D, pp. 35–45, 1960

    1960

  16. [24]

    Enkf data-driven reduced order assimilation system,

    C. Liu, R. Fu, D. Xiao, R. Stefanescu, P. Sharma, C. Zhu, S. Sun, and C. Wang, “Enkf data-driven reduced order assimilation system,” Engineering Analysis with Boundary Elements, vol. 139, pp. 46–55, 2022

    2022

  17. [25]

    Mfcc in audio signal processing for voice disorder: a review,

    M. S. Sidhu, N. A. A. Latib, and K. K. Sidhu, “Mfcc in audio signal processing for voice disorder: a review,”Multimedia Tools and Applications, vol. 84, no. 10, pp. 8015–8035, 2025

    2025

  18. [26]

    Drone detection and tracking system based on fused acoustical and optical approaches,

    S. Ding, X. Guo, T. Peng, X. Huang, and X. Hong, “Drone detection and tracking system based on fused acoustical and optical approaches,” Advanced Intelligent Systems, vol. 5, no. 10, p. 2300251, 2023

    2023

  19. [27]

    Mdetr-modulated detection for end-to-end multi-modal understanding,

    A. Kamath, M. Singh, Y . LeCun, G. Synnaeve, I. Misra, and N. Carion, “Mdetr-modulated detection for end-to-end multi-modal understanding,” inProceedings of the IEEE/CVF international conference on computer vision, 2021, pp. 1780–1790

    2021

  20. [28]

    Mmaud: A comprehensive multi-modal anti-uav dataset for modern miniature drone threats,

    S. Yuan, Y . Yang, T. H. Nguyen, T.-M. Nguyen, J. Yang, F. Liu, J. Li, H. Wang, and L. Xie, “Mmaud: A comprehensive multi-modal anti-uav dataset for modern miniature drone threats,” in2024 IEEE International Conference on Robotics and Automation (ICRA). IEEE, 2024, pp. 2745–2751

    2024

  21. [29]

    Fast r-cnn,

    R. Girshick, “Fast r-cnn,” inProceedings of the IEEE international conference on computer vision, 2015, pp. 1440–1448

    2015

  22. [30]

    Attention is all you need,

    A. Vaswani, N. Shazeer, N. Parmar, J. Uszkoreit, L. Jones, A. N. Gomez, Ł. Kaiser, and I. Polosukhin, “Attention is all you need,”Advances in neural information processing systems, vol. 30, 2017

    2017

  23. [31]

    Feature pyramid networks for object detection,

    T.-Y . Lin, P. Doll´ar, R. Girshick, K. He, B. Hariharan, and S. Belongie, “Feature pyramid networks for object detection,” inProceedings of the IEEE conference on computer vision and pattern recognition, 2017, pp. 2117–2125

    2017

  24. [32]

    wav2vec 2.0: A framework for self-supervised learning of speech representations,

    A. Baevski, H. Zhou, A. Mohamed, and M. Auli, “wav2vec 2.0: A framework for self-supervised learning of speech representations,”

  25. [33]

    Mixed precision training,

    P. Micikevicius, S. Narang, J. Albenet al., “Mixed precision training,”

  26. [36]

    We perform this analysis over the testing datasets

    Spectral analysis:We performed a spectral analysis on the in-distribution and out-of-distribution datasets in order to analyze the difference between background and drone audio samples to estimate how well an audio classifier may perform when tasked with differentiating betwee...

  27. [37]

    First we had to re-sample the audio segments to 16kHz to match the sampling rate of the audio samples used for training the foundation model

    Fine-tuning setup:We use a pre-trained Wav2Vec2 model as our foundation model for the downstream drone clas- sification task. First we had to re-sample the audio segments to 16kHz to match the sampling rate of the audio samples used for training the foundation model. Next, we ...

  28. [2018]

    Available: https://arxiv.org/abs/1710.03740 9 APPENDIX A

    [Online]. Available: https://arxiv.org/abs/1710.03740 9 APPENDIX A. Wav2Vec2 Drone Classification Model We leverage Wav2Vec2’s low data requirements for the fine- tuning stage to fine-tune a pre-trained Wav2Vec2 model on approximately 80 minutes of audio samples and achieve hi...

    arXiv

  29. [2020]

    Available: https://arxiv.org/abs/2006.11477

    [Online]. Available: https://arxiv.org/abs/2006.11477

    2006 arXiv

Pith tools

Reviewed August 4, 2026 · model on record in the stance chip above.