{"id":"349782ea-33f5-46f1-be58-cdf9e9442852","arxiv_id":"2509.09859","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":2,"one_line_summary":"Fusing Wav2Vec2 audio features into a Deformable DETR detector improves small-drone detection mAP by 11-15% on a new synchronized audio-RGB dataset.","lead":"This paper tests whether sound helps a drone-spotting camera: it fuses audio features from a speech model into a visual drone detector and reports 11-15% better small-drone detection on a new synchronized audio-video dataset. The work adds a dataset and compares four fusion designs.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Audio-gain claim is confounded: Local-Init fusion models start from the already fine-tuned RGB checkpoint and receive 60 additional training epochs, with no RGB-only retrained control.","rationale":"The reader's verdict is CONDITIONAL with moderate confidence, which is appropriate, and their rationale explicitly lists a 'confounded baseline (Local-Init fusion vs. COCO-Init RGB without an equivalent RGB-only additional-training control)' as one of the experimental-protocol issues. However, the reader's formal weakest_assumption is the auto-generated labels, not this baseline confound. My stress-test identifies the unmatched baseline as the single most load-bearing concern because it directly undermines the causal attribution of the headline improvement to audio: the Local-Init fusion models are essentially the baseline checkpoint plus 60 more epochs of training, and no RGB-only control receives that extra training. The pattern of results—Local-Init variants outperforming COCO-Init variants for every fusion type, while COCO-Init fusion gains are small or negative—is consistent with a pure training/initialization effect. The proposed concrete test (an RGB-only Local-Init model trained for the same 60 epochs) would settle this. Since the reader already reached CONDITIONAL based partly on this issue, my read does not change the verdict; it sharpens the exact condition that must be met. The auto-label concern is legitimate but would affect both modalities symmetrically unless the fusion specifically overfits label noise, and it is less likely to explain the full magnitude of the reported gains. I therefore recommend keeping the CONDITIONAL verdict with the additional-training control as an explicit acceptance condition.","tokens_in":13991,"tokens_out":4913,"duration_ms":52530,"concrete_test":"Train an RGB-only Deformable DETR initialized from the same locally fine-tuned checkpoint and trained for the same 60 epochs, with the same optimizer, backbone learning rate (1e-5), cosine annealing, batch size, and mixed precision. Evaluate small-drone mAP at IoU 0.5:0.95 on both in-distribution (compare to Local-Init Gated Fusion: 0.540 with dropout 0.1) and out-of-distribution (compare to COCO-Init Gated Fusion: 0.511 with dropout 0.2) test sets. If the RGB-only control meets or exceeds these values, the audio-gain claim is not supported.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim that audio fusion improves small-drone detection is not supported by a matched comparison. The baseline, COCO-Init RGB, is a Deformable DETR fine-tuned from COCO on ARDrone. Local-Init Gated Fusion initializes its RGB backbone from this exact fine-tuned local model and then trains for another 60 epochs with the fusion layer (Section V.D/E). Thus the fusion model has the advantage of additional training epochs and a better starting point, while the baseline is the pre-further-training checkpoint. No RGB-only control receives the same extra training from the same local init. If the reported gains were due to continued fine-tuning rather than audio, they would appear exactly as observed: Local-Init fusion variants outperform their COCO-Init counterparts consistently across all fusion types (Tables I–IV), and COCO-Init fusion models show much smaller, sometimes negative, gains. The 11.1% in-distribution small-drone gain (Table II: 0.540 vs 0.486) and the 15.3% out-of-distribution gain (Table VI: 0.511 vs 0.443) could therefore be artifacts of training length/init rather than acoustic information. A proper control—an RGB-only Deformable DETR initialized from the same local checkpoint and trained for the same 60 epochs with identical hyperparameters—is absent; without it, the causal attribution of the improvement to audio is unestablished. The auto-label quality issue raised in the reader's weakest assumption is real but secondary: even with perfect labels, the unmatched baseline confounds the comparison.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes WAVE-DETR, a multi-modal object detector that fuses wav2vec2 audio embeddings with Deformable DETR multi-scale RGB feature maps through four fusion layers: linear, MLP, gating, and cross-attention. The authors also introduce the ARDrone dataset of ~7,500 synchronized RGB/audio pairs. They report that gated fusion improves the mAP of a Deformable DETR baseline by 11.1% on in-distribution small drones and by 15.3% on out-of-distribution small drones, with smaller gains for medium and large drones. The central claim is that acoustic information materially improves small-drone detection.","tokens_in":14320,"tokens_out":5749,"duration_ms":65337,"significance":"If the claim holds, the paper offers a useful application of audio-visual fusion to a practical small-drone detection problem and contributes a new multimodal dataset. The four-way fusion-layer ablation is a reasonable exploration, and the idea that acoustic cues help exactly where RGB is weakest (small objects) is plausible. However, the experimental design currently does not isolate the contribution of audio. The Local-Init fusion models are warm-started from the RGB-only baseline checkpoint and receive 60 additional training epochs, with no RGB-only control retrained under the same conditions. Headline numbers mix initialization and dropout settings, and all results are single runs without error bars. The ground-truth boxes are auto-generated without reported manual verification. These issues prevent the results from supporting the paper's central causal claim in their present form.","major_comments":[{"comment":"The central comparison is confounded. Local-Init fusion models are initialized from the fine-tuned COCO-Init RGB checkpoint and trained for 60 additional epochs, whereas the baseline COCO-Init RGB is the checkpoint before this additional training. No RGB-only model is retrained from the same local initialization for the same 60 epochs. The consistent pattern in Tables I–IV—Local-Init fusion variants always beat COCO-Init variants, which in turn sometimes fall below baseline—is exactly what one would expect from extra training rather than from acoustic information. Add an RGB-only control initialized from the locally fine-tuned checkpoint and trained for 60 epochs with identical hyperparameters. Without this control, the 11.1% in-distribution small-drone gain (Table II, Local-Init Gated Fusion, dropout 0.1) is not causally attributable to audio.","section":"Section V.D, Tables I–II"},{"comment":"The headline 11.1%–15.3% range mixes Local-Init (in-distribution) and COCO-Init (out-of-distribution) results and selects the best dropout rate post hoc. The out-of-distribution 15.3% figure is COCO-Init Gated Fusion at dropout 0.2 reaching 0.511 vs baseline 0.443, but the same architecture at dropout 0 scores 0.422, below baseline. This instability, combined with single runs and no error bars, means the improvement is not established as a stable property of the fusion. Report results over multiple random seeds with error bars, and pre-specify how dropout/initialization configurations are selected before stating headline numbers.","section":"Tables II and VI"},{"comment":"Ground-truth bounding boxes in the ARDrone dataset are produced automatically by a Deformable DETR detector trained on Drone-vs-Bird, and the paper does not report manual verification or any label-quality statistics. If the auto-labeler is systematically biased for small drones, both training and evaluation are biased, and the audio benefit could reflect fitting to label noise. Provide a quantitative evaluation of the auto-labeler against manually verified boxes (e.g., on a random sample stratified by drone size), and report the agreement.","section":"Section III"},{"comment":"The number of training epochs for the RGB-only baseline is not explicitly stated. The text says 'The architectures were trained using 60 epochs' after describing both unimodal and multimodal training, but the baseline section for COCO-Init RGB does not state its epoch count. This must be clarified to verify that COCO-Init fusion models are matched in training budget to the baseline. If the baseline was trained for 60 epochs, state that explicitly; otherwise, the comparison is not controlled.","section":"Section V.D"},{"comment":"The naming-convention paragraph is self-contradictory: it defines 'Local-Init Gated Fusion' as a Wav2Vec2+RGB gated fusion 'with the RGB weights initialized from COCO dataset.' This contradicts the tables, where Local-Init is used for locally initialized models. The mismatch makes it difficult to map descriptions to table entries and should be corrected.","section":"Section V.E"}],"minor_comments":[{"comment":"Numerous typographical errors and inconsistencies: 'binarry' (Section V.D), 'syncronization' (Section III), 'backround' (Appendix), 'Wav2Vec2 binarry classifier' (Section V.D), and inconsistent use of 'mAP' vs 'MAP'. A careful proofreading pass is needed.","section":"Throughout"},{"comment":"Several entries are missing for COCO-Init Cross Attention Fusion at dropout 0.2 and 0.3 (shown as '- -'). The paper should explain why these runs are absent (e.g., training divergence, resource limits), as missing data without explanation is a reproducibility concern.","section":"Table V and VI"},{"comment":"The text says 'The Local-Init Linear Fusion method priorities the detection of small and medium drones'; the word 'priorities' should be 'prioritizes.' Also, some percentage gains are computed relative to baseline but the basis (relative vs absolute) is not always clear; state explicitly.","section":"Section V.F"},{"comment":"The audio classifier fine-tuning uses the same ARDrone dataset splits as the detection experiments. This is acceptable if the splits are strictly held out, but the paper should state that the test split is not used for fine-tuning the Wav2Vec2 backbone used in the fusion model.","section":"Appendix A"}],"recommendation":"major_revision","confidential_remarks":"The central claim is plausible but not currently established. The confounded Local-Init comparison is the main obstacle; however, the COCO-Init out-of-distribution result provides some initial evidence that audio can help, so I do not recommend outright rejection. I would ask for a matched RGB-only retrained control, multi-seed results, and label-quality verification in revision. The naming inconsistency in Section V.E is minor but should be fixed. The paper's contribution of the ARDrone dataset is real, but the dataset's auto-labeling without manual verification needs to be addressed before it can serve as a sound benchmark."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Best read as a dataset-plus-architecture paper, not a clean causal demonstration. The ARDrone dataset (7,500+ synchronized RGB/audio pairs, varied backgrounds, distances, weather) is genuinely useful, and the idea of inserting Wav2Vec2 embeddings into Deformable DETR before the encoder is a reasonable combination that I haven't seen in prior work. The paper runs a broad ablation over four fusion layers, dropout rates, and two initializations, and the spectral analysis in the appendix is a nice sanity check that the audio actually carries a drone signature.\n\nThe soft spots are real and one is load-bearing. The abstract's claim that gated fusion improves mAP by 11.1–15.3% over the RGB Deformable DETR compares against a baseline that was fine-tuned from COCO and then stopped, while the Local-Init fusion models start from that already fine-tuned checkpoint and train for another 60 epochs. No RGB-only model receives the same extra training. The tables show the pattern you'd expect from continued fine-tuning: Local-Init beats COCO-Init across fusion types, while COCO-Init fusion gains are modest or negative. So the causal attribution to audio is not established. A single matched control—RGB-only Deformable DETR initialized from the local checkpoint and trained for 60 epochs—would fix this, and it's conspicuous by its absence.\n\nThe auto-generated labels are a secondary concern. They used a Deformable DETR trained on Drone-vs-Bird to produce ground truth, with no manual verification or label-quality statistics reported. If those labels are biased for small drones, both training and evaluation inherit the bias. I'd want that addressed before trusting the absolute numbers.\n\nMinor: all results are single runs with no error bars, the headline picks the best dropout configuration post hoc, and the naming in Section V.E has an apparent typo ('Local-Init' for COCO-init) that should be cleaned up.\n\nWho should read it: researchers working on drone detection or audio-visual fusion will find the dataset useful and the architecture worth building on. It deserves a serious referee, but the referee should ask for the missing control and the label-quality analysis before accepting the causal claim.","headline":"A useful new dataset and a plausible fusion architecture, but the headline audio-gain claim is confounded by unmatched training budgets.","tokens_in":14818,"tokens_out":1760,"would_cite":false,"duration_ms":19445,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Adding audio to a vision drone detector improves small-drone detection substantially, the paper claims.","keywords":["drone detection","multi-modal fusion","Deformable DETR","Wav2Vec2","acoustic features","small object detection","ARDrone dataset","gated fusion"],"falsifier":"Train the same gated fusion with audio segments randomly swapped across videos, so the audio does not correspond to the drone in the frame; if the small-drone mAP gain over RGB-only persists, the gain is not from acoustic content. Separately, manually verify a random sample of small-drone ground-truth boxes in ARDrone to check whether auto-label bias explains part of the improvement.","tokens_in":13891,"feed_emoji":"🛸","tokens_out":5862,"duration_ms":53344,"temperature":0.7,"pith_summary":"Drone detection from a camera alone struggles when the drone is far away and occupies only a handful of pixels. This paper claims that adding the drone's sound, captured by a single microphone, helps a transformer-based detector find those small drones. WAVE-DETR fuses audio embeddings from Wav2Vec2 with the multi-scale visual feature maps of Deformable DETR before the transformer encoder, and compares four fusion layers. The best is a gated mechanism that learns a per-location weight between RGB and audio features; it improves small-drone mean average precision by 11.1% to 15.3% over the vision-only baseline on both in-distribution and out-of-distribution splits. The paper also contributes the ARDrone dataset, with more than 7,500 synchronized RGB-audio pairs gathered from a real drone in varied weather, backgrounds, and distances.","feed_headline":"Sound lifts small-drone detection up to 15%","feed_subtitle":"Fusing audio with RGB in a transformer detector improves small-drone accuracy over vision alone.","key_machinery":"The central mechanism is the gated fusion layer placed before the Deformable DETR encoder. Audio embeddings from Wav2Vec2 are reshaped and linearly interpolated to match each RGB feature map, concatenated with the visual features, passed through a linear layer, and then a sigmoid generates per-location weights that blend the RGB and audio feature maps. This lets the model learn when to trust the sound over the picture. The supporting resource is the ARDrone dataset: more than 7,500 synchronized pairs of one-second audio and center-aligned RGB frames, with bounding boxes, distance metadata, and deliberate variation in weather, background, and viewing angle.","core_discovery":"WAVE-DETR extends Deformable DETR, a transformer detector with sparse attention over multi-scale feature maps, by injecting Wav2Vec2 audio embeddings before the encoder. Among four fusion designs, a gated mechanism—sigmoid-weighted blending of RGB and audio features—works best. On the new ARDrone dataset, gated fusion raises small-drone mAP from 0.486 to 0.540 in-distribution and from 0.443 to 0.511 out-of-distribution, relative gains of 11.1% and 15.3% across IoU 0.5-0.9. Medium and large drones improve too, with overall gains of 3.27% to 5.84%. The authors credit the drone's clear acoustic signature, which stays informative when the drone is only a few pixels wide.","pith_inferences":["The stated gains could partly reflect the auto-generated labels: because the ground truth comes from a Deformable DETR trained on Drone-vs-Bird, small-drone boxes may be systematically biased; a manual-verification study on small boxes would clarify how much of the audio benefit is real.","A direct test of whether the gate actually uses acoustic content would be to train the same architecture with shuffled, misaligned audio; if the mAP gain persists, the improvement is not from the drone's sound.","The gated fusion design is not specific to drones; the same 'gate audio into the encoder' recipe could be tried for other small-object tasks, such as bird vs. drone discrimination or detecting distant vehicles from microphone-equipped cameras.","The single-microphone setup limits spatial localization; using a microphone array could turn the same acoustic features into direction cues and potentially improve the gate's decisions."],"forward_implications":["Adding a gated audio channel improves small-drone detection by 11.1% to 15.3% relative mAP over a vision-only baseline at IoU 0.5-0.9.","The gated fusion is the best of the four tested fusions (linear, MLP, cross-attention) for all drone sizes on both in-distribution and out-of-distribution splits.","Initializing the RGB backbone from a locally fine-tuned detector, rather than COCO, yields consistently larger fusion gains.","The ARDrone dataset, with over 7,500 synchronized RGB-audio pairs and varied conditions, can support further multimodal drone detection research."],"fun_headline_variants":["Audio fusion lifts small-drone detection by 11-15%","Sound plus vision boosts small-drone accuracy up to 15%","Gated audio features improve small-drone mAP by 15%","Acoustic cues enhance detection of tiny drones by 15%","WAVE-DETR: audio strengthens small-drone detection 15%"],"cache_read_input_tokens":2304,"weakest_assumption_plain":"The ground-truth bounding boxes in the ARDrone dataset were generated automatically by a Deformable DETR detector trained on Drone-vs-Bird, and the paper does not report manual verification of those labels; if those auto boxes are biased for small drones, both training and evaluation could be biased and the audio gain may partly be fitting to label noise.","fun_headline_variants_meta":{"raw":{"variants":["Audio fusion lifts small-drone detection by 11-15%","Sound plus vision boosts small-drone accuracy up to 15%","Gated audio features improve small-drone mAP by 15%","Acoustic cues enhance detection of tiny drones by 15%","WAVE-DETR: audio strengthens small-drone detection 15%"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000283,"raw_usage":{"total_tokens":1550,"prompt_tokens":827,"completion_tokens":723,"prompt_tokens_details":{"cached_tokens":256},"prompt_cache_hit_tokens":256,"prompt_cache_miss_tokens":571,"completion_tokens_details":{"reasoning_tokens":630}},"tokens_in":571,"tokens_out":723,"duration_ms":8016,"temperature":1.0,"reasoning_tokens":630,"cache_read_input_tokens":256,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-04T18:33:28.601008+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Train the same gated fusion with audio segments randomly swapped across videos, so the audio does not correspond to the drone in the frame; if the small-drone mAP gain over RGB-only persists, the gain is not from acoustic content. Separately, manually verify a random sample of small-drone ground-truth boxes in ARDrone to check whether auto-label bias explains part of the improvement.","supporting_citations":[],"review_version":1}