{"id":"df95088f-cf3f-46c6-b311-534676f24e05","arxiv_id":"2412.16928","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":3,"one_line_summary":"AV-DTEC fuses audio and visual features using a state-space model and an adaptive teacher-student mechanism to estimate drone trajectories and classify drone types, achieving state-of-the-art results on the MMAUD dataset without manual annotation.","lead":"This paper introduces AV-DTEC, a system that combines audio and video to detect, track, and classify small drones, trained without human labels. It reports state-of-the-art accuracy on a public multi-sensor drone dataset while using a lightweight network.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Table IV shows the proposed FEM+AAM full model (0.67 m APE) is worse than a simple concatenation baseline (0.64 m APE), undermining the state-of-the-art trajectory claim.","rationale":"The reader identified pseudo-label quality as the weakest assumption. While that is a legitimate concern, the paper's own Table IV reveals a more direct problem: the proposed full model (FEM+AAM) has higher mean APE (0.67 m) than a simple concatenation of the same backbones (0.64 m). This directly contradicts the central claim that AV-DTEC achieves state-of-the-art trajectory estimation. The pseudo-label concern, by contrast, is about training data quality and could be addressed by more detailed noise analysis; but even with perfect labels, the fusion mechanism's benefit is not demonstrated by the ablation. Hence I disagree with the reader's choice of the single most load-bearing concern. The verdict should remain CONDITIONAL (the reader's original vote is appropriate) because the issue is an internal inconsistency that could be resolved by additional experiments or by re-framing the claims, but the paper as written does not support its headline trajectory result. I recommend no change to the verdict label, but the condition should now include a demand to reconcile Table IV with the state-of-the-art claim, not just to analyze pseudo-labels.","tokens_in":9222,"tokens_out":5260,"duration_ms":47348,"concrete_test":"Re-run the Table IV ablation for the concatenation baseline (AMamba+Vim+Concate) and the full model (AMamba+Vim+FEM+AAM) using the same data split, with at least 5 random seeds, and report mean ± std APE for light, dark, and overall. If the concatenation baseline remains at or below the full model's APE (or the difference is not statistically significant), the paper must retract or qualify the claim that FEM+AAM improves trajectory estimation.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim of state-of-the-art trajectory estimation (mean APE 0.67 m, Table I) is contradicted by the paper's own ablation study. In Table IV, the row with AMamba + Vim + Concate (no FEM, no AAM) achieves a mean APE of 0.64 m and a dark APE of 0.75 m, while the full proposed model with FEM and AAM achieves 0.67 m mean APE and 0.75 m dark APE. Thus the proposed feature enhancement module and adaptive adjustment mechanism do not improve trajectory estimation over a plain concatenation of the same audio and visual features; on light conditions the full model is actually worse (0.58 m vs 0.52 m). The paper's discussion of Table IV claims AAM 'solves the problem of alignment' and that 'the performance of AV-DTEC is less degraded,' but the data show the full model is not better than a simple baseline on the primary metric. Because the paper explicitly labels AV-DTEC as state-of-the-art in both APE and Acc, this internal inconsistency is load-bearing: even if the pseudo-labels are perfect, the specific fusion contribution does not support the headline result. The only advantage of the full model over concatenation is classification accuracy (99.3% vs 98.9%), which is not the paper's principal focus and may be within noise. Without error bars or significance testing, the claimed superiority of the proposed fusion mechanism is unsupported by the paper's own evidence.","agreement_with_reader":"disagree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes AV-DTEC, a lightweight audio-visual fusion system for drone trajectory estimation and classification on the MMAUD dataset. The method uses a Mamba/SSM-based audio feature extractor (AMamba with temporal and spectral branches), a Vision Mamba branch, a feature enhancement module based on residual cross-attention, and an adaptive adjustment mechanism driven by a Faster R-CNN teacher. Training uses pseudo-labels generated by an unsupervised LiDAR-based clustering pipeline, avoiding manual annotation. The paper reports mean APE of 0.67 m and classification accuracy of 99.3% on the test set, together with low computational cost (1.98 GFLOPs, 10.16M parameters), and open-sources the code.","tokens_in":9493,"tokens_out":3281,"duration_ms":31832,"significance":"If the reported results are reliable, the paper would make a useful contribution to practical anti-UAV systems: it demonstrates that a relatively small audio-visual model can estimate drone trajectories and classify drone types on real-world multi-modal data without manual annotations, and the choice of audio as the primary modality is well motivated by robustness to lighting changes. The open-source release of code and trained models, the use of a real-world dataset, and the lightweight architecture are clear strengths that increase reproducibility and practical value. However, the central claim of state-of-the-art trajectory estimation is undermined by an internal inconsistency in the ablation study, and the lack of multiple runs or error bars leaves the main quantitative claims unverified.","major_comments":[{"comment":"The paper's headline claim that AV-DTEC achieves state-of-the-art trajectory estimation (mean APE 0.67 m in Table I) is contradicted by the paper's own ablation study. In Table IV, the row 'AMamba + Vim + Concate' (no FEM, no AAM) achieves mean APE 0.64 m and light-condition APE 0.52 m, while the full model with FEM and AAM achieves mean APE 0.67 m and light-condition APE 0.58 m. Thus the proposed feature enhancement module and adaptive adjustment mechanism do not improve trajectory estimation over a plain concatenation of the same audio and visual features; on the light condition the full model is worse. The discussion in Section IV-D claims that AAM 'solves the problem of alignment' and that performance is 'less degraded,' but the data do not show that the full model is better than the concatenation baseline on the primary metric. This is load-bearing because the paper explicitly claims state-of-the-art APE. The authors should either provide a statistically significant demonstration that FEM/AAM improve trajectory estimation, or revise the claim that the full model is state-of-the-art in trajectory estimation.","section":"Table IV and Section IV-D"},{"comment":"The pseudo-label accuracy is the load-bearing premise for the sub-meter trajectory results, but the paper provides only the statement that LiDAR pseudo-labels have 'up to 1-meter accuracy.' Since the reported mean APE is 0.67 m, the model is trained against labels that may be noisier than the errors it reports; without a quantitative analysis of the pseudo-label error distribution (bias, variance, and per-axis errors) and a sensitivity analysis showing that the reported APE is not an artifact of label noise, the trajectory numbers are not fully supported. Figure 6 shows a difference distribution, but no summary statistics or discussion of how this noise affects the L1 training loss in Eq. (5). This concern is distinct from the Table IV inconsistency and needs to be addressed before the trajectory claim can be accepted.","section":"Section III-D and Fig. 6"},{"comment":"The adaptive adjustment mechanism (AAM) is not specified well enough to be reproduced from the text. Equation (4) states the fusion computation, and Eq. (7) refers to a teacher-student loss Lt-s, but the paper does not define how the binary cross-entropy existence loss and the position loss are combined in Lt-s, how the Faster R-CNN teacher's outputs are converted into the adjustment factor alpha, or how the student model's predictions are used to 'reversely weaken' visual features during inference. Since AAM is one of the two main claimed contributions and the paper argues it is essential for alignment, the absence of these details makes the method under-specified. The code release helps, but the manuscript itself should contain the full formulation.","section":"Section III-B2 and Eqs. (4), (7)"},{"comment":"All results are based on a single 7:3 train/test split with no multiple runs, error bars, or significance tests. Differences such as 99.3% vs. 98.9% accuracy, 0.67 m vs. 0.64 m APE, and 99.7% vs. 99.6% light-condition accuracy are small and could easily be within run-to-run variation. The paper should report results over multiple seeds or provide confidence intervals, especially for the ablation comparisons in Tables III and IV that support the claimed benefits of each module.","section":"All experimental tables"}],"minor_comments":[{"comment":"The system name is inconsistently written: AV-DTEC appears in the title and abstract, but 'AV-DETC' appears in the GitHub URL and in Section IV-D. Please use one consistent spelling.","section":"Title, Abstract, GitHub URL"},{"comment":"Typo: 'Dateset' should be 'Dataset'.","section":"Section IV-A"},{"comment":"The text says 'teacher-teacher model' in the description of the adaptive adjustment mechanism; this should be 'teacher-student model'.","section":"Section III"},{"comment":"The caption says 'Overline: the mean value of day and night,' but it is unclear whether 'Overline' refers to the last two columns or to an overline symbol in the table. Please clarify the notation for the mean columns.","section":"Table I caption"},{"comment":"The pseudo-label generation relies on 'empirically filtering outliers based on point size, shape, and spatial-temporal continuity,' but the exact thresholds are not given, which makes the label generation pipeline hard to reproduce independently.","section":"Section III-D"},{"comment":"The related work section mentions several prior methods but does not clearly state what is missing in them relative to the proposed work, especially with respect to 3D trajectory estimation with audio-visual fusion; a brief comparative table in the text would help.","section":"Section II-A"}],"recommendation":"major_revision","confidential_remarks":"The paper's core idea is interesting and the open-source code is a plus, but the internal inconsistency between Table I and Table IV is serious: the headline trajectory claim is not supported by the paper's own ablation. This is not a case of minor over-claiming; the proposed fusion modules appear to hurt the primary metric. The 'self-supervised' terminology is also somewhat over-stated, since the training uses LiDAR pseudo-labels and a supervised Faster R-CNN teacher; this may be acceptable as 'without manual annotation' but should be described accurately. I would not reject the paper because the classification accuracy and efficiency results are still potentially valuable, and the trajectory issue could be resolved by reframing the claims and adding proper statistical evaluation. However, the authors must address the ablation inconsistency and the pseudo-label noise analysis in a major revision."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"The paper is a practical engineering contribution to anti-UAV systems: it combines Mamba-based audio feature extraction with vision and trains on LiDAR-generated pseudo-labels, achieving strong results on the MMAUD dataset with a lightweight model. The code is open, and the authors compare against several baselines and report compute cost. That part is genuine.\n\nThe soft spot is load-bearing. Table IV shows the concatenation baseline (AMamba + Vim + Concate) achieves 0.64 m mean APE and 0.52 m light APE, while the full model with FEM and AAM gets 0.67 m and 0.58 m. Dark APE is identical at 0.75 m. So the proposed feature enhancement and adaptive adjustment actually degrade trajectory accuracy in light conditions and provide no gain in the dark. The paper's discussion claims AAM \"solves the problem of alignment\" and that performance is \"less degraded,\" but the numbers do not support that. The only improvement is classification accuracy (99.3% vs 98.9%), which is within noise without error bars. The central fusion contribution is therefore not validated by the paper's own evidence.\n\nA second issue is the \"self-supervised\" labeling. LiDAR pseudo-labels are external supervision, and the Faster R-CNN teacher is supervised. The pseudo-label pipeline is under-specified: we are told \"up to 1-meter\" accuracy, but not the error distribution, outlier thresholds, or how label noise affects training. Since the model achieves 0.67 m APE on ground truth, either the pseudo-labels are much more accurate than claimed or something else is happening; the paper should report this.\n\nThere is also no statistical rigor: a single train/test split, no error bars, no multiple runs. The gains over prior work are large (e.g., 0.67 vs 1.24 mean APE for AV-FDTI), so the overall system probably works, but the source of the gain is not the fusion mechanism.\n\nThis is a solid desk-level paper that a reviewer could improve with a clear revision path: add error bars, analyze pseudo-label noise, and either demonstrate that FEM/AAM help with significance or reframe the contribution as the complete self-supervised pipeline rather than the fusion modules. I would send it to peer review because the system is practical and the flaw is addressable, but I would not accept the current claims.","headline":"Useful system with open code, but the paper's own ablation shows the proposed fusion modules don't improve trajectory estimation, so the SOTA claim doesn't hold as stated.","tokens_in":10056,"tokens_out":3190,"would_cite":false,"duration_ms":28818,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"The paper claims that AV-DTEC, a self-supervised audio-visual fusion model, estimates drone trajectories with 0.67 m mean error and 99.3% classification accuracy on the MMAUD benchmark, trained without manual annotation.","keywords":["anti-UAV","audio-visual fusion","self-supervised learning","trajectory estimation","drone classification","state-space model","Mamba","LiDAR pseudo-labels"],"falsifier":"One concrete check: on the MMAUD test split, compute the residual distribution between the LiDAR-generated training labels and the survey-grade ground truth. If the mean absolute or signed bias of those residuals approaches or exceeds 0.67 m, AV-DTEC's reported trajectory error cannot be trusted as true localization accuracy; retraining the same architecture on ground-truth labels should not improve APE if the pseudo-labels are adequate.","tokens_in":8998,"feed_emoji":"📡","tokens_out":5499,"duration_ms":46135,"temperature":0.7,"pith_summary":"The paper claims that a drone-detection system can be trained without any manual annotation and still beat fully supervised audio-visual baselines at both 3D trajectory estimation and drone-type classification. The system, AV-DTEC, uses LiDAR-generated pseudo-labels as its only supervision, then learns to track and classify drones from audio and video. On the MMAUD real-world dataset it reports a mean trajectory error of 0.67 m and 99.3% classification accuracy while running at 1.98 GFLOPs with 10.16M parameters. If correct, this means an inexpensive camera-plus-microphone setup could replace costly LiDAR-based counter-drone systems after a one-time LiDAR-labelled training phase.","feed_headline":"No-label drone tracker reaches 0.67 m trajectory error","feed_subtitle":"Audio-first self-supervised model keeps 99.3% classification accuracy from day to night at under 2 GFLOPs.","key_machinery":"The load-bearing machinery is the Audio Mamba, a selective state-space model applied separately to the temporal and spectral axes of multi-channel mel-spectrograms. Temporal Mamba scans patches left-to-right along time to capture the time-difference of arrival, and Spectral Mamba scans top-to-bottom to capture spectral attenuation. A learnable token in the sequence carries the aggregated information into a prediction head. The second piece is the feature enhancement module, a residual cross-attention block that fuses auxiliary visual or spectral features into the primary audio feature, and the adaptive adjustment mechanism, a teacher-student setup in which a Faster R-CNN teacher supervises a visual student; the student's predicted drone-existence probability becomes the fusion weight during inference.","core_discovery":"The central discovery is a self-supervised training scheme plus architecture that makes audio the primary modality for drone localization. Rather than asking annotators to label drone positions, the method clusters LiDAR point clouds with DBSCAN to produce pseudo-labels accurate to about one meter; these labels supervise an audio-visual network. The network extracts audio features with a parallel selective state-space model (Audio Mamba) that scans mel-spectrograms along both time and frequency axes, and it extracts visual features with Vision Mamba. A feature enhancement module injects visual features into audio features through residual cross-attention, while a teacher-student adaptive adjustment mechanism learns a scalar that down-weights visual features when the drone is not confidently visible. On the MMAUD benchmark the model reports state-of-the-art results: mean APE 0.67 m and accuracy 99.3%, and it degrades far less than visual or fusion baselines when lighting changes.","pith_inferences":["The reported accuracy should be read against the pseudo-label floor: with LiDAR labels accurate to about one meter, a mean APE of 0.67 m suggests the model partially corrects label noise, but the claim is only as strong as the unmeasured bias distribution of those labels.","The teacher-student weighting could be tested in reverse: a version that also learns to ignore audio when wind noise dominates might improve robustness beyond the lighting changes tested here.","Because the method estimates a 3D trajectory in a hemisphere rather than a full sphere, extending it to 360-degree coverage would require either multiple microphone arrays or a different fusion geometry; the paper does not address that regime.","A direct test of the self-supervision hypothesis would be to compare against the same architecture trained on survey-grade ground truth; if performance does not improve, the pseudo-labels are effectively as good as manual annotation."],"forward_implications":["Deployment-time cost drops to a camera and microphone array; LiDAR is needed only to generate training labels, not during inference.","Audio-first design keeps performance nearly flat from day to night: the reported mean APE moves from 0.58 m in daylight to 0.75 m at night, while pure-visual baselines collapse to errors above 8 m.","Because the model needs no manual annotations, counter-drone systems can be retrained for new environments or new drone types using only unlabelled recordings plus a temporary LiDAR setup.","The small footprint (1.98 GFLOPs, 10.16M parameters) puts real-time drone tracking within reach of embedded or mobile devices."],"supporting_citations":[{"why":"Supplies the MMAUD dataset, the real-world multi-modal data and survey-grade ground truth that the method is trained and evaluated on.","marker":"[31]"},{"why":"Provides the Mamba selective state-space model that is the basis of Audio Mamba's temporal and spectral scanning.","marker":"[8]"},{"why":"Supplies Vision Mamba, the visual feature extractor used to detect the drone in images and map its trajectory to 2D positions.","marker":"[27]"},{"why":"AV-pedaware is the self-supervised audio-visual fusion baseline whose architecture and teacher-student ideas the paper extends, and it contributes the AudioNet/VisualNet comparison models.","marker":"[7]"},{"why":"AV-FDTI is the supervised audio-visual anti-UAV baseline that AV-DTEC must beat on trajectory estimation, classification, and resource use.","marker":"[25]"},{"why":"DroneChase provides an audio-based drone tracking baseline that uses visual pseudo-labels, giving the paper a cross-modality self-supervised comparison.","marker":"[5]"},{"why":"Faster R-CNN is the teacher network in the adaptive adjustment mechanism that supervises the visual student model.","marker":"[28]"},{"why":"Supplies the RGB drone dataset used to train the Faster R-CNN teacher network.","marker":"[10]"},{"why":"Provides the LiDAR-based clustering trajectory estimation framework that the pseudo-label generation extends with spatial-temporal filtering.","marker":"[6]"}],"fun_headline_variants":["Self-supervised audio-visual drone tracking: 0.67 m error","Audio-first self-supervised drone tracker: 0.67 m error","Self-supervised AV fusion achieves 0.67 m drone tracking error","No-label audio-visual drone tracker hits 0.67 m error","Self-supervised AV fusion: 0.67 m error, 99.3% accuracy"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The entire training signal rests on LiDAR-generated training labels being accurate to roughly one meter; if those labels carry a larger or biased error, the reported sub-meter trajectory accuracy would not represent true localization quality.","fun_headline_variants_meta":{"raw":{"variants":["Self-supervised audio-visual drone tracking: 0.67 m error","Audio-first self-supervised drone tracker: 0.67 m error","Self-supervised AV fusion achieves 0.67 m drone tracking error","No-label audio-visual drone tracker hits 0.67 m error","Self-supervised AV fusion: 0.67 m error, 99.3% accuracy"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.001106,"raw_usage":{"total_tokens":4592,"prompt_tokens":909,"completion_tokens":3683,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":525,"completion_tokens_details":{"reasoning_tokens":3591}},"tokens_in":525,"tokens_out":3683,"duration_ms":38090,"temperature":1.0,"reasoning_tokens":3591,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-11T05:58:20.299273+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"One concrete check: on the MMAUD test split, compute the residual distribution between the LiDAR-generated training labels and the survey-grade ground truth. If the mean absolute or signed bias of those residuals approaches or exceeds 0.67 m, AV-DTEC's reported trajectory error cannot be trusted as true localization accuracy; retraining the same architecture on ground-truth labels should not improve APE if the pseudo-labels are adequate.","supporting_citations":[{"cited_title":"Clustering-based Learning for UAV Tracking and Pose Estimation","cited_arxiv_id":"2405.16867","evidence_quote":"Supplies the MMAUD dataset, the real-world multi-modal data and survey-grade ground truth that the method is trained and evaluated on."},{"cited_title":"An image is worth 16x16 words: Transformers for image recogni- tion at scale,","cited_arxiv_id":null,"evidence_quote":"Supplies Vision Mamba, the visual feature extractor used to detect the drone in images and map its trajectory to 2D positions."},{"cited_title":"Self-supervised moving vehicle tracking with stereo sound,","cited_arxiv_id":null,"evidence_quote":"AV-FDTI is the supervised audio-visual anti-UAV baseline that AV-DTEC must beat on trajectory estimation, classification, and resource use."},{"cited_title":"Dronechase: A mobile and automated cross-modality system for continuous drone tracking,","cited_arxiv_id":null,"evidence_quote":"DroneChase provides an audio-based drone tracking baseline that uses visual pseudo-labels, giving the paper a cross-modality self-supervised comparison."},{"cited_title":"Investigation of uav detection in images with complex backgrounds and rainy artifacts,","cited_arxiv_id":null,"evidence_quote":"Supplies the RGB drone dataset used to train the Faster R-CNN teacher network."}],"review_version":1}