{"id":"27c664aa-346e-4de7-8ace-66a277bafe35","arxiv_id":"2504.15609","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":4,"one_line_summary":"The paper introduces SonarT165, a large benchmark for underwater acoustic object tracking, and STFTrack, a tracker with multi-view template fusion and trajectory correction that reports state-of-the-art results on it.","lead":"Underwater sonar videos get a new tracking benchmark called SonarT165, with 330 test sequences and over 200,000 annotated frames, plus a tracker called STFTrack that reports top scores on it. The paper is useful mainly because it gives the field a common testbed and baseline numbers for acoustic object tracking.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"SOTA claim is confounded by training data: STFTrack uses UATD sonar and LasHeR thermal data while baselines use generic RGB pretraining, so reported margins may reflect data access rather than MTFM/OTCM/FEM.","rationale":"The paper has real strengths: a new benchmark with 205K annotations, evaluation of many trackers, a code link, and internal ablations that individually show contributions from FEM, MTFM, and OTCM. The reader's conditional verdict is appropriate. My stress-test, however, locates the weakest load-bearing condition elsewhere. The SOTA claim in Tables III and IV is an uncontrolled comparison: STFTrack benefits from UATD (sonar) and LasHeR (thermal-as-binary) training data, which no baseline receives. Because Table V shows UATD improves even the LiteTrack baseline, the reported 2 to 4 point SR gaps cannot be attributed to the proposed modules without same-data baselines. The OTCM brightness assumption flagged by the reader is genuinely a limitation on LAR/BI sequences, and Figure 14 shows it; but even if that assumption fails on some sequences, the module still helps on average as shown in Table VIII, so it is not the primary threat to the central claim. A single controlled retraining experiment can settle whether the SOTA margin is real. Until that is done, and until the dataset is released, the conditional verdict should stand.","tokens_in":24853,"tokens_out":7842,"duration_ms":76005,"concrete_test":"Retrain LiteTrack-B8 (the direct baseline) and at least one strong general tracker, e.g. ARTrackV2Seq-B256, using STFTrack's exact two-stage recipe (first stage LaSOT + GOT10k + UATD with grayscale conversion; second stage LasHeR with thermal images as binary simulant), keeping each architecture and their standard heads and losses unchanged. Then rerun the SonarT165 evaluation with the same metrics. If either retrained baseline reaches or exceeds STFTrack's SR within the reported margin (e.g., SR > 59.2 on fan for STFTrack-B, or > 57.6 for STFTrack-S), the SOTA claim is unsupported. If the margins persist, the claim survives this test.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim is that STFTrack is state-of-the-art on SonarT165 (Section V.B, Tables III and IV). The comparison is not controlled for training data. Section V.A states that STFTrack is trained in two stages: first on LaSOT + GOT10k + UATD (a forward-looking sonar dataset), with RGB converted to grayscale; second on LasHeR, with thermal images used to simulate acoustic binary images. The general and lightweight baselines in Tables III and IV are evaluated directly, with no indication that any of them received UATD or LasHeR, or was fine-tuned on sonar-like data. The ablation in Table V shows that adding UATD to the LiteTrack baseline already increases SR (fan 55.1 to 55.9, square 54.8 to 54.9), so the data are not neutral. Consequently, the headline margins (STFTrack-B 59.2 vs ARTrackV2Seq 57.4 SR; STFTrack-S 57.6 vs LiteTrack-B8 55.0) could be due to domain-specific pretraining rather than to MTFM, OTCM, or FEM. The absence of a validation split and of released benchmark data makes this hard to audit. The OTCM brightness assumption is a real limitation on low-reflection and background-interference sequences (Figure 14), but it is not the most load-bearing issue for the SOTA claim; the data confound is.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper introduces SonarT165, a benchmark for underwater acoustic object tracking (UAOT) containing 165 square-format and 165 fan-format sequences with 205K manual annotations, together with attribute annotations and evaluations of a wide range of general and lightweight trackers. The paper also proposes STFTrack, built on LiteTrack, with three components: a multi-view template fusion module (MTFM), an optimal trajectory correction module (OTCM), and a frequency enhancement module (FEM), plus an acoustic image high-frequency enhancement preprocessing step. STFTrack-B and STFTrack-S are reported to achieve state-of-the-art success rate (SR) on SonarT165 among general and lightweight trackers, respectively, in Tables III and IV.","tokens_in":25136,"tokens_out":3730,"duration_ms":34582,"significance":"If the benchmark and the comparisons are reliable, SonarT165 would be a valuable community resource for UAOT: it is substantially larger than the existing RGBS50 sonar test set, includes both square and fan sonar formats, and provides attribute annotations that expose concrete failure modes such as low acoustic reflection and background interference. The proposed tracking pipeline is lightweight, and the reported speed/parameter numbers in Table X suggest practical deployment potential. However, the central SOTA claim is currently not established as stated, because the training data used for STFTrack differs from that used for the baselines and because all ablations and hyperparameter choices are made on the test sequences. The benchmark contribution is therefore conditionally significant; it would become fully significant if the evaluation protocol were made controlled and a validation split were provided.","major_comments":[{"comment":"The state-of-the-art claim is confounded by training data. STFTrack is trained in two stages on LaSOT, GOT10k, UATD, and LasHeR (Section V.A), with thermal images used to simulate acoustic binary images, while the general and lightweight baselines in Tables III and IV are evaluated directly without any indication that they received UATD, LasHeR, or sonar-like fine-tuning. Table V shows that adding UATD to the LiteTrack baseline already improves SR (fan 55.1 to 55.9, square 54.8 to 54.9), so the reported margins of STFTrack-B over ARTrackV2Seq (59.2 vs 57.4 SR) and STFTrack-S over LiteTrack-B8 (57.6 vs 55.0 SR) could be driven by domain-specific pretraining rather than by MTFM, OTCM, or FEM. To support the SOTA claim, the authors should retrain or fine-tune the strongest baselines on the same acoustic/thermal data, or alternatively present a strictly controlled comparison in which every component is ablated with identical training data and report the marginal gains of the proposed modules over such matched baselines.","section":"Section V.A, V.B; Tables III, IV, V"},{"comment":"The ablation protocol uses the test set for model selection, creating a test-set feedback loop. All ablations in Tables V through IX are reported on SonarT165-Fan and SonarT165-Square, which are the same sequences used for the final performance claim, and Table IX selects the final enhancement variant (High x2) based on these test scores; note that High x3 is better on the fan split (60.7 vs 60.3 SR) but worse on the square split (57.9 vs 58.1 SR). Section VI.B acknowledges that the benchmark contains only test sequences and no training or validation split. Consequently, the chosen configuration, and hence the reported gains, may be optimistically biased by test-set feedback. The authors should add a validation split (or cross-validation over sequences) and perform module selection and threshold tuning on it, then report test results once for the selected configuration.","section":"Section V.D, Table IX; Section VI.B"},{"comment":"The OTCM module depends on hand-set thresholds and on an acoustic-brightness assumption that the paper's own failure analysis shows is violated. The binarization threshold in Eq. (8) is fixed at 30, the brightness segmentation threshold in Eq. (13) is the average pixel value of the target in the previous frame, and the IoB decision threshold in Eq. (14) is fixed at 0.6, with no sensitivity analysis for any of these values. The underlying assumption that target regions are reliably brighter than the background is contradicted by the LAR and BI attributes and by the failure cases in Figure 14 (notably sequences 015 fan and 055 fan), where low reflection and background interference cause accuracy degradation or drift. The authors should provide a threshold sensitivity study, a learned or otherwise justified derivation of these thresholds, and an explicit evaluation of OTCM behavior on LAR/BI sequences.","section":"Section IV.C, Eqs. (8), (12)-(14); Figure 14"}],"minor_comments":[{"comment":"The row for SonarT165 lists 300 sequences, but the text and the rest of the paper consistently state 330 sequences (165 square plus 165 fan); this inconsistency should be corrected.","section":"Table I"},{"comment":"The implementation details state 'Python 2.4.0 and Python 3.10'; this is likely a typo for PyTorch 2.4.0, and the two versions should be clarified.","section":"Section V.A"},{"comment":"The caption lists '(b)' twice; the fourth panel should be labeled '(d)' to match the references in the text.","section":"Figure 4"},{"comment":"The first metric column is labeled 'AUC' in the lightweight comparison table but 'SR' elsewhere; the metric names should be made consistent across tables, or the difference should be explicitly defined.","section":"Table IV"},{"comment":"The text refers to 'UOT00 [50] and [51]', which appears to be a typo for 'UOT100 [50] and VMAT [51]'; the citation formatting should be corrected.","section":"Section III.B.1"},{"comment":"The text says 'as shown in Table 7' when referring to the FEM ablation, but the table is numbered VII; the cross-reference should be fixed.","section":"Section V.D.1"},{"comment":"The attribute labels in the caption use 'BC' and 'TBI' instead of the attribute names 'TBC' and 'BI' defined in Table II; these should be aligned.","section":"Figure 3 captions"}],"recommendation":"major_revision","confidential_remarks":"The benchmark itself is a potentially useful contribution, but the SOTA claim and the module-level ablation claims are not currently supported because of the training-data confound and the test-set feedback loop. I would encourage the editor to request a revision that adds a validation split, re-runs ablations and hyperparameter selection on it, and provides a controlled comparison with baselines trained on the same acoustic/thermal data. If such a revision is infeasible, the authors should at minimum narrow their claims and prominently disclose the data advantage."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Two things to know. First, the benchmark is genuinely useful: 330 sequences, 205K annotations, pool and field, first of its scale for acoustic tracking, and it exposes that mainstream trackers get high PR but mediocre SR on sonar. Second, the method's headline result is not trustworthy as reported. STFTrack is trained on UATD (sonar) and LasHeR (thermal) while the baselines are evaluated with their stock RGB-pretrained weights; Table V shows that adding UATD to the LiteTrack baseline alone improves SR on fan sequences by 0.8 points, so the data are not neutral. The claimed margins over ARTrackV2 (1.8 SR) and LiteTrack-B8 (2.6 SR) could easily shrink or vanish under a controlled comparison. The paper's Limitation section admits the benchmark contains only test sequences, which means every ablation and hyperparameter choice (e.g., High x2 vs x3 enhancement) was made on the test set. That is a textbook test-set feedback loop, and it weakens the state-of-the-art claim substantially. There are no error bars or significance tests, and the margins are small. Minor: the OTCM brightness assumption is acknowledged to fail under low acoustic reflection and background interference (Figure 14), and the benchmark data are not released (only code is). None of this kills the dataset contribution. The benchmark fills a real gap, the annotation effort is substantial, and the baseline evaluation across dozens of trackers is a useful service to the community. The method is described clearly, and the ablations are systematic, but the evaluation protocol needs major revision. To make the SOTA claim credible, the authors should release the data, provide a proper training/validation split or at least evaluate baselines fine-tuned on the same sonar-like data, and report variance. Then SonarT165 could indeed become the standard UAOT testbed. My recommendation: send to peer review, but with major revisions. The dataset deserves referee time; the method paper does not, in its current form.","headline":"SonarT165 is a real benchmark contribution, but the STFTrack SOTA claim is undermined by an uncontrolled training-data comparison and a test-set feedback loop that the paper itself acknowledges.","tokens_in":25726,"tokens_out":2404,"would_cite":true,"duration_ms":22758,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"This paper proposes the first large-scale benchmark for underwater acoustic object tracking, SonarT165, and a tracker called STFTrack that outperforms existing general and lightweight trackers on it.","keywords":["underwater acoustic object tracking","sonar imaging","tracking benchmark","single object tracking","template fusion","trajectory correction","Kalman filter","frequency enhancement"],"falsifier":"Take the low-acoustic-reflection sequences from SonarT165 (attribute LAR) and compare full STFTrack against STFTrack with the OTCM brightness term $R_{np}$ removed: if the success rate gap between the two versions essentially vanishes, then the acoustic-response-equivalent pixel prior is not what carries the improvement, and the paper's central mechanism would be disconfirmed.","tokens_in":24604,"feed_emoji":"🌊","tokens_out":7779,"duration_ms":67815,"temperature":0.7,"pith_summary":"The paper argues that underwater acoustic object tracking has been held back by the lack of a common, large-scale evaluation set, and it supplies one: SonarT165, with 330 sequences in square and fan sonar formats and 205K manually checked annotations. It then evaluates a broad set of existing single-object trackers on this benchmark and finds that their localization precision is high while their overlap-based success is low, which it reads as evidence that acoustic appearance variation is a distinct challenge. To meet that challenge, it introduces STFTrack, built on the LiteTrack pipeline, with a multi-view template fusion module, an optimal trajectory correction module, and frequency-based image and feature enhancement. The paper's central claim is that STFTrack achieves state-of-the-art results on SonarT165 among both general and lightweight trackers, at a small computational cost over its baseline.","feed_headline":"STFTrack tops a new 330-video underwater tracking benchmark","feed_subtitle":"SonarT165 supplies 330 sequences and 205K annotations, giving acoustic trackers a common ground for comparison.","key_machinery":"The central mechanism is the acoustic-response-equivalent pixel property: in forward-looking sonar images, a pixel's brightness equals the strength of the acoustic reflection, so the target region can be separated from background by binarization at a threshold derived from the previous frame. This property carries the OTCM, where the normalized pixel brightness response $R_{np} = \\text{mean}(\\text{extract\\_patch}(x_m, B_c))$ (with $x_m$ the binarized search image) is multiplied with an IoU score between Kalman prediction and candidate boxes to reject suboptimal matches, and the IoB score then checks whether the chosen box is just a neighbor of the maximum-response box. The other load-bearing piece is MTFM, which builds a multi-view dynamic template from original and binary images, enhances it with channel and spatial attention, and fuses it with the fixed template using cross-attention layers, giving the tracker a spatio-temporal target representation that adapts as the target's acoustic signature changes.","core_discovery":"STFTrack's design is built on a property of acoustic imaging: the pixel value in a sonar image is proportional to the acoustic reflection intensity at that point, so targets appear as bright, well-edged regions against a dark background. The multi-view template fusion module (MTFM) exploits this by feeding both the original dynamic template and a binarized version of it through a shared backbone, applying spatial and channel attention, and then fusing the resulting multi-view dynamic template with the fixed template through a cross-attention-style layer. The optimal trajectory correction module (OTCM) uses the same property to repair the standard Kalman-filter trajectory: it scores candidate boxes by their normalized pixel brightness response, combines that with an IoU score against the Kalman prediction, and uses an intersection-over-box2 check against the maximum-response box to suppress suboptimal matches. Added to this are an acoustic image enhancement step that adds back double high-frequency content and a frequency enhancement module that learns separate high- and low-frequency feature branches. On SonarT165, the paper reports STFTrack-B reaching 59.2 SR and 90.8 PR, and STFTrack-S reaching 57.6 SR and 89.9 PR, surpassing the best general tracker and the best lightweight baselines.","pith_inferences":["The paper leaves implicit that SonarT165 being test-only could be addressed by releasing a training split; doing so might reduce the domain gap that currently forces the model to train on RGB and thermal data.","The acoustic-response-equivalent pixel prior likely transfers to other reflection-imaging modalities such as medical ultrasound, where brightness likewise encodes reflected intensity; the paper points to this connection but does not demonstrate it.","A concrete extension would be to make the OTCM binarization threshold adaptive per frame or learned, since the failure cases show the previous-frame threshold degrades under low acoustic reflection and background interference.","If SonarT165 becomes a standard underwater acoustic tracking testbed, evaluating square and fan sequences separately, as this paper does, would be a natural protocol because tracker behavior differs systematically between the two formats."],"forward_implications":["SonarT165 gives the underwater acoustic tracking community a common testbed of 330 sequences and 205K annotations, so future acoustic trackers can be compared on equal ground instead of on private or tiny datasets.","Because general trackers already score near 80 PR but below 60 SR, the benchmark isolates overlap accuracy, getting the full box right rather than just the center, as the open problem in acoustic tracking.","STFTrack's efficiency numbers, roughly 283 FPS on an RTX 3090 Ti and 25 FPS on an Orin NX for the S variant, indicate the approach can run on embedded hardware of the kind used on underwater vehicles.","The benchmark's attribute analysis points to specific unsolved challenges, including acoustic object crossover, small targets, out-of-view, and low acoustic reflection, that next-generation trackers will need to address."],"supporting_citations":[{"why":"Supplies the LiteTrack backbone, asynchronous feature extraction design, and pretrained weights that STFTrack-B and STFTrack-S build on.","marker":"[14]"},{"why":"Provides the Kalman-filter trajectory fusion scheme and response-map candidate reuse that the OTCM takes as its starting point and corrects.","marker":"[27]"},{"why":"The earlier RGB-sonar benchmark whose object categories, PR/NPR metrics, and two-format sonar setup SonarT165 extends.","marker":"[1]"},{"why":"One of the underwater object tracking benchmarks used in the scale comparison that establishes SonarT165 as larger.","marker":"[50]"},{"why":"Part of the first-stage training set; the paper's ablations show LaSOT contributes to acoustic tracking performance.","marker":"[69]"},{"why":"Another first-stage training set whose positive contribution to the tracking pipeline is verified by ablation on SonarT165.","marker":"[70]"},{"why":"UATD supplies sonar-domain data in the first training stage, adapting the model to acoustic image statistics.","marker":"[71]"},{"why":"LasHeR provides thermal images used in the second training stage to simulate binary acoustic images when training MTFM.","marker":"[72]"},{"why":"ARTrackV2Seq-B256 is the strongest general tracker baseline that STFTrack-B must beat for its state-of-the-art claim.","marker":"[61]"}],"fun_headline_variants":["SonarT165: first large-scale acoustic tracking benchmark","New benchmark exposes tracker flaws; STFTrack fixes them","Acoustic trick: brightness-based tracking beats Kalman fixes","Sonar tracking benchmark: 165 sequences, 205K labels","From darkness to targets: STFTrack's acoustic imaging insight"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that the target is brighter in the sonar image than its surroundings, so a binarization threshold taken from the previous frame picks out the true target region; the paper's own failure cases show this stops holding under low acoustic reflection and background interference.","fun_headline_variants_meta":{"raw":{"variants":["SonarT165: first large-scale acoustic tracking benchmark","New benchmark exposes tracker flaws; STFTrack fixes them","Acoustic trick: brightness-based tracking beats Kalman fixes","Sonar tracking benchmark: 165 sequences, 205K labels","From darkness to targets: STFTrack's acoustic imaging insight"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000937,"raw_usage":{"total_tokens":4088,"prompt_tokens":1105,"completion_tokens":2983,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":721,"completion_tokens_details":{"reasoning_tokens":2899}},"tokens_in":721,"tokens_out":2983,"duration_ms":19877,"temperature":1.0,"reasoning_tokens":2899,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-16T11:21:49.521898+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Take the low-acoustic-reflection sequences from SonarT165 (attribute LAR) and compare full STFTrack against STFTrack with the OTCM brightness term $R_{np}$ removed: if the success rate gap between the two versions essentially vanishes, then the acoustic-response-equivalent pixel prior is not what carries the improvement, and the paper's central mechanism would be disconfirmed.","supporting_citations":[{"cited_title":"Litetrack: Layer pruning with asynchronous feature extraction for lightweight and efficient visual tracking,","cited_arxiv_id":null,"evidence_quote":"Supplies the LiteTrack backbone, asynchronous feature extraction design, and pretrained weights that STFTrack-B and STFTrack-S build on."},{"cited_title":"Underwater object tracker: Uostrack for marine organism grasping of underwater vehicles,","cited_arxiv_id":null,"evidence_quote":"Provides the Kalman-filter trajectory fusion scheme and response-map candidate reuse that the OTCM takes as its starting point and corrects."},{"cited_title":"Rgb-sonar tracking bench- mark and spatial cross-attention transformer tracker,","cited_arxiv_id":null,"evidence_quote":"The earlier RGB-sonar benchmark whose object categories, PR/NPR metrics, and two-format sonar setup SonarT165 extends."},{"cited_title":"Comprehensive underwater object tracking benchmark dataset and underwater image enhancement with gan,","cited_arxiv_id":null,"evidence_quote":"One of the underwater object tracking benchmarks used in the scale comparison that establishes SonarT165 as larger."},{"cited_title":"Lasot: A high-quality benchmark for large-scale single object tracking,","cited_arxiv_id":null,"evidence_quote":"Part of the first-stage training set; the paper's ablations show LaSOT contributes to acoustic tracking performance."},{"cited_title":"A dataset with multibeam forward-looking sonar for underwater object detection,","cited_arxiv_id":null,"evidence_quote":"UATD supplies sonar-domain data in the first training stage, adapting the model to acoustic image statistics."},{"cited_title":"Lasher: A large-scale high-diversity benchmark for rgbt tracking,","cited_arxiv_id":null,"evidence_quote":"LasHeR provides thermal images used in the second training stage to simulate binary acoustic images when training MTFM."},{"cited_title":"Artrackv2: Prompting autore- gressive tracker where to look and how to describe,","cited_arxiv_id":null,"evidence_quote":"ARTrackV2Seq-B256 is the strongest general tracker baseline that STFTrack-B must beat for its state-of-the-art claim."}],"review_version":1}