{"id":"658d9163-4b60-4d4b-ba96-75b8dd87b905","arxiv_id":"2502.05129","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":2,"one_line_summary":"A ResNet-18 trained on compressed sonar echograms predicts salmon counts in 200-frame video windows with 23% count error on in-distribution data, suggesting a low-compute alternative to detection and tracking.","lead":"The authors show that a lightweight neural network can count migrating salmon directly from echograms, compressed images that represent many frames of sonar video. This could make automated fish counting practical at remote river sites where frame-by-frame object detection and tracking models are too computationally heavy.","discovery_kind":"new_application","skeptic_critique":null,"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes a method to count migrating salmon directly from echograms—two-dimensional temporal max-intensity projections of multi-beam sonar video—using a ResNet-18 regressor that outputs upstream and downstream counts per 200-frame window. Training combines human-validated 'strong' labels from the Caltech Fish Counting dataset with weakly supervised labels generated by the authors' own detector-tracker pipeline. The authors introduce domain-specific augmentations and report an nMAE of 23% on KL-val and 30.7% on KR, claiming this as a low-compute alternative to per-frame detection and tracking.","tokens_in":6825,"tokens_out":4769,"duration_ms":47084,"significance":"If the numbers hold, the work is a useful feasibility demonstration: counting regression on a compressed temporal representation is markedly cheaper at inference than per-frame YOLO plus tracking, and echograms are already the native review medium for sonar technicians. Positive contributions include the use of the public CFC benchmark, a direct comparison against prior tracking-by-detection results, clear ablations, and a domain-motivated 'realistic horizontal flip' augmentation. The central caveat is that the primary result is measured on the development set and lacks uncertainty quantification; the compute motivation is also not yet demonstrated end-to-end. As a feasibility study, the idea is worth pursuing, but the current evidence is conditional.","major_comments":[{"comment":"The headline 23% nMAE on KL-val is not an independent test statistic. KL-val is used for early stopping (Sec. 3.2) and for selecting echogram generation thresholds (Sec. 5.2, Table 2), and the final model is the result of choices made by looking at this set. With only N=66 clips, the reported number is likely to be optimistically biased. Please report results on a properly held-out split, for example repeated cross-validation or a separate validation set, or present KR as the primary held-out result with confidence intervals.","section":"Sec. 3.2, Sec. 5.2, Table 2"},{"comment":"All reported nMAE values are single point estimates without error bars, repeated seeds, or bootstrap intervals. The ablations in Table 3 differ by only a few points (e.g., 43.2 vs. 39.3 on KL-train with different augmentations), and without variance estimates it is impossible to tell which differences are meaningful. Please provide confidence intervals or results across multiple random seeds.","section":"Sec. 5, Tables 1–3"},{"comment":"The claimed compute advantage is not quantified. The method still requires per-frame background subtraction at three thresholds and connected-component analysis for every frame before the echogram is constructed; the paper does not measure the runtime or energy cost of this preprocessing, nor does it compare end-to-end cost against the detector-tracker. The statement 'requiring only a single forward pass every 200 frames' omits the cost of building the input. Please measure end-to-end processing time on representative field hardware.","section":"Sec. 3.1 and Introduction"},{"comment":"The downstream count errors are very large: 37.5% nMAE on KL-val and 96% on KR for the best model, while the abstract reports a single 23% aggregate. Since downstream fish are part of the management goal, the aggregate metric obscures a failure mode. Additionally, the weak labels are produced by the same CFC detector-tracker used as the baseline, so the model may inherit the tracker's biases (especially for downstream fish); the paper does not analyze weak-label noise. Please report downstream and upstream errors separately in the abstract-level claims and provide a characterization of weak-label bias.","section":"Sec. 4.1, Sec. 5, Tables 1 and 3"}],"minor_comments":[{"comment":"The summation in Eq. (1) is written from i=0 to N, but with N clips the index should start at 1.","section":"Sec. 4.2, Eq. (1)"},{"comment":"The sentence 'We ensure there is no overlap between the KL validation set and the detector-tracker annotated training or validation set' is unclear; please specify how the 33,437 weak-labeled clips were split between training and validation.","section":"Sec. 4.1"},{"comment":"The use of bullet points to indicate augmentation settings makes the table hard to parse; a table with explicit True/False columns would be clearer.","section":"Table 3"},{"comment":"The caption does not explain how ground-truth counts are binned to determine dot sizes; please add a short description.","section":"Figure 2"},{"comment":"Reference [11] lists 'Trout Unlimited Caltech' as an author; this appears to be an organization rather than a person and should be formatted consistently with the other references.","section":"References"}],"recommendation":"major_revision","confidential_remarks":"The authors are transparent about their limitations, which is commendable. My main concern is that the evaluation methodology needs to be upgraded before the headline claim can be trusted. I would also encourage the editor to consider whether a feasibility study with these accuracy levels is sufficient, though I believe it is within the scope of the journal."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Worth a look: this is the first paper I've seen that regresses salmon counts directly from echograms, skipping detection and tracking entirely, and the basic feasibility claim holds up. The numbers are real but need some scrutiny before you trust them.\n\nWhat's genuinely new: the echogram representation (max-intensity projection over beams plus a lateral-position channel) plus a ResNet-18 regressing counts for 200-frame windows. That's a clean, lightweight pipeline: one forward pass per window, no per-frame detector. The realistic horizontal flip augmentation—flipping and then inverting the lateral-position channel—is a small, correct domain-specific contribution. And the weak-supervision protocol, labels from the CFC detector-tracker, is not circular because the final evaluation is against human strong counts; it's a sensible way to exploit 33k unlabeled clips.\n\nCredit where due: the paper is honest about its limitations and compares against the strong detector-tracker baselines. The observation that downstream fish are badly underrepresented (175 vs 8 in KL-val) is stated plainly.\n\nSoft spots, in order of severity. First, the headline 23% nMAE is on a 66-clip validation set that the authors used for early stopping and threshold selection. That is an optimistic point estimate, with no error bars or repeated splits. It could easily be 30% on a true held-out set; the out-of-distribution 30.7% is actually more informative. Second, downstream counts are bad: 37.5% in-distribution and 96% out-of-distribution. That is not a minor blemish; escapement includes downstream movers. Third, the compute advantage is partially unearned: echogram generation still runs per-frame background subtraction and connected components, and the paper doesn't measure that cost. If that preprocessing is slow on field laptops, the lightweight claim weakens. Fourth, no code or data are released, so the numbers can't be checked independently.\n\nThese don't sink the central idea. The paper is a proof of concept, not a deployment-ready system, and it reads like one. The experiments support feasibility, the ablation table is coherent, and the honest discussion of underprediction at high density is a good sign.\n\nRecommendation: send it to review. A serious referee should ask for error bars or a held-out test set, a measurement of echogram generation time, and disaggregated results by density. But the work deserves referee time; it's new, the writing is clear, and the limitations are stated rather than buried.","headline":"Direct count regression from echograms is a genuinely new and plausible feasibility result for low-compute salmon counting, but the headline 23% error is measured on a tiny validation set used for model selection and deserves scrutiny.","tokens_in":7283,"tokens_out":2508,"would_cite":true,"duration_ms":23096,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"A single compressed sonar image per 200 video frames predicts salmon counts with 23% error, replacing per-frame detection and tracking.","keywords":["sonar fish counting","echogram","salmon escapement","temporal video representation","weakly supervised learning","image augmentation","ResNet-18","river monitoring"],"falsifier":"Measure end-to-end processing time of echogram generation plus ResNet-18 inference per 200-frame window on a typical field laptop and compare it with a detector-tracker on the same hardware; if preprocessing dominates and the total time is comparable to frame-wise detection, the central practical claim fails.","tokens_in":6670,"feed_emoji":"🐟","tokens_out":4543,"duration_ms":44000,"temperature":0.7,"pith_summary":"This paper argues that counting migrating salmon can be done by compressing hundreds of sonar video frames into a single temporal image, called an echogram, and reading the fish count directly from that image with a small neural network. This replaces frame-by-frame object detection and tracking, which requires heavy compute that remote river sites often lack. The authors report a count error of 23 percent on in-distribution validation data and 30.7 percent on an out-of-distribution test set from the Kenai River, and they show that mixing weak labels generated by a detector-tracker with human labels improves accuracy. If the approach holds, salmon escapement monitoring could run on consumer laptops with one forward pass per 200 frames.","feed_headline":"Count salmon from one echogram per 200 video frames","feed_subtitle":"A lightweight model reads compressed sonar images instead of tracking fish frame by frame, bringing automated escapement counts to…","key_machinery":"The echogram is the central object: it compresses a multi-beam sonar clip into a single image whose columns are per-frame maximum-intensity profiles over range, with a second channel storing the lateral beam position of that maximum. The model is a ResNet-18 that outputs two non-negative counts, one for upstream and one for downstream fish, for the window. The weakly-supervised training protocol expands the labeled set to 33,437 weak-label images generated by an existing detector-tracker, and the domain-specific augmentations include vertical flipping, superposition of echograms, and a 'realistic horizontal flip' that reflects the image and inverts the lateral-position channel so fish motion direction is preserved.","core_discovery":"The central claim is that upstream and downstream fish counts within a 200-frame time window can be predicted directly from an echogram using a ResNet-18 model, bypassing per-frame detection and tracking. An echogram is a two-dimensional image in which time runs horizontally and each column encodes the strongest sonar echo at each range for one video frame, so a few hundred frames collapse into one image. With weak labels from an existing detector-tracker and strong human labels, plus domain-specific augmentations, the model reaches a normalized mean absolute error of 23 percent on the KL-val split and 30.7 percent on the right-bank test set. Error is much larger for downstream-moving fish, which are rare in the training data, and the model undercounts in clips with many overlapping fish. The authors position this as a feasibility result: the accuracy is higher than state-of-the-art tracking pipelines but comparable to early proof-of-concept counts, while the compute per window is much lower.","pith_inferences":["Beyond the paper, the advertised compute advantage depends on echogram generation being cheap; the reported timings omit background subtraction and connected-components preprocessing, so a fair end-to-end comparison on field hardware could change the conclusion.","The temporal-compression idea may transfer to other sonar counting tasks, such as eel or trout passage, if the weak-label and augmentation recipe carries over.","The systematic undercounting at high fish density suggests that a detection-free regressor could benefit from density-aware losses or multiple time-resolution windows, a testable extension not explored here.","Since the model uses only intensity and lateral-position channels, adding beam-level or motion-direction information might specifically improve downstream counting, which the current setup handles worst."],"forward_implications":["Counting cost drops to one lightweight forward pass per 200-frame window, enabling deployment on low-compute or offline hardware at remote river sites.","Weak labels alone almost match strong labels on out-of-distribution data, so previously unlabeled sonar footage can be harnessed for training.","Domain-specific augmentation, especially the realistic horizontal flip, improves accuracy while naive flipping hurts, giving a recipe for similar temporal-compression counting tasks.","The largest remaining error is downstream fish due to class imbalance, so reducing that imbalance is the main lever for improving total counts.","At 23 percent error the method is not yet a replacement for tracking-based counts, but it is a viable low-cost screening tool or complement to manual review."],"supporting_citations":[{"why":"Supplies the fish counting benchmark dataset (KL-train, KL-val, KR), the strong labels, and the public detector-tracker pipeline used to generate weak labels, as well as the tracking-by-detection baselines of 4.9% and 11.8% error.","marker":"[8]"},{"why":"Supplies the ResNet-18 backbone that the model finetunes for direct count regression.","marker":"[5]"},{"why":"Provides the earlier proof-of-concept counting result of 19.3% error that the paper uses as a comparison point for its own feasibility claim.","marker":"[11]"},{"why":"Defines the ARIS sonar format from which echograms are generated.","marker":"[14]"}],"fun_headline_variants":["Fish counts from compressed sonar: 23% error without tracking","One image per 200 frames: lightweight AI counts migrating salmon","ResNet reads sonar echograms for salmon counts at low compute","Echogram shortcut: fish counts from a single compressed image","Sonar video compressed to echograms cuts counting compute"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The assumption that generating an echogram is cheap enough to preserve the method's compute advantage: the paper's advertised single pass per 200 frames does not count the background subtraction and connected-components preprocessing per video frame, and that cost is not measured.","fun_headline_variants_meta":{"raw":{"variants":["Fish counts from compressed sonar: 23% error without tracking","One image per 200 frames: lightweight AI counts migrating salmon","ResNet reads sonar echograms for salmon counts at low compute","Echogram shortcut: fish counts from a single compressed image","Sonar video compressed to echograms cuts counting compute"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000188,"raw_usage":{"total_tokens":1309,"prompt_tokens":902,"completion_tokens":407,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":518,"completion_tokens_details":{"reasoning_tokens":321}},"tokens_in":518,"tokens_out":407,"duration_ms":4837,"temperature":1.0,"reasoning_tokens":321,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-08T20:08:57.289459+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Measure end-to-end processing time of echogram generation plus ResNet-18 inference per 200-frame window on a typical field laptop and compare it with a detector-tracker on the same hardware; if preprocessing dominates and the total time is comparable to frame-wise detection, the central practical claim fails.","supporting_citations":[{"cited_title":"The caltech fish counting dataset: A benchmark for multiple- object tracking and counting, 2022","cited_arxiv_id":null,"evidence_quote":"Supplies the fish counting benchmark dataset (KL-train, KL-val, KR), the strong labels, and the public detector-tracker pipeline used to generate weak labels, as well as the tracking-by-detection baselines of 4.9% and 11.8% error."},{"cited_title":"Au- tomated salmonid counting in sonar data","cited_arxiv_id":null,"evidence_quote":"Provides the earlier proof-of-concept counting result of 19.3% error that the paper uses as a comparison point for its own feasibility claim."},{"cited_title":"Aris sonars","cited_arxiv_id":null,"evidence_quote":"Defines the ARIS sonar format from which echograms are generated."}],"review_version":1}