pith. sign in

arxiv: 2605.18038 · v1 · pith:24HWM4S5new · submitted 2026-05-18 · 💻 cs.CV

Patch Ensembles for Robust Salmon Re-Identification with Weak Trajectory Labels

Espen Uri H{\o}gstedt , Christian Schellewald , Annette Stahl , Rudolf Mester This is my paper

Pith reviewed 2026-05-20 11:50 UTC · model grok-4.3

classification 💻 cs.CV
keywords salmon re-identificationpatch ensemblelateral lineweak trajectory labelscross-camera testingaquaculturecomputer visionrobust identification
0
0 comments X

The pith

Predicting the lateral line on salmon images allows a patch ensemble to re-identify individuals more robustly than full-image approaches.

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

This paper develops a re-identification method that breaks salmon images into smaller patches anchored on the predicted lateral line and combines the identity predictions from those patches. It addresses the practical barriers of labeling thousands of fish in large net-pen groups and the bias that arises when trajectory IDs serve as weak proxy labels. The evaluation uses two cameras placed six meters apart so the same fish can be seen in separate trajectories, with matches verified by hand to create a realistic cross-camera test. A sympathetic reader would care because reliable individual tracking supports better health and growth monitoring in commercial aquaculture without requiring impractical amounts of precise labels. The approach shows its largest gains precisely when fish move to a new camera, pointing to stronger generalization than models that classify entire images at once.

Core claim

The authors claim that a lateral line predictor enables consistent extraction of texture-anchored patches and slices, and that fusing the patch-level predictions produces more accurate and robust salmon identity decisions than a full-image baseline. This advantage appears in same-trajectory validation and becomes especially clear in cross-camera testing built from manual match confirmation, indicating that the patch ensemble reduces sensitivity to trajectory bias and viewpoint shifts.

What carries the argument

The lateral line predictor that anchors extraction of texture patches and slices whose individual predictions are fused into a final identity decision.

If this is right

  • The patch ensemble reduces the distorting effect of trajectory-ID bias in weak labels.
  • Accuracy rises in both same-trajectory and cross-camera evaluations.
  • The system supports reliable re-identification inside large populations where exhaustive labeling is impossible.
  • Better results across separate camera positions show improved handling of changes in viewpoint.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • Anchoring patches on a stable midline feature such as the lateral line could transfer to re-identification of other fish or patterned animals.
  • Pairing the method with automated video tracking could generate larger weakly labeled datasets over time without extra human effort.
  • Testing under wider ranges of lighting, water clarity, or fish density would clarify how far the robustness extends.
  • Adoption in commercial farms might lower the cost of continuous individual monitoring systems.

Load-bearing premise

The manual confirmation of matches across camera views produces an accurate and unbiased test set, and the lateral line predictor yields consistently useful patches across trajectories.

What would settle it

A study that rebuilds the cross-camera test set with an independent matching process or replaces the lateral line predictor with random patch selection and finds no remaining performance gain over the full-image baseline.

read the original abstract

Salmon re-identification in commercial net-pens is challenging due to large populations, which impose strict accuracy requirements and make large-scale labeled data acquisition infeasible. Trajectory IDs can be used as proxy labels, but this introduces trajectory-ID bias. To address these challenges, we propose a patch-based re-identification framework that fuses patch-level predictions into a salmon identity decision. A key component is the prediction of the salmon's lateral line, enabling extraction of texture-anchored patches and patch slices. To enable realistic evaluation, we introduce an experimental setup using multiple cameras placed 6 m apart, allowing the same fish to be recorded in different trajectories. This enables the construction of a cross-camera test set through manual match confirmation. Our ensemble approach outperforms the full-image baseline in same-trajectory validation (0.932 to 0.965 mAP) and cross-camera testing (0.609 to 0.860 mAP). The substantial improvements in the cross-camera setting demonstrate improved generalizability and robustness. Code and data: https://github.com/espenbh/salmon-reid-patch-ensemble.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit. Tearing a paper down is the easy half of reading it; the pith above is the substance, this is the friction.

Referee Report

2 major / 2 minor

Summary. The paper proposes a patch ensemble framework for salmon re-identification that uses weak trajectory IDs as proxy labels. A lateral-line predictor extracts texture-anchored patches and slices whose predictions are fused for identity decisions. Evaluation employs a multi-camera setup (cameras 6 m apart) to construct a cross-camera test set via manual match confirmation, reporting mAP gains over a full-image baseline from 0.932 to 0.965 on same-trajectory validation and from 0.609 to 0.860 on cross-camera testing. The authors argue the cross-camera gains demonstrate improved robustness and generalizability; code and data are released.

Significance. If the evaluation holds, the work supplies a practical, weakly supervised approach to re-identification under trajectory bias and limited annotations, with concrete mAP improvements and an explicit multi-camera protocol that strengthens ecological validity. Open-sourcing of code and data is a clear asset for reproducibility.

major comments (2)
  1. [§4.3] §4.3 (Cross-camera evaluation): The headline claim of improved generalizability rests on the 0.609-to-0.860 mAP gain, yet the cross-camera test set is assembled solely by manual match confirmation with no reported protocol, inter-annotator agreement, blinding procedure, or bias-mitigation steps. Given visual similarity among salmon, confirmation errors would directly affect the reported robustness improvement and must be quantified or validated.
  2. [§3.2] §3.2 (Lateral-line predictor): Patch extraction is anchored on the predicted lateral line, but no accuracy metric (pixel error, IoU on held-out frames) or ablation on predictor quality is supplied. Because this component is load-bearing for the texture-anchored patches and the ensemble gains, its reliability needs explicit measurement.
minor comments (2)
  1. [Table 2] Table 2: add the exact number of unique identities and images per split to allow direct comparison with prior salmon re-id datasets.
  2. [Figure 4] Figure 4 caption: explicitly state the ensemble fusion rule (e.g., mean, max, or learned) rather than referring only to the method section.

Simulated Author's Rebuttal

2 responses · 0 unresolved

Thank you for the constructive feedback on our manuscript. We address each major comment below and outline the revisions we will make to strengthen the paper.

read point-by-point responses

  1. Referee: [§4.3] §4.3 (Cross-camera evaluation): The headline claim of improved generalizability rests on the 0.609-to-0.860 mAP gain, yet the cross-camera test set is assembled solely by manual match confirmation with no reported protocol, inter-annotator agreement, blinding procedure, or bias-mitigation steps. Given visual similarity among salmon, confirmation errors would directly affect the reported robustness improvement and must be quantified or validated.

    Authors: We agree that the current description of the cross-camera test set construction lacks sufficient detail on the manual matching protocol. In the revised manuscript we will expand §4.3 with a step-by-step account of the matching procedure, including the specific visual criteria applied, steps taken to reduce annotator bias (such as randomized image presentation and independent review), and any available consistency checks. While formal inter-annotator agreement was not computed in the original study, we will explicitly discuss this limitation and provide additional qualitative examples of confirmed matches to support the reliability of the test set. revision: yes

  2. Referee: [§3.2] §3.2 (Lateral-line predictor): Patch extraction is anchored on the predicted lateral line, but no accuracy metric (pixel error, IoU on held-out frames) or ablation on predictor quality is supplied. Because this component is load-bearing for the texture-anchored patches and the ensemble gains, its reliability needs explicit measurement.

    Authors: We acknowledge that the reliability of the lateral-line predictor should be quantified. In the revised version we will add an evaluation subsection reporting pixel-level error and IoU metrics on a held-out set of frames with ground-truth lateral-line annotations. We will also include an ablation that measures the effect of predictor accuracy on downstream re-identification mAP, thereby demonstrating the sensitivity of the ensemble to this component. revision: yes

Circularity Check

0 steps flagged

Empirical patch-ensemble method evaluated on held-out cross-camera data; no derivation circularity

full rationale

The paper presents an empirical computer-vision method for salmon re-identification that fuses patch-level predictions. Reported gains (same-trajectory mAP 0.932→0.965; cross-camera 0.609→0.860) are obtained by direct evaluation on held-out trajectory and manually confirmed cross-camera test sets. No equations, first-principles derivations, or fitted parameters are shown that reduce any claimed prediction to a quantity defined by the inputs themselves. No self-citation load-bearing steps, uniqueness theorems, or ansatzes appear in the provided derivation chain. The work is therefore self-contained against external benchmarks.

Axiom & Free-Parameter Ledger

2 free parameters · 1 axioms · 0 invented entities

Relies on standard deep-learning training assumptions plus domain choices for patch extraction and weak-label handling; no new physical entities postulated.

free parameters (2)
  • Patch extraction hyperparameters
    Size, number, and slicing strategy for texture-anchored patches; chosen to enable fusion.
  • Ensemble fusion rule
    How patch-level predictions are aggregated into final identity score.
axioms (1)
  • domain assumption Trajectory IDs provide usable proxy labels once bias is mitigated by patch fusion
    Central to the weak-supervision strategy described in the abstract.

pith-pipeline@v0.9.0 · 5732 in / 1271 out tokens · 34972 ms · 2026-05-20T11:50:01.830429+00:00 · methodology

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

29 extracted references · 29 canonical work pages · 4 internal anchors

  1. [1]

    INTRODUCTION Monitoring individual salmon over long periods will enable researchers to study how individuals develop over time, and aquaculture operators to tailor operations to the needs of each salmon. Today, fish re-identification relies on electronically This work was supported by the Research Council of Norway (RCN) under project number 344022, title...

    work page internal anchor Pith review Pith/arXiv arXiv 2026

  2. [2]

    RELATED WORK Fish re-identification studies typically build on well-established methods adapted from larger domains, such as vehicle and person re-identification. Existing approaches include clas- sification models with one class per identity [4, 1], metric- learning methods that train networks to generate discrimina- tive embeddings [5, 6, 7], and classi...

    work page

  3. [3]

    or the parr mark region [7]). However, no prior work in fish re-identification extracts multiple informative patches and combines them in an ensemble, nor does any method seg- ment texture-anchored patches by leveraging the salmon lat- eral line. Together, these gaps motivate our approach: a lateral line-anchored patch extraction method, an ensemble of co...

    work page

  4. [4]

    METHOD In this section, we describe our approach in the order of the data processing pipeline. We first outline the raw data col- lection procedure and the construction of the re-identification dataset (3.1–3.4), then describe the re-identification pipeline (3.5-3.6), and finally present the ensemble method (3.7) and the evaluation protocol (3.8). 3.1. Da...

    work page 2024

  5. [5]

    and the full-image baseline network (model 1, Table 1). To estimate the uncertainty of our calculated test and val- idation mAP values, we performed a non-parametric boot- strap over the per-query average precision (AP) scores with B= 50,000resamples. This yielded 95% confidence in- tervals (CI) for all models from the 2.5th and 97.5th per- centiles of th...

    work page

  6. [6]

    First, the ensemble approach outperforms the full-image baseline on the test (p <10 −4) and validation (p <10 −4) sets

    RESULTS Test and validation performance of our models (Table 1) to- gether with p-values for mAP differences (Section 3.8) reveal four main findings. First, the ensemble approach outperforms the full-image baseline on the test (p <10 −4) and validation (p <10 −4) sets. The performance gap between the ensemble and baseline is much larger on the test set th...

    work page

  7. [7]

    The lack of labeled data was addressed by using trajectory IDs as proxy labels, which in turn introduced trajectory-ID bias

    CONCLUSION In this work, we presented a salmon re-identification frame- work that addresses two central challenges of large-scale salmon re-identification: the lack of labeled data and the high accuracy requirements imposed by commercial net-pens. The lack of labeled data was addressed by using trajectory IDs as proxy labels, which in turn introduced traj...

    work page

  8. [8]

    Automated com- puter vision based individual salmon (salmo salar) breathing rate estimation (sabre) for improved state ob- servability,

    Espen Berntzen Høgstedt, Christian Schellewald, Rudolf Mester, and Annette Stahl, “Automated com- puter vision based individual salmon (salmo salar) breathing rate estimation (sabre) for improved state ob- servability,”Aquaculture, vol. 595, pp. 741535, 2025

    work page 2025

  9. [9]

    Computer vision based individual fish identification us- ing skin dot pattern,

    Petr Cisar, Dinara Bekkozhayeva, Oleksandr Movchan, Mohammadmehdi Saberioon, and Rudolf Schraml, “Computer vision based individual fish identification us- ing skin dot pattern,”Scientific Reports, vol. 11, no. 1, pp. 16904, 2021

    work page 2021

  10. [10]

    A multi-purpose tracking framework for salmon welfare monitoring in challeng- ing environments,

    Espen Uri Høgstedt, Christian Schellewald, Annette Stahl, and Rudolf Mester, “A multi-purpose tracking framework for salmon welfare monitoring in challeng- ing environments,” inProceedings of the IEEE/CVF International Conference on Computer Vision (ICCV) Workshops, 2025, pp. 2132–2141

    work page 2025

  11. [11]

    Efficient individual identification of ze- brafish using hue/saturation/value color model,

    Qussay Al-Jubouri, RJ Al-Azawi, Majid Al-Taee, and Iain Young, “Efficient individual identification of ze- brafish using hue/saturation/value color model,”The Egyptian Journal of Aquatic Research, vol. 44, no. 4, pp. 271–277, 2018

    work page 2018

  12. [12]

    Pigmentation-based visual learning for salvelinus fontinalis individual re- identification,

    Zhongliang Zhou, Nathaniel P. Hitt, Benjamin H. Letcher, Weili Shi, and Sheng Li, “Pigmentation-based visual learning for salvelinus fontinalis individual re- identification,” in2022 IEEE International Conference on Big Data (Big Data), 2022, pp. 6850–6852

    work page 2022

  13. [13]

    Fishnet: A unified embedding for salmon recognition,

    Bjørn Magnus Mathisen, Kerstin Bach, Espen Meidell, Håkon Måløy, and Edvard Schreiner Sjøblom, “Fishnet: A unified embedding for salmon recognition,”arXiv preprint arXiv:2010.10475, 2020

    work page arXiv 2010

  14. [14]

    Aging contrast: A contrastive learning framework for fish re- identification across seasons and years,

    Weili Shi, Zhongliang Zhou, Benjamin H. Letcher, Nathaniel Hitt, Yoichiro Kanno, Ryo Futamura, Os- amu Kishida, Kentaro Morita, and Sheng Li, “Aging contrast: A contrastive learning framework for fish re- identification across seasons and years,” inAI 2023: Advances in Artificial Intelligence. 2024, pp. 252–264, Springer Nature Singapore

    work page 2023

  15. [15]

    Re-identification of giant sunfish using keypoint matching,

    Malte Pedersen, Joakim Bruslund Haurum, Thomas B Moeslund, and Marianne Nyegaard, “Re-identification of giant sunfish using keypoint matching,” inProceed- ings of the Northern Lights Deep Learning Workshop, 2022, vol. 3

    work page 2022

  16. [16]

    14, 2016

    Renato B Dala-Corte, Júlia B Moschetta, and Fer- nando G Becker, “Photo-identification as a technique for recognition of individual fish: a test with the fresh- water armored catfish rineloricaria aequalicuspis reis & cardoso, 2001 (siluriformes: Loricariidae),”Neotropi- cal Ichthyology, vol. 14, 2016

    work page 2001

  17. [17]

    Re-identification of individuals from images using spot constellations: a case study in arctic charr (salvelinus alpinus),

    Ignacy T De ¸bicki, Elizabeth A Mittell, Bjarni K Kristjánsson, Camille A Leblanc, Michael B Morris- sey, and Kasim Terzi´c, “Re-identification of individuals from images using spot constellations: a case study in arctic charr (salvelinus alpinus),”Royal Society Open Science, vol. 8, no. 7, pp. 201768, 2021

    work page 2021

  18. [18]

    Consistent melanophore spot patterns allow long-term individual recognition of at- lantic salmon salmo salar,

    Lars Helge Stien, Jonatan Nilsson, Samantha Bui, Jan Erik Fosseidengen, Tore S Kristiansen, Øyvind Øverli, and Ole Folkedal, “Consistent melanophore spot patterns allow long-term individual recognition of at- lantic salmon salmo salar,”Journal of Fish Biology, vol. 91, no. 6, pp. 1699–1712, 2017

    work page 2017

  19. [19]

    Ultra- lytics yolov8,

    Glenn Jocher, Ayush Chaurasia, and Jing Qiu, “Ultra- lytics yolov8,” 2023

    work page 2023

  20. [20]

    The OpenCV Library,

    G. Bradski, “The OpenCV Library,”Dr. Dobb’s Journal of Software Tools, 2000

    work page 2000

  21. [21]

    Albumentations: fast and flexible image augmentations,

    A. Buslaev, A. Parinov, E. Khvedchenya, V . I. Iglovikov, and A. A. Kalinin, “Albumentations: fast and flexible image augmentations,”ArXiv e-prints, 2018

    work page 2018

  22. [22]

    Oriane Siméoni, Huy V V o, Maximilian Seitzer, Fed- erico Baldassarre, Maxime Oquab, Cijo Jose, Vasil Khalidov, Marc Szafraniec, Seungeun Yi, Michaël Ramamonjisoa, et al., “Dinov3,”arXiv preprint arXiv:2508.10104, 2025

    work page internal anchor Pith review Pith/arXiv arXiv 2025

  23. [23]

    Smooth-ap: Smoothing the path to- wards large-scale image retrieval,

    Andrew Brown, Weidi Xie, Vicky Kalogeiton, and An- drew Zisserman, “Smooth-ap: Smoothing the path to- wards large-scale image retrieval,” inEuropean confer- ence on computer vision. Springer, 2020, pp. 677–694

    work page 2020

  24. [24]

    Arcface: Additive angular margin loss for deep face recognition,

    Jiankang Deng, Jia Guo, Niannan Xue, and Stefanos Zafeiriou, “Arcface: Additive angular margin loss for deep face recognition,” inProceedings of the IEEE/CVF conference on computer vision and pattern recognition, 2019, pp. 4690–4699

    work page 2019

  25. [25]

    Multi-similarity loss with gen- eral pair weighting for deep metric learning,

    Xun Wang, Xintong Han, Weilin Huang, Dengke Dong, and Matthew R Scott, “Multi-similarity loss with gen- eral pair weighting for deep metric learning,” inPro- ceedings of the IEEE/CVF conference on computer vi- sion and pattern recognition, 2019, pp. 5022–5030

    work page 2019

  26. [26]

    Pytorch metric learning,

    Kevin Musgrave, Serge Belongie, and Ser-Nam Lim, “Pytorch metric learning,”arXiv preprint arXiv:2008.09164, 2020

    work page arXiv 2008

  27. [27]

    Decoupled Weight Decay Regularization

    Ilya Loshchilov and Frank Hutter, “Decoupled weight decay regularization,”arXiv preprint arXiv:1711.05101, 2017

    work page internal anchor Pith review Pith/arXiv arXiv 2017

  28. [28]

    SGDR: Stochastic Gradient Descent with Warm Restarts

    Ilya Loshchilov and Frank Hutter, “Sgdr: Stochastic gradient descent with warm restarts,”arXiv preprint arXiv:1608.03983, 2016

    work page internal anchor Pith review Pith/arXiv arXiv 2016

  29. [29]

    Tables 3, 4 and 5 report the test mAP for model 10 across different parameter settings

    SUPPLEMENTAL MATERIALS Ablations Table 2 presents the test mAP for model 10 (ensemble sliced) when each patch is removed in turn, showing that the perfor- mance decreases when a patch is removed. Tables 3, 4 and 5 report the test mAP for model 10 across different parameter settings. Within the rangesλ∈[0,0.8], τ∈[0.7,2.0], andk∈[20,500](which are sub-rang...

    work page