REVIEW 3 major objections 5 minor 38 references
SLAP: Selective Local Vision-Language Alignment for Fish Re-Identification via Partial Optimal Transport
T0 review · 3 major / 5 minor · reviewed 2026-08-14 · deepseek-v4-flash
Pith's one-line read Selective local alignment, not global image-text matching, lifts fish re-ID accuracy.
desk verdict Clean experiments wrapped around a cost matrix that is currently undefined; the POT mechanism is not established without a patch projection or dimension fix. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The load-bearing mechanism is the Partial Optimal Transport alignment loss. Visual patch embeddings and identity-aware prompt embeddings define a cosine-distance cost matrix; the transport plan is optimized with relaxed marginal constraints, an entropy regularizer, and a transported mass ratio that controls how much of the visual area can be matched. This plan gives soft prompt-to-patch correspondences that concentrate supervision on the strongest cross-modal matches. It is trained jointly with batch-hard triplet losses and auxiliary identity classification, while LoRA adapts the visual encoder and learnable prompt tokens adapt the text encoder. At inference only the LoRA-adapted visual encoder remains, so the selective alignment adds no deployment compute.
What would settle it
Inspect the learned transport plan on Melops test images: if the transported mass does not concentrate on identity-relevant regions such as the operculum, eye, and snout, or if replacing POT with uniformly random patch selection of equal mass produces no drop in mAP, then the selective-alignment explanation is not supported.
Extended reading notes
Core claim
The paper's central claim is that conventional global alignment between a whole image embedding and a text embedding is a bottleneck for fine-grained animal re-identification, because it lets background and weakly discriminative regions share in cross-modal supervision. SLAP instead computes a transport plan between local visual patch embeddings and multiple identity-aware prompt embeddings, with partially relaxed marginal constraints so that only a fraction of patches need participate. On the Melops dataset this selective local alignment outperforms all compared CLIP-based methods on every reported metric, with the largest relative gains under the open-set protocol where test identities are partly unseen. Ablations show the POT formulation beats no cross-modal supervision, global alignment, and full optimal transport, indicating that both locality and selectivity contribute to the improvement.
Load-bearing premise
The method assumes that after LoRA adaptation, visual patch embeddings and identity-aware prompt embeddings lie in a shared space where cosine distance reliably marks identity-relevant body regions, so that the transport plan selects informative patches and suppresses background.
Editorial extensions
If this is right
- If the central claim holds, CLIP-based ReID methods should move away from single global image-text alignment when identity cues are localized.
- The open-set improvements suggest selective local alignment transfers better to previously unseen identities than global alignment, which matters for long-term wildlife monitoring.
- The method retains a purely visual inference pipeline, so practitioners can adopt it without additional text or transport computation at deployment time.
- The ablation pattern indicates that triplet supervision, POT-based alignment, and auxiliary identity loss each play distinct roles, meaning future methods should treat selective cross-modal alignment as complementary to metric learning rather than a replacement.
- Consistent gains on sea star and sea turtle datasets suggest the benefit is not specific to fish or to the Melops collection protocol.
Reading between the lines
- If the proposed mechanism is genuinely driving the gains, a testable extension is to visualize the learned transport plan and check whether mass concentrates on the operculum, eye, and snout regions the paper cites as identity-relevant; that would confirm the selectivity interpretation rather than a generic regularization effect.
- A stronger stress test would compare POT against random or attention-based patch selection with the same transported mass; if random selection performs equally, the paper's selectivity story would need revision.
- The same selective alignment idea may transfer to other fine-grained recognition tasks where discriminative information is localized, such as person re-identification or vehicle re-identification, though the paper does not test those settings.
- The paper does not directly verify that LoRA-adapted patch embeddings and prompt embeddings reside in a shared space where cosine distance marks identity-relevant locality; that premise is inferred only from downstream retrieval gains.
Signed reviews
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper proposes SLAP, a CLIP-based framework for fish re-identification that replaces global image-text alignment with selective local alignment between visual patch embeddings and multiple identity-aware prompt embeddings, computed via Partial Optimal Transport (POT). The visual encoder is a frozen CLIP ViT-B/16 with LoRA adapters; learnable prompt context tokens adapt the frozen text encoder. The POT loss is trained jointly with triplet and classification losses, and at inference only the adapted image encoder is retained. Experiments on the Melops dataset under closed-set and open-set protocols report consistent improvements over several CLIP-based ReID baselines, with ablations showing a monotone progression from no cross-modal supervision to global alignment to full OT to partial OT. Additional experiments on SeaStarReID2023 and SeaTurtleID2022 report higher mAP than the same baselines.
Significance. If the reported results hold, the paper makes a useful contribution to animal re-identification by introducing a principled selective-alignment mechanism into vision-language ReID and by demonstrating its practical value on a challenging longitudinal fish dataset. The experimental methodology is a strength: five random seeds are used, all methods share the same backbone, data splits, optimizer, and preprocessing, and the central ablation in Table 2 isolates the alignment strategy. The parameter-efficient design (LoRA + prompt tuning, visual-only inference) is also well motivated. However, the core computational step as written is not well-defined because the cost matrix compares vectors of different dimensions, and the paper does not provide direct evidence that the transport plan actually selects identity-discriminative body regions rather than acting as an unstructured regularizer. These issues must be resolved before the central claims can be accepted.
major comments (3)
- [Section 3, Partial Optimal Transport; Visual Feature and Prompt Encoding] The cost matrix C_ik = 1 - v_i^T t_k^y / (||v_i|| ||t_k^y||) is not well-defined as written. For CLIP ViT-B/16, the local patch embeddings V have dimension d_v = 768, while the prompt embeddings T_y have dimension d_t = 512. The paper explicitly projects only the global representation f_v via W_proj in R^{d_v x d_t} and never defines a projection for patch tokens. The sentence claiming that V and T_y 'naturally reside in the shared vision-language embedding space' is not sufficient, because CLIP's shared space is learned for the global [CLS] representation after the visual projection, not for raw patch tokens. This is a load-bearing issue: if the implementation actually projects patch tokens to d_t, that projection must be described (including its parameters and whether it is trainable), and if it does not, the reported POT-based training cannot be reproduced from the paper. Please clarify and, if necessary, re-run the experiments with the corrected formulation.
- [Section 3 and Section 4, Table 2] The paper attributes the gains of POT over OT and global alignment to selective suppression of background and weakly discriminative regions, but no direct evidence of selectivity is provided. The ablation in Table 2 shows monotone improvements, yet those improvements could also arise from a different effective loss landscape or from partial mass truncation acting as a regularizer. To support the mechanistic claim, the authors should provide an analysis of the learned transport plan, for example by visualizing average transport mass per image patch, comparing mass allocated to foreground versus background regions, or showing that the selected patches correspond to biologically plausible features such as the operculum, eye, and snout. Without such evidence, the title and central interpretation of the method overstate what the current experiments establish.
- [Section 4, Evaluation Protocols] The open-set protocol description is underspecified in a way that affects interpretation of the results. The text states that approximately 20% of test identities are excluded from training while query and gallery contain both seen and unseen identities, but it does not report how many unseen identities are used, how their query and gallery images are balanced, or how mAP and Rank-1 are computed when an unseen identity may appear only once. Please provide these details, because the claimed generalization benefit of POT under the open-set protocol depends on the precise composition of the query and gallery sets.
minor comments (5)
- [Section 4, Table 4] The additional dataset results report only mAP without standard deviations or the number of seeds, and the text states improvements of +1.7 and +1.9 percentage points without statistical comparison; please report variability or clearly label these as single-run results.
- [Section 3, Learnable Prompt-Based Text Representation] The identity token is described both as 'fixed identity-specific token embedding' and as 'shared across all samples belonging to the same identity, remains fixed throughout training,' but the initialization and construction of this token are not specified; please clarify how the identity token is obtained and why it is not optimized.
- [Section 4, Implementation Details] The text states that the transported mass ratio was set to rho = 0.8 'based on an ablation study,' but no table or figure for that ablation appears in the main text or supplementary material; please include the full rho sensitivity results.
- [References] Several references are incomplete: reference [1] lacks author names, and references [4], [23], and [26] use 'et al.' without listing the full author list; please complete the bibliographic entries.
- [Abstract and Section 1] There are minor typographical and formatting issues, including a line break in the abstract ('demon-stratethegeneralizationcapability') and irregular spacing in a few places; a careful proofread would improve readability.
Circularity Check
No load-bearing circularity: the POT-vs-global improvement is established by an internally controlled ablation, and self-citations serve only as baselines and hyperparameter sources.
full rationale
The derivation chain is self-contained: the central claim that POT-based selective local alignment improves over global image-text alignment is established by the paper's own controlled ablations (Table 2), which compare No CM, Global Alignment, OT, and POT under identical optimization settings, and by the loss-component ablation (Table 3). The Global Alignment row is numerically identical to the MetaPrompt-ReID baseline with metadata conditioning disabled, so the comparison against the authors' prior work is an internal control rather than an imported ground truth. Self-citations such as [30] and [26] appear as hyperparameter sources (LoRA choice, lambda=0.1, K=4, prompt layouts) and as baseline implementations, but these choices are not used to derive the reported improvements; the POT effect is isolated by ablation. The Melops dataset [27,28] is the authors' own, but generalization is checked on SeaStarReID2023 and SeaTurtleID2022 with fixed hyperparameters, providing external falsifiability. No equation reduces to its inputs: the partial mass constraint m=rho*min(...) literally defines 'selective' transport, but the empirical claim that selectivity improves retrieval is an ablation-supported observation, not a consequence of the definition. The paper does assert, without proof, that 768-d patch embeddings V and 512-d prompt embeddings T_y 'naturally reside in the shared vision-language embedding space' despite no projection being specified for patch tokens; this is a technical validity concern about the POT cost matrix, not a circularity, because the assertion is not used to define the measured outcome and the reported gains are empirical rather than derived from that premise.
Assumptions & free parameters
free parameters (5)
- Transported mass ratio rho =
0.8
- Entropy regularization lambda =
0.1
- Loss balancing weights lambda_tri and lambda_pot =
not reported
- LoRA configuration (rank r, scaling alpha) =
r=16, alpha=16
- Number of prompts K and context tokens M =
K=4, M=4
assumptions (4)
- domain assumption Identity-discriminative cues in corkwing wrasse are localized in specific morphological regions such as operculum, eye, and snout.
- domain assumption CLIP patch embeddings and identity prompt embeddings lie in a shared space where cosine distance reflects identity-relevant similarity after LoRA adaptation.
- standard math The Sinkhorn algorithm with entropic regularization converges to a useful transport plan for gradient-based training.
- domain assumption Using a per-sample transport plan as a training loss, rather than as a prediction target, provides gradients that improve the visual encoder.
Cite this review
Pith. "Pith review of SLAP: Selective Local Vision-Language Alignment for Fish Re-Identification via Partial Optimal Transport." pith.science (2026). https://pith.science/paper/NT2FRMCL
@misc{pith2026260808840,
author = {Pith},
title = {Pith review of: SLAP: Selective Local Vision-Language Alignment for Fish Re-Identification via Partial Optimal Transport},
year = {2026},
howpublished = {\url{https://pith.science/paper/NT2FRMCL}},
note = {Machine review of arXiv:2608.08840}
}
read the original abstract
Individual fish re-identification (ReID) is a fine-grained recognition problem in which identity-discriminative cues are often localized to specific body regions rather than distributed uniformly across the animal. Nevertheless, recent CLIP-based ReID methods rely predominantly on global image-text alignment, allowing background and weakly discriminative regions to contribute to cross-modal supervision. We propose a selective local vision-language alignment framework that establishes localized correspondences between visual patch embeddings and multiple identity-aware prompt embeddings through Partial Optimal Transport (POT). Rather than enforcing exhaustive correspondence, POT enables selective matching between visual patches and prompt embeddings, allowing the model to emphasize the strongest cross-modal correspondences while avoiding forced alignment of weakly matching regions, thereby yielding more discriminative visual representations for retrieval. The framework is trained end-to-end, while only the adapted visual encoder is retained during inference. Experiments on the longitudinal Symphodus melops dataset demonstrate consistent improvements over recent CLIP-based ReID methods under both closed-set and open-set evaluation protocols. Additional evaluations on other datasets further demonstrate the generalization capability of the proposed method across diverse marine ReID benchmarks.
Figures
Reference graph
Works this paper leans on
-
[1]
From species-specific models to universal re-id: a survey of animal re-identification. Information Fusion133, 104323 (2026) SLAP for Fish Re-Identification 17 T able 7:Comparison of trainable parameters and inference time. Method Trainable Parameters (M) Inference (ms) Full FT 150.0 8.0 CLIP-ReID [16] 154.3 8.0 IndivAID [34] 154.3 8.0 ReID-AW [12] 155.0 8...
work page 2026
-
[2]
Adam, L., Čermák, V., Papafitsoros, K., Picek, L.: Seaturtleid2022: A long-span dataset for reliable sea turtle re-identification. In: IEEE WACV (2024)
work page 2024
-
[3]
arXiv preprint arXiv:2408.12934 (2024)
Cermak, V., Picek, L., Adam, L., Neumann, L., Matas, J.: Wildfusion: In- dividual animal identification with calibrated similarity fusion. arXiv preprint arXiv:2408.12934 (2024)
arXiv 2024
-
[4]
arXiv preprint arXiv:2210.01253 (2022)
Chen, G., et al.: Plot: Prompt learning with optimal transport for vision-language models. arXiv preprint arXiv:2210.01253 (2022)
arXiv 2022
-
[5]
Expert Systems with Applications293, 128466 (2025)
Compte, A., Yan, Y., Cortés, X., Escalera, S., Jacques-Junior, J.C.: Housed pig identification and tracking for precision livestock farming. Expert Systems with Applications293, 128466 (2025)
work page 2025
-
[6]
Ellis, B.A., Sørdalen, T.K., Briffa, M., Skiftesvik, A.B., Wilson, A.D.M., Halvorsen, K.T.: Visual cues elicit differential aggression towards female and female mimics in the corkwing wrasse. Behavioral Ecology p. arag022 (Feb 2026).https://doi. org/10.1093/beheco/arag022
-
[7]
In: CVPR
Feng, C., et al.: Ot-filter: An optimal transport filter for learning with noisy labels. In: CVPR. pp. 16164–16174 (2023)
2023
-
[8]
In: International conference on machine learning
Houlsby, N., Giurgiu, A., Jastrzebski, S., Morrone, B., De Laroussilhe, Q., Ges- mundo, A., Attariyan, M., Gelly, S.: Parameter-efficient transfer learning for nlp. In: International conference on machine learning. pp. 2790–2799. PMLR (2019)
2019
Show all 38 references
-
[9]
In: ICLR (2022)
Hu, E.J., Shen, Y., Wallis, P., Allen-Zhu, Z., Li, Y., Wang, S., Wang, L., Chen, W.: Lora: Low-rank adaptation of large language models. In: ICLR (2022)
2022
-
[10]
Expert Systems with Applications264, 125653 (2025)
Huang,J.,Yu,X.,An,D.,Ning,X.,Liu,J.,Tiwari,P.:Uniformityanddeformation: A benchmark for multi-fish real-time tracking in the farming. Expert Systems with Applications264, 125653 (2025)
2025
-
[11]
In: European conference on computer vision
Jia, M., Tang, L., Chen, B.C., Cardie, C., Belongie, S., Hariharan, B., Lim, S.N.: Visual prompt tuning. In: European conference on computer vision. pp. 709–727. Springer (2022)
2022
-
[12]
Neurips36, 40042–40053 (2023)
Jiao, B., Liu, L., Gao, L., Wu, R., Lin, G., Wang, P., Zhang, Y.: Toward re- identifying any animal. Neurips36, 40042–40053 (2023)
2023
-
[13]
In: AAAI
Li, B., Shi, Y., Yu, Q., Wang, J.: Unsupervised cross-domain image retrieval via prototypical optimal transport. In: AAAI. vol. 38, pp. 3009–3017 (2024)
2024
-
[14]
In: CVPR
Li, H., et al.: Global and local prompts cooperation via optimal transport for federated learning. In: CVPR. pp. 12151–12161 (2024)
2024
-
[15]
In: ACM MM (2020)
Li, S., Li, J., Tang, H., Qian, R., Lin, W.: Atrw: A benchmark for amur tiger re-identification in the wild. In: ACM MM (2020)
2020
-
[16]
In: Proceedings of the AAAI conference on artificial intelligence
Li, S., Sun, L., Li, Q.: Clip-reid: exploiting vision-language model for image re- identification without concrete text labels. In: Proceedings of the AAAI conference on artificial intelligence. vol. 37, pp. 1405–1413 (2023) 18 C. Beyan et al
2023
-
[17]
In: Proceedings of the 33rd ACM International Conference on Multimedia
Li, Y., Zhao, D., Qiao, T., Wu, Y., Pang, B., Koh, Y.S.: Metawild: A multimodal dataset for animal re-identification with environmental metadata. In: Proceedings of the 33rd ACM International Conference on Multimedia. pp. 13009–13015 (2025)
2025
-
[18]
Expert Systems with Applications 238, 122194 (2024)
Liu, Y., Li, B., Zhou, X., Li, D., Duan, Q.: Fishtrack: Multi-object tracking method for fish using spatiotemporal information fusion. Expert Systems with Applications 238, 122194 (2024)
2024
-
[19]
arXiv preprint arXiv:1711.05101 (2017)
Loshchilov, I., Hutter, F.: Decoupled weight decay regularization. arXiv preprint arXiv:1711.05101 (2017)
2017 arXiv
-
[20]
Methods in Ecology and Evolution5(12), 1294–1304 (2014)
McClintock, B.T., Hill, J.M., Fritz, L., Chumbley, K., Luxa, K., Diefenbach, D.R.: Mark-resight abundance estimation under incomplete identification of marked in- dividuals. Methods in Ecology and Evolution5(12), 1294–1304 (2014)
2014
-
[21]
In: IEEE WACVw (2020)
Nepovinnykh, E., Eerola, T., Kalviainen, H.: Siamese network based pelage pattern matching for ringed seal re-identification. In: IEEE WACVw (2020)
2020
-
[22]
Foundations and Trends in Machine Learning11(5-6), 355–607 (2019)
Peyré, G.e.a.: Computational optimal transport: With applications to data science. Foundations and Trends in Machine Learning11(5-6), 355–607 (2019)
2019
-
[23]
In: ICLR (2023)
Phatak, A., et al.: Computing all optimal partial transports. In: ICLR (2023)
2023
-
[24]
In: ICML
Radford, A., Kim, J.W., Hallacy, C., et al.: Learning transferable visual models from natural language supervision. In: ICML. pp. 8748–8763 (2021)
2021
-
[25]
Computer Science Review (2020)
Ravoor, P., T.s.b., S.: Deep learning methods for multi-species animal re- identification and tracking: a survey. Computer Science Review (2020)
2020
-
[26]
In: International Conference on Image Analysis and Processing
Shiri, M., Beyan, C., Murino, V.: Madpot: Medical anomaly detection with clip adaptation and partial optimal transport. In: International Conference on Image Analysis and Processing. pp. 247–259. Springer (2025)
2025
-
[27]
Scientific data (2026)
Sordalen, T.K., Malde, K., Skiftesvik, A.B., Sauvaitre, C., Beyan, C., Larsen, T.and Halvorsen, K.T.: A wild fish image dataset for individual re-identification and phenotyping. Scientific data (2026)
2026
-
[28]
Zenodo (2025).https://doi.org/10.5281/zenodo.17099925, https://doi.org/10.5281/zenodo.17099925, version 1.0
Sørdalen, T.K., Halvorsen, K.T.: MelopsReID: A wild fish image dataset for re-identification. Zenodo (2025).https://doi.org/10.5281/zenodo.17099925, https://doi.org/10.5281/zenodo.17099925, version 1.0
2025 doi
-
[29]
arXiv:2201.09366 (2022)
Tu, R., et al.: Optimal transport for causal discovery. arXiv:2201.09366 (2022)
2022 arXiv
-
[30]
Expert Systems with Applications332, 133618 (2026)
Tur, A.O., Sørdalen, T.K., Halvorsen, K.T., Beyan, C.: Parameter-efficient vision-language adaptation with continuous metadata conditioning for animal re- identification. Expert Systems with Applications332, 133618 (2026)
2026
-
[31]
338 (2008)
Villani, C., et al.: Optimal transport: old and new, vol. 338 (2008)
2008
-
[32]
Methods in Ecology and Evolution15(2), 373–387 (2024)
Wahltinez, O., Wahltinez, S.J.: An open-source general purpose machine learning framework for individual animal re-identification using few-shot learning. Methods in Ecology and Evolution15(2), 373–387 (2024)
2024
-
[33]
Pattern Recognition110(2021)
Wu, W., Tao, D., Li, H., Yang, Z., Cheng, J.: Deep features for person re- identification on metric learning. Pattern Recognition110(2021)
2021
-
[34]
arXiv preprint arXiv:2410.22927 (2024)
Wu, Y., Zhao, D., Zhang, J., Koh, Y.S.: An individual identity-driven framework for animal re-identification. arXiv preprint arXiv:2410.22927 (2024)
2024 arXiv
-
[35]
In: Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers)
Zaken, E.B., Goldberg, Y., Ravfogel, S.: Bitfit: Simple parameter-efficient fine- tuning for transformer-based masked language-models. In: Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers). pp. 1–9 (2022)
2022
-
[36]
In: Euro- pean Conference on Computer Vision (ECCV) (2022)
Zhang, R., Fang, R., Zhang, W., Gao, P., Li, K., Dai, J., Qiao, Y., Li, H.: Tip- adapter: Training-free clip-adapter for better vision-language modeling. In: Euro- pean Conference on Computer Vision (ECCV) (2022)
2022
-
[37]
In: CVPR (2022) SLAP for Fish Re-Identification 19
Zhou, K., Yang, J., Loy, C.C., Liu, Z.: Learning to prompt for vision-language models. In: CVPR (2022) SLAP for Fish Re-Identification 19
2022
-
[38]
In: IEEE WACV (2024)
Čermák, V., Picek, L., Adam, L., Papafitsoros, K.: Wildlifedatasets: An open- source toolkit for animal re-identification. In: IEEE WACV (2024)
2024
Reviewed August 14, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.