REVIEW 2 major objections 5 minor 48 references
Whether a frozen vision backbone applies global attention at high resolution decides if raising input size improves fine-grained segmentation under low-rank adaptation.
Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →
T0 review · grok-4.5
2026-07-12 07:34 UTC pith:LPPJXKLT
load-bearing objection Clean controlled study: under fixed LoRA, global attention on a high-res token grid predicts whether resolution keeps buying accuracy; the claim is real and correctly scoped to attention-level adapters. the 2 major comments →
When Does Resolution Help a Frozen Backbone? Global Attention at Resolution Predicts Scalable Adaptation for Camouflaged and Marine Animal Segmentation
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
Core claim
Under one fixed low-rank adaptation pipeline, whether a frozen backbone applies global attention to a high-resolution token set predicts whether raising input resolution improves fine-grained segmentation accuracy. Isotropic ViTs keep improving with resolution; hierarchical backbones saturate by roughly 512 px. Pooling an isotropic grid preserves the scaling benefit; removing global attention removes most of it. The dichotomy is specific to attention-level (LoRA) adaptation rather than every adapter.
What carries the argument
Global attention at resolution: the property that an isotropic ViT attends globally over the full high-resolution token grid at every depth, whereas hierarchical backbones reach global stages only after pooling (or never apply global attention). This property, not capacity, pretraining, or token-grid preservation alone, carries the resolution-scaling dichotomy under LoRA.
Load-bearing premise
The observed scaling rule is treated as a general backbone-selection guide even though the paper itself shows it holds under low-rank attention adaptation and can reverse under a different feature-space adapter.
What would settle it
Repeat the identical resolution sweep under the same LoRA pipeline on a new isotropic backbone whose global attention has been fully replaced by local windows (or on a new hierarchical backbone that applies true global attention before pooling); if the isotropic model still scales or the hierarchical one no longer saturates, the causal claim fails.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper argues that, for low-rank adaptation of frozen vision foundation models on fine-grained binary segmentation (COD, MAS, SOD), a single structural property predicts whether resolution converts into accuracy: whether the backbone applies global attention to a high-resolution token set. Under a fixed lightweight pipeline (SALT: rank-8 LoRA + convolutional neck + attention-gated U-Net + CNN side-stem), three isotropic ViTs (SAM3 PE, DINOv2, DINOv3) improve monotonically with resolution while three hierarchical backbones (Hiera-L, Hiera-B+, Swin-L) saturate near 512 px (Table 5, Fig. 1). Confound controls (capacity, objective, pretraining) and direct edits on DINOv2/DINOv3 show that pooling the grid preserves scaling whereas removing global attention collapses it (Table 6); the dichotomy is specific to attention-level (LoRA) adaptation and disappears under a feature-space bottleneck (Table 7). With SALT-DINOv3 the method sets new SOTA on both MAS benchmarks (MAS3K mIoU 0.878) and obtains the best or tied S-measure on the four COD and five SOD sets in a single RGB pass at 672 px.
Significance. If the result holds, the work supplies a concrete, falsifiable backbone-selection rule for practitioners performing LoRA-style PEFT on resolution-sensitive dense tasks, grounded in a controlled six-backbone sweep rather than a new module. Credit is due for the isolation of geometry (pipeline held fixed), the three confound controls, the causal attention-editing experiments on two isotropic models, the FLOPs/throughput reporting that turns the dichotomy into a cost-accuracy trade-off, the isolation study showing that pretrained features alone underperform prior work, and the strong external-benchmark results (new SOTA on MAS3K/RMAS, best S_m on all four COD sets). The authors carefully scope the mechanism to low-rank adaptation and acknowledge residual correlational aspects of the isotropic-versus-hierarchical comparison. This is a useful empirical contribution to understanding when frozen VFMs remain adaptable under lightweight fine-grained adaptation.
major comments (2)
- [Abstract, Sec. 1 contributions, Sec. 5.2, Table 7, Limitations, Sec. 6] The practical backbone-selection rule ('choose an isotropic ViT that applies global attention at high resolution') is stated in the abstract, contributions list, and conclusion, yet Table 7 and Sec. 5.2 show that a matched-rank feature-space bottleneck adapter lets the hierarchical Swin-L scale (+0.034) while the LoRA dichotomy vanishes. Although the Limitations section scopes the mechanism to LoRA, the framing of the rule itself remains broader. The manuscript should qualify the guidance as LoRA-/attention-level-specific in the abstract, introduction, and conclusion, and briefly discuss when a practitioner might prefer hierarchical geometry plus a feature-space adapter.
- [Table 6, Sec. 5.2, Limitations] Several load-bearing attention edits in Table 6 (fully windowed DINOv2, pool-after-block-2, reconstructed Hiera pattern) remain single-seed, while the unedited baselines and the DINOv3 no-global rows are two-seed means. The causal claim that 'removing global attention collapses scaling' rests on these slope reversals (e.g., +0.001 vs +0.030). Multi-seed means (or at least seed-band reporting) for every edited configuration, plus a short clarification of why the reconstructed hierarchical pattern saturates on DINOv2 but not DINOv3 (already noted in Limitations), would make the causal account more robust without changing the experimental design.
minor comments (5)
- [Table 5, Sec. 4.3] Resolution tiers are matched by design (336/504/672 vs 352/512/704) but are not pixel-identical. A one-sentence note of the resulting token-grid sizes (e.g., 24/36/48 vs the corresponding hierarchical stages) next to Table 5 would make the FLOPs/throughput comparison easier to interpret.
- [Table 1, Sec. 4.4] The component ablation (Table 1) is performed only at 336 px on the SAM3 PE. A single additional row confirming that LoRA remains the dominant term at the 672 px operating point used for the SOTA claims would strengthen the adaptation-isolation narrative.
- [Sec. 4.3, Sec. 6] Code, training configs, and the edited-backbone implementations are not mentioned as released. For a controlled empirical study whose primary value is the protocol and the attention edits, releasing the SALT pipeline and the Table-6 configurations would materially improve reproducibility.
- [Sec. 5.5, Fig. 3] The qualitative scan against BiRefNet maps (53 % / 48 % win rates on NC4K/CAMO) is helpful; a short quantitative summary of the IoU-delta distribution or one representative failure case would balance the high-margin successes shown in Fig. 3.
- [Sec. 3.4, Eq. (2)] Minor notation consistency: the structure loss is written L = L_w^BCE + L_w^IoU in Eq. (2) while the text sometimes refers to 'boundary-weighted' without restating the weighting scheme; a brief pointer to the source papers would help.
Circularity Check
No significant circularity: empirical backbone-resolution study on public benchmarks with fixed public checkpoints; no derivation that reduces to its inputs by construction.
full rationale
The paper's load-bearing claim is an empirical dichotomy under a fixed adapter (SALT/LoRA): isotropic ViTs that apply global attention over a high-resolution token grid keep converting input resolution into accuracy, while hierarchical backbones that confine global attention to post-pooling stages saturate near 512 px. That claim is established by a controlled six-backbone resolution sweep (Table 5, Fig. 1), confound controls (capacity, objective, pretraining), and causal edits that pool the grid or remove global attention on DINOv2/DINOv3 (Table 6), then evaluated on external public MAS/COD/SOD benchmarks with public foundation-model checkpoints. No parameter is fitted to a subset and re-reported as a prediction; global attention is not defined in terms of the scaling outcome; and the authors do not import a uniqueness theorem or ansatz from their own prior work. The adapter-specificity finding (Table 7: a feature-space bottleneck lets Swin-L scale) is an explicit scope limitation, not a circular reduction. Isolation of pretrained features alone (0.784 mIoU, below SAM2-UNet) further shows the result is not tautological with the backbone. The derivation chain is therefore self-contained measurement, not circular.
Axiom & Free-Parameter Ledger
free parameters (3)
- LoRA rank r and scale α =
r=8, α=16
- Resolution tiers (336/504/672 vs 352/512/704) =
three matched tiers
- SALT decoder capacity (2.7-5.6 M trainable params) =
2.7-5.6 M
axioms (3)
- domain assumption Isotropic ViTs maintain a uniform token grid and apply global self-attention in every block; hierarchical backbones (Hiera, Swin) use early local windows and pool before global stages.
- domain assumption Public pretrained checkpoints (SAM3 PE, DINOv2/v3, SAM2 Hiera, Swin-L) are representative of their geometry classes and can be frozen without further pretraining.
- ad hoc to paper The fixed SALT pipeline (LoRA + convolutional neck + attention-gated U-Net + CNN side-stem) does not itself favor one geometry; any residual neck asymmetry favors hierarchical stages.
invented entities (1)
-
SALT pipeline
no independent evidence
Cite this review
Pith. "Pith review of When Does Resolution Help a Frozen Backbone? Global Attention at Resolution Predicts Scalable Adaptation for Camouflaged and Marine Animal Segmentation." pith.science (2026). https://pith.science/paper/LPPJXKLT
@misc{pith2026260702708,
author = {Pith},
title = {Pith review of: When Does Resolution Help a Frozen Backbone? Global Attention at Resolution Predicts Scalable Adaptation for Camouflaged and Marine Animal Segmentation},
year = {2026},
howpublished = {\url{https://pith.science/paper/LPPJXKLT}},
note = {Machine review of arXiv:2607.02708}
}
read the original abstract
Adapting frozen vision foundation models to fine-grained segmentation now largely depends on backbone selection. Whether the backbone applies global attention to a high-resolution token set predicts whether a low-rank adapter turns resolution into accuracy. Isotropic ViTs attend globally over the full grid and keep improving with resolution; hierarchical backbones confine early attention to local windows and pool the grid before their global stages, plateauing at lower resolutions. A controlled six-backbone study establishes the pattern, and editing the backbone points to the cause: pooling keeps the benefit, removing global attention does not. The effect is specific to low-rank adaptation. Under one fixed pipeline, SALT (Side-stem, Attention-gated U-Net, Low-rank Tuning), one RGB-only pass on a strong isotropic backbone wins the best S-measure on the four data-matched camouflaged sets, and leads every marine and salient set. It reaches a new state of the art on both marine-animal benchmarks (MAS3K mIoU 0.878).
Figures
Reference graph
Works this paper leans on
-
[1]
Token merging: Your ViT but faster
Daniel Bolya, Cheng-Yang Fu, Xiaoliang Dai, Peizhao Zhang, Christoph Feichtenhofer, and Judy Hoffman. Token merging: Your ViT but faster. InICLR, 2023. 3
2023
-
[2]
SAM 3: Segment anything with concepts
Nicolas Carion, Laura Gustafson, Yuan-Ting Hu, et al. SAM 3: Segment anything with concepts. InICLR, 2026. 1, 2, 3, 4
2026
-
[3]
SAM-Adapter: Adapting segment anything in under- performed scenes
Tianrun Chen, Lanyun Zhu, Chaotao Ding, Runlong Cao, Yan Wang, Zejian Li, Lingyun Sun, Papa Mao, and Ying Zang. SAM-Adapter: Adapting segment anything in under- performed scenes. InICCV Workshops, 2023. 6
2023
-
[4]
Tianrun Chen, Lanyun Zhu, Chaotao Ding, et al. SAM2- Adapter: Evaluating and adapting segment anything 2 in downstream tasks.arXiv preprint arXiv:2408.04579, 2024. 1, 3
Pith/arXiv arXiv 2024
-
[5]
Vision transformer adapter for dense predictions
Zhe Chen, Yuchen Duan, Wenhai Wang, Junjun He, Tong Lu, Jifeng Dai, and Yu Qiao. Vision transformer adapter for dense predictions. InICLR, 2023. 2
2023
-
[6]
Structure- measure: A new way to evaluate foreground maps
Deng-Ping Fan, Ming-Ming Cheng, Yun Liu, et al. Structure- measure: A new way to evaluate foreground maps. InICCV,
-
[7]
Enhanced- alignment measure for binary foreground map evaluation
Deng-Ping Fan, Cheng Gong, Yang Cao, et al. Enhanced- alignment measure for binary foreground map evaluation. In IJCAI, 2018. 4
2018
-
[8]
Camouflaged object detection
Deng-Ping Fan, Ge-Peng Ji, Guolei Sun, et al. Camouflaged object detection. InCVPR, 2020. 1, 2, 4
2020
-
[9]
PraNet: Par- allel reverse attention network for polyp segmentation
Deng-Ping Fan, Ge-Peng Ji, Tao Zhou, et al. PraNet: Par- allel reverse attention network for polyp segmentation. In MICCAI, 2020. 4
2020
-
[10]
MASNet: A robust deep ma- rine animal segmentation network.IEEE Journal of Oceanic Engineering, 2023
Zhenqi Fu, Ruizhe Chen, Yue Huang, En Cheng, Xinghao Ding, and Kai-Kuang Ma. MASNet: A robust deep ma- rine animal segmentation network.IEEE Journal of Oceanic Engineering, 2023. 4, 6
2023
-
[11]
H2Former: An efficient hierarchical hybrid transformer for medical image segmentation.IEEE Transac- tions on Medical Imaging, 2023
Along He, Kai Wang, Tao Li, Chengkun Du, Shuang Xia, and Huazhu Fu. H2Former: An efficient hierarchical hybrid transformer for medical image segmentation.IEEE Transac- tions on Medical Imaging, 2023. 6
2023
-
[12]
Deep residual learning for image recognition
Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition. InCVPR, 2016. 2
2016
-
[13]
LoRA: Low- rank adaptation of large language models
Edward J Hu, Yelong Shen, Phillip Wallis, et al. LoRA: Low- rank adaptation of large language models. InICLR, 2022. 1, 3
2022
-
[14]
Segment anything
Alexander Kirillov, Eric Mintun, Nikhila Ravi, et al. Segment anything. InICCV, 2023. 1, 2, 6
2023
-
[15]
Anabranch network for camouflaged object segmentation
Trung-Nghia Le, Tam V Nguyen, Zhongliang Nie, et al. Anabranch network for camouflaged object segmentation. Computer Vision and Image Understanding, 2019. 2, 4
2019
-
[16]
Visual saliency based on multi- scale deep features
Guanbin Li and Yizhou Yu. Visual saliency based on multi- scale deep features. InCVPR, 2015. 4
2015
-
[17]
MAS3K: An open dataset for marine animal segmentation
Lin Li, Eric Rigall, Junyu Dong, and Geng Chen. MAS3K: An open dataset for marine animal segmentation. InInter- national Symposium on Benchmarking, Measuring and Op- timizing (Bench), 2020. 4
2020
-
[18]
The secrets of salient object segmentation
Yin Li, Xiaodi Hou, Christof Koch, et al. The secrets of salient object segmentation. InCVPR, 2014. 4
2014
-
[19]
Exploring plain vision transformer backbones for object de- tection
Yanghao Li, Hanzi Mao, Ross Girshick, and Kaiming He. Exploring plain vision transformer backbones for object de- tection. InECCV, 2022. 2, 3
2022
-
[20]
Swin transformer: Hierarchical vision transformer using shifted windows
Ze Liu, Yutong Lin, Yue Cao, Han Hu, Yixuan Wei, Zheng Zhang, Stephen Lin, and Baining Guo. Swin transformer: Hierarchical vision transformer using shifted windows. In ICCV, 2021. 2, 3, 4
2021
-
[21]
Decoupled weight decay regularization
Ilya Loshchilov and Frank Hutter. Decoupled weight decay regularization. InICLR, 2019. 4
2019
-
[22]
VSCode: Gen- eral visual salient and camouflaged object detection with 2d prompt learning
Ziyang Luo, Nian Liu, Wangbo Zhao, et al. VSCode: Gen- eral visual salient and camouflaged object detection with 2d prompt learning. InCVPR, 2024. 2
2024
-
[23]
Simultaneously lo- calize, segment and rank the camouflaged objects
Yunqiu Lv, Jing Zhang, Yuchao Dai, et al. Simultaneously lo- calize, segment and rank the camouflaged objects. InCVPR,
-
[24]
How to evaluate foreground maps? InCVPR, 2014
Ran Margolin, Lihi Zelnik-Manor, and Ayellet Tal. How to evaluate foreground maps? InCVPR, 2014. 4
2014
-
[25]
Vo, et al
Maxime Oquab, Timoth ´ee Darcet, Th´eo Moutakanni, Huy V. Vo, et al. DINOv2: Learning robust visual features without supervision.Transactions on Machine Learning Research (TMLR), 2024. 1, 2, 3, 4
2024
-
[26]
Zoom in and out: A mixed-scale triplet network for camouflaged object detection
Youwei Pang, Xiaoqi Zhao, Tian-Zhu Xiang, Lihe Zhang, and Huchuan Lu. Zoom in and out: A mixed-scale triplet network for camouflaged object detection. InCVPR, 2022. 6
2022
-
[27]
Zoom- NeXt: A unified collaborative pyramid network for camou- flaged object detection.IEEE TPAMI, 2024
Youwei Pang, Xiaoqi Zhao, Tian-Zhu Xiang, et al. Zoom- NeXt: A unified collaborative pyramid network for camou- flaged object detection.IEEE TPAMI, 2024. 2, 6
2024
-
[28]
Learning transferable visual models from natural language supervision
Alec Radford, Jong Wook Kim, Chris Hallacy, et al. Learning transferable visual models from natural language supervision. InICML, 2021. 1
2021
-
[29]
SAM 2: Segment anything in images and videos
Nikhila Ravi, Valentin Gabeur, Yuan-Ting Hu, et al. SAM 2: Segment anything in images and videos. InICLR, 2025. 1, 2, 3, 4
2025
-
[30]
U-Net: Convolutional networks for biomedical image segmentation
Olaf Ronneberger, Philipp Fischer, and Thomas Brox. U-Net: Convolutional networks for biomedical image segmentation. InMICCAI, 2015. 2, 4
2015
-
[31]
Hi- era: A hierarchical vision transformer without the bells-and- whistles
Chaitanya Ryali, Yuan-Ting Hu, Daniel Bolya, et al. Hi- era: A hierarchical vision transformer without the bells-and- whistles. InICML, 2023. 2
2023
-
[32]
Vo, Maximilian Seitzer, Federico Baldassarre, Maxime Oquab, et al
Oriane Sim ´eoni, Huy V. Vo, Maximilian Seitzer, Federico Baldassarre, Maxime Oquab, et al. DINOv3.arXiv preprint arXiv:2508.10104, 2025. 1, 2, 3, 4
Pith/arXiv arXiv 2025
-
[33]
Very deep convo- lutional networks for large-scale image recognition
Karen Simonyan and Andrew Zisserman. Very deep convo- lutional networks for large-scale image recognition. InICLR,
-
[34]
Animal camouflage analysis: CHAMELEON database
Przemys law Skurowski, Hassan Abdulameer, et al. Animal camouflage analysis: CHAMELEON database. InUnpub- lished, 2018. 2, 4
2018
-
[35]
Learning to detect salient objects with image-level supervision
Lijun Wang, Huchuan Lu, Yifan Wang, et al. Learning to detect salient objects with image-level supervision. InCVPR,
-
[36]
Pixels, regions, and objects: Multiple enhancement for salient object detection
Yi Wang, Ruili Wang, Xin Fan, et al. Pixels, regions, and objects: Multiple enhancement for salient object detection. InCVPR, 2023. 6
2023
-
[37]
F3Net: Fu- sion, feedback and focus for salient object detection
Jun Wei, Shuhui Wang, and Qingming Huang. F3Net: Fu- sion, feedback and focus for salient object detection. InAAAI,
-
[38]
Pyramid grafting network for one-stage high-resolution saliency de- tection
Chenxi Xie, Changqun Xia, Mingcan Ma, et al. Pyramid grafting network for one-stage high-resolution saliency de- tection. InCVPR, 2022. 4
2022
-
[39]
SAM3- UNet: Simplified adaptation of segment anything model 3
Xinyu Xiong, Zihuang Wu, Lei Lu, and Yufa Xia. SAM3- UNet: Simplified adaptation of segment anything model 3. arXiv preprint arXiv:2512.01789, 2025. 3, 6, 7
arXiv 2025
-
[40]
SAM2-UNet: Segment anything 2 makes strong encoder for natural and medical image segmentation.Visual Intelligence, 4(1):2, 2026
Xinyu Xiong, Zihuang Wu, Shuangyi Tan, Wenxue Li, Fei- long Tang, Ying Chen, Siying Li, Jie Ma, and Guanbin Li. SAM2-UNet: Segment anything 2 makes strong encoder for natural and medical image segmentation.Visual Intelligence, 4(1):2, 2026. 1, 2, 3, 4, 5, 6, 7
2026
-
[41]
Hierarchical saliency detection
Qiong Yan, Li Xu, Jianping Shi, and Jiaya Jia. Hierarchical saliency detection. InCVPR, 2013. 4
2013
-
[42]
MAS-SAM: Segment any marine animal with aggregated features
Tianyu Yan, Zifu Wan, Xinhao Deng, Pingping Zhang, Yang Liu, and Huchuan Lu. MAS-SAM: Segment any marine animal with aggregated features. InIJCAI, 2024. 2, 5, 6
2024
-
[43]
Saliency de- tection via graph-based manifold ranking
Chuan Yang, Lihe Zhang, Huchuan Lu, et al. Saliency de- tection via graph-based manifold ranking. InCVPR, 2013. 4
2013
-
[44]
FOCUS: Towards universal foreground segmentation
Zuyao You, Lingyu Kong, Lingchen Meng, and Zuxuan Wu. FOCUS: Towards universal foreground segmentation. In AAAI, 2025. 2
2025
-
[45]
Towards high-resolution salient object detection
Yi Zeng, Pingping Zhang, Jianming Zhang, et al. Towards high-resolution salient object detection. InICCV, 2019. 4
2019
-
[46]
Fantastic animals and where to find them: Segment any ma- rine animal with dual SAM
Pingping Zhang, Tianyu Yan, Yang Liu, and Huchuan Lu. Fantastic animals and where to find them: Segment any ma- rine animal with dual SAM. InCVPR, 2024. 6
2024
-
[47]
Bilateral reference for high-resolution dichotomous image segmenta- tion.CAAI Artificial Intelligence Research, 2024
Peng Zheng, Dehong Gao, Deng-Ping Fan, et al. Bilateral reference for high-resolution dichotomous image segmenta- tion.CAAI Artificial Intelligence Research, 2024. 1, 2, 4, 6, 7
2024
-
[48]
Salient object detection via integrity learning.IEEE TPAMI, 2023
Mingchen Zhuge, Deng-Ping Fan, Nian Liu, et al. Salient object detection via integrity learning.IEEE TPAMI, 2023. 6
2023
discussion (0)
Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.