Pith. sign in

REVIEW 3 major objections 5 minor 1 cited by

Benchmarking Feature Upsampling Methods for Vision Foundation Models using Interactive Segmentation

T0 review · 3 major / 5 minor · reviewed 2026-08-16 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read This paper argues that the choice of feature upsampler strongly shapes what frozen vision foundation models can do, and that LoftUp, a coordinate-based cross-attention upsampler, beats bilinear interpolation by up to 50 percent in…

desk verdict Useful benchmark for feature upsamplers on VFMs, but the LoftUp advantage rests on an unverified click-injection interaction. read the letter →

arxiv 2505.02075 v1 pith:XYM43UBG submitted 2025-05-04 cs.CV cs.AIcs.LG

Volodymyr Havrylov , Haiwen Huang , Dan Zhang , Andreas Geiger This is my paper · ORCID
classification cs.CVcs.AIcs.LG
keywords interactivesegmentationfeatureupsamplingvisionfoundationmodelsLoftUpcoordinate-basedcross-attentionclickinjectiondensepredictionbenchmarking
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper sets out to show that feature upsampling is not a minor detail: for frozen vision foundation models, the method used to restore full-resolution features can change downstream dense-prediction quality substantially. It proposes interactive segmentation, where a model turns an image plus user clicks into an object mask, as a sensitive benchmark for comparing feature upsamplers. Across four datasets and two click-encoding designs, the authors find that LoftUp consistently outperforms bilinear interpolation, LiFT, and FeatUp's joint bilateral upsampler, with gains up to 50 percent reported over bilinear interpolation. If the finding holds, it implies that upsampler quality, not just backbone quality, should be treated as a first-order factor when building dense predictors from foundation models.

What carries the argument

The load-bearing component is LoftUp, a coordinate-based cross-attention transformer that maps a low-resolution feature map to a full-resolution target by attending to the feature tokens from a continuous coordinate grid, trained against full-resolution pseudo-ground-truth features. The benchmark architecture around it freezes the vision backbone and the upsampler, trains only the click encoder and a three-layer segmentation head, and injects click features either early after patch embedding or late after upsampling. This setup isolates the upsampler's contribution and makes interactive segmentation a controlled probe of feature quality.

What would settle it

Re-run the benchmark with every upsampler under both early and late click injection and compare the full matrix. If bilinear interpolation or FeatUp's JBU matches or beats LoftUp under its preferred injection, the headline improvement would be due to click-injection choice rather than upsampling quality.

Watch

Extended reading notes

Core claim

The central claim is that a task-agnostic feature upsampler trained on full-resolution pseudo-ground-truth features, with global attention over a coordinate grid, can substantially improve the quality of frozen VFM features, and that interactive segmentation exposes these differences clearly. In the authors' benchmark, LoftUp outperforms the other upsamplers on every dataset and with both click encoders; for example, on GrabCut with the symmetric patch embedding encoder, the number of clicks needed to reach 80 percent IoU drops from 4.32 with bilinear feature upsampling to 1.72 with LoftUp. The paper interprets this as evidence that dense, global supervision during upsampler training matters more than local kernel prediction or fixed-factor upscaling.

Load-bearing premise

The paper assumes that the click-injection strategy chosen once with a single upsampler, early for one click encoder and late for the other, is also the best strategy for the other upsamplers it compares, so its ranking is not an artifact of mismatched click placement.

Editorial extensions

If this is right

  • Interactive segmentation can serve as a standard probing task for feature upsamplers, complementing existing dense-prediction benchmarks.
  • A frozen vision foundation model plus a strong upsampler plus a lightweight head can approach the performance of task-specific multiscale decoders with much lower training cost.
  • Upsampler design choices, especially global cross-attention and full-resolution supervision, matter more than local upsampling kernels for recovering fine detail.
  • Methods that only double resolution or rely on low-resolution pseudo-ground-truth targets are at a systematic disadvantage in this benchmark.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The early-versus-late click-injection choice was tuned using FeatUp's JBU alone; a natural extension is to re-run the benchmark with each upsampler under both injection strategies to confirm the ranking is not an artifact of that choice.
  • If the result generalizes, feature upsampling quality should matter just as much for other click- or prompt-driven dense tasks, such as open-vocabulary segmentation and interactive editing.
  • The small gains on SBD, the training distribution, hint that the benchmark's discriminative power may come mostly from out-of-distribution datasets; testing on held-out domains deliberately would make that explicit.
  • One could test whether LoftUp's advantage persists with larger backbones, since the coordinate-attention budget scales with feature tokens.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. The paper proposes interactive segmentation (IS) as a benchmark for evaluating feature upsampling methods on frozen vision foundation models (VFMs). It introduces a benchmark architecture that freezes the VFM and upsampler while training only a click encoder and a lightweight segmentation head, and it compares five upsampling strategies (low-resolution baseline, bilinear interpolation, LiFT, FeatUp's JBU, and LoftUp) on four IS datasets using NoC and IoU metrics. The main finding is that LoftUp yields the best performance, with up to roughly 50% improvement in NoC metrics over bilinear interpolation, and that appropriate upsamplers generally improve over the low-resolution baseline. The paper also reports unsuccessful explorations of multiscale FPN and multi-granularity configurations in the supplementary material.

Significance. If the main finding holds, the paper establishes interactive segmentation as a sensitive and inexpensive downstream probe for feature upsampler quality, complementing existing linear-probing benchmarks. The frozen-backbone protocol is simple and reproducible, and the code is released as iSegProbe. The performance gaps among upsamplers are large and visually interpretable. However, the central ranking currently depends on a click-injection protocol that was tuned using only FeatUp's JBU and was not independently validated for the other upsamplers; for this reason the specific significance of the LoftUp result is conditional on additional ablations.

major comments (3)
  1. [Sec. 5.2, Tab. 2] The optimal click-injection strategy (early for Symmetric Patch Embedding, late for SimpleViT) was determined using only FeatUp's JBU and then applied to all upsamplers in Tab. 1. Because LoftUp uses a coordinate-based cross-attention transformer while FeatUp and LiFT rely on locally predicted kernels, a method-specific interaction with click placement is plausible. An ablation of early vs. late injection for at least LoftUp and LiFT is needed to rule out that the reported ranking is an artifact of a protocol tuned to FeatUp.
  2. [Sec. 5.2, Tab. 2 (separate upsampling)] The paper reports that upsampling click features separately and merging them with upsampled image features improves FeatUp (NoC80 2.30 vs. 2.72 for late injection) but abandons this configuration for simplicity. Since this changes where click information enters the pipeline, it may interact with the upsampler architecture. The paper should either adopt this configuration uniformly across all upsamplers or provide an ablation showing that the main ranking in Tab. 1 is unchanged when separate upsampling is enabled for all methods.
  3. [Sec. 5.1, Tabs. 1-5] All quantitative results are single-run point estimates without confidence intervals or statistical tests. Even though the LoftUp improvements are large, differences between some conditions (e.g., Low-res vs. Bilinear on GrabCut) are small. For a benchmarking contribution that claims 'significant' improvements, at least three training seeds per configuration should be reported to establish that the ranking is not due to optimization noise.
minor comments (5)
  1. [Sec. A.2, Tab. 3] The conclusion that the multiscale benchmark is inferior is not fully supported because the LoftUp entries are marked as 'non-final checkpoints.' Please rerun with final checkpoints or state this limitation clearly in the main text if it affects the benchmark design choice.
  2. [Sec. 5.1] Clarify how NoC is computed when a model does not reach the target IoU within the maximum of 20 clicks (e.g., is NoC set to 20, or is the instance excluded?).
  3. [Sec. 5.2] The hypothesis that LiFT 'shows almost no learning' would be more convincing if accompanied by training loss curves or an intermediate feature analysis, since the observed degradation relative to the low-resolution baseline is unusually severe.
  4. [Author list and main text] Minor typos: 'V olodymyr' in the author list and 'DA VIS' should be 'DAVIS'.
  5. [Sec. 2] The citation of reference [45] for resize-convolution may be imprecise; Odena et al. focus primarily on deconvolution artifacts, so a more direct reference for resize-convolution would help readers.

Circularity Check

0 steps flagged · score 2.0 of 10

No circular derivation found: the benchmark is empirical, and the self-cited LoftUp is the method under test rather than load-bearing evidence; the injection-type selection is a fairness caveat, not a circular reduction.

full rationale

This paper is an empirical benchmark rather than a derivation chain, so the classic circularity patterns largely do not apply. The central claim — that LoftUp achieves up to 50% improvement over bilinear interpolation — is supported by direct measurements in Tab. 1 (NoC and IoU) on four datasets, with the backbone and upsamplers frozen and only the click encoder and segmentation head trained. No equation in the paper defines the reported performance in terms of the inputs by construction, and no fitted parameter is later renamed as a prediction. LoftUp [26] is a self-citation, but it is used as the experimental method being benchmarked, not as the evidence for the ranking; the ranking is established by the present paper's own experiments and is externally falsifiable by rerunning the protocol. The one structurally questionable decision is the click-injection type: early injection for the symmetric patch embedding encoder and late injection for SimpleViT were selected using FeatUp's JBU alone (Tab. 2) and then applied to all upsamplers. This raises a legitimate protocol-fairness concern — LoftUp's coordinate-based cross-attention may interact differently with click placement than FeatUp's local JBU — but it is a potential confound, not a circular step, because LoftUp's results are not forced by the selection. The score of 2 reflects the minor author-overlap self-citation and the protocol caveat, without any load-bearing circularity.

Assumptions & free parameters 2 free parameters · 3 assumptions · 0 invented entities

This is an empirical benchmark rather than a derivation, so the ledger records the design choices and domain assumptions that the comparison rests on. The main risk is that the click-injection selection and head choice may interact unevenly with upsamplers.

free parameters (2)
  • Segmentation head architecture = Conv Head (3x3 first two layers, inner channels 384)
    Chosen via ablation on ViT without upsampling (Tab. 4); assumed to transfer to all upsampler conditions.
  • Click injection strategy = Early for Symmetric Patch Embedding, late for SimpleViT
    Selected based on experiments with FeatUp's JBU (Tab. 2); assumed to be optimal for all upsamplers, including LoftUp.
assumptions (3)
  • domain assumption The RITM/SimpleClick click simulation protocol produces clicks that mimic natural human behavior
    Section 3, Training and Evaluation Protocols. The benchmark validity depends on this proxy for human interaction.
  • domain assumption Feature upsamplers pretrained on other tasks transfer to interactive segmentation when frozen
    Sections 4 and 5.2. The central comparison freezes upsamplers; if transfer is uneven, the ranking may not reflect intrinsic quality.
  • domain assumption The lightweight segmentation head is expressive enough to expose differences between upsamplers
    Section A.1. The head was chosen on a ViT baseline, and this choice may interact with upsampler feature distributions.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Benchmarking Feature Upsampling Methods for Vision Foundation Models using Interactive Segmentation." pith.science (2026). https://pith.science/paper/XYM43UBG

@misc{pith2026250502075,
  author       = {Pith},
  title        = {Pith review of: Benchmarking Feature Upsampling Methods for Vision Foundation Models using Interactive Segmentation},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/XYM43UBG}},
  note         = {Machine review of arXiv:2505.02075}
}
read the original abstract

Vision Foundation Models (VFMs) are large-scale, pre-trained models that serve as general-purpose backbones for various computer vision tasks. As VFMs' popularity grows, there is an increasing interest in understanding their effectiveness for dense prediction tasks. However, VFMs typically produce low-resolution features, limiting their direct applicability in this context. One way to tackle this limitation is by employing a task-agnostic feature upsampling module that refines VFM features resolution. To assess the effectiveness of this approach, we investigate Interactive Segmentation (IS) as a novel benchmark for evaluating feature upsampling methods on VFMs. Due to its inherent multimodal input, consisting of an image and a set of user-defined clicks, as well as its dense mask output, IS creates a challenging environment that demands comprehensive visual scene understanding. Our benchmarking experiments show that selecting appropriate upsampling strategies significantly improves VFM features quality. The code is released at https://github.com/havrylovv/iSegProbe

Figures

Figures reproduced from arXiv: 2505.02075 by the authors.

Figure 1
Figure 1. General architecture of a typical IS setup (a) and the proposed benchmark (b, c). The benchmark considers two options for injecting click features. head. Additionally, we define modules for encoding user clicks (hereafter referred to as click encoders) and determine an appropriate strategy for integrating their features with the VFM output. Depending on the complexity of the click en￾coder, its features are injected… view at source ↗
Figure 2
Figure 2. Comparison of features from upsamplers with a single input click. The click is indicated by a green dot on the original images. Backbone is DINOv2 (S/14) [46]. Click encoder is a symmetric patch embedding with early injection. Visualization method follows the PCA-based approach introduced in FeatUp [18]. specific learnable upsamplers have been proposed, tailored to different model architectures and downstream applic… view at source ↗
Figure 3
Figure 3. Detailed example of IS inference. The model takes an input image along with a stacked representation of two disk maps indicating positive and negative clicks. Positive and negative clicks are indicated by green and red dots, respectively. Additionally, the model may receive a probability map from the previous iteration. The input image is not shown for clarity. 4. Multiscale segmentation head: aggregates features fr… view at source ↗
Figures from the paper (6 more)
Figure 4
Figure 4. Figure 4: Segmentation results on GrabCut [54] with a single input click. Successful cases. The click is indicated by a green dot. Backbone is DINOv2 (S/14) [46]. Click encoder is a symmetric patch embedding with early injection. native click encoder and injection mechanism. The…
Figure 5
Figure 5. Figure 5: Segmentation results on GrabCut [54] with a single input click. Failure cases. The click is indicated by a green dot. Backbone is DINOv2 (S/14) [46]. Click encoder is a symmetric patch embedding with early injection. same data augmentation techniques as in [37] are app…
Figure 6
Figure 6. Figure 6: Convergence of IoU with increasing user clicks. Backbone is DINOv2 (S/14) [46]. Click encoder is a symmetric patch embedding with early injection. Injection Type GrabCut NoC80 NoC85 NoC90 IoU@1 Click Encoder: Symmetric Patch Embedding [37] Early Injection 3.68 5.02 8.0…
Figure 7
Figure 7. Figure 7: Additional visualizations of upsampler features with a single input click. The click is indicated by a green dot on the original images. 13 [PITH_FULL_IMAGE:figures/full_fig_p013_7.png]
Figure 8
Figure 8. Figure 8: Additional segmentation results with a single input click. The click is indicated by a green dot. 14 [PITH_FULL_IMAGE:figures/full_fig_p014_8.png]
Figure 9
Figure 9. Figure 9: Additional segmentation results with three input clicks. Positive and negative clicks are indicated by green and red dots, respectively. 15 [PITH_FULL_IMAGE:figures/full_fig_p015_9.png]

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. Maybe you don't need a U-Net: convolutional feature upsampling for materials micrograph segmentation

    cs.CV 2025-08 conditional novelty 5.0 of 10

    A lightweight CNN upsampler, distilled from FeatUp features, makes frozen DINOv2 patch features sharp enough for interactive segmentation of micrographs with sparse labels, and its workflow beats fine-tuning a U-Net i...

Reference graph

Works this paper leans on

73 extracted references · 60 canonical work pages · cited by 1 Pith paper

  1. [1]

    Understanding in- termediate layers using linear classifier probes

    Guillaume Alain and Yoshua Bengio. Understanding in- termediate layers using linear classifier probes. CoRR, abs/1610.01644, 2016. 4

    arXiv 2016

  2. [2]

    Mario P Amrehn, Stefan Steidl, Reinier Kortekaas, Mad- dalena Strumia, Markus Weingarten, Markus Kowarschik, and Andreas K. Maier. A semi-automated usability evalua- tion framework for interactive image segmentation systems. International Journal of Biomedical Imaging, 2019, 2019. 1

    work page 2019

  3. [3]

    Can visual foundation models achieve long-term point tracking? ArXiv, abs/2408.13575, 2024

    G¨orkay Aydemir, Weidi Xie, and Fatma G¨uney. Can visual foundation models achieve long-term point tracking? ArXiv, abs/2408.13575, 2024. 4

    arXiv 2024

  4. [4]

    Guibas, Justin Johnson, and Varun Jampani

    Mohamed El Banani, Amit Raj, Kevis-Kokitsi Maninis, Ab- hishek Kar, Yuanzhen Li, Michael Rubinstein, Deqing Sun, Leonidas J. Guibas, Justin Johnson, and Varun Jampani. Prob- ing the 3d awareness of visual foundation models. 2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 21795–21806, 2024. 4

    work page 2024

  5. [5]

    Large- scale interactive object segmentation with human annotators

    Rodrigo Benenson, Stefan Popov, and Vittorio Ferrari. Large- scale interactive object segmentation with human annotators. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 11700–11709, 2019. 1, 4

    work page 2019

  6. [6]

    Better plain vit baselines for imagenet-1k

    Lucas Beyer, Xiaohua Zhai, and Alexander Kolesnikov. Better plain vit baselines for imagenet-1k. CoRR, abs/2205.01580,

    arXiv

  7. [7]

    Interactive graph cuts for optimal boundary and region segmentation of objects in N-D images

    Yuri Boykov and Marie-Pierre Jolly. Interactive graph cuts for optimal boundary and region segmentation of objects in N-D images. In Proceedings of the Eighth International Con- ference On Computer Vision (ICCV-01), Vancouver, British Columbia, Canada, July 7-14, 2001 - Volume 1, pages 105–

    work page 2001

  8. [8]

    An experimental comparison of min-cut/max-flow algorithms for energy min- imization in vision

    Yuri Boykov and Vladimir Kolmogorov. An experimental comparison of min-cut/max-flow algorithms for energy min- imization in vision. In Energy Minimization Methods in Computer Vision and Pattern Recognition, Third Interna- tional Workshop, EMMCVPR 2001, Sophia Antipolis, France, September 3-5, 2001, Proceedings, pages 359–374. Springer,

    work page 2001

Show all 73 references
  1. [9]

    Ledits++: Limitless image editing us- ing text-to-image models

    Manuel Brack, Felix Friedrich, Katharia Kornmeier, Linoy Tsaban, Patrick Schramowski, Kristian Kersting, and Apolin´ario Passos. Ledits++: Limitless image editing us- ing text-to-image models. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition...

    2024

  2. [10]

    Tim Brooks, Aleksander Holynski, and Alexei A. Efros. In- structpix2pix: Learning to follow image editing instructions. In CVPR, pages 18392–18402. IEEE, 2023. 1

    2023

  3. [11]

    Focalclick: Towards practical interactive image segmentation

    Xi Chen, Zhiyan Zhao, Yilei Zhang, Manni Duan, Donglian Qi, and Hengshuang Zhao. Focalclick: Towards practical interactive image segmentation. In CVPR, pages 1290–1299. IEEE, 2022. 2, 4, 5

    2022

  4. [12]

    Feat2gs: Probing visual foundation models with gaussian splatting

    Yue Chen, Xingyu Chen, Anpei Chen, Gerard Pons-Moll, and Yuliang Xiu. Feat2gs: Probing visual foundation models with gaussian splatting. CoRR, abs/2412.09606, 2024. 4

    2024

  5. [13]

    Diffedit: Diffusion-based semantic image editing with mask guidance

    Guillaume Couairon, Jakob Verbeek, Holger Schwenk, and Matthieu Cord. Diffedit: Diffusion-based semantic image editing with mask guidance. In ICLR. OpenReview.net, 2023. 1

    2023

  6. [14]

    Surgical-dino: adapter learning of foundation models for depth estimation in endoscopic surgery.International Journal of Computer Assisted Radiology and Surgery, 19:1013 – 1020,

    Beilei Cui, Mobarakol Islam, Long Bai, and Hongliang Ren. Surgical-dino: adapter learning of foundation models for depth estimation in endoscopic surgery.International Journal of Computer Assisted Radiology and Surgery, 19:1013 – 1020,

  7. [15]

    Learning affinity- aware upsampling for deep image matting

    Yutong Dai, Hao Lu, and Chunhua Shen. Learning affinity- aware upsampling for deep image matting. In CVPR, pages 6841–6850. Computer Vision Foundation / IEEE, 2021. 3

    2021

  8. [16]

    An image is worth 16x16 words: Transformers for image recognition at scale

    Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xiaohua Zhai, Thomas Unterthiner, Mostafa Dehghani, Matthias Minderer, Georg Heigold, Syl- vain Gelly, Jakob Uszkoreit, and Neil Houlsby. An image is worth 16x16 words: Transformers for image recognition ...

    2021

  9. [17]

    A guide to convo- lution arithmetic for deep learning

    Vincent Dumoulin and Francesco Visin. A guide to convo- lution arithmetic for deep learning. CoRR, abs/1603.07285,

    arXiv

  10. [18]

    Brandt, Axel Feld- mann, Zhoutong Zhang, and William T

    Stephanie Fu, Mark Hamilton, Laura E. Brandt, Axel Feld- mann, Zhoutong Zhang, and William T. Freeman. Featup: A model-agnostic framework for features at any resolution. In ICLR. OpenReview.net, 2024. 1, 3, 4, 5, 7, 12

    2024

  11. [19]

    Bros- tow

    Cl´ement Godard, Oisin Mac Aodha, and Gabriel J. Bros- tow. Unsupervised monocular depth estimation with left-right consistency. In CVPR, pages 6602–6611. IEEE Computer Society, 2017. 1 8

    2017

  12. [20]

    Leo J. Grady. Random walks for image segmentation. IEEE Trans. Pattern Anal. Mach. Intell., 28(11):1768–1783, 2006. 2

    2006

  13. [21]

    Geodesic star convexity for interactive image segmentation

    Varun Gulshan, Carsten Rother, Antonio Criminisi, Andrew Blake, and Andrew Zisserman. Geodesic star convexity for interactive image segmentation. In The Twenty-Third IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2010, San Francisco, CA, USA, 13-18 June 2010,...

    2010

  14. [22]

    Bourdev, Subhransu Maji, and Jitendra Malik

    Bharath Hariharan, Pablo Arbel ´aez, Lubomir D. Bourdev, Subhransu Maji, and Jitendra Malik. Semantic contours from inverse detectors. In ICCV, pages 991–998. IEEE Computer Society, 2011. 5, 12

    2011

  15. [23]

    Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen- Zhu, Yuanzhi Li, Shean Wang, Lu Wang, and Weizhu Chen

    Edward J. Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen- Zhu, Yuanzhi Li, Shean Wang, Lu Wang, and Weizhu Chen. Lora: Low-rank adaptation of large language models. InICLR. OpenReview.net, 2022. 12

    2022

  16. [24]

    Learning implicit feature alignment function for semantic segmentation

    Hanzhe Hu, Yinbo Chen, Jiarui Xu, Shubhankar Borse, Hong Cai, Fatih Porikli, and Xiaolong Wang. Learning implicit feature alignment function for semantic segmentation. In ECCV (29), pages 487–505. Springer, 2022. 1, 3

    2022

  17. [25]

    Renovating names in open-vocabulary segmentation benchmarks

    Haiwen Huang, Songyou Peng, Dan Zhang, and Andreas Geiger. Renovating names in open-vocabulary segmentation benchmarks. In NeurIPS, 2024. 1

    2024

  18. [26]

    Loftup: Learning a coordinate-based feature upsampler for vision foundation models, 2025

    Haiwen Huang, Anpei Chen, V olodymyr Havrylov, Andreas Geiger, and Dan Zhang. Loftup: Learning a coordinate-based feature upsampler for vision foundation models, 2025. 2, 3, 4, 5, 12

    2025

  19. [27]

    Gir- shick

    Alexander Kirillov, Yuxin Wu, Kaiming He, and Ross B. Gir- shick. Pointrend: Image segmentation as rendering. In CVPR, pages 9796–9805. Computer Vision Foundation / IEEE, 2020. 1, 3

    2020

  20. [28]

    Berg, Wan-Yen Lo, Piotr Doll´ar, and Ross Girshick

    Alexander Kirillov, Eric Mintun, Nikhila Ravi, Hanzi Mao, Chloe Rolland, Laura Gustafson, Tete Xiao, Spencer White- head, Alexander C. Berg, Wan-Yen Lo, Piotr Doll´ar, and Ross Girshick. Segment anything. arXiv:2304.02643, 2023. 1

    2023 arXiv

  21. [29]

    User-centric learning and evaluation of interactive segmentation systems

    Pushmeet Kohli, Hannes Nickisch, Carsten Rother, and Christoph Rhemann. User-centric learning and evaluation of interactive segmentation systems. Int. J. Comput. Vis., 100 (3):261–274, 2012. 1

    2012

  22. [30]

    Cohen, Dani Lischinski, and Matthew Uyttendaele

    Johannes Kopf, Michael F. Cohen, Dani Lischinski, and Matthew Uyttendaele. Joint bilateral upsampling. ACM Trans. Graph., 26(3):96, 2007. 2

    2007

  23. [31]

    Controlnet++: Improv- ing conditional controls with efficient consistency feedback

    Ming Li, Taojiannan Yang, Huafeng Kuang, Jie Wu, Zhaoning Wang, Xuefeng Xiao, and Chen Chen. Controlnet++: Improv- ing conditional controls with efficient consistency feedback. In ECCV (7), pages 129–147. Springer, 2024. 1

    2024

  24. [32]

    Girshick, and Kaiming He

    Yanghao Li, Hanzi Mao, Ross B. Girshick, and Kaiming He. Exploring plain vision transformer backbones for object detection. In ECCV (9), pages 280–296. Springer, 2022. 1, 3, 5, 11

    2022

  25. [33]

    Interactive image segmentation with latent diversity

    Zhuwen Li, Qifeng Chen, and Vladlen Koltun. Interactive image segmentation with latent diversity. InCVPR, pages 577–

  26. [34]

    Interactive image segmentation with first click attention

    Zheng Lin, Zhao Zhang, Lin-Zhuo Chen, Ming-Ming Cheng, and Shao-Ping Lu. Interactive image segmentation with first click attention. In 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition, CVPR 2020, Seattle, WA, USA, June 13-19, 2020, pages 13336–13345. Compute...

    2020

  27. [35]

    Focuscut: Diving into a focus view in interactive segmentation

    Zheng Lin, Zheng-Peng Duan, Zhao Zhang, Chun-Le Guo, and Ming-Ming Cheng. Focuscut: Diving into a focus view in interactive segmentation. In CVPR, pages 2627–2636. IEEE,

  28. [36]

    isegformer: Interactive image segmentation with transformers

    Qin Liu. isegformer: Interactive image segmentation with transformers. CoRR, abs/2112.11325, 2021. 2

    2021 arXiv

  29. [37]

    Simpleclick: Interactive image segmentation with simple vi- sion transformers

    Qin Liu, Zhenlin Xu, Gedas Bertasius, and Marc Niethammer. Simpleclick: Interactive image segmentation with simple vi- sion transformers. In ICCV, pages 22233–22243. IEEE, 2023. 1, 2, 3, 4, 5, 6, 7, 11, 12

    2023

  30. [38]

    Swin transformer: Hierarchical vision transformer using shifted windows

    Ze Liu, Yutong Lin, Yue Cao, Han Hu, Yixuan Wei, Zheng Zhang, Stephen Lin, and Baining Guo. Swin transformer: Hierarchical vision transformer using shifted windows. In ICCV, pages 9992–10002. IEEE, 2021. 2

    2021

  31. [39]

    Index networks

    Hao Lu, Yutong Dai, Chunhua Shen, and Songcen Xu. Index networks. IEEE Trans. Pattern Anal. Mach. Intell. , 44(1): 242–255, 2022. 3

    2022

  32. [40]

    FADE: fusing the assets of decoder and encoder for task-agnostic upsampling

    Hao Lu, Wenze Liu, Hongtao Fu, and Zhiguo Cao. FADE: fusing the assets of decoder and encoder for task-agnostic upsampling. In ECCV (27), pages 231–247. Springer, 2022. 1, 3

    2022

  33. [41]

    SAPA: similarity-aware point affiliation for feature upsampling

    Hao Lu, Wenze Liu, Zixuan Ye, Hongtao Fu, Yuliang Liu, and Zhiguo Cao. SAPA: similarity-aware point affiliation for feature upsampling. In NeurIPS, 2022. 1, 3

    2022

  34. [42]

    Deep interactive segmentation of medical images: A systematic review and taxonomy

    Zdravko Marinov, Paul F J ¨ager, Jan Egger, Jens Kleesiek, and Rainer Stiefelhagen. Deep interactive segmentation of medical images: A systematic review and taxonomy. IEEE transactions on pattern analysis and machine intelligence ,

  35. [43]

    Martin, Charless C

    David R. Martin, Charless C. Fowlkes, Doron Tal, and Jiten- dra Malik. A database of human segmented natural images and its application to evaluating segmentation algorithms and measuring ecological statistics. In ICCV, pages 416–425. IEEE Computer Society, 2001. 5

    2001

  36. [44]

    Learn- ing deconvolution network for semantic segmentation

    Hyeonwoo Noh, Seunghoon Hong, and Bohyung Han. Learn- ing deconvolution network for semantic segmentation. In ICCV, pages 1520–1528. IEEE Computer Society, 2015. 2

    2015

  37. [45]

    Decon- volution and checkerboard artifacts

    Augustus Odena, Vincent Dumoulin, and Chris Olah. Decon- volution and checkerboard artifacts. Distill, 1(10):e3, 2016. 2

    2016

  38. [46]

    Maxime Oquab, Timoth´ee Darcet, Th´eo Moutakanni, Huy V . V o, Marc Szafraniec, Vasil Khalidov, Pierre Fernandez, Daniel Haziza, Francisco Massa, Alaaeldin El-Nouby, Mido Ass- ran, Nicolas Ballas, Wojciech Galuba, Russell Howes, Po- Yao Huang, Shang-Wen Li, Ishan Misra, Michae...

    2024

  39. [47]

    Taglab: Ai-assisted annotation for the fast and accurate semantic segmentation of coral reef 9 orthoimages

    Gaia Pavoni, Massimiliano Corsini, Federico Ponchio, Alessandro Muntoni, Clinton Edwards, Nicole Pedersen, Stu- art Sandin, and Paolo Cignoni. Taglab: Ai-assisted annotation for the fast and accurate semantic segmentation of coral reef 9 orthoimages. Journal of Field Robotics ...

  40. [48]

    Gross, and Alexander Sorkine- Hornung

    Federico Perazzi, Jordi Pont-Tuset, Brian McWilliams, Luc Van Gool, Markus H. Gross, and Alexander Sorkine- Hornung. A benchmark dataset and evaluation methodology for video object segmentation. In CVPR, pages 724–732. IEEE Computer Society, 2016. 5

    2016

  41. [49]

    Learning transferable visual models from natural language supervision

    Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, Gretchen Krueger, and Ilya Sutskever. Learning transferable visual models from natural language supervision. In ICML, pages 8748–...

    2021

  42. [50]

    Using goms and nasa-tlx to evaluate human–computer inter- action process in interactive segmentation

    Anjana Ramkumar, Pieter Jan Stappers, Wiro J Niessen, Sonja Adebahr, Tanja Schimek-Jasch, Ursula Nestle, and Yu Song. Using goms and nasa-tlx to evaluate human–computer inter- action process in interactive segmentation. International Jour- nal of Human–Computer Interaction, 33...

    2017

  43. [51]

    Am-radio: Agglomerative vision foundation model reduce all domains into one

    Michael Ranzinger, Greg Heinrich, Jan Kautz, and Pavlo Molchanov. Am-radio: Agglomerative vision foundation model reduce all domains into one. 2024 IEEE/CVF Confer- ence on Computer Vision and Pattern Recognition (CVPR), pages 12490–12500, 2023. 4

    2024

  44. [52]

    Girshick, Piotr Doll´ar, and Christoph Feichtenhofer

    Nikhila Ravi, Valentin Gabeur, Yuan-Ting Hu, Ronghang Hu, Chaitanya Ryali, Tengyu Ma, Haitham Khedr, Roman R¨adle, Chlo´e Rolland, Laura Gustafson, Eric Mintun, Junting Pan, Kalyan Vasudev Alwala, Nicolas Carion, Chao-Yuan Wu, Ross B. Girshick, Piotr Doll´ar, and Christoph Fei...

    2024 arXiv

  45. [53]

    High-resolution image syn- thesis with latent diffusion models

    Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Bj¨orn Ommer. High-resolution image syn- thesis with latent diffusion models. In CVPR, pages 10674– 10685. IEEE, 2022. 1

    2022

  46. [54]

    ”grabcut”: interactive foreground extraction using iterated graph cuts

    Carsten Rother, Vladimir Kolmogorov, and Andrew Blake. ”grabcut”: interactive foreground extraction using iterated graph cuts. ACM Trans. Graph., 23(3):309–314, 2004. 2, 5, 6, 12

    2004

  47. [55]

    Aitken, Christian Ledig, and Zehan Wang

    Wenzhe Shi, Jose Caballero, Lucas Theis, Ferenc Huszar, Andrew P. Aitken, Christian Ledig, and Zehan Wang. Is the deconvolution layer the same as a convolutional layer? CoRR, abs/1609.07009, 2016. 2

    2016 arXiv

  48. [56]

    Petrov, and Anton Konushin

    Konstantin Sofiiuk, Ilya A. Petrov, and Anton Konushin. Re- viving iterative training with mask guidance for interactive segmentation. In 2022 IEEE International Conference on Im- age Processing, ICIP 2022, Bordeaux, France, 16-19 October 2022, pages 3141–3145. IEEE, 2022. 2, 4, 5, 6

    2022

  49. [57]

    Lift: A surprisingly simple lightweight feature transform for dense vit descriptors

    Saksham Suri, Matthew Walmer, Kamal Gupta, and Abhinav Shrivastava. Lift: A surprisingly simple lightweight feature transform for dense vit descriptors. In ECCV (7), pages 110–

  50. [58]

    Winoground: Probing vision and language models for visio- linguistic compositionality

    Tristan Thrush, Ryan Jiang, Max Bartolo, Amanpreet Singh, Adina Williams, Douwe Kiela, and Candace Ross. Winoground: Probing vision and language models for visio- linguistic compositionality. 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) , pages 52...

    2022

  51. [59]

    Gritsenko, Xiao Wang, Muhammad Ferjad Naeem, Ibrahim Alabdulmohsin, Nikhil Parthasarathy, Talfan Evans, Lucas Beyer, Ye Xia, Basil Mustafa, Olivier J

    Michael Tschannen, Alexey A. Gritsenko, Xiao Wang, Muhammad Ferjad Naeem, Ibrahim Alabdulmohsin, Nikhil Parthasarathy, Talfan Evans, Lucas Beyer, Ye Xia, Basil Mustafa, Olivier J. H ´enaff, Jeremiah Harmsen, Andreas Steiner, and Xiaohua Zhai. Siglip 2: Multilingual vision- lan...

    2025 arXiv

  52. [60]

    Stewart, Zhitong Xiong, Xiao Xiang Zhu, Stefan Bauer, and John Chuang

    Leonard Waldmann, Ando Shah, Yi Wang, Nils Lehmann, Adam J. Stewart, Zhitong Xiong, Xiao Xiang Zhu, Stefan Bauer, and John Chuang. Panopticon: Advancing any-sensor foundation models for earth observation. 2025. 1

    2025

  53. [61]

    CARAFE: content-aware reassembly of features

    Jiaqi Wang, Kai Chen, Rui Xu, Ziwei Liu, Chen Change Loy, and Dahua Lin. CARAFE: content-aware reassembly of features. In ICCV, pages 3007–3016. IEEE, 2019. 1, 3

    2019

  54. [62]

    Fouhey, and Abhinav Gupta

    Xiaolong Wang, David F. Fouhey, and Abhinav Gupta. De- signing deep networks for surface normal estimation. In CVPR, pages 539–547. IEEE Computer Society, 2015. 1

    2015

  55. [63]

    ´Alvarez, and Ping Luo

    Enze Xie, Wenhai Wang, Zhiding Yu, Anima Anandkumar, Jos´e M. ´Alvarez, and Ping Luo. Segformer: Simple and efficient design for semantic segmentation with transformers. In NeurIPS, pages 12077–12090, 2021. 1, 2, 3, 11

    2021

  56. [64]

    Price, Scott Cohen, Jimei Yang, and Thomas S

    Ning Xu, Brian L. Price, Scott Cohen, Jimei Yang, and Thomas S. Huang. Deep interactive object selection. In 2016 IEEE Conference on Computer Vision and Pattern Recog- nition, CVPR 2016, Las Vegas, NV , USA, June 27-30, 2016, pages 373–381. IEEE Computer Society, 2016. 2, 4

    2016

  57. [65]

    Convolutions die hard: Open-vocabulary seg- mentation with single frozen convolutional CLIP

    Qihang Yu, Ju He, Xueqing Deng, Xiaohui Shen, and Liang- Chieh Chen. Convolutions die hard: Open-vocabulary seg- mentation with single frozen convolutional CLIP. InNeurIPS,

  58. [66]

    Open-vocabulary sam: Segment and recognize twenty-thousand classes interactively

    Haobo Yuan, Xiangtai Li, Chong Zhou, Yining Li, Kai Chen, and Chen Change Loy. Open-vocabulary sam: Segment and recognize twenty-thousand classes interactively. In ECCV,

  59. [67]

    Adding conditional control to text-to-image diffusion models

    Lvmin Zhang, Anyi Rao, and Maneesh Agrawala. Adding conditional control to text-to-image diffusion models. In Proceedings of the IEEE/CVF international conference on computer vision, pages 3836–3847, 2023. 1

    2023

  60. [68]

    Graco: Granularity-controllable interactive segmentation

    Yian Zhao, Kehan Li, Zesen Cheng, Pengchong Qiao, Xi- awu Zheng, Rongrong Ji, Chang Liu, Li Yuan, and Jie Chen. Graco: Granularity-controllable interactive segmentation. In CVPR, pages 3501–3510. IEEE, 2024. 2, 3, 4, 5, 11, 12 10 Benchmarking Feature Upsampling Methods for Vis...

    2024

  61. [72]

    The 1 14 resolution is derived directly through a convolutional layer, while the 1 28 resolution is obtained by applying a 2× 2 max pooling operation prior to convolution

    The highest resolutions ( 1 and 1 4) are obtained via upsampling, followed by a single convolu- tional layer. The 1 14 resolution is derived directly through a convolutional layer, while the 1 28 resolution is obtained by applying a 2× 2 max pooling operation prior to convolut...

  62. [112]

    IEEE Computer Society, 2001. 2

    2001

  63. [128]

    1, 3, 4, 5

    Springer, 2024. 1, 3, 4, 5

    2024

  64. [256]

    Finally, a classifi- cation layer is applied

    The features are then bilinearly interpolated to match the resolution of the largest feature map (either 1 or 2 7 in our setup), concatenated along the channel dimension, and passed through an additional convolutional layer with the same output channels and a 1× 1 kernel. Fina...

  65. [585]

    Computer Vision Foundation / IEEE Computer Society,

Pith tools

Reviewed August 16, 2026 · model on record in the stance chip above.