Pith. sign in

REVIEW 2 major objections 4 minor 56 references

An iterative local-attention upsampler can match or beat cross-attention feature upsamplers at a fraction of the cost.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · deepseek-v4-flash

2026-08-03 08:08 UTC pith:RJB3OHRP

load-bearing objection Strong efficiency and SOTA dense-prediction results with a clever local-attention operator; the generative arm needs ablations before the CFM comparison is credible. the 2 major comments →

arxiv 2601.17950 v2 pith:RJB3OHRP submitted 2026-01-25 cs.CV

UPLiFT: Efficient Pixel-Dense Feature Upsampling with Local Attenders

Matthew Walmer , Saksham Suri , Anirud Aggarwal , Abhinav Shrivastava This is my paper
classification cs.CV
keywords feature upsamplingpixel-dense featureslocal attentioniterative upsamplingsemantic stabilityVAE latent upsamplinglinear scalingself-supervised representation
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

This paper claims that iterative 2x feature upsampling, long thought to cause semantic drift, can still beat modern cross-attention upsamplers if each step uses a cheap local attention operator. The proposed UPLiFT decoder applies the same compact 2x block repeatedly, guided by dense features extracted once from the input image, and a Local Attender pools from a fixed local neighborhood instead of global queries and keys. On four segmentation datasets it reports the highest mIoU among upsamplers, and it ties for best depth error, with inference times and memory that grow linearly rather than quadratically. Applied to VAE latents, it reports text-to-image and super-resolution quality competitive with a flow-matching upsampler while using roughly one-sixth the parameters and one-two-hundredth the training data.

Core claim

The central claim is that the semantic drift that plagued earlier iterative upsamplers comes from unconstrained per-step transformations, and that a strictly local, linear-combination pooling step fixes it. UPLiFT's decoder first applies a convolutional 2x lift, then a Local Attender forms the output as a weighted sum of the original backbone features in a small fixed neighborhood around each cell, with weights predicted by a 1x1 convolution over guide features. Because the output is constrained to live in the span of the input features, the upsampled distribution stays close to the backbone's, and because the neighborhood has constant size, cost scales linearly. The paper reports that this

What carries the argument

Local Attender: an attention operator that replaces Query-Key-Value cross-attention with a fixed set of 2D directional offsets. For each output cell, a 1x1 convolution on the guide feature predicts softmax weights over a local neighborhood (a star of 17 offsets in the main experiments); the output is the weighted sum of the value features at those offsets, optionally upsampling by grouping guide tokens into c x c cells. This keeps outputs in the linear span of the input features, the same regularization that cross-attention provides, while making compute O(nT) instead of O(T^2).

Load-bearing premise

The generative claims rest on the assumption that the Local Attender's linear-combination output is already what a pre-trained VAE decoder expects; Appendix B.1 shows that omitting the post-attender Refiner Block yields blocky artifacts, so the contribution of that add-on (and the color-correction step) to the reported image-quality numbers is never quantified.

What would settle it

Ablate the Refiner Block and the per-channel color-correction module from the generative UPLiFT (Appendix B.1) and rerun the COCO and reLAION text-to-image and FacesHQ/LHQ super-resolution metrics; if PSNR, SSIM, and FID collapse toward bilinear or nearest-neighbor baselines, the claim that the Local Attender itself preserves the VAE-decoder feature distribution fails.

Watch this falsifier — get emailed when new claim-graph text bears on it.

If this is right

  • Iterative 2x upsampling can match or beat direct cross-attention upsampling when each step is a local linear combination; the earlier conclusion that iteration necessarily degrades features does not hold for UPLiFT.
  • Dense-feature upsampling can run on much larger images: UPLiFT processes roughly 2500 visual tokens on a 24 GB GPU where cross-attention baselines exhaust memory near 1500, with 2.5-5x speedup at that scale.
  • A single general-purpose VAE latent upsampler can serve both text-to-image upscaling and 4x super-resolution without task-specific fine-tuning, extending Stable Diffusion 1.5 outputs from 512 to 2048 while adding only about 8% latency over bilinear latent upsampling.
  • The multi-depth, multi-step self-supervised loss makes repeated applications of the same decoder stable, so one epoch on ImageNet-1K (predictive) or 25k images (generative) transfers to unseen datasets and backbones.
  • For patch-size-16 backbones, 16x upsampling becomes the natural target, matching UPLiFT's fixed 2x iteration count without awkward over-upsampling.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • The paper's generative ablations stop at qualitative images: the Refiner Block is shown to matter but never scored, and the color-correction module is described but never ablated; a reader wanting to attribute the PSNR/FID gains to the Local Attender itself would need those numbers.
  • Because the Local Attender is a generic linear-combination pooling operator, it should transfer to other iterative settings where feature distributions must be preserved, such as multi-scale decoder pyramids or feature fusion, as long as a guide signal at the target resolution exists.
  • A natural stress test is to apply UPLiFT beyond its training depth, for example 1024 to 4096 or 8192, and measure per-step feature drift; the paper only trains with up to four depths, so extrapolation behavior is unverified.
  • The multi-depth training schedule suggests a testable extension: depth-aware weighting or curriculum scheduling might push the depth ceiling beyond d=4, which the paper identifies as a performance floor at small token grids.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

2 major / 4 minor

Summary. The paper proposes UPLiFT, an iterative 2x feature upsampler for pre-trained ViT backbones, and a Local Attender operator that performs attention over a fixed local offset neighborhood, constraining output features to be linear combinations of input features. UPLiFT is trained self-supervised by L2 reconstruction of the frozen backbone's own high-resolution features, using a multi-depth training loss. The predictive arm reports state-of-the-art mIoU on four segmentation datasets and competitive depth estimation with recomputed baselines under a fixed protocol, plus a DINOv3 transfer experiment. The generative arm applies a larger UPLiFT variant to VAE latent upsampling for text-to-image diffusion upscaling and image super-resolution, claiming CFM-competitive quality with substantially fewer parameters, less training data, and lower latency.

Significance. If the predictive results hold, UPLiFT is a genuinely useful contribution: it demonstrates that iterative upsampling can compete with recent cross-attention-based upsamplers at lower inference cost, and the paper's evaluation protocol is careful in several respects. The strengths include recomputing all baselines under the same protocol (Table 1), checking the LoftUp backbone-distribution confound (Table 5), evaluating DINOv3 transfer against available baselines (Table 8), and ablating the training-depth and neighborhood design (Tables 6 and 7). The generative claim, however, is currently not securely attributable to the core method because the pipeline includes unablated post-hoc components outside the Local Attender; this needs to be fixed before the CFM-competitive claim can be accepted. The overall contribution is promising and the missing experiments are within the paper's scope.

major comments (2)
  1. [Section 5, App. B.1, Tables 2-4] The generative results are the sole evidence for the claim that UPLiFT achieves CFM-competitive performance, but the pipeline contains two components outside the Local Attender whose quantitative contribution is never measured. App. B.1 states that removing the post-attender Refiner Block leads to 'significant blocky artifacts, which are likely the result of the strict, linear-combination feature upsampling approach,' and it also introduces a per-channel color-correction step after the VAE decoder to remove color shifts. Neither is ablated in Tables 2-4, so the reported SSIM/PSNR/FID/pFID may be carried by these add-ons rather than by the Local Attender. I request an ablation table reporting all generative metrics with (i) no Refiner, (ii) no color correction, (iii) neither, and (iv) the full pipeline. If the Refiner is essential, it should be reported as part of the proposed method with
  2. [Table 1 and Abstract] The 'state-of-the-art' claim rests on small margins over strong baselines, e.g. COCO mIoU 62.55 vs 62.08 (AnyUp), VOC 85.21 vs 84.63 (LoftUp), and ADE20K 42.97 vs 42.25 (AnyUp). No error bars or multiple-seed experiments are reported. Since the baselines were recomputed with an altered learning-rate schedule, and the differences are on the order of a few tenths of a point, it is not possible to tell whether the claimed improvements are significant. I ask for mean +/- standard deviation over at least three seeds for UPLiFT and the closest baselines, or at minimum a statement about evaluation stochasticity. This is a support issue for the headline claim rather than a flaw in the architecture.
minor comments (4)
  1. [Table 7] In the n=5 row, the VOC accuracy column appears to duplicate the mIoU value ('84.80 84.80'); this is likely a typo and should be corrected.
  2. [Section 3.3, Eq. (2)] The notation F' is overloaded: earlier in the section F' denotes the low-resolution backbone features B(I'), while in Eq. (2) F' is a set of upsampled feature maps. Please use a different symbol, e.g. hat-F, to avoid confusion.
  3. [Section 5, Table 4] The phrase 'single general-purpose module' should be clarified: the VAE variant is a much larger network (53.5M parameters) with additional components (Refiner Block, noise channels, LayerNorm, color correction) compared with the predictive 0.8M model. Specify that this refers to one module for VAE features across datasets, not the same exact model used in the predictive arm.
  4. [Throughout] Minor typographical and formatting issues: 'UPLiFT0.8' in Table 1 lacks a space, 'V AE' appears with an unusual space in several places, and the caption of Figure 6 cites 'CFM [15]' but CFM should be [41].

Circularity Check

0 steps flagged

No circular derivation found; generative-claim attribution risk from unablated Refiner/color correction is experimental design, not circularity.

full rationale

No load-bearing circular step is present. UPLiFT's training objective (Eq. 1, Eq. 5) is a self-supervised L2 reconstruction of the same frozen backbone's high-resolution features, so feature reconstruction is self-referential by design. But the paper's central claims are not evaluated on that reconstruction: Table 1 uses external segmentation/depth labels; Tables 2-4 compare decoded image metrics against CFM's published numbers. Those downstream benchmarks are not determined by the training target, so the SOTA claims retain independent content. The self-citations to LiFT [43] and to the offset-attention observation [48] are lineage/inspiration. The Local Attender is ablated in-paper (Table 7, no-LA vs neighborhood sizes) and compared against external baselines, so no load-bearing premise rests solely on those citations. App. B.1 does honestly flag a limitation: 'Without the refiner block, the upsampled images have significant blocky artifacts, which are likely the result of the strict, linear-combination feature upsampling approach,' and it introduces a post-decoder per-channel color-correction module to 'remove any minor color shifts.' This is a real attribution gap for the generative arm, because the Refiner and color correction are not ablated in the headline tables and could carry much of the CFM-competitive result. However, this is an experimental-design/ablation concern, not circularity: no equation in the paper equates the reported PSNR/SSIM/FID/pFID to the training loss or to a fitted parameter. The predictive arm is externally validated, and the generative numbers are external benchmarks, so there is no reduction of a prediction to its inputs. Score 1 reflects the minor same-group lineage and the unquantified post-hoc components, but neither is a circular step.

Axiom & Free-Parameter Ledger

4 free parameters · 4 axioms · 0 invented entities

No new physical entities are postulated; the Local Attender is a learned operator, not an entity. The paper's load-bearing choices are empirical: neighborhood geometry (n=17), training depth set, model scale for generative tasks, and a post-hoc color-correction transform. The most consequential hidden cost is that the generative headline numbers depend on add-on components outside the Local Attender.

free parameters (4)
  • Local Attender neighborhood size/shape n=17 = n=17 star-shaped offset set
    Chosen by ablation on COCO/VOC segmentation (Table 7); all main predictive and generative results use this specific neighborhood without further task-specific tuning.
  • Training depth set D = D={1,2,3} for predictive model; D={1,2,3,4} for VAE model
    Selected by ablation (Table 6): d=4 alone hurts, and adding d=4 to multi-depth training is slightly detrimental; this is a hand-tuned recipe rather than a principled choice.
  • Per-channel color-correction difference vector = Computed at inference from low-res input image means
    App. B.1: after VAE decoding, per-color-channel mean difference between input and output is subtracted. This post-hoc inference-time normalization is not part of the trained UPLiFT module and its contribution to headline PSNR/SSIM/FID is not quantified.
  • Generative UPLiFT model size = 53.5M parameters (vs 0.8M predictive)
    App. B.1: 'we empirically find that small parameter count upsampling models... are insufficient for generative downstream tasks'; model depth and channels scaled up by hand, not by a search.
axioms (4)
  • domain assumption Backbone features extracted from a downsampled image are a valid supervision target for upsampling (Eq. 1, L_simple = L2(F'_2d×, F)).
    Sec. 3.3: the self-supervised target is the same frozen backbone applied to ground-truth images. Assumes this reconstruction task transfers to downstream dense tasks and to VAE latent spaces.
  • domain assumption A weighted sum of local backbone features preserves the backbone feature distribution ('feature consistency').
    Sec. 3.2: 'the output features are linear sums of features in V, ensuring feature consistency with respect to V.' This is the core premise of Local Attender; it is contradicted in the generative setting by the need for a Refiner Block and color correction (App. B.1).
  • domain assumption A decoder trained with up to 3 iterative steps remains stable for 4 steps at inference.
    Sec. 3.3 trains with d∈{1,2,3}; Table 1 uses 16× upsampling (4 decoder applications) followed by downsampling to 14×. Table 6 gives indirect empirical support, but the 3rd-to-4th step extrapolation is not analyzed in isolation.
  • domain assumption Fixed-offset local attention captures all information needed for feature upsampling, based on the observation that ViT attention heads learn local offsets.
    Sec. 3.2: 'we draw inspiration from [48], which showed that attention heads... learn to attend to local positions with a fixed directional offset.' An empirical observation about trained attention heads is extended into a design principle.

pith-pipeline@v1.3.0-alltime-deepseek · 22168 in / 16272 out tokens · 181150 ms · 2026-08-03T08:08:35.975636+00:00 · methodology

0 comments
read the original abstract

The space of task-agnostic feature upsampling has emerged as a promising area of research to efficiently create denser features from pre-trained visual backbones. These methods act as a shortcut to achieve dense features for a fraction of the cost by learning to map low-resolution features to high-resolution versions. While early works in this space used iterative upsampling approaches, more recent works have switched to cross-attention-based methods, which risk falling into the same efficiency scaling problems of the backbones they are upsampling. In this work, we demonstrate that iterative upsampling methods can still compete with cross-attention-based methods; moreover, they can achieve state-of-the-art performance with lower inference costs. We propose UPLiFT, an architecture for Universal Pixel-dense Lightweight Feature Transforms. We also propose an efficient Local Attender operator to overcome the limitations of prior iterative feature upsampling methods. This operator uses an alternative attentional pooling formulation defined fully locally. We show that our Local Attender allows UPLiFT to maintain stable features throughout upsampling, enabling state-of-the-art performance with lower inference costs than existing pixel-dense feature upsamplers. In addition, we apply UPLiFT to generative downstream tasks and show that it achieves competitive performance with state-of-the-art Coupled Flow Matching models for VAE feature upsampling. Altogether, UPLiFT offers a versatile and efficient approach to creating denser features.

Figures

Figures reproduced from arXiv: 2601.17950 by Abhinav Shrivastava, Anirud Aggarwal, Matthew Walmer, Saksham Suri.

Figure 1
Figure 1. Figure 1: UPLiFT time-scaling and dense features. We present UPLiFT, an efficient feature-upsampler that leverages our new Local Attender to extract semantically-stable, pixel-dense fea￾tures. (Top) UPLiFT’s inference time and memory scales linearly with the number of visual tokens, while other recent SOTA meth￾ods face quadratic scaling. (Bottom) PCA visualization of low￾resolution DINOv2 features and pixel-dense U… view at source ↗
Figure 2
Figure 2. Figure 2: UPLiFT Tasks. We demonstrate our UPLiFT feature upsampler for applications in both predictive and generative tasks. This includes semantic segmentation, monocular depth estimation, image super-resolution, and efficient text-to-image generation. higher costs of comparable recent approaches. • State-of-the-art performance on several dense predictive tasks, surpassing prior feature upsamplers while having fas… view at source ↗
Figure 3
Figure 3. Figure 3: UPLiFT Inference. At inference time, our UPLiFT Encoder (EUPLiFT) produces shallow but dense features to guide all sub￾sequent upsampling steps. Iterative application of the UPLiFT Decoder (DUPLiFT) upsamples the low-resolution backbone features to pixel-density. Our proposed Local Attender module is integrated with the UPLiFT Decoder to maintain iterative feature consistency. 3.1. Architecture Overview We… view at source ↗
Figure 4
Figure 4. Figure 4: Local Attender Operator. We propose a streamlined and efficient local attention operator, which gathers features over a set neighborhood defined by fixed direction offsets. vert it to shape H×W×n. This convolutional layer is the only learnable element of the Local Attender. We then ap￾ply position-wise softmax to this feature, creating the “At￾tender Map” A. For a given (x, y) position in A, the value Ax,y… view at source ↗
Figure 6
Figure 6. Figure 6: 2048×2048 Image Generation Comparison with CFM. We show that UPLiFT achieves comparable visual upsampling quality to CFM [15], while using only 1/6 th the network parameters, 1/200th the training data, and only 2 iterative upsampling steps [PITH_FULL_IMAGE:figures/full_fig_p007_6.png] view at source ↗
Figure 7
Figure 7. Figure 7: Visual ablation of design choices for VAE UPLiFT. (Left) UPLiFT achieves high quality 512→1024 upsampling with a larger parameter count model. (Middle) A smaller-scale UPLiFT has insufficient capacity to upsample all high-frequency information and produces blurry results. (Right) Ablation of the Refiner Block leads to blocky artifacts in upsampled images. feature upsampling, similar to CFM, which must deco… view at source ↗
Figure 8
Figure 8. Figure 8: Local Attender neighborhood designs. We visualize the neighborhood designs tested with our Local Attender module. The center token (orange) is always included in the neighborhood, and the blue tokens represent the offset relative local positions that are included in feature pooling through local attention [PITH_FULL_IMAGE:figures/full_fig_p015_8.png] view at source ↗
Figure 9
Figure 9. Figure 9: Speed and Memory Usage Comparison. We compare UPLiFT with recent cross-attention-based feature upsampling methods when running with gradually increasing image sizes. We report the average inference time and memory usage against the visual token count for two different GPU types: an NVIDIA A5000 with 24 GB of memory and an NVIDIA A6000 with 48 GB. UPLiFT maintains linear time and memory scaling with respect… view at source ↗
Figure 10
Figure 10. Figure 10: Comparison of semantic drift in LiFT and semantic stability in UPLiFT. We visualize intermediate feature upsampling steps through PCA, following [15]. LiFT shows signs of feature drift, with local features becoming murkier and more distorted in deeper steps. This drift can lead to poor performance in downstream tasks, as the strength of the original backbone representation is lost. UPLiFT maintains consis… view at source ↗
Figure 11
Figure 11. Figure 11: UPLiFT vs. Latent Space Bilinear 4× upsampling for image super-resolution. We compare the latency of UPLiFT versus applying bilinear upsampling in latent space for image super-resolution to demonstrate UPLiFT’s state-of-the-art efficiency. While only 8.47% slower end-to-end, UPLiFT produces significantly better visual fidelity, as shown by the zoomed-in views (bottom) which display the source low-resoluti… view at source ↗
Figure 12
Figure 12. Figure 12: UPLiFT 512×512 → 2048×2048 upsampled images using Stable Diffusion 1.5. We apply our VAE UPLiFT model to this task in a 4× upsampling configuration. UPLiFT upsamples latents corresponding to 512×512 images generated using 50 diffusion steps on Stable Diffusion 1.5. Best viewed zoomed in. 20 [PITH_FULL_IMAGE:figures/full_fig_p020_12.png] view at source ↗
Figure 13
Figure 13. Figure 13: UPLiFT 512×512 → 1024×1024 upsampled images using Stable Diffusion 1.5. We apply our VAE UPLiFT model to this task in a 2× upsampling configuration. UPLiFT upsamples latents corresponding to 512×512 images generated using 50 diffusion steps on Stable Diffusion 1.5, and the end-to-end latency is 2.75 seconds on an NVIDIA A100 GPU. The UPLiFT model itself takes only 104 milliseconds of this time. Best viewe… view at source ↗
Figure 14
Figure 14. Figure 14: UPLiFT 256×256 → 1024×1024 super-resolution samples from FacesHQ. We use our VAE UPLiFT model that is not fine-tuned for image super-resolution and is only trained in latent space. Our end-to-end upsampling time is only 270.9 milliseconds on an NVIDIA A100 GPU. Best viewed zoomed in. 22 [PITH_FULL_IMAGE:figures/full_fig_p022_14.png] view at source ↗
Figure 15
Figure 15. Figure 15: UPLiFT 256×256 → 1024×1024 super-resolution samples from LHQ. We use our VAE UPLiFT model, which is trained as a generalist model and is not specifically fine-tuned for this dataset. The LHQ dataset presents a greater challenge than FacesHQ, based on the diversity of visual textures present. Despite this challenge, we see good performance with our generalist UPLiFT. In comparison, [41] uses a fine-tuned m… view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

56 extracted references · 19 linked inside Pith

  1. [1]

    Stochastic interpolants: A unifying framework for flows and diffusions.arXiv preprint arXiv:2303.08797,

    Michael S Albergo, Nicholas M Boffi, and Eric Vanden- Eijnden. Stochastic interpolants: A unifying framework for flows and diffusions.arXiv preprint arXiv:2303.08797,

    Pith/arXiv arXiv

  2. [2]

    Deep vit features as dense visual descriptors.arXiv preprint arXiv:2112.05814, 2(3):4, 2021

    Shir Amir, Yossi Gandelsman, Shai Bagon, and Tali Dekel. Deep vit features as dense visual descriptors.arXiv preprint arXiv:2112.05814, 2(3):4, 2021. 1

    Pith/arXiv arXiv 2021

  3. [3]

    Layer normalization.arXiv preprint arXiv:1607.06450,

    Jimmy Lei Ba, Jamie Ryan Kiros, and Geoffrey E Hin- ton. Layer normalization.arXiv preprint arXiv:1607.06450,

    Pith/arXiv arXiv

  4. [4]

    Emerg- ing properties in self-supervised vision transformers

    Mathilde Caron, Hugo Touvron, Ishan Misra, Herv ´e J´egou, Julien Mairal, Piotr Bojanowski, and Armand Joulin. Emerg- ing properties in self-supervised vision transformers. InPro- ceedings of the IEEE/CVF international conference on com- puter vision, pages 9650–9660, 2021. 1, 2

    2021

  5. [5]

    Improved baselines with momentum contrastive learning

    Xinlei Chen, Haoqi Fan, Ross Girshick, and Kaiming He. Improved baselines with momentum contrastive learning. arXiv preprint arXiv:2003.04297, 2020. 2

    Pith/arXiv arXiv 2003

  6. [6]

    An empirical study of training self-supervised vision transformers

    Xinlei Chen, Saining Xie, and Kaiming He. An empirical study of training self-supervised vision transformers. InPro- ceedings of the IEEE/CVF international conference on com- puter vision, pages 9640–9649, 2021. 2

    2021

  7. [7]

    Fsrnet: End-to-end learning face super-resolution with facial priors

    Yu Chen, Ying Tai, Xiaoming Liu, Chunhua Shen, and Jian Yang. Fsrnet: End-to-end learning face super-resolution with facial priors. InProceedings of the IEEE conference on computer vision and pattern recognition, pages 2492–2501,

  8. [8]

    Learning continuous image representation with local implicit image function

    Yinbo Chen, Sifei Liu, and Xiaolong Wang. Learning continuous image representation with local implicit image function. InProceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 8628–8638,

  9. [9]

    The cityscapes dataset for semantic urban scene understanding

    Marius Cordts, Mohamed Omran, Sebastian Ramos, Timo Rehfeld, Markus Enzweiler, Rodrigo Benenson, Uwe Franke, Stefan Roth, and Bernt Schiele. The cityscapes dataset for semantic urban scene understanding. InProceed- ings of the IEEE conference on computer vision and pattern recognition, pages 3213–3223, 2016. 6

    2016

  10. [10]

    Jafar: Jack up any feature at any resolution.arXiv preprint arXiv:2506.11136, 2025

    Paul Couairon, Loick Chambon, Louis Serrano, Jean- Emmanuel Haugeard, Matthieu Cord, and Nicolas Thome. Jafar: Jack up any feature at any resolution.arXiv preprint arXiv:2506.11136, 2025. 1, 2, 3, 4, 6, 7, 12, 16

    arXiv 2025

  11. [11]

    Pixel recursive super resolution

    Ryan Dahl, Mohammad Norouzi, and Jonathon Shlens. Pixel recursive super resolution. InProceedings of the IEEE inter- national conference on computer vision, pages 5439–5448,

  12. [12]

    Imagenet: A large-scale hierarchical image database

    Jia Deng, Wei Dong, Richard Socher, Li-Jia Li, Kai Li, and Li Fei-Fei. Imagenet: A large-scale hierarchical image database. In2009 IEEE conference on computer vision and pattern recognition, pages 248–255. Ieee, 2009. 6

    2009

  13. [13]

    An image is worth 16x16 words: Transformers for image recognition at scale.arXiv preprint arXiv:2010.11929, 2020

    Alexey Dosovitskiy. An image is worth 16x16 words: Transformers for image recognition at scale.arXiv preprint arXiv:2010.11929, 2020. 1

    Pith/arXiv arXiv 2010

  14. [14]

    The pascal visual object classes challenge: A retrospective.Inter- national journal of computer vision, 111(1):98–136, 2015

    Mark Everingham, SM Ali Eslami, Luc Van Gool, Christo- pher KI Williams, John Winn, and Andrew Zisserman. The pascal visual object classes challenge: A retrospective.Inter- national journal of computer vision, 111(1):98–136, 2015. 6

    2015

  15. [15]

    Featup: A model- agnostic framework for features at any resolution.arXiv preprint arXiv:2403.10516, 2024

    Stephanie Fu, Mark Hamilton, Laura Brandt, Axel Feldman, Zhoutong Zhang, and William T Freeman. Featup: A model- agnostic framework for features at any resolution.arXiv preprint arXiv:2403.10516, 2024. 1, 2, 6, 7, 12, 16, 18

    Pith/arXiv arXiv 2024

  16. [16]

    Momentum contrast for unsupervised visual rep- resentation learning

    Kaiming He, Haoqi Fan, Yuxin Wu, Saining Xie, and Ross Girshick. Momentum contrast for unsupervised visual rep- resentation learning. InProceedings of the IEEE/CVF con- ference on computer vision and pattern recognition, pages 9729–9738, 2020. 2

    2020

  17. [17]

    Masked autoencoders are scalable vision learners

    Kaiming He, Xinlei Chen, Saining Xie, Yanghao Li, Piotr Doll´ar, and Ross Girshick. Masked autoencoders are scalable vision learners. InProceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 16000– 16009, 2022. 2

    2022

  18. [18]

    Loftup: Learning a coordinate- based feature upsampler for vision foundation models.arXiv preprint arXiv:2504.14032, 2025

    Haiwen Huang, Anpei Chen, V olodymyr Havrylov, Andreas Geiger, and Dan Zhang. Loftup: Learning a coordinate- based feature upsampler for vision foundation models.arXiv preprint arXiv:2504.14032, 2025. 1, 2, 3, 4, 6, 12, 13, 16

    Pith/arXiv arXiv 2025

  19. [19]

    Upsample guidance: Scale up diffusion models without training.arXiv preprint arXiv:2404.01709, 2024

    Juno Hwang, Yong-Hyun Park, and Junghyo Jo. Upsample guidance: Scale up diffusion models without training.arXiv preprint arXiv:2404.01709, 2024. 3

    Pith/arXiv arXiv 2024

  20. [20]

    Batch normalization: Accelerating deep network training by reducing internal co- variate shift

    Sergey Ioffe and Christian Szegedy. Batch normalization: Accelerating deep network training by reducing internal co- variate shift. InInternational conference on machine learn- ing, pages 448–456. pmlr, 2015. 13

    2015

  21. [21]

    Auto-encoding varia- tional bayes.arXiv preprint arXiv:1312.6114, 2013

    Diederik P Kingma and Max Welling. Auto-encoding varia- tional bayes.arXiv preprint arXiv:1312.6114, 2013. 2, 3

    Pith/arXiv arXiv 2013

  22. [22]

    Joint bilateral upsampling.ACM Transactions on Graphics (ToG), 26(3):96–es, 2007

    Johannes Kopf, Michael F Cohen, Dani Lischinski, and Matt Uyttendaele. Joint bilateral upsampling.ACM Transactions on Graphics (ToG), 26(3):96–es, 2007. 3, 12

    2007

  23. [23]

    Flux.1 kontext: Flow matching for in-context image generation and editing in latent space,

    Black Forest Labs, Stephen Batifol, Andreas Blattmann, Frederic Boesel, Saksham Consul, Cyril Diagne, Tim Dock- horn, Jack English, Zion English, Patrick Esser, Sumith Ku- lal, Kyle Lacey, Yam Levi, Cheng Li, Dominik Lorenz, Jonas M¨uller, Dustin Podell, Robin Rombach, Harry Saini, Axel Sauer, and Luke Smith. Flux.1 kontext: Flow matching for in-context i...

  24. [24]

    Photo- realistic single image super-resolution using a generative ad- versarial network

    Christian Ledig, Lucas Theis, Ferenc Husz´ar, Jose Caballero, Andrew Cunningham, Alejandro Acosta, Andrew Aitken, Alykhan Tejani, Johannes Totz, Zehan Wang, et al. Photo- realistic single image super-resolution using a generative ad- versarial network. InProceedings of the IEEE conference on computer vision and pattern recognition, pages 4681–4690,

  25. [25]

    Asgdiffusion: Parallel high-resolution generation with asynchronous struc- ture guidance.arXiv preprint arXiv:2412.06163, 2024

    Yuming Li, Peidong Jia, Daiwei Hong, Yueru Jia, Qi She, Rui Zhao, Ming Lu, and Shanghang Zhang. Asgdiffusion: Parallel high-resolution generation with asynchronous struc- ture guidance.arXiv preprint arXiv:2412.06163, 2024. 3

    Pith/arXiv arXiv 2024

  26. [26]

    Microsoft coco: Common objects in context

    Tsung-Yi Lin, Michael Maire, Serge Belongie, James Hays, Pietro Perona, Deva Ramanan, Piotr Doll´ar, and C Lawrence Zitnick. Microsoft coco: Common objects in context. In European conference on computer vision, pages 740–755. Springer, 2014. 6

    2014

  27. [27]

    Accdiffusion v2: Towards more accurate higher-resolution diffusion extrapolation.IEEE Transactions on Pattern Anal- ysis and Machine Intelligence, 2025

    Zhihang Lin, Mingbao Lin, Wengyi Zhan, and Rongrong Ji. Accdiffusion v2: Towards more accurate higher-resolution diffusion extrapolation.IEEE Transactions on Pattern Anal- ysis and Machine Intelligence, 2025. 3

    2025

  28. [28]

    Flow matching for generative mod- eling.arXiv preprint arXiv:2210.02747, 2022

    Yaron Lipman, Ricky TQ Chen, Heli Ben-Hamu, Maximil- ian Nickel, and Matt Le. Flow matching for generative mod- eling.arXiv preprint arXiv:2210.02747, 2022. 3

    Pith/arXiv arXiv 2022

  29. [29]

    Learn- ing to upsample by learning to sample

    Wenze Liu, Hao Lu, Hongtao Fu, and Zhiguo Cao. Learn- ing to upsample by learning to sample. InProceedings of the IEEE/CVF international conference on computer vision, pages 6027–6037, 2023. 2

    2023

  30. [30]

    Flow straight and fast: Learning to generate and transfer data with rectified flow.arXiv preprint arXiv:2209.03003, 2022

    Xingchao Liu, Chengyue Gong, and Qiang Liu. Flow straight and fast: Learning to generate and transfer data with rectified flow.arXiv preprint arXiv:2209.03003, 2022. 3

    Pith/arXiv arXiv 2022

  31. [31]

    Sapa: Similarity-aware point affiliation for feature upsampling.Advances in Neural Information Pro- cessing Systems, 35:20889–20901, 2022

    Hao Lu, Wenze Liu, Zixuan Ye, Hongtao Fu, Yuliang Liu, and Zhiguo Cao. Sapa: Similarity-aware point affiliation for feature upsampling.Advances in Neural Information Pro- cessing Systems, 35:20889–20901, 2022. 2

    2022

  32. [32]

    Lcm-lora: A universal stable-diffusion acceler- ation module.arXiv preprint arXiv:2311.05556, 2023

    Simian Luo, Yiqin Tan, Suraj Patil, Daniel Gu, Patrick V on Platen, Apolin´ario Passos, Longbo Huang, Jian Li, and Hang Zhao. Lcm-lora: A universal stable-diffusion acceler- ation module.arXiv preprint arXiv:2311.05556, 2023. 14

    Pith/arXiv arXiv 2023

  33. [33]

    Pulse: Self-supervised photo upsam- pling via latent space exploration of generative models

    Sachit Menon, Alexandru Damian, Shijia Hu, Nikhil Ravi, and Cynthia Rudin. Pulse: Self-supervised photo upsam- pling via latent space exploration of generative models. In Proceedings of the ieee/cvf conference on computer vision and pattern recognition, pages 2437–2445, 2020. 3

    2020

  34. [34]

    Dinov2: Learning robust visual features without supervision

    Maxime Oquab, Timoth ´ee Darcet, Th ´eo Moutakanni, Huy V o, Marc Szafraniec, Vasil Khalidov, Pierre Fernandez, Daniel Haziza, Francisco Massa, Alaaeldin El-Nouby, et al. Dinov2: Learning robust visual features without supervision. arXiv preprint arXiv:2304.07193, 2023. 1, 2, 6, 12, 13, 16

    Pith/arXiv arXiv 2023

  35. [35]

    Sdxl: Improving latent diffusion mod- els for high-resolution image synthesis.arXiv preprint arXiv:2307.01952, 2023

    Dustin Podell, Zion English, Kyle Lacey, Andreas Blattmann, Tim Dockhorn, Jonas M ¨uller, Joe Penna, and Robin Rombach. Sdxl: Improving latent diffusion mod- els for high-resolution image synthesis.arXiv preprint arXiv:2307.01952, 2023. 14

    Pith/arXiv arXiv 2023

  36. [36]

    Freescale: Unleashing the resolution of diffusion models via tuning-free scale fusion

    Haonan Qiu, Shiwei Zhang, Yujie Wei, Ruihang Chu, Hangjie Yuan, Xiang Wang, Yingya Zhang, and Ziwei Liu. Freescale: Unleashing the resolution of diffusion models via tuning-free scale fusion. InProceedings of the IEEE/CVF International Conference on Computer Vision, pages 16893– 16903, 2025. 3

    2025

  37. [37]

    Learning transferable visual models from natural language supervi- sion

    Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, et al. Learning transferable visual models from natural language supervi- sion. InInternational conference on machine learning, pages 8748–8763. PmLR, 2021. 2

    2021

  38. [38]

    High-resolution image syn- thesis with latent diffusion models, 2021

    Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Bj¨orn Ommer. High-resolution image syn- thesis with latent diffusion models, 2021. 3, 8, 13

    2021

  39. [39]

    High-resolution image synthesis with latent diffusion models

    Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Bj ¨orn Ommer. High-resolution image synthesis with latent diffusion models. InProceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 10684–10695, 2022. 3

    2022

  40. [40]

    Image super- resolution via iterative refinement.IEEE transactions on pattern analysis and machine intelligence, 45(4):4713–4726,

    Chitwan Saharia, Jonathan Ho, William Chan, Tim Sali- mans, David J Fleet, and Mohammad Norouzi. Image super- resolution via iterative refinement.IEEE transactions on pattern analysis and machine intelligence, 45(4):4713–4726,

  41. [41]

    Fmboost: Boosting latent diffusion with flow matching

    Johannes Schusterbauer, Ming Gui, Pingchuan Ma, Nick Stracke, Stefan Andreas Baumann, Vincent Tao Hu, and Bj¨orn Ommer. Fmboost: Boosting latent diffusion with flow matching. InEuropean Conference on Computer Vision, pages 338–355. Springer, 2024. 2, 3, 7, 8, 13, 14, 15, 23

    2024

  42. [42]

    Dinov3.arXiv preprint arXiv:2508.10104, 2025

    Oriane Sim ´eoni, Huy V V o, Maximilian Seitzer, Federico Baldassarre, Maxime Oquab, Cijo Jose, Vasil Khalidov, Marc Szafraniec, Seungeun Yi, Micha ¨el Ramamonjisoa, et al. Dinov3.arXiv preprint arXiv:2508.10104, 2025. 1, 2, 12, 16

    Pith/arXiv arXiv 2025

  43. [43]

    Lift: A surprisingly simple lightweight feature transform for dense vit descriptors

    Saksham Suri, Matthew Walmer, Kamal Gupta, and Abhinav Shrivastava. Lift: A surprisingly simple lightweight feature transform for dense vit descriptors. InEuropean Conference on Computer Vision, pages 110–128. Springer, 2024. 1, 2, 3, 6, 12, 16

    2024

  44. [44]

    Is one gpu enough? pushing image generation at higher-resolutions with founda- tion models.arXiv preprint arXiv:2406.07251, 2(3):5, 2024

    Athanasios Tragakis, Marco Aversa, Chaitanya Kaul, Roder- ick Murray-Smith, and Daniele Faccio. Is one gpu enough? pushing image generation at higher-resolutions with founda- tion models.arXiv preprint arXiv:2406.07251, 2(3):5, 2024. 3

    Pith/arXiv arXiv 2024

  45. [45]

    Siglip 2: Multilingual vision-language en- coders with improved semantic understanding, localization, and dense features.arXiv preprint arXiv:2502.14786, 2025

    Michael Tschannen, Alexey Gritsenko, Xiao Wang, Muham- mad Ferjad Naeem, Ibrahim Alabdulmohsin, Nikhil Parthasarathy, Talfan Evans, Lucas Beyer, Ye Xia, Basil Mustafa, et al. Siglip 2: Multilingual vision-language en- coders with improved semantic understanding, localization, and dense features.arXiv preprint arXiv:2502.14786, 2025. 2

    Pith/arXiv arXiv 2025

  46. [46]

    Unsplash Full, Lite Dataset 1.3.0, 2025

    Unsplash. Unsplash Full, Lite Dataset 1.3.0, 2025. Ac- cessed: 14 November 2025. 8

    2025

  47. [47]

    Attention is all you need.Advances in neural information processing systems, 30, 2017

    Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszko- reit, Llion Jones, Aidan N Gomez, Łukasz Kaiser, and Illia Polosukhin. Attention is all you need.Advances in neural information processing systems, 30, 2017. 3

    2017

  48. [48]

    Teaching matters: Investigating the role of supervision in vision transformers

    Matthew Walmer, Saksham Suri, Kamal Gupta, and Abhi- nav Shrivastava. Teaching matters: Investigating the role of supervision in vision transformers. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 7486–7496, 2023. 2, 4

    2023

  49. [49]

    Carafe: Content-aware reassembly of fea- tures

    Jiaqi Wang, Kai Chen, Rui Xu, Ziwei Liu, Chen Change Loy, and Dahua Lin. Carafe: Content-aware reassembly of fea- tures. InProceedings of the IEEE/CVF international confer- ence on computer vision, pages 3007–3016, 2019. 2

    2019

  50. [50]

    Image quality assessment: from error visibility to 10 structural similarity.IEEE transactions on image processing, 13(4):600–612, 2004

    Zhou Wang, Alan C Bovik, Hamid R Sheikh, and Eero P Si- moncelli. Image quality assessment: from error visibility to 10 structural similarity.IEEE transactions on image processing, 13(4):600–612, 2004. 14

    2004

  51. [51]

    Anyup: Universal feature upsampling

    Thomas Wimmer, Prune Truong, Marie-Julie Rakotosaona, Michael Oechsle, Federico Tombari, Bernt Schiele, and Jan Eric Lenssen. Anyup: Universal feature upsampling. arXiv preprint arXiv:2510.12764, 2025. 1, 2, 3, 4, 6, 12, 16

    arXiv 2025

  52. [52]

    Rectifiedhr: Enable efficient high-resolution image generation via energy rectification.arXiv e-prints, pages arXiv–2503, 2025

    Zhen Yang, Guibao Shen, Liang Hou, Mushui Liu, Luozhou Wang, Xin Tao, Pengfei Wan, Di Zhang, and Ying-Cong Chen. Rectifiedhr: Enable efficient high-resolution image generation via energy rectification.arXiv e-prints, pages arXiv–2503, 2025. 3

    2025

  53. [53]

    Sigmoid loss for language image pre-training

    Xiaohua Zhai, Basil Mustafa, Alexander Kolesnikov, and Lucas Beyer. Sigmoid loss for language image pre-training. InProceedings of the IEEE/CVF international conference on computer vision, pages 11975–11986, 2023. 2

    2023

  54. [54]

    Semantic under- standing of scenes through the ade20k dataset.International Journal of Computer Vision, 127(3):302–321, 2019

    Bolei Zhou, Hang Zhao, Xavier Puig, Tete Xiao, Sanja Fi- dler, Adela Barriuso, and Antonio Torralba. Semantic under- standing of scenes through the ade20k dataset.International Journal of Computer Vision, 127(3):302–321, 2019. 6

    2019

  55. [55]

    LiFT” runs the model four times for16×upsampling and “LiFT-2×

    Minghao Zhou, Hong Wang, Yefeng Zheng, and Deyu Meng. A refreshed similarity-based upsampler for di- rect high-ratio feature upsampling.arXiv preprint arXiv:2407.02283, 2024. 2 11 A. Additional Details for Predictive Tasks In this work, we focus on comparing UPLiFT with other task-agnostic feature upsamplers, which have grown in pop- ularity in recent yea...

    Pith/arXiv arXiv 2024

  56. [56]

    Best viewed zoomed in

    uses a fine-tuned model with3×the parameter count for evaluations on LHQ versus FacesHQ. Best viewed zoomed in. 23