Pith. sign in

REVIEW 4 major objections 6 minor 71 references

DiffRIS: Enhancing Referring Remote Sensing Image Segmentation with Pre-trained Text-to-Image Diffusion Models

T0 review · 4 major / 6 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read DiffRIS claims that frozen text-to-image diffusion features, refined by a language adapter and a progressive cross-modal decoder, achieve state-of-the-art referring remote sensing image segmentation on three benchmarks.

desk verdict Solid first diffusion-based RRSIS paper with genuine mIoU gains, but the abstract overclaims 'all standard metrics' and the ablation tables appear to swap mIoU/oIoU; worth reviewing after corrections. read the letter →

arxiv 2506.18946 v1 pith:3BM67OIS submitted 2025-06-23 cs.CV cs.AI

Zhe Dong , Yuzhe Sun , Tianzhu Liu , Yanfeng Gu This is my paper
classification cs.CVcs.AI
keywords referringremotesensingimagesegmentationtext-to-imagediffusionmodelscross-modalalignmentStablecontextperceptionadapterprogressivereasoningdecoderGumbel-softmaxhardassignmentRRSISbenchmarks
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper introduces DiffRIS, a framework that turns a pre-trained text-to-image diffusion model into the visual backbone for referring remote sensing image segmentation (RRSIS), the task of delineating objects described in natural language. The authors claim that, with the diffusion model kept frozen, a context-perception adapter (CP-adapter) that refines the text features and a progressive cross-modal reasoning decoder (PCMRD) that aligns those features with multi-scale visual regions are enough to outperform all prior methods on three benchmarks. The reported gains are substantial on the RefSegRS test split, where mIoU rises to 62.38 versus 57.30 for the closest diffusion-based competitor, and on RISBench the test mIoU reaches 67.04. If correct, the result would show that generative text-to-image models' latent features carry reusable cross-modal alignment for a discriminative recognition task.

What carries the argument

The load-bearing machinery is the frozen denoising UNet of Stable Diffusion treated as a multi-scale feature extractor: the clean latent image, with no added noise, is passed through once and the four stages' outputs become the visual representations $V_i$. Around it sit two trainable components: the CP-adapter, which runs the CLIP text features through a Transformer encoder, an object-aware attention over the original text, and a zero-initialized low-rank adjustment layer to produce refined linguistic features $\hat{L}$; and the PCMRD, whose object-aware query interaction layers (OAQIL) first inject text into learnable query tokens $Q$ via cross-attention, then assign fused cross-scale visual features to tokens through a Gumbel-softmax hard-assignment step $S_{\mathrm{onehot}}$ with a straight-through gradient. Only the adapter and decoder are optimized, so the pretrained weights are preserved.

What would settle it

Run DiffRIS with the same CP-adapter and PCMRD but replace the frozen denoising UNet with a standard vision backbone such as Swin-B or a plain ViT, training under the identical protocol; if the mIoU gap on the RRSIS-D test set (62.19) largely disappears, the central attribution to diffusion features would be falsified. Alternatively, feed the UNet a slightly noised latent or vary the timestep conditioning and show that performance collapses, confirming that the clean single-pass assumption is the active ingredient.

Watch

Extended reading notes

Core claim

The paper's central claim is that the latent features of a pre-trained text-to-image diffusion model, extracted by a single forward pass through a denoising UNet on a clean latent, encode semantic alignment between language and aerial imagery that transfers to referring remote sensing image segmentation. DiffRIS feeds the image through VQGAN into the latent space, encodes the referring expression with CLIP, refines the language features through the CP-adapter, and routes both into the frozen UNet to obtain four scales of visual features; the PCMRD then lets learnable query tokens alternately attend to text and group visual features, using a Gumbel-softmax hard assignment. The authors report state-of-the-art results on RRSIS-D, RefSegRS, and RISBench across precision thresholds Pr@0.5 to Pr@0.9, oIoU, and mIoU, with test mIoU values of 62.19, 62.38, and 67.04, respectively. They also show performance improving monotonically with more pre-training iterations of the Stable Diffusion weights (SD-1-1 through SD-1-5), which they read as evidence that the pretrained knowledge is what carries the gain.

Load-bearing premise

The load-bearing premise is that a single forward pass of a clean, noise-free latent through a denoising UNet, which was trained for iterative denoising, yields semantically aligned multi-scale features for remote sensing segmentation; the paper does not separately validate that this unusual operating mode is what drives the improvement.

Editorial extensions

If this is right

  • Diffusion-based backbones can serve as frozen visual encoders for discriminative dense prediction, not just generators.
  • The text-refinement adapter and query-based decoder together close much of the domain gap between general vision-language pretraining and remote sensing imagery.
  • Performance scaling with Stable Diffusion's pretraining iterations implies that better generative checkpoints may directly yield better segmentation without redesigning the framework.
  • The reported state-of-the-art numbers give concrete baselines (test mIoU of 62.19, 62.38, and 67.04 on RRSIS-D, RefSegRS, and RISBench) for later RRSIS work to beat.
  • Because all pretrained components stay frozen, the method could be adapted to new datasets by training only the small adapter and decoder.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The paper does not isolate how much of the gain comes from the diffusion UNet versus the CP-adapter and PCMRD; a natural test is to swap the UNet for a non-diffusion backbone under identical adapters, which the ablations do not include.
  • The single clean forward pass through a denoiser is an unusual operating mode; probing different timestep conditioning or small noise levels could reveal whether the features exploit the denoising inductive bias or simply act as a strong pretrained encoder.
  • The same recipe of CLIP text plus frozen diffusion features plus a query-token decoder with Gumbel assignment should transfer to referring segmentation in natural images or other vision-language dense prediction tasks, since nothing in the method is remote-sensing-specific except the evaluation.
  • A practical consequence the authors do not develop is that, because most parameters are frozen, DiffRIS may be deployable with a distilled or precomputed UNet, though the paper itself notes real-time deployment remains an open limitation.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 6 minor

Summary. The paper proposes DiffRIS, an RRSIS method that uses a frozen Stable Diffusion UNet (single clean forward pass) and a CLIP text encoder as backbones, a CP-adapter to refine text features, and a PCMRD decoder with learnable query tokens and Gumbel-softmax hard assignment. Experiments on RRSIS-D, RefSegRS, and RISBench report mIoU improvements over prior methods on all three test sets, and the paper claims state-of-the-art across all standard metrics.

Significance. If the mIoU gains hold, this is a valuable demonstration that text-to-image diffusion features transfer to remote sensing referring segmentation, with particularly large margins at high precision thresholds on RefSegRS and RISBench. The paper includes extensive comparisons, ablations, and visualizations. However, the universal SOTA claim is contradicted by the paper's own oIoU columns, and the ablation tables have a label-swapping inconsistency; these issues must be resolved before the results can be fully trusted.

major comments (4)
  1. [Abstract, Section 4.2.1, Tables 1-3] The abstract and conclusion claim that DiffRIS 'consistently outperforms existing methods across all standard metrics,' but the tables show otherwise. In Table 1 (RRSIS-D test), DiffRIS oIoU is 76.05, below robust-ref-seg (77.40) and CARIS (77.17). In Table 3 (RISBench test), DiffRIS oIoU is 73.00, below CARIS (75.10), RIS-DMMI (74.82), and CrossVLT (74.33). In Table 2 (RefSegRS test), DiffRIS oIoU is 76.18, below VPD (76.24). Thus the state-of-the-art claim holds for mIoU and several Pr@X thresholds, but not for oIoU; the claim and the conclusion must be narrowed or the tables corrected.
  2. [Tables 4-6, Section 4.3] The ablation tables appear to swap the mIoU and oIoU columns. Table 1 reports RRSIS-D test mIoU=62.19 and oIoU=76.05 for DiffRIS, but Tables 4-6 report the full model as mIoU=76.05 and oIoU=62.19. The text in Section 4.3.2 follows the same reversed pattern. Since the ablated values are described relative to these numbers, the quantitative support for the ablations is internally inconsistent as presented. Please correct the labels or the values and ensure the discussion matches.
  3. [Section 3.2] The paper does not specify how the refined linguistic features L' condition the denoising UNet. The text states that L' and the latent image representation z are fed into the UNet, but it does not state whether text is injected via cross-attention, concatenation, or another mechanism, nor whether the UNet's original text-conditioning layers are used. This is a load-bearing architectural detail for a method whose central claim is that pre-trained text-to-image diffusion knowledge improves cross-modal alignment; please provide the exact formulation and a diagram or equation.
  4. [Section 3.2 / Section 4.3] The framework uses a clean latent (no noise) and a single forward pass through the denoising UNet, which is a non-standard use of the model. This choice is not validated: the paper does not compare against timestep-conditioned features, a small added noise, or multiple forward passes, and it does not ablate the UNet against the CLIP visual encoder to isolate the contribution of the diffusion backbone. Without such an analysis, the reader cannot determine how much of the reported gain is attributable to the diffusion model rather than to the CP-adapter/PCMRD and the CLIP text encoder.
minor comments (6)
  1. [Section 2.3] There is a duplicated sentence describing the LGCE module: the sentence beginning 'They also introduced a language-guided cross-scale enhancement...' is immediately followed by a near-repetition in the next sentence.
  2. [Equation (2)] Equation (2) in Section 3.1 appears to have a typo: the mean of q(xt|xt-1) should be sqrt(1 - beta_t) xt-1, not sqrt(1 - beta_{t-1}).
  3. [Table 5] In Table 5, the CAMD row has '76.4161.86' without a separator; please fix the formatting.
  4. [Figure 2 caption] The caption contains the typo 'Difussion Image Encoder'; it should be 'Diffusion Image Encoder'.
  5. [Table 3 caption] The caption uses 'RisBench' while the text and other tables use 'RISBench'; please standardize.
  6. [Equation (12)] The 'low-rank linear transformation matrix' in Eq. (12) is not defined as a product of two low-rank factors; please clarify whether this is a LoRA-style decomposition and specify the rank.

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity found: the method is evaluated against external benchmarks and the architecture is not derived from its own conclusions; the internal table inconsistencies are correctness issues, not circularity.

full rationale

The paper's central claim is an empirical performance comparison on three public benchmarks (RRSIS-D, RefSegRS, RISBench) against externally published methods. The proposed DiffRIS is assembled from frozen pre-trained diffusion/CLIP encoders plus a trained CP-adapter and PCMRD; there is no fitted parameter that is later renamed as a prediction, and no uniqueness theorem or design choice is imported solely from the authors' prior work. The only self-citation that is potentially load-bearing in the broadest sense is Ref. [49], the authors' CroBIM paper, which introduced the RISBench dataset; however, RISBench is a public benchmark with independently defined triplets and is used as an external test bed, not as a source of the method's derivation or of its state-of-the-art conclusion. The paper does exhibit internal quantitative inconsistencies that should be flagged as correctness risks, not circularity: the abstract claims DiffRIS 'consistently outperforms existing methods across all standard metrics,' but Tables 1 and 3 show that on the oIoU metric DiffRIS is not the best on RRSIS-D test (76.05 vs. robust-ref-seg 77.40 and CARIS 77.17) or on RISBench test (73.00 vs. CARIS 75.10, RIS-DMMI 74.82, CrossVLT 74.33). Additionally, Tables 4-6 appear to swap mIoU and oIoU relative to Table 1 (the full model is listed as mIoU=76.05/oIoU=62.19 in Table 4, whereas Table 1 gives mIoU=62.19/oIoU=76.05 on the same RRSIS-D test set). These inconsistencies affect the credibility of the reported numbers but do not constitute a circular derivation. The conclusion's stated limitations (limited interpretability, extreme-condition robustness, computational cost) are acknowledged shortcomings and do not undermine the independence of the empirical evaluation. Because the central claims rest on external benchmark comparisons rather than on a self-referential derivation or on fitted inputs being passed off as predictions, no significant circularity is present. A score of 0 is therefore appropriate, with the caveat that the numerical tables need correction before the headline SOTA claim can be accepted as stated.

Assumptions & free parameters 3 free parameters · 4 assumptions · 0 invented entities

This is a supervised deep learning paper; the only new components are network modules (CP-adapter and PCMRD) rather than physical entities. Free parameters listed are hyperparameters or learnable coefficients whose values are not reported and on which performance may depend.

free parameters (3)
  • Number of query tokens M = Not reported
    The decoder uses M learnable query tokens; the paper does not state M or ablate it, yet the hard-assignment capacity depends on this choice.
  • Gumbel temperature tau = Learnable
    Equation 17 uses a learnable tau for the Gumbel-softmax; the paper does not report its final value or sensitivity.
  • CP-adapter fusion weight alpha = Learnable
    Equation 13 blends adapted and original linguistic features with a learnable alpha; its learned value is not reported.
assumptions (4)
  • domain assumption Frozen pre-trained Stable Diffusion and CLIP encoders provide useful cross-modal features for remote sensing imagery
    The entire framework rests on this transfer; the paper argues it but never isolates whether the diffusion encoder or CLIP alone drives gains.
  • ad hoc to paper A single clean forward pass through the denoising UNet yields meaningful visual features
    Section 3.2: no noise is added and the denoising process is reduced to one pass, which is unconventional for a model trained on noisy inputs.
  • domain assumption Benchmark datasets (RRSIS-D, RefSegRS, RISBench) are correctly labeled and the standard evaluation protocol is appropriate
    The paper relies on these datasets without independent verification; one benchmark (RISBench) was introduced by the same group.
  • standard math Gumbel-softmax with straight-through estimator is a valid differentiable assignment
    The paper uses this technique (Equations 17-19) without proof; it is a standard method in deep learning.

how reviews work

0 comments
Cite this review

Pith. "Pith review of DiffRIS: Enhancing Referring Remote Sensing Image Segmentation with Pre-trained Text-to-Image Diffusion Models." pith.science (2026). https://pith.science/paper/3BM67OIS

@misc{pith2026250618946,
  author       = {Pith},
  title        = {Pith review of: DiffRIS: Enhancing Referring Remote Sensing Image Segmentation with Pre-trained Text-to-Image Diffusion Models},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/3BM67OIS}},
  note         = {Machine review of arXiv:2506.18946}
}
read the original abstract

Referring remote sensing image segmentation (RRSIS) enables the precise delineation of regions within remote sensing imagery through natural language descriptions, serving critical applications in disaster response, urban development, and environmental monitoring. Despite recent advances, current approaches face significant challenges in processing aerial imagery due to complex object characteristics including scale variations, diverse orientations, and semantic ambiguities inherent to the overhead perspective. To address these limitations, we propose DiffRIS, a novel framework that harnesses the semantic understanding capabilities of pre-trained text-to-image diffusion models for enhanced cross-modal alignment in RRSIS tasks. Our framework introduces two key innovations: a context perception adapter (CP-adapter) that dynamically refines linguistic features through global context modeling and object-aware reasoning, and a progressive cross-modal reasoning decoder (PCMRD) that iteratively aligns textual descriptions with visual regions for precise segmentation. The CP-adapter bridges the domain gap between general vision-language understanding and remote sensing applications, while PCMRD enables fine-grained semantic alignment through multi-scale feature interaction. Comprehensive experiments on three benchmark datasets-RRSIS-D, RefSegRS, and RISBench-demonstrate that DiffRIS consistently outperforms existing methods across all standard metrics, establishing a new state-of-the-art for RRSIS tasks. The significant performance improvements validate the effectiveness of leveraging pre-trained diffusion models for remote sensing applications through our proposed adaptive framework.

Figures

Figures reproduced from arXiv: 2506.18946 by the authors.

Figure 1
Figure 1. Schematic illustration of diffusion probabilistic models. The forward diffusion process (top) gradually adds noise to the original image. The backward diffusion process (bottom) learns to reverse this degradation through a denoising UNet, progressively recovering the original image from pure noise. trained via variational inference. The primary objective of diffusion models is to facilitate sample generation by pres… view at source ↗
Figure 2
Figure 2. Architecture of the proposed DiffRIS framework for referring remote sensing image segmentation. The framework consists of three main components: (1) pre-trained CLIP text encoder, diffusion image encoder and denoising UNet that collaboratively extract multi-modal features, (2) a context perception adapter (CP-adapter) that refines linguistic features and bridges the domain gap, and (3) a progressive cross-modal reas… view at source ↗
Figure 3
Figure 3. Detailed architecture of the proposed CP-adapter. The adapter [PITH_FULL_IMAGE:figures/full_fig_p006_3.png] view at source ↗
Figures from the paper (5 more)
Figure 4
Figure 4. Figure 4: For simplicity, we take the first decoding process as [PITH_FULL_IMAGE:figures/full_fig_p007_4.png]
Figure 5
Figure 5. Figure 5: Visualization of segmentation results for Di [PITH_FULL_IMAGE:figures/full_fig_p010_5.png]
Figure 6
Figure 6. Figure 6: Visualization of segmentation results for Di [PITH_FULL_IMAGE:figures/full_fig_p010_6.png]
Figure 7
Figure 7. Figure 7: Visualization of segmentation results for Di [PITH_FULL_IMAGE:figures/full_fig_p010_7.png]
Figure 8
Figure 8. Figure 8: Performance comparison of DiffRIS using different pre-trained weights from Stable Diffusion models on the RRSIS-D test set. The metrics mIoU and oIoU are plotted for four versions of Stable Diffusion (SD-1-1, SD￾1-2, SD-1-4, SD-1-5), demonstrating a consistent improvem…

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

71 extracted references · 56 canonical work pages

  1. [1]

    Kaplan, T

    G. Kaplan, T. Rashid, M. Gasparovic, A. Pietrelli, V . Ferrara, Monitoring war-generated environmental security using remote sensing: A review, Land degradation & development 33 (10) (2022) 1513–1526

    work page 2022

  2. [2]

    Rolnick, P

    D. Rolnick, P. L. Donti, L. H. Kaack, K. Kochanski, A. Lacoste, K. Sankaran, A. S. Ross, N. Milojevic- Dupont, N. Jaques, A. Waldman-Brown, et al., Tackling climate change with machine learning, ACM Computing Surveys (CSUR) 55 (2) (2022) 1–96

    work page 2022

  3. [3]

    Huang, J

    J. Huang, J. Stoter, R. Peters, L. Nan, City3d: Large-scale building reconstruction from airborne lidar point clouds, Remote Sensing 14 (9) (2022) 2254

    work page 2022

  4. [4]

    Avtar, A

    R. Avtar, A. A. Komolafe, A. Kouser, D. Singh, A. P. Yunus, J. Dou, P. Kumar, R. D. Gupta, B. A. Johnson, H. V . T. Minh, et al., Assessing sustainable development prospects through remote sensing: A review, Remote sensing applications: Society and environment 20 (2020) 100402

    work page 2020

  5. [5]

    Z. Dong, G. Gao, T. Liu, Y . Gu, X. Zhang, Distilling segmenters from cnns and transformers for remote sensing images semantic segmentation, IEEE Transactions on Geoscience and Remote Sensing (2023)

    work page 2023

  6. [6]

    Z. Dong, T. Liu, Y . Gu, Spatial and semantic consistency contrastive learning for self-supervised semantic segmentation of remote sensing images, IEEE Transactions on Geoscience and Remote Sensing (2023)

    work page 2023

  7. [7]

    G. Luo, Y . Zhou, R. Ji, X. Sun, J. Su, C.-W. Lin, Q. Tian, Cascade grouped attention network for referring expression segmentation, in: Proceedings of the 28th ACM International Conference on Multimedia, 2020, pp. 1274–1282

    work page 2020

  8. [8]

    Z. Hu, G. Feng, J. Sun, L. Zhang, H. Lu, Bi- directional relationship inferring network for referring image segmentation, in: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, 2020, pp. 4424–4433

    work page 2020

Show all 71 references
  1. [9]

    S. Yang, M. Xia, G. Li, H.-Y . Zhou, Y . Yu, Bottom-up shift and reasoning for referring image segmentation, in: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2021, pp. 11266–11275

    2021

  2. [10]

    Zhang, Y

    Z. Zhang, Y . Zhu, J. Liu, X. Liang, W. Ke, Coupalign: Coupling word-pixel with sentence-mask alignments for referring image segmentation, Advances in Neural Information Processing Systems 35 (2022) 14729–14742

    2022

  3. [11]

    B. Chen, Z. Hu, Z. Ji, J. Bai, W. Zuo, Position-aware contrastive alignment for referring image segmentation, arXiv preprint arXiv:2212.13419 (2022)

    2022 arXiv

  4. [12]

    C. Zhu, Y . Zhou, Y . Shen, G. Luo, X. Pan, M. Lin, C. Chen, L. Cao, X. Sun, R. Ji, Seqtr: A simple yet universal network for visual grounding, in: European Conference on Computer Vision, Springer, 2022, pp. 598– 615

    2022

  5. [13]

    J. Liu, H. Ding, Z. Cai, Y . Zhang, R. K. Satzoda, V . Mahadevan, R. Manmatha, Polyformer: Referring image segmentation as sequential polygon generation, in: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2023, pp. 18653–18663

    2023

  6. [14]

    G. Luo, Y . Zhou, X. Sun, L. Cao, C. Wu, C. Deng, R. Ji, Multi-task collaborative network for joint referring expression comprehension and segmentation, in: Proceedings of the IEEE/CVF Conference on computer vision and pattern recognition, 2020, pp. 10034–10043

    2020

  7. [15]

    Huang, T

    S. Huang, T. Hui, S. Liu, G. Li, Y . Wei, J. Han, L. Liu, B. Li, Referring image segmentation via cross- modal progressive comprehension, in: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, 2020, pp. 10488–10497

    2020

  8. [16]

    H. Ding, C. Liu, S. Wang, X. Jiang, Vision- language transformer and query generation for referring segmentation, in: Proceedings of the IEEE/CVF International Conference on Computer Vision, 2021, pp. 16321–16330

    2021

  9. [17]

    Zhang, A

    L. Zhang, A. Rao, M. Agrawala, Adding conditional control to text-to-image diffusion models, in: Proceedings of the IEEE/CVF International Conference on Computer Vision, 2023, pp. 3836–3847

    2023

  10. [18]

    Saharia, W

    C. Saharia, W. Chan, S. Saxena, L. Li, J. Whang, E. L. Denton, K. Ghasemipour, R. Gontijo Lopes, B. Karagol Ayan, T. Salimans, et al., Photorealistic text-to-image diffusion models with deep language understanding, Advances in neural information processing systems 35 (2022) 36...

    2022

  11. [19]

    Schuhmann, R

    C. Schuhmann, R. Beaumont, R. Vencu, C. Gordon, R. Wightman, M. Cherti, T. Coombes, A. Katta, C. Mullis, M. Wortsman, et al., Laion-5b: An open large-scale dataset for training next generation image-text models, Advances in Neural Information Processing Systems 35 (2022) 25278–25294

    2022

  12. [20]

    R. Ou, H. Yan, M. Wu, C. Zhang, A method of efficient synthesizing post-disaster remote sensing image with diffusion model and llm, in: 2023 Asia Pacific Signal and Information Processing Association Annual Summit and Conference (APSIPA ASC), IEEE, 2023, pp. 1549–1555

    2023

  13. [21]

    Rombach, A

    R. Rombach, A. Blattmann, D. Lorenz, P. Esser, B. Ommer, High-resolution image synthesis with latent diffusion models, in: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, 2022, pp. 10684–10695

    2022

  14. [22]

    Khanna, P

    S. Khanna, P. Liu, L. Zhou, C. Meng, R. Rombach, M. Burke, D. B. Lobell, S. Ermon, Diffusionsat: A generative foundation model for satellite imagery, in: The Twelfth International Conference on Learning Representations, 2023

    2023

  15. [23]

    D. Tang, X. Cao, X. Hou, Z. Jiang, D. Meng, Crs-diff: Controllable generative remote sensing foundation model, arXiv preprint arXiv:2403.11614 (2024)

    2024 arXiv

  16. [24]

    Z. Wu, J. Qian, M. Zhang, Y . Cao, T. Wang, L. Yang, High-confidence sample augmentation based on label- guided denoising diffusion probabilistic model for active deception jamming recognition, IEEE Geoscience and Remote Sensing Letters (2023)

    2023

  17. [25]

    Espinosa, E

    M. Espinosa, E. J. Crowley, Generate your own scotland: Satellite image generation conditioned on maps, arXiv preprint arXiv:2308.16648 (2023)

    2023 arXiv

  18. [26]

    Z. Yuan, C. Hao, R. Zhou, J. Chen, M. Yu, W. Zhang, H. Wang, X. Sun, Efficient and controllable remote sensing fake sample generation based on diffusion model, IEEE Transactions on Geoscience and Remote Sensing 61 (2023) 1–12

    2023

  19. [27]

    C. Zhao, Y . Ogawa, S. Chen, Z. Yang, Y . Sekimoto, Label freedom: Stable diffusion for remote sensing image semantic segmentation data generation, in: 2023 IEEE International Conference on Big Data (BigData), IEEE, 2023, pp. 1022–1030

    2023

  20. [28]

    Baghirli, H

    O. Baghirli, H. Askarov, I. Ibrahimli, I. Bakhishov, N. Nabiyev, Satdm: Synthesizing realistic satellite image with semantic layout conditioning using diffusion models, arXiv preprint arXiv:2309.16812 (2023)

    2023 arXiv

  21. [29]

    S. Shi, L. Zhang, J. Chen, Hyperspectral and multispectral image fusion using the conditional denoising diffusion probabilistic model, arXiv preprint arXiv:2307.03423 (2023)

    2023 arXiv

  22. [30]

    S. Xie, Z. Tu, Holistically-nested edge detection, in: Proceedings of the IEEE international conference on computer vision, 2015, pp. 1395–1403

    2015

  23. [31]

    Czerkawski, C

    M. Czerkawski, C. Tachtatzis, Exploring the capability of text-to-image diffusion models with structural edge guidance for multi-spectral satellite image inpainting, IEEE Geoscience and Remote Sensing Letters (2024)

    2024

  24. [32]

    W. Yu, Y . Xu, P. Ghamisi, Universal adversarial defense in remote sensing based on pre-trained denoising diffusion models, International Journal of Applied Earth Observation and Geoinformation 133 (2024) 104131

    2024

  25. [33]

    Antol, A

    S. Antol, A. Agrawal, J. Lu, M. Mitchell, D. Batra, C. L. Zitnick, D. Parikh, Vqa: Visual question answering, in: Proceedings of the IEEE international conference on computer vision, 2015, pp. 2425–2433

    2015

  26. [34]

    Kottur, J

    S. Kottur, J. M. Moura, D. Parikh, D. Batra, M. Rohrbach, Visual coreference resolution in visual dialog using neural module networks, in: Proceedings of the European conference on computer vision (ECCV), 2018, pp. 153– 169

    2018

  27. [35]

    Greff, R

    K. Greff, R. K. Srivastava, J. Koutník, B. R. Steunebrink, J. Schmidhuber, Lstm: A search space odyssey, IEEE transactions on neural networks and learning systems 28 (10) (2016) 2222–2232

    2016

  28. [36]

    K. He, X. Zhang, S. Ren, J. Sun, Deep residual learning for image recognition, in: Proceedings of the IEEE conference on computer vision and pattern recognition, 2016, pp. 770–778

    2016

  29. [37]

    C. Liu, Z. Lin, X. Shen, J. Yang, X. Lu, A. Yuille, Recurrent multimodal interaction for referring image segmentation, in: Proceedings of the IEEE international conference on computer vision, 2017, pp. 1271–1280

    2017

  30. [38]

    R. Hu, M. Rohrbach, T. Darrell, Segmentation from natural language expressions, in: Computer Vision– ECCV 2016: 14th European Conference, Amsterdam, The Netherlands, October 11–14, 2016, Proceedings, Part I 14, Springer, 2016, pp. 108–124

    2016

  31. [39]

    R. Li, K. Li, Y .-C. Kuo, M. Shu, X. Qi, X. Shen, J. Jia, Referring image segmentation via recurrent refinement networks, in: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2018, pp. 5745–5753

    2018

  32. [40]

    Z. Yang, J. Wang, Y . Tang, K. Chen, H. Zhao, P. H. Torr, Lavt: Language-aware vision transformer for referring image segmentation, in: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2022, pp. 18155–18165

    2022

  33. [41]

    Z. Liu, Y . Lin, Y . Cao, H. Hu, Y . Wei, Z. Zhang, S. Lin, B. Guo, Swin transformer: Hierarchical vision transformer using shifted windows, in: Proceedings of the 14 IEEE/CVF international conference on computer vision, 2021, pp. 10012–10022

    2021

  34. [42]

    H. Ding, C. Liu, S. Wang, X. Jiang, Vlt: Vision- language transformer and query generation for referring segmentation, IEEE Transactions on Pattern Analysis and Machine Intelligence 45 (6) (2022) 7900–7916

    2022

  35. [43]

    Z. Wang, Y . Lu, Q. Li, X. Tao, Y . Guo, M. Gong, T. Liu, Cris: Clip-driven referring image segmentation, in: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, 2022, pp. 11686–11695

    2022

  36. [44]

    N. Kim, D. Kim, C. Lan, W. Zeng, S. Kwak, Restr: Convolution-free referring image segmentation using transformers, in: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2022, pp. 18145–18154

    2022

  37. [45]

    J. Tang, G. Zheng, C. Shi, S. Yang, Contrastive grouping with transformer for referring image segmentation, in: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2023, pp. 23570–23580

    2023

  38. [46]

    C. Liu, H. Ding, X. Jiang, Gres: Generalized referring expression segmentation, in: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, 2023, pp. 23592–23601

    2023

  39. [47]

    Z. Yuan, L. Mou, Y . Hua, X. X. Zhu, Rrsis: Referring remote sensing image segmentation, IEEE Transactions on Geoscience and Remote Sensing (2024)

    2024

  40. [48]

    S. Liu, Y . Ma, X. Zhang, H. Wang, J. Ji, X. Sun, R. Ji, Rotated multi-scale interaction network for referring remote sensing image segmentation, in: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2024, pp. 26658–26668

    2024

  41. [49]

    Z. Dong, Y . Sun, Y . Gu, T. Liu, Cross-modal bidirectional interaction model for referring remote sensing image segmentation, arXiv preprint arXiv:2410.08613 (2024)

    2024 arXiv

  42. [50]

    Sohl-Dickstein, E

    J. Sohl-Dickstein, E. Weiss, N. Maheswaranathan, S. Ganguli, Deep unsupervised learning using nonequilibrium thermodynamics, in: International conference on machine learning, PMLR, 2015, pp. 2256–2265

    2015

  43. [51]

    J. Ho, A. Jain, P. Abbeel, Denoising diffusion probabilistic models, Advances in neural information processing systems 33 (2020) 6840–6851

    2020

  44. [52]

    L. Yang, Z. Zhang, Y . Song, S. Hong, R. Xu, Y . Zhao, W. Zhang, B. Cui, M.-H. Yang, Diffusion models: A comprehensive survey of methods and applications, ACM Computing Surveys 56 (4) (2023) 1–39

    2023

  45. [53]

    Zhang, C

    C. Zhang, C. Zhang, M. Zhang, I. S. Kweon, Text-to- image diffusion models in generative ai: A survey, arXiv preprint arXiv:2303.07909 (2023)

    2023 arXiv

  46. [54]

    Ronneberger, P

    O. Ronneberger, P. Fischer, T. Brox, U-net: Convolutional networks for biomedical image segmentation, in: Medical image computing and computer-assisted intervention– MICCAI 2015: 18th international conference, Munich, Germany, October 5-9, 2015, proceedings, part III 18, Sprin...

    2015

  47. [55]

    Esser, R

    P. Esser, R. Rombach, B. Ommer, Taming transformers for high-resolution image synthesis, in: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, 2021, pp. 12873–12883

    2021

  48. [56]

    Radford, J

    A. Radford, J. W. Kim, C. Hallacy, A. Ramesh, G. Goh, S. Agarwal, G. Sastry, A. Askell, P. Mishkin, J. Clark, et al., Learning transferable visual models from natural language supervision, in: International conference on machine learning, PMLR, 2021, pp. 8748–8763

    2021

  49. [57]

    E. Jang, S. Gu, B. Poole, Categorical reparameterization with gumbel-softmax, arXiv preprint arXiv:1611.01144 (2016)

    2016 arXiv

  50. [58]

    C. J. Maddison, A. Mnih, Y . W. Teh, The concrete distribution: A continuous relaxation of discrete random variables, arXiv preprint arXiv:1611.00712 (2016)

    2016 arXiv

  51. [59]

    L. Ye, M. Rochan, Z. Liu, Y . Wang, Cross-modal self- attention network for referring image segmentation, in: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, 2019, pp. 10502–10511

    2019

  52. [60]

    T. Hui, S. Liu, S. Huang, G. Li, S. Yu, F. Zhang, J. Han, Linguistic structure guided context modeling for referring image segmentation, in: Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part X 16, Springer, 2020, pp. 59–75

    2020

  53. [61]

    S. Liu, T. Hui, S. Huang, Y . Wei, B. Li, G. Li, Cross-modal progressive comprehension for referring segmentation, IEEE Transactions on Pattern Analysis and Machine Intelligence 44 (9) (2021) 4761–4775

    2021

  54. [62]

    H. Ding, S. Zhang, Q. Wu, S. Yu, J. Hu, L. Cao, R. Ji, Bilateral knowledge interaction network for referring image segmentation, IEEE Transactions on Multimedia (2023)

    2023

  55. [63]

    Z. Xu, Z. Chen, Y . Zhang, Y . Song, X. Wan, G. Li, Bridging vision and language encoders: Parameter- efficient tuning for referring image segmentation, in: Proceedings of the IEEE/CVF International Conference on Computer Vision, 2023, pp. 17503–17512

    2023

  56. [64]

    Ouyang, H

    S. Ouyang, H. Wang, S. Xie, Z. Niu, R. Tong, Y .-W. Chen, L. Lin, Slvit: Scale-wise language-guided vision transformer for referring image segmentation., in: IJCAI, 2023, pp. 1294–1302

    2023

  57. [65]

    J. Wu, X. Li, X. Li, H. Ding, Y . Tong, D. Tao, Towards robust referring image segmentation, IEEE Transactions on Image Processing (2024). 15

    2024

  58. [66]

    Y . Cho, H. Yu, S.-J. Kang, Cross-aware early fusion with stage-divided vision and language transformer encoders for referring image segmentation, IEEE Transactions on Multimedia (2023)

    2023

  59. [67]

    Y . Hu, Q. Wang, W. Shao, E. Xie, Z. Li, J. Han, P. Luo, Beyond one-to-one: Rethinking the referring image segmentation, in: Proceedings of the IEEE/CVF International Conference on Computer Vision, 2023, pp. 4067–4077

    2023

  60. [68]

    S.-A. Liu, Y . Zhang, Z. Qiu, H. Xie, Y . Zhang, T. Yao, Caris: Context-aware referring image segmentation, in: Proceedings of the 31st ACM International Conference on Multimedia, 2023, pp. 779–788

    2023

  61. [69]

    Lavreniuk, S

    M. Lavreniuk, S. F. Bhat, M. Müller, P. Wonka, Evp: Enhanced visual perception using inverse multi-attentive feature refinement and regularized image-text alignment, arXiv preprint arXiv:2312.08548 (2023)

    2023 arXiv

  62. [70]

    W. Zhao, Y . Rao, Z. Liu, B. Liu, J. Zhou, J. Lu, Unleashing text-to-image diffusion models for visual perception, in: Proceedings of the IEEE/CVF International Conference on Computer Vision, 2023, pp. 5729–5739

    2023

  63. [71]

    L. Yu, Z. Lin, X. Shen, J. Yang, X. Lu, M. Bansal, T. L. Berg, Mattnet: Modular attention network for referring expression comprehension, in: Proceedings of the IEEE conference on computer vision and pattern recognition, 2018, pp. 1307–1315. 16

    2018

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.