Pith. sign in

REVIEW 2 major objections 8 minor 55 references

Cross-Modal Pre-Aligned Method with Global and Local Information for Remote-Sensing Image and Text Retrieval

T0 review · 2 major / 8 minor · reviewed 2026-08-12 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read A pre-aligned global-local transformer with similarity-matrix reweighting reports the best mean Recall on four remote-sensing image-text retrieval benchmarks, including a 4.65% R@1 gain and a 2.28% mean Recall gain on RSITMD.

desk verdict Solid engineering, but the headline RSITMD numbers are in-sample for the SMR tuning. read the letter →

arxiv 2411.14704 v1 pith:GSIH6BJZ submitted 2024-11-22 cs.CV cs.IR

Zengbao Sun , Ming Zhao , Gaorui Liu , André Kaup This is my paper · ORCID
classification cs.CVcs.IR
keywords remotesensingcross-modalretrievalimage-textpre-alignmentglobalandlocalinformationtransformerattentionsimilaritymatrixreweightingtripletlossimagecaptioningdatasets
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Remote sensing cross-modal text-image retrieval lets users find satellite images with natural-language queries, but matching text to small objects and complex spatial relations in overhead scenes remains difficult. This paper sets out to show that a specific pipeline—pre-aligning image and text features before fusing them, extracting visual features with a transformer block that mixes global and local window attention, tightening the triplet loss with an intra-class term, and reweighting the retrieval similarity matrix—improves retrieval over existing methods. The central claim is that CMPAGL with SMR reranking achieves the best mean Recall on all four benchmarks (RSICD, RSITMD, UCM-Captions, Sydney-Captions) and the best score on every metric on RSICD and RSITMD, with gains of up to 4.65% in R@1 and 2.28% in mean Recall on RSITMD. If the result holds, the same recipe of alignment before fusion plus multi-scale attention and matrix reweighting is a practical option for remote sensing information mining.

What carries the argument

The load-bearing machinery is the Gswin transformer block, a dual-branch attention module in which local window self-attention and shifted local window self-attention each run cross-attention against the same global-information window, so multi-scale features are fused without needing a separate detection stage. Around this block, the paper wraps three further mechanisms: a pre-alignment stage (image-text contrastive loss plus an optimized triplet loss) applied before the image and text features enter the multimodal encoder; a BERT text encoder split in half so its second half doubles as the multimodal encoder; and an SMR reranking step that multiplies the original similarity matrix $S_{raw}$ by a weight matrix $W_{map} = w_{i2t} + \gamma_1 w_{t2i} + \gamma_2 w_{md}$, where $w_{i2t}$ and $w_{t2i}$ are forward and reverse ranking probabilities and $w_{md}$ is the extreme-difference-ratio term. The optimized triplet loss includes the intra-class term $\sum_i [1 - S(I_i, T_i)]$, which is what pushes matching pairs' similarity toward 1. The paper's ablations attribute each stage a distinct role: Gswin improves multi-scale and small-target representation, pre-alignment lowers fusion difficulty, the triplet term tightens matched-pair distances, and SMR exploits bidirectional ranking information that the raw cosine similarity matrix ignores.

What would settle it

Fix $\gamma_1$ and $\gamma_2$ using only a held-out validation split of RSITMD (or using RSICD), then evaluate on the RSITMD test split; if the mean Recall advantage over IEFT or KAMCL shrinks below 2.28% or reverses, the tuned SMR is not responsible for the claimed state-of-the-art result.

Watch

Extended reading notes

Core claim

On its own terms, the paper's discovery is that retrieval accuracy in remote sensing cross-modal text-image retrieval is improved by explicitly pre-aligning visual and textual features before they enter a multimodal fusion transformer, rather than letting the fusion layers learn alignment from scratch. The paper demonstrates this with a model whose image encoder stacks Gswin transformer blocks: each block computes local window self-attention and shifted-window self-attention in two separate branches, and both branches cross-attend to a shared global window produced by a convolutional global-window generation module, so fine local detail and scene-level context are mixed without an object-detection stage. Pre-alignment is driven by image-text contrastive loss plus an optimized triplet loss whose extra intra-class term pushes the similarity of matched pairs toward 1; after fusion, masked language modeling and image-text matching losses refine the shared representation. At retrieval time, the similarity matrix reweighting (SMR) algorithm rescales the raw similarity matrix using forward and reverse ranking probabilities together with an extreme-difference-ratio term, and the paper reports that this final step alone adds further gains on top of the trained model.

Load-bearing premise

The load-bearing premise is that the SMR reranking weights $\gamma_1=0.9$ and $\gamma_2=1.9$, chosen by grid search on RSITMD to maximize mean Recall, still work out-of-sample on RSITMD; if those weights are overfit to that dataset's queries, the headline 4.65% R@1 and 2.28% mean Recall gains are not a fair test.

Editorial extensions

If this is right

  • On RSICD and RSITMD, CMPAGL with SMR reports the highest R@1, R@5, and R@10 in both text retrieval and image retrieval, and the ablations attribute each gain to a specific component.
  • Pre-alignment is shown to outperform a standard modal-interaction architecture with the same total number of transformer layers (Tables V and VI), so the paper claims the benefit comes from aligning before fusing, not from added capacity.
  • The Gswin encoder outperforms ViT, Swin, and GCViT encoders under the same comparison protocol (Table VII), supporting the claim that global-local window cross-attention is responsible for the visual gains.
  • SMR adds a further 1.53% mean Recall on RSICD and 2.28% on RSITMD over the identical model without reranking, and does so without any additional training.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Because SMR operates on the final similarity matrix, it could be applied to any dual-encoder remote sensing retrieval model without retraining, so the reported reweighting gains are likely portable across architectures.
  • The paper's grid-search protocol selects $\gamma_1$ and $\gamma_2$ on RSITMD before reporting RSITMD results; a stricter procedure would fix these coefficients on a validation split or a different dataset, and the stated RSITMD improvements should be read with that caveat.
  • The pre-alignment recipe of contrastive loss plus an intra-class triplet term before a shared multimodal encoder may transfer to other fine-grained retrieval domains, such as vehicle, building, or scene-attribute retrieval, where the captions describe object relationships rather than simple labels.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

2 major / 8 minor

Summary. The paper proposes CMPAGL, a cross-modal image-text retrieval method for remote sensing data, comprising a Gswin transformer visual encoder that fuses global and local window attention, a BERT-based text encoder, a pre-alignment stage using ITC and an optimized triplet loss, a multi-modal encoder trained with ITM and MLM, and an SMR reranking algorithm that reweights the retrieval similarity matrix. The method is evaluated on RSICD, RSITMD, UCM-Captions, and Sydney-Captions, with reported state-of-the-art results, e.g., a 4.65% absolute R@1 improvement over IEFT on RSITMD and a 2.28% mR gain over the non-reranked variant. Ablations show monotone performance gains as each component is added.

Significance. If the results are trustworthy, the paper makes a useful contribution: the Gswin block is a plausible extension of Swin/GCViT for remote sensing images, the pre-alignment idea is well motivated, and the optimized triplet loss is a sensible modification. The paper provides three-seed averaging and releases code. However, the evaluation protocol for the SMR reranking hyperparameters is flawed on RSITMD: the weights γ1 and γ2 are selected by grid search on the RSITMD dataset itself, and the same weights are then used to report the RSITMD results, making the headline gains in-sample estimates. This compromises the central state-of-the-art claim on that dataset, although the variant without SMR already outperforms all baselines in mR on RSITMD.

major comments (2)
  1. [Section IV-E, Table II] The grid search for the SMR weights γ1 and γ2 is performed on the RSITMD dataset, and the same tuned values (γ1=0.9, γ2=1.9) are used to produce the RSITMD results reported in Table II and cited in the Abstract (4.65% R@1 improvement over IEFT; 2.28% mR gain). No held-out validation split or cross-validation procedure is described anywhere in the paper. As written, the grid search appears to evaluate on the same data used for reporting, so the RSITMD numbers for CMPAGL with SMR are not out-of-sample estimates. This is a test-set overfitting issue and invalidates the size of the SMR gain and the state-of-the-art claim on RSITMD. The authors should either (a) clearly state that the grid search was performed on a training/validation split and not on the test set, or (b) redo the protocol by tuning on a validation split and reporting test results only once. The same concern applies to any other hyperparameters (e.g., α, K) not listed in the implementation details.
  2. [Section III-E, Eq. (14)] The definition of the extreme difference ratio component w_md is inconsistent with its explanation. Equation (14) writes w_md = s(t_j, i_k)/Maxrow_p1(Sraw) + s(t_j, i_k)/Maxcol_p2(Sraw), but the text then describes the quantity as reflecting the discrepancy between s(i_q, t_j) and the row/column maxima, and uses the query image i_q. The roles of i_q, t_j, and i_k are not clearly defined, and the relationship between the numerator and the stated goal is unclear. Please rewrite Eq. (14) with precise indices and a verbal description that matches the formula exactly; otherwise, the SMR algorithm is not reproducible.
minor comments (8)
  1. [Section IV-B] The paper states that the alpha parameter of the optimized triplet loss will be discussed in Section D, but no such discussion appears; please provide the value or a selection procedure for alpha.
  2. [Section IV-B] The value of K (number of top candidates) in the SMR reranking is never specified; please state it in the implementation details.
  3. [Section IV-E] The grid search range description is unclear: 'the value range to 0.5, 2.0 with an interval of 0.1' should read 'a value range from 0.5 to 2.0 with an interval of 0.1'.
  4. [Section III-C] The citation 'Inspired by VILT [26]' refers to reference [26] (Swin Transformer); the intended reference appears to be ViLT [48].
  5. [Section IV-F] There is a duplicated phrase in the last paragraph: 'results demonstrates the effectiveness of the effectiveness of'.
  6. [Section IV-E] The 'MR' reranking algorithm is used in comparisons but is never defined or cited; define it or give a reference.
  7. [Section IV-C.4] The text says CMPAGL outperforms on 'the R@1, R@5 metrics for the image-to-text retrieval task, and the mR metric,' but Table II shows that on Sydney text retrieval (image-to-text), CMPAGL does not outperform IEFT on R@5 (49.34 vs 56.25). If the intended direction is text-to-image (image retrieval), please correct the wording.
  8. [Abstract] The statement 'up to 4.65% improvement in R@1 and 2.28% in mean Recall (mR) over state-of-the-art methods' is ambiguous: the 4.65% is relative to IEFT, while the 2.28% mR is relative to CMPAGL w/o SMR. Please specify the exact comparison baselines for each number.

Circularity Check

1 steps flagged · score 6.0 of 10

RSITMD headline gains for SMR depend on γ1=0.9 and γ2=1.9 tuned by grid search on the same RSITMD benchmark that is then reported; the 2.28% mR SMR gain is in-sample for those weights.

  1. fitted input called prediction [Section IV-E (Reranking Parameter Analysis); results reported in Table II and Table IV on RSITMD; Abstract]
    "For γ1 and γ2, we set the value range to 0.5, 2.0 with an interval of 0.1, and traversed all possible parameter combinations through grid search. ... we selected the average recall rates mR of R@1, R@5, and R@10 for text retrieval and image retrieval to measure performance. ... The SMR method achieves the best performance when γ1 = 0.9 and γ2 = 1.9."

    The grid search is run on the RSITMD dataset and optimized for the exact mR metric that is later reported as the SMR improvement on RSITMD. No validation split is described. Table IV gives RSITMD mR 37.18 without SMR and 39.46 with SMR, a +2.28 gain that the Abstract cites; Table II gives the RSITMD image R@1 of 15.67 vs IEFT's 11.02, the cited +4.65 gain. Selecting γ1 and γ2 to maximize RSITMD mR and then reporting the RSITMD SMR gain as validation makes that gain a fitted quantity rather than an out-of-sample prediction: the reranking weights are not independently predicting an unseen benchmark but are optimized against the very metric and dataset being reported.

full rationale

The paper's main derivation chain is architectural: Gswin combines local window self-attention with global-local cross-attention, pre-alignment is adopted from ALBEF with an external citation, and the triplet loss is extended with an intra-class distance term. These components are evaluated through ablations and comparisons that are not equivalent to their inputs by construction, and the self-citations present (e.g., to the authors' earlier SAR work) are not load-bearing for the central retrieval claim. The one significant circular step is the SMR reranking evaluation on RSITMD. Section IV-E performs a grid search on the RSITMD dataset to maximize mR, selects γ1=0.9 and γ2=1.9, and then the same dataset's SMR results are reported as the headline improvements (2.28% mR and 4.65% R@1). Because no held-out validation split is described, the reported SMR gain is a fitted result rather than a prediction. However, CMPAGL without SMR already outperforms previous methods on RSITMD, and the SMR weights are not tuned on RSICD, UCM-Captions, or Sydney-Captions, so the circularity affects the magnitude and interpretation of the SMR-specific gain on RSITMD rather than the entire model's novelty or all benchmark claims. Overall score 6: one fitted-input-called-prediction step creates partial circularity in the central RSITMD SOTA claim, while the architecture itself remains independently supported.

Assumptions & free parameters 5 free parameters · 3 assumptions · 0 invented entities

The method rests on standard pretrained encoders, an assumed transfer from general-domain pretraining to remote sensing, and experimental assumptions about comparability of baselines and generalization of grid-searched reranking weights. No new physical or conceptual entities are introduced.

free parameters (5)
  • gamma1 (forward/reverse ranking weight in SMR) = 0.9
    Eq. (15), Section IV-E: selected by grid search on the RSITMD dataset to maximize mR; used to report RSITMD results in Table II.
  • gamma2 (extreme difference ratio weight in SMR) = 1.9
    Eq. (15), Section IV-E: selected by grid search on the RSITMD dataset to maximize mR; used to report RSITMD results in Table II.
  • K (number of top candidates in SMR reranking) = not reported
    Eqs. (10)-(12) define top-K reranking; the experimental section never states the K used, so the reranking procedure is underspecified.
  • alpha (margin in optimized triplet loss) = not reported
    Eq. (19) includes a boundary parameter alpha; Section IV-B says it will be discussed later, but the provided text does not state its value.
  • tau (ITC temperature) = not reported
    Eq. (18) uses a temperature parameter tau; no value is given in the implementation details.
assumptions (3)
  • domain assumption Pretrained general-domain models (BERT and ImageNet-style pretraining for the visual encoder) transfer useful representations to remote sensing imagery and text.
    Section III-C selects a 12-layer BERT and splits it into text and multimodal encoders; Section III-B initializes the image encoder from standard vision transformer weights. The entire method assumes this transfer is beneficial, which is empirically plausible but not proven within the paper.
  • ad hoc to paper The grid-searched SMR weights (gamma1, gamma2) chosen on RSITMD remain valid for UCM, Sydney, and RSICD.
    Section IV-E fixes gamma1=0.9 and gamma2=1.9 using RSITMD and applies them to all datasets in Table II without a validation protocol for the other datasets.
  • domain assumption The reported baseline numbers in Table II are directly comparable with CMPAGL's results despite different training protocols, backbones, and hyperparameters.
    Section IV-C compares against numbers from prior papers; the paper does not re-run baselines under shared conditions.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Cross-Modal Pre-Aligned Method with Global and Local Information for Remote-Sensing Image and Text Retrieval." pith.science (2026). https://pith.science/paper/GSIH6BJZ

@misc{pith2026241114704,
  author       = {Pith},
  title        = {Pith review of: Cross-Modal Pre-Aligned Method with Global and Local Information for Remote-Sensing Image and Text Retrieval},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/GSIH6BJZ}},
  note         = {Machine review of arXiv:2411.14704}
}
read the original abstract

Remote sensing cross-modal text-image retrieval (RSCTIR) has gained attention for its utility in information mining. However, challenges remain in effectively integrating global and local information due to variations in remote sensing imagery and ensuring proper feature pre-alignment before modal fusion, which affects retrieval accuracy and efficiency. To address these issues, we propose CMPAGL, a cross-modal pre-aligned method leveraging global and local information. Our Gswin transformer block combines local window self-attention and global-local window cross-attention to capture multi-scale features. A pre-alignment mechanism simplifies modal fusion training, improving retrieval performance. Additionally, we introduce a similarity matrix reweighting (SMR) algorithm for reranking, and enhance the triplet loss function with an intra-class distance term to optimize feature learning. Experiments on four datasets, including RSICD and RSITMD, validate CMPAGL's effectiveness, achieving up to 4.65% improvement in R@1 and 2.28% in mean Recall (mR) over state-of-the-art methods.

Figures

Figures reproduced from arXiv: 2411.14704 by the authors.

Figure 1
Figure 1. Overview of the CMPAGL architecture. The proposed framework comprises three principal components: a Gswin-based image encoder, a text encoder, N [PITH_FULL_IMAGE:figures/full_fig_p004_1.png] view at source ↗
Figure 2
Figure 2. (a) GWG block. (b) Schematic diagram of the interaction between [PITH_FULL_IMAGE:figures/full_fig_p005_2.png] view at source ↗
Figure 3
Figure 3. Gswin transformer block. This method fully combines global and [PITH_FULL_IMAGE:figures/full_fig_p006_3.png] view at source ↗
Figures from the paper (5 more)
Figure 4
Figure 4. Figure 4: Schematic diagram of the re-ranking algorithm. In this schematic, [PITH_FULL_IMAGE:figures/full_fig_p007_4.png]
Figure 5
Figure 5. Figure 5: Optimized triplet loss positive and negative sample optimization [PITH_FULL_IMAGE:figures/full_fig_p009_5.png]
Figure 6
Figure 6. Figure 6: Parameter analysis of reranking algorithms: comparative study and [PITH_FULL_IMAGE:figures/full_fig_p014_6.png]
Figure 7
Figure 7. Figure 7: Visualization of image encoder output in cross-modal retrieval network. (a)Visualization under baseball field scene. (b)Visualization under airport [PITH_FULL_IMAGE:figures/full_fig_p015_7.png]
Figure 8
Figure 8. Figure 8: Comparison of query text and retrieved images by different methods. [PITH_FULL_IMAGE:figures/full_fig_p016_8.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

55 extracted references · 41 canonical work pages

  1. [1]

    Remote sensing big data computing: Challenges and opportunities,

    Y . Ma, H. Wu, L. Wang, B. Huang, R. Ranjan, A. Zomaya, and W. Jie, “Remote sensing big data computing: Challenges and opportunities,” Future Generation Computer Systems , vol. 51, pp. 47–60, 2015

    work page 2015

  2. [2]

    Big data for remote sensing: Challenges and opportunities,

    M. Chi, A. Plaza, J. A. Benediktsson, Z. Sun, J. Shen, and Y . Zhu, “Big data for remote sensing: Challenges and opportunities,” Proceedings of the IEEE , vol. 104, no. 11, pp. 2207–2219, 2016

    work page 2016

  3. [3]

    Understanding urban landuse from the above and ground perspectives: A deep learning, multi- modal solution,

    S. Srivastava, J. E. Vargas-Munoz, and D. Tuia, “Understanding urban landuse from the above and ground perspectives: A deep learning, multi- modal solution,” Remote Sensing of Environment, vol. 228, pp. 129–143, 2019

    work page 2019

  4. [4]

    Hyperspectral data analysis for arid vegetation species: Smart & sustainable growth,

    S. Borana, S. Yadav, and S. Parihar, “Hyperspectral data analysis for arid vegetation species: Smart & sustainable growth,” in 2019 International Conference on Computing, Communication, and Intelligent Systems (ICCCIS), pp. 495–500, IEEE, 2019

    work page 2019

  5. [5]

    Google earth engine cloud computing platform for remote sensing big data applications: A comprehensive review,

    M. Amani, A. Ghorbanian, S. A. Ahmadi, M. Kakooei, A. Moghimi, S. M. Mirmazloumi, S. H. A. Moghaddam, S. Mahdavi, M. Ghahreman- loo, S. Parsian, et al. , “Google earth engine cloud computing platform for remote sensing big data applications: A comprehensive review,”IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, vol. 13, ...

    work page 2020

  6. [6]

    Nwpu- captions dataset and mlca-net for remote sensing image captioning,

    Q. Cheng, H. Huang, Y . Xu, Y . Zhou, H. Li, and Z. Wang, “Nwpu- captions dataset and mlca-net for remote sensing image captioning,” IEEE Transactions on Geoscience and Remote Sensing , vol. 60, pp. 1– 19, 2022

    2022

  7. [7]

    Textrs: Deep bidirectional triplet network for matching text to remote sensing images,

    T. Abdullah, Y . Bazi, M. M. Al Rahhal, M. L. Mekhalfi, L. Rangarajan, and M. Zuair, “Textrs: Deep bidirectional triplet network for matching text to remote sensing images,” Remote Sensing , vol. 12, no. 3, p. 405, 2020

    work page 2020

  8. [8]

    A deep semantic alignment network for the cross-modal image-text retrieval in remote sensing,

    Q. Cheng, Y . Zhou, P. Fu, Y . Xu, and L. Zhang, “A deep semantic alignment network for the cross-modal image-text retrieval in remote sensing,” IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing , vol. 14, pp. 4284–4297, 2021

    2021

Show all 55 references
  1. [9]

    Fusion-based correlation learning model for cross-modal remote sensing image retrieval,

    Y . Lv, W. Xiong, X. Zhang, and Y . Cui, “Fusion-based correlation learning model for cross-modal remote sensing image retrieval,” IEEE Geoscience and Remote Sensing Letters , vol. 19, pp. 1–5, 2021

    2021

  2. [10]

    Cross spectral image reconstruction using a deep guided neural network,

    F. Sippel, J. Seiler, and A. Kaup, “Cross spectral image reconstruction using a deep guided neural network,” in 2023 IEEE International Conference on Image Processing (ICIP) , pp. 226–230, IEEE, 2023

    2023

  3. [11]

    Image super-resolution using t-tetromino pixels,

    S. Grosche, A. Regensky, J. Seiler, and A. Kaup, “Image super-resolution using t-tetromino pixels,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pp. 9989–9998, 2023

    2023

  4. [12]

    Exploring a fine-grained multiscale method for cross-modal remote sensing image retrieval,

    Z. Yuan, W. Zhang, K. Fu, X. Li, C. Deng, H. Wang, and X. Sun, “Exploring a fine-grained multiscale method for cross-modal remote sensing image retrieval,” IEEE Transactions on Geoscience and Remote Sensing, vol. 60, pp. 1–19, 2022

    2022

  5. [13]

    Remote sensing cross-modal text-image retrieval based on global and local information,

    Z. Yuan, W. Zhang, C. Tian, X. Rong, Z. Zhang, H. Wang, K. Fu, and X. Sun, “Remote sensing cross-modal text-image retrieval based on global and local information,” IEEE Transactions on Geoscience and Remote Sensing , vol. 60, pp. 1–16, 2022

    2022

  6. [14]

    A lightweight multi-scale crossmodal text-image retrieval method in remote sensing,

    Z. Yuan, W. Zhang, X. Rong, X. Li, J. Chen, H. Wang, K. Fu, and X. Sun, “A lightweight multi-scale crossmodal text-image retrieval method in remote sensing,” IEEE Transactions on Geoscience and Remote Sensing , vol. 60, pp. 1–19, 2021

    2021

  7. [15]

    Exploring uni-modal feature learning on entities and relations for remote sensing cross-modal text-image re- trieval,

    S. Zhang, Y . Li, and S. Mei, “Exploring uni-modal feature learning on entities and relations for remote sensing cross-modal text-image re- trieval,” IEEE Transactions on Geoscience and Remote Sensing , vol. 61, pp. 1–17, 2023. JOURNAL OF LATEX CLASS FILES, VOL. 14, NO. 8, AUG...

    2023

  8. [16]

    Hypersphere-based remote sensing cross-modal text-image retrieval via curriculum learning,

    W. Zhang, J. Li, S. Li, J. Chen, W. Zhang, X. Gao, and X. Sun, “Hypersphere-based remote sensing cross-modal text-image retrieval via curriculum learning,” IEEE Transactions on Geoscience and Remote Sensing, 2023

    2023

  9. [17]

    An image is worth 16x16 words: Trans- formers for image recognition at scale,

    A. Dosovitskiy, L. Beyer, A. Kolesnikov, D. Weissenborn, X. Zhai, T. Unterthiner, M. Dehghani, M. Minderer, G. Heigold, S. Gelly, J. Uszkoreit, and N. Houlsby, “An image is worth 16x16 words: Trans- formers for image recognition at scale,” ArXiv, vol. abs/2010.11929, 2020

    2010 arXiv

  10. [18]

    Long short-term memory recurrent neural network architectures for large scale acoustic modeling,

    H. Sak, A. W. Senior, and F. Beaufays, “Long short-term memory recurrent neural network architectures for large scale acoustic modeling,” in Interspeech, 2014

    2014

  11. [19]

    Empirical evalua- tion of gated recurrent neural networks on sequence modeling,

    J. Chung, C ¸ aglar G¨ulc ¸ehre, K. Cho, and Y . Bengio, “Empirical evalua- tion of gated recurrent neural networks on sequence modeling,” ArXiv, vol. abs/1412.3555, 2014

    2014 arXiv

  12. [20]

    Attention is all you need,

    A. Vaswani, N. Shazeer, N. Parmar, J. Uszkoreit, L. Jones, A. N. Gomez, Ł. Kaiser, and I. Polosukhin, “Attention is all you need,” Advances in Neural Information Processing Systems , vol. 30, 2017

    2017

  13. [21]

    Multiscale salient alignment learning for remote sensing image-text retrieval,

    Y . Chen, J. Huang, X. Li, S. Xiong, and X. Lu, “Multiscale salient alignment learning for remote sensing image-text retrieval,” IEEE Trans- actions on Geoscience and Remote Sensing , 2023

    2023

  14. [22]

    Interacting- enhancing feature transformer for cross-modal remote sensing image and text retrieval,

    X. Tang, Y . Wang, J. Ma, X. Zhang, F. Liu, and L. Jiao, “Interacting- enhancing feature transformer for cross-modal remote sensing image and text retrieval,” IEEE Transactions on Geoscience and Remote Sensing , 2023

    2023

  15. [23]

    Align before fuse: Vision and language representation learning with momentum distillation,

    J. Li, R. Selvaraju, A. Gotmare, S. Joty, C. Xiong, and S. C. H. Hoi, “Align before fuse: Vision and language representation learning with momentum distillation,” in Advances in Neural Information Processing Systems (M. Ranzato, A. Beygelzimer, Y . Dauphin, P. Liang, and J. W....

    2021

  16. [24]

    Deep saliency smoothing hashing for drone image retrieval,

    Y . Chen, J. Huang, L. Mou, P. Jin, S. Xiong, and X. X. Zhu, “Deep saliency smoothing hashing for drone image retrieval,” IEEE Transac- tions on Geoscience and Remote Sensing , vol. 61, pp. 1–13, 2023

    2023

  17. [25]

    Multitask learning for sar ship detection with gaussian-mask joint segmentation,

    M. Zhao, X. Zhang, and A. Kaup, “Multitask learning for sar ship detection with gaussian-mask joint segmentation,” IEEE Transactions on Geoscience and Remote Sensing , 2023

    2023

  18. [26]

    Swin transformer: Hierarchical vision transformer using shifted windows,

    Z. Liu, Y . Lin, Y . Cao, H. Hu, Y . Wei, Z. Zhang, S. Lin, and B. Guo, “Swin transformer: Hierarchical vision transformer using shifted windows,” in Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 10012–10022, 2021

    2021

  19. [27]

    Global context vision transformers,

    A. Hatamizadeh, H. Yin, G. Heinrich, J. Kautz, and P. Molchanov, “Global context vision transformers,” in International Conference on Machine Learning , pp. 12633–12646, PMLR, 2023

    2023

  20. [28]

    Matching images and text with multi-modal tensor fusion and re- ranking,

    T. Wang, X. Xu, Y . Yang, A. Hanjalic, H. T. Shen, and J. Song, “Matching images and text with multi-modal tensor fusion and re- ranking,” in Proceedings of the 27th ACM International Conference on Multimedia, pp. 12–20, 2019

    2019

  21. [29]

    Vse++: Improving visual-semantic embeddings with hard negatives,

    F. Faghri, D. J. Fleet, J. R. Kiros, and S. Fidler, “Vse++: Improving visual-semantic embeddings with hard negatives,” in British Machine Vision Conference, 2017

    2017

  22. [30]

    Exploring models and data for remote sensing image caption generation,

    X. Lu, B. Wang, X. Zheng, and X. Li, “Exploring models and data for remote sensing image caption generation,” IEEE Transactions on Geoscience and Remote Sensing , vol. 56, no. 4, pp. 2183–2195, 2017

    2017

  23. [31]

    Deep semantic understanding of high resolution remote sensing image,

    B. Qu, X. Li, D. Tao, and X. Lu, “Deep semantic understanding of high resolution remote sensing image,” in 2016 International Conference on Computer , Information and Telecommunication Systems (Cits) , pp. 1–5, IEEE, 2016

    2016

  24. [32]

    End-to-end convolutional semantic embeddings,

    Q. You, Z. Zhang, and J. Luo, “End-to-end convolutional semantic embeddings,” in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition , pp. 5735–5744, 2018

    2018

  25. [33]

    Cross-modal semantic correlation learning by bi-cnn network,

    C. Wang, L. Li, C. Yan, Z. Wang, Y . Sun, and J. Zhang, “Cross-modal semantic correlation learning by bi-cnn network,” IET Image Processing, vol. 15, no. 14, pp. 3674–3684, 2021

    2021

  26. [34]

    Dual-path convolutional image-text embeddings with instance loss,

    Z. Zheng, L. Zheng, M. Garrett, Y . Yang, M. Xu, and Y .-D. Shen, “Dual-path convolutional image-text embeddings with instance loss,” ACM Transactions on Multimedia Computing, Communications, and Applications (TOMM) , vol. 16, no. 2, pp. 1–23, 2020

    2020

  27. [35]

    Deep supervised cross-modal retrieval,

    L. Zhen, P. Hu, X. Wang, and D. Peng, “Deep supervised cross-modal retrieval,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pp. 10394–10403, 2019

    2019

  28. [36]

    Learning semantic concepts and order for image and sentence matching,

    Y . Huang, Q. Wu, C. Song, and L. Wang, “Learning semantic concepts and order for image and sentence matching,” in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition , pp. 6163– 6171, 2018

    2018

  29. [37]

    Stacked cross attention for image-text matching,

    K.-H. Lee, X. Chen, G. Hua, H. Hu, and X. He, “Stacked cross attention for image-text matching,” in Proceedings of the European Conference on Computer Vision (ECCV) , pp. 201–216, 2018

    2018

  30. [38]

    Cross- modal attention with semantic consistence for image–text matching,

    X. Xu, T. Wang, Y . Yang, L. Zuo, F. Shen, and H. T. Shen, “Cross- modal attention with semantic consistence for image–text matching,” IEEE Transactions on Neural Networks and Learning Systems , vol. 31, no. 12, pp. 5412–5425, 2020

    2020

  31. [39]

    Visual semantic reasoning for image-text matching,

    K. Li, Y . Zhang, K. Li, Y . Li, and Y . Fu, “Visual semantic reasoning for image-text matching,” in Proceedings of the IEEE/CVF International Conference on Computer Vision , pp. 4654–4662, 2019

    2019

  32. [40]

    Image-text embedding learning via visual and textual semantic reasoning,

    K. Li, Y . Zhang, K. Li, Y . Li, and Y . Fu, “Image-text embedding learning via visual and textual semantic reasoning,”IEEE Transactions on Pattern Analysis and Machine Intelligence , vol. 45, no. 1, pp. 641–656, 2022

    2022

  33. [41]

    Vilbert: Pretraining task-agnostic visiolinguistic representations for vision-and-language tasks,

    J. Lu, D. Batra, D. Parikh, and S. Lee, “Vilbert: Pretraining task-agnostic visiolinguistic representations for vision-and-language tasks,” Advances in Neural Information Processing Systems , vol. 32, 2019

    2019

  34. [42]

    Lxmert: Learning cross-modality encoder representations from transformers,

    H. H. Tan and M. Bansal, “Lxmert: Learning cross-modality encoder representations from transformers,” in Conference on Empirical Methods in Natural Language Processing , 2019

    2019

  35. [43]

    Fashionbert: Text and image matching with adaptive loss for cross- modal retrieval,

    D. Gao, L. Jin, B. Chen, M. Qiu, P. Li, Y . Wei, Y . Hu, and H. Wang, “Fashionbert: Text and image matching with adaptive loss for cross- modal retrieval,” in Proceedings of the 43rd International ACM SIGIR Conference on Research and Development in Information Retrieval , pp. ...

    2020

  36. [44]

    Learning the best pooling strategy for visual semantic embedding,

    J. Chen, H. Hu, H. Wu, Y . Jiang, and C. Wang, “Learning the best pooling strategy for visual semantic embedding,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pp. 15789–15798, 2021

    2021

  37. [45]

    Pixel-bert: Aligning image pixels with text by deep multi-modal transformers,

    Z. Huang, Z. Zeng, B. Liu, D. Fu, and J. Fu, “Pixel-bert: Aligning image pixels with text by deep multi-modal transformers,” ArXiv, vol. abs/2004.00849, 2020

    2004 arXiv

  38. [46]

    Learning transferable visual models from natural language supervision,

    A. Radford, J. W. Kim, C. Hallacy, A. Ramesh, G. Goh, S. Agarwal, G. Sastry, A. Askell, P. Mishkin, J. Clark, et al., “Learning transferable visual models from natural language supervision,” in International Conference on Machine Learning , pp. 8748–8763, PMLR, 2021

    2021

  39. [47]

    Vista: Vision and scene text aggregation for cross-modal retrieval,

    M. Cheng, Y . Sun, L. Wang, X. Zhu, K. Yao, J. Chen, G. Song, J. Han, J. Liu, E. Ding, et al. , “Vista: Vision and scene text aggregation for cross-modal retrieval,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pp. 5184–5193, 2022

    2022

  40. [48]

    Vilt: Vision-and-language transformer without convolution or region supervision,

    W. Kim, B. Son, and I. Kim, “Vilt: Vision-and-language transformer without convolution or region supervision,” in International Conference on Machine Learning , pp. 5583–5594, PMLR, 2021

    2021

  41. [49]

    Vlmo: Unified vision-language pre-training with mixture-of-modality-experts,

    H. Bao, W. Wang, L. Dong, Q. Liu, O. K. Mohammed, K. Aggarwal, S. Som, S. Piao, and F. Wei, “Vlmo: Unified vision-language pre-training with mixture-of-modality-experts,” Advances in Neural Information Pro- cessing Systems , vol. 35, pp. 32897–32912, 2022

    2022

  42. [50]

    Knowledge-aided momentum contrastive learning for remote-sensing image text retrieval,

    Z. Ji, C. Meng, Y . Zhang, Y . Pang, and X. Li, “Knowledge-aided momentum contrastive learning for remote-sensing image text retrieval,” IEEE Transactions on Geoscience and Remote Sensing , vol. 61, pp. 1– 13, 2023

    2023

  43. [51]

    Multi- scale interactive transformer for remote sensing cross-modal image-text retrieval,

    Y . Wang, J. Ma, M. Li, X. Tang, X. Han, and L. Jiao, “Multi- scale interactive transformer for remote sensing cross-modal image-text retrieval,” in IGARSS 2022-2022 IEEE International Geoscience and Remote Sensing Symposium , pp. 839–842, IEEE, 2022

    2022

  44. [52]

    Parameter-efficient transfer learning for remote sensing image-text retrieval,

    Y . Yuan, Y . Zhan, and Z. Xiong, “Parameter-efficient transfer learning for remote sensing image-text retrieval,” IEEE Transactions on Geoscience and Remote Sensing , 2023

    2023

  45. [53]

    Integrating multisubspace joint learning with multilevel guidance for cross-modal retrieval of remote sensing images,

    Y . Chen, J. Huang, S. Xiong, and X. Lu, “Integrating multisubspace joint learning with multilevel guidance for cross-modal retrieval of remote sensing images,” IEEE Transactions on Geoscience and Remote Sensing , vol. 62, pp. 1–17, 2024

    2024

  46. [54]

    Bert: Pre-training of deep bidirectional transformers for language understanding,

    J. Devlin, M.-W. Chang, K. Lee, and K. Toutanova, “Bert: Pre-training of deep bidirectional transformers for language understanding,” in North American Chapter of the Association for Computational Linguistics , 2019

    2019

  47. [55]

    Momentum contrast for unsupervised visual representation learning,

    K. He, H. Fan, Y . Wu, S. Xie, and R. Girshick, “Momentum contrast for unsupervised visual representation learning,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pp. 9729–9738, 2020

    2020

Pith tools

Reviewed August 12, 2026 · model on record in the stance chip above.