Pith. sign in

REVIEW 3 major objections 5 minor 62 references

CoMiX: Cross-Modal Fusion with Deformable Convolutions for HSI-X Semantic Segmentation

T0 review · 3 major / 5 minor · reviewed 2026-08-12 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read CoMiX, a modality-agnostic fusion network, reports the highest overall accuracy among compared methods on three HSI-X semantic segmentation benchmarks.

desk verdict Solid engineering contribution with a legitimate new fusion module, but the comparative claim rests on baseline fairness that the paper does not yet establish. read the letter →

arxiv 2411.09023 v1 pith:FMX3BETG submitted 2024-11-13 cs.CV

Xuming Zhang , Xingfa Gu , Qingjiu Tian , Lorenzo Bruzzone This is my paper · ORCID
classification cs.CV
keywords hyperspectralimagesemanticsegmentationcross-modalfusiondeformableconvolutionattentionmechanismland-coverclassificationmultimodalremotesensingencoder-decoder
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper proposes CoMiX, a deep network for pixel-level semantic segmentation of hyperspectral images (HSIs) combined with any secondary sensor type, such as LiDAR-derived DSM, SAR, or multispectral LiDAR. The authors claim that by using deformable convolutions tailored to each modality and a cross-modality module that recalibrates and exchanges features, CoMiX learns modality-specific, modality-shared, and complementary information more effectively than existing fusion architectures. On three public benchmarks (Houston2013 HSI-DSM, Berlin HSI-SAR, DFC2018 HSI-MS-LiDAR) it reports the highest overall accuracy among all compared methods: 95.75%, 76.81%, and 68.26%, respectively. If correct, this suggests that a single modality-agnostic architecture can replace specialized HSI-LiDAR or HSI-SAR designs while improving accuracy and keeping near-real-time inference.

What carries the argument

The load-bearing mechanism is the CMFeX module working together with the paired 2D/3D deformable-convolution backbones. CMFeX builds spatial and spectral attention maps for each modality, then cross-applies them: HSI spatial values are recalibrated by the X modality's spatial attention map and vice versa, and similarly for spectral attention, so each modality highlights what the other finds informative before residual addition and fusion. The deformable convolutions supply adaptive receptive fields, with 3x3 DCNv4-style windows in early stages and 7x7 windows in later stages, and the HSI branch uses an efficient paired-attention module to keep 3D processing affordable.

What would settle it

A third-party reproduction that tunes every baseline per dataset, including larger patch sizes and longer training for Fusion-FCN, and then compares overall accuracy on the Berlin and DFC2018 test sets; if any baseline matches or exceeds CoMiX's reported OA, or the reported margins shrink to within one standard deviation, the central claim fails.

Watch

Extended reading notes

Core claim

CoMiX is an asymmetric encoder-decoder in which two parallel backbones extract features from HSI and X data with deformable convolutions: 2D DCN blocks for the X modality adapt sampling locations to geometric variation, while 3D DCN blocks for the HSI treat it as a spatial-spectral cube and aggregate spectral and spatial context. At each of four stages a Cross-Modality Feature enhancement and eXchange (CMFeX) module computes spatial and spectral attention maps for both modalities, uses each modality's attention to recalibrate the other's values, and adds the result back as a residual; a feature fusion module then concatenates and mixes the recalibrated features, and the per-stage outputs feed a lightweight all-MLP decoder. The paper's central claim is that this combination extracts, calibrates, and fuses modality-specific and modality-shared information better than prior fusion networks, yielding the best overall accuracy on all three datasets.

Load-bearing premise

The evaluation assumes that the seven comparison methods, run from their public code with the same preprocessing, training and test samples, and a 128x128 patch input, are sufficiently well tuned to be fair baselines; if any is under-tuned for the Berlin HSI-SAR setting or the large-patch DFC2018 setting, the reported accuracy gaps would overstate CoMiX's advantage.

Editorial extensions

If this is right

  • If correct, one modality-agnostic architecture handles HSI fused with DSM, SAR, or MS-LiDAR without per-pair redesign.
  • Combining 2D DCN for X and 3D DCN for HSI preserves high-resolution parsing (only one 2x downsampling), yielding sharper boundaries in the reported qualitative maps.
  • The method reports near-real-time inference (7.72 seconds on Houston2013) despite a larger parameter count, making it potentially viable where patch-based fusion is too slow.
  • Accuracy holds across reduced training-sample percentages on Houston2013, suggesting lower data dependence than the transformer-based baselines.
  • Ablation results attribute the gains to both spatial and spectral calibration in CMFeX, plus the fusion module, so the full pipeline matters rather than any single component alone.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Beyond the paper: the CMFeX cross-rectification idea should transfer to other paired modalities (RGB-depth, MRI-PET, optical-radar) because it assumes only that two feature maps share spatial support, not specific sensor physics.
  • Beyond the paper: because the baselines were run with 128x128 patches, the reported advantage on DFC2018 could partly reflect input-size fairness; larger patches for segmentation baselines might narrow the gap, a point the paper does not test.
  • Beyond the paper: testing CoMiX on an HSI paired with an X modality it has never seen, such as optical RGB or thermal imagery, would directly probe the claimed modality-agnostic property.
  • Beyond the paper: an ablation isolating the 3D DCN branch from the cross-attention module could clarify how much of the gain comes from true 3D spectral-spatial processing versus the attention-based recalibration.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. The paper proposes CoMiX, an asymmetric encoder-decoder architecture for HSI-X semantic segmentation. The encoder uses two interacting backbones: 2D DCN blocks for the X modality and a block labeled "3D DCN" for HSI, together with Cross-Modality Feature enhancement and eXchange (CMFeX) modules and feature fusion modules (FFM), followed by a lightweight ALL-MLP decoder. Experiments on Houston2013 (HSI-DSM), Berlin (HSI-SAR), and DFC2018 (HSI-MS-LiDAR) report overall accuracies of 95.75%, 76.81%, and 68.26%, respectively, claimed as the best among seven compared methods. Ablation studies on Houston2013 attribute successive gains to the 2D DCN block, the 3D DCN block, the CMFeX module, and the FFM.

Significance. If the comparative results hold, CoMiX is a potentially strong general fusion backbone for heterogeneous HSI-X tasks. The manuscript has several concrete strengths: the ablations are internally consistent; the experiments span three different supplementary modalities (DSM, SAR, MS-LiDAR); the CMFeX and FFM modules are clearly specified; and the reported numbers are reproducible from the tables. However, two issues are load-bearing for the central claims. First, the block called "3D DCN" is not a deformable convolution, because Eqs. (8)-(10) define it with EPA attention and 3D convolutions, not learnable offsets. Second, the comparison protocol for patch-based baselines versus the dense segmentation framework is not demonstrated to be fair, and one entry in Table VII is physically implausible. These issues must be resolved before the performance and efficiency claims can be accepted.

major comments (3)
  1. [Section II.B, Eqs. (8)-(10)] The block labeled "3D DCN" does not use deformable convolution. Equation (8) is LN(EPA(X_hsi)) + X_hsi, Eq. (9) is a residual Conv1 after a 3D FFN, and Eq. (10) is a 3D FFN; no learnable sampling offsets or deformable sampling locations are defined anywhere in the HSI branch. This directly contradicts the title, the abstract, and Contribution 2, which state that 3D DCN blocks adaptively aggregate spatial-spectral features for HSIs. The authors should either rename the block (e.g., "EPA-based 3D feature extraction block") and revise all claims that refer to 3D deformable convolutions, or implement and evaluate a genuinely deformable 3D convolution. As written, a central advertised novelty of the paper is not supported by the method description.
  2. [Section III.B.2 and Tables IV-VI] The comparison protocol is not demonstrably fair. FusAtNet, CALC, Fusion HCT, MFT, and Flex-MCFNet are patch-based classification networks that predict the central pixel of each patch, while CoMiX is a dense segmentation network trained and evaluated on 128x128 crops. Stating that all methods use the "same training and test sample sets" does not guarantee the same supervision density, sampling strategy, or inference protocol; the manuscript does not report the patch sizes, numbers of training patches, or per-dataset hyperparameters for any baseline. Several of the baselines were designed for HSI-LiDAR fusion and may require adaptation for Berlin HSI-SAR or for the 20-class DFC2018 split. Without evidence that the baselines were configured competitively, the reported margins of 3.99 to 11.91 percentage points may overstate CoMiX's advantage. The authors should provide complete configuration details for each baseline, tune baselines per dataset where appropriate, or adopt a common dense-label evaluation protocol.
  3. [Table VII and Section IV] Table VII lists the CACL training time as 1,696,081,351.24 seconds, which is physically impossible (over 50 years) and indicates a unit or reporting error. Since Section IV's efficiency analysis and the "accuracy-efficiency trade-off" discussion rely on this table, the entry must be corrected or the experiment must be re-run and reported accurately. In addition, the statement in Section IV that CoMiX achieves faster inference "despite having higher Params and FLOPs compared to others" is not uniformly true: CoMiX (21.87M parameters, 197.02G FLOPs) has fewer parameters and FLOPs than FusAtNet (36.90M parameters, 221.61G FLOPs), and more than Fusion-FCN (0.09M parameters, 6.22G FLOPs). The text should be reconciled with the table, and the Section II.E claim that the decoder gives CoMiX "lower computational cost, fewer parameters, and increased efficiency" should be removed or heavily qualified.
minor comments (5)
  1. [Throughout] The acronym for the coupled adversarial learning method is inconsistently written as CALC and CACL (e.g., Section III.B.1 vs. Tables IV and VII), and "Fusion HCF" appears where "Fusion HCT" is intended (Section III.C.1). Please standardize the names.
  2. [Section II.B] The text says "Conv1 and Conv2 represent 1x1x1 and 3x3x3 convolutions," but Eq. (9) uses Conv1 and Eq. (10) uses Conv3. The notation should be aligned.
  3. [Table VIII] The check-mark layout of Table VIII is difficult to interpret because the rows and columns both use the same configuration labels. A conventional ablation table with one column per configuration (listing which components are enabled) would be clearer.
  4. [Section III.B.2] The experimental section does not report the hardware used for the timing measurements or the data augmentation strategy. These details are needed for the efficiency comparison in Table VII.
  5. [Tables IV-VI] Standard deviations are reported only for aggregate metrics, not for per-class accuracies. Reporting per-class variability would strengthen the claims about class-level improvements.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: CoMiX's architecture is defined from external components and its claims are supported by measured benchmarks and controlled ablations.

full rationale

The paper's derivation chain is not circular. CoMiX is an encoder-decoder whose blocks are explicitly defined in Eqs. (1)-(25) from inputs: the X branch uses DCNv4 [54], the HSI branch uses the EPA attention module [61] with a 3D FFN, and CMFeX/FFM implement cross-attention and residual MLP fusion. None of these equations references the target benchmark accuracies or the final OA values. The central superiority claim is an empirical comparison on three public datasets with stated train/test splits (Tables I-III and IV-VI), so the reported OAs are measurements, not fitted quantities renamed as predictions. The ablation study (Table VIII) varies one component at a time while keeping the data and protocol fixed, providing independent evidence for each module. The few self-citations ([1], [29]) are background related-work citations and do not carry the derivation. The baseline-fairness concern raised by the reader and the anomalous CACL training time in Table VII are experimental-reporting or comparison-protocol risks, not cases where a result reduces to its input by construction. Therefore no circular step is exhibited and the score is 0.

Assumptions & free parameters 3 free parameters · 4 assumptions · 0 invented entities

The central empirical claim rests on standard deep learning assumptions and a few paper-specific design substitutions. There are no fitted theoretical constants; the free parameters are architectural hyperparameters chosen by hand. The most consequential ad hoc assumption is the substitution of EPA attention for the advertised 3D DCN, which changes the method's actual components. No new physical entities are introduced.

free parameters (3)
  • DCN kernel window sizes per stage = 3x3 in stages 1-2, 7x7 in stages 3-4
    Chosen by hand for trade-off between fine detail and context; stated in Section II-A without ablation.
  • Number of encoder stages = 4
    Set by design, matching common backbone choices; no sensitivity analysis.
  • Input patch size and batch size = 128x128, batch 4
    Set due to GPU memory constraints; affects receptive field and comparisons.
assumptions (4)
  • ad hoc to paper EPA attention is a sufficient substitute for 3D deformable convolution for HSI feature extraction.
    Section II-B replaces the promised 3D DCN with the EPA module to avoid parameter explosion, without evidence that it preserves geometric adaptivity.
  • domain assumption Flattening the HSI cube into a sequence (H*W*D x C) and applying shared query/key attention does not destroy spatial-spectral structure.
    Used in Section II-B and II-C for spatial and spectral attention; assumes the linear projections and attention can mix spectral and spatial dimensions appropriately.
  • ad hoc to paper Cross-modality attention exchange (using one modality's attention map to recalibrate the other's values) improves fusion.
    Core of CMFeX, Equations (15)-(18). Supported only by ablations on one dataset, no theoretical justification.
  • domain assumption DCNs preserve convolutional inductive bias and thus reduce dependence on large datasets.
    Invoked in Section IV.B to explain robust performance under reduced training samples; not experimentally isolated from other factors.

how reviews work

0 comments
Cite this review

Pith. "Pith review of CoMiX: Cross-Modal Fusion with Deformable Convolutions for HSI-X Semantic Segmentation." pith.science (2026). https://pith.science/paper/FMX3BETG

@misc{pith2026241109023,
  author       = {Pith},
  title        = {Pith review of: CoMiX: Cross-Modal Fusion with Deformable Convolutions for HSI-X Semantic Segmentation},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/FMX3BETG}},
  note         = {Machine review of arXiv:2411.09023}
}
read the original abstract

Improving hyperspectral image (HSI) semantic segmentation by exploiting complementary information from a supplementary data type (referred to X-modality) is promising but challenging due to differences in imaging sensors, image content, and resolution. Current techniques struggle to enhance modality-specific and modality-shared information, as well as to capture dynamic interaction and fusion between different modalities. In response, this study proposes CoMiX, an asymmetric encoder-decoder architecture with deformable convolutions (DCNs) for HSI-X semantic segmentation. CoMiX is designed to extract, calibrate, and fuse information from HSI and X data. Its pipeline includes an encoder with two parallel and interacting backbones and a lightweight all-multilayer perceptron (ALL-MLP) decoder. The encoder consists of four stages, each incorporating 2D DCN blocks for the X model to accommodate geometric variations and 3D DCN blocks for HSIs to adaptively aggregate spatial-spectral features. Additionally, each stage includes a Cross-Modality Feature enhancement and eXchange (CMFeX) module and a feature fusion module (FFM). CMFeX is designed to exploit spatial-spectral correlations from different modalities to recalibrate and enhance modality-specific and modality-shared features while adaptively exchanging complementary information between them. Outputs from CMFeX are fed into the FFM for fusion and passed to the next stage for further information learning. Finally, the outputs from each FFM are integrated by the ALL-MLP decoder for final prediction. Extensive experiments demonstrate that our CoMiX achieves superior performance and generalizes well to various multimodal recognition tasks. The CoMiX code will be released.

Figures

Figures reproduced from arXiv: 2411.09023 by the authors.

Figure 1
Figure 1. Overview of the proposed CoMiX framework for HSI-X semantic segmentation. [PITH_FULL_IMAGE:figures/full_fig_p003_1.png] view at source ↗
Figure 2
Figure 2. Illustration of the (a) 2D DCN block and (b) 3D DCN block. [PITH_FULL_IMAGE:figures/full_fig_p004_2.png] view at source ↗
Figure 3
Figure 3. Illustration of the proposed CMFeX and FFM modules. [PITH_FULL_IMAGE:figures/full_fig_p005_3.png] view at source ↗
Figures from the paper (7 more)
Figure 5
Figure 5. Figure 5: Berlin dataset: spatial distribution of (a) the training samples [PITH_FULL_IMAGE:figures/full_fig_p007_5.png]
Figure 4
Figure 4. Figure 4: Houston2013 dataset: spatial distribution of (a) the training [PITH_FULL_IMAGE:figures/full_fig_p007_4.png]
Figure 6
Figure 6. Figure 6: DFC2018 dataset: spatial distribution of training (green box) [PITH_FULL_IMAGE:figures/full_fig_p008_6.png]
Figure 7
Figure 7. Figure 7: Classification maps of different methods on the Houston2013 dataset: (a) False Color Image, (b) Ground Truth, (c) SVM-X, (d) [PITH_FULL_IMAGE:figures/full_fig_p010_7.png]
Figure 8
Figure 8. Figure 8: Classification maps of different methods on the Berlin dataset: (a) False Color Image, (b) Ground Truth, (c) SVM-X, (d) SVM-HSI, [PITH_FULL_IMAGE:figures/full_fig_p012_8.png]
Figure 9
Figure 9. Figure 9: Classification maps of different methods on the DFC2018 dataset: (a) False Color Image, (b) Ground Truth, (c) SVM-X, (d) SVM-HSI, [PITH_FULL_IMAGE:figures/full_fig_p013_9.png]
Figure 10
Figure 10. Figure 10: Classification accuracy of different methods versus the [PITH_FULL_IMAGE:figures/full_fig_p014_10.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

62 extracted references · 56 canonical work pages

  1. [1]

    A lightweight transformer network for hyperspectral image classification,

    X. Zhang, Y . Su, L. Gao, L. Bruzzone, X. Gu, and Q. Tian, “A lightweight transformer network for hyperspectral image classification,” IEEE Trans. Geosci. Remote Sens. , vol. 61, pp. 1–17, 2023

    work page 2023

  2. [2]

    LMFFNet: A well-balanced lightweight network for fast and accurate semantic segmentation,

    M. Shi et al., “LMFFNet: A well-balanced lightweight network for fast and accurate semantic segmentation,” IEEE Trans. Neural Netw. Learn. Syst., vol. 34, no. 6, pp. 3205–3219, 2023

    work page 2023

  3. [3]

    Context enhancing representation for semantic segmentation in remote sensing images,

    L. Fang, P. Zhou, X. Liu, P. Ghamisi, and S. Chen, “Context enhancing representation for semantic segmentation in remote sensing images,” IEEE Trans. Neural Netw. Learn. Syst. , vol. 35, no. 3, pp. 4138–4152, 2024

    work page 2024

  4. [4]

    Deep fusion of localized spectral features and multi-scale spatial features for effective classification of hyperspectral images,

    G. Sun et al., “Deep fusion of localized spectral features and multi-scale spatial features for effective classification of hyperspectral images,”Int.J. Appl. Earth Obs. Geoinf. , vol. 91, p. 102157, Sep. 2020

    work page 2020

  5. [5]

    Application of hyperspectral imaging systems and artificial intelligence for quality assessment of fruit, vegetables and mushrooms: A review,

    J. Wieme et al. , “Application of hyperspectral imaging systems and artificial intelligence for quality assessment of fruit, vegetables and mushrooms: A review,” Biosyst. Eng., vol. 222, pp. 156–176, 2022

    work page 2022

  6. [6]

    Ground-based hyperspectral analysis of the urban nightscape,

    Alam ´us et al. , “Ground-based hyperspectral analysis of the urban nightscape,” ISPRS J. Photogramm. Remote Sens. , vol. 124, pp. 16–26, Feb. 2017

    work page 2017

  7. [7]

    SegFormer: Simple and efficient design for semantic segmentation with transformers,

    E. Xie, W. Wang, Z. Yu, A. Anandkumar, J. M. Alvarez, and P. Luo, “SegFormer: Simple and efficient design for semantic segmentation with transformers,” in Adv. Neural Inf. Process. Syst. , Dec. 2021

    work page 2021

  8. [8]

    Multimodal remote sensing benchmark datasets for land cover classification with a shared and specific feature learning model,

    D. Hong, J. Hu, J. Yao, J. Chanussot, and X. X. Zhu, “Multimodal remote sensing benchmark datasets for land cover classification with a shared and specific feature learning model,” ISPRS J. Photogramm. Remote Sens., vol. 178, pp. 68–80, 2021

    work page 2021

Show all 62 references
  1. [9]

    A multi-scale progressive collaborative attention network for remote sensing fusion classification,

    W. Ma et al., “A multi-scale progressive collaborative attention network for remote sensing fusion classification,” IEEE Trans. Neural Netw. Learn. Syst., vol. 34, no. 8, pp. 3897–3911, 2023

    2023

  2. [10]

    Mitigating modality discrepancies for rgb-t semantic segmentation,

    S. Zhao, Y . Liu, Q. Jiao, Q. Zhang, and J. Han, “Mitigating modality discrepancies for rgb-t semantic segmentation,” IEEE Trans. Neural Netw. Learn. Syst. , pp. 1–15, 2023

    2023

  3. [11]

    Pixel-level image fusion: A survey of the state of the art,

    S. Li, X. Kang, L. Fang, J. Hu, and H. Yin, “Pixel-level image fusion: A survey of the state of the art,” Inf. Fusion, vol. 33, pp. 100–112, 2017

    2017

  4. [12]

    Hyperspectral and LiDAR fusion using extinction profiles and total variation component analysis,

    B. Rasti, P. Ghamisi, and R. Gloaguen, “Hyperspectral and LiDAR fusion using extinction profiles and total variation component analysis,” IEEE Trans. Geosci. Remote Sens., vol. 55, no. 7, pp. 3997–4007, 2017

    2017

  5. [13]

    Application of majority voting to pattern recog- nition: An analysis of its behavior and performance,

    L. Lam and S. Suen, “Application of majority voting to pattern recog- nition: An analysis of its behavior and performance,” IEEE Trans Syst Man Cybern A , vol. 27, no. 5, pp. 553–568, 1997

    1997

  6. [14]

    Extinction profiles: A novel approach for the analysis of remote sensing data,

    P. Ghamisi, R. Souza, L. Rittner, J. A. Benediktsson, R. Lotufo, and X. X. Zhu, “Extinction profiles: A novel approach for the analysis of remote sensing data,” in IEEE Int. Geosci. Remote Sens. Symp. (IGARSS), pp. 5122–5125, 2016

    2016

  7. [15]

    Multi-source remotely sensed data fusion for improving land cover classification,

    B. Chen, B. Huang, and B. Xu, “Multi-source remotely sensed data fusion for improving land cover classification,” ISPRS J. Photogramm. Remote Sens., vol. 124, pp. 27–39, 2017

    2017

  8. [16]

    Fusion of hyperspectral and LiDAR data using sparse and low-rank component analysis,

    B. Rasti, P. Ghamisi, J. Plaza, and A. Plaza, “Fusion of hyperspectral and LiDAR data using sparse and low-rank component analysis,” IEEE Trans. Geosci. Remote Sens. , vol. 55, no. 11, pp. 6354–6365, 2017

    2017

  9. [17]

    Manifold learning: what, how, and why,

    M. Meil ˘a and H. Zhang, “Manifold learning: what, how, and why,”Annu. Rev. Stat. Appl., vol. 11, no. V olume 11, 2024, pp. 393–417, 2024

    2024

  10. [18]

    Hyperspec- tral and multispectral image fusion based on a sparse representation,

    Q. Wei, J. Bioucas-Dias, N. Dobigeon, and J.-Y . Tourneret, “Hyperspec- tral and multispectral image fusion based on a sparse representation,” IEEE Trans. Geosci. Remote Sens., vol. 53, no. 7, pp. 3658–3668, 2015

    2015

  11. [19]

    CMX: Cross-modal fusion for RGB-X semantic segmentation with transform- ers,

    J. Zhang, H. Liu, K. Yang, X. Hu, R. Liu, and R. Stiefelhagen, “CMX: Cross-modal fusion for RGB-X semantic segmentation with transform- ers,” IEEE trans. Intell. Transp. Syst., vol. 24, no. 12, pp. 14679–14694, 2023

    2023

  12. [20]

    Hyperspectral and sar image classification via multiscale interactive fusion network,

    J. Wang, W. Li, Y . Gao, M. Zhang, R. Tao, and Q. Du, “Hyperspectral and sar image classification via multiscale interactive fusion network,” IEEE Trans. Neural Netw. Learn. Syst. , vol. 34, no. 12, pp. 10823– 10837, 2023

    2023

  13. [21]

    Robust self-ensembling network for hyperspectral image classification,

    Y . Xu, B. Du, and L. Zhang, “Robust self-ensembling network for hyperspectral image classification,” IEEE Trans. Neural Netw. Learn. Syst., vol. 35, no. 3, pp. 3780–3793, 2024

    2024

  14. [22]

    Deep feature aggregation frame- work driven by graph convolutional network for scene classification in remote sensing,

    K. Xu, H. Huang, P. Deng, and Y . Li, “Deep feature aggregation frame- work driven by graph convolutional network for scene classification in remote sensing,” IEEE Trans. Neural Netw. Learn. Syst., vol. 33, no. 10, pp. 5751–5765, 2022

    2022

  15. [23]

    MGML: Multigranularity multilevel feature ensemble network for remote sensing scene classifica- tion,

    Q. Zhao, S. Lyu, Y . Li, Y . Ma, and L. Chen, “MGML: Multigranularity multilevel feature ensemble network for remote sensing scene classifica- tion,” IEEE Trans. Neural Netw. Learn. Syst. , vol. 34, no. 5, pp. 2308– 2322, 2023

    2023

  16. [24]

    Escaping the big data paradigm with compact trans- formers,

    H. Ali, W. Steven, S. Nikhil, A. Abulikemu, L. Jiachen, and S. Humphrey, “Escaping the big data paradigm with compact trans- formers,” arXiv:2104.05704, 2022

    2022 arXiv

  17. [25]

    Deep fusion of remote sensing data for accurate classification,

    Y . Chen, C. Li, P. Ghamisi, X. Jia, and Y . Gu, “Deep fusion of remote sensing data for accurate classification,” IEEE Geosci. Remote Sens. Lett., vol. 14, no. 8, pp. 1253–1257, 2017

    2017

  18. [26]

    Deep encoder–decoder networks for classification of hyperspectral and LiDAR data,

    D. Hong, L. Gao, R. Hang, B. Zhang, and J. Chanussot, “Deep encoder–decoder networks for classification of hyperspectral and LiDAR data,” IEEE Geosci. Remote Sens. Lett. , vol. 19, pp. 1–5, 2022

    2022

  19. [27]

    Deep residual network- based fusion framework for hyperspectral and LiDAR data,

    C. Ge, Q. Du, W. Sun, K. Wang, J. Li, and Y . Li, “Deep residual network- based fusion framework for hyperspectral and LiDAR data,”IEEE J. Sel. Topics Appl. Earth Observ. Remote Sens., vol. 14, pp. 2458–2472, 2021

    2021

  20. [28]

    More diverse means better: multimodal deep learn- ing meets remote-sensing imagery classification,

    D. Hong et al. , “More diverse means better: multimodal deep learn- ing meets remote-sensing imagery classification,” IEEE Trans. Geosci. Remote Sens., vol. 59, no. 5, pp. 4340–4354, 2021

    2021

  21. [29]

    Objective evaluation-based efficient learning framework for hyperspectral image classification,

    X. Zhang, J. Yan, J. Tian, W. Li, X. Gu, and Q. Tian, “Objective evaluation-based efficient learning framework for hyperspectral image classification,” GISci. Remote Sens. , vol. 60, no. 1, p. 2225273, 2023

    2023

  22. [30]

    Squeeze-and-excitation networks,

    J. Hu, L. Shen, and G. Sun, “Squeeze-and-excitation networks,” in Proc. IEEE CVPR, pp. 7132–7141, 2018

    2018

  23. [31]

    Dual attention network for scene segmentation,

    J. Fu et al., “Dual attention network for scene segmentation,” in Proc. IEEE CVPR, pp. 3141–3149, 2019

    2019

  24. [32]

    Imbalance knowledge-driven multi-modal network for land-cover semantic seg- mentation using aerial images and LiDAR point clouds,

    Y . Wang, Y . Wan, Y . Zhang, B. Zhang, and Z. Gao, “Imbalance knowledge-driven multi-modal network for land-cover semantic seg- mentation using aerial images and LiDAR point clouds,” ISPRS J. Photogramm. Remote Sens. , vol. 202, pp. 385–404, 2023

    2023

  25. [33]

    Multimodal attention- aware convolutional neural networks for classification of hyperspectral and LiDAR data,

    H. Zhang, J. Yao, L. Ni, L. Gao, and M. Huang, “Multimodal attention- aware convolutional neural networks for classification of hyperspectral and LiDAR data,” IEEE J. Sel. Topics Appl. Earth Observ. Remote Sens., vol. 16, pp. 3635–3644, 2023

    2023

  26. [34]

    Coupled adversarial learning for fusion classification of hyperspectral and LiDAR data,

    T. Lu, K. Ding, W. Fu, S. Li, and A. Guo, “Coupled adversarial learning for fusion classification of hyperspectral and LiDAR data,” Inf. Fusion, vol. 93, pp. 118–131, 2023

    2023

  27. [35]

    FusAtNet: Dual attention based spectrospatial multimodal fusion network for hyperspec- tral and LiDAR classification,

    S. Mohla, S. Pande, B. Banerjee, and S. Chaudhuri, “FusAtNet: Dual attention based spectrospatial multimodal fusion network for hyperspec- tral and LiDAR classification,” in Proc. IEEE Conf. Comput. Vis. Pattern Recognit. (CVPR), pp. 416–425, 2020

    2020

  28. [36]

    A triplet semisupervised deep network for fusion classification of hyperspectral and LiDAR data,

    J. Li, Y . Ma, R. Song, B. Xi, D. Hong, and Q. Du, “A triplet semisupervised deep network for fusion classification of hyperspectral and LiDAR data,” IEEE Trans. Geosci. Remote Sens., vol. 60, pp. 1–13, 2022. JOURNAL OF LATEX CLASS FILES 16

    2022

  29. [37]

    Multi-attentive hierarchical dense fusion net for fusion classification of hyperspectral and LiDAR data,

    X. Wang, Y . Feng, R. Song, Z. Mu, and C. Song, “Multi-attentive hierarchical dense fusion net for fusion classification of hyperspectral and LiDAR data,” Inf. Fusion, vol. 82, pp. 1–18, 2022

    2022

  30. [38]

    Multiscale deep learning network with self-calibrated convolution for hyperspectral and LiDAR data collaborative classification,

    Z. Xue, X. Yu, X. Tan, B. Liu, A. Yu, and X. Wei, “Multiscale deep learning network with self-calibrated convolution for hyperspectral and LiDAR data collaborative classification,” IEEE Trans. Geosci. Remote Sens., vol. 60, pp. 1–16, 2022

    2022

  31. [39]

    Cross hyperspectral and lidar attention transformer: An extended self-attention for land use and land cover classification,

    S. K. Roy, A. Sukul, A. Jamali, J. M. Haut, and P. Ghamisi, “Cross hyperspectral and lidar attention transformer: An extended self-attention for land use and land cover classification,” IEEE Trans. Geosci. Remote Sens., vol. 62, pp. 1–15, 2024

    2024

  32. [40]

    An image is worth 16x16 words: Transformers for image recognition at scale,

    D. Alexey et al. , “An image is worth 16x16 words: Transformers for image recognition at scale,” in Proc. Int. Conf. Learn. Represent. , Jan. 2021

    2021

  33. [41]

    Swin Transformer: Hierarchical vision transformer using shifted windows,

    Z. Liu et al., “Swin Transformer: Hierarchical vision transformer using shifted windows,” in Proc. IEEE Int. Conf. Comput. Vis. (ICCV) , pp. 10012–10022, October 2021

    2021

  34. [42]

    Rethinking semantic segmentation from a sequence- to-sequence perspective with transformers,

    S. Zheng et al. , “Rethinking semantic segmentation from a sequence- to-sequence perspective with transformers,” in Proc. IEEE CVPR , pp. 6877–6886, 2021

    2021

  35. [43]

    Multimodal fusion transformer for remote sensing image classification,

    S. K. Roy, A. Deria, D. Hong, B. Rasti, A. Plaza, and J. Chanussot, “Multimodal fusion transformer for remote sensing image classification,” IEEE Trans. Geosci. Remote Sens. , vol. 61, pp. 1–20, 2023

    2023

  36. [44]

    Deep hierarchical vision transformer for hyperspectral and LiDAR data classification,

    Z. Xue, X. Tan, X. Yu, B. Liu, A. Yu, and P. Zhang, “Deep hierarchical vision transformer for hyperspectral and LiDAR data classification,” IEEE Trans. Image Process. , vol. 31, pp. 3095–3110, 2022

    2022

  37. [45]

    Local information interaction transformer for hyperspectral and LiDAR data classification,

    Y . Zhang, Y . Peng, B. Tu, and Y . Liu, “Local information interaction transformer for hyperspectral and LiDAR data classification,” IEEE J. Sel. Topics Appl. Earth Observ. Remote Sens. , vol. 16, pp. 1130–1143, 2023

    2023

  38. [46]

    Global–local transformer network for HSI and LiDAR data joint classification,

    K. Ding, T. Lu, W. Fu, S. Li, and F. Ma, “Global–local transformer network for HSI and LiDAR data joint classification,” IEEE Trans. Geosci. Remote Sens. , vol. 60, pp. 1–13, 2022

    2022

  39. [47]

    CoT: Contourlet transformer for hierarchical semantic segmentation,

    Y . Shao et al. , “CoT: Contourlet transformer for hierarchical semantic segmentation,” IEEE Trans. Neural Netw. Learn. Syst. , pp. 1–15, 2024

    2024

  40. [49]

    Scaling up your kernels to 31×31: Revisiting large kernel design in CNNs,

    X. Ding, X. Zhang, J. Han, and G. Ding, “Scaling up your kernels to 31×31: Revisiting large kernel design in CNNs,” in Proc. IEEE CVPR , pp. 11953–11965, 2022

    2022

  41. [50]

    UniRepLKNet: A universal perception large-kernel con- vnet for audio, video, point cloud, time-series and image recognition,

    X. Ding et al., “UniRepLKNet: A universal perception large-kernel con- vnet for audio, video, point cloud, time-series and image recognition,” arXiv, vol. abs/2311.15599, 2023

    2023 arXiv

  42. [51]

    InternImage: Exploring large-scale vision foundation models with deformable convolutions,

    W. Wang et al. , “InternImage: Exploring large-scale vision foundation models with deformable convolutions,” Proc. IEEE CVPR , pp. 14408– 14419, 2022

    2022

  43. [52]

    Mamba: Linear-time sequence modeling with selective state spaces,

    A. Gu and T. Dao, “Mamba: Linear-time sequence modeling with selective state spaces,” 2023

    2023

  44. [53]

    Deformable ConvNets V2: More deformable, better results,

    X. Zhu, H. Hu, S. Lin, and J. Dai, “Deformable ConvNets V2: More deformable, better results,” in Proc. IEEE CVPR, pp. 9300–9308, 2019

    2019

  45. [54]

    Efficient Deformable ConvNets: Rethinking dynamic and sparse operator for vision applications,

    Y . Xiong et al. , “Efficient Deformable ConvNets: Rethinking dynamic and sparse operator for vision applications,” ArXiv, vol. abs/2401.06197, 2024

    2024 arXiv

  46. [55]

    Advanced multi-sensor optical remote sensing for urban land use and land cover classification: Outcome of the 2018 IEEE GRSS data fusion contest,

    Y . Xu et al. , “Advanced multi-sensor optical remote sensing for urban land use and land cover classification: Outcome of the 2018 IEEE GRSS data fusion contest,” IEEE J. Sel. Topics Appl. Earth Observ. Remote Sens., vol. 12, no. 6, pp. 1709–1724, 2019

    2018

  47. [56]

    Joint classification of hyperspectral and LiDAR data using a hierarchical CNN and trans- former,

    G. Zhao, Q. Ye, L. Sun, Z. Wu, C. Pan, and B. Jeon, “Joint classification of hyperspectral and LiDAR data using a hierarchical CNN and trans- former,” IEEE Trans. Geosci. Remote Sens. , vol. 61, pp. 1–16, 2023

    2023

  48. [57]

    Deformable convolutional networks,

    J. Dai et al., “Deformable convolutional networks,” in Proc. IEEE ICCV, pp. 764–773, 2017

    2017

  49. [58]

    Vmamba: Visual state space model,

    Y . Liu et al., “Vmamba: Visual state space model,” 2024

    2024

  50. [59]

    Layer normalization,

    J. Ba, J. R. Kiros, and G. E. Hinton, “Layer normalization,” ArXiv, vol. abs/1607.06450, Jul. 2016

    2016 arXiv

  51. [60]

    Beyond self-attention: Deformable large kernel attention for medical image segmentation,

    R. Azad et al., “Beyond self-attention: Deformable large kernel attention for medical image segmentation,” 2024 IEEE/CVF Winter Conf. Appl. Comput. Vis. (WACV), pp. 1276–1286, 2023

    2024

  52. [61]

    UNETR++: Delving into efficient and accurate 3D medical image segmentation,

    A. M. Shaker, M. Maaz, H. Rasheed, S. Khan, M.-H. Yang, and F. S. Khan, “UNETR++: Delving into efficient and accurate 3D medical image segmentation,” IEEE Trans. Med. Imaging , pp. 1–1, 2024

    2024

  53. [62]

    Support-vector networks,

    C. Cortes and V . Vapnik, “Support-vector networks,” Mach. Learn. , vol. 20, pp. 273–297, 1995

    1995

  54. [63]

    A multistage information complementary fusion network based on flexible-mixup for hsi-x image classification,

    J. Wang, M. Zhang, W. Li, and R. Tao, “A multistage information complementary fusion network based on flexible-mixup for hsi-x image classification,” IEEE Trans. Neural Netw. Learn. Syst. , pp. 1–13, 2023

    2023

Pith tools

Reviewed August 12, 2026 · model on record in the stance chip above.