Pith. sign in

REVIEW 67 references

EGFormer: Towards Efficient and Generalizable Multimodal Semantic Segmentation

T0 review · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read EGFormer dynamically scores and drops the least useful sensor modality at each processing stage, cutting parameters by up to 91 percent and GFLOPs by half while keeping segmentation accuracy competitive.

arxiv 2505.14014 v1 pith:H3JDVWWX submitted 2025-05-20 cs.CV

Zelin Zhang , Tao Zhang , KediLI , Xu Zheng This is my paper · ORCID
classification cs.CV
keywords egformersegmentationmodalitiesmultimodalperformancesemanticwhileachieves
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

EGFormer is a neural network for semantic segmentation, the task of labeling every pixel of an image with a category such as road, building, or sky. It is designed to take several sensor inputs at once, for example an RGB camera, a depth sensor, an event camera, and LiDAR. The key idea is to decide, at each processing stage, which sensor is contributing the least useful information and to stop processing it fully, while still letting some of its features influence the remaining sensors through learned attention weights. A second component scores each sensor with a small network that pools its feature map to a number, and the sensor with the lowest average score is the one that gets dropped.

The authors report that this scheme cuts the number of parameters to about one tenth to one fifth of a strong baseline, CMNeXt, and roughly halves the computational cost in GFLOPs, while keeping segmentation accuracy within a small margin or slightly above it. They also test a synthetic-to-real setting: a model trained on the simulator dataset DELIVER is used to label real MUSES images. Their method generates pseudo-labels by taking several modality subsets, letting a teacher network predict on each subset, and keeping only pixels where at least two subsets agree. Those labels then train a student network.

The main concerns are about the evidence. The paper does not release code, reports no error bars, and the comparison tables contain several apparent inconsistencies, including different modality pairs being compared on MCubeS and a baseline parameter count that changes between text and table. The domain adaptation comparison does not include any baseline that was given the same adaptation procedure.

Extended reading notes

Core claim

The paper claims EGFormer "flexibly integrates an arbitrary number of modalities while significantly reducing model parameters and inference time without sacrificing performance," reaching "up to 88% reduction in parameters and 50% fewer GFLOPs" and "state-of-the-art transfer performance" under unsupervised domain adaptation. If correct, the framework delivers near-CMNeXt accuracy with roughly 80 to 90 percent fewer parameters and about half the compute, plus robust segmentation when sensors fail and improved synthetic-to-real transfer through pseudo-label voting.

Load-bearing premise

The robustness and train-once/test-everywhere claims rest on the unstated premise that when a modality is missing, EGFormer and CMNeXt receive the same placeholder representation in the same format. The paper never specifies how absent modalities are encoded (zero tensors, masked features, or repeated input), and the extreme score gaps in Tables 4 and 5 (CMNeXt Depth-only 0.49 mIoU vs. EGFormer 43.89) suggest the baseline may have been fed an input distribution it was not trained on. If the encodings differ, the claimed robustness advantage is an artifact of evaluation protocol, not of the ASM/MDM modules. Location: Section 4.2, Tables 4 and 5.

Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Assumptions & free parameters 4 free parameters · 5 assumptions · 0 invented entities

The central efficiency-accuracy trade-off rests on learned module weights and hand-chosen hyperparameters rather than on a derivation; the main assumptions are about evaluation protocol fairness and the reliability of pseudo-labels.

free parameters (4)
  • Consensus threshold T in pseudo-label voting = 2
    Set by hand in Eq. (7); no sensitivity analysis or justification beyond 'e.g., T=2'.
  • Fusion coefficient 0.5 in MDM update = 0.5
    Fixed constant in Eq. (6); no ablation for this weight.
  • Number of modalities dropped per stage = 1
    Ablation in Figure 5(a) shows one drop per stage is best; this is a design choice fitted to validation performance.
  • Equal weights for L_adapt and L_KL = 1 and 1
    Eq. (10) sums the two losses without balancing coefficients; no ablation reported.
assumptions (5)
  • ad hoc to paper The argmin in Eq. (5) is treated as differentiable, but no surrogate (e.g., Gumbel-softmax) is described.
    Section 3.2 calls the selection 'differentiable and efficient' but no straight-through or soft relaxation is specified; if gradients cannot flow through the discrete choice, training is not as described.
  • domain assumption Absent modalities are encoded in a form all compared methods handle equivalently.
    Section 4.2's train-once/test-everywhere protocol does not state how missing modalities are represented; Table 4/5 gaps imply the baseline may be disadvantaged by input formatting.
  • domain assumption Pseudo-label majority voting with T=2 yields reliable targets for UDA.
    Eq. (7) assumes at least two modality-subset predictions agreeing is a trustworthy label; no validation of this threshold is given.
  • domain assumption DELIVER-to-MUSES 11-class mapping is valid for domain-gap comparison.
    Tables 6 and 9 compare source and target per-class mIoU, which requires a correct label mapping; the paper does not describe it.
  • domain assumption All methods use the same SegFormer-B0 backbone and ImageNet-1K initialization.
    Stated in implementation details; this is an external benchmark assumption needed for fair comparison.

how reviews work

0 comments
Cite this review

Pith. "Pith review of EGFormer: Towards Efficient and Generalizable Multimodal Semantic Segmentation." pith.science (2026). https://pith.science/paper/H3JDVWWX

@misc{pith2026250514014,
  author       = {Pith},
  title        = {Pith review of: EGFormer: Towards Efficient and Generalizable Multimodal Semantic Segmentation},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/H3JDVWWX}},
  note         = {Machine review of arXiv:2505.14014}
}
read the original abstract

Recent efforts have explored multimodal semantic segmentation using various backbone architectures. However, while most methods aim to improve accuracy, their computational efficiency remains underexplored. To address this, we propose EGFormer, an efficient multimodal semantic segmentation framework that flexibly integrates an arbitrary number of modalities while significantly reducing model parameters and inference time without sacrificing performance. Our framework introduces two novel modules. First, the Any-modal Scoring Module (ASM) assigns importance scores to each modality independently, enabling dynamic ranking based on their feature maps. Second, the Modal Dropping Module (MDM) filters out less informative modalities at each stage, selectively preserving and aggregating only the most valuable features. This design allows the model to leverage useful information from all available modalities while discarding redundancy, thus ensuring high segmentation quality. In addition to efficiency, we evaluate EGFormer on a synthetic-to-real transfer task to demonstrate its generalizability. Extensive experiments show that EGFormer achieves competitive performance with up to 88 percent reduction in parameters and 50 percent fewer GFLOPs. Under unsupervised domain adaptation settings, it further achieves state-of-the-art transfer performance compared to existing methods.

Figures

Figures reproduced from arXiv: 2505.14014 by the authors.

Figure 1
Figure 1. (a) Multi-modal data are processed by EGFormer, where robust modality features are [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 2
Figure 2. Overall framework of our proposed EGFormer. [PITH_FULL_IMAGE:figures/full_fig_p004_2.png] view at source ↗
Figure 3
Figure 3. Illustration of MDM module. Building upon the importance-aware ranking obtained from ASM, we further introduce the Modality Dropping Mod￾ule (MDM) to enhance model robustness against modality perturbations, as shown in [PITH_FULL_IMAGE:figures/full_fig_p005_3.png] view at source ↗
Figures from the paper (6 more)
Figure 4
Figure 4. Figure 4: Visualization of networks trained with four modalities under missing modality scenarios [PITH_FULL_IMAGE:figures/full_fig_p008_4.png]
Figure 5
Figure 5. Figure 5: Trade-off Between mIoU and GFLOPs in MDM Ablation on the MCubeS Dataset. [PITH_FULL_IMAGE:figures/full_fig_p009_5.png]
Figure 6
Figure 6. Figure 6: Supplementary Visualization of our EGFormer trained with four modalities under missing [PITH_FULL_IMAGE:figures/full_fig_p017_6.png]
Figure 7
Figure 7. Figure 7: Supplementary Visualization of EGFormer trained with four modalities under missing [PITH_FULL_IMAGE:figures/full_fig_p018_7.png]
Figure 8
Figure 8. Figure 8: Supplementary visualization of EGFormer trained with all modalities on DELIVER. [PITH_FULL_IMAGE:figures/full_fig_p019_8.png]
Figure 9
Figure 9. Figure 9: Supplementary visualization of EGFormer trained with all modalities on MCubeS. [PITH_FULL_IMAGE:figures/full_fig_p020_9.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

67 extracted references · 57 canonical work pages

  1. [1]

    Ev-segnet: Semantic segmentation for event-based cameras,

    I. Alonso and A. C. Murillo, “Ev-segnet: Semantic segmentation for event-based cameras,” inProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW), pp. 0–0, IEEE, 2019

    work page 2019

  2. [2]

    Shapeconv: Shape-aware convolutional layer for indoor rgb-d semantic segmentation,

    J. Cao, H. Leng, D. Lischinski,et al., “Shapeconv: Shape-aware convolutional layer for indoor rgb-d semantic segmentation,” inProceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), (Montreal, Canada), pp. 7088–7097, IEEE, 2021

    work page 2021

  3. [3]

    Bi-directional cross-modality feature propagation with separation-and-aggregation gate for rgb-d semantic segmentation,

    X. Chen, K. Lin, J. Wang,et al., “Bi-directional cross-modality feature propagation with separation-and-aggregation gate for rgb-d semantic segmentation,” inProceedings of the Euro- pean Conference on Computer Vision (ECCV), (Cham), pp. 561–577, Springer, 2020

    work page 2020

  4. [4]

    Event-based semantic segmentation with posterior attention,

    Z. Jia, K. You, W. He,et al., “Event-based semantic segmentation with posterior attention,” IEEE Transactions on Image Processing, vol. 32, pp. 1829–1842, 2023

    work page 2023

  5. [5]

    Rtfnet: Rgb-thermal fusion network for semantic segmentation of urban scenes,

    Y . Sun, W. Zuo, and M. Liu, “Rtfnet: Rgb-thermal fusion network for semantic segmentation of urban scenes,”IEEE Robotics and Automation Letters, vol. 4, no. 3, pp. 2576–2583, 2019

    2019

  6. [6]

    Delivering arbitrary-modal semantic segmentation,

    J. Zhang, R. Liu, H. Shi,et al., “Delivering arbitrary-modal semantic segmentation,” inPro- ceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), (New Orleans, LA), pp. 1136–1147, IEEE, 2023

    work page 2023

  7. [7]

    Omnibind: Teach to build unequal-scale modality interaction for omni-bind of all,

    Y . Lyu, X. Zheng, D. Kim, and L. Wang, “Omnibind: Teach to build unequal-scale modality interaction for omni-bind of all,”arXiv preprint arXiv:2405.16108, 2024

    arXiv 2024

  8. [8]

    Fusenet: Incorporating depth into semantic segmentation via fusion-based cnn architecture,

    C. Hazirbas, L. Ma, C. Domokos, and D. Cremers, “Fusenet: Incorporating depth into semantic segmentation via fusion-based cnn architecture,” inProceedings of the Asian Conference on Computer Vision (ACCV), (Cham), pp. 213–228, Springer, 2016

    work page 2016

Show all 67 references
  1. [9]

    Rgb and lidar fusion based 3d semantic seg- mentation for autonomous driving,

    K. El Madawi, H. Rashed, A. El Sallab,et al., “Rgb and lidar fusion based 3d semantic seg- mentation for autonomous driving,” inProceedings of the 2019 IEEE Intelligent Transportation Systems Conference (ITSC), pp. 7–12, IEEE, 2019

    2019

  2. [10]

    Mseg3d: Multi-modal 3d semantic segmentation for autonomous driving,

    J. Li, H. Dai, H. Han,et al., “Mseg3d: Multi-modal 3d semantic segmentation for autonomous driving,” inProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recog- nition (CVPR), pp. 21694–21704, IEEE, 2023

    2023

  3. [11]

    Unibind: Llm-augmented unified and balanced representation space to bind them all,

    Y . Lyu, X. Zheng, J. Zhou, and L. Wang, “Unibind: Llm-augmented unified and balanced representation space to bind them all,” inProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 26752–26762, 2024

    2024

  4. [12]

    Sam-event-adapter: Adapting segment anything model for event-rgb semantic segmentation,

    B. Yao, Y . Deng, Y . Liu,et al., “Sam-event-adapter: Adapting segment anything model for event-rgb semantic segmentation,” inProceedings of the 2024 IEEE International Conference on Robotics and Automation (ICRA), pp. 9093–9100, IEEE, 2024

    2024

  5. [13]

    Eseg: Event-based segmentation boosted by explicit edge- semantic guidance,

    Y . Zhao, G. Lyu, K. Li,et al., “Eseg: Event-based segmentation boosted by explicit edge- semantic guidance,” inProceedings of the AAAI Conference on Artificial Intelligence, vol. 39, pp. 10510–10518, 2025

    2025

  6. [14]

    Eventdance: Unsupervised source-free cross-modal adaptation for event-based object recognition,

    X. Zheng and L. Wang, “Eventdance: Unsupervised source-free cross-modal adaptation for event-based object recognition,” inProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 17448–17458, 2024

    2024

  7. [15]

    Exact: Language-guided conceptual reasoning and uncertainty estimation for event-based action recognition and more,

    J. Zhou, X. Zheng, Y . Lyu, and L. Wang, “Exact: Language-guided conceptual reasoning and uncertainty estimation for event-based action recognition and more,” inProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 18633–18643, 2024

    2024

  8. [16]

    Cafuser: Condition-aware multimodal fusion for robust semantic perception of driving scenes,

    T. Brödermann, C. Sakaridis, Y . Fu,et al., “Cafuser: Condition-aware multimodal fusion for robust semantic perception of driving scenes,”IEEE Robotics and Automation Letters, 2025

    2025

  9. [17]

    Stitchfusion: Weaving any visual modalities to enhance multimodal semantic segmentation,

    B. Li, D. Zhang, Z. Zhao,et al., “Stitchfusion: Weaving any visual modalities to enhance multimodal semantic segmentation,”arXiv preprint arXiv:2408.01343, 2024. 10

    2024 arXiv

  10. [18]

    Centering the value of every modality: Towards efficient and resilient modality-agnostic semantic segmentation,

    X. Zheng, Y . Lyu, J. Zhou,et al., “Centering the value of every modality: Towards efficient and resilient modality-agnostic semantic segmentation,” inProceedings of the European Conference on Computer Vision (ECCV), (Cham), pp. 192–212, Springer Nature Switzerland, 2024

    2024

  11. [19]

    Multimodal token fusion for vision transformers,

    Y . Wang, X. Chen, L. Cao,et al., “Multimodal token fusion for vision transformers,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 12186–12195, IEEE, 2022

    2022

  12. [20]

    360sfuda++: Towards source-free uda for panoramic segmentation by learning reliable category prototypes,

    X. Zheng, P. Y . Zhou, A. V . Vasilakos, and L. Wang, “360sfuda++: Towards source-free uda for panoramic segmentation by learning reliable category prototypes,”IEEE Transactions on Pattern Analysis and Machine Intelligence, 2024

    2024

  13. [21]

    Semantics distortion and style matter: Towards source-free uda for panoramic segmentation,

    X. Zheng, P. Zhou, A. V . Vasilakos, and L. Wang, “Semantics distortion and style matter: Towards source-free uda for panoramic segmentation,” inProceedings of the IEEE/CVF Confer- ence on Computer Vision and Pattern Recognition, pp. 27885–27895, 2024

    2024

  14. [22]

    Both style and distortion matter: Dual- path unsupervised domain adaptation for panoramic semantic segmentation,

    X. Zheng, J. Zhu, Y . Liu, Z. Cao, C. Fu, and L. Wang, “Both style and distortion matter: Dual- path unsupervised domain adaptation for panoramic semantic segmentation,” inProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 1285–1295, 2023

    2023

  15. [23]

    Look at the neighbor: Distortion-aware unsupervised domain adaptation for panoramic semantic segmentation,

    X. Zheng, T. Pan, Y . Luo, and L. Wang, “Look at the neighbor: Distortion-aware unsupervised domain adaptation for panoramic semantic segmentation,” inProceedings of the IEEE/CVF ICCV, pp. 18687–18698, 2023

    2023

  16. [24]

    Multimodal material segmentation,

    Y . Liang, R. Wakaki, S. Nobuhara,et al., “Multimodal material segmentation,” inProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 19800– 19808, IEEE, 2022

    2022

  17. [25]

    Multi-shot temporal event localization: A benchmark,

    X. Liu, Y . Hu, S. Bai,et al., “Multi-shot temporal event localization: A benchmark,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 12596–12606, IEEE, 2021

    2021

  18. [26]

    Beyond rgb: Very high resolution urban remote sensing with multimodal deep networks,

    N. Audebert, B. Le Saux, and S. Lefèvre, “Beyond rgb: Very high resolution urban remote sensing with multimodal deep networks,”ISPRS Journal of Photogrammetry and Remote Sensing, vol. 140, pp. 20–32, 2018

    2018

  19. [27]

    U-net ensemble for enhanced semantic segmentation in remote sensing imagery,

    I. Dimitrovski, V . Spasev, S. Loshkovska,et al., “U-net ensemble for enhanced semantic segmentation in remote sensing imagery,”Remote Sensing, vol. 16, no. 12, p. 2077, 2024

    2024

  20. [28]

    Metasegnet: Metadata-collaborative vision-language representation learning for semantic segmentation of remote sensing images,

    L. Wang, S. Dong, Y . Chen,et al., “Metasegnet: Metadata-collaborative vision-language representation learning for semantic segmentation of remote sensing images,”IEEE Transactions on Geoscience and Remote Sensing, 2024

    2024

  21. [29]

    Transformer-cnn cohort: Semi-supervised semantic segmentation by the best of both students,

    X. Zheng, Y . Luo, C. Fu, K. Liu, and L. Wang, “Transformer-cnn cohort: Semi-supervised semantic segmentation by the best of both students,” in2024 IEEE International Conference on Robotics and Automation (ICRA), pp. 11147–11154, IEEE, 2024

    2024

  22. [30]

    Distilling efficient vision transformers from cnns for semantic segmentation,

    X. Zheng, Y . Luo, P. Zhou, and L. Wang, “Distilling efficient vision transformers from cnns for semantic segmentation,”Pattern Recognit., vol. 158, p. 111029, 2025

    2025

  23. [31]

    Frozen is better than learning: A new design of prototype-based classifier for semantic segmentation,

    J. Chen, D. Deguchi, C. Zhang, X. Zheng, and H. Murase, “Frozen is better than learning: A new design of prototype-based classifier for semantic segmentation,”Pattern Recognit., vol. 152, p. 110431, 2024

    2024

  24. [32]

    Covered: Collaborative robot environment dataset for 3d semantic segmentation,

    C. Munasinghe, F. M. Amin, D. Scaramuzza,et al., “Covered: Collaborative robot environment dataset for 3d semantic segmentation,” inProceedings of the 2022 IEEE 27th International Conference on Emerging Technologies and Factory Automation (ETFA), pp. 1–4, IEEE, 2022

    2022

  25. [33]

    Fully convolutional networks for semantic segmentation,

    J. Long, E. Shelhamer, and T. Darrell, “Fully convolutional networks for semantic segmentation,” inProceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 3431–3440, IEEE, 2015. 11

    2015

  26. [34]

    Deeplab: Semantic image segmentation with deep convolutional nets, atrous convolution, and fully connected crfs,

    L.-C. Chen, G. Papandreou, I. Kokkinos, K. Murphy, and A. L. Yuille, “Deeplab: Semantic image segmentation with deep convolutional nets, atrous convolution, and fully connected crfs,” IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 40, no. 4, pp. 834–848, 2018

    2018

  27. [35]

    Encoder-decoder with atrous separable convolution for semantic image segmentation,

    L.-C. Chen, Y . Zhu, G. Papandreou, F. Schroff, and H. Adam, “Encoder-decoder with atrous separable convolution for semantic image segmentation,” inProceedings of the European Conference on Computer Vision (ECCV), pp. 801–818, Springer, 2018

    2018

  28. [36]

    Strip pooling: Rethinking spatial pooling for scene pars- ing,

    Q. Hou, L. Zhang, M.-M. Cheng,et al., “Strip pooling: Rethinking spatial pooling for scene pars- ing,” inProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 4003–4012, IEEE, 2020

    2020

  29. [37]

    Pyramid scene parsing network,

    H. Zhao, J. Shi, X. Qi, X. Wang, and J. Jia, “Pyramid scene parsing network,” inProceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 2881–2890, IEEE, 2017

    2017

  30. [38]

    Cars can’t fly up in the sky: Improving urban-scene segmen- tation via height-driven attention networks,

    S. Choi, J. T. Kim, and J. Choo, “Cars can’t fly up in the sky: Improving urban-scene segmen- tation via height-driven attention networks,” inProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 9373–9382, IEEE, 2020

    2020

  31. [39]

    Dual attention network for scene segmentation,

    J. Fu, J. Liu, H. Tian, Y . Li, Y . Bao, Z. Fang, and H. Lu, “Dual attention network for scene segmentation,” inProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 3146–3154, IEEE, 2019

    2019

  32. [40]

    Ccnet: Criss-cross attention for semantic segmentation,

    Z. Huang, X. Wang, L. Huang, C. Huang, Y . Wei, and W. Liu, “Ccnet: Criss-cross attention for semantic segmentation,” inProceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), pp. 603–612, IEEE, 2019

    2019

  33. [41]

    Ocnet: Object context for semantic segmentation,

    Y . Yuan, L. Huang, J. Guo, C. Zhang, X. Chen, and J. Wang, “Ocnet: Object context for semantic segmentation,”International Journal of Computer Vision (IJCV), vol. 129, no. 5, pp. 1106–1121, 2021

    2021

  34. [42]

    Inverseform: A loss function for structured boundary-aware segmentation,

    S. Borse, Y . Wang, Y . Zhang, and F. Porikli, “Inverseform: A loss function for structured boundary-aware segmentation,” inProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 5901–5911, IEEE, 2021

    2021

  35. [43]

    Boundary-aware feature propagation for scene segmentation,

    H. Ding, X. Jiang, A. Q. Liu, N. M. Thalmann, and G. Wang, “Boundary-aware feature propagation for scene segmentation,” inProceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), pp. 6819–6829, IEEE, 2019

    2019

  36. [44]

    Improving semantic segmentation via decoupled body and edge supervision,

    X. Li, X. Li, L. Zhang, G. Cheng, J. Shi, Z. Lin, S. Tan, and Y . Tong, “Improving semantic segmentation via decoupled body and edge supervision,” inComputer Vision – ECCV 2020: 16th European Conference, Proceedings, Part XVII, (Glasgow, UK), pp. 435–452, Springer, 2020

    2020

  37. [45]

    Vision transformers for dense prediction without convolutions,

    R. Ranftl, A. Bochkovskiy, and V . Koltun, “Vision transformers for dense prediction without convolutions,”arXiv preprint arXiv:2106.06195, 2021

    2021 arXiv

  38. [46]

    Multi- scale high-resolution vision transformer for semantic segmentation,

    J. Gu, H. Kwon, D. Wang, W. Ye, M. Li, Y .-H. Chen, L. Lai, V . Chandra, and D. Z. Pan, “Multi- scale high-resolution vision transformer for semantic segmentation,” inProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 16256–16266, IEEE, 2022

    2022

  39. [47]

    Segnext: Rethinking convolutional attention design for semantic segmentation,

    M.-H. Guo, C.-Z. Lu, Q. Hou, Z. Liu, M.-M. Cheng, and S.-M. Hu, “Segnext: Rethinking convolutional attention design for semantic segmentation,” inAdvances in Neural Information Processing Systems (NeurIPS), 2022

    2022

  40. [48]

    Swin transformer v2: Scaling up capacity and resolution,

    Z. Liu, H. Hu, Y . Lin, Z. Yao, Z. Xie, Y . Wei, J. Ning, Y . Cao, Z. Zhang, L. Dong,et al., “Swin transformer v2: Scaling up capacity and resolution,” inProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 12009–12019, IEEE, 2022. 12

    2022

  41. [49]

    Swin transformer: Hierarchical vision transformer using shifted windows,

    Z. Liu, Y . Lin, Y . Cao, H. Hu, Y . Wei, Z. Zhang, S. Lin, and B. Guo, “Swin transformer: Hierarchical vision transformer using shifted windows,” inProceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), pp. 10012–10022, IEEE, 2021

    2021

  42. [50]

    Segmenter: Transformer for semantic segmen- tation,

    R. Strudel, R. Garcia, I. Laptev, and C. Schmid, “Segmenter: Transformer for semantic segmen- tation,” inProceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), pp. 7262–7272, IEEE, 2021

    2021

  43. [51]

    Segformer: Simple and efficient design for semantic segmentation with transformers,

    E. Xie, W. Wang, Z. Yu, A. Anandkumar, J. M. Alvarez, and P. Luo, “Segformer: Simple and efficient design for semantic segmentation with transformers,”Advances in Neural Information Processing Systems (NeurIPS), vol. 34, pp. 12077–12090, 2021

    2021

  44. [52]

    Segvit: Semantic segmenta- tion with plain vision transformers,

    B. Zhang, Z. Tian, Q. Tang, X. Chu, X. Wei, C. Shen, and Y . Liu, “Segvit: Semantic segmenta- tion with plain vision transformers,” inAdvances in Neural Information Processing Systems (NeurIPS), 2022

    2022

  45. [53]

    Rethinking semantic segmentation from a sequence-to-sequence perspective with transformers,

    S. Zheng, J. Lu, H. Zhao, X. Zhu, Z. Luo, Y . Wang, Y . Fu, J. Feng, T. Xiang, P. H. S. Torr, and L. Zhang, “Rethinking semantic segmentation from a sequence-to-sequence perspective with transformers,” inProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Rec...

    2021

  46. [54]

    X- align: Cross-modal cross-view alignment for bird’s-eye-view segmentation,

    S. Borse, M. Klingner, V . R. Kumar, H. Cai, A. Almuzairee, S. Yogamani, and F. Porikli, “X- align: Cross-modal cross-view alignment for bird’s-eye-view segmentation,” inProceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision (WACV), pp. 3287–3297, IEEE, 2023

    2023

  47. [55]

    Modality-induced transfer-fusion network for rgb-d and rgb-t salient object detection,

    G. Chen, F. Shao, X. Chai, H. Chen, Q. Jiang, X. Meng, and Y .-S. Ho, “Modality-induced transfer-fusion network for rgb-d and rgb-t salient object detection,”IEEE Transactions on Circuits and Systems for Video Technology, vol. 33, no. 4, pp. 1787–1801, 2022

    2022

  48. [56]

    Bridging search region interaction with template for rgb-t tracking,

    T. Hui, Z. Xun, F. Peng, J. Huang, X. Wei, X. Wei, J. Dai, J. Han, and S. Liu, “Bridging search region interaction with template for rgb-t tracking,” inProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 13630–13639, IEEE, 2023

    2023

  49. [57]

    Cross-collaborative fusion-encoder network for robust rgb-thermal salient object detection,

    G. Liao, W. Gao, G. Li, J. Wang, and S. Kwong, “Cross-collaborative fusion-encoder network for robust rgb-thermal salient object detection,”IEEE Transactions on Circuits and Systems for Video Technology, vol. 32, no. 11, pp. 7646–7661, 2022

    2022

  50. [58]

    Glass segmentation using intensity and spectral polarization cues,

    H. Mei, B. Dong, W. Dong, J. Yang, S.-H. Baek, F. Heide, P. Peers, X. Wei, and X. Yang, “Glass segmentation using intensity and spectral polarization cues,” inProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 12622–12631, IEEE, 2022

    2022

  51. [59]

    Caver: Cross-modal view-mixed transformer for bi- modal salient object detection,

    Y . Pang, X. Zhao, L. Zhang, and H. Lu, “Caver: Cross-modal view-mixed transformer for bi- modal salient object detection,”IEEE Transactions on Image Processing, vol. 32, pp. 892–904, 2023

    2023

  52. [60]

    Benchmarking multi-modal semantic segmentation under sensor failures: Missing and noisy modality robustness,

    C. Liao, K. Lei, X. Zheng, J. Moon, Z. Wang, Y . Wang, D. P. Paudel, L. Van Gool, and X. Hu, “Benchmarking multi-modal semantic segmentation under sensor failures: Missing and noisy modality robustness,”arXiv preprint arXiv:2503.18445, 2025

    2025 arXiv

  53. [61]

    Reducing unimodal bias in multi-modal semantic segmentation with multi-scale functional entropy regularization,

    X. Zheng, Y . Lyu, L. Jiang, D. P. Paudel, L. Van Gool, and X. Hu, “Reducing unimodal bias in multi-modal semantic segmentation with multi-scale functional entropy regularization,”arXiv preprint arXiv:2505.06635, 2025

    2025 arXiv

  54. [62]

    Memorysam: Memorize modalities and semantics with segment anything model 2 for multi-modal semantic segmentation,

    C. Liao, X. Zheng, Y . Lyu, H. Xue, Y . Cao, J. Wang, K. Yang, and X. Hu, “Memorysam: Memorize modalities and semantics with segment anything model 2 for multi-modal semantic segmentation,”arXiv preprint arXiv:2503.06700, 2025

    2025 arXiv

  55. [63]

    Unveiling the potential of segment anything model 2 for rgb-thermal semantic segmentation with language guidance,

    J. Zhao, F. Teng, K. Luo, G. Zhao, Z. Li, X. Zheng, and K. Yang, “Unveiling the potential of segment anything model 2 for rgb-thermal semantic segmentation with language guidance,” arXiv preprint arXiv:2503.02581, 2025. 13

    2025 arXiv

  56. [64]

    Learning modality-agnostic representation for semantic segmentation from any modalities,

    X. Zheng, Y . Lyu, and L. Wang, “Learning modality-agnostic representation for semantic segmentation from any modalities,” inEuropean Conference on Computer Vision, pp. 146–165, Springer, 2024

    2024

  57. [65]

    Magic++: Efficient and resilient modality-agnostic semantic segmentation via hierarchical modality selection,

    X. Zheng, Y . Lyu, L. Jiang, J. Zhou, L. Wang, and X. Hu, “Magic++: Efficient and resilient modality-agnostic semantic segmentation via hierarchical modality selection,”arXiv preprint arXiv:2412.16876, 2024

    2024 arXiv

  58. [66]

    Customize segment anything model for multi- modal semantic segmentation with mixture of lora experts,

    C. Zhu, B. Xiao, L. Shi, S. Xu, and X. Zheng, “Customize segment anything model for multi- modal semantic segmentation with mixture of lora experts,”arXiv preprint arXiv:2412.04220, 2024

    2024 arXiv

  59. [67]

    Deep learn- ing for event-based vision: A comprehensive survey and benchmarks,

    X. Zheng, Y . Liu, Y . Lu, T. Hua, T. Pan, W. Zhang, D. Tao, and L. Wang, “Deep learn- ing for event-based vision: A comprehensive survey and benchmarks,”arXiv preprint arXiv:2302.08890, 2023. 14 A Technical Appendices and Supplementary Material In this Appendices, we provide ...

    2023 arXiv

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.