Pith. sign in

REVIEW 3 major objections 4 minor 24 references

Omni-Fusion of Spatial and Spectral for Hyperspectral Image Segmentation

T0 review · 3 major / 4 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read Omni-Fuse, a spatial-spectral omni-fusion network, claims to improve hyperspectral image segmentation by fusing spatial and spectral features with bidirectional cross-attention and a coarse-to-fine decoder, beating the strongest baseline…

desk verdict The architecture and ablation are plausible, but the headline margins and p-values are wrong, so the strong claims don't survive contact with Table 1. read the letter →

arxiv 2507.06606 v1 pith:7AJKLSJF submitted 2025-07-09 cs.CV

Qing Zhang , Guoquan Pei , Yan Wang This is my paper
classification cs.CV
keywords hyperspectralimagesegmentationmedicalimagingspatial-spectralfusioncross-dimensionalattentiondual-streamnetworkMambaSwinTransformerDicesimilaritycoefficient
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Omni-Fuse is a dual-stream network that fuses spatial and spectral information from medical hyperspectral images at every stage rather than late fusion. The paper claims this omni-fusion raises tumor segmentation Dice scores to 84.12% on the MDC dataset and 79.78% on the GPCC dataset, beating the next-best method by 5.73 and 4.02 percentage points. The authors argue that bidirectional cross-dimensional attention, spectral-guided selection of spatial queries, and a coarse-to-fine decoder are what make the fusion effective. If the reported gains hold, this is a step toward using hyperspectral microscopy in computational pathology for more accurate tumor boundary detection.

What carries the argument

The carrying mechanism is the Omni-Fuse architecture: a dual-branch extractor (CNN plus Swin Transformer for spatial features, a Mamba-based block for spectral tokens), a cross-dimensional feature enhancement module that refines spatial tokens with deformable self-attention and spectral tokens with vanilla self-attention, followed by two-layer bidirectional cross-attention; a spectral-guided spatial query selection that keeps the top-$N_q$ spatial tokens most correlated with spectral tokens; and a two-stage spatial-spectral decoder that combines a coarse mask from attention layers and SAM's pixel decoder with Mask2Former-style foreground-aware mask refinement.

What would settle it

Take the per-case DSC scores that produced Table 1, run a paired test between Omni-Fuse and QSQL-FL, and compare the resulting p-values with the printed ones. Since a p-value must lie in [0, 1], any printed value above 1 cannot be correct; if the recomputed tests yield any p-value above 0.05, the paper's claim of significant improvement fails even though mean Dice differences may remain.

Watch

Extended reading notes

Core claim

The central claim is that abundant cross-dimensional fusion, applied in the feature enhancer, the query selector, and the decoder, extracts more discriminative spatial-spectral features than the sequential or parallel fusion used by prior methods. In the authors' telling, bidirectional cross-attention lets spatial and spectral features guide each other, spectral-guided spatial query selection compacts redundant spatial tokens, and the two-stage decoder produces a coarse mask then refines it with foreground-aware attention. The resulting model outperforms thirteen comparison methods on both a public choledoch dataset and a private gastric-cancer dataset across DSC, IoU, and Hausdorff distance.

Load-bearing premise

The load-bearing premise is that the reported improvements are statistically significant, but the p-values in Table 1 are numerically impossible (all between 3 and 4.6 when p-values must lie between 0 and 1), so if the real p-values are not all below 0.05 the significance claim collapses even if the mean Dice differences are real.

Editorial extensions

If this is right

  • Omni-Fuse reports the highest DSC, highest IoU, and lowest Hausdorff distance among thirteen comparison methods on both the MDC and GPCC datasets.
  • The ablation study attributes specific gains to each module: adding the cross-dimensional enhancer raises DSC by 2.78% on MDC and 2.58% on GPCC, and replacing the two-stage decoder with a plain SAM decoder drops MDC DSC from 84.12% to 79.39%.
  • Using only pseudo-color RGB instead of the full hyperspectral cube lowers DSC by 9.74% on MDC and 8.21% on GPCC, so spectral information carries substantial task-relevant signal.
  • The two-stage decoder, combining spatial-spectral cross-attention with SAM's pixel decoder and Mask2Former-style mask attention, is what the paper credits for fine boundary refinement.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The paper's printed p-values (3.06 to 4.66) are outside the valid range of 0 to 1, so a reader should recompute significance from per-case scores before trusting the 'statistically significant' claim.
  • A natural follow-up the paper does not run is replacing the two-layer bidirectional cross-attention with a single shared attention block; if most of the DSC gain survives, the efficiency story strengthens.
  • The spectral-redundancy reduction (from 0.5755 to 0.4633 on MDC and from 0.6450 to 0.5154 on GPCC) is stated without a formula; an independent check would compute mean adjacent-band correlation before and after the enhancement module.
  • If the margins survive re-analysis, the fusion pattern likely transfers to other hyperspectral pathology tasks with different band counts; if not, the method may still be practically useful but the statistical claim needs revision.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 4 minor

Summary. The paper proposes Omni-Fuse, a spatial-spectral fusion network for microscopic hyperspectral image segmentation. The architecture combines a CNN/Swin-Transformer spatial branch and a Mamba-based spectral branch with a cross-dimensional feature enhancement module based on bidirectional cross-attention, a spectral-guided spatial query selection mechanism, and a two-stage coarse-to-fine decoder that reuses SAM's pixel decoder and Mask2Former-style mask attention. Experiments are reported on the public MDC dataset and a private GPCC dataset, with DSC, IoU, and Hausdorff distance metrics. The paper claims state-of-the-art performance, including over 5.73% DSC improvement on MDC and 4.02% on GPCC, and states that the improvements are statistically significant.

Significance. If the reported results survive correction, the architectural contribution is meaningful for hyperspectral medical imaging: the bidirectional cross-attention between spatial and spectral branches, the spectral-guided query selection, and the two-stage decoder are sensible and potentially transferable ideas. The use of a public dataset, a patient-centric hard split, and a public code link are strengths for reproducibility. However, the three reporting problems identified below affect exactly the headline quantitative claims, so the contribution cannot be assessed as stated. The paper's value will depend on correcting the comparison baseline, providing valid statistical evidence, and fixing the ablation description.

major comments (3)
  1. [Section 3.2, Table 1] The central claim that Omni-Fuse outperforms the second-best method QSQL-FL by 5.73% DSC on MDC and 4.02% on GPCC is contradicted by the table itself. QSQL-FL obtains 78.39 DSC on MDC and 75.76 on GPCC, whereas SpectralFormer obtains 82.68 and 79.47 and VM-UNet obtains 81.53 and 77.53; QSQL-FL is therefore not the runner-up on either dataset. Against the actual runner-up (SpectralFormer), the margins are 1.44 DSC on MDC and 0.31 DSC on GPCC, and even against VM-UNet the MDC margin is 2.59 DSC rather than 5.73. The abstract, Section 3.2, and Section 4 should be corrected, and the significance discussion should be re-based on the correct comparison.
  2. [Table 1, p-value columns] The reported p-values range from 2.96 to 4.66, which is outside the valid [0,1] range for a p-value, so the sentence in Section 3.2 that 'all p-values below 0.05' cannot be true for these numbers. No statistical test, sample size, pairing scheme, or multiple-comparison correction is described, and no error bars or confidence intervals are given. The significance claim is therefore unsupported; please report valid p-values from a clearly described paired test over per-case or per-patient scores, along with effect sizes and confidence intervals.
  3. [Section 3.3, Table 2] The text states that replacing the two-stage spatial-spectral decoder (SSD+MR) with the SAM decoder drops performance from 84.12% to 79.39% on MDC. However, the 79.39% value appears in the row with CNN, Mamba, and CFE enabled and SQS, SSD, and MR disabled; that row also removes the spectral-guided spatial query selector and does not indicate that a SAM decoder was substituted. The ablation therefore does not isolate the contribution of SSD+MR, and the described comparison is not what Table 2 shows. Please revise the ablation description and either report a dedicated SAM-decoder baseline or state explicitly which modules are removed.
minor comments (4)
  1. [Eq. (3)] The notation max(-1) is undefined; specify that the maximum is taken over the spectral-token dimension and define how TopNq selects indices from the resulting vector.
  2. [Throughout] Typos include 'non-invasice' (Introduction), 'deided' (Conclusion), 'selctor' and 'os' (Table 2 caption), 'steam' (Fig. 2 caption), and 'prove' should be 'improve' (Section 2.4).
  3. [Section 3.1] The claim that Omni-Fuse 'maintains low computational complexity and operates at high speed' is not quantified; please report parameters, FLOPs, and inference time.
  4. [Eqs. (1)-(2)] The definitions of dspa and dspec are ambiguous because the query and key dimensions in the cross-attention are not specified; clarify the tensor shapes.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: Omni-Fuse's central claims are empirical results on held-out test splits, not derivations from fitted parameters.

full rationale

The paper's core contribution is an architecture and its evaluation on two datasets. The reported DSC, IoU, and HD values come from training on a training split and evaluating on a patient-centric held-out test split (Section 3.1), so the segmentation scores are empirical outcomes rather than quantities forced by construction. The loss weights (lambda_ce = 0.25, lambda_dice = 0.75, lambda_w = 0.8) are hand-set hyperparameters and do not predetermine test-set Dice scores. The ablation study in Table 2 compares component combinations on the same evaluation protocol, which is a standard empirical analysis rather than a circular derivation. The authors do cite their own prior work (MDN [12] for the spectral backbone, QSQL-FL [6] as a baseline), but these citations are used as components and comparison points, not as proof of the central claim; the central claim is supported by the held-out evaluation. There are serious internal-consistency problems that affect the validity of the quantitative claims but are not circularity: Table 1 lists p-values between 2.96 and 4.66, which are outside the valid [0,1] range, contradicting the statement that 'all p-values below 0.05'; and the claim of a 5.73% margin over 'the second best approach, QSQL-FLs' is contradicted by the same table, where SpectralFormer (82.68) and VM-UNet (81.53) score higher than QSQL-FL (78.39) on MDC, and similarly on GPCC. These are correctness and reporting issues, not instances where a prediction reduces to its input by definition. No circular step meeting the required evidentiary standard was found.

Assumptions & free parameters 3 free parameters · 3 assumptions · 0 invented entities

The central claim is an empirical comparison, so the ledger captures the hand-chosen training weights, the unspecified query count, the correctness of annotations, and the validity of the statistical test, which is the main unverified input.

free parameters (3)
  • Loss weights λ_ce, λ_dice, λ_w = 0.25, 0.75, 0.8
    Hand-chosen weights for the combined cross-entropy and Dice losses, set in Section 2.4.
  • Number of selected spatial queries Nq = Not reported
    The TopNq selection in Eq. 3 requires a value for Nq, but the paper never specifies it.
  • Training hyperparameters = batch size 8, epochs 300, lr 0.005
    Standard choices for the optimizer; the central results depend on them only through training.
assumptions (3)
  • domain assumption The MDC and GPCC annotations are correct and consistent.
    Supervised training and evaluation assume ground-truth labels are accurate; stated in Section 3.1.
  • domain assumption The patient-centric hard split prevents data leakage between training and test.
    Section 3.1 states the split is patient-centric, but no patient ID details are given to verify independence.
  • ad hoc to paper The reported p-values are computed from a valid statistical test.
    The p-value numbers in Table 1 are numerically impossible, so this assumption appears violated.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Omni-Fusion of Spatial and Spectral for Hyperspectral Image Segmentation." pith.science (2026). https://pith.science/paper/7AJKLSJF

@misc{pith2026250706606,
  author       = {Pith},
  title        = {Pith review of: Omni-Fusion of Spatial and Spectral for Hyperspectral Image Segmentation},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/7AJKLSJF}},
  note         = {Machine review of arXiv:2507.06606}
}
read the original abstract

Medical Hyperspectral Imaging (MHSI) has emerged as a promising tool for enhanced disease diagnosis, particularly in computational pathology, offering rich spectral information that aids in identifying subtle biochemical properties of tissues. Despite these advantages, effectively fusing both spatial-dimensional and spectral-dimensional information from MHSIs remains challenging due to its high dimensionality and spectral redundancy inherent characteristics. To solve the above challenges, we propose a novel spatial-spectral omni-fusion network for hyperspectral image segmentation, named as Omni-Fuse. Here, we introduce abundant cross-dimensional feature fusion operations, including a cross-dimensional enhancement module that refines both spatial and spectral features through bidirectional attention mechanisms, a spectral-guided spatial query selection to select the most spectral-related spatial feature as the query, and a two-stage cross-dimensional decoder which dynamically guide the model to focus on the selected spatial query. Despite of numerous attention blocks, Omni-Fuse remains efficient in execution. Experiments on two microscopic hyperspectral image datasets show that our approach can significantly improve the segmentation performance compared with the state-of-the-art methods, with over 5.73 percent improvement in DSC. Code available at: https://github.com/DeepMed-Lab-ECNU/Omni-Fuse.

Figures

Figures reproduced from arXiv: 2507.06606 by the authors.

Figure 1
Figure 1. (a) Microscopic hyperspectral image (MHSI) of Multi-Dimensional Chole [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 2
Figure 2. Framework architecture of Omni-Fuse. The MHSI is primarily processed [PITH_FULL_IMAGE:figures/full_fig_p003_2.png] view at source ↗
Figure 3
Figure 3. tSNE visualization of the spatial-spectral features processed by several [PITH_FULL_IMAGE:figures/full_fig_p007_3.png] view at source ↗

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

24 extracted references · 21 canonical work pages

  1. [1]

    Applied Spectroscopy Reviews59(7), 935–958 (2024)

    Banu, K.S., Lerma, M., Ahmed, S.U., Gardea-Torresdey, J.L.: Hyperspectral microscopy-applications of hyperspectral imaging techniques in different fields of science: a review of recent advances. Applied Spectroscopy Reviews59(7), 935–958 (2024)

    work page 2024

  2. [2]

    In: Proceedings of Medical Image Computing and Computer-Assisted Intervention

    Bengs, M., Gessert, N., Laffers, W., Eggert, D., Westermann, S., Mueller, N.A., Gerstner, A.O., Betz, C., Schlaefer, A.: Spectral-spatial recurrent-convolutional networks for in-vivo hyperspectral tumor type classification. In: Proceedings of Medical Image Computing and Computer-Assisted Intervention. pp. 690–699. Springer (2020)

    work page 2020

  3. [3]

    Chen,X.,He,K.:Exploringsimplesiameserepresentationlearning.In:Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. pp. 15750–15758 (2021)

    work page 2021

  4. [4]

    Cheng, B., Schwing, A.G., Kirillov, A.: Per-pixel classification is not all you need for semantic segmentation (2021) 10 Qing Zhang, Guoquan Pei, and Yan Wang ( )

    work page 2021

  5. [5]

    In: Proceedings of Medical Image Computing and Computer-Assisted Intervention

    Çiçek, Ö., Abdulkadir, A., Lienkamp, S.S., Brox, T., Ronneberger, O.: 3d u-net: learning dense volumetric segmentation from sparse annotation. In: Proceedings of Medical Image Computing and Computer-Assisted Intervention. pp. 424–432. Springer (2016)

    work page 2016

  6. [6]

    In: Proceedings of Medical Image Computing and Computer-Assisted Intervention

    Dong, H., Zhou, R., Yun, B., Zhou, H., Zhang, B., Li, Q., Wang, Y.: Multi-stage multi-granularity focus-tuned learning paradigm for medical hsi segmentation. In: Proceedings of Medical Image Computing and Computer-Assisted Intervention. pp. 456–466. Springer (2024)

    work page 2024

  7. [7]

    Advances in Neural Information Processing Systems 33, 21271–21284 (2020)

    Grill, J.B., Strub, F., Altsché, F., Tallec, F., Richemond, C., Buchatskaya, P., Doersch, E., Pires, C.A., Guo, B., Azar, M.G.: Bootstrap your own latent-a new approach to self-supervised learning. Advances in Neural Information Processing Systems 33, 21271–21284 (2020)

    work page 2020

  8. [8]

    IEEE Transactions on Geoscience and Remote Sensing60, 1–15 (2021)

    Hong, D., Han, Z., Yao, J., Gao, L., Zhang, B., Plaza, A., Chanussot, J.: Spec- tralformer: Rethinking hyperspectral image classification with transformers. IEEE Transactions on Geoscience and Remote Sensing60, 1–15 (2021)

    work page 2021

Show all 24 references
  1. [9]

    Nature Methods 18(2), 203–211 (2021)

    Isensee, F., Jaeger, P.F., Kohl, S.A., Petersen, J., Maier-Hein, K.H.: nnu-net: A self-configuring method for deep learning-based biomedical image segmentation. Nature Methods 18(2), 203–211 (2021)

    2021

  2. [10]

    In: Proceedings of the IEEE/CVF international conference on computer vision

    Kirillov, A., Mintun, E., Ravi, N., Mao, H., Rolland, C., Gustafson, L., Xiao, T., Whitehead, S., Berg, A.C., Lo, W.Y., et al.: Segment anything. In: Proceedings of the IEEE/CVF international conference on computer vision. pp. 4015–4026 (2023)

    2023

  3. [11]

    IEEE Transactions on Neural Networks and Learning Systems34(10), 8057–8070 (2022)

    Li, W., Gao, Y., Zhang, M., Tao, R., Du, Q.: Asymmetric feature fusion network for hyperspectral and sar image classification. IEEE Transactions on Neural Networks and Learning Systems34(10), 8057–8070 (2022)

    2022

  4. [12]

    In: ICASSP 2025- 2025 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)

    Lin, S., Yun, B., Shen, W., Li, Q., Yang, A., Wang, Y.: Mdn: Mamba-driven dual- stream network for medical hyperspectral image segmentation. In: ICASSP 2025- 2025 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP). pp. 1–5. IEEE (2025)

    2025

  5. [13]

    Liu, B., Yu, A., Yu, X., Wang, R., Gao, K., Guo, W.: Deep multiview learning for hyperspectral image classification. vol. 59(9), pp. 7758–7772 (2020)

    2020

  6. [14]

    Liu, Z., Lin, Y., Cao, Y., Hu, H., Wei, Y., Zhang, Z., Lin, S., Guo, B.: Swin transformer:Hierarchical visiontransformerusingshiftedwindows.In:Proceedings of the IEEE/CVF international conference on computer vision. pp. 10012–10022 (2021)

    2021

  7. [15]

    arXiv preprint arXiv:2402.02491 (2024)

    Ruan, J., Li, J., Xiang, S.: Vm-unet: Vision mamba unet for medical image seg- mentation. arXiv preprint arXiv:2402.02491 (2024)

    2024 arXiv

  8. [16]

    IEEE Transactions on Circuits and Systems for Video Technology (2024)

    Song, L., Feng, Z., Yang, S., Zhang, X., Jiao, L.: Interactive spectral-spatial trans- former for hyperspectral image classification. IEEE Transactions on Circuits and Systems for Video Technology (2024)

    2024

  9. [17]

    In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition

    Tang, Y., Yang, Y., Li, D., Roth, H.R., Landman, H., Xu, B., Nath, V., Hatamizadeh, V.: Self-supervised pre-training of swin transformers for 3d medi- cal image analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. pp. 20730–20740 (2022)

    2022

  10. [18]

    IEEE Transactions on Medical Imaging40(1), 218–227 (2021)

    Wang, Q., Sun, L., Wang, X., Zhou, Y., Hu, M., Chen, M., Wen, J., Li, Y., Li, Q.: Identification of melanoma from hyperspectral pathology image using 3d convolu- tional networks. IEEE Transactions on Medical Imaging40(1), 218–227 (2021)

    2021

  11. [19]

    In: Proceedings of the 31st ACM International Conference on Multimedia

    Xie, X., Jin, T., Yun, B., Li, B., Wang, Q.: Exploring hyperspectral histopathology image segmentation from a deformable perspective. In: Proceedings of the 31st ACM International Conference on Multimedia. pp. 242–251 (2023) Omni-Fusion of Spatial and Spectral for Hyperspectra...

    2023

  12. [20]

    In: Proceed- ings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition

    Yao, Z., Liu, S., Yuan, X., Fang, L.: Specat: Spatial-spectral cumulative-attention transformer for high-resolution hyperspectral image reconstruction. In: Proceed- ings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. pp. 25368–25377 (2024)

    2024

  13. [21]

    IEEE Transactions on Circuits and Systems for Video Technology (2023)

    Yun, B., Lei, B., Chen, J., Wang, J., Qiu, H., Shen, S., Li, W., Wang, Q.: Spectral transformer for microscopic hyperspectral pathology image segmentation. IEEE Transactions on Circuits and Systems for Video Technology (2023)

    2023

  14. [22]

    In: Proceedings of Medical Image Computing and Computer-Assisted Intervention

    Yun, B., Li, Q., Mitrofanova, Q., Zhou, C., Wang, Y.: Factor space and spectrum for medical hyperspectral image segmentation. In: Proceedings of Medical Image Computing and Computer-Assisted Intervention. pp. 152–162. Springer (2023)

    2023

  15. [23]

    Remote Sensing15(12), 2990 (2023)

    Zhang, J., Zhao, L., Jiang, H., Shen, S., Wang, J., Zhang, P., Zhang, W., Wang, L.: Hyperspectral image classification based on dense pyramidal convolution and multi-feature fusion. Remote Sensing15(12), 2990 (2023)

    2023

  16. [24]

    IEEE Access7, 149414–149421 (2019)

    Zhang, Q., Li, Q., Yu, G., Sun, L., Zhou, M., Chu, J.: A multidimensional chole- doch database and benchmarks for cholangiocarcinoma diagnosis. IEEE Access7, 149414–149421 (2019)

    2019

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.