Pith. sign in

REVIEW 5 major objections 5 minor 14 references

HSLiNets: Hyperspectral Image and LiDAR Data Fusion Using Efficient Dual Non-Linear Feature Learning Networks

T0 review · 5 major / 5 minor · reviewed 2026-08-12 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read A dual reversed CNN with a spatial block claims top Houston 2013 accuracy, 96.68%, without self-attention.

desk verdict Plausible incremental architecture, but the central SOTA claim is unverifiable from the manuscript because the training/test protocol is unspecified and the text has multiple internal inconsistencies. read the letter →

arxiv 2412.00302 v2 pith:RGU5PXPM submitted 2024-11-30 cs.CV eess.IV

Judy X Yang , Jing Wang , Chen Hong Sui , Zekun Long , Jun Zhou This is my paper · ORCID
classification cs.CVeess.IV
keywords hyperspectralimageclassificationLiDARfusiondualnon-linearfeaturespacebidirectionalreversedCNNHouston2013datasetmulti-sensorefficientdeeplearning
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper tries to show that fusing hyperspectral and LiDAR data does not require a large attention-based backbone. It proposes HSLiNet, which forms a dual non-linear feature space from two reversed CNN scans over fused patches plus a spatial branch, and reports that this light architecture reaches 96.68% overall accuracy, 97.32% average accuracy, and a Kappa coefficient of 96.39% on the Houston 2013 dataset, beating six published fusion baselines. The significance, if the result holds, is that high-accuracy multimodal remote sensing classification can be achieved with low memory and a 0.48-second test time, which would make the method practical in GPU-limited or real-time settings.

What carries the argument

The carrying mechanism is the Fused BiNetworks Block (B-Net) paired with the LiDAR Spatial Feature Processing Block (S-Block). On a fused patch, one path computes $x_{\text{forward}} = f_{\text{activation}}(\operatorname{Conv1d}(x_{\text{proj}}))$ while the reversed path computes $x_{\text{backward}} = f_{\text{activation}}(\operatorname{Conv1d}(z_{\text{proj}}, \text{reversed}))$; each direction then applies a delta-modulated tanh state update with learnable matrices $A$ and $B$, and the two resulting states are reduced and summed into $h_{\text{combined}}$. The S-Block independently processes the same patch through $\operatorname{ReLU}(\operatorname{BatchNorm}(\operatorname{Conv2d}(\cdot)))$ to produce $h_{l_{\text{processed}}}$, and the fusion is $y_{\text{fusion}} = \operatorname{concat}(h_{\text{combined}}, h_{l_{\text{processed}}})$, followed by a linear classifier. The double sweep in opposite spectral directions is the module that replaces self-attention, letting the model read spectral dependencies both forward and backward without a global attention matrix.

What would settle it

Re-run the Table I benchmark with a fixed patch size and identical per-class training counts on the standard Houston 2013 split; if HSLiNet's overall accuracy does not remain above the strongest baseline's 94.19%, the superiority claim is falsified.

Watch

Extended reading notes

Core claim

The authors claim that a deliberately small network can outperform attention-based fusion models when hyperspectral and LiDAR patches are fused early and then passed through two reversed 1D convolutional pathways. The forward path uses a transform matrix $A$ to accumulate spectral dependencies in normal band order, while the backward path uses a transform $B$ in reverse order; after the delta-modulated tanh update $h = \tanh(x + A\,\Delta_{\text{expanded}})$ and its backward counterpart, the two reduced hidden states are summed. A separate spatial block applies $\operatorname{ReLU}(\operatorname{BatchNorm}(\operatorname{Conv2d}(\cdot)))$ and its output is concatenated with the bidirectional state to form the vector sent to the classifier. On Houston 2013 the full model reports the best overall accuracy (96.68%), average accuracy (97.32%), and Kappa (96.39%) among the methods in Table I, and the ablation attributes the gain to the joint action of the forward, backward, and spatial components.

Load-bearing premise

The state-of-the-art claim rests on all six comparison models being trained and evaluated under exactly the same Houston 2013 data split, patch size, and training budget, and the paper does not state those numbers.

Editorial extensions

If this is right

  • Attention is not a necessary ingredient: the model's 96.68% overall accuracy is the best in Table I without any self-attention module, so reversed convolutional passes can substitute for attention in HSI-LiDAR fusion.
  • Fusion is what earns the accuracy: the same architecture reaches 96.68% overall accuracy with both modalities, 96.13% with HSI alone, and 27.38% with LiDAR alone, so LiDAR contributes almost nothing by itself but lifts the fused result.
  • Every module pays: removing both directional passes drops overall accuracy by 3.26 points, removing the spatial block drops it by 0.83 points, and removing only the backward path costs 0.53 points.
  • The method is cheap: testing takes 0.48 seconds and training time grows only with the number of epochs, so the reported accuracy comes with a runtime profile that fits resource-constrained settings.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • If the reversed dual-pass structure is what carries the gain, the same block could be transplanted to other paired 1D-2D sensor tasks, such as multispectral plus synthetic-aperture radar or spectral plus depth, without redesigning the fusion module.
  • Because the delta-modulated update resembles a discretized state transition, sweeping the number of reversed passes and the choice of nonlinearity would show whether the gain saturates quickly, which would clarify whether the mechanism behaves more like a state-space model than like plain convolution.
  • A direct transfer test on a second fusion benchmark with different land-cover classes would show whether the margin over the strongest baseline is specific to Houston 2013 or generalizes.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

5 major / 5 minor

Summary. The paper proposes HSLiNets, a dual non-linear feature learning network that fuses hyperspectral image (HSI) and LiDAR data using bidirectional (forward-backward) convolutional pathways and a spatial analysis block. The authors claim state-of-the-art classification accuracy on the Houston 2013 dataset (OA 96.68%, AA 97.32%, Kappa 96.39%) while avoiding the computational cost of self-attention and transformer architectures. The paper also presents an ablation study and a limited efficiency discussion. However, the experimental protocol is critically under-specified, the reported results contain internal contradictions, and the baseline comparisons are difficult to verify due to inconsistent citations and missing details.

Significance. If the performance and efficiency claims were fully validated, the architecture could offer a practical alternative to attention-based fusion models for HSI-LiDAR classification in resource-constrained settings. The idea of learning bidirectional spectral dependencies with a delta-modulated non-linearity is interesting, and the paper explicitly states that attention mechanisms are not required. However, the current manuscript does not substantiate these claims: the missing experimental protocol, the inconsistent numbers between tables, and the unclear baseline definitions mean that the central result cannot be confirmed from the text alone. The contribution as presented is therefore not yet at the standard expected for a journal publication.

major comments (5)
  1. [Section III-A and III-B; Table I] The paper never specifies the exact training/test split, patch size, per-class sample counts, or number of epochs used for the main comparison. Section III-A only says the counts follow 'standard training and test data quantity requirements,' and Section III-B gives batch size and learning rate but omits the patch dimension and the training duration for Table I. Without a fully specified protocol, the claimed 2.49-point OA improvement over S2ENet cannot be distinguished from a protocol difference, especially since baseline per-class accuracies vary widely across methods. This is a load-bearing issue for the central SOTA claim.
  2. [Section III-D, Tables II and III] The ablation results are internally inconsistent: Table II lists Model 1 with OA=0.9668, AA=0.9722, Kappa=0.9639, while the text in Section III-D and the Table III caption report OA=0.9736, AA=0.9786, Kappa=0.9713 for the same Model 1. These numbers cannot both be correct, and the discrepancy directly affects the paper's headline result. The authors must resolve this inconsistency and provide a single, verified set of metrics.
  3. [Section III-C and Reference list] The baseline citations are inconsistent and therefore the comparison set is unclear. In the text, FusAtNet is cited as [7], but in Table I the FusAtNet column is labeled [14], which is the S2ENet reference; TwoBranch is also labeled [7] in the table, yet [7] is a different paper (a multisource CNN classification paper). This makes it impossible to know which specific implementations were compared and whether the baselines are the intended state-of-the-art models.
  4. [Section II-A and II-C, Eqs. (3)-(4)] The transformation parameters A and B, and the delta parameter, are not sufficiently defined. The text describes A and B as capturing forward and backward spectral dependencies, but it never states their shape, initialization, whether they are learned, or how they are applied in Eqs. (3)-(4) (elementwise multiplication is implied but not stated). The 'delta-expanded' variable is also unexplained. This lack of detail prevents reproduction of the core mechanism.
  5. [Section III-C and Fig. 3] The efficiency claim is unsupported. The paper states that testing time is 'always 0.48 seconds' but provides no comparison with any baseline, no parameter counts, no FLOPs, and no memory measurements. The claim of 'linear memory usage' in the conclusion is not demonstrated. Without quantitative comparisons on runtime or complexity, the contribution of reduced computational demand cannot be evaluated.
minor comments (5)
  1. [Throughout] The model name is inconsistent: it is called HSLiNet, FusedBiRNet, 'SS non-linear Model,' and 'Hi-Vim' in different places. Please use one consistent name throughout.
  2. [Section III-A] The dataset description contains an empty citation '[]' in the Houston 2013 paragraph; a proper reference to the GRSS Data Fusion Contest dataset is needed.
  3. [Section III-C] The reference to 'TwoBranch CNN []' has an empty citation; also the sentence 'HSLiNet again led with a score of 97.54, outperforms' is grammatically incomplete.
  4. [Section III-D and Conclusion] The term 'kapa' appears instead of 'Kappa' in the conclusion, and the model is called 'Hi-Vim' only in the conclusion; please unify terminology and fix the typo.
  5. [Figure 3] The figure is referenced but the relationship between epochs and OA is not described in detail; also the statement 'testing time is always 0.48 seconds' should clarify whether this is per epoch, per patch, or for the entire dataset.

Circularity Check

0 steps flagged · score 1.0 of 10

No significant circularity: the central claim is an empirical benchmark comparison, not a derivation from fitted inputs, and the only self-citation is non-load-bearing related work.

full rationale

The paper's equations (1)-(8) define a standard neural architecture with learned parameters A, B, Delta and trainable convolutions and linear layers. None of the reported outputs (OA, AA, Kappa) enters the model definition, and no parameter is fitted to the benchmark numbers and then renamed as a prediction. The only self-citation is reference [4] (Yang et al., TGRS 2024), used in the introduction as related work on dual-CNN fusion; the proposed HSLiNet architecture and its Houston2013 evaluation do not depend on that citation for correctness or uniqueness, so it is not load-bearing. The comparison is not circular, but its completeness is questionable: Section III-A specifies neither the exact train/test split nor per-class training counts, saying only that the numbers are 'extracted based on standard training and test data quantity requirements,' and Table III's text (OA 0.9736) contradicts the table entries (0.9668). These are protocol-transparency and consistency problems, not circularity. Because the benchmark comparison is external to the method's construction, the derivation chain is self-contained.

Assumptions & free parameters 5 free parameters · 5 assumptions · 0 invented entities

The central empirical claim rests almost entirely on the Houston 2013 experiment. The learned matrices A, B, and delta are internal fitted parameters; patch size and split counts are unstated hyperparameters. No analytical derivation is offered, so the model is a fitted architecture supported by a single table of results.

free parameters (5)
  • A (forward transformation matrix) = learned during training
    Appears in Eq. (3) as h_forward = tanh(x_forward + A * Delta_expanded); no constraints or analytical derivation, so the spectral state update is fit to the training data.
  • B (backward transformation matrix) = learned during training
    Eq. (4) uses B to update the backward hidden state; same status as A.
  • Delta (delta modulation parameter) = learned during training
    Eqs. (3)-(4) use Delta_expanded to modulate the state updates; no formula for Delta is given, it is simply learned.
  • Patch size p = not stated
    Sections II-B and II-C define patches as p x p x CH and p x p x 1, but p is never specified; it is a critical hyperparameter chosen without analysis.
  • Training and test sample counts = not stated
    Section III-A says numbers follow 'standard training and test data quantity requirements' but does not give them; the result depends on this split.
assumptions (5)
  • domain assumption The Houston 2013 dataset split named 'standard' in the paper is the same split used by the baseline papers whose numbers appear in Table I.
    Section III-A refers to 'standard training and test data quantity requirements' without defining them; the comparison to prior papers depends on this.
  • domain assumption The baseline accuracy values in Table I were obtained under comparable training conditions (same training samples, patches, epochs).
    Section III-C compares across methods but no protocol harmonization is described.
  • ad hoc to paper The state update h = tanh(x + parameter * Delta) is a valid and sufficient way to model spectral dependencies.
    Equations (3)-(4) in Section II-C are asserted without derivation or comparison against a plain 1D convolution.
  • ad hoc to paper The 'fused non-linear space' improves feature selection without a dedicated experiment.
    Contribution 1 (Section I) claims better feature selection, but no band-selection or interpretability analysis is shown.
  • domain assumption Standard PyTorch training with the stated learning rate and batch size is sufficient to reproduce the reported accuracy.
    Section III-B gives only batch size 32, learning rate 1e-4, and RTX 3090; no seed or schedule is provided.

how reviews work

0 comments
Cite this review

Pith. "Pith review of HSLiNets: Hyperspectral Image and LiDAR Data Fusion Using Efficient Dual Non-Linear Feature Learning Networks." pith.science (2026). https://pith.science/paper/RGU5PXPM

@misc{pith2026241200302,
  author       = {Pith},
  title        = {Pith review of: HSLiNets: Hyperspectral Image and LiDAR Data Fusion Using Efficient Dual Non-Linear Feature Learning Networks},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/RGU5PXPM}},
  note         = {Machine review of arXiv:2412.00302}
}
read the original abstract

The integration of hyperspectral imaging (HSI) and LiDAR data within new linear feature spaces offers a promising solution to the challenges posed by the high-dimensionality and redundancy inherent in HSIs. This study introduces a dual linear fused space framework that capitalizes on bidirectional reversed convolutional neural network (CNN) pathways, coupled with a specialized spatial analysis block. This approach combines the computational efficiency of CNNs with the adaptability of attention mechanisms, facilitating the effective fusion of spectral and spatial information. The proposed method not only enhances data processing and classification accuracy, but also mitigates the computational burden typically associated with advanced models such as Transformers. Evaluations of the Houston 2013 dataset demonstrate that our approach surpasses existing state-of-the-art models. This advancement underscores the potential of the framework in resource-constrained environments and its significant contributions to the field of remote sensing.

Figures

Figures reproduced from arXiv: 2412.00302 by the authors.

Figure 1
Figure 1. The architectural overview of the Proposed HSI and Lidar Fusion [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 2
Figure 2. Visualization and classification maps for the Houston2013 dataset. [PITH_FULL_IMAGE:figures/full_fig_p003_2.png] view at source ↗
Figure 3
Figure 3. OA and Epoch relationship based on different Training loops [PITH_FULL_IMAGE:figures/full_fig_p005_3.png] view at source ↗

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

14 extracted references · 12 canonical work pages

  1. [7]

    Multisource remote sensing data classification based on convolutional neural network,

    X. Xu, W. Li, Q. Ran, Q. Du, L. Gao, and B. Zhang, “Multisource remote sensing data classification based on convolutional neural network,” IEEE Transactions on Geoscience and Remote Sensing , vol. 56, no. 2, pp. 937–949, 2017

    work page 2017

  2. [14]

    S 2enet: Spatial–spectral cross-modal en- hancement network for classification of hyperspectral and lidar data,

    S. Fang, K. Li, and Z. Li, “S 2enet: Spatial–spectral cross-modal en- hancement network for classification of hyperspectral and lidar data,” IEEE Geoscience and Remote Sensing Letters , vol. 19, pp. 1–5, 2021

    work page 2021

  3. [1]

    A joint convolutional cross vit network for hyperspectral and light detection and ranging fusion classification,

    H. Xu, T. Zheng, Y . Liu, Z. Zhang, C. Xue, and J. Li, “A joint convolutional cross vit network for hyperspectral and light detection and ranging fusion classification,” Remote Sensing , vol. 16, no. 3, p. 489, 2024

    work page 2024

  4. [2]

    A survey on deep learning for multimodal data fusion,

    J. Gao, P. Li, Z. Chen, and J. Zhang, “A survey on deep learning for multimodal data fusion,” Neural Computation , vol. 32, no. 5, pp. 829– 864, 2020

    work page 2020

  5. [3]

    An adaptive multi-sensor data fusion method based on deep convolutional neural networks for fault diagnosis of planetary gearbox,

    L. Jing, T. Wang, M. Zhao, and P. Wang, “An adaptive multi-sensor data fusion method based on deep convolutional neural networks for fault diagnosis of planetary gearbox,” Sensors, vol. 17, no. 2, p. 414, 2017

    work page 2017

  6. [4]

    Lidar- guided cross-attention fusion for hyperspectral band selection and image classification,

    J. X. Yang, J. Zhou, J. Wang, H. Tian, and A. W. C. Liew, “Lidar- guided cross-attention fusion for hyperspectral band selection and image classification,” IEEE Transactions on Geoscience and Remote Sensing , 2024

    work page 2024

  7. [5]

    Deep encoder– decoder networks for classification of hyperspectral and lidar data,

    D. Hong, L. Gao, R. Hang, B. Zhang, and J. Chanussot, “Deep encoder– decoder networks for classification of hyperspectral and lidar data,” IEEE Geoscience and Remote Sensing Letters , vol. 19, pp. 1–5, 2020

    work page 2020

  8. [6]

    Collaborative contrastive learning for hyperspectral and lidar classification,

    S. Jia, X. Zhou, S. Jiang, and R. He, “Collaborative contrastive learning for hyperspectral and lidar classification,” IEEE Transactions on Geo- science and Remote Sensing , vol. 61, pp. 1–14, 2023

    work page 2023

Show all 14 references
  1. [8]

    Deep fusion of hyperspectral and lidar images using attention-based cnn,

    S. Falahatnejad and A. Karami, “Deep fusion of hyperspectral and lidar images using attention-based cnn,” SN Computer Science , vol. 4, no. 1, p. 1, 2022

    2022

  2. [9]

    Vision transformers: From semantic segmentation to dense prediction,

    L. Zhang, J. Lu, S. Zheng, X. Zhao, X. Zhu, Y . Fu, T. Xiang, J. Feng, and P. H. Torr, “Vision transformers: From semantic segmentation to dense prediction,” International Journal of Computer Vision , pp. 1–21, 2024

    2024

  3. [10]

    Spectralformer: Rethinking hyperspectral image classification with transformers,

    D. Hong, Z. Han, J. Yao, L. Gao, B. Zhang, A. Plaza, and J. Chanus- sot, “Spectralformer: Rethinking hyperspectral image classification with transformers,” IEEE Transactions on Geoscience and Remote Sensing , vol. 60, pp. 1–15, 2021

    2021

  4. [11]

    Hsi-bert: Hyperspectral image classification using the bidirectional encoder representation from transformers,

    J. He, L. Zhao, H. Yang, M. Zhang, and W. Li, “Hsi-bert: Hyperspectral image classification using the bidirectional encoder representation from transformers,” IEEE Transactions on Geoscience and Remote Sensing , vol. 58, no. 1, pp. 165–178, 2019

    2019

  5. [12]

    Efficiently modeling long sequences with structured state spaces,

    A. Gu, K. Goel, and C. R ´e, “Efficiently modeling long sequences with structured state spaces,” arXiv preprint arXiv:2111.00396 , 2021

    2021 arXiv

  6. [13]

    More diverse means better: Multimodal deep learning meets remote- sensing imagery classification,

    D. Hong, L. Gao, N. Yokoya, J. Yao, J. Chanussot, Q. Du, and B. Zhang, “More diverse means better: Multimodal deep learning meets remote- sensing imagery classification,” IEEE Transactions on Geoscience and Remote Sensing , vol. 59, no. 5, pp. 4340–4354, 2020

    2020

Pith tools

Reviewed August 12, 2026 · model on record in the stance chip above.