Pith. sign in

REVIEW 2 major objections 2 minor 37 references

Partially trainable tensor decompositions let RGB-pretrained convolutional filters adapt to hyperspectral images by replacing only the spectral component while retaining spatial patterns.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · grok-4.3

2026-06-29 13:29 UTC pith:53YJHGVO

load-bearing objection The core idea of decomposing RGB conv filters into fixed spatial and trainable spectral parts is a reasonable engineering move for hyperspectral transfer, but the abstract gives almost no math or controls so the accuracy claim is hard to evaluate. the 2 major comments →

arxiv 2605.28331 v1 pith:53YJHGVO submitted 2026-05-27 cs.CV

Transfer learning RGB models to hyperspectral images with trainable tensor decompositions

Mariette Sch\"onfeld , Laurens Devos , Wannes Meert , Hendrik Blockeel This is my paper
classification cs.CV
keywords transfer learninghyperspectral imagingtensor decompositionconvolutional neural networkscomputer visionmultispectral imagesmodel adaptation
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper shows how to adapt large vision networks trained on RGB images to hyperspectral data that have many more spectral channels. It decomposes each pretrained filter into a spatial part and a spectral part, then replaces the spectral part with a trainable tensor that matches the new channel count. This keeps the original spatial structure intact and allows the model to specialize on hyperspectral datasets. Experiments across multiple hyperspectral datasets indicate the resulting models achieve higher accuracy and greater robustness than prior transfer-learning approaches for this domain. The approach avoids the information loss that occurs when either the input image or the pretrained model is altered to fit the channel mismatch.

Core claim

The method creates hyperspectral filters by forming partially trainable tensor decompositions of pretrained RGB convolutional filters, separating each filter into its spatial and spectral components, then substituting the spectral component with a trainable tensor of matching higher dimensionality; the resulting filters retain the spatial patterns of the original RGB filters while allowing specialization to hyperspectral data.

What carries the argument

Partially trainable tensor decompositions that split convolutional filters into fixed spatial components and trainable spectral components of higher channel count.

Load-bearing premise

The spatial patterns learned by RGB-pretrained filters stay useful after the spectral part is replaced and retrained on hyperspectral data.

What would settle it

A head-to-head accuracy comparison on a held-out hyperspectral dataset in which a model using the tensor-decomposition transfer underperforms a baseline that simply expands input channels or fine-tunes the full network without decomposition.

Watch this falsifier — get emailed when new claim-graph text bears on it.

If this is right

  • Large RGB-pretrained networks become directly usable on hyperspectral imagery without discarding spatial information or model capacity.
  • The spectral components can be retrained independently on new hyperspectral datasets while spatial knowledge remains frozen.
  • The same decomposition can be applied to any convolutional layer, enabling layer-wise transfer across an entire network.
  • Robustness gains appear across varied hyperspectral datasets, suggesting the retained spatial patterns generalize beyond the original RGB domain.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • The approach could extend to other high-dimensional inputs such as video or 3-D medical volumes by treating an additional dimension as the trainable component.
  • If the spatial patterns prove domain-invariant, the method might reduce the need for large hyperspectral training sets by leveraging existing RGB corpora.
  • Hardware implementations could cache the fixed spatial kernels and only compute the trainable spectral part, lowering memory for hyperspectral inference.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

2 major / 2 minor

Summary. The manuscript proposes a transfer learning technique for hyperspectral images that decomposes pretrained RGB convolutional filters via partially trainable tensor decompositions into spatial and spectral components. The spectral components are replaced by trainable higher-dimensional tensors to match the input channel count, while the spatial components are retained to preserve patterns learned from RGB data. Experiments on multiple hyperspectral datasets are reported to show higher accuracy and robustness than prior hyperspectral transfer learning methods.

Significance. If the central empirical claims hold after addressing the decomposition validity, the work could provide a practical route for leveraging large-scale RGB models in data-scarce hyperspectral domains such as remote sensing without discarding either spectral information or model capacity. The approach is presented as an engineering contribution with empirical validation across datasets; no parameter-free derivations or machine-checked proofs are claimed.

major comments (2)
  1. [Method (decomposition and filter construction)] The load-bearing assumption that the spatial component isolated by the tensor decomposition preserves transferable behavior when the spectral component is retrained on hyperspectral inputs is not yet substantiated. If the decomposition (CP/Tucker or similar) is approximate or if the original 3-channel kernels entangle spatial and spectral factors, the performance gains may derive primarily from added capacity in the new spectral weights rather than retained RGB patterns. This requires explicit validation, such as feature-map similarity metrics or an ablation isolating the spatial reuse effect.
  2. [Experiments and results] The abstract states that experiments demonstrate superior accuracy and robustness, yet the provided text supplies no equations, implementation details, error bars, dataset descriptions, or quantitative tables. Without these, it is impossible to determine whether the reported improvements are statistically significant, reproducible, or attributable to the proposed decomposition rather than hyperparameter choices or extra parameters.
minor comments (2)
  1. Clarify the exact tensor decomposition employed and any rank or factorization hyperparameters in the method description.
  2. Add standard reproducibility elements such as code release statement, exact dataset splits, and training protocol details.

Simulated Author's Rebuttal

2 responses · 0 unresolved

We thank the referee for the constructive comments on our manuscript. We address each major point below and will revise the manuscript to incorporate additional validation and details as outlined.

read point-by-point responses

  1. Referee: [Method (decomposition and filter construction)] The load-bearing assumption that the spatial component isolated by the tensor decomposition preserves transferable behavior when the spectral component is retrained on hyperspectral inputs is not yet substantiated. If the decomposition (CP/Tucker or similar) is approximate or if the original 3-channel kernels entangle spatial and spectral factors, the performance gains may derive primarily from added capacity in the new spectral weights rather than retained RGB patterns. This requires explicit validation, such as feature-map similarity metrics or an ablation isolating the spatial reuse effect.

    Authors: The comparative results across hyperspectral datasets indicate that retaining the spatial components yields measurable gains over baselines that do not preserve RGB-derived spatial patterns. Nevertheless, we agree that direct evidence isolating this effect would strengthen the central claim. In the revised manuscript we will add an ablation that randomizes or retrains the spatial factors while keeping the trainable spectral components, together with cosine similarity measurements between feature maps produced by the original RGB filters and the adapted hyperspectral versions. revision: yes

  2. Referee: [Experiments and results] The abstract states that experiments demonstrate superior accuracy and robustness, yet the provided text supplies no equations, implementation details, error bars, dataset descriptions, or quantitative tables. Without these, it is impossible to determine whether the reported improvements are statistically significant, reproducible, or attributable to the proposed decomposition rather than hyperparameter choices or extra parameters.

    Authors: The full manuscript contains the tensor decomposition equations, dataset descriptions, and quantitative tables. To improve reproducibility and address the concern about statistical rigor, the revised version will expand the experimental section with error bars from multiple random seeds, full hyperparameter specifications, and significance testing of the accuracy differences. revision: yes

Circularity Check

0 steps flagged

No circularity: empirical method with independent experimental validation

full rationale

The paper describes an engineering method that decomposes pretrained RGB convolutional filters into spatial and spectral factors via tensor decomposition, replaces the spectral factors with trainable higher-dimensional weights, and evaluates the resulting model on hyperspectral datasets through direct accuracy and robustness comparisons against baselines. No derivation chain exists that reduces a claimed result to its own fitted parameters or inputs by construction; performance is measured externally on held-out data rather than being a statistical consequence of the decomposition itself. No self-citation load-bearing steps, uniqueness theorems, or ansatzes imported from prior author work appear in the load-bearing claims. The contribution is self-contained against external benchmarks.

Axiom & Free-Parameter Ledger

0 free parameters · 0 axioms · 0 invented entities

Only the abstract is available, so free parameters, axioms, and invented entities cannot be enumerated from the text. The approach implicitly assumes a tensor decomposition exists that cleanly separates spatial and spectral factors without loss of information.

pith-pipeline@v0.9.1-grok · 5688 in / 1073 out tokens · 32403 ms · 2026-06-29T13:29:57.727120+00:00 · methodology

0 comments
read the original abstract

Transfer learning makes it possible to use large vision networks on a variety of domains, by specializing their models' general filters to new tasks. However, these networks assume the input images to have 3 input channels, making them incompatible with multi- or hyperspectral images. Current approaches that mitigate this incompatibility sacrifice information in either the image, or the model. This work proposes a novel approach that preserves the image and spatial information present in the model by using partially trainable tensor decompositions. We create such decompositions of pretrained convolutional filters, separating the filters into spatial and spectral components. The spectral components are then replaced with trainable components of higher channel dimensionality. This creates hyperspectral filters that can specialize to new datasets, while retaining the spatial patterns of the original filter. Experiments on a variety of hyperspectral datasets show that our approach is more accurate and robust than other hyperspectral transfer learning methods.

Figures

Figures reproduced from arXiv: 2605.28331 by Hendrik Blockeel, Laurens Devos, Mariette Sch\"onfeld, Wannes Meert.

Figure 1
Figure 1. Figure 1: Illustration of our general approach. Values for the new hyperspectral [PITH_FULL_IMAGE:figures/full_fig_p005_1.png] view at source ↗
Figure 2
Figure 2. Figure 2: Illustration of how a filter is decomposed into its spectral ( [PITH_FULL_IMAGE:figures/full_fig_p005_2.png] view at source ↗
Figure 4
Figure 4. Figure 4: Architecture of the different transfer learning approaches with regards to [PITH_FULL_IMAGE:figures/full_fig_p010_4.png] view at source ↗
Figure 5
Figure 5. Figure 5: RGB-extracts of filters learned when using ‘Grape Leaves’ for training. [PITH_FULL_IMAGE:figures/full_fig_p012_5.png] view at source ↗
Figure 6
Figure 6. Figure 6: Two example loss curves over ‘Grape leaves’ for a learning rate of 0.01 [PITH_FULL_IMAGE:figures/full_fig_p012_6.png] view at source ↗
Figure 7
Figure 7. Figure 7: Final accuracy of different hyperspectral transfer learning methods with [PITH_FULL_IMAGE:figures/full_fig_p013_7.png] view at source ↗
Figure 8
Figure 8. Figure 8: (a) Number of experiments where a higher rank results in a higher accu [PITH_FULL_IMAGE:figures/full_fig_p014_8.png] view at source ↗
Figure 9
Figure 9. Figure 9: Loss curves over several combinations of learning rates and batch sizes on [PITH_FULL_IMAGE:figures/full_fig_p019_9.png] view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

37 extracted references · 2 canonical work pages · 2 internal anchors

  1. [1]

    In: Proceedings of the IEEE conference on computer vision and pattern recognition

    Chollet, F.: Xception: Deep learning with depthwise separable convolutions. In: Proceedings of the IEEE conference on computer vision and pattern recognition. pp. 1251–1258 (2017)

    2017

  2. [2]

    In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition

    Corley, I., Robinson, C., Dodhia, R., Ferres, J.M.L., Najafirad, P.: Revisiting pre- trained remote sensing model benchmarks: resizing and normalization matters. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. pp. 3162–3172 (2024)

    2024

  3. [3]

    In: 2009 IEEE conference on computer vision and pattern recognition

    Deng, J., Dong, W., Socher, R., Li, L.J., Li, K., Fei-Fei, L.: Imagenet: A large- scale hierarchical image database. In: 2009 IEEE conference on computer vision and pattern recognition. pp. 248–255. Ieee (2009)

    2009

  4. [4]

    Advances in neural information processing systems27(2014)

    Denton, E.L., Zaremba, W., Bruna, J., LeCun, Y., Fergus, R.: Exploiting linear structure within convolutional networks for efficient evaluation. Advances in neural information processing systems27(2014)

    2014

  5. [5]

    In: 2022 IEEE 4th International Conference on Cybernetics, Cognition and Machine Learning Applications (ICCCMLA)

    Giri, R.N., Janghel, R.R., Govil, H., Pandey, S.K.: Spatial feature extraction using pretrained convolutional neural network for hyperspectral image classification. In: 2022 IEEE 4th International Conference on Cybernetics, Cognition and Machine Learning Applications (ICCCMLA). pp. 386–389. IEEE (2022)

    2022

  6. [6]

    Gra˜ na, M., Veganzones, M.A., Ayerdi, B.: Hyperspectral remote sens- ing scenes.https://www.ehu.eus/ccwintco/index.php/Hyperspectral_Remote_ Sensing_Scenes(1996), accessed on November 4 2025

    1996

  7. [7]

    In: Proceedings of the IEEE conference on computer vision and pattern recognition

    He, K., Zhang, X., Ren, S., Sun, J.: Deep residual learning for image recognition. In: Proceedings of the IEEE conference on computer vision and pattern recognition. pp. 770–778 (2016)

    2016

  8. [8]

    ICLR1(2), 3 (2022)

    Hu, E.J., Shen, Y., Wallis, P., Allen-Zhu, Z., Li, Y., Wang, S., Wang, L., Chen, W., et al.: Lora: Low-rank adaptation of large language models. ICLR1(2), 3 (2022)

    2022

  9. [9]

    In: Proceedings of the IEEE conference on computer vision and pattern recognition

    Hu, J., Shen, L., Sun, G.: Squeeze-and-excitation networks. In: Proceedings of the IEEE conference on computer vision and pattern recognition. pp. 7132–7141 (2018)

    2018

  10. [10]

    In: Proceedings of the IEEE conference on computer vision and pattern recognition

    Huang, G., Liu, Z., Van Der Maaten, L., Weinberger, K.Q.: Densely connected convolutional networks. In: Proceedings of the IEEE conference on computer vision and pattern recognition. pp. 4700–4708 (2017)

    2017

  11. [11]

    In: Proceedings of the British Machine Vision Conference 2014

    Jaderberg, M., Vedaldi, A., Zisserman, A.: Speeding up convolutional neural net- works with low rank expansions. In: Proceedings of the British Machine Vision Conference 2014. pp. 88–1. British Machine Vision Association (2014)

    2014

  12. [12]

    Compression of Deep Convolutional Neural Networks for Fast and Low Power Mobile Applications

    Kim, Y.D., Park, E., Yoo, S., Choi, T., Yang, L., Shin, D.: Compression of deep convolutional neural networks for fast and low power mobile applications. arXiv preprint arXiv:1511.06530 (2015)

    work page internal anchor Pith review Pith/arXiv arXiv 2015

  13. [13]

    SIAM review 51(3), 455–500 (2009)

    Kolda, T.G., Bader, B.W.: Tensor decompositions and applications. SIAM review 51(3), 455–500 (2009)

    2009

  14. [14]

    Journal of Machine Learning Research20(26), 1–6 (2019)

    Kossaifi, J., Panagakis, Y., Anandkumar, A., Pantic, M.: Tensorly: Tensor learning in python. Journal of Machine Learning Research20(26), 1–6 (2019)

    2019

  15. [15]

    One weird trick for parallelizing convolutional neural networks

    Krizhevsky, A.: One weird trick for parallelizing convolutional neural networks. arXiv preprint arXiv:1404.5997 (2014)

    work page internal anchor Pith review Pith/arXiv arXiv 2014

  16. [16]

    In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition

    Laprade, W.M., Pieta, P., Kutuzova, S., Westergaard, J.C., Nielsen, M., Chris- tensen, S., Dahl, A.B.: Hyperleaf2024-a hyperspectral imaging dataset for classifi- cation and regression of wheat leaves. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. pp. 1234–1243 (2024)

    2024

  17. [17]

    In: 3rd Inter- national Conference on Learning Representations, ICLR 2015-Conference Track Proceedings (2015)

    Lebedev, V., Ganin, Y., Rakhuba, M., Oseledets, I., Lempitsky, V.: Speeding-up convolutional neural networks using fine-tuned cp-decomposition. In: 3rd Inter- national Conference on Learning Representations, ICLR 2015-Conference Track Proceedings (2015)

    2015

  18. [18]

    Journal of biomedical optics19(1), 010901–010901 (2014)

    Lu, G., Fei, B.: Medical hyperspectral imaging: a review. Journal of biomedical optics19(1), 010901–010901 (2014)

    2014

  19. [19]

    Postharvest Biology and Technology170, 111318 (2020)

    Lu, Y., Saeys, W., Kim, M., Peng, Y., Lu, R.: Hyperspectral imaging technology for quality and safety evaluation of horticultural products: A review and celebration of the past 20-year progress. Postharvest Biology and Technology170, 111318 (2020)

    2020

  20. [20]

    In: Proceedings of the 18th ACM international conference on Multimedia

    Marcel, S., Rodriguez, Y.: Torchvision the machine-vision package of torch. In: Proceedings of the 18th ACM international conference on Multimedia. pp. 1485– 1488 (2010)

    2010

  21. [21]

    Proceedings of the IEEE109(5), 863–890 (2021)

    Panagakis, Y., Kossaifi, J., Chrysos, G.G., Oldfield, J., Nicolaou, M.A., Anand- kumar, A., Zafeiriou, S.: Tensor methods in computer vision and deep learning. Proceedings of the IEEE109(5), 863–890 (2021)

    2021

  22. [22]

    In: European Conference on Com- puter Vision

    Phan, A.H., Sobolev, K., Sozykin, K., Ermilov, D., Gusak, J., Tichavsk` y, P., Glukhov, V., Oseledets, I., Cichocki, A.: Stable low-rank tensor decomposition for compression of convolutional neural network. In: European Conference on Com- puter Vision. pp. 522–539. Springer (2020)

    2020

  23. [23]

    In: Pro- ceedings of the IEEE conference on computer vision and pattern recognition

    Rigamonti, R., Sironi, A., Lepetit, V., Fua, P.: Learning separable filters. In: Pro- ceedings of the IEEE conference on computer vision and pattern recognition. pp. 2754–2761 (2013)

    2013

  24. [24]

    Scientific Data 10(1), 743 (2023)

    Ryckewaert, M., H´ eran, D., Trani, J.P., Mas-Garcia, S., Feilhes, C., Prezman, F., Serrano, E., Bendoula, R.: Hyperspectral images of grapevine leaves includ- ing healthy leaves and leaves with biotic and abiotic symptoms. Scientific Data 10(1), 743 (2023)

    2023

  25. [25]

    IEEE transactions on neural networks and learning systems26(5), 1019–1034 (2014)

    Shao, L., Zhu, F., Li, X.: Transfer learning for visual categorization: A survey. IEEE transactions on neural networks and learning systems26(5), 1019–1034 (2014)

    2014

  26. [26]

    MIT Press (2024)

    Torralba, A., Isola, P., Freeman, W.T.: Foundations of computer vision. MIT Press (2024)

    2024

  27. [27]

    In: 6th International Conference on Optical Character- ization of Materials, OCM

    Varga, L.A., Frank, H., Zell, A.: Self-supervised pretraining for hyperspectral clas- sification of fruit ripeness. In: 6th International Conference on Optical Character- ization of Materials, OCM. pp. 97–108 (2023)

    2023

  28. [28]

    In: 2021 International Joint Conference on Neural Networks (IJCNN)

    Varga, L.A., Makowski, J., Zell, A.: Measuring the ripeness of fruit with hyperspec- tral imaging and deep learning. In: 2021 International Joint Conference on Neural Networks (IJCNN). pp. 1–8. IEEE (2021)

    2021

  29. [29]

    In: Proceedings of the IEEE/CVF Winter Con- ference on Applications of Computer Vision

    Varga, L.A., Messmer, M., Benbarka, N., Zell, A.: Wavelength-aware 2d convolu- tions for hyperspectral imaging. In: Proceedings of the IEEE/CVF Winter Con- ference on Applications of Computer Vision. pp. 3788–3797 (2023)

    2023

  30. [30]

    IEEE Transactions on Geoscience and Remote Sensing61, 1–20 (2022)

    Wang, D., Zhang, J., Du, B., Xia, G.S., Tao, D.: An empirical study of remote sensing pretraining. IEEE Transactions on Geoscience and Remote Sensing61, 1–20 (2022)

    2022

  31. [31]

    IEEE Transactions on Geoscience and Remote Sensing56(5), 2798–2810 (2018)

    Windrim, L., Melkumyan, A., Murphy, R.J., Chlingaryan, A., Ramakrishnan, R.: Pretraining for hyperspectral convolutional neural network classification. IEEE Transactions on Geoscience and Remote Sensing56(5), 2798–2810 (2018)

    2018

  32. [32]

    In: Proceedings of the European conference on computer vision (ECCV)

    Woo, S., Park, J., Lee, J.Y., Kweon, I.S.: Cbam: Convolutional block attention module. In: Proceedings of the European conference on computer vision (ECCV). pp. 3–19 (2018)

    2018

  33. [33]

    The Twelfth International Conference on Learning Representations (2024)

    Wu, X., Huang, S., Wei, F.: Mixture of lora experts. The Twelfth International Conference on Learning Representations (2024)

    2024

  34. [34]

    In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition

    Yin, M., Sui, Y., Liao, S., Yuan, B.: Towards efficient tensor decomposition- based dnn model compression with optimization framework. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. pp. 10674– 10683 (2021)

    2021

  35. [35]

    In: European conference on computer vision

    Zeiler, M.D., Fergus, R.: Visualizing and understanding convolutional networks. In: European conference on computer vision. pp. 818–833. Springer (2014)

    2014

  36. [36]

    IEEE Transactions on Geoscience and Remote Sensing57(8), 5813–5828 (2019)

    Zhang, H., Li, Y., Jiang, Y., Wang, P., Shen, Q., Shen, C.: Hyperspectral classifi- cation based on lightweight 3-d-cnn with transfer learning. IEEE Transactions on Geoscience and Remote Sensing57(8), 5813–5828 (2019)

    2019

  37. [37]

    Zhu, X.X., Tuia, D., Mou, L., Xia, G.S., Zhang, L., Xu, F., Fraundorfer, F.: Deep learning in remote sensing: A comprehensive review and list of resources. IEEE geoscience and remote sensing magazine5(4), 8–36 (2017) A Loss curves 1 2Loss lr:0.0001, batch_size:4, 1 2 lr:0.001, batch_size:4, 1 2 lr:0.01, batch_size:4, 1 2Loss lr:0.0001, batch_size:8, 1 2 l...

    2017