Pith. sign in

REVIEW 3 major objections 7 minor 14 references

HSI-Drive v2.0: More Data for New Challenges in Scene Understanding for Autonomous Driving

T0 review · 3 major / 7 minor · reviewed 2026-08-12 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read HSI-Drive v2.0, a four-season hyperspectral driving dataset with 272% more images, lifts FCN segmentation global IoU from 91.50 to 96.87.

desk verdict Useful dataset, but the claimed gains from data expansion are not backed by a controlled comparison. read the letter →

arxiv 2411.17530 v1 pith:SEGQB334 submitted 2024-11-26 cs.CV cs.AIcs.LGeess.IV

Jon Gutiérrez-Zaballa , Koldo Basterretxea , Javier Echanobe , M. Victoria Martínez , Unai Martínez-Corral This is my paper · ORCID
classification cs.CVcs.AIcs.LGeess.IV
keywords hyperspectralimagingsemanticsegmentationautonomousdrivingdatasetfullyconvolutionalnetworkssceneunderstandingHSI-Drive
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper introduces HSI-Drive v2.0, an expanded hyperspectral image dataset for autonomous driving research, containing 752 manually labeled images recorded in all four seasons and roughly 44 million labeled pixels in 10 classes. The authors claim that training fully convolutional networks on this larger, more diverse dataset yields substantial accuracy gains over the earlier v1.1 version, with global IoU rising from 91.50 to 96.87 for road and road-mark segmentation and from 87.66 to 94.51 for a five-class scene segmentation. They also show that the model can segment new safety-critical categories, Painted Metal (vehicles and signs) and Pedestrian, with mean IoUs near 59 and 62 respectively. The work matters because it suggests that hyperspectral imaging, not just RGB vision, can support reliable scene understanding in autonomous vehicles across varied lighting and weather, using models that can be pruned to a few hundred thousand parameters.

What carries the argument

The central object is HSI-Drive v2.0, a structured dataset of 752 hyperspectral images (25 bands, 535 to 975 nm, 216 by 409 spatial resolution after demosaicing) with 10 manually labeled material-based classes and about 44 million labeled pixels, partitioned by season, weather, daytime, and road type. The dataset's organization matters because it lets the authors train and evaluate FCNs both on the full seasonal diversity and on condition-specific subsets. The accompanying model is an encoder-decoder FCN with 31.1 million parameters, trained on 192 by 384 overlapping patches with per-pixel spectral normalization and inverse-frequency weighted cross-entropy, and later pruned to 320K parameters. The mechanism that carries the argument is scale plus diversity: more labeled pixels, especially for minority classes like Road Marks, Painted Metal, and Pedestrian, give the fully convolutional model a stronger spectral and spatial basis for separating classes under real driving conditions.

What would settle it

Train the exact same v2.0-optimized FCN, with identical architecture, preprocessing, and patch size, on the 276 images of v1.1 and compare global IoU on the same test partitions; if the v1.1-trained model matches the v2.0-trained model's accuracy (for example, global IoU within a point or two), then the paper's claim that the expanded dataset drives the improvement is falsified.

Watch

Extended reading notes

Core claim

The central claim is that adding 476 new annotated images from fall and winter to the HSI-Drive dataset, a 272% increase over v1.1, together with a redesigned deeper encoder-decoder FCN, produces significant improvements in hyperspectral road-scene segmentation. Across five-fold cross-validation, the global IoU in the road-marking experiment improves from 91.50 to 96.87, and the five-class experiment improves from 87.66 to 94.51; the Road Marks class shows the largest gains, with precision rising from 77.22 to 95.53 in the first experiment. The expanded dataset also supports two new six-class experiments that add Painted Metal and Pedestrian categories, achieving mean IoUs of 58.61 and 61.94 despite high intra-class spectral variability. The paper further reports that segmentation remains dependable across weather and lighting subsets, with cloudy conditions yielding the best global IoU (96.29) and sunny conditions the worst (91.84), and that the final model can be pruned to 320K parameters and 8.49 GFLOPS without noticeable accuracy loss.

Load-bearing premise

The reported improvements are attributed to the larger, more diverse dataset, but no experiment isolates the data contribution from simultaneous changes to model architecture, preprocessing, and training patch size.

Editorial extensions

If this is right

  • Global IoU for road and road-mark segmentation rises from 91.50 to 96.87, and for the five-class scene from 87.66 to 94.51, when models are trained on v2.0.
  • Road Marks, the class most relevant to lane keeping, sees precision jump from 77.22 to 95.53 in Experiment 1 and IoU from 64.90 to 86.08 in Experiment 2.
  • New six-class models can segment Painted Metal (vehicles and signs) and Pedestrian classes with mean IoUs of 58.61 and 61.94, respectively.
  • Segmentation is most accurate under cloudy conditions (global IoU 96.29) and least accurate in sunny, high-contrast scenes (91.84), with rainy conditions mainly degrading Road Marks.
  • The pruned model, at 320K parameters and 8.49 GFLOPS, is well below the complexity typically considered feasible for real-time embedded deployment.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Because the authors changed model depth, preprocessing, and training patch size at the same time as adding data, the specific accuracy gain attributable to the new images alone remains an open question; a controlled ablation on v1.1 versus v2.0 with identical architecture would settle it.
  • The four-season structure invites targeted training and testing per condition, so future work could specialize models for dawn or dusk and sunny contrast rather than relying on one universal model.
  • The qualitative video results suggest that temporal continuity, such as tracking a cyclist across frames, could compensate for single-frame segmentation failures, a direction the paper names as future work.
  • If the hyperspectral advantage over RGB holds up in more controlled tests, it could open a niche for HSI sensors in safety-critical perception, especially under overexposure and shadow conditions.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 7 minor

Summary. This manuscript presents HSI-Drive v2.0, an extended hyperspectral driving dataset containing 752 manually annotated images (about 44 million labeled pixels) spanning four seasons, and evaluates fully convolutional network (FCN) segmentation models in four experiments. The authors report large Intersection-over-Union (IoU) gains over previously published v1.1 results, analyze performance under different lighting and weather conditions, and provide qualitative evaluations of representative driving scenes. The paper's central claim is that the expanded dataset, together with a redesigned model, enables substantially more accurate HSI segmentation for autonomous driving, and the dataset is publicly released.

Significance. If the empirical results are interpreted carefully, HSI-Drive v2.0 is a valuable community resource: it is one of the few hyperspectral driving datasets with real-world, multi-season recordings, structured metadata, and a substantial number of labeled pixels. The paper also demonstrates that an encoder-decoder FCN can segment 25-band HSI images at reasonable accuracy, and it reports an aggressive pruning result (31.1M to 320K parameters) that, if substantiated, would be relevant for embedded deployment. However, the headline improvement from v1.1 to v2.0 is confounded by simultaneous changes in architecture, preprocessing, training configuration, and test split, so the specific contribution of the added fall/winter data is not established by the experiments as designed. The paper also introduces two new six-class tasks (Painted Metal, Pedestrian) without any baseline comparison, and several metric-definition details are missing.

major comments (3)
  1. [Abstract, Sec. IV-A] The claimed improvements (global IoU 91.50 to 96.87 in Exp. 1 and 87.66 to 94.51 in Exp. 2) are presented as resulting from the expanded v2.0 dataset, but the comparison to v1.1 changes the model (the tiny FCN of [12] vs. the 31.1M-parameter depth-5 encoder-decoder, Sec. III-C), the preprocessing (removal of the median filter and addition of per-pixel normalization, Sec. III-B), the training patch size (192x384 overlapping patches, Sec. III-C), and the test split (a new 5-fold split of 752 images vs. the previous 276-image split, Sec. III-B). Because these factors change simultaneously, the specific contribution of the added fall/winter images to the reported IoU gains is not established. A controlled experiment that trains the same architecture with identical preprocessing and test protocol on a v1.1-equivalent subset and on v2.0 is necessary to support the central claim.
  2. [Sec. III-C] The paper explicitly states that deeper encoder structures were chosen 'to make the most of the availability of new data' and that training on larger images 'implies using deeper networks,' but it offers no ablation that varies dataset size while holding the model fixed. The hyperparameter grid search (encoder depth, filter count, kernel size, dropout placement/rate, and learning rate) appears to have been run on the full v2.0 data, so it is unknown whether the selected configuration would also be optimal on v1.1. This further complicates the interpretation of the v1.1-vs-v2.0 comparison in Sec. IV-A.
  3. [Sec. IV-A, Tables II-V] The new experiments 3 and 4 (Painted Metal and Pedestrian classes) are presented as 'widening' scene understanding, but they have no comparison to any baseline or to v1.1, and the recall values for these new classes are low (65.40 for Painted Metal, 70.02 for Pedestrian). While this may be acceptable as a feasibility demonstration, the paper should state this explicitly and avoid implying that the new classes are handled with the same level of confidence as the established classes.
minor comments (7)
  1. [Sec. IV-A, Tables II-V] It is unclear whether the reported means and standard deviations are computed over the 5 folds, the 3 weight initializations, or the 15 possible combinations; please specify the aggregation procedure for the reported metrics.
  2. [Sec. III-B, Figs. 3-15] The text mentions that metrics are computed on 'complete 216x409 images' and that performance 'extends robustly to unlabeled pixels', but the figures show a 'NoLab' class and the class frequencies in Table I sum to 100% without a NoLab category; please clarify whether unlabeled pixels are excluded from the metric computation or assigned to a class.
  3. [Sec. IV-B, Table VI] The lighting/weather analysis in Table VI does not report the number of images per condition, and the subsets may differ in season and road-type composition. Conclusions such as 'the poorest results were obtained in the Sunny subset' would be more robust if per-condition sample sizes and a breakdown by season/road type were provided.
  4. [Sec. IV-C] There are typographical errors: 't explore' in Sec. IV-B should be 'to explore', and 'situatoins' in Sec. IV-C should be 'situations'.
  5. [Sec. III-C, Sec. IV-C] The pruning claim (31.10M parameters and 34.87 GFLOPS reduced to 320K parameters and 8.49 GFLOPS 'with no noticeable impact on the model's accuracy') is deferred to future work without presenting any accuracy comparison. Since this claim is not supported in the current manuscript, it should be either removed or accompanied by the corresponding accuracy numbers.
  6. [Sec. II, Table I] The class abbreviations in Table I ('Pain.', 'Unpain. Met.', 'R. Marks') are not expanded in the table caption; please provide full names for readability and to avoid ambiguity.
  7. [References and author block] There are formatting inconsistencies in the reference list (e.g., 'V .' in [1]) and in the author block ('5rd' instead of '5th'); a thorough copyedit would improve the presentation.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the paper contributes a dataset and reports measured segmentation metrics; the improvement attribution is confounded but not circular.

full rationale

This paper is an empirical dataset paper, not a derivation. The claimed improvements (e.g., global IoU rising from 91.50 to 96.87 in Experiment 1, and from 87.66 to 94.51 in Experiment 2) are measured outcomes on a held-out test fold of the v2.0 data, not quantities predicted from fitted parameters. No parameter is fitted to a target and then relabeled as a prediction, and no equation reduces a result to its own input. The self-citations to [11] and [12] provide the v1.1 baseline numbers and the base architecture, but the v2.0 metrics are computed on a new 5-fold split of 752 images and are checkable against the released dataset; the self-citations are therefore not load-bearing in a circular sense. The paper's main weakness is that the comparison to v1.1 changes several factors simultaneously: the model grows from the tiny FCN of [12] to a 31.1M-parameter encoder-decoder (Section III-C), the median filter is removed and per-pixel normalization is added (Section III-B), training uses 192x384 overlapping patches, and the test split differs. Consequently, the sentence 'This improvement is mainly attributed to the increase in IoU of the Road Marks class' and the broader claim that added data drives the gains are causal attributions without an ablation. That is a confound or correctness risk, not a circular derivation, and it does not make the dataset contribution equivalent to its inputs.

Assumptions & free parameters 0 free parameters · 3 assumptions · 0 invented entities

No free parameters or invented entities: the paper makes an empirical dataset and trains models, so the ledger is dominated by measurement and domain assumptions about annotation quality and spectral discriminability.

assumptions (3)
  • domain assumption Manual labeling is accurate.
    All reported metrics depend on the correctness of the manually annotated ground truth in the dataset; errors or label noise would propagate into every IoU number.
  • domain assumption Spectral reflectance is sufficient to distinguish the defined classes.
    The classes (e.g., Painted Metal, Pedestrian) are defined by materials/objects with high intra-class spectral variability; the authors assume the 25-band VIS-NIR signatures carry enough information for segmentation, an assumption they themselves note is challenged by metamerism and mixing.
  • domain assumption Sensor calibration via dark and flat images and per-pixel normalization is valid.
    The preprocessing described in Section III-B assumes that reflectance correction and per-pixel normalization remove illumination effects without destroying spectral discriminability; this is invoked before all experiments.

how reviews work

0 comments
Cite this review

Pith. "Pith review of HSI-Drive v2.0: More Data for New Challenges in Scene Understanding for Autonomous Driving." pith.science (2026). https://pith.science/paper/SEGQB334

@misc{pith2026241117530,
  author       = {Pith},
  title        = {Pith review of: HSI-Drive v2.0: More Data for New Challenges in Scene Understanding for Autonomous Driving},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/SEGQB334}},
  note         = {Machine review of arXiv:2411.17530}
}
read the original abstract

We present the updated version of the HSI-Drive dataset aimed at developing automated driving systems (ADS) using hyperspectral imaging (HSI). The v2.0 version includes new annotated images from videos recorded during winter and fall in real driving scenarios. Added to the spring and summer images included in the previous v1.1 version, the new dataset contains 752 images covering the four seasons. In this paper, we show the improvements achieved over previously published results obtained on the v1.1 dataset, showcasing the enhanced performance of models trained on the new v2.0 dataset. We also show the progress made in comprehensive scene understanding by experimenting with more capable image segmentation models. These models include new segmentation categories aimed at the identification of essential road safety objects such as the presence of vehicles and road signs, as well as highly vulnerable groups like pedestrians and cyclists. In addition, we provide evidence of the performance and robustness of the models when applied to segmenting HSI video sequences captured in various environments and conditions. Finally, for a correct assessment of the results described in this work, the constraints imposed by the processing platforms that can sensibly be deployed in vehicles for ADS must be taken into account. Thus, and although implementation details are out of the scope of this paper, we focus our research on the development of computationally efficient, lightweight ML models that can eventually operate at high throughput rates. The dataset and some examples of segmented videos are available in https://ipaccess.ehu.eus/HSI-Drive/.

Figures

Figures reproduced from arXiv: 2411.17530 by the authors.

Figure 1
Figure 1. Grayscale images of the most saturated (left) and least saturated (right) bands and number of saturated pixels by band (center) of image 566, captured [PITH_FULL_IMAGE:figures/full_fig_p007_1.png] view at source ↗
Figure 2
Figure 2. Boxplots (top) and number of outliers (bottom) of Road Marks (left), Painted Metal (middle) and Pedestrian (right) classes using the spectral signatures [PITH_FULL_IMAGE:figures/full_fig_p007_2.png] view at source ↗
Figure 8
Figure 8. Image 228 (f8, AG1, 10ms), captured during a spring, sunny midday, in an urban environment with shadows: (far left) Exp1 segmentation, (left) Exp1 [PITH_FULL_IMAGE:figures/full_fig_p008_8.png] view at source ↗

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

14 extracted references · 10 canonical work pages

  1. [12]

    On-chip hyperspectral image segmentation with fully convolutional networks for scene understanding in autonomous driving,

    J. Guti ´errez-Zaballa, K. Basterretxea, J. Echanobe, M. V . Mart ´ınez, U. Martinez-Corral, ´Oscar Mata-Carballeira, and I. del Campo, “On-chip hyperspectral image segmentation with fully convolutional networks for scene understanding in autonomous driving,” Journal of Systems Architecture, vol. 139, p. 102878, 2023. [Online]. Available: https://www.scie...

    work page 2023

  2. [1]

    HSI-Drive: A Dataset for the Research of Hyperspectral Image Processing Applied to Autonomous Driving Systems,

    K. Basterretxea, V . Mart ´ınez, J. Echanobe, J. Guti ´errez–Zaballa, and I. Del Campo, “HSI-Drive: A Dataset for the Research of Hyperspectral Image Processing Applied to Autonomous Driving Systems,” in 2021 IEEE Intelligent Vehicles Symposium (IV) , 2021, pp. 866–873

    work page 2021

  3. [2]

    MV1-D2048x1088-HS02-96-G2

    Photonfocus, “MV1-D2048x1088-HS02-96-G2.” [Online]. Avail- able: https://www.photonfocus.com/products/camerafinder/camera/ mv1-d2048x1088-hs02-96-g2

    work page

  4. [3]

    Efficient hyper- spectral image segmentation for biosecurity scanning using knowledge distillation from multi-head teacher,

    M. H. Phan, S. L. Phung, K. Luu, and A. Bouzerdoum, “Efficient hyper- spectral image segmentation for biosecurity scanning using knowledge distillation from multi-head teacher,” Neurocomputing, vol. 504, pp. 189–203, 2022

    work page 2022

  5. [4]

    Comparison of hyperspectral imaging with conventional RGB imaging for quality evaluation of Agaricus bisporus mushrooms,

    M. Taghizadeh, A. A. Gowen, and C. P. O’Donnell, “Comparison of hyperspectral imaging with conventional RGB imaging for quality evaluation of Agaricus bisporus mushrooms,” Biosystems engineering , vol. 108, no. 2, pp. 191–194, 2011

    work page 2011

  6. [5]

    Ro- bust deep learning-based semantic organ segmentation in hyperspectral images,

    S. Seidlitz, J. Sellner, J. Odenthal, B. ¨Ozdemir, A. Studier-Fischer, S. Kn ¨odler, L. Ayala, T. J. Adler, H. G. Kenngott, M. Tizabi et al., “Ro- bust deep learning-based semantic organ segmentation in hyperspectral images,” Medical Image Analysis , p. 102488, 2022

    2022

  7. [6]

    A convolutional neural network classifier identifies tree species in mixed-conifer forest from hyperspectral imagery,

    G. A. Fricker, J. D. Ventura, J. A. Wolf, M. P. North, F. W. Davis, and J. Franklin, “A convolutional neural network classifier identifies tree species in mixed-conifer forest from hyperspectral imagery,” Remote Sensing, vol. 11, no. 19, p. 2326, 2019

    2019

  8. [7]

    HyKo: A Spectral Dataset for Scene Understanding,

    C. Winkens, F. Sattler, V . Adams, and D. Paulus, “HyKo: A Spectral Dataset for Scene Understanding,” in Proceedings of the IEEE Interna- tional Conference on Computer Vision Workshops , 2017, pp. 254–261

    work page 2017

Show all 14 references
  1. [8]

    Hsi Road: A Hyper Spectral Image Dataset For Road Segmentation,

    J. Lu, H. Liu, Y . Yao, S. Tao, Z. Tang, and J. Lu, “Hsi Road: A Hyper Spectral Image Dataset For Road Segmentation,” in 2020 IEEE International Conference on Multimedia and Expo (ICME) , 2020, pp. 1–6

    2020

  2. [9]

    Hyperspectral city v1. 0 dataset and benchmark,

    S. You, E. Huang, S. Liang, Y . Zheng, Y . Li, F. Wang, S. Lin, Q. Shen, X. Cao, D. Zhang et al. , “Hyperspectral city v1. 0 dataset and benchmark,” arXiv preprint arXiv:1907.10270 , 2019

    1907 arXiv

  3. [10]

    HSI-Drive,

    University of the Basque Country UPV/EHU, “HSI-Drive,” 2023. [Online]. Available: https://ipaccess.ehu.eus/HSI-Drive/

    2023

  4. [11]

    Exploring Fully Convolutional Networks for the Segmentation of Hyperspectral Imaging Applied to Advanced Driver Assistance Systems,

    J. Guti ´errez-Zaballa, K. Basterretxea, J. Echanobe, M. V . Mart ´ınez, and I. del Campo, “Exploring Fully Convolutional Networks for the Segmentation of Hyperspectral Imaging Applied to Advanced Driver Assistance Systems,” in Design and Architecture for Signal and Image Proc...

    2022 doi

  5. [13]

    Colour Constancy from Hyper-Spectral Data,

    T. Gevers, H. M. Stokman, and J. van de Weijer, “Colour Constancy from Hyper-Spectral Data,” in BMVC, 2000, pp. 1–10

    2000

  6. [14]

    On the removal of shadows from images,

    G. D. Finlayson, S. D. Hordley, C. Lu, and M. S. Drew, “On the removal of shadows from images,” IEEE transactions on pattern analysis and machine intelligence, vol. 28, no. 1, pp. 59–68, 2005. 5 10 15 20 25 1 1.5 2 2.5 104 Fig. 1. Grayscale images of the most saturated (left) ...

    2005

Pith tools

Reviewed August 12, 2026 · model on record in the stance chip above.