Pith. sign in

REVIEW 4 major objections 5 minor 42 references

Pixel-aligned RGB-NIR Stereo Imaging and Dataset for Robot Vision

T0 review · 4 major / 5 minor · reviewed 2026-08-12 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read Pixel-aligned RGB and near-infrared stereo data cuts depth error by a quarter and lifts detection in dark scenes.

desk verdict A genuinely useful pixel-aligned RGB-NIR stereo + LiDAR dataset for robotics, with reasonable fusion methods; the unmeasured alignment claim and a few reporting choices need attention, but this deserves a serious referee. read the letter →

arxiv 2411.18025 v2 pith:LQASLQGG submitted 2024-11-27 cs.CV

Jinnyeong Kim , Seung-Hwan Baek This is my paper · ORCID
classification cs.CV
keywords RGB-NIRimagingpixel-alignedmultispectralstereodepthestimationimagefusionfeaturerobotvisiondatasetnear-infrared
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper argues that pixel-level alignment between RGB and near-infrared (NIR) stereo images is what lets robot vision systems exploit both spectra, and it builds a capture rig and dataset to demonstrate the claim. The authors introduce two fusion routes: an image-fusion method that encodes RGB and NIR into a single three-channel image usable by any pretrained RGB model without fine-tuning, and a feature-fusion method that fine-tunes a pretrained stereo network on fused RGB-NIR features. On their collected data the feature-fusion stereo estimator reaches 6.747 m depth RMSE versus 8.943 m for RGB-only input, and the learned fused image raises YOLO detection mAP from 0.756 to 0.828. The paper also releases a real-world dataset of 43 scenes and 80,000 RGB-NIR stereo frames with LiDAR ground truth, plus a synthetic RGB-NIR dataset with dense disparity labels.

What carries the argument

The load-bearing hardware is a pair of JAI FS-1600D-10GE cameras: a dichroic prism inside each one reflects NIR onto one CMOS sensor and lets RGB pass to another, so the two spectral images share one viewpoint and one lens. The central computational operator is a learned pair of spatially-varying weights $\alpha$ and $\beta$ that blend the HSV brightness channel with the NIR image before converting back to RGB, producing a three-channel fused image for pretrained models. The feature-level variant adds an attentional feature-fusion module and alternates between fused-feature and NIR correlation volumes inside the GRU updates of RAFT-Stereo, which the paper finds is the most accurate way to use both spectra.

What would settle it

Photograph a high-contrast checkerboard illuminated by both visible light and 850 nm NIR light with the same camera, and measure the per-corner offset between the RGB and NIR captures; if the median offset is even one pixel, the dataset's pixel-aligned premise and the attribution of the depth and detection gains to alignment would need to be revisited.

Watch

Extended reading notes

Core claim

The central discovery is that a single dichroic-prism camera can produce RGB and NIR frames that are pixel-aligned, so no registration or color-transform step is needed before fusion. With that alignment in place, the paper shows two ways to combine the spectra: fuse them at the image level with learned spatially-varying weights, which lets pretrained RGB-only models consume NIR data directly, or fuse them at the feature level inside a recurrent stereo network, which yields the largest accuracy gain. The reported numbers are a stereo depth RMSE of 6.747 m for feature fusion versus 8.943 m for RGB-only RAFT-Stereo, and detection mAP of 0.828 for the fused image versus 0.756 for RGB-only YOLO. These results support the paper's claim that pixel-aligned RGB-NIR data, rather than post-hoc registration, is what improves downstream vision in low-light and high-contrast scenes.

Load-bearing premise

Everything downstream rests on the claim that the JAI dichroic-prism cameras deliver true pixel-level RGB-NIR alignment, but the paper reports no measured registration error between the two spectral images.

Editorial extensions

If this is right

  • Pretrained RGB-only stereo, detection, and structure-from-motion pipelines can exploit NIR information with zero fine-tuning when fed the learned fused image.
  • Fine-tuned feature fusion beats image fusion and all compared cross-spectral baselines, with the largest gains appearing in nighttime scenes.
  • The 80,000-frame real dataset plus LiDAR-derived dense depth gives other methods a directly comparable RGB-NIR stereo training and evaluation target.
  • The alternating fused-NIR correlation volume result indicates that NIR features carry cues RGB features miss, especially under active NIR illumination.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Pith inference (not a paper claim): the image-fusion module contains no geometry, so it may transfer to other prism-based RGB-NIR cameras without retraining; that transferability is a natural follow-up.
  • Pith inference (not a paper claim): if residual RGB-NIR misregistration is actually subpixel, the fusion gains may partly come from the spectral content itself; an artificial-misalignment experiment would separate the two.
  • Pith inference (not a paper claim): the dataset records separate exposure times for RGB and NIR sensors, so a future method could make fusion exposure-aware, which the current fixed-weight formulation does not exploit.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. The paper presents a robotic imaging system with two prism-based RGB-NIR cameras, active NIR illumination, and LiDAR, plus a dataset of synchronized pixel-aligned RGB-NIR stereo images and LiDAR depth for indoor/outdoor day/night scenes. It also proposes two fusion methods: an image-level fusion that lets pretrained RGB models consume RGB-NIR data without finetuning, and a feature-level fusion that extends RAFT-Stereo with attentional RGB-NIR feature fusion and alternating correlation volumes. Experiments report improvements over RGB-only, NIR-only, and several RGB-NIR fusion baselines on stereo depth estimation, object detection, and structure-from-motion.

Significance. If the main claims hold, the dataset and hardware design would be a useful community resource: the paper is among few to provide stereo RGB and NIR pairs with LiDAR ground truth, and the image-fusion method has practical value because it enables off-the-shelf RGB-pretrained models to exploit NIR without retraining. The paper is also stronger than a pure dataset paper because it includes controlled comparisons against single-modality baselines and several established fusion methods, and it states limitations explicitly. However, the central 'pixel-aligned' property is asserted from the camera datasheet rather than measured, and several ablation-based architecture choices are supported by only one metric while other metrics in the supplement point differently. These issues need to be resolved before the specific claims about alignment-driven gains and the proposed fusion designs can be accepted.

major comments (4)
  1. [Section 3 (Calibration)] The 'pixel-aligned' property is load-bearing for the dataset, the title, and the fusion equations (Eq. 3 and Eq. 7), but it is never verified quantitatively. Section 3 states that 'calibration between RGB and NIR images is unnecessary' and reports only stereo-pose and LiDAR calibration; the supplemental calibration section likewise contains no measured RGB-NIR registration error. Please report a measured alignment error (e.g., residual reprojection error of a checkerboard or dot pattern visible in both bands, or the epipolar error between RGB and NIR images) and, ideally, a sensitivity analysis showing that the reported downstream gains persist under small residual misregistration. Without this, the claimed advantage over pixel-misaligned datasets such as CS-Stereo is not established.
  2. [Section 4.2 and Table 3; Supplement Tables 15 and 16] The architecture choice of alternating Fusion-NIR correlation volumes is justified by Table 3, which reports only RMSE. However, Supplemental Table 16 shows that on MAE the 'Fusion-RGB-NIR' variant is better than the selected 'Fusion-NIR' variant (2.5238 m vs. 2.5886 m), and Supplemental Table 15 shows that simple feature multiplication yields better RMSE and MAE than the proposed attentional feature fusion. Because these ablations are used to justify the core feature-fusion design, the paper must report all evaluation metrics (MAE, RMSE, d1-d3) for every ablation and either justify the chosen metric or explain why the alternative is preferred despite worse quantitative results. As written, the claim in the Table 3 caption that the selected variant gives 'the highest depth-estimation accuracy' is not supported by the full set of reported metrics.
  3. [Supplemental Section 4.2.1 and Table 11 / Table 1] The real-world object-detection evaluation relies on pseudo-labels, but the manuscript gives no information about how these pseudo-labels were generated, whether they were manually verified, or what their accuracy is. The detection mAP numbers are also inconsistent between the main paper and the supplement: Table 1(b) reports Ours = 0.809, while Supplemental Table 11 reports Ours = 0.828 for the same task. Please clarify the labeling protocol, provide a verification procedure or error estimate for the pseudo-labels, and reconcile the two mAP values.
  4. [Section 5.2 / Table 2] It is not stated whether the test split was used to select among the correlation-volume variants in Table 3 and among the feature-fusion variants in Supplemental Table 15. If these architecture choices were made using the same test data on which final results are reported, the reported gains are potentially optimistic. Please specify the validation procedure (e.g., a held-out split used for model selection, or a statement that the test set was used only once) and, if no validation set was used, discuss the risk of selection on the test set.
minor comments (5)
  1. [Section 1, first paragraph] There is a typo: 'where where RGB imaging is inadequate' should have only one 'where'.
  2. [Section 3 (Calibration)] The main text says temporal synchronization is maintained to 'less than 1 microsecond', while the supplemental Section 1.1.4 says the cameras were aligned 'within 100 microseconds' and packets with differences exceeding 1 ms were discarded. Please reconcile these numbers.
  3. [Section 4.2] The sentence 'inspired by the cross-spectral time-of-flight imaging method [5]' is unexplained; please state what design element is borrowed from that method.
  4. [Table 2] The caption groups 'different RGB-NIR stereo depth estimation model [64]' with methods that were fine-tuned on the proposed dataset, but the text says only methods [16,51] were retrained. Please clarify whether CS-Stereo [64] was fine-tuned or used with its original checkpoint, since this affects fairness of the comparison.
  5. [Supplemental Section 2.1.2] The dense disparity labels are produced by a BPNet-based reconstruction with an occlusion threshold; the paper would benefit from a short validation of this pseudo-ground-truth against raw LiDAR, e.g., the median error of the densified depth at held-out LiDAR points.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the paper is an empirical dataset and fusion study whose headline results are evaluated on held-out real test videos against external baselines, with no load-bearing self-citation and no prediction that reduces to a fitted input by construction.

full rationale

This is an empirical systems/dataset paper rather than a derivation, so the circularity burden is modest and is not met. The central claims—that pixel-aligned RGB-NIR fusion improves depth estimation, object detection, and structure-from-motion—are supported by evaluations on a held-out test split (4 of 43 videos, 7000 frames) against external baselines including RGB-only and NIR-only inputs, CS-Stereo, CSPD, DPSNet, and several prior fusion methods. The learned fusion weights alpha and beta in Section 4.1 are trained on synthetic and real training data with photometric and disparity losses and then evaluated on test frames; the feature-fusion network in Section 4.2 is fine-tuned on training data and tested on the held-out split. No equation in the paper defines a predicted quantity in terms of the same measured quantity used to fit it. The choice of alternating Fusion-NIR correlation volumes was made by comparing test-set RMSE values (Table 3), which is a test-set selection concern and a validity risk, but it is not a circular reduction: the reported 6.747 m is the honest test result for that configuration, not a quantity forced by construction. Similarly, the unverified premise that the JAI FS-1600D-10GE dichroic-prism camera provides pixel-level RGB-NIR alignment with no residual error is an untested hardware assumption, not a circular argument. There are no load-bearing self-citations: the cited pretrained networks and fusion modules (RAFT-Stereo, attentional feature fusion, BPNet, Gated2Depth) are external, code-reproduced components, and the paper does not invoke any uniqueness theorem or prior work by the same authors to forbid alternatives. Overall, the empirical comparisons are self-contained against external benchmarks, so the appropriate circularity score is 0.

Assumptions & free parameters 5 free parameters · 5 assumptions · 0 invented entities

The main claim rests on hardware (prism alignment), a sensor model, synthetic NIR simulation, densified LiDAR ground truth, and pseudo-label quality. None of these are verified with explicit measurements, and several involve hand-chosen thresholds. This is typical for a systems and dataset paper, but it means the reader is asked to accept a chain of domain assumptions.

free parameters (5)
  • Image fusion weights alpha and beta = learned spatially-varying maps
    Section 4.1: the fused image is alpha*I_V + beta*I_NIR, with alpha and beta predicted by a decoder trained on photometric and disparity losses; the claimed improvements depend on these fitted maps.
  • Synthetic lighting parameters phi_j and phi_active = randomized in simulation
    Supplement Section 2.2.1: ambient light brightness phi_j and active NIR brightness phi_active are chosen to render synthetic RGB-NIR images; the realism and transfer of the synthetic dataset depend on these choices.
  • LiDAR densification thresholds alpha and beta = 0.75 and 0.85
    Supplement Algorithm 1: distance threshold alpha=0.75 and disparity ratio beta=0.85 prune LiDAR points before BPNet densification, shaping the ground-truth depth maps used for training.
  • Occlusion mask threshold = not specified
    Supplement Equation (8): the left-right disparity discrepancy threshold for the occlusion mask is described but no numeric value is given, affecting the dense GT masks.
  • Photometric and SSIM loss weights = 0.85 and 0.15
    Supplement Section 3.1.2: photometric loss weights gamma_L1=0.85 and gamma_ssim=0.15; the fusion training objective depends on these values.
assumptions (5)
  • domain assumption The JAI FS-1600D-10GE dichroic prism camera provides true pixel-level alignment between the RGB and NIR sensors with no residual misregistration.
    Section 3 'Calibration: One advantage of using pixel-aligned cameras is that calibration between RGB and NIR images is unnecessary.' This is load-bearing: without it, the pixel-aligned claim and the end-to-end fusion results lose their basis. No alignment-error measurement is reported.
  • domain assumption The RGB-to-NIR color synthesis of Gated2depth [15] produces NIR reflectance maps realistic enough for synthetic training to transfer to real scenes.
    Supplement Section 2.2.1: 'NIR albedo is pseudo driven from RGB albedo by [12]'; the synthetic dataset and one-epoch pretraining rely on this learned mapping.
  • domain assumption BPNet-densified LiDAR disparity maps are sufficiently accurate to serve as ground truth for training and evaluation.
    Supplement Section 2.1.2: dense depth is reconstructed from sparse LiDAR via BPNet [36]; if BPNet errors correlate with lighting, the training labels are biased.
  • ad hoc to paper Pseudo-labels created for real object-detection evaluation are accurate and unbiased.
    Supplement Section 4.2.1: 'For the real dataset, we prepared the data by directly creating pseudo labels to serve as ground-truth annotations.' The procedure is not described, making mAP numbers hard to interpret.
  • domain assumption Camera-to-LiDAR synchronization is adequate for moving scenes even though only the two cameras are formally synchronized.
    Supplement Section 1.1.4: camera pair sync is quantified as under 100 microseconds, but LiDAR-to-camera sync is not directly validated, yet LiDAR points are used as depth ground truth.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Pixel-aligned RGB-NIR Stereo Imaging and Dataset for Robot Vision." pith.science (2026). https://pith.science/paper/LQASLQGG

@misc{pith2026241118025,
  author       = {Pith},
  title        = {Pith review of: Pixel-aligned RGB-NIR Stereo Imaging and Dataset for Robot Vision},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/LQASLQGG}},
  note         = {Machine review of arXiv:2411.18025}
}
read the original abstract

Integrating RGB and NIR stereo imaging provides complementary spectral information, potentially enhancing robotic 3D vision in challenging lighting conditions. However, existing datasets and imaging systems lack pixel-level alignment between RGB and NIR images, posing challenges for downstream vision tasks. In this paper, we introduce a robotic vision system equipped with pixel-aligned RGB-NIR stereo cameras and a LiDAR sensor mounted on a mobile robot. The system simultaneously captures pixel-aligned pairs of RGB stereo images, NIR stereo images, and temporally synchronized LiDAR points. Utilizing the mobility of the robot, we present a dataset containing continuous video frames under diverse lighting conditions. We then introduce two methods that utilize the pixel-aligned RGB-NIR images: an RGB-NIR image fusion method and a feature fusion method. The first approach enables existing RGB-pretrained vision models to directly utilize RGB-NIR information without fine-tuning. The second approach fine-tunes existing vision models to more effectively utilize RGB-NIR information. Experimental results demonstrate the effectiveness of using pixel-aligned RGB-NIR images across diverse lighting conditions.

Figures

Figures reproduced from arXiv: 2411.18025 by the authors.

Figure 1
Figure 1. RGB-NIR stereo imaging system. (a) Stereo camera setup integrated with LiDAR and active NIR illumination. (b) Spectral sensitivity profiles of the camera sensors and the irradiance profile of the NIR illumination. (c) Diagram of the pixel-aligned RGB-NIR camera featuring a dichroism prism, where RGB light penetrates the prism and NIR light is reflected, achieving spectral separation. installed a LiDAR (OUSTER OS1) t… view at source ↗
Figure 2
Figure 2. RGB-NIR stereo dataset. (a) Our dataset comprises stereo RGB image pairs, stereo NIR image pairs, and sparse depth point clouds, all captured from continuous video sequences. (b) A comparison of our dataset with other RGB-NIR image datasets, specifically curated for 3D vision tasks. (c) A quantitative analysis of our dataset is presented, illustrating the distribution of frames under varying lighting conditions, a h… view at source ↗
Figure 3
Figure 3. RGB-NIR image fusion. We fuse the RGB image IRGB and the image INIR as a weighted sum in the brightness domain V after converting RGB image into HSV channel. The spatially-varying weights α, β are learned to effectively fuse RGB and NIR images. The fused image can be used as inputs to vision models such as object detection, stereo depth estimation, and structure from motion. where M is the RGB-to-HSV conversion matr… view at source ↗
Figures from the paper (19 more)
Figure 4
Figure 4. Figure 4: RGB-NIR stereo depth estimation model. We modi￾fied RAFT-Stereo [36] with attentional feature fusion and alterna￾tive correlation search for RGB-NIR depth estimation. We extract features from RGB and NIR images, fuse them, and build cost vol￾umes. We estimate disparity…
Figure 5
Figure 5. Figure 5: Image fusion for pretrained RAFT-stereo [36]. (a)&(b) Using a single modality either RGB or NIR images often results in sub-optimal disparity estimation in challenging lighting conditions. (c) Our RGB-NIR fused image enables robust disparity estimation without finetuni…
Figure 6
Figure 6. Figure 6: Image fusion for pretrained YOLO [45]. (a) Us￾ing the RGB images struggle with object detection in scenes with low light conditions or high saturation due to the under- and over￾exposed regions by the limited dynamic range of the cameras. (b) NIR images suffer from cha…
Figure 8
Figure 8. Figure 8: RGB-NIR feature-based depth estimation. (a)&(b) Using single-modality RGB/NIR images for stereo depth estimation [36] often fails to obtain high-quality depth. (c) Our RGB-NIR image-fusion method (Section 4.1) mitigates the issue without finetuning the RAFT stereo, alt…
Figure 1
Figure 1. Figure 1: Sparse Depth Reconstruction. (a) RGB image and (b) NIR image are used as guidance for reconstruction. LiDAR points are projected onto the left image plane (c) and the right image plane (d). The points are then densified following [36] in (e) and (f). Left and right con…
Figure 2
Figure 2. Figure 2: Various real dataset samples RGB stereo datasets [27, 31] and developed a synthetic rendering pipeline leveraging an image formation model to generate realistic RGB images based on provided depth maps and material segmentation maps [PITH_FULL_IMAGE:figures/full_fig_p0…
Figure 3
Figure 3. Figure 3: Various real dataset samples on challenging lighting condition. [PITH_FULL_IMAGE:figures/full_fig_p022_3.png]
Figure 4
Figure 4. Figure 4: Statistics of our real dataset. (a) Rendered RGB image (b) Disparity map (c) Material segmentation (d) RGB diffuse albedo (e) Normal map (f) NIR diffuse albedo (g) Augmented RGB image (h) Augmented NIR image [PITH_FULL_IMAGE:figures/full_fig_p023_4.png]
Figure 5
Figure 5. Figure 5: RGB-NIR synthetic data augmentation. Sceneflow dataset [27] provide (a) rendered RGB image, (b) disparity map and (c) material index. We assumed RGB albedo (d) with (a) & (b), normal map (e) from (b). We simulated NIR albedo (f) from (d) by [12]. We rendered some light…
Figure 6
Figure 6. Figure 6: Samples of augmented synthetic dataset. Sceneflow [27] includes both ourdoor environment (a) and indoor environment (b) dataset. We augmented these datasets relighting RGB stereo images and NIR stereo images. Dataset Pixel -aligned Multi -View RGB Stereo NIR Stereo GT …
Figure 7
Figure 7. Figure 7: Attention-enhanced feature fusion. (a) The structure of attention based feature fusion module, proposed by [9]. MS-CAM is a attention enhancer module. The fusion operation is served by weight sum of attention-enhanced features with w. (b,c) Examples of input RGB and NI…
Figure 8
Figure 8. Figure 8: and 9 illustrate different image fusion methods applied to our real RGB-NIR dataset. We conducted comparative studies on downstream vision applications using our image fusion model alongside other color fusion techniques. (a) RGB (b) NIR (c) RGB Channel sum (d) YCrCb f…
Figure 9
Figure 9. Figure 9: Image fusion visualization of ours and comparison methods, indoor sample. (a,b) Pixel-aligned RGB and NIR images. (c) Image driven by simple channel sum. (d) YCrCb channel fusion [15]. (e) Bayesian estimation based fusion [40]. (f) Gradient adaptive fusion [1]. (g) Dar…
Figure 10
Figure 10. Figure 10: COLMAP reconstruction examples for RGB, NIR and our fused images. (a) RGB images. (b) NIR images. (c) Our fused images. For comparisons using RAFT-Stereo, we tested various input images while maintaining consistent weights in the RAFT￾Stereo model to ensure fair pairw…
Figure 11
Figure 11. Figure 11: Depth estimation samples of our feature fusion model. (a) RGB images and outputs of [25] with them. (b) NIR images and outputs with them. (c) Fused images by our image fusion method and outputs with them. (d) Stereo depth estimation with our feature fusion based metho…
Figure 12
Figure 12. Figure 12: Depth estimation samples on challenging lighting conditions of our feature fusion model. (a) RGB images and outputs of [25] with them. (b) NIR images and outputs with them. (c) Fused images by our image fusion method and outputs with them. (d) Stereo depth estimation …
Figure 13
Figure 13. Figure 13: Ablation study on pretrained encoder. (a) Pretrained with [33]. (b) Pretrained with [27]. (c) Pretrained with [31]. 29 [PITH_FULL_IMAGE:figures/full_fig_p040_13.png]
Figure 14
Figure 14. Figure 14: Ablation study on feature fusion method. (a) Feature addition. (b) Feature concatenation. (c) Feature multiplication. (d) Feature weight summation. (e) Attentional feature fusion [9] Pretrained MAE (m) RMSE (m) δ1 δ2 δ3 Feature addition 2.611 6.5365 0.6526 0.8772 0.95…
Figure 15
Figure 15. Figure 15: Ablation study on alternative correlation volume searching. (a) Fusion correlation volume only. (b) RGB and NIR correla￾tion volumes. (c) Fusion, RGB and NIR correlation volumes. (d) Fusion and NIR correlation volumes. Correlation volumes for disparity estimation MAE …

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

42 extracted references · 41 canonical work pages

  1. [1]

    Adaptive near-infrared and visible fusion for fast image enhancement

    Mohamed Awad, Ahmed Elliethy, and Hussein A Aly. Adaptive near-infrared and visible fusion for fast image enhancement. IEEE Transactions on Computational Imaging, 6:408–418, 2019. 21, 22, 23, 24, 25, 27, 28

    work page 2019

  2. [2]

    Camera calibration toolbox for matlab

    Jean-Yves Bouguet. Camera calibration toolbox for matlab. http://www. vision. caltech. edu/bouguetj/calib doc/, 2004. 4

    work page 2004

  3. [3]

    Agricultural robot dataset for plant classification, localization and mapping on sugar beet fields

    Nived Chebrolu, Philipp Lottes, Alexander Schaefer, Wera Winterhalter, Wolfram Burgard, and Cyrill Stachniss. Agricultural robot dataset for plant classification, localization and mapping on sugar beet fields. The International Journal of Robotics Research , 36(10):1045–1052, 2017. 13, 14, 15

    work page 2017

  4. [4]

    Ranus: Rgb and nir urban scene dataset for deep scene parsing

    Gyeongmin Choe, Seong-Heum Kim, Sunghoon Im, Joon-Young Lee, Srinivasa G Narasimhan, and In So Kweon. Ranus: Rgb and nir urban scene dataset for deep scene parsing. IEEE Robotics and Automation Letters, 3(3):1808–1815, 2018. 13, 14

    work page 2018

  5. [5]

    Kaist multi-spectral day/night data set for autonomous and assisted driving

    Yukyung Choi, Namil Kim, Soonmin Hwang, Kibaek Park, Jae Shin Yoon, Kyounghwan An, and In So Kweon. Kaist multi-spectral day/night data set for autonomous and assisted driving. IEEE Transactions on Intelligent Transportation Systems , 19(3):934–948, Mar. 2018. 13, 14, 15

    work page 2018

  6. [6]

    The cityscapes dataset for semantic urban scene understanding

    Marius Cordts, Mohamed Omran, Sebastian Ramos, Timo Rehfeld, Markus Enzweiler, Rodrigo Benenson, Uwe Franke, Stefan Roth, and Bernt Schiele. The cityscapes dataset for semantic urban scene understanding. In 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), page 3213–3223, Las Vegas, NV , USA, June 2016. IEEE. 13, 14

    work page 2016

  7. [7]

    Indoor semantic segmentation using depth information

    Camille Couprie, Cl ´ement Farabet, Laurent Najman, and Yann Lecun. Indoor semantic segmentation using depth information. In First International Conference on Learning Representations (ICLR 2013), pages 1–8, 2013. 13, 14, 15

    work page 2013

  8. [8]

    Scannet: Richly-annotated 3d reconstructions of indoor scenes

    Angela Dai, Angel X Chang, Manolis Savva, Maciej Halber, Thomas Funkhouser, and Matthias Nießner. Scannet: Richly-annotated 3d reconstructions of indoor scenes. In Proceedings of the IEEE conference on computer vision and pattern recognition , pages 31 5828–5839, 2017. 13, 14, 15

    work page 2017

Show all 42 references
  1. [9]

    Attentional feature fusion

    Yimian Dai, Fabian Gieseke, Stefan Oehmcke, Yiquan Wu, and Kobus Barnard. Attentional feature fusion. In 2021 IEEE Winter Conference on Applications of Computer Vision (WACV), page 3559–3568, Waikoloa, HI, USA, Jan. 2021. IEEE. 16, 17, 19, 21, 30

    2021

  2. [10]

    Colouring the near-infrared

    Cl ´ement Fredembach and Sabine S ¨usstrunk. Colouring the near-infrared. In Color and imaging conference , volume 16, pages 176–182. Society of Imaging Science and Technology, 2008. 21, 22, 23, 24, 25, 27, 28

    2008

  3. [11]

    Geiger, P

    A. Geiger, P. Lenz, and R. Urtasun. Are we ready for autonomous driving? the kitti vision benchmark suite. In2012 IEEE Conference on Computer Vision and Pattern Recognition, page 3354–3361, Providence, RI, June 2012. IEEE. 13, 14, 15

    2012

  4. [12]

    Gated2depth: Real-time dense lidar from gated images

    Tobias Gruber, Frank Julca-Aguilar, Mario Bijelic, and Felix Heide. Gated2depth: Real-time dense lidar from gated images. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 1506–1516, 2019. 10, 12

    2019

  5. [13]

    Unsupervised cross-spectrum depth estimation by visible-light and thermal cameras

    Yubin Guo, Xinlei Qi, Jin Xie, Cheng-Zhong Xu, and Hui Kong. Unsupervised cross-spectrum depth estimation by visible-light and thermal cameras. IEEE Transactions on Intelligent Transportation Systems, 24(10):10937–10947, Oct. 2023. 13, 14, 15, 25, 26, 27, 28

    2023

  6. [14]

    Deep residual learning for image recognition

    Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 770–778, 2016. 15

    2016

  7. [15]

    Color outdoor image enhancement by v-nir fusion and weighted luminance

    Mar ´ıa A Herrera-Arellano, Hayde Peregrina-Barreto, and Iv ´an Terol-Villalobos. Color outdoor image enhancement by v-nir fusion and weighted luminance. In 2019 IEEE International Autumn Meeting on Power, Electronics and Computing (ROPEC), pages 1–6. IEEE, 2019. 21, 22, 24, 2...

    2019

  8. [16]

    Surface reconstruction from unorganized points

    Hugues Hoppe, Tony DeRose, Tom Duchamp, John McDonald, and Werner Stuetzle. Surface reconstruction from unorganized points. In Proceedings of the 19th annual conference on computer graphics and interactive techniques, pages 71–78, 1992. 12

    1992

  9. [17]

    Darkvisionnet: Low-light imaging via rgb-nir fusion with deep inconsistency prior

    Shuangping Jin, Bingbing Yu, Minhao Jing, Yi Zhou, Jiajun Liang, and Renhe Ji. Darkvisionnet: Low-light imaging via rgb-nir fusion with deep inconsistency prior. InProceedings of the AAAI Conference on Artificial Intelligence, volume 36, pages 1104–1112,

  10. [18]

    CostDCNet: Cost Volume Based Depth Completion for a Single RGB-D Image, volume 13662 of Lecture Notes in Computer Science, page 257–274

    Jaewon Kam, Jungeon Kim, Soongjin Kim, Jaesik Park, and Seungyong Lee. CostDCNet: Cost Volume Based Depth Completion for a Single RGB-D Image, volume 13662 of Lecture Notes in Computer Science, page 257–274. Springer Nature Switzerland, Cham,

  11. [19]

    3d gaussian splatting for real-time radiance field rendering

    Bernhard Kerbl, Georgios Kopanas, Thomas Leimk ¨uhler, and George Drettakis. 3d gaussian splatting for real-time radiance field rendering. ACM Trans. Graph., 42(4):1–14, 2023. 9

    2023

  12. [20]

    Vivid++: Vision for visibility dataset

    Alex Junho Lee, Younggun Cho, Young-sik Shin, Ayoung Kim, and Hyun Myung. Vivid++: Vision for visibility dataset. IEEE Robotics and Automation Letters, 7(3):6282–6289, July 2022. 13, 14

    2022

  13. [21]

    Infrared and visible image fusion using a deep learning framework

    Hui Li, Xiao-Jun Wu, and Josef Kittler. Infrared and visible image fusion using a deep learning framework. In2018 24th international conference on pattern recognition (ICPR), pages 2705–2710. IEEE, 2018. 21, 22, 24, 25, 27, 28

    2018

  14. [22]

    Practical stereo matching via cascaded recurrent network with adaptive correlation

    Jiankun Li, Peisen Wang, Pengfei Xiong, Tao Cai, Ziwei Yan, Lei Yang, Jiangyu Liu, Haoqiang Fan, and Shuaicheng Liu. Practical stereo matching via cascaded recurrent network with adaptive correlation. In 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR...

    2022

  15. [23]

    Saputra, Zhuangzhuang Dai, and Chris Xiaoxuan Lu

    Peize Li, Kaiwen Cai, Muhamad Risqi U. Saputra, Zhuangzhuang Dai, and Chris Xiaoxuan Lu. Odombeyondvision: An indoor multi-modal multi-platform odometry dataset beyond the visible spectrum. In2022 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), page...

    2022

  16. [24]

    Image fusion with guided filtering

    Shutao Li, Xudong Kang, and Jianwen Hu. Image fusion with guided filtering. IEEE Transactions on Image Processing , 22(7):2864–2875, July 2013. 18

    2013

  17. [25]

    Raft-stereo: Multilevel recurrent field transforms for stereo matching

    Lahav Lipson, Zachary Teed, and Jia Deng. Raft-stereo: Multilevel recurrent field transforms for stereo matching. In 2021 Interna- tional Conference on 3D Vision (3DV), pages 218–227. IEEE, 2021. 7, 15, 17, 18, 20, 21, 25, 27, 28, 29

    2021

  18. [26]

    Multispectral deep neural networks for pedestrian detection

    Jingjing Liu, Shaoting Zhang, Shu Wang, and Dimitris N Metaxas. Multispectral deep neural networks for pedestrian detection. In 27th British Machine Vision Conference, BMVC 2016, 2016. 13, 14, 15

    2016

  19. [27]

    A large dataset to train convolutional networks for disparity, optical flow, and scene flow estimation

    Nikolaus Mayer, Eddy Ilg, Philip Hausser, Philipp Fischer, Daniel Cremers, Alexey Dosovitskiy, and Thomas Brox. A large dataset to train convolutional networks for disparity, optical flow, and scene flow estimation. In Proceedings of the IEEE conference on computer vision and ...

    2016

  20. [28]

    The goose dataset for perception in unstructured environments

    Peter Mortimer, Raphael Hagmanns, Miguel Granero, Thorsten Luettel, Janko Petereit, and Hans-Joachim Wuensche. The goose dataset for perception in unstructured environments. 2024. 13, 14, 15

    2024

  21. [29]

    Linear n-point camera pose determination

    Long Quan and Zhongdan Lan. Linear n-point camera pose determination. IEEE Transactions on pattern analysis and machine intelligence, 21(8):774–780, 1999. 5

    1999

  22. [30]

    You only look once: Unified, real-time object detection

    Joseph Redmon, Santosh Divvala, Ross Girshick, and Ali Farhadi. You only look once: Unified, real-time object detection. In 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pages 779–788, 2016. 24

    2016

  23. [31]

    High-resolution stereo datasets with subpixel-accurate ground truth

    Daniel Scharstein, Heiko Hirschm ¨uller, York Kitajima, Greg Krathwohl, Nera Neˇsi´c, Xi Wang, and Porter Westling. High-resolution stereo datasets with subpixel-accurate ground truth. In Pattern Recognition: 36th German Conference, GCPR 2014, M ¨unster, Ger- many, September 2...

    2014

  24. [32]

    Structure-from-motion revisited

    Johannes L Schonberger and Jan-Michael Frahm. Structure-from-motion revisited. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 4104–4113, 2016. 9

    2016

  25. [33]

    Bad slam: Bundle adjusted direct rgb-d slam

    Thomas Schops, Torsten Sattler, and Marc Pollefeys. Bad slam: Bundle adjusted direct rgb-d slam. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 134–144, 2019. 27, 28, 29

    2019

  26. [34]

    Deep depth estimation from thermal image

    Ukcheol Shin, Jinsun Park, and In So Kweon. Deep depth estimation from thermal image. In 2023 IEEE/CVF Conference on 32 Computer Vision and Pattern Recognition (CVPR), page 1043–1053, Vancouver, BC, Canada, June 2023. IEEE. 13, 14, 15

    2023

  27. [35]

    Multispectral object detection for autonomous vehicles

    Karasawa Takumi, Kohei Watanabe, Qishen Ha, Antonio Tejero-De-Pablos, Yoshitaka Ushiku, and Tatsuya Harada. Multispectral object detection for autonomous vehicles. In Proceedings of the on Thematic Workshops of ACM Multimedia 2017 , pages 35–43,

    2017

  28. [36]

    Bilateral propagation network for depth completion

    Jie Tang, Fei-Peng Tian, Boshi An, Jian Li, and Ping Tan. Bilateral propagation network for depth completion. In 2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), page 9763–9772, Seattle, W A, USA, June 2024. IEEE. 6, 7, 8, 25

    2024

  29. [37]

    Dps-net: Deep polarimetric stereo depth estimation

    Chaoran Tian, Weihong Pan, Zimo Wang, Mao Mao, Guofeng Zhang, Hujun Bao, Ping Tan, and Zhaopeng Cui. Dps-net: Deep polarimetric stereo depth estimation. In Proceedings of the IEEE/CVF International Conference on Computer Vision , pages 3569– 3579, 2023. 25, 26, 27, 28

    2023

  30. [38]

    The triclobs dynamic multi-band image data set for the development and evaluation of image fusion methods

    Alexander Toet, Maarten A Hogervorst, and Alan R Pinkus. The triclobs dynamic multi-band image data set for the development and evaluation of image fusion methods. PloS one, 11(12):e0165016, 2016. 13, 14, 15

    2016

  31. [39]

    Deep multispectral semantic scene understanding of forested environments using multimodal fusion

    Abhinav Valada, Gabriel L Oliveira, Thomas Brox, and Wolfram Burgard. Deep multispectral semantic scene understanding of forested environments using multimodal fusion. In 2016 International Symposium on Experimental Robotics , pages 465–477. Springer, 2017. 13, 14, 15

    2016

  32. [40]

    Bayesian fusion for infrared and visible images

    Zixiang Zhao, Shuang Xu, Chunxia Zhang, Junmin Liu, and Jiangshe Zhang. Bayesian fusion for infrared and visible images. Signal Processing, 177:107734, 2020. 21, 22, 24, 25, 27, 28

    2020

  33. [41]

    Deep material-aware cross-spectral stereo matching

    Tiancheng Zhi, Bernardo R Pires, Martial Hebert, and Srinivasa G Narasimhan. Deep material-aware cross-spectral stereo matching. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 1916–1925, 2018. 13, 14, 25, 26, 27, 28 33

    1916

  34. [2022]

    21, 22, 24, 25, 27, 28

Pith tools

Reviewed August 12, 2026 · model on record in the stance chip above.