Pith. sign in

REVIEW 3 major objections 5 minor 2 cited by

Filterless Snapshot Hyperspectral Imaging using Guided Patch Diffusion

T0 review · 3 major / 5 minor · reviewed 2026-08-11 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read Hyperspectral images can be reconstructed from the chromatic aberration in a single grayscale snapshot through a flat-optic lens, using patch-based diffusion with point-spread-function guidance.

desk verdict Strong simulation-first paper with a real gap: it claims experimental validation but reports none, and the method is closed under the same forward model it uses for guidance. read the letter →

arxiv 2412.02798 v3 pith:IWY5FTX5 submitted 2024-12-03 cs.CV eess.IVphysics.optics

Dean Hazineh , Luca Sacchi , Davide Cassara , Federico Capasso , Todd Zickler This is my paper · ORCID
classification cs.CVeess.IVphysics.optics
keywords hyperspectralimagingsnapshotpatch-baseddiffusionconditionalmodelchromaticaberrationflatopticsmetalensinverseproblems
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper claims that a single grayscale image taken through a flat diffractive lens, with no color filters or dispersive prisms, carries enough information to reconstruct a 31-channel hyperspectral image. The key physical idea is engineered chromatic aberration: because each wavelength creates its own blur and shift in the point-spread function, the measured grayscale pattern encodes the scene's spectrum. The reconstruction is produced by a conditional denoising diffusion model trained on small patches and synchronized during inference by an optical-consistency guidance step that forces patch predictions to agree with the full measurement. The paper's experiments, all in simulation, show this beats prior full-field models on the ARAD1K benchmark and that the same trained model generalizes to larger images from other datasets without retraining. If the claim holds, compact snapshot hyperspectral cameras could be built from a single flat optic and an ordinary filterless sensor.

What carries the argument

The load-bearing object is the measurement operator $\mathcal{M}$ of Eq. (1): a shift-invariant, wavelength-dependent point-spread function $f(u,v,\lambda)$ convolved with each spectral channel and summed with sensor response $o(\lambda)$ to produce the grayscale frame. Around it, the method builds a conditional denoising diffusion model that denoises 31-channel image patches given corresponding grayscale measurement patches, then a guided-sampling loop that stitches full-field predictions and iteratively minimizes $\mathcal{L} = \|\mathcal{M}(\mathrm{Stitch}(c_{\mathrm{lsq}} \cdot \hat{x}_0)) - y\|^2$, with per-patch scale factors $c_{\mathrm{lsq}}$ recovered by least squares. This guidance step is what resolves the ambiguity of patch-based processing when the PSF support reaches the patch size; removing it drops PSNR from roughly 34.6 dB to 32.2 dB.

What would settle it

Fabricate a T4 or R1 metalens, measure its PSF at each wavelength on a bench, then image a scene with known hyperspectral ground truth on a filterless sensor; if the per-pixel spectral angle of the reconstruction is substantially worse than the simulated value of roughly 0.11 or the uncertainty-error correlation drops far below the reported 0.80, the central claim that chromatic aberration alone suffices is falsified.

Watch

Extended reading notes

Core claim

On the paper's own terms, the central claim is that the inverse problem $y = \mathcal{M}(x)$, formed by convolving each of 31 wavelength channels of the scene with the lens's wavelength-dependent point-spread function and summing with the sensor response, is solvable from a single grayscale frame: the chromatic aberration itself is the spectral code. The solution is a conditional denoising diffusion model trained on small, shift-invariant patches, with inference synchronized by repeatedly rendering the stitched patch predictions through the same optical operator and nudging them toward agreement with the measured image. The paper reports that this guided sampling lifts PSNR from 32.32 dB without guidance to 34.63 dB with guidance on the ARAD1K test set, that patch sizes can shrink to the support of the PSF, and that multiple stochastic draws yield uncertainty maps with a 0.80 Pearson correlation to true error. The evidence reported in the body is entirely simulated; the abstract's phrase "in simulation and experiment" is not matched by a hardware experiment in the text.

Load-bearing premise

The load-bearing premise is that the wavelength-dependent point-spread function and sensor response used in the guidance model exactly match the real physical lens and sensor, and the paper provides no hardware experiment to test that match.

Editorial extensions

If this is right

  • A snapshot hyperspectral imager can in principle use one flat optic and a filterless sensor with no pixel-count penalty, avoiding color filter arrays and multi-element relay optics.
  • Because the diffusion prior is patch-based, a single trained model handles arbitrary sensor resolutions; the paper demonstrates reconstructions at 512x512, 1024x1344, and 1280x1280 without finetuning.
  • Lens-design choice matters: the T4 and R1 point-spread functions, which balance spectral mixing against spatial blur, give the best grayscale reconstructions, and the best encoder for this task differs from established RGB-oriented designs.
  • Drawing multiple samples yields per-pixel uncertainty that flags unreliable spectra: the reported correlation with true error is 0.80 over 12.5K sampled pixels.
  • Training the model on noise-matched measurements extends reliable reconstruction to SNR of about 20 dB and above, which the paper argues is attainable in practice.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Editorial extension: the same patch-diffusion-plus-guidance recipe could apply to any shift-invariant linear encoder whose kernel support exceeds the patch size, such as coded apertures or scattering layers, not only chromatic lenses.
  • Editorial extension: a hardware experiment is the natural next test the paper leaves open; because the guidance loss assumes a known, shift-invariant PSF, a practical device would likely need per-lens calibration, and the simulation-to-hardware gap is unmeasured.
  • Editorial extension: the uncertainty-error correlation suggests an adaptive imaging loop in which high-uncertainty pixels receive more guidance iterations or a second exposure; the paper does not develop this.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. The paper proposes a patch-based conditional denoising diffusion model for reconstructing a 31-channel hyperspectral image (HSI) from a single grayscale snapshot acquired through a diffractive flat-optic lens. Training uses HSI–measurement patch pairs rendered with a known shift-invariant, wavelength-dependent point-spread function, and inference stitches independently denoised patches and applies global guidance by enforcing consistency with the full-field measurement. The authors report strong simulation results on ARAD1K, including improvements over eight baselines, a lens-design study, ablations, cross-dataset generalization, noise robustness, and per-pixel uncertainty estimates with a reported Pearson correlation of 0.80 with reconstruction error. The abstract claims validation "in simulation and experiment," but the full text and supplement contain only simulation results; the conclusion also states an unqualified "first demonstration" of reconstruction from a flat-optic capture.

Significance. If the simulation results transfer to hardware, the contribution would be significant: a single filterless optic and photosensor with the same pixel count as the output HSI, combined with a diffusion prior and optical guidance, would enable compact, light-efficient snapshot hyperspectral imaging with per-pixel uncertainty. The simulation study itself is thorough and thoughtfully designed: the authors compare against eight baselines retrained on the same rendered measurements, ablate the guidance and patch-size choices, study eight PSF designs, demonstrate generalization to three external datasets at varying resolutions, and test robustness to measurement noise. The patch-based diffusion formulation with global PSF guidance is a sensible and well-motivated contribution, and the uncertainty estimation is a useful byproduct. However, the central hardware claim is not established by the reported evidence, and the paper's significance as stated depends on that claim. The reported results support a conditional statement about reconstruction under an exactly known simulated forward model, not an unconditional hardware demonstration.

major comments (3)
  1. [Abstract and §4, Conclusion] The abstract states the method produces "high-quality results in simulation and experiment," but §4 opens with "We extensively evaluate our method in simulation" and neither the main text nor the supplement contains a hardware measurement. The Conclusion repeats an unqualified "first demonstration that hyperspectral images can be reconstructed solely from the chromatic aberration in a single grayscale measurement, captured through a flat-optic lens." This is load-bearing because the paper's headline contribution is about a physical capture. The authors must either add a hardware experiment or explicitly reframe all claims as simulation-only; deleting "and experiment" alone would not resolve the overclaim in the Conclusion.
  2. [§3.1 and §3.4, Eqs. (1) and (5)] All training and testing use the exact same known, shift-invariant measurement operator M for both rendering and guidance, including the cross-dataset experiments in §4.5. The method therefore assumes that the simulated PSF f(u,v,λ) and sensor response o(λ) match a real lens and sensor exactly. A physical metalens will exhibit field-dependent PSFs, fabrication and alignment errors, and an approximately known sensor response; under model mismatch the guidance loss in Eq. (5) would enforce consistency with an incorrect M. The paper provides no calibration or self-calibration procedure and no mismatch analysis. The authors should include experiments with perturbed or spatially varying PSFs, sensor-response errors, or a hardware validation; without one of these, the claim that a real flat-optic capture suffices is unsupported.
  3. [Table 1 and Supplement Section 11] The entries labeled "Bayer" in Table 1 are not single-snapshot Bayer measurements. Supplement Section 11 states that the RGB measurements "effectively assume three sequential captures, each using a uniform spectral filter" and that the main-paper results "do not account for spatial demosaicing." A sequential three-channel capture is substantially better conditioned than a single Bayer-filtered snapshot, so the Bayer+Optic comparison in Table 1 overstates the method's advantage in the claimed single-snapshot scenario. The authors should simulate a true Bayer mosaic (including demosaicing) or relabel and caveat the comparison accordingly.
minor comments (5)
  1. [Equation (8)] The uncertainty formula uses Var over N samples but does not define the variance operator or state whether the sum over λ is normalized; please clarify the estimator.
  2. [§4.5] The sentence "ICVL HSIs exhibit more pronounced lens blur" refers to dataset-specific image characteristics, not to the simulated optical encoding; rephrase to avoid implying a different PSF was used.
  3. [Supplement Section 9] The text says all source code is available in the project repository but no URL is provided; please add a link or footnote.
  4. [Figure 7] The reported Pearson correlation of 0.80 is computed over 12.5K sampled pixels from 50 test images; please state whether this is pooled across scenes and report a confidence interval or per-image variability.
  5. [Supplement Section 11] The discussion of RGB measurements should appear in the main text near Table 1, because the distinction between sequential captures and a true Bayer mosaic materially affects how the table is interpreted.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the reconstruction is trained and tested on held-out HSIs with measurements synthesized by the known forward operator, and evaluation is against ground truth; the abstract's 'experiment' claim is unsupported by the simulation-only body, but that is a support gap, not circularity.

full rationale

The derivation chain is self-contained. Measurements are generated by the known shift-invariant operator M in Eq. (1), the diffusion model is trained on (x_patch, y_patch) pairs from ARAD1K, and inference is evaluated against held-out ground-truth HSIs from ARAD1K and external ICVL/Harvard/CAVE datasets. The guidance loss in Eq. (5) enforces consistency with the same M, but this does not force the HSI prediction: the network must still recover 31 spectral channels per pixel, and the reported PSNR/SSIM/SAM are computed against true HSIs, so a wrong spectral estimate cannot pass by construction. The only fitted quantities are the per-patch scale factors c_p_lsq in Eq. (4), which correct a known training-time normalization ambiguity and do not determine spectral content. Self-cited optical tools such as DFlat [17,18] are used to simulate PSFs from stated physical parameters (Fresnel propagation, FDTD-computed phase), not to assume the reconstruction result, so they are not load-bearing circular evidence. One in-scope issue is the claim gap: the abstract says results are shown 'in simulation and experiment,' while Sec. 4 says 'We extensively evaluate our method in simulation,' and no hardware experiment appears in the main text or supplement. This makes the conclusion's 'first demonstration ... captured through a flat-optic lens' unsupported, but the failure is missing evidence for a physical premise (the simulated PSF matches a real lens), not a reduction of the prediction to its inputs. Accordingly, the circularity score is 0.

Assumptions & free parameters 0 free parameters · 4 assumptions · 0 invented entities

No free parameters are fit to the test results; the method learns from training data and uses the known optical forward model. The main axioms are the exactness of the shift-invariant measurement model, the accuracy of the wave-optics PSF simulations, the representativeness of ARAD1K, and standard diffusion theory. No new physical entities are introduced.

assumptions (4)
  • domain assumption The measurement model y = sum_lambda o(lambda) f(u,v,lambda) * x(u,v,lambda) (Eq. 1) is exact, shift-invariant, and known during inference.
    Guidance in Eq. (5) and evaluation assume this forward model; no hardware calibration or tolerance analysis is provided.
  • domain assumption The PSFs computed by the DFlat wave-optics simulator (Eq. 17) accurately represent the physical metalenses.
    All measurements are rendered with these PSFs; fabrication and alignment errors are not modeled.
  • domain assumption A diffusion prior trained on random 64x64 patches from ARAD1K generalizes to other HSI distributions.
    Cross-dataset tests on ICVL and Harvard support this, but performance drops and larger uncertainty are observed.
  • standard math Standard denoising diffusion and DDIM sampling mathematics are valid.
    Used for training and sampling in Section 3.2.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Filterless Snapshot Hyperspectral Imaging using Guided Patch Diffusion." pith.science (2026). https://pith.science/paper/IWY5FTX5

@misc{pith2026241202798,
  author       = {Pith},
  title        = {Pith review of: Filterless Snapshot Hyperspectral Imaging using Guided Patch Diffusion},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/IWY5FTX5}},
  note         = {Machine review of arXiv:2412.02798}
}
abstract

We consider the problem of reconstructing a HxWx31 hyperspectral image from a $H\times W$ grayscale snapshot measurement that is captured using only a single diffractive lens and a filterless panchromatic photosensor. This problem is severely ill-posed, but we present a model that produces high-quality results in simulation and experiment. We make efficient use of limited training data by creating a conditional denoising diffusion model that operates on small patches in a shift-invariant manner. During inference, we synchronize per-patch hyperspectral predictions using guidance by physical consistency with the system's optical point spread function. Our experiments reveal that the patch size can be as small as the point spread function, with local optical cues being the main source of information about complete spectra. Also, by drawing multiple samples, our model provides per-pixel uncertainty estimates that strongly correlate with reconstruction error.

Figures

Figures reproduced from arXiv: 2412.02798 by the authors.

Figure 1
Figure 1. The RGB-projection and two representative spectra [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. (a) A hyperspectral scene is imaged through a diffrac [PITH_FULL_IMAGE:figures/full_fig_p002_2.png] view at source ↗
Figure 3
Figure 3. PSFs (middle row, projected to RGB) used in our exper [PITH_FULL_IMAGE:figures/full_fig_p003_3.png] view at source ↗
Figures from the paper (10 more)
Figure 4
Figure 4. Figure 4: Reconstructing a full-field HSI from an input measurement is achieved by splitting the measurement into patches. Each patch [PITH_FULL_IMAGE:figures/full_fig_p004_4.png]
Figure 6
Figure 6. Figure 6: Comparison of grayscale-to-HSI reconstructions from [PITH_FULL_IMAGE:figures/full_fig_p005_6.png]
Figure 7
Figure 7. Figure 7: (a) Our model’s estimated HSIs, projected to RGB, are [PITH_FULL_IMAGE:figures/full_fig_p006_7.png]
Figure 8
Figure 8. Figure 8: Reconstruction accuracy on the ARAD1K test as a func [PITH_FULL_IMAGE:figures/full_fig_p007_8.png]
Figure 9
Figure 9. Figure 9: (a) Saliency map (bottom row) for diffusion models [PITH_FULL_IMAGE:figures/full_fig_p007_9.png]
Figure 11
Figure 11. Figure 11: Marker pairs denote pixel locations sampled from a real [PITH_FULL_IMAGE:figures/full_fig_p008_11.png]
Figure 10
Figure 10. Figure 10: Cross-dataset reconstructions using measurements ren [PITH_FULL_IMAGE:figures/full_fig_p008_10.png]
Figure 12
Figure 12. Figure 12: (Top) Example noisy grayscale measurements rendered [PITH_FULL_IMAGE:figures/full_fig_p011_12.png]
Figure 13
Figure 13. Figure 13: Nanocylinder Optical Response. The images displays [PITH_FULL_IMAGE:figures/full_fig_p012_13.png]
Figure 14
Figure 14. Figure 14: Visualization of strided patching and stitching. The [PITH_FULL_IMAGE:figures/full_fig_p014_14.png]

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 2 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. MetaH2: A Snapshot Metasurface HDR Hyperspectral Camera

    eess.IV 2025-07 conditional novelty 6.0 of 10

    A single metasurface snapshot encoding both HDR and hyperspectral data is demonstrated, with simulations showing higher reconstruction accuracy than prior snapshot hyperspectral methods.

  2. Emergence of Classical Dynamics from a Random Matrix Schr\"odinger Model

    quant-ph 2026-03 unverdicted novelty 5.0 of 10

    Newtonian macroscopic motion is derived from the linear Schrödinger equation plus a GUE random Hamiltonian modeling environmental interaction, via state-space random-walk parameters and equivalence classes of indistin...

Reference graph

Works this paper leans on

62 extracted references · 57 canonical work pages · cited by 2 Pith papers

  1. [1]

    Sparse recovery of hyper- spectral signal from natural rgb images

    Boaz Arad and Ohad Ben-Shahar. Sparse recovery of hyper- spectral signal from natural rgb images. In Computer Vision – ECCV 2016, pages 19–34. Springer International Publish- ing, 2016. 2, 8

    work page 2016

  2. [2]

    Mohamed Mansoor Roomi

    Boaz Arad, Radu Timofte, Rony Yahel, Nimrod Morag, Amir Bernat, Yuanhao Cai, Jing Lin, Zudi Lin, Haoqian Wang, Yulun Zhang, Hanspeter Pfister, Luc Van Gool, Shuai Liu, Yongqiang Li, Chaoyu Feng, Lei Lei, Jiaojiao Li, Songcheng Du, Chaoxiong Wu, Yihong Leng, Rui Song, Mingwei Zhang, Chongxing Song, Shuyi Zhao, Zhiqiang Lang, Wei Wei, Lei Zhang, Renwei Dian...

    work page 2022

  3. [3]

    Multiwavelength metasurfaces through spatial multiplexing

    Ehsan Arbabi, Amir Arbabi, Seyedeh Mahsa Kamali, Yu Horie, and Andrei Faraon. Multiwavelength metasurfaces through spatial multiplexing. Scientific Reports , 6:32803,

    work page

  4. [4]

    Align your latents: High-resolution video synthesis with la- tent diffusion models, 2023

    Andreas Blattmann, Robin Rombach, Huan Ling, Tim Dock- horn, Seung Wook Kim, Sanja Fidler, and Karsten Kreis. Align your latents: High-resolution video synthesis with la- tent diffusion models, 2023. 5

    work page 2023

  5. [5]

    Metahdr: single shot high-dynamic range imaging and sensing using a multifunctional metasurface

    Charles Brookshire, Yuxuan Liu, Yuanrui Chen, Wei Ting Chen, and Qi Guo. Metahdr: single shot high-dynamic range imaging and sensing using a multifunctional metasurface. Opt. Express, 32(15):26690–26707, 2024. 3

    work page 2024

  6. [6]

    Mask-guided spectral-wise transformer for efficient hyper- spectral image reconstruction

    Yuanhao Cai, Jing Lin, Xiaowan Hu, Haoqian Wang, Xin Yuan, Yulun Zhang, Radu Timofte, and Luc Van Gool. Mask-guided spectral-wise transformer for efficient hyper- spectral image reconstruction. In CVPR, pages 17481– 17490, 2022. 2

    work page 2022

  7. [7]

    Mst++: Multi-stage spectral-wise transformer for efficient spectral reconstruction

    Yuanhao Cai, Jing Lin, Zudi Lin, Haoqian Wang, Yulun Zhang, Hanspeter Pfister, Radu Timofte, and Luc Van Gool. Mst++: Multi-stage spectral-wise transformer for efficient spectral reconstruction. In CVPRW, 2022. 2, 5, 6

    work page 2022

  8. [8]

    Degradation-aware unfolding half-shuffle transformer for spectral compressive imaging, 2022

    Yuanhao Cai, Jing Lin, Haoqian Wang, Xin Yuan, Henghui Ding, Yulun Zhang, Radu Timofte, and Luc Van Gool. Degradation-aware unfolding half-shuffle transformer for spectral compressive imaging, 2022. 2, 5, 6

    work page 2022

Show all 62 references
  1. [9]

    Chakrabarti and T

    A. Chakrabarti and T. Zickler. Statistics of real-world hyper- spectral images. In Proc. IEEE Conf. on Computer Vision and Pattern Recognition (CVPR), pages 193–200, 2011. 8

    2011

  2. [10]

    Spec- traldiff: A generative framework for hyperspectral image classification with diffusion models

    Ning Chen, Jun Yue, Leyuan Fang, and Shaobo Xia. Spec- traldiff: A generative framework for hyperspectral image classification with diffusion models. IEEE Trans. Geo- science and Remote Sensing, 61:1–16, 2023. 2

    2023

  3. [11]

    Mccann, Marc L

    Hyungjin Chung, Jeongsol Kim, Michael T. Mccann, Marc L. Klasky, and Jong Chul Ye. Diffusion posterior sam- pling for general noisy inverse problems, 2024. 4, 1

    2024

  4. [12]

    Improving diffusion models for inverse prob- lems using manifold constraints, 2024

    Hyungjin Chung, Byeongsu Sim, Dohoon Ryu, and Jong Chul Ye. Improving diffusion models for inverse prob- lems using manifold constraints, 2024. 4

    2024

  5. [13]

    Snapshot spec- tral imaging: from spatial-spectral mapping to metasurface- based imaging

    Kaiyang Ding, Ming Wang, Mengyuan Chen, Xiaohao Wang, Kai Ni, Qian Zhou, and Benfeng Bai. Snapshot spec- tral imaging: from spatial-spectral mapping to metasurface- based imaging. Nanophotonics, 13(8):1303–1330, 2024. 1

    2024

  6. [14]

    J. W. Goodman. Introduction to Fourier Optics. Roberts & Co., Englewood, Colorado, 3rd edition, 2005. 3

    2005

  7. [15]

    Com- pact single-shot metalens depth sensors inspired by eyes of jumping spiders

    Qi Guo, Zhujun Shi, Yao-Wei Huang, Emma Alexander, Cheng-Wei Qiu, Federico Capasso, and Todd Zickler. Com- pact single-shot metalens depth sensors inspired by eyes of jumping spiders. Proceedings of the National Academy of Sciences, 116(46):22959–22965, 2019. 2

    2019

  8. [16]

    Efficient diffu- sion training via min-snr weighting strategy, 2024

    Tiankai Hang, Shuyang Gu, Chen Li, Jianmin Bao, Dong Chen, Han Hu, Xin Geng, and Baining Guo. Efficient diffu- sion training via min-snr weighting strategy, 2024. 4

    2024

  9. [17]

    Polarization multi-image synthesis with birefringent metasurfaces

    Dean Hazineh, Soon Wei Daniel Lim, Qi Guo, Federico Ca- passo, and Todd Zickler. Polarization multi-image synthesis with birefringent metasurfaces. In ICCP, pages 1–12, 2023. 3

    2023

  10. [18]

    Hazineh, Soon Wei Daniel Lim, Zhujun Shi, Fed- erico Capasso, Todd Zickler, and Qi Guo

    Dean S. Hazineh, Soon Wei Daniel Lim, Zhujun Shi, Fed- erico Capasso, Todd Zickler, and Qi Guo. D-flat: A dif- ferentiable flat-optics framework for end-to-end metasurface visual sensor design, 2022. 3, 2

    2022

  11. [19]

    Denoising diffu- sion probabilistic models, 2020

    Jonathan Ho, Ajay Jain, and Pieter Abbeel. Denoising diffu- sion probabilistic models, 2020. 3, 5, 4

    2020

  12. [20]

    Jonathan Ho, Tim Salimans, Alexey Gritsenko, William Chan, Mohammad Norouzi, and David J. Fleet. Video diffu- sion models, 2022. 5

    2022

  13. [21]

    Fessler, and Liyue Shen

    Jason Hu, Bowen Song, Jeffrey A. Fessler, and Liyue Shen. Patch-based diffusion models beat whole-image models for mismatched distribution inverse problems, 2024. 1, 8

    2024

  14. [22]

    Jason Hu, Bowen Song, Xiaojian Xu, Liyue Shen, and Jeffrey A. Fessler. Learning image priors through patch- based diffusion models for solving inverse problems. arXiv preprint arXiv:2406.02462, 2024. 1, 2

    2024

  15. [23]

    Hd- net: High-resolution dual-domain learning for spectral com- pressive imaging, 2022

    Xiaowan Hu, Yuanhao Cai, Jing Lin, Haoqian Wang, Xin Yuan, Yulun Zhang, Radu Timofte, and Luc Van Gool. Hd- net: High-resolution dual-domain learning for spectral com- pressive imaging, 2022. 2, 5, 6

    2022

  16. [24]

    Deep gaussian scale mixture prior for spec- tral compressive imaging, 2021

    Tao Huang, Weisheng Dong, Xin Yuan, Jinjian Wu, and Guangming Shi. Deep gaussian scale mixture prior for spec- tral compressive imaging, 2021. 2, 5, 6

    2021

  17. [25]

    Jeon, Seung-Hwan Baek, Shinyoung Yi, Qiang Fu, Xiong Dun, Wolfgang Heidrich, and Min H

    Daniel S. Jeon, Seung-Hwan Baek, Shinyoung Yi, Qiang Fu, Xiong Dun, Wolfgang Heidrich, and Min H. Kim. Com- pact snapshot hyperspectral imaging with diffracted rotation. ACM TOG, 38(4):117:1–13, 2019. 2, 3, 7

    2019

  18. [26]

    Metalenses: Versatile multifunctional photonic components

    Mohammadreza Khorasaninejad and Federico Capasso. Metalenses: Versatile multifunctional photonic components. Science, 358(6367):eaam8100, 2017. 3, 2

    2017

  19. [27]

    Devlin, Jaewon Oh, Alexander Y

    Mohammadreza Khorasaninejad, Wei Ting Chen, Robert C. Devlin, Jaewon Oh, Alexander Y . Zhu, and Federico Ca- passo. Metalenses at visible wavelengths: Diffraction- limited focusing and subwavelength resolution imaging.Sci- ence, 352(6290):1190–1194, 2016. 2 9

    2016

  20. [28]

    Khorasaninejad, A

    M. Khorasaninejad, A. Y . Zhu, C. Roques-Carmes, W. T. Chen, J. Oh, I. Mishra, R. C. Devlin, and F. Capasso. Polarization-insensitive metalenses at visible wavelengths. Nano Letters, 16(11):7229–7234, 2016. 3, 2

    2016

  21. [29]

    Jointly learning band selection and filter array design for hyperspectral imag- ing

    Ke Li, Dengxin Dai, and Luc Van Gool. Jointly learning band selection and filter array design for hyperspectral imag- ing. In WACV, pages 6373–6383, 2023. 2

    2023

  22. [30]

    Spatial-spectral transformer for hyperspectral image denoising

    Miaoyu Li, Ying Fu, and Yulun Zhang. Spatial-spectral transformer for hyperspectral image denoising. In Proceed- ings of the Thirty-Seventh AAAI Conference on Artificial In- telligence and Thirty-Fifth Conference on Innovative Appli- cations of Artificial Intelligence and Thir...

    2023

  23. [31]

    Patchscaler: An efficient patch-independent diffusion model for image super- resolution, 2024

    Yong Liu, Hang Dong, Jinshan Pan, Qingji Dong, Kai Chen, Rongxiang Zhang, Lean Fu, and Fei Wang. Patchscaler: An efficient patch-independent diffusion model for image super- resolution, 2024. 7

    2024

  24. [32]

    End-to-end low cost compressive spectral imaging with spatial-spectral self- attention

    Ziyi Meng, Jiawei Ma, and Xin Yuan. End-to-end low cost compressive spectral imaging with spatial-spectral self- attention. In European Conference on Computer Vision ,

  25. [33]

    lambda-net: Reconstruct hyperspectral images from a snap- shot measurement

    Xin Miao, Xin Yuan, Yunchen Pu, and Vassilis Athitsos. lambda-net: Reconstruct hyperspectral images from a snap- shot measurement. In ICCV, pages 4058–4068, 2019. 2

    2019

  26. [34]

    Spectral diffusercam: lensless snapshot hyper- spectral imaging with a spectral filter array

    Kristina Monakhova, Kyrollos Yanny, Neerja Aggarwal, and Laura Waller. Spectral diffusercam: lensless snapshot hyper- spectral imaging with a spectral filter array. Optica, 7(10): 1298–1307, 2020. 2

    2020

  27. [35]

    Improved denoising dif- fusion probabilistic models, 2021

    Alex Nichol and Prafulla Dhariwal. Improved denoising dif- fusion probabilistic models, 2021. 5, 4

    2021

  28. [36]

    Deeply learned filter response functions for hyperspectral reconstruction

    Shijie Nie, Lin Gu, Yinqiang Zheng, Antony Lam, Nobutaka Ono, and Imari Sato. Deeply learned filter response functions for hyperspectral reconstruction. InICCV, pages 4767–4776,

  29. [37]

    Restoring vision in adverse weather conditions with patch-based denoising dif- fusion models, 2022

    Ozan Ozdenizci and Robert Legenstein. Restoring vision in adverse weather conditions with patch-based denoising dif- fusion models, 2022. 1, 2

    2022

  30. [38]

    Z. Pan, H. Zeng, J. Cao, K. Zhang, and Y . Chen. Diffsci: Zero-shot snapshot compressive imaging via iterative spec- tral diffusion model. In CVPR, pages 25297–25306, Los Alamitos, CA, USA, 2024. IEEE Computer Society. 2

    2024

  31. [39]

    L. Pang, X. Rui, L. Cui, H. Wang, D. Meng, and X. Cao. Hir-diff: Unsupervised hyperspectral image restoration via improved diffusion models. InCVPR, pages 3005–3014, Los Alamitos, CA, USA, 2024. IEEE Computer Society. 2

    2024

  32. [40]

    High-resolution image syn- thesis with latent diffusion models

    Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Bjorn Ommer. High-resolution image syn- thesis with latent diffusion models. In CVPR, pages 10674– 10685, Los Alamitos, CA, USA, 2022. IEEE Computer So- ciety. 5

    2022

  33. [41]

    Diy hyperspectral imaging via polarization-induced spectral filters

    Katherine Salesin, Dario Seyb, Sarah Friday, and Wojciech Jarosz. Diy hyperspectral imaging via polarization-induced spectral filters. In ICCP, pages 1–12, 2022. 2

    2022

  34. [42]

    Sigger, Q

    N. Sigger, Q. T. Vien, S. V . Nguyen, et al. Unveiling the po- tential of diffusion model-based framework with transformer for hyperspectral image classification. Scientific Reports, 14: 8438, 2024. 2

    2024

  35. [43]

    Deep inside convolutional networks: Visualising image clas- sification models and saliency maps

    Karen Simonyan, Andrea Vedaldi, and Andrew Zisserman. Deep inside convolutional networks: Visualising image clas- sification models and saliency maps. CoRR, abs/1312.6034,

    arXiv

  36. [44]

    Denois- ing diffusion implicit models, 2022

    Jiaming Song, Chenlin Meng, and Stefano Ermon. Denois- ing diffusion implicit models, 2022. 3

    2022

  37. [45]

    Single disperser design for coded aperture snapshot spectral imaging

    Ashwin Wagadarikar, Renu John, Rebecca Willett, and David Brady. Single disperser design for coded aperture snapshot spectral imaging. Appl. Opt. , 47(10):B44–B51,

  38. [46]

    Kim, and Hua Huang

    Lizhi Wang, Chen Sun, Ying Fu, Min H. Kim, and Hua Huang. Hyperspectral image reconstruction using a deep spatial-spectral prior. In CVPR, pages 8024–8033, 2019. 2

    2019

  39. [47]

    In2set: Intra-inter similarity exploit- ing transformer for dual-camera compressive hyperspectral imaging

    Xin Wang, Lizhi Wang, Xiangtian Ma, Maoqing Zhang, Lin Zhu, and Hua Huang. In2set: Intra-inter similarity exploit- ing transformer for dual-camera compressive hyperspectral imaging. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) , p...

    2024

  40. [48]

    Patch diffusion: Faster and more data-efficient training of diffusion models

    Zhendong Wang, Yifan Jiang, Huangjie Zheng, Peihao Wang, Pengcheng He, Zhangyang Wang, Weizhu Chen, and Mingyuan Zhou. Patch diffusion: Faster and more data-efficient training of diffusion models. arXiv preprint arXiv:2304.12526, 2023. 1, 2

    2023 arXiv

  41. [49]

    Specat: Spatial-spectral cumulative-attention transformer for high-resolution hyperspectral image reconstruction

    Zhiyang Yao, Shuyang Liu, Xiaoyun Yuan, and Lu Fang. Specat: Spatial-spectral cumulative-attention transformer for high-resolution hyperspectral image reconstruction. In 2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 25368–25377, 2024. 5, 6

    2024

  42. [50]

    Fumihito Yasuma, Tomoo Mitsunaga, Daisuke Iso, and Shree K. Nayar. Generalized assorted pixel camera: Post- capture control of resolution, dynamic range, and spectrum. IEEE TIP, 19(9):2241–2253, 2010. 8

    2010

  43. [51]

    Yuhas, Alexander F

    Roberta H. Yuhas, Alexander F. H. Goetz, and Joe W. Board- man. Discrimination among semi-arid landscape endmem- bers using the spectral angle mapper (sam) algorithm. In Summaries of the Third Annual JPL Airborne Geoscience Workshop. JPL, 1992. 6

    1992

  44. [52]

    H. Zeng, J. Cao, K. Zhang, Y . Chen, H. Luong, and W. Philips. Unmixing diffusion for self-supervised hyperspec- tral image denoising. In CVPR, pages 27820–27830, Los Alamitos, CA, USA, 2024. IEEE Computer Society. 2

    2024

  45. [53]

    Dual prior unfolding for snapshot compressive imaging

    Jiancheng Zhang, Haijin Zeng, Jiezhang Cao, Yongyong Chen, Dengxiu Yu, and Yin-Ping Zhao. Dual prior unfolding for snapshot compressive imaging. In CVPR, pages 25742– 25752, 2024. 2

    2024

  46. [54]

    R2h- ccd: Hyperspectral imagery generation from rgb images based on conditional cascade diffusion probabilistic models

    Lei Zhang, Xiaoyan Luo, Sen Li, and Xiaofeng Shi. R2h- ccd: Hyperspectral imagery generation from rgb images based on conditional cascade diffusion probabilistic models. In IGARSS 2023 - 2023 IEEE International Geoscience and Remote Sensing Symposium, pages 7392–7395, 2023. 2

    2023

  47. [55]

    End-to-end joint optimization of metasur- face and image processing for compact snapshot hyperspec- tral imaging

    Qiangbo Zhang, Zeqing Yu, Xinyu Liu, Chang Wang, and Zhenrong Zheng. End-to-end joint optimization of metasur- face and image processing for compact snapshot hyperspec- tral imaging. Optics Communications, 530:129154, 2023. 2 10 Grayscale to Hyperspectral at Any Resolution Usi...

    2023

  48. [56]

    Robustness to Measurement Noise We demonstrate that our method remains effective when applied to optically-encoded measurements corrupted by noise, provided that the diffusion model is trained using similarly noisy measurements. To simulate noisy measure- ments ynoisy, we add ...

  49. [57]

    See main text for more details

    Pseudo-Code for Guided Sampling Pseudo-code for the guided sampling step is given in Algo- rithm 1. See main text for more details. Algorithm 1 Guided Sampling 1: Initialize xp T ∼ {N(0, I)}p 2: Initialize yp = Patch(y, p) 3: while t >0 do 4: ϵp θ = Model(xp t , t; yp) ▷ Compu...

  50. [58]

    Runtime in seconds to reconstruct at256 × 256 × 31 HSI using a patch size of 32 or 64 pixels as a function of the number of guidance iterations

    Run Time on the ARAD1K Test Set Table 4. Runtime in seconds to reconstruct at256 × 256 × 31 HSI using a patch size of 32 or 64 pixels as a function of the number of guidance iterations. See Sec. 4.2 for more details. Bold denotes the condition used throughout the main paper. I...

  51. [59]

    While any optic that produces similar PSFs would work well, metalenses serve as a convenient ex- ample due to their maturity and strong dispersion proper- ties [27, 28]

    PSF Design Theory Our grayscale-to-hyperspectral approach leverages various point-spread functions (PSFs) generated by metalenses- —flat optics patterned with fixed-height, transparent nanos- tructures [26]. While any optic that produces similar PSFs would work well, metalense...

  52. [60]

    3.3 of the main paper, our diffusion model can only generate hyperspectral image (HSI) patches up to a global scale factor when conditioned on measurement patches

    Patch Normalization during Training As noted in Sec. 3.3 of the main paper, our diffusion model can only generate hyperspectral image (HSI) patches up to a global scale factor when conditioned on measurement patches. Here, we explain the origin of this ambiguity and why we opt...

  53. [61]

    We denote the full-field ground truth HSI as x(i, j, λ) and the reconstructed HSI as ˆx(i, j, λ), each of size (H × W × C)

    Additional Experiment Details HSI Evaluation Metrics. We denote the full-field ground truth HSI as x(i, j, λ) and the reconstructed HSI as ˆx(i, j, λ), each of size (H × W × C). The formulas used to compute our evaluation metrics follow standard practices in grayscale-to-hyper...

  54. [62]

    Here, we provide additional technical details

    Model Summary Our diffusion model adopts a UNet backbone similar to the approaches described in [19] and [35]. Here, we provide additional technical details. Our UNet has five downsam- pling/upsampling stages and uses one ResBlock per stage, rather than the two or three as is ...

Pith tools

Reviewed August 11, 2026 · model on record in the stance chip above.