REVIEW 3 major objections 5 minor 1 cited by
Rethinking High-speed Image Reconstruction Framework with Spike Camera
T0 review · 3 major / 5 minor · reviewed 2026-08-10 · deepseek-v4-flash
Pith's one-line read SpikeCLIP trains low-light spike-to-image reconstruction from CLIP text and unpaired high-quality images, and reports better reconstruction quality on U-CALTECH and U-CIFAR than prior methods.
desk verdict Novel CLIP-based training paradigm for spike reconstruction, undermined by an evaluation loop where NIQE selects the training targets and then scores the results. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The load-bearing mechanism is CLIP as a joint text-image embedding space, used as the only perceptual supervisor. Two learnable prompts, initialized in the style of CoOp, are optimized to represent the distributions of high-quality and low-quality reconstructions; the reconstruction network is then trained with a decoupled class loss and prompt loss in CLIP's shared embedding space. A supporting mechanism is the HQ-image pipeline, which merges outputs of several reconstruction algorithms (TFP, TFI, Spk2ImgNet, SSML, WGSE) and keeps the lowest-NIQE image per sample, while a TFI-based coarse reconstruction gives the lightweight network a stable initialization.
What would settle it
Take a held-out set of real low-light spike streams not used in training, reconstruct with SpikeCLIP and with WGSE or TFI, and run a forced-choice human perceptual study with equalized display. If human raters prefer SpikeCLIP no more often than chance, or if a perceptual metric that was not used to build the HQ dataset (for example, a trained opinion-aware image quality model) shows no advantage, the claimed perceptual improvement would not hold.
Extended reading notes
Core claim
On its own terms, the paper's central claim is that SpikeCLIP surpasses previous spike-to-image reconstruction methods on the real-world low-light datasets U-CALTECH and U-CIFAR, and that it does so by replacing ground-truth supervision with two CLIP-based losses. The method has three stages: a coarse stage trains a lightweight reconstruction network to output the model-based TFI estimate; a prompt-learning stage builds a high-quality image dataset from synthetic and real reconstructions and learns CLIP prompts that separate high-quality from low-quality images; and a fine stage optimizes the network with a class loss (InfoNCE against the class label's text feature) plus a prompt loss that pulls reconstructed images toward the high-quality prompt. The paper reports NIQE reductions of 37.9% on U-CALTECH and 43.1% on U-CIFAR over the best compared method, along with lower BRISQUE and PIQE scores, a 64% classification accuracy on U-CALTECH, and a 0.186M-parameter network with 0.540 ms latency.
Load-bearing premise
The evaluation assumes that NIQE and the other no-reference metrics measure true reconstruction quality, even though the training's high-quality dataset was built by selecting the lowest-NIQE image, so the reported gains may partly reflect optimizing that same statistic.
Editorial extensions
If this is right
- If the claim holds, low-light spike-to-image reconstruction no longer requires paired sharp ground truth; class labels plus unpaired high-quality images suffice, removing the synthetic-to-real domain gap as a training obstacle.
- A 0.186M-parameter network with 0.540 ms latency is enough to beat much larger supervised models, suggesting CLIP supervision can make low-power spike cameras practical for edge deployment.
- Because outputs are aligned with CLIP's text-image features, reconstructed images can be directly classified or matched to text without a separate recognition head.
- On U-CALTECH and U-CIFAR, the method reports the best NIQE, BRISQUE, and PIQE among the compared methods, with the reported margins shown in the results table.
Reading between the lines
- Editorial inference: because the supervision is purely semantic, the same recipe should transfer to other degradation types such as motion blur, haze, or extreme noise wherever a CLIP-recognizable label exists; a direct test would be applying SpikeCLIP to normal-light or deblurring spike datasets.
- Editorial inference: the HQ dataset is built by minimizing NIQE, and the headline metrics are also NIQE-family, so a cleaner test of perceptual quality would use a held-out human rating study or a metric not involved in dataset construction.
- Editorial inference: the reported classification accuracy suggests the reconstruction network is learning semantic features from spikes even before image quality is perfect, hinting that joint reconstruction-recognition systems could be trained end-to-end with the same losses, a direction the paper does not develop.
Signed reviews
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper proposes SpikeCLIP, a three-stage spike-to-image reconstruction framework for real-world low-light spike streams. Stage 1 trains a lightweight reconstruction network to reproduce TFI results; Stage 2 constructs a high-quality (HQ) image set from synthetic and real datasets, selecting candidate reconstructions with the lowest NIQE score (Eq. 6), and learns prompt embeddings that separate HQ from low-quality images in CLIP space; Stage 3 fine-tunes the network with a class-label InfoNCE loss (Eq. 10) and a prompt loss (Eq. 9). Experiments on U-CALTECH and U-CIFAR report large improvements over prior methods in NIQE, BRISQUE, and PIQE, higher CLIP classification accuracy, and very low parameter count, FLOPs, and latency. The central claim is that this CLIP-based supervision, using class labels and unpaired HQ images, suffices to train a lightweight network that recovers texture-rich and brightness-balanced images under low light without paired ground truth.
Significance. If the quantitative claims hold, the framework is a meaningful contribution: it removes the requirement of paired spike-sharp supervision in real-world low-light spike reconstruction, exploits CLIP's text-image alignment as a supervisory signal, and demonstrates that a lightweight network can suffice when the supervision is semantic. The paper also releases code, which supports reproducibility. However, the evaluation protocol is too weak to support the central claim as it stands: the training-target selection and the headline evaluation metric are not independent, and the method uses class labels that the baselines do not. The core idea is promising, but the evidence for superior reconstruction quality requires independent validation.
major comments (3)
- [Prompt Learning, Eq. (6) and Experimental Results, Table 1] The evaluation of reconstruction quality is circular with respect to the HQ dataset construction. In Eq. (6), the HQ images used as positive training targets are selected by minimizing NIQE among candidate reconstructions on real-world datasets, and the main results in Table 1 are reported as NIQE improvements. Because the prompt-loss and fine-tuning stages align the LRN output with embeddings of these NIQE-selected images, a large part of the reported NIQE gain is expected by construction rather than being evidence of genuine perceptual improvement. The claim that SpikeCLIP 'significantly enhances texture details and luminance balance' is therefore not established by Table 1. Please provide evaluation with a metric not involved in dataset construction, such as a human perceptual study, or report scores on an HQ set selected without NIQE.
- [Fine Reconstruction, Eq. (10), and Table 1] The comparison with prior methods is unequal because SpikeCLIP is trained with ground-truth class labels for the reconstructed scene, whereas TFP, TFI, TFSTP, SSML, SpikeFormer, RSIR, Spk2ImgNet, and WGSE do not use label information. The gains in Table 1 may reflect the additional semantic supervision rather than a fundamentally better spike-to-image mapping. In particular, the classification accuracy reported in Table 2 measures alignment with the text labels, not image reconstruction quality. Please compare against baselines that also receive the same class-label information, or explicitly justify why the comparison remains fair for the reconstruction-quality claim.
- [Experimental Results, Table 1 and Fig. 5] All quantitative quality evidence rests on no-reference metrics NIQE, BRISQUE, and PIQE, which are known to correlate imperfectly with perceptual quality and can be optimized artificially. There is no real-world paired ground truth, and Fig. 5 shows only selected qualitative examples. Please report variability across the 1,000 test samples, include statistical significance tests and error bars, and provide a more complete qualitative panel rather than a single selected frame per scene. This would substantially increase confidence that the improvements are perceptual and not metric-specific.
minor comments (5)
- [Fig. 5 caption] The caption contains a typo: 'U-CALTEHC' should be 'U-CALTECH'.
- [Eq. (9)] The prompt loss in Eq. (9) is the negative softmax probability without a logarithm; please clarify whether this is intentional or whether a cross-entropy formulation was intended, since the loss magnitude and gradient scale differ.
- [Table 3] The columns 'Params' and 'Flops' in Table 3 appear to omit units; they are presumably millions and giga-FLOPs, respectively, and should be labeled consistently with Table 1.
- [Eq. (6)] The text says the image with the 'best non-reference metric NIQE score' is selected, but because lower NIQE is better, 'best' should be 'lowest' for clarity.
- [Table 1] The text says bold and underline distinguish best and second-best results, but Table 1 shows only bold entries and no underlined entries; please align the formatting with the stated policy.
Circularity Check
Reported NIQE gains are partly circular: HQ training targets are selected by minimizing NIQE and the same NIQE metric is used as the headline evaluation.
-
fitted input called prediction
[Section 'Prompt Learning / HQ Images Generation Pipeline', Eq. (6); Section 'Experiments / Training Details', evaluation description; Table 1]
"Among them, image Ihq with the best non-reference metric NIQE (Mittal, Soundararajan, and Bovik 2012) score is added to the HQ-Images dataset, i.e.: Ihq = arg min_{m in M} NIQE(Im) (6) ... Since paired spike-sharp datasets are not available in real-world scenarios, we use non-reference metrics, NIQE ..., BRISQUE ..., and PIQE ..., to evaluate the image reconstruction quality."
The supervision dataset is constructed by selecting, per scene, the reconstruction with the lowest NIQE among TFP, TFI, Spk2ImgNet, SSML, and WGSE (Eq. 6). The fine-reconstruction prompt loss then trains the lightweight network to align its outputs with the CLIP embedding of these NIQE-selected HQ images. The paper's central quantitative claim of superiority is then reported in Table 1 using the same NIQE statistic. Thus part of the reported NIQE improvement is a selection effect: the model was guided toward images that were explicitly chosen to minimize NIQE, so NIQE is not an independent measure of the claimed perceptual gains.
full rationale
The derivation chain is not fully self-contained: the CLIP prompt supervision is anchored to a high-quality dataset whose construction criterion (minimum NIQE, Eq. 6) is identical to the headline evaluation metric in Table 1. This makes the NIQE column of Table 1 a partly fitted target rather than a fully independent prediction. However, the method also uses class labels and CLIP text features as external supervision, and it improves on BRISQUE and PIQE, which were not used in the HQ selection. The LRN architecture is borrowed from prior work and the framework is tested against external baselines on U-CALTECH/U-CIFAR, so the central claim has independent content beyond the NIQE-selection bias. No self-citation chain is load-bearing here, and no uniqueness theorem is imported. The main circularity risk is confined to the NIQE evaluation channel; it does not make the entire reconstruction result an equivalence by definition.
Assumptions & free parameters
free parameters (3)
- prompt loss weight lambda =
100
- InfoNCE temperature tau =
not specified
- voxelized spike length =
50 bins
assumptions (5)
- domain assumption The integrate-and-fire spike camera model in Eq. 1 describes the real capture process.
- domain assumption TFI provides a valid coarse reconstruction target for low-light spike streams.
- domain assumption CLIP embeddings are sensitive enough to image quality to distinguish high-quality from low-quality reconstructions and to guide low-level restoration.
- domain assumption NIQE is a reliable perceptual quality measure for spike-camera low-light images.
- domain assumption Class labels for captured scenes are available during training and match the CLIP vocabulary.
Cite this review
Pith. "Pith review of Rethinking High-speed Image Reconstruction Framework with Spike Camera." pith.science (2026). https://pith.science/paper/HTIDWAPY
@misc{pith2026250104477,
author = {Pith},
title = {Pith review of: Rethinking High-speed Image Reconstruction Framework with Spike Camera},
year = {2026},
howpublished = {\url{https://pith.science/paper/HTIDWAPY}},
note = {Machine review of arXiv:2501.04477}
}
read the original abstract
Spike cameras, as innovative neuromorphic devices, generate continuous spike streams to capture high-speed scenes with lower bandwidth and higher dynamic range than traditional RGB cameras. However, reconstructing high-quality images from the spike input under low-light conditions remains challenging. Conventional learning-based methods often rely on the synthetic dataset as the supervision for training. Still, these approaches falter when dealing with noisy spikes fired under the low-light environment, leading to further performance degradation in the real-world dataset. This phenomenon is primarily due to inadequate noise modelling and the domain gap between synthetic and real datasets, resulting in recovered images with unclear textures, excessive noise, and diminished brightness. To address these challenges, we introduce a novel spike-to-image reconstruction framework SpikeCLIP that goes beyond traditional training paradigms. Leveraging the CLIP model's powerful capability to align text and images, we incorporate the textual description of the captured scene and unpaired high-quality datasets as the supervision. Our experiments on real-world low-light datasets U-CALTECH and U-CIFAR demonstrate that SpikeCLIP significantly enhances texture details and the luminance balance of recovered images. Furthermore, the reconstructed images are well-aligned with the broader visual features needed for downstream tasks, ensuring more robust and versatile performance in challenging environments.
Figures
Figures from the paper (2 more)
Forward citations
Cited by 1 Pith paper
-
USP-Gaussian: Unifying Spike-based Image Reconstruction, Pose Correction and Gaussian Splatting
USP-Gaussian jointly optimizes spike-to-image reconstruction, camera poses, and 3D Gaussian Splatting, reducing cascaded errors and improving 3D reconstruction quality on synthetic and real-world spike-camera data.
Reference graph
Works this paper leans on
-
[1]
, " * write output.state after.block = add.period write newline
ENTRY address archivePrefix author booktitle chapter edition editor eid eprint howpublished institution isbn journal key month note number organization pages publisher school series title type volume year label extra.label sort.label short.list INTEGERS output.state before.all mid.sentence after.sentence after.block FUNCTION init.state.consts #0 'before.a...
-
[2]
write newline
" write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 global.max substring 't := if while FUNCTION word.in bbl.in capitalize " " * FUNCT...
-
[3]
Chen, K.; Chen, S.; Zhang, J.; Zhang, B.; Zheng, Y.; Huang, T.; and Yu, Z. 2024. SpikeReveal: Unlocking Temporal Sequences from Real Blurry Inputs with Spike Streams. arXiv preprint arXiv:2403.09486
work page Pith review arXiv 2024
-
[4]
Chen, K.; Ye, Z.; Huang, T.; and Yu, Z. 2025. Spike-Zoo : A Toolbox for Spike-to-Image Reconstruction. https://github.com/chenkang455/Spike-Zoo
work page 2025
-
[5]
Chen, K.; and Yu, L. 2024. Motion Deblur by Learning Residual from Events. IEEE TMM, 26: 6632--6647
work page 2024
-
[6]
Chen, S.; Duan, C.; Yu, Z.; Xiong, R.; and Huang, T. 2022. Self-supervised mutual learning for dynamic scene reconstruction of spiking camera. IJCAI
work page 2022
-
[7]
Chen, S.; Yu, Z.; and Huang, T. 2023. Self-supervised joint dynamic scene reconstruction and optical flow estimation for spiking camera. In AAAI, volume 37, 350--358
work page 2023
-
[8]
Cheng, J.; Liang, D.; and Tan, S. 2024. Transfer CLIP for Generalizable Image Denoising. In CVPR, 25974--25984
work page 2024
Show all 33 references
-
[9]
Cho, H.; Kim, H.; Chae, Y.; and Yoon, K.-J. 2023. Label-free event-based object recognition via joint learning with image reconstruction from events. In ICCV, 19866--19877
2023
-
[10]
Fei-Fei, L.; Fergus, R.; and Perona, P. 2004. Learning generative visual models from few training examples: An incremental bayesian approach tested on 101 object categories. In CVPR, 178--178. IEEE
2004
-
[11]
Huang, T.; Zheng, Y.; Yu, Z.; Chen, R.; Li, Y.; Xiong, R.; Ma, L.; Zhao, J.; Dong, S.; Zhu, L.; et al. 2023. 1000 faster camera and machine vision with ordinary devices. Engineering, 25: 110--119
2023
-
[12]
Krizhevsky, A.; Hinton, G.; et al. 2009. Learning multiple layers of features from tiny images
2009
-
[13]
Liang, Z.; Li, C.; Zhou, S.; Feng, R.; and Loy, C. C. 2023. Iterative prompt learning for unsupervised backlit image enhancement. In ICCV, 8094--8103
2023
-
[14]
K.; and Bovik, A
Mittal, A.; Moorthy, A. K.; and Bovik, A. C. 2012. No-reference image quality assessment in the spatial domain. TIP, 21(12): 4695--4708
2012
-
[15]
completely blind
Mittal, A.; Soundararajan, R.; and Bovik, A. C. 2012. Making a “completely blind” image quality analyzer. SPL, 20(3): 209--212
2012
-
[16]
Oord, A. v. d.; Li, Y.; and Vinyals, O. 2018. Representation learning with contrastive predictive coding. arXiv preprint arXiv:1807.03748
2018 arXiv
-
[17]
W.; Hallacy, C.; Ramesh, A.; Goh, G.; Agarwal, S.; Sastry, G.; Askell, A.; Mishkin, P.; Clark, J.; et al
Radford, A.; Kim, J. W.; Hallacy, C.; Ramesh, A.; Goh, G.; Agarwal, S.; Sastry, G.; Askell, A.; Mishkin, P.; Clark, J.; et al. 2021. Learning transferable visual models from natural language supervision. In ICML, 8748--8763. PMLR
2021
-
[18]
Ronneberger, O.; Fischer, P.; and Brox, T. 2015. U-net: Convolutional networks for biomedical image segmentation. In Medical Image Computing and Computer-Assisted Intervention--MICCAI 2015: 18th International Conference, Munich, Germany, October 5-9, 2015, Proceedings, Part II...
2015
-
[19]
She, C.; and Qing, L. 2022. SpikeFormer: Image Reconstruction from the Sequence of Spike Camera Based on Transformer. In Proceedings of the 2022 5th International Conference on Image and Graphics Processing, 72--78
2022
-
[20]
Song, C.; Huang, Q.; and Bajaj, C. 2022. E-cir: Event-enhanced continuous intensity recovery. In CVPR, 7803--7812
2022
-
[21]
C.; Channappayya, S
Venkatanath, N.; Praneeth, D.; Bh, M. C.; Channappayya, S. S.; and Medasani, S. S. 2015. Blind image quality evaluation using perception based features. In 2015 twenty first national conference on communications (NCC), 1--6. IEEE
2015
-
[22]
Yang, H.; Pan, L.; Yang, Y.; Hartley, R.; and Liu, M. 2024. LDP: Language-driven Dual-Pixel Image Defocus Deblurring Network. In CVPR, 24078--24087
2024
-
[23]
Zhang, J.; Jia, S.; Yu, Z.; and Huang, T. 2023. Learning temporal-ordered representation for spike streams based on discrete wavelet transforms. In AAAI, volume 37, 137--147
2023
-
[24]
Zhao, J.; Xiong, R.; Liu, H.; Zhang, J.; and Huang, T. 2021. Spk2imgnet: Learning to reconstruct dynamic scene from continuous spike stream. In CVPR, 11996--12005
2021
-
[25]
Zhao, J.; Zhang, S.; Yu, Z.; and Huang, T. 2024 a . Recognizing Ultra-High-Speed Moving Objects with Bio-Inspired Spike Camera. In AAAI, volume 38, 7478--7486
2024
-
[26]
Zhao, R.; Xiong, R.; Zhao, J.; Zhang, J.; Fan, X.; Yu, Z.; and Huang, T. 2024 b . Boosting Spike Camera Image Reconstruction from a Perspective of Dealing with Spike Fluctuations. In CVPR, 24955--24965
2024
-
[27]
Zheng, Y.; Yu, Z.; Wang, S.; and Huang, T. 2022. Spike-based motion estimation for object tracking through bio-inspired unsupervised learning. TIP, 32: 335--349
2022
-
[28]
Zheng, Y.; Zhang, J.; Zhao, R.; Ding, J.; Chen, S.; Xiong, R.; Yu, Z.; and Huang, T. 2023. SpikeCV: Open a Continuous Computer Vision Era. arXiv preprint arXiv:2303.11684
2023 arXiv
-
[29]
Zheng, Y.; Zheng, L.; Yu, Z.; Shi, B.; Tian, Y.; and Huang, T. 2021. High-speed image reconstruction through short-term plasticity for spiking cameras. In CVPR, 6358--6367
2021
-
[30]
C.; and Liu, Z
Zhou, K.; Yang, J.; Loy, C. C.; and Liu, Z. 2022. Learning to prompt for vision-language models. IJCV, 130(9): 2337--2348
2022
-
[31]
Zhu, L.; Dong, S.; Huang, T.; and Tian, Y. 2019. A retina-inspired sampling method for visual texture reconstruction. In ICME, 1432--1437. IEEE
2019
-
[32]
Zhu, L.; Dong, S.; Li, J.; Huang, T.; and Tian, Y. 2020. Retina-like visual image reconstruction via spiking neural model. In CVPR, 1438--1446
2020
-
[33]
Zhu, L.; Zheng, Y.; Geng, M.; Wang, L.; and Huang, H. 2023. Recurrent spike-based image restoration under general illumination. In ACMMM, 8251--8260
2023
Reviewed August 10, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.