Pith. sign in

REVIEW 3 major objections 6 minor 72 references

Video Denoising in Fluorescence Guided Surgery

T0 review · 3 major / 6 minor · reviewed 2026-08-12 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read The paper argues that fluorescence guided surgery video can be denoised by simulating laser leakage light from a co-located reference video, and that a simple recurrent network built on a single-image denoiser outperforms adapted…

desk verdict A solid FGS denoising paper with a genuinely useful simulate-then-train pipeline and a surprising single-image-over-video result, though the 'accurate' noise simulation claim is only partially validated and a few numbers need explanation. read the letter →

arxiv 2411.09798 v1 pith:HBC3EQSS submitted 2024-11-14 cs.CV cs.LGeess.IV

Trevor Seets , Andreas Velten This is my paper · ORCID
classification cs.CVcs.LGeess.IV
keywords fluorescenceguidedsurgeryvideodenoisinglaserleakagelightnoisesimulationreferencerecurrentneuralnetworkPoissonshotmedicalimaging
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Fluorescence guided surgery (FGS) produces dim videos corrupted not only by shot and read noise but also by laser leakage light (LLL), an additive bias whose brightness can rival the fluorescence signal itself. The paper tries to establish that LLL can be predicted from a co-located reference video and inserted into a noise simulator, so denoising networks can be trained on realistic synthetic data. It reports that a single-image denoiser, NafNet32, outperforms adapted state-of-the-art video denoisers on this task, and that a recurrent baseline built on it, BL-RNN, gives the best perceptual quality in the noise regime where FGS systems operate. If correct, the work gives the field a validated simulate-and-train pipeline, strong baselines, and a way to evaluate hardware choices over the signal/LLL tradeoff space.

What carries the argument

Two objects carry the argument. The LLL-PN (a lightweight NafNet32 trained on OL-LLL with a median-seeking L1 loss) predicts the laser leakage frame from the reference video and supplies the bias term for Eq. (1); it is load-bearing because without it the simulated LLL distribution would not match the real one. The second is the noise simulator of Eq. (1), which combines clean fluorescence frames, predicted LLL, Poisson shot noise with calibrated camera gain K, and real dark frames for read noise. The third is BL-RNN, a recurrent network that feeds the previous denoised frame back into a NafNet32 U-net with ReLU activations, giving temporal integration without the unstable propagation of the BasicVSR++ style recurrence the paper tried first.

What would settle it

Acquire human FGS frames with the excitation laser on and no fluorophore present, as in the paper's OL-LLL protocol, and measure how much of the true LLL energy a LLL-PN trained on chicken-thigh data can predict; if the predicted energy share falls well below the 40% observed in OL-LLL, or if a denoiser trained on LLL-PN-simulated data fails to remove a hand or tool shadow on real human video, the transfer claim collapses.

Watch

Extended reading notes

Core claim

The central claim is that FGS video denoising should be treated as a noise-formation problem with three terms—Poisson shot noise, sampled read noise, and spatially varying LLL—and that the LLL term can be approximated by a network fLLL that maps each reference-video frame Rv_t to a predicted LLL frame. The noisy fluorescence frame is modeled as F̃ = Quant(K Pois(Sm S + Lm LLL_pred) + R) / (K Sm), with calibrated gain K = 1/1764 and dark frames sampled from OL-Dark. Trained on OL-LLL in a Noise2Noise fashion with L1 loss, the LLL-PN accounts for 40% of the energy in noisy LLL frames, capturing large structure while missing specular reflections and 3D-dependent shadows. On simulated test data, NafNet32 reaches 34.8 dB PSNR versus 30.5 dB for causal BasicVSR++, while FastDVDnetC and OFDVDnetC fall below 7 dB; the paper attributes this to the bias and causality violations in FGS, and shows on the Davis dataset that the advantage reverses in conventional video denoising. The paper's proposed baselines combine NafNet blocks with sliding-window, align-and-merge, and recurrent temporal strategies, with BL-RNN giving the best LPIPS in the middle noise region and being the only model that removes LLL from a real no-fluorescence scene.

Load-bearing premise

The load-bearing premise is that the reference video can predict the laser leakage light strongly enough that the simulated LLL distribution matches reality; the authors find this true in their mock-surgery data but flag that it must be checked on human tissue, and their LLL-PN only explains 40% of the energy in noisy LLL frames, missing specular reflections and 3D-dependent shadows.

Editorial extensions

If this is right

  • Simulated training data that includes LLL should let denoisers trained on mock-surgery videos transfer to real FGS systems, as long as the reference video remains informative.
  • Causal FGS denoising is feasible: the proposed baselines use only past and current frames, which fits the clinical workflow.
  • Conventional video denoisers designed for zero-mean, non-causal noise will need structural changes—especially access to the reference video and robustness to biased noise—before they can serve FGS.
  • The two-dimensional (Sm, Lm) noise space provides a cost landscape that could rank candidate hardware configurations by denoiser performance.
  • Dim fluorescent contrast agents become more clinically viable if post-capture denoising can recover signal near the noise floor.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Editorial inference: if the RV-to-LLL correlation holds in human tissue, the same simulator could model autofluorescence, which is also correlated with the reference view; the paper notes this possibility.
  • Editorial inference: the repeated-frame experiment suggests current recurrent models underuse temporal information, so an architecture that explicitly forces long averages, like the align-and-merge baseline, may be necessary as signal drops.
  • Editorial inference: the qualitative success on the no-fluorescence real scene implies BL-RNN has learned a physical prior about LLL structure rather than memorizing simulated noise; a quantitative real-data test with injected fluorescence of known concentration would settle how far this extends.
  • Editorial inference: the gain calibration and dark-frame sampling procedures are transferable to other FGS cameras, so the simulate-and-train pipeline could be reproduced without the authors' hardware.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 6 minor

Summary. The paper addresses video denoising for fluorescence-guided surgery (FGS), where the fluorescence video is corrupted by shot noise, read noise, and a spatially varying laser leakage light (LLL) term. The authors propose a noise simulation pipeline that predicts LLL from a co-located reference video using a learned LLL-PN, generates synthetic noisy FV frames via Eq. (1), and trains denoising networks on these data. They introduce OL-2024, expanding existing FGS datasets, and propose three causal baseline models (BL-SW, BL-A&M, BL-RNN) built on a NAFNet backbone. They report that an image denoiser (NafNet32) outperforms adapted causal video denoisers (BasicVSR++, FastDVDnet, OFDVDnet) on their simulated FGS test set, and they analyze LLL robustness, ablations, and temporal behavior.

Significance. If the claims hold, the paper provides a valuable simulate-and-train pipeline for FGS, a new public dataset, and strong baselines for a clinically relevant problem. The surprising result that a single-image denoiser outperforms temporal video denoisers on LLL-dominated noise, with a Davis control showing the opposite on conventional noise, is an important empirical finding. The paper also includes useful robustness analyses (mlll, LLL-PN switching) and makes data/code available. However, the significance is contingent on whether the simulated LLL distribution matches real deployment conditions, which the current validation does not establish quantitatively.

major comments (3)
  1. [Sec. III / Eq. (1) / Sec. III-A / Fig. 4] The 'accurate noise simulation' claim (Abstract; Sec. III) is not validated at the point that matters. Eq. (1) generates both training and test noise using the same LLL-PN, fLLL(Rv_t), so the test set is in-distribution by construction; this does not demonstrate that the simulated noise matches real LLL. Sec. III-A reports that fLLL accounts for 40% of the L2 energy of noisy LLL frames and that specular reflections and 3D-dependent shadows (Fig. 4) are not predicted. Those structures are absent from the synthetic noise, so the trained models are never asked to remove them, while the OL-Real evaluation (Sec. V-E) has no ground truth and cannot show whether they are handled. The authors' own caveat in Sec. III-A that the RV-to-LLL assumption 'will be important to check in human data' concedes exactly this gap. I request a quantitative comparison of synthetic and real noise (e.g., patch statistics, noise power spectra, or detection rates on held-out OL-LLL frames with known LLL) and either simulation of the missing LLL structures or real-data evaluation with ground truth LLL.
  2. [Table I / Sec. IV-A2] The reported PSNR/SSIM values for FastDVDnetC (6.1/0.020) and OFDVDnetC (2.5/0.000) are below what a trivial baseline (e.g., outputting the noisy input) would achieve, strongly suggesting an implementation or training failure in the adaptations rather than a genuine property of temporal denoisers. As written, the conclusion that 'SOTA video denoisers are unsuitable' (Sec. IV-A2) rests on a comparison in which the baseline models may not be functioning correctly. The authors should verify that their causal adaptations train correctly, for example by reporting training curves, comparing against the original non-causal models on the same data, or checking performance on clean/unnoisy inputs. Without this, the headline comparison of Table I is not interpretable.
  3. [Sec. V-B / Table III] The LLL robustness test with a second LLL-PN f2 changes only the training data split of the same OL-LLL set. It therefore measures sensitivity to LLL-PN weights, but not to the structural failure modes the paper identifies in Sec. III-A: specular reflections and shadows that depend on 3D scene geometry. The mlll slope measure is useful for quantifying sensitivity to Lm, but it cannot test whether models handle those missing components. To support the claim that the proposed baselines are robust to LLL in practice, the authors need a test that includes those structures, e.g., by injecting synthetic specular/shadow LLL into the simulation or by collecting real LLL frames with paired RV and evaluating where the denoisers fail.
minor comments (6)
  1. [Sec. IV] There are typographical errors: 'We simulate simulate noisy frames' in Sec. IV and 'LLL rmeoval' in Sec. V-C should be corrected.
  2. [Eq. (1)] The notation '1/KS m' is ambiguous; please write it as a single fraction with K and Sm clearly separated, and state explicitly whether Quant is applied before or after the multiplication by 1/(K Sm).
  3. [Sec. III-A] The statement that fLLL 'accounts for 40% of the total energy' is ambiguous because it does not specify what fraction of the remaining energy is shot/read noise versus unmodeled LLL structure; please provide a decomposition of the residual energy.
  4. [Fig. 7] The red polygon indicating 'realistic noise scenarios' is not defined in the text; please provide the criterion used to draw it.
  5. [Sec. IV-C] The training budget of 'maximum 2 weeks' is a hardware-specific statement; report iterations/epochs and early-stopping criteria instead of wall-clock time as the primary convergence measure.
  6. [References] The dataset reference [12] is listed as 'Forthcoming 2024'; if the dataset and code are now available, update the citation to the final DOI.

Circularity Check

1 steps flagged · score 3.0 of 10

Simulated LLL test frames are generated by the same LLL-PN used to synthesize training noise, so the reported PSNR/SSIM rankings measure inversion of the LLL-PN's own output rather than real LLL removal; the paper's own 40%-energy and specular/shadow caveats confirm the loop omits the hardest real LLL structures.

  1. fitted input called prediction [Sec. III Eq. (1); Sec. III-A LLL-PN; Sec. IV (test set); Table I]
    "Let St be the fluorescent signal of interest, LLLt be a predicted LLL frame, and Rt be the read noise of the camera at frame t, all scaled between 0 and 1. Then a noisy FV frame, ˜Fvt, is given by, ... (1) ... We simulate simulate noisy frames using the clean FV in OL-Combined as St in our noise model and train the algorithms to recover St. Unless otherwise noted the test set used is from OL-Combined."

    The LLL injected by Eq. (1) is the output of the LLL-PN, fLLL(Rvt), trained on OL-LLL (Sec. III-A). The same Eq. (1) generates both training and test frames from OL-Combined, so the LLL component of the test distribution is produced by the very function the denoisers are trained to invert. Tables I-III therefore rank models on removal of the LLL-PN's own predictions, not on removal of real LLL. The paper concedes fLLL captures only 40% of the L2 energy of noisy LLL frames and misses specular reflections and 3D-dependent shadows, structures that never appear in the closed-loop benchmark. The Davis control and f2lll robustness test are independent checks, but f2lll is itself another fitted LLL-PN, so neither breaks the same-generator loop for real LLL validation.

full rationale

No load-bearing self-citation circularity was found: the authors' prior OFDVDnet work is used as a dataset source, an alignment strategy, and a retrained comparison baseline, all of which are empirical resources rather than assumed conclusions. The paper also provides independent grounding for its central simulation claim: the LLL-PN is evaluated against real OL-LLL frames (qualitatively and via the 40% energy figure), the noise model's gain and read noise are calibrated on physical phantoms and dark frames, the Davis experiment shows the surprising NafNet-over-BasicVSR++ ranking reverses on conventional noise, and a second LLL-PN is used to test robustness to simulator changes. The remaining mild circularity is that the main simulated benchmark uses the same LLL-PN to create both training and test LLL, so the reported PSNR/SSIM tables measure in-distribution inversion of the fitted LLL-PN, not generalization to the real LLL structures the paper itself identifies as missing (specular reflections, 3D shadows). The authors acknowledge the load-bearing RV-to-LLL assumption must 'be important to check in human data,' which further limits external validity. This partial same-generator circularity warrants a score of 3, not higher, because the central claim has independent content and the circular element is confined to the simulated evaluation loop.

Assumptions & free parameters 4 free parameters · 5 assumptions · 0 invented entities

The paper introduces no new physical entities, forces, or conserved quantities. Its new artifacts are learned networks (LLL-PN and denoisers) and datasets, which are not invented entities in the sense of this ledger. The load-bearing content is carried by the fitted camera gain K, the hand-chosen noise simulation scales (Sm, Lm, Rm), and the five domain assumptions listed above, most notably the RV-predicts-LLL premise that the authors themselves flag as needing human-data validation.

free parameters (4)
  • Camera gain K = 1/1764 (scaled to [0,1])
    Fitted from OL-Phantom Quel-well data using the mean-equals-variance Poisson relation (Supplement Sec. I-A). The entire noise model in Eq. 1 scales with K, so all simulated PSNR values depend on this calibration.
  • Signal photon scaling Sm = Evaluated at 10-200; training range [10, 1/(2K)]
    Hand-chosen simulation parameter defining the fluorescence signal level in Eq. 1. Performance conclusions in Table II and Fig. 7 are functions of Sm, so the chosen range shapes the claims.
  • LLL photon scaling Lm = Evaluated at 0 to Sm
    Hand-chosen simulation parameter defining the laser leakage light level. Central to the claim that LLL degrades SOTA video denoisers; the tested Lm/Sm ratios determine where each baseline appears strongest.
  • Read-noise scaling Rm = 6 at test; trained over [4,8]
    Sampled from OL-Dark dark-frame calibration; adjusts read noise magnitude in Eq. 1. The test value 6 is calibrated to the digital gain setting (Supplement Sec. I-C), so it is a fitted scale rather than a pure augmentation choice.
assumptions (5)
  • domain assumption Reference video (RV) predicts near-infrared LLL.
    The LLL-PN maps RV to LLL. The authors state: 'our LLL-PN hinges on the assumption that the RV can be used to predict near infrared reflectivity, which we find to be true in OL-LLL, but will be important to check in human data' (Sec. III-A). The whole simulation pipeline rests on this.
  • domain assumption Clean FV from high-concentration ICG is LLL-free ground truth.
    The paper uses 'high concentrations of indocyanine green (ICG) to generate very bright fluorescence that can be used as ground truth FV without LLL' (Sec. II). If residual LLL remains in those frames, the training targets are contaminated.
  • domain assumption Noise decomposes as Eq. 1: read noise, shot noise, and additive LLL photon count.
    Eq. 1 models LLL as an additive spatially varying photon count inside the Poisson parameter. This assumes LLL is independent of the fluorescence signal and that filter leakage is additive, which is plausible but not proven for all scenes.
  • domain assumption Time-contiguous dark frames from OL-Dark model the sensor read noise.
    Read noise Rt is sampled from OL-Dark frames following [21] (Sec. III). This transfers a prior method to this camera and assumes the dark-frame statistics are stationary.
  • domain assumption Simulated noise transfers to real FGS data.
    The entire training strategy assumes a model trained on simulated noise (with a fixed LLL-PN) generalizes to real clinical noise. Only qualitative OL-Real results are provided; there is no quantitative distributional comparison between simulated and real noisy frames.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Video Denoising in Fluorescence Guided Surgery." pith.science (2026). https://pith.science/paper/HBC3EQSS

@misc{pith2026241109798,
  author       = {Pith},
  title        = {Pith review of: Video Denoising in Fluorescence Guided Surgery},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/HBC3EQSS}},
  note         = {Machine review of arXiv:2411.09798}
}
read the original abstract

Fluorescence guided surgery (FGS) is a promising surgical technique that gives surgeons a unique view of tissue that is used to guide their practice by delineating tissue types and diseased areas. As new fluorescent contrast agents are developed that have low fluorescent photon yields, it becomes increasingly important to develop computational models to allow FGS systems to maintain good video quality in real time environments. To further complicate this task, FGS has a difficult bias noise term from laser leakage light (LLL) that represents unfiltered excitation light that can be on the order of the fluorescent signal. Most conventional video denoising methods focus on zero mean noise, and non-causal processing, both of which are violated in FGS. Luckily in FGS, often a co-located reference video is also captured which we use to simulate the LLL and assist in the denoising processes. In this work, we propose an accurate noise simulation pipeline that includes LLL and propose three baseline deep learning based algorithms for FGS video denoising.

Figures

Figures reproduced from arXiv: 2411.09798 by the authors.

Figure 1
Figure 1. Measurement and Noise: (a) In the FGS measurement process, excitation laser light and reference light shine onto the scene where the excitation light produces fluorescence at a higher wavelength. Three relevant spectral bands are imaged by the FGS system. The reference band (yellow) is isolated using a dichoric beamsplitter and imaged by the RV camera. The excitation laser band (red) is attenuated by the emission fi… view at source ↗
Figure 2
Figure 2. Dataset Example Images: Here we show two example images for both OL-2023 and OL-2024. OL-2023 focuses on vasculature where as OL￾2024 focuses on local fluorescent regions. scenarios. We expand the scope of this data with the new OL-2024 dataset with challenging motion scenarios and a focus on cancer and lymphatic surgeries. We follow a similar experimental setup that was used to create OL-2023; we use [PITH_FULL_IM… view at source ↗
Figure 3
Figure 3. Real and Simulated Data: (a) frames from our OL-Real test set. (b) simulated frames with increasing signal levels and fixed Lm = 50. Qualitatively, Sm = Lm = 50 closely matches many of the real data frames. Notice the hand (red arrow) has similar signal levels in both the real and simulated data at these parameters. (c) simulated frames with increasing Lm and Sm = 50. Notice that the small fluorescent features (gree… view at source ↗
Figures from the paper (11 more)
Figure 4
Figure 4. Figure 4: LLL and LLL-PN: This figure shows examples from OL-LLL and our corresponding LLL-PN predictions. The last row shows the absolute difference between the LLL frame and prediction. The LLL-PN is able to correctly predict most of the large structure leaving shot and read n…
Figure 5
Figure 5. Figure 5: Baseline Model Overview: We propose 3 baseline: (a) SW-BL uses a fixed temporal receptive field, (b) BL-A&M uses a recursive non-learning based align and merge module followed by a U-net block, and (c) RNN-BL uses a simple recursive input. 1) BL-SW: A sliding window st…
Figure 6
Figure 6. Figure 6: Proposed Baseline Results: This figure shows the results of our baseline models over different numbers of photons with constant Lm Sm . Notice as signal gets lower the read noise structure becomes more significant and BL-SW outperforms, whereas at higher signal levels …
Figure 7
Figure 7. Figure 7: Best Model over Sm and Lm: This figure shows the model with the lowest LPIPS over a range of different Sm and Lm values. Inside of the red polygon indicates an estimate of realistic noise scenarios that may be seen on current systems in scenes that require denoising. t…
Figure 8
Figure 8. Figure 8: Example Denoised Results: This figure shows a test case with Sm = Lm = 50. In this example our three baseline models all perform reasonable well whereas the comparison models tend to over-smooth (red arrow). TABLE IV BL-RNN ABLATION RESULTS: THIS TABLE SHOWS THE PSNR C…
Figure 9
Figure 9. Figure 9: Repeated Frames: In this experiment, we copy the first frame of each test video 100 times to simulate a scene without motion. We find RNN-BL and BasicVSR++C have constant performance after the first frame, whereas BL￾A&M improves for the first 50 frames. After 50 frame…
Figure 10
Figure 10. Figure 10: OL-2024 Results: This figure shows the results of our baseline models (a-b) on 2 scenes from OL-Real. Notice how the hands in (a-b) are correctly removed by our baselines. (c) shows a scene with no fluorescent agent injected, BL-RNN is able to correctly remove most of…
Figure 1
Figure 1. Figure 1: Gain Calibration Images: This figure shows the (a) mean, (b) variance, and (c) the mean divided by the variance of the Quel calibration phantom from OL-Phantom used to calibrate for our K parameter. (d) shows the histogram of mean divided by variance values for the 9 w…
Figure 2
Figure 2. Figure 2: Flicker Noise: (a) shows the histogram of the per-frame average values of the OL-Dark dataset. The read noise pattern of our camera sensor exhibits strong flicker noise leading to a bimodal distribution, one example from each peak is shown. (b) shows the per-frame aver…
Figure 3
Figure 3. Figure 3: Instability of Recurrent Candidate Models: (a) Training and validation curves for the BasicNaf32 and BasicNaf32-Preload candidate models. The BasicNaf32 model exhibits training instability after 100 epochs of training (60 hours). While our BasicNaf32-Preload exhibits s…
Figure 4
Figure 4. Figure 4: No Reference: This figure shows a case where the reference frame is key in the denoising process to properly remove the LLL associated with a hand. The noise parameters used for this example are Sm = Lm = 25. V. NO REFERENCE FRAME QUALITATIVE RESULT We find that the RV…

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

72 extracted references · 46 canonical work pages

  1. [1]

    Trauma of major surgery: a global problem that is not going away,

    G. P. Dobson, “Trauma of major surgery: a global problem that is not going away,” pp. 47–54, 2020

    work page 2020

  2. [2]

    Positive surgical margins in the 10 most common solid cancers,

    R. K. Orosco, V . J. Tapia, J. A. Califano, B. Clary, E. E. Cohen, C. Kane, S. M. Lippman, K. Messer, A. Molinolo, J. D. Murphy et al., “Positive surgical margins in the 10 most common solid cancers,” Scientific reports, vol. 8, no. 1, p. 5686, 2018

    work page 2018

  3. [3]

    Fluorescence-guided surgery: comprehensive review,

    P. A. Sutton, M. A. van Dam, R. A. Cahill, S. Mieog, K. Polom, A. L. Vahrmeijer, and J. van der V orst, “Fluorescence-guided surgery: comprehensive review,” BJS open, vol. 7, no. 3, p. zrad049, 2023

    work page 2023

  4. [4]

    Review of fluorescence guided surgery systems: identification of key performance capabilities beyond indocyanine green imaging,

    A. V . DSouza, H. Lin, E. R. Henderson, K. S. Samkoe, and B. W. Pogue, “Review of fluorescence guided surgery systems: identification of key performance capabilities beyond indocyanine green imaging,” Journal of biomedical optics , vol. 21, no. 8, pp. 080 901–080 901, 2016

    work page 2016

  5. [5]

    Fluorescence guidance in surgical oncology: challenges, opportunities, and translation,

    M. T. Olson, Q. P. Ly, and A. M. Mohs, “Fluorescence guidance in surgical oncology: challenges, opportunities, and translation,” Molecular imaging and biology , vol. 21, pp. 200–218, 2019

    work page 2019

  6. [6]

    Guidance for performance evaluation for fluorescence guided surgery systems: evaluation for fluorescence guided surgery systems,

    B. Pogue, T. Zhu, V . Ntziachristos, B. Wilson, K. Paulsen, S. Gioux, R. Nordstrom, J. Pfefer, B. Tromberg, H. Wabnitz et al. , “Guidance for performance evaluation for fluorescence guided surgery systems: evaluation for fluorescence guided surgery systems,” 2023

    work page 2023

  7. [7]

    Transient room lighting for ambient light multiphoton microscopy,

    A. Velten, A. Uselmann, S. Prajapati, J. Bredfeldt, T. Mackie, and K. Eliceiri, “Transient room lighting for ambient light multiphoton microscopy,” 2020

    work page 2020

  8. [8]

    Ofdvdnet: A sensor fusion approach for video denoising in fluorescence-guided surgery,

    T. Seets, W. Lin, Y . Lu, C. Lin, A. Uselmann, and A. Velten, “Ofdvdnet: A sensor fusion approach for video denoising in fluorescence-guided surgery,” in Medical Imaging with Deep Learning , ser. Proceedings of Machine Learning Research, I. Oguz, J. Noble, X. Li, M. Styner, C. Baumgartner, M. Rusu, T. Heinmann, D. Kontos, B. Landman, and B. Dawant, Eds., ...

    work page 2024

Show all 72 references
  1. [9]

    Basicvsr++: Improving video super-resolution with enhanced propagation and alignment,

    K. C. Chan, S. Zhou, X. Xu, and C. C. Loy, “Basicvsr++: Improving video super-resolution with enhanced propagation and alignment,” in Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, 2022, pp. 5972–5981

    2022

  2. [10]

    Simple baselines for image restoration,

    L. Chen, X. Chu, X. Zhang, and J. Sun, “Simple baselines for image restoration,” in European Conference on Computer Vision . Springer, 2022, pp. 17–33

    2022

  3. [11]

    Augmentation of chicken thigh model with fluorescence imaging allows for real-time, high fidelity assessment in supermicrosurgery training,

    N. J. Albano, W. Zeng, C. Lin, A. J. Uselmann, K. W. Eliceiri, and S. O. Poore, “Augmentation of chicken thigh model with fluorescence imaging allows for real-time, high fidelity assessment in supermicrosurgery training,” Journal of reconstructive microsurgery , vol. 37, no. 6, 2021

    2021

  4. [12]

    Dataset and trained models for video de- noising in fluorescence guided surgery,

    T. Seets and A. Velten, “Dataset and trained models for video de- noising in fluorescence guided surgery,” https://doi.org/10.5061/dryad. 8gtht76x9, (Forthcoming 2024)

    2024 doi

  5. [13]

    Indocyanine green matching phantom for fluorescence-guided surgery imaging system characterization and performance assessment,

    A. J. Ruiz, M. Wu, E. P. LaRochelle, D. Gorpas, V . Ntziachristos, T. J. Pfefer, and B. W. Pogue, “Indocyanine green matching phantom for fluorescence-guided surgery imaging system characterization and performance assessment,” Journal of Biomedical Optics , vol. 25, no. 5, pp....

    2020

  6. [14]

    S. W. Hasinoff, Photon, Poisson Noise . Boston, MA: Springer US, 2014, pp. 608–610

    2014

  7. [15]

    A statistical model of flicker noise,

    J. Barnes and D. Allan, “A statistical model of flicker noise,” Proceed- ings of the IEEE , vol. 54, no. 2, pp. 176–178, 1966

    1966

  8. [16]

    High-level numerical simulations of noise in ccd and cmos photosensors: review and tutorial,

    M. Konnik and J. Welsh, “High-level numerical simulations of noise in ccd and cmos photosensors: review and tutorial,” arXiv preprint arXiv:1412.4031, 2014

    2014 arXiv

  9. [17]

    A model for dark current characterization and simulation,

    R. L. Baer, “A model for dark current characterization and simulation,” in Sensors, Cameras, and Systems for Scientific/Industrial Applications VII, M. M. Blouke, Ed., vol. 6068, International Society for Optics and Photonics. SPIE, 2006, p. 606805

    2006

  10. [18]

    A cmos imager with column-level adc using dynamic column fixed- pattern noise reduction,

    M. F. Snoeij, A. J. P. Theuwissen, K. A. A. Makinwa, and J. H. Huijsing, “A cmos imager with column-level adc using dynamic column fixed- pattern noise reduction,” IEEE Journal of Solid-State Circuits , vol. 41, no. 12, pp. 3007–3015, 2006

    2006

  11. [19]

    Enhancing low light videos by exploring high sensitivity camera noise,

    W. Wang, X. Chen, C. Yang, X. Li, X. Hu, and T. Yue, “Enhancing low light videos by exploring high sensitivity camera noise,” in 2019 IEEE/CVF International Conference on Computer Vision (ICCV) , 2019, pp. 4110–4118

    2019

  12. [20]

    A physics-based noise formation model for extreme low-light raw denoising,

    K. Wei, Y . Fu, J. Yang, and H. Huang, “A physics-based noise formation model for extreme low-light raw denoising,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , 2020, pp. 2758–2767

    2020

  13. [21]

    Rethinking noise synthesis and modeling in raw denoising,

    Y . Zhang, H. Qin, X. Wang, and H. Li, “Rethinking noise synthesis and modeling in raw denoising,” in Proceedings of the IEEE/CVF International Conference on Computer Vision , 2021, pp. 4593–4601

    2021

  14. [22]

    Noise2noise: Learning image restoration without clean data,

    J. Lehtinen, J. Munkberg, J. Hasselgren, S. Laine, T. Karras, M. Aittala, and T. Aila, “Noise2noise: Learning image restoration without clean data,” arXiv preprint arXiv:1803.04189 , 2018

    2018 arXiv

  15. [23]

    Kbnet: Kernel basis network for image restoration,

    Y . Zhang, D. Li, X. Shi, D. He, K. Song, X. Wang, H. Qin, and H. Li, “Kbnet: Kernel basis network for image restoration,” arXiv preprint arXiv:2303.02881, 2023

    2023 arXiv

  16. [24]

    Restormer: Efficient transformer for high-resolution image restoration,

    S. W. Zamir, A. Arora, S. Khan, M. Hayat, F. S. Khan, and M.-H. Yang, “Restormer: Efficient transformer for high-resolution image restoration,” in Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, 2022, pp. 5728–5739

    2022

  17. [25]

    Multi-stage progressive image restoration,

    S. W. Zamir, A. Arora, S. Khan, M. Hayat, F. S. Khan, M.-H. Yang, and L. Shao, “Multi-stage progressive image restoration,” in Proceedings of the IEEE/CVF conference on computer vision and pattern recognition , 2021, pp. 14 821–14 831

    2021

  18. [26]

    Hinet: Half instance normalization network for image restoration,

    L. Chen, X. Lu, J. Zhang, X. Chu, and C. Chen, “Hinet: Half instance normalization network for image restoration,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , 2021, pp. 182–192

    2021

  19. [27]

    Uformer: A general u-shaped transformer for image restoration,

    Z. Wang, X. Cun, J. Bao, W. Zhou, J. Liu, and H. Li, “Uformer: A general u-shaped transformer for image restoration,” in Proceedings of the IEEE/CVF conference on computer vision and pattern recognition , 2022, pp. 17 683–17 693

    2022

  20. [28]

    Maxim: Multi-axis mlp for image processing,

    Z. Tu, H. Talebi, H. Zhang, F. Yang, P. Milanfar, A. Bovik, and Y . Li, “Maxim: Multi-axis mlp for image processing,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , 2022, pp. 5769–5780

    2022

  21. [29]

    Nbnet: Noise basis learning for image denoising with subspace projection,

    S. Cheng, Y . Wang, H. Huang, D. Liu, H. Fan, and S. Liu, “Nbnet: Noise basis learning for image denoising with subspace projection,” in Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, 2021, pp. 4896–4906

    2021

  22. [30]

    Learning enriched features for real image restoration and enhancement,

    S. W. Zamir, A. Arora, S. Khan, M. Hayat, F. S. Khan, M.-H. Yang, and L. Shao, “Learning enriched features for real image restoration and enhancement,” in Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XXV

    2020

  23. [31]

    Springer, 2020, pp. 492–511

    2020

  24. [32]

    Dvdnet: A fast network for deep video denoising,

    M. Tassano, J. Delon, and T. Veit, “Dvdnet: A fast network for deep video denoising,” in 2019 IEEE International Conference on Image Processing (ICIP). IEEE, 2019, pp. 1805–1809

    2019

  25. [33]

    Bp-evd: Forward block-output propagation for efficient video denoising,

    P. K. Ostrowski, E. Katsaros, D. W˛ esierski, and A. Jezierska, “Bp-evd: Forward block-output propagation for efficient video denoising,” IEEE Transactions on Image Processing , vol. 31, pp. 3809–3824, 2022

    2022

  26. [34]

    Fastdvdnet: Towards real-time deep video denoising without flow estimation,

    M. Tassano, J. Delon, and T. Veit, “Fastdvdnet: Towards real-time deep video denoising without flow estimation,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), June 2020

    2020

  27. [35]

    Edvr: Video restoration with enhanced deformable convolutional networks,

    X. Wang, K. C. Chan, K. Yu, C. Dong, and C. Change Loy, “Edvr: Video restoration with enhanced deformable convolutional networks,” in Proceedings of the IEEE/CVF conference on computer vision and pattern recognition workshops , 2019, pp. 0–0

    2019

  28. [36]

    Supervised raw video denoising with a benchmark dataset on dynamic scenes,

    H. Yue, C. Cao, L. Liao, R. Chu, and J. Yang, “Supervised raw video denoising with a benchmark dataset on dynamic scenes,” in Proceedings 12 of the IEEE/CVF conference on computer vision and pattern recognition, 2020, pp. 2301–2310

    2020

  29. [37]

    Real-time streaming video denoising with bidirectional buffers,

    C. Qi, J. Chen, X. Yang, and Q. Chen, “Real-time streaming video denoising with bidirectional buffers,” in Proceedings of the 30th ACM International Conference on Multimedia , 2022, pp. 2758–2766

    2022

  30. [38]

    Real-time video denoising to reduce ionizing radiation exposure in fluoroscopic imaging,

    D. Van Veen, B. A. Duffy, L. Wang, K. Datta, T. Zhang, G. Zaharchuk, and E. Gong, “Real-time video denoising to reduce ionizing radiation exposure in fluoroscopic imaging,” in International Workshop on Ma- chine Learning for Medical Image Reconstruction . Springer, 2021, pp. 109–119

    2021

  31. [39]

    Bidirectional recurrent convolu- tional networks for multi-frame super-resolution,

    Y . Huang, W. Wang, and L. Wang, “Bidirectional recurrent convolu- tional networks for multi-frame super-resolution,” Advances in neural information processing systems , vol. 28, 2015

    2015

  32. [40]

    Efficient multi-stage video denoising with recurrent spatio-temporal fusion,

    M. Maggioni, Y . Huang, C. Li, S. Xiao, Z. Fu, and F. Song, “Efficient multi-stage video denoising with recurrent spatio-temporal fusion,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2021, pp. 3466–3475

    2021

  33. [41]

    Basicvsr: The search for essential components in video super-resolution and beyond,

    K. C. Chan, X. Wang, K. Yu, C. Dong, and C. C. Loy, “Basicvsr: The search for essential components in video super-resolution and beyond,” in Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, 2021, pp. 4947–4956

    2021

  34. [42]

    Learning task-oriented flows to mutually guide feature alignment in synthesized and real video denoising,

    J. Cao, Q. Wang, J. Liang, Y . Zhang, K. Zhang, R. Timofte, and L. V . Gool, “Learning task-oriented flows to mutually guide feature alignment in synthesized and real video denoising,” 2023

    2023

  35. [43]

    Remonet: Recurrent multi-output network for efficient video denoising,

    L. Xiang, J. Zhou, J. Liu, Z. Wang, H. Huang, J. Hu, J. Han, Y . Guo, and G. Ding, “Remonet: Recurrent multi-output network for efficient video denoising,” in Proceedings of the AAAI Conference on Artificial Intelligence, vol. 36, no. 3, 2022, pp. 2786–2794

    2022

  36. [44]

    Deep rnns for video denoising,

    X. Chen, L. Song, and X. Yang, “Deep rnns for video denoising,” in Applications of digital image processing XXXIX, vol. 9971. SPIE, 2016, pp. 573–582

    2016

  37. [45]

    Video denoising via empirical bayesian estimation of space-time patches,

    P. Arias and J.-M. Morel, “Video denoising via empirical bayesian estimation of space-time patches,” Journal of Mathematical Imaging and Vision, vol. 60, pp. 70–93, 2017

    2017

  38. [46]

    Video denoising, deblocking, and enhancement through separable 4-d nonlocal spatiotem- poral transforms,

    M. Maggioni, G. Boracchi, A. Foi, and K. Egiazarian, “Video denoising, deblocking, and enhancement through separable 4-d nonlocal spatiotem- poral transforms,” IEEE Transactions on Image Processing , vol. 21, no. 9, pp. 3952–3966, 2012

    2012

  39. [47]

    Burst photography for high dynamic range and low-light imaging on mobile cameras,

    S. W. Hasinoff, D. Sharlet, R. Geiss, A. Adams, J. T. Barron, F. Kainz, J. Chen, and M. Levoy, “Burst photography for high dynamic range and low-light imaging on mobile cameras,” ACM Transactions on Graphics (ToG), vol. 35, no. 6, pp. 1–12, 2016

    2016

  40. [48]

    Learning spatial and spatio- temporal pixel aggregations for image and video denoising,

    X. Xu, M. Li, W. Sun, and M.-H. Yang, “Learning spatial and spatio- temporal pixel aggregations for image and video denoising,” IEEE Transactions on Image Processing , vol. 29, pp. 7153–7165, 2020

    2020

  41. [49]

    Non-local video denoising by cnn,

    A. Davy, T. Ehret, J.-M. Morel, P. Arias, and G. Facciolo, “Non-local video denoising by cnn,” arXiv preprint arXiv:1811.12758 , 2018

    2018 arXiv

  42. [50]

    Patch craft: Video denoising by deep modeling and patch matching,

    G. Vaksman, M. Elad, and P. Milanfar, “Patch craft: Video denoising by deep modeling and patch matching,” in Proceedings of the IEEE/CVF International Conference on Computer Vision , 2021, pp. 2157–2166

    2021

  43. [51]

    Automated motion artifact correction for dynamic contrast-enhanced fluorescence imaging during open orthopedic surgery,

    Y . Tang, I. L. Gitajn, X. Cao, X. Han, J. T. Elliott, X. Yu, L. M. Bateman, B. S. Malskis, L. A. Fisher, J. M. Sin et al. , “Automated motion artifact correction for dynamic contrast-enhanced fluorescence imaging during open orthopedic surgery,” in Molecular-Guided Surgery: M...

    2023

  44. [52]

    Squeeze-and-excitation networks,

    J. Hu, L. Shen, and G. Sun, “Squeeze-and-excitation networks,” in Proceedings of the IEEE conference on computer vision and pattern recognition, 2018, pp. 7132–7141

    2018

  45. [53]

    Demystifying local vision transformer: Sparse connectivity, weight sharing, and dynamic weight,

    Q. Han, Z. Fan, Q. Dai, L. Sun, M.-M. Cheng, J. Liu, and J. Wang, “Demystifying local vision transformer: Sparse connectivity, weight sharing, and dynamic weight,” arXiv preprint arXiv:2106.04263, vol. 2, no. 3, 2021

    2021 arXiv

  46. [54]

    A convnet for the 2020s,

    Z. Liu, H. Mao, C.-Y . Wu, C. Feichtenhofer, T. Darrell, and S. Xie, “A convnet for the 2020s,” in Proceedings of the IEEE/CVF conference on computer vision and pattern recognition , 2022, pp. 11 976–11 986

    2022

  47. [55]

    Layer normalization,

    J. L. Ba, J. R. Kiros, and G. E. Hinton, “Layer normalization,” arXiv preprint arXiv:1607.06450, 2016

    2016 arXiv

  48. [56]

    U-net: Convolutional networks for biomedical image segmentation,

    O. Ronneberger, P. Fischer, and T. Brox, “U-net: Convolutional networks for biomedical image segmentation,” in Medical Image Computing and Computer-Assisted Intervention–MICCAI 2015: 18th International Conference, Munich, Germany, October 5-9, 2015, Proceedings, Part III

    2015

  49. [57]

    Springer, 2015, pp. 234–241

    2015

  50. [58]

    On the generaliza- tion of basicvsr++ to video deblurring and denoising,

    K. C. Chan, S. Zhou, X. Xu, and C. C. Loy, “On the generaliza- tion of basicvsr++ to video deblurring and denoising,” arXiv preprint arXiv:2204.05308, 2022

    2022 arXiv

  51. [59]

    Understanding deformable alignment in video super-resolution,

    K. C. Chan, X. Wang, K. Yu, C. Dong, and C. C. Loy, “Understanding deformable alignment in video super-resolution,” in Proceedings of the AAAI conference on artificial intelligence, vol. 35, no. 2, 2021, pp. 973– 981

    2021

  52. [60]

    Deformable convolutional networks,

    J. Dai, H. Qi, Y . Xiong, Y . Li, G. Zhang, H. Hu, and Y . Wei, “Deformable convolutional networks,” in Proceedings of the IEEE international conference on computer vision , 2017, pp. 764–773

    2017

  53. [61]

    Deformable convnets v2: More deformable, better results,

    X. Zhu, H. Hu, S. Lin, and J. Dai, “Deformable convnets v2: More deformable, better results,” in Proceedings of the IEEE/CVF conference on computer vision and pattern recognition , 2019, pp. 9308–9316

    2019

  54. [62]

    The 2017 davis challenge on video object segmen- tation,

    J. Pont-Tuset, F. Perazzi, S. Caelles, P. Arbeláez, A. Sorkine-Hornung, and L. Van Gool, “The 2017 davis challenge on video object segmen- tation,” arXiv:1704.00675, 2017

    2017 arXiv

  55. [63]

    Adam: A method for stochastic optimization,

    D. P. Kingma and J. Ba, “Adam: A method for stochastic optimization,” CoRR, vol. abs/1412.6980, 2014

    2014 arXiv

  56. [64]

    Two deterministic half-quadratic regularization algorithms for computed imaging,

    P. Charbonnier, L. Blanc-Feraud, G. Aubert, and M. Barlaud, “Two deterministic half-quadratic regularization algorithms for computed imaging,” in Proceedings of 1st International Conference on Image Processing, vol. 2, 1994, pp. 168–172 vol.2

    1994

  57. [65]

    Deep laplacian pyramid networks for fast and accurate super-resolution,

    W.-S. Lai, J.-B. Huang, N. Ahuja, and M.-H. Yang, “Deep laplacian pyramid networks for fast and accurate super-resolution,” in Proceedings of the IEEE conference on computer vision and pattern recognition , 2017, pp. 624–632

    2017

  58. [66]

    Image quality assessment: from error visibility to structural similarity,

    Z. Wang, A. C. Bovik, H. R. Sheikh, and E. P. Simoncelli, “Image quality assessment: from error visibility to structural similarity,” IEEE transactions on image processing , vol. 13, no. 4, pp. 600–612, 2004

    2004

  59. [67]

    The unreasonable effectiveness of deep features as a perceptual metric,

    R. Zhang, P. Isola, A. A. Efros, E. Shechtman, and O. Wang, “The unreasonable effectiveness of deep features as a perceptual metric,” in Proceedings of the IEEE conference on computer vision and pattern recognition, 2018, pp. 586–595

    2018

  60. [68]

    Noise2void-learning denoising from single noisy images,

    A. Krull, T.-O. Buchholz, and F. Jug, “Noise2void-learning denoising from single noisy images,” in Proceedings of the IEEE/CVF conference on computer vision and pattern recognition , 2019, pp. 2129–2137

    2019

  61. [69]

    Noise2Self: Blind denoising by self- supervision,

    J. Batson and L. Royer, “Noise2Self: Blind denoising by self- supervision,” in Proceedings of the 36th International Conference on Machine Learning , ser. Proceedings of Machine Learning Research, K. Chaudhuri and R. Salakhutdinov, Eds., vol. 97. PMLR, 09–15 Jun 2019, pp. 524–...

    2019

  62. [70]

    real_short.mp4

    "real_short.mp4" : baseline results on OL-Real

  63. [71]

    Short_sm25_lm_25.mp4

    "Short_sm25_lm_25.mp4" : baseline results on Lm = 25 = Sm simulated data

  64. [72]

    Short_sm50_lm_50.mp4

    "Short_sm50_lm_50.mp4" : all model results on Lm = 50 = Sm simulated data. In general the baseline models obtain better temporal consistency than either NafNet32 or BasicVSR++ C. Full video results are available at [12]. Reviewer link to dataset with video results: https://dat...

Pith tools

Reviewed August 12, 2026 · model on record in the stance chip above.