BiDeMem: Bidirectional Degradation Memory for Explainable Image Restoration
Pith reviewed 2026-06-29 03:55 UTC · model grok-4.3
The pith
Bidirectional memory retrieves degradation slots to support both image restoration and falsifiable explanations of degradation type.
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
Core claim
BiDeMem retrieves a compact top-k subset of memory slots from a query built on restoration features and input statistics; the same slot identities drive the restoration path at inference and a training-only forward-degradation explanation path. Intervention probes confirm that this arrangement preserves restoration quality while increasing measurable sensitivity to wrong-prior and native-prior swaps, outperforming correction-head, dense-query-prior, and static-global-prior variants by 0.2588, 0.2586, and 0.2839 dB respectively.
What carries the argument
Bidirectional degradation memory: query-retrieved top-k slots whose identities serve both a restoration path and a training-only explanation path.
If this is right
- The full bidirectional model outperforms a correction-head variant by 0.2588 dB.
- It outperforms a dense-query-prior variant by 0.2586 dB.
- It outperforms a static-global-prior variant by 0.2839 dB.
- Strong residual supervision and a wider degradation head both fall short of the bidirectional memory model.
- Intervention on slot identity raises wrong-prior and native-prior sensitivity while leaving restoration quality intact.
Where Pith is reading between the lines
- If slot identities reliably encode degradation type, the same memory could be used after training to diagnose which degradation a model is mishandling on a given input.
- The training-only explanation path offers a template for adding interpretability modules that impose no cost at inference time.
- The retrieval-plus-intervention design could be tested on other conditional vision tasks where the semantic content of a condition must be verified independently of final task accuracy.
Load-bearing premise
That the identity of a retrieved memory slot can be treated as a semantically meaningful representation of degradation type so that swapping it produces interpretable, measurable changes rather than mere capacity effects.
What would settle it
An experiment that forces the model to use a wrong-prior slot on images whose degradation type is known in advance and checks whether PSNR or output statistics remain statistically unchanged from the correct-prior case.
Figures
read the original abstract
Degradation-aware prompts, conditions, and latent priors are increasingly used in image restoration, yet they are usually judged by a single endpoint: whether the restored image obtains higher PSNR. This is a weak test of semantics. A condition can help by adding capacity, acting as a global correction bias, or exploiting dataset shortcuts, without becoming an interpretable degradation prior. We propose BiDeMem, a bidirectional degradation memory for explainable image restoration. A query built from restoration features and input statistics retrieves a compact top-k subset of memory slots. The same selected slot identity supports the restoration path at inference time and a training-only forward-degradation explanation path. The study centers on verifiability in a controlled multi-degradation NAFNet setting. New controls separate the gain from a correction head alone, a dense query prior, and a static global prior: these variants are 0.2588, 0.2586, and 0.2839 dB below BiRank, respectively. Strong residual supervision and a wider degradation head also remain below the full bidirectional memory model. Intervention probes show that BiRank preserves restoration quality while increasing wrong-prior and native-prior sensitivity, framing degradation memory as both a restoration module and a falsifiable explanation mechanism.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper proposes BiDeMem, a bidirectional degradation memory for explainable image restoration. A query derived from restoration features and input statistics retrieves a top-k subset of memory slots; the slot identities are used both in the restoration forward pass and in a training-only forward-degradation explanation path. Experiments are conducted in a controlled multi-degradation NAFNet setting. The full bidirectional model is reported to outperform three explicit variant baselines (correction head, dense query prior, static global prior) by 0.2588 dB, 0.2586 dB, and 0.2839 dB respectively. Intervention probes (wrong-prior and native-prior swaps) are claimed to increase sensitivity while preserving restoration quality, positioning the memory as both a performance module and a falsifiable explanation mechanism.
Significance. If the central claim holds—that retrieved memory slot identities constitute a semantically meaningful and falsifiable representation of degradation type rather than an additional learned bias—the work would provide a concrete methodological advance in evaluating degradation-aware priors beyond endpoint PSNR. The explicit construction of multiple ablation variants and the use of intervention probes are positive design choices that ground the comparison. The small reported margins and absence of supporting validation details, however, limit the immediate impact.
major comments (3)
- [Abstract] Abstract: the reported PSNR gaps (0.2588 dB, 0.2586 dB, 0.2839 dB) are presented without error bars, standard deviations, or statistical significance tests across multiple runs or dataset splits; this directly affects the claim that the bidirectional model is superior to the three named variants.
- [Abstract] Abstract (and method description): no specification is given for memory-slot initialization, population rules, update mechanism, or the precise contents of each slot; without these details the assertion that slot identities function as a falsifiable degradation representation cannot be evaluated.
- [Abstract] Abstract (intervention probes paragraph): the claim that BiRank 'increases wrong-prior and native-prior sensitivity' while preserving quality is consistent with the prior acting simply as an extra learned input channel; the manuscript supplies no auxiliary check (e.g., clustering of slot identities against known degradation categories or alignment metrics) that would distinguish semantic representation from capacity augmentation.
minor comments (1)
- [Abstract] The term 'BiRank' appears in the abstract without prior definition; clarify its relation to BiDeMem.
Simulated Author's Rebuttal
We thank the referee for the thoughtful and constructive review. The comments highlight important aspects of statistical rigor, methodological transparency, and validation of semantic claims. We address each major comment below and commit to revisions that strengthen the manuscript without altering its core contributions.
read point-by-point responses
-
Referee: [Abstract] Abstract: the reported PSNR gaps (0.2588 dB, 0.2586 dB, 0.2839 dB) are presented without error bars, standard deviations, or statistical significance tests across multiple runs or dataset splits; this directly affects the claim that the bidirectional model is superior to the three named variants.
Authors: We agree that reporting variability and significance is essential for robust claims. The current results reflect single-run evaluations in the controlled NAFNet setting. In the revision we will rerun all experiments over five independent random seeds, report mean PSNR with standard deviations, and include paired t-test p-values comparing BiDeMem against each baseline. This will directly address the concern about the small margins. revision: yes
-
Referee: [Abstract] Abstract (and method description): no specification is given for memory-slot initialization, population rules, update mechanism, or the precise contents of each slot; without these details the assertion that slot identities function as a falsifiable degradation representation cannot be evaluated.
Authors: The method section describes a fixed-size learnable memory bank whose slots are retrieved by top-k similarity and whose identities are used in both paths, but we concede that explicit initialization, population, and update details are insufficiently specified. We will add a dedicated subsection detailing: random normal initialization scaled by slot dimension, fixed population of 128 slots, end-to-end gradient updates with no explicit population rule beyond back-propagation, and slot contents as 256-dimensional embedding vectors. These additions will make the falsifiability claim evaluable. revision: yes
-
Referee: [Abstract] Abstract (intervention probes paragraph): the claim that BiRank 'increases wrong-prior and native-prior sensitivity' while preserving quality is consistent with the prior acting simply as an extra learned input channel; the manuscript supplies no auxiliary check (e.g., clustering of slot identities against known degradation categories or alignment metrics) that would distinguish semantic representation from capacity augmentation.
Authors: The intervention probes (wrong-prior and native-prior swaps) already show that swapping slot identities alters restoration behavior in a degradation-dependent way while quality remains comparable, which goes beyond generic capacity. Nevertheless, we accept that auxiliary checks would better separate semantic representation from bias. In revision we will add (i) k-means clustering of learned slot identities colored by ground-truth degradation category and (ii) normalized mutual information between slot assignments and degradation labels, providing quantitative evidence that the memory captures more than an extra channel. revision: yes
Circularity Check
No significant circularity in derivation chain
full rationale
The paper's central claims rest on empirical comparisons to explicitly constructed variant architectures (correction head, dense query prior, static global prior) and intervention probes within a controlled NAFNet multi-degradation setting. These provide independent external benchmarks rather than reducing to self-definition or fitted inputs renamed as predictions. No equations or self-citations are shown that force the explanatory status of memory slots by construction; the bidirectional path is tested for sensitivity while preserving restoration quality, keeping the argument self-contained against those controls.
Axiom & Free-Parameter Ledger
free parameters (1)
- top-k
axioms (1)
- domain assumption Memory slots populated from restoration features and input statistics can represent distinct degradation types
invented entities (1)
-
Bidirectional degradation memory slots
no independent evidence
Reference graph
Works this paper leans on
-
[1]
Beyond a gaussian denoiser: Residual learning of deep cnn for image denoising.IEEE transactions on image processing, 26(7):3142–3155, 2017
Kai Zhang, Wangmeng Zuo, Yunjin Chen, Deyu Meng, and Lei Zhang. Beyond a gaussian denoiser: Residual learning of deep cnn for image denoising.IEEE transactions on image processing, 26(7):3142–3155, 2017. 1, 2
2017
-
[2]
Benchmarking single- image dehazing and beyond.IEEE transactions on image processing, 28(1):492–505, 2018
Boyi Li, Wenqi Ren, Dengpan Fu, Dacheng Tao, Dan Feng, Wenjun Zeng, and Zhangyang Wang. Benchmarking single- image dehazing and beyond.IEEE transactions on image processing, 28(1):492–505, 2018. 2, 5
2018
-
[3]
Deep joint rain detection and removal from a single image
Wenhan Yang, Robby T Tan, Jiashi Feng, Jiaying Liu, Zong- ming Guo, and Shuicheng Yan. Deep joint rain detection and removal from a single image. InProceedings of the IEEE conference on computer vision and pattern recogni- tion, pages 1357–1366, 2017. 2, 5
2017
-
[4]
Multi-stage progressive image restoration
Syed Waqas Zamir, Aditya Arora, Salman Khan, Munawar Hayat, Fahad Shahbaz Khan, Ming-Hsuan Yang, and Ling Shao. Multi-stage progressive image restoration. InPro- ceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 14821–14831, 2021. 3
2021
-
[5]
Restormer: Efficient transformer for high-resolution image restoration
Syed Waqas Zamir, Aditya Arora, Salman Khan, Mu- nawar Hayat, Fahad Shahbaz Khan, and Ming-Hsuan Yang. Restormer: Efficient transformer for high-resolution image restoration. InProceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 5728–5739,
-
[6]
Simple baselines for image restoration
Liangyu Chen, Xiaojie Chu, Xiangyu Zhang, and Jian Sun. Simple baselines for image restoration. InEuropean confer- ence on computer vision, pages 17–33. Springer, 2022. 1, 3, 5
2022
-
[7]
All-in-one image restoration for unknown corruption
Boyun Li, Xiao Liu, Peng Hu, Zhongqin Wu, Jiancheng Lv, and Xi Peng. All-in-one image restoration for unknown corruption. InProceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 17452– 17462, 2022. 1, 3, 7
2022
-
[8]
Transweather: Transformer-based restoration of im- ages degraded by adverse weather conditions
Jeya Maria Jose Valanarasu, Rajeev Yasarla, and Vishal M Patel. Transweather: Transformer-based restoration of im- ages degraded by adverse weather conditions. InProceed- ings of the IEEE/CVF conference on computer vision and pattern recognition, pages 2353–2363, 2022. 3
2022
-
[9]
Promptir: Prompting for all-in- one image restoration.Advances in neural information pro- cessing systems, 36:71275–71293, 2023
Vaishnav Potlapalli, Syed Waqas Zamir, Salman H Khan, and Fahad Shahbaz Khan. Promptir: Prompting for all-in- one image restoration.Advances in neural information pro- cessing systems, 36:71275–71293, 2023. 1, 3, 4, 7
2023
-
[10]
Ffdnet: Toward a fast and flexible solution for cnn-based image denoising
Kai Zhang, Wangmeng Zuo, and Lei Zhang. Ffdnet: Toward a fast and flexible solution for cnn-based image denoising. IEEE Transactions on Image Processing, 27(9):4608–4622,
-
[11]
Toward convolutional blind denoising of real pho- tographs
Shi Guo, Zifei Yan, Kai Zhang, Wangmeng Zuo, and Lei Zhang. Toward convolutional blind denoising of real pho- tographs. InProceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 1712–1722,
-
[12]
Noise2Noise: Learning Image Restoration without Clean Data
Jaakko Lehtinen, Jacob Munkberg, Jon Hasselgren, Samuli Laine, Tero Karras, Miika Aittala, and Timo Aila. Noise2noise: Learning image restoration without clean data. arXiv preprint arXiv:1803.04189, 2018. 2
work page internal anchor Pith review Pith/arXiv arXiv 2018
-
[13]
Noise2void-learning denoising from single noisy images
Alexander Krull, Tim-Oliver Buchholz, and Florian Jug. Noise2void-learning denoising from single noisy images. In Proceedings of the IEEE/CVF conference on computer vi- sion and pattern recognition, pages 2129–2137, 2019. 2
2019
-
[14]
Single image haze removal using dark channel prior.IEEE transactions on pat- tern analysis and machine intelligence, 33(12):2341–2353,
Kaiming He, Jian Sun, and Xiaoou Tang. Single image haze removal using dark channel prior.IEEE transactions on pat- tern analysis and machine intelligence, 33(12):2341–2353,
-
[15]
Dehazenet: An end-to-end system for single image haze removal.IEEE transactions on image process- ing, 25(11):5187–5198, 2016
Bolun Cai, Xiangmin Xu, Kui Jia, Chunmei Qing, and Dacheng Tao. Dehazenet: An end-to-end system for single image haze removal.IEEE transactions on image process- ing, 25(11):5187–5198, 2016. 2
2016
-
[16]
Aod-net: All-in-one dehazing network
Boyi Li, Xiulian Peng, Zhangyang Wang, Jizheng Xu, and Dan Feng. Aod-net: All-in-one dehazing network. InPro- ceedings of the IEEE international conference on computer vision, pages 4770–4778, 2017. 2
2017
-
[17]
Density-aware single image de-raining using a multi-stream dense network
He Zhang and Vishal M Patel. Density-aware single image de-raining using a multi-stream dense network. InProceed- ings of the IEEE conference on computer vision and pattern recognition, pages 695–704, 2018. 2
2018
-
[18]
Removing rain from single images via a deep detail network
Xueyang Fu, Jiabin Huang, Delu Zeng, Yue Huang, Xinghao Ding, and John Paisley. Removing rain from single images via a deep detail network. InProceedings of the IEEE con- ference on computer vision and pattern recognition, pages 3855–3863, 2017. 2
2017
-
[19]
Recurrent squeeze-and-excitation context aggregation net for single image deraining
Xia Li, Jianlong Wu, Zhouchen Lin, Hong Liu, and Hongbin Zha. Recurrent squeeze-and-excitation context aggregation net for single image deraining. InProceedings of the Eu- ropean conference on computer vision (ECCV), pages 254– 269, 2018. 2
2018
-
[20]
Spatial attentive single-image deraining with a high quality real rain dataset
Tianyu Wang, Xin Yang, Ke Xu, Shaozhe Chen, Qiang Zhang, and Rynson WH Lau. Spatial attentive single-image deraining with a high quality real rain dataset. InProceed- ings of the IEEE/CVF conference on computer vision and pattern recognition, pages 12270–12279, 2019. 2
2019
-
[21]
Deblurgan: Blind mo- tion deblurring using conditional adversarial networks
Orest Kupyn, V olodymyr Budzan, Mykola Mykhailych, Dmytro Mishkin, and Ji ˇr´ı Matas. Deblurgan: Blind mo- tion deblurring using conditional adversarial networks. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 8183–8192, 2018. 2 8
2018
-
[22]
Deblurgan-v2: Deblurring (orders-of-magnitude) faster and better
Orest Kupyn, Tetiana Martyniuk, Junru Wu, and Zhangyang Wang. Deblurgan-v2: Deblurring (orders-of-magnitude) faster and better. InProceedings of the IEEE/CVF inter- national conference on computer vision, pages 8878–8887,
-
[23]
Enhanced deep residual networks for single image super-resolution
Bee Lim, Sanghyun Son, Heewon Kim, Seungjun Nah, and Kyoung Mu Lee. Enhanced deep residual networks for single image super-resolution. InProceedings of the IEEE confer- ence on computer vision and pattern recognition workshops, pages 136–144, 2017. 2
2017
-
[24]
Image super-resolution using very deep residual channel attention networks
Yulun Zhang, Kunpeng Li, Kai Li, Lichen Wang, Bineng Zhong, and Yun Fu. Image super-resolution using very deep residual channel attention networks. InProceedings of the European conference on computer vision (ECCV), pages 286–301, 2018. 2
2018
-
[25]
Learning enriched features for real image restoration and enhancement
Syed Waqas Zamir, Aditya Arora, Salman Khan, Munawar Hayat, Fahad Shahbaz Khan, Ming-Hsuan Yang, and Ling Shao. Learning enriched features for real image restoration and enhancement. InEuropean conference on computer vi- sion, pages 492–511. Springer, 2020. 3
2020
-
[26]
Swinir: Image restoration us- ing swin transformer
Jingyun Liang, Jiezhang Cao, Guolei Sun, Kai Zhang, Luc Van Gool, and Radu Timofte. Swinir: Image restoration us- ing swin transformer. InProceedings of the IEEE/CVF inter- national conference on computer vision, pages 1833–1844,
-
[27]
Uformer: A general u-shaped transformer for image restoration
Zhendong Wang, Xiaodong Cun, Jianmin Bao, Wengang Zhou, Jianzhuang Liu, and Houqiang Li. Uformer: A general u-shaped transformer for image restoration. InProceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 17683–17693, 2022. 3
2022
-
[28]
Learning to see in the dark
Chen Chen, Qifeng Chen, Jia Xu, and Vladlen Koltun. Learning to see in the dark. InProceedings of the IEEE con- ference on computer vision and pattern recognition, pages 3291–3300, 2018. 3
2018
-
[29]
Zero-reference deep curve estimation for low-light image enhancement
Chunle Guo, Chongyi Li, Jichang Guo, Chen Change Loy, Junhui Hou, Sam Kwong, and Runmin Cong. Zero-reference deep curve estimation for low-light image enhancement. In Proceedings of the IEEE/CVF conference on computer vi- sion and pattern recognition, pages 1780–1789, 2020. 3
2020
-
[30]
Film: Visual reasoning with a general conditioning layer
Ethan Perez, Florian Strub, Harm De Vries, Vincent Du- moulin, and Aaron Courville. Film: Visual reasoning with a general conditioning layer. InProceedings of the AAAI con- ference on artificial intelligence, 2018. 3
2018
-
[31]
Mod- ulating early visual processing by language.Advances in neural information processing systems, 30, 2017
Harm De Vries, Florian Strub, J ´er´emie Mary, Hugo Larochelle, Olivier Pietquin, and Aaron C Courville. Mod- ulating early visual processing by language.Advances in neural information processing systems, 30, 2017. 4
2017
-
[32]
Dynamic filter networks.Advances in neural informa- tion processing systems, 29, 2016
Xu Jia, Bert De Brabandere, Tinne Tuytelaars, and Luc V Gool. Dynamic filter networks.Advances in neural informa- tion processing systems, 29, 2016. 4
2016
-
[33]
Jason Weston, Sumit Chopra, and Antoine Bordes. Memory networks.arXiv preprint arXiv:1410.3916, 2014. 4
work page internal anchor Pith review Pith/arXiv arXiv 2014
-
[34]
End- to-end memory networks.Advances in neural information processing systems, 28, 2015
Sainbayar Sukhbaatar, Jason Weston, Rob Fergus, et al. End- to-end memory networks.Advances in neural information processing systems, 28, 2015. 4
2015
-
[35]
Alex Graves, Greg Wayne, and Ivo Danihelka. Neural turing machines.arXiv preprint arXiv:1410.5401, 2014. 4
work page internal anchor Pith review Pith/arXiv arXiv 2014
-
[36]
Hybrid computing using a neural network with dynamic external memory.Nature, 538(7626): 471–476, 2016
Alex Graves, Greg Wayne, Malcolm Reynolds, Tim Harley, Ivo Danihelka, Agnieszka Grabska-Barwi ´nska, Ser- gio G ´omez Colmenarejo, Edward Grefenstette, Tiago Ra- malho, John Agapiou, et al. Hybrid computing using a neural network with dynamic external memory.Nature, 538(7626): 471–476, 2016. 4
2016
-
[37]
Object- centric learning with slot attention.Advances in neural in- formation processing systems, 33:11525–11538, 2020
Francesco Locatello, Dirk Weissenborn, Thomas Un- terthiner, Aravindh Mahendran, Georg Heigold, Jakob Uszkoreit, Alexey Dosovitskiy, and Thomas Kipf. Object- centric learning with slot attention.Advances in neural in- formation processing systems, 33:11525–11538, 2020. 4
2020
-
[38]
Large memory layers with product keys.Advances in Neural Information Processing Systems, 32, 2019
Guillaume Lample, Alexandre Sablayrolles, Marc’Aurelio Ranzato, Ludovic Denoyer, and Herv´e J´egou. Large memory layers with product keys.Advances in Neural Information Processing Systems, 32, 2019. 4
2019
-
[39]
Grad-cam: visual explanations from deep networks via gradient-based localization.International journal of com- puter vision, 128(2):336–359, 2020
Ramprasaath R Selvaraju, Michael Cogswell, Abhishek Das, Ramakrishna Vedantam, Devi Parikh, and Dhruv Ba- tra. Grad-cam: visual explanations from deep networks via gradient-based localization.International journal of com- puter vision, 128(2):336–359, 2020. 4
2020
-
[40]
Axiomatic attribution for deep networks
Mukund Sundararajan, Ankur Taly, and Qiqi Yan. Axiomatic attribution for deep networks. InInternational conference on machine learning, pages 3319–3328. PMLR, 2017. 4
2017
-
[41]
Interpretability be- yond feature attribution: Quantitative testing with concept activation vectors (tcav)
Been Kim, Martin Wattenberg, Justin Gilmer, Carrie Cai, James Wexler, Fernanda Viegas, et al. Interpretability be- yond feature attribution: Quantitative testing with concept activation vectors (tcav). InInternational conference on ma- chine learning, pages 2668–2677. PMLR, 2018. 4
2018
-
[42]
” why should i trust you?” explaining the predictions of any classifier
Marco Tulio Ribeiro, Sameer Singh, and Carlos Guestrin. ” why should i trust you?” explaining the predictions of any classifier. InProceedings of the 22nd ACM SIGKDD interna- tional conference on knowledge discovery and data mining, pages 1135–1144, 2016. 4
2016
-
[43]
A unified approach to interpreting model predictions.Advances in neural informa- tion processing systems, 30, 2017
Scott M Lundberg and Su-In Lee. A unified approach to interpreting model predictions.Advances in neural informa- tion processing systems, 30, 2017. 4
2017
-
[44]
Counterfactual Explanations without Opening the Black Box: Automated Decisions and the GDPR
Sandra Wachter, Brent Mittelstadt, and Chris Russell. Counterfactual explanations without opening the black box: Automated decisions and the gdpr.arXiv preprint arXiv:1711.00399, 2017. 4
work page internal anchor Pith review Pith/arXiv arXiv 2017
-
[45]
Explanations based on the missing: Towards contrastive ex- planations with pertinent negatives.Advances in neural in- formation processing systems, 31, 2018
Amit Dhurandhar, Pin-Yu Chen, Ronny Luss, Chun-Chen Tu, Paishun Ting, Karthikeyan Shanmugam, and Payel Das. Explanations based on the missing: Towards contrastive ex- planations with pertinent negatives.Advances in neural in- formation processing systems, 31, 2018. 4
2018
-
[46]
A database of human segmented natural images and its application to evaluating segmentation algorithms and measuring ecological statistics
David Martin, Charless Fowlkes, Doron Tal, and Jitendra Malik. A database of human segmented natural images and its application to evaluating segmentation algorithms and measuring ecological statistics. InProceedings of the IEEE International Conference on Computer Vision, pages 416– 423, 2001. 5
2001
-
[47]
Sin- gle image super-resolution from transformed self-exemplars
Jia-Bin Huang, Abhishek Singh, and Narendra Ahuja. Sin- gle image super-resolution from transformed self-exemplars. InProceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 5197–5206, 2015. 5 9
2015
discussion (0)
Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.