REVIEW 4 major objections 5 minor 31 references
Human Aligned Compression for Robust Models
T0 review · 4 major / 5 minor · reviewed 2026-08-16 · deepseek-v4-flash
Pith's one-line read Learned image compression that matches human perception defends image classifiers against adversarial attacks more effectively than JPEG, especially for Vision Transformers.
desk verdict Learned compression beats JPEG as a black-box defense, especially for ViTs, but the white-box claim is oversold and the learned-vs-JPEG comparison is confounded by bitrate. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The central objects are the two learned lossy codecs: HiFiC, a GAN-based compressor trained to match human perception, and ELIC, an efficient learned codec using unevenly grouped space-channel contextual adaptive coding, with JPEG as the classical baseline. Because both learned models are differentiable, attacks can propagate gradients through them in white-box mode, making the defense evaluation conservative. The mechanism carrying the argument is the lossy bottleneck: it discards high-frequency or perceptually unimportant detail, which is where adversarial perturbations tend to live, while retaining features the classifier relies on. Sequential compression repeatedly applies this bottleneck to erase residual adversarial signal.
What would settle it
Run a white-box PGD attack with, say, 200 iterations and 10 random restarts, propagating gradients through the differentiable HiFiC or ELIC pipeline, and check whether ViT accuracy collapses toward the no-defense baseline; a collapse would falsify the claim of substantial effectiveness in white-box settings.
Extended reading notes
Core claim
On ImageNet and Imagenette subsets, adversarial examples generated by FGSM, iFGSM, PGD, CW, and DeepFool lose much of their power when the input is compressed before classification. Learned codecs HiFiC and ELIC preserve near-baseline accuracy under black-box attacks at all tested epsilon values, while JPEG degrades for ViT at stronger attacks; under white-box attacks, where gradients flow through a differentiable approximation of the defense, all defenses weaken but learned compression still provides substantial effectiveness. Sequential compression increases defense accuracy with each round, with JPEG converging fastest and ELIC and HiFiC improving but at a cost to clean accuracy at low quality settings, and the overhead is small: about 14 ms per image for learned codecs versus 5 ms bare and 8 ms with JPEG on a single GPU.
Load-bearing premise
The paper assumes that 10-step gradient attacks with a differentiable approximation of the compression capture what a determined attacker can do, so a stronger attack with more iterations and restarts could break the defense.
Editorial extensions
If this is right
- Learned compression can be dropped into an existing classifier as a preprocessing step without retraining or modifying the model.
- For ViT classifiers, which the paper finds less robust to begin with, HiFiC and ELIC recover almost all clean accuracy under black-box iFGSM and PGD attacks, outperforming JPEG at every tested strength.
- Sequential compression is a tunable dial: more cycles give more robustness, but for HiFiC low quality it quickly degrades clean accuracy, while JPEG stays near baseline for 50 cycles.
- The measured overhead of single-digit to tens of milliseconds per image makes the defense practical for inference pipelines.
Reading between the lines
- A natural next test is combining compression with adversarially trained models; since compression removes non-robust features, it may lower the accuracy cost that adversarial training usually pays on clean inputs.
- The paper's white-box attacks use 10-step iFGSM/PGD and 50-step CW/DeepFool; stronger adaptive attacks with restarts and more iterations would show how much of the apparent robustness is gradient masking rather than true invariance.
- If the mechanism is really the perceptual bottleneck, then other human-aligned input transformations such as learned denoising or super-resolution should confer similar defense, which is testable with the same experimental setup.
- Sequential compression changes the lossy artifact distribution per round; understanding which artifacts the classifier keys on could inform future codecs designed explicitly for downstream task robustness.
Signed reviews
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper investigates learned lossy image compression (HiFiC and ELIC) as a preprocessing defense against adversarial attacks, comparing them with JPEG on ImageNet subsets using ResNet50 and ViT-B/16. The authors report that learned compression outperforms JPEG especially for ViT, that defenses remain partially effective in white-box settings where gradients are propagated through differentiable approximations of the compression, and that repeated sequential compression cycles increase robustness. The paper includes detailed accuracy tables for FGSM, iFGSM, PGD, CW, and DeepFool attacks, quality ablations, computational overhead measurements, and sequential-compression experiments on Imagenette and a 1000-image ImageNet sample.
Significance. If the white-box robustness claim were supported, the paper would provide a practical, computationally cheap input-transformation defense that also works for ViT, for which compression-based defenses are comparatively understudied. The manuscript is strong on breadth: it compares three compression families, two architectures, five attacks, and both adaptive and non-adaptive settings, and it reports per-setting accuracy tables, quality ablations, bitrate measurements, and inference-time overhead. The code link is provided. The main significance is conditional on the robustness evaluation being trustworthy; as it stands, the white-box results are generated by weak, low-iteration attacks through approximate gradients, so the headline claim of 'substantial effectiveness' in white-box settings is not yet established.
major comments (4)
- [§3.2, Table 3, §5] The white-box evaluation is too weak to support the abstract claim of 'substantial effectiveness' against attackers who can access the defense. iFGSM and PGD are run for only 10 steps with no restarts, and CW/DeepFool for 50 steps, through differentiable approximations of inherently non-differentiable compression pipelines. This is exactly the gradient-masking scenario: the attack's gradients may be uninformative because they are computed through a proxy, so high defended accuracy can reflect attack failure rather than genuine robustness. The paper itself cites Uesato et al. [30] on the dangers of evaluating against weak attacks. Please add stronger adaptive attacks (e.g., PGD with 100 or more steps and multiple random restarts, AutoAttack, or BPDA with expectation over transformation) and report convergence or attack success as a function of iterations.
- [§3.4, Table 1, §4.2] The learned-versus-JPEG comparison confounds compression method with compression strength. The selected operating points are JPEG q=25 at 0.78 BPP, ELIC 0016 at 0.14 BPP, and HiFiC low at 0.15 BPP, so the learned methods compress roughly 5x more aggressively than the chosen JPEG setting. The observed advantage of HiFiC and ELIC over JPEG may therefore reflect stronger compression rather than 'human-aligned' preservation of semantic content. To support the paper's central comparison, the authors should either use bitrate-matched operating points (e.g., JPEG at the lowest available quality near 0.15 BPP, or otherwise interpolate/extrapolate) or explicitly analyze how much of the performance gap remains after controlling for BPP.
- [§3.2, Figures 2-4] The 'black-box' results are not black-box in the standard adversarial-robustness sense. The attacks are generated on the classifier alone without passing gradients through the defense, which is an attack that is unaware of the defense, not an attack against a system whose internals are hidden. This terminology materially inflates the apparent robustness: a defense that simply destroys all low-amplitude structure will score highly under such 'black-box' evaluation. Please rename these settings 'non-adaptive' or 'defense-unaware,' or run proper black-box attacks (e.g., transfer attacks from surrogate models or query-based attacks) if the black-box claim is intended.
- [§4.5, Figures 8-9, Tables 9-10] The abstract's claim that sequential compression 'significantly enhances defense efficacy while maintaining classification performance' is not supported for the learned methods. For HiFiC low, seven compression cycles reduce clean accuracy to 0.987 and adversarial accuracy at epsilon 8/255 reaches only 0.385; for ELIC 0016, the corresponding adversarial accuracy is 0.41 after seven cycles. Only JPEG maintains both high clean accuracy (0.994) and high robust accuracy (0.96+) across many iterations. The sequential-compression conclusion should be restricted to settings where clean accuracy is preserved, or the claim should be reformulated to emphasize the JPEG result.
minor comments (5)
- [§2.1] The text reads 'Projected Gradient Decent'; this should be 'Projected Gradient Descent'.
- [Table 8 caption] Table 8 is captioned as comparing HiFiC quality levels, but the rows are ELIC checkpoints (0004 through 0450). This appears to be a copy-paste error from Table 7.
- [Figure 1 caption] The caption labels panel (a) as 'ELIC low quality' while the panel list refers to 'ELIC 0004'; the naming is inconsistent and should be unified.
- [§4.2.1, Figure 4] The ImageNet results are based on 1000 randomly sampled images, but no confidence intervals, seeds, or repeated sampling are reported. Given the visible jaggedness and the claim of generalization to ImageNet, error bars or a fixed seed/description of sampling would aid interpretability.
- [§3.4, §4.4] The defense quality levels are selected using ablations on the same datasets used for the main results, which can mildly overestimate the selected configurations' performance. A brief acknowledgment or a small held-out tuning set would strengthen the evaluation.
Circularity Check
No circularity: the paper is an empirical comparison whose claims rest on reported measurements and external baselines, not on definitions, fitted parameters, or self-citations.
full rationale
The paper's central claims are empirical evaluations, not derivations. The finding that HiFiC and ELIC preserve classifier accuracy under adversarial attacks better than JPEG in these experiments is a measured result reported in Tables 4-5 and Figures 2-11; no equation defines a predicted quantity in terms of the defense's own outputs, so nothing is true by construction. The compression strengths were selected using ablations on the same datasets used for the main results (Section 3.4, Figures 5-7), which is a hyperparameter-selection and comparability concern rather than circularity, especially because the chosen settings are not bitrate-matched (Table 1). The full ablation tables are provided, and the selection is not presented as an independent prediction. The white-box evaluation uses short, low-iteration attacks and differentiable approximations of non-differentiable compressors, which may overstate robustness, but the Conclusion explicitly acknowledges the scope: 'There are some limitations, as we only experiment with gradient-based attacks.' That is a threat-model validity caveat, not a logical circle. Citations to prior work (Dziugaite et al., Shin and Song, HiFiC, ELIC) are external baselines and do not carry the paper's load-bearing argument, and no uniqueness theorem or ansatz is imported from the authors' own prior work. No fitted parameter is relabeled as a prediction, and no self-definitional reduction is present.
Assumptions & free parameters
free parameters (3)
- JPEG quality level =
25.0
- HiFiC weight level =
low (target BPP 0.14)
- ELIC checkpoint =
0016 (BPP 0.14)
assumptions (4)
- domain assumption Kornia's differentiable JPEG approximation faithfully represents JPEG for adaptive attack gradient computation
- domain assumption White-box attacks with 10 PGD/iFGSM iterations and 50 CW/DeepFool steps are sufficient to approximate the worst-case attack
- domain assumption HiFiC and ELIC are 'human-aligned' and preserve semantically meaningful content
- domain assumption Imagenette and the 1000 random ImageNet images are representative
Cite this review
Pith. "Pith review of Human Aligned Compression for Robust Models." pith.science (2026). https://pith.science/paper/VYUNUXM4
@misc{pith2026250412255,
author = {Pith},
title = {Pith review of: Human Aligned Compression for Robust Models},
year = {2026},
howpublished = {\url{https://pith.science/paper/VYUNUXM4}},
note = {Machine review of arXiv:2504.12255}
}
read the original abstract
Adversarial attacks on image models threaten system robustness by introducing imperceptible perturbations that cause incorrect predictions. We investigate human-aligned learned lossy compression as a defense mechanism, comparing two learned models (HiFiC and ELIC) against traditional JPEG across various quality levels. Our experiments on ImageNet subsets demonstrate that learned compression methods outperform JPEG, particularly for Vision Transformer architectures, by preserving semantically meaningful content while removing adversarial noise. Even in white-box settings where attackers can access the defense, these methods maintain substantial effectiveness. We also show that sequential compression--applying rounds of compression/decompression--significantly enhances defense efficacy while maintaining classification performance. Our findings reveal that human-aligned compression provides an effective, computationally efficient defense that protects the image features most relevant to human and machine understanding. It offers a practical approach to improving model robustness against adversarial threats.
Figures
Figures from the paper (12 more)
Reference graph
Works this paper leans on
-
[30]
Adversarial risk and the dangers of evaluating against weak attacks, 2018
Jonathan Uesato, Brendan O’Donoghue, Aaron van den Oord, and Pushmeet Kohli. Adversarial risk and the dangers of evaluating against weak attacks, 2018. 8
work page 2018
-
[1]
Generative adversar- ial networks for extreme learned image compression
Eirikur Agustsson, Michael Tschannen, Fabian Mentzer, Radu Timofte, and Luc Van Gool. Generative adversar- ial networks for extreme learned image compression. In Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), 2019. 2
work page 2019
-
[2]
Towards evaluating the robustness of neural networks, 2017
Nicholas Carlini and David Wagner. Towards evaluating the robustness of neural networks, 2017. 3
work page 2017
-
[3]
Learned image compression with discretized gaussian mixture likelihoods and attention modules, 2020
Zhengxue Cheng, Heming Sun, Masaru Takeuchi, and Jiro Katto. Learned image compression with discretized gaussian mixture likelihoods and attention modules, 2020. 2
2020
-
[4]
Imagenet: A large-scale hierarchical image database
Jia Deng, Wei Dong, Richard Socher, Li-Jia Li, Kai Li, and Li Fei-Fei. Imagenet: A large-scale hierarchical image database. In 2009 IEEE conference on computer vision and pattern recognition, pages 248–255. Ieee, 2009. 3
2009
-
[5]
An image is worth 16x16 words: Transformers for image recognition at scale, 2021
Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xiaohua Zhai, Thomas Unterthiner, Mostafa Dehghani, Matthias Minderer, Georg Heigold, Syl- vain Gelly, Jakob Uszkoreit, and Neil Houlsby. An image is worth 16x16 words: Transformers for image recognition at scale, 2021. 3
2021
-
[6]
Gintare Karolina Dziugaite, Zoubin Ghahramani, and Daniel M. Roy. A study of the effect of jpg compression on adversarial images, 2016. 1, 2
work page 2016
-
[7]
Exploring the landscape of spatial robustness, 2019
Logan Engstrom, Brandon Tran, Dimitris Tsipras, Ludwig Schmidt, and Aleksander Madry. Exploring the landscape of spatial robustness, 2019. 8
work page 2019
Show all 31 references
-
[8]
Deep universal generative adversarial compression artifact removal
Leonardo Galteri, Lorenzo Seidenari, Marco Bertini, and Alberto Del Bimbo. Deep universal generative adversarial compression artifact removal. IEEE Transactions on Multi- media, 21(8):2131–2145, 2019. 2
2019
-
[9]
Adams, Ian Goodfellow, David An- dersen, and George E
Justin Gilmer, Ryan P. Adams, Ian Goodfellow, David An- dersen, and George E. Dahl. Motivating the rules of the game for adversarial example research, 2018. 8
2018
-
[10]
Explaining and harnessing adversarial examples
Ian J Goodfellow, Jonathon Shlens, and Christian Szegedy. Explaining and harnessing adversarial examples. arXiv preprint arXiv:1412.6572, 2014. 3
2014 arXiv
-
[11]
Checkerboard context model for effi- cient learned image compression
Dailan He, Yaoyan Zheng, Baocheng Sun, Yan Wang, and Hongwei Qin. Checkerboard context model for effi- cient learned image compression. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 14771–14780, 2021. 2
2021
-
[12]
Elic: Efficient learned image compres- sion with unevenly grouped space-channel contextual adap- tive coding, 2022
Dailan He, Ziming Yang, Weikun Peng, Rui Ma, Hongwei Qin, and Yan Wang. Elic: Efficient learned image compres- sion with unevenly grouped space-channel contextual adap- tive coding, 2022. 2, 3
2022
-
[13]
Deep residual learning for image recognition, 2015
Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition, 2015. 1, 3
2015
-
[14]
Distilling the knowledge in a neural network, 2015
Geoffrey Hinton, Oriol Vinyals, and Jeff Dean. Distilling the knowledge in a neural network, 2015. 2
2015
-
[15]
Adversar- ial examples are not bugs, they are features
Andrew Ilyas, Shibani Santurkar, Dimitris Tsipras, Logan Engstrom, Brandon Tran, and Aleksander Madry. Adversar- ial examples are not bugs, they are features. Advances in neural information processing systems, 32, 2019. 1
2019
-
[16]
Torchattacks: A pytorch repository for adversar- ial attacks
Hoki Kim. Torchattacks: A pytorch repository for adversar- ial attacks. arXiv preprint arXiv:2010.01950, 2020. 3
2010 arXiv
-
[17]
Adver- sarial examples in the physical world, 2017
Alexey Kurakin, Ian Goodfellow, and Samy Bengio. Adver- sarial examples in the physical world, 2017. 2, 3
2017
-
[18]
Learned image compression with mixed transformer-cnn architectures
Jinming Liu, Heming Sun, and Jiro Katto. Learned image compression with mixed transformer-cnn architectures. In Proceedings of the IEEE/CVF Conference on Computer Vi- sion and Pattern Recognition (CVPR), pages 14388–14397,
-
[19]
Towards deep learning models resistant to adversarial attacks, 2019
Aleksander Madry, Aleksandar Makelov, Ludwig Schmidt, Dimitris Tsipras, and Adrian Vladu. Towards deep learning models resistant to adversarial attacks, 2019. 1, 2, 3
2019
-
[20]
High-fidelity generative image compres- sion, 2020
Fabian Mentzer, George Toderici, Michael Tschannen, and Eirikur Agustsson. High-fidelity generative image compres- sion, 2020. 2
2020
-
[21]
Joint autoregressive and hierarchical priors for learned image compression
David Minnen, Johannes Ball ´e, and George D Toderici. Joint autoregressive and hierarchical priors for learned image compression. In Advances in Neural Information Processing Systems. Curran Associates, Inc., 2018. 2
2018
-
[22]
Deepfool: a simple and accurate method to fool deep neural networks, 2016
Seyed-Mohsen Moosavi-Dezfooli, Alhussein Fawzi, and Pascal Frossard. Deepfool: a simple and accurate method to fool deep neural networks, 2016. 3
2016
-
[23]
Distillation as a defense to adversar- ial perturbations against deep neural networks, 2016
Nicolas Papernot, Patrick McDaniel, Xi Wu, Somesh Jha, and Ananthram Swami. Distillation as a defense to adversar- ial perturbations against deep neural networks, 2016. 2
2016
-
[24]
Pytorch: An im- perative style, high-performance deep learning library, 2019
Adam Paszke, Sam Gross, Francisco Massa, Adam Lerer, James Bradbury, Gregory Chanan, Trevor Killeen, Zeming Lin, Natalia Gimelshein, Luca Antiga, Alban Desmaison, Andreas K ¨opf, Edward Yang, Zach DeVito, Martin Raison, Alykhan Tejani, Sasank Chilamkurthy, Benoit Steiner, Lu F...
2019
-
[25]
Kornia: an open source differentiable computer vision library for pytorch
Edgar Riba, Dmytro Mishkin, Daniel Ponsa, Ethan Rublee, and Gary Bradski. Kornia: an open source differentiable computer vision library for pytorch. In Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pages 3674–3683, 2020. 2
2020
-
[26]
Deep learning in medical image analysis
Dinggang Shen, Guorong Wu, and Heung-Il Suk. Deep learning in medical image analysis. Annual Review of Biomedical Engineering , 19(V olume 19, 2017):221–248,
2017
-
[27]
Jpeg-resistant adversarial im- ages
Richard Shin and Dawn Song. Jpeg-resistant adversarial im- ages. In NIPS 2017 workshop on machine learning and com- puter security, page 8, 2017. 1, 2, 3, 5, 6
2017
-
[28]
Deep neural networks for object detection
Christian Szegedy, Alexander Toshev, and Dumitru Erhan. Deep neural networks for object detection. In Advances in Neural Information Processing Systems. Curran Associates, Inc., 2013. 1
2013
-
[29]
In- triguing properties of neural networks, 2014
Christian Szegedy, Wojciech Zaremba, Ilya Sutskever, Joan Bruna, Dumitru Erhan, Ian Goodfellow, and Rob Fergus. In- triguing properties of neural networks, 2014. 1, 2
2014
-
[31]
Object detection in 20 years: A survey.Proceed- ings of the IEEE, 111(3):257–276, 2023
Zhengxia Zou, Keyan Chen, Zhenwei Shi, Yuhong Guo, and Jieping Ye. Object detection in 20 years: A survey.Proceed- ings of the IEEE, 111(3):257–276, 2023. 1 9 A. Tables Attack Hyperparameters FGSM eps=epsilon iFGSM eps=epsilon,alpha=epsilon/4, steps = 10 PGD eps=epsilon, alpha...
2023
Reviewed August 16, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.