REVIEW 2 major objections 4 minor 45 references
Exploring adversarial robustness of JPEG AI: methodology, comparison and new methods
T0 review · 2 major / 4 minor · reviewed 2026-08-12 · deepseek-v4-flash
Pith's one-line read The paper demonstrates that JPEG AI, the first end-to-end neural image compression standard, is relatively more robust to white-box adversarial attacks than other neural codecs, with base-operation-point versions more robust than…
desk verdict Broad, useful robustness benchmark for JPEG AI, but the central ranking may be inflated by attacking the core model instead of the full codec. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The load-bearing machinery is the $\Delta$score methodology: the difference in a full-reference quality metric (PSNR, MSE, MS-SSIM, VMAF) between the codec's reconstruction of a clean image and of its adversarial counterpart. This turns robustness into a single comparable number across codecs, attacks, and loss functions. The methodology is paired with a modified attack interface that perturbs the JPEG AI core model while evaluating effects on the full codec with its optional tools enabled, plus a transferability variant $\hat{\Delta}$score that measures how well an attack aimed at one codec degrades another codec's output.
What would settle it
Run the same six white-box attacks against the complete JPEG AI configuration with RVS, filters, and gain enabled, and compare the $\Delta$VMAF rankings. If the high-operation-point versions no longer appear less robust than base-operation-point versions, or the version-to-version robustness differences change sign, the core-model ranking does not represent the shipping codec.
Extended reading notes
Core claim
The central discovery is a robustness ranking: under a battery of six white-box attacks (FTDA, I-FGSM, MADC, PGD, SSAH, cAdv) plus Gaussian noise, JPEG AI shows relatively high robustness compared to other NIC models, while the diffusion-based CDC shows the lowest robustness. Within JPEG AI, the high-operation-point (HOP) versions are less robust than base-operation-point (BOP) versions, and robustness improves from version 5.1 to 6.1. Attacks constructed on the core model transfer across JPEG AI bitrates and versions, and adversarial inputs also increase the bitrate of compressed images, harming both quality and bandwidth. Simple reversible transformations\u2014flip, random roll, and random ensemble\u2014serve as effective purification defenses against these attacks.
Load-bearing premise
The main assumption is that attacking the stripped-down core encoder\u2013decoder of JPEG AI, rather than the full codec with its optional tools (RVS, filters, and gain) enabled, reveals the robustness that matters in real deployments.
Editorial extensions
If this is right
- If JPEG AI is deployed in consumer devices, the base operation point is a safer default than the high operation point under a white-box threat model.
- Simple reversible preprocessing defenses\u2014flip, random roll, and random ensemble\u2014can nullify several of the tested attacks without retraining the codec.
- Adversarial attacks increase compressed-file size even when they do not target bitrate, so a small perturbation can inflate storage and bandwidth costs.
- Attack success transfers across JPEG AI bitrates and versions, so each new release of the codec should be re-evaluated for robustness before wide adoption.
- JPEG AI's robustness improved from version 5.1 to 6.1, indicating that the standard's development process is already making the codec harder to attack.
Reading between the lines
- Because adversarial attacks transfer between JPEG AI bitrates and versions, an attacker who can craft perturbations against any public version could plausibly aim them at a newer, unreleased version, making robustness certification a release-gate issue for the standard.
- The finding that attacks inflate compressed bitrate even when not targeting BPP suggests a denial-of-service path: a small, invisible perturbation could multiply storage or bandwidth costs in large-scale deployments.
- The defense results hint that random transforms are effective not because JPEG AI is inherently weak but because the attack's gradient is brittle; a testable extension would be to attack the defended pipeline end-to-end and see whether robustness collapses.
- The reliance on full-reference quality metrics leaves open whether attacks that fool these metrics also degrade perceived quality; a human-study extension could connect the $\Delta$metric scale to subjective annoyance.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper proposes a methodology for measuring the adversarial robustness of neural image codecs (NICs) using ΔPSNR, ΔMSE, ΔMS-SSIM, and ΔVMAF, and applies it in a large-scale comparison of 10 NIC models, including three versions of JPEG AI at two operation points each. Six white-box attacks, ten optimization losses, four datasets, and several preprocessing defenses are evaluated. The central empirical claims are that JPEG AI is relatively robust compared with other NICs, that high-operation-point (HOP) versions are less robust than base-operation-point (BOP) versions, and that robustness improved from version 5.1 to 6.1. The paper also reports attack transferability, artifact analyses, and bitrate-increase effects, and releases code and results online.
Significance. If the robustness ranking is correct, this is an important result: JPEG AI is the first neural compression standard, and knowing whether standard white-box attacks can break it, and whether higher-efficiency configurations are riskier, is directly relevant to deployment. The study is one of the largest of its kind, covering many codecs, attacks, losses, and defenses, and the use of standard full-reference quality metrics rather than fitted models makes the delta scores transparent and reproducible in principle. The transferability experiment across JPEG AI versions is a useful contribution, and the public release of code and evaluation data is a strength.
major comments (2)
- [4.6] The attacks against JPEG AI are optimized using a modified interface that attacks only the core encoder-decoder, while the measured effect is computed on the full codec including RVS, filters, and Channel-Wise Gain, per the text: 'a modified interface for attacking the core model was used' and 'the attack's impact was assessed not only on the core model's result but also on the entire codec with additional tools included.' For the other NICs, the description indicates standard full-model attacks. This asymmetry is load-bearing for the central claim in Section 5.2 (that JPEG AI shows relatively high robustness, that HOP is less robust than BOP, and that 6.1 improves on 5.1). If the additional JPEG AI tools act as implicit defenses that the optimizer never sees, the measured robustness is not the robustness of the shipped codec. The authors should either run at least a subset of attacks end-to-end against the full JPEG AI configuration or explicitly justify why core-only attacks are the correct threat model, and they should report how the HOP/BOP and version comparisons change when the additional tools are included in the attack graph.
- [4.6 and 5.2] The implementation details state that each attack was run four times with varied learning rate, number of iterations, and perturbation bound, but none of these values or ranges are reported. Given that the paper's main comparisons are qualitative rankings without error bars or significance tests, the reader cannot assess whether the reported differences between codecs, between HOP and BOP, and between JPEG AI versions are larger than run-to-run variability or attack-strength variability. At minimum, the paper should report the exact attack settings and per-condition standard deviations or confidence intervals, and ideally a statistical test for the key pairwise claims in Section 5.2.
minor comments (4)
- [5.4] The text says 'except for ΔNIQE' but NIQE is not introduced among the quality metrics in Section 4.5; either define it (and the corresponding delta) or remove the reference.
- [5.4] The caption of Figure 5 says 'Spearman Correlation Coefficients' while the text in the same section says 'pairwise Pearson correlation coefficients'; please make the statistic used consistent.
- [References] Reference [11] and reference [12] are the same paper, and the entry for [24] has the author order malformed ('et al Liu, Kang'); please correct these.
- [5.1] The sentence 'Fig. 2 shows how they influence the quality of uncompressed images' is unclear; since the figure shows Δ-metrics, it should probably say how the losses affect the adversarial images before compression or how each loss changes the delta metrics.
Circularity Check
No significant circularity: robustness rankings are empirical measurements, and the author self-citations are auxiliary tools rather than load-bearing premises.
full rationale
The paper's central claims in Section 5.2 (JPEG AI robustness relative to other NICs, HOP versus BOP, and version 5.1 versus 6.1) are direct empirical measurements computed from Equation 5, Delta score = FR(x, C(x)) - FR(x', C(x')), applied to standard full-reference metrics (PSNR, MSE, MS-SSIM, VMAF). No parameter is fitted to the robustness ranking and then used to predict that same ranking, and no equation reduces by construction to its own input. The attacks are specified independently in Sections 4.2 and 4.6, and the robustness comparison is read off the resulting delta-metric values, so the ranking is not definitionally forced. The author self-citations appear in auxiliary roles: [2] supports the choice of VMAF as a perceptually aligned metric, [33] provides artifact-detection tools for Section 5.4, and [45] defines BSQ-rate for the no-attack compression comparison in Section 5.7. None of these citations is used to justify the central robustness conclusion; the robustness conclusion would stand on the measured delta metrics even if those citations were removed. One potential concern is the asymmetric attack setup in Section 4.6, where JPEG AI is attacked through a modified core-model interface while effectiveness is assessed on the full codec with additional tools enabled; this is an experimental-validity and threat-model question, not a circularity, because the reported numbers are still measurements rather than derived equivalences. Overall, the derivation chain is self-contained, and the paper's main claims are externally falsifiable empirical comparisons.
Assumptions & free parameters
free parameters (2)
- Attack hyperparameters (learning rate, iterations, perturbation bound) =
not reported
- Defense parameters (random roll size, rotation angle, ensemble weights) =
not reported
assumptions (3)
- domain assumption White-box access to the codec core is an appropriate threat model for JPEG AI deployment.
- domain assumption The core-model attack interface represents the attack surface of the full codec.
- domain assumption Delta score computed with PSNR, MSE, MS-SSIM, and VMAF captures adversarial harm.
Cite this review
Pith. "Pith review of Exploring adversarial robustness of JPEG AI: methodology, comparison and new methods." pith.science (2026). https://pith.science/paper/7QLEPVPG
@misc{pith2026241111795,
author = {Pith},
title = {Pith review of: Exploring adversarial robustness of JPEG AI: methodology, comparison and new methods},
year = {2026},
howpublished = {\url{https://pith.science/paper/7QLEPVPG}},
note = {Machine review of arXiv:2411.11795}
}
read the original abstract
Adversarial robustness of neural networks is an increasingly important area of research, combining studies on computer vision models, large language models (LLMs), and others. With the release of JPEG AI - the first standard for end-to-end neural image compression (NIC) methods - the question of its robustness has become critically significant. JPEG AI is among the first international, real-world applications of neural-network-based models to be embedded in consumer devices. However, research on NIC robustness has been limited to open-source codecs and a narrow range of attacks. This paper proposes a new methodology for measuring NIC robustness to adversarial attacks. We present the first large-scale evaluation of JPEG AI's robustness, comparing it with other NIC models. Our evaluation results and code are publicly available online (link is hidden for a blind review).
Figures
Figures from the paper (5 more)
Reference graph
Works this paper leans on
-
[1]
Soft-to-hard vector quantization for end-to-end learn- ing compressible representations
Eirikur Agustsson, Fabian Mentzer, Michael Tschannen, Lukas Cavigelli, Radu Timofte, Luca Benini, and Luc V Gool. Soft-to-hard vector quantization for end-to-end learn- ing compressible representations. Advances in Neural Infor- mation Processing Systems, 30, 2017. 2
work page 2017
-
[2]
Video compression dataset and benchmark of learning-based video-quality metrics
Anastasia Antsiferova, Sergey Lavrushkin, Maksim Smirnov, Aleksandr Gushchin, Dmitriy Vatolin, and Dmitriy Kulikov. Video compression dataset and benchmark of learning-based video-quality metrics. In Advances in Neural Information Processing Systems , pages 13814–13825. Curran Associates, Inc., 2022. 5
work page 2022
-
[3]
Anastasia Antsiferova, Khaled Abud, Aleksandr Gushchin, Ekaterina Shumitskaya, Sergey Lavrushkin, and Dmitriy Va- tolin. Comparing the robustness of modern no-reference image- and video-quality metrics to adversarial attacks. Pro- ceedings of the AAAI Conference on Artificial Intelligence , 38(2):700–708, 2024. 1
work page 2024
-
[4]
Contour detection and hierarchical image seg- mentation
Pablo Arbelaez, Michael Maire, Charless Fowlkes, and Ji- tendra Malik. Contour detection and hierarchical image seg- mentation. IEEE Transactions on Pattern Analysis and Ma- chine Intelligence, 33:898–916, 2011. 5
work page 2011
-
[5]
The jpeg ai standard: Providing efficient human and machine vi- sual data consumption
Jo ˜ao Ascenso, Elena Alshina, and Touradj Ebrahimi. The jpeg ai standard: Providing efficient human and machine vi- sual data consumption. IEEE MultiMedia, 30(1):100–111,
-
[6]
End-to-end optimized image compression
Johannes Ball ´e, Valero Laparra, and Eero P Simoncelli. End-to-end optimized image compression. arXiv preprint arXiv:1611.01704, 2016. 2
arXiv 2016
-
[7]
Variational image compres- sion with a scale hyperprior
Johannes Ball ´e, David Minnen, Saurabh Singh, Sung Jin Hwang, and Nick Johnston. Variational image compres- sion with a scale hyperprior. In International Conference on Learning Representations, 2018. 2, 4
work page 2018
-
[8]
Unrestricted adversarial examples via semantic manipulation
Anand Bhattad and et al. Unrestricted adversarial examples via semantic manipulation. arXiv preprint arXiv:1904.06347, 2019. 4
arXiv 1904
Show all 45 references
-
[9]
Towards evaluating the robustness of neural networks
Nicholas Carlini and David Wagner. Towards evaluating the robustness of neural networks. In 2017 IEEE Symposium on Security and Privacy, 2017. 3
2017
-
[10]
A survey on adversarial attacks and defences
Anirban Chakraborty, Manaar Alam, Vishal Dey, Anupam Chattopadhyay, and Debdeep Mukhopadhyay. A survey on adversarial attacks and defences. CAAI Transactions on In- telligence Technology, 6(1):25–45, 2021. 1
2021
-
[11]
Toward robust neural image com- pression: Adversarial attack and model finetuning
Tong Chen and Zhan Ma. Toward robust neural image com- pression: Adversarial attack and model finetuning. IEEE Transactions on Circuits and Systems for Video Technology, 33(12):7842–7856, 2023. 3, 4, 5
2023
-
[12]
Toward robust neural image com- pression: Adversarial attack and model finetuning
Tong Chen and Zhan Ma. Toward robust neural image com- pression: Adversarial attack and model finetuning. IEEE Transactions on Circuits and Systems for Video Technology, 33(12):7842–7856, 2023. 1, 5
2023
-
[13]
Learned image compression with discretized gaussian mixture likelihoods and attention modules
Zhengxue Cheng, Heming Sun, Masaru Takeuchi, and Jiro Katto. Learned image compression with discretized gaussian mixture likelihoods and attention modules. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 7939–7948, 2020. 2, 4
2020
-
[14]
Nips 2017: Adversarial learning devel- opment set
Competition Page. Nips 2017: Adversarial learning devel- opment set. https://www.kaggle.com/datasets/ google - brain / nips - 2017 - adversarial - learning-development-set, 2017. 5
2017
-
[15]
The cityscapes dataset for semantic urban scene understanding
Marius Cordts, Mohamed Omran, Sebastian Ramos, Timo Rehfeld, Markus Enzweiler, Rodrigo Benenson, Uwe Franke, Stefan Roth, and Bernt Schiele. The cityscapes dataset for semantic urban scene understanding. In Proc. of the IEEE Conference on Computer Vision and Pattern Recognitio...
2016
-
[16]
Boosting adversarial at- tacks with momentum
Yinpeng Dong, Fangzhou Liao, Tianyu Pang, Hang Su, Jun Zhu, Xiaolin Hu, and Jianguo Li. Boosting adversarial at- tacks with momentum. pages 9185–9193, 2018. 1
2018
-
[17]
Lossy image compression with quantized hierarchical vaes
Zhihao Duan, Ming Lu, Zhan Ma, and Fengqing Zhu. Lossy image compression with quantized hierarchical vaes. InPro- ceedings of the IEEE/CVF Winter Conference on Applica- tions of Computer Vision, pages 198–207, 2023. 2, 4
2023
-
[18]
Neural image com- pression via attentional multi-scale back projection and fre- quency decomposition
Ge Gao, Pei You, Rong Pan, Shunyuan Han, Yuanyuan Zhang, Yuchao Dai, and Hojae Lee. Neural image com- pression via attentional multi-scale back projection and fre- quency decomposition. In Proceedings of the IEEE/CVF In- ternational Conference on Computer Vision , pages 14677–...
2021
-
[19]
Elic: Efficient learned image compres- sion with unevenly grouped space-channel contextual adap- tive coding
Dailan He, Ziming Yang, Weikun Peng, Rui Ma, Hongwei Qin, and Yan Wang. Elic: Efficient learned image compres- sion with unevenly grouped space-channel contextual adap- tive coding. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages 57...
2022
-
[20]
Kodak lossless true color image suite
Eastman Kodak. Kodak lossless true color image suite. In PhotoCD PCD0992, 1991. 4
1991
-
[21]
Goodfellow, and Samy Bengio
Alexey Kurakin, Ian J. Goodfellow, and Samy Bengio. Ad- versarial examples in the physical world. In Artificial intel- ligence safety and security. Chapman and Hall/CRC , pages 99–112, 2018. 3, 4
2018
-
[22]
Deep contextual video com- pression
Jiahao Li, Bin Li, and Yan Lu. Deep contextual video com- pression. Advances in Neural Information Processing Sys- tems, 34, 2021. 4
2021
-
[23]
Vmaf: The journey continues
Zhi Li, Christos Bampis, Julie Novak, Anne Aaron, Kyle Swanson, Anush Moorthy, and JD Cock. Vmaf: The journey continues. Netflix Technology Blog, 25, 2018. 5
2018
-
[24]
Manipulation attacks on learned image com- pression
et al Liu, Kang. Manipulation attacks on learned image com- pression. In IEEE Transactions on Artificial Intelligence ,
-
[25]
Learned image compression with mixed transformer-cnn architectures
Jinming Liu, Heming Sun, and Jiro Katto. Learned image compression with mixed transformer-cnn architectures. In Proceedings of the IEEE/CVF Conference on Computer Vi- sion and Pattern Recognition, pages 14388–14397, 2023. 2, 4
2023
-
[26]
Frequency-driven imperceptible ad- versarial attack on semantic similarity
Cheng Luo and et al. Frequency-driven imperceptible ad- versarial attack on semantic similarity. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 15315–15324, 2022. 4
2022
-
[27]
Towards deep learn- ing models resistant to adversarial attacks
Aleksander Madry, Aleksandar Makelov, Ludwig Schmidt, Dimitris Tsipras, and Adrian Vladu. Towards deep learn- ing models resistant to adversarial attacks. In International Conference on Learning Representations, 2018. 4
2018
-
[28]
High-fidelity generative image compres- sion
Fabian Mentzer, George D Toderici, Michael Tschannen, and Eirikur Agustsson. High-fidelity generative image compres- sion. Advances in Neural Information Processing Systems , 33:11913–11924, 2020. 2, 4
2020
-
[29]
Joint autoregressive and hierarchical priors for learned im- age compression
David Minnen, Johannes Ball ´e, and George D Toderici. Joint autoregressive and hierarchical priors for learned im- age compression. Advances in Neural Information Process- ing Systems, 31, 2018. 2, 4
2018
-
[30]
Pick-object-attack: Type-specific adver- sarial attack for object detection
Omid Mohamad Nezami, Akshay Chaturvedi, Mark Dras, and Utpal Garain. Pick-object-attack: Type-specific adver- sarial attack for object detection. Computer Vision and Im- age Understanding, 211:103257, 2021. 1
2021
-
[31]
Diffusion models for adversarial purification
Weili Nie, Brandon Guo, Yujia Huang, Chaowei Xiao, Arash Vahdat, and Anima Anandkumar. Diffusion models for adversarial purification. arXiv preprint arXiv:2205.07460 ,
-
[32]
The ciede2000 color-difference formula: Implementation notes, supplementary test data, and mathematical observations
Gaurav Sharma, Wencheng Wu, and Edul Dalal. The ciede2000 color-difference formula: Implementation notes, supplementary test data, and mathematical observations. Color Research and Application, 30:21 – 30, 2005. 6
2005
-
[33]
Jpeg ai image compression visual artifacts: Detection methods and dataset, 2024
Daria Tsereh, Mark Mirgaleev, Ivan Molodetskikh, Roman Kazantsev, and Dmitriy Vatolin. Jpeg ai image compression visual artifacts: Detection methods and dataset, 2024. 6
2024
-
[34]
Evc: To- wards real-time neural image compression with mask decay
Guo-Hua Wang, Jiahao Li, Bin Li, and Yan Lu. Evc: To- wards real-time neural image compression with mask decay. arXiv preprint arXiv:2302.05071, 2023. 2
2023 arXiv
-
[35]
Simoncelli
Zhou Wang and Eero P. Simoncelli. Maximum differentia- tion (mad) competition: A methodology for comparing com- putational models of perceptual quantities. In Journal of Vi- sion 8.12, pages 8–8, 2008. 4
2008
-
[36]
Wang, E.P
Z. Wang, E.P. Simoncelli, and A.C. Bovik. Multiscale struc- tural similarity for image quality assessment. In The Thrity- Seventh Asilomar Conference on Signals, Systems and Com- puters, 2003, pages 1398–1402 V ol.2, 2003. 5
2003
-
[37]
Physical adversarial attack meets computer vision: A decade survey
Hui Wei, Hao Tang, Xuemei Jia, Zhixiang Wang, Hanxun Yu, Zhubo Li, Shin’ichi Satoh, Luc Van Gool, and Zheng Wang. Physical adversarial attack meets computer vision: A decade survey. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2024. 1
2024
-
[38]
Slimmable compressive autoencoders for practical neural image compression
Fei Yang, Luis Herranz, Yongmei Cheng, and Mikhail G Mozerov. Slimmable compressive autoencoders for practical neural image compression. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages 4998–5007, 2021. 1
2021
-
[39]
Slimmable compressive autoencoders for practical neural image compression
Fei Yang, Luis Herranz, Yongmei Cheng, and Mikhail G Mozerov. Slimmable compressive autoencoders for practical neural image compression. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages 4998–5007, 2021. 2
2021
-
[40]
Lossy image compression with conditional diffusion models
Ruihan Yang and Stephan Mandt. Lossy image compression with conditional diffusion models. Advances in Neural In- formation Processing Systems, 36, 2024. 2, 4
2024
-
[41]
Improving inference for neural image compression
Yibo Yang, Robert Bamler, and Stephan Mandt. Improving inference for neural image compression. Advances in Neural Information Processing Systems, 33:573–584, 2020. 1
2020
-
[42]
A survey on universal adversarial attack
Chaoning Zhang, Philipp Benz, Chenguo Lin, Adil Karjauv, Jing Wu, and In So Kweon. A survey on universal adversarial attack. arXiv preprint arXiv:2103.01498, 2021. 1
2021 arXiv
-
[43]
Attack and defense analysis of learned image compression
et al Zhu, Tianyu. Attack and defense analysis of learned image compression. arXiv preprint arXiv:2401.10345, 2024. 1
2024 arXiv
-
[44]
The devil is in the details: Window-based attention for image compression
Renjie Zou, Chunfeng Song, and Zhaoxiang Zhang. The devil is in the details: Window-based attention for image compression. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 17492– 17501, 2022. 2
2022
-
[45]
Bsq-rate: a new approach for video-codec performance comparison and drawbacks of current solutions
Anastasia V Zvezdakova, Dmitriy L Kulikov, Sergey V Zvezdakov, and Dmitriy S Vatolin. Bsq-rate: a new approach for video-codec performance comparison and drawbacks of current solutions. Programming and computer software, 46: 183–194, 2020. 7
2020
Reviewed August 12, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.