{"id":"7a244c27-5de0-4d61-8b21-e85366d3fd56","arxiv_id":"2504.12255","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":3,"one_line_summary":"On Imagenette and a 1000-image ImageNet sample, learned compression (HiFiC, ELIC) defends image classifiers against adversarial examples better than JPEG, especially for Vision Transformers, and sequential compression rounds add further robustness.","lead":"Learned image compression (HiFiC and ELIC) is tested as a defense against adversarial attacks on image classifiers, and it beats JPEG, especially for Vision Transformers. The paper also finds that running compression several times in a row strengthens the defense, though white-box attackers who know the defense still reduce accuracy sharply.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"White-box robustness claim rests on weak, non-adaptive attacks; stronger PGD/AutoAttack with restarts could collapse it.","rationale":"The reader's verdict is conditional, and I agree. The black-box results are clearly presented, reproducible in principle (code is provided), and the ViT-specific advantage of learned compression is an interesting empirical finding. However, the white-box component of the headline is not established by 10-step attacks without restarts or convergence checks. If a stronger adaptive attack such as AutoAttack or PGD-100 with restarts collapses the reported accuracies toward the no-defense baseline, the abstract's 'substantial effectiveness' claim would be unsupported. The bitrate mismatch between the selected JPEG and learned-compression settings is a real but secondary confound; it does not invalidate the black-box comparison but weakens the attribution to human-aligned preservation of semantics. Conditional acceptance requiring stronger adaptive attack evaluation and either bitrate-matched JPEG baselines or softened white-box claims is the appropriate outcome. No formal verification is claimed, so correctness rests entirely on the empirical protocol, which is exactly where the risk sits.","tokens_in":17378,"tokens_out":5209,"duration_ms":59172,"concrete_test":"Recompute the white-box rows of Table 5 for ViT at eps=8/255 using AutoAttack (APGD-CE, APGD-DLR, FAB, Square) or at minimum PGD-100 with 10 random restarts, using the same differentiable forward passes and also an identity-backward BPDA variant. Compare ELIC/HiFiC/JPEG accuracies against the no-defense value of 0.006. If ELIC/HiFiC drop to within a few points of 0.006, the 'substantial white-box effectiveness' claim fails. As a secondary check, run JPEG at q≈2 (BPP ≈ 0.15) through the same black-box protocol; if it matches HiFiC/ELIC, the learned-method advantage is bitrate-driven.","verdict_should_be":"CONDITIONAL","load_bearing_attack":"The load-bearing claim is the abstract's 'Even in white-box settings ... substantial effectiveness.' The evidence is Table 5 with 10-step iFGSM/PGD and 50-step CW/DeepFool (Table 3), with no restarts or convergence checks. Lossy compression includes nondifferentiable quantization and entropy coding; the differentiable forward functions in Section 3.1 are approximations, so the gradients passed to the attack may be poor proxies. This is the classic gradient-masking setting: the defense can look robust because the attack's gradient is uninformative rather than because the defense is genuinely robust. The paper itself limits the threat model to gradient-based attacks in Section 5 and cites Uesato et al. [30], which warns against evaluating only weak attacks. A secondary confound: the selected qualities are not bitrate-matched (JPEG q=25 has BPP 0.78, while ELIC 0016 has 0.14 and HiFiC low has 0.15, Table 1), so the learned-vs-JPEG advantage partly reflects compression strength, not 'human alignment' as such.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper investigates learned lossy image compression (HiFiC and ELIC) as a preprocessing defense against adversarial attacks, comparing them with JPEG on ImageNet subsets using ResNet50 and ViT-B/16. The authors report that learned compression outperforms JPEG especially for ViT, that defenses remain partially effective in white-box settings where gradients are propagated through differentiable approximations of the compression, and that repeated sequential compression cycles increase robustness. The paper includes detailed accuracy tables for FGSM, iFGSM, PGD, CW, and DeepFool attacks, quality ablations, computational overhead measurements, and sequential-compression experiments on Imagenette and a 1000-image ImageNet sample.","tokens_in":17568,"tokens_out":3926,"duration_ms":43673,"significance":"If the white-box robustness claim were supported, the paper would provide a practical, computationally cheap input-transformation defense that also works for ViT, for which compression-based defenses are comparatively understudied. The manuscript is strong on breadth: it compares three compression families, two architectures, five attacks, and both adaptive and non-adaptive settings, and it reports per-setting accuracy tables, quality ablations, bitrate measurements, and inference-time overhead. The code link is provided. The main significance is conditional on the robustness evaluation being trustworthy; as it stands, the white-box results are generated by weak, low-iteration attacks through approximate gradients, so the headline claim of 'substantial effectiveness' in white-box settings is not yet established.","major_comments":[{"comment":"The white-box evaluation is too weak to support the abstract claim of 'substantial effectiveness' against attackers who can access the defense. iFGSM and PGD are run for only 10 steps with no restarts, and CW/DeepFool for 50 steps, through differentiable approximations of inherently non-differentiable compression pipelines. This is exactly the gradient-masking scenario: the attack's gradients may be uninformative because they are computed through a proxy, so high defended accuracy can reflect attack failure rather than genuine robustness. The paper itself cites Uesato et al. [30] on the dangers of evaluating against weak attacks. Please add stronger adaptive attacks (e.g., PGD with 100 or more steps and multiple random restarts, AutoAttack, or BPDA with expectation over transformation) and report convergence or attack success as a function of iterations.","section":"§3.2, Table 3, §5"},{"comment":"The learned-versus-JPEG comparison confounds compression method with compression strength. The selected operating points are JPEG q=25 at 0.78 BPP, ELIC 0016 at 0.14 BPP, and HiFiC low at 0.15 BPP, so the learned methods compress roughly 5x more aggressively than the chosen JPEG setting. The observed advantage of HiFiC and ELIC over JPEG may therefore reflect stronger compression rather than 'human-aligned' preservation of semantic content. To support the paper's central comparison, the authors should either use bitrate-matched operating points (e.g., JPEG at the lowest available quality near 0.15 BPP, or otherwise interpolate/extrapolate) or explicitly analyze how much of the performance gap remains after controlling for BPP.","section":"§3.4, Table 1, §4.2"},{"comment":"The 'black-box' results are not black-box in the standard adversarial-robustness sense. The attacks are generated on the classifier alone without passing gradients through the defense, which is an attack that is unaware of the defense, not an attack against a system whose internals are hidden. This terminology materially inflates the apparent robustness: a defense that simply destroys all low-amplitude structure will score highly under such 'black-box' evaluation. Please rename these settings 'non-adaptive' or 'defense-unaware,' or run proper black-box attacks (e.g., transfer attacks from surrogate models or query-based attacks) if the black-box claim is intended.","section":"§3.2, Figures 2-4"},{"comment":"The abstract's claim that sequential compression 'significantly enhances defense efficacy while maintaining classification performance' is not supported for the learned methods. For HiFiC low, seven compression cycles reduce clean accuracy to 0.987 and adversarial accuracy at epsilon 8/255 reaches only 0.385; for ELIC 0016, the corresponding adversarial accuracy is 0.41 after seven cycles. Only JPEG maintains both high clean accuracy (0.994) and high robust accuracy (0.96+) across many iterations. The sequential-compression conclusion should be restricted to settings where clean accuracy is preserved, or the claim should be reformulated to emphasize the JPEG result.","section":"§4.5, Figures 8-9, Tables 9-10"}],"minor_comments":[{"comment":"The text reads 'Projected Gradient Decent'; this should be 'Projected Gradient Descent'.","section":"§2.1"},{"comment":"Table 8 is captioned as comparing HiFiC quality levels, but the rows are ELIC checkpoints (0004 through 0450). This appears to be a copy-paste error from Table 7.","section":"Table 8 caption"},{"comment":"The caption labels panel (a) as 'ELIC low quality' while the panel list refers to 'ELIC 0004'; the naming is inconsistent and should be unified.","section":"Figure 1 caption"},{"comment":"The ImageNet results are based on 1000 randomly sampled images, but no confidence intervals, seeds, or repeated sampling are reported. Given the visible jaggedness and the claim of generalization to ImageNet, error bars or a fixed seed/description of sampling would aid interpretability.","section":"§4.2.1, Figure 4"},{"comment":"The defense quality levels are selected using ablations on the same datasets used for the main results, which can mildly overestimate the selected configurations' performance. A brief acknowledgment or a small held-out tuning set would strengthen the evaluation.","section":"§3.4, §4.4"}],"recommendation":"major_revision","confidential_remarks":"The paper is an empirical study with a clear scope, and the comparison tables are useful. The load-bearing issues are the weak white-box attack protocol and the BPP confound in the learned-versus-JPEG comparison; both are fixable with additional experiments rather than being fundamental flaws. I would not reject on novelty grounds: the contribution is incremental but potentially useful if the robustness claims survive stronger adaptive attacks."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Short version: this is a useful empirical comparison, not a breakthrough. The black-box numbers are convincing: with no gradient through the defense, HiFiC and ELIC keep ImageNet/ViT accuracy high under iFGSM/PGD while JPEG degrades. That's a genuinely new datapoint, and the authors deserve credit for testing two modern learned codecs and for measuring inference overhead (5 ms no defense vs 8-14 ms with defense). The sequential-compression experiments are also a nice addition, even if JPEG's behavior there is the more striking result.\n\nThe soft spots are real. The white-box claim in the abstract is oversold. In Table 5, at eps=8/255 on ViT, ELIC white-box accuracy is 0.196 and HiFiC is 0.08. That's better than the undefended baseline (0.006) but calling it 'substantial effectiveness' is a stretch. The body text is more honest — Section 4.2 says white-box attacks drastically weaken all defenses — so the abstract and the body disagree. More importantly, the white-box protocol is exactly the kind that invites gradient-masking illusions: 10-step iFGSM/PGD, no restarts, and gradients propagated through differentiable approximations of a lossy codec. The paper cites Uesato et al. but doesn't follow its advice. A stronger adaptive attack (AutoAttack with restarts, or even a longer PGD) could plausibly drop those numbers further. I'd want to see that before trusting the robustness claim.\n\nThere's also a confound in the headline learned-vs-JPEG comparison: JPEG q=25 runs at 0.78 bpp, while ELIC 0016 and HiFiC low are 0.14-0.15 bpp (Table 1). The learned methods are compressing much more aggressively, so part of their advantage is simply stronger denoising. That doesn't invalidate the empirical finding, but it undercuts the 'human-aligned' framing — a bitrate-matched JPEG (or a stronger JPEG) might close the gap in black-box settings. The quality ablation partially mitigates this, but the selected operating points are not matched.\n\nMinor points: no error bars or repeats for the 1000-image ImageNet sample; quality levels were chosen from ablations on the same Imagenette test set, which mildly favors the reported results. These are fixable.\n\nWho is this for? People working on input-transformation defenses or robust preprocessing. It's a reasonable empirical starting point, with an honest body and an overstated abstract. It deserves a serious referee, but the referee should send it back for stronger attacks and matched-bitrate comparisons. I'd suggest major revision, not desk rejection.","headline":"Learned compression beats JPEG as a black-box defense, especially for ViTs, but the white-box claim is oversold and the learned-vs-JPEG comparison is confounded by bitrate.","tokens_in":18083,"tokens_out":3229,"would_cite":false,"duration_ms":32167,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Learned image compression that matches human perception defends image classifiers against adversarial attacks more effectively than JPEG, especially for Vision Transformers.","keywords":["adversarial robustness","learned image compression","HiFiC","ELIC","vision transformer","JPEG compression","preprocessing defense","sequential compression"],"falsifier":"Run a white-box PGD attack with, say, 200 iterations and 10 random restarts, propagating gradients through the differentiable HiFiC or ELIC pipeline, and check whether ViT accuracy collapses toward the no-defense baseline; a collapse would falsify the claim of substantial effectiveness in white-box settings.","tokens_in":17176,"feed_emoji":"🛡️","tokens_out":3736,"duration_ms":35386,"temperature":0.7,"pith_summary":"The paper tries to establish that human-aligned learned compression—specifically the HiFiC and ELIC codecs—can serve as a practical preprocessing defense against adversarial image attacks, and that it beats JPEG on this job, especially for Vision Transformers. The reason this matters is that compression can strip away small adversarial perturbations while keeping the image inside the classifier's expected distribution, so accuracy on clean images barely drops. The paper also claims that running several compression/decompression cycles in sequence strengthens the defense, and that the whole pipeline is cheap enough for real-time use.","feed_headline":"Learned compression beats JPEG as an adversarial defense","feed_subtitle":"Human-aligned codecs erase attack noise while keeping images in-distribution, especially for vision transformers.","key_machinery":"The central objects are the two learned lossy codecs: HiFiC, a GAN-based compressor trained to match human perception, and ELIC, an efficient learned codec using unevenly grouped space-channel contextual adaptive coding, with JPEG as the classical baseline. Because both learned models are differentiable, attacks can propagate gradients through them in white-box mode, making the defense evaluation conservative. The mechanism carrying the argument is the lossy bottleneck: it discards high-frequency or perceptually unimportant detail, which is where adversarial perturbations tend to live, while retaining features the classifier relies on. Sequential compression repeatedly applies this bottleneck to erase residual adversarial signal.","core_discovery":"On ImageNet and Imagenette subsets, adversarial examples generated by FGSM, iFGSM, PGD, CW, and DeepFool lose much of their power when the input is compressed before classification. Learned codecs HiFiC and ELIC preserve near-baseline accuracy under black-box attacks at all tested epsilon values, while JPEG degrades for ViT at stronger attacks; under white-box attacks, where gradients flow through a differentiable approximation of the defense, all defenses weaken but learned compression still provides substantial effectiveness. Sequential compression increases defense accuracy with each round, with JPEG converging fastest and ELIC and HiFiC improving but at a cost to clean accuracy at low quality settings, and the overhead is small: about 14 ms per image for learned codecs versus 5 ms bare and 8 ms with JPEG on a single GPU.","pith_inferences":["A natural next test is combining compression with adversarially trained models; since compression removes non-robust features, it may lower the accuracy cost that adversarial training usually pays on clean inputs.","The paper's white-box attacks use 10-step iFGSM/PGD and 50-step CW/DeepFool; stronger adaptive attacks with restarts and more iterations would show how much of the apparent robustness is gradient masking rather than true invariance.","If the mechanism is really the perceptual bottleneck, then other human-aligned input transformations such as learned denoising or super-resolution should confer similar defense, which is testable with the same experimental setup.","Sequential compression changes the lossy artifact distribution per round; understanding which artifacts the classifier keys on could inform future codecs designed explicitly for downstream task robustness."],"forward_implications":["Learned compression can be dropped into an existing classifier as a preprocessing step without retraining or modifying the model.","For ViT classifiers, which the paper finds less robust to begin with, HiFiC and ELIC recover almost all clean accuracy under black-box iFGSM and PGD attacks, outperforming JPEG at every tested strength.","Sequential compression is a tunable dial: more cycles give more robustness, but for HiFiC low quality it quickly degrades clean accuracy, while JPEG stays near baseline for 50 cycles.","The measured overhead of single-digit to tens of milliseconds per image makes the defense practical for inference pipelines."],"supporting_citations":[{"why":"Establishes that JPEG compression can weaken adversarial attacks, the baseline the paper compares against.","marker":"[6]"},{"why":"Shows that a differentiable approximation of JPEG lets adaptive attacks bypass the defense, motivating the white-box evaluation.","marker":"[27]"},{"why":"Introduces HiFiC, the GAN-based human-aligned codec used as one of the learned defenses.","marker":"[20]"},{"why":"Introduces ELIC, the efficient learned codec used as the other learned defense.","marker":"[12]"},{"why":"Provides the non-robust features explanation that motivates why removing small perturbations can protect classifiers.","marker":"[15]"},{"why":"Introduces PGD attacks, one of the main attack families used to evaluate the defenses.","marker":"[19]"},{"why":"Introduces iFGSM, the iterative attack used in the core accuracy tables.","marker":"[17]"},{"why":"Introduces the Carlini-Wagner attack, one of the stronger white-box attacks tested.","marker":"[2]"},{"why":"Introduces DeepFool, the minimal-perturbation attack used in the evaluation.","marker":"[22]"}],"fun_headline_variants":["Learned compression outperforms JPEG against adversarial attacks","Human-aligned codecs erase attack noise better than JPEG","Learned codecs defend ViTs from adversarial noise","Sequential compression boosts adversarial defense"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The paper assumes that 10-step gradient attacks with a differentiable approximation of the compression capture what a determined attacker can do, so a stronger attack with more iterations and restarts could break the defense.","fun_headline_variants_meta":{"raw":{"variants":["Learned compression outperforms JPEG against adversarial attacks","Human-aligned codecs erase attack noise better than JPEG","Learned codecs defend ViTs from adversarial noise","Sequential compression boosts adversarial defense"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.001049,"raw_usage":{"total_tokens":4357,"prompt_tokens":842,"completion_tokens":3515,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":458,"completion_tokens_details":{"reasoning_tokens":3456}},"tokens_in":458,"tokens_out":3515,"duration_ms":23469,"temperature":1.0,"reasoning_tokens":3456,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-16T12:33:46.078496+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run a white-box PGD attack with, say, 200 iterations and 10 random restarts, propagating gradients through the differentiable HiFiC or ELIC pipeline, and check whether ViT accuracy collapses toward the no-defense baseline; a collapse would falsify the claim of substantial effectiveness in white-box settings.","supporting_citations":[{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Establishes that JPEG compression can weaken adversarial attacks, the baseline the paper compares against."},{"cited_title":"Jpeg-resistant adversarial im- ages","cited_arxiv_id":null,"evidence_quote":"Shows that a differentiable approximation of JPEG lets adaptive attacks bypass the defense, motivating the white-box evaluation."},{"cited_title":"Elic: Efficient learned image compres- sion with unevenly grouped space-channel contextual adap- tive coding, 2022","cited_arxiv_id":null,"evidence_quote":"Introduces ELIC, the efficient learned codec used as the other learned defense."},{"cited_title":"Adversar- ial examples are not bugs, they are features","cited_arxiv_id":null,"evidence_quote":"Provides the non-robust features explanation that motivates why removing small perturbations can protect classifiers."},{"cited_title":"Towards deep learning models resistant to adversarial attacks, 2019","cited_arxiv_id":null,"evidence_quote":"Introduces PGD attacks, one of the main attack families used to evaluate the defenses."},{"cited_title":"Adver- sarial examples in the physical world, 2017","cited_arxiv_id":null,"evidence_quote":"Introduces iFGSM, the iterative attack used in the core accuracy tables."},{"cited_title":"Towards evaluating the robustness of neural networks, 2017","cited_arxiv_id":null,"evidence_quote":"Introduces the Carlini-Wagner attack, one of the stronger white-box attacks tested."},{"cited_title":"Deepfool: a simple and accurate method to fool deep neural networks, 2016","cited_arxiv_id":null,"evidence_quote":"Introduces DeepFool, the minimal-perturbation attack used in the evaluation."}],"review_version":1}