Pith. sign in

REVIEW 4 major objections 4 minor 35 references

Post-training quantization preserves clean accuracy but hides a robustness gap; a 6 KB, source-trained head adapter recovers most of it.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · deepseek-v4-flash

2026-08-01 15:05 UTC pith:3N7GRX6C

load-bearing objection A plausible cheap fix for a real PTQ robustness gap, but the reported recovery is selective and it lacks the one control that would make the story stick. the 4 major comments →

arxiv 2607.18540 v1 pith:3N7GRX6C submitted 2026-07-20 cs.CV cs.LGcs.RO

Recti-Q: Feature-Space Rectification for Out-of-Distribution-Robust Quantized Perception in Edge Robotics

Hamidreza Yaghoubi Araghi , Parastoo Pilevar , Ming C. Lin This is my paper
classification cs.CV cs.LGcs.RO
keywords post-training quantizationout-of-distribution robustnessquantization-induced robustness gaplow-rank adapterfeature-space rectificationdomain shiftedge roboticsImageNet-C
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper tries to establish that post-training quantization (PTQ) does not merely preserve accuracy—it quietly sacrifices out-of-distribution robustness. Across ImageNet-C and PACS, 4-bit quantized models show almost no in-distribution loss yet drop sharply on corrupted or domain-shifted inputs, a failure the authors call the Quantization-Induced Robustness Gap. The proposed remedy, Recti-Q, freezes the quantized backbone and trains a tiny low-rank adapter on the pre-classifier features using only clean source images. At less than 1% parameter overhead, the adapter recovers a large share of the lost robustness, sometimes exceeding full-precision performance, while retaining over 99% of the memory savings. The significance is practical: a 6 KB adapter can be transmitted over-the-air to patch deployed robots without retraining.

Core claim

The paper's central claim is twofold. First, PTQ creates a hidden Quantization-Induced Robustness Gap: models that appear reliable on in-distribution test sets degrade disproportionately under common corruptions and domain shift—up to 5.62 percentage points on ImageNet-C contrast and over 4 points on PACS sketch, while clean accuracy moves by less than 1 point. Second, this gap can be repaired by a classifier-head LoRA adapter trained only on source data. The adapter adds a low-rank correction to the frozen quantized head's logits, computed from the pre-classifier features, and recovers a substantial fraction of the lost robustness; in several PACS cases the adapted model exceeds the full-pr

What carries the argument

The load-bearing mechanism is the Classifier-LoRA adapter, a low-rank adaptation module applied to the pre-classifier features u before the head. It consists of a down-projection A and an up-projection B, scaled by alpha/r, and produces a logit-space correction added to the quantized logits: z = z_q + (alpha/r) B A u. Because B is zero-initialized, training begins exactly at the quantized model and the adapter learns a gentle feature-space calibration rather than a destructive overhaul. The frozen quantized backbone is never modified. Training minimizes cross-entropy on source labels plus an optional temperature-scaled distillation loss with a frozen full-precision teacher; a teacher-free va

Load-bearing premise

The load-bearing premise is that quantization perturbs features consistently enough that a rectification learned from clean source images transfers to corrupted or domain-shifted inputs; the paper itself concedes that when quantization destroys too much feature structure (its 2-bit breakdown), head-only repair no longer works.

What would settle it

Train Recti-Q on clean source images, then evaluate on a held-out corruption engineered to produce a feature distortion orthogonal to the clean-image quantization error (for example, a synthetic sensor artifact that rotates the dominant class-discriminative feature). If recovery drops to zero or goes negative, the source-only transfer assumption fails. A ready-made near-falsifier is the paper's own 2-bit result: once quantized features no longer retain usable class structure (in-distribution accuracy falls to 4.61%), adapter recovery collapses to +0.12 points, showing the mechanism is bounded

Watch this falsifier — get emailed when new claim-graph text bears on it.

If this is right

  • Evaluating PTQ models on clean in-distribution data alone is misleading; deployment decisions need corruption and domain-shift benchmarks to expose the robustness gap.
  • Robustness repair for compressed models does not require quantization-aware training or full retraining; a head-level adapter trained on source data can close much of the gap.
  • Because the backbone stays frozen and only the adapter changes, a robustness patch can be as small as 6 KB and transmitted over low-bandwidth links to deployed robots.
  • The technique transfers across CNN and Transformer architectures, and works without a full-precision teacher, making it feasible when only the quantized model is available.
  • The improvement is bounded by information preserved in the frozen backbone; at very low bit widths (e.g., 2-bit) the feature structure collapses and head-only rectification no longer helps.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • If the adapter is learning a stable map of quantization-induced feature distortion rather than a corruption-specific fix, the same adapter should transfer to unseen corruption types with similar feature shifts; this can be tested by training on clean images and evaluating on held-out corruption families not used in the paper.
  • The head-only design is suited to classification; extending the idea to detection or segmentation would likely need a spatially structured adapter on feature maps, which is a natural next application.
  • The 2-bit breakdown point suggests a practical diagnostic: compare the low-rank adapter recovery against bit width to find the compression level at which rectification becomes futile, potentially guiding bit-width selection for edge deployments.
  • The over-the-air patching story has a safety caveat left implicit: a 6 KB patch that improves benchmark robustness does not by itself certify performance on unanticipated physical sensor failures; validation on the target environment would still be needed.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

4 major / 4 minor

Summary. The paper claims that 4-bit post-training quantization (PTQ) preserves in-distribution accuracy but significantly degrades out-of-distribution robustness on ImageNet-C and PACS, a phenomenon it calls the 'Quantization-Induced Robustness Gap.' It proposes Recti-Q, which keeps the quantized backbone frozen and trains a small low-rank LoRA adapter on the pre-classifier features using only source data, with optional knowledge distillation from an FP32 teacher. The reported results show partial recovery of the robustness gap (e.g., +2.68 pp on contrast corruption, +0.88 to +1.62 pp on selected PACS domains) at less than 1% parameter overhead, motivating over-the-air robustness patching for edge robotics.

Significance. If the empirical claims are fully substantiated, Recti-Q would be a useful and practical contribution: it is a simple, architecture-agnostic, source-only repair method for quantized vision models, with very low parameter overhead and a plausible deployment story. The paper's protocol is leakage-free (no target data for training or model selection), and the teacher-free variant is a genuine efficiency contribution. However, the current evidence is too selectively reported to establish the central 'significant recovery' claim: the main tables show only worst-case rows, most gains are 0.27–1.62 pp with no error bars, and one table contains a suspicious +7.37 pp outlier. The significance is therefore conditional on full-benchmark reporting and a robustness analysis of the source-only transfer assumption.

major comments (4)
  1. [Section IV-B, Tables IV and V] The Quantization-Induced Robustness Gap is characterized only through 'representative worst-case domains and corruptions.' No means over all 15 ImageNet-C corruptions or all PACS domain/model pairs are reported, so the reader cannot tell whether the gap is a general phenomenon or an artifact of cherry-picked rows. Please report full-benchmark aggregates and per-corruption/per-domain tables (at least in an appendix) before claiming that PTQ 'substantially degrades reliability.'
  2. [Section IV-C, Tables II, III, VI] The recovery claim rests on point estimates without error bars or seeds. Most recoveries are under 2 pp (+0.27, +0.37, +0.42 in Table III; +0.88 to +1.62 in Table II), and the teacher-free vs. KD comparisons in Table VI differ by as little as 0.08–0.8 pp. These differences are within plausible run-to-run noise. Please provide at least three seeds with standard deviations for the main tables. The ResNet50 impulse-noise recovery (+7.37 pp in Table III) is an unexplained outlier and should either be analyzed or corrected.
  3. [Section III-A, Eq. (1)-(2); Limitations] The method's core transfer premise is that a low-rank linear correction fitted to clean source features remains valid on corrupted or out-of-domain inputs. This premise is not directly tested. The Limitations section itself concedes that head-only rectification 'depends on those features retaining usable class structure.' Please add a diagnostic, e.g., compare the adapter correction g_phi(u) against the true quantization error on OOD inputs, or ablate training with a small unlabeled corruption sample. Absent such evidence, the general-OOD recovery claim is not established.
  4. [Section IV-B, 'Breakdown point' paragraph] This paragraph is inconsistent with the tables and lacks context. It states that the mean OOD accuracy across the four Table III corruptions is 12.05%, but the four W4 accuracies shown in Table III have a mean of approximately 29.5%. It also introduces W3 and W2 experiments with no setup, no table, and an unexplained 'ID accuracy collapses to 4.61%' statement. Please reconcile these numbers or remove the passage; as written, it undermines confidence in the reported data.
minor comments (4)
  1. [General] There are several typos and formatting issues: 'aQuantization-Induced' in the abstract, the code link is given only as 'Recti-Q' with no URL, and Figure 3's overlay text is not fully readable. Please proofread.
  2. [Section IV-D.2, Table VII] The claim that 'feature-space rectification is proven necessary' is based on a single model-domain pair (DeiT-s sketch). A comparison on at least one more domain and model would make the ablation convincing.
  3. [Section III-C, Algorithm 1] Algorithm 1 lists the teacher as a required input even for teacher-free runs; the text later says teacher-free runs skip computing s_t. Please make the algorithm's control flow explicit for lambda=0.
  4. [Section IV-A] The implementation details state lambda in {0, 0.5, 1.0} but do not specify how lambda is chosen for each reported result or whether model selection on the source validation set was used. Please clarify the selection protocol.

Circularity Check

0 steps flagged

No significant circularity: Recti-Q's recovery is an experimentally measured result with source-only training and held-out OOD evaluation.

full rationale

The paper's derivation chain is empirical rather than formal. The Quantization-Induced Robustness Gap is measured as W4-vs-FP32 accuracy on ImageNet-C/PACS (Tables IV-V), and Recti-Q's recovery is measured by training a frozen-backbone LoRA adapter on source data and evaluating on held-out OOD splits (Section III-D, Tables II-III). The objective in Eq. (3) is standard cross-entropy plus optional knowledge distillation; no term in the objective contains the OOD evaluation targets, and no fitted parameter is reported as a prediction of the benchmark. The low-rank update in Eqs. (1)-(2) is an architectural choice, not derived from the results it is used to report. The paper's author self-citations ([1], [23]) are related-work context and are not load-bearing for the central claim. The 'Quantization-Induced Robustness Gap' is a label for a measured phenomenon, not an input-output equivalence. The acknowledged limitation that head-only rectification depends on features retaining class structure is an assumption about transferability, not a circular step.

Axiom & Free-Parameter Ledger

5 free parameters · 4 axioms · 0 invented entities

No new physical or architectural entities are introduced. The method contributes hyperparameters (rank, alpha, lambda, temperature) chosen by hand, and the key unstated assumption is that clean-source feature distortions match OOD distortions. The paper provides no derivation of the gap from quantization theory, so the central claim is empirical and rests on benchmark representativeness and transferability.

free parameters (5)
  • LoRA rank r = 64 (default)
    Adapter rank chosen by hand; ablation shows r=4 already gives 74% of the r=64 gain, so the default is a capacity/runtime tradeoff rather than a quantity derived from the problem.
  • LoRA alpha = 16
    Scaling applied as alpha/r; chosen by hand, not tuned per model or dataset.
  • Distillation weight lambda = 0, 0.5, or 1.0
    Selection among three values; the criterion for choosing the final lambda per model/domain is not specified, creating a potential selection lever.
  • Temperature T = 4
    Standard KD temperature chosen by hand; no sensitivity analysis reported.
  • Learning rate = 3e-4
    AdamW LR chosen by hand; no schedule sensitivity reported.
axioms (4)
  • domain assumption ImageNet-C and PACS are representative of deployment-relevant distribution shifts for robotic perception
    The entire evaluation and the Quantization-Induced Robustness Gap claim rest on these two benchmarks standing in for real sensor noise and domain shift (Section IV-A).
  • domain assumption torchao Int4WeightOnlyConfig with HQQ is a representative PTQ scheme
    The W4 baseline and all claims about PTQ rely on this single uncalibrated weight-only quantizer (Section IV-A).
  • domain assumption Quantization-induced feature distortion is consistent between clean source images and corrupted/shifted inputs, so a correction learned on source data transfers OOD
    Recti-Q trains only on clean source images yet is expected to fix OOD robustness; this transfer is the load-bearing premise of the method (Section III-A).
  • domain assumption A rank-64 LoRA on pre-classifier features has sufficient capacity to repair the quantized model without changing the backbone
    The method assumes head-only correction suffices; the paper's own limitations section states this fails under sufficiently aggressive quantization.

pith-pipeline@v1.3.0-alltime-deepseek · 12040 in / 11985 out tokens · 126084 ms · 2026-08-01T15:05:33.776749+00:00 · methodology

0 comments
read the original abstract

Robotic perception pipelines increasingly rely on large vision backbones deployed on SWaP-constrained edge platforms, making post-training quantization (PTQ) attractive for real-time inference. However, while PTQ often preserves clean in-distribution accuracy, we show that it can substantially degrade reliability under deployment-relevant distribution shifts (e.g., sensor noise, severe weather, and novel operating environments), creating a Quantization-Induced Robustness Gap. Across foundational vision benchmarks (ImageNet-C and PACS), 4-bit PTQ models exhibit pronounced robustness degradation despite negligible ID accuracy loss. To address this, we propose Recti-Q, a lightweight feature-space rectification framework that freezes the quantized backbone and trains a small classifier-head LoRA adapter using only source data. Recti-Q is architecture-agnostic across CNNs and Transformers, supports efficient teacher-free training, and recovers a significant portion of the lost robustness, in some cases matching or exceeding FP32 performance. At less than 1% parameter overhead (as small as 6 KB), Recti-Q preserves over 99% of PTQ memory savings, adds negligible compute, and enables low-bandwidth Over-The-Air (OTA) resilience patching for deployed robotic fleets operating in unpredictable physical environments.

Figures

Figures reproduced from arXiv: 2607.18540 by HamidReza Yaghoubi Araghi, Ming C. Lin, Parastoo Pilevar.

Figure 1
Figure 1. Figure 1: Quantization can severely degrade robustness under realistic perturbations. A clean “STOP” sign is corrupted and passed to three visual classification systems representative of an edge robotic perception setting. The full-precision model predicts STOP (correct). The quantized model misreads it as No Entrance (wrong). Our frame￾work, Recti-Q, keeps the quantized backbone and adds a lightweight adapter, rest… view at source ↗
Figure 2
Figure 2. Figure 2: Quantization degrades robustness under distribu￾tion shift. We plot the accuracy drop of a 4-bit quantized model (relative to its FP32 version) on ImageNet-C at severities 1, 3, and 5 (highest level). The robustness gap grows with corruption severity, showing that PTQ can sig￾nificantly reduce reliability under realistic input degradation. This issue motivates our Recti-Q framework, which recovers a substa… view at source ↗
Figure 3
Figure 3. Figure 3: Recti-Q training and inference. A frozen quantized student produces logits that are refined by a trainable LoRA adapter placed at the classifier head. The adapter applies a down-projection A and an up-projection B (scaled by α/r) to the pre-classifier features. Training uses only source￾domain images and optimizes cross-entropy on labels and a distillation term that compares adapted student logits with tea… view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

35 extracted references · 1 canonical work pages

  1. [1]

    Quantifying and modeling driving styles in trajectory forecasting,

    L. Zheng, H. Y . Araghi, T. Wu, S. Thalapanane, T. Zhou, and M. C. Lin, “Quantifying and modeling driving styles in trajectory forecasting,” in2025 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS). IEEE, 2025, pp. 12 042–12 049

    2025

  2. [2]

    Towards a science of integrated ai and robotics,

    K. Rajan and A. Saffiotti, “Towards a science of integrated ai and robotics,”Artificial Intelligence, vol. 247, pp. 1–9, 2017, special Issue on AI and Robotics. [Online]. Available: https: //www.sciencedirect.com/science/article/pii/S0004370217300310

    2017

  3. [3]

    Revolutionizing robotics with ai,

    P. Ganesan, “Revolutionizing robotics with ai,”Machine Learning, and Deep Learning: A Deep Dive into Current Trends and Challenges. J Artif Intell Mach Learn & Data Sci, vol. 1, no. 4, pp. 1124–1128, 2023

    2023

  4. [4]

    Making sense of text: artificial intelligence-enabled content analysis,

    L. W. Lee, A. Dabirian, I. P. McCarthy, and J. Kietzmann, “Making sense of text: artificial intelligence-enabled content analysis,” European Journal of Marketing, vol. 54, no. 3, pp. 615–644, 02

  5. [5]

    Quantized distillation: Optimizing driver activity recognition models for resource-constrained environments,

    C. Tanama, K. Peng, Z. Marinov, R. Stiefelhagen, and A. Roitberg, “Quantized distillation: Optimizing driver activity recognition models for resource-constrained environments,” in2023 IEEE/RSJ Interna- tional Conference on Intelligent Robots and Systems (IROS). IEEE, 2023, pp. 5479–5486

    2023

  6. [6]

    Design space exploration of low-bit quantized neural networks for visual place recognition,

    O. Grainge, M. Milford, I. Bodala, S. D. Ramchurn, and S. Ehsan, “Design space exploration of low-bit quantized neural networks for visual place recognition,”IEEE Robotics and Automation Letters, vol. 9, no. 6, pp. 5070–5077, 2024

    2024

  7. [7]

    Comprehensive survey of model compression and speed up for vision transformers,

    F. Chen, Z. Luo, L. Zhou, X. Pan, and Y . Jiang, “Comprehensive survey of model compression and speed up for vision transformers,” arXiv preprint arXiv:2404.10407, 2024

    Pith/arXiv arXiv 2024

  8. [8]

    Post-training quantization for vision transformer,

    Z. Liu, Y . Wang, K. Han, W. Zhang, S. Ma, and W. Gao, “Post-training quantization for vision transformer,”Advances in Neural Information Processing Systems, vol. 34, pp. 28 092–28 103, 2021

    2021

  9. [9]

    Benchmarking neural network robustness to common corruptions and perturbations,

    D. Hendrycks and T. Dietterich, “Benchmarking neural network robustness to common corruptions and perturbations,” 2019. [Online]. Available: https://arxiv.org/abs/1903.12261

    Pith/arXiv arXiv 2019

  10. [10]

    Deeper, broader and artier domain generalization,

    D. Li, Y . Yang, Y .-Z. Song, and T. M. Hospedales, “Deeper, broader and artier domain generalization,” inProceedings of the IEEE Inter- national Conference on Computer Vision (ICCV), Oct 2017

    2017

  11. [11]

    An image is worth 16x16 words: Transformers for image recognition at scale,

    A. Dosovitskiy, L. Beyer, A. Kolesnikov, D. Weissenborn, X. Zhai, T. Unterthiner, M. Dehghani, M. Minderer, G. Heigold, S. Gelly, J. Uszkoreit, and N. Houlsby, “An image is worth 16x16 words: Transformers for image recognition at scale,” 2021. [Online]. Available: https://arxiv.org/abs/2010.11929

    Pith/arXiv arXiv 2021

  12. [12]

    Data-free quantization through weight equalization and bias correction,

    M. Nagel, M. van Baalen, T. Blankevoort, and M. Welling, “Data-free quantization through weight equalization and bias correction,” 2019. [Online]. Available: https://arxiv.org/abs/1906.04721

    Pith/arXiv arXiv 2019

  13. [13]

    Quantized neural networks: Training neural networks with low precision weights and activations,

    I. Hubara, M. Courbariaux, D. Soudry, R. El-Yaniv, and Y . Bengio, “Quantized neural networks: Training neural networks with low precision weights and activations,” 2016. [Online]. Available: https://arxiv.org/abs/1609.07061

    Pith/arXiv arXiv 2016

  14. [14]

    TorchAO: Pytorch-native training-to-serving model optimization,

    A. Or, J. Liu, Y .-A. Li, Y .-H. Tang, H. Zuo, J. Yang, C. Shen, H. Huang, W.-C. Wu, P. Tang, P. D’Cunha, X. Jia, R. Raskaret al., “TorchAO: Pytorch-native training-to-serving model optimization,”arXiv preprint arXiv:2507.16099, 2025, published in CODEML@ICML2025. [Online]. Available: https://arxiv.org/abs/ 2507.16099

    Pith/arXiv arXiv 2025

  15. [15]

    Zero-shot adversarial quantization,

    Y . Liu, W. Zhang, and J. Wang, “Zero-shot adversarial quantization,” inProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2021, pp. 1512–1521

    2021

  16. [16]

    Adversarially robust distillation,

    M. Goldblum, L. Fowl, S. Feizi, and T. Goldstein, “Adversarially robust distillation,” inProceedings of the AAAI conference on artificial intelligence, vol. 34, no. 04, 2020, pp. 3996–4003

    2020

  17. [17]

    Robust quantization: One model to rule them all,

    B. Chmiel, R. Banner, G. Shomron, Y . Nahshan, A. Bronstein, U. Weiseret al., “Robust quantization: One model to rule them all,” Advances in neural information processing systems, vol. 33, pp. 5308– 5317, 2020

    2020

  18. [18]

    Parameter-efficient fine-tuning for pre-trained vision models: A survey and benchmark,

    Y . Xin, J. Yang, S. Luo, Y . Du, Q. Qin, K. Cen, Y . He, Z. Zhang, B. Fu, X. Yang, G. Zhai, M.-H. Yang, and X. Liu, “Parameter-efficient fine-tuning for pre-trained vision models: A survey and benchmark,”

  19. [19]

    Emergent Mind

    Adapter-based tuning. Emergent Mind. [Online]. Available: https: //www.emergentmind.com/topics/adapter-based-tuning

  20. [20]

    Lora: Low-rank adaptation of large language models,

    E. J. Hu, Y . Shen, P. Wallis, Z. Allen-Zhu, Y . Li, S. Wang, L. Wang, and W. Chen, “Lora: Low-rank adaptation of large language models,”

  21. [21]

    Efficient unsupervised domain adaptation with peft combinations,

    M. Rawhani, D. Karabo ˘ga, U. Nalbantoglu, A. Bas ¸t¨urk, and B. Akay, “Efficient unsupervised domain adaptation with peft combinations,” in2024 9th International Conference on Computer Science and Engineering (UBMK), 2024, pp. 169–174

    2024

  22. [22]

    Just train twice: Improving group robustness without training group information,

    E. Z. Liu, B. Haghgoo, A. S. Chen, A. Raghunathan, P. W. Koh, S. Sagawa, P. Liang, and C. Finn, “Just train twice: Improving group robustness without training group information,” 2021. [Online]. Available: https://arxiv.org/abs/2107.09044

    Pith/arXiv arXiv 2021

  23. [23]

    Annotation-free group robustness via loss-based resampling,

    M. Ghaznavi, H. Asadollahzadeh, H. Y . Araghi, F. H. Noohdani, M. H. Rohban, and M. S. Baghshah, “Annotation-free group robustness via loss-based resampling,”arXiv preprint arXiv:2312.04893, 2023

    Pith/arXiv arXiv 2023

  24. [24]

    Quadapter: Adapter for gpt-2 quantization,

    M. Park, J. You, M. Nagel, and S. Chang, “Quadapter: Adapter for gpt-2 quantization,” inFindings of the Association for Computational Linguistics: EMNLP 2022, 2022, pp. 2510–2517

    2022

  25. [25]

    Accurate and efficient fine-tuning of quantized large language models through optimal balance in adaptation,

    A. Shen, Z. Lai, Q. Wang, X. Li, L. Zhang, D. Li, and J. Li, “Accurate and efficient fine-tuning of quantized large language models through optimal balance in adaptation,”Transactions of the Association for Computational Linguistics, vol. 13, pp. 861–877, 2025

    2025

  26. [26]

    Repairllama: Efficient represen- tations and fine-tuned adapters for program repair,

    A. Silva, S. Fang, and M. Monperrus, “Repairllama: Efficient represen- tations and fine-tuned adapters for program repair,”IEEE Transactions on Software Engineering, 2025

    2025

  27. [27]

    Robustness feature adapter for efficient adversarial training,

    Q. Wu, J. Guo, W. Wang, and Y . Wang, “Robustness feature adapter for efficient adversarial training,”arXiv preprint arXiv:2508.17680, 2025

    Pith/arXiv arXiv 2025

  28. [28]

    Distilling the knowledge in a neural network,

    G. Hinton, O. Vinyals, and J. Dean, “Distilling the knowledge in a neural network,” 2015. [Online]. Available: https://arxiv.org/abs/1503. 02531

    2015

  29. [29]

    Decoupled weight decay regularization,

    I. Loshchilov and F. Hutter, “Decoupled weight decay regularization,”

  30. [30]

    Out-of-domain unlabeled data improves general- ization,

    A. H. Saberi, A. Najafi, A. Heidari, M. H. Movasaghinia, A. Motahari, and B. H. Khalaj, “Out-of-domain unlabeled data improves general- ization,”arXiv preprint arXiv:2310.00027, 2023

    Pith/arXiv arXiv 2023

  31. [31]

    Pytorch image models (timm),

    R. Wightman, “Pytorch image models (timm),” https://github.com/ rwightman/pytorch-image-models, 2019

    2019

  32. [2019]

    Available: https://arxiv.org/abs/1711.05101

    [Online]. Available: https://arxiv.org/abs/1711.05101

    Pith/arXiv arXiv

  33. [2020]

    Available: https://doi.org/10.1108/EJM-02-2019-0219

    [Online]. Available: https://doi.org/10.1108/EJM-02-2019-0219

    work page doi:10.1108/ejm-02-2019-0219 2019

  34. [2021]

    Available: https://arxiv.org/abs/2106.09685

    [Online]. Available: https://arxiv.org/abs/2106.09685

    Pith/arXiv arXiv

  35. [2025]

    Available: https://arxiv.org/abs/2402.02242

    [Online]. Available: https://arxiv.org/abs/2402.02242

    arXiv