Pith. sign in

REVIEW 4 major objections 6 minor 2 cited by

Analysing the Robustness of Vision-Language-Models to Common Corruptions

T0 review · 4 major / 6 minor · reviewed 2026-08-16 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read The paper establishes that a transformer-based vision-language model fails in task-specific ways: blur and snow most severely degrade scene-text understanding, while frost and impulse noise most severely degrade object reasoning, and both…

desk verdict Useful but unreproducible corruption benchmark for VLMs—single model, no error bars, no release; ranking could be real but needs verification. read the letter →

arxiv 2504.13690 v2 pith:LNJXUWES submitted 2025-04-18 cs.CV

Muhammad Usama , Syeda Aishah Asim , Syed Bilal Ali , Syed Talal Wasim , Umair Bin Mansoor This is my paper · ORCID
classification cs.CV
keywords vision-languagemodelsrobustnesscommoncorruptionsImageNet-CTextVQA-CGQA-CLLaVA1.5low-frequencybias
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper sets out to map how a vision-language model's accuracy degrades when images are hit with the 19 common corruptions from ImageNet-C, at five severity levels. It contributes two new benchmarks, TextVQA-C for reading text in images and GQA-C for object-based reasoning, and runs 190 evaluations on LLaVA 1.5. The central finding is that failure modes are task-dependent: blur and snow hurt text reading most, while frost and impulse noise hurt object reasoning most, with brightness and JPEG compression doing little harm. The paper explains these rankings by the transformer's bias toward low-frequency information, arguing that corruptions which preserve low-frequency structure are tolerated and corruptions that inject high-frequency disruption are not. If correct, the result gives system builders a task-specific robustness map and points to the architectural bias a more robust vision-language model would need to correct.

What carries the argument

The load-bearing object is the ImageNet-C corruption suite, 19 corruptions in four categories (noise, blur, weather, digital) at five severity levels, applied to 5,000 TextVQA and 12,578 GQA samples to form TextVQA-C and GQA-C. The argument is carried by ranking accuracy across the five severity levels and matching those rankings to a frequency-domain classification of each corruption: self-attention acts as a weighted average, giving the transformer a low-frequency bias, so corruptions that preserve low-frequency content should degrade little and those that inject high-frequency patterns should degrade a lot. The evaluated model is LLaVA 1.5, a CLIP ViT-L/14 encoder feeding linear-projected visual tokens into a Vicuna 7B decoder.

What would settle it

Run TextVQA-C and GQA-C on a second vision-language model with a different vision encoder, such as a BLIP-2-style model with a Q-Former or a model built on a convolutional encoder, and compare the per-corruption rankings with LLaVA 1.5's; if blur and snow do not remain the worst for text tasks, or frost and impulse noise do not remain the worst for object reasoning, the claimed task-specific map and its frequency explanation fail. A more direct check is to measure whether a corruption's spectral content, classified as high-frequency versus low-frequency disruption, predicts the observed accuracy drops in the way the paper claims.

Watch

Extended reading notes

Core claim

On the paper's own terms, the discovery is that corruption robustness in a transformer vision-language model is not a single property; it depends on the task and on the spectral footprint of the corruption. Scene text understanding (TextVQA) degrades most under blur and snow corruptions, while object reasoning (GQA) is most sensitive to frost and impulse noise. The offered mechanism is that self-attention acts as a weighted average, emphasizing low frequencies, so corruptions like JPEG compression and brightness that preserve low-frequency structure cause small drops, whereas impulse noise, frost, snow, and blur that disrupt or replace high-frequency detail cause large drops. The paper presents TextVQA-C and GQA-C as reusable benchmarks, with clean baselines of 57.5% and 62.5%, so other models can be compared on the same degradation axis.

Load-bearing premise

The load-bearing premise is that LLaVA 1.5, with a CLIP ViT-L/14 encoder, a linear projection, and a Vicuna 7B decoder, represents transformer-based vision-language models generally; the paper evaluates no other model, so if other architectures, training sets, or projection layers behave differently, the vulnerability rankings and the low-frequency-bias explanation do not generalize.

Editorial extensions

If this is right

  • TextVQA-C and GQA-C become reusable benchmarks: any vision-language model can be scored on the same 19 corruptions at five severities, producing a task-specific robustness map rather than a single average.
  • Deployments that rely on reading text in images should treat blur and snow as primary failure risks, while object-reasoning deployments should treat frost and impulse noise as primary risks.
  • The claimed low-frequency bias predicts that improving text-task robustness requires protecting mid-to-high frequency bands such as character edges, while object reasoning can tolerate more low-frequency-only degradation.
  • Frequency-preserving corruptions such as JPEG compression and brightness change are the least likely to surprise a deployed system, and high-frequency disruptions are the ones to defend against.
  • Designing a more robust vision-language model would mean correcting the encoder's low-frequency bias, especially for applications that need fine-grained visual detail.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Beyond the paper: swapping the vision encoder for a convolutional backbone, while keeping the same decoder and instruction data, would test whether the low-frequency-bias explanation is causal; if the vulnerability rankings invert or flatten, the encoder's spectral behavior rather than the task is the controlling factor.
  • Beyond the paper: the severity-level anomalies, such as an accuracy rise from severity 4 to 5 for snow in TextVQA and frost in GQA, suggest that single-word exact-match scoring may be masking answer-pattern effects; a per-answer analysis would show whether these are evaluation noise or systematic reversion.
  • Beyond the paper: the spectral account implies a concrete, untested remedy: augmenting training with high-frequency-disrupting corruptions, or adding a high-pass branch to the visual encoder, should differentially improve the tasks that currently fail most.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 6 minor

Summary. The paper presents the first systematic robustness study of a vision-language model under common image corruptions. The authors construct two new benchmarks, TextVQA-C and GQA-C, by applying the 19 corruption types from ImageNet-C at five severity levels to subsets of TextVQA and GQA, and they report LLaVA 1.5 accuracy on both tasks. Their central empirical claim is a task-specific vulnerability ranking: scene text understanding (TextVQA) degrades most under blur and snow, while object reasoning (GQA) is most sensitive to frost and impulse noise. They further propose that these patterns are explained by a low-frequency processing bias of transformer-based vision encoders.

Significance. If the empirical ranking and the benchmark datasets were fully substantiated, this would be a useful contribution to VLM robustness evaluation: TextVQA-C and GQA-C address a real gap, the use of concise answers reduces evaluation ambiguity, and the task-level comparison between OCR-centric and object-centric reasoning is a sensible design. The paper also makes a falsifiable architectural claim about frequency bias that could inform future robustness work. However, the current evidence base is narrow: results come from a single model with no error bars or repeated runs, the proposed benchmarks and code are not released, and the frequency-domain mechanism is asserted rather than measured. The strengths of the paper—the benchmark construction and the clear articulation of a task-dependent robustness map—are real, but they currently outrun the evidence presented.

major comments (4)
  1. [Sections 4.1 and 5.5] The experiments evaluate only LLaVA 1.5, yet the abstract, Section 5.5, and the conclusion generalize to 'transformer-based VLMs' and 'transformer models.' LLaVA 1.5's CLIP ViT-L/14 encoder, linear projection, and Vicuna 7B decoder represent one point in a large architectural space, so the reported vulnerability ranking cannot support claims about transformer-based VLMs as a class. Either evaluate additional VLMs with different encoders, projection layers, and decoders, or restrict every claim in the abstract and conclusion to LLaVA 1.5.
  2. [Sections 4.2 and 5.3] The central empirical ranking is reported as single accuracy values with no repeated runs, confidence intervals, or error bars. Section 5.3 explicitly flags non-monotonic 'anomalous' severity points, including frost in GQA rising from severity 3 to 4 and snow in TextVQA rising from severity 4 to 5. A deterministic model evaluated on a fixed test set should not produce unexplained non-monotonicity unless there is finite-sample noise or a pipeline artifact. The paper must release per-sample results and provide bootstrap or paired statistical comparisons across severities to establish that the reported ranking is not noise.
  3. [Sections 5.1-5.5 and 6] The low-frequency-bias explanation is asserted rather than tested. The paper presents no spectral analysis of the corrupted images and no measurements of the model's internal representations, so the frequency-domain account is an interpretation consistent with selected results rather than evidence for the stated mechanism. Add quantitative support, such as Fourier power spectra of each corruption type, controlled high-pass/low-pass filtering experiments, or attention/saliency analyses, or explicitly downgrade the frequency explanation to a hypothesis.
  4. [Section 4.2] The two proposed benchmarks, TextVQA-C and GQA-C, are the paper's main reusable contribution, but no dataset download link, evaluation code, or full raw accuracy table is provided. Without these artifacts, the benchmarks cannot be used by the community and the reported numbers cannot be independently checked. The datasets, evaluation code, and complete per-corruption/per-severity accuracy tables should be released as a condition for the benchmark contribution to be verifiable.
minor comments (6)
  1. [Section 2.3] There is a duplicated and grammatically broken sentence: 'Both methods, Flamingo, and Frozen adopt the language modeling loss, where the language model generates texts conditioned on the image. have demonstrated the benefits...' This should be rewritten.
  2. [Sections 3.3, 2.4] There are several typographical issues, including 'W eather' and 'F rost' in Section 3.3 and 'naturally occuring' in Section 2.4; these should be corrected.
  3. [Figures 4-7] The accuracy plots are difficult to verify because the axis labels and legends are not legible in the provided figures, while the text cites values such as 52.7% and 51.6%; ensure figures have readable axes and legends and include the exact numeric values in a supplementary table.
  4. [Section 4.3] The evaluation protocol is underspecified: the exact matching procedure for TextVQA (normalization, synonyms, or the official TextVQA metric) and the sampling method and seed for the 5,000 TextVQA and 12,578 GQA subsets should be stated.
  5. [Section 2.4] The adversarial-attack paragraph cites several papers that are not connected to the corruption-robustness analysis; this material should be trimmed or explicitly tied to the paper's scope.
  6. [Section 4.1] The text says 'the authors generate multi-turn conversation data' when describing LLaVA's training; this should be rephrased as 'the LLaVA [27] authors,' and the LLaVA-1.5 paper should be cited specifically since the evaluated version is 1.5.

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity: the benchmark results are external measurements with no fitted parameters, self-referential equations, or load-bearing self-citations.

full rationale

The paper's central claims are empirical robustness rankings produced by applying the existing ImageNet-C corruption suite to two public VQA datasets (TextVQA and GQA) and measuring LLaVA 1.5 accuracy. No parameter is fitted to the reported outcomes, no prediction is constructed from the data it claims to predict, and no uniqueness theorem or prior result is imported from the authors' own work to force a conclusion. The equations in Section 4.1 merely restate the standard LLaVA architecture (CLIP encoding, linear projection, autoregressive decoding) and do not encode the reported vulnerability rankings. The frequency-domain explanation in Sections 5.1-5.5 is an interpretive hypothesis rather than a derivation, so it may be unsupported or untested, but it is not circular: the empirical accuracy values come from external benchmarks and are logically independent of that interpretation. The acknowledged non-monotonic 'anomalous' severity points in Section 5.3 raise reproducibility and statistical-concern issues, but those are correctness risks, not circularity. The paper is therefore self-contained against external benchmarks, and the honest finding is no significant circularity.

Assumptions & free parameters 0 free parameters · 4 assumptions · 0 invented entities

The central empirical claims rest on standard model and dataset choices rather than fitted parameters. No free parameters are fit. The main uncharged assumptions are representativeness of one model, transfer of ImageNet-C to VQA tasks, validity of exact-answer accuracy, and the low-frequency bias narrative, which is asserted rather than measured.

assumptions (4)
  • domain assumption LLaVA 1.5 is representative of transformer-based vision-language models for drawing general conclusions.
    The paper generalizes all findings to transformer-based VLMs after evaluating this single model, so the representativeness of LLaVA 1.5 is load-bearing for the broad claims.
  • domain assumption Accuracy on the selected subsets of TextVQA and GQA, with exact answer matching, measures the intended robustness property.
    The metric conflates answer-format effects with OCR or spatial-reasoning degradation, and no validation is provided that corrupted images remain answerable.
  • domain assumption ImageNet-C corruption functions transfer unchanged from classification images to natural VQA images without changing task validity.
    ImageNet-C was designed for object classification, not for OCR or spatial reasoning, so the corruptions may affect VQA tasks in ways not captured by the benchmark design.
  • ad hoc to paper Transformer encoders have a low-frequency processing bias sufficient to explain the observed robustness differences.
    This explanation is asserted repeatedly but never measured; it is introduced after the fact to fit the observed accuracy rankings.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Analysing the Robustness of Vision-Language-Models to Common Corruptions." pith.science (2026). https://pith.science/paper/LNJXUWES

@misc{pith2026250413690,
  author       = {Pith},
  title        = {Pith review of: Analysing the Robustness of Vision-Language-Models to Common Corruptions},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/LNJXUWES}},
  note         = {Machine review of arXiv:2504.13690}
}
read the original abstract

Vision-language models (VLMs) have demonstrated impressive capabilities in understanding and reasoning about visual and textual content. However, their robustness to common image corruptions remains under-explored. In this work, we present the first comprehensive analysis of VLM robustness across 19 corruption types from the ImageNet-C benchmark, spanning four categories: noise, blur, weather, and digital distortions. We introduce two new benchmarks, TextVQA-C and GQA-C, to systematically evaluate how corruptions affect scene text understanding and object-based reasoning, respectively. Our analysis reveals that transformer-based VLMs exhibit distinct vulnerability patterns across tasks: text recognition deteriorates most severely under blur and snow corruptions, while object reasoning shows higher sensitivity to corruptions such as frost and impulse noise. We connect these observations to the frequency-domain characteristics of different corruptions, revealing how transformers' inherent bias toward low-frequency processing explains their differential robustness patterns. Our findings provide valuable insights for developing more corruption-robust vision-language models for real-world applications.

Figures

Figures reproduced from arXiv: 2504.13690 by the authors.

Figure 1
Figure 1. Different noise types across severity levels [PITH_FULL_IMAGE:figures/full_fig_p006_1.png] view at source ↗
Figure 2
Figure 2. Overview of the LLaVA architecture Hv = W · Zv (2) These projected visual tokens Hv are treated equivalently to text tokens and are concatenated with the natural language input tokens before being passed to the LLM. LLaVA [27] employs Vicuna [26], a fine-tuned version of LLaMA 2.0 7B [25], as its lan￾guage model backbone due to its strong instruction-following capabilities. This design ensures that visual and textua… view at source ↗
Figure 3
Figure 3. Sample image-text pairs from TextVQA and GQA. [PITH_FULL_IMAGE:figures/full_fig_p010_3.png] view at source ↗
Figures from the paper (4 more)
Figure 4
Figure 4. Figure 4: Accuracy Results for the Noise Corruption Category [PITH_FULL_IMAGE:figures/full_fig_p011_4.png]
Figure 5
Figure 5. Figure 5: Accuracy Results for the Blur Corruption Category [PITH_FULL_IMAGE:figures/full_fig_p013_5.png]
Figure 6
Figure 6. Figure 6: Accuracy Results for the Weather Corruption Category [PITH_FULL_IMAGE:figures/full_fig_p014_6.png]
Figure 7
Figure 7. Figure 7: Accuracy Results for the Digital Corruption Category [PITH_FULL_IMAGE:figures/full_fig_p015_7.png]

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 2 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. BRUCE: Benchmarking Robustness Under Corruption Escalation for Scientific Vision-Language Reasoning

    cs.CV 2026-08 unverdicted novelty 6.0 of 10

    BRUCE is a new benchmark with metrics RCI and T-RCI that quantify how vision-language model reasoning for chemistry and math degrades as image corruption escalates.

  2. Diagnosing Corruption-Induced Reliability Failures in Vision-Language Models

    cs.CV 2025-11 conditional novelty 6.0 of 10

    Mild visual corruption can boost a vision-language model's top-1 accuracy while its confidence–correctness alignment (measured by the new RAS score) degrades.

Reference graph

Works this paper leans on

39 extracted references · 24 canonical work pages · cited by 2 Pith papers

  1. [1]

    Benchmarking neural network robustness to common corruptions and perturbations,

    D. Hendrycks and T. Dietterich, “Benchmarking neural network robustness to common corruptions and perturbations,” in ICLR, 2019

    work page 2019

  2. [2]

    Learning transferable visual models from natural language supervision,

    A. Radford, J. W. Kim, C. Hallacy, A. Ramesh, G. Goh, S. Agarwal, G. Sastry, A. Askell, P. Mishkin, J. Clark, et al., “Learning transferable visual models from natural language supervision,” in Interna- tional conference on machine learning , pp. 8748–8763, PmLR, 2021

    work page 2021

  3. [3]

    Lxmert: Learning cross-modality encoder representations from transformers,

    H. Tan and M. Bansal, “Lxmert: Learning cross-modality encoder representations from transformers,” arXiv preprint arXiv:1908.07490 , 2019

    arXiv 1908

  4. [4]

    Encoder-decoder architecture for supervised dynamic graph learning: A survey,

    Y. Zhu, F. Lyu, C. Hu, X. Chen, and X. Liu, “Encoder-decoder architecture for supervised dynamic graph learning: A survey,” arXiv preprint arXiv:2203.10480 , 2022

    arXiv 2022

  5. [5]

    Uninet: Unified architecture search with convolution, transformer, and mlp,

    J. Liu, X. Huang, G. Song, H. Li, and Y. Liu, “Uninet: Unified architecture search with convolution, transformer, and mlp,” in European Conference on Computer Vision , pp. 33–49, Springer, 2022

    work page 2022

  6. [6]

    Image difference captioning with pre-training and contrastive learning,

    L. Yao, W. Wang, and Q. Jin, “Image difference captioning with pre-training and contrastive learning,” in Proceedings of the AAAI Conference on Artificial Intelligence , 2022

    work page 2022

  7. [7]

    MELM: Data Augmentation with Masked Entity Language Modeling for Low-Resource NER

    R. Zhou, X. Li, R. He, L. Bing, E. Cambria, L. Si, and C. Miao, “Melm: Data augmentation with masked entity language modeling for low-resource ner,” arXiv preprint arXiv:2108.13655 , 2021

    work page Pith review arXiv 2021

  8. [8]

    Point-bert: Pre-training 3d point cloud transformers with masked point modeling,

    X. Yu, L. Tang, Y. Rao, T. Huang, J. Zhou, and J. Lu, “Point-bert: Pre-training 3d point cloud transformers with masked point modeling,” in Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp. 19313–19322, 2022

    work page 2022

Show all 39 references
  1. [9]

    Ensemble deep learning for automated visual classification using eeg signals,

    X. Zheng, W. Chen, Y. You, Y. Jiang, M. Li, and T. Zhang, “Ensemble deep learning for automated visual classification using eeg signals,” Pattern Recognition, vol. 102, p. 107147, 2020

    2020

  2. [10]

    Lit: Zero-shot transfer with locked-image text tuning,

    X. Zhai, X. Wang, B. Mustafa, A. Steiner, D. Keysers, A. Kolesnikov, and L. Beyer, “Lit: Zero-shot transfer with locked-image text tuning,” in Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp. 18123–18133, 2022

    2022

  3. [11]

    Multimodal few-shot learning with frozen language models,

    M. Tsimpoukelli, J. L. Menick, S. Cabi, S. Eslami, O. Vinyals, and F. Hill, “Multimodal few-shot learning with frozen language models,” Advances in Neural Information Processing Systems , vol. 34, pp. 200–212, 2021

    2021

  4. [12]

    Blip-2: Bootstrapping language-image pre-training with frozen im- age encoders and large language models,

    J. Li, D. Li, S. Savarese, and S. Hoi, “Blip-2: Bootstrapping language-image pre-training with frozen im- age encoders and large language models,” in International conference on machine learning, pp. 19730– 19742, PMLR, 2023

    2023

  5. [13]

    Bert: Pre-training of deep bidirectional trans- formers for language understanding,

    J. Devlin, M.-W. Chang, K. Lee, and K. Toutanova, “Bert: Pre-training of deep bidirectional trans- formers for language understanding,” in Proceedings of the 2019 conference of the North American chapter of the association for computational linguistics: human language technolo...

    2019

  6. [14]

    Language models are unsu- pervised multitask learners,

    A. Radford, J. Wu, R. Child, D. Luan, D. Amodei, I. Sutskever, et al., “Language models are unsu- pervised multitask learners,” OpenAI blog, vol. 1, no. 8, p. 9, 2019

    2019

  7. [15]

    Exploring the limits of transfer learning with a unified text-to-text transformer,

    C. Raffel, N. Shazeer, A. Roberts, K. Lee, S. Narang, M. Matena, Y. Zhou, W. Li, and P. J. Liu, “Exploring the limits of transfer learning with a unified text-to-text transformer,” Journal of machine learning research, vol. 21, no. 140, pp. 1–67, 2020

    2020

  8. [16]

    Language models are few- shot learners,

    T. Brown, B. Mann, N. Ryder, M. Subbiah, J. D. Kaplan, P. Dhariwal, A. Neelakantan, P. Shyam, G. Sastry, A. Askell, S. Agarwal, A. Herbert-Voss, G. Krueger, T. Henighan, R. Child, A. Ramesh, D. Ziegler, J. Wu, C. Winter, C. Hesse, M. Chen, E. Sigler, M. Litwin, S. Gray, B. Che...

    1901

  9. [17]

    Using deepspeed and megatron to train megatron-turing nlg 530b, a large-scale generative language model,

    S. Smith, M. Patwary, B. Norick, P. LeGresley, S. Rajbhandari, J. Casper, Z. Liu, S. Prabhumoye, G. Zerveas, V. Korthikanti, et al., “Using deepspeed and megatron to train megatron-turing nlg 530b, a large-scale generative language model,” arXiv preprint arXiv:2201.11990 , 2022

    2022 arXiv

  10. [18]

    Training compute-optimal large language models,

    J. Hoffmann, S. Borgeaud, A. Mensch, E. Buchatskaya, T. Cai, E. Rutherford, D. d. L. Casas, L. A. Hendricks, J. Welbl, A. Clark,et al., “Training compute-optimal large language models,”arXiv preprint arXiv:2203.15556, 2022

    2022 arXiv

  11. [19]

    Palm: Scaling language modeling with pathways,

    A. Chowdhery, S. Narang, J. Devlin, M. Bosma, G. Mishra, A. Roberts, P. Barham, H. W. Chung, C. Sutton, S. Gehrmann, et al., “Palm: Scaling language modeling with pathways,”Journal of Machine Learning Research, vol. 24, no. 240, pp. 1–113, 2023

    2023

  12. [20]

    Opt: Open pre-trained transformer language models,

    S. Zhang, S. Roller, N. Goyal, M. Artetxe, M. Chen, S. Chen, C. Dewan, M. Diab, X. Li, X. V. Lin, et al., “Opt: Open pre-trained transformer language models,” arXiv preprint arXiv:2205.01068 , 2022

    2022 arXiv

  13. [21]

    Bloom: A 176b-parameter open-access multilingual language model,

    B. Workshop, T. L. Scao, A. Fan, C. Akiki, E. Pavlick, S. Ili´ c, D. Hesslow, R. Castagn´ e, A. S. Luccioni, F. Yvon, et al., “Bloom: A 176b-parameter open-access multilingual language model,” arXiv preprint arXiv:2211.05100, 2022

    2022 arXiv

  14. [22]

    Emergent abilities of large language models,

    J. Wei, Y. Tay, R. Bommasani, C. Raffel, B. Zoph, S. Borgeaud, D. Yogatama, M. Bosma, D. Zhou, D. Metzler, et al. , “Emergent abilities of large language models,” arXiv preprint arXiv:2206.07682 , 2022

    2022 arXiv

  15. [23]

    Training language models to follow instructions with human feedback,

    L. Ouyang, J. Wu, X. Jiang, D. Almeida, C. Wainwright, P. Mishkin, C. Zhang, S. Agarwal, K. Slama, A. Ray, et al., “Training language models to follow instructions with human feedback,” Advances in neural information processing systems, vol. 35, pp. 27730–27744, 2022

    2022

  16. [24]

    Openai: Introducing chatgpt

    OpenAI, “Openai: Introducing chatgpt.” https://openai.com/index/chatgpt, 2022

    2022

  17. [25]

    Llama: Open and efficient foundation language models,

    H. Touvron, T. Lavril, G. Izacard, X. Martinet, M.-A. Lachaux, T. Lacroix, B. Rozi` ere, N. Goyal, E. Hambro, F. Azhar, et al., “Llama: Open and efficient foundation language models,” arXiv preprint arXiv:2302.13971, 2023

    2023 arXiv

  18. [26]

    Vicuna: An open-source chatbot impressing gpt-4 with 90%* chatgpt quality,

    W.-L. Chiang, Z. Li, Z. Lin, Y. Sheng, Z. Wu, H. Zhang, L. Zheng, S. Zhuang, Y. Zhuang, J. E. Gonzalez, I. Stoica, and E. P. Xing, “Vicuna: An open-source chatbot impressing gpt-4 with 90%* chatgpt quality,” March 2023

    2023

  19. [27]

    Visual instruction tuning,

    H. Liu, C. Li, Q. Wu, and Y. J. Lee, “Visual instruction tuning,” in NeurIPS, 2023

    2023

  20. [28]

    Minigpt-4: Enhancing vision-language under- standing with advanced large language models,

    D. Zhu, J. Chen, X. Shen, X. Li, and M. Elhoseiny, “Minigpt-4: Enhancing vision-language under- standing with advanced large language models,” arXiv preprint arXiv:2304.10592 , 2023

    2023 arXiv

  21. [29]

    Visualgpt: Data-efficient adaptation of pretrained language models for image captioning,

    J. Chen, H. Guo, K. Yi, B. Li, and M. Elhoseiny, “Visualgpt: Data-efficient adaptation of pretrained language models for image captioning,” in Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp. 18030–18040, 2022

    2022

  22. [30]

    Flamingo: a visual language model for few-shot learning,

    J.-B. Alayrac, J. Donahue, P. Luc, A. Miech, I. Barr, Y. Hasson, K. Lenc, A. Mensch, K. Millican, M. Reynolds, et al. , “Flamingo: a visual language model for few-shot learning,” Advances in neural information processing systems, vol. 35, pp. 23716–23736, 2022

    2022

  23. [31]

    Palm-e: An embodied multimodal language model,

    D. Driess, F. Xia, M. S. Sajjadi, C. Lynch, A. Chowdhery, A. Wahid, J. Tompson, Q. Vuong, T. Yu, W. Huang, et al. , “Palm-e: An embodied multimodal language model,” arXiv preprint arXiv:2303.03378, 2023

    2023 arXiv

  24. [32]

    Gpt-4 technical report,

    O. AI, “Gpt-4 technical report,” arXiv preprint arXiv:2303.08774 , 2023

    2023 arXiv

  25. [33]

    One pixel attack for fooling deep neural networks,

    J. Su, D. V. Vargas, and K. Sakurai, “One pixel attack for fooling deep neural networks,” IEEE Transactions on Evolutionary Computation , vol. 23, no. 5, pp. 828–841, 2019

    2019

  26. [34]

    Deepfool: a simple and accurate method to fool deep neural networks,

    S.-M. Moosavi-Dezfooli, A. Fawzi, and P. Frossard, “Deepfool: a simple and accurate method to fool deep neural networks,” in Proceedings of the IEEE conference on computer vision and pattern recog- nition, pp. 2574–2582, 2016

    2016

  27. [35]

    Tracing the origin of adversar- ial attack for forensic investigation and deterrence,

    H. Fang, J. Zhang, Y. Qiu, J. Liu, K. Xu, C. Fang, and E.-C. Chang, “Tracing the origin of adversar- ial attack for forensic investigation and deterrence,” in Proceedings of the IEEE/CVF International Conference on Computer Vision , pp. 4335–4344, 2023

    2023

  28. [36]

    The best defense is a good offense: Adversarial augmentation against adversar- ial attacks,

    I. Frosio and J. Kautz, “The best defense is a good offense: Adversarial augmentation against adversar- ial attacks,” in Proceedings of the IEEE/CVF conference on computer vision and pattern recognition , pp. 4067–4076, 2023

    2023

  29. [37]

    Defending against patch-based backdoor attacks on self-supervised learning,

    A. Tejankar, M. Sanjabi, Q. Wang, S. Wang, H. Firooz, H. Pirsiavash, and L. Tan, “Defending against patch-based backdoor attacks on self-supervised learning,” inProceedings of the IEEE/CVF conference on computer vision and pattern recognition , pp. 12239–12249, 2023

    2023

  30. [38]

    Explaining and harnessing adversarial examples,

    I. J. Goodfellow, J. Shlens, and C. Szegedy, “Explaining and harnessing adversarial examples,” arXiv preprint arXiv:1412.6572, 2014

    2014 arXiv

  31. [39]

    Sibling-attack: Rethinking transferable adversarial attacks against face recognition,

    Z. Li, B. Yin, T. Yao, J. Guo, S. Ding, S. Chen, and C. Liu, “Sibling-attack: Rethinking transferable adversarial attacks against face recognition,” in Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp. 24626–24637, 2023. Authors M. Usama is...

    2023

Pith tools

Reviewed August 16, 2026 · model on record in the stance chip above.