On Adversarial Robustness and Out-of-Distribution Robustness of Large Language Models

April Yang; Jordan Tab; Parth Shah; Paul Kotchavong

REVIEW 4 major objections 6 minor 14 references

On Adversarial Robustness and Out-of-Distribution Robustness of Large Language Models

T0 review · 4 major / 6 minor · reviewed 2026-08-11 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read The paper claims the correlation between adversarial and out-of-distribution robustness in large language models is model-specific: neutral for LLaMA2-7b, negative for LLaMA2-13b, and positive for Mixtral-8x7b.

desk verdict Asks a good question about whether adversarial and OOD robustness transfer across LLMs, but the headline correlations are built on a handful of normalized points and fall apart under the paper's own admission. read the letter →

arxiv 2412.10535 v1 pith:5JOCWO7I submitted 2024-12-13 cs.CL cs.AI

April Yang , Jordan Tab , Parth Shah , Paul Kotchavong This is my paper · ORCID

classification cs.CLcs.AI

keywords adversarialrobustnessout-of-distributionlargelanguagemodelscorrelationanalysislinearregressionin-contextrewritingAHPnaturalinference

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper asks whether a large language model that resists adversarial attacks, inputs deliberately crafted to fool it, is also robust to out-of-distribution text, inputs unlike what it was trained on. The authors take two methods, each designed for one of these robustness types, apply both to both kinds of benchmarks across three LLMs, and measure performance with accuracy, precision, recall, and F1 scores. Their central claim is that the correlation between the two robustness types is not universal but model-specific: neutral for LLaMA2-7b, negative for LLaMA2-13b, and positive for Mixtral-8x7b. If this is right, robustness improvements do not transfer between the two settings, and defenses and evaluations must be chosen per model and per domain rather than assumed to generalize.

What carries the argument

The machinery is a per-model regression: for each model, the accuracy of three or four strategies (baseline, AHP, AHP2 where applicable, and ICR) on adversarial benchmarks is paired with the same strategies' accuracy on OOD benchmarks, min-max normalized per benchmark, and a line $Y = \beta_0 + \beta_1 X + \epsilon$ is fitted, with the slope $\beta_1$ read as the correlation coefficient between adversarial and OOD robustness. The Analytic Hierarchy Process, a prompt-driven framework that decomposes inference into safety-and-validity assessment, input refinement, and verified response synthesis, using one LLM to clean or rewrite inputs before the model answers, supplies the adversarial-oriented strategy; In-Context Rewriting, which prompts a model to rewrite OOD or perturbed inputs to look like in-distribution examples before the downstream task, supplies the OOD-oriented strategy. The regression turns the small set of strategy-benchmark accuracy points into a single signed claim, neutral, negative, or positive, for each model.

What would settle it

Recompute the regression for LLaMA2-13b without the single strategy-benchmark pair that drives the negative slope, AHP on PromptRobust, where accuracy drops from .774 to .583 while OOD accuracies rise, and see whether the correlation stays negative. More generally, add one more robustness strategy or one more benchmark per type to each model's point set: if any model's slope changes sign, the claimed neutral/negative/positive trichotomy is an artifact of the small point set rather than a property of the model.

Watch

Extended reading notes

Core claim

The paper's central discovery is that adversarial robustness and out-of-distribution robustness do not move together in any fixed way across language models. Using the slope of a regression line fitted through per-strategy, min-max-normalized accuracies on adversarial versus OOD benchmarks, the authors find a neutral correlation for LLaMA2-7b, a negative correlation for LLaMA2-13b, and a positive correlation for Mixtral-8x7b, and they attribute the differences to parameter scale and architecture. At the same time, no improvement strategy, whether the adversarial-oriented Analytic Hierarchy Process or the OOD-oriented In-Context Rewriting, improved both robustness types consistently on all benchmarks for any model, and the authors state plainly that the correlation signs are fragile, with a single benchmark's outcome able to flip the slope. The conclusion they defend is that the two robustness types are largely decoupled and interact differently depending on the model, so robustness work must be evaluated and designed at the level of a specific model and domain.

Load-bearing premise

The paper treats the slope of a line fitted through min-max-normalized accuracies from only three or four strategies per benchmark as a stable measure of the true correlation between adversarial and OOD robustness, a limitation the authors themselves state in section 8, where they note that a single benchmark's outcome can flip the sign.

Editorial extensions

If this is right

No strategy in the study improves both adversarial and OOD robustness across all models and benchmarks, so hybrid robustness frameworks that combine both kinds of defenses would be needed rather than a single universal method.
Scaling the LLaMA2 family from 7b to 13b flips the correlation from neutral to negative, so parameter count alone does not predict how the two robustness types relate.
Mixtral-8x7b, a sparse mixture-of-experts architecture, shows a positive correlation, indicating that some architectures may balance both robustness types at once.
Robustness conclusions are provisional: the paper shows that the negative correlation for LLaMA2-13b can be driven by one strategy failing on one benchmark, so evaluations should report per-benchmark, per-model results rather than pooled numbers.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

With only three or four data points per model, the regression slope measures which strategy happens to win on each benchmark family more than any intrinsic property of the model; adding strategies or benchmarks could plausibly change the sign for any of the three models.
A clean way to separate scale from architecture would be to run the same protocol on LLaMA2-70b and on a dense model of roughly Mixtral's parameter count; the paper stops at Mixtral versus the LLaMA2 pair, which confounds the two factors.
The AHP collapses on LLaMA models may be partly a prompt-formatting artifact, since the paper notes JSON-format overloading and prompt sensitivity, in which case the reported correlations inherit that artifact rather than reflecting a true robustness trade-off.
The finding connects to an open question in computer vision about whether adversarial and OOD robustness are in tension; a multilingual or multimodal extension of this protocol could test whether the model-specific trichotomy survives across modalities.

Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

You should know two things about this paper. First, the question it asks is legitimate: do robustness interventions that help against adversarial inputs also help out-of-distribution inputs, and does that relationship vary by model? That is a real gap, and the paper deserves credit for trying to look at it across model families and scales. Second, the central claim — neutral correlation for LLaMA2-7b, negative for 13b, positive for Mixtral — is not backed by the evidence in the paper. The correlation is a regression slope over min-max normalized accuracy scores from three or four strategies per benchmark. There are no error bars, no bootstrap, no significance test, and the points are not independent observations of a population; they are outcomes of specific interventions. The authors even admit that the negative LLaMA2-13b result depends heavily on one strategy’s poor performance on one benchmark. That is not a stable estimate.

What the paper does well is mostly in the details. The result tables are transparent about per-attack and per-task performance, which is more than many papers do. The discussion section is appropriately hedged, acknowledging the limited number of benchmarks and the influence of single points. The authors also flag their own sampling shortcuts, like taking the first 300 Flipkart reviews and the DDXPlus size. That honesty is worth acknowledging.

The soft spots are structural, not cosmetic. The correlation analysis is the core of the paper, and it cannot support the abstract’s plain statements of neutral, negative, and positive correlations. With only two adversarial and two OOD benchmarks, and only baseline plus two or three strategies, the regression is fitting a line through a handful of points that are themselves products of the same methods. Changing the normalization or removing one strategy flips the sign. There are also smaller issues: the appendix prompt table is referenced as Table??, the naming is inconsistent (LLaMA vs Llama, AdvGlue vs AdvGLUE++), and there is no code or data to reproduce the numbers.

This reads like a solid course project rather than a research paper. It identifies a real problem and runs some experiments, but it does not yet deliver a result that would survive scrutiny. The right call is to not send this to peer review in its current form. If the authors extend the study with more benchmarks, more independent strategies, proper uncertainty quantification, and full release of prompts and code, it could become a worthwhile contribution. For now, the abstract overstates what the evidence supports.

Referee Report

4 major / 6 minor

Summary. The paper investigates whether adversarial robustness and out-of-distribution (OOD) robustness are correlated in large language models. The authors evaluate Llama2-7b, Llama2-13b, and Mixtral-8x7b on two adversarial benchmarks (PromptRobust, AdvGLUE++) and two OOD benchmarks (Flipkart, DDXPlus), applying three robustness-enhancement strategies (AHP, AHP2, ICR). For each model they compute a linear regression slope over min-max normalized accuracy values of the strategies on adversarial-OOD benchmark pairings, interpreting the sign of the slope as the correlation between the two robustness types. They report a neutral correlation for Llama2-7b, a negative correlation for Llama2-13b, and a positive correlation for Mixtral-8x7b, and discuss implications for model architecture and size. The paper includes per-attack and per-task tables in the appendix and describes the prompts used for each strategy.

Significance. If the correlation findings were reliable, they would inform whether robustness interventions transfer across adversarial and distributional shift threat models, a question of practical importance. The paper also provides a transparent account of its experimental limitations and makes its prompt templates available in the appendix, which aids reproducibility. However, the central claims rest on a fragile statistical analysis: the correlation coefficients are slopes from regressions over only three to four strategies per benchmark, with no error bars, confidence intervals, or significance tests, and the paper itself concedes that a single benchmark-strategy point drives the negative correlation for Llama2-13b. As such, the neutral/negative/positive classification is not supported by the evidence, and the study is better characterized as a preliminary pilot than a definitive finding.

major comments (4)

[Section 8] The central claim of the paper—neutral, negative, and positive correlations—depends entirely on regression slopes computed from 3-4 min-max normalized accuracy values per benchmark, as described in Section 8. The paper provides no confidence intervals, bootstrap, or any significance test for these slopes, and the data points are not independent because each adversarial benchmark is paired with each OOD benchmark. The instability of the result is acknowledged in Section 8: the negative correlation for LLaMA2-13b is 'strongly influenced by AHP on PromptRobust', and removing that single strategy-benchmark point would likely change the sign. This fragility means the abstract's model-specific correlation claims overstate what the data actually support.
[Section 4.1.3 / Section 8] There is a direct inconsistency in the definition of the regression variables. Equation (3) states that Y is the dependent variable (Adversarial Metric) and X is the independent variable (OOD Metric), but Section 8 says 'X values are adversarial benchmark metrics and Y values are OOD benchmark metrics.' This ambiguity makes the reported slope sign impossible to interpret. Moreover, a slope from a linear regression on min-max normalized variables is not a correlation coefficient; a correlation requires standardized variables and an accompanying measure of uncertainty. As reported, the slopes do not provide a valid measure of correlation between adversarial and OOD robustness.
[Section 3.2 / Tables 3-4] The OOD benchmarks are small and possibly not OOD for all models. Flipkart is subsampled to only the first 300 reviews with character lengths between 150 and 160, and DDXPlus contains just 100 samples. The paper asserts that these datasets are out-of-distribution without checking against the models' training corpora; indeed, Section 7.2 suggests that Flipkart 'may not be as out-of-distribution for Mixtral as they are for LLaMA models.' Because the OOD axis is one of the two variables in the correlation analysis, this undermines the validity of the correlation findings.
[Section 7.3 / Section 8] The paper's own 'Higher Level Observations and Shortcomings' paragraph concedes that the negative correlation for LLaMA2-13b is driven by a single benchmark (PromptRobust AHP) and that 'further experimentation is needed.' Despite this explicit admission of instability, the abstract and conclusion present the neutral/negative/positive correlations as established results. Given the load-bearing nature of these correlations, the conclusions should be substantially weakened or the analysis extended with additional benchmarks and proper statistical inference before such claims can be made.

minor comments (6)

[Section 6.1] The prompt table reference is left as 'Table?? in the appendix', which should be resolved to the actual table number.
[Section 7.1] There is a typo: 'LLaMa2:13b outperformed it's smaller counterpart' should use 'its' instead of 'it's'.
[Section 7.2] The distinction between AHP and AHP2 is only introduced in the caption of Table 3; it should be clearly defined in the main text before the results are presented.
[Section 4.1.3] The notation for the error term ε is defined but never used later, which is harmless but could be removed for clarity.
[Section 8] The term 'neutral correlation' is never defined. A threshold or a hypothesis test should specify what slope magnitude counts as neutral versus positive or negative, especially since no significance intervals are provided.
[Section 8] The exact number of points used in each regression is not reported. The reader cannot infer whether the regressions are based on 8, 12, or 16 points, which is essential for assessing the reliability of the slope estimates.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the paper's correlation claims are transparent summaries of its own fitted regression slopes, not predictions derived from the inputs.

full rationale

The paper's central claims (neutral/negative/positive correlations) are descriptive results obtained by min-max normalizing measured accuracies and fitting a regression line; Section 8 explicitly states 'The slope of this line served as our correlation coefficient.' Since the paper defines the correlation coefficient as the regression slope and then reports the sign of that slope, the conclusion is a restatement of the measurement rather than a derivation from a claimed first principle. This is an operationalization, not circularity: no parameter is fitted to a subset and then used to predict the same data, and no load-bearing premise is justified by a self-citation (the authors cite only external prior work for models, benchmarks, and methods). The skeptical concern that a single benchmark point (AHP on PromptRobust for LLaMA2-13b) can change the sign of the slope is a statistical-validity and generalization limitation, not a circularity: it does not show that the reported slope was forced by definition or by the data's construction. Under the stated criteria, absence of circularity is the appropriate finding, so score 0.

Assumptions & free parameters 4 free parameters · 4 assumptions · 0 invented entities

The central claims rest on the fitted correlation slopes, which in turn depend on the specific strategies and small samples. The free parameters include the correlation slope itself and the normalization and sampling choices. The axioms are the assumptions that the datasets are OOD, that the regression slope is a meaningful correlation, and that the limited strategy set is representative.

free parameters (4)

Per-model correlation slope = LLaMA2-7b near zero, LLaMA2-13b negative, Mixtral positive (exact values not reported)
This slope is the central finding, fitted by linear regression to min-max normalized accuracy scores across strategies.
Min-max normalization bounds per benchmark = Not reported numerically
The normalization uses observed min and max accuracy across strategies for each benchmark, which are data-derived and influence the slope.
Flipkart subsample size = 300
Chosen by hand due to compute limits; only reviews of length 150-160 characters were considered, which may bias the dataset.
DDXPlus sample size = 100
The full dataset is only 100 samples; this small size leads to high variance in accuracy estimates.

assumptions (4)

domain assumption Both benchmark datasets are out-of-distribution for the models
Stated in section 3.2 'OOD Benchmark Assumptions'. If either dataset is actually in-distribution for a model (e.g., Flipkart for Mixtral), the OOD robustness measurements are invalid.
ad hoc to paper The linear regression slope is a valid measure of correlation between adversarial and OOD robustness
Used in section 8 without justification. The slope is not normalized to [-1,1], has no confidence intervals, and is computed from a tiny number of points.
ad hoc to paper The set of strategies evaluated (baseline, AHP, AHP2, ICR) spans the relevant robustness landscape
The correlation is derived solely from these strategies. Other interventions (e.g., adversarial training, consistency alignment) could yield different or opposite correlations.
domain assumption AHP and ICR prompts function correctly when transferred to the opposite robustness type
The authors apply AHP to OOD data and ICR to adversarial data with minimal adaptation. They note prompt overloading and inconsistency in section 7.3, which could undermine the measurements.

how reviews work

0 comments

Cite this review

Pith. "Pith review of On Adversarial Robustness and Out-of-Distribution Robustness of Large Language Models." pith.science (2026). https://pith.science/paper/5JOCWO7I

@misc{pith2026241210535,
  author       = {Pith},
  title        = {Pith review of: On Adversarial Robustness and Out-of-Distribution Robustness of Large Language Models},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/5JOCWO7I}},
  note         = {Machine review of arXiv:2412.10535}
}

read the original abstract

The increasing reliance on large language models (LLMs) for diverse applications necessitates a thorough understanding of their robustness to adversarial perturbations and out-of-distribution (OOD) inputs. In this study, we investigate the correlation between adversarial robustness and OOD robustness in LLMs, addressing a critical gap in robustness evaluation. By applying methods originally designed to improve one robustness type across both contexts, we analyze their performance on adversarial and out-of-distribution benchmark datasets. The input of the model consists of text samples, with the output prediction evaluated in terms of accuracy, precision, recall, and F1 scores in various natural language inference tasks. Our findings highlight nuanced interactions between adversarial robustness and OOD robustness, with results indicating limited transferability between the two robustness types. Through targeted ablations, we evaluate how these correlations evolve with different model sizes and architectures, uncovering model-specific trends: smaller models like LLaMA2-7b exhibit neutral correlations, larger models like LLaMA2-13b show negative correlations, and Mixtral demonstrates positive correlations, potentially due to domain-specific alignment. These results underscore the importance of hybrid robustness frameworks that integrate adversarial and OOD strategies tailored to specific models and domains. Further research is needed to evaluate these interactions across larger models and varied architectures, offering a pathway to more reliable and generalizable LLMs.

Figures

Figures reproduced from arXiv: 2412.10535 by the authors.

**Figure 1.** Entire Experiment Workflow 3.1. Model Selection Our study focuses on three large language models that represent the current state-of-the-art in natural language processing. These models were chosen for their diverse architectures and training approaches, allowing us to explore how different design choices affect robustness. The first model is Llama-2-7b, a 7 billion parameter powerful open-source language model … view at source ↗

**Figure 2.** Overview of AHP framework. The framework consists of two key components: (1) Safety and Validity Assessment, which iteratively refines input through inspection, refinement, and evaluation cycles guided by internal and external feedback; and (2) Secure Response Synthesis, which generates and verifies appropriate model outputs. The system first inspects input text for potential adversarial elements (like misspelling… view at source ↗

**Figure 3.** LLM-TTA Prompts. We leverage LLM-TTA with two prompting methods. The placeholder “style input” is replaced with the test input. To experiment with out-of-distribution (OOD) and adversarial inputs, “style transfer exemplars” is replaced with the ID or adversarial examples during the ICR task. For our project, beyond applying In-Context Rewriting to OOD benchmarks, we also tested it on Adversarial Benchmarks. This … view at source ↗

Figures from the paper (3 more)

**Figure 4.** Figure 4: Correlation of Adversarial and OOD Robustness for LLaMA2:7B We analyzed the relationship between adversarial and OOD robustness by grouping the accuracy scores by benchmark, and normalizing the scores for the 3-4 strategies we evaluated per benchmark using min-max norm…

**Figure 7.** Figure 7: Linear Correlation Coefficient (slope) vs Model Parameter Size Conversely, the positive correlation seen in Mixtral:8x7B indicates that larger models with specific architectural features might inherently balance both adversarial and OOD robustness. This could be due …

**Figure 6.** Figure 6: Correlation of Adversarial and OOD Robustness for Mixtral:8x7B that correlation between the two robustness went from neutral to negative. This suggests that increasing number of model parameters has a negative impact on the correlation between adversarial robustness …

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

14 extracted references · 3 canonical work pages

[1]

Q., Sablayrolles, A., Mensch, A., Bamford, C., Chaplot, D

Jiang, A. Q., Sablayrolles, A., Mensch, A., Bamford, C., Chaplot, D. S., Casas, D. d. l., Bressand, F., Lengyel, G., Lample, G., Saulnier, L., et al. Mixtral of experts. arXiv preprint arXiv:2401.04088, 2023

arXiv 2023
[2]

L., and Courville, A

Krueger, D., Caballero, E., Jacobsen, J.-H., Zhang, A., Binas, J., Zhang, D., Priol, R. L., and Courville, A. Out-of-distribution generalization via risk extrapolation (rex). arXiv preprint arXiv:2003.00688, 2021

arXiv 2003
[3]

Translation of Gulmanelli's seminar notes "On a theory of isotopic spin" (1954)

Liu, M., Li, Z., Xu, Z., Chen, Z., Xu, R., Xiong, J., Xia, S., and Yang, K. Enhancing adversarial robustness of llms with analytic hierarchy process. arXiv preprint arXiv:2311.08261, 2023

work page Pith review arXiv 2023
[4]

Adversarial nli: A new benchmark for natural language understanding

Nie, Y., Williams, A., Dinan, E., Bansal, M., Weston, J., and Kiela, D. Adversarial nli: A new benchmark for natural language understanding. arXiv preprint arXiv:1910.14599, 2020

arXiv 1910
[5]

Improving Black-box Robustness with In-Context Rewriting

O'Brien, K., Ng, N., Puri, I., Mendez, J., Palangi, H., Kim, Y., Ghassemi, M., and Hartvigsen, T. Improving black-box robustness with in-context rewriting, 2024. URL https://arxiv.org/abs/2402.08225

work page Pith review arXiv 2024
[6]

F., Goel, R., Wen, Z., Martel, J., and Ghosn, J

Tchango, A. F., Goel, R., Wen, Z., Martel, J., and Ghosn, J. Ddxplus: A new dataset for automatic medical diagnosis, 2022. URL https://arxiv.org/abs/2205.09148

arXiv 2022
[7]

Llama 2: Open foundation and fine-tuned chat models

Touvron, H., Lavril, T., Izacard, G., Martinet, X., Lachaux, M.-A., Lacroix, T., Rozi `e re, B., Goyal, N., Hambro, E., Azhar, F., et al. Llama 2: Open foundation and fine-tuned chat models. arXiv preprint arXiv:2307.09288, 2023

arXiv 2023
[8]

and Thummar, M

Vaghani, N. and Thummar, M. Flipkart product reviews with sentiment dataset, 2023. URL https://www.kaggle.com/datasets/niraliivaghani/flipkart-product-customer-reviews-dataset

work page 2023

Show all 14 references

[9]

Adversarial glue: A multi-task benchmark for robustness evaluation of language models

Wang, B., Zheng, C., Xu, Z., Jiang, Y., and Shen, A. Adversarial glue: A multi-task benchmark for robustness evaluation of language models. arXiv preprint arXiv:2111.02840, 2021

2021 arXiv
[10]

T., Arora, S., Mazeika, M., Hendrycks, D., Lin, Z., Cheng, Y., Koyejo, S., Song, D., and Li, B

Wang, B., Chen, W., Pei, H., Xie, C., Kang, M., Zhang, C., Xu, C., Xiong, Z., Dutta, R., Schaeffer, R., Truong, S. T., Arora, S., Mazeika, M., Hendrycks, D., Lin, Z., Cheng, Y., Koyejo, S., Song, D., and Li, B. Decodingtrust: A comprehensive assessment of trustworthiness in gp...

2024 arXiv
[11]

On the robustness of chatgpt: An adversarial and out-of-distribution perspective, 2023

Wang, J., Hu, X., Hou, W., Chen, H., Zheng, R., Wang, Y., Yang, L., Huang, H., Ye, W., Geng, X., Jiao, B., Zhang, Y., and Xie, X. On the robustness of chatgpt: An adversarial and out-of-distribution perspective, 2023. URL https://arxiv.org/abs/2302.12095

2023 arXiv
[12]

Improving the robustness of large language models via consistency alignment, 2024

Zhao, Y., Yan, L., Sun, W., Xing, G., Wang, S., Meng, C., Cheng, Z., Ren, Z., and Yin, D. Improving the robustness of large language models via consistency alignment, 2024. URL https://arxiv.org/abs/2403.14221

2024 arXiv
[13]

Z., and Xie, X

Zhu, K., Wang, J., Zhou, J., Wang, Z., Chen, H., Wang, Y., Yang, L., Ye, W., Zhang, Y., Gong, N. Z., and Xie, X. Promptrobust: Towards evaluating the robustness of large language models on adversarial prompts, 2024. URL https://arxiv.org/abs/2306.04528

2024 arXiv
[14]

write newline

" write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 gl...

Pith tools

Reviewed August 11, 2026 · model on record in the stance chip above.

[1] [1]

Q., Sablayrolles, A., Mensch, A., Bamford, C., Chaplot, D

Jiang, A. Q., Sablayrolles, A., Mensch, A., Bamford, C., Chaplot, D. S., Casas, D. d. l., Bressand, F., Lengyel, G., Lample, G., Saulnier, L., et al. Mixtral of experts. arXiv preprint arXiv:2401.04088, 2023

arXiv 2023

[2] [2]

L., and Courville, A

Krueger, D., Caballero, E., Jacobsen, J.-H., Zhang, A., Binas, J., Zhang, D., Priol, R. L., and Courville, A. Out-of-distribution generalization via risk extrapolation (rex). arXiv preprint arXiv:2003.00688, 2021

arXiv 2003

[3] [3]

Translation of Gulmanelli's seminar notes "On a theory of isotopic spin" (1954)

Liu, M., Li, Z., Xu, Z., Chen, Z., Xu, R., Xiong, J., Xia, S., and Yang, K. Enhancing adversarial robustness of llms with analytic hierarchy process. arXiv preprint arXiv:2311.08261, 2023

work page Pith review arXiv 2023

[4] [4]

Adversarial nli: A new benchmark for natural language understanding

Nie, Y., Williams, A., Dinan, E., Bansal, M., Weston, J., and Kiela, D. Adversarial nli: A new benchmark for natural language understanding. arXiv preprint arXiv:1910.14599, 2020

arXiv 1910

[5] [5]

Improving Black-box Robustness with In-Context Rewriting

O'Brien, K., Ng, N., Puri, I., Mendez, J., Palangi, H., Kim, Y., Ghassemi, M., and Hartvigsen, T. Improving black-box robustness with in-context rewriting, 2024. URL https://arxiv.org/abs/2402.08225

work page Pith review arXiv 2024

[6] [6]

F., Goel, R., Wen, Z., Martel, J., and Ghosn, J

Tchango, A. F., Goel, R., Wen, Z., Martel, J., and Ghosn, J. Ddxplus: A new dataset for automatic medical diagnosis, 2022. URL https://arxiv.org/abs/2205.09148

arXiv 2022

[7] [7]

Llama 2: Open foundation and fine-tuned chat models

Touvron, H., Lavril, T., Izacard, G., Martinet, X., Lachaux, M.-A., Lacroix, T., Rozi `e re, B., Goyal, N., Hambro, E., Azhar, F., et al. Llama 2: Open foundation and fine-tuned chat models. arXiv preprint arXiv:2307.09288, 2023

arXiv 2023

[8] [8]

and Thummar, M

Vaghani, N. and Thummar, M. Flipkart product reviews with sentiment dataset, 2023. URL https://www.kaggle.com/datasets/niraliivaghani/flipkart-product-customer-reviews-dataset

work page 2023