REVIEW 4 major objections 4 minor 3 references
Evaluating the performance and fragility of large language models on the self-assessment for neurological surgeons
T0 review · 4 major / 4 minor · reviewed 2026-08-07 · deepseek-v4-flash
Pith's one-line read Inserting one clinically irrelevant sentence into a neurosurgery board question lowers accuracy for every LLM tested, by up to 20.4%.
desk verdict Solid extension of the known distraction result to neurosurgery boards, but the distractor design risks conflating lexical priming of wrong answers with general fragility. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The SANS-Noisy distraction framework: it extracts a clinically relevant term from an incorrect answer option, uses GPT-4o to generate a short sentence that uses that medically polysemous word in a clinically irrelevant context, and algorithmically interweaves the sentence into the original question. Because the injected term comes from a wrong answer, it is medically familiar but carries no diagnostic or therapeutic information; the framework isolates how well a model can ignore irrelevant text while answering a medical multiple-choice question.
What would settle it
Ask a panel of neurosurgeons to judge each of the 2,904 augmented questions on whether the inserted sentence is clinically irrelevant and free of clues to the correct answer; if a nontrivial fraction are judged to alter the question or hint at the answer, the measured accuracy drop conflates distraction with question corruption. A complementary test: replace the injected sentence with a random, non-medical filler sentence; if the accuracy drop largely disappears, the effect is specific to clinically familiar polysemous distractors rather than any inserted text.
Extended reading notes
Core claim
The central claim is that LLM performance on clean, highly structured medical board questions substantially overestimates their ability to handle the noisy text of real clinical work. The authors demonstrate this by constructing a distractor version of the SANS benchmark: for each question they take a clinically relevant term from an incorrect answer option, ask GPT-4o to reuse that term in a fluent sentence about a non-clinical context (e.g., 'The patient's zodiac sign is Cancer'), and insert the sentence into the question. Across 28 models, accuracy dropped on the augmented questions for every single model, with the largest decline 20.4% and one model falling below the board-passing threshold. The loss was negatively correlated with baseline accuracy ($\rho=-0.66$), and open-source models, particularly general-purpose ones, were substantially more fragile than proprietary models. The paper concludes that current LLMs lack the distraction-filtering capability that clinicians take for granted.
Load-bearing premise
The load-bearing premise is that the GPT-4o-generated sentences are truly clinically irrelevant and do not change the question's meaning or leak the correct answer; the paper reports no human validation of the distractors.
Editorial extensions
If this is right
- Board-style benchmarks that present only clean, structured questions will keep overstating LLM readiness for clinical environments unless they include explicit noise stress tests.
- Ambient AI documentation tools that insert templated, copy-forwarded, or tangential text into notes may degrade the performance of downstream LLMs that read those notes.
- Open-source general-purpose models, which had the lowest baseline accuracy and the largest distraction-induced drops, need targeted resilience training before being considered for clinical decision support.
- Neuropathology questions were disproportionately vulnerable in general open-source models, so robustness testing should be disaggregated by subspecialty section rather than reported as a single average.
Reading between the lines
- Because the injected distractors are built from terms in the wrong answer options, the noisy question is semantically tilted toward the distractor set; a follow-up study that places the same sentences after the correct answer, or scrambles their position, would isolate whether the drop comes from lexical association or from general attention capture.
- The distractor sentences test lexical disambiguation (e.g., 'Cancer' as a zodiac sign), which is one narrow form of noise; real clinical notes contain semantically relevant but tangential details, so the 20.4% figure may be either an underestimate or an overestimate of real-world degradation, and a corpus of actual notes would settle the direction.
- The paper reports no human validation of the distractors; a neurosurgeon panel rating whether each inserted sentence is truly irrelevant and free of hints would distinguish pure distraction-fragility from accidental question corruption, and the authors' own limitations section implicitly invites this check.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper evaluates 28 large language models on 2,904 text-only CNS-SANS neurosurgery board-style questions, both in their original form and after inserting distractor sentences generated by GPT-4o. The distractor sentences are constructed by taking a clinically relevant term from an incorrect answer option and embedding it into a non-clinical sentence inserted into the question stem. The authors report that 6 of 28 models achieved board-passing outcomes, that accuracy dropped by up to 20.4% when distractors were added, that one previously passing model fell below the passing threshold, and that general-purpose open-source models lost more accuracy than proprietary models. The paper interprets these results as evidence that LLMs are markedly vulnerable to extraneous, irrelevant information in clinical text.
Significance. If the central claim holds, the study provides a large-scale, domain-specific demonstration that clean MCQ accuracy overstates the reliability of LLMs in noisy clinical settings, and it extends prior work by the same group to neurosurgery with a broader model sample. The use of the complete CNS-SANS text bank, 28 models across three classes, and category-level analyses are concrete strengths, as is the explicit comparison of open-source and proprietary model families. However, the central claim depends on the distractors being truly irrelevant and non-priming; the current experimental design does not separate general fragility to noise from targeted lexical bias toward the injected answer-option term, and several reporting gaps (unnamed models, unspecified passing threshold, no human validation of distractors) weaken the support for the headline conclusions.
major comments (4)
- [Methods, 'SANS with distraction benchmark curation' (Fig. 1)] Every distractor sentence is built from a clinically relevant term extracted from an incorrect answer option, and that exact term is then embedded into the question stem. This creates a lexical-overlap confound: the augmented prompt contains a token that exactly matches one of the wrong choices, and transformer-based MCQ models are known to be sensitive to lexical overlap between prompt and options. The observed accuracy drops (up to 20.4%) may therefore reflect targeted priming of the injected option rather than general fragility to irrelevant prose. The paper reports no control condition with distractors that contain no vocabulary from the answer options, and the Limitations section does not mention this confound. Without such a control, the central claim that 'irrelevant' distractions degrade performance is not fully established.
- [Abstract and Results] The board-passing threshold is never specified in the manuscript, so the claims that 6 of 28 models passed and that one model fell below the threshold after distraction cannot be independently evaluated. The authors should state the exact passing score or cutoff used, identify the model that crossed the threshold, and report its baseline and distracted accuracies. Without this information, the board-passing and 'one model failing' claims are not auditable.
- [Methods, 'SANS with distraction benchmark curation'] The distractor sentences are generated by GPT-4o and are not validated by human raters for clinical irrelevance, fluency, or preservation of the original question's meaning. Because the entire fragility argument rests on the premise that these sentences are pure noise, the paper needs at least a clinician audit of a sample of augmented questions, ideally with inter-rater agreement. Absent this, the measured accuracy drops could be caused by altered question semantics or accidental clues rather than by irrelevant text.
- [Methods, 'Model evaluation' and Results] The identities of the 28 models, their exact versions, and their classification into general open-source, medical open-source, and proprietary are not listed in the text or in a table; the model labels in Fig. 2 are not readable in the provided version. This makes the group-level comparisons (n = 19, n = 3, n = 6) and the headline numbers difficult to audit. In addition, the statement that models with disallowed temperature control were run with providers' default inference parameters means that not all evaluations were deterministic; the models run under stochastic default sampling should be identified, and their results should be accompanied by variance estimates or repeated runs.
minor comments (4)
- [Methods, 'Prompting'] The text says prompting follows 'Vishwanath et al15', but reference 15 is Omar et al.; the intended citation appears to be reference 16 (Vishwanath et al., 'Medical large language models are easily distracted'). Please correct the reference numbering.
- [Methods, 'SANS with distraction benchmark curation'] The example distractor 'The patient's zodiac sign is Cancer' capitalizes the term, which could itself act as a clue that the word is used in a non-clinical sense. The methods should specify how capitalization and formatting were handled when embedding distractor terms.
- [Limitations] The Limitations section mentions that only a single distractor style was examined, but it does not mention the lexical-overlap confound or the absence of human validation of distractors; these should be acknowledged explicitly.
- [Results, Fig. 2 and Fig. 3] The figures, particularly the model labels in Fig. 2, are difficult to read in the provided version. A supplemental table with model names, versions, inference configurations, baseline accuracies, and distracted accuracies would substantially improve clarity and reproducibility.
Circularity Check
Empirical benchmark study; no derivation chain reduces to its inputs. The only self-citation is a prior distraction-framework paper by the same group, and it is not load-bearing because the current results are re-measured on an external CNS-SANS benchmark.
full rationale
The paper's central claims are direct measurements: 28 LLMs are scored on 2,904 CNS-SANS questions with and without injected distractor sentences, and accuracy differences are reported. No parameter is fitted to the outcome and then renamed a prediction; the benchmark is an external question bank supplied by the CNS, and the board-passing thresholds are external standards. The distractor construction is a fixed rule (extract a term from a wrong answer option, have GPT-4o embed it in a non-clinical sentence, insert into the stem) applied identically to all models, so the reported accuracy drop is an empirical result rather than a tautology. The only notable self-citation is the prior same-group paper 'Medical large language models are easily distracted' (ref. 16), which introduced the general distraction framework and reported an 18% drop; the present paper does not rely on that earlier result to establish its own numbers but instead re-runs an independent evaluation on a different, external benchmark. The prompting section also cites 'Vishwanath et al15' for prompt engineering, but that is a methodological provenance citation, and the evaluation itself is described in the current Methods; even if the citation is a numbering error, it does not supply the measured accuracy values. A genuine validity concern exists: because distractor sentences are generated from the exact terms appearing in incorrect answer choices, the observed degradation may partly reflect lexical priming toward those options rather than general fragility to irrelevant prose. That confound is not acknowledged in the Limitations section, which only mentions the single distractor style. However, a confound is not circularity: it weakens the interpretation of the measurement, but the measurement is not equivalent to its input by construction. Accordingly, the circularity score is low.
Assumptions & free parameters
free parameters (1)
- Board-passing threshold
assumptions (3)
- domain assumption CNS-SANS text-only questions are a representative benchmark of neurosurgical board knowledge.
- ad hoc to paper GPT-4o-generated distractor sentences are clinically irrelevant and do not alter the correct answer.
- domain assumption Exact string matching of chain-of-thought output is a valid measure of correct answer selection.
Cite this review
Pith. "Pith review of Evaluating the performance and fragility of large language models on the self-assessment for neurological surgeons." pith.science (2026). https://pith.science/paper/ARM4YTP7
@misc{pith2026250523477,
author = {Pith},
title = {Pith review of: Evaluating the performance and fragility of large language models on the self-assessment for neurological surgeons},
year = {2026},
howpublished = {\url{https://pith.science/paper/ARM4YTP7}},
note = {Machine review of arXiv:2505.23477}
}
read the original abstract
The Congress of Neurological Surgeons Self-Assessment for Neurological Surgeons (CNS-SANS) questions are widely used by neurosurgical residents to prepare for written board examinations. Recently, these questions have also served as benchmarks for evaluating large language models' (LLMs) neurosurgical knowledge. This study aims to assess the performance of state-of-the-art LLMs on neurosurgery board-like questions and to evaluate their robustness to the inclusion of distractor statements. A comprehensive evaluation was conducted using 28 large language models. These models were tested on 2,904 neurosurgery board examination questions derived from the CNS-SANS. Additionally, the study introduced a distraction framework to assess the fragility of these models. The framework incorporated simple, irrelevant distractor statements containing polysemous words with clinical meanings used in non-clinical contexts to determine the extent to which such distractions degrade model performance on standard medical benchmarks. 6 of the 28 tested LLMs achieved board-passing outcomes, with the top-performing models scoring over 15.7% above the passing threshold. When exposed to distractions, accuracy across various model architectures was significantly reduced-by as much as 20.4%-with one model failing that had previously passed. Both general-purpose and medical open-source models experienced greater performance declines compared to proprietary variants when subjected to the added distractors. While current LLMs demonstrate an impressive ability to answer neurosurgery board-like exam questions, their performance is markedly vulnerable to extraneous, distracting information. These findings underscore the critical need for developing novel mitigation strategies aimed at bolstering LLM resilience against in-text distractions, particularly for safe and effective clinical deployment.
Figures
Reference graph
Works this paper leans on
-
[10]
Guerra GA, Hofmann H, Sobhani S, et al. GPT-4 Artificial Intelligence Model Outperforms ChatGPT, Medical Students, and Neurosurgery Residents on Neurosurgery Written Board-Like Questions. World Neurosurg. 2023;179:e160-e165. doi:10.1016/j.wneu.2023.08.042 11. Ali R, Tang OY, Connolly ID, et al. Performance of ChatGPT, GPT-4, and Google Bard on a Neurosurg...
-
[12]
Other General Only respond with the topic number (1-12) that BEST fits with the medical question. Do not include any other text in your response. 1 Supplemental Digital Content 2, Figure. Correlation between baseline accuracy and accuracy loss under distractors present of each LLM by model classification. Each circle represents a different LLM, and points ...
-
[19]
Current and Potential Applications of Ambient Artificial Intelligence
Nahar JK, Kachnowski S. Current and Potential Applications of Ambient Artificial Intelligence. Mayo Clin Proc Digit Health. 2023;1(3):241-246. doi:10.1016/j.mcpdig.2023.05.003 20. Zhou J, Zhang J, Wan R, et al. Integrating AI into clinical education: evaluating general practice trainees’ proficiency in distinguishing AI-generated hallucinations and impact...
Reviewed August 7, 2026 · model on record in the stance chip above.
Discussion (0). Sign in to comment.