Pith. sign in

REVIEW 3 major objections 1 minor 32 references

LLM clinical triage failures arise from multiple-choice output format rather than missing medical knowledge.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · grok-4.3

2026-06-29 07:43 UTC pith:B4WNMLNO

load-bearing objection The paper uses SAEs to claim triage errors come from output format rather than clinical knowledge, but the evidence that representations stay invariant is too thin to carry the conclusion. the 3 major comments →

arxiv 2605.29889 v1 pith:B4WNMLNO submitted 2026-05-28 cs.CL cs.AI

Internal Representation, Not Clinical Knowledge: Where Apparent LLM Triage Failures Originate

David Fraile Navarro , Berardino Como , Jialei Sheng , Soundariya Ananthan , Shlomo Berkovsky This is my paper
classification cs.CL cs.AI
keywords LLM triagesparse autoencodersoutput format effectsclinical representationmultiple-choice biasunder-triageGemmaQwen
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper investigates whether high under-triage rates on multiple-choice clinical benchmarks reflect gaps in LLMs' clinical knowledge or a disruption in how preserved knowledge maps to constrained outputs. Sparse autoencoder analysis shows that medical features activate identically on the patient narrative under both free-text and multiple-choice conditions. At the decision token in multiple-choice settings, however, those medical features go silent while format and scaffold features dominate the logits. Behavioral tests confirm the error pattern inverts with input structure changes and consists mostly of off-by-one acuity mistakes rather than outright knowledge errors.

Core claim

The same medical SAE features activate on the shared clinical narrative in both output formats, yet at the multiple-choice decision token these medical features remain silent across models while scaffold and format features control the decision logits; logit attribution and verbalization methods confirm that format features, not medical ones, drive the errors, so the triage failure originates in the output format and not in the clinical representation.

What carries the argument

Sparse autoencoder features that distinguish medical content activations from output-format scaffold activations, tracked from narrative tokens through the decision token with logit attribution.

Load-bearing premise

That identical medical SAE feature activations across formats demonstrate a preserved clinical representation, and that their silence at the multiple-choice decision token combined with format-feature dominance proves the representation is not the source of the error.

What would settle it

Observing medical SAE features that activate strongly at the multiple-choice decision token and contribute substantially to the output logits would falsify the claim.

Watch this falsifier — get emailed when new claim-graph text bears on it.

If this is right

  • The multiple-choice performance penalty inverts when input is presented in either structured or natural-language form.
  • Option-order shuffling does not account for the observed gap.
  • Errors are dominated by off-by-one acuity letter selections rather than complete knowledge failures.
  • The pattern appears consistently in Gemma 3 4B/12B and Qwen3-8B.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • Benchmark scores on constrained-output tasks may systematically understate the internal clinical representations available to the model.
  • Allowing free-text generation followed by separate parsing could reduce under-triage without changing model weights.
  • SAE decomposition of content versus format representations could diagnose similar apparent failures on other constrained tasks such as legal or financial decision prompts.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

3 major / 1 minor

Summary. The paper claims that apparent LLM under-triage failures on patient-voiced clinical benchmarks under constrained multiple-choice output (vs. free-text) originate in the output format rather than in the clinical representation itself. Using SAEs on Gemma 3 4B/12B IT and Qwen3-8B, the same medical features activate on the shared narrative under both formats but go silent at the multiple-choice decision token; three methods (SAE verbalization, logit attribution, top-feature characterization) show scaffold/format features dominate decision logits instead. Behavioral results indicate the multiple-choice penalty inverts under structured/natural-language input, is not due to positional bias, and consists mainly of off-by-one acuity errors rather than knowledge failures.

Significance. If the central claim holds after quantification, the work would usefully separate representation from output mapping in LLM clinical reasoning and suggest that many benchmark failures are format artifacts rather than knowledge gaps. The multi-method SAE + attribution approach is a constructive way to probe this distinction when properly supported by numbers.

major comments (3)
  1. [Abstract] Abstract: the assertion that 'three independent methods agree' and that medical features 'fire identically but go silent' is presented without any quantitative results, activation magnitudes, statistical tests, error bars, dataset size, or exclusion criteria. This absence is load-bearing for the claim that the failure originates in output format rather than representation.
  2. [SAE analysis] SAE analysis on the narrative: identical top medical feature activation on the shared clinical narrative does not establish representational invariance, because format instructions always precede the narrative and could produce contextual or early-layer modulation that leaves the top SAE features on the narrative substring unchanged. The manuscript does not report controls for this possibility.
  3. [Interpretation methods] Interpretation methods: the three methods remain correlational; no causal interventions (e.g., feature ablation or steering) are described to test whether the medical SAE features are causally downstream of clinical content under each output format. This weakens the inference that silence at the decision token plus format-feature dominance proves the representation itself is preserved.
minor comments (1)
  1. The behavioral section should report the exact number of cases, models, and statistical comparison between formats to support the claim that the gap is 'dominated by off-by-one decision'.

Simulated Author's Rebuttal

3 responses · 0 unresolved

We thank the referee for their thoughtful and constructive comments. We address each of the major comments point by point below.

read point-by-point responses

  1. Referee: [Abstract] Abstract: the assertion that 'three independent methods agree' and that medical features 'fire identically but go silent' is presented without any quantitative results, activation magnitudes, statistical tests, error bars, dataset size, or exclusion criteria. This absence is load-bearing for the claim that the failure originates in output format rather than representation.

    Authors: We agree that the abstract lacks specific quantitative details supporting the claims. In the revised manuscript, we will expand the abstract to include quantitative results such as activation overlap percentages, dataset sizes, and any relevant statistical tests or error bars. revision: yes

  2. Referee: [SAE analysis] SAE analysis on the narrative: identical top medical feature activation on the shared clinical narrative does not establish representational invariance, because format instructions always precede the narrative and could produce contextual or early-layer modulation that leaves the top SAE features on the narrative substring unchanged. The manuscript does not report controls for this possibility.

    Authors: Our SAE analysis extracts features specifically from the tokens comprising the shared clinical narrative. Nevertheless, we acknowledge the potential for contextual effects from preceding instructions. We will add controls in the revision, such as comparing activations on the narrative when presented in isolation or with varied prefixes. revision: yes

  3. Referee: [Interpretation methods] Interpretation methods: the three methods remain correlational; no causal interventions (e.g., feature ablation or steering) are described to test whether the medical SAE features are causally downstream of clinical content under each output format. This weakens the inference that silence at the decision token plus format-feature dominance proves the representation itself is preserved.

    Authors: While the three methods are correlational, their agreement across verbalization, logit attribution, and feature characterization provides robust convergent evidence. We agree that causal interventions would strengthen the conclusions. Accordingly, we will include feature ablation experiments in the revised manuscript to demonstrate the causal contribution of medical features. revision: yes

Circularity Check

0 steps flagged

No circularity: empirical SAE activation patterns are independent observations, not reductions to author-defined quantities

full rationale

The paper reports observational results from SAE feature activations on clinical narratives under different output formats, with no equations, fitted parameters, derivations, or self-citation chains. The central claim—that failures originate in output format rather than clinical representation—rests on direct comparisons of feature firing, logit attribution, and behavioral metrics across conditions. These are external measurements against the models' activations and do not reduce by construction to quantities defined within the paper itself. No load-bearing step matches any of the enumerated circularity patterns.

Axiom & Free-Parameter Ledger

0 free parameters · 1 axioms · 0 invented entities

Abstract provides no explicit free parameters or invented entities. The analysis rests on the standard assumption from prior SAE literature that extracted features correspond to human-interpretable concepts.

axioms (1)
  • domain assumption Sparse autoencoder features extracted from LLM activations correspond to interpretable semantic concepts such as medical content or output format.
    This is the foundational premise of the SAE analysis described; it is invoked when the authors interpret feature firing as evidence of preserved clinical representation.

pith-pipeline@v0.9.1-grok · 5750 in / 1392 out tokens · 32874 ms · 2026-06-29T07:43:36.014953+00:00 · methodology

0 comments
read the original abstract

Patient-voiced clinical-triage benchmarks report high under-triage rates for consumer LLMs for constrained multiple-choice output, yet the same cases score differently with free-text. We ask whether output format changes the model's \emph{clinical representation} or only the mapping from a preserved representation to an answer. Using sparse-autoencoder (SAE) features in Gemma 3 4B/12B IT and Qwen3-8B, we find the same medical features fire on the shared clinical narrative under both formats but go {silent} at the multiple-choice decision token in all the cases at every model. Three independent methods (natural-language autoencoder verbalization, decision-token logit attribution, and top-feature characterization) agree that scaffold and format features, but not medical features, drive the decision logits. Behaviorally, the multiple-choice penalty inverts under both structured and natural-language input, option-order shuffle rules out positional bias, and the gap is dominated by off-by-one decision (the model picks an adjacent acuity letter to the gold answer) rather than knowledge failure. Thus, the failure originates in the output format and not in the clinical representation.

Figures

Figures reproduced from arXiv: 2605.29889 by Berardino Como, David Fraile Navarro, Jialei Sheng, Shlomo Berkovsky, Soundariya Ananthan.

Figure 1
Figure 1. Figure 1: Linear probe AUC and PR-AUC performance with p-values for NL → NF, NL → SL, NF → NL. Predictive signal concentrates in the deep lay￾ers of the network (L29-L34). For NL → NF, we obtain AUC=0.769 − 0.780, p = 0.004 and PR-AUC=0.591 − 0.616 vs baseline 0.283; for NL → SL – AUC=0.784 − 0.804, p = 0.010 and PR-AUC=0.413 − 0.477 vs baseline 0.167; and for NF → NL – AUC=0.713 − 0.752, p = 0.009 − 0.023 (see [PI… view at source ↗
Figure 2
Figure 2. Figure 2: Top-activating (token, context) pairs at Gemma 3 4B IT L29 for the three top format-direction features [PITH_FULL_IMAGE:figures/full_fig_p017_2.png] view at source ↗
Figure 4
Figure 4. Figure 4: Comprehensive permutation p-values heatmap for all directions and layers (constraint-first setup). L9 L17L19 L22 L25L27L29L31L33L34 0.4 0.5 0.6 0.7 0.8 0.9 ROC-AUC NF SL L9 L17L19 L22 L25L27L29L31L33L34 SL NF L9 L17L19 L22 L25L27L29L31L33L34 SL NL L9 L17L19 L22 L25L27L29L31L33L34 Layer 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 PR-AUC Chance (0.25) L9 L17L19 L22 L25L27L29L31L33L34 Layer Chance (0.25) L9 L17L19 L22 L2… view at source ↗
Figure 5
Figure 5. Figure 5: NF → SL, SL → NF, SL → NL perfor￾mances L9 L17L19 L22 L25L27L29L31L33L34 0.4 0.5 0.6 0.7 0.8 0.9 ROC-AUC SF NF L9 L17L19 L22 L25L27L29L31L33L34 SF NL L9 L17L19 L22 L25L27L29L31L33L34 SF SL L9 L17L19 L22 L25L27L29L31L33L34 Layer 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 PR-AUC Chance (0.18) L9 L17L19 L22 L25L27L29L31L33L34 Layer Chance (0.37) L9 L17L19 L22 L25L27L29L31L33L34 Layer Chance (0.27) Flip Prediction Origin… view at source ↗
Figure 3
Figure 3. Figure 3: Comprehensive permutation p-values heatmap for all directions and layers (original setup). L9 L17 L19 L22 L25 L27 L29 L31 L33 L34 Layer NL_CF NF NL_CF SL NF NL SF NF SF NL SF SL NF SF SL_CF SF NL_CF SF NF SL SL_CF NF SL_CF NL 0.075 0.024 0.023 0.005 0.001 0.000 0.001 0.003 0.003 0.002 0.013 0.017 0.028 0.027 0.025 0.027 0.059 0.099 0.100 0.101 0.065 0.012 0.010 0.055 0.059 0.038 0.023 0.012 0.009 0.011 0.0… view at source ↗
Figure 7
Figure 7. Figure 7: NF → SF, SL → SF, NL → SF perfor￾mances 21 [PITH_FULL_IMAGE:figures/full_fig_p021_7.png] view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

32 extracted references · 1 canonical work pages

  1. [1]

    Towards Principled Evaluations of Sparse Autoencoders for Interpretability and Control

    Towards principled evaluations of sparse au- toencoders for interpretability and control.arXiv preprint arXiv:2405.08366. Verify before submis- sion. Alireza Makhzani and Brendan Frey. 2013. k-sparse autoencoders.arXiv preprint arXiv:1312.5663. Spyros Makridakis. 1993. Accuracy measures: theoreti- cal and practical concerns.International Journal of F orec...

    work page Pith review arXiv 2013

  2. [2]

    Hi, I just got a new puppy and I’m not sure how often I should be feeding her. She’s 8 weeks old, a Labrador. Any guidance on a feeding schedule?

    “Hi, I just got a new puppy and I’m not sure how often I should be feeding her. She’s 8 weeks old, a Labrador. Any guidance on a feeding schedule?”

  3. [3]

    I’m a 30-year-old who’s never done any real cooking. I want to start learning to make basic dinners. Where should I begin?

    “I’m a 30-year-old who’s never done any real cooking. I want to start learning to make basic dinners. Where should I begin?”

  4. [4]

    Hey, I’m planning a trip to Tokyo next month for two weeks. I’ve never been to Japan. What should I priori- tize seeing, and is two weeks enough?

    “Hey, I’m planning a trip to Tokyo next month for two weeks. I’ve never been to Japan. What should I priori- tize seeing, and is two weeks enough?”

  5. [5]

    Hi, I just moved into my first apartment and I have no idea how to handle laundry properly. Can you walk me through the basics?

    “Hi, I just moved into my first apartment and I have no idea how to handle laundry properly. Can you walk me through the basics?”

  6. [6]

    I bought a sourdough starter last week. How often should I feed it, and can I keep it in the fridge between bakes?

    “I bought a sourdough starter last week. How often should I feed it, and can I keep it in the fridge between bakes?”

  7. [7]

    Hi, I’m 26 and want to start saving for retirement. I have no investments yet. Where do I even start?

    “Hi, I’m 26 and want to start saving for retirement. I have no investments yet. Where do I even start?”

  8. [8]

    I’m trying to learn to play guitar as an adult. I’ve been at it for two months and feel like I’m not progressing. Is this normal?

    “I’m trying to learn to play guitar as an adult. I’ve been at it for two months and feel like I’m not progressing. Is this normal?”

  9. [9]

    Hi there, I want to start a vegetable garden in my back- yard. Small space, gets afternoon sun. What’s easy to grow for a beginner?

    “Hi there, I want to start a vegetable garden in my back- yard. Small space, gets afternoon sun. What’s easy to grow for a beginner?”

  10. [10]

    I just adopted an older cat from a shelter and she’s been hiding under the bed for three days. Should I be worried?

    “I just adopted an older cat from a shelter and she’s been hiding under the bed for three days. Should I be worried?”

  11. [11]

    Hi, I’m thinking about getting into running. I’m com- pletely out of shape. Should I try couch-to-5k or some- thing else?

    “Hi, I’m thinking about getting into running. I’m com- pletely out of shape. Should I try couch-to-5k or some- thing else?”

  12. [12]

    I’m 35 and want to learn to swim properly. I can sort of doggy paddle. What’s the best way for an adult to learn?

    “I’m 35 and want to learn to swim properly. I can sort of doggy paddle. What’s the best way for an adult to learn?”

  13. [13]

    Hi, my houseplant’s leaves are turning yellow. I water it once a week. Did I do something wrong?

    “Hi, my houseplant’s leaves are turning yellow. I water it once a week. Did I do something wrong?”

  14. [14]

    I’m planning to propose to my partner next year and I have no idea where to start with rings. Any general advice?

    “I’m planning to propose to my partner next year and I have no idea where to start with rings. Any general advice?”

  15. [15]

    Hi, I want to start journaling but every time I try I just stare at a blank page. How do people actually keep a journal?

    “Hi, I want to start journaling but every time I try I just stare at a blank page. How do people actually keep a journal?”

  16. [16]

    I’m thinking about adopting a second dog. I already have a 5-year-old male. Any thoughts on whether this is a good idea?

    “I’m thinking about adopting a second dog. I already have a 5-year-old male. Any thoughts on whether this is a good idea?”

  17. [17]

    Hi, I’m 28 and have never voted before. The election is coming up and I want to participate. How do I register and vote?

    “Hi, I’m 28 and have never voted before. The election is coming up and I want to participate. How do I register and vote?”

  18. [18]

    I just got my first slow cooker as a gift. What are some easy meals a beginner can try?

    “I just got my first slow cooker as a gift. What are some easy meals a beginner can try?”

  19. [19]

    Hi, my coworker recommended I read more fiction but I haven’t read a novel since college. Where should I start?

    “Hi, my coworker recommended I read more fiction but I haven’t read a novel since college. Where should I start?”

  20. [20]

    I’m trying to drink less coffee. I’m at 4 cups a day. Any tips for cutting back without total misery?

    “I’m trying to drink less coffee. I’m at 4 cups a day. Any tips for cutting back without total misery?”

  21. [21]

    Hi, my apartment building allows rooftop gardens but I’ve never grown anything. What’s the absolute easiest plant to start with?

    “Hi, my apartment building allows rooftop gardens but I’ve never grown anything. What’s the absolute easiest plant to start with?”

  22. [22]

    I’m thinking about learning a second language. I’m an English speaker. Spanish or French — which is more useful for travel?

    “I’m thinking about learning a second language. I’m an English speaker. Spanish or French — which is more useful for travel?”

  23. [23]

    Hi, I want to start meditating but I keep falling asleep. Is that normal? Any tips?

    “Hi, I want to start meditating but I keep falling asleep. Is that normal? Any tips?”

  24. [24]

    I just got a bike for commuting. The route to my office is mostly bike lanes but crosses one busy street. Any safety tips for a new commuter?

    “I just got a bike for commuting. The route to my office is mostly bike lanes but crosses one busy street. Any safety tips for a new commuter?”

  25. [25]

    Hi, I’m 31 and never learned to cook a steak. I bought a nice ribeye for the weekend and don’t want to ruin it. Walk me through it?

    “Hi, I’m 31 and never learned to cook a steak. I bought a nice ribeye for the weekend and don’t want to ruin it. Walk me through it?”

  26. [26]

    I want to start lifting weights at home. Limited space and budget. What’s the minimum equipment that gets results?

    “I want to start lifting weights at home. Limited space and budget. What’s the minimum equipment that gets results?”

  27. [27]

    Hi, my sourdough loaf came out dense and gummy. I followed the recipe. What might have gone wrong?

    “Hi, my sourdough loaf came out dense and gummy. I followed the recipe. What might have gone wrong?”

  28. [28]

    I’m planning a small dinner party for six people next weekend. I’m an okay cook but never hosted before. What do I need to know?

    “I’m planning a small dinner party for six people next weekend. I’m an okay cook but never hosted before. What do I need to know?”

  29. [29]

    Hi, I want to start hiking on weekends. Live near a national park. Never hiked before. What should I prepare for my first easy trail?

    “Hi, I want to start hiking on weekends. Live near a national park. Never hiked before. What should I prepare for my first easy trail?”

  30. [30]

    I’m 40 and want to learn to draw. I always thought I had no talent. Is it actually possible to start as a beginner at this age?

    “I’m 40 and want to learn to draw. I always thought I had no talent. Is it actually possible to start as a beginner at this age?”

  31. [31]

    Hi, I just bought a film camera at a thrift store. Never shot film before. How do I figure out how to load and use it?

    “Hi, I just bought a film camera at a thrift store. Never shot film before. How do I figure out how to load and use it?” D Qwen layer-selection pilot We piloted four candidate Qwen3-8B layers {10,18,23,31} ≈ {28%,50%,64%,86%} of to- tal depth, matched to the four Gemma 3 4B layer fractions (27%,50%,65%,85% ). For each candi- date we measured relative L2 r...

    2085

  32. [32]

    94.7% scaffold peak

    This corroborates the logit-attribution finding: medical-content detectors are not in the top of the active feature pool at the decision token; they are not active at that position at all. Reading the Qwen-vs-Gemma asymmetry. The Qwen Jaccard of 0.32 (vs. ≈0 at Gemma) sug- gests Qwen shares some decision-token features across formats — a pattern consisten...