Enabling On-Device Medical AI Assistants via Input-Driven Saliency Adaptation

Edward Humes; Rithvik Jonna; Tinoosh Mohsenin; Uttej Kallakurik; Xiaomin Lin

REVIEW 5 major objections 5 minor 25 references

Enabling On-Device Medical AI Assistants via Input-Driven Saliency Adaptation

T0 review · 5 major / 5 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read Input-driven saliency pruning, followed by quantization and LoRA fine-tuning, can shrink medical LLMs to edge-device size while retaining most question-answering accuracy.

desk verdict The accuracy-preservation claim collapses under the paper's own Table I; only the deployment measurements survive scrutiny. read the letter →

arxiv 2506.11105 v3 pith:M3OP2AJU submitted 2025-06-07 cs.CL cs.AIcs.ARcs.SYeess.SY

Uttej Kallakurik , Edward Humes , Rithvik Jonna , Xiaomin Lin , Tinoosh Mohsenin This is my paper

classification cs.CLcs.AIcs.ARcs.SYeess.SY

keywords largelanguagemodelsmodelpruningneuronsaliencyquantizationedgedeploymentmedicalquestionansweringlow-rankadaptationon-deviceAI

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper tries to show that a large language model trained for general text can be aggressively compressed for a specialized domain—medical question answering—by measuring how active each neuron is on medical text and cutting the quiet ones. The authors claim that pruning to 50% (Gemma 7B) or 67% (LLaMA3 8B) of the feed-forward neurons, followed by 4-bit weight quantization and LoRA fine-tuning, keeps most multiple-choice medical QA accuracy while making the model small enough for edge hardware. If true, it means clinics and mobile health apps could run private, low-power medical assistants locally instead of sending patient questions to the cloud. The paper supports the claim with benchmark scores on MedMCQA, MedQA, and PubMedQA, plus power and throughput measurements on two edge boards.

What carries the argument

The central object is the dual-stage saliency score, $\text{Saliency}(v_i) = \mathbb{E}_{x \sim D_{task}}\left[\|\text{activation}_{v_i}(x)\|_2 + \|J_{v_i}(x)\|_2\right]$, where the Jacobian term measures how sensitive a neuron's output is to its input. This score is computed over unlabeled medical samples, and a differentiable binary masking function guided by a resource-constrained loss selects which feed-forward neurons to keep. Pruning is formulated as subgraph selection that minimizes remaining edges subject to a performance-retention ratio, and only FFN layers are pruned because they hold most parameters and dominate latency.

What would settle it

Compute the saliency mask on one medical dataset, prune to 67%, and evaluate on a held-out medical benchmark that was never used to compute saliency or to fine-tune; if accuracy collapses while the original datasets hold up, the mask was fitted to the evaluation distribution rather than to general medical competence.

Watch

Extended reading notes

Core claim

The core claim is that neuron saliency is input-dependent, so a small sample of unlabeled domain text reveals which neurons carry a task. The method scores each feed-forward neuron by the L2 norm of its activation plus the L2 norm of the Jacobian of that activation, computed over medical samples, then learns binary masks that turn off low-scoring neurons while optimizing a loss that balances retained task performance against remaining parameters. After masking, the model is fine-tuned with LoRA and quantized to 4-bit weights. On the reported benchmarks the 50% pruned Gemma 7B retains 30.82% on MedMCQA, 22.20% on MedQA, and 61.70% on PubMedQA; the 67% pruned LLaMA3 8B retains 29.45%, 20.59%, and 43.15% respectively, which the authors describe as minimal accuracy loss given the size reduction.

Load-bearing premise

The whole method rests on the assumption that saliency scores measured on a small set of unlabeled MedQA questions identify neurons that matter universally for medical text, so a mask tuned on those samples keeps working on the full benchmark test sets and on other medical datasets.

Editorial extensions

If this is right

Medical assistants that answer questions locally on a roughly 10-watt edge device become feasible, removing the need to ship patient data to cloud APIs.
The same input-driven saliency recipe could compress an LLM for any narrow domain—legal, financial, or scientific—by swapping the medical text for domain text.
Privacy-preserving point-of-care tools become realistic in settings with unreliable connectivity or strict data-handling rules.
The reported trade-off gives a concrete target: about half the feed-forward parameters can go without catastrophic QA loss, but crossing roughly 67% for LLaMA3 leads to sharp degradation.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

Because the saliency probe uses only unlabeled MedQA samples, the method's generality hinges on those samples resembling the deployment population; a clinic's real patient queries may activate different neurons, so the published accuracy numbers should be seen as upper bounds until tested on unseen clinical transcripts.
The pruning mask is fitted to medical data, so the compressed model likely loses general-purpose ability; the paper implicitly trades broad competence for task retention, and a direct comparison of the pruned model on general benchmarks would make that trade explicit.
The same pipeline could be tested as a data-efficiency probe: measuring how much saliency quality degrades as the number of unlabeled samples shrinks would reveal the minimum corpus needed to justify domain-adaptive pruning.

Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

Colleague—

The short version: the paper's central claim—that saliency-based pruning preserves medical QA accuracy—is not supported by its own Table I. The 50% pruned Gemma drops 12.2 points on MedMCQA, 16.5 on MedQA, and 10.4 on PubMedQA; the 67% pruned LLaMA drops 27.2 on PubMedQA. Those are not "minimal accuracy losses," and the abstract and conclusion overstate what the data show.

What's genuinely useful is the deployment part. They measure throughput and power on Jetson Orin Nano and Raspberry Pi 5 with a 4-bit quantized pruned model, and the numbers look plausible. That's a real contribution for the edge-AI community. The idea of using a dual saliency (L2 magnitude plus Jacobian norm) on domain data is a reasonable extension of existing pruning work, though not new in a structural sense—ATP and LLM-Pruner already do domain-guided pruning.

The soft spots are load-bearing. First, the evaluation is circular: saliency is computed on unlabeled MedQA samples (Section III-C) and the model is evaluated on MedQA (Section IV-C). That doesn't outright invalidate the accuracy numbers, but it means the mask is partly fitted to the test benchmark. Second, the comparison is confounded: the "Baseline (FT)" row is full precision and no pruning, while the pruned row adds pruning plus 4-bit quantization plus LoRA. You can't attribute the accuracy drop to pruning alone. Third, the compression numbers are misleading—"67% pruned" removes about 30% of parameters, and most of the size reduction comes from quantization. Finally, there's no code, no detailed algorithm, and the free parameters (alpha, target fraction, eta, LoRA rank, thresholds) are not specified enough to reproduce the results.

That said, the paper isn't incoherent. The equations are consistent, and the perplexity-vs-pruning curves honestly show collapse points. The problem is that the headline claim doesn't match the reported numbers. A careful reviewer would send this back for major revision, asking for an ablation that separates pruning from quantization, non-circular calibration (e.g., saliency on a different medical corpus than the evaluation set), and honest accuracy reporting.

For you: it's a decent teaching example of how to misread your own results, but not something to build on. I wouldn't cite it. If it lands on your desk, send it to review—the hardware work is worth refereeing—but expect heavy revision or rejection. My recommendation: treat the central accuracy claim as unsupported; if the authors fix the evaluation, the deployment methodology could be a useful reference.

Referee Report

5 major / 5 minor

Summary. The paper proposes an input-driven saliency pruning framework for medical-domain LLMs, combining structured pruning of FFN neurons with post-training 4-bit quantization and LoRA fine-tuning. The method is evaluated on Gemma 7B and LLaMA 3 8B using perplexity on medical corpora, accuracy on MedMCQA/MedQA/PubMedQA, and deployment measurements on Jetson Orin Nano and Raspberry Pi 5. The central claim is that domain-specific saliency allows aggressive compression with minimal accuracy loss and up to 5x smaller models while enabling real-time on-device inference.

Significance. If the central claim were supported, the paper would be a useful practical contribution: it reports real edge-hardware measurements (power, throughput) and applies compression to a realistic medical QA setting, which is more than many pruning papers do. The deployment results in Table II and the perplexity comparison in Fig. 1 are informative. However, the central claim is not supported by the paper's own principal evidence: the accuracy drops in Table I are large, the compression labels are inconsistent with the parameter counts, and the experimental design does not isolate the effect of pruning. These issues are load-bearing rather than presentational.

major comments (5)

[Abstract, Conclusion, Section IV-C, Table I] The claim that pruning preserves performance with 'minimal accuracy loss' is contradicted by Table I. The 50% pruned Gemma 7B drops from 42.98% to 30.82% on MedMCQA, from 38.74% to 22.20% on MedQA, and from 72.14% to 61.70% on PubMedQA; the 67% pruned LLaMA 3 8B drops from 70.30% to 43.15% on PubMedQA. These are absolute drops of 10 to 27 accuracy points on the same benchmarks used to validate the method. The text in Section IV-C states that the compressed models 'maintain high task accuracy' and the conclusion repeats 'minimal accuracy loss'; those sentences are not supported by the reported numbers and should be revised or the experiments redone.
[Table I, Section IV-C] The compression labels and the 'up to 5x smaller' statement are inconsistent with the parameter counts. The LLaMA model labeled '67% Pruned' has 8.03B parameters at baseline and 5.58B after pruning, a 30.5% parameter reduction, not 67%; the Gemma model labeled '50% Pruned' has 9.32B going to 6.59B, a 29.3% reduction. The reported '28 GB to 5 GB' memory reduction and 'up to 5x smaller' appear to be dominated by 4-bit quantization rather than by pruning. The paper should report parameter reduction and memory reduction separately, and should not describe the neuron-pruning fraction as if it were the model-size reduction.
[Table I, Section IV-A] The evaluation does not isolate the effect of pruning. The baseline row is 'Baseline (FT)', while the compressed rows add pruning, 4-bit quantization, and LoRA fine-tuning simultaneously. Any of these components could cause the observed accuracy drop, so Table I cannot support the claim that pruning itself preserves performance. The paper needs ablations that compare (i) pruning only, (ii) quantization only, (iii) LoRA only, and (iv) the full pipeline, with all models evaluated in the same precision and fine-tuning protocol.
[Section III-C, Section IV-C] There is a potential distribution leak in the saliency calibration. Section III-C says saliency scores are computed on 'a small number of unlabeled samples from the application-specific dataset D_task, such as MedQA', and Section IV-C evaluates on MedQA. If those calibration samples are drawn from the same distribution as the MedQA test benchmark, the pruning mask is partly fitted to the evaluation set, which would invalidate the benchmark results as evidence of task-specific preservation. The paper must specify exactly which split of MedQA (or another dataset) is used for saliency computation, and ideally use a calibration set disjoint from all evaluation benchmarks.
[Fig. 3, Fig. 4, Table I] The perplexity trends in Figs. 3 and 4 are used to justify the 50% and 67% pruning targets, but these targets are selected from the same perplexity curves that are then presented as evidence of robustness. Moreover, the perplexity results do not predict the large accuracy drops in Table I: for example, Fig. 4 reports that perplexity 'remains stable up to 67% pruning' for LLaMA 3 8B, yet PubMedQA accuracy falls by 27.15 points at that pruning level. The paper should either explain this disconnect or stop using Harrison-corpus perplexity as a proxy for medical QA retention.

minor comments (5)

[Section III-C] The Jacobian saliency term is written as a norm of a partial derivative without an explicit expectation over the input distribution, while the magnitude term in the same equation has an expectation; the notation should be made consistent.
[Section IV-A] Key hyperparameters are missing: the loss weight alpha in Eq. (1), the TargetFraction in Eq. (3), the retention ratio eta, the LoRA rank/alpha, and the fine-tuning epochs and learning rate are not reported. Without these, the method is not reproducible.
[Abstract and Conclusion] The phrase 'real-time' is not supported by the reported measurements: Table II reports throughput in tokens/s but not end-to-end latency, and no real-time requirement is defined. The authors should either report latency against a defined deadline or temper the real-time claim.
[Fig. 1] The caption says the method achieves 'up to 1.9x lower perplexity', but the text says '1.4-1.6x lower perplexity'; these numbers should be reconciled.
[Throughout] There are several typos and inconsistent spellings, including 'Rasberry-Pi' in the introduction, 'varaints' in Section IV-C, and 'Llama' vs 'LLaMA'; the manuscript should be proofread.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the pruning mask is fit to unlabeled domain text, not to QA labels or to the reported benchmark accuracies, and Table I's large degradation shows the accuracy results are not forced by construction.

full rationale

The derivation chain is self-contained in the sense required here. Saliency (Sec. III-C) is computed from unlabeled activations and Jacobians on domain text; the pruning mask is therefore a function of input statistics, not of the QA labels or of the Table I accuracies. The subsequent evaluation on MedMCQA, MedQA, and PubMedQA (Sec. IV-C) is a separate measurement. Naming MedQA as an example Dtask in Sec. III-C does create an in-domain calibration/evaluation overlap, but it does not make the reported accuracy equal to the saliency by construction: the mask could be perfectly adapted to MedQA input statistics and still fail the QA task, and indeed Table I shows large drops (e.g., Gemma 7B 50%-pruned MedQA 22.20% vs. 38.74% baseline). The 50% and 67% ratios are chosen from measured perplexity curves (Sec. IV-B); choosing the last stable point and then describing the curve as stable is selection, not a circular derivation of the accuracy claim. The citations to the authors' own prior work ([18] and [20]) are contextual and not load-bearing. Hence there is no circular step in the claimed derivation; the empirical contradiction between the abstract/conclusion and Table I is a correctness problem, not a circularity problem.

Assumptions & free parameters 5 free parameters · 4 assumptions · 0 invented entities

The paper introduces no new physical entities, but its central claim depends on several unvalidated assumptions and hand-picked hyperparameters, most importantly the choice to compute saliency on the same benchmark used for evaluation.

free parameters (5)

alpha (loss weight)
In Eq. 1, the balance between TaskLoss and ResourceLoss is not specified; performance results depend on this choice.
TargetFraction = 50% for Gemma, 67% for LLaMA
Chosen by hand from perplexity curves (Figures 3 and 4) before the model collapse point; these targets are then presented as the main compression results.
eta (performance retention ratio)
Defined in the pruning objective but not specified or used in the loss formulation.
LoRA rank
LoRA is used for fine-tuning but rank is not reported, so the fine-tuning capacity is under-specified.
Saliency thresholds
The thresholds that turn soft masks into binary masks are learned via backprop, but no initialization or update rule is given.

assumptions (4)

domain assumption For any specific application, only a subset of neurons show high saliency on that domain's input distribution, and the rest can be pruned without much loss.
Section III-A states this as the central postulation; it is asserted, not derived, and is the basis for the entire pruning approach.
domain assumption Pruning only FFN layers is sufficient; attention layers do not need compression.
Section III-B states pruning is performed exclusively in FFN layers, but no evidence or ablation supports leaving attention untouched.
ad hoc to paper Saliency scores computed on a small number of unlabeled MedQA samples transfer to full medical benchmarks.
Section III-C explicitly uses 'a small number of unlabeled samples from ... MedQA'; the same benchmark is later used for evaluation in Section IV-C, creating circularity.
ad hoc to paper Perplexity on Harrison's Principles of Internal Medicine is a meaningful proxy for medical QA performance.
Section IV-B defines the 'Harrison Score' perplexity; this is a non-standard, self-defined metric not tied to the QA benchmarks.

how reviews work

0 comments

Cite this review

Pith. "Pith review of Enabling On-Device Medical AI Assistants via Input-Driven Saliency Adaptation." pith.science (2026). https://pith.science/paper/M3OP2AJU

@misc{pith2026250611105,
  author       = {Pith},
  title        = {Pith review of: Enabling On-Device Medical AI Assistants via Input-Driven Saliency Adaptation},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/M3OP2AJU}},
  note         = {Machine review of arXiv:2506.11105}
}

read the original abstract

Large Language Models (LLMs) have significant impact on the healthcare scenarios but remain prohibitively large for deployment in real-time, resource-constrained environments such as edge devices. In this work, we introduce a novel medical assistant system, optimized through our general-purpose compression framework, which tailors Large Language Models (LLMs) for deployment in specialized domains. By measuring neuron saliency on domain-specific data, our method can aggressively prune irrelevant neurons, reducing model size while preserving performance. Following pruning, we apply post-training quantization to further reduce the memory footprint, and evaluate the compressed model across medical benchmarks including MedMCQA, MedQA, and PubMedQA. We also deploy the 50\% compressed Gemma and the 67\% compressed LLaMA3 models on Jetson Orin Nano (18.7W peak) and Raspberry Pi 5 (6.3W peak), achieving real-time, energy-efficient inference under hardware constraints.

Figures

Figures reproduced from arXiv: 2506.11105 by the authors.

**Figure 2.** High-level architecture, involves computing saliency measures for [PITH_FULL_IMAGE:figures/full_fig_p002_2.png] view at source ↗

**Figure 3.** Perplexity vs. pruning percentage for Gemma 7B. Models are fine [PITH_FULL_IMAGE:figures/full_fig_p003_3.png] view at source ↗

Figures from the paper (2 more)

**Figure 4.** Figure 4: Perplexity vs. pruning percentage for LLaMA3 8B. Both in-domain [PITH_FULL_IMAGE:figures/full_fig_p003_4.png]

**Figure 5.** Figure 5: Power consumption over time on the Jetson Orin Nano. Peaks [PITH_FULL_IMAGE:figures/full_fig_p004_5.png]

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

25 extracted references · 14 canonical work pages

[1]

ggml-org/llama.cpp. 6 2025. URL https://github.com/ ggml-org/llama.cpp

work page 2025
[2]

A question-entailment approach to question answering

Asma Ben Abacha and Dina Demner-Fushman. A question-entailment approach to question answering. BMC Bioinformatics , 20(1):511, 2019. doi: 10.1186/ s12859-019-3119-4. URL https://bmcbioinformatics. biomedcentral.com/articles/10.1186/s12859-019-3119-4

doi:10.1186/s12859-019-3119-4 2019
[3]

Deploying quantized llms with onnx runtime

ApX Machine Learning. Deploying quantized llms with onnx runtime. https://apxml.com/courses, 2025

work page 2025
[4]

Slicegpt: Compress large language models by delet- ing rows and columns

Saleh Ashkboos, Maximilian L Croci, Marcelo Gen- nari do Nascimento, Torsten Hoefler, and James Hens- man. Slicegpt: Compress large language models by delet- ing rows and columns. arXiv preprint arXiv:2401.15024, 2024

arXiv 2024
[5]

Natural language processing models reveal neural dynamics of human conversation

Jing Cai, Alex E Hadjinicolaou, Angelique C Paulk, Daniel J Soper, Tian Xia, Alexander F Wang, John D Rolston, R Mark Richardson, Ziv M Williams, and Sydney S Cash. Natural language processing models reveal neural dynamics of human conversation. Nature Communications, 16(1):3376, 2025

work page 2025
[6]

Survey of the state of the art in natural language generation: Core tasks, appli- cations and evaluation

Albert Gatt and Emiel Krahmer. Survey of the state of the art in natural language generation: Core tasks, appli- cations and evaluation. Journal of Artificial Intelligence Research, 61:65–170, 2018

work page 2018
[7]

The llama 3 herd of models

Aaron Grattafiori, Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Abhishek Kadian, Ahmad Al-Dahle, Aiesha Letman, Akhil Mathur, Alan Schelten, Alex Vaughan, et al. The llama 3 herd of models. arXiv preprint arXiv:2407.21783, 2024

arXiv 2024
[8]

Efficientllm: Scalable pruning-aware pretraining for architecture-efficient large language models

Boyang Guo, Fenglin Liu, Hongjian Zhou, et al. Efficientllm: Scalable pruning-aware pretraining for architecture-efficient large language models. arXiv preprint arXiv:2502.06663, 2025

arXiv 2025

Show all 25 references

[9]

Lighteval: A lightweight framework for llm evaluation, 2023

Nathan Habib, Cl ´ementine Fourrier, Hynek Kydl ´ıˇcek, Thomas Wolf, and Lewis Tunstall. Lighteval: A lightweight framework for llm evaluation, 2023. URL https://github.com/huggingface/lighteval

2023
[10]

Adams, Jens-Michalis Papaioannou, Paul Grundmann, Tom Oberhauser, Alexei Figueroa, Alexander L ¨oser, Daniel Truhn, and Keno K

Tianyu Han, Lisa C. Adams, Jens-Michalis Papaioannou, Paul Grundmann, Tom Oberhauser, Alexei Figueroa, Alexander L ¨oser, Daniel Truhn, and Keno K. Bressem. Medalpaca – an open-source collection of medical con- versational ai models and training data. arXiv preprint arXiv:2304...

2023 arXiv
[11]

Principles of internal medicine

Tinsley Randolph Harrison. Principles of internal medicine. In Principles of internal medicine , pages 1703–1703. 1954

1954
[12]

What disease does this patient have? a large-scale open domain question answering dataset from medical exams

Di Jin, Eileen Pan, Nassim Oufattole, Wei-Hung Weng, Hanyi Fang, and Peter Szolovits. What disease does this patient have? a large-scale open domain question answering dataset from medical exams. Applied Sciences, 11(14):6421, 2021

2021
[13]

Pubmedqa: A dataset for biomedical research question answering

Qiao Jin, Bhuwan Dhingra, Zhengping Liu, William Cohen, and Xinghua Lu. Pubmedqa: A dataset for biomedical research question answering. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natur...

2019
[14]

Benchmark evaluations, applications, and challenges of large vision language models: A sur- vey

Zongxia Li, Xiyang Wu, Hongyang Du, Huy Nghiem, and Guangyao Shi. Benchmark evaluations, applications, and challenges of large vision language models: A sur- vey. arXiv preprint arXiv:2501.02189 , 1, 2025

2025 arXiv
[15]

All-in-one tuning and struc- tural pruning for domain-specific llms, 2024

Lei Lu, Zhepeng Wang, Runxue Bao, Mengbing Wang, Fangyi Li, Yawen Wu, Weiwen Jiang, Jie Xu, Yanzhi Wang, and Shangqian Gao. All-in-one tuning and struc- tural pruning for domain-specific llms, 2024

2024
[16]

Biogpt: Gen- erative pre-trained transformer for biomedical text gen- eration and mining

Renqian Luo, Liai Sun, Yingce Xia, Tao Qin, Sheng Zhang, Hoifung Poon, and Tie-Yan Liu. Biogpt: Gen- erative pre-trained transformer for biomedical text gen- eration and mining. arXiv preprint arXiv:2210.10341 , 2022

2022 arXiv
[17]

Llm- pruner: On the structural pruning of large language mod- els

Xinyin Ma, Gongfan Fang, and Xinchao Wang. Llm- pruner: On the structural pruning of large language mod- els. Advances in neural information processing systems , 36:21702–21720, 2023

2023
[18]

Multi-rag: A multimodal retrieval- augmented generation system for adaptive video under- standing

Mingyang Mao, Mariela M Perez-Cabarcas, Utteja Kallakuri, Nicholas R Waytowich, Xiaomin Lin, and Tinoosh Mohsenin. Multi-rag: A multimodal retrieval- augmented generation system for adaptive video under- standing. arXiv preprint arXiv:2505.23990 , 2025

2025
[19]

Post-training quantization of llms with nvidia nemo and nvidia tensorrt model optimizer

NVIDIA. Post-training quantization of llms with nvidia nemo and nvidia tensorrt model optimizer. https:// developer.nvidia.com/blog, 2024

2024
[20]

Chatsim: Underwater simulation with natural language prompting

Aadi Palnitkar, Rashmi Kapu, Xiaomin Lin, Cheng Liu, Nare Karapetyan, and Yiannis Aloimonos. Chatsim: Underwater simulation with natural language prompting. In OCEANS 2023-MTS/IEEE US Gulf Coast , pages 1–7. IEEE, 2023

2023
[21]

Large language models encode clinical knowledge

Karan Singhal, Shekoofeh Azizi, Tao Tu, S Sara Mah- davi, Jason Wei, Hyung Won Chung, Nathan Scales, Ajay Tanwani, Heather Cole-Lewis, Stephen Pfohl, et al. Large language models encode clinical knowledge. Nature, 620 (7972):172–180, 2023

2023
[22]

Toward expert-level medical question answering with large language models

Karan Singhal, Shekoofeh Azizi, Tien Tu, Soroush Mah- davi, Jason Wei, Hyung Won Chung, Nathan Scales, Ajay Tanwani, Joon Lee, Andy Liu, et al. Toward expert-level medical question answering with large language models. npj Digital Medicine , 6(1):1–9, 2023

2023
[23]

Gemma: Open models based on gemini research and technology

Gemma Team, Thomas Mesnard, Cassidy Hardin, Robert Dadashi, Surya Bhupatiraju, Shreya Pathak, Laurent Sifre, Morgane Rivi`ere, Mihir Sanjay Kale, Juliette Love, et al. Gemma: Open models based on gemini research and technology. arXiv preprint arXiv:2403.08295, 2024

2024 arXiv
[24]

A comprehensive study on post-training quantization for large language models

Zhewei Yao, Cheng Li, Xiaoxia Wu, Stephen Youn, and Yuxiong He. A comprehensive study on post-training quantization for large language models. arXiv preprint arXiv:2303.08302, 2023

2023 arXiv
[25]

Natural language reasoning, a survey

Fei Yu, Hongbo Zhang, Prayag Tiwari, and Benyou Wang. Natural language reasoning, a survey. ACM Computing Surveys, 56(12):1–39, 2024

2024

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.

[1] [1]

ggml-org/llama.cpp. 6 2025. URL https://github.com/ ggml-org/llama.cpp

work page 2025

[2] [2]

A question-entailment approach to question answering

Asma Ben Abacha and Dina Demner-Fushman. A question-entailment approach to question answering. BMC Bioinformatics , 20(1):511, 2019. doi: 10.1186/ s12859-019-3119-4. URL https://bmcbioinformatics. biomedcentral.com/articles/10.1186/s12859-019-3119-4

doi:10.1186/s12859-019-3119-4 2019

[3] [3]

Deploying quantized llms with onnx runtime

ApX Machine Learning. Deploying quantized llms with onnx runtime. https://apxml.com/courses, 2025

work page 2025

[4] [4]

Slicegpt: Compress large language models by delet- ing rows and columns

Saleh Ashkboos, Maximilian L Croci, Marcelo Gen- nari do Nascimento, Torsten Hoefler, and James Hens- man. Slicegpt: Compress large language models by delet- ing rows and columns. arXiv preprint arXiv:2401.15024, 2024

arXiv 2024

[5] [5]

Natural language processing models reveal neural dynamics of human conversation

Jing Cai, Alex E Hadjinicolaou, Angelique C Paulk, Daniel J Soper, Tian Xia, Alexander F Wang, John D Rolston, R Mark Richardson, Ziv M Williams, and Sydney S Cash. Natural language processing models reveal neural dynamics of human conversation. Nature Communications, 16(1):3376, 2025

work page 2025

[6] [6]

Survey of the state of the art in natural language generation: Core tasks, appli- cations and evaluation

Albert Gatt and Emiel Krahmer. Survey of the state of the art in natural language generation: Core tasks, appli- cations and evaluation. Journal of Artificial Intelligence Research, 61:65–170, 2018

work page 2018

[7] [7]

The llama 3 herd of models

Aaron Grattafiori, Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Abhishek Kadian, Ahmad Al-Dahle, Aiesha Letman, Akhil Mathur, Alan Schelten, Alex Vaughan, et al. The llama 3 herd of models. arXiv preprint arXiv:2407.21783, 2024

arXiv 2024

[8] [8]

Efficientllm: Scalable pruning-aware pretraining for architecture-efficient large language models

Boyang Guo, Fenglin Liu, Hongjian Zhou, et al. Efficientllm: Scalable pruning-aware pretraining for architecture-efficient large language models. arXiv preprint arXiv:2502.06663, 2025

arXiv 2025