Pith. sign in

REVIEW 4 major objections 5 minor 2 cited by

Compact language models match cloud-scale LLMs on wearable health prediction.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · deepseek-v4-flash

2026-08-04 22:31 UTC pith:ALBC42MA

load-bearing objection Useful benchmark, but the 'SLMs match LLMs' headline rests on a comparison to Health-LLM baselines that isn't apples-to-apples. the 4 major comments →

arxiv 2509.07260 v5 pith:ALBC42MA submitted 2025-09-08 cs.AI cs.HCcs.LG

HealthSLM-Bench: Benchmarking Small Language Models for Mobile and Wearable Healthcare Monitoring

Xin Wang , Ting Dang , Xinyu Zhang , Vassilis Kostakos , Michael J. Witbrock , Hong Jia This is my paper
classification cs.AI cs.HCcs.LG
keywords small language modelswearable healthcare monitoringhealth prediction benchmarkzero-shot learningfew-shot learninginstruction tuningLoRAon-device deployment
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper asks whether small language models (SLMs, roughly 1–4 billion parameters) can substitute for cloud-based large language models in predicting health conditions from wearable sensor data. Its central claim is yes: across three public wearable datasets and eight prediction tasks, the nine SLMs tested perform on par with or better than the larger LLM baselines under zero-shot, few-shot, and instruction-tuned settings, while using far less memory and responding faster. The authors build HealthSLM-Bench, a benchmark that turns 14-day summaries of steps, calories, heart rate, sleep, and mood into prompts, and deploy two fine-tuned SLMs on a phone to measure real-world latency and memory use. They also show the main remaining weakness: SLMs collapse toward majority classes on imbalanced tasks and struggle in some few-shot and calorie-regression cases. If the claim holds, privacy-preserving on-device health monitoring becomes feasible without sacrificing prediction quality.

Core claim

On its own terms, the paper establishes that compact language models are not a compromise version of LLMs for this benchmark: across the three datasets, SLM means are better than LLM means for stress, readiness, and fatigue in zero-shot settings, and after low-rank instruction tuning they beat LLMs on fatigue accuracy and calorie error by wide margins. The deployment experiment shows two top SLMs producing complete responses in about 1.8 to 7.6 seconds on a phone, versus about 57 seconds for a 7-billion-parameter LLM, while using 9–28% less RAM. The paper also documents where parity fails: one SLM consistently underperforms, calorie regression is much harder for SLMs in zero-shot settings, a

What carries the argument

HealthSLM-Bench, a prompt-based benchmark that converts 14-day wearable sensor summaries (steps, calories, resting heart rate, sleep, self-reported mood) into instruction prompts, evaluates nine SLMs under zero-shot, few-shot, and instruction-tuning protocols, and compares their mean accuracy and mean absolute error against published LLM numbers. Instruction tuning uses LoRA, low-rank adaptation that trains small injected matrices while freezing the base model. On-device evaluation uses 4-bit quantized models run through a lightweight inference engine, with latency and memory metrics borrowed from the mobile-benchmark literature.

Load-bearing premise

The headline comparison assumes the earlier LLM results were obtained under the same prompts, same greedy decoding, same token cap, and same handling of unparseable outputs as the new SLM runs; if any of these differ, the parity claim could be an artifact.

What would settle it

Re-run one LLM and one SLM on the same split of one dataset with an identical prompt template, greedy decoding, a 30-token cap, and a single parser; then recompute means with failed predictions counted as errors rather than omitted. If the LLM's mean error then drops below the SLM's, the paper's comparable-performance claim is not reproducible.

Watch this falsifier. Get emailed when new claim-graph text bears on it.

Share X Bluesky LinkedIn Reddit HN

If this is right

  • On-device SLMs can handle privacy-sensitive health inference without sending raw physiological data to a server.
  • A phone can run a fine-tuned SLM end-to-end in seconds, making real-time monitoring feasible; the paper reports about 1.8 seconds total for the smallest model versus about 57 seconds for a 7B LLM.
  • LoRA instruction tuning is enough to bring SLMs to LLM-level health accuracy, avoiding the cost of full fine-tuning.
  • Certain tasks, notably fatigue and calorie estimation, favor SLMs after tuning, while stress, readiness, and activity remain somewhat better handled by LLMs.
  • Class imbalance and few-shot collapse are the main remaining barriers to adoption, and the benchmark quantifies exactly where each model fails.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • The paper aggregates 14 days into summary statistics, so it cannot see recovery dynamics or circadian patterns; prompting with raw daily sequences could let larger models pull ahead, which would bracket the parity claim to aggregated inputs.
  • Because the labels are self-reported fatigue, stress, mood, and sleep quality, a natural test is whether the same parity survives on objective clinical endpoints such as polysomnography or blood pressure readings.
  • The majority-class collapse after fine-tuning suggests that imbalance-aware LoRA, such as class-weighted loss or minority oversampling, might convert the current comparable result into a clear SLM advantage on imbalanced tasks.
  • If the few-shot collapse on some tasks stems from prompt format rather than model capacity, instruction-formatting search could be a cheaper fix than scaling model size.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. The paper introduces HealthSLM-Bench, a benchmark that evaluates nine small language models (SLMs) in the 1B-4B range on eight health prediction tasks drawn from three wearable/mobile datasets (PMData, GLOBEM, AW-FB). The evaluation covers zero-shot, few-shot (N=1,3,5,10), and LoRA-based instruction tuning. The authors compare SLM results with LLM baselines from Health-LLM [23], report that SLMs match or exceed LLM performance on several tasks, and deploy two fine-tuned SLMs on an iPhone 15 Pro Max to measure TTFT, token throughput, CPU, and RAM usage against Llama-2-7b. They conclude that SLMs are a promising privacy-preserving and efficient alternative to cloud-based healthcare LLMs, while acknowledging limitations from class imbalance and few-shot collapse.

Significance. If the claims hold, this is a useful contribution: it extends prior SLM health-monitoring work to a broader set of models, datasets, and adaptation protocols, and it includes rare on-device deployment measurements. The efficiency measurements and the systematic comparison of nine SLMs are concrete assets for the community. However, the central SLM-vs-LLM comparison currently depends on cross-paper baseline numbers whose protocol equivalence is not demonstrated, and the averaging over different subsets of models makes the headline means difficult to interpret. The paper does not yet provide code/checkpoints despite promising them, so reproducibility is currently limited to the textual protocol.

major comments (4)
  1. [§5, 5.1; Table 3; Appendix A] The headline claim that SLMs match or outperform LLMs rests on comparing SLM runs with LLM numbers taken from Health-LLM [23]. Section 5 says 'under the same settings as in [23]', but Appendix A specifies greedy decoding, max 30 generated tokens, Alpaca-style prompts, and output-constraint parsing for SLMs; no evidence is given that Health-LLM used the same decoding, max-token limit, prompt template, or failed-output handling. In Table 3, the LLM mean for stress (0.64) is computed over 10 models excluding GPT-3.5/GPT-4, whereas the SLM mean (0.61) averages all 9 models. If failure handling or decoding differs, the observed SLM advantage on stress/readiness/fatigue could be an artifact of averaging over different subsets. This is the load-bearing part of the benchmark and must be resolved by reproducing the baselines or reporting the exact Health-LLM protocol.
  2. [Table 4; §5.1 (Few-shot learning)] The few-shot comparison is asymmetric. LLM rows are labeled 'FS-best' (the best result across shot counts from [23]), while SLM rows are fixed FS-1, FS-3, FS-5, FS-10. Comparing the best-of-k LLM result with a single one-shot SLM result inflates the LLM baseline, not the SLM one, and conflates model capability with prompt-selection budget. For a fair benchmark, either report the same N for both families or provide the full shot-count sweep for LLMs.
  3. [Tables 3–5; dash handling] No explicit rule is given for how '-' (failed predictions) are treated in mean rows. In Table 3 the stress mean 0.64 excludes GPT-3.5/GPT-4 but the SLM mean 0.61 includes all 9 models; in Table 5 Gemma-2-2b-it has dashes for three PMData tasks, and the SLM mean likely drops it. Averages over different model sets are not comparable. The paper should report the number of valid predictions per model/task and compute means either over the same set or with an explicit imputation/failure penalty.
  4. [§5.2; Table 6] The efficiency comparison is narrower than implied. Only two SLMs are deployed and compared with Llama-2-7b as a proxy for the fine-tuned LLM baseline, and the summary statistics come from 10 randomly selected samples with no confidence intervals or variance. The conclusion that SLMs preserve latency and memory advantages is reasonable for these specific models, but it does not support a general efficiency claim across the nine SLMs in the benchmark. Reporting per-model variance and the prompt-length distribution would strengthen the claim.
minor comments (5)
  1. [§4.1] The AW-FB dataset heading appears as 'A W_FB' in several places; use 'AW-FB' consistently.
  2. [Appendix A] Typographical issues: 're-productiveness' should be 'reproducibility' and 'with batch size of 128 with 3 number of epochs' should be rephrased.
  3. [Table 4] Header contains 'across across eight healthcare monitoring tasks'; remove the duplicate word.
  4. [References] References [57] and [58] refer to the same llama.cpp repository; unify and correct the year/attribution.
  5. [§5.2] Table 6 title 'Efficiency & Utilization of LLMs & SLMs' is misleading because only one LLM (Llama-2-7b) is included.

Circularity Check

0 steps flagged

No significant circularity: the paper is an empirical benchmark whose central claim rests on external LLM baselines, not on fitted parameters or self-citation chains.

full rationale

This is an empirical evaluation, not a derivation chain. The headline claim (Section 5.1: 'SLMs achieve comparable or better performance than LLMs across the three health datasets') is supported by two independent inputs: (i) the authors' own zero-shot, few-shot, and LoRA-tuned evaluations of SLMs, described concretely in Appendix A (greedy decoding, max 30 generated tokens, Alpaca-style templates, LoRA fine-tuning), and (ii) LLM baseline numbers taken from Health-LLM [23], an external paper by different authors. The LLM numbers are not outputs of the present paper's pipeline, not fitted parameters, and not defined in terms of the SLM results, so the comparison cannot be circular by construction. No fitted input is renamed as a prediction: the instruction-tuned models are evaluated on test data, and no parameter is tuned to a subset and then reported as a prediction of a closely related quantity. The self-citations [18] and [55] are used only for prompt construction, data-processing conventions, and related-work context; they are not invoked as a uniqueness theorem, a forbidden-alternative argument, or the justification for the central claim. The Appendix also openly discloses the SLM inference protocol, which is a transparency feature rather than a circular step. The legitimate concern raised by the reader—whether the 'same settings as in [23]' claim truly holds given Appendix A's greedy decoding and 30-token cap, and whether Table 3's means are computed over different model subsets—is a threat to external validity and fair comparison, not circularity: even if the protocols differ, the conclusion would be an artifact of uneven evaluation, not a tautology or a fitted-input-renamed-as-prediction. Since the central result depends on independent external baselines and direct held-out evaluation, the paper is self-contained against the circularity failure modes considered here. Score 0.

Axiom & Free-Parameter Ledger

6 free parameters · 4 axioms · 0 invented entities

This benchmark does not introduce a theoretical model, so there are no fitted equations. The load-bearing inputs are hand-chosen protocol and hyperparameter settings (LoRA rank, learning rate, max tokens, 14-day window, split seed, few-shot example selection) plus assumptions that self-reports are valid ground truth and that copied LLM numbers are comparable. The five settings listed as free parameters are all required to reproduce the numbers in Tables 3 to 6.

free parameters (6)
  • LoRA rank
    Not reported in the paper; chosen by hand, it controls the capacity of instruction tuning and thus all fine-tuned results in Table 5.
  • Learning rate = 5e-5
    Appendix A: 'learning rate as 5e-5', a hand-chosen hyperparameter for LoRA fine-tuning.
  • Max generated tokens = 30
    Appendix A caps generation at 30 tokens; this can truncate outputs and affect whether predictions are parsed as valid, influencing invalid-prediction counts.
  • 14-day data window = 14 days
    Section 4.3 follows prior work in standardizing daily sequences to 14-day windows; window length is a modeling choice that affects all predictions.
  • Data split seed
    Section 4.3 says data is 'randomly shuffled and split' 80/20 but no seed is given, so the exact splits underlying every table are not reproducible.
  • Few-shot example selection
    The paper varies N in {1,3,5,10} but does not report how the N examples are chosen or whether selection is seeded; this affects few-shot results in Table 4.
axioms (4)
  • domain assumption Self-reported labels in PMData, GLOBEM, and AW-FB are treated as ground truth for health status.
    Section 4.1 uses self-reported fatigue, stress, readiness, sleep quality, anxiety and depression scores as training and evaluation targets. If these are noisy or biased, all accuracy and MAE numbers inherit that noise.
  • ad hoc to paper The LLM baseline numbers from Health-LLM [23] are directly comparable to the authors' SLM evaluations.
    Section 5.1 says the comparison is 'under the same settings as in [23]', but no evidence is given that prompts, decoding settings, output parsing, or handling of '-' failures match the SLM protocol described in Appendix A.
  • domain assumption Accuracy and MAE are sufficient summary metrics for the health prediction tasks.
    The paper acknowledges class imbalance and models predicting only majority classes (Section 5.1, Appendix C), yet it reports only accuracy and MAE, which do not quantify per-class performance or balanced accuracy.
  • domain assumption Greedy decoding at max 30 tokens yields valid, complete predictions for all tasks.
    Appendix A sets greedy decoding and caps generated tokens at 30; long or complex responses may be truncated, and invalid predictions are marked '-' in tables, meaning some models were excluded from means.

pith-pipeline@v1.3.0-alltime-deepseek · 18950 in / 12963 out tokens · 133264 ms · 2026-08-04T22:31:33.434217+00:00 · methodology

0 comments
Cite this review

Pith. "Pith review of HealthSLM-Bench: Benchmarking Small Language Models for Mobile and Wearable Healthcare Monitoring." pith.science (2026). https://pith.science/paper/ALBC42MA

@misc{pith2026250907260,
  author       = {Pith},
  title        = {Pith review of: HealthSLM-Bench: Benchmarking Small Language Models for Mobile and Wearable Healthcare Monitoring},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/ALBC42MA}},
  note         = {Machine review of arXiv:2509.07260}
}
Share X Bluesky LinkedIn Reddit HN
read the original abstract

Mobile and wearable healthcare monitoring play a vital role in facilitating timely interventions, managing chronic health conditions, and ultimately improving individuals' quality of life. Previous studies on large language models (LLMs) have highlighted their impressive generalization abilities and effectiveness in healthcare prediction tasks. However, most LLM-based healthcare solutions are cloud-based, which raises significant privacy concerns and results in increased memory usage and latency. To address these challenges, there is growing interest in compact models, Small Language Models (SLMs), which are lightweight and designed to run locally and efficiently on mobile and wearable devices. Nevertheless, how well these models perform in healthcare prediction remains largely unexplored. We systematically evaluated SLMs on health prediction tasks using zero-shot, few-shot, and instruction fine-tuning approaches, and deployed the best performing fine-tuned SLMs on mobile devices to evaluate their real-world efficiency and predictive performance in practical healthcare scenarios. Our results show that SLMs can achieve performance comparable to LLMs while offering substantial gains in efficiency and privacy. However, challenges remain, particularly in handling class imbalance and few-shot scenarios. These findings highlight SLMs, though imperfect in their current form, as a promising solution for next-generation, privacy-preserving healthcare monitoring.

Figures

Figures reproduced from arXiv: 2509.07260 by Hong Jia, Michael J. Witbrock, Ting Dang, Vassilis Kostakos, Xin Wang, Xinyu Zhang.

Figure 1
Figure 1. Figure 1: The label distribution of the four tasks in PMData [PITH_FULL_IMAGE:figures/full_fig_p016_1.png] view at source ↗
Figure 2
Figure 2. Figure 2: The label distribution of the two tasks in GLOBEM [PITH_FULL_IMAGE:figures/full_fig_p017_2.png] view at source ↗
Figure 3
Figure 3. Figure 3: The label distribution of the two tasks in AW_FB [PITH_FULL_IMAGE:figures/full_fig_p017_3.png] view at source ↗
Figure 4
Figure 4. Figure 4: Distribution of predictions for the four tasks in PMData under FS setting [PITH_FULL_IMAGE:figures/full_fig_p019_4.png] view at source ↗
Figure 5
Figure 5. Figure 5: Distribution of predictions for the four tasks in PMData [PITH_FULL_IMAGE:figures/full_fig_p020_5.png] view at source ↗
Figure 6
Figure 6. Figure 6: Distribution of predictions for the two tasks in GLOBEM [PITH_FULL_IMAGE:figures/full_fig_p020_6.png] view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Forward citations

Cited by 2 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score.

  1. Little Brains, Big Feats: Exploring Compact Language Models

    cs.CL 2026-06 unverdicted novelty 3.0

    Small language models can run RAG generation on-device without GPUs in reasonable time.

  2. ECG Foundation Models and Medical LLMs for Agentic Cardiovascular Intelligence at the Edge: A Review and Outlook

    eess.SP 2026-04 unverdicted novelty 3.0

    ECG foundation models for signal interpretation and medical LLMs for reasoning can be integrated into agentic systems for real-time cardiovascular intelligence on edge devices.

Reference graph

Works this paper leans on

66 extracted references · 51 canonical work pages · cited by 2 Pith papers · 3 internal anchors

  1. [1]

    Wearable health technology and electronic health record integration: scoping review and future directions.Journal of Medical Internet Research, 21(9):e12861, 2019

    Cecilia Dinh-Le, Rebecca Chuang, Sonia Chokshi, and Devin Mann. Wearable health technology and electronic health record integration: scoping review and future directions.Journal of Medical Internet Research, 21(9):e12861, 2019

    work page 2019

  2. [2]

    Pros: an efficient pattern-driven compressive sensing framework for low-power biopotential-based wearables with on-chip intelligence

    Nhat Pham, Hong Jia, Minh Tran, Tuan Dinh, Nam Bui, Young Kwon, Dong Ma, Phuc Nguyen, Cecilia Mascolo, and Tam Vu. Pros: an efficient pattern-driven compressive sensing framework for low-power biopotential-based wearables with on-chip intelligence. InProceedings of the 28th Annual International Conference on Mobile Computing And Networking, pages 661–675, 2022

    work page 2022

  3. [3]

    Ur2m: Uncertainty and resource-aware event detection on microcontrollers

    Hong Jia, Young D Kwon, Dong Mat, Nhat Pham, Lorena Qendro, Tam Vu, and Cecilia Mascolo. Ur2m: Uncertainty and resource-aware event detection on microcontrollers. In2024 IEEE International Conference on Pervasive Computing and Communications (PerCom), pages 1–10. IEEE, 2024

    work page 2024

  4. [4]

    Udama: Unsupervised domain adaptation through multi-discriminator adversarial training with noisy labels improves cardio-fitness prediction

    Yu Wu, Dimitris Spathis, Hong Jia, Ignacio Perez-Pozuelo, Tomas I Gonzales, Soren Brage, Nicholas Wareham, and Cecilia Mascolo. Udama: Unsupervised domain adaptation through multi-discriminator adversarial training with noisy labels improves cardio-fitness prediction. In Machine Learning for Healthcare Conference, pages 863–883. PMLR, 2023

    work page 2023

  5. [5]

    AI on the Pulse: Real-Time Health Anomaly Detection with Wearable and Ambient Intelligence

    Lucas Gabrielli et al. Ai on the pulse: Integrating wearable sensors, ambient intelligence, and large language models for continuous health monitoring.arXiv preprint arXiv:2508.03436,

    work page internal anchor Pith review Pith/arXiv arXiv

  6. [6]

    Wearable eeg and ai for real-time personalized health monitoring and intervention.Journal of Cloud Computing, 14(1):1–15, 2025

    Yassir Ghadi et al. Wearable eeg and ai for real-time personalized health monitoring and intervention.Journal of Cloud Computing, 14(1):1–15, 2025

    work page 2025

  7. [7]

    A survey on large language models for sensor-based human activity recognition and health monitoring.Sensors, 24(15):5045, 2024

    Emilio Ferrara. A survey on large language models for sensor-based human activity recognition and health monitoring.Sensors, 24(15):5045, 2024

    work page 2024

  8. [8]

    Llasa: Multimodal large language models for interpreting human activity from inertial sensor data

    Muhammad Imran et al. Llasa: Multimodal large language models for interpreting human activity from inertial sensor data. InProceedings of the Annual Meeting of the Association for Computational Linguistics (ACL), 2024

    work page 2024

  9. [9]

    Personal health llms: Towards context-aware and adaptive health monitoring from wearable sensor data.Nature Medicine, 2025

    Amir Khasentino et al. Personal health llms: Towards context-aware and adaptive health monitoring from wearable sensor data.Nature Medicine, 2025. In press

    work page 2025

  10. [10]

    Security and privacy challenges of large language models.ACM Computing Surveys, 58(2):1–38, 2025

    Anish Das. Security and privacy challenges of large language models.ACM Computing Surveys, 58(2):1–38, 2025. doi: 10.1145/3712001. URL https://dl.acm.org/doi/10. 1145/3712001

    doi:10.1145/3712001 2025

  11. [11]

    H. Li, Y . Chen, J. Luo, Y . Kang, X. Zhang, Q. Hu, C. Chan, and Y . Song. Privacy in large language models: Attacks, defences and future directions.arXiv, 2024. URL https://arxiv. org/pdf/2310.10383. Available: https://arxiv.org/pdf/2310.10383

    Pith/arXiv arXiv 2024

  12. [12]

    Camel: Energy-Aware LLM Inference on Resource-Constrained Devices

    Tianqi Xu, Wei Zhang, Chen Li, and Yifan Wang. Camel: Energy-aware llm inference on resource-constrained devices.arXiv preprint arXiv:2508.09173, 2025

    work page internal anchor Pith review Pith/arXiv arXiv 2025

  13. [13]

    Large language models on edge devices: Challenges and opportunities for intelligent data analysis.Frontiers in Computer Science, 7:1538277, 2025

    Hui Wang, Qiang Liu, and Mei Chen. Large language models on edge devices: Challenges and opportunities for intelligent data analysis.Frontiers in Computer Science, 7:1538277, 2025

    work page 2025

  14. [14]

    Phi-3 technical report: A highly capable language model locally on your phone

    Microsoft. Phi-3 technical report: A highly capable language model locally on your phone. Technical Report, 2024. URLhttps://arxiv.org/pdf/2404.14219v4

    Pith/arXiv arXiv 2024

  15. [15]

    Tinyllama: An open-source small language model

    Peiyuan Zhang, Guangtao Zeng, Tianduo Wang, and Wei Lu. Tinyllama: An open-source small language model. https://github.com/jzhang38/TinyLlama, 2024. TinyLlama achieves approximately 70-80% of LLaMA2’s performance on commonsense reasoning tasks such as HellaSwag and ARC-Challenge

    work page 2024

  16. [16]

    Qwen2-1.5b: A new series of large language models, 2024

    Qwen. Qwen2-1.5b: A new series of large language models, 2024. URL https:// huggingface.co/Qwen/Qwen2-1.5B. Accessed: 2024-09-01. 10

    work page 2024

  17. [17]

    Gemma 2: Improving open language models at a practical size

    Gemma Team and Google DeepMind. Gemma 2: Improving open language models at a practical size. Technical report, Google DeepMind, 2024. For full author list, see Contributions and Acknowledgments section. Correspondence togemma-2-report@google.com

    work page 2024

  18. [18]

    Efficient and personalized mobile health event prediction via small language models

    Xin Wang, Ting Dang, Vassilis Kostakos, and Hong Jia. Efficient and personalized mobile health event prediction via small language models. InProceedings of the 30th Annual International Conference on Mobile Computing and Networking, ACM MobiCom ’24, page 2353–2358, New York, NY , USA, 2024. Association for Computing Machinery. ISBN 9798400704895. doi: 10....

    work page arXiv 2024

  19. [19]

    Marcus, Suchi Saria, and Daniel Halperin

    Brandon Ballinger, Joy Hsieh, Avesh Singh, Nitish Sohoni, Jae Wang, Fangfei Li, Amit Sharma, Akshay Sharma, Gregory M. Marcus, Suchi Saria, and Daniel Halperin. Deepheart: Semi-supervised sequence learning for cardiovascular risk prediction. InProceedings of the Thirty-Second AAAI Conference on Artificial Intelligence (AAAI), pages 2079–2086. AAAI Press, ...

    work page 2079

  20. [20]

    Kristján Hallgrímsson, Tom Goodwin, Sujit Ghosh, Peter Bühlmann, Christian Mathys, Vincent Lefort, Ara Darzi, Lionel Tarassenko, and David A. Clifton. Learning individualized cardio- vascular responses from large-scale wearable sensors data. InProceedings of the Thirty-Third AAAI Conference on Artificial Intelligence (AAAI), pages 941–948. AAAI Press, 201...

    work page 2019

  21. [21]

    Mullick, A

    T. Mullick, A. Radovic, S. Shaaban, and A. Doryab. Predicting depression in adolescents using mobile and wearable sensors: Multimodal machine learning–based exploratory study.JMIR Formative Research, 6(6):e35807, 2022. doi: 10.2196/35807. URL https://formative. jmir.org/2022/6/e35807

    work page doi:10.2196/35807 2022

  22. [22]

    Holtzman, Md

    Sebastian Kasl, Nathanael S. Holtzman, Md. Masudul Islam Shandhi, Tanishq Gupta, Jiang Kuang, Gregory D. Hager, Shawn S. Lam, and Suchi Saria. On the generalizability of wearable- based machine learning for respiratory virus detection. InProceedings of the 9th Machine Learning for Healthcare Conference (MLHC), volume 248 ofProceedings of Machine Learning ...

    work page 2024

  23. [23]

    Health-llm: Large language models for health prediction via wearable sensor data

    Yubin Kim, Xuhai Xu, Daniel McDuff, Cynthia Breazeal, and Hae Won Park. Health-llm: Large language models for health prediction via wearable sensor data. In Tom Pollard, Edward Choi, Pankhuri Singhal, Michael Hughes, Elena Sizikova, Bobak Mortazavi, Irene Chen, Fei Wang, Tasmie Sarker, Matthew McDermott, and Marzyeh Ghassemi, editors,Proceedings of the fi...

    work page 2024

  24. [24]

    Exploring Large-Scale Language Models to Evaluate EEG-Based Multimodal Data for Mental Health

    Yongquan Hu, Shuning Zhang, Ting Dang, Hong Jia, Flora D Salim, Wen Hu, and Aaron J Quigley. Exploring large-scale language models to evaluate eeg-based multimodal data for mental health.arXiv preprint arXiv:2408.07313, 2024

    work page internal anchor Pith review Pith/arXiv arXiv 2024

  25. [25]

    Dey, and Dakuo Wang

    Xuhai Xu, Bingsheng Yao, Yuanzhe Dong, Saadia Gabriel, Hong Yu, James Hendler, Marzyeh Ghassemi, Anind K. Dey, and Dakuo Wang. Mental-llm: Leveraging large language models for mental health prediction via online text data. InProceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies, volume 8, pages Article 32, 32 pages. Associati...

    doi:10.1145/3643540 2024

  26. [26]

    Large language models encode clinical knowledge.Nature, 620(7972):172–180, 2023

    Karan Singhal, Shekoofeh Azizi, Tao Tu, S Sara Mahdavi, Jason Wei, Hyung Won Chung, Nathan Scales, Ajay Tanwani, Heather Cole-Lewis, Stephen Pfohl, et al. Large language models encode clinical knowledge.Nature, 620(7972):172–180, 2023

    2023

  27. [27]

    Capa- bilities of gpt-4 on medical challenge problems.arXiv preprint arXiv:2303.13375, 2023

    Harsha Nori, Nicholas King, Scott Mayer McKinney, Dean Carignan, and Eric Horvitz. Capa- bilities of gpt-4 on medical challenge problems.arXiv preprint arXiv:2303.13375, 2023

    Pith/arXiv arXiv 2023

  28. [28]

    Physiollm: Supporting personalized health insights with wearables and large language models.arXiv preprint arXiv:2406.19283, 2024

    Cathy Mengying Fang, Valdemar Danry, Nathan Whitmore, Andria Bao, Andrew Hutchison, Cayden Pierce, and Pattie Maes. Physiollm: Supporting personalized health insights with wearables and large language models.arXiv preprint arXiv:2406.19283, 2024. 11

    Pith/arXiv arXiv 2024

  29. [29]

    Minicpm: Unveiling the potential of small language models with scalable training strategies.arXiv preprint arXiv:2404.06395, 2024

    Shengding Hu, Yuge Tu, Xu Han, Chaoqun He, Ganqu Cui, Xiang Long, Zhi Zheng, Yewei Fang, Yuxiang Huang, Weilin Zhao, et al. Minicpm: Unveiling the potential of small language models with scalable training strategies.arXiv preprint arXiv:2404.06395, 2024

    Pith/arXiv arXiv 2024

  30. [30]

    Lane, and Mengwei Xu

    Zhenyan Lu, Xiang Li, Dongqi Cai, Rongjie Yi, Fangming Liu, Wei Liu, Jian Luan, Xiwen Zhang, Nicholas D. Lane, and Mengwei Xu. Demystifying small language models for edge deployment. InProceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 14747–14764, Vienna, Austria, 2025

    work page 2025

  31. [31]

    Mobileaibench: Benchmarking llms and lmms for on-device use cases

    Rithesh Murthy, Liangwei Yang, Juntao Tan, Tulika Manoj Awalgaonkar, Yilun Zhou, Shelby Heinecke, Sachin Desai, Jason Wu, Ran Xu, Sarah Tan, et al. Mobileaibench: Benchmarking llms and lmms for on-device use cases. 2023

    work page 2023

  32. [32]

    Phi-3-mini-4k-instruct: A lightweight, state-of-the-art open model, 2024

    Microsoft. Phi-3-mini-4k-instruct: A lightweight, state-of-the-art open model, 2024. URL https://phi.microsoft.com/phi-3-mini-4k-instruct. Accessed: 2024-09-01

    work page 2024

  33. [33]

    Tinyllama-1.1b-chat-v1.0: A compact llama model with 1.1b parameters, 2024

    TinyLlama. Tinyllama-1.1b-chat-v1.0: A compact llama model with 1.1b parameters, 2024. URLhttps://github.com/jzhang38/TinyLlama. Accessed: 2024-09-01

    work page 2024

  34. [34]

    Gemma 2: A lightweight, state-of-the-art open model family, 2024

    Google. Gemma 2: A lightweight, state-of-the-art open model family, 2024. URL https: //huggingface.co/google/gemma-2-2b. Accessed: 2024-09-01

    work page 2024

  35. [35]

    Smollm-1.7b-instruct: A series of small language models, 2024

    HuggingFaceTB. Smollm-1.7b-instruct: A series of small language models, 2024. URL https://huggingface.co/HuggingFaceTB/SmolLM-1.7B-Instruct . Accessed: 2024- 09-01

    work page 2024

  36. [36]

    Llama 3.2 model card

    Meta AI. Llama 3.2 model card. Hugging Face, 2024. Release date: September 25, 2024. Includes lightweight text-only (1 B, 3 B) and multimodal (11 B, 90 B) models

    work page 2024

  37. [37]

    Pmdata: A sports logging dataset

    Vajira Thambawita, Steven Alexander Hicks, Hanna Borgli, Håkon Kvale Stensland, Debesh Jha, Martin Kristoffer Svensen, Svein-Arne Pettersen, Dag Johansen, Håvard Dagenborg Johansen, Susann Dahl Pettersen, Simon Nordvang, Sigurd Pedersen, Anders Gjerdrum, Tor-Morten Grønli, Per Morten Fredriksen, Ragnhild Eg, Kjeld Hansen, Siri Fagernes, Christine Claudi, ...

    work page arXiv 2020

  38. [38]

    Tom B. Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, Sandhini Agarwal, Ariel Herbert-V oss, Gretchen Krueger, Tom Henighan, Rewon Child, Aditya Ramesh, Daniel M. Ziegler, Jeffrey Wu, Clemens Winter, Christopher Hesse, Mark Chen, Eric Sigler, Mateusz Litwi...

    Pith/arXiv arXiv 2020

  39. [39]

    Chain-of-thought prompting elicits reasoning in large language models

    John Wei, Michael Bosma, Dale Schuurmans, and et al. Chain-of-thought prompting elicits reasoning in large language models. InProceedings of NeurIPS 2022, 2022. URL https: //arxiv.org/abs/2201.11903

    Pith/arXiv arXiv 2022

  40. [40]

    Self-consistency improves chain-of-thought reasoning in language models

    Xinyang Wang, Hinrich Schütze, and et al. Self-consistency improves chain-of-thought reasoning in language models. InProceedings of NeurIPS 2022, 2022. URL https: //arxiv.org/abs/2203.11171

    Pith/arXiv arXiv 2022

  41. [41]

    Zhao, Kelvin Guu, Adams Wei Yu, Brian Lester, Nan Du, Andrew M

    Jason Wei, Maarten Bosma, Vincent Y . Zhao, Kelvin Guu, Adams Wei Yu, Brian Lester, Nan Du, Andrew M. Dai, and Quoc V . Le. Finetuned language models are zero-shot learners. In Proceedings of the International Conference on Learning Representations (ICLR), 2022. URL https://iclr.cc/virtual/2022/oral/6255

    work page 2022

  42. [42]

    Stanford alpaca: An instruction-following llama model

    Rohan Taori, Ishaan Gulrajani, Tianyi Zhang, Yann Dubois, Xuechen Li, Carlos Guestrin, Percy Liang, and Tatsunori Hashimoto. Stanford alpaca: An instruction-following llama model. https://crfm.stanford.edu/2023/03/13/alpaca.html, 2023. Accessed: 2025-04-27. 12

    work page 2023

  43. [43]

    Smith, Iz Beltagy, and Hannaneh Hajishirzi

    Yizhong Wang, Hamish Ivison, Pradeep Dasigi, Jack Hessel, Tushar Khot, Khyathi Chandu, David Wadden, Kelsey MacMillan, Noah A. Smith, Iz Beltagy, and Hannaneh Hajishirzi. How far can camels go? exploring the state of instruction tuning on open resources. InAd- vances in Neural Information Processing Systems (NeurIPS), Datasets and Benchmarks Track,

    work page

  44. [44]

    Dolly: The first truly open-source instruction-tuned model

    Mike Conover, Matt Hayes, Ankit Mathur, Jianwei Xie, Sam Shah, Ali Ghodsi, Patrick Wendell, and Matei Zaharia. Dolly: The first truly open-source instruction-tuned model. Databricks blog,

    work page

  45. [45]

    Vicuna: An open-source chatbot impressing gpt-4 with 90 LMSYS blog post,

    Vicuna Team. Vicuna: An open-source chatbot impressing gpt-4 with 90 LMSYS blog post,

    work page

  46. [46]

    Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, and Weizhu Chen

    Edward J. Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, and Weizhu Chen. Lora: Low-rank adaptation of large language models. In Proceedings of the Tenth International Conference on Learning Representations (ICLR), 2022. URLhttps://openreview.net/forum?id=nZeVKeeFYf9

    work page 2022

  47. [47]

    Fine-tuned on the Stanford Alpaca dataset

    work page

  48. [48]

    Replication data for: Using machine learning methods to predict physical activity types with apple watch and fitbit data using indirect calorimetry as the criterion, 2020

    Daniel Fuller. Replication data for: Using machine learning methods to predict physical activity types with apple watch and fitbit data using indirect calorimetry as the criterion, 2020. URL https://doi.org/10.7910/DVN/ZS2Z2J. Accessed: 2025-04-29

    work page doi:10.7910/dvn/zs2z2j 2020

  49. [49]

    Instruction tuning inspired by Alpaca’s methodology

    work page

  50. [50]

    Geneactiv: Raw data accelerometer for physical activity and sleep research

    Activinsights Ltd. Geneactiv: Raw data accelerometer for physical activity and sleep research. https://www.activinsights.com/products/geneactiv/, 2015. Accessed: 2025-08- 30

    work page 2015

  51. [51]

    Morris, Eve Riskin, Jennifer Mankoff, and Anind K

    Xuhai Xu, Han Zhang, Yasaman Sefidgar, Yiyi Ren, Xin Liu, Woosuk Seo, Jennifer Brown, Kevin Kuehn, Mike Merrill, Paula Nurius, Shwetak Patel, Tim Althoff, Margaret E. Morris, Eve Riskin, Jennifer Mankoff, and Anind K. Dey. Globem dataset: Multi-year datasets for longitudinal human behavior modeling generalization, 2023. URL https://arxiv.org/abs/ 2211.02733

    Pith/arXiv arXiv 2023

  52. [52]

    Fitbit charge 2: Heart rate + fitness wristband

    Fitbit Inc. Fitbit charge 2: Heart rate + fitness wristband. https://www.fitbit.com/ global/us/products/trackers/charge2, 2016. Accessed: 2025-08-30

    work page 2016

  53. [53]

    Fitbit versa 2: Health & fitness smartwatch

    Fitbit Inc. Fitbit versa 2: Health & fitness smartwatch. https://www.fitbit.com/global/ us/products/smartwatches/versa2, 2019. Accessed: 2025-08-30

    work page 2019

  54. [54]

    Tinyllama: A distilled version of llama for efficient language tasks

    LLaMA Open Source Team. Tinyllama: A distilled version of llama for efficient language tasks. https://github.com/TinyLlama, 2024. Highlights the use of knowledge distillation for TinyLlama-1.1B derived from LLaMA-13B

    work page 2024

  55. [55]

    Apple watch series 2

    Apple Inc. Apple watch series 2. https://support.apple.com/kb/SP745, 2016. Accessed: 2025-08-30

    work page 2016

  56. [56]

    Gguf, 2023

    Hugging Face. Gguf, 2023. URL https://huggingface.co/docs/hub/en/gguf. Ac- cessed: 2025-09-05

    work page 2023

  57. [57]

    Fine-tune small language model (slm) phi-3 using azure machine learning

    Microsoft Corporation. Fine-tune small language model (slm) phi-3 using azure machine learning. https://techcommunity.microsoft.com/t5/ai-machine-learning-blog/ fine-tune-small-language-model-slm-phi-3-using-azure-machine/ba-p/ 4130399, 2024. Accessed: 2025-06-09

    work page 2024

  58. [58]

    llama.cpp: Efficient llm inference in c/c++

    Georgi Gerganov and community. llama.cpp: Efficient llm inference in c/c++. https:// github.com/ggml-org/llama.cpp, 2023. Released March 10, 2023; accessed 2025-09-06. 13

    work page 2023

  59. [59]

    Beyond scale: Small language models are comparable to gpt-4 in mental health understanding.arXiv preprint arXiv:2507.08031, 2025

    Hong Jia, Shiya Fu, Feng Xia, Vassilis Kostakos, and Ting Dang. Beyond scale: Small language models are comparable to gpt-4 in mental health understanding.arXiv preprint arXiv:2507.08031, 2025

    Pith/arXiv arXiv 2025

  60. [60]

    Qwen2.5 technical report, 2025

    Qwen, :, An Yang, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chengyuan Li, Dayiheng Liu, Fei Huang, Haoran Wei, Huan Lin, Jian Yang, Jianhong Tu, Jianwei Zhang, Jianxin Yang, Jiaxi Yang, Jingren Zhou, Junyang Lin, Kai Dang, Keming Lu, Keqin Bao, Kexin Yang, Le Yu, Mei Li, Mingfeng Xue, Pei Zhang, Qin Zhu, Rui Men, Runji Lin, Tianhao Li,...

    Pith/arXiv arXiv 2025

  61. [61]

    Ggerganov/llama.cpp: Llm inference in c/c++

    Ggerganov. Ggerganov/llama.cpp: Llm inference in c/c++. URL https://github.com/ ggerganov/llama.cpp

    work page

  62. [62]

    3” when the true label is “4

    Trevor Hastie, Robert Tibshirani, and Jerome Friedman.The Elements of Statistical Learning: Data Mining, Inference, and Prediction. Springer, 2009. 14 Appendix A Implementation Details We fine-tune our SLMs on a NVIDIA A100 80GB GPUs with a batch size of 128 with 3 number of epochs for the purpose of fine-tuning, with Adam optimizer and a learning rate as...

    work page 2009

  63. [63]

    Kwon, Alessio Orsino, Ting Dang, Domenico Talia, and Cecilia Mascolo

    Hong Jia, Young D. Kwon, Alessio Orsino, Ting Dang, Domenico Talia, and Cecilia Mascolo. Tinytta: Efficient test-time adaptation via early-exit ensembles on edge devices. InAdvances in Neural Information Processing Systems, volume 37, pages 43274–43299, 2024

    work page 2024

  64. [65]

    Bishop.Pattern Recognition and Machine Learning

    Christopher M. Bishop.Pattern Recognition and Machine Learning. Springer, 2006

    2006

  65. [2023]

    URL https://proceedings.neurips.cc/paper_files/paper/2023/hash/ ec6413875e4ab08d7bc4d8e225263398-Abstract-Datasets_and_Benchmarks.html

    work page 2023

  66. [2025]

    Accessed: 2025-09-04

    work page 2025