RECORD
Quantifying Impairment and Disease Severity Using AI Models Trained on Healthy Subjects
Machine Learning · Submitted 21 Nov 2023
abstract compiled · 2026-09-23 · no language-model review
The authors' main claim The COBRA score, computed automatically in under one minute, is shown to be strongly correlated with the FMA on an independent test cohort for two different data modalities: wearable sensors ($\rho = 0.845$, 95% CI [0.743,0.908]) and video ($\rho = 0.746$, 95% C.I [0.594, 0.847]).
The compiled record
The paper's own sentences, arranged. Each label is drawn from a closed vocabulary; the number beside it is the probability the classifier gave that label. Numbers in the tables are the authors', checked by code. No sentence in this record was written by a model.
Open the compiled record
The abstract, sentence by sentence
Automatic assessment of impairment and disease severity is a key challenge in data-driven medicine. We propose a novel framework to address this challenge, which leverages AI models trained exclusively on healthy individuals. The COnfidence-Based chaRacterization of Anomalies (COBRA) score exploits the decrease in confidence of these models when presented with impaired or diseased patients to quantify their deviation from the healthy population. We applied the COBRA score to address a key limitation of current clinical evaluation of upper-body impairment in stroke patients. The gold-standard Fugl-Meyer Assessment (FMA) requires in-person administration by a trained assessor for 30-45 minutes, which restricts monitoring frequency and precludes physicians from adapting rehabilitation protocols to the progress of each patient. The COBRA score, computed automatically in under one minute, is shown to be strongly correlated with the FMA on an independent test cohort for two different data modalities: wearable sensors ($\rho = 0.845$, 95% CI [0.743,0.908]) and video ($\rho = 0.746$, 95% C.I [0.594, 0.847]). To demonstrate the generalizability of the approach to other conditions, the COBRA score was also applied to quantify severity of knee osteoarthritis from magnetic-resonance imaging scans, again achieving significant correlation with an independent clinical assessment ($\rho = 0.644$, 95% C.I [0.585,0.696]).
background contribution method result interpretation main claim
| # | sentence | role | the authors' own work | states the main conclusion | strength |
|---|---|---|---|---|---|
| s0 | Automatic assessment of impairment and disease severity is a key challenge in data-driven medicine. | background 1.00 | 0.04 | 0.03 | asserted 1.00 |
| s1 | We propose a novel framework to address this challenge, which leverages AI models trained exclusively on healthy individuals. | contribution 0.98 | 0.83 | 0.06 | asserted 0.98 |
| s2 | The COnfidence-Based chaRacterization of Anomalies (COBRA) score exploits the decrease in confidence of these models when presented with impaired or diseased patients to quantify their deviation from the healthy population. | method 0.85 | 0.68 | 0.12 | asserted 1.00 |
| s3 | We applied the COBRA score to address a key limitation of current clinical evaluation of upper-body impairment in stroke patients. | contribution 0.84 | 0.93 | 0.05 | asserted 0.99 |
| s4 | The gold-standard Fugl-Meyer Assessment (FMA) requires in-person administration by a trained assessor for 30-45 minutes, which restricts monitoring frequency and precludes physicians from adapting rehabilitation protocols to the progress of each patient. | background 0.68 | 0.07 | 0.03 | asserted 1.00 |
| s5 | The COBRA score, computed automatically in under one minute, is shown to be strongly correlated with the FMA on an independent test cohort for two different data modalities: wearable sensors ($\rho = 0.845$, 95% CI [0.743,0.908]) and video ($\rho = 0.746$, 95% C.I [0.594, 0.847]). | result 1.00 | 0.96 | 0.48 | asserted 1.00 |
| s6 | To demonstrate the generalizability of the approach to other conditions, the COBRA score was also applied to quantify severity of knee osteoarthritis from magnetic-resonance imaging scans, again achieving significant correlation with an independent clinical assessment ($\rho = 0.644$, 95% C.I [0.585,0.696]). | result 0.97 | 0.96 | 0.18 | asserted 1.00 |
What was examined, and what was not
- arXiv listing abstract v1
- full text
- figures and tables
- references and citing sentences
- relations to other papers
- P abstract
- S sections
- T body sentences
- C citations
- R cross-paper relations
The paper
Original abstract
Automatic assessment of impairment and disease severity is a key challenge in data-driven medicine. We propose a novel framework to address this challenge, which leverages AI models trained exclusively on healthy individuals. The COnfidence-Based chaRacterization of Anomalies (COBRA) score exploits the decrease in confidence of these models when presented with impaired or diseased patients to quantify their deviation from the healthy population. We applied the COBRA score to address a key limitation of current clinical evaluation of upper-body impairment in stroke patients. The gold-standard Fugl-Meyer Assessment (FMA) requires in-person administration by a trained assessor for 30-45 minutes, which restricts monitoring frequency and precludes physicians from adapting rehabilitation protocols to the progress of each patient. The COBRA score, computed automatically in under one minute, is shown to be strongly correlated with the FMA on an independent test cohort for two different data modalities: wearable sensors ($\rho = 0.845$, 95% CI [0.743,0.908]) and video ($\rho = 0.746$, 95% C.I [0.594, 0.847]). To demonstrate the generalizability of the approach to other conditions, the COBRA score was also applied to quantify severity of knee osteoarthritis from magnetic-resonance imaging scans, again achieving significant correlation with an independent clinical assessment ($\rho = 0.644$, 95% C.I [0.585,0.696]).
Human review
How the literature uses it
What this paper does with the arXiv papers it cites
What it cites · 3 sentences without a confident label
Uses the method or tool of 2 papers
-
The COBRA score is inspired by a technique proposed in [ref] , which identifies anomalous data points using the confidence of AI models.
A Baseline for Detecting Misclassified and Out-of-Distribution Examples in Neural Networks · Introduction · 0.85
-
This cost function was minimized for 50 epochs using the Adam optimizer [ref] with a learning rate of $5·10^-3$ (selected via cross-validation).
Adam: A Method for Stochastic Optimization · Wearable sensors · 1.00
Cites as background 3 papers
-
Several existing works propose to use confidence values to perform anomaly detection [ref] .
A Baseline for Detecting Misclassified and Out-of-Distribution Examples in Neural Networks · Confidence-based Characterization of Anomalies · 0.86
-
Several existing works propose to use confidence values to perform anomaly detection [ref] .
Learning Confidence for Out-of-Distribution Detection in Neural Networks · Confidence-based Characterization of Anomalies · 0.86
-
Several existing works propose to use confidence values to perform anomaly detection [ref] .
Robust Out-of-distribution Detection for Neural Networks · Confidence-based Characterization of Anomalies · 0.86
Discussion
Know something about this paper? Leave a comment, or highlight any passage above to comment on it.