REVIEW 4 major objections 6 minor 41 references
Rehabilitation Exercise Quality Assessment and Feedback Generation Using Large Language Models with Prompt Engineering
T0 review · 4 major / 6 minor · reviewed 2026-08-07 · deepseek-v4-flash
Pith's one-line read Pre-trained LLMs can assess rehabilitation exercise quality and generate corrective feedback from engineered prompts and joint-angle features alone.
desk verdict A plausible proof-of-concept for LLM-based rehab exercise feedback, with honest but optimistic evaluation that needs held-out prompt selection and repeated runs before the headline claim is fully supported. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The load-bearing mechanism is the pairing of exercise-specific feature sequences with prompt-engineering techniques inside a pre-trained LLM (GPT-4o). Features are extracted frame-by-frame from skeletal joints according to clinician-defined movement criteria, reducing each repetition to a frames-by-features array with three to five columns. The prompts govern how the LLM consumes those arrays: few-shot examples establish the classification task, chain-of-thought elicits step-by-step reasoning, certainty and probability elicitation extract confidence scores, and role-play instructs the model to answer as a physiotherapist. The feature abstraction is what makes skeletal data legible to a text model.
What would settle it
Construct or collect repetitions whose ground-truth incorrectness comes from an error absent from the extracted features, for example a leg lunge with subtle upper-body instability but normal knee angle, trunk angle, and foot distance, and have physiotherapists label them. If GPT-4o with the paper's prompts labels them correct, the central claim fails for any movement whose defining error lies outside the feature list; the false-correct rate on such examples quantifies exactly how far the method generalizes.
Extended reading notes
Core claim
The central discovery is that a generic pre-trained LLM can act as a rehabilitation exercise assessor when raw joint sequences are replaced by three to five clinician-informed features per frame, such as knee valgus angle, thigh angle, and pelvic stability. On REHAB24-6, the prompt-based LLM (best accuracy 0.70 with certainty prompting) outperformed the paper's LSTM baseline (0.60) and ST-GCN baseline (0.63); on UI-PRMD it reached 0.76 best accuracy, below the deep baselines (0.87 and 0.94). The authors treat this as demonstrating feasibility and interpretability: the same model that issues a quality label can explain its reasoning and, in a second role-play step, advise the patient what to fix.
Load-bearing premise
The load-bearing premise is that the few extracted joint angles capture every error that defines an incorrect repetition; if a defining error is not among them, the LLM will call the movement correct no matter how well the prompt is written.
Editorial extensions
If this is right
- A pre-trained LLM with three-shot prompts and exercise-specific features can assess exercise quality on two public datasets without fine-tuning, removing the need for a rehabilitation dataset with ground-truth text feedback for the assessment step.
- Certainty prompting gives the best overall accuracy and F1 on both datasets and is less prone to the hallucinated thresholds observed with chain-of-thought reasoning.
- The LLM beats the reported LSTM and ST-GCN baselines on REHAB24-6 but not on UI-PRMD, so its advantage is dataset-dependent; on easier, more separable data traditional deep models still win.
- Role-play prompting after classification produces corrective feedback that combines trends from the extracted features with general movement knowledge, addressing a gap left by label-only quality-assessment models.
- LLM confidence scores are overconfident (mostly 0.8 to 1.0 regardless of correctness), so probability outputs are not directly usable as calibrated certainty.
Reading between the lines
- An untested extension implied by the paper's feature-set limitation: expanding the features (for example, adding trunk sway or foot-pressure estimates) could lift LLM accuracy on UI-PRMD toward the deep-model baselines; the authors do not run this experiment.
- Because feedback is only qualitatively evaluated, a quantified next step is to have physiotherapists blindly rate LLM feedback against their own corrections; the paper identifies the missing ground-truth data but does not conduct such a study.
- The same feature-plus-prompt recipe could plausibly transfer to other rubric-based movement domains such as sports technique coaching, but this is our extrapolation, not a result the paper claims.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper proposes a method for rehabilitation exercise quality assessment and feedback generation using pre-trained LLMs (GPT-4o) with prompt engineering. Exercise-specific features are extracted from body joint sequences based on clinical criteria, and the LLM is prompted (zero-shot, few-shot, chain-of-thought, certainty, probability, role-play) to classify each repetition as correct or incorrect and to generate textual feedback. The method is evaluated on two public datasets, UI-PRMD and REHAB24-6, with accuracy, precision, recall, and F1 reported, alongside qualitative feedback examples. The main claims are that LLMs can assess exercise quality without fine-tuning and that the approach outperforms or complements existing deep learning baselines.
Significance. If the claims hold, the paper would make a useful contribution by showing that pre-trained LLMs can provide interpretable, feedback-generating rehabilitation assessment without task-specific fine-tuning, which is relevant for virtual rehabilitation platforms where annotated textual feedback is scarce. The paper is honest about several limitations, including the absence of quantitative feedback evaluation and the non-determinism of GPT-4o. It also releases code and reproducible feature-extraction scripts, which is a strength. However, the current evidence is not fully convincing: the evaluation protocol selects hyperparameters on the test set, the feature sets are acknowledged to be incomplete for capturing all movement errors, and the quantitative results on UI-PRMD are substantially below standard deep learning baselines.
major comments (4)
- [Section 3.1, Table 1, and Section 4.3] The method feeds only the hand-engineered features of Table 1 to the LLM, so any movement error not captured by those features is undetectable regardless of prompting. The paper itself concedes this in Section 4.3: a leg lunge with subtle upper-body instability may be classified as correct because upper-body instability is not among the extracted features. Since feedback is generated from the same features (Section 3.2, Figure 3), this incompleteness propagates to the feedback contribution. The authors should either provide evidence that the feature sets are sufficient to explain the ground-truth labels (e.g., an analysis showing that every 'incorrect' label corresponds to a detectable deviation in the extracted features) or explicitly scope the central claim to errors that are reflected in the chosen features.
- [Section 4.2, Figure 2, and Table 3] The experimental protocol selects the optimal number of few-shot examples k and the best prompting technique using the same test data on which the final accuracies are reported. This is test-set selection, which makes the reported accuracies optimistic. In addition, no repeated runs, confidence intervals, or statistical tests are provided, despite the paper's own acknowledgement in Section 5 that GPT-4o's non-determinism affects reproducibility. Consequently, the claims that '3-shot prompting is optimal' and that certainty prompting outperforms other techniques (Table 4) are not statistically supported.
- [Section 4.2, Table 4(a)] On UI-PRMD, the best LLM accuracy (0.76 with certainty prompting) is substantially lower than the LSTM (0.87) and ST-GCN (0.94) baselines. The paper acknowledges this but still describes the results as 'promising results' in the abstract and 'high accuracy' in the conclusion. Given that UI-PRMD is one of the two main datasets, the quantitative support for the central assessment claim is weaker than the paper's overall framing suggests. The authors should qualify their claims by dataset and discuss the implications of the large gap on this benchmark.
- [Section 3.3 and Section 4.3] The feedback generation contribution is evaluated only qualitatively, with three illustrative examples in Figure 3. There is no human evaluation, rubric-based scoring, or comparison against existing feedback systems. While the lack of ground-truth textual feedback is a known limitation and is acknowledged in Section 5, feedback generation is a central claimed contribution of the paper. Without a more structured evaluation, the usefulness of the generated feedback is not demonstrated beyond anecdote.
minor comments (6)
- [Section 3.2] The text defines k-shot prompting with k in {1,2,3,4}, but Figure 2 and Table 3 also report results for 0-shot and 5-shot. Please clarify the exact range of k used in the experiments.
- [Section 3.3 and Table 4] For probability elicitation, the reported accuracy, precision, recall, and F1 scores require a decision threshold, but the threshold choice is not specified. Please state how the threshold was selected for these metrics.
- [Figure 1 and Section 3.1] The text contains formatting artifacts such as 'num_f rames' and 'num_f eatures'; these should be corrected to 'num_frames' and 'num_features'.
- [Table 2] The caption describes the prompt as '2-shot prompting' yet provides four labeled examples. This is consistent if k denotes the number of examples per class, but the caption should state this explicitly to avoid confusion.
- [Section 4.3, Table 5] The per-exercise results in Table 5 are reported without the number of samples per exercise or any variability measure, making it difficult to judge whether the differences among exercises are meaningful.
- [Section 2.2] The sentence describing Wang et al. [18] as 'the first LLM-enabled platform' makes a strong priority claim that may be difficult to verify; consider softening it to 'one of the first'.
Circularity Check
The evaluation loop is partially circular: the best few-shot setting is selected on the same test set later used to report accuracy, so the headline results are selection-maximized rather than independent predictions.
-
fitted input called prediction
[Section 4.2, Few-shot Prompting and Reasoning Elicitation]
"The optimal k-shot setting, which achieved the highest exercise quality classification accuracy, was used in the following more advanced prompting techniques. Building on the best-performing setting from the few-shot prompting experiments, three-shot prompting with feature sequences was selected for reasoning elicitation and subsequent experiments."
The hyperparameter k (and later the 'best-performing setting') is chosen by maximizing classification accuracy on the same test split that is subsequently used to report the accuracy of CoT, certainty, probability, and cross-exercise results. The reported numbers for the selected configuration are therefore the maximum of a model-selection sweep evaluated on the test data, not an independent out-of-sample prediction. The evaluation loop is self-referential: the test labels determine which prompt configuration is reported, and the same labels then score that configuration, so the headline accuracy is forced upward by the selection criterion.
full rationale
The central derivation — extracting hand-crafted exercise-specific features and asking a pre-trained LLM to classify and give feedback — is not circular by itself. The features are derived from clinical criteria, but the ground-truth labels are not literally contained in the prompt, and the few-shot examples are needed to convey thresholds. Section 4.3's admission that subtle upper-body instability is missed shows the feature set is incomplete, which is a feature-completeness limitation rather than a definitional equivalence. The few citations to the authors' own prior work ([13], [22]) are background and feature-engineering context, not a load-bearing uniqueness or ansatz argument. The one concrete circular step is the test-set-based selection of the few-shot setting: the paper chooses the prompt configuration that maximizes accuracy on the test set and then reports accuracy on that same test set as evidence for the method. This makes the reported evaluation partially in-sample and the numeric comparisons optimistically biased, but it does not reduce the entire contribution to a fit or to a self-citation chain. Score 4 reflects this partial circularity in the evaluation loop while acknowledging the method itself has independent content.
Assumptions & free parameters
free parameters (2)
- Number of few-shot examples k =
3
- Prompting technique =
3-shot + certainty elicitation
assumptions (3)
- domain assumption The hand-crafted exercise-specific features fully capture the clinically relevant movement errors for each exercise.
- domain assumption The binary correct/incorrect labels in UI-PRMD and REHAB24-6 are valid ground truth for exercise quality.
- ad hoc to paper GPT-4o's pre-trained knowledge is sufficient to interpret feature magnitudes without explicit clinical thresholds being supplied.
Cite this review
Pith. "Pith review of Rehabilitation Exercise Quality Assessment and Feedback Generation Using Large Language Models with Prompt Engineering." pith.science (2026). https://pith.science/paper/RKJBHCJW
@misc{pith2026250518412,
author = {Pith},
title = {Pith review of: Rehabilitation Exercise Quality Assessment and Feedback Generation Using Large Language Models with Prompt Engineering},
year = {2026},
howpublished = {\url{https://pith.science/paper/RKJBHCJW}},
note = {Machine review of arXiv:2505.18412}
}
read the original abstract
Exercise-based rehabilitation improves quality of life and reduces morbidity, mortality, and rehospitalization, though transportation constraints and staff shortages lead to high dropout rates from rehabilitation programs. Virtual platforms enable patients to complete prescribed exercises at home, while AI algorithms analyze performance, deliver feedback, and update clinicians. Although many studies have developed machine learning and deep learning models for exercise quality assessment, few have explored the use of large language models (LLMs) for feedback and are limited by the lack of rehabilitation datasets containing textual feedback. In this paper, we propose a new method in which exercise-specific features are extracted from the skeletal joints of patients performing rehabilitation exercises and fed into pre-trained LLMs. Using a range of prompting techniques, such as zero-shot, few-shot, chain-of-thought, and role-play prompting, LLMs are leveraged to evaluate exercise quality and provide feedback in natural language to help patients improve their movements. The method was evaluated through extensive experiments on two publicly available rehabilitation exercise assessment datasets (UI-PRMD and REHAB24-6) and showed promising results in exercise assessment, reasoning, and feedback generation. This approach can be integrated into virtual rehabilitation platforms to help patients perform exercises correctly, support recovery, and improve health outcomes.
Figures
Reference graph
Works this paper leans on
-
[1]
World Health Organization, “Rehabilitation,” https://www.who.int/news-room/ fact-sheets/detail/rehabilitation, 2023, Accessed: January 30, 2023
work page 2023
-
[2]
Exercise-based cardiac rehabilitation for coronary heart disease: a meta-analysis,
G. O. Dibben, J. Faulkner, N. Oldridge, K. Rees, D. R. Thompson, A.-D. Zwisler, and R. S. Taylor, “Exercise-based cardiac rehabilitation for coronary heart disease: a meta-analysis,” European heart journal, vol. 44, no. 6, pp. 452–469, 2023
work page 2023
-
[3]
S. Shirozhan, N. Arsalani, S. S. B. Maddah, and F. Mohammadi-Shahboulaghi, “Barriers and facilitators of rehabilitation nursing care for patients with disability in the rehabilitation hospital: A qualitative study,” Frontiers in Public Health, vol. 10, 2022
work page 2022
-
[4]
Usage of auxiliary systems and artificial intelligence in home-based rehabilitation: A review,
R. Ferreira, R. Santos, and A. Sousa, “Usage of auxiliary systems and artificial intelligence in home-based rehabilitation: A review,”Exploring the Convergence of Computer and Medical Science Through Cloud Healthcare, pp. 163–196, 2023
work page 2023
-
[5]
Effective- ness of telerehabilitation in physical therapy: a rapid overview,
P. Seron, M.-J. Oliveros, R. Gutierrez-Arias, R. Fuentes-Aspe, R. C. Torres-Castro, C. Merino-Osorio, P. Nahuelhual, J. Inostroza, Y. Jalil, R. Solanoet al., “Effective- ness of telerehabilitation in physical therapy: a rapid overview,”Physical therapy, vol. 101, no. 6, p. pzab053, 2021
work page 2021
-
[6]
I. Boukhennoufa, X. Zhai, V. Utti, J. Jackson, and K. D. McDonald-Maier, “Wear- able sensors and machine learning in post-stroke rehabilitation assessment: A sys- tematic review,” Biomedical Signal Processing and Control, vol. 71, p. 103197, 2022
work page 2022
-
[7]
Artificial intelligence-driven virtual rehabilitation for people living in the community: A scoping review,
A. Abedi, T. J. Colella, M. Pakosh, and S. S. Khan, “Artificial intelligence-driven virtual rehabilitation for people living in the community: A scoping review,”NPJ Digital Medicine, vol. 7, no. 1, p. 25, 2024
2024
-
[8]
S. Sardari, S. Sharifzadeh, A. Daneshkhah, B. Nakisa, S. W. Loke, V. Palade, and M. J. Duncan, “Artificial intelligence for skeleton-based physical rehabilitation action evaluation: A systematic review,”Computers in Biology and Medicine, p. 106835, 2023
work page 2023
Show all 41 references
-
[9]
Technological advances in lower-limb tele- rehabilitation:Areviewofliterature,
A. Ettefagh and A. Roshan Fekr, “Technological advances in lower-limb tele- rehabilitation:Areviewofliterature,” Journal of Rehabilitation and Assistive Tech- nologies Engineering, vol. 11, p. 20556683241259256, 2024
2024
-
[10]
Feedback design in targeted exercise digital biofeedback systems for home rehabilitation: A scoping review,
L. Brennan, E. Dorronzoro Zubiete, and B. Caulfield, “Feedback design in targeted exercise digital biofeedback systems for home rehabilitation: A scoping review,” Sensors, vol. 20, no. 1, p. 181, 2019
2019
-
[11]
3d human pose estimation in video with temporal convolutions and semi-supervised training,
D. Pavllo, C. Feichtenhofer, D. Grangier, and M. Auli, “3d human pose estimation in video with temporal convolutions and semi-supervised training,” inProceedings of the IEEE/CVF conference on computer vision and pattern recognition, 2019, pp. 7753–7762. Rehabilitation Exercise...
2019
-
[12]
Mediapipe: A framework for building perception pipelines,
C. Lugaresi, J. Tang, H. Nash, C. McClanahan, E. Uboweja, M. Hays, F. Zhang, C.-L. Chang, M. G. Yong, J. Leeet al., “Mediapipe: A framework for building perception pipelines,” arXiv preprint arXiv:1906.08172, 2019
1906 arXiv
-
[13]
Cross-modal video to body-joints augmentation for rehabilitation exercise quality assessment,
A. Abedi, M. Malmirian, and S. S. Khan, “Cross-modal video to body-joints augmentation for rehabilitation exercise quality assessment,” arXiv preprint arXiv:2306.09546, 2023
2023 arXiv
-
[14]
Spatial temporal graph convolutional networks for skeleton-based action recognition,
S. Yan, Y. Xiong, and D. Lin, “Spatial temporal graph convolutional networks for skeleton-based action recognition,” inProceedings of the AAAI conference on artificial intelligence, vol. 32, no. 1, 2018
2018
-
[15]
The kimore dataset: Kinematic assessment of movement and clinical scores for remote monitoring of physical rehabilitation,
M. Capecci, M. Ceravolo, F. Ferracuti, S. Iarlori, A. Monteriu, L. Romeo, and F. Verdini, “The kimore dataset: Kinematic assessment of movement and clinical scores for remote monitoring of physical rehabilitation,” IEEE Transactions on Neural Systems and Rehabilitation Enginee...
2019
-
[16]
Relevance of therapist feedback in the context of group-based exercise programs in medical rehabilitation–results from a qualitative study with patients and exercise therapists,
A. A. Schuber and A. Schaller, “Relevance of therapist feedback in the context of group-based exercise programs in medical rehabilitation–results from a qualitative study with patients and exercise therapists,”European Journal of Physiotherapy, pp. 1–9, 2024
2024
-
[17]
Feedback- mediated upper extremities exercise: Increasing patient motivation in poststroke rehabilitation,
M. D. Popović, M. D. Kostić, S. Z. Rodić, and L. M. Konstantinović, “Feedback- mediated upper extremities exercise: Increasing patient motivation in poststroke rehabilitation,” BioMed research international, vol. 2014, no. 1, p. 520374, 2014
2014
-
[18]
Ubiphysio: Support daily functioning, fitness, and rehabilitation with action understanding and feedback in natural language,
C. Wang, Y. Feng, L. Zhong, S. Zhu, C. Zhang, S. Zheng, C. Liang, Y. Wang, C. He, C. Yuet al., “Ubiphysio: Support daily functioning, fitness, and rehabilitation with action understanding and feedback in natural language,”Proceedings of the ACM on Interactive, Mobile, Wearable...
2024
-
[19]
A data set of human body movements for physical rehabilitation exercises,
A. Vakanski, H.-p. Jun, D. Paul, and R. Baker, “A data set of human body movements for physical rehabilitation exercises,” Data, vol. 3, no. 1, 2018. [Online]. Available: https://www.mdpi.com/2306-5729/3/1/2
2018
-
[20]
Rehab24-6: Physical therapy dataset for analyzing pose estimation methods,
A. Černek, J. Sedmidubsky, and P. Budikova, “Rehab24-6: Physical therapy dataset for analyzing pose estimation methods,” inInternational Conference on Similarity Search and Applications. Springer, 2024, pp. 18–33
2024
-
[21]
A deep learning framework for assessing physical rehabilitation exercises,
Y. Liao, A. Vakanski, and M. Xian, “A deep learning framework for assessing physical rehabilitation exercises,”IEEE Transactions on Neural Systems and Re- habilitation Engineering, vol. 28, no. 2, pp. 468–477, 2020
2020
-
[22]
Exercise-specific feature extraction approach for assessing physical rehabilitation,
Q. Guo and S. S. Khan, “Exercise-specific feature extraction approach for assessing physical rehabilitation,” in 4th IJCAI Workshop on AI for Aging, Rehabilitation and Intelligent Assisted Living. IJCAI, 2021
2021
-
[23]
Supervised sequential contrastive regression: Improving performance on imbalanced rehabilitation exercises datasets,
B. Karagoz, A. Ashraf, and S. Khan, “Supervised sequential contrastive regression: Improving performance on imbalanced rehabilitation exercises datasets,”preprint, 12 2023
2023
-
[24]
Graph convolutional networks for assessment of physical rehabilitation exercises,
S. Deb, M. F. Islam, S. Rahman, and S. Rahman, “Graph convolutional networks for assessment of physical rehabilitation exercises,”IEEE Transactions on Neural Systems and Rehabilitation Engineering, vol. 30, pp. 410–419, 2022
2022
-
[25]
A skeleton-based rehabilitation exer- cise assessment system with rotation invariance,
K. Zheng, J. Wu, J. Zhang, and C. Guo, “A skeleton-based rehabilitation exer- cise assessment system with rotation invariance,”IEEE Transactions on Neural Systems and Rehabilitation Engineering, 2023
2023
-
[26]
Graph transformer for phys- ical rehabilitation evaluation,
K. Réby, I. Dulau, G. Dubrasquet, and M. B. Aimar, “Graph transformer for phys- ical rehabilitation evaluation,” in 2023 IEEE 17th International Conference on Automatic Face and Gesture Recognition (FG). IEEE, 2023, pp. 1–8. 16 Tang et al
2023
-
[27]
Rehabilitation exercise quality assessment through supervised contrastive learning with hard and soft negatives,
M. Karlov, A. Abedi, and S. S. Khan, “Rehabilitation exercise quality assessment through supervised contrastive learning with hard and soft negatives,”Medical & Biological Engineering & Computing, pp. 1–14, 2024
2024
-
[28]
Egcn++: A new fusion strategy for ensemble learning in skeleton-based rehabilitation exercise assessment,
X. Bruce, Y. Liu, K. C. Chan, and C. W. Chen, “Egcn++: A new fusion strategy for ensemble learning in skeleton-based rehabilitation exercise assessment,”IEEE Transactions on Pattern Analysis and Machine Intelligence, 2024
2024
-
[29]
A review of the evidence underpin- ning the use of visual and auditory feedback for computer technology in post- stroke upper-limb rehabilitation,
J. Parker, G. Mountain, and J. Hammerton, “A review of the evidence underpin- ning the use of visual and auditory feedback for computer technology in post- stroke upper-limb rehabilitation,”Disability and rehabilitation: Assistive technol- ogy, vol. 6, no. 6, pp. 465–472, 2011
2011
-
[30]
Finerehab: A multi- modality and multi-task dataset for rehabilitation analysis,
J. Li, J. Xue, R. Cao, X. Du, S. Mo, K. Ran, and Z. Zhang, “Finerehab: A multi- modality and multi-task dataset for rehabilitation analysis,” inProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2024, pp. 3184–3193
2024
-
[31]
Intellirehabds (irds)—a dataset of physical rehabilitation movements,
A. Miron, N. Sadawi, W. Ismail, H. Hussain, and C. Grosan, “Intellirehabds (irds)—a dataset of physical rehabilitation movements,”Data, vol. 6, no. 5, 2021. [Online]. Available: https://www.mdpi.com/2306-5729/6/5/46
2021
-
[32]
Prompt engineering paradigms for medical applications: Scoping review,
J. Zaghir, M. Naguib, M. Bjelogrlic, A. Névéol, X. Tannier, and C. Lovis, “Prompt engineering paradigms for medical applications: Scoping review,”Journal of Med- ical Internet Research, vol. 26, p. e60501, 2024
2024
-
[33]
Language models are few-shot learners,
T. Brown, B. Mann, N. Ryder, M. Subbiah, J. D. Kaplan, P. Dhariwal, A. Nee- lakantan, P. Shyam, G. Sastry, A. Askellet al., “Language models are few-shot learners,” Advances in neural information processing systems, vol. 33, pp. 1877– 1901, 2020
1901
-
[34]
Llama: Open and efficient foundation language models,
H. Touvron, T. Lavril, G. Izacard, X. Martinet, M.-A. Lachaux, T. Lacroix, B. Roz- ière, N. Goyal, E. Hambro, F. Azharet al., “Llama: Open and efficient foundation language models,” arXiv preprint arXiv:2302.13971, 2023
2023 arXiv
-
[35]
Can llms express their uncertainty? an empirical evaluation of confidence elicitation in llms,
M. Xiong, Z. Hu, X. Lu, Y. Li, J. Fu, J. He, and B. Hooi, “Can llms express their uncertainty? an empirical evaluation of confidence elicitation in llms,”arXiv preprint arXiv:2306.13063, 2023
2023 arXiv
-
[36]
Cycles of thought: Measuring llm confidence through stable explanations,
E. Becker and S. Soatto, “Cycles of thought: Measuring llm confidence through stable explanations,” arXiv preprint arXiv:2406.03441, 2024
2024 arXiv
-
[37]
Chain-of-thought prompting elicits reasoning in large language models,
J. Wei, X. Wang, D. Schuurmans, M. Bosma, F. Xia, E. Chi, Q. V. Le, D. Zhou et al., “Chain-of-thought prompting elicits reasoning in large language models,” Advances in neural information processing systems,vol.35,pp.24824–24837,2022
2022
-
[38]
Probabilistic medical predictions of large language models,
B. Gu, R. J. Desai, K. J. Lin, and J. Yang, “Probabilistic medical predictions of large language models,”npj Digital Medicine, vol. 7, no. 1, p. 367, 2024
2024
-
[39]
Role play with large language mod- els,
M. Shanahan, K. McDonell, and L. Reynolds, “Role play with large language mod- els,” Nature, vol. 623, no. 7987, pp. 493–498, 2023
2023
-
[40]
Gpt-4o announcement,
OpenAI, “Gpt-4o announcement,” 2024, accessed: 2025-02-17. [Online]. Available: https://openai.com/index/hello-gpt-4o
2024
-
[41]
Think twice be- fore assure: Confidence estimation for large language models through reflection on multiple answers,
M. Li, W. Wang, F. Feng, F. Zhu, Q. Wang, and T.-S. Chua, “Think twice be- fore assure: Confidence estimation for large language models through reflection on multiple answers,”arXiv preprint arXiv:2403.09972, 2024
2024 arXiv
Reviewed August 7, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.