REVIEW 3 major objections 6 minor 32 references
A Small Dataset May Go a Long Way: Process Duration Prediction in Clinical Settings
T0 review · 3 major / 6 minor · reviewed 2026-08-05 · deepseek-v4-flash
Pith's one-line read This paper claims that simple cluster-average durations predict surgical process times as accurately as complex machine-learning models, and sometimes better, using only a year of data from one hospital.
desk verdict The paper's practical claim is plausible but its headline numbers are not trustworthy because clustering and outlier removal happen before the train/test split, so test information leaks into the evaluation. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The load-bearing mechanism is the semantic cluster. More than 11,000 distinct free-text operation descriptions are cleaned and standardized with clinical expertise, converted into numeric vectors with a standard text-weighting scheme, and grouped by a common clustering algorithm so that differently worded descriptions of the same intervention land in the same cluster. Each cluster yields a historical mean duration, and that cluster mean is the entire predictor. The paper's experiments show that this single number matches or beats random forests, gradient boosting, and linear regression on the same features, and that adding demographic variables does not improve it. The same cluster structure
What would settle it
Take the same one-year dataset and same train/test split, and run a published complex model—for example, gradient boosting with all the paper's features—then compare test MAE with the cluster-mean baseline. If the complex model is significantly more accurate on this shared data, the central claim of parity would be refuted.
Extended reading notes
Core claim
The paper's central claim is that effective surgical duration prediction does not need large datasets or complex models. On one year of records from a major university hospital, a baseline that groups free-text procedure descriptions into semantically similar clusters and predicts the historical mean duration of the cluster matched or beat published results from random forests, gradient boosting, and neural approaches, while cutting mean percentage deviation from the plan by 18.32 percentage points for the surgical phase and 7.94 for induction. The authors argue this works because expert interviews guided data cleaning, text normalization, and feature selection, so the small dataset's inform
Load-bearing premise
The claim that the model 'yields better results than previous researchers' assumes that published error figures from other hospitals, with different case mixes and preprocessing choices, can be directly compared with this study's MAE and percentage deviations.
Editorial extensions
If this is right
- A hospital can replace manual duration estimates with automatic cluster means and expect substantially tighter schedules: the paper reports mean deviation from plan dropping by 18.32 percentage points for the surgical phase and 7.94 for induction.
- Large multi-year training datasets may be unnecessary; one year of local records appears sufficient for a useful predictor, which matters for smaller hospitals.
- Complex models such as random forest and gradient boosting add tuning effort without a clear accuracy payoff over the cluster mean, so data preparation and feature design deserve priority.
- Because patient age and sex have negligible practical effect, duration prediction can be built without personal data, easing privacy and consent concerns.
- The semantically clustered descriptions could become the basis of a standardized procedure catalog, improving consistency across departments and information systems.
Reading between the lines
- If the recipe is 'expert-guided text normalization plus central tendency,' the same pattern may hold in other domains with free-text case labels and small local data—for example, repair work orders or legal case durations—where cluster means would be a cheap first model to test.
- The headline comparison with previous studies is indirect; a direct head-to-head on one shared dataset could change the ranking. A fair test would report both MAE and scheduling cost (overrun versus underrun) for each model.
- Since preparation-phase timestamps exist for only about 7% of workflows, the paper effectively validates the method on induction and surgery, not the whole perioperative process; hospitals with complete timestamping could test whether the same simple baseline holds for preparation and recovery.
Signed reviews
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper studies duration prediction for perioperative sub-processes (induction, preparation, surgical procedure) at LMU University Hospital using one year of clinical workflow data. The authors combine qualitative expert interviews with quantitative analysis. They cluster free-text procedure and induction descriptions using TF-IDF with K-Means/GMM, then compare simple mean-based plans, linear regression, random forest, and GBM against manual planning. The central claims are that simple methods of central tendency perform on par with or better than complex models from prior literature, that combining expert knowledge with a small dataset yields large improvements over manual planning (e.g., 18.32 percentage points for the surgical phase), and that the authors 'yield better results than previous researchers.'
Significance. If the main claims held, the paper would provide a valuable and practical result: a small, local dataset plus a few well-chosen features and a cluster-mean baseline could substantially improve operating-room scheduling accuracy, with less data and less model complexity than commonly assumed. The qualitative-expert integration and the focus on the whole perioperative process are also worthwhile contributions. However, the current evaluation protocol contains a load-bearing data-leakage issue in preprocessing, and the cross-study superiority claim is not supported by the evidence presented. The practical recommendation could survive after re-analysis, but the manuscript in its present form does not establish it.
major comments (3)
- [§5.1 and §6] The evaluation protocol leaks test information into the preprocessing pipeline. Section 5.1 applies the 1.5×IQR exclusion rule to the full dataset before the 80/20 split, so test-set extremes are removed before any model is evaluated. It also fits TF-IDF vectorization, K-Means/GMM clustering, and selects the number of clusters using the silhouette coefficient on the full dataset before the split. This means the cluster structure and outlier thresholds used for the mean-based plan are informed by test-set texts and test-set duration distributions. The reported improvement of 18.32 percentage points for the surgical phase in §6 is therefore computed on a truncated, cluster-informed test distribution. The fix is to split the data first, fit all preprocessing (IQR thresholds, vectorizer, cluster centers, cluster-count selection) on the training portion only, and score the full original hold-
- [§6 and §9] The claim that the authors 'yield better results than previous researchers' is unsupported. No previous model is run on the same dataset; the comparison is made by reading off MAE/percentage deviations from other studies with different hospitals, case mixes, outcome definitions, and preprocessing pipelines. Section 8.2 itself notes that LMU is a major university hospital with an unusual case mix, so cross-study metric comparison is not valid for establishing superiority. The authors should either implement and evaluate benchmark models from prior work on their own data or substantially weaken the cross-study claim.
- [Fig. 4 and §6] The internal comparison between manual planning, the mean-based plan, GBM, and random forest is presented via boxplots without error bars, confidence intervals, or significance tests. The claim that simple methods perform 'on a par' with complex methods cannot be assessed from the figure. Moreover, the statement that 'the predictions generated by the GBM model showed no statistically significant differences of whether age was included' is given without reporting the test used or its results. Paired comparisons or confidence intervals on the error metrics are needed to support the central parsimony claim.
minor comments (6)
- [§5.1 and §8.1] The number of excluded outliers is inconsistent: §8.1 says 'we excluded 812 outliers,' but §5.1 describes a 1.5×IQR exclusion without reporting the count. Clarify whether the 812 are the IQR-based exclusions or only the implausible-timestamp exclusions.
- [Fig. 3 and §8.1] Figure 3 labels the preparation subset as '7%' while §8.1 states 6.97%; the caption and text should be consistent.
- [§6] The '3-minute interval' in the caption of Fig. 5 is unclear; presumably it refers to binning of planned durations, but it should be explained.
- [§5.2 and §6] The grid search details are not reported (hyperparameter ranges, final values, number of folds). This makes the comparison between simple and complex models hard to reproduce.
- [§7] The claim that patient characteristics have little impact is supported only by a sentence; the paper would benefit from a table of the statistical test results (t-test/ANOVA/Kruskal-Wallis) with effect sizes.
- [General] The dataset and code are not mentioned as available; providing an anonymized version and a reproducible pipeline would strengthen the paper substantially.
Circularity Check
Clustering and IQR filtering are applied to the full dataset before the train/test split, so the test-set text informs the very cluster means used for 'prediction,' making the headline error reductions partly in-sample.
-
fitted input called prediction
[Section 5.1 (Data Preparation), before the 80/20 split; results reported in Section 6 (Observations and Findings)]
"The text data was transformed into numerical representations using TF-IDF vectorization. To cluster descriptions, we applied K-Means for procedures and Gaussian Mixture Models (GMM) for induction data. The number of clusters was determined with the mean Silhouette Coefficient. The dataset was split into training (80%) and test (20%) sets."
The clustering is completed before the split, so the cluster model—including the silhouette-selected number of clusters and the cluster centroids—is fitted on the full dataset, test records included. Section 6 then evaluates the 'Mean-based plan (procedure clusters only)' on the test set: each test case is assigned to a cluster whose structure was already influenced by that same test case's free-text description, and the cluster mean is used as the prediction. This is not a purely training-derived forecast; it is partly an in-sample fit. Additionally, the 1.5×IQR outlier removal is applied to the full data before the split, excluding test-set extremes and truncating the distribution on which the reported 7.94 and 18.32 percentage-point improvements are computed.
full rationale
The core derivation—compute a cluster-conditional arithmetic mean on training data and use it to predict test durations—is standard supervised learning and is not circular by itself. The circularity is in the evaluation protocol: the paper fits the clustering and the outlier filter on the entire dataset before the 80/20 split, so the test set leaks into the feature grouping and into the distribution used for error measurement. This inflates the headline claim of large improvements over manual planning. However, the internal comparison between simple mean and more complex models (GBM, Random Forest) is less affected because all models share the same leaky clusters, and the paper contains no load-bearing self-citations. The cross-study claim of outperforming previous researchers is unsupported due to non-comparable settings, but that is a validity problem rather than a circularity. Overall, the central practical result is partially circular but not fully forced; score 4.
Assumptions & free parameters
free parameters (4)
- Number of procedure clusters (K-Means k) =
Not reported; chosen by mean Silhouette Coefficient
- Number of induction clusters (GMM k) =
15 after Sqior normalization; chosen by silhouette
- Target encoding smoothing parameter =
40
- IQR multiplier for outlier exclusion =
1.5
assumptions (4)
- domain assumption Expert interviews and the resulting causal model (Fig. 2) correctly identify the relevant predictive factors.
- domain assumption TF-IDF semantic clusters group procedures with similar true durations.
- ad hoc to paper Performance metrics from other studies are directly comparable to this study's MAE and percentage deviations.
- domain assumption Outliers removed by the IQR rule are data errors rather than genuine long or short procedures.
Cite this review
Pith. "Pith review of A Small Dataset May Go a Long Way: Process Duration Prediction in Clinical Settings." pith.science (2026). https://pith.science/paper/76M7VJP7
@misc{pith2026250903522,
author = {Pith},
title = {Pith review of: A Small Dataset May Go a Long Way: Process Duration Prediction in Clinical Settings},
year = {2026},
howpublished = {\url{https://pith.science/paper/76M7VJP7}},
note = {Machine review of arXiv:2509.03522}
}
read the original abstract
Context: Utilization of operating theaters is a major cost driver in hospitals. Optimizing this variable through optimized surgery schedules may significantly lower cost and simultaneously improve medical outcomes. Previous studies proposed various complex models to predict the duration of procedures, the key ingredient to optimal schedules. They did so perusing large amounts of data. Goals: We aspire to create an effective and efficient model to predict operation durations based on only a small amount of data. Ideally, our model is also simpler in structure, and thus easier to use. Methods: We immerse ourselves in the application domain to leverage practitioners expertise. This way, we make the best use of our limited supply of clinical data, and may conduct our data analysis in a theory-guided way. We do a combined factor analysis and develop regression models to predict the duration of the perioperative process. Findings: We found simple methods of central tendency to perform on a par with much more complex methods proposed in the literature. In fact, they sometimes outperform them. We conclude that combining expert knowledge with data analysis may improve both data quality and model performance, allowing for more accurate forecasts. Conclusion: We yield better results than previous researchers by integrating conventional data science methods with qualitative studies of clinical settings and process structure. Thus, we are able to leverage even small datasets.
Figures
Figures from the paper (2 more)
Reference graph
Works this paper leans on
-
[1]
Econometrica 70(1), 191–221 (2002).https://doi.org/https://doi.org/10
Bai, J., Ng, S.: Determining the number of factors in approximate factor mod- els. Econometrica 70(1), 191–221 (2002).https://doi.org/https://doi.org/10. 1111/1468-0262.00273, https://onlinelibrary.wiley.com/doi/abs/10.1111/ 1468-0262.00273
arXiv 2002
-
[2]
Grounded Theory Review08(2) (Jun 2009), Posted on Jun 30, 2009
Breckenridge, J., Jones, D.: Demystifying Theoretical Sampling in Grounded Theory Research. Grounded Theory Review08(2) (Jun 2009), Posted on Jun 30, 2009
work page 2009
-
[3]
Machine Learning45(1), 5–32 (2001).https://doi
Breiman, L.: Random forests. Machine Learning45(1), 5–32 (2001).https://doi. org/10.1023/A:1010933404324, https://link.springer.com/article/10.1023/ A:1010933404324
-
[4]
JAMA Surg.153(4), e176233 (2018).https://doi.org/10.1001/jamasurg
Childers, C., Maggard-Gibbons, M.: Understanding costs of care in the operating room. JAMA Surg.153(4), e176233 (2018).https://doi.org/10.1001/jamasurg. 2017.6233
arXiv 2018
-
[5]
Health Care Manage Sci10, 13–24 (2007)
Denton, B., Viapiano, J., Vogl, A.: Optimization of surgery sequencing and scheduling decisions under uncertainty. Health Care Manage Sci10, 13–24 (2007). https://doi.org/10.1007/s10729-006-9005-4
-
[6]
(eds.): The SAGE Handbook of Qualitative Research
Denzin, N.K., Lincoln, Y.S. (eds.): The SAGE Handbook of Qualitative Research. SAGE, 5th edn. (2018)
work page 2018
-
[7]
In: Wiley Encyclopedia of Operations Research and Management Science
Erdogan, S., Denton, B., Cochran, J.: Surgery planning and scheduling. In: Wiley Encyclopedia of Operations Research and Management Science. John Wiley & Sons, Ltd (2011). https://doi.org/10.1002/9780470400531.eorms0861
-
[8]
JMIR Perioper Med6, e39650 (2023)
Gabriel, R., Harjai, B., Simpson, S., Du, A., Tully, J., George, O., Waterman, R.: An ensemble learning approach to improving prediction of case duration for spine surgery: algorithm development and validation. JMIR Perioper Med6, e39650 (2023). https://doi.org/10.2196/39650
Show all 32 references
-
[9]
Sociology Press (1967)
Glaser, B., Strauss, A.: Discovery of Grounded Theory. Sociology Press (1967)
1967
-
[10]
Journal of the Royal Statistical Society
Hartigan, J.A., Wong, M.A.: Algorithm as 136: A k-means clustering algorithm. Journal of the Royal Statistical Society. Series C (Applied Statistics)28(1), 100–108 (1979)
1979
-
[11]
He, Z., Lin, D., Lau, T., Wu, M.: Gradient boosting machine: A survey (2019), https://arxiv.org/abs/1908.06951
2019 arXiv
-
[12]
In: AMIA Annual Symposium Proceedings
Hosseini, N., Sir, M., Jankowski, C., Pasupathy, K.: Surgical duration estimation via data mining and predictive modeling: A case study. In: AMIA Annual Symposium Proceedings. pp. 640–648. American Medical Informatics Association (2015)
2015
-
[13]
Oxford University Press (2022)
Hughes, S.J.: Oxford handbook of perioperative practice. Oxford University Press (2022)
2022
-
[14]
James, G., Witten, D., Hastie, T., Tibshirani, R., Taylor, J.: Linear Regression, pp. 69–134. Springer International Publishing, Cham (2023).https://doi.org/10. 1007/978-3-031-38747-0_3, https://doi.org/10.1007/978-3-031-38747-0_3
2023 doi
-
[15]
Br J Anaesth128(5), 829–837 (2022).https: //doi.org/10.1016/j.bja.2022.01.017
Jiao, Y., et al.: Continuous real-time prediction of surgical case duration using a modular artificial neural network. Br J Anaesth128(5), 829–837 (2022).https: //doi.org/10.1016/j.bja.2022.01.017
2022 doi
-
[16]
JMIR AI2, e44909 (2023)
Kendale, S., Bishara, A., Burns, M., Solomon, S., Corriere, M., Mathis, M.: Machine learning for the prediction of procedural case durations developed using a large multicenter database: algorithm development and validation study. JMIR AI2, e44909 (2023). https://doi.org/10.2196/44909
2023 doi
-
[17]
Addison-Wesley Professional, Hoboken, NJ, first edition edn
Levene, M., Harris, M.: Just Enough Data Science and Machine Learning: Essential Tools and Techniques. Addison-Wesley Professional, Hoboken, NJ, first edition edn. (2025), https://learning.oreilly.com/library/view/-/9780138340773/ ?ar, kapitel 2: Basic Statistics. Includes bib...
2025
-
[18]
Comput Methods Programs Biomed208, 106220 (2021)
Martinez, O., Martinez, C., Parra, C., Rugeles, S., Suarez, D.: Machine learning for surgical time prediction. Comput Methods Programs Biomed208, 106220 (2021). https://doi.org/10.1016/j.cmpb.2021.106220
2021
-
[19]
SIGKDD Explor
Micci-Barreca, D.: A preprocessing scheme for high-cardinality categorical at- tributes in classification and prediction problems. SIGKDD Explor. Newsl.3(1), 27–32 (Jul 2001).https://doi.org/10.1145/507533.507538, https://doi.org/ 10.1145/507533.507538
2001
-
[20]
Otolaryngol Head Neck Surg168, 241–247 (2023).https://doi
Miller, L., Goedicke, W., Crowson, M., Rathi, V., Naunheim, M., Agarwala, A.: Using machine learning to predict operating room case duration: a case study in otolaryngology. Otolaryngol Head Neck Surg168, 241–247 (2023).https://doi. org/10.1177/01945998221076480
2023 doi
-
[21]
Routledge, Abingdon, Oxon; New York, NY, fifth edition edn
Murphy, K., Myors, B.: Statistical Power Analysis. Routledge, Abingdon, Oxon; New York, NY, fifth edition edn. (2023), https://learning.oreilly.com/library/ view/-/9781000843255/?ar, revised edition of Statistical Power Analysis, 2014. Includes bibliographical references and i...
2023
-
[22]
Papineni, K.: Why inverse document frequency? In: Proceedings of the Second Meeting of the North American Chapter of the Association for Computational Linguistics. pp. 1–8 (2001),https://aclanthology.org/N01-1004
2001
-
[23]
J Med Syst49, 8 (2025)
Park, J., Roh, G., Kim, K., et al.: Development of predictive model of surgical case durations using machine learning approach. J Med Syst49, 8 (2025). https: //doi.org/10.1007/s10916-025-02141-y
2025 doi
-
[24]
BMC Health Serv Res23, 1343 (2023)
Riahi, V., Hassanzadeh, H., Khanna, S., et al.: Improving preoperative prediction of surgery duration. BMC Health Serv Res23, 1343 (2023). https://doi.org/10. 1186/s12913-023-10264-6
2023
-
[25]
Semin Pediatr Surg27(2), 79–85 (2018)
Rothstein, D., Raval, M.: Operating room efficiency. Semin Pediatr Surg27(2), 79–85 (2018). https://doi.org/10.1053/j.sempedsurg.2018.02.004
2018 doi
-
[26]
Oxford University Press (2011)
Saldaña, J.: Fundamentals of Qualitative Research. Oxford University Press (2011)
2011
-
[27]
J Med Syst44, 42 (2020)
Soh, K., Walker, C., O’Sullivan, M., et al.: An evaluation of the hybrid model for predicting surgery duration. J Med Syst44, 42 (2020). https://doi.org/10. 1007/s10916-019-1501-4
2020
-
[28]
JAMA Surg.156(4), 315–321 (2021).https: //doi.org/10.1001/jamasurg.2020.6361
Strömblad, C., Baxter-King, R., Meisami, A., et al.: Effect of a predictive model on planned surgical duration accuracy, patient wait time, and use of presurgical resources: A randomized clinical trial. JAMA Surg.156(4), 315–321 (2021).https: //doi.org/10.1001/jamasurg.2020.6361
2021
-
[29]
verywellhealth.com/perioperative-defined-3157137, accessed: 2 January 2025
Whitlock, J.: What does perioperative mean? (2025), https://www. verywellhealth.com/perioperative-defined-3157137, accessed: 2 January 2025
2025
-
[30]
Informatics Med Unlocked 25, 100633 (2021).https://doi.org/10.1016/j.imu.2021.100633
Yuniartha, D., Masruroh, N., Herliansyah, M.: An evaluation of a simple model for predicting surgery duration using a set of surgical procedure parameters. Informatics Med Unlocked 25, 100633 (2021).https://doi.org/10.1016/j.imu.2021.100633
2021
-
[31]
ACM Trans
Zhang, Y., Li, M., Wang, S., Dai, S., Luo, L., Zhu, E., Xu, H., Zhu, X., Yao, C., Zhou, H.: Gaussian mixture model clustering with incomplete data. ACM Trans. Multimedia Comput. Commun. Appl.17(1s) (Mar 2021). https://doi.org/10. 1145/3408318, https://doi.org/10.1145/3408318
2021 doi
-
[32]
J Comb Optim 37, 757–805 (2019)
Zhu, S., Fan, W., Yang, S., et al.: Operating room planning and surgical case scheduling: a review of literature. J Comb Optim 37, 757–805 (2019). https: //doi.org/10.1007/s10878-018-0322-6
2019 doi
Reviewed August 5, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.