{"id":"85e97eb6-58cd-4fae-bd42-8e4c326d6a29","arxiv_id":"2509.03522","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":4,"one_line_summary":"Cluster averages of historical surgery durations match or beat much more complex ML models and outperform manual plans on a small single-hospital dataset.","lead":"This paper shows that averaging past durations of similar surgeries predicts operating room times about as well as complex machine learning models, even on one year of hospital data. It suggests hospitals can improve scheduling accuracy with small, expert-guided datasets instead of large multicentric ones.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Pre-split outlier removal and clustering leak test information; the headline improvement over manual planning may be inflated.","rationale":"I read the paper in good faith. The internal comparison is coherent and the general finding that simple models perform comparably to more complex ones is plausible. However, the text in Section 5.1 explicitly describes preprocessing steps—IQR outlier removal and clustering—applied before the 80/20 split. Outlier removal uses duration labels from the full dataset, which is a direct form of leakage: the test set is truncated by removing extreme cases, making any model's error appear smaller. Clustering before the split is a softer leakage but still allows test descriptions to influence the feature representation. The reader's verdict correctly identified the clustering issue but selected cross-study comparability as the weakest assumption. I believe the preprocessing leakage is more load-bearing because it threatens the internal result that cluster-mean beats manual planning, which is the core practical message. If a clean rerun with train-only preprocessing preserves the improvement, the paper's central claim holds and the conditional verdict can be maintained; if not, the claim is overstated. I therefore propose a single concrete reanalysis that would settle this. I do not see a need to change the reader's conditional verdict, but the specific condition should be made explicit.","tokens_in":10677,"tokens_out":7048,"duration_ms":76158,"concrete_test":"Rerun the full pipeline with a strict train-only preprocessing discipline: (1) compute IQR thresholds using only the 80% training partition and exclude outliers only from training; (2) fit TF-IDF, cluster models, and silhouette-based cluster count on training only; (3) apply the fitted representations/encodings to the complete original test partition without removing any test outliers; (4) compare cluster-mean baseline vs manual plan MAE/percentage deviation. If the improvement drops materially (e.g., from 18.32 pp to below 10 pp, or the 95% CI includes zero), the paper's central claim is overstated and the verdict should remain conditional pending a clean reanalysis.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central practical claim—that a cluster-mean baseline using a few well-prepared features yields large error reductions (e.g., 18.32 percentage points for the surgical phase in Section 6)—rests on an evaluation protocol that applies target-dependent preprocessing before the train/test split. Section 5.1 removes records using the 1.5xIQR rule over the full dataset, so test-set extremes are excluded before any model is trained; the model is then evaluated on a truncated test distribution. Section 5.1 also fits TF-IDF/K-Means/GMM clustering and selects the cluster count using the silhouette coefficient on the full dataset before the 80/20 split, so test procedure descriptions shape the clusters used to define the mean-based plan. Only target encoding is explicitly restricted to the training set. This is not merely a cross-study comparability problem: it threatens the internal comparison against manual plans, which is the paper's most useful finding. If IQR thresholds and clustering are instead fit only on training and the full original hold-out (including outliers) is scored, the 18.32 pp improvement may shrink or disappear. The cross-study 'outperform previous researchers' claim in Sections 6 and 9 is also unsupported, but it is secondary; the leakage issue is more load-bearing because it undermines the main practical recommendation regardless of external benchmarks.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper studies duration prediction for perioperative sub-processes (induction, preparation, surgical procedure) at LMU University Hospital using one year of clinical workflow data. The authors combine qualitative expert interviews with quantitative analysis. They cluster free-text procedure and induction descriptions using TF-IDF with K-Means/GMM, then compare simple mean-based plans, linear regression, random forest, and GBM against manual planning. The central claims are that simple methods of central tendency perform on par with or better than complex models from prior literature, that combining expert knowledge with a small dataset yields large improvements over manual planning (e.g., 18.32 percentage points for the surgical phase), and that the authors 'yield better results than previous researchers.'","tokens_in":10953,"tokens_out":2402,"duration_ms":30108,"significance":"If the main claims held, the paper would provide a valuable and practical result: a small, local dataset plus a few well-chosen features and a cluster-mean baseline could substantially improve operating-room scheduling accuracy, with less data and less model complexity than commonly assumed. The qualitative-expert integration and the focus on the whole perioperative process are also worthwhile contributions. However, the current evaluation protocol contains a load-bearing data-leakage issue in preprocessing, and the cross-study superiority claim is not supported by the evidence presented. The practical recommendation could survive after re-analysis, but the manuscript in its present form does not establish it.","major_comments":[{"comment":"The evaluation protocol leaks test information into the preprocessing pipeline. Section 5.1 applies the 1.5×IQR exclusion rule to the full dataset before the 80/20 split, so test-set extremes are removed before any model is evaluated. It also fits TF-IDF vectorization, K-Means/GMM clustering, and selects the number of clusters using the silhouette coefficient on the full dataset before the split. This means the cluster structure and outlier thresholds used for the mean-based plan are informed by test-set texts and test-set duration distributions. The reported improvement of 18.32 percentage points for the surgical phase in §6 is therefore computed on a truncated, cluster-informed test distribution. The fix is to split the data first, fit all preprocessing (IQR thresholds, vectorizer, cluster centers, cluster-count selection) on the training portion only, and score the full original hold-","section":"§5.1 and §6"},{"comment":"The claim that the authors 'yield better results than previous researchers' is unsupported. No previous model is run on the same dataset; the comparison is made by reading off MAE/percentage deviations from other studies with different hospitals, case mixes, outcome definitions, and preprocessing pipelines. Section 8.2 itself notes that LMU is a major university hospital with an unusual case mix, so cross-study metric comparison is not valid for establishing superiority. The authors should either implement and evaluate benchmark models from prior work on their own data or substantially weaken the cross-study claim.","section":"§6 and §9"},{"comment":"The internal comparison between manual planning, the mean-based plan, GBM, and random forest is presented via boxplots without error bars, confidence intervals, or significance tests. The claim that simple methods perform 'on a par' with complex methods cannot be assessed from the figure. Moreover, the statement that 'the predictions generated by the GBM model showed no statistically significant differences of whether age was included' is given without reporting the test used or its results. Paired comparisons or confidence intervals on the error metrics are needed to support the central parsimony claim.","section":"Fig. 4 and §6"}],"minor_comments":[{"comment":"The number of excluded outliers is inconsistent: §8.1 says 'we excluded 812 outliers,' but §5.1 describes a 1.5×IQR exclusion without reporting the count. Clarify whether the 812 are the IQR-based exclusions or only the implausible-timestamp exclusions.","section":"§5.1 and §8.1"},{"comment":"Figure 3 labels the preparation subset as '7%' while §8.1 states 6.97%; the caption and text should be consistent.","section":"Fig. 3 and §8.1"},{"comment":"The '3-minute interval' in the caption of Fig. 5 is unclear; presumably it refers to binning of planned durations, but it should be explained.","section":"§6"},{"comment":"The grid search details are not reported (hyperparameter ranges, final values, number of folds). This makes the comparison between simple and complex models hard to reproduce.","section":"§5.2 and §6"},{"comment":"The claim that patient characteristics have little impact is supported only by a sentence; the paper would benefit from a table of the statistical test results (t-test/ANOVA/Kruskal-Wallis) with effect sizes.","section":"§7"},{"comment":"The dataset and code are not mentioned as available; providing an anonymized version and a reproducible pipeline would strengthen the paper substantially.","section":"General"}],"recommendation":"major_revision","confidential_remarks":"The leakage issue described in the major comments is load-bearing: the paper's most defensible finding, the improvement of a simple cluster-mean plan over manual planning, is currently evaluated on a test set that has been filtered and clustered with full-data preprocessing. This is fixable within the manuscript's scope by re-running the pipeline with a pre-registered train/test split before any preprocessing. If the re-analysis confirms the improvement, the paper could be a solid practical contribution; if the improvement shrinks to insignificance, the central claim would fail. The cross-study superiority claim should also be removed or replaced with a same-data benchmark comparison."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Let me give you my read on arXiv:2509.03522. The real news here, if it holds, is narrow but useful: on one year of data from a large university hospital, a cluster-mean baseline built from free-text procedure descriptions performs about as well as random forest or gradient boosting, and both beat manual planning. That is a legitimate practical finding for operating room scheduling, and the qualitative work (expert-informed semantic clustering, hypothesis-driven feature selection) is a genuine strength. The authors also do the right thing by breaking the perioperative process into phases rather than treating surgery as a single black box.\n\nThe soft spots are real and load-bearing. Section 5.1 does the 1.5×IQR outlier exclusion on the full dataset before the 80/20 split, and K-Means/GMM clustering with silhouette-based cluster count selection also runs on the full dataset. Only target encoding is explicitly restricted to training. That means test-set extremes are removed before the model is scored, and test procedure descriptions shape the clusters used to generate the mean-based plan. The 18.32 percentage point improvement over manual planning for the surgical phase is therefore likely inflated. It would not surprise me if the qualitative conclusion survives a clean split, but the reported magnitude is not credible as-is.\n\nThe other major weakness is the claim of outperforming previous researchers. That comparison is made across different hospitals, case mixes, and evaluation metrics, with no prior model re-run on this dataset. The paper itself acknowledges the unusual case mix of a university hospital in Section 8.2, which undermines the cross-study superiority claim. I would drop that claim unless the authors benchmark against a published method on their own data.\n\nSmaller issues: no confidence intervals on the headline improvements, no code or data release, and the free-text clustering procedure is described with enough detail to reproduce but not enough to verify the exact pipeline (e.g., how cluster means are computed relative to the split). The authors do state limitations around small cluster sizes and missing context variables in Section 6, which I appreciate.\n\nWho gets value from this: applied health informatics researchers and OR scheduling practitioners, especially those with limited data. It is not a methodological breakthrough, but the empirical question it raises is worth answering cleanly.\n\nMy recommendation: engage with it, but treat it as a preprint requiring a major revision. A simple re-run with clustering and outlier removal done only on training, and a full hold-out scored including outliers, would tell us whether the central finding holds. I would send it to peer review with that expectation.","headline":"The paper's practical claim is plausible but its headline numbers are not trustworthy because clustering and outlier removal happen before the train/test split, so test information leaks into the evaluation.","tokens_in":11427,"tokens_out":1592,"would_cite":false,"duration_ms":20475,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"This paper claims that simple cluster-average durations predict surgical process times as accurately as complex machine-learning models, and sometimes better, using only a year of data from one hospital.","keywords":["surgery duration prediction","perioperative process","small dataset","semantic clustering","central tendency baseline","expert knowledge","operating room scheduling","process mining"],"falsifier":"Take the same one-year dataset and same train/test split, and run a published complex model—for example, gradient boosting with all the paper's features—then compare test MAE with the cluster-mean baseline. If the complex model is significantly more accurate on this shared data, the central claim of parity would be refuted.","tokens_in":10542,"feed_emoji":"🏥","tokens_out":8235,"duration_ms":83716,"temperature":0.7,"pith_summary":"The paper tries to establish that accurate prediction of how long surgeries and their surrounding perioperative steps take does not require huge data volumes or elaborate models. Using one year of records from a single large university hospital, the authors show that a baseline predicting the historical mean duration of semantically similar procedure clusters performs on par with—and sometimes better than—more complex machine-learning models previously reported in the literature. The reason, they argue, is that expert interviews guided the cleaning and normalization of messy free-text fields and the choice of a few relevant features, so the small dataset is used efficiently. If the result holds, hospitals without multi-year data warehouses could build reliable scheduling tools quickly, and patient-privacy-sensitive personal data could be omitted from models. The strongest comparative claim—that they beat prior researchers—depends on comparing error numbers across different studies rather than head-to-head on the same data.","feed_headline":"Simple averages rival complex machine learning for surgery duration","feed_subtitle":"For hospitals, accurate schedules may need only one year of local data plus expert-guided cleanup.","key_machinery":"The load-bearing mechanism is the semantic cluster. More than 11,000 distinct free-text operation descriptions are cleaned and standardized with clinical expertise, converted into numeric vectors with a standard text-weighting scheme, and grouped by a common clustering algorithm so that differently worded descriptions of the same intervention land in the same cluster. Each cluster yields a historical mean duration, and that cluster mean is the entire predictor. The paper's experiments show that this single number matches or beats random forests, gradient boosting, and linear regression on the same features, and that adding demographic variables does not improve it. The same cluster structure","core_discovery":"The paper's central claim is that effective surgical duration prediction does not need large datasets or complex models. On one year of records from a major university hospital, a baseline that groups free-text procedure descriptions into semantically similar clusters and predicts the historical mean duration of the cluster matched or beat published results from random forests, gradient boosting, and neural approaches, while cutting mean percentage deviation from the plan by 18.32 percentage points for the surgical phase and 7.94 for induction. The authors argue this works because expert interviews guided data cleaning, text normalization, and feature selection, so the small dataset's inform","pith_inferences":["If the recipe is 'expert-guided text normalization plus central tendency,' the same pattern may hold in other domains with free-text case labels and small local data—for example, repair work orders or legal case durations—where cluster means would be a cheap first model to test.","The headline comparison with previous studies is indirect; a direct head-to-head on one shared dataset could change the ranking. A fair test would report both MAE and scheduling cost (overrun versus underrun) for each model.","Since preparation-phase timestamps exist for only about 7% of workflows, the paper effectively validates the method on induction and surgery, not the whole perioperative process; hospitals with complete timestamping could test whether the same simple baseline holds for preparation and recovery."],"forward_implications":["A hospital can replace manual duration estimates with automatic cluster means and expect substantially tighter schedules: the paper reports mean deviation from plan dropping by 18.32 percentage points for the surgical phase and 7.94 for induction.","Large multi-year training datasets may be unnecessary; one year of local records appears sufficient for a useful predictor, which matters for smaller hospitals.","Complex models such as random forest and gradient boosting add tuning effort without a clear accuracy payoff over the cluster mean, so data preparation and feature design deserve priority.","Because patient age and sex have negligible practical effect, duration prediction can be built without personal data, easing privacy and consent concerns.","The semantically clustered descriptions could become the basis of a standardized procedure catalog, improving consistency across departments and information systems."],"supporting_citations":[{"why":"Supplies the machine-learning procedure-duration baseline and the text-preprocessing approach for procedure names that this work adapts.","marker":"[16]"},{"why":"Provides a comparable machine-learning surgical-time prediction study and encoding techniques this paper tests against.","marker":"[18]"},{"why":"Supplies the smoothed target-encoding scheme used to encode procedure clusters without leaking test-set information.","marker":"[19]"},{"why":"Defines the random forest model used as a complex comparison method.","marker":"[3]"},{"why":"Defines the gradient boosting machine used as a complex comparison method.","marker":"[11]"},{"why":"Supplies the clustering algorithm used to group free-text procedure descriptions.","marker":"[10]"},{"why":"Supplies the inverse-document-frequency text weighting used to vectorize procedure and anesthesia descriptions.","marker":"[22]"},{"why":"Shows the precedent for combining clinical staff input with predictive modeling to improve scheduling outcomes.","marker":"[28]"}],"fun_headline_variants":["Small data, simple math outsmart complex models for surgery times","Expert-guided averages beat deep learning for operation duration","One year of local data beats AI models for surgery duration","Simple cluster means beat complex models on small clinical data"],"cache_read_input_tokens":2688,"weakest_assumption_plain":"The claim that the model 'yields better results than previous researchers' assumes that published error figures from other hospitals, with different case mixes and preprocessing choices, can be directly compared with this study's MAE and percentage deviations.","fun_headline_variants_meta":{"raw":{"variants":["Small data, simple math outsmart complex models for surgery times","Expert-guided averages beat deep learning for operation duration","One year of local data beats AI models for surgery duration","Simple cluster means beat complex models on small clinical data"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.001227,"raw_usage":{"total_tokens":4874,"prompt_tokens":733,"completion_tokens":4141,"prompt_tokens_details":{"cached_tokens":256},"prompt_cache_hit_tokens":256,"prompt_cache_miss_tokens":477,"completion_tokens_details":{"reasoning_tokens":4076}},"tokens_in":477,"tokens_out":4141,"duration_ms":28411,"temperature":1.0,"reasoning_tokens":4076,"cache_read_input_tokens":256,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-05T18:42:29.049513+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Take the same one-year dataset and same train/test split, and run a published complex model—for example, gradient boosting with all the paper's features—then compare test MAE with the cluster-mean baseline. If the complex model is significantly more accurate on this shared data, the central claim of parity would be refuted.","supporting_citations":[{"cited_title":"JMIR AI2, e44909 (2023)","cited_arxiv_id":null,"evidence_quote":"Supplies the machine-learning procedure-duration baseline and the text-preprocessing approach for procedure names that this work adapts."},{"cited_title":"Comput Methods Programs Biomed208, 106220 (2021)","cited_arxiv_id":null,"evidence_quote":"Provides a comparable machine-learning surgical-time prediction study and encoding techniques this paper tests against."},{"cited_title":"SIGKDD Explor","cited_arxiv_id":null,"evidence_quote":"Supplies the smoothed target-encoding scheme used to encode procedure clusters without leaking test-set information."},{"cited_title":"Journal of the Royal Statistical Society","cited_arxiv_id":null,"evidence_quote":"Supplies the clustering algorithm used to group free-text procedure descriptions."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Supplies the inverse-document-frequency text weighting used to vectorize procedure and anesthesia descriptions."},{"cited_title":"JAMA Surg.156(4), 315–321 (2021).https: //doi.org/10.1001/jamasurg.2020.6361","cited_arxiv_id":null,"evidence_quote":"Shows the precedent for combining clinical staff input with predictive modeling to improve scheduling outcomes."}],"review_version":1}