REVIEW 4 major objections 6 minor 35 references
Universal Laboratory Model: prognosis of abnormal clinical outcomes based on routine tests
T0 review · 4 major / 6 minor · reviewed 2026-08-15 · deepseek-v4-flash
Pith's one-line read The paper claims that a set-transformer with fixed GPT-style embeddings, called the Universal Laboratory Model, predicts low ferritin, high uric acid, glucose, and cholesterol levels from routine CBC and biochemistry tests, improving AUC…
desk verdict A clever set-transformer for missing lab values, but the evaluation has a patient-leakage risk and too few baselines to believe the AUC gains. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The central object is the Universal Laboratory Model (ULM), a set-transformer encoder-decoder that is permutation-invariant by construction. It encodes each available test as a pair of the test's fixed GPT embedding and its numerical value, applies multi-head attention in the encoder, and then lets the decoder attend to the encoded set using embeddings of the target tests as queries. Because the model operates on sets, reports of different sizes and compositions are accepted without imputation, and the GPT embeddings provide semantic context for what each test measures.
What would settle it
Re-run the same 80/20 split but group all rows belonging to the same patient into either the training or the test set (using whatever patient identifier exists in the source laboratory information system) and check whether the AUC advantage of ULM over the MLP survives; if the gap collapses to noise, the reported improvement is an artifact of patient leakage.
Extended reading notes
Core claim
In its own terms, the paper demonstrates that predicting abnormal laboratory outcomes reduces to a set-translation problem: the input is a set of (test embedding, value) pairs, and the output is a set of test embeddings for the predicted tests. The model, ULM, uses a multi-head attention encoder and decoder, where the decoder queries are the embeddings of the target tests, and the GPT-derived embeddings of test names are kept fixed and transformed only by scalar multiplication and a trainable bias. On held-out rows, the model achieves AUCs of 82.4% for glucose ≥7 mmol/L, 77.9% for cholesterol ≥5.2 mmol/L, 89.7% for ferritin ≤12 ng/mL, and 79.7% for uric acid at sex-specific thresholds, gains of 0.8 to 8 percentage points over a comparable MLP baseline.
Load-bearing premise
The evaluation randomly splits rows of the database into training and testing sets and treats each row as independent, so the same patient's repeated laboratory reports can appear in both parts of the split.
Editorial extensions
If this is right
- If the central claim holds, a laboratory information system could automatically add a test predicted to be abnormal when biological material from the same patient is still available, as the paper proposes.
- The architecture can be extended to other tests or demographic features without retraining on a fixed feature matrix.
- The model's permutation invariance makes it applicable to any subset of the 31 available measurements, so the same trained ULM serves patients with different test panels.
- The public deployment at https://ulm.roslis.ru would let any laboratory integrate the prognostic service into its workflow.
Reading between the lines
- We infer that the reported 8-point AUC advantage for uric acid is the strongest evidence that the set-transformer benefits from cross-test patterns, so a patient-level or temporal split would test whether that gain survives when repeated visits from the same patient are kept together.
- We infer that the design choice to freeze GPT embeddings and avoid matrix multiplications on them, rather than learning column embeddings as in earlier tabular transformers, is the key ablation to try; if learned embeddings perform equally, the improvement would be attributed more to architecture than to semantic prior.
- We infer that the model's current dichotomized outcomes could be extended to continuous value prediction or to percentiles, which would make the warnings more informative for clinicians.
Signed reviews
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper proposes the Universal Laboratory Model (ULM), a set-transformer with fixed GPT-based embeddings for laboratory test names and values, designed to predict binary abnormalities of four target tests (glucose, cholesterol, ferritin, uric acid) from a variable-size set of available routine laboratory results. The authors evaluate ULM on a large commercial laboratory dataset (1,173,403 rows) and report AUC improvements of up to 8 percentage points over a three-layer MLP, with the largest gain for uric acid. The claimed contribution is a tabular-modeling formulation as set translation that handles missing values without explicit imputation.
Significance. If the reported results survive proper validation, the work addresses a practically important problem: predicting non-prescribed abnormal laboratory tests from routine CBC and biochemistry panels, which could enable earlier clinical warnings. The dataset is substantially larger than those used in prior studies, and the authors provide a public model and code. However, the current evidence is insufficient to support the broad claims of effectiveness: the evaluation uses only a single weak baseline (a three-layer MLP), the data split is row-level with no patient grouping, and the outcome definitions differ from prior work. The architectural idea of fixed GPT-based embeddings for both input and target features is interesting and worth further study.
major comments (4)
- [Section II-E] The 80/20 random split is performed at the row level, and the manuscript states that each row is one laboratory report but does not report or use any patient identifier. In a real laboratory information system, the same patient contributes multiple reports over time, and repeated reports share stable person-specific values. If any patient appears on both sides of the split, a set-transformer can memorize patient-specific joint value patterns rather than learn a generalizable rule, which would inflate the reported test AUCs, especially the uric acid gain of 8 percentage points. The manuscript neither reports the number of unique patients nor performs a group-aware split. The authors must either stratify the split by patient or provide strong evidence (e.g., unique patient counts and an appropriate split) that this leakage does not occur.
- [Section III, Table IV] The only comparator is a three-layer MLP with architecture 27 x 256 x 4. No classical machine learning baselines (e.g., random forest, gradient boosting) and no imputation-based methods are evaluated, even though the introduction and Table I emphasize comparisons to such methods. The headline claim of 'improvement up to 8% AUC' is therefore only a comparison to one specific MLP architecture; it does not establish that ULM is competitive with or superior to the standard methods used in the prior work cited. The authors should add reasonable baselines, including at least one tree-based method and one imputation-plus-classifier pipeline, on the same train and test sets to support the claim that ULM is a practically useful approach.
- [Table IV vs. Table I] The uric acid positive-class threshold used in Table IV (M: ≥0.48 mmol/L, F: ≥0.38 mmol/L) differs from the threshold used in the prior studies listed in Table I (M: ≥0.42 mmol/L, F: ≥0.36 mmol/L). The choice of threshold changes the prevalence and the difficulty of the classification task, so any implicit comparison of the reported AUC (79.7%) with the prior AUCs in Table I (0.78–0.85) is invalid. The authors should either adopt the thresholds from the prior literature for at least a subset of experiments, or explicitly state that the comparison is not apples-to-apples and discuss the clinical rationale for the higher thresholds.
- [Table V] The confusion matrix counts for each target do not match the test set sizes implied by Table IV and the stated 80/20 split. For example, the glucose counts sum to 142,928, whereas 20% of the glucose sample size 680,502 is 136,100; for cholesterol the sum is 55,477 versus 49,761 expected; for ferritin 22,924 versus 21,186; for uric acid 19,906 versus 18,580. The discrepancies are too large to be rounding artifacts. The authors must correct these numbers or clarify what the sizes in Table IV represent and how the confusion matrices are computed.
minor comments (6)
- [Section II-C, Eq. (3)] The notation in Eq. (3) should be clarified: GPT_i denotes a fixed embedding vector for feature i, and the operation 'GPT_i * v_i + B' is a scalar-vector product plus a (presumably vector) bias B. Please state the dimensions explicitly and explain how the resulting vector is incorporated into the encoder.
- [Section II-B] The sentence 'the sum of all WBC parts from a 5-DIFF analyzer ... must equal100±4%' has a formatting error (missing space before '100'), and the tolerance of 4% is an arbitrary free parameter. Please also state whether this condition is applied to the 3-DIFF analyzers (MID, GRA, LYMP) with the same tolerance.
- [Table I] There are typographical errors in Table I: 'Iron deficiancy anemia' should be 'Iron deficiency anemia', and 'Na ¨ıveBayes' should be 'Naive Bayes' with a space.
- [Section II-C] The manuscript refers to Figure 1 for the architecture, but the figure is not included in the text; please ensure the figure appears with its caption and is referenced correctly.
- [Section IV] The limitations section does not mention the lack of a patient-stratified split or the absence of external validation. Given that the dataset comes from commercial laboratories, the authors should also discuss calibration and the potential for overfitting to the specific analyzer or laboratory population.
- [Section II-E] The sentence 'The statistics in Table IV are the result of three different training runs for each variant' is helpful, but the paper does not report how the test set is held fixed across runs or whether the same random seed is used for the split. Please describe the experimental protocol for the random splits and runs.
Circularity Check
No circular steps found; the reported AUC gains are empirical outcomes of a held-out evaluation and do not reduce to the model's inputs or to self-citations.
full rationale
The paper's core derivation is a supervised classification setup: the target tests (GLU, CHOL, FER, URIC) are explicitly removed from the encoder input (the input set x is defined with f not in P), and the decoder uses only the target feature embeddings as queries with keys and values taken from the encoder. The predicted binary outcomes are therefore not fed back into the model as inputs, and the AUC numbers are computed on a separate 20% test split after training on 80%. No fitted parameter is renamed as a prediction, and no equation in the paper makes the target a restatement of the input. The only self-citation, reference [29] on multi-task learning, is used as background and is not load-bearing for the ULM design or the reported results. The applicability-domain ranges are fitted on the full dataset before splitting, which is a potential data-leakage or validity concern rather than a circularity, because the reported predictions are not constructed from those ranges. Similarly, the row-level random split raises a patient-identity independence question, but that is an evaluation-risk issue, not a circular derivation. The paper's central quantitative claim is therefore not circular, though its external validity depends on the unverified independence of the split and on the absence of external benchmarks.
Assumptions & free parameters
free parameters (3)
- Positive-class thresholds for target tests =
GLU 7 mmol/L, CHOL 5.2 mmol/L, FER 12 ng/mL, URIC 0.48/0.38 mmol/L (M/F)
- Applicability-domain coverage threshold =
99.99%
- WBC-part sum tolerance =
100 +/- 4%
assumptions (4)
- domain assumption GPT embeddings from Yandex GPT carry semantic information about test names that improves predictions.
- domain assumption Rows in the dataset are independent patient reports, so random row splitting gives an unbiased test set.
- standard math Multi-head attention pooling with order-independent queries yields a permutation-invariant set encoder.
- domain assumption Logarithmic transformation and scaling to [0.1, 0.9] preserve clinically meaningful signal.
Cite this review
Pith. "Pith review of Universal Laboratory Model: prognosis of abnormal clinical outcomes based on routine tests." pith.science (2026). https://pith.science/paper/XI7LHM5E
@misc{pith2026250615330,
author = {Pith},
title = {Pith review of: Universal Laboratory Model: prognosis of abnormal clinical outcomes based on routine tests},
year = {2026},
howpublished = {\url{https://pith.science/paper/XI7LHM5E}},
note = {Machine review of arXiv:2506.15330}
}
read the original abstract
Clinical laboratory results are ubiquitous in any diagnosis making. Predicting abnormal values of not prescribed tests based on the results of performed tests looks intriguing, as it would be possible to make early diagnosis available to everyone. The special place is taken by the Common Blood Count (CBC) test, as it is the most widely used clinical procedure. Combining routine biochemical panels with CBC presents a set of test-value pairs that varies from patient to patient, or, in common settings, a table with missing values. Here we formulate a tabular modeling problem as a set translation problem where the source set comprises pairs of GPT-like label column embedding and its corresponding value while the target set consists of the same type embeddings only. The proposed approach can effectively deal with missing values without implicitly estimating them and bridges the world of LLM with the tabular domain. Applying this method to clinical laboratory data, we achieve an improvement up to 8% AUC for joint predictions of high uric acid, glucose, cholesterol, and low ferritin levels.
Figures
Reference graph
Works this paper leans on
-
[1]
World Health Organization, “The selection and use of essential in vitro diagnostics: Report of the third meeting of the strategic advisory group of experts on in vitro diagnostics,” World Health Organization, Tech. Rep. 1031, 2021. [Online]. Available: https://iris.who.int/bitstream/ handle/10665/339064/9789240019102-eng.pdf?sequence=1
work page 2021
-
[2]
Artificial intelligence in routine blood tests,
M. A. Santos-Silva, N. Sousa, and J. C. Sousa, “Artificial intelligence in routine blood tests,”Frontiers in Medical Engineering, vol. 2, 2024. [Online]. Available: https://www.frontiersin.org/journals/ medical-engineering/articles/10.3389/fmede.2024.1369265
arXiv 2024
-
[3]
I.-H. Seo and Y . Lee, “Usefulness of complete blood count (cbc) to assess cardiovascular and metabolic diseases in clinical settings: A comprehensive literature review,”Biomedicines, vol. 10, 2022. [Online]. Available: https://api.semanticscholar.org/CorpusID:253148030
work page 2022
-
[4]
Automated prediction of low ferritin concentrations using a machine learning algorithm,
S. Kurstjens, T. de Bel, A. van der Horst, R. Kusters, J. Krabbe, and J. van Balveren, “Automated prediction of low ferritin concentrations using a machine learning algorithm,”Clinical Chemistry and Laboratory Medicine (CCLM), vol. 60, no. 12, pp. 1921–1928, 2022. [Online]. Available: https://doi.org/10.1515/cclm-2021-1194
-
[5]
Using machine learning to predict laboratory test results,
Y . Luo, P. Szolovits, A. S. Dighe, and J. M. Baron, “Using machine learning to predict laboratory test results,”American Journal of Clinical Pathology, vol. 145, no. 6, pp. 778–788, 06 2016. [Online]. Available: https://doi.org/10.1093/ajcp/aqw064
-
[6]
S. Pullakhandam and S. McRoy, “Classification and Explanation of Iron Deficiency Anemia from Complete Blood Count Data Using Machine Learning,”BioMedInformatics, vol. 4, no. 1, pp. 661–672,
-
[7]
Use of machine learning and routine laboratory tests for diabetes mellitus screening,
G. Cardozo, G. B. Pintarelli, G. R. Andreis, A. C. W. Lopes, and J. L. B. Marques, “Use of machine learning and routine laboratory tests for diabetes mellitus screening,”BioMed Research International, vol. 2022, no. 1, p. 8114049, 2022. [Online]. Available: https://onlinelibrary.wiley.com/doi/abs/10.1155/2022/8114049
-
[8]
S. L. Cichosz, C. Bender, and O. Hejlesen, “A comparative analysis of machine learning models for the detection of undiagnosed diabetes patients,”Diabetology, vol. 5, no. 1, pp. 1–11, 2024. [Online]. Available: https://www.mdpi.com/2673-4540/5/1/1
work page 2024
Show all 35 references
-
[9]
Ensemble machine learning prediction of hyperuricemia based on a prospective health checkup population,
Y . Zhang, L. Zhang, H. Lv, and G. Zhang, “Ensemble machine learning prediction of hyperuricemia based on a prospective health checkup population,”Frontiers in Physiology, vol. 15, 2024. [Online]. Available: https://www.frontiersin.org/journals/physiology/ articles/10.3389/fph...
2024
-
[10]
Exploration of machine learning for hyperuricemia prediction models based on basic health checkup tests,
S. Lee, E. K. Choe, and B. Park, “Exploration of machine learning for hyperuricemia prediction models based on basic health checkup tests,” Journal of Clinical Medicine, vol. 8, no. 2, 2019. [Online]. Available: https://www.mdpi.com/2077-0383/8/2/172
2019
-
[11]
Prediction model of random forest for the risk of hyperuricemia in a chinese basic health checkup test,
Y . Gao, S. Jia, D. Li, C. Huang, Z. Meng, Y . Wang, M. Yu, T. Xu, M. Liu, J. Sun, Q. Jia, Q. Zhang, Y . Gao, K. Song, X. Wang, and Y . Fan, “Prediction model of random forest for the risk of hyperuricemia in a chinese basic health checkup test,”Bioscience Reports, vol. 41, no...
2021 doi
-
[12]
A survey of large language models in medicine: Progress, application, and challenge,
H. Zhou, F. Liu, B. Gu, X. Zou, J. Huang, J. Wu, Y . Li, S. S. Chen, P. Zhou, J. Liu, Y . Hua, C. Mao, C. You, X. Wu, Y . Zheng, L. Clifton, Z. Li, J. Luo, and D. A. Clifton, “A survey of large language models in medicine: Progress, application, and challenge,” 2024. [Online]....
2024 arXiv
-
[13]
Revisiting deep learning models for tabular data,
Y . Gorishniy, I. Rubachev, V . Khrulkov, and A. Babenko, “Revisiting deep learning models for tabular data,” 2023. [Online]. Available: https://arxiv.org/abs/2106.11959
2023 arXiv
-
[14]
Blood uric acid prediction with machine learning: Model development and performance comparison,
M. B. Sampa, M. N. Hossain, M. R. Hoque, R. Islam, F. Yokota, M. Nishikitani, and A. Ahmed, “Blood uric acid prediction with machine learning: Model development and performance comparison,” JMIR Med Inform, vol. 8, no. 10, p. e18331, Oct 2020. [Online]. Available: https://medi...
2020
-
[15]
Multi-class classification algorithms for the diagnosis of anemia in an outpatient clinical setting,
R. V ohra, A. Hussain, A. K. Dudyala, J. Pahareeya, and W. Khan, “Multi-class classification algorithms for the diagnosis of anemia in an outpatient clinical setting,”PLOS ONE, vol. 17, no. 7, pp. 1–18, 07
-
[16]
A survey on missing data in machine learning,
T. Emmanuel, T. Maupong, D. Mpoeleng, T. Semong, B. Mphago, and O. Tabona, “A survey on missing data in machine learning,”Journal of Big data, vol. 8, pp. 1–37, 2021
2021
-
[17]
A review of missing data handling techniques for machine learning,
L. O. Joel, W. Doorsamy, and B. S. Paul, “A review of missing data handling techniques for machine learning,”International Journal of Innovative Technology and Interdisciplinary Sciences, vol. 5, no. 3, pp. 971–1005, 2022
2022
-
[18]
A review on genetic algorithm: past, present, and future,
S. Katoch, S. S. Chauhan, and V . Kumar, “A review on genetic algorithm: past, present, and future,”Multimedia tools and applications, vol. 80, pp. 8091–8126, 2021
2021
-
[19]
Development of optimal transmission rate of the kinematic chain by using genetic algorithms coded in mathcad,
A. Sulejmani and O. Koc ¸a, “Development of optimal transmission rate of the kinematic chain by using genetic algorithms coded in mathcad,” International Journal of Innovative Technology and Interdisciplinary Sciences, vol. 4, no. 4, pp. 792–803, 2021
2021
-
[20]
Gamin: Generative adversarial multiple imputation network for highly missing data,
S. Yoon and S. Sull, “Gamin: Generative adversarial multiple imputation network for highly missing data,” inProceedings of the IEEE/CVF conference on computer vision and pattern recognition, 2020, pp. 8456– 8464
2020
-
[21]
A survey of missing data imputation using generative adversarial networks,
J. Kim, D. Tae, and J. Seok, “A survey of missing data imputation using generative adversarial networks,” in2020 International conference on artificial intelligence in information and communication (ICAIIC). IEEE, 2020, pp. 454–456
2020
-
[22]
A machine learning model for hemoglobin estimation and anemia classification,
E. El-Kenawy and E. SM, “A machine learning model for hemoglobin estimation and anemia classification,”International Journal of Computer Science and Information Security (IJCSIS), vol. 17, no. 2, pp. 100–108, 2019
2019
-
[23]
Set transformer: A framework for attention-based permutation-invariant neural networks,
J. Lee, Y . Lee, J. Kim, A. R. Kosiorek, S. Choi, and Y . W. Teh, “Set transformer: A framework for attention-based permutation-invariant neural networks,” 2019. [Online]. Available: https://arxiv.org/abs/1810. 00825
2019
-
[24]
Attention is All you Need,
A. Vaswani, N. Shazeer, N. Parmar, J. Uszkoreit, L. Jones, A. N. Gomez, Ł. Kaiser, and I. Polosukhin, “Attention is All you Need,” inAdvances in Neural Information Processing Systems, I. Guyon, U. V . Luxburg, S. Bengio, H. Wallach, R. Fergus, S. Vishwanathan, and R. Garnett, ...
2017
-
[25]
Available: https://yandex.cloud/en-ru/services/yandexgpt
[Online]. Available: https://yandex.cloud/en-ru/services/yandexgpt
-
[26]
Tabtransformer: Tabular data modeling using contextual embeddings,
X. Huang, A. Khetan, M. Cvitkovic, and Z. Karnin, “Tabtransformer: Tabular data modeling using contextual embeddings,” 2020. [Online]. Available: https://arxiv.org/abs/2012.06678
2020 arXiv
-
[27]
Cholletet al., “Keras,” https://keras.io, 2015
F. Cholletet al., “Keras,” https://keras.io, 2015
2015
-
[28]
TensorFlow: Large-scale machine learning on heterogeneous systems,
M. Abadi, A. Agarwal, P. Barham, E. Brevdo, Z. Chen, C. Citro, G. S. Corrado, A. Davis, J. Dean, M. Devin, S. Ghemawat, I. Goodfellow, A. Harp, G. Irving, M. Isard, Y . Jia, R. Jozefowicz, L. Kaiser, M. Kudlur, J. Levenberg, D. Man ´e, R. Monga, S. Moore, D. Murray, C. Olah, M...
2015
-
[29]
A survey of multi-task learning methods in chemoinformatics,
S. Sosnin, M. Vashurina, M. Withnall, P. Karpov, M. Fedorov, and I. V . Tetko, “A survey of multi-task learning methods in chemoinformatics,”Molecular Informatics, vol. 38, no. 4, p. 1800108,
-
[30]
On embeddings for numerical features in tabular deep learning,
Y . Gorishniy, I. Rubachev, and A. Babenko, “On embeddings for numerical features in tabular deep learning,” 2023. [Online]. Available: https://arxiv.org/abs/2203.05556
2023 arXiv
-
[31]
Tabm: Advancing tabular deep learning with parameter-efficient ensembling,
Y . Gorishniy, A. Kotelnikov, and A. Babenko, “Tabm: Advancing tabular deep learning with parameter-efficient ensembling,” 2025. [Online]. Available: https://arxiv.org/abs/2410.24210
2025 arXiv
-
[32]
Federated learning: Overview, strategies, applications, tools and future directions,
B. Yurdem, M. Kuzlu, M. K. Gullu, F. O. Catak, and M. Tabassum, “Federated learning: Overview, strategies, applications, tools and future directions,”Heliyon, vol. 10, no. 19, p. e38137, 2024. [Online]. Available: https://www.sciencedirect.com/science/article/pii/ S2405844024141680
2024
-
[2019]
Available: https://onlinelibrary.wiley.com/doi/abs/10
[Online]. Available: https://onlinelibrary.wiley.com/doi/abs/10. 1002/minf.201800108
-
[2022]
Available: https://doi.org/10.1371/journal.pone.0269685
[Online]. Available: https://doi.org/10.1371/journal.pone.0269685
-
[2024]
Available: https://www.mdpi.com/2673-7426/4/1/36
[Online]. Available: https://www.mdpi.com/2673-7426/4/1/36
Reviewed August 15, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.