{"id":"a7238085-a6fb-40e2-8ae8-05299f013053","arxiv_id":"2412.11839","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":4.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":3,"one_line_summary":"Adding global electric heterogeneity features derived from standard ECGs to risk factors improves machine learning prediction of cardiovascular events in a tertiary cardiology referral cohort.","lead":"Using standard 12-lead ECGs, this study computes global electric heterogeneity features and combines them with risk factors to train XGBoost models that triage patients referred to a tertiary cardiology center. In 274 patients, the combined model achieved the best performance, with 94% sensitivity at 31% specificity, and the spatial ventricular gradient was the top feature.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Test-set model selection invalidates comparative performance: the 'best of 50' XGBoost instance is chosen by AUC on the same 30% test set used for reported metrics, so SRG's apparent advantage may be a selection artifact rather than a real GEH benefit.","rationale":"The paper's central claim is that VCG/GEH features add predictive value for cardiovascular triage beyond standard ECG and risk factors. That claim requires a valid comparison showing SRG outperforms S, R, and G. The most load-bearing threat to that comparison is test-set leakage through model selection: the authors explicitly choose the best of 50 XGBoost instances by AUC on the test set, then report test-set metrics for that chosen instance. This guarantees inflated performance and makes the reported differences between feature sets unreliable, especially given the small outcome count (51 events) and modest observed gaps. The phone follow-up exclusion raised by the reader is a real limitation, but it affects all models and would not by itself explain SRG's apparent advantage; the selection/evaluation leakage directly undermines the comparative evidence for GEH enhancement. I agree partially with the reader because their rationale already mentions test-set selection, though their formal weakest_assumption focuses on outcome ascertainment. I keep the verdict CONDITIONAL because the underlying hypothesis is plausible and consistent with prior GEH literature, and a properly blinded reanalysis could still support it. The requested final verdict label is CONDITIONAL, the same as the reader's; the contribution here is a sharper, more specific condition: unbiased evaluation must precede any claim of enhancement.","tokens_in":8290,"tokens_out":5379,"duration_ms":49374,"concrete_test":"Re-run all four models (S, G, R, SRG) with nested cross-validation: for each outer fold, split the 274 subjects into train/validation/test (e.g., 10-fold), fit the 50 XGBoost instances on the training portion, select the representative and the 90%-sensitivity threshold on the validation portion, and evaluate only on the held-out test portion. Report the mean and 95% CI of AUC and F2 over folds for each model. If the SRG advantage over S and R does not exceed the confidence intervals, the conclusion that GEH features enhance triage is unsupported. A supplementary check should report the distribution of AUC across the 50 instances rather than only the maximum.","verdict_should_be":"CONDITIONAL","load_bearing_attack":"Section 'XGBoost' states that 50 instances are trained per parameter set and 'from all the 50 instances in one model, the one with the higher AUC was selected to be the final XGBoost tree.' The only split is 70/30 train/test, and Table 2/Figure 4 report metrics for this selected representative on the same test set. Because test labels are used both to choose the representative and to compute reported AUC, F2, specificity, and sensitivity, each Table 2 entry is a maximum over 50 random seeds rather than an unbiased estimate. Different feature sets have different variance across seeds, so SRG's higher selected AUC (67.6% vs 62.5% for R) and specificity (30.8% vs 20.0% for S) can be inflated by selection even if the true predictive difference is zero. The threshold producing 94.12% sensitivity is also chosen from the test ROC, adding further leakage. The central claim that GEH features 'significantly enhance' triage therefore rests on a comparison that is not internally valid. In addition, the Discussion reports AUC 76.3%, F1 0.48, and specificity 49.25% for SRG, none of which appear in Table 2, so it is unclear which analysis produced the headline numbers. An unbiased nested or repeated cross-validation estimate is required before any enhancement can be attributed to GEH features.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper evaluates whether vectorcardiographic (VCG) features obtained from a standard 12-lead ECG via the Kors transform (Global Electric Heterogeneity, GEH) improve machine-learning-based triage of patients referred to a tertiary cardiology center, relative to using only standard ECG parameters or risk factors. The authors train XGBoost models on four feature sets (S, R, G, SRG), select the best of 50 trained instances by highest AUC, and report that the combined SRG model achieves the best F2 score, AUC, sensitivity, and specificity (Table 2). The conclusion is that incorporating VCG features significantly enhances triage and prediction of cardiovascular outcomes.","tokens_in":8608,"tokens_out":3519,"duration_ms":31169,"significance":"If the central claim were supported by valid evaluation, the paper would provide a practical, low-cost decision-support tool for cardiology triage and add to the evidence that GEH markers carry incremental prognostic information beyond standard ECG intervals and risk factors. The univariate associations in Table 1 (e.g., QRST angles, SVG, age, previous MI/PCI/diabetes) are plausible and consistent with prior literature. The study has the strength of using real-world referred-population data and a transparent decision-tree model with feature-importance reporting. However, the comparative performance claim is currently undermined by a model-selection protocol that uses test-set labels, and by inconsistent reporting of the main results; these issues must be fixed before the significance of the clinical claim can be assessed.","major_comments":[{"comment":"The representative model for each feature set is selected as the single instance with the highest AUC among 50 trained instances, and this selection is made using the same 30% test set on which all metrics in Table 2 and Figure 4 are reported. This constitutes selection on the test set: each reported AUC is a maximum over 50 random seeds rather than an unbiased estimate of model performance. Because the variance across seeds is likely to differ across feature sets (S, R, G, SRG), the observed advantage of SRG (AUC 67.6% vs 62.5% for R) could be inflated or even entirely artifactual, even if the true predictive difference is zero. An unbiased evaluation protocol—such as nested cross-validation, or selection of the representative within each training fold only—is required before any statement about the superiority of SRG can be made. This issue directly affects the central conclusion in §5 that VCG features 'significantly enhance' triage.","section":"§2 (XGBoost) and Table 2"},{"comment":"The Discussion states: 'When all parameters were combined, the model achieved a higher Area Under the Curve (AUC) of 76.3%, an F1 score of 0.48, and a specificity of 49.25%, while maintaining a fixed sensitivity of 94.12%.' These numbers appear nowhere in Table 2, which reports for SRG an AUC of 67.6%, an F2 score of 0.62, and a specificity of 30.77%. The reader cannot determine which analysis produced the headline values. If the 76.3% AUC comes from a different threshold, a different run, or a different evaluation split, the manuscript must state this explicitly and reconcile the discrepancy; as written, the results section and discussion contradict each other.","section":"§4 Discussion and Table 2"},{"comment":"The classification threshold is chosen from the test-set ROC curve to yield approximately 90% sensitivity, and the sensitivity (94.12%) and specificity values in Table 2 are then computed on the same test set at that threshold. This is a second, independent source of test-set leakage: the threshold is a free parameter tuned on the same data used to report performance, which optimistically biases the reported specificity and F2 scores. The threshold should be selected inside the training loop (e.g., by cross-validation on the training portion) or fixed a priori by a rule that does not use test labels.","section":"§2 (Model's performance)"},{"comment":"The outcome is determined solely by phone follow-up, and the 29 patients (out of 303) who did not answer any follow-up call are excluded from the analysis. If non-responders differ systematically from responders in their event rates or risk-factor distribution, the model's performance and feature-importance estimates could be distorted. The manuscript should compare baseline characteristics of included versus excluded patients, or at minimum discuss the likely direction and magnitude of this selection bias, since the outcome status is the target variable of the study.","section":"§2 (Population)"},{"comment":"The claim that GEH features 'significantly enhance' prediction is not supported by any statistical hypothesis test comparing the models. The univariate p-values in Table 1 do not establish incremental predictive value in a multivariable model, and no test is reported for the difference between the SRG and R (or S, G) AUCs. The paper should include a formal comparison of model performance—for example, a DeLong test for paired AUCs, a likelihood-ratio test in a logistic-regression framework with the same features, or confidence intervals from a properly conducted cross-validation—to substantiate the 'significant enhancement' language used in the abstract and conclusion.","section":"§5 Conclusion and §2 (XGBoost)"}],"minor_comments":[{"comment":"The phrase 'Kors Matriz' should be 'Kors matrix' (also appears in the Data Pre-processing section).","section":"Abstract and §2"},{"comment":"The sentence 'The goal is to provide an improved diagnostic tool having a high Sensitivity with the biggest sensibility possible' is awkward and should be reworded; 'sensibility' is not the correct term here.","section":"§1 Introduction"},{"comment":"The Discussion reports SVG values of 66.0 [43.4, 86.2] for the no-event group and 45.6 [31.4, 64.4] for the event group, but Table 1 lists 65.0 [42.4, 85.9] and 45.6 [29.5, 61.9]. The discrepancy in IQRs should be corrected or explained.","section":"§4 Discussion"},{"comment":"The keyword 'Survival Prediction' is misleading because the study analyzes a binary 1-year outcome, not time-to-event survival data; 'Outcome Prediction' would be more accurate.","section":"Keywords"},{"comment":"The description of the GEH computation says an 'adapted version' of Tereshchenko's algorithm was used, but the details of the adaptation are not provided; the manuscript should specify what was modified relative to the published algorithm, since this affects reproducibility.","section":"§2 (Data Pre-processing)"}],"recommendation":"major_revision","confidential_remarks":"The core scientific question is legitimate, and the GEH features are well-motivated, but the evaluation protocol (test-set model selection plus threshold tuning on the test set) is a serious methodological flaw that currently invalidates the comparative performance claim. The inconsistency between the Discussion and Table 2 also needs to be resolved. The paper would likely be suitable for a clinical informatics or cardiovascular digital-health journal after a rigorous reanalysis with nested cross-validation or a pre-specified model-selection rule. I would also encourage the authors to make the data and code available to support reproducibility."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Quick take: the paper contains a genuinely useful clinical observation—GEH markers, especially QRS-T angle and SVG, are univariately associated with cardiovascular outcomes in a Brazilian referral cohort—but the ML comparison that supposedly shows GEH adds predictive value is not to be trusted. The stress-test note is right: the authors trained 50 XGBoost instances per feature set, picked the one with highest AUC on the test set, and then reported that same test set's metrics. That selection invalidates the comparison between models. The SRG model's 67.6% AUC and 30.8% specificity are maxima over 50 random seeds, not unbiased estimates. The Discussion even reports a different set of numbers (AUC 76.3%, F1 0.48) that don't appear in Table 2, so it's unclear which analysis produced the headline claims.\n\nWhat's actually new: the application of GEH to a real triage setting with standard 12-lead ECG and a Brazilian public-health population. Table 1 is the strongest part—the univariate comparisons are clean and consistent with prior GEH literature (Waks et al.). The authors also deserve credit for describing their preprocessing and for using a published GEH algorithm (Kors transform, Tereshchenko's code). The feature importance analysis is illustrative, though it inherits the same selection problem.\n\nSoft spots: the test-set model selection is the big one. Another minor issue is the exclusion of 29 patients who didn't answer follow-up; non-response could correlate with outcomes, but that's a standard limitation and probably not fatal. The sample is small (274, 51 events) and the population narrow, so external validity is limited.\n\nWho is this for? A clinical informatics audience interested in cheap ECG-derived triage markers. The univariate findings might be worth citing, but the machine-learning claims need reanalysis with nested cross-validation or a proper validation set.\n\nRecommendation: I'd send it to peer review, but with a clear message that the central comparison needs to be redone. The raw data collection effort and the univariate results are worth preserving. If the authors can't provide an unbiased estimate of the GEH increment, the paper should be reduced to the descriptive analysis.","headline":"Real univariate signal in GEH markers, but the ML comparison is sunk by test-set selection; needs reanalysis before the main claim can be believed.","tokens_in":9156,"tokens_out":2225,"would_cite":false,"duration_ms":20567,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"This paper tries to establish that vectorcardiographic markers of global electric heterogeneity, computed from standard 12-lead ECG traces, materially improve prediction of cardiovascular outcomes in a population already referred to…","keywords":["Cardiology","Survival Prediction","Machine Learning","Vectorcardiogram","Electrocardiogram","Global Electric Heterogeneity","Tertiary Care","XGBoost"],"falsifier":"An external validation with adjudicated outcomes, or with non-responders tracked through registries, would settle the claim if it showed that adding GEH parameters to risk factors and standard ECG intervals does not raise the AUC, or that the combined model's specificity advantage disappears when follow-up is complete.","tokens_in":8129,"feed_emoji":"🫀","tokens_out":8347,"duration_ms":69830,"temperature":0.7,"pith_summary":"What the paper tries to establish is that vectorcardiographic markers of global electric heterogeneity (GEH), computed mathematically from an ordinary 12-lead ECG, carry predictive information for cardiovascular outcomes in patients already referred to a tertiary cardiology service. In 274 patients with phone-confirmed outcomes, GEH parameters — particularly the QRS-T angle and the spatial ventricular gradient — differed significantly between patients who later had an event and those who did not, while most standard ECG intervals did not. A gradient-boosted decision-tree model that combined risk factors, standard ECG intervals, and GEH parameters achieved the best triage performance, with an AUC of 67.6%, an F2 score of 0.62, a fixed sensitivity of 94.12%, and a specificity of 30.77%. The intended payoff is a transparent, low-cost way to send fewer low-risk patients to scarce tertiary care without raising the rate of missed events.","feed_headline":"VCG markers from standard ECG sharpen heart-risk triage","feed_subtitle":"Adding QRS-T angle and SVG to risk factors cuts false-positive referrals at 94% sensitivity.","key_machinery":"The load-bearing object is the derived vectorcardiogram: a Kors transformation matrix converts the standard 12-lead ECG into three orthogonal leads, and an adapted GEH analysis algorithm computes global electric heterogeneity markers from the measured wave boundaries. The decisive quantities are the spatial ventricular gradient (SVG) and the QRS-T angle, which summarize the direction and magnitude of repolarization relative to depolarization. The prediction machinery is a gradient-boosted decision-tree ensemble trained with class resampling, tuned on the area under the precision-recall curve, and represented by the best of 50 instances; the model's gain importance is used to attribute each feature's contribution. The argument works by showing the derived VCG markers survive in a model alongside clinical risk factors, where standard ECG intervals do not.","core_discovery":"The paper's central claim, stated in its conclusion, is that incorporating VCG features derived from standard ECG signals significantly enhances the triage and prediction of cardiovascular outcomes in a referred population. The evidence offered is comparative: models built from only standard ECG intervals, only risk factors, and only GEH parameters each performed worse than the combined model, and the combined model's feature-importance ranking put GEH measures at the top, with age and prior PCI also contributing. The QRS-T angle and the spatial ventricular gradient were the GEH parameters with the strongest statistical separation between outcome groups. The implication the authors draw is that the same 12-lead ECG already recorded at first attendance can be post-processed into VCG-based markers that improve allocation of tertiary cardiology care.","pith_inferences":["The authors do not compare the derived VCG markers against a separately recorded Frank-lead VCG, so this study leaves open whether direct VCG acquisition would strengthen or change the predictive gain.","Because the outcome definition mixes mortality with revascularization procedures such as PCI and cardiac surgery, which are themselves clinical decisions, part of the model's signal may reflect who was offered treatment rather than who had spontaneous disease progression.","A natural external check would be to validate the combined model in a primary-care or general-population cohort, where the prevalence of prior MI and PCI is much lower and the spectrum of ECG abnormalities is wider."],"forward_implications":["If the result holds in larger cohorts, tertiary referral algorithms can be upgraded using only software on existing ECG traces, with no extra patient visit or equipment.","The combined model's fixed 94% sensitivity with specificity improved from roughly 3–20% to about 31%, implying that adopting it would reduce false-positive referrals to tertiary care while keeping event capture essentially unchanged.","Standard ECG intervals alone had little discriminating power in this referred population; the gain came from VCG-derived heterogeneity markers and clinical history.","Because the winning model is a decision tree, its cutoff rules can be inspected and audited by clinicians, which eases adoption in triage workflows."],"supporting_citations":[{"why":"Supplies the Kors transformation used to convert the 12-lead ECG into orthogonal vectorcardiogram signals.","marker":"[10]"},{"why":"Provides the adapted GEH analysis algorithm that turns the ECG into a VCG and extracts the global electric heterogeneity parameters.","marker":"[11]"},{"why":"Establishes global electric heterogeneity markers as outcome predictors in larger population cohorts, the hypothesis transferred to this referred triage population.","marker":"[7]"},{"why":"Gives the maximum-sensitivity AUROC threshold logic that the study adapts into its fixed 90% sensitivity triage cutoff.","marker":"[16]"},{"why":"Supplies reference spatial ventricular gradient values for event-free individuals, used to interpret the normal-beat comparison.","marker":"[17]"},{"why":"Supports the choice of gradient-boosted decision trees over deep learning for tabular clinical data.","marker":"[13]"}],"fun_headline_variants":["ECG-derived vectors enhance heart risk triage","VCG markers from standard ECG improve cardiology triage","QRS-T angle and SVG strengthen outcome prediction","Decision tree with VCG features allocates cardiology care better","Standard ECG plus VCG analysis refines tertiary referrals"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing assumption is that the outcome status obtained by phone follow-up is correct and that excluding the 29 patients who never answered a follow-up call does not bias the cohort; if those patients have different event rates, both the measured AUC and the feature-importance ranking could shift.","fun_headline_variants_meta":{"raw":{"variants":["ECG-derived vectors enhance heart risk triage","VCG markers from standard ECG improve cardiology triage","QRS-T angle and SVG strengthen outcome prediction","Decision tree with VCG features allocates cardiology care better","Standard ECG plus VCG analysis refines tertiary referrals"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000219,"raw_usage":{"total_tokens":1472,"prompt_tokens":1002,"completion_tokens":470,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":618,"completion_tokens_details":{"reasoning_tokens":393}},"tokens_in":618,"tokens_out":470,"duration_ms":5169,"temperature":1.0,"reasoning_tokens":393,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-11T14:30:35.706199+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"An external validation with adjudicated outcomes, or with non-responders tracked through registries, would settle the claim if it showed that adding GEH parameters to risk factors and standard ECG intervals does not raise the AUC, or that the combined model's specificity advantage disappears when follow-up is complete.","supporting_citations":[{"cited_title":"doi:10.22489/CINC.2018.165","cited_arxiv_id":null,"evidence_quote":"Supplies the Kors transformation used to convert the 12-lead ECG into orthogonal vectorcardiogram signals."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Provides the adapted GEH analysis algorithm that turns the ECG into a VCG and extracts the global electric heterogeneity parameters."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Establishes global electric heterogeneity markers as outcome predictors in larger population cohorts, the hypothesis transferred to this referred triage population."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Supplies reference spatial ventricular gradient values for event-free individuals, used to interpret the normal-beat comparison."},{"cited_title":"Shwartz-Ziv, A","cited_arxiv_id":null,"evidence_quote":"Supports the choice of gradient-boosted decision trees over deep learning for tabular clinical data."}],"review_version":1}