{"id":"fbfe5754-1d77-4ac7-95a1-cde33ff17a56","arxiv_id":"2608.06430","paper_version":1,"verdict":"REJECT","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"high","formal_verification":"none","parameter_count":7,"one_line_summary":"A heterogeneous graph transformer with temporal attention and balanced multi-task training reports state-of-the-art average performance on four EHR prediction tasks on MIMIC-III and MIMIC-IV, but the evaluation may leak test information into graph construction.","lead":"MiGHT-EHR builds a graph from patient records and trains one model to predict drug recommendations, length of stay, death, and readmission, reporting the best average performance across two large ICU databases. If the evaluation is sound, the model could improve early warning and care planning, but the graph is built on all data including the test split, which may inflate the reported gains.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"MiGHT-EHR's graph construction and TransE pretraining use all visits including the test set before the 90/10 split; this transductive leakage makes the reported +8.71% average improvement over inductive baselines unreliable.","rationale":"The reader's weakest_assumption correctly identifies the full-data graph construction and pretraining as the load-bearing flaw, and I agree. The central claim is that MiGHT-EHR achieves the only positive average improvement across all four tasks, so the evaluation protocol must compare methods under an identical inductive split. Section 4.1's NPMI computation over N = |V_vis| and Section 4.2's TransE on the entire graph break that protocol for the proposed method alone, giving it access to test-visit co-occurrence structure and test-visit node embeddings before the split. Baselines such as GRU, Transformer, and MulT-EHR are trained from the same 90% split but do not receive this transductive signal. A second flaw, checkpoint selection on test AUROC, also inflates reported numbers, but if applied uniformly to all methods it is at least a shared evaluation rule; the graph and pretraining leakage is specific to MiGHT-EHR and therefore more directly undermines the fairness of the comparison. The proposed concrete re-run with train-only preprocessing would settle whether the concern actually lands. I do not see an internal inconsistency in the architecture itself; the DB-MTL balancing and temporal attention are plausible components, and the appendix honestly flags the MIMIC-IV mortality calibration limitation. However, the evaluation protocol is not sound enough to support the claimed state-of-the-art result, so the reader's REJECT verdict is appropriate and unchanged.","tokens_in":18810,"tokens_out":6237,"duration_ms":61713,"concrete_test":"Re-run MiGHT-EHR exactly as in §5.2, but compute the NPMI co-occurrence statistics (eq. 3 with threshold tau=0.10 and count floor kappa=5) and the TransE pretraining (§4.2) using only the 90% training visits and the induced graph of those visits, leaving the test visits entirely out of both preprocessing stages. Keep the same 90/10 split, seed 612, hyperparameters, and the paper's checkpoint rule. Then compare the resulting test AUROC and AUPR with Tables 1–2 and recompute the average relative improvement delta_p over MulT-EHR. If delta_p drops substantially (e.g., below +3%) or MiGHT-EHR no longer ranks first on average, the transductive leakage is confirmed as the load-bearing cause of the claimed gains.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim is purely empirical: that a single shared model achieves the best average performance across all four tasks. For that claim to hold, the evaluation must compare methods under an identical inductive protocol. In the protocol described in §5.1, a fixed 90/10 visit-level split is used, but the method's two preprocessing stages are not split-aware. §4.1 (eq. 3) computes NPMI over N = |V_vis|, the full visit set, so co-occurrence edges among diagnosis, procedure, and prescription nodes encode co-occurrence statistics from the test visits as well as the train visits. §4.2 then runs TransE pretraining on the full graph, with membership and temporal relations incident to test visits, so patient and visit embeddings, including those of test visits, are learned with knowledge of the test visits' own structure. When the shared encoder and task heads are subsequently trained on the 90% split, the test representation already depends on test-set structure. The comparison baselines are not given this transductive advantage. Even if no test labels are used in these stages, the reported test AUROC and AUPR numbers are not produced under an inductive evaluation protocol, so the central claim that MiGHT-EHR is the best multi-task predictor among the fourteen methods is not established. The stated checkpoint rule (maximizing mean test AUROC) compounds the problem, but the graph and pretraining leakage is the method-specific unfairness that most directly undermines the comparison.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes MiGHT-EHR, a heterogeneous graph transformer for multi-task prediction from temporal EHRs. It constructs a five-node-type graph with membership, temporal, and NPMI-based co-occurrence edges; initializes concept nodes with Bio_ClinicalBERT text embeddings and patient/visit nodes with TransE embeddings; and trains a shared two-layer relation-aware message-passing encoder with cross-visit temporal attention and DB-MTL gradient balancing. On MIMIC-III and MIMIC-IV it reports the best average relative improvement over MulT-EHR across mortality, readmission, length-of-stay, and drug recommendation, and presents post-hoc analyses of neighborhood purity, linear concept probes, and calibration. The central claim is empirical: a single shared model achieves the highest average performance across four tasks among fourteen methods.","tokens_in":19135,"tokens_out":8055,"duration_ms":84706,"significance":"If the empirical claim were established under a clean inductive protocol, the paper would be a useful contribution: it jointly models heterogeneity, temporality, and task sharing in one architecture; it uses a statistically motivated edge criterion (NPMI) rather than raw co-occurrence counts; and it reports graph statistics, hyperparameters, and an honest limitation on MIMIC-IV mortality calibration. The post-hoc analyses include sensible control conditions, such as comparing outcome-neighborhood purity with comorbidity-burden purity. However, the current evaluation does not establish the central claim because the graph and pretrained features are built on the full dataset including test visits, and because the checkpoint is selected on test AUROC. These are protocol-level issues that affect every reported comparison and must be fixed before the performance claims can be assessed.","major_comments":[{"comment":"The graph construction and node-feature pretraining are not split-aware. In §4.1, NPMI in Eq. (3) is computed with N = |V_vis| over all visits, so co-occurrence edges are selected using test-visit co-occurrence statistics; in §4.2 and Appendix E, TransE is pretrained on the full graph with membership and next_visit relations incident to test visits. Because message passing in §4.3 operates on this graph and the pretrained features are frozen into test-visit representations, the reported test AUROC/AUPR values for MiGHT-EHR are produced under a transductive protocol, whereas the baselines in Tables 1–2 are trained on the 90% split without access to test visits or test-derived statistics. The claim in §5.4 and Table 3 that MiGHT-EHR is the only method with positive average improvement therefore does not follow from an apples-to-apples comparison. The experiments need to be re-run with graph construction and TransE pretraining restricted to the training split, or all baselines must be given the same transductive access as MiGHT-EHR.","section":"§4.1, §4.2, §5.1"},{"comment":"Model selection is performed on the test set: the reported results come from \"the checkpoint maximizing the mean test AUROC across the four tasks.\" Selecting a checkpoint over 1000 training epochs using the test labels invalidates the test AUROC/AUPR as an unbiased estimate and can systematically inflate the reported numbers, especially for a high-capacity model. A validation split should be used for checkpoint selection, and the test set should be reserved for a single final evaluation. This issue affects the validity of the headline comparison even if the same rule were applied to all methods.","section":"§5.1, Appendix E"},{"comment":"The average improvement claim is based on a single run with one seed (seed 612, Appendix E), and no confidence intervals or significance tests are reported. Since the headline result is a single aggregate number and the gains are concentrated in two rare-event tasks, the absence of uncertainty quantification makes it impossible to tell whether the reported +8.71% is robust or within run-to-run variation. Multiple seeds with error bars, or a paired significance test, are needed to support the central empirical claim.","section":"Table 3, §5.4"}],"minor_comments":[{"comment":"In the MiGHT-EHR row, the MIMIC-III length-of-stay accuracy is reported as 642.22, which is impossible for a percentage; this appears to be a typo for 64.22.","section":"Table 1"},{"comment":"The text in §5.4 states Δp = +8.61%, while Table 3 and Eq. (15) give +8.71%; these numbers should be made consistent.","section":"§5.4 vs Table 3"},{"comment":"GraphCare (Jiang et al. 2023) is listed as a comparable method in §5.3 but its results do not appear in Tables 1 or 2; either its results should be included or it should be removed from the comparison list.","section":"§5.3, Tables 1–2"},{"comment":"The appendix candidly reports high raw ECE for MIMIC-IV mortality (0.461) and readmission (0.241); this disclosure is appreciated, but the main text's claim that predicted probabilities are \"well calibrated\" should be qualified to note that this holds only for drug recommendation and length-of-stay on MIMIC-IV.","section":"Appendix F, Table 12"}],"recommendation":"major_revision","confidential_remarks":"The blocking issue is the transductive leakage and test-set checkpoint selection. These are methodological errors that can be fixed by re-running the pipeline with split-aware graph construction and pretraining and with validation-based model selection. If the headline improvement disappears after the fix, the central claim will need to be substantially revised. The authors' candid disclosure of calibration limitations is a positive sign, but it does not address the evaluation-protocol concerns."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"The paper's central claim—that a single shared model beats all fourteen baselines on average across four tasks—is not reliably established by the current evaluation. The graph construction in Sec 4.1 computes NPMI over all visits, including test visits, and the TransE pretraining in Sec 4.2 runs on the full graph. That means the test representations already encode test-set co-occurrence structure before any training happens. The baselines do not get this transductive advantage. On top of that, the reported checkpoint is selected by maximizing mean test AUROC, which is another form of test-set leakage. These two problems alone invalidate the headline numbers, and the results are based on a single seed.\n\nThat said, the paper has real strengths. The architecture is a sensible integration of three known components: NPMI-based edge construction, a heterogeneous graph transformer with temporal attention, and DB-MTL gradient balancing. The qualitative analysis is thoughtful and unusually honest—the authors explicitly flag the high mortality calibration error on MIMIC-IV in Appendix F, which is the kind of self-aware reporting you rarely see. The use of CCS categories to handle the ICD-9/10 transition in MIMIC-IV is also a good practical decision.\n\nThe soft spots are concentrated in the experimental protocol. I already mentioned the leakage and test-set checkpoint selection. There is also a small inconsistency in the reported average improvement: the text says +8.61% while Table 3 says +8.71%. The drug recommendation results on MIMIC-III are actually below several baselines, so the \"best on average\" claim is carried almost entirely by mortality and readmission—exactly the tasks most likely to be inflated by the transductive effects. The post-hoc analyses are nice but they are not a substitute for a sound evaluation.\n\nWho is this paper for? Someone working on multi-task clinical prediction with heterogeneous graphs will find the architecture and the honest analysis of representation quality useful, but they should not cite the performance numbers. The method itself is plausible enough that it deserves a serious referee—but only if the authors are willing to redo the evaluation with a clean inductive split, validation-based checkpoint selection, and multiple seeds. As submitted, I would not accept it.\n\nRecommendation: desk reject with an invitation to resubmit after fixing the protocol. If the authors can show competitive results under a clean split and with proper model selection, the paper could be a solid contribution.","headline":"The architecture is coherent and the post-hoc analysis is honest, but the central empirical claim is undermined by transductive leakage in graph construction and checkpoint selection on test AUROC.","tokens_in":19662,"tokens_out":3052,"would_cite":false,"duration_ms":32091,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"One shared graph-transformer model outperforms all comparison methods on average across four clinical tasks.","keywords":["electronic health records","heterogeneous graph transformer","multi-task learning","temporal attention","normalized pointwise mutual information","clinical prediction","mortality prediction","readmission prediction"],"falsifier":"Rebuild the graph and pretrained features using only training-side visits, or only visits before a cutoff date, keeping everything else fixed, and compare test AUROC across the four tasks; if the average improvement over the reference multi-task baseline collapses, the pre-split graph construction is what carries the result.","tokens_in":18597,"feed_emoji":"🩺","tokens_out":14873,"duration_ms":133694,"temperature":0.7,"pith_summary":"MiGHT-EHR is a single model that tries to capture all three structures hidden in electronic health records at once: the web of clinical entities (patients, visits, diagnoses, procedures, prescriptions), the order of a patient's visits over time, and the statistical links between prediction tasks. The paper claims that a heterogeneous graph built with statistically grounded co-occurrence edges, combined with temporal attention and gradient-balanced multi-task training, learns one visit representation that serves four clinical tasks better on average than all comparison methods on two large critical-care databases. The headline result is an 8.71% relative average improvement over the reference multi-task baseline, driven mainly by mortality and readmission prediction, while drug recommendation and length-of-stay stay competitive. If true, a single shared model could support several clinical predictions at once while retaining interpretable structure in its representations.","feed_headline":"Single shared model gains 8.7% average over reference baseline","feed_subtitle":"One encoder serves four clinical tasks; rare-event mortality and readmission gains are largest.","key_machinery":"The load-bearing construction is the heterogeneous temporal EHR graph: five node types (patient, visit, diagnosis, procedure, prescription) joined by membership edges, directed next-visit edges between a patient's consecutive visits, and co-occurrence edges between concept nodes retained only when the normalized pointwise mutual information $\\mathrm{NPMI}(a,b)=\\frac{1}{-\\log p(a,b)}\\log\\frac{p(a,b)}{p(a)p(b)}$ passes a threshold with a count floor. The encoder is a relation-aware graph transformer: two message-passing layers compute per-relation attention, per-node inter-relation attention, and gated residuals, followed by a causal cross-visit temporal attention in which each visit attends directly to all of its earlier visits; task losses are combined by DB-MTL, a gradient-balancing objective that equalizes the scale and magnitude of each task's update.","core_discovery":"The paper's central claim is that representing electronic health records as a heterogeneous temporal graph, rather than as a sequence or a plain heterogeneous graph, lets four clinical tasks be learned from one shared encoder. Each visit is a node whose features come from frozen clinical text embeddings for concept nodes and structure-based embeddings for patients and visits; edges are of three kinds: membership links tying each visit to its patient and its diagnoses, procedures, and prescriptions; directed next-visit edges that keep the encoding causal; and co-occurrence edges between concepts retained only when normalized pointwise mutual information passes a threshold. A two-layer relation-aware message-passing transformer followed by cross-visit temporal attention produces one representation per visit, and dual-balancing gradient equalization prevents any single task from dominating the shared parameters. On the two evaluation databases the model is the only compared method with a positive average relative improvement over the reference multi-task baseline ($\\Delta_p = +8.71\\%$), with the gains concentrated in mortality ($+22.16\\%$) and readmission ($+17.90\\%$); drug recommendation and length of stay remain competitive. Post-hoc probing shows patient neighborhoods become organized by outcome rather than by diagnosis, clinical concepts appear as linear directions in the representation space, and calibration is good for the drug and length-of-stay tasks.","pith_inferences":["Rebuilding the graph and pretrained features only from admissions before the prediction window would test whether the method's advantage survives real deployment, where future visits are not available when the model is built.","Because the three relation categories are not hospital-specific, the same recipe of association-thresholded edges plus causal temporal attention could transfer to insurance claims or outpatient registries with different concept vocabularies and visit structures.","An ablation that replaces the gradient-balancing objective with plain loss weighting would isolate how much of the rare-event gain comes from optimization balance rather than from graph structure or temporal attention.","The paper's pairwise co-occurrence edges only approximate multi-drug interactions; modeling the set of concepts in a visit as a hyperedge is a concrete next step that would likely help the drug-recommendation task most."],"forward_implications":["A single shared encoder can serve four heterogeneous tasks and is the only compared method with a positive average relative improvement over the reference multi-task baseline, so multi-task sharing does not have to come at the price of single-task accuracy.","The rare-event tasks of mortality and readmission are where the model gains most (+22.16% and +17.90% relative), while drug recommendation and length of stay remain within about one point of specialized methods.","Because the NPMI criterion discards 75–90% of observed concept pairs, the reported results imply that association-based edges, not raw co-occurrence counts, are what make the graph informative.","The post-hoc analyses show patient neighborhoods increasingly share clinical outcomes with depth while diagnostic and comorbidity controls stay flat, implying the representation separates prognosis from diagnosis.","Calibration is task-dependent: drug and length-of-stay probabilities are nearly calibrated on both databases, while mortality is mildly overconfident on the smaller database and strongly overconfident on the larger one, so those probabilities should be recalibrated before clinical use."],"supporting_citations":[{"why":"Supplies the frozen clinical text encoder whose pooled token embeddings initialize concept node features.","marker":"Alsentzer et al. 2019"},{"why":"Supplies the relation-embedding pretraining that initializes patient and visit node features.","marker":"Bordes et al. 2013"},{"why":"Provides the reference multi-task heterogeneous graph baseline against which the average improvement is measured.","marker":"Chan et al. 2024"},{"why":"Provides the dual-balancing multi-task gradient objective used to combine the four task losses.","marker":"Lin et al. 2023"},{"why":"Supplies the first critical-care evaluation database used for all four tasks.","marker":"Johnson et al. 2016"},{"why":"Supplies the second, larger critical-care evaluation database used for all four tasks.","marker":"Johnson et al. 2023"},{"why":"Supplies the CCS mapping that unifies ICD-9 and ICD-10 diagnosis and procedure codes into shared concept nodes.","marker":"Agency for Healthcare Research and Quality 2019"},{"why":"Supplies the attention machinery on which the relation-aware and cross-visit temporal attention layers build.","marker":"Vaswani et al. 2017"}],"fun_headline_variants":["Graph transformer for EHRs gains 8.7% over multi-task baseline","One model, four tasks: biggest wins in mortality and readmission","MiGHT-EHR: one shared encoder outperforms multi-task baseline","Heterogeneous temporal graph lifts EHR predictions, best for mortality and readmission"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The graph statistics and the pretrained node features are computed from all records, test visits included, before the 90/10 split, so the reported test gains rest on the assumption that those test visits do not leak label-relevant information into the representation.","fun_headline_variants_meta":{"raw":{"variants":["Graph transformer for EHRs gains 8.7% over multi-task baseline","One model, four tasks: biggest wins in mortality and readmission","MiGHT-EHR: one shared encoder outperforms multi-task baseline","Heterogeneous temporal graph lifts EHR predictions, best for mortality and readmission"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.001289,"raw_usage":{"total_tokens":5327,"prompt_tokens":1074,"completion_tokens":4253,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":690,"completion_tokens_details":{"reasoning_tokens":4173}},"tokens_in":690,"tokens_out":4253,"duration_ms":29619,"temperature":1.0,"reasoning_tokens":4173,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-10T04:25:28.887861+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Rebuild the graph and pretrained features using only training-side visits, or only visits before a cutoff date, keeping everything else fixed, and compare test AUROC across the four tasks; if the average improvement over the reference multi-task baseline collapses, the pre-split graph construction is what carries the result.","supporting_citations":[],"review_version":1}