{"id":"ac02af3d-ba2f-4dca-92a8-453d72d25ed6","arxiv_id":"2412.12909","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":4.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":5,"one_line_summary":"PT, a plain Transformer with modality-specific blocks, achieves 0.896 AUC on a MIMIC-based readmission benchmark, outperforming LSTM (0.878) and GRU (0.880) on the same setup.","lead":"A simple Transformer that combines electronic health records, chest X-rays, and clinical notes predicts 30-day hospital readmission with about 0.90 AUC, slightly better than LSTM and GRU baselines. The authors claim it stays accurate when timestamps are unclear or missing, making it a potentially easy-to-use baseline for hospitals.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Headline AUC is not yet shown to measure generalization: no patient-level split is reported, and hyperparameters (RF feature count, noise schedule, K) appear tuned on the reported AUC.","rationale":"The reader's CONDITIONAL verdict is appropriate. I partly agree with the reader's weakest assumption about leakage from RF feature selection and noise tuning, but I would sharpen it: the more concrete and more serious issue is that no patient-level split is described, and the paper's own hyperparameter sweeps (Tables 3, 4, 6, 7) report AUCs above the headline 0.896, which suggests the headline number may be a post-selection result. I am not alleging misconduct; under-specification of the protocol is sufficient to block a firm acceptance. A single rerun with patient-level grouping and locked hyperparameters would settle whether the claimed advantage over LSTM/GRU is real. The robustness claim about missing temporal information is also an overstatement, because no experiment actually removes or corrupts temporal structure, but the core unresolved issue is the evaluation protocol.","tokens_in":10611,"tokens_out":4843,"duration_ms":44667,"concrete_test":"Re-run the full pipeline with an explicit patient-level split (no patient appears in both train and test), freeze all hyperparameters (RF feature count k, noise schedule, ensemble K) on the training/validation folds, and compute a single test AUC for PT, LSTM, and GRU. If PT's test AUC falls below the 0.878 LSTM baseline or the ordering changes, the central claim is not supported.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The most load-bearing assumption is that the reported AUCs measure held-out generalization. Section 4.1 reports 13,763 admissions from 11,041 patients, so many patients contribute multiple admissions, yet the paper never states whether the split is by admission or by patient; an admission-level split allows the same patient in train and test and can inflate AUC. Section 3.4 and Tables 3, 4, 6, and 7 show that the RF feature count k, the dynamic-noise schedule, and the K-fold ensemble size are selected by comparing AUC values. If those AUCs come from the same split used for the headline result, Table 1's 0.896 is an optimistic selection artifact. The internal numbers support this worry: Table 3 reports a noise configuration with AUC 0.900 and Table 7 reports K=10 with AUC 0.9014, both higher than the headline, so the reported figure is not the best configuration of the stated pipeline. The robustness claim is also untested: 'missing temporal information' is asserted, not manipulated, and Table 2 shows adding images lowers AUC from 0.896 to 0.881, so multimodal superiority is not monotone. Without the exact split and selection protocol, the LSTM/GRU comparison is not interpretable.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper presents PT, a Transformer-based model for predicting 30-day hospital readmissions from three modalities: EHR tabular data, chest radiographs, and clinical notes. The model extracts features per modality, processes each with a separate Transformer block, pools with attention, concatenates, and passes through an MLP. Additional components include Random Forest feature selection for EHR, dynamic noise scheduling, label smoothing focal loss, and a K-fold ensemble. On a MIMIC-IV-derived dataset of 13,763 admissions, PT reports an AUC of 0.896 for EHR+Notes, compared to 0.878 for LSTM and 0.880 for GRU; ablations explore noise schedulers, feature counts, loss functions, and ensemble sizes. The paper claims superior accuracy, scalability, and robustness, including when temporal information is missing.","tokens_in":10857,"tokens_out":5029,"duration_ms":40328,"significance":"If the reported results are reliable, PT would provide a simple, competitive Transformer baseline for multimodal readmission prediction, and the modular design could be useful for future benchmarks. The paper's strengths are its use of a public dataset (MIMIC-IV/MIMIC-CXR) and the transparency of the architecture description. However, the evaluation protocol is currently under-specified: no patient-level split is documented, hyperparameters appear to be selected on the reported AUC, strong cited baselines are not compared, and the robustness claim is not directly tested. These issues prevent the reader from assessing whether the 0.896 AUC reflects true generalization. No code or repeated-run statistics are provided, so the precision of the comparison is unknown.","major_comments":[{"comment":"The paper reports 13,763 admissions from 11,041 unique patients but never states whether the train/test split is by admission or by patient. If the split is by admission, the same patient can appear in both training and test sets, and the reported AUCs in Tables 1 and 2 would be inflated by within-patient correlation. This is load-bearing because the central claim of superiority depends on the integrity of the evaluation. Please specify the split granularity and, if it is admission-level, re-run the experiments with a patient-level split.","section":"Section 4.1 and all experiments"},{"comment":"The Random Forest feature count k, the noise-scheduler parameters, and the K-fold value are selected by comparing AUC values, yet the paper does not describe a nested validation procedure or a held-out test set that is distinct from the selection process. If the AUCs in Tables 3, 4, 6, and 7 come from the same split as the headline result in Table 1, then the reported 0.896 is a selection artifact. Moreover, Table 3 reports a configuration with AUC 0.900 and Table 7 reports K=10 with AUC 0.9014, both higher than the headline 0.896, so the paper must clarify which configuration yields the headline and how the selection was shielded from test-set information.","section":"Section 3.4 and Tables 3, 4, 6, 7"},{"comment":"The abstract claims PT 'outperforms existing ones,' but the only baselines presented are LSTM and GRU. The cited state-of-the-art models MuST (Ref. [33]) and ClinicalBERT (Ref. [3]) are not included in any experiment, so the claim of superiority over existing multimodal or text-based methods is not supported. The paper should either add these or comparable baselines, or restrict the claim to the recurrent architectures actually evaluated. In addition, all AUCs are reported as point estimates from what appears to be a single run; without error bars or significance tests, the differences (0.896 vs. 0.878) may not be statistically meaningful.","section":"Section 4.3 and Related Work"},{"comment":"The robustness claim—that PT 'performs well even when temporal information is missing'—is never tested experimentally. The paper does not define which temporal information is missing, nor does it compare against a variant with complete or shuffled temporal order. Furthermore, Table 2 shows that adding images to EHR+Notes decreases AUC from 0.896 to 0.881, which contradicts the intuition that more modalities help; the paper should explain this non-monotonicity and directly test the robustness claim by ablating temporal information.","section":"Section 1 and Table 2"}],"minor_comments":[{"comment":"The CXR representation is defined as Rq×1024, but the notes section later says the feature dimension is '1024 or 4096 depending on the extraction method'; please clarify which dimensions are used in the experiments.","section":"Section 3.1"},{"comment":"The focal loss equation defines pt = σ(z), but Algorithm 1 computes pt = exp(-BCE_loss); these are not equivalent, so the pseudo-code and the equation should be reconciled.","section":"Section 3.3"},{"comment":"Hyperparameters such as embedding dimension, feed-forward size, dropout, and batch size are not reported; please provide a complete configuration to enable reproducibility.","section":"Section 4.2"},{"comment":"The description 'identical design configurations' for LSTM and GRU is vague; please state exactly what attention and positional encoding were applied to the recurrent baselines.","section":"Section 4.3"},{"comment":"The phrase 'a innegligible gap' contains a typo and should read 'a non-negligible gap.'","section":"Section 2"},{"comment":"The running time for EHR+Note+Image (12.5 s/epoch) is much higher than others; if the image features are pre-extracted rather than trained end-to-end, this should be clarified in the experimental setup.","section":"Table 2"}],"recommendation":"major_revision","confidential_remarks":"The paper's evaluation protocol is the main barrier. The authors should be required to either provide a proper patient-level split and a nested validation scheme, or revise the claims to the level of what the current protocol can support. The absence of comparisons to MuST and ClinicalBERT is also a concern for fit with the readmission-prediction literature."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Quick take: clean, plainly-written baseline paper—per-modality Transformer blocks over EHR, notes, and chest X-rays, attention pooling, smoothed focal loss. Reported AUC 0.896 on a MIMIC readmission cohort edges out LSTM (0.878) and GRU (0.880) at comparable size and runtime. If that number holds, it's a useful default baseline for a mature task; the gain is modest but not trivial.\n\nCredit where due: the architecture is a straightforward composition of established pieces, the ablations on feature count, K, and loss are sensible, and the authors are honest that images don't always help (Table 2: EHR+Notes+Image 0.881, below EHR+Notes 0.896). The writing is accessible, and testing a plain transformer as a strong baseline is a legitimate question.\n\nSoft spots, in order:\n\n1. The split is unspecified. 13,763 admissions come from 11,041 patients, so many patients contribute multiple admissions. If the split is admission-level rather than patient-level, the same patient appears in both train and test, and the AUC is optimistic. This changes the meaning of every number in the paper.\n\n2. Hyperparameters look selected on the reported AUC. Tables 3–4 show noise-scheduler searches over AUC; Table 6 selects k=100 features by AUC; Table 7 selects K=10 by AUC. The best AUC in those tables (0.900, 0.9014) is higher than the headline 0.896, so the paper is not reporting the best configuration it found. That pattern suggests selection on the same split as the headline, which biases the comparison against baselines.\n\n3. No comparison to the state of the art they cite (MuST, ClinicalBERT), so the superiority claim over existing methods is unsupported.\n\n4. The robustness claim is asserted, not tested. They never manipulate or remove temporal information; they just note that their data lacks clear timestamps. That's a description, not an experiment.\n\n5. Minor: no error bars, no repeated runs, no code or data release. A single AUC difference of 0.016 is hard to interpret without variance.\n\nOverall: the paper is worth engaging with as a baseline idea, but the central quantitative claim is not yet supported. A serious referee could help the authors fix the evaluation: patient-level splitting, fold-local preprocessing, transparent selection protocol, and missing baselines. That is heavy revision, not a reject. I'd send it to review rather than desk reject, but only if the venue wants methodologically honest empirical work.","headline":"A useful baseline idea whose reported AUC is probably optimistic—split unspecified, hyperparameters tuned on the reported metric; needs heavy revision before the numbers can be trusted.","tokens_in":11409,"tokens_out":3703,"would_cite":false,"duration_ms":33782,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"A plain Transformer with separate per-modality blocks and simple training add-ons predicts 30-day hospital readmissions with an area-under-the-ROC-curve of 0.896, beating LSTM and GRU at comparable parameter counts and runtime.","keywords":["hospital readmission prediction","transformer","multimodal learning","electronic health records","clinical notes","chest radiographs","AUC","classification"],"falsifier":"Re-run the EHR+Notes experiment with the random-forest feature selector and the noise-scheduler search nested inside each training fold, then apply the final model once to a held-out set of admissions; if the AUC falls to the 0.878–0.880 range of the recurrent baselines, the claimed advantage is leakage rather than architecture.","tokens_in":10391,"feed_emoji":"🏥","tokens_out":15061,"duration_ms":120154,"temperature":0.7,"pith_summary":"PT is a plain Transformer for 30-day hospital readmission prediction that combines electronic health records, chest radiographs, and clinical notes. The paper's central claim is that this simple per-modality Transformer architecture beats recurrent baselines like LSTM and GRU at the same parameter and runtime budget, reaching an area-under-the-ROC-curve (AUC) of 0.896 on EHR plus notes. The model is also claimed to stay accurate when only clinical notes or only EHR data are used, and when temporal information is missing or unclear. A strong, simple baseline matters because hospitals can deploy and maintain it more easily, and researchers can compare future models against a meaningful lower bound.","feed_headline":"Plain transformer beats recurrent networks at readmission prediction","feed_subtitle":"Same run time and parameters, better accuracy: 0.896 vs 0.878/0.880 AUC for 30-day readmission.","key_machinery":"The load-bearing mechanism is a per-modality Transformer encoder: each data type gets its own self-attention block with its own parameters, with positional encodings preserving sequence order, followed by attention pooling to collapse the sequence into one vector, concatenation of the modality vectors, and an MLP for the final probability. Feature preparation uses random-forest selection to keep the top-$k$ EHR variables, a self-supervised chest X-ray encoder for radiographs, and TF-IDF vectors for clinical notes. Training adds label-smoothing focal loss, a cosine learning-rate schedule, dynamic Gaussian noise, and a 10-fold test-time ensemble, and the paper reports these components individually raise the headline AUC.","core_discovery":"The paper's central claim is that a plain Transformer—one self-attention block per modality with its own parameters—can outperform recurrent sequence models for 30-day all-cause readmission without requiring explicit timestamps. Each modality is fed in as an ordered sequence: hospital-day rows for EHR, radiographs in recorded order, and notes in note order. After attention pooling and concatenation, an MLP produces the readmission probability. On 13,763 admissions the paper reports an area-under-the-ROC-curve (AUC) of 0.896 for EHR plus notes, against 0.878 for LSTM and 0.880 for GRU with comparable parameter counts and runtime, and notes-only and EHR-only configurations reach 0.832 and 0.761. The authors attribute the outcome to treating each modality as a sequence rather than requiring timestamped events, supported by random-forest feature selection, dynamic noise, and a 10-fold ensemble.","pith_inferences":["A fully nested validation protocol—fitting the random-forest selector and noise schedule only on training folds—would make the 0.896 figure directly comparable to models tuned the same way; if the gap persists, plain Transformers should become the default baseline in clinical readmission papers.","The large gap between notes-only and EHR-only performance suggests that free-text notes carry more predictive signal than the structured features used here; replacing TF-IDF with a modern clinical language model might push the single-modality ceiling higher.","The drop when chest radiographs are added (0.896 to 0.881) is worth probing: if the images are noisy or misaligned with the admission, simple late fusion may dilute the EHR+notes signal, and gating or weighted fusion could recover the loss.","A clean test of the missing-timestamps claim would be to strip all timestamps and compare PT against a recurrent model that requires event times; the paper's assumptions imply PT should lose little or nothing."],"forward_implications":["Recurrent models LSTM and GRU are not required to reach strong readmission AUC; PT matches their runtime and parameter count and reports higher AUC.","Institutions without imaging data can still deploy the model: notes alone give AUC 0.832 and EHR alone gives 0.761.","The approach works when explicit timestamps are missing or unclear, because each modality is treated as an ordered sequence rather than requiring event times.","The reported gains come from a bundle of training techniques—feature selection, dynamic noise, and ensembling—not from the architecture alone; ablations show each component contributes.","Because the design is modular, adding or dropping a modality only requires adding or removing a Transformer block, supporting the scalability claim."],"supporting_citations":[{"why":"Supplies the EHR preprocessing pipeline and the multimodal spatiotemporal graph baseline this work builds from.","marker":"[16]"},{"why":"Supplies the self-supervised chest X-ray feature encoder used to embed radiographs.","marker":"[17]"},{"why":"Supplies the TF-IDF text feature extraction used to vectorize clinical notes.","marker":"[18]"},{"why":"Supplies the Transformer architecture and attention mechanism underlying each modality block.","marker":"[20]"},{"why":"Supplies the Random Forest feature selector that reduces EHR dimensionality to the top-k features.","marker":"[43]"},{"why":"Supplies the Focal Loss component of the label-smoothing focal-loss training criterion.","marker":"[45]"},{"why":"Supplies the K-fold cross-validation ensemble procedure used at test time.","marker":"[47]"},{"why":"Provides the EHR and clinical notes data used for training and evaluation.","marker":"[50]"},{"why":"Provides the LSTM recurrent baseline whose AUC and runtime are compared against PT.","marker":"[55]"},{"why":"Provides the GRU recurrent baseline whose AUC and runtime are compared against PT.","marker":"[56]"}],"fun_headline_variants":["Plain transformer beats LSTM and GRU for readmission prediction","Transformer without timestamps outshines recurrent models for readmissions","Simple transformer wins over RNNs in readmission AUC","No timestamps needed: plain transformer predicts readmissions best"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The headline results depend on an evaluation in which every tuning choice—the random-forest feature count and the noise-scheduling parameters—is made inside the training folds and never sees the held-out admissions; if that separation is breached, the reported AUC would not reflect real-world generalization.","fun_headline_variants_meta":{"raw":{"variants":["Plain transformer beats LSTM and GRU for readmission prediction","Transformer without timestamps outshines recurrent models for readmissions","Simple transformer wins over RNNs in readmission AUC","No timestamps needed: plain transformer predicts readmissions best"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000252,"raw_usage":{"total_tokens":1550,"prompt_tokens":921,"completion_tokens":629,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":537,"completion_tokens_details":{"reasoning_tokens":560}},"tokens_in":537,"tokens_out":629,"duration_ms":6145,"temperature":1.0,"reasoning_tokens":560,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-11T13:35:52.800005+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Re-run the EHR+Notes experiment with the random-forest feature selector and the noise-scheduler search nested inside each training fold, then apply the final model once to a held-out set of admissions; if the AUC falls to the 0.878–0.880 range of the recurrent baselines, the claimed advantage is leakage rather than architecture.","supporting_citations":[{"cited_title":"Multimodal spatiotemporal graph neural networks for improved prediction of 30-day all-cause hospital readmission","cited_arxiv_id":"2204.06766","evidence_quote":"Supplies the EHR preprocessing pipeline and the multimodal spatiotemporal graph baseline this work builds from."},{"cited_title":"Moco pretraining improves representation and transferability of chest x-ray models","cited_arxiv_id":null,"evidence_quote":"Supplies the self-supervised chest X-ray feature encoder used to embed radiographs."},{"cited_title":"Text mining: use of TF-IDF to examine the relevance of words to documents","cited_arxiv_id":null,"evidence_quote":"Supplies the TF-IDF text feature extraction used to vectorize clinical notes."},{"cited_title":"Attention is all you need","cited_arxiv_id":null,"evidence_quote":"Supplies the Transformer architecture and attention mechanism underlying each modality block."},{"cited_title":"Random forests","cited_arxiv_id":null,"evidence_quote":"Supplies the Random Forest feature selector that reduces EHR dimensionality to the top-k features."},{"cited_title":"The’K’in K-fold Cross Validation","cited_arxiv_id":null,"evidence_quote":"Supplies the K-fold cross-validation ensemble procedure used at test time."},{"cited_title":"MIMIC-IV , a freely accessible electronic health record dataset","cited_arxiv_id":null,"evidence_quote":"Provides the EHR and clinical notes data used for training and evaluation."},{"cited_title":"Long short-term memory","cited_arxiv_id":null,"evidence_quote":"Provides the LSTM recurrent baseline whose AUC and runtime are compared against PT."}],"review_version":1}