{"id":"23a444a5-2c9a-46fd-9ebd-dbb266a74f9f","arxiv_id":"2508.12418","paper_version":1,"verdict":"REJECT","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"high","formal_verification":"none","parameter_count":3,"one_line_summary":"A transformer that attends along both time and clinical-variable axes improves sepsis prediction and missingness robustness on EHR benchmarks.","lead":"The paper introduces BAT, a transformer model that pays attention to both time points and clinical measurements in electronic health records, rather than only one axis. It reports better sepsis prediction than its baseline models and argues that the model handles missing data more robustly.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Hyperparameter selection on test splits (§7.1) invalidates the reported benchmark numbers, so the P19 SOTA claim is not established.","rationale":"The reader's weakest_assumption identifies exactly the concern I consider most load-bearing: Supp. §7.1 appears to select hyperparameters using test-set AUROC. The abstract's central claim—state-of-the-art sepsis prediction on P19—rests entirely on the Table 2 numbers, and if those numbers come from test-set selection, they are optimistically biased and do not support a SOTA claim. The paper's phrasing is ambiguous, but the burden is on the authors to provide a protocol that does not touch the test set during model selection; the repository was not run, so there is no independent verification. I also considered other issues: the P19 baseline modifications (§4.3.1) are disclosed and make the comparison more conservative rather than unfair, and the transfer-learning evidence is mixed but secondary. The test-selection issue is more fundamental because it invalidates every reported benchmark metric. Therefore I agree with the reader's REJECT verdict; the concrete check I propose would either confirm the flaw or, if the wording was misleading and validation selection was used, rehabilitate the empirical claim.","tokens_in":15877,"tokens_out":4399,"duration_ms":43490,"concrete_test":"Inspect the hyperparameter sweep script in the GitHub repository to determine whether the AUROC used for candidate selection is computed on the validation split or the test split. If it is computed on the test split, rerun the P19 sweep for BAT and all baselines with selection performed on validation AUROC only, then evaluate the selected models on the test splits and compare the resulting metrics. If BAT's P19 AUROC/AUPRC advantage over GRU-D and IP-Nets shrinks or reverses, the SOTA claim should be rejected.","verdict_should_be":"UNCHANGED","load_bearing_attack":"Supp. §7.1 states that each of 20 hyperparameter candidates was 'trained and tested on 5 splits of train, validation, and test data' and that 'the model with the highest resulting AUROC was selected.' Taken literally, the test splits are used for model selection, so the AUROC and AUPRC values reported in Table 2 are not unbiased estimates of generalization—they are the result of selecting the best candidate on the test set. This inflates BAT's reported margin over the baselines and directly undermines the abstract's central claim that BAT achieves state-of-the-art performance on sepsis prediction (P19). Even if every model used the same protocol, test-set selection overfits the test data and makes the numbers incomparable to published SOTA results, which are obtained with proper held-out evaluation. The paper's wording is ambiguous, and the repository was not executed, so this cannot be resolved without inspecting the sweep code. This is the load-bearing concern because the paper's contribution is empirical: without unbiased test numbers, the architecture's benefit over existing methods is not demonstrated.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes the Bi-Axial Transformer (BAT), an architecture for EHR classification that applies axial attention over both the sensor and time axes of multivariate irregularly sampled time series, while retaining explicit missingness indicators and learning per-sensor embeddings. The authors evaluate BAT on mortality prediction (P12, MIMIC-III), sepsis prediction (P19), and a controlled sparsity experiment on HAR, comparing against SeFT, GRU-D, IP-Nets, a standard Transformer, and iTransformer. They report that BAT achieves state-of-the-art sepsis prediction on P19, is competitive on mortality tasks, is more robust to missing data than other transformers, and that shared sensor embeddings enable joint training across datasets. All baselines were re-implemented in PyTorch and released in a public repository. The core empirical claim is the P19 SOTA result and the associated architectural advantages.","tokens_in":16004,"tokens_out":4840,"duration_ms":50295,"significance":"If the empirical results are unbiased, BAT is a sensible and modular extension of axial attention to sparse, irregular EHR time series. The sensor-embedding transfer experiment is a useful idea for multi-dataset pretraining, and the reimplementation of several baselines in a modern framework is a practical community contribution. The ablations and sparsity experiments are thoughtful and give insight into where the architecture helps. However, the central claims rest entirely on the benchmark numbers. The hyperparameter selection protocol described in the supplement appears to use test-set labels for model selection, which would invalidate the reported metrics and the SOTA statement. Until that issue is resolved, the significance of the contribution cannot be assessed.","major_comments":[{"comment":"The hyperparameter sweep protocol states that 'each candidate was trained and tested on 5 splits of train, validation, and test data' and that 'the model with the highest resulting AUROC was selected.' Taken literally, this selects hyperparameters based on test-set AUROC, which makes the reported AUROC and AUPRC values in Table 2 in-sample selection results rather than unbiased estimates of generalization. This directly affects the central claim in the Abstract and Section 5 that BAT achieves state-of-the-art performance on P19 sepsis prediction. Please clarify whether the selection AUROC was computed on the validation splits or the test splits. If test splits were used, the experiments must be re-run with hyperparameters selected on validation data (or via nested cross-validation) and all tables regenerated; the repository contains the sweep code, so this is feasible.","section":"Supplementary §7.1"},{"comment":"The modification of GRU-D and IP-Nets for P19 — pooling hidden states across all time points instead of using the final hidden state — is a substantial deviation from the original model definitions. The authors state this was done to avoid missed diagnoses when sepsis resolves before the end of the sequence, but this changes the inductive bias of the baselines. To establish a fair comparison, please report results for the original final-state versions as well, or justify with a citation that this pooling modification is standard practice for P19. Without this, it is unclear whether the reported baseline numbers are the strongest reasonable comparators. This issue is compounded by the hyperparameter-selection problem in §7.1, since the modified baselines' hyperparameters were selected under the same protocol.","section":"§4.3.1"},{"comment":"The claim of 'state-of-the-art performance on sepsis prediction' is supported only by comparisons against the authors' reimplemented baselines. No published state-of-the-art numbers for the PhysioNet 2019 sepsis challenge or other recent P19 sepsis-prediction results are cited or compared. If the term 'state-of-the-art' is meant globally, the paper needs to benchmark against published results; if it is meant only relative to the included baselines, the wording should be changed accordingly. This is a substantive point because the P19 result is the paper's main empirical contribution.","section":"Abstract; §5"}],"minor_comments":[{"comment":"In the paragraph after Eq. (3), 'concatentated' should be 'concatenated.' Also, the description of tau as 'the maximum possible time point' is ambiguous; since the encoding uses tau in the denominator as a scaling factor, please clarify whether tau is the largest timestamp in the dataset or a fixed hyperparameter analogous to max_timescale in SeFT.","section":"§3.2.1"},{"comment":"The entry for Transformer on P19 AUPRC appears to be missing from the table (only the AUROC is visible in the text). Please ensure all cells are reported consistently, as the claim of a 2.32-point AUPRC improvement over the second-best model depends on that value.","section":"Table 2"},{"comment":"In the HAR sparsity experiment, the statement that all models perform 'equivalently to random guessing' at 99% sparsity would be more informative if the random-guess performance level were stated explicitly and shown as a reference line in Fig. 6.","section":"§4.4.2"},{"comment":"The sentences describing the average improvements over iTransformer and Transformer would benefit from a small table or explicit arithmetic, since the numbers are not directly readable from Table 2 due to the formatting of the bold/italic markers.","section":"§4.3.1"},{"comment":"The attention visualization is described in the text, but the figure itself is hard to interpret without a more detailed legend explaining the circles, arrows, and density plot. Consider moving some of the supplementary description into the main caption.","section":"Fig. 4"}],"recommendation":"major_revision","confidential_remarks":"The decisive issue for me is the hyperparameter sweep protocol in Supp. §7.1. If the authors selected hyperparameters using test AUROC, the empirical contribution is not established, and the paper would need a full re-run of all experiments. The wording is ambiguous enough that I would not reject outright without giving the authors the chance to clarify and correct, especially since the code repository is promised. I would ask the editor to request the actual sweep script or, failing that, a precise statement of which split was used for selection. The architecture itself is coherent and the baseline reimplementation effort is valuable, so a major revision is appropriate if the experiments can be redone correctly."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Quick take: this is a sensible and reasonably clear architecture paper—two parallel axial-attention tracks over time and sensor axes, with explicit sensor identity embeddings—but the headline claim of SOTA sepsis prediction on P19 is not established as reported. Supp. §7.1 says hyperparameter candidates were 'trained and tested on 5 splits of train, validation, and test data' and 'the model with the highest resulting AUROC was selected.' Read literally, that is test-set model selection, which would inflate every number in Table 2 and void the comparison to any published baseline. The wording is ambiguous enough that it might mean validation selection, but the authors need to fix this before anyone cites the sepsis result.\n\nWhat's genuinely new: the combination of axial attention in two parallel tracks for EHR classification, plus explicit sensor embeddings that can be shared across datasets. Axial attention itself is not new, and per-sensor embeddings are used in GRU-D and iTransformer, but the specific two-track shared-weight design and the cross-dataset shared-embedding experiment are new. The paper also does something useful: it reimplements several popular baselines in PyTorch and makes them public. The ablation study on P12 is informative, and the HAR sparsity stress test is a nice way to show robustness—though it is a synthetic test, not real EHR missingness.\n\nThe soft spots are real but mostly fixable. The §7.1 issue is the load-bearing one, and it has to be resolved. The SOTA claim also rests entirely on self-implemented baselines; there is no direct comparison to published numbers for P19 sepsis, so 'state-of-the-art' is a stretch even if the internal comparison is fair. The transfer-learning claim is weaker than the abstract implies: shared embeddings help P12 AUROC and reduce variance, but MIMIC-III AUPRC drops, so the conclusion should be tempered.\n\nOverall: the architecture is plausible, the writing is clear, and the code release is a plus. I'd send it to review, not desk-reject it, because the central problem is a protocol ambiguity that might have a benign explanation. But I'd ask the authors to state exactly which split was used for model selection and to re-run the benchmark if it was the test set.","headline":"A clear architecture paper for EHR classification whose headline sepsis result depends on an ambiguous hyperparameter-selection protocol that must be resolved before the SOTA claim can be trusted.","tokens_in":16604,"tokens_out":3347,"would_cite":true,"duration_ms":34408,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Bi-Axial Transformer reaches state-of-the-art sepsis prediction","keywords":["Electronic Health Records","Predictive Models","Time Series Analysis","Axial Attention","Transformer","Missing Data","Sepsis Prediction","Transfer Learning"],"falsifier":"Re-run the P19 comparison with a nested procedure: hold out one test split untouched until after hyperparameter selection, or select hyperparameters using only training and validation data; if BAT's reported 2.32-point AUPRC margin over the second-best model disappears or reverses, the state-of-the-art claim is not established.","tokens_in":15621,"feed_emoji":"🏥","tokens_out":6665,"duration_ms":66504,"temperature":0.7,"pith_summary":"This paper introduces the Bi-Axial Transformer (BAT), a classifier that applies attention along both the time axis and the sensor axis of electronic health record data instead of attending along a single flattened axis. The paper claims BAT reaches state-of-the-art sepsis prediction on the P19 benchmark and is competitive with the best RNN methods for mortality prediction on two ICU datasets, while outperforming other transformers under increasing data sparsity. It argues that retaining informative missingness (the pattern of which readings are absent) and learning explicit per-sensor identity embeddings are what give BAT its edge. If true, the model is a practical route to combining multiple EHR datasets through shared sensor embeddings for large-scale pretraining, and a reusable benchmark suite for EHR classification.","feed_headline":"Two-axis attention transformer tops sepsis prediction","feed_subtitle":"New Bi-Axial Transformer keeps missing-data signals and learns sensor embeddings that transfer across EHR datasets.","key_machinery":"The central object is bi-axial attention: instead of flattening every (sensor, time) observation into one long sequence or attending along only one dimension, BAT clones the embedding matrix and runs self-attention along the time axis in one track and along the sensor axis in the other, switching the axis order between tracks. This keeps the sequence length at $T + D$ per track rather than $T \\times D$, avoiding the quadratic blow-up of a dense tuple representation while still letting information flow across both axes. The design also preserves the binary observation mask inside every embedding and learns a separate embedding per sensor identity, which is what later enables sharing sensors across datasets.","core_discovery":"BAT's central claim is that EHR classification improves when a transformer is allowed to attend along both axes of the observation matrix at once. Each observation embedding concatenates the sensor reading with a binary observed/missing indicator, a learned sensor-identity embedding, and a sinusoidal time encoding; the data are then duplicated into two attention tracks that perform self-attention over time points and over sensors in opposite orders, with weights shared inside each track. On sepsis prediction for P19, BAT reports AUPRC 77.05 and AUROC 95.41, ahead of every baseline; on P12 and MIMIC-III mortality it is competitive with GRU-D, the best overall method. The paper also reports that BAT maintains higher AUPRC than two transformer baselines at every induced sparsity level on a human-activity dataset, and that sharing sensor embeddings between P12 and MIMIC-III reduces variance and mostly improves performance. The authors read this as evidence that cross-axis attention and explicit sensor identities address sparsity, informative missingness, and dataset integration.","pith_inferences":["A direct extension the authors leave implicit: because sensor identities are stored as separate embeddings, a model pretrained on one EHR system could be adapted to a new hospital by only learning new sensor embeddings and freezing the rest—this is testable on any pair of datasets with overlapping measurements.","The sparsity result on HAR suggests the advantage may extend beyond EHR to other irregularly sampled multivariate signals such as wearables; the same induced-missingness protocol would settle that.","If the supplementary selection procedure is taken literally—choosing the candidate with highest AUROC on the same five test splits—reported margins over baselines could shrink under a properly nested validation scheme; the authors do not discuss this, so the robustness of the state-of-the-art claim remains an open question."],"forward_implications":["On the P19 sepsis benchmark, BAT reports the best AUPRC and AUROC among all compared models; if the comparison is sound, bi-axial attention is a stronger transformer design for sparse ICU time series than one-axis attention.","Attention visualizations show observed measurements attending strongly to absent time points, supporting the claim that BAT uses informative missingness rather than treating missingness as noise.","In HAR experiments with induced missingness, BAT outperforms both Transformer and iTransformer at every sparsity level up to 99%, where all models reach chance; this points to cross-axis attention as the source of robustness.","Shared sensor embeddings across P12 and MIMIC-III reduce variance and improve most metrics, suggesting that datasets with overlapping sensors can be trained jointly or used for transfer.","The re-implemented baselines and BAT are released in one open-source repository, making the benchmark reproducible."],"supporting_citations":[{"why":"Supplies the transformer architecture and sinusoidal positional encoding that BAT adapts for its embedding.","marker":"[39]"},{"why":"Provides the SeFT tuple representation, the temporal-encoding formula, and the preprocessing used for P12, MIMIC-III, and P19.","marker":"[43]"},{"why":"Introduces axial attention over rows and columns, the mechanism BAT adapts into two attention tracks.","marker":"[53]"},{"why":"Defines GRU-D and documents informative missingness in EHR data, motivating BAT's use of observation masks.","marker":"[27]"},{"why":"Defines iTransformer, the sensor-axis transformer baseline BAT is compared against.","marker":"[46]"},{"why":"Defines IP-Nets, an interpolation-prediction baseline compared on all three EHR tasks.","marker":"[51]"},{"why":"Provides the P19 sepsis-prediction benchmark on which BAT reports state-of-the-art results.","marker":"[54]"},{"why":"Provides the P12 ICU dataset used for mortality classification.","marker":"[14]"},{"why":"Provides the MIMIC-III ICU dataset used for mortality classification and the joint sensor-embedding experiments.","marker":"[52]"},{"why":"Supplies the HAR smartphone-sensor dataset used to measure robustness under induced sparsity.","marker":"[55]"}],"fun_headline_variants":["BAT: dual-axis attention tops sepsis benchmarks","Missingness-aware BAT transformer bests sepsis","BAT attends to time and sensors, wins sepsis","Robust to missingness: BAT tops sepsis, transfers embeddings","Cross-axis attention with missingness flag boosts sepsis"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The reported state-of-the-art sepsis result rests on the assumption that the benchmark comparison is unbiased—in particular, that selecting hyperparameters by best AUROC on the same five splits used to report test performance, and the modified GRU-D and IP-Nets baselines, did not favor BAT.","fun_headline_variants_meta":{"raw":{"variants":["BAT: dual-axis attention tops sepsis benchmarks","Missingness-aware BAT transformer bests sepsis","BAT attends to time and sensors, wins sepsis","Robust to missingness: BAT tops sepsis, transfers embeddings","Cross-axis attention with missingness flag boosts sepsis"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.001087,"raw_usage":{"total_tokens":4552,"prompt_tokens":960,"completion_tokens":3592,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":576,"completion_tokens_details":{"reasoning_tokens":3518}},"tokens_in":576,"tokens_out":3592,"duration_ms":26157,"temperature":1.0,"reasoning_tokens":3518,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-15T17:22:30.802442+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Re-run the P19 comparison with a nested procedure: hold out one test split untouched until after hyperparameter selection, or select hyperparameters using only training and validation data; if BAT's reported 2.32-point AUPRC margin over the second-best model disappears or reverses, the state-of-the-art claim is not established.","supporting_citations":[{"cited_title":"Gomez, Łukasz Kaiser, and Illia Polosukhin","cited_arxiv_id":null,"evidence_quote":"Supplies the transformer architecture and sinusoidal positional encoding that BAT adapts for its embedding."},{"cited_title":"Borgwardt","cited_arxiv_id":null,"evidence_quote":"Provides the SeFT tuple representation, the temporal-encoding formula, and the preprocessing used for P12, MIMIC-III, and P19."},{"cited_title":"itransformer: Inverted transformers are effective for time series forecasting","cited_arxiv_id":null,"evidence_quote":"Defines iTransformer, the sensor-axis transformer baseline BAT is compared against."},{"cited_title":"Interpolation-prediction networks for irregularly sampled time series","cited_arxiv_id":null,"evidence_quote":"Defines IP-Nets, an interpolation-prediction baseline compared on all three EHR tasks."},{"cited_title":"Early prediction of sepsis from clinical data: The Phys- ioNet/computing in cardiology challenge 2019, 2019","cited_arxiv_id":null,"evidence_quote":"Provides the P19 sepsis-prediction benchmark on which BAT reports state-of-the-art results."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Provides the MIMIC-III ICU dataset used for mortality classification and the joint sensor-embedding experiments."}],"review_version":2}