{"id":"039a1627-7022-452c-b760-5a018d2a744f","arxiv_id":"2412.08873","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":10,"one_line_summary":"A causal transformer that predicts each patient's future disease diagnoses repeatedly as their health record grows, producing a continuous risk trajectory over time.","lead":"This paper trains a transformer model on Finnish national health register data so that it makes a new disease-risk prediction at every point in a person's medical history, not just once. The model matches the accuracy of standard predictors while allowing doctors and researchers to watch how risk changes over time.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The claim that the trained model can explore other forecast intervals by changing t2f at inference is undermined by a confound: in training, t2f is a deterministic function of age and birth year, so the model has never seen the same history paired with a different t2f; this core 'conditioned on…","rationale":"The central claim is that Evolve provides per-timestep evolving predictions conditioned on the forecast time without sacrificing accuracy. The accuracy parity with CLS and XGBoost is well supported, and the per-position predictions for the fixed 2016-2020 interval are plausible. But the 'conditioned on the time of the forecast period' property is the novel, load-bearing contribution: without it, the model is just a causal transformer with a repeated-label training objective, still useful for fixed-interval trajectory analysis but not for the claimed generalizability to other forecast intervals. The Discussion explicitly makes this a capability of the trained model. Because t2f is confounded with birth cohort, the inference-time shift is not a valid manipulation as stated. This is not an internal inconsistency in the fixed-interval experiments, but an unsupported external validity claim. The reader identified this as the weakest assumption; we agree and provide the mechanistic reason. The appropriate verdict remains CONDITIONAL: the model's core accuracy results stand, but the trajectory-generalization claim must be demonstrated on a second forecast interval before it can be accepted.","tokens_in":14569,"tokens_out":9737,"duration_ms":99517,"concrete_test":"Train the Evolve model on a forecast interval A (e.g., 2006-2010) using the same architecture; then take the model trained on 2016-2020, recompute t2f for the historical codes relative to A, and compare its AUROC to a model trained directly on A. If the shifted model does not match the retrained model within a small margin (e.g., 0.01 macro AUROC), the inference-time t2f generalization claim fails. An analytical companion check is to train a linear probe on the t2f embeddings to predict (2016 - birth_year); high accuracy would confirm that t2f encodes birth cohort rather than forecast distance.","verdict_should_be":"UNCHANGED","load_bearing_attack":"Section 3.1 introduces t2f (years to the forecast interval) as a learned embedding, and the Discussion claims that 'the trained model should allow exploration of health trajectories relative to other forecast intervals simply by changing t2f at inference time.' This is the only mechanism supporting the abstract's 'conditioned on the time of the forecast period.' However, the training data use a single forecast interval starting 2016-01-01, so for every code t2f = 2016 - code_year and age = code_year - birth_year. Hence t2f = (2016 - birth_year) - age: within each individual, t2f is perfectly collinear with age, with an intercept determined by birth year. The model can therefore learn t2f as a birth-cohort or calendar-time feature, not as a semantic distance to a changeable forecast interval. At inference, shifting t2f by Δ is equivalent to holding age fixed and changing the implied birth year, not to changing the forecast period; the same code sequence has never been observed with two different t2f values, so the model cannot have learned a transferable conditioning signal. The fixed-interval results in Table 2 are unaffected, but the claimed generality of the trajectory modeling is not supported.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"This manuscript introduces Evolve, a decoder-only transformer trained with causal masking on longitudinal Finnish register data, producing multi-label disease-onset predictions at every code position. The model is conditioned on code, age, position, and 'years to forecast interval' (t2f). On a 20% held-out test of about 1.04 million individuals, Evolve's final-position AUROC, AUPRC, and Recall@4 are statistically indistinguishable from the bidirectional CLS transformer and slightly below XGBoost. The remaining sections present qualitative trajectory analyses: sigmoid jumps, embedding-neighborhood changes around bereavement and childbirth, and nearest-neighbor similarity trajectories.","tokens_in":14849,"tokens_out":6656,"duration_ms":74178,"significance":"If the trajectory claims were quantitatively supported, this would be a useful tool for monitoring evolving health trajectories in national register data. The paper deserves credit for a large real-world evaluation with bootstrapped confidence intervals, a reasonable head-to-head comparison, and careful separation of historical and forecast intervals with a buffer. However, the distinguishing contribution is currently supported mainly by selected visualizations and descriptive statistics, and the one general claim about changing the forecast interval is not supported by the training setup.","major_comments":[{"comment":"The Discussion claims that 'the trained model should allow exploration of health trajectories relative to other forecast intervals simply by changing t2f at inference time.' This is not supported by the training setup. Because there is a single forecast interval (2016-2020) for every individual, each code at year y and age a satisfies t2f = 2016 - y = (2016 - birth_year) - a. Thus, within each individual, t2f is perfectly collinear with age, and across individuals it is an encoding of birth cohort. The model never sees the same code history paired with two different t2f values, so the t2f embedding cannot have learned a transferable 'distance to forecast' semantic; changing t2f at inference is an out-of-distribution intervention equivalent to holding age fixed and changing an unobserved calendar or birth-cohort variable. The fixed-interval results in Table 2 are unaffected, but the abstract's 'conditioned on the time of the forecast period' and the transferability claim need either retraction or a new experiment with multiple forecast intervals or varied t2f offsets.","section":"Section 3.1 and Discussion"},{"comment":"The paper's central contribution, 'promising trajectory modeling properties,' is not quantitatively evaluated. Section 5.2's event analysis uses the model's own sigmoid jumps to define important codes; Table 3 reports descriptive frequencies with no comparison to a null model (for example, codes expected by base rate or random jump positions), so the listed associations cannot be distinguished from artifacts of the jump-extraction procedure. Figure 4 shows mean neighborhood-change curves for groups of 18-27 mothers without confidence intervals or a statistical test for the target-control difference, making the claim that bereavement 'seem[s] to have an impact' unverifiable. Figures 5-7 are selected individual examples. I request at least one quantitative validation of the trajectory output, such as a frequency-matched null or permutation test for Table 3, bootstrap confidence intervals and a difference test for Figure 4, or an early-warning metric with time-to-detection compared against a non-evolving baseline.","section":"Section 5.2, Figures 4-7, Table 3"},{"comment":"Table 2 evaluates Evolve only at the final position t = T_n. Since the model's defining property is a prediction at every code position, the paper should report the quality of intermediate predictions, for example AUROC(t) or AUPRC(t) as a function of age, or time until the prediction first exceeds a high-precision threshold. Without this, it is unknown whether the intermediate sigmoid values used in Figures 1, 5, 6, and 7 carry meaningful information beyond a smoothed version of the final prediction. This is a load-bearing gap because the trajectory analyses are predicated on those intermediate outputs.","section":"Section 5.1 and Section 3.1"}],"minor_comments":[{"comment":"The phrase 'our approach enable the model' should be 'our approach enables the model.'","section":"Abstract"},{"comment":"Please define the reference population over which the k-nearest neighbors are computed; with 18-27 target individuals and k=1000, it is unclear whether neighbors are drawn from the whole test set, the validation set, or the matched control group.","section":"Equation (5) and Figure 4"},{"comment":"The description of position-weighted mean pooling should clarify how age embeddings are handled for ages with no recorded codes and whether the same reference individuals are used at every age when computing neighborhood changes.","section":"Section 3.3"},{"comment":"For the CLS baseline, please clarify whether it receives the same input features, including t2f, and how its position and age embeddings are constructed; this matters for the comparability of the Table 2 results.","section":"Section 3.2"},{"comment":"Several class and code labels are abbreviated or inconsistently formatted (for example, 'CHD Heart failure, not strict' and 'Noninflammatory disorders of female genital tract'); mapping these to the class definitions in Appendix B would improve readability.","section":"Table 3 and Table 5"},{"comment":"The statement that class weighting makes the latent embedding space more discriminative is made in the main text's Discussion but the supporting Figure 8 is only in the appendix; consider adding a brief pointer in Section 5 or 6.","section":"Appendix A"}],"recommendation":"major_revision","confidential_remarks":"The main concern is that the manuscript's title and abstract promise trajectory modeling, while the current evidence is largely qualitative and the t2f generalization claim is not supported by the training setup. I recommend major revision rather than rejection because the fixed-interval prediction result is honest and the causal-mask objective may still be useful if intermediate outputs are properly evaluated. The authors should either add multi-window training or remove the transferability claim, and should add at least one quantitative validation of the trajectory properties."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"First, the thing to know: this paper proposes a simple and sensible modification to transformer training for longitudinal EHR data — causal attention mask while always predicting the same set of future diagnoses at every historical code position — and shows that it performs about as well as a bidirectional transformer and slightly worse than XGBoost on a large Finnish registry cohort. That part is honestly done, with a proper test split, bootstrapped confidence intervals, and a fair comparison. The trajectory-oriented analyses built on this training scheme, like tracking sigmoid jumps and embedding neighborhoods over age, are genuinely new and could be useful for early-detection research.\n\nWhat is actually new is the training objective itself and the associated visualization tools. Nothing in the cited BEHRT/Med-BERT/Hi-BEHRT line does exactly this. The paper does not oversell accuracy; it explicitly notes that transformers don't beat simpler models. Credit where due: this is a clean, reproducible setup on a huge dataset, and the authors are clear about their limitations.\n\nThe soft spots are in the added value, not the core comparison. The trajectory analyses are exploratory and lack statistical validation. The sigmoid-jump method just lists codes that co-occur with jumps; there's no baseline or test showing these are more informative than, say, frequent codes in the same age window. The mother-child neighborhood plots have tiny samples and no error bars, so the visual separation is suggestive but not established. These are presented as preliminary, so I don't hold that against the paper much.\n\nThe more serious problem is the claim in the Discussion that you can explore other forecast intervals by changing the 'time to forecast' (t2f) embedding at inference. The training data use a single forecast interval starting 2016, so for every person, t2f = (2016 - birth year) - age. Within an individual, t2f is a deterministic linear function of age. The model has never seen the same code sequence paired with a different t2f. So changing t2f at inference is equivalent to changing the implied birth year, not shifting the forecast period. The fixed-interval results in Table 2 don't depend on this claim, but the generality of the trajectory tool does. The authors should either remove that sentence or demonstrate transfer by retraining with a different forecast interval. A referee should ask for that.\n\nBottom line: for someone working on EHR trajectory modeling, this is worth reading and citing for the training trick. It deserves peer review, but the trajectory evaluation needs to be tightened and the t2f generalization claim either backed or cut.","headline":"Honest, novel training trick for evolving EHR trajectories, but the t2f generalization claim is not supported by the training setup.","tokens_in":15401,"tokens_out":2755,"would_cite":true,"duration_ms":30459,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"The paper claims that a causally masked transformer trained to predict the same future diagnoses at every past health-code event yields evolving risk trajectories that match a bidirectional transformer and classical baselines on…","keywords":["EHR","longitudinal health trajectories","disease prediction","transformers","causal attention mask","trajectory analysis","forecast interval","risk evolution"],"falsifier":"Train the same Evolve architecture on a second, differently positioned forecast interval (e.g., 2010–2014) and check whether the model, with only $t2f$ changed at inference, produces sensible per-position risk trajectories; if the predictions degrade or the embedding must be retrained for each interval, the claimed generality of the evolving-prediction mechanism is refuted.","tokens_in":14362,"feed_emoji":"🩺","tokens_out":10433,"duration_ms":101487,"temperature":0.7,"pith_summary":"This paper tries to show that a transformer can be trained to give a running, per-event prediction of whether a person will develop any of 21 health outcomes in a fixed five-year window, instead of a single prediction at the end of the person's history. The method, called Evolve, applies a causal attention mask and asks the model, at every historical code, to predict the same label set for the forecast interval; the model is conditioned on how many years remain until that interval through a time-to-forecast input. On a nationwide register cohort, Evolve's discrimination is on par with a bidirectional BERT-style transformer and with logistic regression and XGBoost, so the extra trajectory information does not come at a large accuracy cost. The authors then show two uses of the per-time predictions: finding which diagnoses and drug codes most often trigger upward jumps in predicted risk, and measuring how a person's nearest neighbours in the embedding space change from year to year, with shifts visible around childbirth and bereavement. The payoff, if the approach holds, is continuous monitoring of health trajectories and earlier detection of events that precede disease onset.","feed_headline":"One transformer predicts disease risk at every age","feed_subtitle":"It matches bidirectional models on accuracy while revealing when and why a person's health trajectory shifts.","key_machinery":"The load-bearing mechanism is the combination of a causal attention mask with a per-position training objective. At every input position $t$, the model must predict the same label vector $y_n$ for the forecast interval, so the total loss is the average over positions of the binary cross-entropy between $\\hat{y}_n(t)$ and $y_n$; this forces each position to act as a standalone predictor using only the history up to that point. The other essential input is the time-to-forecast embedding, $t2f$, which encodes the number of years from each code to the start of the forecast interval and is summed with code, age, and position embeddings, letting the model condition each per-position prediction on how far the forecast window lies ahead. A sigmoid decision layer on top of the final hidden state at each position converts the representation into evolving class probabilities, and position-weighted mean pooling of the per-position embeddings yields age-wise embeddings for neighbourhood analysis.","core_discovery":"The central claim is that a unidirectional transformer trained with a causal attention mask and a position-wise multi-label objective produces evolving predictions, $\\hat{y}_n(1:T_n)$, where each $\\hat{y}_n(t)$ depends only on the person's health history up to time $t$, and that these predictions perform comparably to a bidirectional CLS transformer while enabling trajectory analysis. For each individual, the model outputs sigmoid probabilities for 19 diagnoses plus death and a 'none' class at every code position; as more of the person's history is observed, these probabilities are intended to track the changing risk of outcomes in the 2016–2020 forecast interval. The paper demonstrates that the largest upward jumps in these sigmoid probabilities are frequently caused by clinically sensible codes (for example, selective $\\beta$-blockers before coronary heart disease and SSRIs before depression), and that the rate at which a person's $k$-nearest neighbours in the embedding space change from one age to the next rises around major life events such as childbirth and the death of a child. The paper positions this as an alternative to change-point detection: instead of looking for discontinuities, it models continuously evolving changes in predictions and in the latent neighborhood.","pith_inferences":["The paper does not test whether the $t2f$ embedding transfers to a second forecast interval, so a direct experiment would be to train Evolve on 2010–2014 and check whether changing $t2f$ at inference still yields sensible per-position trajectories.","The per-position sigmoid jumps could be turned into an individual-level early-warning score by comparing each person's jump magnitudes against their own baseline trajectory; the paper only aggregates jumps across the validation set.","The neighbourhood-change measure $r_n(a,k)$ is presented as a population-level signal, and a natural extension is to test whether large deviations from a person's own average rate of change precede a diagnosis, giving a personalized alarm.","Because the data are coded Finnish register entries, the trajectory properties may be tied to that specific code vocabulary and healthcare system; transfer to other coding systems is an open question the paper does not address."],"forward_implications":["If the central claim holds, a single trained model can display a person's predicted risk for each of 21 outcomes at every point in their recorded history, without retraining for different cutoff times.","Because Evolve matches a bidirectional transformer and classical baselines on AUROC, AUPRC, and Recall@4, the causal mask and position-wise loss do not force a meaningful accuracy trade-off for this task.","The sigmoid-jump analysis can surface early predictors: codes such as beta-blockers before coronary heart disease and SSRIs before depression appear years before the forecast interval, supporting the idea of early-detection screening.","The embedding neighbourhood-change measure rises around life events like childbirth and bereavement, so the model can be used retrospectively to locate when a person's health trajectory shifted.","The time-to-forecast conditioning, if it generalizes, would let the same model be queried for other forecast horizons by changing $t2f$ at inference time, as the paper hypothesizes in its discussion."],"supporting_citations":[{"why":"Supplies the base transformer architecture with multi-head self-attention that Evolve and CLS build on.","marker":"[2]"},{"why":"Provides the BEHRT-style input scheme of code, age, and position embeddings for EHR transformers.","marker":"[21]"},{"why":"Defines the bidirectional BERT-style CLS reference model used for comparison.","marker":"[32]"},{"why":"Introduces the causal attention masking from decoder-style GPT models that Evolve adapts.","marker":"[33]"},{"why":"Provides the XGBoost baseline that Evolve must match on the prediction task.","marker":"[36]"},{"why":"Supplies the nationwide register dataset of coded health histories used for training and evaluation.","marker":"[38]"},{"why":"Introduces the position-weighted mean pooling used to form age embeddings from per-position outputs.","marker":"[37]"},{"why":"Reports the increasing uniqueness of individual health trajectories with age, motivating the nearest-neighbour analysis.","marker":"[43]"}],"fun_headline_variants":["Causal transformer predicts risk at every age","Continuous risk tracking with a causal transformer","Watch disease risk evolve via a transformer","Evolving health trajectories from a causal transformer","Model shows how disease risk shifts over time"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The approach assumes the model has learned a general notion of time remaining until the forecast window rather than simply the distance to the year 2016, because it is trained on only one fixed forecast interval; if that is false, changing the time-to-forecast input at inference would not produce meaningful predictions for other intervals, and the method's main advantage over a fixed-window classifier disappears.","fun_headline_variants_meta":{"raw":{"variants":["Causal transformer predicts risk at every age","Continuous risk tracking with a causal transformer","Watch disease risk evolve via a transformer","Evolving health trajectories from a causal transformer","Model shows how disease risk shifts over time"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000368,"raw_usage":{"total_tokens":2004,"prompt_tokens":1006,"completion_tokens":998,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":622,"completion_tokens_details":{"reasoning_tokens":948}},"tokens_in":622,"tokens_out":998,"duration_ms":11739,"temperature":1.0,"reasoning_tokens":948,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-11T17:28:15.288731+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Train the same Evolve architecture on a second, differently positioned forecast interval (e.g., 2010–2014) and check whether the model, with only $t2f$ changed at inference, produces sensible per-position risk trajectories; if the predictions degrade or the embedding must be retrained for each interval, the claimed generality of the evolving-prediction mechanism is refuted.","supporting_citations":[{"cited_title":"BEHRT: transformer for electronic health records","cited_arxiv_id":null,"evidence_quote":"Provides the BEHRT-style input scheme of code, age, and position embeddings for EHR transformers."},{"cited_title":"BERT: Pre-training of deep bidirectional transformers for language understanding","cited_arxiv_id":null,"evidence_quote":"Defines the bidirectional BERT-style CLS reference model used for comparison."},{"cited_title":"XGBoost: A scalable tree boosting system","cited_arxiv_id":null,"evidence_quote":"Provides the XGBoost baseline that Evolve must match on the prediction task."},{"cited_title":"Data resource profile: Nationwide registry data for high-throughput epidemiology and machine learning (FinRegistry)","cited_arxiv_id":null,"evidence_quote":"Supplies the nationwide register dataset of coded health histories used for training and evaluation."},{"cited_title":"Biological age predictors","cited_arxiv_id":null,"evidence_quote":"Reports the increasing uniqueness of individual health trajectories with age, motivating the nearest-neighbour analysis."}],"review_version":1}