{"id":"3e9156c0-23a1-4e39-91e3-2bc08d969534","arxiv_id":"2505.07431","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":5,"one_line_summary":"A diffusion-denoised heterogeneous graph combined with a KAN-based spatiotemporal transformer is proposed for next medical examination recommendation, along with a new MIMIC-III-derived benchmark dataset and reported state-of-the-art results.","lead":"This paper introduces a two-stage model for recommending which medical examinations a patient should receive next, first denoising the patient's medical record graph with a diffusion model and then fusing spatial and temporal features with a KAN-based transformer. It also introduces a new EHR benchmark dataset, MeExam, and reports the best results on it against ten baselines.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"MeExam entity extraction lacks any reported accuracy, so the benchmark validity and the SOTA claim rest on an unverified NLP pipeline.","rationale":"The reader's weakest assumption identifies the same burden-bearing link: MeExam's construction is never validated quantitatively. I agree that this is the most load-bearing concern because the paper's novelty claims are twofold—a new task/dataset and a new model—and the model's \"state-of-the-art\" status can only be measured on that dataset. If the dataset is noisy, the comparison is meaningless regardless of model architecture. The reader's CONDITIONAL verdict is therefore appropriate: the paper should not be fully accepted until the dataset is released and its extraction quality is demonstrated. I considered other candidate concerns, including the internal inconsistency in Table IV where \"w/o. Diffusion\" achieves HR@10 0.7338 versus Ours 0.7302 despite the text claiming that removing any component invariably diminishes performance. That inconsistency undermines the claim that the diffusion component is beneficial, but it does not directly invalidate the central benchmark comparison. Similarly, the absence of error bars and the lack of a stated validation split for hyperparameter tuning weaken the \"significantly outperforms\" claim but are secondary to dataset validity. Thus the verdict remains CONDITIONAL, unchanged from the reader's assessment.","tokens_in":20213,"tokens_out":8115,"duration_ms":73777,"concrete_test":"Release MeExam along with the MIMIC-III note identifiers used for extraction. Sample 500 patient sequences stratified by entity type and have two independent clinicians (or trained annotators) judge, against the original notes, whether each extracted disease, symptom, and examination is correct, complete, and temporally ordered. Compute entity-level precision, recall, F1, and Cohen's kappa. If entity-level F1 is below 0.90 or kappa below 0.80, or if the two annotators disagree on the merging of synonymous exam names (e.g., \"MRI\" vs. \"Head MRI\"), the benchmark is too noisy to support the SOTA claim; additionally, re-run the main comparison on the cleaned subset to check whether DST-GKAN retains its lead over LightSANs.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim that DST-GKAN significantly outperforms all compared methods for medical examination recommendation depends entirely on MeExam being a faithful extraction of real examination decisions from MIMIC-III notes. Section V describes the pipeline: clinical notes → NEEDLE/ClinicalBERT NER (trained on roughly 5,000 manually annotated labels plus weak labels) → LLM-based cleaning, disambiguation, and standardization. Yet the paper reports no precision, recall, F1, human agreement, or error analysis for any stage of this extraction. The only quality evidence is anecdotal (Table I and one prompt example in Fig. 6). Because MeExam is a brand-new dataset with no external validation, the benchmark itself is the unverified foundation for every subsequent comparison. If the NER or LLM cleaning merges distinct examinations (e.g., \"Head MRI\" and \"MRI\" may be the same exam or different exams in Table I), omits relevant examinations, or misorders temporality, the leave-one-out next-examination prediction task becomes ill-posed and the reported ranking of methods may reflect annotation artifacts rather than clinically meaningful performance. Moreover, the improvement over the strongest baseline (LightSANs: HR@10 0.7302 vs. 0.7069, NDCG@10 0.4646 vs. 0.4540) is small and reported without error bars or statistical tests, so the word \"significantly\" is not supported even before accounting for dataset noise.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper formalizes a new task, medical examination recommendation, and proposes DST-GKAN, a two-stage model that first applies a task-adaptive diffusion process to denoise a heterogeneous patient-entity graph and then learns spatiotemporal representations with a relation-aware graph attention network (RGAT) and a KAN-based Transformer (KANsformer). The authors also construct and release a new benchmark dataset, MeExam, extracted from MIMIC-III notes using a few-shot NER framework (NEEDLE/ClinicalBERT) and LLM-based cleaning. Experiments on MeExam report HR@10 0.7302 and NDCG@10 0.4646, claimed to significantly outperform ten baselines, with additional ablations and hyperparameter sensitivity analyses.","tokens_in":20538,"tokens_out":4938,"duration_ms":47286,"significance":"If the claims hold, this is the first competitive benchmark and method for next-examination recommendation from heterogeneous temporal EHR data, and the proposed architecture—diffusion-based subgraph distillation coupled with a spatiotemporal KANsformer—would be a meaningful addition to sequential and graph-based recommendation. However, the significance is currently undercut by three load-bearing weaknesses: the MeExam extraction pipeline is not validated with any quantitative accuracy measure; the evaluation uses a single split with no error bars or significance tests and hyperparameter tuning without an explicit validation split; and the main ablation table contradicts the prose claim that every component helps, since removing the diffusion module improves HR@10. The architectural novelty is real but the empirical evidence does not yet support the stated conclusions.","major_comments":[{"comment":"The MeExam dataset is the sole basis for all experiments, but its construction pipeline is never quantitatively validated. The paper reports approximately 5,000 manually annotated entity labels for NEEDLE training and an LLM-based cleaning stage, yet it provides no precision, recall, F1, human agreement, or error analysis for either the NER step or the LLM refinement. Without such evaluation, the possibility remains that entity merging/splitting errors, omissions, or temporal misorderings in the extracted sequences make the leave-one-out next-examination prediction task ill-posed. Since the entire SOTA claim rests on MeExam, the authors must report NER accuracy on a held-out annotation set and a quality analysis of the LLM cleaning step before the benchmark can be considered reliable.","section":"Section V-A"},{"comment":"The evaluation protocol is not sufficient to support the word 'significantly' in the RQ1 conclusion. All numbers in Table III come from a single leave-one-out split with no error bars, no multiple random seeds, and no significance tests. Furthermore, the hyperparameters (rebuilt neighbors, RGAT layers, task-adaptive threshold) are tuned by grid search, but no validation split is described; this creates a risk of test-set contamination. The observed advantage over the strongest baseline, LightSANs, is HR@10 0.7302 vs. 0.7069 and NDCG@10 0.4646 vs. 0.4540, which could easily be within run-to-run variance. Please add validation-based hyperparameter selection, report results over multiple seeds with standard deviations, and run paired significance tests.","section":"Section VI-A and VI-B"},{"comment":"The ablation study contradicts the prose. The text in Section VI-C states that 'removing any component invariably diminishes medical examination recommendation efficacy' and that removing the diffusion module causes a decline, but Table IV shows that 'w/o. Diffusion' achieves HR@10 0.7338, which is higher than Ours at 0.7302, and NDCG@10 0.4639 vs. 0.4646. Thus the ablation provides no evidence that the diffusion module is beneficial, and the claim that every component contributes is false as stated. The authors must either reconcile this discrepancy with corrected experiments or temper the claim about the diffusion component's contribution.","section":"Table IV and Section VI-C"},{"comment":"The derivation of the denoised subgraph G_d is underspecified and appears to involve a distributional mismatch. The forward process in Eq. (1)-(3) adds continuous Gaussian noise to x_0, which is the binary adjacency matrix A, and the reverse process in Eq. (7)-(8) predicts x_0 with a squared-error loss. For binary adjacency entries, a Gaussian likelihood is misspecified. More importantly, the paper never defines how the continuous prediction x_0 is converted into the discrete subgraph G_d used by RGAT; the later hyperparameter 'number of patients' rebuilt neighbors' in Section VI-E suggests a top-k thresholding step, but this step is not formalized in the method section. Please specify the exact mapping from predicted x_0 to G_d and justify the Gaussian assumption for binary data, or adopt a discrete diffusion formulation.","section":"Section IV-A"}],"minor_comments":[{"comment":"The first bullet under the ablation study lists 'w/o. cross-attention' but describes it as removing the task-adaptive diffusion model. This label-description mismatch should be corrected, especially because Section VI-D uses 'w/o. cross-attention' for the linear fusion variant.","section":"Section VI-C"},{"comment":"The adjacency matrix A is defined as |U| x (|V|-|U|), but the RGAT in Section IV-B.1 operates on relations r_{m,n} among all entity types. Please clarify how the initial graph is extended to edges between all node types and how relation types are assigned to those edges.","section":"Section III Definition 2"},{"comment":"The quantities ∇L_rec and ∇L'_rec are not defined precisely; 'difference between the current training step and the last step' is ambiguous (loss difference versus gradient norm). Please provide formal definitions so the sign convention in Eq. (10) and the surrounding text can be checked.","section":"Equation (10)"},{"comment":"The text says the threshold range is [0.0, 0.2, 0.4, 0.6, 0.8, 1.0], while Section VI-A lists the grid search range as [0.2, 0.4, 0.6, 0.8, 1.0]. Please unify these ranges.","section":"Section VI-E.3"},{"comment":"There are minor typos: 'V omiting' in Table I, and 'GSCAN' in Table III appears as 'GCSAN' in the baselines list and the reference. Also, for a claimed benchmark contribution, a public dataset URL or hosting plan should be provided; otherwise the dataset contribution is not reproducible.","section":"Various"}],"recommendation":"major_revision","confidential_remarks":"The paper's central contribution is a new task and dataset, but the dataset construction is unvalidated and the experimental protocol is not yet rigorous enough for the claims made. The authors appear capable of addressing the issues (reporting NER accuracy, adding multiple seeds and significance tests, clarifying the subgraph construction). I am not recommending rejection because the core architectural idea is plausible and the benchmark, once validated, could be useful to the community. However, the current ablation contradiction in Table IV is a red flag that the main empirical claim may not survive closer scrutiny; the authors should be asked to re-run or carefully explain that result."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Colleague,\n\nHere's my read on 2505.07431. The paper's genuine contributions are the formalization of medical examination recommendation and the MeExam dataset; the model is a competent recombination of known pieces (diffusion denoising, RGAT, KAN-based transformer, cross-attention), and the writing is clear. I agree with the reader on both counts: novelty is in the task and dataset, not in a new principle.\n\nWhat I value: the problem is real and under-served, the dataset construction is described in enough detail to be reproduced if released, and the two-stage training with task-adaptive feedback to the denoiser is a sensible way to align denoising with the downstream objective.\n\nThe soft spots are real and several land on the central claim. MeExam's extraction pipeline reports no precision, recall, or F1 for either the NER stage or the LLM cleaning stage. Since every downstream comparison lives on that benchmark, an unvalidated extraction is a load-bearing gap. Second, the improvement over LightSANs is small (HR@10 +0.023) and no error bars or significance tests are given, so the word \"significantly\" is doing work it can't support. Third, the ablation is internally inconsistent: removing the diffusion module gives HR@10 of 0.7338 versus 0.7302 for the full model, contradicting the prose claim of a decline, and the \"w/o. cross-attention\" variant is misdescribed as removing the diffusion module. That needs correcting, not just cosmetically. Fourth, grid search is described without a validation split, leaving open test-set contamination. The lack of released code or data also blocks independent checking.\n\nThese are not deep theoretical flaws; the architecture is coherent. But they are exactly the issues that need referee time. I'd send this to peer review with major revision: release the dataset, validate the extraction, report statistics, and reconcile the ablation.\n\nFor a reading group, it's a useful case study in dataset construction pitfalls.","headline":"New task and dataset are the real contributions; the model is a plausible stack, but the evaluation infrastructure (dataset validity, statistics, ablation consistency) is not yet at publishable strength.","tokens_in":21033,"tokens_out":2743,"would_cite":false,"duration_ms":24621,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"The paper introduces DST-GKAN, a two-stage model that denoises a heterogeneous patient graph with a task-adaptive diffusion process and then encodes spatial and temporal structure with graph attention and a KAN-based transformer…","keywords":["medical examination recommendation","electronic health records","heterogeneous graph","diffusion model","Kolmogorov-Arnold Networks","sequential recommendation","temporal modeling","healthcare AI benchmark"],"falsifier":"Have two clinicians independently annotate a random sample of the source notes for diseases, symptoms, and examinations, then compute precision and recall of the dataset's extracted entities against their labels; if recall or precision is low, the benchmark comparisons would measure an artifact of the extraction pipeline. A complementary test is to rerun DST-GKAN and the baselines on a second independently constructed examination dataset and check whether the reported ranking gains persist.","tokens_in":20036,"feed_emoji":"🩺","tokens_out":6063,"duration_ms":53250,"temperature":0.7,"pith_summary":"The paper formalizes a new recommender task: given a patient's chronological record of diseases, symptoms, and prior examinations, predict which medical examination should come next. To support it, the authors build a heterogeneous temporal dataset from real anonymized clinical notes, then propose DST-GKAN, a two-stage model. A task-adaptive diffusion process first strips noise from the patient-entity graph, and a spatiotemporal encoder combining graph attention, a KAN-based transformer, and cross-attention fusion learns the patient representation. On their benchmark the model reports the best HR@10 of 0.7302 and NDCG@10 of 0.4646 against ten sequential-recommendation baselines. If the task and dataset are sound, this opens a new line of diagnostic decision support.","feed_headline":"New model beats 10 baselines at recommending the next medical exam","feed_subtitle":"It pairs a denoising diffusion stage with a spatiotemporal graph-KAN transformer and introduces a new EHR benchmark.","key_machinery":"The load-bearing mechanism is the two-stage DST-GKAN architecture. In stage one, a task-adaptive diffusion model treats the adjacency matrix of the heterogeneous patient-entity graph as the signal to be denoised: a forward process adds Gaussian noise, a reverse process reconstructs the graph, and a gating function that watches the downstream recommendation loss decides whether to keep training the denoiser or freeze it, yielding a denoised subgraph that keeps recommendation-relevant edges. In stage two, a Relation-aware Graph Attention Network aggregates spatial neighbor information over that subgraph; a KANsformer, where KAN means Kolmogorov-Arnold Network (a network with learnable spline activations), encodes the patient's sequence of entities over time; and cross-attention fuses spatial query features with temporal key-value features into the final patient embedding. The recommendation score is an inner product between patient and examination embeddings, trained with cross-entropy.","core_discovery":"The paper's central claim is that medical examination recommendation can be treated as a sequential prediction problem over heterogeneous electronic health records, and that a carefully structured two-stage architecture solves it better than existing general-purpose recommenders. Concretely, DST-GKAN constructs a patient-entity heterogeneous graph, uses a diffusion model to regenerate a recommendation-relevant subgraph, encodes spatial structure with a relation-aware graph attention network, encodes temporal structure with a KANsformer (a Transformer whose feed-forward blocks are replaced by Kolmogorov-Arnold-style learnable spline layers), and fuses the two views by cross-attention. On the introduced MeExam dataset, the authors report state-of-the-art results: HR@10 of 0.7302 and NDCG@10 of 0.4646. Ablation studies indicate that the temporal KANsformer contributes the largest performance share among the model components.","pith_inferences":["Because the paper reports no extraction accuracy for the entities pulled from clinical notes, the benchmark's reliability is untested; a natural extension is to publish entity-level precision and recall alongside a human-agreement study.","The task-adaptive diffusion stage produces a denoised subgraph that could double as an interpretable summary of a patient's relevant conditions if its edge weights were inspected, which the paper does not pursue.","If the benchmark proves reliable, the same two-stage pattern of denoising a heterogeneous graph and then encoding space and time may transfer to other clinical prediction tasks such as procedure or imaging orders.","Even at the reported best, the model places the correct examination in the top ten only about 73 percent of the time, so clinical deployment would require calibration and explicit handling of rare examinations."],"forward_implications":["Medical examination recommendation becomes a concrete, benchmarked task: predicting the next examination from a patient's chronological disease-symptom-examination history.","On the introduced MeExam dataset, DST-GKAN reports HR@10 of 0.7302 and NDCG@10 of 0.4646, surpassing the ten general-purpose sequential recommenders tested, with the closest baseline at HR@10 of 0.7069.","Ablations show that removing the temporal KANsformer costs more performance than removing the graph encoder, indicating that temporal modeling carries much of the predictive signal.","The task-adaptive diffusion mechanism improves results over a fixed denoiser, and the reported sensitivity sweep finds a threshold of 0.2 gives the best performance.","The dataset itself, with 37,774 patients and 701 examination items, is offered as a public resource for future work on this task."],"supporting_citations":[{"why":"It supplies the few-shot named-entity recognition training procedure used to extract medical entities from clinical notes.","marker":"[58]"},{"why":"It provides the clinical language model used as the backbone for the named-entity recognition stage.","marker":"[59]"},{"why":"It supplies the source clinical notes database from which the dataset entities are extracted.","marker":"[40]"},{"why":"It provides the denoising diffusion probabilistic model formalism that grounds the task-adaptive denoising stage.","marker":"[6]"},{"why":"It introduces the Kolmogorov-Arnold Network architecture that the paper adapts into the KANsformer temporal encoder.","marker":"[8]"},{"why":"It supplies the self-attention mechanism at the core of the KANsformer and the cross-attention fusion module.","marker":"[48]"},{"why":"It provides a recurrent sequential recommender used as one of the comparison baselines.","marker":"[28]"},{"why":"It provides a self-attentive sequential recommender used as a comparison baseline and as a reference for Transformer-based sequence encoding.","marker":"[34]"},{"why":"It provides a bidirectional Transformer sequential recommender used as a comparison baseline and as the source of the final-vector sequence pooling strategy.","marker":"[35]"},{"why":"It provides a diffusion-based recommender baseline, allowing the paper to isolate the contribution of its task-adaptive diffusion design.","marker":"[38]"}],"fun_headline_variants":["Diffusion-graph KAN model beats 10 baselines in exam prediction","Denoising diffusion plus graph KANsformer predicts next medical exam","MeExam dataset and diffusion-graph KANsformer achieve SOTA results","Graph attention plus Kolmogorov splines: new SOTA for exam recommendation","Diffusion-denoiser + graph KAN transformer: top exam recommendations"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The whole evaluation rests on the assumption that the automatically extracted entities in the new dataset faithfully reflect the examinations, diseases, and symptoms that actually occurred in the source clinical notes, but extraction accuracy is asserted rather than measured.","fun_headline_variants_meta":{"raw":{"variants":["Diffusion-graph KAN model beats 10 baselines in exam prediction","Denoising diffusion plus graph KANsformer predicts next medical exam","MeExam dataset and diffusion-graph KANsformer achieve SOTA results","Graph attention plus Kolmogorov splines: new SOTA for exam recommendation","Diffusion-denoiser + graph KAN transformer: top exam recommendations"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.001035,"raw_usage":{"total_tokens":4388,"prompt_tokens":1004,"completion_tokens":3384,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":620,"completion_tokens_details":{"reasoning_tokens":3285}},"tokens_in":620,"tokens_out":3384,"duration_ms":24242,"temperature":1.0,"reasoning_tokens":3285,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-15T22:16:44.978520+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Have two clinicians independently annotate a random sample of the source notes for diseases, symptoms, and examinations, then compute precision and recall of the dataset's extracted entities against their labels; if recall or precision is low, the benchmark comparisons would measure an artifact of the extraction pipeline. A complementary test is to rerun DST-GKAN and the baselines on a second independently constructed examination dataset and check whether the reported ranking gains persist.","supporting_citations":[{"cited_title":"Named entity recognition with small strongly labeled and large weakly labeled data,","cited_arxiv_id":null,"evidence_quote":"It supplies the few-shot named-entity recognition training procedure used to extract medical entities from clinical notes."},{"cited_title":"Publicly available clinical BERT embeddings,","cited_arxiv_id":null,"evidence_quote":"It provides the clinical language model used as the backbone for the named-entity recognition stage."},{"cited_title":"Session-based recommendations with recurrent neural networks,","cited_arxiv_id":null,"evidence_quote":"It provides a recurrent sequential recommender used as one of the comparison baselines."},{"cited_title":"Bert4rec: Sequential recommendation with bidirectional encoder representations from transformer,","cited_arxiv_id":null,"evidence_quote":"It provides a bidirectional Transformer sequential recommender used as a comparison baseline and as the source of the final-vector sequence pooling strategy."},{"cited_title":"Diffusion recommender model,","cited_arxiv_id":null,"evidence_quote":"It provides a diffusion-based recommender baseline, allowing the paper to isolate the contribution of its task-adaptive diffusion design."}],"review_version":1}