{"id":"dada7301-0477-42ed-8244-7f41af8087a3","arxiv_id":"2501.16373","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":6,"one_line_summary":"UDC aligns text descriptions and EHR co-occurrence signals in a shared discrete code space, improving rare-disease diagnosis and medication prediction across three ICU datasets.","lead":"A new prediction model called UDC improves diagnosis and medication recommendations for rare diseases by translating disease text descriptions into the same collaborative signal space derived from common diseases. It uses discrete, code-like representations so rare diseases can borrow knowledge from common ones, and it can be plugged into existing healthcare prediction models.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"No significant objection identified: the central rare-disease transfer claim is coherent, but the absence of error bars and the single-dataset rare-disease analysis leave its statistical strength unverified.","rationale":"The paper's central claim is that UDC improves healthcare prediction for rare diseases by enriching their representation semantics via text-to-CO mapping. The mechanism is internally coherent: a PCM is trained, a DRL aligns text and CO representations using only common diseases, and rare-disease embeddings are replaced with text-derived reconstructions. The ablations (Table 3) and plug-in experiments support the design at the level of point estimates, and the group analysis (Figure 3) on MIMIC-III shows the intended rare-disease benefit. However, the decisive empirical evidence for the rare-disease subgroup is thin: Figure 3 is presented without error bars, seed counts, or statistical tests, and the same analysis is not repeated for MIMIC-IV or eICU. Given the reader's conditional verdict already hinges on exactly this assumption, my stress-test confirms rather than overturns it. The most load-bearing concern is therefore the statistical robustness of the rare-disease subgroup gains, not a flaw in the method's logic. A concrete test—repeating the group analysis on the other two datasets with confidence intervals—would settle whether the claim generalizes. I therefore recommend keeping the conditional verdict.","tokens_in":22564,"tokens_out":1375,"duration_ms":11715,"concrete_test":"Report the G1 (rarest) subgroup Acc@K/Jaccard for UDC and the strongest baselines on MIMIC-IV and eICU, with 5 or more random seeds and 95% confidence intervals. If G1 gains over the best baseline are not significant on at least one additional dataset, the rare-disease claim is not established beyond MIMIC-III point estimates.","verdict_should_be":"CONDITIONAL","load_bearing_attack":"The reader's weakest assumption is the load-bearing point: DRL is trained only on D_com (Section 3.5, after Eq. 14), then Eq. 15 applies the common-disease-trained decoder psi_co to rare-disease text embeddings. The success of UDC's central claim therefore depends on whether a Text-to-CO mapping learned purely on common diseases transfers to the tail of the distribution. The paper does not provide an explicit quantitative analysis of exactly this transfer: Figure 3 shows group analysis on MIMIC-III only, and the main tables report aggregate metrics. If the rare-disease gains in Figure 3 are within run-to-run noise (no error bars or seeds are reported anywhere), the headline claim that UDC 'significantly enriched the semantics of rare diseases' is not statistically supported. In addition, the manuscript uses a single threshold eta=20% and a single codebook size (|C_l|=64); the appendix hyperparameter study (Figure 9-10) reports only MIMIC-III Diag Pred, not the rare-disease subgroup. The weakness is not an internal inconsistency: the mechanism is plausible and ablations are consistent. But the empirical basis for the rare-disease claim is a single dataset, a single split, and point estimates without variance, so the central claim is under-evidenced rather than contradicted.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes UDC, a discrete representation learning framework for healthcare predictions that aims to improve rare-disease performance. UDC first trains a standard collaborative model (PCM) to obtain CO embeddings, then trains a residual-quantization VQ-VAE (DRL) that reconstructs and aligns CO and text (Sap-BERT) embeddings in a shared codebook, using condition-aware calibration, task-aware contrastive learning with synthetic and mixed-domain hard negatives, and co-teacher distillation for codebook updates. The DRL is trained only on common diseases, and at inference time rare-disease text embeddings are mapped through the common-trained decoder to replace the original CO embeddings. The framework is evaluated on diagnosis prediction and medication recommendation over MIMIC-III, MIMIC-IV, and eICU, reporting consistent improvements over a broad set of baselines, along with ablations, plug-in experiments with different PCMs and PLMs, and a case study.","tokens_in":22864,"tokens_out":2749,"duration_ms":22513,"significance":"If the reported results hold, UDC offers a practically useful and conceptually appealing way to leverage textual descriptions as a bridge for rare diseases in EHR prediction, and its design as a plug-in module means it could benefit several existing architectures. The manuscript is strong in scope: it evaluates across three datasets and two tasks, compares against many recent baselines, and includes ablations that isolate condition-aware calibration, task-aware calibration, hard-negative sampling, and co-teacher distillation. The code is promised on GitHub, and the method is presented as model-agnostic. However, the central claim of significantly enriching rare-disease semantics currently rests on point estimates without variance or significance testing, and the rare-disease-specific analysis is shown for only one dataset, so the statistical strength of the headline claim is not yet established.","major_comments":[{"comment":"All reported results are single-run point estimates with no standard deviations, confidence intervals, or significance tests across seeds or test-set resamples. The paper's key contribution is worded as 'significantly enriched the semantics of rare diseases' (Section 1), but the improvements over the strongest baselines are often modest (for example, Diag Pred Acc@K gains of roughly 1–2 points on MIMIC-III and MIMIC-IV, and 0.5–1 point on several Med Rec metrics). Without repeated runs or a paired significance test, these margins cannot be distinguished from run-to-run noise. I ask the authors to report mean and standard deviation over at least five random seeds for the main tables and to run a paired significance test (e.g., Wilcoxon signed-rank on per-admission metrics or bootstrap over admissions) for the UDC-vs-strongest-baseline comparisons.","section":"§4.1–§4.2, Tables 1–2"},{"comment":"The group-level rare-disease analysis, which is the direct evidence for the paper's central claim, is presented only for MIMIC-III and without any error bars or repeated-seed information. The main tables report aggregate metrics, so the rare-disease subgroup gains are not quantified on MIMIC-IV or eICU. Since the paper's title and contribution center on rare-disease prediction, the authors should provide group-level results (e.g., the G1–G5 breakdown used in Figure 3) for at least MIMIC-IV and ideally eICU, with variance estimates. Without this, the claim that UDC improves rare-disease predictions is supported by exactly one dataset and one split, which is insufficient evidence for the headline.","section":"§4.3.2, Figure 3"},{"comment":"The DRL is trained exclusively on common diseases (D_com), and rare-disease text embeddings are mapped through the common-trained decoder psi_co in Eq. (15). This transfer is the load-bearing assumption of the method: a Text→CO mapping learned on the head of the distribution must generalize to the tail. The manuscript does not directly test this assumption. I request a quantitative transfer analysis, for example: (a) reconstruction error (Eq. 14 components) for rare versus common diseases, (b) an ablation in which the DRL is trained on all diseases (D_com ∪ D_rar) instead of only D_com, and (c) rare-disease subgroup performance with this alternative training. This would show whether the rare-disease gains come from the common-to-rare transfer or simply from training the DRL on more data.","section":"§3.5, Eq. (14)–(15)"}],"minor_comments":[{"comment":"The text refers to 'UDC-CO' as the configuration without condition-aware calibration, but Table 3 labels this configuration 'UDC-NCO'. Please make the naming consistent.","section":"§4.3.1"},{"comment":"The case study shows 'before DRL' and 'after DRL' representations but does not specify the visualization method (e.g., t-SNE, PCA) or how the plotted points are colored/selected. Please add this information so the figure can be interpreted precisely.","section":"§4.3.4, Figure 6"},{"comment":"The sentence 'We retain patients with more than one visit in MIMIC-III and eICU, while for MIMIC-IV, we include patients with two or more visits' appears to say the same thing twice. Please clarify the intended inclusion criteria.","section":"§4.1, Datasets"},{"comment":"The related work section states that our method 'aligns with the last genre' of generative retrieval and extends VQ-VAE, but the connection to generative retrieval is not made explicit in the method or experiments. A brief sentence connecting the discrete-code reconstruction to generative retrieval would help situate the contribution.","section":"§2.2"},{"comment":"The hyperparameter study reports only MIMIC-III diagnosis-prediction results and does not break down performance by rare-disease group. Since the central claim concerns rare diseases, adding the G1-group or a rare/common split to at least the eta sensitivity analysis would strengthen the robustness story.","section":"Appendix E.2, Figures 9–10"}],"recommendation":"major_revision","confidential_remarks":"The paper is a plausible and well-scoped contribution, but the central rare-disease claim needs stronger empirical support before it can be accepted. The absence of any repeated-seed or significance testing is surprising for a venue like WWW and should be addressed head-on. I would also encourage the editor to check whether the GitHub link is live and whether the code indeed reproduces the reported numbers, since the paper's reproducibility claim is part of its value."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Zhao et al. report a method for improving rare-disease prediction in ICU records: UDC trains a discrete VQ-VAE to align text descriptions (from Sap-BERT) with collaborative signals (from a pretrained PCM) at the code level, then swaps in text-derived embeddings for rare diseases at inference. The central idea—learn the text-to-collaborative mapping on common diseases only, then apply it to the tail—is coherent, and the paper is the first to apply this discrete-code alignment strategy in healthcare prediction. The experiments are broad: three datasets, two tasks, many baselines, ablations, plug-in tests with different backbones. The reported margins over the strongest baselines are consistent, if modest (roughly 1-3 points in most metrics). The ablations support each of the three main components (condition-aware calibration, task-aware contrastive loss, co-teacher codebook updates), which is rare in this literature.\n\nThe soft spots are exactly where the reader put them. First, there are no error bars or significance tests anywhere. Every number is a single run. Given the size of the reported gains, the headline claim that UDC 'significantly enriched the semantics of rare diseases' is not statistically supported. Second, the rare-disease subgroup analysis is only shown for MIMIC-III. The transfer from D_com to D_rar is the load-bearing assumption, and the single-dataset group figure is the closest thing to a direct test of it. Third, the code repository appears to be a placeholder; the footnote points to a README rather than a working repo. That matters for a method paper.\n\nNone of these are internal contradictions. The mechanism is plausible and the ablations hang together. The concern is under-evidence, not evidence of error. The authors would need multi-seed runs with variance, a subgroup analysis on at least one more dataset, and a real code drop to make the central claim convincing.\n\nThis is a solid method paper for healthcare ML people working on long-tailed EHR prediction. It is not a breakthrough, but it is a genuine, well-organized contribution. I would send it to peer review and ask for the additional evidence. If the results hold under proper statistical reporting, this becomes a useful citeable method.","headline":"A solid, well-executed method paper for rare-disease prediction whose central claim rests on single-run point estimates and a one-dataset subgroup analysis; deserves peer review but needs statistical rigor and a working code repo.","tokens_in":23344,"tokens_out":2529,"would_cite":true,"duration_ms":23072,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"This paper claims that rare disease prediction is substantially improved by transferring text knowledge into collaborative signal space through a discrete VQ-VAE variant, replacing sparse disease embeddings with reconstructed text-guided…","keywords":["discrete representation learning","healthcare prediction","rare disease","vector quantization","contrastive learning","co-teacher distillation","medication recommendation","diagnosis prediction"],"falsifier":"If UDC's rare-disease gains come from the Text-to-CO transfer, then feeding the frozen DRL scrambled or permuted text embeddings for rare diseases should collapse the reported G1 improvement to near baseline levels; a concrete test is to evaluate the authors' released code with shuffled rare-disease text descriptions on MIMIC-IV and see whether the Acc@K gain on the rarest 20% of diseases disappears.","tokens_in":22388,"feed_emoji":"🏥","tokens_out":3389,"duration_ms":31642,"temperature":0.7,"pith_summary":"The paper tries to show that the poor predictive performance of electronic health record (EHR) models on rare diseases can be repaired by using clinical text as a bridge into the space of collaborative signals. It proposes UDC, a discrete representation learning framework that aligns textual descriptions of diseases with their collaborative embeddings in a shared codebook, then substitutes the unreliable rare-disease embeddings with text-derived reconstructions. If correct, the method would lift worst-case performance on the rarest disease groups while preserving or improving average accuracy, and it would do so without changing the underlying prediction model. The paper reports state-of-the-art results on diagnosis prediction and medication recommendation across three datasets, with the largest gains concentrated in the rarest disease groups.","feed_headline":"Rare disease forecasts improve via text-to-code transfer","feed_subtitle":"A discrete VQ-VAE variant maps clinical text into collaborative space, lifting prediction on the rarest disease groups across three EHR…","key_machinery":"The central mechanism is the discrete representation learning (DRL) module built on RQ-VAE, which maps both collaborative and textual disease embeddings into the same residual codebook of four code levels. Condition-aware calibration modulates each quantized code by normalized multi-head attention over the procedures and medications co-occurring in the same visit, making similar text descriptions produce distinct codes. Task-aware calibration adds contrastive losses whose negative samples are synthetic targets (randomly substituted next-visit entities) and mixed-domain targets (text counterparts of collaborative samples), pushing reconstructions toward task-relevant distinctions. Co-teacher distillation updates each codebook vector as an exponential moving average of the aggregated representations of both domains, with mutual cross-attention terms that enforce a shared code semantics. The final mapping $\\hat{e}_d = \\psi_{co}[\\varphi(\\phi_{te}(\\tilde{e}_d); e_p, e_m)]$ for rare diseases $d \\in \\mathcal{D}_{rar}$ and the analogous reconstruction for common diseases transform text embeddings into collaborative-space embeddings that then replace the original embeddings during fine-tuning.","core_discovery":"UDC establishes that a tailored VQ-VAE variant, trained only on common diseases, can learn a Text-to-CO mapping that enriches rare disease representations enough to significantly improve downstream healthcare predictions. The framework refines the vector quantization process with condition-aware calibration, which injects visit-level co-occurring entities to distinguish clinically different diseases with similar text, and task-aware calibration, which uses synthetic and mixed-domain hard negatives to keep reconstructions relevant to the prediction target. A co-teacher distillation updates the shared codebook by aggregating both textual and collaborative signals, aligning them at the code level. After training, the model substitutes each rare disease's original collaborative embedding with the reconstruction obtained from its text description, then fine-tunes the prediction model. Experiments on MIMIC-III, MIMIC-IV, and eICU report the best Acc@K, Pres@K, AUPRC, and AUROC for diagnosis prediction and Jaccard, F1, AUPRC, and AUROC for medication recommendation, with group analysis showing the largest relative gains on the rarest disease group.","pith_inferences":["The central assumption that a Text-to-CO map trained on common diseases transfers to rare diseases could be tested directly by evaluating UDC on rare diseases whose text descriptions are deliberately degraded; if the reported gains persist, the transfer is not the active mechanism.","The condition-aware calibration idea, treating co-occurring entities as contextual conditions for quantized representations, could generalize to other long-tail prediction problems beyond healthcare, such as sparse item recommendation or rare event forecasting.","Co-teacher distillation in a shared discrete space may be a general recipe for aligning two views of the same entities (e.g., text and behavior) under severe label sparsity, and its ablation sensitivity suggests the codebook update is a key bottleneck for such alignment.","The reported performance drop on the most common disease group (G5) hints that the method may trade some accuracy on high-frequency, low-specificity diagnoses for gains on rarer ones; whether this trade-off is clinically desirable depends on the deployment's cost structure."],"forward_implications":["UDC improves diagnosis prediction and medication recommendation across all three datasets, with the largest gains on the rarest disease group (G1), according to the reported group analysis.","The framework is plug-in: it works with different collaborative prediction backbones (GRU, Transformer, multi-head attention) and different clinical language models (Sap-BERT, BioGPT, Clinical-BERT), so the gains are not tied to one architecture.","The authors report that DRL training collapses when trained jointly with the prediction model, and that skipping fine-tuning loses most of the benefit, implying that a staged pipeline is necessary for the method to work.","The method achieves competitive time complexity, making it feasible to apply in large-scale EHR settings.","Ablations show that all three components—condition-aware calibration, task-aware calibration, and co-teacher distillation—contribute, with condition-aware calibration the most impactful for diagnosis prediction."],"supporting_citations":[{"why":"Provides the residual quantized VQ-VAE (RQ-VAE) framework that UDC extends with condition-aware and task-aware calibrations and co-teacher distillation.","marker":"[22]"},{"why":"Introduces VQ-VAE-2, the base discrete modeling approach whose vector quantization process UDC refines for healthcare.","marker":"[39]"},{"why":"Supplies Sap-BERT, the clinical pre-trained language model UDC uses to obtain text embeddings of disease descriptions.","marker":"[29]"},{"why":"The MIMIC-III EHR dataset used in all experiments for both diagnosis prediction and medication recommendation.","marker":"[18]"},{"why":"The MIMIC-IV EHR dataset, the most challenging and imbalanced of the three test beds.","marker":"[17]"},{"why":"The eICU dataset, used to test generalizability to a different critical-care EHR source without standardized coding.","marker":"[37]"},{"why":"GraphCare, a strong baseline that also uses external knowledge; UDC is compared against it and shown to improve on it.","marker":"[15]"},{"why":"RAREMed, a recent rare-disease-focused baseline that UDC outperforms and contrasts with in the group analysis.","marker":"[63]"}],"fun_headline_variants":["Text-to-code trick boosts rare disease predictions","Rare disease AI gets a boost from text knowledge","Discrete clues: how text improves rare disease forecasts","New method maps text to codes for better rare disease prediction","Unveiling discrete clues for superior rare disease predictions"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The DRL is trained only on common diseases, so the method assumes that a text-to-collaborative mapping learned on common diseases transfers to rare diseases despite the distribution shift, and that Sap-BERT text embeddings of rare disease descriptions are informative enough to reconstruct useful collaborative representations.","fun_headline_variants_meta":{"raw":{"variants":["Text-to-code trick boosts rare disease predictions","Rare disease AI gets a boost from text knowledge","Discrete clues: how text improves rare disease forecasts","New method maps text to codes for better rare disease prediction","Unveiling discrete clues for superior rare disease predictions"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000194,"raw_usage":{"total_tokens":1375,"prompt_tokens":985,"completion_tokens":390,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":601,"completion_tokens_details":{"reasoning_tokens":315}},"tokens_in":601,"tokens_out":390,"duration_ms":3851,"temperature":1.0,"reasoning_tokens":315,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-10T16:14:17.748632+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"If UDC's rare-disease gains come from the Text-to-CO transfer, then feeding the frozen DRL scrambled or permuted text embeddings for rare diseases should collapse the reported G1 improvement to near baseline levels; a concrete test is to evaluate the authors' released code with shuffled rare-disease text descriptions on MIMIC-IV and see whether the Acc@K gain on the rarest 20% of diseases disappears.","supporting_citations":[{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Provides the residual quantized VQ-VAE (RQ-VAE) framework that UDC extends with condition-aware and task-aware calibrations and co-teacher distillation."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Introduces VQ-VAE-2, the base discrete modeling approach whose vector quantization process UDC refines for healthcare."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Supplies Sap-BERT, the clinical pre-trained language model UDC uses to obtain text embeddings of disease descriptions."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"The eICU dataset, used to test generalizability to a different critical-care EHR source without standardized coding."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"GraphCare, a strong baseline that also uses external knowledge; UDC is compared against it and shown to improve on it."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"RAREMed, a recent rare-disease-focused baseline that UDC outperforms and contrasts with in the group analysis."}],"review_version":1}