{"id":"db8e2e95-28f3-44c1-835a-6c779780a45c","arxiv_id":"2412.11671","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":4.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":2,"one_line_summary":"BioBridge improves emergency triage classification on Korean-English code-switched EMRs by adding language segment tokens and BioSent2Vec medical features to transformer encoders, with modest gains over baselines.","lead":"A new framework called BioBridge adds language markers and medical word embeddings to pretrained language models to classify emergency cases from Korean-English mixed hospital notes. It reports small accuracy and calibration gains over standard models, but the single-hospital dataset and tuning choices limit the strength of the findings.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Test-set hyperparameter selection in Section IV-F and Section V invalidates the reported held-out comparisons; without a dev-only re-run and significance testing, the claimed BioBridge improvements are unsupported.","rationale":"I agree with the reader that label leakage is a serious threat to the clinical interpretation of the task, but the most load-bearing issue for the paper's central quantitative claim is the evaluation protocol. The paper's own text says the optimal hyperparameters were chosen 'for the test set', which makes the reported test numbers optimistic by construction. The effect sizes are close to the noise level expected from fine-tuned transformers, and no significance testing is provided, so the claimed 'significant' improvement is not established. The proposed check directly settles this: if the gains persist under dev-only hyperparameter selection with multiple seeds, the quantitative claim survives; if not, it does not. I therefore recommend rejecting the current version, while noting that the flaw is correctable by redoing the experiments under a valid protocol and, separately, by analyzing whether PI notes mention the interventions used to define the labels.","tokens_in":16826,"tokens_out":7405,"duration_ms":78936,"concrete_test":"Obtain the released code and rerun the full comparison with a corrected protocol: (1) keep the same train/dev/test split; (2) for each model, perform the learning-rate grid search on the development set only and select the learning rate by dev F1; (3) train each configuration with five random seeds using that fixed learning rate; (4) evaluate on the test set exactly once, reporting mean and 95% confidence intervals for F1, AUROC, AUPRC, and Brier. The concern lands if BioBridge-XLM's +0.85% F1 advantage over XLM falls within the confidence interval or reverses, or if any BioBridge gain over its baseline is no longer consistent across seeds.","verdict_should_be":"REJECT","load_bearing_attack":"The most load-bearing concern is the evaluation protocol described in Section IV-F. The paper states that a grid search over learning rates was run 'for all experiments' and that 'the optimal hyperparameters for the test set are shown in Table 2'; the ablation section repeats the same wording for Table 4. Taken literally, this means the test set was used to select hyperparameters, so the comparisons in Tables 3 and 5 are not independent held-out evaluations. The reported improvements are small (e.g., +0.85% F1, +0.75% AUROC, +0.76% AUPRC for BioBridge-XLM) and no confidence intervals, error bars, or multiple-seed results are reported. The dev-set checkpoint selection mentioned in Section IV-F does not repair hyperparameter selection on the test set. If the wording is a mistake, the paper must clarify; if it is not, the central claim of significant improvement is unsupported as written.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes BioBridge, a two-module framework for classifying emergency versus non-emergency cases from Korean-English code-switched Present Illness (PI) notes in a pediatric emergency department. The first module, 'bridging modality in context,' inserts segment tokens [B-K] and [B-E] to separate Korean and English spans in the input. The second module, 'unified bio-embedding,' extracts English medical word features with a fixed BioSent2Vec encoder, projects them through a fully connected layer, and integrates them into a pre-trained transformer encoder. Experiments on a private dataset of 87,759 PI notes compare four machine-learning baselines and five encoder-based models (KR-BERT, KoBERT, mBERT-cased/uncased, XLM, XLM-R), reporting small F1, AUROC, AUPRC, and Brier-score improvements for BioBridge variants; the headline result is BioBridge-XLM with +0.85% F1 over XLM. An ablation study decomposes the two modules. The paper also describes a preprocessing method that decodes common medical abbreviations and symbols.","tokens_in":16944,"tokens_out":7914,"duration_ms":71955,"significance":"If the findings were valid, BioBridge would be a useful early exploration of segment-token-based language-modality separation and BioSent2Vec fusion for code-switched EMR classification, an understudied problem. The framework is simple, and the paper targets a practically relevant decision-support task with a large private dataset. However, the reported improvements are small (at most 1.1% F1), and the evaluation protocol is compromised as written by explicit test-set hyperparameter selection; no confidence intervals or significance tests are provided. The method definition also omits the exact fusion operation and the handling of multiple code-switches. The paper states that source code will be made public, which is commendable, but the current evidence does not support the central claim of significant improvement.","major_comments":[{"comment":"Section IV-F states that a grid search over learning rates was run 'for all experiments' and that 'the optimal hyperparameters for the test set are shown in Table 2'; Section V repeats the same wording for Table 4. Taken literally, this means the test set was used to select hyperparameters, so the results in Tables 3 and 5 are not independent held-out evaluations. The reported gains are small (e.g., BioBridge-XLM +0.85% F1, +0.75% AUROC), and no confidence intervals, error bars, or multiple-seed results are given. Please re-run the experiments selecting hyperparameters on the development set only (or nested cross-validation), report test metrics for the dev-selected configuration, and include at least five seeds and a significance test (e.g., bootstrap or paired test) for the headline comparisons. If the wording is a mistake, correct it and explain the actual selection protocol.","section":"Section IV-F, Tables 2 and 4; Section V"},{"comment":"The emergency label is defined by whether the patient received interventions such as blood tests, urinalysis, IV hydration, nebulizer treatment, immediate drugs, or admission (Section IV-C), while the predictive input is the free-text PI note (Section IV-B). The paper does not establish that the PI note was written before these interventions or that intervention-related mentions are excluded from the input. If the notes contain phrases such as 'IV started' or 'admitted', the model can learn to detect documented care rather than predict the need for it, which would invalidate the decision-support claim. Please provide evidence of temporal ordering or a redaction protocol, and report how performance changes when intervention-related terms are removed from the input (e.g., a vocabulary-based ablation).","section":"Section IV-C and Section IV-B"},{"comment":"The fusion operation for the unified bio-embedding is not specified. Equation (7) only states that f_theta is in R^{m x h_M}, and the text says the projected BioSent2Vec features are 'integrated into' the encoder, but it does not state whether they are added to the token embeddings of English subword tokens, concatenated with the encoder's hidden states, or combined in another way. The position-alignment procedure from m word-level features to the token-level sequence is also missing. Please give the exact fusion equation, including the role of the [B-E] spans, and the alignment rule; without this, the method is not reproducible.","section":"Section III-B2, Eq. (7)"},{"comment":"The formal definition of the bridging-modality input, Eq. (5), covers only a single Korean-to-English switch: x_bri = {[CLS], [B-K], [tokens]_kor, [B-E], [tokens]_eng, [SEP]}. The real PI notes shown in Figure 1 (and implied by the statistics in Table 1) contain multiple alternating Korean and English spans, e.g., '환아가 vomiting 10 회로 emergency department 내원'. The paper does not define how [B-K] and [B-E] tokens are inserted for arbitrary multi-switch sentences, nor how the language-span boundaries are detected. Please specify the span-detection and token-insertion algorithm, or restrict the method and experiments to the single-switch case.","section":"Section III-B, Eq. (5) and Figure 1"}],"minor_comments":[{"comment":"The sentence 'the proposed BioBridge significantly performance traditional machine learning and pre-trained encoder-based models' contains a grammatical error ('performance' should be 'outperforms'); also, 'significantly' is not supported by significance tests.","section":"Abstract"},{"comment":"The F1 threshold is described as set at 0.595 'to align with the label frequency ratio of the label 1 ratio'. Please state explicitly whether this ratio is computed from the training, development, or test set; if it is computed from the test set, it is another test-set-dependent choice affecting the F1 comparison.","section":"Section IV-E"},{"comment":"The heading 'Uninifed bio-embedding' contains a typo ('Uninifed' should be 'Unified'). There are also typographical errors elsewhere, including 'MIMMIC-III' in Section II-A (should be 'MIMIC-III') and 'Herhert M Adler' in reference [76]; please proofread the manuscript.","section":"Section V.A"},{"comment":"The column header 'Param' appears to be an artifact; the tables show only Model, Batch Size, and Learning Rate. Please remove or fill that column so the tables are unambiguous.","section":"Tables 2 and 4"},{"comment":"The narrative around Table 5 selectively highlights improvements: for BioBridge-XLM-Rbase the Brier score worsens from 18.78 to 19.59 relative to the XLM-Rbase baseline, and the 'w/ Bio-embedding' variant's Brier score is 23.33. The text says the module 'consistently enhanced performance on almost all metrics' but does not discuss these negative cases; please report and discuss all results transparently.","section":"Section IV-G and Table 5"},{"comment":"The conclusion claims 'state-of-the-art performance' but the comparisons are only against internal baselines; no prior code-switched EMR classification systems are compared. Please temper the claim or provide external comparisons.","section":"Section VII"}],"recommendation":"major_revision","confidential_remarks":"The main issue is the explicit statement of test-set hyperparameter selection, which appears twice (Section IV-F and Section V). If it is an error, the authors must correct it and re-run; if it is not, the central claim is unsupported as written. The label-definition leakage risk also needs a concrete response, as does the unspecified fusion operation in Eq. (7). The authors should be asked for a dev-only re-run, a significance analysis, and a temporal/redaction audit of the PI notes before any acceptance decision."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Dear colleague,\n\nThe paper is a plausible engineering contribution to a narrow but real problem: triage classification on Korean-English code-switched pediatric EMR notes. BioBridge adds language segment tokens ([B-K]/[B-E]) and injects BioSent2Vec word features into standard encoders. Neither idea is new, but applying them jointly to this setting and evaluating across five encoder architectures is a legitimate, useful effort. The preprocessing of medical abbreviations is also a practical plus, and the ablation study is a good habit.\n\nThe problem is the evaluation protocol. Section IV-F says the grid search over learning rates picked 'the optimal hyperparameters for the test set,' and the ablation section repeats that wording. Taken at face value, the test set was used for model selection, so the numbers in Tables 3 and 5 are not independent held-out results. The dev-set checkpoint selection does not repair this. With effect sizes around 0.85% F1 and no confidence intervals or significance tests, the claimed advantage of BioBridge is unsupported as written. This is the load-bearing weakness, and it is fixable: re-run the grid on dev only, then evaluate once on test.\n\nThe second concern is the label. Emergency is defined by whether the patient received blood tests, IV hydration, nebulizer, medication, or admission. If the PI notes mention these interventions—phrases like 'IV started' or 'admitted'—the model can learn to detect documented care rather than predict the need for it. The paper doesn't address this. It's a legitimate worry, though not fatal if the notes are truly symptom-oriented; the authors need to show a leakage check.\n\nThere are smaller inconsistencies. On XLM-R, the full BioBridge has a worse Brier score than the baseline (19.59 vs 18.78), and the bio-embedding-only ablation degrades Brier to 23.33. That undercuts the 'consistent enhancement' claim.\n\nWho should read this? Anyone working on code-switched clinical NLP, especially Korean EMRs. The framework is worth trying, but the empirical evidence as presented is not enough to trust. The paper deserves a serious referee, not a desk reject, because the core idea is sound and the flaws are methodological rather than conceptual. I'd send it to peer review with a demand for dev-only tuning, significance testing, and a leakage analysis.\n\nRegards.","headline":"Sensible framework for Korean-English code-switched EMR triage, but test-set hyperparameter selection and a possible label leak undercut the reported gains; worth a revision, not a reject.","tokens_in":17521,"tokens_out":3843,"would_cite":false,"duration_ms":34779,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Adding language-segment tokens and medical word embeddings to transformer encoders improves emergency classification of Korean-English code-switched pediatric EMR notes.","keywords":["code-switching","electronic medical records","pediatric emergency department","emergency triage classification","BioSent2Vec","segment tokens","multilingual BERT","Korean-English clinical NLP"],"falsifier":"Mask or remove from each test-set Present Illness note every phrase that names a label-defining intervention (blood test, urinalysis, IV, nebulizer, drug administration, admission) and retrain or re-evaluate BioBridge; if its F1, AUROC, and AUPRC advantages over the plain encoder disappear or shrink to near zero, the reported gains were carried by direct documentation of the label rather than by clinical reasoning from the note.","tokens_in":16594,"feed_emoji":"🏥","tokens_out":6705,"duration_ms":57720,"temperature":0.7,"pith_summary":"BioBridge is a fine-tuning framework for transformer encoders that classifies emergency versus non-emergency visits from the free-text Present Illness notes of Korean-English code-switched pediatric EMRs. It adds two mechanisms: language segment tokens that mark whether each span is Korean or English, and a unified bio-embedding that injects English medical word vectors, from a biomedical sentence-embedding model, into the encoder's token representations. In the paper's reported results, BioBridge improves F1, AUROC, and AUPRC over the plain encoder for every backbone tested, and it lowers the Brier score for most backbones; BioBridge-XLM, for instance, gains 0.85% F1, 0.75% AUROC, 0.76% AUPRC, and a 3.04% lower Brier score relative to XLM. The claim matters because code-switched clinical text is common in non-English-speaking hospitals and is poorly served by general-domain multilingual models.","feed_headline":"Code-switch tags boost emergency triage from bilingual EMR notes","feed_subtitle":"Adding language markers and medical word vectors lifts F1 and calibration over plain transformer models.","key_machinery":"The load-bearing object is the pair of segment tokens [B-K] and [B-E], which are prepended to the Korean and English spans of each tokenized input sentence so the encoder can treat the two languages as distinct modalities, mirroring how multimodal transformers separate video, text, and audio inputs. The second mechanism is the unified bio-embedding: English subword tokens are re-joined into words, passed through the fixed BioSent2Vec medical feature extractor, and projected by a fully connected layer into the encoder's hidden dimension, where they are combined with the token embeddings. The classification readout is the usual [CLS] embedding, and the framework is trained end-to-end with only the encoder and projection layer updated.","core_discovery":"The paper's central claim is that code-switched clinical notes can be handled more effectively by treating each language as a separate modality inside a pre-trained transformer and by supplementing the encoder with medical-domain embeddings at the word level. Concretely, BioBridge inserts [B-K] and [B-E] segment tokens so the encoder can tell Korean spans from English spans, and it reconstructs the English subword tokens into whole words, feeds those words through BioSent2Vec to get medical features, and projects those features into the encoder's hidden space. Tested on emergency/non-emergency classification of 87,759 pediatric EMR Present Illness notes, the framework improves all four reported metrics over the corresponding baseline for both Korean-specific encoders (KR-BERT, KoBERT) and multilingual encoders (XLM, mBERT, XLM-R). The authors also show by ablation that each module contributes to the gain.","pith_inferences":["Because the emergency label is defined by whether the patient received interventions such as blood tests, IV hydration, or admission, and the model reads the notes that document those visits, the reported gains may partly reflect detecting documentation of treatment rather than predicting the need for it; a strong test would be to mask intervention-related phrases and see whether the gap survives.","The bio-embedding module only covers English medical words, so Korean medical terminology is still handled by the encoder's general-domain Korean knowledge; a Korean medical embedding or a cross-lingual medical feature extractor could plausibly give larger gains.","The framework's transferability likely depends on how cleanly the tokenizer splits the non-English language; languages without clear subword boundaries may need different segment markers or a different word-reconstruction rule."],"forward_implications":["If the framework works as reported, code-switched clinical text can be classified with off-the-shelf multilingual encoders plus a lightweight preprocessing step, without biomedical resources like UMLS that are unavailable for Korean.","The Brier-score reductions on most backbones suggest better-calibrated probability estimates, which matters for triage support where decision thresholds are set by clinicians.","Because the modules attach to any transformer encoder, the same recipe could be reused for other code-switched language pairs; the paper identifies this as its planned next step.","The method's gains are additive: ablations show both the segment tokens and the bio-embedding contribute, so hospitals could adopt either module alone if data or compute is limited."],"supporting_citations":[{"why":"Supplies the fixed medical word-feature extractor that injects biomedical knowledge in the unified bio-embedding module.","marker":"[63]"},{"why":"Provides the segment-token technique for separating input modalities, which BioBridge adapts to distinguish Korean from English spans.","marker":"[54]"},{"why":"Defines the BERT encoder architecture and the [CLS]/[SEP] conventions that the framework modifies and fine-tunes.","marker":"[23]"},{"why":"Multilingual encoder baseline; BioBridge-XLM is the paper's headline improvement.","marker":"[42]"},{"why":"Multilingual encoder baseline and ablation backbone used to test both modules.","marker":"[43]"},{"why":"Korean-specific encoder baseline showing the framework also helps monolingual models.","marker":"[40]"},{"why":"Korean-specific encoder baseline with subword tokenization, also improved by BioBridge.","marker":"[41]"},{"why":"Supplies the clinical definition of emergency that the classification labels are based on.","marker":"[59]"}],"fun_headline_variants":["BioBridge treats code-switched EMR as dual-modality for triage","Language-aware model sharpens pediatric ER predictions from notes","Bridging medical embeddings lifts code-switch EMR classification","Code-switch aware encoder improves ER triage on bilingual notes"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The whole result depends on the emergency label meaning what the framework is asked to predict: cases are labeled emergency when the record shows the patient received blood tests, urinalysis, IV hydration, nebulizer treatment, immediate drugs, or admission, so if the free-text notes already mention those interventions, the model can learn to spot documented care instead of anticipating clinical need.","fun_headline_variants_meta":{"raw":{"variants":["BioBridge treats code-switched EMR as dual-modality for triage","Language-aware model sharpens pediatric ER predictions from notes","Bridging medical embeddings lifts code-switch EMR classification","Code-switch aware encoder improves ER triage on bilingual notes"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000239,"raw_usage":{"total_tokens":1560,"prompt_tokens":1038,"completion_tokens":522,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":654,"completion_tokens_details":{"reasoning_tokens":450}},"tokens_in":654,"tokens_out":522,"duration_ms":6302,"temperature":1.0,"reasoning_tokens":450,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-11T14:41:27.385107+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Mask or remove from each test-set Present Illness note every phrase that names a label-defining intervention (blood test, urinalysis, IV, nebulizer, drug administration, admission) and retrain or re-evaluate BioBridge; if its F1, AUROC, and AUPRC advantages over the plain encoder disappear or shrink to near zero, the reported gains were carried by direct documentation of the label rather than by clinical reasoning from the note.","supporting_citations":[{"cited_title":"Biosentvec: creating sentence embeddings for biomedical texts","cited_arxiv_id":null,"evidence_quote":"Supplies the fixed medical word-feature extractor that injects biomedical knowledge in the unified bio-embedding module."},{"cited_title":"Bridging text and video: A universal multimodal transformer for audio-visual scene- aware dialog","cited_arxiv_id":null,"evidence_quote":"Provides the segment-token technique for separating input modalities, which BioBridge adapts to distinguish Korean from English spans."},{"cited_title":"Cross-lingual language model pretraining","cited_arxiv_id":null,"evidence_quote":"Multilingual encoder baseline; BioBridge-XLM is the paper's headline improvement."},{"cited_title":"Unsupervised cross-lingual represen- tation learning at scale","cited_arxiv_id":null,"evidence_quote":"Multilingual encoder baseline and ablation backbone used to test both modules."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Korean-specific encoder baseline showing the framework also helps monolingual models."},{"cited_title":"Definitions and assessment approaches for emergency medical services for children","cited_arxiv_id":null,"evidence_quote":"Supplies the clinical definition of emergency that the classification labels are based on."}],"review_version":1}