{"id":"ddf7b32d-4742-4c08-b44f-a6170307ab68","arxiv_id":"2412.15957","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":4.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":3,"one_line_summary":"A method constructs patient-specific hard prompts, refined only by word deletion with a BERTScore reward, to make proprietary LLMs output more personalized medical responses.","lead":"This paper introduces PMLM, a system that builds a patient-specific text prompt from predicted health labels and similar past patients, then uses reinforcement learning to delete words from the prompt before asking a proprietary LLM for medical advice. The authors report higher automatic text-similarity scores on real obstetrics and gynecology records, but the evidence lacks statistical tests, public code, and a mechanism for adding new personalized content.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"BERTScore serving as both RL reward and evaluation metric, with deletion-only refinement, leaves the central personalization claim unvalidated until an independent or clinician-blinded check confirms it.","rationale":"The reader's weakest assumption, that deletion-only refinement limits expressiveness, is real and is acknowledged in the paper's Limitations. I would sharpen it further: the more load-bearing issue is that BERTScore is simultaneously the training reward and the evaluation metric. The policy optimizes exactly the metric on which the paper's main comparison rests, and no clinician or independent metric validates that the improved BERTScore corresponds to more personalized or more clinically sound advice. The deletion-only operation compounds this by restricting the prompt to a subsequence of the coarse prompt, so the improvement mechanism is most plausibly the selection of a better-aligned template, not the addition of patient-specific content. A blinded expert rating would settle whether the effect is genuine personalization or metric alignment. I also note a smaller, concrete inconsistency: the claim that 'all evaluated LLMs exhibit performance improvements exceeding 10%' is not supported by the BERTScore F1 column in Table 1, where four of five models improve by roughly 5–10% relative. This does not change the conditional verdict but should be corrected if the paper is revised. The verdict remains CONDITIONAL, hence UNCHANGED relative to the reader's assessment, because the available evidence is suggestive but not sufficient for acceptance as stated.","tokens_in":18009,"tokens_out":6391,"duration_ms":60751,"concrete_test":"Pre-register a blinded clinician evaluation: sample 100 test patients stratified by the 12 labels; for each, show two obstetricians the before-prompt and after-prompt responses in randomized order, and ask them to rate which response is more individualized and more clinically appropriate. If PMLM's after-prompt responses are not preferred at a pre-specified rate (e.g., >60% with the lower bound of a 95% CI above 50%), then the BERTScore-based claim of fine-grained personalization is not established. A secondary computational check would replace the learned deletion policy with random deletion of the same length; if random deletion recovers most of the BERTScore gain, the RL policy is not doing patient-specific work.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim that RL word-deletion yields fine-grained, patient-level personalization is not yet supported because the only measured signal is BERTScore, which is used both as the RL reward (Eq. 9) and as the headline evaluation metric (Tables 1–3). The policy is trained to maximize BS(LLMr(P), r) relative to the initial prompt, and evaluation reports the same BERTScore. Because refinement is deletion-only (Eqs. 5–11; Limitations), P is a subsequence of the coarse prompt and cannot introduce new individualized content; the observed BERTScore gains could therefore reflect a learned deletion pattern that makes LLMr echo the predicted-label and similar-patient block already present in the prompt, pushing replies closer to the reference responses, rather than adapting advice to each patient. The personalization study (Fig. 6) only counts which word indices are deleted across patients; it does not demonstrate semantic or clinical personalization. Without a clinician-blinded evaluation or an independent metric not used in training, the paper's central assertion remains a correlation with its own training objective.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes PMLM, a method for generating personalized hard prompts for medical LLMs. It first fine-tunes LLaVA1.5-7B as a label predictor, then encodes patient data and retrieves k similar patients via cosine similarity to construct a coarse-grained prompt containing the predicted label and similar patients' diagnoses. A BERT-based policy network refines this prompt by deleting words over n steps, using a REINFORCE-style loss whose reward is the BERTScore improvement of the downstream proprietary LLM response over the initial prompt. The final prompt is fed to a closed LLM such as GPT-4 to produce a response. Experiments on real obstetrics data report BLEU, ROUGE, and BERTScore improvements over the initial prompt and over fine-tuned 7-9B open LLMs.","tokens_in":18163,"tokens_out":6239,"duration_ms":53034,"significance":"If the central claim held, the contribution would be practically significant: it would show that text-only hard prompts can be optimized for individual patients and transferred to proprietary LLMs without weight updates or raw data transfer. The paper has several strengths: real hospital data, evaluation across five proprietary LLMs, an ablation of the three components, a hyperparameter study, and qualitative case studies. However, the current evidence does not establish fine-grained personalization because the evaluation metric is also the training reward, the deletion-only action space limits expressivity, and the state-of-the-art comparison confounds method with base model. The result is a useful proposal with suggestive evidence rather than a validated system.","major_comments":[{"comment":"The RL reward is BERTScore against doctor reference responses, and BERTScore is also the headline evaluation metric in Tables 1-3. The authors explicitly argue (Section 5.2) that greater emphasis should be placed on BERTScore. Hence the reported BERTScore gains are, to an unknown extent, a consequence of optimizing the same score. The held-out year split reduces overfitting but does not remove the objective-alignment problem. The paper needs an independent evaluation (e.g., clinician ratings, or a metric not used in training, or a check that the refinement changes responses in ways correlated with patient-specific data beyond the predicted label).","section":"Section 4.2, Eq. (9) vs. Section 5.2, Tables 1-3"},{"comment":"The claim that PMLM 'achieves state-of-the-art performance' is confounded: the PMLM row in Table 2 is identical to the GPT-4 'after' row in Table 1. The comparison is thus between GPT-4 with a refined prompt and fine-tuned 7-9B open models with an unrefined prompt. This does not show that the PMLM method, as opposed to the base LLM scale, is responsible for the improvement. The authors should include fine-tuned lightweight baselines with the same personalized prompt (or at least the coarse prompt) and a GPT-4 baseline without PMLM in the same table.","section":"Section 5.2, Tables 1-2"},{"comment":"The personalization study only plots counts of deleted word indices; it does not show that different patients receive semantically different refinements. Since the action space is deletion-only (Eq. 7) and the Limitations section explicitly states that addition and replacement are left to future work, the final prompt is a subsequence of the coarse prompt and cannot introduce patient-specific instructions absent from the initial prompt. The BERTScore gains could therefore come from selecting a common deletion pattern that emphasizes the predicted label and similar-patient block, rather than from adapting the advice to the individual. A per-patient analysis or a clinician-blinded evaluation is needed to support the claim of individual-level personalization.","section":"Section 5.3, Eqs. 5-11, Limitations"},{"comment":"The sentence 'all evaluated LLMs exhibit performance improvements exceeding 10%' is contradicted by Table 1 on the BERTScore F1 metric: Gemini1.5-pro improves from 63.94 to 70.13 (about 9.7%), GPT3.5-turbo from 65.19 to 68.89 (about 5.7%), and GLM4-plus from 61.26 to 67.31 (about 9.9%). The claim is only true for BLEU-4 or ROUGE-N for some models. Since the paper highlights BERTScore as the most important metric, this overstatement should be corrected and the claim restated per metric.","section":"Section 5.2, Table 1"},{"comment":"No variance, confidence intervals, or significance tests are reported despite the statement that averages over five runs are used. Given the small test set (the second half of 2022 is a small slice of the 2,373 patients) and the modest differences in the ablation study (e.g., Variant 2 vs. Variant 3 differ by about 2.4 BERTScore F1), the improvements could be within run-to-run noise. The authors should report standard deviations and significance tests, or at least the minimum and maximum over runs.","section":"Section 5.1, Experimental settings"}],"minor_comments":[{"comment":"There is a typo in the first contribution bullet: 'healthcare filed' should be 'healthcare field'.","section":"Section 1, Contributions"},{"comment":"The state indexing is inconsistent: the paper defines S = {s0, ..., s_{n-1}} but says the process is repeated n times and the final state is s_{n-1}. After n deletions the final state should be s_n; please correct the off-by-one and ensure the loss in Eq. (11) aligns with the number of steps.","section":"Section 4.2, Eqs. 5-11"},{"comment":"The caption of Table 2 does not state that the PMLM row uses GPT-4 as the base LLM; since PMLM is model-agnostic, please specify the base model for each row in the caption or table notes.","section":"Section 5.2, Table 2"},{"comment":"The text says the final dataset contains '38,817 records for 2,373 pregnant patients', but the subsequent sentence 'It contains 56 entries from 2020, 1,638 from 2021, and 679 from 2022' sums to 2,373 and appears to refer to patients, not records; please clarify the units.","section":"Appendix A.2, Dataset details"},{"comment":"The paper states that code will be released upon formal publication. For reproducibility, please provide a link to the code or at least a synthetic data generator and detailed preprocessing scripts, since the real data cannot be shared for privacy reasons.","section":"Appendix A.1 and Ethical statement"}],"recommendation":"major_revision","confidential_remarks":"The paper is an interesting application of hard-prompt RL to a real medical dataset, but the reliance on BERTScore as both reward and metric is likely to be a central sticking point for reviewers. The identical PMLM/GPT4 rows are visible in the tables, and the limitations are acknowledged, so I see no intent to mislead. The main work for revision is to provide an independent evaluation of the personalization claim and to de-confound the base model comparison. With those additions, the paper could become a solid contribution; without them, the evidence remains suggestive only."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Quick take: this is a competent application of known prompt-optimization ingredients to a new domain, but the evidence for \"fine-grained individual-level personalization\" doesn't yet support the claim. The pipeline is coherent: LoRA predictor, collaborative filtering, RL word-deletion prompt refinement, evaluated on a private OB/GYN dataset. That combination is new and the authors ablate each piece; the ablations and hyperparameter sweeps are the best part. Case studies are illustrative, not proof.\n\nThe soft spots: (1) BERTScore is both the RL reward and the headline evaluation metric. The stress-test note is right to flag this. It's mitigated by the fact that BLEU and ROUGE also improve and by the temporal split, but those metrics are not fully independent of the training objective; the gains could still be from deletion patterns that happen to make outputs more reference-like on average. (2) More importantly, refinement is deletion-only (the Limitations admit this). A subsequence of the initial prompt cannot add new patient-specific content. So the RL stage can only select which pieces of the already-included information to emphasize or discard. That might explain the BERTScore gains without constituting fine-grained personalization. The \"personalization study\" counts deleted word indices, which does not demonstrate semantic adaptation. (3) The comparison against fine-tuned 7-9B LLMs is confounded: PMLM uses GPT4 as the generator, so \"state-of-the-art\" is largely model size, not method. (4) No error bars, no significance tests, and code/data are unavailable.\n\nI'd send this to review with the expectation of major revision: demand an independent metric (e.g., clinician rating) and a comparison that matches the generator model, or at least includes a GPT4-only baseline with the same prompt template. The method is worth discussing but the personalization claim needs stronger support.","headline":"A competent but under-evidenced pipeline for prompt-level medical personalization; the central claim of fine-grained individual adaptation is not yet supported.","tokens_in":18735,"tokens_out":2062,"would_cite":false,"duration_ms":19885,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"A patient-specific text prompt, built from a predicted diagnosis and similar patients' diagnoses, then refined by deleting words with a reinforcement-learned policy, is enough to make closed proprietary LLMs give more personalized medical…","keywords":["personalized medicine","large language models","prompt optimization","reinforcement learning","hard prompts","collaborative filtering","healthcare NLP","obstetrics"],"falsifier":"Take a held-out set of patients, shuffle the predicted labels among patients so that each prompt contains a wrong label, and run the trained PMLM policy to produce refined prompts and responses; if BERTScore against doctor references does not drop markedly relative to the unshuffled case, the method is responding to template-level patterns and not to the patient-specific content.","tokens_in":17749,"feed_emoji":"🩺","tokens_out":7616,"duration_ms":55102,"temperature":0.7,"pith_summary":"The paper tries to establish that a patient-specific text prompt, built from a predicted diagnosis plus the diagnoses of similar past patients and then refined by deleting words through reinforcement learning, is enough to make closed commercial LLMs give more personalized medical advice. It proposes PMLM, which first uses a fine-tuned LLM to predict each patient's condition and an encoder plus cosine similarity to find similar patients, then composes a coarse personalized prompt and trains a word-level policy to delete tokens so that the final prompt improves the LLM's response as measured by BERTScore against doctor-written replies. On real obstetric and gynecological records, every proprietary LLM tested, including GPT-4, Gemini1.5-pro, and GLM4 variants, improved by more than 10% on BERTScore when driven by the refined prompt, and the GPT-4-based PMLM outperformed fine-tuned open models like Llama3-8B and Qwen2-7B. If this holds, a text-only prompt layer can adapt proprietary LLMs to individual patients without retraining them or sending raw examination tables to the closed service.","feed_headline":"Text-prompt edits lift medical LLM personalization by over 10%","feed_subtitle":"Text-only RL-refined prompts steer closed LLMs to patient-specific advice without retraining or exposing raw records.","key_machinery":"The load-bearing object is the personalized hard prompt, a text prompt that differs for every patient and is passed unchanged to a closed LLM. Its construction is carried by two mechanisms: (1) self-informed personalization, where a LoRA-fine-tuned LLM predictor turns variable-length health check records into a predicted label, and peer-informed personalization, where an MLP encoder projects padded records into a shared space and cosine similarity selects the $k$ most similar past patients whose diagnoses are listed in the prompt; and (2) an RL word-deletion policy that, starting from the coarse prompt, samples one word to delete per iteration based on BERT embeddings concatenated with the mean embedding and the patient vector, using BERTScore gain over the initial prompt's response as reward. Because the final prompt is a subsequence of the coarse prompt and is pure text, it needs no access to the proprietary LLM's parameters.","core_discovery":"The central claim, stated on the paper's own terms, is that PMLM achieves individual-level personalization for medical LLMs by constructing a coarse-grained hard prompt from self-informed and peer-informed information and refining it with an RL policy that deletes words. The self-informed part is a predicted label from a LoRA-fine-tuned predictor; the peer-informed part is the top-$k$ most similar patients by cosine similarity in an MLP-encoded space, whose ground-truth diagnoses are inserted into the prompt. The RL policy computes deletion probabilities from BERT word embeddings, their mean, and the patient representation, deletes one word per step for $n$ steps, and is trained with a loss that rewards BERTScore gain between the refined and initial prompt responses. The result, the paper claims, is that all evaluated proprietary LLMs exceed a 10% performance improvement and the GPT-4-based PMLM reaches the best BERTScore, exceeding fine-tuned lightweight LLMs while requiring no weight changes to the proprietary model.","pith_inferences":["Editorial inference: because the reward is BERTScore against doctor-written reference responses, the method can only rediscover patterns present in the reference corpus; it would not generate novel advice that no doctor in the training set wrote.","Editorial inference: a testable extension is to allow insertion and replacement in the policy; if the current method's gains are primarily from removing generic text, an insertion-capable policy trained on the same reward should yield equal or better personalization on the same obstetric data.","Editorial inference: the similarity-based peer-informed component had the smallest ablation impact, suggesting that the predicted label carries most of the personalization signal; a cheaper predictor or a similarity based on raw records rather than zero-padded MLP embeddings might change this balance.","Editorial inference: the method transfers naturally to other closed-API personalization tasks, such as email drafting or fitness advice, whenever a reference response is available to define the BERTScore reward; the paper only demonstrates the healthcare setting."],"forward_implications":["A text-only prompt layer can steer closed, proprietary LLMs into producing more individualized responses without fine-tuning or weight access.","Patients' raw multi-visit examination tables do not need to be sent to the proprietary model; only the predicted label and selected similar-patient diagnoses appear in the prompt, which the paper argues reduces privacy exposure.","The same refined text prompt can in principle be reused across different LLMs, though the policy itself is trained with reward from a specific downstream model.","On semantic similarity metrics the method beats fine-tuned 7B-9B open LLMs, but it trails on ROUGE-L, indicating the gains are semantic rather than lexical overlap with doctor-written replies.","Since only word deletion is used, the refined prompt cannot introduce new patient-specific content; the authors list addition and replacement as future work."],"supporting_citations":[{"why":"Supplies the RL-based hard prompt optimization paradigm that PMLM adapts to word-level deletion.","marker":"(Mingkai and Jianyu, 2022)"},{"why":"Defines BERTScore, the reward used to train the policy and the primary semantic evaluation metric.","marker":"(Zhang et al., 2020)"},{"why":"Provides LoRA, used to fine-tune the lightweight LLM predictor for self-informed personalization.","marker":"(Hu et al., 2022)"},{"why":"Basis of the collaborative filtering approach for selecting similar patients for peer-informed personalization.","marker":"(Linden et al., 2003)"},{"why":"Provides BERT word embeddings that the policy network uses to compute deletion probabilities.","marker":"(Devlin, 2018)"},{"why":"GPT-4 is the proprietary LLM that yields the best PMLM results in the experiments.","marker":"(Achiam et al., 2023)"},{"why":"Used for fine-tuning the lightweight baseline LLMs that PMLM is compared against.","marker":"(Zheng et al., 2024b)"},{"why":"LLaVA1.5-7B is the specific predictor used for label prediction after comparing several LLMs and time-series methods.","marker":"(Liu et al., 2024)"}],"fun_headline_variants":["RL-refined hard prompts personalize medical LLMs","RL-tuned prompts boost medical LLM personalization","Hard-prompt RL personalizes medical LLMs per patient","Deletion-RL refines prompts for individualized LLM advice","RL-refined prompts lead to 10%+ gain in medical LLM personalization"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The claim that the RL stage enables fine-grained personalization rests on the assumption that deleting words from the initial prompt is enough to adapt it to each patient, since the paper's Limitations state that addition and replacement are left to future work; if no deletion can express the needed new guidance, the measured gains may reflect selection of a common template that scores well on BERTScore rather than true per-patient adaptation.","fun_headline_variants_meta":{"raw":{"variants":["RL-refined hard prompts personalize medical LLMs","RL-tuned prompts boost medical LLM personalization","Hard-prompt RL personalizes medical LLMs per patient","Deletion-RL refines prompts for individualized LLM advice","RL-refined prompts lead to 10%+ gain in medical LLM personalization"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.001034,"raw_usage":{"total_tokens":4349,"prompt_tokens":936,"completion_tokens":3413,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":552,"completion_tokens_details":{"reasoning_tokens":3328}},"tokens_in":552,"tokens_out":3413,"duration_ms":21013,"temperature":1.0,"reasoning_tokens":3328,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-11T10:55:06.166438+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Take a held-out set of patients, shuffle the predicted labels among patients so that each prompt contains a wrong label, and run the trained PMLM policy to produce refined prompts and responses; if BERTScore against doctor references does not drop markedly relative to the unshuffled case, the method is responding to template-level patterns and not to the patient-specific content.","supporting_citations":[],"review_version":1}