{"id":"17915867-0ea3-4d3f-a31c-cd27fc8b5364","arxiv_id":"2504.14361","paper_version":2,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":4.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":3,"one_line_summary":"Swapping DeepCDR's cell encoder for scGPT embeddings raises IC50 prediction PCC over scFoundation and baseline, but the result lacks randomized splits, error bars, and released code.","lead":"This paper replaces the cell-line encoder in the DeepCDR drug response model with embeddings from the scGPT single-cell foundation model and reports higher Pearson correlations between predicted and measured drug sensitivity than the scFoundation-based version. The comparison is plausible but rests on a non-random data split, single training runs, and no released code.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Section 3.2 says scGPT inputs were ordered and zero-padded with scFoundation's gene list; if literal, the scGPT embeddings are scrambled and the central comparison is invalid.","rationale":"The reader's weakest assumption was the non-random 95/5 split and the 90,000-instance training cap. That is a genuine validity concern and is acknowledged in the paper. However, the more load-bearing issue is upstream: the description of how scGPT embeddings were generated is ambiguous in a way that could invalidate the entire scGPT comparison. Sections 3.2 and 3.3 explicitly say a scFoundation-provided gene list was used for compatibility, but scGPT and scFoundation have different pretrained gene vocabularies. If the scGPT model received expression values ordered by scFoundation's gene list, the input positions would not match the positions learned during pretraining, making the embeddings arbitrary. This is not an external-consensus disagreement; it is an internal correctness risk in the central pipeline. The paper's own text does not rule it out, and the absence of released code prevents verification. The leave-one-drug-out and training-stability results do not mitigate this, because they all use the same embeddings. A CONDITIONAL verdict remains appropriate, but the conditions should include verifying the gene-vocabulary alignment, not only randomizing the data split. Since the reader already arrived at CONDITIONAL, I do not change the overall verdict, but I would add this as a primary condition for acceptance.","tokens_in":5408,"tokens_out":7572,"duration_ms":73980,"concrete_test":"Obtain the authors' embedding-generation script, or re-implement the scGPT zero-shot tutorial. For a fixed set of CCLE profiles, generate scGPT embeddings twice: (i) using scGPT_human's own vocab.json and gene ordering, and (ii) using the scFoundation gene-list ordering described in Section 3.2. Compare the two embedding sets per profile, then rerun the 95/5 PCC evaluation with embedding set (i). If the ranking between scGPT and scFoundation changes, or if PCC values shift materially, the paper's central claim rests on invalid scGPT inputs and is not supported.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim depends on scGPT embeddings being faithful outputs of the pretrained scGPT checkpoint. Sections 3.2-3.3 state that gene expression data were made compatible with scGPT and scFoundation by using a gene list provided by scFoundation and zero-padding to that list. scFoundation and scGPT use different pretrained gene vocabularies: scGPT's human checkpoint defines its own ordered vocabulary, and the model interprets input expression values as aligned to that vocabulary. If the CCLE expression vector was ordered and zero-padded according to scFoundation's list and then fed to scGPT, the gene values are permuted relative to scGPT's trained gene embeddings, so the resulting 512-dimensional representations would not be meaningful scGPT embeddings. The paper never states that scGPT's own vocab.json or gene ordering was used, and no code is released to verify this. Since the headline result is a comparison of scGPT against scFoundation on exactly this representation, an input-ordering mismatch would invalidate the central claim rather than merely weaken it. The acknowledged non-random 90,000-instance slicing bias in Section 3.4 is a separate, secondary concern.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The manuscript adapts the DeepCDR drug-response framework by replacing its cell representation with embeddings from the pretrained scGPT single-cell foundation model, using bulk RNA-seq from CCLE and IC50 labels from GDSC. It compares against two baselines—DeepCDR and scFoundation, both stripped to gene expression only—using Pearson correlation across cell lines, cancer types, and drugs, plus a leave-one-drug-out experiment on 20 drugs. The paper reports that scGPT outperforms both baselines and exhibits smoother training curves, and concludes that scGPT embeddings improve cancer drug response prediction.","tokens_in":5649,"tokens_out":4341,"duration_ms":38675,"significance":"If the central comparison is valid, the paper provides a useful, if incremental, benchmark of single-cell foundation model embeddings for bulk drug-response prediction, and it is one of few direct comparisons of scGPT and scFoundation in this setting. The evaluation is externally grounded: CCLE/GDSC data, public pretrained checkpoints, and two implemented baselines are used, and no target-label information enters the embedding generation, so the reported gains are not circular. The authors also state that their implementation builds on publicly available codebases. However, the evaluative basis is thin: a single run, a non-random data split, no released code, and a potentially fatal ambiguity about gene ordering.","major_comments":[{"comment":"The manuscript states that gene expression data were made compatible with scGPT and scFoundation 'using a gene list provided by scFoundation' and by zero-padding to that list. scGPT's human checkpoint has its own pretrained gene vocabulary and ordering. If the CCLE expression values were ordered and zero-padded according to scFoundation's gene list and then passed to scGPT, the values would be permuted relative to the positions learned by scGPT's gene embeddings, making the resulting 512-dimensional vectors meaningless. The paper never states that scGPT's own vocab.json or gene ordering was used, and no code is released to verify this. Because the headline claim is that scGPT embeddings outperform scFoundation on exactly this representation, the paper must clarify and verify the gene-order handling before the results can be interpreted.","section":"3.2–3.3"},{"comment":"The evaluation rests on a single run of each configuration on a 95/5 split in which the 90,000-instance training subset was 'selected by slicing rather than random sampling, which may have introduced bias and affected the evaluation results.' No error bars, confidence intervals, or repeated-seed results are reported for Figures 2–5. Since the scGPT-versus-scFoundation differences are described as 'small but consistent,' the absence of uncertainty estimates and the acknowledged bias mean the ranking could plausibly change under a randomized split or a different seed. I request repeated runs with different seeds, a randomized or stratified split, and reporting of per-category variability.","section":"3.4"},{"comment":"The baselines are not 'original DeepCDR': Section 3.4 states that both baselines were modified to exclude methylation and mutation data, relying solely on gene expression inputs. The abstract and conclusion nevertheless claim superiority over 'the original DeepCDR framework' and describe 'the original DeepCDR model' as failing to generalize to unseen drugs. This conflates a gene-expression-only DeepCDR variant with the published multi-omics DeepCDR model. The comparisons are reasonable for isolating the cell-embedding contribution, but the wording should be corrected, and ideally the published multi-omics DeepCDR should be included or its omission explicitly justified.","section":"3.4 and 4"}],"minor_comments":[{"comment":"There are multiple typos in the introduction, including 'heterogenity', 'vaiability', 'reserach', and 'predcition'; these should be corrected.","section":"1"},{"comment":"The caption says each dot represents a cell line, cancer type, or drug, and the red dashed identity line y=x is used, but the axes are not clearly identified; the reader cannot tell which model is on the x-axis and which is on the y-axis.","section":"Figure 2"},{"comment":"The x-axis is described as 'the rank of drugs based on their improvement,' but it is unclear whether the rank is based on scGPT, scFoundation, or both, and no error bars or per-drug sample sizes are shown.","section":"Figure 4"},{"comment":"The leave-one-drug-out analysis covers only 20 of 223 drugs, with no description of how the random selection was performed; a seed or selection criterion should be reported.","section":"3.4"},{"comment":"The claim that the 90,000-instance cap 'did not seem to have too big of an impact' is unsupported because no comparison against a full-data run is provided.","section":"4"},{"comment":"Details of the normalization and preprocessing are incomplete: the text mentions CPM and log1p, but not whether the scGPT checkpoint's own recommended preprocessing (including any gene filtering) was applied.","section":"3.3"}],"recommendation":"major_revision","confidential_remarks":"The gene-ordering ambiguity in Sections 3.2–3.3 is the most serious issue and should be resolved before the paper is considered further; if the scFoundation gene list was literally used for scGPT, the central comparison is invalid. The evaluation-reporting weaknesses are fixable but require additional experiments. The paper may be better suited to a workshop or short-report venue unless the evaluation is substantially strengthened."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Main thing to know: the headline result—scGPT beats scFoundation and DeepCDR—rests on scGPT embeddings produced from inputs prepared with scFoundation's gene list (Sections 3.2–3.3). scGPT's checkpoint uses its own vocabulary ordering. If the bulk expression vector was ordered and zero-padded to scFoundation's list, the values are permuted relative to scGPT's gene embeddings, and the resulting 512-dim vectors are not meaningful scGPT representations. The paper never says it used scGPT's vocab.json, and no code is released. This would invalidate the central comparison, not just weaken it.\n\nWhat's new: it's the first public attempt to use scGPT specifically inside DeepCDR, and the comparison to scFoundation is a reasonable test. The authors also flag the non-random 90,000-instance slicing themselves, which is honest. The training stability curve is a nice extra observation.\n\nSoft spots in proportion: the ordering issue is load-bearing and needs an explicit correction. Second, the evaluation is single-run PCC with no error bars or repeated seeds; the acknowledged slicing bias means the test set may not be representative. The leave-one-drug-out covers only 20 of 223 drugs, randomly chosen but without seeds, so it's suggestive rather than conclusive. These are fixable but currently make the margin between scGPT and scFoundation look thinner than the paper implies.\n\nWho it's for: people working on drug response prediction who want to know whether scGPT embeddings transfer to bulk expression. If the ordering issue is resolved, it's a useful incremental data point.\n\nRecommendation: I would not publish as is. I'd send it to peer review only if the authors first clarify the gene ordering and ideally release code; without that, the central claim is unverifiable. A serious referee could check this quickly, so if you're considering it, ask for the implementation details before investing more.","headline":"Potentially invalid cell embeddings from a gene-ordering mismatch undermine an otherwise honest incremental comparison.","tokens_in":6157,"tokens_out":2904,"would_cite":false,"duration_ms":26375,"reading_group":"maybe","serious_thinker":"unclear","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"scGPT embeddings improve cancer drug response prediction when swapped into a graph-network framework.","keywords":["drug response prediction","single-cell foundation models","scGPT","graph neural networks","IC50","transfer learning","cancer cell lines"],"falsifier":"Re-run the same three-model comparison with a randomly sampled 95/5 split, or a stratified split that preserves drug and cancer-type coverage, drawn from the full dataset and using the same 90,000-instance cap selected uniformly at random. If the scGPT-based model's Pearson correlation no longer exceeds the scFoundation-based model across the cell-line, cancer-type, and drug filters, the claimed ordering rests on the slicing artifact rather than on scGPT embeddings.","tokens_in":5214,"feed_emoji":"🧬","tokens_out":8745,"duration_ms":71679,"temperature":0.7,"pith_summary":"This paper asks whether cell representations from a pretrained single-cell foundation model, scGPT, can replace multi-omics profiles in an existing drug-response prediction framework. It claims they can: a pipeline that encodes drug structures with a graph neural network and cancer cell lines with scGPT embeddings predicts IC50 values (the drug concentration needed to inhibit a response by half) with higher correlation between predicted and measured values than the same pipeline using scFoundation embeddings or the original multi-omics encoder. The improvement appears across cell-line, cancer-type, and drug-level filters, extends to unseen drugs in leave-one-drug-out tests, and is accompanied by smoother training curves. The paper's bet is that pretrained single-cell knowledge compensates for the small, expensive drug-response datasets that would otherwise limit accuracy.","feed_headline":"scGPT beats scFoundation and baseline on drug response","feed_subtitle":"Plugging scGPT into a graph-network drug model raises IC50 correlation and training stability.","key_machinery":"The load-bearing element is the pretrained scGPT cell encoder: a generative transformer trained on 33 million single cells, repurposed here as a fixed embedding function that maps bulk RNA-seq gene-expression values for a cancer cell line into a 512-dimensional vector. That vector replaces the multi-omics feature set used in the original DeepCDR and is concatenated with a drug representation produced by a graph neural network over the drug's molecular graph; the combined vector is passed through a neural network with dropout and batch normalization to regress IC50. The paper's comparisons attribute the performance gap to the quality of these scGPT embeddings rather than to any change in the drug-encoding side.","core_discovery":"The paper's central claim is that replacing the cell-side representation in DeepCDR—normally built from methylation, mutation, and expression data—with a 512-dimensional scGPT embedding of bulk gene expression improves cancer drug response prediction. In its evaluation, the scGPT-based model outperforms both an scFoundation-based adaptation and the original DeepCDR for IC50 prediction, as measured by Pearson correlation between predicted and observed responses at three filter levels: cell lines, cancer types, and drugs. It also shows higher and more consistent correlation gains than scFoundation in leave-one-drug-out runs, indicating better generalization to unseen drugs, and its validation score climbs steadily from about 0.80 to above 0.90 over 20 epochs while the scFoundation baseline fluctuates. The paper interprets this as evidence that scGPT's pretrained gene representations capture cellular state more effectively for this task, despite using a lower-dimensional embedding than scFoundation.","pith_inferences":["Editorial inference: if the ranking survives a randomized evaluation split, the strongest claim would shift from 'scGPT works here' to 'cell foundation model choice systematically changes transfer quality,' making model selection a standard axis of comparison in drug-response pipelines.","Editorial inference: a natural paired test is to swap the drug encoder for a molecular foundation model, which the paper lists as future work, and ask whether cell-side and drug-side gains compound.","Editorial inference: because the paper omits methylation and mutation data in all compared variants, an ablation that re-adds those modalities would reveal whether scGPT embeddings capture enough biology to make them redundant or whether the two are complementary."],"forward_implications":["Future drug-response models can treat scGPT as a drop-in cell encoder, replacing multi-omics preprocessing with a single gene-expression input without sacrificing accuracy.","Because the advantage persists in leave-one-drug-out tests, the approach is directly applicable to screening candidate drugs that have no prior response data in the training set.","The smoother training trajectory means less sensitivity to epoch selection and could reduce computational tuning for new datasets.","Since scGPT's 512-dim embeddings beat scFoundation's 768-dim embeddings, the choice of pretraining objective may matter more than embedding size when transferring to drug response."],"supporting_citations":[{"why":"Defines the DeepCDR framework whose architecture is adapted and whose original version is the baseline.","marker":"[1]"},{"why":"Describes the scFoundation-based integration and evaluation setup that this paper extends and compares against.","marker":"[2]"},{"why":"Introduces scGPT, the pretrained model whose embeddings are the paper's central new ingredient.","marker":"[3]"},{"why":"Supplies the gene list and preprocessing conventions used to align CCLE expression data to foundation-model inputs.","marker":"[4]"},{"why":"Provides the bulk RNA expression profiles for cancer cell lines used as model input.","marker":"[5]"},{"why":"Provides the IC50 ground-truth drug-cell line responses used for training and evaluation.","marker":"[6]"},{"why":"Gives the pretrained scGPT-human checkpoint and the embedding-generation protocol used to create the cell representations.","marker":"[7]"}],"fun_headline_variants":["scGPT embedding boosts drug response prediction","scGPT outperforms scFoundation on IC50 prediction","scGPT bests baselines for cancer drug response","scGPT representation improves drug response accuracy","scGPT tops scFoundation in drug response task"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The evaluation assumes that the 95/5 split obtained by slicing ordered data, with training capped at 90,000 instances, still produces a test set representative enough to rank the three models fairly; the paper itself notes that this slicing may have introduced bias.","fun_headline_variants_meta":{"raw":{"variants":["scGPT embedding boosts drug response prediction","scGPT outperforms scFoundation on IC50 prediction","scGPT bests baselines for cancer drug response","scGPT representation improves drug response accuracy","scGPT tops scFoundation in drug response task"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000292,"raw_usage":{"total_tokens":1670,"prompt_tokens":881,"completion_tokens":789,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":497,"completion_tokens_details":{"reasoning_tokens":720}},"tokens_in":497,"tokens_out":789,"duration_ms":7166,"temperature":1.0,"reasoning_tokens":720,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-16T11:49:58.388796+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Re-run the same three-model comparison with a randomly sampled 95/5 split, or a stratified split that preserves drug and cancer-type coverage, drawn from the full dataset and using the same 90,000-instance cap selected uniformly at random. If the scGPT-based model's Pearson correlation no longer exceeds the scFoundation-based model across the cell-line, cancer-type, and drug filters, the claimed ordering rests on the slicing artifact rather than on scGPT embeddings.","supporting_citations":[{"cited_title":"Deepcdr: a hybrid graph convolutional network for predicting cancer drug response","cited_arxiv_id":null,"evidence_quote":"Defines the DeepCDR framework whose architecture is adapted and whose original version is the baseline."},{"cited_title":"Large-scale foundation model on single-cell transcriptomics","cited_arxiv_id":null,"evidence_quote":"Describes the scFoundation-based integration and evaluation setup that this paper extends and compares against."},{"cited_title":"scgpt: towards building a foundation model for single-cell multi-omics using generative ai","cited_arxiv_id":null,"evidence_quote":"Introduces scGPT, the pretrained model whose embeddings are the paper's central new ingredient."},{"cited_title":"scfoundation github repository - https://github.com/biomap-research/scfoundation, 2023","cited_arxiv_id":null,"evidence_quote":"Supplies the gene list and preprocessing conventions used to align CCLE expression data to foundation-model inputs."},{"cited_title":"The cancer cell line encyclopedia enables predictive modelling of anticancer drug sensitivity","cited_arxiv_id":null,"evidence_quote":"Provides the bulk RNA expression profiles for cancer cell lines used as model input."},{"cited_title":"A landscape of pharmacogenomic interactions in cancer","cited_arxiv_id":null,"evidence_quote":"Provides the IC50 ground-truth drug-cell line responses used for training and evaluation."},{"cited_title":"scgpt github repository - https://github.com/bowang-lab/scgpt, 2024","cited_arxiv_id":null,"evidence_quote":"Gives the pretrained scGPT-human checkpoint and the embedding-generation protocol used to create the cell representations."}],"review_version":1}