{"id":"29c1343f-40d0-420b-bc48-229d3c61dbcb","arxiv_id":"2412.00068","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":6,"one_line_summary":"Semi-supervised pseudo-labeling with head-and-neck cancer data improved lung cancer survival prediction accuracy to 0.85 versus 0.65 for supervised learning alone.","lead":"Adding pseudo-labeled head-and-neck cancer PET/CT images to a lung cancer training set, with labels generated by a random forest, is reported to improve two-year survival classification accuracy from 0.65 to 0.85. The paper also reports a survival model with c-index about 0.80 on CT features, but the evaluation has methodological weaknesses that should be addressed.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Headline SSL-vs-SL comparison is confounded: best SSL (DRF-PET+MLP) is compared with best SL (DRF-CT+KNN), changing both modality and classifier; the claimed p<0.05 does not isolate pseudo-labeling.","rationale":"The reader's weakest_assumption focuses on cross-cancer pseudo-label distribution shift, which is a real empirical risk. However, I see a more immediate load-bearing flaw: the reported headline comparison changes two variables at once, so even perfectly informative pseudo-labels would not be isolated. This is visible directly from the numbers in Section 3.1.2 and can be settled by re-analysis without new data. The reader did list 'selection of the best model for the headline comparison' among the weaknesses, so I mark partial agreement rather than disagreement. I recommend keeping the CONDITIONAL verdict: the matched-comparison reanalysis is a necessary condition before the SSL gain can be accepted, and the domain-shift question should then also be tested. This is not a rejection of the underlying idea, but the current evidence for the central claim is insufficient as reported.","tokens_in":15752,"tokens_out":7813,"duration_ms":74978,"concrete_test":"Re-analyze the released GitHub code to produce a matched comparison: for every combination of feature set (HRF, DRF), modality (CT, PET), and classifier (MLP, SVM, KNN, EV), compute 5-fold CV accuracy under SL and SSL, then test paired fold-wise differences within each configuration. The decisive cell is DRF-PET+MLP: report the SL accuracy for that exact configuration; if SL DRF-PET+MLP is already ≥0.8, the SSL effect disappears. More generally, if the median matched SSL−SL difference across all configurations is not consistently positive and significant, the paper's claim is unsupported.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim that SSL outperforms SL rests on a non-matched comparison. In Section 3.1.2, the best SSL result (DRF-PET, PCA, MLP: 0.85±0.05) is contrasted with the best SL result (DRF-CT, PCA, KNN: 0.65±0.08). These runs differ in two factors unrelated to SSL: imaging modality (PET vs CT) and classifier (MLP vs KNN). A paired t-test across CV folds cannot validly compare these two models because the paired entries are not the same model under the two strategies; moreover, the SSL fold results reuse the same 408 HNCa cases, making the five folds non-independent and the paired test anti-conservative. The main text does not report the SL accuracy for the exact SSL-winning configuration (DRF-PET+MLP) or the SSL accuracy for the SL-winning configuration (DRF-CT+KNN), so the reader cannot determine whether the gain is due to pseudo-labeling or to the change in base model. Without matched comparisons that hold feature set, modality, and classifier fixed while toggling only the pseudo-labeled augmentation, the headline accuracy improvement is not attributable to SSL.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"This manuscript proposes a semi-supervised learning (SSL) pseudo-labeling framework for lung cancer (LCa) overall-survival prediction using handcrafted radiomic features (HRF) and deep radiomic features (DRF) from PET/CT. The authors extract features from 199 LCa patients and 408 head-and-neck cancer (HNCa) patients, pseudo-label the HNCa cases with a Random Forest trained on the LCa training folds, and compare SSL to supervised learning (SL) across PCA+classifier hybrids. They also perform survival analysis using PCA combined with four survival algorithms, reporting that SSL with DRF-PET and MLP reaches 0.85±0.05 accuracy versus 0.65±0.08 for SL with DRF-CT and KNN, and that CT-based CWGB survival models reach a c-index of about 0.80 with external testing.","tokens_in":15962,"tokens_out":5274,"duration_ms":45881,"significance":"If the central result were established, the study would offer a practical recipe for reducing labeled-data requirements in radiomics-based outcome prediction, and the finding that CT alone can match or exceed PET in SSL settings would have cost and availability implications. The code is publicly shared, and the systematic comparison of four classifiers and four survival algorithms within a fixed pipeline is a useful engineering contribution. However, the headline SSL-versus-SL comparison is confounded, and the statistical evidence as presented is not sufficient to attribute the observed gains to pseudo-labeling; the paper's practical significance therefore hinges on a matched re-analysis.","major_comments":[{"comment":"The central claim that SSL outperforms SL is not supported by the comparison reported here. The best SSL configuration (DRF-PET, PCA, MLP: 0.85±0.05) is contrasted with the best SL configuration (DRF-CT, PCA, KNN: 0.65±0.08), which differs in both imaging modality (PET vs CT) and classifier (MLP vs KNN). A paired t-test across the five folds cannot validly attribute the difference to SSL, because the paired entries are not the same underlying model. The manuscript should report matched comparisons that hold modality, feature set, and classifier fixed and toggle only the pseudo-labeled augmentation, including the SL accuracy for DRF-PET+MLP and the SSL accuracy for DRF-CT+KNN; without such a table, the headline p<0.05 does not isolate pseudo-labeling.","section":"Section 3.1.2, Figure 4"},{"comment":"The five-fold paired t-test used throughout the results is anti-conservative because the same 408 pseudo-labeled HNCa samples are added to every training fold (Section 2.2). The fold-level accuracies are therefore not independent, so the effective sample size for the test is not five independent measurements. The authors should use a test that accounts for the shared pseudo-labeled data (e.g., a mixed-effects model with fold as a random effect, or repeated random train/test splits with re-fitting of the pseudo-labeling step), or at minimum report the per-fold accuracies and a confidence interval for the mean difference.","section":"Section 2.2 and Section 3.1"},{"comment":"The external nested testing C-indices are reported as '0.80±0' and '0.59±0.03' for the CWGB models. A zero standard deviation across external-test folds is implausible for a survival C-index and suggests either a single external split or a formatting error. The manuscript should state the number of external test patients (nominally 20% of 199, i.e., about 40) and report the distribution of the C-index across the external-test folds. As written, the '±0' notation obscures the reliability of the external validation, and the identical values reported for the HRF-CT and DRF-CT models are suspicious.","section":"Section 3.2"},{"comment":"The pseudo-labeling procedure assumes that the 408 HNCa cases can be labeled with a Random Forest trained on the LCa training folds, i.e., that the HNCa feature distribution is sufficiently aligned with LCa. The manuscript provides no diagnostic for this assumption: no overlap analysis of the feature spaces, no confidence threshold on the pseudo-labels, and no sensitivity analysis that, for example, retains only high-confidence pseudo-labeled cases. Given the known domain shift between head-and-neck and lung cancers, this is a correctness risk for the SSL gains; a quantitative check is needed before the claim 'SSL strategy outperformed SL method' can be accepted as generalizable.","section":"Section 2.2"}],"minor_comments":[{"comment":"The list of feature-set combinations contains 'DRF-CT plus HRF-CT' twice; the second instance should presumably be 'DRF-PET plus HRF-PET'.","section":"Section 2.2"},{"comment":"The legends include 'BR: Bagging Regression', but no Bagging Regression results are reported in the text; either remove the entry or report the corresponding results.","section":"Figures 3 and 4"},{"comment":"There are several typos in headings and front matter, for example 'MATRIALS AND METHODS', 'CONFILICT OF INTREST', and 'ACHNOWLEDGEMNTS'; these should be corrected.","section":"Throughout"},{"comment":"The abstract and conclusion state '0.85' in places without the '±0.05' reported in the main text; the uncertainty should be reported consistently.","section":"Section 3.1.2 and Abstract"},{"comment":"The sentence beginning 'As depicted in Figure 6, the HRF frameworks...' is ambiguous about which models the word 'respectively' refers to, and the repeated external-test C-index values for the HRF and DRF models should be disentangled.","section":"Section 3.2"}],"recommendation":"major_revision","confidential_remarks":"The paper relies heavily on the authors' own ViSERA and autoencoder tools, which is acceptable, but the reproducibility claim would be stronger if the feature-extraction code were included in the shared repository rather than only the predictor code. The main technical issue is the confounded SSL-versus-SL comparison; the revision should prioritize matched analyses and a corrected statistical test."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Short version: the paper does something reasonably new — using 408 head-and-neck cancer PET/CTs, pseudo-labeled by a Random Forest trained on lung cancer, to augment a 199-patient lung cancer survival model — and it ships code. That is worth a look. But the headline claim that SSL beats SL is not backed by a matched comparison, so the size of the gain, and even its direction, is not established.\n\nThe good: the idea is concrete and the domain-adaptation rationale (shared smoking etiology, overlapping SCC histology) is argued directly. The authors show SSL improving over SL across HRF/DRF and CT/PET configurations, not just one lucky run. Code and data (TCIA) are public. The survival analysis part is honestly labeled as supervised-only, and CT+DRF with CWGB reaching c-index ~0.80 is a plausible, if modest, result.\n\nThe soft spots are real. First, the stress-test note is correct: the abstract compares the best SSL run (DRF-PET, PCA+MLP, 0.85) with the best SL run (DRF-CT, PCA+KNN, 0.65). That changes modality and classifier at the same time. The p<0.05 claim is a paired t-test over five folds, but the folds are not independent because the same 408 pseudo-labeled HNCa cases are reused in every fold, and the test cannot validly compare two different models. The paper never reports SL accuracy for the SSL-winning configuration (DRF-PET+MLP) or SSL accuracy for the SL-winning configuration (DRF-CT+KNN), so we cannot tell how much of the gain is pseudo-labeling versus a better base model. Second, the external test set is ~40 patients (20% of 199), and the paper reports external values like 0.80±0, which looks like a typo or a too-small sample. Third, the supplement with methods and hyperparameters is not in the preprint, so reproducibility is partially blocked. Fourth, the SSL pseudo-labels are used with no check of distribution shift; the authors acknowledge the noise risk but don't quantify it.\n\nOverall: this is a plausible incremental result in radiomics, not a breakthrough. It deserves a serious referee because the question matters and the code is there, but the revision needs matched comparisons, a proper validation protocol, and better reporting. I would not cite it as evidence until those are fixed.","headline":"Cross-cancer pseudo-labeling for lung cancer survival is a reasonable idea, but the headline SSL-vs-SL comparison isn't matched, so the claimed gain isn't yet attributable to SSL.","tokens_in":16569,"tokens_out":1884,"would_cite":false,"duration_ms":16741,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Adding pseudo-labeled head-and-neck scans to lung-cancer training lifts two-year survival prediction accuracy from 0.65 to 0.85, the authors report.","keywords":["lung cancer","semi-supervised learning","pseudo-labeling","radiomics","PET/CT imaging","overall survival prediction","deep radiomic features","survival analysis"],"falsifier":"Run the same SSL pipeline with pseudo-labels randomly shuffled on the 408 head-and-neck cases; if accuracy stays at 0.85, the reported gain is not caused by pseudo-label content. Alternatively, evaluate the full pipeline on an independent lung-cancer cohort and require SSL to beat supervised-only training with p<0.05.","tokens_in":15498,"feed_emoji":"🫁","tokens_out":8125,"duration_ms":70278,"temperature":0.7,"pith_summary":"The paper tries to establish that semi-supervised pseudo-labeling can break the small-labeled-data bottleneck in lung-cancer survival prediction by borrowing PET/CT images from head-and-neck cancer, a related disease with overlapping risk factors and radiomic appearance. The authors report that adding 408 pseudo-labeled head-and-neck cases to 199 labeled lung-cancer cases raises binary two-year overall-survival classification accuracy from 0.65±0.08 (supervised, deep features from CT, PCA+KNN) to 0.85±0.05 (semi-supervised, deep features from PET, PCA+MLP), a difference they call statistically significant. On the survival-analysis side, they report that PCA with Component-wise Gradient Boosting Survival Analysis on CT-derived features reaches an average concordance index of about 0.80 with log-rank p<0.001, confirmed on external testing. They conclude that combining deep radiomic features with semi-supervised learning lets a single modality, especially CT, reach high predictive performance in data-limited settings.","feed_headline":"Pseudo-labeling lifts lung cancer survival prediction to 0.85","feed_subtitle":"Adding 408 head-and-neck scans labeled by a lung-trained model beats supervised-only training, study finds.","key_machinery":"The load-bearing mechanism is pseudo-labeling with a cross-disease data pool. A Random Forest trained on four training folds of 199 lung-cancer cases assigns each of 408 head-and-neck PET/CT cases a binary label (survived beyond two years or not); those pseudo-labeled cases are added to the training folds, and the expanded set is fed through PCA dimensionality reduction into one of four classifiers. The other central component is the feature representation: 215 handcrafted radiomic features per image and 1,024 deep radiomic features from the bottleneck of a 3D autoencoder, both extracted from manually segmented primary tumors. For survival analysis, the key mechanism is PCA coupled with Component-wise Gradient Boosting Survival Analysis on CT-derived features, which stratifies patients into low- and high-risk groups with concordance around 0.80.","core_discovery":"The paper's central claim is that a semi-supervised pseudo-labeling scheme can improve lung-cancer overall-survival classification beyond what supervised learning achieves on the same labeled data, and that deep radiomic features extracted from PET/CT drive the gain. Specifically, a Random Forest trained on four of five folds of 199 lung-cancer cases labels 408 head-and-neck cancer cases; these pseudo-labeled cases are folded into the training set, and PCA plus an MLP classifier on 1,024-dimensional deep features from PET reaches an average accuracy of 0.85±0.05, versus 0.65±0.08 for supervised PCA+KNN on deep features from CT. On the survival-analysis side, the paper reports that PCA with Component-wise Gradient Boosting Survival Analysis on CT-derived features, whether handcrafted or deep, gives an average c-index around 0.80 with log-rank p<0.001, validated on external testing. The authors interpret these results as showing that SSL with related-disease data plus deep features can make CT-only prediction nearly as accurate as PET-based prediction in small-sample settings.","pith_inferences":["We infer that the gain should depend on how much the head-and-neck feature distribution overlaps the lung-cancer distribution; measuring that overlap directly and correlating it with the accuracy lift would give a sharper test than the paired t-test the paper uses.","We infer that confidence-thresholding pseudo-labels (keeping only head-and-neck cases the Random Forest labels with high probability) would likely reduce the noise the authors acknowledge; this is a direct extension of their own limitation note.","We infer that the survival-analysis pipeline could be extended to SSL if follow-up times for head-and-neck patients became available, since the paper currently excludes SSL from survival tasks for that reason; such an extension would require a principled way to pseudo-label censored times."],"forward_implications":["If the finding holds, semi-supervised pseudo-labeling could let hospitals with small labeled cohorts draw on unlabeled or differently-labeled scans from related cancers.","If the finding holds, CT-only prediction with deep features plus SSL could offer a cheaper, more accessible prognostic alternative in settings where PET is unavailable.","If the finding holds, the specific recipe (PCA+MLP, deep features from PET, SSL) provides a concrete baseline for future lung-cancer overall-survival prediction studies.","If the finding holds, the reported survival models (CT features, PCA+CWGB, c-index around 0.80) could be used to stratify patients into risk groups for treatment planning."],"supporting_citations":[{"why":"Semi-supervised self-training for lung cancer prediction, the methodological precursor this paper adapts to cross-disease pseudo-labeling.","marker":"[16]"},{"why":"Establishes shared risk factors between lung and head-and-neck cancers, motivating the use of head-and-neck data for lung-cancer prediction.","marker":"[18]"},{"why":"Supports shared tobacco-related carcinogenesis and tumor-microenvironment similarities between the two diseases.","marker":"[19]"},{"why":"Reports elevated risk of second primary lung cancer in head-and-neck cancer survivors, evidence of disease relatedness.","marker":"[20]"},{"why":"Supports shared biological susceptibility underlying both cancers, part of the rationale for cross-disease transfer.","marker":"[21]"},{"why":"Provides radiomics-based risk assessment in head-and-neck cancer, underpinning the feature-level transfer.","marker":"[22]"},{"why":"Describes the standardized radiomics environment used to extract handcrafted and deep radiomic features.","marker":"[63]"},{"why":"Supplies the component-wise gradient boosting survival analysis algorithm used in the top-performing survival model.","marker":"[70]"},{"why":"Provides the high-dimensional Hotelling's T-squared test used to compare high- and low-risk feature distributions.","marker":"[73]"}],"fun_headline_variants":["Pseudo-labeling boosts lung cancer survival accuracy to 0.85","Semi-supervised pseudo-labels lift lung cancer survival to 0.85","Pseudo-labeling diverse scans lifts lung cancer survival to 0.85","Semi-supervised pseudo-labeling hits 0.85 lung cancer survival"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing assumption is that head-and-neck cancer images labeled by a lung-cancer-trained Random Forest are similar enough to lung-cancer images that the pseudo-labels add useful supervision rather than systematic noise.","fun_headline_variants_meta":{"raw":{"variants":["Pseudo-labeling boosts lung cancer survival accuracy to 0.85","Semi-supervised pseudo-labels lift lung cancer survival to 0.85","Pseudo-labeling diverse scans lifts lung cancer survival to 0.85","Semi-supervised pseudo-labeling hits 0.85 lung cancer survival"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000678,"raw_usage":{"total_tokens":3181,"prompt_tokens":1144,"completion_tokens":2037,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":760,"completion_tokens_details":{"reasoning_tokens":1954}},"tokens_in":760,"tokens_out":2037,"duration_ms":13427,"temperature":1.0,"reasoning_tokens":1954,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-12T12:38:42.735836+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run the same SSL pipeline with pseudo-labels randomly shuffled on the 408 head-and-neck cases; if accuracy stays at 0.85, the reported gain is not caused by pseudo-label content. Alternatively, evaluate the full pipeline on an independent lung-cancer cohort and require SSL to beat supervised-only training with p<0.05.","supporting_citations":[{"cited_title":"A new semi-supervised self-training method for lung cancer prediction","cited_arxiv_id":"2012.09472","evidence_quote":"Semi-supervised self-training for lung cancer prediction, the methodological precursor this paper adapts to cross-disease pseudo-labeling."},{"cited_title":"Risk factors for lung cancer worldwide,","cited_arxiv_id":null,"evidence_quote":"Establishes shared risk factors between lung and head-and-neck cancers, motivating the use of head-and-neck data for lung-cancer prediction."},{"cited_title":"Tobacco -related carcinogenesis in head and neck cancer,","cited_arxiv_id":null,"evidence_quote":"Supports shared tobacco-related carcinogenesis and tumor-microenvironment similarities between the two diseases."},{"cited_title":"Cramer , J","cited_arxiv_id":null,"evidence_quote":"Reports elevated risk of second primary lung cancer in head-and-neck cancer survivors, evidence of disease relatedness."},{"cited_title":"A mendelian randomisation study in UK Biobank and international genetic consortia particip ants,","cited_arxiv_id":null,"evidence_quote":"Supports shared biological susceptibility underlying both cancers, part of the rationale for cross-disease transfer."},{"cited_title":"Radiomics strategies for risk assessment of tumour failure in head -and-neck cancer.,","cited_arxiv_id":null,"evidence_quote":"Provides radiomics-based risk assessment in head-and-neck cancer, underpinning the feature-level transfer."},{"cited_title":"ViSERA: Visualized & Standardized Environment for Radiomics Analysis - A Shareable, Executable, and Reproducible Workflow Generator,","cited_arxiv_id":null,"evidence_quote":"Describes the standardized radiomics environment used to extract handcrafted and deep radiomic features."},{"cited_title":"Gradient Boosted Models,","cited_arxiv_id":null,"evidence_quote":"Supplies the component-wise gradient boosting survival analysis algorithm used in the top-performing survival model."},{"cited_title":"Inference for the mean of large p small n data: A finite -sample high -dimensional generalization of Hotelling’s theorem,","cited_arxiv_id":null,"evidence_quote":"Provides the high-dimensional Hotelling's T-squared test used to compare high- and low-risk feature distributions."}],"review_version":1}