{"id":"9e67c38e-fc77-4ae5-be34-8b29660f43db","arxiv_id":"1908.06278","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":6,"one_line_summary":"OmiVAE, an end-to-end variational autoencoder with a classifier and chromosome-wise methylation encoding, reports 97.49% accuracy on 34-class TCGA pan-cancer classification and 97.88% on 33 tumor types.","lead":"This paper introduces OmiVAE, a variational autoencoder combined with a classifier that integrates gene expression and DNA methylation data to classify 33 cancer types and normal samples. On TCGA data the model reports 97.49% accuracy in 10-fold cross-validation and claims multi-omics input outperforms either omics type alone.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The headline accuracies hinge on whether unsupervised VAE pretraining is rerun inside each CV fold; the paper does not state this, and Figure 2 implies at least one unsupervised model is trained on all 9,081 samples.","rationale":"The paper is a straightforward, well-structured application of a VAE plus classifier to a substantial TCGA multi-omics dataset, and it reports useful internal comparisons (unsupervised versus end-to-end, single-omics versus multi-omics). The model architecture and loss are described in enough detail to be reimplemented, and a public code link is provided, which is a real independent check even though I did not execute it here. My reading confirms the reader's weakest assumption: the text never explicitly states that the unsupervised pretraining phase is nested inside the 10-fold cross-validation. The 2-D visualization experiment in Section IV-A is explicitly trained on all 9,081 samples, and the same section describes the unsupervised phase as a pretraining step whose encoder/decoder parameters are transferred to the supervised phase. If that pretraining is done once on the full dataset, the reported accuracies are not proper cross-validated estimates. This is a genuine ambiguity in the manuscript rather than an internal contradiction, and it can be resolved by inspecting the released code or by rerunning the experiment with nested pretraining. Since the concern is real but checkable, and since the reader already conditioned the verdict on exactly this issue, I keep the CONDITIONAL verdict unchanged. I do not see a reason to reject the paper outright, and I would not accept it unconditionally until the nested-CV question is resolved.","tokens_in":10285,"tokens_out":7061,"duration_ms":75427,"concrete_test":"Inspect the public GitHub repository (github.com/zhangxiaoyu11/OmiVAE) and locate the training loop for the supervised classification experiments. Check whether the unsupervised VAE pretraining is invoked inside each StratifiedKFold iteration on the training folds only, or once before the folds on all 9,081 samples. If the code does not make this explicit, instrument it and rerun the 34-class experiment with pretraining nested per fold: for each fold, pretrain the VAE on the 9 training folds, fine-tune encoder/decoder/classifier on the same 9 folds with early stopping on a validation subset, then evaluate on the held-out fold. Compare the mean accuracy to 97.49%; a drop of more than about 1 percentage point would confirm the optimism and require revising the headline.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The 97.49% (34-class) and 97.88% (33-class) numbers are valid held-out estimates only if the unsupervised VAE pretraining is repeated inside each fold of the stratified 10-fold cross-validation, using only the 9 training folds. Section IV states only that the data were 'randomly divided into training, validation and testing sets' and that stratified 10-fold CV was used, and Section IV-B says encoder/decoder parameters 'learned in the unsupervised phase were transferred to the supervised phase'; it never says this transfer is per-fold. In the same section, the unsupervised 2-D embedding shown in Figure 2 is described as learned from 'all 9,081 samples', which demonstrates that at least one unsupervised-phase model is fit on the full dataset. If the 128-d unsupervised pretraining used for the supervised classifier was likewise fit once on all samples before the 10 folds, then the encoder weights have already adapted to the test-fold feature distributions, so the 97.49% and 97.88% accuracies are not independent generalization estimates and are likely optimistically biased. This is the load-bearing assumption for the central accuracy claim; the weaker baselines comparison (cross-paper numbers from Li, Lyu, Mostavi) is a second, separate issue but would not by itself undermine the internal accuracy figure.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes OmiVAE, an end-to-end variational autoencoder combined with a classifier for pan-cancer classification from multi-omics data (gene expression and DNA methylation) on TCGA. The model is trained in two phases: an unsupervised phase with only the VAE reconstruction loss, followed by a supervised phase in which the encoder output feeds a classifier. The authors report 97.49% average accuracy for 34 classes (33 tumour types plus normal) and 97.88% for 33 tumour types after stratified 10-fold cross-validation, and claim that multi-omics input outperforms either single-omics input and that the model outperforms previously reported methods.","tokens_in":10520,"tokens_out":6387,"duration_ms":63058,"significance":"If the evaluation protocol is sound, the paper makes a useful empirical contribution: it demonstrates that a VAE-based latent representation can separate tumour types from multi-omics data, and it provides a public implementation. The explicit comparison between single-omics and multi-omics inputs is valuable, as is the quantitative evaluation of unsupervised embeddings against PCA, t-SNE, and UMAP. However, the central accuracy claims hinge on whether the unsupervised pretraining is properly nested inside the cross-validation folds, which the manuscript does not make clear. The cross-paper comparisons with prior methods are also weaker than the internal comparisons. These issues make the manuscript suitable for major revision rather than acceptance in its current form.","major_comments":[{"comment":"The text does not state that the unsupervised VAE pretraining is repeated inside each fold of the 10-fold cross-validation. It says only that the data were 'randomly divided into training, validation and testing sets' and that encoder/decoder parameters learned in the unsupervised phase were transferred to the supervised phase. If the pretraining is performed once on all 9,081 samples — as the description of Figure 2 suggests for the 2D embedding — then the test-fold feature distributions have already been seen by the encoder before supervised training begins, so the reported 97.49% and 97.88% accuracies are not standard held-out generalization estimates and are likely optimistically biased. Please specify clearly that the entire unsupervised-plus-supervised pipeline is nested within each fold, or re-run the evaluation with per-fold pretraining.","section":"Section IV-B"},{"comment":"The unsupervised-phase comparison suffers from the same ambiguity. The 2D embedding is described as learned from 'all 9,081 samples', and then an SVM is evaluated on the resulting representations with 10-fold cross-validation. If the dimensionality-reduction method (OmiVAE, PCA, KPCA, t-SNE, or UMAP) is fit on the full dataset before the SVM folds are created, the test representations are not independent of the training data. The dimensionality-reduction step must be fit on the training fold only for each of the 10 folds, and this must be stated explicitly for the reported Table I numbers to be valid.","section":"Section IV-A and Table I"},{"comment":"The claim that OmiVAE 'outperformed other existing methods' is based on accuracies reported in references [16]-[18], which are gene-expression-only models evaluated on possibly different sample sets, preprocessing pipelines, and cross-validation schemes. These are not direct comparisons under a common protocol. To support the superiority claim, the baseline methods should be reimplemented and evaluated on the same preprocessed data (or at least the same expression-only subset) under identical cross-validation, or the claim should be softened to state that the reported accuracies are higher than previously reported figures.","section":"Section IV-B, Tables II and III"},{"comment":"The reported advantage of multi-omics over single-omics (97.49% versus 96.37% for 34 classes; 97.88% versus 96.86/96.89% for 33 classes) is not accompanied by any statistical significance test or confidence interval beyond the standard deviations across the 10 folds. Given the small margins, please add a paired significance test (for example, McNemar's test on the combined predictions or a paired test across folds) before concluding that complementary information from multiple omics types improves classification.","section":"Section IV-B, Tables II and III"}],"minor_comments":[{"comment":"The abstract and the introduction report 96.42% accuracy for gene-expression-only OmiVAE, while Table II reports 96.37%; please correct this inconsistency.","section":"Abstract and Section I"},{"comment":"The total loss is defined with weights α and β, and the text states that β is 0 in the unsupervised phase and 1 in the supervised phase, but the value of α is never reported. Please provide all hyperparameters, including α, the latent dimension, learning rate, batch size, and the early-stopping rule, for reproducibility.","section":"Section III-C, Equation (7)"},{"comment":"For the rows labeled 'OmiVAE+SVM (unsupervised phase)', the latent dimension is not specified; Table I uses a 2D bottleneck, but Section IV-B says the latent dimension is 'normally set to 128' in the supervised phase. Please state which latent dimension is used for each table.","section":"Tables II and III"},{"comment":"The VAE reconstruction loss uses binary cross-entropy for the log2-transformed and normalized gene-expression values, which are continuous rather than Bernoulli; please justify this choice or switch to a continuous loss such as mean squared error for the expression component.","section":"Section III-C, Equation (4)"},{"comment":"The text says the data were 'randomly divided into training, validation and testing sets' and also that stratified 10-fold cross-validation was used; please clarify how the validation set is formed within each fold and how early stopping is applied relative to training and testing.","section":"Section IV"},{"comment":"There are several typographical and LaTeX issues: 'In OmiV AE, we combines' in Section III-C, the GitHub URL rendered as 'OmiV AE' with a space, and 'Beat values' instead of 'Beta values' in Section III-B.","section":"General presentation"}],"recommendation":"major_revision","confidential_remarks":"The main risk is the pretraining/CV nesting ambiguity. If the authors confirm that the unsupervised phase is repeated per fold, the headline numbers may stand; if not, the accuracies need to be recomputed with a properly nested protocol. The cross-paper comparison with prior methods is likely to be considered weak by the journal, so I would encourage the editors to require a direct reimplementation or a clearly softened claim."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"OmiVAE is a practical, reusable multi-omics classifier, and the chromosome-block encoding of 392K methylation features is a neat engineering contribution. But the headline 97.49%/97.88% accuracies depend on an unstated cross-validation protocol for the unsupervised pretraining, and the \"better than existing methods\" line rides on cross-paper numbers. The reader's conditional verdict is right; the stress-test note identifies the exact load-bearing assumption.\n\nWhat's new and good: the model combines a VAE with a classifier in two phases, and the chromosome-wise block encoding makes it feasible to consume genome-wide methylation data. The paper runs a clean internal comparison: multi-omics beats either single omics in both the unsupervised+SVM and end-to-end modes, and the 2D embeddings show sensible hierarchical structure. The code and data are public, which counts for something.\n\nWhere it wobbles: the biggest issue is that the paper never states whether the unsupervised pretraining is repeated inside each of the 10 folds. Section IV says data were \"randomly divided into training, validation and testing sets\" and Section IV-B says parameters learned in the unsupervised phase were transferred to the supervised phase, but there's no sentence saying this transfer is per-fold. The 2D embedding in Figure 2 is explicitly described as learned from all 9,081 samples, so at least one full-data unsupervised model exists. If the 128-d pretraining used for classification is also fit on all samples, the test folds are not independent and the numbers are optimistically biased. This is not a hypothetical nit; it's the difference between a solid benchmark result and a leaky one. The authors need to either confirm that each fold does its own pretraining or re-run the experiment that way.\n\nSecond issue: the comparisons to Li, Lyu, and Mostavi are just numbers from those papers, not re-run baselines. Different preprocessing and data versions can move these numbers by a few points, so the claim of superiority is credible only as a rough comparison. That's a softer problem, and it doesn't damage the internal multi-omics vs single-omics comparison.\n\nMinor: hyperparameter sensitivity is not reported, but that's a normal omission for an application paper.\n\nBottom line: this is a genuinely useful application paper with a solid architecture idea and a reproducible evaluation setup -- provided the CV nesting is resolved. I'd send it to a serious referee, with a specific instruction to ask for the per-fold pretraining clarification. If the authors can confirm it, the numbers stand; if not, the abstract needs to be reworked.","headline":"OmiVAE is a genuinely useful practical multi-omics classifier, but the central accuracy numbers hinge on an unstated cross-validation nesting assumption that the paper never confirms.","tokens_in":11081,"tokens_out":3253,"would_cite":false,"duration_ms":32582,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"OmiVAE, an end-to-end variational autoencoder plus classifier, reaches 97.49% average accuracy across 33 tumour types and normal samples from a pan-cancer multi-omics dataset, and beats single-omics models.","keywords":["variational autoencoder","pan-cancer classification","multi-omics integration","DNA methylation","gene expression","deep learning","latent space","pan-cancer genomics"],"falsifier":"Re-run the 10-fold experiment with the autoencoder trained from scratch on each training fold alone, leaving the test fold out of both the unsupervised and supervised phases; if the 34-class average falls materially below 97.49% or below the single-omics 96.37%, the claimed multi-omics advantage is partly an artifact of pre-training on the test folds.","tokens_in":10042,"feed_emoji":"🧬","tokens_out":9219,"duration_ms":84955,"temperature":0.7,"pith_summary":"OmiVAE is built to solve a mismatch: cancer omics profiles have tens of thousands to hundreds of thousands of molecular features but only thousands of labelled samples, so classifiers overfit. The paper's proposal is to train a variational autoencoder first without labels, letting a low-dimensional latent space organise samples into a hierarchy that mirrors organs, subtypes, and normal tissue, then attach a classifier to that latent space and fine-tune with labels. On 9,081 pan-cancer samples covering 33 tumour types plus normal tissue, the end-to-end model reaches 97.49% average accuracy after stratified 10-fold cross-validation, and 97.88% when normal samples are excluded. In both its unsupervised and end-to-end modes, combining DNA methylation with gene expression outperforms using either alone. If the result holds, integrated multi-omics classification can be improved by a single architecture that does representation learning and classification together, rather than by manual feature selection followed by a separate classifier.","feed_headline":"One VAE model classifies 33 cancer types and normal tissue at 97.49%","feed_subtitle":"Combining gene expression and DNA methylation in a single end-to-end network beats single-omics and prior pan-cancer classifiers.","key_machinery":"The load-bearing machine is the joint loss $\\mathcal{L}_{\\mathrm{total}} = \\alpha \\mathcal{L}_{\\mathrm{vae}} + \\beta \\mathcal{L}_{\\mathrm{class}}$, coupling a variational-autoencoder reconstruction objective (binary cross-entropy on reconstructed methylation blocks and expression plus a KL term) with a cross-entropy classification term on the latent mean $\\mu$. Two design choices make it feasible: methylation probes are split into 23 fully-connected blocks by target chromosome before being merged, which cuts the parameter count and captures intra-chromosome structure; and the classifier branch regularises the latent space so it must both reconstruct the sample and separate classes. The two-phase schedule—$\\beta=0$ first, then $\\beta=1$ with transferred encoder and decoder weights—is what lets the model first organise unlabelled samples and then tune that organisation to the classification task.","core_discovery":"The central claim is that task-oriented feature extraction and classification can live in one network: OmiVAE's encoder compresses 450,804 molecular features into a 128-dimensional latent Gaussian, the decoder reconstructs the input, and a classifier attached to the latent mean predicts tissue of origin and normality. Training happens in two phases—an unsupervised VAE phase, after which the latent space already shows separable clusters for organs, tumour subtypes, and normal tissue, and a supervised phase in which the encoder weights are transferred and the classifier loss is added. In the paper's evaluation this end-to-end model reaches 97.49% average accuracy among 33 tumour types and normal samples (34 classes) and 97.88% among tumour types alone (33 classes), outperforming prior pan-cancer classifiers built from gene expression alone. The same comparison shows multi-omics input beating single-omics input in every mode, which the paper reads as evidence that the latent vector combines complementary information from DNA methylation and gene expression.","pith_inferences":["A natural next test, which the paper does not run, is to ablate the classifier branch by keeping $\\beta=0$ throughout and using the same latent dimension; that would isolate how much of the 97.49% comes from the classifier loss as opposed to the VAE's unsupervised geometry.","The chromosome-block encoding suggests a general recipe for other genomics assays with positional structure: group features by genomic location when feeding a VAE; this could be tried on chromatin-state or other genome-wide data.","If the multi-omics gain replicates on independent cohorts outside the paper's dataset, the latent vector might serve as a shared representation for downstream tasks such as survival or drug response, not just tissue classification."],"forward_implications":["Combining gene expression and DNA methylation in one latent space improves classification over either omics alone, in both the unsupervised-embedding and end-to-end settings.","Because the unsupervised phase is a parametric encoder, a trained OmiVAE can embed new samples without re-running the whole dataset, unlike t-SNE.","The latent structure learned without labels separates tissues, tumour subtypes, and normal samples, so the model doubles as an exploratory tool for unannotated cohorts.","End-to-end training with the classifier branch beats feeding OmiVAE's unsupervised embeddings to an SVM, showing that task-oriented fine-tuning of the latent space contributes to the accuracy gain.","On the same pan-cancer task, the reported numbers exceed prior classifiers that used gene expression only."],"supporting_citations":[{"why":"Supplies the variational autoencoder objective and reparameterization trick that the encoder-decoder part of OmiVAE is built on.","marker":"[4]"},{"why":"Supplies the pan-cancer multi-omics dataset with 33 tumour types and normal samples from which all training and test splits are drawn.","marker":"[5]"},{"why":"Previous pan-cancer classifier on gene expression; its 95.6% accuracy is a baseline OmiVAE claims to exceed.","marker":"[16]"},{"why":"Previous CNN pan-cancer classifier on gene expression; the 95.59% result is compared against.","marker":"[17]"},{"why":"Previous CNN model evaluated on the 34-class tumour-plus-normal task; its 95.0% is the direct comparison for OmiVAE's 97.49%.","marker":"[18]"},{"why":"Baseline dimensionality reduction method whose 2D embeddings are compared with OmiVAE's unsupervised phase.","marker":"[23]"},{"why":"Another baseline dimensionality reduction method included in the same unsupervised-phase comparison.","marker":"[24]"}],"fun_headline_variants":["Multi-omics VAE hits 97.49% across 33 cancers and normal","OmiVAE: one network, two omics, 97% pan-cancer accuracy","End-to-end VAE fuses methylation and expression for cancer ID","Deep VAE with classifier nails 33 cancer types at 97.49%","Multi-omics autoencoder beats single-omics in pan-cancer test"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The accuracy numbers assume each test group is kept out of the entire training process, including the label-free pre-training step; if that step saw all samples before the test splits were made, the reported figures are optimistic.","fun_headline_variants_meta":{"raw":{"variants":["Multi-omics VAE hits 97.49% across 33 cancers and normal","OmiVAE: one network, two omics, 97% pan-cancer accuracy","End-to-end VAE fuses methylation and expression for cancer ID","Deep VAE with classifier nails 33 cancer types at 97.49%","Multi-omics autoencoder beats single-omics in pan-cancer test"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000847,"raw_usage":{"total_tokens":3722,"prompt_tokens":1019,"completion_tokens":2703,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":635,"completion_tokens_details":{"reasoning_tokens":2600}},"tokens_in":635,"tokens_out":2703,"duration_ms":17025,"temperature":1.0,"reasoning_tokens":2600,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-14T12:51:56.205383+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Re-run the 10-fold experiment with the autoencoder trained from scratch on each training fold alone, leaving the test fold out of both the unsupervised and supervised phases; if the 34-class average falls materially below 97.49% or below the single-omics 96.37%, the claimed multi-omics advantage is partly an artifact of pre-training on the test folds.","supporting_citations":[{"cited_title":"The cancer genome atlas pan-cancer analysis project,","cited_arxiv_id":null,"evidence_quote":"Supplies the pan-cancer multi-omics dataset with 33 tumour types and normal samples from which all training and test splits are drawn."},{"cited_title":"A comprehensive genomic pan-cancer classification using the cancer genome atlas gene expression data,","cited_arxiv_id":null,"evidence_quote":"Previous pan-cancer classifier on gene expression; its 95.6% accuracy is a baseline OmiVAE claims to exceed."},{"cited_title":"Deep learning based tumor type classification using gene expression data,","cited_arxiv_id":null,"evidence_quote":"Previous CNN pan-cancer classifier on gene expression; the 95.59% result is compared against."},{"cited_title":"Convolutional neural network models for cancer type prediction based on gene expression","cited_arxiv_id":"1906.07794","evidence_quote":"Previous CNN model evaluated on the 34-class tumour-plus-normal task; its 95.0% is the direct comparison for OmiVAE's 97.49%."}],"review_version":1}