{"id":"86796b1c-e780-453d-b151-f9e24661a0a4","arxiv_id":"2506.16921","paper_version":1,"verdict":"REJECT","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"high","formal_verification":"none","parameter_count":4,"one_line_summary":"A GCN with wavelet-denoised targets predicts activity for the TEAS enzyme on a 9-site hypercube, with test R-squared up to 0.66.","lead":"This paper combines wavelet denoising with a graph convolutional network to predict enzyme activity for 419 mutant sequences of a tobacco enzyme. It reports that denoising improves test-set accuracy, but the evaluation measures prediction of denoised values, not the raw experimental data.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Reported generalization is not established: the 1-D DWT in Sec. 2.2 is applied to all 419 measured values with unspecified ordering/missing-value handling, and the resulting denoised values are used as both training and test targets; test R² therefore reflects fit to a data-dependent smoothed…","rationale":"The paper's stated goal is protein fitness estimation, and the strongest claim is generalization to unseen mutants. For that claim to hold, test performance must be measured against values that the model was not trained to reproduce and that are not altered by the test values themselves. The current protocol violates both requirements: the DWT is computed over the entire 419-point set before the train/test split, so training targets depend on test measurements, and the reported test R² is computed against denoised targets rather than raw experimental activity. These are not cosmetic issues; they directly determine the magnitude of the reported improvement. This is why I focused on the denoising pipeline rather than on the GCN architecture, which is standard and, taken alone, not the source of the problem. The reader's verdict of REJECT is supported: with the current protocol, the central claim is unverifiable. My concrete test would settle whether a corrected protocol preserves the result, and if it does, the paper could be revised to CONDITIONAL or ACCEPT. Until then, the reported numbers should not be read as evidence of fitness-prediction quality.","tokens_in":12369,"tokens_out":4319,"duration_ms":51485,"concrete_test":"Re-run the 4-EE experiment with a leak-free protocol: (i) apply the DWT separately to the 287 training labels only (using the exact Gray-code order and missing-value rule stated by the authors), and (ii) evaluate the trained model on the 132 raw, non-denoised test values. Also report the same split's R² against the denoised test targets for comparison. If raw-test R² is substantially below the reported 0.66, the headline result is an artifact of test-set-inclusive denoising; if raw-test R² remains approximately 0.66, the claim survives this check and the remaining question is only the unspecified ordering.","verdict_should_be":"REJECT","load_bearing_attack":"The central claim that EHCube4P predicts protein fitness depends on the validity of the wavelet-denoised labels used in Sec. 3.2. Eq. 2 defines the DWT over M=419 samples, but the paper never states the order in which the 419 measured sequences are fed to the transform, nor how the 93 missing vertices of the H(9,2) cube are handled. If only measured sequences are used, consecutive 'samples' can differ by more than one mutation, so the locality that Definition 2 claims for Gray-code ordering is lost; if all 512 positions are used, the missing values must be filled by some unstated rule. Either way the detail coefficients in Eq. 3 and the thresholded reconstruction are not uniquely defined by the data. More seriously, the denoised signal is a global function of all 419 raw measurements. Training labels for the 287 training vertices therefore contain information from the 132 test vertices whenever the wavelet support spans a gap or a nearby test vertex, and the test targets are themselves smoothed versions of the test measurements. Thus the reported R² values (0.66 for 4-EE, 0.50 for 5-EA/PSD) are not evidence that the model predicts unseen experimental fitness; they are evidence that the model fits a smoothed curve that was constructed with access to the test set. The absence of an evaluation on raw, unsmoothed test values makes the central claim unsupported.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The manuscript proposes EHCube4P, a graph convolutional network for protein fitness prediction. The method represents the sequence space of k binary mutation sites as a hypercube H(k,2), applies a 1-D discrete wavelet transform to denoise the 419 measured fitness values, and trains a two-layer GCN followed by an MLP with a smoothness-regularized loss to predict fitness for all 512 vertices. The authors report test R² values of 0.66, 0.50, 0.50, and 0.18 for four enzyme activities of TEAS, and an ablation study on the 4-EE activity. The central claim is that wavelet preprocessing plus graph convolution improves generalization and captures higher-order epistatic interactions.","tokens_in":12703,"tokens_out":7008,"duration_ms":73973,"significance":"If the reported performance were obtained under a valid evaluation protocol, the framework would be a useful contribution to semi-supervised fitness prediction on sparse combinatorial sequence spaces. The paper has several strengths: the hypercube graph formulation is natural for this dataset, the use of unlabeled vertices in aggregation is well motivated, and the ablation study explicitly isolates wavelet denoising, smoothness, and the MLP. However, the current evaluation is compromised by the use of globally denoised test targets and by underspecified preprocessing, so the empirical claims are not yet established.","major_comments":[{"comment":"The test-set evaluations are performed on wavelet-denoised activity values, and the denoising is a global operation over all 419 measured sequences. The detail coefficients in Eq. (2) and the universal threshold λ in Eq. (3) are computed from the full signal, so each denoised test target is a function of the entire dataset, including the training vertices and the other test vertices. Consequently, the reported test R² values (0.66 for 4-EE, 0.50 for 5-EA and PSD, 0.18 for MP) do not measure prediction of unseen experimental fitness; they measure agreement with a smoothed reconstruction that was built with access to the test measurements. To support the generalization claim, the authors should evaluate on raw experimental test values, or construct denoised targets using a procedure that does not use test data, and report both raw and denoised metrics.","section":"Sec. 3.2, Eqs. (2)-(4), Fig. 4"},{"comment":"The 1-D DWT is applied to M=419 measured fitness values, but the manuscript never states the ordering of these 419 sequences or how the 93 missing vertices of H(9,2) are handled. Definition 2 defines a Gray-code ordering on the full hypercube {0,1}^k, not on the measured subset; if only the measured vertices are ordered, consecutive entries can differ by more than one bit and the locality used to justify the transform is lost, while if the full 512-vertex ordering is used, the missing values must be imputed by an unstated rule. As a result, Eq. (2) and the denoised signals shown in Fig. 3 are not uniquely defined by the data as presented, and the preprocessing is not reproducible.","section":"Sec. 2.2, Definition 2, Fig. 3"},{"comment":"All reported results come from a single train/test split using seed 40 (287 training vertices, 132 test vertices) with no repeated runs, confidence intervals, or alternative splits. Because the test set is small and the split is arbitrary, the differences among activities and the ablation comparisons in Table 1 may be split-specific rather than stable properties of the method. The authors should report mean and standard deviation over multiple seeds and, ideally, multiple splits, and compare against a model trained on raw, undenoiosed targets to place the R² values in context.","section":"Sec. 3.2, Fig. 4, Table 1"},{"comment":"The claim that the model 'effectively captures higher-order mutational interactions' is not directly tested. The two GCN layers aggregate information up to distance two in the hypercube, but no experiment isolates this effect from the contributions of wavelet denoising and the smoothness loss. A comparison against a pairwise model (e.g., a one-layer GCN or a linear epistasis model) on the same preprocessed targets is needed before the higher-order interaction claim can be supported.","section":"Sec. 2.4, Eq. (9), Sec. 3.2"}],"minor_comments":[{"comment":"The caption contains the typo 'Denosing', and the x-axis label 'Mutants' does not state which ordering of the 419 sequences is used; this should be clarified.","section":"Fig. 3"},{"comment":"Equation (6) has notation errors: the neighborhood is written as 'N⟩' and the normalization term is typeset as '√dj √di' without a division sign; the intended formula appears to be the normalized aggregation in Eq. (7).","section":"Eq. (6)"},{"comment":"In Eq. (9), the smoothness weight wi,j is not defined; the authors should state explicitly that it is the edge weight 1/2^(i⊙j) introduced in Sec. 2.3.","section":"Eq. (9)"},{"comment":"The schedule for λ_smooth is given only in Sec. 3 ('min(10.0, epoch/100)'), while Table 1 reports experiments with λ_smooth in different ranges; the table should state the schedule used for each row.","section":"Sec. 3, Table 1"},{"comment":"The statement says the code will be released upon acceptance; providing the code and processed data with the revision would greatly improve reproducibility, especially given the underspecified preprocessing.","section":"Data Availability Statement"}],"recommendation":"major_revision","confidential_remarks":"The evaluation flaw is serious but addressable: re-running the analysis on raw targets with a well-defined ordering and multiple splits could support or refute the central claim. I do not see evidence of deliberate misconduct, but the current version substantially overstates generalization, and the preprocessing must be specified before any conclusions can be drawn."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Two things you should know about this paper. First, the combination of a hypercube GCN with wavelet denoising for fitness prediction is a reasonable idea, and the ablation study is a genuine attempt to isolate the contribution of each component. Second, the evaluation is not valid. The test targets are denoised values produced by a DWT run over all 419 measurements, including the test sequences, and the universal threshold in Eq. 3 depends on the median of all detail coefficients. So the reported test R² values (0.66, 0.50, 0.18) describe fit to a smoothed curve that was constructed with access to the test set, not prediction of unseen experimental fitness.\n\nWhat is new and worth credit: modeling the binary sequence space as H(9,2) and using two GCN layers plus an MLP is a straightforward but sensible architecture for this small-mutant library. The ablation on 4-EE is a plus: it shows denoising, smoothness loss, and MLP each matter under their setup. The authors are also honest about the minor-products case where performance collapses.\n\nThe soft spots are more than minor. The paper never states the 1-D ordering used for the DWT, nor how the 93 missing vertices are handled. That makes the denoised signal not uniquely defined. More importantly, the evaluation target is data-dependent: test labels are smoothed using test measurements. There are also no baseline comparisons on the same data (no ridge, random forest, or a simple epistasis model), and the results are based on a single random split. The claim that the method 'generalizes well across different enzyme activity datasets' is overstated; it is one enzyme with four product readouts.\n\nThere is also a conceptual tension: they want to capture higher-order epistasis, but wavelet thresholding removes local variation, which is exactly where epistatic signal lives. They acknowledge this in the conclusion but don't resolve it.\n\nWho this is for: someone working on fitness prediction for sparse binary mutational libraries might find the architecture idea worth a quick look, but the evidence doesn't support the central claim. As is, I would not send this to a serious referee. The right move is to ask the authors to redo the evaluation on raw test labels (or denoise using only training data), specify the ordering and missing-value handling, and add baselines. If that fixed the numbers, resubmission would make sense.","headline":"The architecture idea is reasonable, but the evaluation is invalid: test labels are wavelet-denoised with access to test data, so the reported R² does not measure prediction of unseen fitness.","tokens_in":13212,"tokens_out":2747,"would_cite":false,"duration_ms":30318,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":false},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Treating mutant sequences as a hypercube graph and denoising their fitness values with a wavelet transform before GCN training yields test R-squared of 0.66 versus 0.38 without denoising on a nine-site enzyme landscape.","keywords":["protein fitness prediction","epistasis","graph convolutional network","wavelet denoising","hypercube graph","semi-supervised regression"],"falsifier":"Evaluate the trained EHCube4P model directly against the raw, non-denoised experimental fitness values on the held-out 132 mutants; if agreement is markedly worse than against the denoised values, then the model is fitting an artifact of the wavelet pipeline rather than the measured fitness. A second check is to re-run the same pipeline with several different random permutations of the 419-sequence ordering; if test R-squared swings by more than the denoising gain, the ordering assumption, not epistasis, is carrying the result.","tokens_in":12184,"feed_emoji":"🧬","tokens_out":7180,"duration_ms":73671,"temperature":0.7,"pith_summary":"The paper tries to establish that protein fitness prediction improves when experimental noise is removed from a fitness signal before a graph neural network learns from it, and that the mutant space is naturally structured as a hypercube. The authors build EHCube4P, a two-layer graph convolution network on the hypercube H(9,2), where each vertex is a mutant sequence and edges connect single-mutation neighbors. They denoise the 419 measured TEAS enzyme fitness values with a one-dimensional discrete wavelet transform (Daubechies-3 basis) and then train the GCN with a smoothness regularizer to predict held-out and unmeasured mutants. The ablation study reports that removing the wavelet step drops test R-squared from 0.66 to 0.38 on the 4-EE product spectrum, which they interpret as evidence that denoising is the load-bearing component. If the claim is right, sparse and noisy fitness datasets could be modeled more reliably, easing the search for high-fitness protein variants.","feed_headline":"Denoising first lifts enzyme fitness prediction R2 from 0.38 to 0.66","feed_subtitle":"A two-layer graph network on the mutant hypercube captures epistasis once noise is filtered out of the training signal.","key_machinery":"The central machinery is the hypercube graph H(k,2) used as the domain for a fitness signal: each vertex is a binary sequence of length k (wild-type vs mutant at each site), edges connect single-mutation neighbors, and edge weights are 1/$2^{{i⊙j}}$, reflecting interaction order. Onto this graph the paper places two GCN layers with tanh activation and normalized adjacency, followed by an MLP, and learning happens only on labeled vertices with a loss that is 70 percent mean-squared error and 30 percent graph smoothness. Before training, the experimental fitness values are treated as a one-dimensional signal and denoised by a discrete wavelet transform with universal-threshold soft shrinkage of detail coefficients, which is the step the ablation shows to be load-bearing.","core_discovery":"On its own terms, the central claim is that combining wavelet-based denoising of fitness signals with a hypercube-structured GCN produces accurate fitness estimates for unmeasured mutants, and that the improvement is driven by the denoising step. Using the TEAS/HPS library of 419 experimentally characterized mutants, the model first linearizes the fitness values by Gray code ordering, applies first-level Daubechies-3 discrete wavelet thresholding with universal soft shrinkage, and then runs two GCN layers followed by an MLP. The reported test R-squared is 0.66 for 4-epi-eremophilene production versus 0.38 without denoising, with 0.50 on two other product spectra and 0.18 on minor products. The paper also shows that performance degrades as the local variance of the fitness signal increases, concluding that smoother landscapes are more predictable and that the two-layer GCN captures beyond-pairwise epistatic interactions.","pith_inferences":["Because the paper does not state the ordering of the 419 sequences used for the wavelet transform, the method's dependence on that ordering is untested; since one-dimensional wavelet denoising is not permutation invariant, I would expect a random permutation of the same fitness values to change the denoised target and therefore the reported R-squared.","The ablation compares the full model against raw fitness, but the training target is already denoised, so the 0.38 versus 0.66 gap measures both denoising and a changed target; a fairer test would train on raw values and evaluate on raw held-out values.","The framework could transfer directly to other binary combinatorial settings such as antibody libraries or regulatory DNA sequences, where a Hamming or hypercube graph and a scalar phenotypic readout are similarly available."],"forward_implications":["Wavelet denoising could become a standard cheap preprocessing step for fitness landscape datasets before any graph-based or even non-graph model is trained.","The graph structure supports semi-supervised regression: unlabeled mutants participate in message passing, so predictions can be made for the complete 2^k space from a sparse sample.","The observed link between signal smoothness and test R-squared implies the model could flag which fitness landscapes are too rugged to be reliably predicted, guiding experimentalists toward smoother measurable traits.","Two layers of aggregation offer a concrete way to model higher-order epistatic interactions without explicitly enumerating interaction terms, which would be combinatorially expensive at large k."],"supporting_citations":[{"why":"Supplies the TEAS/HPS mutant library dataset with 419 measured product spectra used as fitness labels.","marker":"[19]"},{"why":"Provides the discrete wavelet transform technique used for denoising the fitness signal.","marker":"[22]"},{"why":"Gives the soft-thresholding shrinkage procedure applied to detail coefficients.","marker":"[27]"},{"why":"Supplies the graph convolution propagation rule (Eq. 7) that the two-layer GCN builds on.","marker":"[32]"},{"why":"Supports the noise-plus-epistasis decomposition y = f + ε used to justify denoising.","marker":"[21]"},{"why":"Supplies the smoothness prior for sequence-function relationships, which the graph smoothness loss resembles.","marker":"[35]"}],"fun_headline_variants":["Wavelet denoising lifts enzyme fitness R2 from 0.38 to 0.66","Hypercube GCN predicts protein fitness after wavelet denoising","Denoised fitness signals improve hypercube GCN protein fitness estimates","Beyond pairwise epistasis via hypercube GCN on smooth fitness landscapes"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The wavelet denoising step assumes that a sensible one-dimensional ordering of the 419 measured mutants exists and that the 93 missing sequences can be ignored, but the paper never states what ordering is used or how missing values are treated.","fun_headline_variants_meta":{"raw":{"variants":["Wavelet denoising lifts enzyme fitness R2 from 0.38 to 0.66","Hypercube GCN predicts protein fitness after wavelet denoising","Denoised fitness signals improve hypercube GCN protein fitness estimates","Beyond pairwise epistasis via hypercube GCN on smooth fitness landscapes"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000274,"raw_usage":{"total_tokens":1670,"prompt_tokens":1009,"completion_tokens":661,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":625,"completion_tokens_details":{"reasoning_tokens":580}},"tokens_in":625,"tokens_out":661,"duration_ms":7324,"temperature":1.0,"reasoning_tokens":580,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-06T23:37:13.224181+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Evaluate the trained EHCube4P model directly against the raw, non-denoised experimental fitness values on the held-out 132 mutants; if agreement is markedly worse than against the denoised values, then the model is fitting an artifact of the wavelet pipeline rather than the measured fitness. A second check is to re-run the same pipeline with several different random permutations of the 419-sequence ordering; if test R-squared swings by more than the denoising gain, the ordering assumption, not epistasis, is carrying the result.","supporting_citations":[{"cited_title":"Quantitative exploration of the catalytic landscape separating divergent plant sesquiterpene synthases","cited_arxiv_id":null,"evidence_quote":"Supplies the TEAS/HPS mutant library dataset with 419 measured product spectra used as fitness labels."},{"cited_title":"Wavelets: Mathematical Theory","cited_arxiv_id":null,"evidence_quote":"Provides the discrete wavelet transform technique used for denoising the fitness signal."},{"cited_title":"Ideal spatial adaptation by wavelet shrinkage","cited_arxiv_id":null,"evidence_quote":"Gives the soft-thresholding shrinkage procedure applied to detail coefficients."},{"cited_title":"Semi-Supervised Classification with Graph Convolutional Networks","cited_arxiv_id":null,"evidence_quote":"Supplies the graph convolution propagation rule (Eq. 7) that the two-layer GCN builds on."},{"cited_title":"In: Sikosek T, editor","cited_arxiv_id":null,"evidence_quote":"Supports the noise-plus-epistasis decomposition y = f + ε used to justify denoising."},{"cited_title":"Minimum epistasis interpolation for sequence-function relationships","cited_arxiv_id":null,"evidence_quote":"Supplies the smoothness prior for sequence-function relationships, which the graph smoothness loss resembles."}],"review_version":1}