{"id":"711c06a1-51a8-484d-8e09-dd2b537e4f67","arxiv_id":"2605.18552","paper_version":1,"verdict":"UNVERDICTED","confidence":"LOW","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":1,"one_line_summary":"Introduces TEDBench benchmark and MiAE self-supervised framework that outperforms baselines for large-scale protein fold classification.","lead":"This paper introduces TEDBench, a large non-redundant benchmark for protein fold classification built from TED and clustered AlphaFold structures, along with a self-supervised MiAE model using high-ratio masking. A smart generalist might read it to understand scalable ways to learn useful representations from protein structures for applications in biology and medicine.","discovery_kind":"new_method","skeptic_critique":{"model":"grok-4.3","headline":"TEDBench non-redundancy via Foldseek clustering on AlphaFold data lacks explicit validation against fold-level leakage or prediction artifacts.","rationale":"The identified concern directly matches the reader's weakest assumption. Full-text details on clustering parameters and split statistics would be needed to confirm the benchmark's integrity; absent those, the outperformance claim remains conditional on unverified non-redundancy. No other internal inconsistency (e.g., in the MiAE masking or invariance description) rises to the same load-bearing level from the supplied abstract.","tokens_in":1693,"tokens_out":369,"duration_ms":34486,"concrete_test":"Compute all-vs-all TM-scores (or Foldseek E-values) between TEDBench train and test proteins; if any pair exceeds TM-score 0.5, create a de-duplicated test subset excluding those proteins and re-evaluate MiAE vs. baselines on the cleaned split. A >5% drop in MiAE accuracy relative to baselines would indicate the original gains partly relied on leakage.","verdict_should_be":"CONDITIONAL","load_bearing_attack":"The headline claim that MiAE scales and outperforms supervised/SOTA baselines on TEDBench (thereby establishing a strong recipe) requires TEDBench to be a genuinely non-redundant, unbiased test of fold classification. The construction combines TED domains with Foldseek-clustered AlphaFold structures, yet the abstract supplies no quantitative details on clustering thresholds, maximum inter-set TM-scores, sequence-identity cutoffs, or checks that train/test folds are structurally dissimilar. If residual similarities or AlphaFold-specific geometric biases persist, reported gains could reflect exploitation of dataset artifacts rather than learned fold representations. The additional CATH experimental benchmark is mentioned but its size and difficulty relative to TEDBench are unspecified, leaving the generalization argument under-supported.","agreement_with_reader":"agree"},"referee_report":{"model":"grok-4.3","summary":"The paper introduces TEDBench, a large-scale non-redundant benchmark for protein fold classification built from the Encyclopedia of Domains (TED) combined with Foldseek-clustered AlphaFold structures. It proposes Masked Invariant Autoencoders (MiAE), a self-supervised framework using up to 90% masking, an SE(3)-invariant encoder, and a lightweight decoder that reconstructs backbone coordinates. The central claim is that MiAE scales well and outperforms both supervised baselines and prior state-of-the-art methods on TEDBench while transferring to a curated experimental CATH v4.4 dataset; TEDBench is released publicly.","tokens_in":1858,"tokens_out":568,"duration_ms":48303,"significance":"If the results hold after validation of the benchmark, the work would be significant for structural bioinformatics and representation learning by supplying a scalable self-supervised recipe for fold classification and a new public benchmark that addresses duplication issues in existing resources. The high-masking invariant autoencoder approach and the github release of TEDBench constitute concrete contributions to reproducibility.","major_comments":[{"comment":"TEDBench construction section: the description of combining TED domains with Foldseek-clustered AlphaFold structures supplies no quantitative details on clustering thresholds, maximum inter-set TM-scores, sequence-identity cutoffs, or explicit checks that train/test folds are structurally dissimilar. Without these metrics it is impossible to confirm that reported gains reflect learned fold representations rather than residual similarities or AlphaFold geometric biases; this directly undermines the central outperformance claim.","section":"TEDBench construction"},{"comment":"Results and experimental sections: the claim that MiAE 'outperforms supervised counterparts and state-of-the-art baselines' on TEDBench and transfers to CATH requires tabulated metrics with error bars, ablation studies on the masking ratio, and precise baseline implementations. The abstract states clear superiority but the manuscript must supply these numbers to substantiate scaling behavior and rule out dataset-specific artifacts.","section":"Results"}],"minor_comments":[{"comment":"Abstract: the phrase 'up to 90%' masking ratio should be accompanied by the exact ratio(s) used in the reported experiments for clarity.","section":"Abstract"},{"comment":"Notation: the SE(3)-invariant encoder is introduced without an explicit equation or reference to the invariance property; adding a short formal definition would improve readability.","section":"Methods"}],"recommendation":"major_revision","confidential_remarks":"The manuscript fits the scope of a machine-learning-for-biology venue, but the absence of benchmark-validation statistics in the abstract and early sections makes the empirical claims harder to assess at first reading; the authors should be encouraged to add a dedicated validation subsection."},"author_rebuttal":{"model":"grok-4.3","summary":"We thank the referee for their thorough and constructive review of our manuscript. We have carefully considered the two major comments and provide point-by-point responses below. We agree that both points identify areas where the manuscript can be strengthened and plan to make the corresponding revisions.","responses":[{"response":"We agree that additional quantitative details on benchmark construction are required for full reproducibility and to allow independent verification that train/test splits are structurally dissimilar. In the revised manuscript we will expand the TEDBench construction section to report the specific Foldseek clustering parameters, maximum inter-set TM-scores, sequence-identity cutoffs, and the results of explicit structural dissimilarity checks (e.g., TM-align) between folds in different splits. These metrics will be presented in the main text together with a supplementary table summarizing the final dataset statistics and split properties.","revision_made":"yes","referee_comment":"[TEDBench construction] TEDBench construction section: the description of combining TED domains with Foldseek-clustered AlphaFold structures supplies no quantitative details on clustering thresholds, maximum inter-set TM-scores, sequence-identity cutoffs, or explicit checks that train/test folds are structurally dissimilar. Without these metrics it is impossible to confirm that reported gains reflect learned fold representations rather than residual similarities or AlphaFold geometric biases; this directly undermines the central outperformance claim."},{"response":"We acknowledge that the experimental reporting can be made more rigorous. In the revised manuscript we will augment the results section with complete tabulated performance metrics (including accuracy, F1, and other relevant measures) accompanied by error bars from multiple independent runs, dedicated ablation experiments varying the masking ratio, and precise descriptions of all baseline implementations (model architectures, hyperparameters, and training protocols). These additions will directly support the scaling claims and outperformance statements.","revision_made":"yes","referee_comment":"[Results] Results and experimental sections: the claim that MiAE 'outperforms supervised counterparts and state-of-the-art baselines' on TEDBench and transfers to CATH requires tabulated metrics with error bars, ablation studies on the masking ratio, and precise baseline implementations. The abstract states clear superiority but the manuscript must supply these numbers to substantiate scaling behavior and rule out dataset-specific artifacts."}],"tokens_in":1394,"tokens_out":475,"duration_ms":46000,"standing_objections":[]},"desk_editor":{"model":"grok-4.3","letter":"The main contribution is TEDBench, a larger deduplicated collection of protein domains drawn from TED and Foldseek-clustered AlphaFold structures, together with a masked autoencoder that runs at up to 90 percent masking on an SE(3)-invariant encoder and a lightweight coordinate decoder. The authors show that this recipe scales better than prior self-supervised methods and beats both supervised training and existing baselines on their new benchmark, with an additional transfer test on experimental CATH structures. That combination of a cleaned benchmark and a workable pretraining scheme is the part worth paying attention to. The high masking ratio is a concrete engineering choice that appears to work without collapsing the representation, which is not automatic for structure data. The decision to release the benchmark at the GitHub link is also useful on its own. The soft spot is the non-redundancy claim. The construction relies on Foldseek clustering, yet the abstract and stress-test note give no numbers on maximum TM-scores between train and test splits, sequence-identity cutoffs, or direct checks that folds remain structurally dissimilar. If those controls are missing or weak in the full paper, the reported gains could partly reflect dataset artifacts rather than learned fold features. The lack of error bars or ablation tables in the summary also makes it hard to judge how robust the outperformance really is. This is aimed at groups that build or benchmark structure-based protein models and need something bigger than the classic small fold sets. Readers who care about practical self-supervised recipes for geometric encoders will get value from the method details and the benchmark itself. It is solid enough to deserve a serious referee who can verify the clustering metrics and ask for the missing quantitative comparisons.","headline":"TEDBench plus the high-masking MiAE pretrainer is the real deliverable here, but the non-redundancy of the benchmark still needs explicit checks against structural leakage.","tokens_in":2338,"tokens_out":416,"would_cite":true,"duration_ms":33162,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":{"model":"grok-4.3","evidence":[],"headline":"MiAE masked reconstruction on SE(3) protein frames has no contact with RS cost, ladder or periodicity machinery","alignment":"orthogonal","rationale":"The paper's core objects (high-ratio masking of residue frames, SE(3)-invariant geometric attention, composite distance/direction reconstruction loss, linear probing on CATH topology labels) operate entirely within standard geometric deep learning. No J-cost functional, ratio-symmetric cost, φ-ladder spacing, 8-tick orbit, or parameter-free constant derivation appears. RS theorems on AbsoluteFloorClosure, Cost.FunctionalEquation (J-uniqueness), AlexanderDuality (D=3 forcing) and DimensionForcing are never paralleled. The domain (protein fold classification) lies outside the RS forcing chain.","tokens_in":59471,"confidence":"high","tokens_out":173,"duration_ms":31902,"cache_read_input_tokens":128,"cache_creation_input_tokens":0},"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"grok-4.3","headline":"Masked Invariant Autoencoders with up to 90 percent masking outperform supervised methods for protein fold classification on a new large benchmark.","keywords":["protein fold classification","self-supervised learning","masked autoencoders","protein structure","TEDBench","benchmark","SE(3) invariance","MiAE"],"falsifier":"Re-evaluate all models on a version of the benchmark rebuilt by clustering only experimental CATH structures with an independent algorithm; if MiAE loses its performance advantage or if substantial fold duplicates appear in the test split, the central scaling claim would be falsified.","tokens_in":2605,"feed_emoji":"🧬","tokens_out":729,"duration_ms":40697,"temperature":0.7,"pith_summary":"The paper constructs TEDBench, a large non-redundant test set for protein fold classification drawn from the Encyclopedia of Domains and Foldseek-clustered AlphaFold predictions. It introduces Masked Invariant Autoencoders, a self-supervised method that masks most input coordinates and trains an SE(3)-invariant encoder plus lightweight decoder to reconstruct the backbone. On this benchmark the approach scales better than prior self-supervised or supervised models and also transfers to experimental structures. A sympathetic reader would care because reliable large-scale fold classification supports understanding of biological function without depending on massive labeled datasets or enormous model sizes.","feed_headline":"Self-supervised autoencoders beat supervised ones on protein fold tasks","feed_subtitle":"A new non-redundant benchmark shows high-masking MiAE models scale better than large supervised networks for recognizing protein topologies.","key_machinery":"Masked Invariant Autoencoders (MiAE) that apply extreme masking to protein backbone coordinates and use an SE(3)-invariant encoder with a lightweight decoder to reconstruct the full structure from the masked latent representation.","core_discovery":"We introduce TEDBench, a large-scale, non-redundant benchmark for protein fold classification constructed from the Encyclopedia of Domains (TED) and Foldseek-clustered AlphaFold structures. We propose Masked Invariant Autoencoders (MiAE), a self-supervised framework for protein structure representation learning that uses an extremely high masking ratio of up to 90 percent with an SE(3)-invariant encoder and a lightweight decoder that reconstructs backbone coordinates from the latent representation and mask tokens. MiAE scales well and outperforms supervised counterparts and state-of-the-art baselines on TEDBench.","pith_inferences":["The same high-masking recipe might reduce the model size needed for other structure-related prediction tasks in biology.","If AlphaFold models contain systematic local errors, future benchmarks could combine experimental and predicted data with explicit error weighting.","Similar masking-based pretraining could be tested on other chain-like biomolecules such as RNA backbones."],"forward_implications":["Protein fold classification benefits from self-supervised pretraining on large unlabeled structure sets instead of relying solely on supervised training.","Very high masking ratios remain effective for learning useful structural representations in this domain.","The non-redundant benchmark construction removes duplicate-induced inflation that affected earlier protein fold datasets.","Gains from MiAE pretraining carry over when models are tested on experimental rather than predicted structures."],"fun_headline_variants":["MiAE uses high masking for protein fold classification on TEDBench","TEDBench constructed as non-redundant benchmark from AlphaFold data","Self-supervised MiAE with SE(3) invariant encoder for proteins","High masking MiAE pretrains protein representations at scale"],"cache_read_input_tokens":64,"weakest_assumption_plain":"The construction of TEDBench from TED and Foldseek-clustered AlphaFold structures yields a truly non-redundant and unbiased test of fold classification that generalizes beyond prediction artifacts in AlphaFold models.","fun_headline_variants_meta":{"raw":{"variants":["MiAE uses high masking for protein fold classification on TEDBench","TEDBench constructed as non-redundant benchmark from AlphaFold data","Self-supervised MiAE with SE(3) invariant encoder for proteins","High masking MiAE pretrains protein representations at scale"]},"model":"grok-4.3","cost_usd":0.015982,"raw_usage":{"total_tokens":6770,"prompt_tokens":701,"num_sources_used":0,"completion_tokens":70,"cost_in_usd_ticks":159815500,"prompt_tokens_details":{"text_tokens":701,"audio_tokens":0,"image_tokens":0,"cached_tokens":64},"completion_tokens_details":{"audio_tokens":0,"reasoning_tokens":5999,"accepted_prediction_tokens":0,"rejected_prediction_tokens":0}},"tokens_in":701,"tokens_out":70,"duration_ms":86727,"temperature":1.0,"reasoning_tokens":5999,"cache_read_input_tokens":64,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-05-20T13:11:49.648923+00:00","model_set":{"reader":"grok-4.3"},"falsifier":"Re-evaluate all models on a version of the benchmark rebuilt by clustering only experimental CATH structures with an independent algorithm; if MiAE loses its performance advantage or if substantial fold duplicates appear in the test split, the central scaling claim would be falsified.","supporting_citations":[],"review_version":1}