{"id":"31beb3e0-457f-4a83-96de-d89b963e9686","arxiv_id":"2509.03885","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":5,"one_line_summary":"Topotein introduces Protein Combinatorial Complexes and the SE(3)-equivariant TCPNet, improving fold classification and structure-only protein prediction over several geometric GNN baselines.","lead":"Topotein is a new machine learning framework that represents proteins hierarchically, from individual residues up through secondary structures to the whole protein, and passes messages across these levels. It shows gains on fold classification and structure-only protein tasks, though it does not beat all baselines on every benchmark.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Abstract's 'consistently outperforms' claim is contradicted by Table 1, and single-run experiments leave the remaining margins without statistical support.","rationale":"The paper's key contribution is a hierarchical, SE(3)-equivariant topological architecture, and its headline evidence is the empirical comparison. For the central claim to hold, TCPNet would need to convincingly beat all compared GGNNs across the four tasks. Table 1 undermines this directly: GVP-GNN wins inverse folding, and GCPNet wins two structure+sequence settings. This is not a consensus disagreement or a subtle failure mode; it is an internal inconsistency between the abstract and the reported results.\n\nEven where TCPNet is ahead, Appendix E explicitly acknowledges the absence of multiple trials and limited hyperparameter tuning. With single runs, a 3% fold-classification gap or a 0.4% F1 gap is not evidence of consistent superiority. The reader's DSSP-sensitivity concern is legitimate but secondary: it concerns robustness of the 2-cell construction if the method were otherwise validated, whereas the empirical claim fails before that point. Agreeing with the reader's overall CONDITIONAL verdict, I would not change the verdict, but I would sharpen the condition: the authors must correct the overclaim and provide seed-level variance estimates before the headline result can be accepted.","tokens_in":19054,"tokens_out":9613,"duration_ms":90993,"concrete_test":"Run each model (TCPNet, GCPNet, GVP-GNN, EGNN) with at least 5 independent seeds on all four tasks, using identical featurization (same node/edge features including 3Di one-hot and the same auxiliary 3Di denoising objective), and report mean±std. The decisive check is whether the Table 1 ordering persists: specifically, whether GVP-GNN remains ahead of TCPNet in inverse folding, whether GCPNet remains ahead in structure+sequence cellular-component and antibody tasks, and whether TCPNet's margins on fold classification are significant (e.g., paired bootstrap or t-test over seeds). If these orderings reproduce, the abstract's 'consistently outperforms' claim must be revised to reflect task-specific, often structure-only, superiority.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim (Abstract; Section 6.2) is that TCPNet 'consistently outperforms state-of-the-art geometric graph neural networks' and 'consistently outperforms its direct GGNN counterpart GCPNet across all tasks.' The paper's own Table 1 contradicts both statements. On inverse folding, GVP-GNN achieves accuracy 0.474 and perplexity 5.280 versus TCPNet's 0.441 and 5.822. On structure+sequence cellular component prediction, GCPNet achieves F1max 0.408 versus TCPNet's 0.398, and on structure+sequence antibody developability, GCPNet achieves AUPRC 0.878 versus TCPNet's 0.874. Thus the 'consistent' outperformance is false as stated even for the narrower GCPNet comparison.\n\nMoreover, Appendix E states: 'Due to computational constraints, we were unable to perform extensive hyperparameter tuning, try various feature combinations, or run multiple trials for each experiment.' With only one trial per model, the reported favorable margins—e.g., Fold accuracy 0.433 vs 0.384, structure-only cellular F1max 0.392 vs 0.388—cannot be distinguished from seed noise. The entire empirical case for the topological enhancement rests on these single-run differences. The DSSP-based 2-cell construction (Section 5.1) is a plausible secondary risk, but the primary load-bearing weakness is that the central empirical claim is both internally contradicted and statistically unverified.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper introduces Topotein, a topological deep learning framework for protein representation learning. It defines a Protein Combinatorial Complex (PCC) that organizes residues, interactions, secondary structure elements (SSEs), and whole proteins as cells of ranks 0--3, and proposes the Topology-Complete Perceptron Network (TCPNet), an SE(3)-equivariant message-passing architecture over PCCs. The main claimed contributions are the hierarchical data structure, the TCPNet architecture, and an empirical evaluation on four tasks: inverse folding, fold classification, cellular component prediction, and antibody developability. The paper compares TCPNet with geometric GNN baselines and with two topological adaptations, GVP-TNN and ETNN, reporting tables of test-set metrics.","tokens_in":19462,"tokens_out":2988,"duration_ms":29578,"significance":"If the central claims were established, the paper would be a useful contribution to protein representation learning: it articulates a principled way to incorporate secondary-structure-level information into equivariant message passing, and it provides an open-source implementation. The hierarchical PCC construction is a sensible and flexible representation that is well motivated by the role of SSE arrangements in protein classification. The architectural design of TCPNet, including outer-edge neighborhoods and edge-centric scalarization, is technically sound as an equivariant mechanism. However, the paper's empirical evidence is not currently sufficient to support the claimed consistent state-of-the-art performance, for the concrete reasons detailed below.","major_comments":[{"comment":"The abstract claims TCPNet 'consistently outperforms state-of-the-art geometric graph neural networks,' and Section 6.2 claims it 'consistently outperforms its direct GGNN counterpart GCPNet across all tasks.' Table 1 contradicts both statements. GVP-GNN beats TCPNet on inverse folding (perplexity 5.280 vs 5.822; accuracy 0.474 vs 0.441). GCPNet beats TCPNet on structure+sequence cellular component prediction (F1max 0.408 vs 0.398) and on structure+sequence antibody developability (AUPRC 0.878 vs 0.874). The 'consistent' outperformance claim must be corrected, or the evaluation must be rerun and reported in a way that supports the claim.","section":"Abstract and Section 6.2, Table 1"},{"comment":"The entire empirical case rests on single runs. Appendix E states 'we were unable to perform extensive hyperparameter tuning, try various feature combinations, or run multiple trials for each experiment.' No error bars or seed-level results are reported. Several favorable margins are small in absolute terms (e.g., Fold accuracy 0.433 vs 0.384; structure-only cellular component F1max 0.392 vs 0.388) and could easily be within seed noise. This is load-bearing because the central claim is empirical superiority. The authors should provide multiple seeds with standard deviations or at least clearly label the results as preliminary single-run observations and soften the claims accordingly.","section":"Appendix E, Table 1"},{"comment":"The design choices are described as based on 'preliminary experiments' on the same datasets: the auxiliary 3Di denoising task was added because 'our preliminary results show that this auxiliary task consistently improves all model's performance across our evaluation tasks,' and training time limits were 'carefully chosen through preliminary experiments to allow all models to reach convergence.' This is a selection-on-validation-protocol procedure applied to the same benchmark data, which can bias the comparison in favor of the proposed model. The paper should either report results under a pre-registered protocol or analyze sensitivity to these choices. At minimum, this should be acknowledged as a limitation in the main text, not only in the appendix.","section":"Section 6.1 and Appendix E (auxiliary 3Di task, time limits)"},{"comment":"The 2-cells are defined from DSSP labels: each residue has exactly one SSE label and only consecutive runs of at least three residues with the same label become 2-cells. The outer-edge neighborhoods in Eqs. (2)--(3) subtract B^{2->1} to remove self-edges; if DSSP assignments are noisy, or if biologically relevant SSEs are short, discontinuous, or overlapping, the construction will misrepresent the protein and the outer-edge subtraction may drop exactly the inter-SSE edges that matter. The manuscript provides no sensitivity analysis to the DSSP assignment quality, the minimum SSE length, or the k=16 edge cutoff. This is a concrete risk to the main methodological contribution and should be tested (e.g., by perturbing DSSP labels or varying the minimum SSE length).","section":"Section 5.1, Eqs. (2)--(3)"}],"minor_comments":[{"comment":"The reported cellular component F1max for TCPNet structure+sequence differs between Table 1 (0.398) and Table 2 (0.397). This inconsistency should be resolved or explained.","section":"Tables 1 and 2"},{"comment":"The choice of 16 nearest neighbors for the residue interaction edges is a free parameter, as is the minimum SSE size of three residues. No sensitivity analysis or ablation is provided; at least a sentence justifying these choices would help.","section":"Section 5.1"},{"comment":"The notation in the GVP layer normalization LN(H) = (LN_s(H_s), H_v / sqrt(||H_v||^2_2 / |H|)) is ambiguous: the denominator should be clarified (what is |H|? the number of vector channels times 3?). This is important for reproducibility.","section":"Eq. (14)"},{"comment":"The auxiliary 3Di denoising description is terse: 'we randomly permute half of the residues' 3Di types while masking the other half as unknown' leaves unclear whether the model is trained to reconstruct the masked half, the permuted half, or both. Please specify the exact loss target.","section":"Appendix E"},{"comment":"Several typos and infelicities appear, e.g., 'vertice set' (Appendix A), 'GVP norms' (caption of Figure 8), and inconsistent use of 'GTNN' (Section 6.3) vs 'TNN.' These should be corrected in revision.","section":"General"},{"comment":"Two readout strategies are described (pooling node embeddings vs. using the protein embedding), but no comparison or ablation is reported. Since the protein-level message passing is optional (Figure 8), the reader cannot tell which strategy produced Table 1.","section":"Section 5.4"}],"recommendation":"major_revision","confidential_remarks":"The paper's core architectural idea is interesting and worth further development, but the current empirical claims are not supported by the reported results. The authors should be encouraged to rerun with multiple seeds, report error bars, correct the overclaimed 'consistent outperformance' statements, and address the DSSP-sensitivity concern. If after revision the central claims remain unsupported by additional experiments, a reject may be warranted. A fitted-parameter circularity concern is not the main issue here; the main issue is statistical and internal-consistency."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Quick take: this is a genuinely new architectural idea—PCC gives you directed 1-cells, outer-edge neighborhoods, and per-rank scalar/vector features—and the fold classification result is the one place where the topological hierarchy shows a real advantage. That said, the two headline claims—'consistently outperforms SOTA' and 'consistently outperforms GCPNet across all tasks'—are both false as written in the paper's own Table 1. GVP-GNN wins inverse folding (5.280 vs 5.822 perplexity, 0.474 vs 0.441 accuracy), and GCPNet wins the structure+sequence variants of cellular component (0.408 vs 0.398 F1max) and antibody (0.878 vs 0.874 AUPRC). The paper would be more accurate if it claimed improvements on fold classification and structure-only robustness.\n\nWhat's actually new is worth taking seriously. The directed edge construction with SO(3) frames is a non-obvious fix for getting SSE-to-SSE communication without overlapping 2-cells, and the four-step hierarchical message passing is a coherent design. The authors are also honest in Appendix E that they ran no multiple trials and barely any hyperparameter tuning, which puts all the favorable margins—fold accuracy 0.433 vs 0.384, structure-only cellular F1max 0.392 vs 0.388—within seed noise. The auxiliary 3Di denoising task and per-task time limits were chosen after preliminary experiments on the same datasets, so there is at least some selection bias in the design.\n\nThe DSSP-based 2-cell construction is a plausible secondary concern: if SSE assignment is noisy, the outer-edge neighborhoods could drop exactly the edges connecting meaningful structural units. The paper never tests this. I wouldn't block on it, but it's the kind of ablation a referee should ask for.\n\nBottom line: the architecture is worth a serious referee, but not as is. The abstract and Section 6.2 need to be corrected, and the main numbers need seed variance before I'd trust the margins. Send it to review with a strong request for revision.","headline":"A real, novel topological representation for proteins, but the empirical case is overstated and rests on single runs.","tokens_in":19908,"tokens_out":2591,"would_cite":false,"duration_ms":24073,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"TCPNet embeds proteins as residues, interactions, secondary structures, and whole proteins in one SE(3)-equivariant model and reports consistent gains over state-of-the-art geometric GNNs, strongest on fold classification.","keywords":["topological deep learning","protein representation learning","combinatorial complexes","SE(3)-equivariant message passing","secondary structure","fold classification","geometric graph neural networks"],"falsifier":"Run TCPNet and its direct GNN counterpart on the same fold-classification benchmark with two versions of each structure: the original DSSP-derived secondary-structure assignments, and corrupted assignments in which a fraction of residues are relabeled or long SSE runs are split into shorter ones. If fold-classification accuracy does not drop materially under corruption, or TCPNet no longer beats the residue-only baseline, then the hierarchy built from SSE labels is not the source of the reported gains.","tokens_in":19007,"feed_emoji":"🧬","tokens_out":7898,"duration_ms":68866,"temperature":0.7,"pith_summary":"Protein structure is hierarchical, but most learned representations treat a protein as a flat graph of residue nodes. This paper argues that the missing level is secondary structure: the helices and strands whose spatial arrangement defines a protein's fold. It introduces the Protein Combinatorial Complex (PCC), a four-rank object holding residues, residue-residue interactions, secondary-structure elements, and the whole protein, and TCPNet, a network that passes scalar and vector messages across these ranks while remaining equivariant to rotations and translations. On four protein tasks, TCPNet is reported to beat standard geometric graph networks across the board, with the clearest wins on fold classification and the smallest accuracy loss when sequence information is withheld.","feed_headline":"Hierarchical protein model tops geometric GNNs on fold classification","feed_subtitle":"Four-level protein representation keeps structure-only accuracy high and boosts fold classification.","key_machinery":"The Protein Combinatorial Complex (PCC): a four-rank cell complex with residues as 0-cells, directed nearest-neighbor edges as 1-cells, DSSP-derived secondary-structure runs as 2-cells, and the protein as a 3-cell. The argument is carried by TCPNet, whose Topology-Complete Perceptron (TCP) module scalarizes vector features against edge-localized frames at every rank, and whose four-step hierarchical message passing—edge messages, SSE updates via outer-edge neighborhoods (which subtract self-edges so SSEs communicate through edges that cross between different SSEs), residue refinement, and global protein readout—moves information up and down the hierarchy while preserving SE(3)-equivariance.","core_discovery":"The paper's central claim is that proteins are best represented not as flat residue graphs but as a four-rank combinatorial complex—residues (rank 0), directed residue-residue interaction edges (rank 1), secondary structure elements (rank 2), and the whole protein (rank 3)—and that neural networks can exchange messages across all four ranks while staying equivariant to rotations and translations. The authors define the Protein Combinatorial Complex (PCC), featurize each rank with scalar and vector geometric features, and design TCPNet, whose Topology-Complete Perceptron scalarizes vector features through edge-centered reference frames. A four-step message-passing scheme first computes edge m","pith_inferences":["The reported structure-only robustness suggests a sequence-free inductive bias: a testable extension is to use TCPNet embeddings as pre-sequence filters or structural priors in inverse folding and de novo antibody design, which the paper only hints at in the conclusion.","Because 2-cells are built from sequential runs of identical DSSP labels with minimum length three, short, discontinuous, or overlapping structural elements are invisible to the hierarchy; corrupting or re-assigning SSE labels would reveal how much of the gain depends on exactly these assignments.","The outer-edge neighborhoods subtract self-edges; if real secondary-structure elements are occasionally non-contiguous or interleaved, this subtraction may drop biologically meaningful contacts, so an extension is to allow overlapping or probabilistic SSE membership and compare.","The paper's comparison places TCPNet above its direct GNN counterpart but below GVP-GNN on inverse folding; an inference is that topological structure helps most at scales above the local residue environment, so hybrid residue/topology pretraining objectives may combine both strengths."],"forward_implications":["Fold classification improves by about 3 percentage points on the hardest Fold split over the second-best model, suggesting that SSE-level message passing captures the structural motifs that define folds.","With sequence information removed, TCPNet's cellular-component and antibody-developability metrics drop only 0.6% and 2%, respectively, while graph baselines degrade by up to 10%, making pure-structure representations more practical.","Topological enhancement helps only when integrated deeply: TCPNet beats its direct GNN counterpart GCPNet everywhere, while a shallow SSE feature addition gives mixed or worse results.","Because the PCC carries scalar and vector features at every rank, the same encoder can be plugged into downstream tasks that need a single protein embedding or per-residue predictions."],"supporting_citations":[{"why":"Defines combinatorial complexes, the mathematical object the PCC is built on.","marker":"Hajij, Zamzmi, et al. 2022"},{"why":"Supplies the equivariant topological message-passing framework (ETNN) that TCPNet extends and that is used as a baseline.","marker":"Battiloro et al. 2024"},{"why":"Provides the GCP perceptron that TCP generalizes and the GCPNet model used as the direct GNN counterpart.","marker":"Morehead and Cheng 2024"},{"why":"Gives the GVP machinery (layer norms and message functions) reused in TCPNet and the GVP-GNN baseline.","marker":"Jing et al. 2020"},{"why":"Provides the ProteinWorkshop evaluation tasks, feature pipeline, and baselines used for comparison.","marker":"Jamasb et al. 2024"},{"why":"Defines the SCOP 1.75 dataset and the family/superfamily/fold test splits used for fold classification.","marker":"Murzin et al. 1995"},{"why":"Supplies the DSSP secondary-structure assignment used to construct the rank-2 SSE cells.","marker":"Kabsch and Sander 1983"},{"why":"Provides backbone featurization details and the auxiliary 3Di denoising task used for graph-level training.","marker":"Tan et al. 2024"},{"why":"Supplies the SabDab antibody structure dataset used for developability prediction.","marker":"Dunbar et al. 2014"},{"why":"Supplies the Gene Ontology cellular-component prediction dataset.","marker":"Gligorijevi´c et al. 2021"}],"fun_headline_variants":["Topotein's 4-level protein complex beats GNNs on fold tasks","Four-rank protein topology bests GNNs on fold classification","Protein as 4-level complex: TCPNet beats GNNs on folds","Hierarchical protein complex tops GNNs on fold classification","Equivariant 4-rank protein model beats GNNs on fold classification"],"cache_read_input_tokens":2688,"weakest_assumption_plain":"The method assumes DSSP secondary-structure labels are accurate, complete, and non-overlapping—each residue has exactly one label and only sequential runs of at least three residues with the same label become structural units—so noisy or fragmented assignments would corrupt the very edges the hierarchy relies on.","fun_headline_variants_meta":{"raw":{"variants":["Topotein's 4-level protein complex beats GNNs on fold tasks","Four-rank protein topology bests GNNs on fold classification","Protein as 4-level complex: TCPNet beats GNNs on folds","Hierarchical protein complex tops GNNs on fold classification","Equivariant 4-rank protein model beats GNNs on fold classification"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.001552,"raw_usage":{"total_tokens":6002,"prompt_tokens":668,"completion_tokens":5334,"prompt_tokens_details":{"cached_tokens":256},"prompt_cache_hit_tokens":256,"prompt_cache_miss_tokens":412,"completion_tokens_details":{"reasoning_tokens":5239}},"tokens_in":412,"tokens_out":5334,"duration_ms":36287,"temperature":1.0,"reasoning_tokens":5239,"cache_read_input_tokens":256,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-05T10:33:39.824129+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run TCPNet and its direct GNN counterpart on the same fold-classification benchmark with two versions of each structure: the original DSSP-derived secondary-structure assignments, and corrupted assignments in which a fraction of residues are relabeled or long SSE runs are split into shorter ones. If fold-classification accuracy does not drop materially under corruption, or TCPNet no longer beats the residue-only baseline, then the hierarchy built from SSE labels is not the source of the reported gains.","supporting_citations":[],"review_version":1}