{"id":"b41aec2a-6c00-4542-bb94-0505be598025","arxiv_id":"2505.04939","paper_version":1,"verdict":"CONDITIONAL","confidence":"LOW","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":4,"one_line_summary":"Knowledge graph link prediction and KGEM hyperparameter preference can be modelled from graph structural features alone; TWIG and TWIG-I demonstrate this and enable cross-KG transfer.","lead":"This thesis argues that link prediction on knowledge graphs can be understood and performed using only graph structure, without learned node or edge embeddings. It introduces TWIG and TWIG-I, open-source systems that simulate embedding-model performance and predict links from structural features, and reports that cross-graph transfer becomes feasible.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Near-zero cross-seed correlation of local ranked lists (Sec 4.2.1) undercuts TWIG's local-level Claim 1: per-query ranks appear dominated by random seed, so local structural features cannot deterministically explain KGEM output.","rationale":"The reader's weakest_assumption concerned sufficiency of the 22 structural features; that is a real limitation, but the more load-bearing issue is the internal stability of TWIG's local target. Section 4.2.1 is the thesis's own randomness control, and it shows that per-query ranked lists are nearly uncorrelated across seeds even when hyperparameters are fixed. Since TWIG's inputs do not include the seed, a deterministic function of structure and hyperparameters cannot reproduce seed-specific local ranks. This matters because TWIG is presented as a local/global simulator and Claim 1 includes hyperparameter preference and LP performance as functions of structure. The global MRR stability (>0.99) is exactly what an aggregate would look like even if local ranks were noise, so it does not resolve the issue. The proposed test is cheap and decisive because the code and data are open-sourced. I credit the author for reporting the near-zero local correlation and for releasing the code; the concern is about the argument, not the conduct. If the full thesis shows TWIG was only evaluated on global MRR, the verdict should remain conditional on reframing the local claim rather than on new experiments.","tokens_in":48769,"tokens_out":9775,"duration_ms":109439,"concrete_test":"Using the released TWIG code and data, train TWIG on per-query ranks for ComplEx/UMLS (or another KGEM-KG pair), holding out a subset of hyperparameter combinations. Compare Spearman rank correlation of TWIG's predicted per-query ranks against (a) individual held-out seed runs and (b) seed-averaged ranks, with a hyperparameters-only baseline. If correlation with per-seed ranks is near 0 and no better than the baseline, the local simulation claim fails; if correlation with seed-averaged ranks is high, the claim must be reframed as prediction of the expectation, not of individual KGEM outputs.","verdict_should_be":"UNCHANGED","load_bearing_attack":"TWIG's data model pairs each query's structural features with the rank assigned by a single KGEM run (Sec 4.1.3), and TWIG is introduced as simulating KGEM output 'both locally and globally' (Sec 1.1). However, Sec 4.2.1 reports that while global MRR is stable across seeds (>0.99 correlation), per-query ranked lists are nearly uncorrelated across seeds. A target that is essentially seed-specific noise cannot be a deterministic function of the structural features and hyperparameters that TWIG receives, because the random seed is not among TWIG's inputs. This does not refute global MRR prediction or the separate TWIG-I evidence, but it directly contradicts the local half of Claim 1 as instantiated: structure-based simulation of individual link-prediction outcomes. The thesis would need to show either that TWIG is trained on seed-averaged ranks and that those averages are structurally determined, or that local simulation was never actually claimed. The reported MRR stability is a useful control, but as an aggregate it is consistent with per-query noise and does not rescue local prediction.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The thesis proposes Structural Alignment, a hypothesis that knowledge-graph learning and link prediction can be modelled as a function of graph structure without learned node or edge embeddings. It instantiates this idea with 22 structural features (six fine-grained and sixteen coarse-grained, all computed only from the training split), and presents two systems: TWIG, which takes structural features and KGEM hyperparameters and predicts KGEM link-prediction performance both globally (MRR) and locally (per-query ranks), and TWIG-I, a structure-based link predictor intended to show that link prediction can be performed directly from structural features and that cross-KG transfer learning is viable. The thesis reports that KGEM hyperparameter preference and performance are determined by structure, that TWIG can predict KGEM output, and that TWIG-I is competitive with state-of-the-art predictors. All code and data are open-sourced, and the thesis includes a bilingual English/Irish presentation.","tokens_in":49042,"tokens_out":5165,"duration_ms":52806,"significance":"If the main claims hold, the work would be a significant reorientation of link prediction research: it would show that embedding-free, structure-only models can both explain KGEM behaviour and perform link prediction, with the additional practical benefit of cross-KG transferability. The manuscript has real strengths: the MRR stability check across random seeds (Section 4.2.1) is a careful reliability control; the worked examples of structural feature computation are concrete; and the open-sourced code and data (TWIG, TWIG-I, PyPI packages, Figshare data) materially support reproducibility. The intended distinction between structural features and learned embeddings is also clearly drawn. However, the local-level claim of TWIG is undercut by the manuscript's own near-zero cross-seed rank correlation result, and the decisive TWIG-I quantitative results are not present in the supplied text. Both issues are load-bearing for the thesis's central conclusions.","major_comments":[{"comment":"The manuscript reports that per-query ranked lists are nearly uncorrelated across random seeds (\"the correlation of ranked lists trained with different random initialisations, but identical hyperparameters, was near 0\"), while global MRR is very stable (correlation > 0.99 across runs). This directly contradicts the local half of Claim 1 as instantiated by TWIG. TWIG's data model (Section 4.1.3) pairs each query's structural features with the rank assigned by a single KGEM run, and Section 1.1 states that TWIG simulates KGEM output 'both locally and globally'. Since the random seed is not among TWIG's inputs, a target that is essentially seed-specific noise cannot be a deterministic function of the structural features and hyperparameters TWIG receives. Global MRR stability does not rescue local prediction, because an aggregate over many queries can be stable while individual ranks are dominated by noise. The thesis needs either to train TWIG on seed-averaged ranks and demonstrate that those averages are structurally determined, or to explicitly retract the local-level simulation claim; without one of these, the local half of Claim 1 is not supported by the reported evidence.","section":"Section 4.2.1 and Section 4.1.3"},{"comment":"The supplied version of the manuscript does not contain the quantitative results of the TWIG-I experiments, nor the transfer-learning results. The abstract and summary assert that structure-based link prediction is 'highly effective compared to state-of-the-art approaches' and that cross-KG transfer learning becomes viable, but Section 5.2.1 and Section 5.4 are presented only as headings or descriptions in the text available for review. These numbers are load-bearing for Claim 2 and for one of the thesis's three stated impacts. The manuscript must include the actual results, or must clearly indicate where in the submitted version they appear, before the claims can be verified.","section":"Section 5.2.1 and Section 5.4"},{"comment":"The Structural Alignment Hypothesis is stated in general terms ('KG learning and link prediction can be modelled as a function of graph structure'), but the experiments instantiate only a single set of 22 hand-selected, 1-hop, frequency-based features. Multi-hop paths, global topological statistics, and label-derived semantic information are excluded by the feature definition. The conclusion that 'link prediction can be understood and modelled as a structural task' is therefore broader than the evidence supplied. For a feasibility claim this scope restriction may be acceptable, but the thesis should either explicitly formulate the validated hypothesis as restricted to this feature family, or add experiments with richer structural features (e.g., path counts, graphlet counts, or global statistics) to show that the omitted information is not needed.","section":"Section 3.2.1 and Section 6"},{"comment":"The hyperparameter grid in Table 4.1 fixes batch size, optimiser, and regulariser, and the case-study uses a single KGEM (ComplEx) on a single KG (UMLS). Sections 4.3 and 4.4 are described as extending this to other KGEMs and KGs, but the generalisation claim in Chapter 6 (that Structural Alignment explains KGEM learning 'in the general case') rests on a small set of three KGEMs and five benchmark KGs. This is not necessarily a flaw for a feasibility study, but the general-case phrasing should be tempered to match the empirical coverage, and the effect of the fixed hyperparameters on the conclusions should be discussed explicitly.","section":"Section 4.1.2 and Section 4.3"}],"minor_comments":[{"comment":"In the worked example for (Gondor, At-War-With, Isengard), the subject-side node degrees are listed as 1 (Osgiliath), 2 (Minas Tirith), 2 (Aragorn), 7 (Rohan), and 5 (Isengard), but the text reports the minimum degree as 2; the correct minimum is 1.","section":"Section 3.2.2"},{"comment":"In the summary of Mohamed et al. (2019), the second category of loss functions is labelled 'Pointwise losses' but the sentence describes pairwise losses; this should read 'Pairwise losses' for consistency with the surrounding text.","section":"Section 2.4.2"},{"comment":"The near-zero ranked-list correlation result is reported through a figure and a sentence; a brief table reporting the mean and median correlation across the four runs would make the severity of the local-level noise easier for readers to assess.","section":"Section 4.2.1"}],"recommendation":"major_revision","confidential_remarks":"The local-level rank instability in Section 4.2.1 is likely to be the decisive issue in any revision: it directly contradicts a stated design goal of TWIG. The missing TWIG-I quantitative results must also be supplied. If the author can successfully retarget TWIG to global MRR prediction or to seed-averaged local ranks, and if the TWIG-I numbers support the stated claims, the thesis could be suitable for publication after a careful revision."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Here's my read after going through the manuscript and the stress-test note.\n\nThe thesis is worth engaging with. The genuinely new piece is the Structural Alignment Framework: it pulls together the scattered evidence on degree and frequency effects into one hypothesis, and it actually ships code and data for TWIG and TWIG-I. The writing is clear, the definitions are careful, and the explicit stance that the studies test feasibility, not optimality, is the right frame. The MRR stability check across four random seeds (Section 4.2.1) is a good control and supports the global side of Claim 1.\n\nThe soft spot is real and it is in their own results. They claim TWIG simulates KGEM output both locally and globally, but Section 4.2.1 reports near-zero correlation of per-query ranked lists across seeds. Since the random seed is not an input to TWIG, local ranks cannot be a deterministic function of the structural features and hyperparameters TWIG receives. That directly contradicts the local half of Claim 1 as instantiated. It does not kill the global MRR prediction, and it does not touch TWIG-I, but the thesis needs to either drop the local simulation claim or show that seed-averaged ranks are structurally determined. The stress-test note has this right.\n\nTwo smaller reservations. First, the decisive TWIG-I comparisons against state-of-the-art predictors and the transfer-learning gains are not in the provided excerpt, so I could not verify the headline numbers; the full thesis presumably has them, but they need careful checking. Second, the 22 hand-selected structural features are a limitation for the general claim. The author acknowledges some of this in the feature calculation examples, so it is a known boundary rather than a hidden one.\n\nWho is this for? Researchers working on KGEM hyperparameter prediction, structure-based link prediction, or cross-graph transfer. They will get a solid framework, a reproducible baseline, and a genuinely interesting hypothesis to argue with.\n\nMy recommendation: send it to peer review. The framework is significant, the code is open, and the central empirical question is sharp enough to justify referee time. Acceptance should be conditional on resolving the local-vs-global contradiction and on the missing TWIG-I results surviving scrutiny.","headline":"A careful, open-sourced PhD thesis proposing a structure-first view of link prediction; the framework is coherent but the local-level claim is undercut by the thesis's own seed-sensitivity result.","tokens_in":49521,"tokens_out":1943,"would_cite":false,"duration_ms":21616,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"This thesis argues that knowledge graph learning and link prediction can be modelled entirely from graph structural features, and presents two systems, TWIG and TWIG-I, that instantiate this claim.","keywords":["link prediction","knowledge graphs","graph structure","structural features","knowledge graph embeddings","transfer learning","hyperparameter preference","Structural Alignment"],"falsifier":"Take two triples in a knowledge graph that receive identical 22-dimensional structural feature vectors but whose correct completions differ; if an embedding model ranks both correctly while a structure-only predictor cannot, the claim that the chosen structural features are sufficient for link prediction is refuted.","tokens_in":48535,"feed_emoji":"🔗","tokens_out":7384,"duration_ms":68409,"temperature":0.7,"pith_summary":"This thesis proposes that knowledge graph learning and link prediction can be understood as structural tasks. It argues that a small set of frequency-based graph features, computed from the training triples alone, is enough to predict both how well an embedding model will perform and which missing links are plausible. To support the claim, the author builds two systems: TWIG, which simulates the output of embedding models from structure plus hyperparameters, and TWIG-I, which performs link prediction directly from structure with no learned embeddings. The reported experiments suggest the structural approach is viable, competitive with embedding-based models, and enables cross-graph transfer learning. The thesis states explicitly that it aims to show feasibility, not optimality.","feed_headline":"Graph shape alone can predict missing links","feed_subtitle":"Proposes that 22 structural features capture what embedding models learn, enabling structure-only link prediction.","key_machinery":"The load-bearing object is the 22-dimensional structural feature vector defined in Table 3.1. Six fine-grained features describe the core triple directly: subject degree, object degree, predicate frequency, subject-predicate co-frequency, object-predicate co-frequency, and subject-object co-frequency. Sixteen coarse-grained features summarise the neighbourhoods of the subject and object: min, max, and mean neighbour degree; number of distinct neighbours; min, max, and mean frequency of incident edges; and number of distinct incident edges. All features are computed from the training split only, in a position-aware way, so that directionality is preserved and no test information leaks. This vector turns a triple into a fixed-size, label-free representation; TWIG and TWIG-I are then neural networks that map these representations, together with hyperparameters in TWIG's case, to predicted ranks and plausibility scores.","core_discovery":"Structural Alignment is the claim that the structural features of a training knowledge graph, such as node degrees, relation frequencies, co-frequencies, and neighbourhood summaries, can model hyperparameter preference and link prediction performance, and that link prediction can be performed from those features alone, without learned embeddings. The thesis instantiates this claim in two systems. TWIG takes KG structure plus a hyperparameter configuration and predicts the ranks an embedding model would produce, and experiments on three KGEMs and five KGs report that global performance is highly predictable from structure, with MRR correlations above 0.99 across random seeds. TWIG-I replaces learned embeddings with a fixed 22-dimensional structural feature vector for each candidate triple and learns to score triples directly; the reported results show it is competitive with embedding baselines and that pre-training on KGs from other domains improves its accuracy, making cross-KG transfer viable. The author states the aim is feasibility, not optimality, and concludes that a structure-first view of link prediction is viable and useful.","pith_inferences":["A direct test the thesis leaves implicit: permute node and relation labels while preserving all 22 structural features; a purely structural predictor would give identical predictions, so any accuracy change on the relabelled graph would reveal label-derived signal the features miss.","The feature set is deliberately local; extending it with multi-hop path counts, motif frequencies, or global degree distributions is a natural next test and could close cases where co-frequency vectors collapse.","Because structural encodings are label-free and graph-agnostic, they are a concrete candidate input representation for graph foundation models aiming at cross-graph transfer; the thesis notes the connection but does not build such a model."],"forward_implications":["Hyperparameter search for embedding models could be shortcut by predicting performance from structure plus hyperparameters, skipping expensive training runs.","Structure-based link prediction can act as a strong, faster baseline alongside embedding models, with competitive accuracy on standard benchmark KGs.","Cross-KG and cross-domain transfer learning works for link prediction when triples are represented by structural features rather than KG-specific embeddings.","KGEM outputs can be simulated from structure, meaning at least part of what embedding models learn is expressible in structural terms.","KG construction and curation could be guided by structural learnability, since low-degree and low-frequency elements are the hard cases."],"supporting_citations":[{"why":"Large-scale hyperparameter study showing optimal KGEM hyperparameters depend on the KG and KGEM; supplies the performance data and search protocol TWIG builds on.","marker":"[2]"},{"why":"Provides the reference implementations of ComplEx, DistMult, TransE and the benchmark KGs used in all TWIG and TWIG-I experiments.","marker":"[3]"},{"why":"Shows node degree drives link prediction bias in biomedical KGs, supporting degree as a structural feature.","marker":"[9]"},{"why":"Shows negative sampler preference is governed by KG connectivity and co-frequencies, directly motivating the structural feature set.","marker":"[45]"},{"why":"Shows loss-function preference depends on KGEM and KG, evidence that hyperparameter preference is not model-internal only.","marker":"[61]"},{"why":"Shows node-relation co-frequency and relation path support correlate with link prediction ranks, supporting co-frequency features.","marker":"[70]"},{"why":"Shows higher-degree nodes are learned better by TransE and DistMult, foundational evidence for structure determining performance.","marker":"[71]"},{"why":"Shows re-evaluated KGEMs beat newer models under better hyperparameters and that hyperparameters depend on KG and KGEM, motivating structure-based performance prediction.","marker":"[73]"},{"why":"Shows frequency-based edge priors can reconstruct unlabelled graphs, evidence that structure alone can predict links.","marker":"[110]"}],"fun_headline_variants":["Link prediction is a structural task, not an embedding one","Structure-only link prediction rivals embedding models","22 structural features replace embeddings for link prediction","Structural alignment: link prediction without embeddings"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that the 22 hand-selected structural features, computed only from the training split, capture the information a triple and its neighbourhood carry for link prediction; if multi-hop paths, global topology, or label-derived semantics carry essential signal, the central claim gives way.","fun_headline_variants_meta":{"raw":{"variants":["Link prediction is a structural task, not an embedding one","Structure-only link prediction rivals embedding models","22 structural features replace embeddings for link prediction","Structural alignment: link prediction without embeddings"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000478,"raw_usage":{"total_tokens":2423,"prompt_tokens":1056,"completion_tokens":1367,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":672,"completion_tokens_details":{"reasoning_tokens":1323}},"tokens_in":672,"tokens_out":1367,"duration_ms":10375,"temperature":1.0,"reasoning_tokens":1323,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-15T23:17:20.459501+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Take two triples in a knowledge graph that receive identical 22-dimensional structural feature vectors but whose correct completions differ; if an embedding model ranks both correctly while a structure-only predictor cannot, the claim that the chosen structural features are sufficient for link prediction is refuted.","supporting_citations":[{"cited_title":"Analysis of the Impact of Negative Sampling on Link Prediction in Knowledge Graphs","cited_arxiv_id":null,"evidence_quote":"Shows negative sampler preference is governed by KG connectivity and co-frequencies, directly motivating the structural feature set."},{"cited_title":"Loss Functions in Knowledge Graph Embedding Models","cited_arxiv_id":null,"evidence_quote":"Shows loss-function preference depends on KGEM and KG, evidence that hyperparameter preference is not model-internal only."},{"cited_title":"Knowledge Graph Embedding for Link Prediction: A Comparative Anal- ysis","cited_arxiv_id":null,"evidence_quote":"Shows node-relation co-frequency and relation path support correlate with link prediction ranks, supporting co-frequency features."},{"cited_title":"Knowledge graph embeddings: Are relation- learning models learning relations?","cited_arxiv_id":null,"evidence_quote":"Shows higher-degree nodes are learned better by TransE and DistMult, foundational evidence for structure determining performance."},{"cited_title":"The probability of edge existence due to node degree: a baseline for network-based predictions","cited_arxiv_id":null,"evidence_quote":"Shows frequency-based edge priors can reconstruct unlabelled graphs, evidence that structure alone can predict links."}],"review_version":1}