{"id":"cd43355f-9e04-4570-ae5d-ba49f05e586f","arxiv_id":"2412.00521","paper_version":2,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":3,"one_line_summary":"MPS-GNN learns predictive meta-paths in relational databases using aggregate statistics over their occurrences, not just existence, and outperforms prior heterogeneous GNNs in experiments.","lead":"This paper presents a graph neural network that automatically finds the important chains of connections, or meta-paths, in relational databases, and uses statistics such as counts of counts rather than just the existence of a connection. It reports accuracy gains over previous methods on medical, geographic, and sports databases, and argues the selected meta-paths serve as transparent explanations of the model's predictions.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Per-node scoring weights in Eq. (2) are unconstrained and can memorize node identities, so the greedy meta-path search may select a spurious relation whose apparent informativeness no meta-path extension can materialize; this makes the central aggregate-statistics claim conditional on an…","rationale":"I read the paper's central claim as twofold: (1) MPS-GNN can learn meta-paths whose informativeness comes from aggregate count-of-count statistics rather than existential occurrence, and (2) the resulting predictions depend only on the meta-path-induced subgraph, making the meta-paths self-explanations. The second claim is architecturally true by construction and is further supported by the necessity experiments in Section 5.3. The first claim, however, depends on the scoring function in Section 4.2 being a reliable guide to meta-paths that are not only potentially separable by free per-node weights but actually realizable as statistics computed along meta-path extensions. The reader's weakest assumption points exactly at this: the per-node weights w_u in Eq. (2) can memorize node identities. On closer inspection, Eq. (6) passes the materialization burden forward rather than resolving it, and the greedy search has no backtracking mechanism, so a single bad first choice can be fatal. The toy example illustrates the mechanism but does not prove it always resolves; the real-world successes in Section 5.2 may rely on the datasets happening to have extensions that realize the learned weights. This does not refute the empirical results, but it makes the theoretical claim conditional. I therefore agree with the conditional verdict: the paper should either constrain w_u to be a function of node attributes or prove that low Eq. (4) loss guarantees the existence of a realizing extension, and the synthetic decoy experiment would settle whether the current formulation is actually vulnerable.","tokens_in":23710,"tokens_out":7985,"duration_ms":193231,"concrete_test":"Construct a synthetic graph with three relations, a, b, c. Let the true rule be: a target node is positive iff it has at least two a-neighbors that each have at least two c-neighbors (a length-2 count-of-counts pattern). Add a decoy relation b such that each positive target has exactly one private b-neighbor, with no attributes and no outgoing edges, and each negative target has none. Because Eq. (3) can set w_u=1 on those private b-neighbors, b should achieve near-zero loss and be selected first; but no extension from b can materialize the separation, so MPS-GNN should fail to reach the correct a,c meta-path and return F1 near 0.5. Run the authors' released code on this dataset with the same hyperparameters as the synthetic experiments.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim is that MPS-GNN automatically identifies meta-paths whose predictive power comes from aggregate statistics over multiple occurrences. The load-bearing step is the scoring function in Section 4.2: Eq. (2) introduces a free per-node weight w_u for each r-neighbor u, and Eq. (4) minimizes the resulting bag-separation loss over all such w_u. Nothing constrains w_u to be a function of node attributes, of the meta-path prefix, or of any statistic computable from a meta-path extension. In the extreme, if every positive bag contains a unique r-successor that no negative bag contains, setting w_u=1 for those nodes and w_u=0 elsewhere drives the loss in Eq. (3) to zero, even though the r-neighborhoods may be entirely uninformative for any realizable feature. The toy example in Section 4.2 is itself an instance of this phenomenon: relation b is selected only because w_u can separate the bags, and the b-only meta-path is explicitly non-discriminative; the construction succeeds only because a later relation, d, happens to materialize the required count statistic. Equation (6) does not remove the risk: the next-iteration bag weights are built from Theta_i^T x_v and previous alpha values, not from the optimized w_u, so the burden of turning w_u into a real feature is deferred entirely to the next scoring step. If no extension can express the memorized separation, the greedy search is stuck on a spurious first relation, and Algorithm 1's validation-based F1 check cannot recover because the correct prefix was never explored. Thus the claimed automatic identification of statistically meaningful meta-paths rests on an implicit and unverified materializability assumption: low Eq. (4) loss must imply existence of a meta-path extension realizing the learned weights as node features. This is the same soft spot the reader identified, and it is load-bearing for the paper's main contribution.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes MPS-GNN, a heterogeneous GNN for relational databases that automatically learns meta-paths whose predictive power derives from aggregate statistics over multiple occurrences (e.g., counts and counts-of-counts), rather than merely from the existence of a single occurrence. The method extends MP-GNN by replacing max aggregation with sum aggregation in a weighted multi-instance scoring function, and by iteratively constructing meta-paths through a greedy, local search guided by a surrogate loss. The final classifier is a multi-relational GNN restricted to the learned meta-path-induced subgraph, which the paper argues makes the model self-explainable by construction. Experimental evaluation on eight synthetic count-based scenarios and three real-world databases (EICU, MONDIAL, ErgastF1) reports state-of-the-art F1 scores, and the identified meta-paths are shown to be faithful explanations via deletion-based necessity tests.","tokens_in":24104,"tokens_out":6569,"duration_ms":68484,"significance":"If the claims hold, the paper addresses a real limitation of existing meta-path learning methods in relational deep learning: moving beyond existential quantification to aggregate statistics. The method is computationally efficient relative to full meta-path enumeration, the code is released, and the synthetic experiments provide a clear falsifiable testbed for the central claim. The self-explainability framing is also valuable, as the final model's predictions are, by construction, functions only of the selected meta-path subgraphs. These strengths make the paper a potentially useful contribution to the community.","major_comments":[{"comment":"The per-node weights w_u in Eq. (2) are unconstrained free parameters, and the loss L(r) in Eq. (4) is minimized over all such weights. This allows w_u to memorize node identities: if each positive bag contains an r-successor not present in any negative bag, setting those w_u to 1 and all others to 0 drives the loss to zero, even when the r-neighborhood is uninformative for any realizable feature. The paper does not constrain w_u to be a function of node attributes or of meta-path extensions, nor does it analyze when the greedy search could get stuck on a spurious first relation. The toy example in Section 4.2 is an instance of this: relation b is selected solely because w_u separates the bags, while the b-only meta-path is non-discriminative; the method succeeds only because a later relation d materializes the required count statistic. Please either (a) constrain or regularize w_u so that they correspond to materializable features, or (b) provide an empirical or theoretical analysis showing that the validation-based F1 check in Algorithm 1 reliably prevents the selection of meta-paths whose apparent informativeness cannot be materialized. As written, the central claim that MPS-GNN automatically identifies meta-paths for aggregate statistics is conditional on an unverified assumption about the scoring function.","section":"Section 4.2, Eq. (2)-(4)"},{"comment":"The necessity metric Nec = 1/N Σ (p_v(G) − p_v(G′)) is computed by randomly deleting meta-path occurrences and measuring the change in predicted probabilities. Because the model's computational graph only contains the meta-path subgraph, any deletion changes predictions, so this test confirms sensitivity to the explanation but does not specifically verify that predictions depend on aggregate counts rather than on the existence of any occurrence. A targeted experiment that reduces occurrence counts from c to c−1 while preserving at least one occurrence (or that compares predictions under count perturbations versus existence-preserving perturbations) would directly support the aggregate-statistics claim. Without such a test, the self-explainability evidence is weaker than the synthetic experiments, which already address this distinction.","section":"Section 5.3 and Eq. (for necessity)"},{"comment":"The complexity claim that MPS-GNN 'scales linearly in the number of relations and nodes' is imprecise. Minimizing Eq. (4) for each candidate relation involves optimizing over a vector w with one entry per node reachable via that relation, which is O(|V|) parameters per relation per iteration, giving a worst-case cost of O(L·|R|·|V|) for the full meta-path construction. This is still far cheaper than enumerating all meta-paths, but the 'linear' statement should be qualified, and the per-iteration optimization cost should be stated explicitly so that readers can assess scalability on large relational databases.","section":"Section 4.2.1"}],"minor_comments":[{"comment":"The sentence 'Existing approaches for heterogeneous GNNs either rely on domain experts to provide relevant meta-paths a priori, or attempt to learn them from data by assigning different weights to various relations, a solution that fails to scale with the number of candidate relations' is grammatically incomplete; consider rephrasing to clarify which solution fails to scale.","section":"Section 1"},{"comment":"The stopping criterion in Algorithm 1, 'if min_r L(r) ≥ η Linit(r)', uses a quantity Linit(r) that is not defined in the algorithm or in the surrounding text; please define it and specify how the 30% improvement threshold is chosen and how sensitive the results are to it.","section":"Section 4.2 and Algorithm 1"},{"comment":"The preprocessing step that clusters rows of auxiliary tables to improve connectivity is described only briefly; its potential effect on the results (e.g., information leakage or sensitivity to the clustering method) is not discussed. Please add a sentence or an ablation assessing this.","section":"Section 5.2 / Appendix A.3"},{"comment":"The RDL row for rel-f1-top3 reports a standard deviation of 0.7, which appears to be a typo (likely 0.07); please correct it.","section":"Appendix A.8, Table 13"},{"comment":"The claim that MPS-GNN is 'the first truly self-explainable GNN designed for relational deep learning applications' is too strong without a more thorough survey; please soften it to 'to the best of our knowledge' and add references to any prior self-explainable heterogeneous GNNs or meta-path-based explanation methods.","section":"Section 2"},{"comment":"There are numerous typos and spacing errors throughout the text (e.g., 'behindMP-GNN', 'in both synthetic and real-world scenario', missing spaces before citations). A careful proofreading pass is recommended.","section":"General"}],"recommendation":"major_revision","confidential_remarks":"The paper's empirical results are strong and the code availability is a plus. The main risk is the unconstrained per-node weights in the scoring function, which, if not addressed, could limit the method's applicability to more complex databases. The authors should be encouraged to provide a constraint or a robustness analysis for this issue. The necessity metric also deserves strengthening to directly probe the count-based mechanism. These are fixable within the scope of a major revision."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"The thing to know is that this paper delivers a real, working extension of MP-GNN: it replaces the existential (max-aggregation) meta-path test with sum aggregation, adds a skip connection, and demonstrates on synthetic count-based tasks that the resulting MPS-GNN recovers the planted meta-path where MP-GNN fails. It also beats the listed baselines on three real relational databases, with code released. That is a solid, reproducible contribution, and the authors are appropriately clear that the advance is incremental relative to their own prior work.\n\nThe strongest part is the experimental demonstration: all eight synthetic scenarios are recovered, and the real-world gains are consistent and non-trivial. The self-explainability claim is also handled honestly: sufficiency of the meta-path subgraph is true by construction because the network only sees that subgraph, and the necessity experiments show that removing meta-path occurrences degrades predictions. That is the right kind of evidence.\n\nThe soft spots are real but not fatal. The main one is exactly what the stress-test note identifies: the per-node weights w_u in Eq. (2) are free parameters, so a low score in Eq. (4) can come from memorizing node identities rather than from any feature that a meta-path extension could materialize. The toy example itself shows this: relation b is selected only because the weights separate the bags, while the b-only path is non-discriminative; the construction works only because relation d later materializes the count statistic. The paper calls w_u \"putative features\" and assumes they can be realized as node attributes or meta-path extension features, but that materializability step is never proven or systematically tested. Algorithm 1's validation check cannot recover if the greedy search gets stuck on a spurious first relation with no realizable extension. This makes the central claim conditional, not false: the synthetic results suggest the assumption often holds, but the paper should either provide conditions under which low Eq. (4) loss guarantees a realizable extension, or add experiments that deliberately test for memorization (e.g., permuted node identities).\n\nMinor issues: the real-world preprocessing is heavy (clustering, DBSCAN, supernodes), and there is a duplicated sentence in the introduction that should be cleaned up.\n\nWho is this for: anyone working on relational deep learning or heterogeneous GNNs will want to know this result, and the method is worth building on. It deserves a serious referee; the materializability question should be pushed in revision, but the empirical evidence and the clean formulation justify engagement.","headline":"A genuinely useful extension of MP-GNN to count-based meta-path statistics, with real experimental gains, but the scoring function's per-node weights carry an unverified materializability assumption that should be pinned down before the main claim is taken as settled.","tokens_in":24621,"tokens_out":1715,"would_cite":true,"duration_ms":21598,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":["68T07","68R10"],"pacs":[],"model":"deepseek-v4-flash","headline":"MPS-GNN learns meta-paths whose informative content is aggregate statistics over occurrences, making predictions depend only on the meta-path-induced subgraph.","keywords":["meta-path learning","heterogeneous graph neural networks","relational deep learning","self-explainable GNN","weighted multi-instance classification","aggregate statistics","counts-of-counts","relational databases"],"falsifier":"Build a synthetic database whose label depends on at least $c$ occurrences of a known meta-path, and add a decoy relation that connects each target node to a unique dummy node carrying a one-hot identifier. If the scoring function gives the decoy relation a lower loss than the correct one, or if the trained MPS-GNN's predictions change when the decoy subgraph is modified, then the central claim that the weights represent materializable features is falsified.","tokens_in":23519,"feed_emoji":"🧮","tokens_out":7856,"duration_ms":71118,"temperature":0.7,"pith_summary":"The paper introduces MPS-GNN (Meta-Path Statistics GNN), a heterogeneous graph neural network that discovers meta-paths—sequences of relations—whose predictive value lies in aggregate statistics over their occurrences, such as counts-of-counts patterns ('at least two exempt prescriptions, each containing at least two medications'), rather than in the mere existence of a single occurrence. The authors argue that this is the right inductive bias for relational databases, where labels depend on statistics computed over related rows. They claim the method finds these meta-paths automatically, scales linearly in the number of relations, and is genuinely self-explainable because the trained network can only access the subgraph induced by the identified meta-paths. Experiments on synthetic count-based scenarios and on medical, geographic, and Formula 1 databases report large F1 gains over non-relational, heterogeneous, and meta-path baselines.","feed_headline":"GNN learns database patterns that count linked rows","feed_subtitle":"MPS-GNN scores relations by aggregate statistics, so predictions explain themselves through the meta-paths they rely on.","key_machinery":"The load-bearing object is the node feature function $f(v,r,\\Theta,w)=\\Theta^\\top x_v$ when $v$ has no $r$-neighbors, and $f(v,r,\\Theta,w)=\\Theta^\\top x_v \\sum_{u\\in N^r_v} w_u$ otherwise, with per-node weights $w_u\\in[0,1]$; the sum over the $r$-neighborhood is what lets the score count occurrences instead of checking existence. Relation selection minimizes $L(r)=\\min_{\\Theta,w}\\sum_{B^+\\in S^+, B^-\\in S^-}\\sigma(F(B^-)-F(B^+))$, and newly created bags with weights from Equations (5)–(6) propagate informativeness to the next iteration. The MPS-GNN layer $h^{(l+1)}_v=\\sigma(W^{(l)}_0 h^{(l)}_v + W^{(l)}_{\\mathrm{neigh}}\\sum_{u\\in N^{r_{L-l}}_v}h^{(l)}_u + W^{(l)}_1 h^{(0)}_v)$ includes a skip connection from the input features at every layer, which the ablation shows is needed to keep node attributes available when statistics are computed.","core_discovery":"The central claim is that class membership in relational data can be determined by learnable statistics over meta-path realizations, and that such meta-paths can be identified by a greedy, local search without user supervision. The search keeps a growing meta-path prefix; at each step it scores every candidate next relation by solving a weighted multi-instance classification problem whose node weights act as 'putative features' that later steps materialize as real features. A sum aggregation over neighbors replaces the existential max of the predecessor method, so multiple occurrences contribute to the score. The final MPS-GNN uses only the subgraph spanned by the occurrences of the selected meta-paths, which makes the meta-paths sufficient explanations by construction; necessity is then verified by removing occurrences and observing a drop in F1 and a rise in predicted-probability distance.","pith_inferences":["Because the scoring function is already a weighted multi-instance regression in disguise, the same machinery should extend to multiclass and regression targets by replacing scalar weights with vector-valued versions; the authors note this extension but leave it to future work.","The discovered meta-paths could be compiled directly into SQL aggregate features (COUNT, SUM, AVG over joined tables), offering a testable bridge between the learned explanations and classical relational feature engineering.","A caveat the paper itself states: the scoring function relies on a well-connected graph, and disconnected neighborhoods require supernode preprocessing; datasets with sparse connectivity may need that step before the guarantees apply.","One can test the 'putative feature' assumption directly by checking whether the learned per-node weights correlate with node attributes on held-out bags; if they instead memorize node identities, the greedy search could be misled in larger graphs."],"forward_implications":["Relational databases with many tables can be handled without a domain expert listing the relevant meta-paths, because the scoring function selects relations by their potential to support aggregate features.","The search cost for a meta-path of length $L$ drops from testing all $|R|^L$ paths to $O(|R|\\cdot L)$ scoring steps, at fixed beam size.","The discovered meta-paths double as model-level explanations: predictions cannot change when parts of the graph outside the meta-path-induced subgraph change, and removing meta-path occurrences measurably degrades performance.","Count-based synthetic tasks—where at least $c$ occurrences of a length-$l$ meta-path decide the label—are solved near-optimally, while the existential-only predecessor fails on the same tasks.","On EICU, MONDIAL, and ErgastF1 the method reports the highest F1 among the compared models, with the identified meta-paths matching domain intuition about vital signs, language/border/ethnic-group information, and standings."],"supporting_citations":[{"why":"Defines MP-GNN, the predecessor whose existential meta-path assumption MPS-GNN generalizes; supplies the scoring-function and meta-path search machinery.","marker":"Ferrini et al. (2024)"},{"why":"Establishes the relational deep learning paradigm that motivates treating relational databases as heterogeneous graphs.","marker":"Fey et al. (2023)"},{"why":"Provides the relation-specific GCN baseline (RGCN) that MPS-GNN must beat and that scales poorly with the number of relations.","marker":"Schlichtkrull et al. (2018)"},{"why":"Raises the faithfulness question for self-explainable GNNs, which the paper answers with sufficiency-by-construction and necessity experiments.","marker":"Christiansen et al. (2023)"},{"why":"Supplies the RelBench temporal benchmark used in the extended temporal experiment.","marker":"Robinson et al. (2024)"},{"why":"Provides the necessity metric (predicted-probability difference after masking) used to evaluate explanation faithfulness.","marker":"Azzolin et al. (2025)"}],"fun_headline_variants":["Self-explaining GNN counts linked rows for predictions","GNN picks meta-paths that explain its own predictions","Relational GNN learns counting rules that explain itself","Meta-path GNN uses aggregate counts to self-explain","Heterogeneous GNN finds explainable data patterns automatically"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The search assumes that the per-node weights learned to separate positive and negative bags stand for features that can be materialized from node attributes or from meta-path extensions; if those weights simply memorize node identities, the relation scores can point the greedy search at uninformative meta-paths.","fun_headline_variants_meta":{"raw":{"variants":["Self-explaining GNN counts linked rows for predictions","GNN picks meta-paths that explain its own predictions","Relational GNN learns counting rules that explain itself","Meta-path GNN uses aggregate counts to self-explain","Heterogeneous GNN finds explainable data patterns automatically"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000572,"raw_usage":{"total_tokens":2670,"prompt_tokens":880,"completion_tokens":1790,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":496,"completion_tokens_details":{"reasoning_tokens":1711}},"tokens_in":496,"tokens_out":1790,"duration_ms":12118,"temperature":1.0,"reasoning_tokens":1711,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-12T05:16:54.285186+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Build a synthetic database whose label depends on at least $c$ occurrences of a known meta-path, and add a decoy relation that connects each target node to a unique dummy node carrying a one-hot identifier. If the scoring function gives the decoy relation a lower loss than the correct one, or if the trained MPS-GNN's predictions change when the decoy subgraph is modified, then the central claim that the weights represent materializable features is falsified.","supporting_citations":[{"cited_title":"Meta-path learning for multi-relational graph neural networks","cited_arxiv_id":null,"evidence_quote":"Defines MP-GNN, the predecessor whose existential meta-path assumption MPS-GNN generalizes; supplies the scoring-function and meta-path search machinery."},{"cited_title":"Modeling relational data with graph convolutional networks","cited_arxiv_id":null,"evidence_quote":"Provides the relation-specific GCN baseline (RGCN) that MPS-GNN must beat and that scales poorly with the number of relations."},{"cited_title":"Reconsidering faithfulness in regular, self-explainable and domain invariant GNN s","cited_arxiv_id":null,"evidence_quote":"Provides the necessity metric (predicted-probability difference after masking) used to evaluate explanation faithfulness."}],"review_version":1}