{"id":"2fdfcf7a-019a-4d15-93fd-aba67dd4e44c","arxiv_id":"2412.05218","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":6,"one_line_summary":"A new neural architecture, DBFORMER, applies Transformer self-attention within rows and cross-attention across foreign-key-linked rows, and reports superior average performance over a wide relational database benchmark suite.","lead":"This paper introduces a message-passing architecture family that lets Transformer models learn directly from relational databases, treating tables, rows, and foreign keys as structured objects rather than flattened tables. Its flagship model, DBFORMER, reports the best average rank across 19 classification and 16 regression benchmarks compared with tabular, propositionalization, and statistical-relational baselines.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"DBFORMER's 'superior performance' is unverified against the closest deep relational line of work: Sec. V-B omits GFS/SPARE/ATJ-Net on the assumption they reduce to DBGNN-style attribute combination, but that equivalence is asserted, not demonstrated.","rationale":"The reader's weakest assumption identifies exactly the load-bearing gap: the evaluation compares DBFORMER against tabular, propositionalization, and neuro-symbolic baselines, but not against the recent deep-relational GNN line of work. The paper tries to close this gap with a conceptual-closeness claim in Sec. V-B. That claim is doing real epistemic work: it converts an absence of comparison into an implicit comparison, and it is not justified by any concrete reduction in the paper. I checked the cited methods and they are not obviously reduced variants of the proposed scheme: ATJ-Net, SPARE, and GFS each contain mechanisms (automatic join selection, single-pass relation-specific message passing, feature-store updates) that go beyond flattening attributes into a graph convolution. The architecture itself is coherent, the code and data are public, and the ablations in App. C show that text and timestamp embeddings materially help, which is genuine supporting evidence. But none of that evidence bears on whether DBFORMER is superior to GFS or SPARE. The conditional verdict remains appropriate: the paper should either run these baselines or explicitly narrow its claim from 'superior performance' to 'superior performance among the baselines evaluated.' The proposed concrete test would settle the matter by measuring whether DBFORMER actually holds its rank when the closest deep-relational methods are included.","tokens_in":20818,"tokens_out":4663,"duration_ms":51037,"concrete_test":"Use the public code for GFS (arXiv:2312.02037) and SPARE (arXiv:2310.13581) to run five representative relational classification datasets (Carcinogenesis, Mondial, financial, imdb-ijs, tpcd) from the authors' public relational.fel.cvut.cz server, using the paper's 70/30 split, 16-trial Optuna budget, and accuracy metric; repeat DBFORMER, GFS, and SPARE with 5 seeds. If either baseline matches or beats DBFORMER's reported accuracies (75.51, 100.00, 88.73, 93.29, 73.35) on three or more datasets, or if the average-rank gap between DBFORMER and the best baseline is within run-to-run noise, the omitted-baseline assumption lands and the superiority claim needs qualification.","verdict_should_be":"CONDITIONAL","load_bearing_attack":"The abstract claims 'superior performance' for the proposed class, and Tables I-II support it with DBFORMER's average ranks of 1.95 (classification) and 2.44 (regression). However, the only modern deep-relational competitor actually run is the authors' own DBGNN reduced variant. Section V-B explicitly says the recent deep relational proposals could not be put into operation and asserts they are 'conceptually close to the reduced (attribute combination) variants of the scheme (Sec. IV-B).' That assertion is the load-bearing premise for extending the comparison to the closest line of work, and it is not supported in the text. ATJ-Net performs automatic join-path selection and table integration before prediction; SPARE uses a single-pass message-passing scheme with relation-specific feature memories and no row-column flattening; GFS aggregates neighbor features through feature synthesis and updates feature vectors, not just tuple embeddings. None of these is equivalent to DBGNN's concatenation of all attributes followed by GraphSAGE convolution, so they are not covered by the reduced-variant argument. Consequently the central claim is unverified against modern deep relational learning; no experiment bounds how much better or worse DBFORMER is than these methods. The same absence weakens the CILP++ comparison, which is emulated by propositionalization plus an FNN rather than the actual CILP++ algorithm.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"This paper proposes a modular two-level neural message-passing scheme for supervised learning from relational databases, preserving per-attribute structure inside tables and propagating information across foreign-key relationships. The DBFORMER instantiation combines Transformer encoders for intra-table attribute self-attention with cross-attention for inter-table joins, and the authors compare several instantiations with tabular FNN, RDN-boost, getML propositionalization, an emulated CILP++ baseline, and tabular Transformer baselines across 19 classification and 16 regression datasets from the CTU Prague relational repository. The headline result is that DBFORMER achieves the best average rank (1.95 classification, 2.44 regression), supporting the paper's claim of superior performance for the proposed architecture class.","tokens_in":21133,"tokens_out":5672,"duration_ms":60467,"significance":"If the empirical claims hold, this is a useful contribution: it gives a clean formalization of relational databases as two-level hypergraphs, proposes a flexible scheme that subsumes both tabular Transformers and GNN-style message passing, and provides a public implementation, public datasets, and a broad benchmark suite. The ablation studies for text and time embeddings and the fixed-hyperparameter robustness table are also valuable. However, the central claim is an empirical comparison, and the current evaluation has load-bearing gaps: the closest modern deep relational baselines are not run, the CILP++ baseline is an emulation rather than the original method, and the reported numbers lack variance or significance information. These gaps currently make the 'superior performance' claim stronger than the evidence supports.","major_comments":[{"comment":"The abstract and Sec. II-F claim superior performance against the deep relational learning line, but no recent deep relational baseline (GFS, SPARE, ATJ-Net, or a RelBench-style GNN) is actually run. Sec. V-B explains this by asserting that the closest GNN-based works are 'conceptually close to the reduced (attribute combination) variants of the scheme (Sec. IV-B).' That assertion is load-bearing for the claim against the deep relational line, yet it is not demonstrated: as described in Sec. II-E, ATJ-Net performs join-path selection, SPARE uses single-pass message passing with relation-specific feature memories, and GFS performs feature synthesis and updates feature vectors, none of which reduces to DBGNN's attribute concatenation plus SAGE convolution. The paper should either run at least one recent deep relational baseline on the same benchmark or explicitly restrict the superiority claim to the baselines actually compared.","section":"Sec. V-B; Sec. II-F"},{"comment":"The headline average-rank results are reported as single numbers with no standard deviation, confidence interval, or significance test. Because the comparison involves stochastic neural training and a 16-run hyperparameter search (App. A-B), differences such as Carcinogenesis 75.51 vs. 73.47, Pima 83.04 vs. 83.48, and Triazine NRMSE 0.1354 vs. 0.1174 are not interpretable without repeated-seed statistics or pairwise significance tests. Additionally, the treatment of N/A entries when computing average ranks in Tables I and II is unspecified; if ranks are computed over different dataset subsets per model, the averages are not directly comparable. This is a load-bearing issue for the central empirical claim of DBFORMER's superiority.","section":"Tables I and II; Sec. V-E"},{"comment":"The CILP++ baseline is not the actual algorithm: the paper emulates it by propositionalization followed by an FNN. Therefore the category labeled 'Ne-Sy' in Tables I and II is a propositionalization-plus-FNN baseline, and statements in Sec. II-D and V-E about outperforming neuro-symbolic models are not supported by the experiments. The authors should either run the real CILP++ implementation or relabel the baseline and reword the corresponding claims.","section":"Sec. V-B"},{"comment":"The tuning effort for the baselines appears not to be matched. RDN-boost is described in Sec. V-E as using a 'rather straightforward' mode setup that 'possibly explain[s] its generally weaker performance,' and getML uses a custom BFS depth implemented by the authors. Since the central claim is empirical superiority over representative related work, the protocol should document and equalize the hyperparameter search budgets for all methods, or restrict conclusions to the actual configurations used.","section":"Sec. V-D; App. A-B"}],"minor_comments":[{"comment":"The text says 'with Trompt [11]' but Trompt is reference [12]; the citation numbering for Trompt and SAINT should be checked.","section":"Sec. V-E"},{"comment":"The equation block contains a spacing/typesetting error in 'N RM SEfunction', and the text should state explicitly whether the NRMSE normalization in Eq. (2) is applied using the training mean for validation/test predictions as well.","section":"App. A"},{"comment":"The diagram is dense and would benefit from clearer annotations distinguishing the optional residual connection from the transformation, combination, and aggregation operators, since those terms are central to the proposed scheme.","section":"Fig. 1"},{"comment":"The N/A entries and the average-rank computation should be explained in the table captions or in Sec. V-E, because missing models affect the comparability of the reported average ranks.","section":"Tables III and IV"}],"recommendation":"major_revision","confidential_remarks":"The authors maintain the CTU Prague relational learning repository and build directly on their own prior workshop paper [41], which is also cited as the predecessor of this work. This is not improper, but the editor may want assurance that the public benchmark and code will remain available and that the revision addresses the absence of independent deep relational baselines. The paper fits the journal's scope; the main risk is that the empirical superiority claim currently overreaches the evidence."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Pal, here's my read of Peleška and Šír.\n\nThe genuinely new thing is the two-level message-passing scheme. Instead of flattening a database into one big table, they keep tuple structure intact, run self-attention over the attributes of each tuple, and cross-attention between tuples linked by foreign keys. The scheme is modular and the DBFORMER instantiation makes sense as a bridge between tabular Transformers and GNN message passing. That is a real contribution. The paper also ships code, data, and a public benchmark server, and the ablations on text and time embeddings are useful.\n\nThe soft spots are exactly where the reader's report puts them. The headline claim of 'superior performance' rests on Tables I and II, but those tables report single numbers with no standard deviations or significance tests. Average ranks over 19 and 16 datasets are suggestive, but without variance we don't know how stable the rankings are. More importantly, the evaluation omits the recent deep relational baselines — GFS, SPARE, ATJ-Net, RelBench. The paper says these are 'conceptually close' to reduced attribute-combination variants of the scheme, but that is asserted, not argued. DBGNN, the one GNN baseline they do run, is their own reduced variant, so it doesn't cover the features that distinguish those methods, like GFS's feature synthesis or SPARE's single-pass memory design. The CILP++ baseline is also emulated with propositionalization plus an FNN, not the actual algorithm. So the central superiority claim is not fully verified against the closest line of work.\n\nI don't think any of this is fatal. The architecture is well specified, the benchmark suite is broad, and the direction is worthwhile. But the evaluation needs to be tightened before the paper's main claim is credible. A referee should ask for error bars, actual implementations of the omitted baselines (or at least a real argument for equivalence), and clearer criteria for dataset filtering.\n\nThis paper deserves a serious referee. It is a legitimate contribution to an under-served area, and the code and data make the work reproducible. I'd engage with it, and I'd cite the architecture if I were working on relational deep learning.","headline":"A well-specified architecture for learning on relational databases, but the 'superior performance' claim is weaker than the tables look because recent deep relational baselines are omitted.","tokens_in":21648,"tokens_out":2527,"would_cite":true,"duration_ms":25227,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"This paper claims that a Transformer-based message-passing scheme built directly on the relational database model — keeping each row's columns intact and passing messages across foreign keys — outperforms propositionalization, statistical…","keywords":["relational databases","Transformers","message passing","relational learning","tabular data","propositionalization","graph neural networks","end-to-end deep learning"],"falsifier":"Running one of the recent deep relational models from the paper's own related-work section on the same 19 classification and 16 regression datasets and observing an average rank below 1.95 or 2.44 would overturn the claimed superiority; a simpler check is to show that on a database with rich foreign-key structure, the attribute-combination (flattened) variant matches DBFORMER's accuracy, which would indicate the cross-attention machinery is not doing the work.","tokens_in":20597,"feed_emoji":"🗄️","tokens_out":6294,"duration_ms":60419,"temperature":0.7,"pith_summary":"The paper claims that a neural message-passing scheme built directly on the relational database model lets Transformer architectures learn end-to-end from SQL databases without denormalizing the data into flat tables. It proposes DBFORMER, an instantiation that keeps the rows and columns of each table intact, passes messages between tuples linked by foreign keys, and uses self-attention inside a table and cross-attention between related tables. Across 19 classification and 16 regression relational benchmarks, it reports the best average rank (1.95 and 2.44) against propositionalization, statistical relational learning, neuro-symbolic, and tabular Transformer baselines. If true, this would make Transformers a practical end-to-end alternative to manual feature engineering on relational data.","feed_headline":"Transformer variant tops relational database benchmarks","feed_subtitle":"A message-passing design preserving table rows and foreign keys averages rank 1.95 on 19 classification tasks.","key_machinery":"The load-bearing object is a two-level multi-relational hypergraph representation of a database: each relation's tuples are $n$-ary hyperedges over their attribute values, and each foreign-key pair between tuples forms bidirectional hyperedges. The architecture scheme composes modular operations — transformations ($1{:}1$), combinations ($N{:}1$), and permutation-invariant aggregations ($M{:}1$) — over attributes and tuples, with DBFORMER instantiating the tuple transformation as a Transformer Encoder (self-attention over columns), the tuple combination as cross-attention between related tuples, and the aggregation as attention-sum. This machinery lets the model preserve intra-row column structure while propagating information across the relational schema, and it reduces to standard heterogeneous GNNs when the attribute-combination operation flattens columns.","core_discovery":"The central discovery is that the relational model can be turned into a two-level hypergraph — tuples of a relation form $n$-ary hyperedges over their attribute values, and foreign-key links form bidirectional hyperedges between tuple pairs — and that a Transformer can be integrated into a message-passing scheme over this hypergraph. In DBFORMER, a Transformer Encoder acts as the tuple transformation (self-attention over the attributes of a row), cross-attention acts as the tuple combination (queries from the referencing tuple, keys and values from the referenced tuple), and attention-sum aggregates messages from all related tables. The paper argues that this corresponds to the select-join-aggregate operations of SQL done in a differentiable, end-to-end way, with foreign-key, primary-key, and column-value roles assigned to query, key, and value. Its experiments show that DBFORMER achieves the best average rank among all compared methods on both classification and regression suites.","pith_inferences":["If the method's advantage is real, the common practice of denormalizing databases into flat feature vectors before learning should be revisitable; the natural test is a head-to-head on a newer, larger relational benchmark suite with modern deep relational baselines actually run.","The cross-attention weights between referencing and referenced tuples may be interpretable as learned join importance, suggesting a tool for automated feature selection over the schema.","The authors' reliance on heuristics for schema type detection leaves room for substantial gains; better automatic classification of cyclic, ordinal, and text columns could improve the model without changing its architecture.","The scheme is generic enough to apply to multi-relational settings beyond SQL stores, such as knowledge graphs with rich per-node attributes, as long as a two-level hypergraph with primary-foreign-key links is available."],"forward_implications":["Relational databases can be used for end-to-end deep learning without manual propositionalization, because the scheme loads samples through breadth-first search over foreign keys and handles large stores with in-database, lazy minibatch fetching.","The architecture space includes the tested competitors as special cases: flattening the attribute-combination step turns the scheme into a heterogeneous GNN, and using tabular Transformer encoders as the tuple transformation turns it into a relational tabular model.","DBFORMER's cross-attention gives a differentiable analogue of select-join-aggregate operations, so the relational features that propositionalization constructs by hand are in principle learnable by gradient descent.","Preserving text and timestamp columns through dedicated embedders produces large accuracy gains in the reported experiments, meaning schema-aware type detection is load-bearing for the method's performance.","Any single-target supervised task on a relational database, including databases with cycles, composite keys, and very large row counts, fits the same loading and message-passing pipeline."],"supporting_citations":[{"why":"Supplies the Transformer self-attention and cross-attention blocks that DBFORMER instantiates as tuple transformation and tuple combination.","marker":"[5]"},{"why":"Defines the relational model and integrity constraints (primary and foreign keys) that the nested hypergraph representation follows.","marker":"[42]"},{"why":"Provides the relational database repository whose benchmarks and datasets are used for the classification and regression experiments.","marker":"[51]"},{"why":"Supplies the propositionalization baseline that DBFORMER must beat in the comparison.","marker":"[20]"},{"why":"Provides the statistical relational learning baseline used in the classification comparison.","marker":"[59]"},{"why":"Provides the neuro-symbolic baseline that connects propositionalization with a neural network.","marker":"[25]"},{"why":"One of the tabular Transformer instantiations used inside the scheme and as a comparison model.","marker":"[10]"},{"why":"Supplies the inter-sample attention tabular Transformer variant used as a DBFORMER competitor and scheme instantiation.","marker":"[11]"},{"why":"Presents the deep relational blueprint that this work directly extends.","marker":"[41]"}],"fun_headline_variants":["Relational Transformer tops 19 benchmarks via hypergraph messaging","DBFORMER: end-to-end Transformer for relational databases","A Transformer that speaks SQL earns top rank on tabular tasks","Hypergraph message-passing Transformer outranks tabular models"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The claim that DBFORMER beats the full deep relational line rests on the assumption that recent deep relational proposals are conceptually close to flattened variants of the same scheme, since none of those proposals was actually run.","fun_headline_variants_meta":{"raw":{"variants":["Relational Transformer tops 19 benchmarks via hypergraph messaging","DBFORMER: end-to-end Transformer for relational databases","A Transformer that speaks SQL earns top rank on tabular tasks","Hypergraph message-passing Transformer outranks tabular models"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000819,"raw_usage":{"total_tokens":3529,"prompt_tokens":831,"completion_tokens":2698,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":447,"completion_tokens_details":{"reasoning_tokens":2641}},"tokens_in":447,"tokens_out":2698,"duration_ms":18203,"temperature":1.0,"reasoning_tokens":2641,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-11T20:49:43.168551+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Running one of the recent deep relational models from the paper's own related-work section on the same 19 classification and 16 regression datasets and observing an average rank below 1.95 or 2.44 would overturn the claimed superiority; a simpler check is to show that on a database with rich foreign-key structure, the attribute-combination (flattened) variant matches DBFORMER's accuracy, which would indicate the cross-attention machinery is not doing the work.","supporting_citations":[{"cited_title":"Attention is all you need,","cited_arxiv_id":null,"evidence_quote":"Supplies the Transformer self-attention and cross-attention blocks that DBFORMER instantiates as tuple transformation and tuple combination."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Defines the relational model and integrity constraints (primary and foreign keys) that the nested hypergraph representation follows."},{"cited_title":"Available: https: //getml.com","cited_arxiv_id":null,"evidence_quote":"Supplies the propositionalization baseline that DBFORMER must beat in the comparison."},{"cited_title":"Gradient-based boosting for statistical relational learning: The rela- tional dependency network case,","cited_arxiv_id":null,"evidence_quote":"Provides the statistical relational learning baseline used in the classification comparison."},{"cited_title":"Fast relational learning using bottom clause propositionalization with artificial neural networks,","cited_arxiv_id":null,"evidence_quote":"Provides the neuro-symbolic baseline that connects propositionalization with a neural network."},{"cited_title":"Saint: Improved neural networks for tabular data via row attention and contrastive pre-training,","cited_arxiv_id":null,"evidence_quote":"Supplies the inter-sample attention tabular Transformer variant used as a DBFORMER competitor and scheme instantiation."},{"cited_title":"A deep learning blueprint for relational databases,","cited_arxiv_id":null,"evidence_quote":"Presents the deep relational blueprint that this work directly extends."}],"review_version":1}