Pith. sign in

REVIEW 4 major objections 4 minor 69 references

Transformers Meet Relational Databases

T0 review · 4 major / 4 minor · reviewed 2026-08-11 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read This paper claims that a Transformer-based message-passing scheme built directly on the relational database model — keeping each row's columns intact and passing messages across foreign keys — outperforms propositionalization, statistical…

desk verdict A well-specified architecture for learning on relational databases, but the 'superior performance' claim is weaker than the tables look because recent deep relational baselines are omitted. read the letter →

arxiv 2412.05218 v1 pith:MFL2F3OF submitted 2024-12-06 cs.LG cs.DB

Jakub Peleška , Gustav Šír This is my paper · ORCID
classification cs.LGcs.DB
keywords relationaldatabasesTransformersmessagepassinglearningtabulardatapropositionalizationgraphneuralnetworksend-to-enddeep
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper claims that a neural message-passing scheme built directly on the relational database model lets Transformer architectures learn end-to-end from SQL databases without denormalizing the data into flat tables. It proposes DBFORMER, an instantiation that keeps the rows and columns of each table intact, passes messages between tuples linked by foreign keys, and uses self-attention inside a table and cross-attention between related tables. Across 19 classification and 16 regression relational benchmarks, it reports the best average rank (1.95 and 2.44) against propositionalization, statistical relational learning, neuro-symbolic, and tabular Transformer baselines. If true, this would make Transformers a practical end-to-end alternative to manual feature engineering on relational data.

What carries the argument

The load-bearing object is a two-level multi-relational hypergraph representation of a database: each relation's tuples are $n$-ary hyperedges over their attribute values, and each foreign-key pair between tuples forms bidirectional hyperedges. The architecture scheme composes modular operations — transformations ($1{:}1$), combinations ($N{:}1$), and permutation-invariant aggregations ($M{:}1$) — over attributes and tuples, with DBFORMER instantiating the tuple transformation as a Transformer Encoder (self-attention over columns), the tuple combination as cross-attention between related tuples, and the aggregation as attention-sum. This machinery lets the model preserve intra-row column structure while propagating information across the relational schema, and it reduces to standard heterogeneous GNNs when the attribute-combination operation flattens columns.

What would settle it

Running one of the recent deep relational models from the paper's own related-work section on the same 19 classification and 16 regression datasets and observing an average rank below 1.95 or 2.44 would overturn the claimed superiority; a simpler check is to show that on a database with rich foreign-key structure, the attribute-combination (flattened) variant matches DBFORMER's accuracy, which would indicate the cross-attention machinery is not doing the work.

Watch

Extended reading notes

Core claim

The central discovery is that the relational model can be turned into a two-level hypergraph — tuples of a relation form $n$-ary hyperedges over their attribute values, and foreign-key links form bidirectional hyperedges between tuple pairs — and that a Transformer can be integrated into a message-passing scheme over this hypergraph. In DBFORMER, a Transformer Encoder acts as the tuple transformation (self-attention over the attributes of a row), cross-attention acts as the tuple combination (queries from the referencing tuple, keys and values from the referenced tuple), and attention-sum aggregates messages from all related tables. The paper argues that this corresponds to the select-join-aggregate operations of SQL done in a differentiable, end-to-end way, with foreign-key, primary-key, and column-value roles assigned to query, key, and value. Its experiments show that DBFORMER achieves the best average rank among all compared methods on both classification and regression suites.

Load-bearing premise

The claim that DBFORMER beats the full deep relational line rests on the assumption that recent deep relational proposals are conceptually close to flattened variants of the same scheme, since none of those proposals was actually run.

Editorial extensions

If this is right

  • Relational databases can be used for end-to-end deep learning without manual propositionalization, because the scheme loads samples through breadth-first search over foreign keys and handles large stores with in-database, lazy minibatch fetching.
  • The architecture space includes the tested competitors as special cases: flattening the attribute-combination step turns the scheme into a heterogeneous GNN, and using tabular Transformer encoders as the tuple transformation turns it into a relational tabular model.
  • DBFORMER's cross-attention gives a differentiable analogue of select-join-aggregate operations, so the relational features that propositionalization constructs by hand are in principle learnable by gradient descent.
  • Preserving text and timestamp columns through dedicated embedders produces large accuracy gains in the reported experiments, meaning schema-aware type detection is load-bearing for the method's performance.
  • Any single-target supervised task on a relational database, including databases with cycles, composite keys, and very large row counts, fits the same loading and message-passing pipeline.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • If the method's advantage is real, the common practice of denormalizing databases into flat feature vectors before learning should be revisitable; the natural test is a head-to-head on a newer, larger relational benchmark suite with modern deep relational baselines actually run.
  • The cross-attention weights between referencing and referenced tuples may be interpretable as learned join importance, suggesting a tool for automated feature selection over the schema.
  • The authors' reliance on heuristics for schema type detection leaves room for substantial gains; better automatic classification of cyclic, ordinal, and text columns could improve the model without changing its architecture.
  • The scheme is generic enough to apply to multi-relational settings beyond SQL stores, such as knowledge graphs with rich per-node attributes, as long as a two-level hypergraph with primary-foreign-key links is available.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 4 minor

Summary. This paper proposes a modular two-level neural message-passing scheme for supervised learning from relational databases, preserving per-attribute structure inside tables and propagating information across foreign-key relationships. The DBFORMER instantiation combines Transformer encoders for intra-table attribute self-attention with cross-attention for inter-table joins, and the authors compare several instantiations with tabular FNN, RDN-boost, getML propositionalization, an emulated CILP++ baseline, and tabular Transformer baselines across 19 classification and 16 regression datasets from the CTU Prague relational repository. The headline result is that DBFORMER achieves the best average rank (1.95 classification, 2.44 regression), supporting the paper's claim of superior performance for the proposed architecture class.

Significance. If the empirical claims hold, this is a useful contribution: it gives a clean formalization of relational databases as two-level hypergraphs, proposes a flexible scheme that subsumes both tabular Transformers and GNN-style message passing, and provides a public implementation, public datasets, and a broad benchmark suite. The ablation studies for text and time embeddings and the fixed-hyperparameter robustness table are also valuable. However, the central claim is an empirical comparison, and the current evaluation has load-bearing gaps: the closest modern deep relational baselines are not run, the CILP++ baseline is an emulation rather than the original method, and the reported numbers lack variance or significance information. These gaps currently make the 'superior performance' claim stronger than the evidence supports.

major comments (4)
  1. [Sec. V-B; Sec. II-F] The abstract and Sec. II-F claim superior performance against the deep relational learning line, but no recent deep relational baseline (GFS, SPARE, ATJ-Net, or a RelBench-style GNN) is actually run. Sec. V-B explains this by asserting that the closest GNN-based works are 'conceptually close to the reduced (attribute combination) variants of the scheme (Sec. IV-B).' That assertion is load-bearing for the claim against the deep relational line, yet it is not demonstrated: as described in Sec. II-E, ATJ-Net performs join-path selection, SPARE uses single-pass message passing with relation-specific feature memories, and GFS performs feature synthesis and updates feature vectors, none of which reduces to DBGNN's attribute concatenation plus SAGE convolution. The paper should either run at least one recent deep relational baseline on the same benchmark or explicitly restrict the superiority claim to the baselines actually compared.
  2. [Tables I and II; Sec. V-E] The headline average-rank results are reported as single numbers with no standard deviation, confidence interval, or significance test. Because the comparison involves stochastic neural training and a 16-run hyperparameter search (App. A-B), differences such as Carcinogenesis 75.51 vs. 73.47, Pima 83.04 vs. 83.48, and Triazine NRMSE 0.1354 vs. 0.1174 are not interpretable without repeated-seed statistics or pairwise significance tests. Additionally, the treatment of N/A entries when computing average ranks in Tables I and II is unspecified; if ranks are computed over different dataset subsets per model, the averages are not directly comparable. This is a load-bearing issue for the central empirical claim of DBFORMER's superiority.
  3. [Sec. V-B] The CILP++ baseline is not the actual algorithm: the paper emulates it by propositionalization followed by an FNN. Therefore the category labeled 'Ne-Sy' in Tables I and II is a propositionalization-plus-FNN baseline, and statements in Sec. II-D and V-E about outperforming neuro-symbolic models are not supported by the experiments. The authors should either run the real CILP++ implementation or relabel the baseline and reword the corresponding claims.
  4. [Sec. V-D; App. A-B] The tuning effort for the baselines appears not to be matched. RDN-boost is described in Sec. V-E as using a 'rather straightforward' mode setup that 'possibly explain[s] its generally weaker performance,' and getML uses a custom BFS depth implemented by the authors. Since the central claim is empirical superiority over representative related work, the protocol should document and equalize the hyperparameter search budgets for all methods, or restrict conclusions to the actual configurations used.
minor comments (4)
  1. [Sec. V-E] The text says 'with Trompt [11]' but Trompt is reference [12]; the citation numbering for Trompt and SAINT should be checked.
  2. [App. A] The equation block contains a spacing/typesetting error in 'N RM SEfunction', and the text should state explicitly whether the NRMSE normalization in Eq. (2) is applied using the training mean for validation/test predictions as well.
  3. [Fig. 1] The diagram is dense and would benefit from clearer annotations distinguishing the optional residual connection from the transformation, combination, and aggregation operators, since those terms are central to the proposed scheme.
  4. [Tables III and IV] The N/A entries and the average-rank computation should be explained in the table captions or in Sec. V-E, because missing models affect the comparability of the reported average ranks.

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity found: the central superiority claim rests on external benchmark comparisons, not on a derivation that reduces to its inputs.

full rationale

The paper's central claim is empirical: DBFORMER and its sibling instantiations of the proposed message-passing scheme achieve the best average ranks in Tables I and II. The architecture is defined by a modular message-passing scheme (Sec. IV-B) with choices such as Transformer encoders, cross-attention, and attention-sum aggregations; these definitions are not written in terms of the benchmark results, and the datasets come from the public CTU Prague relational repository [51] rather than being generated by the model. There is no fitted parameter that is later renamed as a prediction, and no equation in Sec. IV-C or Appendix C reduces a reported result to an assumed value by construction. The one self-citation, [41] (Zahradník, Neumann, and Šír), appears in Sec. II-F as prior work that the paper 'directly expands'; it is contextual and does not carry the weight of the performance claim. The most serious limitation, stated in Sec. V-B, is that modern deep relational baselines (e.g., RelBench, GFS, SPARE, ATJ-Net) were not run and are asserted to be 'conceptually close' to reduced attribute-combination variants of the scheme. That is a substantiation and coverage gap in the comparison, not a circular reduction: the assertion is unsupported, but the DBFORMER results are not made true by it. Accordingly, no circular step is identified under the required standard of exhibiting a specific equation-level or fit-level reduction.

Assumptions & free parameters 6 free parameters · 5 assumptions · 1 invented entities

The empirical claim rests on hyperparameter choices, sampling assumptions, and benchmark representativeness. The formal hypergraph representation is deterministic from the database, so it is not an invented physical entity. No new particles, forces, or unobservable quantities are introduced.

free parameters (6)
  • embedding dimension D = 16, 32, or 64 (chosen by Optuna)
    Every neural model tunes D per dataset; the architecture's capacity depends on it (App. A-B).
  • number of scheme layers N = 1 to 5 (random choice)
    Depth of relational message passing is a tuned hyperparameter, affecting the receptive field (App. A-B).
  • learning rate = log-uniform 0.00005 to 0.002
    Standard Adam learning rate tuned per model and dataset.
  • batch size = 16 to 16384, from a scale factor
    Batch size is tuned and interacts with the heterogeneous graph sampling strategy.
  • decoder head layers M = 1, 2, or 3
    Prediction head capacity is tuned as a hyperparameter.
  • BFS depth limit = matches number of scheme layers in inductive setting, otherwise optional
    Controls how much relational context each training sample sees; affects memory and performance (Section IV-A4).
assumptions (5)
  • domain assumption The two-level multi-relational hypergraph with bidirectional foreign-key hyperedges fully captures the relevant relational database semantics.
    Used in Section IV-A1 to define the learning representation; if it loses information, the architecture cannot recover it.
  • domain assumption Automated schema detection from heuristics and data statistics correctly identifies nominal, ordinal, numeric, cyclic, and text attribute types.
    Section IV-A2 relies on this to choose embeddings; errors would change model inputs and degrade results.
  • domain assumption BFS and HGSampling produce representative minibatches, and the label masking prevents target leakage.
    Section IV-A3 and IV-A4 describe sampling; non-i.i.d. sampling can bias gradients if neighborhoods are not representative.
  • ad hoc to paper Cross-attention with query, key, and value roles assigned to foreign key, primary key, and attribute values can express SQL-like select-join-aggregate features.
    Section IV-C states this as a hypothesis; the empirical superiority claim depends on this learnability.
  • domain assumption The selected 19 classification and 16 regression datasets represent relational database learning tasks, and the filtering of smaller or trivial datasets does not bias results.
    Section V-A describes dataset selection; average-rank claims depend on representativeness.
invented entities (1)
  • Two-level multi-relational hypergraph representation of a database independent evidence
    purpose: Enables attribute-level and tuple-level message passing directly on relational structure.
    Section IV-A1 introduces it as the learning representation; it is constructed deterministically from the database schema and tuples, so any database instance yields a checkable graph. It is not an unobservable postulate.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Transformers Meet Relational Databases." pith.science (2026). https://pith.science/paper/MFL2F3OF

@misc{pith2026241205218,
  author       = {Pith},
  title        = {Pith review of: Transformers Meet Relational Databases},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/MFL2F3OF}},
  note         = {Machine review of arXiv:2412.05218}
}
read the original abstract

Transformer models have continuously expanded into all machine learning domains convertible to the underlying sequence-to-sequence representation, including tabular data. However, while ubiquitous, this representation restricts their extension to the more general case of relational databases. In this paper, we introduce a modular neural message-passing scheme that closely adheres to the formal relational model, enabling direct end-to-end learning of tabular Transformers from database storage systems. We address the challenges of appropriate learning data representation and loading, which are critical in the database setting, and compare our approach against a number of representative models from various related fields across a significantly wide range of datasets. Our results demonstrate a superior performance of this newly proposed class of neural architectures.

Figures

Figures reproduced from arXiv: 2412.05218 by the authors.

Figure 1
Figure 1. The relational message-passing scheme of the proposed neural architecture space, instantiated with operations of the leading DBF [PITH_FULL_IMAGE:figures/full_fig_p005_1.png] view at source ↗
Figure 1
Figure 1. Firstly, the model instantiates a Transformer Encoder in place of the tuple transformation, facilitating self-attention over the relations’ attributes in the standard spirit of the tabular Transformers [8], but repeated across the database and over the layers, as part of the relational scheme. Secondly, the model also uses cross-attention in place of the tuple combination as Ct(ti , tj ) = attn(Q = ti , K = tj , V =… view at source ↗

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

69 extracted references · 44 canonical work pages

  1. [1]

    James, D

    G. James, D. Witten, T. Hastie, R. Tibshirani et al., An introduction to statistical learning. Springer, 2013, vol. 112

    2013

  2. [2]

    Halpin and T

    T. Halpin and T. Morgan, Information modeling and relational databases. Morgan Kaufmann, 2010

    work page 2010

  3. [3]

    Greedy function approximation: a gradient boosting machine,

    J. H. Friedman, “Greedy function approximation: a gradient boosting machine,” Annals of statistics , pp. 1189–1232, 2001

    2001

  4. [4]

    Tabular data: Deep learning is not all you need,

    R. Shwartz-Ziv and A. Armon, “Tabular data: Deep learning is not all you need,” Information Fusion, vol. 81, pp. 84–90, 2022

    2022

  5. [5]

    Attention is all you need,

    A. Vaswani, N. Shazeer, N. Parmar, J. Uszkoreit, L. Jones, A. N. Gomez, L. Kaiser, and I. Polosukhin, “Attention is all you need,” Neural Information Processing Systems , 2017

    work page 2017

  6. [6]

    Well-tuned simple nets excel on tabular datasets,

    A. Kadra, M. Lindauer, F. Hutter, and J. Grabocka, “Well-tuned simple nets excel on tabular datasets,” Advances in neural information process- ing systems, vol. 34, pp. 23 928–23 941, 2021

    work page 2021

  7. [7]

    Getoor and B

    L. Getoor and B. Taskar, Introduction to Statistical Relational Learning. The MIT Press, 08 2007. [Online]. Available: https: //doi.org/10.7551/mitpress/7432.001.0001

    doi:10.7551/mitpress/7432.001.0001 2007

  8. [8]

    Transformers for tabular data representation: A survey of models and applications,

    G. Badaro, M. Saeed, and P. Papotti, “Transformers for tabular data representation: A survey of models and applications,” Transactions of the Association for Computational Linguistics , vol. 11, pp. 227–249,

    work page

Show all 69 references
  1. [9]

    Tabnet: Attentive interpretable tabular learning,

    S. O. Arik and T. Pfister, “Tabnet: Attentive interpretable tabular learning,” 2020

    2020

  2. [11]

    Saint: Improved neural networks for tabular data via row attention and contrastive pre-training,

    G. Somepalli, M. Goldblum, A. Schwarzschild, C. B. Bruss, and T. Goldstein, “Saint: Improved neural networks for tabular data via row attention and contrastive pre-training,” 2021

    2021

  3. [12]

    Trompt: Towards a better deep neural network for tabular data,

    K.-Y . Chen, P.-H. Chiang, H.-R. Chou, T.-W. Chen, and T.-H. Chang, “Trompt: Towards a better deep neural network for tabular data,” 2023

    2023

  4. [13]

    Turning 30: New ideas in inductive logic programming,

    A. Cropper, S. Duman ˇci´c, and S. H. Muggleton, “Turning 30: New ideas in inductive logic programming,” arXiv preprint arXiv:2002.11002 , 2020

    2002 arXiv

  5. [14]

    De Raedt, Logical and relational learning

    L. De Raedt, Logical and relational learning . Springer Science & Business Media, 2008

    2008

  6. [15]

    J. H. Gallier, Logic for computer science: foundations of automatic theorem proving. Courier Dover Publications, 2015

    2015

  7. [16]

    Inductive logic programming: Theory and methods,

    S. Muggleton and L. De Raedt, “Inductive logic programming: Theory and methods,” The Journal of Logic Programming , vol. 19, 1994

    1994

  8. [17]

    Lifted graphical models: a survey,

    A. Kimmig, L. Mihalkova, and L. Getoor, “Lifted graphical models: a survey,” Machine Learning, vol. 99, no. 1, pp. 1–45, 2015

    2015

  9. [18]

    Propositionalization approaches to relational data mining,

    S. Kramer, N. Lavra ˇc, and P. Flach, “Propositionalization approaches to relational data mining,” Relational data mining , pp. 262–291, 2001

    2001

  10. [19]

    Featuretools

    The Alteryx, “Featuretools.” [Online]. Available: https://www. featuretools.com

  11. [20]

    Available: https: //getml.com

    The SQLNet Company GmbH, “getml.” [Online]. Available: https: //getml.com

  12. [21]

    Hammer and P

    B. Hammer and P. Hitzler, Perspectives of neural-symbolic integration . Springer, 2007, vol. 77

    2007

  13. [22]

    Learning knowledge base inference with neural theorem provers,

    T. Rockt ¨aschel and S. Riedel, “Learning knowledge base inference with neural theorem provers,” Proceedings of the 5th Workshop on Automated Knowledge Base Construction (AKBC) , pp. 45–50, 2016

    2016

  14. [23]

    Learning and reasoning with logic tensor networks,

    A. Serafini, Luciano, Garcez, L. Serafini, and A. S. d’Avila Garcez, “Learning and reasoning with logic tensor networks,” Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics) , vol. 10037, pp. 334–...

    2016

  15. [24]

    Lifted relational neural networks: Efficient learning of latent relational structures,

    G. ˇSourek, V . Aschenbrenner, F.ˇZelezn´y, S. Schockaert, and O. Kuˇzelka, “Lifted relational neural networks: Efficient learning of latent relational structures,” Journal of Artificial Intelligence Research , vol. 62, pp. 69– 100, 2018

    2018

  16. [25]

    Fast relational learning using bottom clause propositionalization with artificial neural networks,

    M. V . Franca, G. Zaverucha, and A. Garcez, “Fast relational learning using bottom clause propositionalization with artificial neural networks,” Machine learning, vol. 94, no. 1, pp. 81–104, 2014

    2014

  17. [26]

    A comprehensive survey on graph neural networks,

    Z. Wu, S. Pan, F. Chen, G. Long, C. Zhang, and S. Y . Philip, “A comprehensive survey on graph neural networks,” IEEE Transactions on Neural Networks and Learning Systems , 2020

    2020

  18. [27]

    The graph neural network model,

    F. Scarselli, M. Gori, A. C. Tsoi, M. Hagenbuchner, and G. Monfardini, “The graph neural network model,” IEEE Transactions on Neural Networks, vol. 20, no. 1, pp. 61–80, 2008

    2008

  19. [28]

    Hypergraph neural net- works,

    Y . Feng, H. You, Z. Zhang, R. Ji, and Y . Gao, “Hypergraph neural net- works,” in Proceedings of the AAAI Conference on Artificial Intelligence, vol. 33, 2019, pp. 3558–3565

    2019

  20. [29]

    Modeling relational data with graph convolutional networks,

    M. Schlichtkrull, T. N. Kipf, P. Bloem, R. Van Den Berg, I. Titov, and M. Welling, “Modeling relational data with graph convolutional networks,” in European Semantic Web Conference. Springer, 2018, pp. 593–607

    2018

  21. [30]

    Knowledge graph embed- ding: A survey of approaches and applications,

    Q. Wang, Z. Mao, B. Wang, and L. Guo, “Knowledge graph embed- ding: A survey of approaches and applications,” IEEE Transactions on Knowledge and Data Engineering, vol. 29, no. 12, pp. 2724–2743, 2017

    2017

  22. [31]

    Supervised learning on relational databases with graph neural networks,

    M. Cvitkovic, “Supervised learning on relational databases with graph neural networks,” arXiv preprint arXiv:2002.02046 , 2020

    2002 arXiv

  23. [32]

    Flaky performances when pre-training on relational databases with a plan for future characteriza- tion efforts,

    S. Liu, D. Vazquez, J. Tang, and P.-A. Noel, “Flaky performances when pre-training on relational databases with a plan for future characteriza- tion efforts,” in First Workshop on Pre-training: Perspectives, Pitfalls, and Paths Forward at ICML 2022 , 2022

    2022

  24. [33]

    Atj-net: Auto- table-join network for automatic learning on relational databases,

    J. Bai, J. Wang, Z. Li, D. Ding, J. Zhang, and J. Gao, “Atj-net: Auto- table-join network for automatic learning on relational databases,” in Proceedings of the Web Conference 2021 , 2021, pp. 1540–1551

    2021

  25. [34]

    Turl: Table understanding through representation learning,

    X. Deng, H. Sun, A. Lees, Y . Wu, and C. Yu, “Turl: Table understanding through representation learning,” ACM SIGMOD Record, vol. 51, no. 1, pp. 33–40, 2022

    2022

  26. [35]

    Teaching old db neu(ral) tricks: Learning embeddings on multi-tabular databases,

    G. Gaur, R. Singh, S. Arora, V . Gupta, and S. Bedathur, “Teaching old db neu(ral) tricks: Learning embeddings on multi-tabular databases,” in Proceedings of the 6th Joint International Conference on Data Science & Management of Data (10th ACM IKDD CODS and 28th COMAD) , ser. ...

    2023

  27. [36]

    Towards foundation models for relational databases [vision paper],

    L. V ogel, B. Hilprecht, and C. Binnig, “Towards foundation models for relational databases [vision paper],” arXiv preprint arXiv:2305.15321 , 2023

    2023 arXiv

  28. [37]

    Spare: A single-pass neural model for relational databases,

    B. Hilprecht, K. Kersting, and C. Binnig, “Spare: A single-pass neural model for relational databases,” arXiv preprint arXiv:2310.13581, 2023

    2023 arXiv

  29. [38]

    Position: Relational deep learning - graph representation learning on relational databases,

    M. Fey, W. Hu, K. Huang, J. E. Lenssen, R. Ranjan, J. Robinson, R. Ying, J. You, and J. Leskovec, “Position: Relational deep learning - graph representation learning on relational databases,” in Forty- first International Conference on Machine Learning , 2024. [Online]. Availa...

    2024

  30. [39]

    Beyond graphs: Learning with relational DBs,

    J. You and G. Liu, “Beyond graphs: Learning with relational DBs,” 2024. [Online]. Available: https://openreview.net/forum?id=ZQIgnzxMKJ

    2024

  31. [40]

    Gfs: Graph-based fea- ture synthesis for prediction over relational databases,

    H. Zhang, Q. Gan, D. Wipf, and W. Zhang, “Gfs: Graph-based fea- ture synthesis for prediction over relational databases,” arXiv preprint arXiv:2312.02037, 2023

    2023 arXiv

  32. [41]

    A deep learning blueprint for relational databases,

    L. Zahradn ´ık, J. Neumann, and G. ˇS´ır, “A deep learning blueprint for relational databases,” in NeurIPS 2023 Second Table Representation Learning Workshop, 2023

    2023

  33. [42]

    E. F. Codd, The relational model for database management: version 2 . Addison-Wesley Longman Publishing Co., Inc., 1990

    1990

  34. [43]

    Goodfellow, Y

    I. Goodfellow, Y . Bengio, and A. Courville, Deep learning. MIT press, 2016

    2016

  35. [44]

    Sentence-bert: Sentence embeddings using siamese bert-networks,

    N. Reimers and I. Gurevych, “Sentence-bert: Sentence embeddings using siamese bert-networks,” arXiv preprint arXiv:1908.10084 , 2019

    1908 arXiv

  36. [45]

    Pytorch frame: A modular framework for multi-modal tabular learning,

    W. Hu, Y . Yuan, Z. Zhang, A. Nitta, K. Cao, V . Kocijan, J. Leskovec, and M. Fey, “Pytorch frame: A modular framework for multi-modal tabular learning,” arXiv preprint arXiv:2404.00776 , 2024

    2024 arXiv

  37. [46]

    Inductive representation learning on large graphs,

    W. Hamilton, Z. Ying, and J. Leskovec, “Inductive representation learning on large graphs,” in Advances in neural information processing systems, 2017, pp. 1024–1034

    2017

  38. [47]

    Heterogeneous graph trans- former,

    Z. Hu, Y . Dong, K. Wang, and Y . Sun, “Heterogeneous graph trans- former,” 2020

    2020

  39. [48]

    Heterogeneous graph neural network,

    C. Zhang, D. Song, C. Huang, A. Swami, and N. V . Chawla, “Heterogeneous graph neural network,” in Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining , ser. KDD ’19. New York, NY , USA: Association for Computing Machinery, 2019, p....

    2019

  40. [49]

    Tabtransformer: Tab- ular data modeling using contextual embeddings,

    X. Hu, W. Tang, C.-K. Hsieh, and S. Shi, “Tabtransformer: Tab- ular data modeling using contextual embeddings,” arXiv preprint arXiv:2012.06678, 2020

    2012 arXiv

  41. [50]

    Tracr: Compiled transformers as a laboratory for interpretability,

    D. Lindner, J. Kram ´ar, M. Rahtz, T. McGrath, and V . Mikulik, “Tracr: Compiled transformers as a laboratory for interpretability,” arXiv preprint arXiv:2301.05062, 2023. TRANSFORMERS MEET RELATIONAL DATABASES - A PREPRINT 13

    2023 arXiv

  42. [51]

    The ctu prague relational learning repository,

    J. Motl and O. Schulte, “The ctu prague relational learning repository,” arXiv preprint arXiv:1511.03086 , 2015

    2015 arXiv

  43. [52]

    UCI Machine Learning Repository,

    D. Aha, “UCI Machine Learning Repository,” UCI Machine Learning Repository, 1987

    1987

  44. [53]

    Explain- ing queries over web tables to non-experts,

    J. Berant, D. Deutch, A. Globerson, T. Milo, and T. Wolfson, “Explain- ing queries over web tables to non-experts,” 2018

    2018

  45. [54]

    Mimic-iv (version 1.0),

    A. Johnson, L. Bulgarelli, T. Pollard, S. Horng, L. Celi, and R. Mark, “Mimic-iv (version 1.0),” 2020

    2020

  46. [55]

    SNAP Datasets: Stanford large network dataset collection,

    J. Leskovec and A. Krevl, “SNAP Datasets: Stanford large network dataset collection,” http://snap.stanford.edu/data, Jun. 2014

    2014

  47. [56]

    Open graph benchmark: Datasets for machine learning on graphs,

    W. Hu, M. Fey, M. Zitnik, Y . Dong, H. Ren, B. Liu, M. Catasta, and J. Leskovec, “Open graph benchmark: Datasets for machine learning on graphs,” 2021

    2021

  48. [57]

    Relbench: A benchmark for deep learning on relational databases,

    J. Robinson, R. Ranjan, W. Hu, K. Huang, J. Han, A. Dobles, M. Fey, J. E. Lenssen, Y . Yuan, Z. Zhang, X. He, and J. Leskovec, “Relbench: A benchmark for deep learning on relational databases,” in The Thirty-eight Conference on Neural Information Processing Systems Datasets an...

    2024

  49. [58]

    Wikidbs: A large-scale corpus of relational databases from wikidata,

    L. V ogel, J.-M. Bodensohn, and C. Binnig, “Wikidbs: A large-scale corpus of relational databases from wikidata,” in The Thirty-eight Conference on Neural Information Processing Systems Datasets and Benchmarks Track , 2024. [Online]. Available: https://openreview.net/ pdf?id=a...

    2024

  50. [59]

    Gradient-based boosting for statistical relational learning: The rela- tional dependency network case,

    S. Natarajan, T. Khot, K. Kersting, B. Gutmann, and J. Shavlik, “Gradient-based boosting for statistical relational learning: The rela- tional dependency network case,” Machine Learning, vol. 86, pp. 25–56, 2012

    2012

  51. [60]

    Xgboost: A scalable tree boosting system,

    T. Chen and C. Guestrin, “Xgboost: A scalable tree boosting system,” in Proceedings of the 22nd acm sigkdd international conference on knowledge discovery and data mining , 2016, pp. 785–794

    2016

  52. [61]

    Inductive representation learning on large graphs,

    W. L. Hamilton, R. Ying, and J. Leskovec, “Inductive representation learning on large graphs,” 2018

    2018

  53. [62]

    Layer normalization,

    J. L. Ba, J. R. Kiros, and G. E. Hinton, “Layer normalization,” 2016

    2016

  54. [63]

    Krogel, S

    M.-A. Krogel, S. Rawles, F. ˇZelezn´y, P. A. Flach, N. Lavra ˇc, and S. Wrobel, Comparative evaluation of approaches to propositionaliza- tion. Springer, 2003

    2003

  55. [64]

    Batch normalization: Accelerating deep network training by reducing internal covariate shift,

    S. Ioffe and C. Szegedy, “Batch normalization: Accelerating deep network training by reducing internal covariate shift,” 2015

    2015

  56. [65]

    Ray: A distributed framework for emerging ai applications,

    P. Moritz, R. Nishihara, S. Wang, A. Tumanov, R. Liaw, E. Liang, M. Elibol, Z. Yang, W. Paul, M. I. Jordan, and I. Stoica, “Ray: A distributed framework for emerging ai applications,” 2018

    2018

  57. [66]

    Optuna: A next-generation hyperparameter optimization framework,

    T. Akiba, S. Sano, T. Yanase, T. Ohta, and M. Koyama, “Optuna: A next-generation hyperparameter optimization framework,” 2019

    2019

  58. [67]

    Accelerat- ing the machine learning lifecycle with mlflow

    M. Zaharia, A. Chen, A. Davidson, A. Ghodsi, S. A. Hong, A. Konwin- ski, S. Murching, T. Nykodym, P. Ogilvie, M. Parkhe et al., “Accelerat- ing the machine learning lifecycle with mlflow.” IEEE Data Eng. Bull. , vol. 41, no. 4, pp. 39–45, 2018

    2018

  59. [68]

    Adam: A method for stochastic optimization,

    D. P. Kingma and J. Ba, “Adam: A method for stochastic optimization,” arXiv preprint arXiv:1412.6980 , 2014

    2014 arXiv

  60. [69]

    Excelformer: A neural network surpassing gbdts on tabular data,

    J. Chen, J. Yan, D. Z. Chen, and J. Wu, “Excelformer: A neural network surpassing gbdts on tabular data,” 2023

    2023

  61. [2023]

    Available: https://aclanthology.org/2023.tacl-1.14

    [Online]. Available: https://aclanthology.org/2023.tacl-1.14

    2023

Pith tools

Reviewed August 11, 2026 · model on record in the stance chip above.