Parameter-Free Encoders Remain Viable for RDB Foundation Models

David Wipf; Linjie Xu

Even when neighborhood labels are fed to the encoder, trainable parameters bring no general advantage over simple parameter-free RDB encoders for foundation models.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · grok-4.5

2026-07-11 11:46 UTC pith:3I2M3PY7

load-bearing objection Clean theory + strong open baseline showing parameter-free RDB encoders stay competitive even when labels are available as inputs. the 2 major comments →

arxiv 2607.05476 v2 pith:3I2M3PY7 submitted 2026-07-06 cs.LG

Parameter-Free Encoders Remain Viable for RDB Foundation Models

Linjie Xu , David Wipf This is my paper

classification cs.LG

keywords relational databasefoundation modelparameter-free encoderin-context learningsubgraph encoderneighborhood labelshomophily

verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

Relational databases contain countless possible prediction targets, so enterprise settings need frozen foundation models that work on new tasks without retraining. Recent work has split over whether the RDB encoder itself needs trainable parameters that exploit observable neighborhood labels, or whether a fixed, parameter-free subgraph encoder plus a strong single-table foundation model already suffices. This paper proves that, once labels appear as encoder inputs, trainable parameters cannot reliably turn those labels into either stable discriminative features or reliable feature-importance signals that transfer across tasks. Empirically, a deliberately simple parameter-free encoder still matches or beats both open-source and most closed-source foundation models across six diverse RDB benchmarks, without any RDB-specific pre-training.

Core claim

When labels are supplied as encoder inputs, there always exist distributions over the unobserved relational context such that any fixed trainable encoder plus prediction head performs no better than chance on the test label, while a parameter-free encoder plus a shallow ReLU head can still fit any finite supervised sample drawn from the same distribution; likewise, local labels supply no information about which feature columns are actually informative. Consequently parameter-free encoders remain competitive for RDB foundation models.

What carries the argument

Propositions 3.1 and 4.1, which construct adversarial parity-style labelings over partially observed neighborhoods to show that neither the discriminative role nor the feature-selection role of neighborhood labels can be stably exploited by any fixed set of encoder parameters.

Load-bearing premise

The paper's impossibility claims rest on the idea that the adversarial parity-style label distributions it constructs are realistic enough to constrain how encoders should be designed for actual enterprise databases.

What would settle it

Construct a large suite of real-world RDB tasks whose label-generation processes are known to be strongly heterophilic or parity-like; if a carefully tuned parameterized encoder that receives neighborhood labels then systematically outperforms every parameter-free baseline by a clear margin, the claimed practical limitation of trainable parameters would be refuted.

Watch this falsifier — get emailed when new claim-graph text bears on it.

If this is right

RDB foundation-model pipelines can continue to pair simple fixed aggregators with frontier single-table models and still expect near-SOTA accuracy on new tasks.
Pre-training effort that is spent only on learning encoder parameters to exploit local labels is unlikely to yield robust gains that transfer across arbitrary RDBs.
In-context-learning heads already have enough flexibility to re-weight diverse label aggregations, so the encoder itself need not be made task-adaptive.
Benchmark rankings that currently favor closed-source models with parameterized encoders may be explained by homophily or other non-transferable regularities rather than by the trainable parameters themselves.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

If the dominant failure mode of fixed encoders is merely missing a few useful aggregators, then expanding the fixed aggregator catalogue (quantiles, entropy, etc.) may close most remaining gaps without ever introducing trainable weights.
The same partial-observability argument used for labels may also limit the value of trainable parameters that try to fuse multi-hop text embeddings or other high-cardinality side information.
Future RDB benchmarks that deliberately inject controlled heterophily would give a sharper test of whether parameterized encoders ever become necessary.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Desk Editor's Note

Clean theory + strong open baseline showing parameter-free RDB encoders stay competitive even when labels are available as inputs.

read the letter

The punchline is simple: even when neighborhood labels are fed to the encoder, a fixed (parameter-free) design still has limited ability to exploit them across tasks, and the authors’ updated RDBLearn pipeline is already competitive with—or better than—most open- and closed-source RDB foundation models on six public suites.

What is new are the two existence results (Props. 3.1 and 4.1). They cleanly separate the foundation-model regime (encoder must stay fixed) from ordinary supervised learning, using standard parity constructions over partially observed neighborhoods. The proofs in Appendix D are explicit and correctly scoped; they do not claim every real RDB is adversarial, only that no fixed encoder can guarantee good use of labels across arbitrary tasks. That is the right claim for the setting. Empirically they ship an open, zero-RDB-pretraining pipeline that tops the open-source rankings and beats most closed-source numbers (Fig. 1 + Tables 1–6). Running everything from a public repo with no task-specific tuning is real evidence, not marketing.

Soft spots are minor and already flagged by the paper itself. The parity counter-examples may overstate the gap if enterprise RDBs are overwhelmingly homophilic (they note the near-parity of entity-mean vs. RT). Public RDB benchmarks remain scarce, closed-source hyper-parameters are opaque, and RDBLearn still ignores text columns and does not yet feed labels into its own encoder. None of these break the central argument.

This is for people building or choosing RDB foundation models who need a concrete design principle rather than another leaderboard entry. The math is solid, the data are reproducible, and the citation pattern is fair. I would send it to peer review without hesitation; a workshop is the right venue and the contribution is already useful.

Referee Report

2 major / 4 minor

Summary. The paper argues that, even when neighborhood labels are supplied as encoder inputs, trainable RDB encoder parameters have limited general efficacy for frozen foundation models, so simpler parameter-free subgraph encoders remain viable. It proves two existence results: (Prop. 3.1) there exist distributions over unobserved RDB context such that y_test is a deterministic function of the full labeled graph yet any fixed encoder+head is no better than chance, while a per-task supervised ReLU head on a parameter-free map can still fit; (Prop. 4.1) neighborhood labels need not identify which feature-column support set generates the labels. Empirically, an updated open-source RDBLearn pipeline (parameter-free encoder + single-table FMs, no RDB pre-training) matches or exceeds open-source FMs and most closed-source ones across six public benchmarks (SALT, RelBench v1/v2, 4DBInfer).

Significance. The result is timely: several concurrent RDB foundation models (KumoRFM-2, RT, OpenRFM, etc.) emphasize parameterized encoders that ingest neighborhood labels, while Xu et al. (2026) and RDBLearn advocate parameter-free designs. The paper supplies explicit, self-contained existence proofs (Appendix D) that cleanly separate the fixed-encoder FM regime from the per-task supervised regime, and pairs them with reproducible public-code experiments covering six diverse suites and both open- and closed-source comparators. If the claims hold, they lower the bar for practical RDB FMs (no RDB-specific pre-training required) and clarify when label-aware encoders can and cannot help. Strengths include machine-checkable proof structure, open RDBLearn v1.1 release, and transparent ranking tables.

major comments (2)

Props. 3.1 and 4.1 are correctly scoped as existence results over adversarial (parity) distributions; the paper itself notes that real RDBs may be homophily-dominated (discussion after Prop. 3.1, citing entity-mean vs. RT). The central claim therefore rests on the empirical half. Tables 5–6 show that on RelBench-v1 the closed-source TabPFN-REL and (original) KumoRFM-2 still lead RDBLearn on several tasks. The manuscript should either (i) quantify how much of that gap is attributable to label-aware encoder parameters versus other undisclosed components (text fields, ensembling, ICL sampling), or (ii) explicitly bound the claim to “parameter-free encoders remain competitive / best open-source” rather than “remain viable against the best closed-source industry architectures.” Without that clarification the strongest reading of the abstract is only partially supported by the tables.
Appendix A states that RDBLearn v1.1 still does not feed neighborhood labels into its parameter-free encoder, even though Sections 3–4 analyze exactly that setting and note that labels “may well etaεnefit parameter-free encoder models.” A controlled ablation that adds the same label aggregations used by KumoRFM-2 / RT to the RDBLearn encoder (still parameter-free) would directly test whether the theory’s “no unequivocal advantage for trainable parameters” holds once both sides receive labels. Its absence leaves a load-bearing empirical gap between the propositions and the reported rankings.

minor comments (4)

Figure 1 normalizes average rank to [0,1] but the caption does not state the exact ranking procedure (ties, missing entries). Tables 3 and 5 already exclude incomplete methods from averages; the figure should cite the same rule.
Notation: G*_H(x) vs. eG*_H(xtest) is introduced cleanly in §3, but the switch between G*_test and G*_H in the KL statements of Prop. 3.1 could be restated once in the main text for readers who skip the appendix.
Appendix B.3 correctly flags LLM-agent leakage risk on public RDB benchmarks; a one-sentence reminder in the main-text experimental section would help non-specialist readers interpret RelAgent numbers.
Typos / polish: “Intuition behind of Proposition 3.1”; “w.r.t. downstream performance” is fine but a few sentences in §4 are long; “βεnefit” encoding artifact in the source.

Circularity Check

0 steps flagged

No significant circularity: Props. 3.1/4.1 are self-contained existence constructions; empirics evaluate an open parameter-free encoder on public held-out benchmarks without fitted-as-prediction steps.

full rationale

The load-bearing theoretical claims are Propositions 3.1 and 4.1. Both are pure existence results: the authors construct adversarial joint distributions (via a latent parity selector z and carefully placed extra nodes/features outside the H-hop ego-network) under which any fixed encoder g_ϕ plus head q_θ cannot beat chance (Prop. 3.1) or cannot recover the informative support set S from neighborhood labels (Prop. 4.1). The constructions and the KL/uniform lower-bound arguments appear fully in Appendix D and do not invoke fitted parameters, uniqueness theorems, or prior results of the same authors as premises. The only external citation inside the proofs (Daniely & Malach 2020) is used solely for an optional learnability remark and is not required for the impossibility statements. Empirically, RDBLearn-v1.1 is an open, parameter-free encoder (vertical aggregations + frozen single-table FM) evaluated zero-shot on six public suites (RelBench v1/v2, 4DBInfer, SALT) with no RDB pre-training and no task-specific hyper-parameter search beyond a fixed default; performance numbers are therefore external measurements, not re-statements of training objectives. Self-citations to Xu et al. (2026) and Zhang et al. (2026) merely locate the encoder architecture that is being stress-tested; they do not supply the impossibility proofs or the ranking tables. Consequently none of the six circularity patterns is present, and the derivation chain does not reduce to its own inputs by construction.

Axiom & Free-Parameter Ledger

3 free parameters · 3 axioms · 0 invented entities

The paper rests on standard relational-database and graph-neural-network background plus two carefully constructed adversarial distributions used only for existence proofs. No continuous free parameters are fitted to obtain the theoretical claims; a handful of discrete design choices appear only in the empirical pipeline.

free parameters (3)

encoder hop depth H = 4
Default H=4 chosen for the no-validation-set setting; affects which neighborhood labels and features are visible.
aggregation function set = expanded set (default)
Choice between original (mean/min/max/std/count/median) and expanded (plus 25/75 quantiles, entropy) sets is selected by validation performance when available.
base tabular FM = TabPFN-v2.5
Default TabPFN-v2.5 (or TabICL-v2 / TabPFN-v3 when validation exists); the paper treats the choice as a free plug-in.

axioms (3)

domain assumption An RDB can be losslessly converted to a heterogeneous graph via PK-FK edges and H-hop ego-networks can be sampled without temporal leakage.
Standard construction used throughout Sections 2–4 and all experiments; taken from the relational deep-learning literature.
ad hoc to paper Existence of distributions over the unobserved RDB context eG*_H such that labels are deterministic parity functions of partially overlapping neighborhoods.
Core technical device of Props. 3.1 and 4.1 (Appendix D); used only to prove existence of failure cases for fixed encoders.
standard math A ReLU network of finite width and depth can realize the parity function on a scalar sum of binary labels.
Invoked in part 2 of Prop. 3.1; classical universal-approximation fact.

pith-pipeline@v1.1.0-grok45 · 23708 in / 2669 out tokens · 25109 ms · 2026-07-11T11:46:28.751376+00:00 · methodology

0 comments

read the original abstract

Given a relational database (RDB) storing heterogeneous tabular information, how can we predict missing (or future) values in some target column of interest? As the space of potential targets is vast across enterprise settings, it is preferable to avoid learning a new model from scratch each time there is a new prediction task. Frozen foundation models based on RDB-specific encoders provide a viable solution, but ideal design remains an open question. On the one hand, it has recently been argued that certain parameter-free subgraph encoders combined with single-table foundation models can achieve near SOTA performance, with no RDB-specific pre-training required. Meanwhile, other contemporary studies advocate for parameterized encoders pre-trained to exploit observable labels for learning task-specific representations. To address this ambiguity, we analyze RDB encoder properties specifically when labels are present as inputs, proving limitations on the potential efficacy of trainable encoder parameters. As empirical validation, we demonstrate that considerably simpler parameter-free encoders are still capable of strong performance across many relevant benchmarking tasks.

Figures

Figures reproduced from arXiv: 2607.05476 by David Wipf, Linjie Xu.

Parameter-Free Encoders Remain Viable for RDB Foundation Models

Core claim

What carries the argument

Load-bearing premise

What would settle it

If this is right

Where Pith is reading between the lines

discussion (0)