REVIEW 3 major objections 5 minor 10 references
This paper claims that a general-purpose synthetic database generator can replace a massive purpose-built pretraining corpus for relational in-context learning, provided training begins with a fixed real-world schema rather than ending with
Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →
T0 review · deepseek-v4-flash
2026-08-03 13:09 UTC pith:F4WAAWFM
load-bearing objection Plausible external-generator pipeline and a curriculum order that looks real, but recovery percentages and the schema-first claim are both confounded by unmatched budgets and post-hoc selection. the 3 major comments →
PluRel-to-RDB-PFN: Schema-Guided Synthetic Relational Pretraining
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
Core claim
The paper's claim is that an external, schema-flexible synthetic generator can provide a useful pretraining signal for a relational in-context learner, and that the ordering of training stages determines whether that signal is usable. Its Schema-Guided First curriculum (real-world ERP schema with PluRel-generated content, then fully synthetic schemas of increasing diversity) reaches 0.6346 ROC-AUC at 1024-shot and 0.6116 at 64-shot across 19 RelBench/DBInfer tasks, using only about 33K tasks and no single-table warm-up. The authors frame this as recovering 87.6% and 93.8% of the published RDB-PFN performance at the two context sizes, and they observe that late introduction of the real-world
What carries the argument
The load-bearing object is the PluRel-to-RDB-PFN conversion pipeline: external binary task construction on top of PluRel databases, DFS linearization into fixed-width tables, and HDF5 assembly for the RDB-PFN transformer. The second mechanism is the curriculum itself—a staged training order (Schema-Guided First, Fully Synthetic, Schema-Guided Last) that isolates whether a fixed real-world schema anchor early in training stabilizes relational reasoning. The pipeline decouples generation from training, and the curriculum study is what converts a data-format exercise into a claim about learning order.
Load-bearing premise
The recovery percentages assume the published RDB-PFN scores (0.7245 and 0.6517) were produced under the same task definitions, data splits, feature preprocessing, and random-seed protocol used in this paper's evaluation; the paper reports the 55x scale difference but does not verify protocol equivalence.
What would settle it
Re-run the published RDB-PFN checkpoints on the same 19 tasks through this paper's exact DFS preprocessing, seed handling, and context sampling. If the reference averages fail to reproduce 0.7245 and 0.6517 under those conditions, the recovery ratios are comparison artifacts rather than measurements of relative capability.
If this is right
- External synthetic generators can serve as usable pretraining signals for relational in-context learners when curriculum order is designed appropriately.
- Starting with a real-world schema yields 2–3 points over fully synthetic training and 5–6 points over putting the schema last, suggesting that early schema anchoring is a generalizable training principle.
- A roughly 55x reduction in pretraining tasks (33K vs 1.8M) recovers 87.6–93.8% of published performance, implying that schema control can partially substitute for raw scale.
- The residual gap grows with context length, so original-style generators with learned content completion remain stronger for long-context relational structure.
- Data generation and model training for relational foundation models can be decoupled, enabling synthetic pretraining under user-controlled schemas without sharing real data.
Where Pith is reading between the lines
- A direct test would be to run the same curriculum study with several real-world schemas (e.g., from diverse enterprise domains) instead of one ERP schema; the paper's own hypothesis predicts additional gains, but it only tests a single anchor schema.
- A hybrid generator that keeps PluRel's schema and foreign-key control but replaces its SCM feature generation with the original model's learned content completion could isolate whether the remaining 1024-shot gap is caused by generator richness or by the 55x data-scale difference. The paper lists this as future work; it is my inference that the gap is at least partly architectural.
- Because the reference numbers are taken from a published protocol rather than re-run under this paper's exact evaluation pipeline, the recovery percentages should be read as upper-bound estimates until a scale-matched, pipeline-matched comparison is performed.
- The fact that context size changes the recovery ratio (93.8% at 64-shot vs 87.6% at 1024-shot) suggests that short-context relational reasoning is easier to synthesize than long-range statistical dependencies; testing even longer contexts (e.g., 2048-shot) would sharpen this boundary.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper proposes a pipeline that connects PluRel, a general-purpose synthetic relational database generator, to RDB-PFN, a relational in-context learner. PluRel cannot generate prediction tasks, so the pipeline adds an external binary-task-construction step, then applies RDB-PFN's DFS linearization and training format. The authors compare three curriculum strategies - schema-guided first, fully synthetic, and schema-guided last - using roughly 33K tasks and no single-table warm-up. They report that the schema-guided-first curriculum achieves 0.6346 average ROC-AUC across 19 real benchmark tasks at 1024-shot context, recovering 87.6% of the published RDB-PFN performance (0.7245), and 93.8% at 64-shot. The paper concludes that external synthetic generators can provide useful pretraining signals for RFMs and that early exposure to a real-world schema is substantially more effective than late adaptation.
Significance. If the results held as stated, the paper would make a useful contribution: it would show that data generation can be decoupled from the model training pipeline, that schema flexibility can partially compensate for training-corpus scale, and that curriculum ordering matters for relational pretraining. The use of public components (PluRel, RDB-PFN, RelBench) is a strength, and the 55x reduction in task count is an interesting empirical data point. However, the central quantitative claims currently rest on an unmatched comparison to published numbers and on curriculum comparisons that are not controlled for training budget or per-stage allocation. The paper also reports post-hoc selection of runs without variance information. These issues need to be addressed before the conclusions can be considered reliable.
major comments (3)
- [§4.1, Tables 2 and 3] The headline 'recovery' percentages (87.6% at 1024-shot, 93.8% at 64-shot) are computed by comparing the authors' ROC-AUC averages to 'Published RDB-PFN' values. The paper does not demonstrate that the two evaluations are protocol-equivalent: no evidence is given that the published reference used the same task preprocessing, data splits, DFS feature pipeline, context-window construction, evaluation seeds, or metric aggregation. Since the authors' own evaluation uses an external task-construction heuristic and a different data source, the reference numbers may reflect a different task setup. To make the recovery claim load-bearing, the authors should either re-run the original RDB-PFN model under their own evaluation protocol or verify in detail that the published numbers are directly comparable.
- [§3.2, Table 6] The comparison that supports the central 'schema-first is best' claim is not controlled. In Table 2, SCHEMA-GUIDED FIRST (SG→S1→S2→L1) is compared to FULLY SYNTHETIC (S1→S2), but these differ in total training steps (100K vs. 50K) and in the SG and Large pools. The only order-matched comparison, SGF vs. SGL, both at 100K steps, is not reported with per-stage step allocations. If SGF allocates more steps to the SG stage than SGL does, then order is conflated with training budget. The authors should equalize per-stage budgets across curricula, report the allocations, or restrict the order claim to a matched setting.
- [Appendix B.1, Tables 6 and 8-9] The statement 'When a run was repeated with a longer training schedule, we report the better average result and omit repeat identifiers for readability' is a post-hoc selection rule that inflates reported numbers. Combined with the absence of standard deviations or confidence intervals in the main and extended result tables (despite '10 random seeds' being claimed in §4.1), the 5-6 point advantage of SGF over SGL could be within run-to-run noise or the result of favorable schedule selection. Please report variance and all runs, or provide a principled selection criterion.
minor comments (5)
- [Table 4] Table 4 contains malformed entries (e.g., '30003,000' under S1, 'SHOULD BE Full' and 'SHOULD BE SALT' in SG rows) that obscure the corpus statistics. These should be corrected.
- [Table 6] The rows labeled 'SG→Fully Synthetic' correspond to 'SCHEMA-GUIDED FIRST' elsewhere, and the 'FULLY SYNTHETIC' main result uses S1→S2 while Table 6 also reports S1→S2→L1. Please clarify which configuration is the main comparison and unify the naming.
- [§3.1] The task construction says 'up to 6 candidate target columns' are selected per database, but the actual distribution of tasks per database is not reported. The total task count (~33K) is consistent with ~5.5 tasks per database, but a sentence on this would help.
- [Appendix A] The sentence 'The D Large1 1000 statistics appeared twice in the raw logs with identical values; we include them once' is unclear and grammatically awkward. Please rephrase and clarify what was deduplicated.
- [§5 Limitations] The limitations paragraph acknowledges the scale mismatch but not the potential protocol mismatch with the published RDB-PFN reference. This should be stated explicitly given the central comparison.
Circularity Check
No significant circularity: pretraining is externally generated, test tasks are real-world, and no prediction reduces to fitted inputs.
full rationale
The paper's central claims are empirically tested rather than derived from their own inputs. PluRel-generated databases are produced by an external generator (Kothapalli et al., 2026) and are not constructed from the 19 RelBench/DBInfer evaluation tasks; the schema-guided corpus uses the rel-salt enterprise schema, which is not among the test tasks, so there is no label or schema leakage from the evaluation set. The RDB-PFN architecture and DFS preprocessing are reused as fixed infrastructure, not fitted to test outcomes. The comparison against 'Published RDB-PFN' references (Table 2) is a benchmark comparison, not a derivation; the paper explicitly acknowledges in Limitations that the comparison is not scale-matched and that the reference numbers come from the original protocol, which is a comparability caveat rather than a circular step. The only notable issue is Appendix B.1's statement that 'When a run was repeated with a longer training schedule, we report the better average result and omit repeat identifiers for readability,' which is post-hoc selection of reported runs and a potential evaluation-integrity concern, but it does not make the central result true by construction. Similarly, the SGF vs. FS comparison in Table 6 confounds training budget with corpus composition (FS is 50K steps without the Large pool), and per-stage step allocations for matched-budget curricula are not reported; these are experimental confounds that could affect the curriculum-order conclusion, but they are not circularity. No parameter is fitted to the evaluation labels, no uniqueness theorem is imported from the authors' own prior work, and no load-bearing premise is justified solely by a self-citation. The derivation chain is self-contained: synthetic pretraining data → fixed preprocessing → evaluated on held-out real benchmarks.
Axiom & Free-Parameter Ledger
free parameters (3)
- Number of tasks per database (up to 6) =
6
- Curriculum pool sizes =
500 SG, 3000 Small1, 1000 Small2, 1000 Large
- DFS hop configurations per pool =
Small1: 1-hop, Small2: 2-hop, Large: 1-hop
axioms (4)
- domain assumption PluRel-generated databases with rank-based normalization approximate real relational data distributions sufficiently for pretraining.
- domain assumption The fixed rel-salt schema is representative of real-world enterprise schemas.
- ad hoc to paper The external binary task construction (column selection, median splits) produces tasks that resemble the real benchmark tasks.
- domain assumption RDB-PFN's DFS preprocessing and architecture operate on PluRel output in the same way as on its native generator's output.
read the original abstract
Relational Foundation Models (RFMs) require large-scale synthetic relational databases for pretraining, but existing approaches tightly couple data generation with the model training pipeline. We study whether PluRel, a general-purpose synthetic relational database generator, can serve as an external data source for RDB-PFN, a relational in-context learner originally pretrained with a 600K-task single-table warm-up followed by an approximately 1.8M-task adaptation stage. We build a conversion pipeline that maps PluRel-generated databases, including externally constructed binary prediction tasks, into the RDB-PFN training format and evaluate three curriculum strategies: SCHEMA-GUIDED FIRST (real-world schema then fully synthetic), FULLY SYNTHETIC (diverse synthetic schemas throughout), and SCHEMA-GUIDED LAST (fully synthetic then real-world schema). Using only approximately 5,500 relational databases (approximately 33K tasks), roughly 55x fewer tasks than the original protocol, and no single-table warm-up, our best curriculum (SCHEMA-GUIDED FIRST) achieves 0.6346 average ROC-AUC across 19 real benchmark tasks at 1024-shot context, recovering 87.6% of the published RDB-PFN performance (0.7245). At 64-shot context, the gap narrows to 93.8% (0.6116 vs. 0.6517). Our results demonstrate that external synthetic generators can provide useful pretraining signals for RFMs when combined with appropriate curriculum design and that exposure to a real-world schema early in training is substantially more effective than late-stage schema adaptation.
Reference graph
Works this paper leans on
-
[1]
arXiv preprint arXiv:2603.03805 , year=
Relational In-Context Learning via Synthetic Pre-training with Structural Prior , author=. arXiv preprint arXiv:2603.03805 , year=
-
[2]
arXiv preprint arXiv:2602.04029 , year=
PluRel: Synthetic Data unlocks Scaling Laws for Relational Foundation Models , author=. arXiv preprint arXiv:2602.04029 , year=
-
[3]
Advances in Neural Information Processing Systems , volume=
Relbench: A benchmark for deep learning on relational databases , author=. Advances in Neural Information Processing Systems , volume=
-
[4]
arXiv preprint arXiv:2207.01848 , year=
Tabpfn: A transformer that solves small tabular classification problems in a second , author=. arXiv preprint arXiv:2207.01848 , year=
-
[5]
arXiv preprint arXiv:2505.05568 , year=
Griffin: Towards a graph-centric relational database foundation model , author=. arXiv preprint arXiv:2505.05568 , year=
-
[6]
arXiv preprint arXiv:2509.21489 , year=
GraphPFN: A prior-data fitted graph foundation model , author=. arXiv preprint arXiv:2509.21489 , year=
-
[7]
Advances in Neural Information Processing Systems , volume=
Clavaddpm: Multi-relational data synthesis with cluster-guided diffusion models , author=. Advances in Neural Information Processing Systems , volume=
-
[8]
arXiv preprint arXiv:2404.18209 , year=
4dbinfer: A 4d benchmarking toolbox for graph-centric predictive modeling on relational dbs , author=. arXiv preprint arXiv:2404.18209 , year=
-
[9]
arXiv preprint arXiv:2506.00710 , year=
RelDiff: Relational Data Generative Modeling with Graph-Based Diffusion Models , author=. arXiv preprint arXiv:2506.00710 , year=
-
[10]
Proceedings of the 26th annual international conference on machine learning , pages=
Curriculum learning , author=. Proceedings of the 26th annual international conference on machine learning , pages=
discussion (0)
Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.