Pith. sign in

REVIEW 4 major objections 3 minor 13 references

The paper argues that curriculum ordering — not the synthetic-data generator or its scale — is the dominant training lever for relational Prior-Data Fitted Networks.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · deepseek-v4-flash

2026-08-03 13:19 UTC pith:VHPLR6PL

load-bearing objection Striking curriculum-vs-all-at-once gap, but single-run numbers with no seeds or code leave the central claim provisional; worth a serious referee if the authors add variance. the 4 major comments →

arxiv 2607.29120 v1 pith:VHPLR6PL submitted 2026-07-31 cs.LG cs.DB

Curriculum Matters: Data-Efficient Relational PFN Pretraining with Synthetic Data

Mohammad Sadeq Abolhasani , Viswanath Ganapathy This is my paper
classification cs.LG cs.DB
keywords Prior-Data Fitted Networkscurriculum learningsynthetic data generationrelational learningDeep Feature Synthesisin-context learningtabular benchmarksdata efficiency
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

Prior-Data Fitted Networks (PFNs) are small transformers that learn to solve tabular or relational tasks by training on many synthetic examples. This paper claims that the biggest factor in how well such a model performs is the order in which those examples are shown, not which generator produced them or how many there are. Using a single lightweight synthetic-database generator, the authors show a width-progressive curriculum over ~13,300 single tables reaches 0.703 average ROC-AUC on 23 tabular tasks, while the same data shuffled and presented at once collapses to 0.541. A relational curriculum from scratch, using ~5,500 databases, recovers 88% of a much larger published model's reported performance, and a model trained only on single tables nearly matches a dedicated relational pipeline when tested on relational tasks. If true, this redirects relational foundation-model research toward curriculum design and single-table diversity rather than elaborate relational generators.

Core claim

On the paper's own terms, the central discovery is that curriculum ordering is a first-order training-recipe variable for relational PFN pretraining: an identical corpus of PluRel-generated synthetic tables yields 0.703 average ROC-AUC when presented through a width-progressive curriculum (7 to 17 columns) but only 0.541 when trained all-at-once, a 16-point gap. The paper further finds that a structurally orthogonal generator (PluRel) can substitute for RDB-PFN's native neural generator, recovering 88% of its reported relational performance with about 220x less relational synthetic data, and that a single-table-only PFN transfers to relational tasks almost as well as a dedicated relational p

What carries the argument

The carrying mechanism is the curriculum schedule itself: a staged, width-progressive ordering of synthetic single tables (TF07 through TF17), with each stage continuing from the previous checkpoint. The other load-bearing piece is Deep Feature Synthesis (DFS) linearization, which converts a multi-table relational database into a fixed-width feature matrix, effectively reducing relational tasks to structured tabular tasks; this is what allows a single-table curriculum to transfer to relational benchmarks.

Load-bearing premise

The paper reports no seed variance or significance tests, so its comparative claims assume that each configuration's average ROC-AUC is stable enough to rank them; if run-to-run variation is on the order of a few points, the precise ordering and 'primary lever' interpretation would not be supported.

What would settle it

Re-run Family A (curriculum) and Family B (all-at-once) over multiple random seeds with identical data and compute; if the average gap between them is not consistently close to the reported 16 points (e.g., shrinks below a few points across seeds), the central claim that curriculum ordering is the primary lever would be falsified.

Watch this falsifier. Get emailed when new claim-graph text bears on it.

If this is right

  • Relational PFN pretraining can be made dramatically more data-efficient: roughly 45x fewer single-table datasets and 220x less relational data suffice when the data is ordered as a curriculum.
  • A structurally different synthetic generator can replace a heavily engineered one with limited loss, implying generator identity is less important than the distributional properties of the synthetic data.
  • A single-table-only curriculum can nearly match a dedicated relational pipeline, suggesting the marginal value of explicit relational synthetic data is smaller than the two-stage recipe implies.
  • The single-table warm-up is the more critical stage: ablating it costs more than ablating the relational curriculum ordering.
  • Future effort should shift toward single-table diversity, curriculum design, and backbone capacity rather than elaborate relational generators.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • A testable extension: design curricula around DFS feature statistics (e.g., block-diagonal correlation patterns) rather than raw schema width; the paper's per-task analysis hints that task-structure-aware ordering could outperform width-progression.
  • If the transfer result is robust, one could investigate whether a pure tabular pretraining corpus (no relational structure at any stage) fully closes the gap, which would further undermine the need for relational generators.
  • The all-at-once collapse suggests PFN optimization is highly sensitive to distribution shift; a possible connection to curriculum learning in other in-context learning settings, where stage-wise exposure mitigates catastrophic interference, could be explored.
  • A multi-seed reproduction study on the close comparisons (0.631 vs 0.638) would determine whether the single-table model truly matches the relational pipeline, a question the paper leaves open.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

4 major / 3 minor

Summary. The paper investigates three questions about relational Prior-Data Fitted Networks (PFNs) trained on synthetic data: whether a structurally different generator (PluRel) can substitute for RDB-PFN's native generator, whether the order of synthetic task presentation (curriculum) affects downstream quality, and how much relational ability emerges from single-table pretraining alone. Using PluRel as the sole generator, the authors report that a width-progressive single-table curriculum (7 to 17 columns, ~13,300 tables) reaches 0.703 average ROC-AUC on 23 tabular tasks, while the same data trained all-at-once collapses to 0.541. A relational curriculum from scratch on ~5,500 PluRel databases reaches 0.638 on 19 relational tasks, recovering 88% of RDB-PFN's reported 0.725, and the single-table curriculum model transfers to 0.631 on the relational benchmark. The paper concludes that curriculum design is the primary lever for relational PFN pretraining.

Significance. If the findings are substantiated, they would substantially reframe relational PFN pretraining: synthetic generator identity and raw data scale may matter less than training trajectory, and data-efficient curricula could replace million-scale pretraining. The paper's strengths include full per-task tables in Appendix A and Section B showing that the headline averages are exactly recoverable from those tables, which is a useful transparency practice. The use of an external generator (PluRel) and external baselines is also appropriate for the substitution question. However, the central comparative claims rest entirely on single-run point estimates with no seed variation or uncertainty quantification, which is a serious gap given that several ordering conclusions are based on differences of less than 2 points.

major comments (4)
  1. [Section 4.1, Table 1] No seed count, variance, or significance test is reported for any configuration. The headline 16.2-point gap between Family A (0.703) and Family B (0.541) is large, but Appendix A.2 shows task-level collapses of up to 48 points (MiniBooNE .362 vs .841, house_16H .397 vs .821). Such collapses are consistent with divergent training runs or initialization effects rather than a systematic effect of curriculum ordering. Because the paper's central claim ('curriculum is the primary lever') is entirely comparative, the authors must report multiple seeds, mean ± standard deviation, and ideally paired comparisons or a significance test across seeds. Without this, the central claim is not verifiable from the reported evidence.
  2. [Section 4.1, Section 4.4] The assertion of 'identical data, identical optimizer, identical compute budget' is not backed by concrete reporting. No number of optimizer steps, epoch count, learning-rate schedule, or early-stopping criterion is given for the curriculum and all-at-once runs. If the all-at-once run used an unmatched training budget or a suboptimal schedule, the 16-point gap would be a training artifact rather than evidence about curriculum. Please report the exact training configuration for each family, including total steps and LR schedule, and ideally verify that both runs reach comparable training loss saturation.
  3. [Section 4.2, Section 4.4, Tables 2 and 4] Several secondary comparisons used to rank curricula are small and may reverse with plausible run-to-run variance. Family C vs E (0.638 vs 0.620, 1.8 points), Family D vs C (0.631 vs 0.638, 0.7 points), and Family F vs C (0.596 vs 0.638) are all within a range that could be noise for a 0.7M-parameter model trained on synthetic data. The claim that the warm-up is 'the more critical stage' (Section 4.4) rests on a 3.5-point difference that is not statistically supported. The authors should either provide seed variance for these comparisons or temper the ordering claims accordingly.
  4. [Section 3, Family G] There are two conflicting definitions of Family G in the same paragraph. The first says 'Relational-only, no single tables at all. The full relational corpus was mixed at once; there was no single-table data at any stage' — which is indistinguishable from Family F. The second says 'Relational curriculum without Rel-SALT... only PluRel-generated databases are used, with no single-table warm-up.' Table 4 reports 'Family G: no single tables 0.624' but it is unclear which definition this corresponds to. This ambiguity affects the reproducibility of the ablations and should be corrected in a revision.
minor comments (3)
  1. [Abstract/Introduction] The abstract states 'approximately 45x fewer single-table datasets' while the introduction also says 'roughly 270x less total tabular content'; these are different metrics but the relationship is not explained. Please clarify whether the 45x refers to dataset count and 270x to total rows/features, and ensure the numbers are consistent.
  2. [Section 4.1, Table 1] The row labeled 'RDB-PFN (paper) ∼600,000 0.800' is a published baseline, not a run in this paper. It would be helpful to note explicitly that these numbers are taken from the original paper and have not been re-run, to avoid implying a matched comparison.
  3. [Section B] The verification of headline averages is a good transparency practice, but it only confirms arithmetic consistency. It would be stronger to include the per-task standard deviations across seeds, or at least a statement about the number of runs, in the same appendix.

Circularity Check

0 steps flagged

No significant circularity: the central claims rest on controlled pretraining experiments against external benchmarks and external generators, with no fitted-input-as-prediction or self-citation chain.

full rationale

The paper's derivation chain is experimental rather than definitional or theorem-driven. Q1 (generator substitutability) is tested by replacing RDB-PFN's native generator with the external PluRel generator while keeping the external RDB-PFN consumer architecture and DFS linearization fixed; Q2 (curriculum ordering) is tested by comparing the same PluRel corpus presented in a progressive curriculum versus all-at-once; Q3 (single-table transfer) is tested by evaluating a single-table checkpoint directly on the relational benchmark. None of these comparisons reduces by construction to an input: the curriculum schedule is not fitted to the benchmark targets, the performance numbers are measured on external benchmarks (23 Grinsztajn tasks, 19 RelBench/4DBInfer tasks), and the baselines cited (PluRel, RDB-PFN, RelBench, 4DBInfer) are from non-overlapping author groups and released codebases, so no load-bearing self-citation chain exists. The claim that 'curriculum ordering, not synthetic generator identity and not raw synthetic scale, is the dominant lever' is an interpretation of directly measured point estimates, not a renaming of an assumed result. The main weaknesses flagged by the manuscript itself are reproducibility-related rather than circular: Section 4 and Appendix A report single-run point estimates with no seed count, variance, or significance test, and Section 3 contains two conflicting definitions of 'Family G' (one described as relational-only all-at-once and one described as a relational curriculum without Rel-SALT). These affect the evidentiary strength of the comparative claims but do not amount to a circular derivation. Accordingly, the circularity score is 0.

Axiom & Free-Parameter Ledger

5 free parameters · 4 axioms · 0 invented entities

No invented entities are introduced. The central claim rests on hand-chosen curriculum schedules and on inherited DFS/architecture choices; these are inputs from prior work or chosen by the authors, not fitted to benchmark targets. The most consequential free parameter is the curriculum schedule itself, which is the paper's treatment variable.

free parameters (5)
  • Single-table curriculum width schedule and per-stage counts = TF07: 3,300; TF08-TF11: 1,000 each; TF12: 2,000; TF13-TF17: 1,000 each (cumulative ~13,300)
    Hand-chosen schedule; no sensitivity analysis or search is reported. The paper's main result depends on this exact ordering and these counts.
  • Number of rows per synthetic table = 150
    Fixed without ablation; row count interacts with in-context learning and could affect the curriculum result.
  • Post-DFS feature width = 30 columns via random feature subsampling
    Standardized feature width chosen by hand; if benchmark results are sensitive to this, the single-table/relational comparisons may not transfer to other widths.
  • Relational curriculum corpus split = Rel-SALT 500, PluRelSmall1 3,000, PluRelSmall2 1,000, PluRelLarge1 1,000
    Stage sizes are hand-picked; no ablations on stage composition other than all-at-once comparisons.
  • Number of target columns per schema = 6 random target columns
    Inherited from RDB-PFN protocol; a modeling choice about task density, not justified here.
axioms (4)
  • domain assumption DFS linearization with mean/max/min/count/mode aggregations is sufficient to represent relational databases for binary classification.
    The paper inherits RDB-PFN's DFS pipeline without re-validation and bases the single-table transfer result on it. Invoked in Section 2.1 and Section 3 'Linearization and targets'.
  • domain assumption RDB-PFN's published baseline numbers (0.800 single-table, 0.725 relational) were obtained under the same evaluation protocol and are directly comparable.
    The 88% recovery claims and the 'dominant lever' conclusion are computed against those published numbers; no re-run of RDB-PFN is performed. See Sections 4.1, 4.2, and Appendix B.
  • domain assumption PluRel-generated data is sufficiently diverse and realistic to transfer zero-shot to RelBench/4DBInfer tasks.
    PluRel is the sole generator; no distributional closeness analysis is provided. See Sections 2.2 and 3.
  • domain assumption The 0.7M-parameter consumer model and its training hyperparameters transfer unchanged from RDB-PFN's recipe to PluRel data.
    All non-curriculum hyperparameters 'follow the original paper'; if PluRel data needs different optimization settings, the curriculum-vs-all-at-once comparison could be confounded. See Section 3.

pith-pipeline@v1.3.0-daily-deepseek · 10698 in / 15096 out tokens · 141732 ms · 2026-08-03T13:19:22.265037+00:00 · methodology

0 comments
read the original abstract

Relational Prior-Data Fitted Networks (PFNs) such as RDB-PFN approximate Bayesian inference over multi-table relational databases by pretraining on millions of synthetic tasks. We investigate three intertwined questions about this paradigm. First, can a structurally different synthetic generator PluRel substitute for RDB-PFN's prior? Second, how much does the order in which synthetic data is presented to the PFN affect downstream performance? Third, how much relational reasoning can a PFN acquire from single-table synthetic pretraining alone, before any relational data is introduced? Using PluRel as the sole synthetic data source across all experiments, we find: (i) a progressive single-table curriculum that gradually widens schema complexity from 7 to 17 columns reaches 0.703 average ROC-AUC on the 23-task tabular benchmark using only approximately 13,300 synthetic tables (approximately 45x fewer single-table datasets than RDB-PFN's reported warm-up recipe), while the same data trained all-at-once collapses to 0.541 ROC-AUC; (ii) a relational curriculum trained from scratch on only approximately 5,500 PluRel databases reaches 0.638 average ROC-AUC on the 19-task RelBench/4DBInfer benchmark, recovering 88% of RDB-PFN's reported performance with approximately 220x less relational synthetic data; and (iii) the single-table curriculum model, evaluated directly on the relational benchmark without any relational adaptation, achieves 0.631, nearly matching the dedicated relational pipeline. Together, these findings suggest that curriculum design and synthetic data diversity may matter more for relational PFN pretraining than the specific relational generator or raw synthetic scale alone.

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

13 extracted references · 1 linked inside Pith

  1. [1]

    Léo Grinsztajn, Edouard Oyallon, and Gaël Varoquaux. 2022. Why do tree-based models still outperform deep learning on typical tabular data?Advances in Neural Information Processing Systems35 (2022), 507-520

    2022

  2. [2]

    Noah Hollmann, Samuel Müller, Katharina Eggensperger, and Frank Hutter. 2023. TabPFN: A Transformer that Solves Small Tabular Classification Problems in a Second. InThe Eleventh International Conference on Learning Representations

    2023

  3. [3]

    Noah Hollmann, Samuel Müller, Lennart Purucker, Arjun Krishnakumar, Max Körfer, Shi Bin Hoo, Robin Tibor Schirrmeister, and Frank Hutter. 2025. Accurate predictions on small data with a tabular foundation model.Nature637, 8045 (2025), 319-326

    2025

  4. [4]

    James Max Kanter and Kalyan Veeramachaneni. 2015. Deep Feature Synthe- sis: Towards Automating Data Science Endeavors. In2015 IEEE International Conference on Data Science and Advanced Analytics (DSAA). IEEE, 1-10

    2015

  5. [5]

    Vignesh Kothapalli, Rishabh Ranjan, Valter Hudovernik, Vijay Prakash Dwivedi, Johannes Hoffart, Carlos Guestrin, and Jure Leskovec. 2026. PLUREL: Synthetic Data unlocks Scaling Laws for Relational Foundation Models. InarXiv preprint arXiv:2602.04029. https://arxiv.org/abs/2602.04029

    Pith/arXiv arXiv 2026

  6. [6]

    Samuel Müller, Noah Hollmann, Sebastian Pineda Arango, Josif Grabocka, and Frank Hutter. 2022. Transformers Can Do Bayesian Inference. InInternational Conference on Learning Representations

    2022

  7. [7]

    Jingang Qu, David Holzmüller, Gaël Varoquaux, and Marine Le Morvan. 2025. TabICL: A Tabular Foundation Model for In-Context Learning on Large Data. In Forty-second International Conference on Machine Learning (ICML)

    2025

  8. [8]

    Jingang Qu, David Holzmüller, Gaël Varoquaux, and Marine Le Morvan. 2026. TabICLv2: A Better, Faster, Scalable, and Open Tabular Foundation Model.arXiv preprint arXiv:2602.11139(2026). https://arxiv.org/abs/2602.11139 4

    arXiv 2026

  9. [9]

    Rishabh Ranjan, Valter Hudovernik, Matej Znidar, Charilaos Kanatsoulis, Raghav Upendra, Mahmoud Mohammadi, Joe Meyer, Tom Palczewski, Carlos Guestrin, and Jure Leskovec. 2025. Relational Transformer: Toward Zero-Shot Foundation Models for Relational Data. InarXiv preprint arXiv:2510.06377. https://arxiv.org/ abs/2510.06377

    arXiv 2025

  10. [10]

    Joshua Robinson, Rishabh Ranjan, Weihua Hu, Kexin Huang, Jiaqi Han, Alejandro Dobles, Matthias Fey, Jan Eric Lenssen, Yiwen Yuan, Zecheng Zhang, et al. 2024. RelBench: A Benchmark for Deep Learning on Relational Databases.Advances in Neural Information Processing Systems37 (2024), 21330-21341

    2024

  11. [11]

    Minjie Wang, Quan Gan, David Wipf, Zheng Zhang, Christos Faloutsos, Weinan Zhang, Muhan Zhang, Zhenkun Cai, Jiahang Li, Zunyao Mao, et al. 2024. 4DBIn- fer: A 4D Benchmarking Toolbox for Graph-Centric Predictive Modeling on RDBs.Advances in Neural Information Processing Systems37 (2024), 27236-27273

    2024

  12. [12]

    Yanbo Wang, Jiaxuan You, Chuan Shi, and Muhan Zhang. 2026. Relational In- Context Learning via Synthetic Pre-training with Structural Prior. InProceedings of the 43rd International Conference on Machine Learning (ICML). PMLR

    2026

  13. [13]

    Family B all-at-once

    Xiyuan Zhang, Danielle C. Maddix, Junming Yin, Nick Erickson, Abdul Fatir Ansari, Boran Han, Shuai Zhang, Leman Akoglu, Christos Faloutsos, Michael W. Mahoney, et al. 2025. Mitra: Mixed Synthetic Priors for Enhancing Tabular Foundation Models. InarXiv preprint arXiv:2510.21204. A PER-TASK RESULTS This appendix supplements the main paper with full per-task...

    arXiv 2025