REVIEW 3 major objections 4 minor 37 references
Hippasus: Effective and Efficient Automatic Feature Augmentation for Machine Learning Tasks on Relational Data
T0 review · 3 major / 4 minor · reviewed 2026-08-03 · deepseek-v4-flash
Pith's one-line read Hippasus claims that decoupling semantic path scoring from join execution makes automatic feature augmentation over relational schemas more accurate and faster than prior systems, reporting accuracy gains up to 26.8 percent.
desk verdict Well-structured LLM feature-augmentation system, but the evaluation may leak test labels into feature selection; the headline gains are unsubstantiated until the authors show a proper train/test split. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The central mechanism is the decoupled score-then-execute pipeline. The load-bearing object is the hybrid path scoring function Score(pi) = (S_sem + S_stat) / (2(ell-1)), which averages LLM semantic scores across tables in a path with three statistical connection metrics (coverage, uniqueness, size ratio) per hop; normalizing by path length keeps long and short paths comparable. A second mechanism, the suffix multi-way join, applies semi-join reductions to the suffix of a join path before left-joining it to the base table, guaranteeing row and distribution preservation while reducing intermediate sizes. The final mechanism is the LLM feature-ranking prompt that fuses mutual information and P
What would settle it
Take a synthetic join graph where the target column depends on a feature located in a table the LLM rates low (or which the embedding prefilter drops), while statistically poor paths rank high; if Hippasus prunes the useful table and its accuracy drops below a statistics-only explorer that visits all tables, the claim that LLM scores predict feature utility is falsified. Concretely, construct such a dataset, run the pipeline, and compare against a variant that gives every table a neutral semantic score.
Extended reading notes
Core claim
The paper claims that feature augmentation—enriching a base table with predictive columns from other tables joined through primary/foreign keys—can be made both more accurate and cheaper by decoupling three steps that prior systems fuse together. A Path Explorer scores all candidate join paths before materializing them, combining one batch LLM call that rates each table's semantic relevance to the target with cheap metadata statistics (coverage, uniqueness, size ratio). Only the top paths are materialized, using a suffix-style multi-way join with semi-join reductions and left-join semantics that preserves every base row and the target distribution. A consolidated table is then passed to a hy
Load-bearing premise
The load-bearing premise is that a single LLM batch call can rank tables by semantic relevance to the target well enough that paths passing through low-ranked tables can be safely pruned before any join is executed; when the schema exceeds the LLM context window, tables that fall outside an embedding-based prefilter get a semantic score of zero and are never reconsidered.
Editorial extensions
If this is right
- Feature augmentation can reach multi-hop distances: allowing paths up to length 7 and letting an LLM rank paths finds informative features that single-hop systems miss, with accuracy improving as depth increases on most datasets.
- Join execution cost can be contained: materializing only about 10 top-ranked paths with a semi-join multi-way strategy keeps augmentation time at seconds to a few minutes, avoiding exhaustive materialization.
- Semantic context matters most when column names are cryptic: generated feature descriptions produce large gains (for example over 36% on a steel-plate classification task) and little harm on self-descriptive schemas.
- Neither statistics nor semantics alone suffice: hybrid LLM-statistical feature selection outperforms either signal alone on 10 of 12 datasets, and pure-LLM selection can degrade accuracy substantially.
- The same approach is not tied to one LLM: open-weight models in the 8B-72B range perform competitively, suggesting privacy-preserving deployments are feasible.
Reading between the lines
- If LLM semantic scores are cheap and stable across model families, the same score-then-prune pattern could be lifted to data-lake settings where the join graph is not precomputed; the paper names this as future work, and the architecture suggests discovery tools could feed candidate edges instead of a static schema.
- The hybrid selection step suggests a general recipe for LLM-based feature selection in other tabular pipelines: give the model statistical evidence and instruct it to defer to that evidence when it conflicts with semantic intuition; this 'statistics ground truth' instruction is a testable design choice.
- A testable extension: replace the single-batch table scoring with a sampling-based scoring when schemas exceed the context window; the paper's embedding prefilter currently assigns unscored tables a semantic score of zero, so a sensitivity analysis on prefilter recall would reveal how much accuracy depends on that top-T selection.
- If the accuracy gains generalize, the practical consequence is that ordinary ML teams can augment features from enterprise relational schemas without hand-writing join logic, using one LLM call per stage plus standard statistics—no training data or reinforcement loop.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper presents Hippasus, a modular framework for automatic feature augmentation over relational data. It decouples the pipeline into four components: a Feature Description Generator that uses LLMs to enrich column names, a Path Explorer that ranks multi-hop join paths before materialization by combining LLM semantic table scores with lightweight statistical join-quality metrics, a Join Executor that uses a suffix-Yannakakis multi-way left-join strategy with feature consolidation, and a hybrid Feature Selector that feeds mutual information and Pearson correlation to an LLM for final feature ranking. The authors evaluate on 12 real-world datasets and report an average improvement of 26.8% over ARDA, 14.5% over AutoFeat, and 18.6% over FeatPilot, with runtime speedups up to 60x over FeatPilot. The central claim is that an LLM-plus-statistics pipeline can prune joins before execution, materialize fewer paths, and still select features that outperform prior exhaustive, RL, and LSTM-based augmentation systems.
Significance. If the empirical claims hold, this is a timely and useful contribution: it addresses a real scalability bottleneck in feature augmentation by explicitly separating path exploration from join execution, and it uses LLM semantic signals in a cost-aware way. The paper also ships an artifact link, provides component-level ablations, and gives asymptotic analysis for the join executor, which are strengths. However, the headline effectiveness result is currently not cleanly supported because the evaluation protocol does not state where the supervised feature statistics are computed relative to the train/test split. This is a load-bearing issue: mutual information and Pearson correlation are computed against the target variable, and if they are computed on the full data before splitting, every effectiveness table is biased. A second issue is that default parameters are chosen on the same datasets used for the final comparison. These concerns can be addressed in revision, and the architectural ideas remain valuable.
major comments (3)
- [§8, §9.1] The Feature Selector computes mutual information and Pearson correlation between each candidate feature and the target y (§8). Section 9.1 states only that all reported results are averages over five runs with different random seeds; it never describes a train/test split before these statistics are computed. If MI/Pearson are computed on the full augmented table, feature ranking is supervised by the same labels that are later used for evaluation, biasing all effectiveness tables (Tables 2, 4, 5, 6, 7, 8, 9) and invalidating the headline 26.8% average improvement. Please specify the exact split protocol, state clearly on which rows the statistical signals are computed, and recompute the comparisons if necessary.
- [§9.5, Tables 7–9] The default parameters (ℓ=7, π=10, κ=10) appear to be selected by inspecting sensitivity results on the same 12 datasets that are used for the headline comparison. Since the final Tables 2 and 3 are reported with these chosen defaults, the main comparison is at least partially selected on the test data. To support the 26.8% claim, the authors should report results across all parameter settings, use a held-out tuning set, or provide nested cross-validation.
- [§9.1] For FeatPilot, the paper states that 'we train the LSTM on the same exploration data used by Hippasus.' If this means the LSTM is trained on data produced by Hippasus's path explorer rather than on FeatPilot's official exploration data, the comparison may not reflect the baseline as intended. Please clarify what 'exploration data' means, and if this deviates from FeatPilot's released pipeline, re-run the baseline with its standard training procedure.
minor comments (4)
- [Table 4] The Northwind row reports an 'Improvement' of +2.41% for FDG, but the MAE worsens from 11.925 to 12.310. Since lower MAE is better, this should be a negative improvement; please correct the sign and the associated discussion.
- [Abstract, §9.2] The abstract says 'up to 26.8%' while the body reports an average improvement of 26.8% over ARDA. These are different statements; the abstract should say 'average' or state the actual maximum per-dataset improvement.
- [§9.1] The paper reports averages over five runs but provides no standard deviations or significance tests. Given the small number of runs and the variability of AutoGluon, adding error bars would strengthen the comparison.
- [§8] The text says the feature selector 'extends LLM-Rank [22]', but reference [22] is described as a data-centric study of LLM feature selection. Please clarify the precise relationship, or cite the specific LLM-Rank method if it is different.
Circularity Check
Feature selection uses the evaluation target before any documented train/test split, so the headline accuracy gain is partly self-fulfilling.
-
fitted input called prediction
[Section 8 (Feature Selector); Section 9.1 (Experimental Setup)]
"we employ Mutual Information (MI), which measures the dependency between a feature f and target y ... we utilize Pearson Correlation (ρ), which measures the linear association between numeric features and targets ... For each feature in the consolidated table, we provide its name, the description generated by FDG, and the two statistical measures computed earlier ... From the LLM’s ranked output, we select the top-κ features to produce the final augmented table T_aug, which is then provided to the ML Trainer for model training."
The feature ranking is computed from MI(f,y) and Pearson(f,y), i.e., from the target labels themselves. Section 9.1 only says 'all reported results are averages over five runs with different random seeds' and never describes a train/test split before these statistics are computed. If MI/Pearson are computed on the same rows that later enter AutoGluon's training/evaluation, then the y values used to select top-κ features are the same y values on which accuracy/F1/MAE are measured. The selection signal and the evaluation signal coincide, making the reported effectiveness in Tables 2, 4, 5, 6, 7, 8, and 9 partly manufactured rather than an independent measurement of the pipeline.
full rationale
No self-citation chain or imported uniqueness theorem is load-bearing here; the paper does not derive its headline result from equations that are equivalent to its inputs by construction. The main circularity risk is the target-label leakage in the Feature Selector: mutual information and Pearson correlation against y are fed to the LLM, used to choose features, and then the same y is used to score the downstream model, with no stated split. This biases all effectiveness tables, including the 26.8% average improvement claim and the 9-of-12 'best' count. There is also a milder benchmark-selection issue: default hyperparameters (ℓ=7, π=10, κ=10) appear to be chosen from the same datasets used for final reporting, which inflates the main result but is a statistical overfitting concern rather than circular derivation. The efficiency and join-execution results are less affected by this leakage, but the central effectiveness claim is partially circular as reported.
Assumptions & free parameters
free parameters (6)
- α, β, γ join-quality weights =
1/3 each (default)
- max path length ℓ =
7 (default)
- path budget π =
10 (default)
- number of selected features κ =
10 (default)
- LLM temperature =
0.1
- statistical prefilter size K =
100 (example)
assumptions (5)
- standard math Yannakakis algorithm correctly and efficiently computes acyclic joins via semi-join reductions.
- domain assumption PK-FK relationships are given as input and the join graph is correct and complete.
- domain assumption Only acyclic paths from the base table are considered.
- domain assumption LLM outputs valid JSON scores and rankings that follow the prompt's integration rules.
- domain assumption Deduplicating foreign-key duplicates by first occurrence preserves the row and distribution invariants without destroying useful signal.
Cite this review
Pith. "Pith review of Hippasus: Effective and Efficient Automatic Feature Augmentation for Machine Learning Tasks on Relational Data." pith.science (2026). https://pith.science/paper/VI5UQ45R
@misc{pith2026260202025,
author = {Pith},
title = {Pith review of: Hippasus: Effective and Efficient Automatic Feature Augmentation for Machine Learning Tasks on Relational Data},
year = {2026},
howpublished = {\url{https://pith.science/paper/VI5UQ45R}},
note = {Machine review of arXiv:2602.02025}
}
read the original abstract
ML models critically depend on feature quality, yet in real-world settings, useful features are often distributed across multiple relational tables rather than a single dataset. Feature augmentation addresses this problem by automatically discovering and joining additional tables to enrich a base table with predictive features. However, scaling feature augmentation to complex schemas with many tables and multi-hop relationships is challenging. It requires exploring a large space of join paths, executing costly joins, and selecting useful features from noisy results. Existing approaches suffer from either limited effectiveness or efficiency. Restricting exploration to simple joins limits predictive performance, while more expressive methods rely on expensive training data, lack scalability, or fail to fully exploit schema-level semantics. We present Hippasus, a cost-aware, LLM-augmented feature discovery framework over relational schemas that addresses these challenges. Hippasus combines lightweight statistical signals with adaptive semantic reasoning, invoking stronger (LLM-based) analysis only when necessary. It further introduces efficient multi-way join execution with cross-path feature consolidation, and a hybrid feature selection strategy that integrates statistical relevance with semantic refinement. Experiments on real-world datasets show that Hippasus improves feature augmentation accuracy by up to 26.8% over state-of-the-art methods, while achieving a favorable effectiveness-cost tradeoff.
Figures
Figures from the paper (4 more)
Reference graph
Works this paper leans on
-
[1]
Liese Bekkers, Frank Neven, Stijn Vansummeren, and Yisu Remy Wang. 2025. Instance-Optimal Acyclic Join Processing Without Regret: Engineering the Yan- nakakis Algorithm in Column Stores.Proc. VLDB Endow.18, 8 (2025), 2413–2426
2025
-
[2]
Sonia Castelo, Rémi Rampin, Aécio S. R. Santos, Aline Bessa, Fernando Chirigati, and Juliana Freire. 2021. Auctus: A Dataset Search Engine for Data Discovery and Augmentation.Proc. VLDB Endow.14, 12 (2021), 2791–2794
2021
-
[3]
Nadiia Chepurko, Ryan Marcus, Emanuel Zgraggen, Raul Castro Fernandez, Tim Kraska, and David R. Karger. 2020. ARDA: Automatic Relational Data Augmentation for Machine Learning.Proc. VLDB Endow.13, 9 (2020), 1373–1387
2020
-
[4]
Xin Luna Dong and Theodoros Rekatsinas. 2018. Data Integration and Machine Learning: A Natural Synergy. InProceedings of the 2018 International Conference on Management of Data (SIGMOD). 1645–1650
2018
-
[5]
Yuyang Dong, Kunihiro Takeoka, Chuan Xiao, and Masafumi Oyamada. 2021. Efficient Joinable Table Discovery in Data Lakes: A High-Dimensional Similarity- Based Approach. In37th IEEE International Conference on Data Engineering (ICDE). 456–467
2021
-
[6]
Yuyang Dong, Chuan Xiao, Takuma Nozawa, Masafumi Enomoto, and Masafumi Oyamada. 2023. DeepJoin: Joinable Table Discovery with Pre-trained Language Models.Proc. VLDB Endow.16, 10 (2023), 2458–2470
2023
-
[7]
Dyer and Camille F
John N. Dyer and Camille F. Rogers. 2015.Teaching Case: Adapting the Access Northwind Database to Support a Database Course.J. Inf. Syst. Educ.26, 2 (2015), 85–102
2015
-
[8]
Nick Erickson, Jonas Mueller, Alexander Shirkov, Hang Zhang, Pedro Larroy, Mu Li, and Alexander J. Smola. 2020. AutoGluon-Tabular: Robust and Accurate AutoML for Structured Data.CoRRabs/2003.06505 (2020)
arXiv 2020
Show all 37 references
-
[9]
Meihao Fan, Xiaoyue Han, Ju Fan, Chengliang Chai, Nan Tang, Guoliang Li, and Xiaoyong Du. 2024. Cost-effective in-context learning for entity resolution: A design space exploration. In40th International Conference on Data Engineering (ICDE). 3696–3709
2024
-
[10]
Raul Castro Fernandez, Ziawasch Abedjan, Famien Koko, Gina Yuan, Samuel Madden, and Michael Stonebraker. 2018. Aurum: A Data Discovery System. In 34th IEEE International Conference on Data Engineering (ICDE). 1001–1012
2018
-
[11]
Raul Castro Fernandez, Jisoo Min, Demitri Nava, and Samuel Madden. 2019. Lazo: A Cardinality-Based Method for Coupled Estimation of Jaccard Similarity and Containment. In35th IEEE International Conference on Data Engineering (ICDE). 1190–1201
2019
-
[12]
Benjamin Feuer, Yurong Liu, Chinmay Hegde, and Juliana Freire. 2024. ArcheType: A Novel Framework for Open-Source Column Type Annotation using Large Language Models.Proc. VLDB Endow.17, 9 (2024), 2279–2292
2024
-
[13]
Juliana Freire, Grace Fan, Benjamin Feuer, Christos Koutras, Yurong Liu, Eduardo Peña, Aécio S. R. Santos, Cláudio T. Silva, and Eden Wu. 2025. Large Language Models for Data Discovery and Integration: Challenges and Opportunities.IEEE Data Eng. Bull.49, 1 (2025), 3–31
2025
-
[14]
Sainyam Galhotra, Yue Gong, and Raul Castro Fernandez. 2023. Metam: Goal- Oriented Data Discovery. In39th IEEE International Conference on Data Engineer- ing (ICDE). 2780–2793
2023
-
[15]
Andra Ionescu, Kiril Vasilev, Florena Buse, Rihan Hai, and Asterios Katsifodimos
-
[16]
Jeong, Zachary Chase Lipton, and Pradeep Kumar Ravikumar
Daniel P. Jeong, Zachary Chase Lipton, and Pradeep Kumar Ravikumar. 2025. LLM-Select: Feature Selection with Large Language Models.Trans. Mach. Learn. Res.2025 (2025)
2025
-
[17]
Pengyue Jia, Zhaocheng Du, Yichao Wang, Xiangyu Zhao, Xiaopeng Li, Yuhao Wang, Qidong Liu, Huifeng Guo, and Ruiming Tang. 2024. AltFS: Agency-light Feature Selection with Large Language Models in Deep Recommender Systems. CoRRabs/2412.08516 (2024)
2024 arXiv
-
[18]
James Max Kanter and Kalyan Veeramachaneni. 2015. Deep feature synthesis: Towards automating data science endeavors. InIEEE International Conference on Data Science and Advanced Analytics (DSAA). 1–10
2015
-
[19]
Moe Kayali, Anton Lykov, Ilias Fountalis, Nikolaos Vasiloglou, Dan Olteanu, and Dan Suciu. 2024. CHORUS: Foundation Models for Unified Data Discovery and Exploration.Proc. VLDB Endow.17, 8 (2024), 2104–2114
2024
-
[20]
Miller, and Mirek Riedewald
Aamod Khatiwada, Grace Fan, Roee Shraga, Zixuan Chen, Wolfgang Gatter- bauer, Renée J. Miller, and Mirek Riedewald. 2023. SANTOS: Relationship-based Semantic Table Union Search.Proc. ACM Manag. Data1, 1 (2023), 9:1–9:25
2023
-
[21]
Naughton, Jignesh M
Arun Kumar, Jeffrey F. Naughton, Jignesh M. Patel, and Xiaojin Zhu. 2016. To Join or Not to Join?: Thinking Twice about Joins before Feature Selection. In Proceedings of the 2016 International Conference on Management of Data (SIGMOD). 19–34
2016
-
[22]
Dawei Li, Zhen Tan, and Huan Liu. 2024. Exploring Large Language Models for Feature Selection: A Data-centric Perspective.SIGKDD Explor.26, 2 (2024), 44–53
2024
-
[23]
Jianhao Li and Xianchao Xiu. 2025. LLM4FS: Leveraging Large Language Models for Feature Selection and How to Improve It.CoRRabs/2503.24157 (2025)
2025
-
[24]
Peng Li, Yeye He, Dror Yashar, Weiwei Cui, Song Ge, Haidong Zhang, Danielle Rifinski Fainman, Dongmei Zhang, and Surajit Chaudhuri. 2024. Table-GPT: Table Fine-tuned GPT for Diverse Table Tasks.Proceedings of the ACM on Management of Data2, 3 (2024), 1–28
2024
-
[25]
Jiaming Liang, Chuan Lei, Xiao Qin, Jiani Zhang, Asterios Katsifodimos, Christos Faloutsos, and Huzefa Rangwala. 2025. FeatPilot: Automatic Feature Augmenta- tion on Tabular Data. In41st IEEE International Conference on Data Engineering (ICDE). 2148–2160
2025
-
[26]
Jiabin Liu, Chengliang Chai, Yuyu Luo, Yin Lou, Jianhua Feng, and Nan Tang. 2022. Feature Augmentation with Reinforcement Learning. In38th IEEE International Conference on Data Engineering (ICDE). 3360–3372
2022
-
[27]
Danrui Qi, Weiling Zheng, and Jiannan Wang. 2024. FeatAug: Automatic Feature Augmentation From One-to-Many Relationship Tables.IEEE 40th International Conference on Data Engineering (ICDE)(2024), 1805–1818
2024
-
[28]
Vraj Shah, Arun Kumar, and Xiaojin Zhu. 2017. Are Key-Foreign Key Joins Safe to Avoid when Learning High-Capacity Classifiers?Proc. VLDB Endow.11, 3 (2017), 366–379
2017
-
[29]
Qichen Wang, Bingnan Chen, Binyang Dai, Ke Yi, Feifei Li, and Liang Lin. 2025. Yannakakis+: Practical Acyclic Query Evaluation with Theoretical Guarantees. Proc. ACM Manag. Data3, 3 (2025), 235:1–235:28
2025
-
[30]
Siyi Wang, Qi Deng, Shiwei Feng, Hong Zhang, and Chao Liang. 2024. A sur- vey on rank aggregation. InProceedings of the Thirty-Third International Joint Conference on Artificial Intelligence (IJCAI-24). 8281–8289
2024
-
[31]
Lindsey Linxi Wei, Guorui Xiao, and Magdalena Balazinska. 2024. RACOON: An LLM-based Framework for Retrieval-Augmented Column Type Annotation with a Knowledge Graph. InNeurIPS 2024 Third Table Representation Learning Workshop
2024
-
[32]
Mihalis Yannakakis. 1981. Algorithms for Acyclic Database Schemes. InVLDB. 82–94
1981
-
[33]
Alizadeh, Kangwook Lee, Jose Blanchet, Mert Pilanci, and Robert Tibshirani
Erica Zhang, Ryunosuke Goto, Naomi Sagan, Jurik Mutter, Nick Phillips, Ash A. Alizadeh, Kangwook Lee, Jose Blanchet, Mert Pilanci, and Robert Tibshirani
-
[34]
Haoxiang Zhang, Yurong Liu, Wei-Lun Hung, Aécio S. R. Santos, and Juliana Freire. 2025. AutoDDG: Automated Dataset Description Generation using Large Language Models.CoRRabs/2502.01050 (2025)
2025
-
[35]
Erkang Zhu, Dong Deng, Fatemeh Nargesian, and Renée J. Miller. 2019. JOSIE: Overlap Set Similarity Search for Finding Joinable Tables in Data Lakes. In Proceedings of the 2019 International Conference on Management of Data (SIGMOD). 847–864
2019
-
[2024]
In40th IEEE International Conference on Data Engineering (ICDE)
AutoFeat: Transitive Feature Discovery over Join Paths. In40th IEEE International Conference on Data Engineering (ICDE). 1861–1873
-
[2025]
LLM-Lasso: A Robust Framework for Domain-Informed Feature Selection and Regularization.CoRRabs/2502.10648 (2025)
2025 arXiv
Reviewed August 3, 2026 · model on record in the stance chip above.
Discussion (0). Sign in to comment.