Pith. sign in

REVIEW 4 major objections 5 minor 101 references

TopoFE: topology-aware LLM-guided Automated Feature Engineering

T0 review · 4 major / 5 minor · reviewed 2026-07-31 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read Automated feature engineering improves when LLM search is split into family-specialized islands whose cross-island transfers are learned online.

desk verdict TopoFE's accuracy story is plausible, but its diversity and topology claims rest on a correlation filter and a by-construction metric. read the letter →

arxiv 2607.23286 v1 pith:IIQQYV3S submitted 2026-07-25 cs.AI cs.LG

Sha Li , Naren Ramakrishnan This is my paper
classification cs.AIcs.LG
keywords automatedfeatureengineeringlargelanguagemodelsevolutionarysearchprogramsynthesismulti-islandevolutiontopologylearningtabulardatatransfer
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper argues that the weak spot in LLM-based automated feature engineering is not the language model but the search architecture: a single undifferentiated population locks onto one family of transformations and never reaches features that combine several families. TOPOFE splits the program space into five islands—arithmetic interactions, statistical aggregates, temporal dynamics, relational encodings, and nonlinear maps—each evolving features under its own assumption about where signal comes from. A directed topology graph learns, from observed transfer gains, which island to borrow from when another island's local search is genuinely exhausted, and the LLM is then prompted to synthesize a hybrid program that carries one structural idea from the donor family into the target family. The paper's central claim is that this structured decomposition plus adaptive, saturation-triggered transfer is what delivers consistent gains on 24 of 29 benchmarks, and that the gains are architectural rather than a side effect of LLM strength.

What carries the argument

The central object is the topology graph G(t), a directed weighted graph over islands whose edge weights encode the empirically observed utility of transferring knowledge from one transformation family to another, updated online by exponential moving average after each transfer event. It orchestrates three decisions: when to transfer (a saturation detector measuring marginal improvement over a sliding window of five generations), where to transfer (choosing the donor island with the highest learned edge weight), and what to transfer (LLM-mediated hybrid synthesis that composes a target-family program with one structural element from the donor family, producing features in the joint space nei

What would settle it

Run TOPOFE on the same benchmark suite with the saturation mechanism replaced by fixed-interval transfer and by random-precursor hybrid synthesis, keeping all other components identical; if the margin over single-population search disappears, then learning when and where to transfer is the active ingredient. Also sweep the saturation window and threshold across the suite: if accuracy is insensitive, the specific constants are non-essential; if it is highly sensitive, the mechanism is brittle.

Watch

Extended reading notes

Core claim

TOPOFE models feature engineering as program synthesis over a heterogeneous space partitioned into five transformation families (arithmetic interactions, statistical aggregates, temporal features, relational encodings, nonlinear univariate maps). Each family gets an island with its own population, elite archive, and a Prompt Adaptation Memory that distills accepted/rejected history into natural-language 'prefer/avoid' signals. When an island's marginal improvement over a five-generation window falls below a threshold, the topology graph selects the most promising donor island by learned edge weights, and the LLM is prompted to hybridize a target-family program with one structural idea from t

Load-bearing premise

The load-bearing premise is that the saturation detector's five-generation sliding window and 0.01 improvement threshold reliably distinguish genuine local exhaustion from a transient plateau; if this criterion fires too early or too late, the claimed diversity gains and cross-family transfer effects collapse.

Editorial extensions

If this is right

  • Feature engineering search should be decomposed by transformation family: single-population evolution collapses onto early-successful motifs, while family-specialized islands preserve diversity by construction.
  • Cross-family compositional features—such as a lagged rolling mean expressed as a ratio—become reachable only when transfer is triggered by measured stagnation and routed through learned complementarity, not by fixed schedules or random migration.
  • The discovered features encode dataset-intrinsic structure: they transfer to downstream predictors not used during search and remain stable across LLM backbones of different capability, so gains are attributable to the search mechanism rather than to any particular model.
  • The learned topology graph acquires dataset-specific knowledge: its specialization score grows monotonically, and datasets with near-zero specialization are exactly those where multi-island transfer yields little benefit.
  • Search becomes more stable: lower run-to-run variance indicates that structured, memory-conditioned search tames the instability of unconstrained LLM proposal generation.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The saturation detector's hyperparameters (window W=5, threshold ε=0.01) are hand-set; a natural extension is to make them adaptive to per-island fitness noise, and a sensitivity sweep would reveal how much of the claimed gain rests on these exact values.
  • The five transformation families are a fixed cover of program space; the same architecture could be extended to automatically discover family schemas from data, and the topology specialization score could serve as a stopping rule for retiring an island that has failed to specialize.
  • The parallel-speedup argument assumes cross-island LLM calls are independent and that fitness evaluation dominates wall-clock time; if LLM latency dominates, the practical speedup will be smaller than the analysis suggests.
  • Since the final feature set is capped at five programs via greedy correlation-thresholded selection, a reader could test whether the diversity gains come from the multi-island search itself or from the final redundancy filter by applying the same selection procedure to a single-population archive.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. The paper proposes TOPOFE, an LLM-guided evolutionary framework for automated feature engineering. It partitions the feature-program search space into five transformation-family islands (arithmetic, statistical, temporal, relational, nonlinear), equips each island with a Prompt Adaptation Memory that summarizes accept/reject history, learns a directed topology graph of cross-family transfer utilities, triggers cross-island hybrid synthesis when an island is detected as saturated, and finally selects a low-redundancy feature set via a greedy correlation-filtered procedure. The authors evaluate TOPOFE on 19 classification and 10 regression datasets against six AutoFE baselines, report accuracy/RMSE, backbone robustness, downstream predictor transferability, two diversity metrics (MPOC and EffRank), a topology specialization metric (TGSS), and an ablation study. The central claim is that the architectural mechanisms—family-specialized islands, adaptive prompt memory, and topology-guided transfer—rather than raw LLM capability are responsible for the reported gains, and that TOPOFE discovers more diverse, transferable feature programs.

Significance. If the mechanism-level claims were established, this would be a meaningful contribution: it targets a real failure mode of LLM-based AutoFE (cross-family lock-in), and the design of family-specialized islands with learned transfer topology is a plausible and generalizable idea. The benchmark is broad (29 datasets, three LLM backbones, downstream predictor transfer, ablations), which is above average for this area. However, several load-bearing concerns prevent accepting the paper as-is: the multi-term objective stated in Eq. (15) is not what Algorithm 1 optimizes; the diversity metrics in Table 5 are confounded by a final correlation filter not applied to baselines; TGSS as defined in Eq. (19) is close to a mathematical artifact of the EMA update; and no statistical significance tests or code/data release are provided. These issues do not necessarily invalidate the predictive performance claims, but they do undermine the paper's stronger conclusion that the specific architectural components cause the gains.

major comments (4)
  1. [§3.3.4 (Eq. 15) and Algorithm 1] The paper states in Eq. (15) that TOPOFE jointly optimizes predictive performance, redundancy, stability, and cost with weights λ1, λ2, λ3. Algorithm 1 never references these weights or any stability/cost term; the only fitness signal used to accept/reject candidates and update archives is the cross-validated score Φ̂ (Eq. 6). Redundancy enters only in the final greedy selection described after Eq. (15) ('discarding candidates whose absolute Spearman correlation with any already-selected feature exceeds τ_red=0.9'), and stability/cost appear nowhere in the search loop. Consequently, Appendix A.vi's statement that 'TOPOFE alone optimizes an explicitly multi-term objective' is not supported by the algorithm as written. Moreover, because this final τ_red filter is not applied to any baseline, the low MPOC / high EffRank values in Table 5 may be an artifact of that filter rather than of the
  2. [§5.5 (Eq. 19, Fig. 2)] TGSS is defined as the empirical variance of the off-diagonal entries of the transfer-weight matrix W(t). Since W(t) is initialized to a uniform vector and updated via the EMA in Eq. (11), the variance increases mechanically as transfer events accumulate with different observed gains, even if the weights carry no task-specific signal. The monotonic increase in Fig. 2 is therefore not evidence that the topology graph 'acquires task-specific knowledge.' To support RQ5, the paper needs a control or a series of controls: for example, compare TGSS under learned precursor selection against TGSS under random or fixed precursor selection with the same number of transfer events, or test whether learned weights predict held-out transfer gains better than random weights. As it stands, the conclusion that 'the topology graph reliably acquires task-specific transfer knowledge' is not established.
  3. [§5.1 (Tables 1–2)] The headline claim is 'consistent improvements' over state-of-the-art methods, with best performance on 15/19 classification and 9/10 regression datasets. However, no statistical significance testing is reported, and several top-1 margins are smaller than the reported standard deviations (e.g., adult: 0.9268±0.0025 vs OpenFE 0.9267±0.0013). In addition, Table 2 contains a clear counterexample on the insurance dataset, where OpenFE (4.8053±0.2061) outperforms TopoFE (4.8843±0.1802). Please report paired significance tests across the five seeds or across datasets (e.g., Wilcoxon signed-rank test over the 29 datasets, or paired t-tests with appropriate multiple-testing correction), and state which comparisons remain significant. This is necessary to support the 'consistent improvements' claim and to quantify the size of the architectural advantage.
  4. [§3.3.1 (Eq. 12) and §5.6 (Fig. 3)] The saturation detector is central to the transfer mechanism, but Eq. (12) defines Δ_i(τ) as an expectation and gives no estimator; Algorithm 1 calls Saturated(i, W, ε) without specifying how the expectation is computed from the observed fitness sequence. The ablation study in §5.6 introduces 'Correlation-Only' and 'Multi-Signal Saturation' variants, yet §3.3.1 defines only a single-signal criterion based on marginal improvement. The 'multi-signal composite criterion' mentioned in the ablation text is not defined anywhere in the paper. Please specify the exact implemented detector (e.g., empirical mean of the last W observed deltas, with a rule for handling noisy fitness), and define the multi-signal variant. Without this, Fig. 3 cannot be interpreted as evidence about saturation detection.
minor comments (5)
  1. [General] No code or data release is provided. For an empirical paper with this many tables and ablations, a reproducibility statement with a code link and dataset split identifiers is essential, especially because the method involves many prompt templates and implementation details.
  2. [§5.2, Table 3] The backbone name in Table 3 ('QwenCode-7B') is inconsistent with the text, which refers to 'Qwen2.5-Coder-7B.' Please use one consistent name throughout.
  3. [§3.3.4] The stability term Stability(S) is described as 'the inverse variance of the fitness estimate Φ̂(S) across cross-validation folds,' but no formula is given and no results using it are reported. If the term is not used in the final implementation, say so explicitly; otherwise, provide the formal definition.
  4. [Eq. (3)] The composition notation p = o^{(L)} ∘ ⋯ ∘ o^{(1)} is ambiguous: it is not clear whether the outermost operator is o^{(L)} or o^{(1)}. Please define the evaluation order explicitly, since the paper later uses tree-edit distance and structural comparisons.
  5. [Appendix B] The prompt templates are helpful, but the paper would benefit from one worked example of an actual generated program (input columns, generated code, and whether it was accepted) to make the method concrete and to help readers understand the family constraints.

Circularity Check

0 steps flagged · score 1.0 of 10

No significant circularity: the central empirical claims rest on held-out benchmarks and external baselines; the unified-objective mismatch and TGSS construct-validity issue are correctness risks, not circular reductions.

full rationale

TOPOFE's central claim is an empirical comparison against external baselines on held-out test splits, so the accuracy results are self-contained and falsifiable; they do not reduce to fitted constants or to a self-citation chain. The paper's search procedure (Algorithm 1) is an evolutionary loop whose accept/reject signal is the cross-validated predictive fitness, not a derivation from Eq. (15). The stated multi-term objective (Eq. 15) is not literally what Algorithm 1 optimizes: the search loop uses only the predictive fitness, and the redundancy penalty appears only through the final τ_red-filtered greedy selection. This is an implementation-consistency weakness, not a circularity, because Eq. (15) is not used to generate the reported predictions. Similarly, TGSS (Eq. 19) is defined as the variance of EMA-updated transfer weights, and the paper treats its increase as evidence of task-specific transfer knowledge; but variance increase is largely a property of the update dynamics, so RQ5's interpretive claim is weakly supported. That is a metric-validity concern, not a case where a predicted quantity is identical to an input by construction. The paper's baselines and ablations provide independent points of comparison, and no load-bearing argument reduces to a self-citation or to a renamed known result. Given the external benchmarks and held-out evaluation, the appropriate circularity score is low.

Assumptions & free parameters 10 free parameters · 6 assumptions · 4 invented entities

The method's behavior and performance rest on a small set of hand-set hyperparameters, a manually chosen family taxonomy, and assumptions about LLM proposal quality and saturation detection. None of these are derived or externally benchmarked independently of the paper's own experiments.

free parameters (10)
  • EMA decay α = 0.3
    Controls adaptive topology weight updates in Eq. 11; hand-set without sensitivity analysis.
  • saturation threshold ε = 0.01
    Determines when cross-island transfer triggers in Eq. 12; hand-set and crucial to transfer timing.
  • history window W = 5
    Used for both saturation detection and PAM sliding window; hand-set.
  • redundancy penalty λ1 = 0.3
    Weight in unified objective Eq. 15; hand-set, no sensitivity analysis.
  • stability weight λ2 = 0.1
    Weight in Eq. 15; not actually used in the final greedy selection, and hand-set.
  • cost weight λ3 = 0.1
    Weight in Eq. 15; not observed in Algorithm 1, and hand-set.
  • correlation threshold τ_red = 0.9
    Spearman correlation cutoff in final greedy selection; hand-set.
  • tree-edit novelty threshold δ_min = 0.1
    Structural novelty filter for the elite archive; hand-set.
  • archive capacity |A|max = 20
    Elite archive size per island; hand-set.
  • number and choice of transformation families = M=5 (P1–P5)
    Family taxonomy is hand-defined in §3.1.1; no data-driven validation that these categories are the right or complete cover.
assumptions (6)
  • domain assumption The five canonical transformation families P1–P5 approximately cover the useful feature program space P.
    Definition 1 and §3.1.1; if the cover misses important families or overlaps badly, multi-island search is biased.
  • domain assumption LLM proposals conditioned on family prompts, memory, and in-context examples are sufficiently valid and diverse for evolutionary search to make progress.
    All effectiveness depends on LLM generation quality; no syntactic/semantic validity rate is reported.
  • ad hoc to paper Saturation measured by sliding-window marginal improvement (Eq. 12 with ε=0.01, W=5) reliably detects when an island's local search is exhausted.
    Threshold and window are arbitrary; no theoretical or empirical justification that they generalize across datasets.
  • ad hoc to paper The EMA update in Eq. 11 is a valid estimator of cross-family transfer utility.
    Assumes stationary utility and additive gains; no consistency properties are shown.
  • ad hoc to paper TGSS, the variance of learned transfer weights, reflects task-specific transfer knowledge.
    Eq. 19; the variance can increase from noise even without task signal, so it is not a validated probe.
  • domain assumption Feature programs discovered under the XGBoost surrogate transfer without modification to other predictors.
    §5.3 tests this for CatBoost/MLP/TabPFN but only reports averages and does not control for distribution shift in the augmentation matrix.
invented entities (4)
  • Prompt Adaptation Memory (PAM)
    purpose: Per-island summary of accepted/rejected program patterns used to condition LLM proposals.
    Validated only through internal ablations; no external falsifiable handle.
  • Topology graph over islands
    purpose: Learn a directed weighted graph for routing cross-island transfers.
    The graph is a method component; TGSS is an internal measure, not independent evidence.
  • Saturation-triggered cross-island transfer
    purpose: Trigger hybrid synthesis when an island's local search plateaus.
    Depends on hand-set ε and W; no external validation.
  • Five canonical transformation families
    purpose: Partition the feature program space into specialized search islands.
    Hand-defined taxonomy with no external or data-driven justification.

how reviews work

0 comments
Cite this review

Pith. "Pith review of TopoFE: topology-aware LLM-guided Automated Feature Engineering." pith.science (2026). https://pith.science/paper/IIQQYV3S

@misc{pith2026260723286,
  author       = {Pith},
  title        = {Pith review of: TopoFE: topology-aware LLM-guided Automated Feature Engineering},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/IIQQYV3S}},
  note         = {Machine review of arXiv:2607.23286}
}
read the original abstract

Automatic feature engineering (AutoFE) for tabular learning can be naturally formulated as a program synthesis problem, where the objective is to discover predictive feature transformations from an exponentially large search space. Recent advances in large language models (LLMs) have expanded the expressiveness of AutoFE by enabling feature program generation beyond predefined operator libraries. However, existing LLM-based approaches remain fundamentally limited by stateless generation and homogeneous search: feature proposals are produced from static prompts without accumulating search experience, while single-population exploration quickly converges to dominant transformation patterns and rarely discovers complementary feature compositions across transformation families. We propose TOPOFE, a topology-aware multi-island evolutionary framework for LLM-guided feature engineering. TOPOFE combines family-specialized exploration, adaptive prompt memory, and topology-guided knowledge transfer to efficiently discover diverse and compositional feature programs. Experiments on 29 public tabular datasets demonstrate consistent improvements over state-of-the-art AutoFE methods across classification and regression tasks. Beyond predictive performance, TOPOFE discovers more diverse and transferable feature programs that generalize across multiple downstream predictors and LLM backbones.

Figures

Figures reproduced from arXiv: 2607.23286 by the authors.

Figure 1
Figure 1. Overview of TOPOFE. It decomposes FE-program search into multiple transformation-family islands with LLM-guided local evolution, adaptive saturation detection, topology-aware cross-island synthesis, and archive-based selection to generate diverse engineered features that improve downstream tabular prediction performance. 3.1 Intra-island Evolution 3.1.1 Multi-island Decomposition The feature program space P is not h… view at source ↗
Figure 2
Figure 2. TGSS trajectories across all classification (left) and regression (right) datasets over 10 generations. [PITH_FULL_IMAGE:figures/full_fig_p012_2.png] view at source ↗
Figure 3
Figure 3. Ablation study of TOPOFE. Each bar reports the performance degradation relative to the full TOPOFE. Results are averaged across all datasets. 6 Related Work 6.1 Automatic Feature Engineering Automated feature engineering (AutoFE) has evolved through three broad paradigms. Search-based methods treat feature construction as a combinatorial optimization problem, through genetic programming [Olson and Moore, 2016], exha… view at source ↗

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

101 extracted references · 12 linked inside Pith

  1. [1]

    Feature engineering for machine learning and data analytics

    Guozhu Dong and Huan Liu. Feature engineering for machine learning and data analytics. CRC press, 2018

    2018

  2. [2]

    Openfe: Automated feature generation with expert-level performance

    Tianping Zhang, Zheyu Aqa Zhang, Zhiyuan Fan, Haoyan Luo, Fengyuan Liu, Qian Liu, Wei Cao, and Li Jian. Openfe: Automated feature generation with expert-level performance. In International Conference on Machine Learning, pages 41880--41901. PMLR, 2023 a

    2023

  3. [3]

    Large language models for automated data science: Introducing caafe for context-aware automated feature engineering

    Noah Hollmann, Samuel M \"u ller, and Frank Hutter. Large language models for automated data science: Introducing caafe for context-aware automated feature engineering. Advances in Neural Information Processing Systems, 36: 0 44753--44775, 2023 a

    2023

  4. [4]

    Tpot: A tree-based pipeline optimization tool for automating machine learning

    Randal S Olson and Jason H Moore. Tpot: A tree-based pipeline optimization tool for automating machine learning. In Workshop on automatic machine learning, pages 66--74. PMLR, 2016

    2016

  5. [5]

    The autofeat python library for automated feature engineering and selection

    Franziska Horn, Robert Pack, and Michael Rieger. The autofeat python library for automated feature engineering and selection. In Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pages 111--120. Springer, 2019

    2019

  6. [6]

    Large language models can automatically engineer features for few-shot tabular learning

    Sungwon Han, Jinsung Yoon, Sercan O Arik, and Tomas Pfister. Large language models can automatically engineer features for few-shot tabular learning. Forty-first International Conference on Machine Learning, 2024

    2024

  7. [8]

    Evolutionary large language model for automated feature transformation

    Nanxu Gong, Chandan K Reddy, Wangyang Ying, Haifeng Chen, and Yanjie Fu. Evolutionary large language model for automated feature transformation. In Proceedings of the AAAI conference on artificial intelligence, volume 39, pages 16844--16852, 2025

    2025

  8. [9]

    Uci machine learning repository, 2007

    Arthur Asuncion, David Newman, et al. Uci machine learning repository, 2007

    2007

Show all 101 references
  1. [10]

    Openml: networked science in machine learning

    Joaquin Vanschoren, Jan N Van Rijn, Bernd Bischl, and Luis Torgo. Openml: networked science in machine learning. ACM SIGKDD Explorations Newsletter, 15 0 (2): 0 49--60, 2014

    2014

  2. [11]

    Tabpfn: A transformer that solves small tabular classification problems in a second

    Noah Hollmann, Samuel M \"u ller, Katharina Eggensperger, and Frank Hutter. Tabpfn: A transformer that solves small tabular classification problems in a second. In NeurIPS 2022 First Table Representation Workshop

    2022

  3. [12]

    Optimized feature generation for tabular data via llms with decision tree reasoning

    Jaehyun Nam, Kyuyoung Kim, Seunghyuk Oh, Jihoon Tack, Jaehyung Kim, and Jinwoo Shin. Optimized feature generation for tabular data via llms with decision tree reasoning. Advances in neural information processing systems, 37: 0 92352--92380, 2024

    2024

  4. [16]

    Lift: Language-interfaced fine-tuning for non-language machine learning tasks

    Tuan Dinh, Yuchen Zeng, Ruisu Zhang, Ziqian Lin, Michael Gira, Shashank Rajput, Jy-yong Sohn, Dimitris Papailiopoulos, and Kangwook Lee. Lift: Language-interfaced fine-tuning for non-language machine learning tasks. Advances in Neural Information Processing Systems, 35: 0 1176...

    2022

  5. [17]

    Catboost: unbiased boosting with categorical features

    Liudmila Prokhorenkova, Gleb Gusev, Aleksandr Vorobev, Anna Veronika Dorogush, and Andrey Gulin. Catboost: unbiased boosting with categorical features. Advances in neural information processing systems, 31, 2018

    2018

  6. [18]

    In Geomatic approaches for modeling land change scenarios, pages 451--455

    Multilayer perceptron (mlp). In Geomatic approaches for modeling land change scenarios, pages 451--455. Springer, 2017

    2017

  7. [19]

    Tabpfn: A transformer that solves small tabular classification problems in a second

    Noah Hollmann, Samuel M \"u ller, Katharina Eggensperger, and Frank Hutter. Tabpfn: A transformer that solves small tabular classification problems in a second. In International Conference on Learning Representations 2023, 2023 b

    2023

  8. [20]

    Batch normalization: Accelerating deep network training by reducing internal covariate shift

    Sergey Ioffe and Christian Szegedy. Batch normalization: Accelerating deep network training by reducing internal covariate shift. In International conference on machine learning, pages 448--456. pmlr, 2015

    2015

  9. [21]

    Ferg-llm: Feature engineering by reason generation large language models

    Jeonghyun Ko, Gyeongyun Park, Donghoon Lee, and Kyunam Lee. Ferg-llm: Feature engineering by reason generation large language models. In Findings of the Association for Computational Linguistics: NAACL 2025, pages 4211--4228, 2025

    2025

  10. [22]

    Dynamic and adaptive feature generation with llm

    Xinhao Zhang, Jinghan Zhang, Banafsheh Rekabdar, Yuanchun Zhou, Pengfei Wang, and Kunpeng Liu. Dynamic and adaptive feature generation with llm. In Proceedings of the Thirty-Fourth International Joint Conference on Artificial Intelligence, pages 7029--7037, 2025

    2025

  11. [27]

    Large scale transfer learning for tabular data via language modeling

    Josh Gardner, Juan C Perdomo, and Ludwig Schmidt. Large scale transfer learning for tabular data via language modeling. Advances in Neural Information Processing Systems, 37: 0 45155--45205, 2024

    2024

  12. [28]

    Tab STAR : A tabular foundation model for tabular data with text fields

    Alan Arazi, Eilam Shapira, and Roi Reichart. Tab STAR : A tabular foundation model for tabular data with text fields. In The Thirty-ninth Annual Conference on Neural Information Processing Systems, 2026. URL https://openreview.net/forum?id=FrXHdcTEzE

    2026

  13. [29]

    u ller, Ga \

    Jingang Qu, David Holzm \"u ller, Ga \"e l Varoquaux, and Marine Le Morvan. Tabicl: A tabular foundation model for in-context learning on large data. In International Conference on Machine Learning, pages 50817--50847. PMLR, 2025

    2025

  14. [30]

    Cresswell, Keyvan Golestan, Guangwei Yu, Anthony L

    Junwei Ma, Valentin Thomas, Rasa Hosseinzadeh, Alex Labach, Hamidreza Kamkari, Jesse C. Cresswell, Keyvan Golestan, Guangwei Yu, Anthony L. Caterini, and Maksims Volkovs. Tab DPT : Scaling tabular foundation models on real data. In The Thirty-ninth Annual Conference on Neural ...

    2025

  15. [31]

    Tabllm: Few-shot classification of tabular data with large language models

    Stefan Hegselmann, Alejandro Buendia, Hunter Lang, Monica Agrawal, Xiaoyi Jiang, and David Sontag. Tabllm: Few-shot classification of tabular data with large language models. In International conference on artificial intelligence and statistics, pages 5549--5581. PMLR, 2023

    2023

  16. [32]

    Towards cross-table masked pretraining for web data mining

    Chao Ye, Guoshan Lu, Haobo Wang, Liyao Li, Sai Wu, Gang Chen, and Junbo Zhao. Towards cross-table masked pretraining for web data mining. In Proceedings of the ACM Web Conference 2024, pages 4449--4459, 2024 a

    2024

  17. [33]

    Carte: Pretraining and transfer for tabular learning

    Myung Jun Kim, Leo Grinsztajn, and Gael Varoquaux. Carte: Pretraining and transfer for tabular learning. In International Conference on Machine Learning, pages 23843--23866. PMLR, 2024

    2024

  18. [34]

    Making pre-trained language models great on tabular prediction

    Jiahuan Yan, Bo Zheng, Hongxia Xu, Yiheng Zhu, Danny Chen, Jimeng Sun, Jian Wu, and Jintai Chen. Making pre-trained language models great on tabular prediction. In The Twelfth International Conference on Learning Representations

  19. [35]

    Generative table pre-training empowers models for tabular prediction

    Tianping Zhang, Shaowen Wang, Shuicheng Yan, Li Jian, and Qian Liu. Generative table pre-training empowers models for tabular prediction. In Proceedings of the 2023 conference on empirical methods in natural language processing, pages 14836--14854, 2023 b

    2023

  20. [36]

    Language models are realistic tabular data generators

    Vadim Borisov, Kathrin Sessler, Tobias Leemann, Martin Pawelczyk, and Gjergji Kasneci. Language models are realistic tabular data generators. In The Eleventh International Conference on Learning Representations

  21. [37]

    Exploring llm agents for cleaning tabular machine learning datasets

    Tommaso Bendinelli, Artur Dox, and Christian Holz. Exploring llm agents for cleaning tabular machine learning datasets. In ICLR 2025 Workshop on Foundation Models in the Wild

    2025

  22. [38]

    Evoprompt: Connecting llms with evolutionary algorithms yields powerful prompt optimizers

    Qingyan Guo, Rui Wang, Junliang Guo, Bei Li, Kaitao Song, Xu Tan, Guoqing Liu, Jiang Bian, and Yujiu Yang. Evoprompt: Connecting llms with evolutionary algorithms yields powerful prompt optimizers. arXiv e-prints, pages arXiv--2309, 2023

    2023

  23. [39]

    Promptbreeder: Self-referential self-improvement via prompt evolution

    Chrisantha Fernando, Dylan Sunil Banarse, Henryk Michalewski, Simon Osindero, and Tim Rockt \"a schel. Promptbreeder: Self-referential self-improvement via prompt evolution. In Forty-first International Conference on Machine Learning

  24. [40]

    Dynamic cheatsheet: Test-time learning with adaptive memory

    Mirac Suzgun, Mert Yuksekgonul, Federico Bianchi, Dan Jurafsky, and James Zou. Dynamic cheatsheet: Test-time learning with adaptive memory. In Proceedings of the 19th Conference of the European Chapter of the Association for Computational Linguistics (Volume 1: Long Papers), p...

    2026

  25. [41]

    Gepa: Reflective prompt evolution can outperform reinforcement learning

    Lakshya A Agrawal, Shangyin Tan, Dilara Soylu, Noah Ziems, Rishi Khare, Krista Opsahl-Ong, Arnav Singhvi, Herumb Shandilya, Michael J Ryan, Meng Jiang, et al. Gepa: Reflective prompt evolution can outperform reinforcement learning. In First Workshop on Foundations of Reasoning...

  26. [42]

    Algorithm discovery with llms: Evolutionary search meets reinforcement learning

    Anja S urina, Amin Mansouri, Lars CPM Quaedvlieg, Amal Seddas, Maryna Viazovska, Emmanuel Abbe, and Caglar Gulcehre. Algorithm discovery with llms: Evolutionary search meets reinforcement learning. In Second Conference on Language Modeling

  27. [45]

    Openevolve: an open-source evolutionary coding agent, 2025

    Asankhaya Sharma. Openevolve: an open-source evolutionary coding agent, 2025. URL https://github. com/codelion/openevolve, 5

    2025

  28. [47]

    Llm-sr: Scientific equation discovery via programming with large language models

    Parshin Shojaee, Kazem Meidani, Shashank Gupta, Amir Barati Farimani, and Chandan K Reddy. Llm-sr: Scientific equation discovery via programming with large language models. In The Thirteenth International Conference on Learning Representations

  29. [50]

    Nikhil Abhyankar, Sanchit Kabra, Saaketh Desai, and Chandan K. Reddy. LLEMA : Evolutionary search with LLM s for multi-objective materials discovery. In The Fourteenth International Conference on Learning Representations, 2026. URL https://openreview.net/forum?id=TIqzhBvCNB

    2026

  30. [51]

    Evolution of heuristics: Towards efficient automatic algorithm design using large language model

    Fei Liu, Tong Xialiang, Mingxuan Yuan, Xi Lin, Fu Luo, Zhenkun Wang, Zhichao Lu, and Qingfu Zhang. Evolution of heuristics: Towards efficient automatic algorithm design using large language model. In Forty-first International Conference on Machine Learning

  31. [52]

    Reevo: Large language models as hyper-heuristics with reflective evolution

    Haoran Ye, Jiarui Wang, Zhiguang Cao, Federico Berto, Chuanbo Hua, Haeyeon Kim, Jinkyoo Park, and Guojie Song. Reevo: Large language models as hyper-heuristics with reflective evolution. Advances in neural information processing systems, 37: 0 43571--43608, 2024 b

    2024

  32. [53]

    Evoagent: Towards automatic multi-agent generation via evolutionary algorithms

    Siyu Yuan, Kaitao Song, Jiangjie Chen, Xu Tan, Dongsheng Li, and Deqing Yang. Evoagent: Towards automatic multi-agent generation via evolutionary algorithms. In Proceedings of the 2025 Conference of the Nations of the Americas Chapter of the Association for Computational Lingu...

    2025

  33. [54]

    Agentsquare: Automatic llm agent search in modular design space

    Yu Shang, Yu Li, Keyu Zhao, Likai Ma, Jiahe Liu, Fengli Xu, and Yong Li. Agentsquare: Automatic llm agent search in modular design space. In The Thirteenth International Conference on Learning Representations

  34. [55]

    Findings of the Association for Computational Linguistics: NAACL 2025 , pages=

    Ferg-llm: Feature engineering by reason generation large language models , author=. Findings of the Association for Computational Linguistics: NAACL 2025 , pages=

    2025

  35. [56]

    International Conference on Learning Representations 2023 , year=

    TabPFN: A transformer that solves small tabular classification problems in a second , author=. International Conference on Learning Representations 2023 , year=

    2023

  36. [57]

    2025 , eprint=

    TabPFN-2.5: Advancing the State of the Art in Tabular Foundation Models , author=. 2025 , eprint=

    2025

  37. [58]

    TabPFN: A Transformer That Solves Small Tabular Classification Problems in a Second , author=

  38. [59]

    Joint European Conference on Machine Learning and Knowledge Discovery in Databases , pages=

    The autofeat Python Library for Automated Feature Engineering and Selection , author=. Joint European Conference on Machine Learning and Knowledge Discovery in Databases , pages=. 2019 , organization=

    2019

  39. [60]

    Proceedings of the Thirty-Fourth International Joint Conference on Artificial Intelligence , pages=

    Dynamic and adaptive feature generation with LLM , author=. Proceedings of the Thirty-Fourth International Joint Conference on Artificial Intelligence , pages=

  40. [61]

    arXiv preprint arXiv:2309.16609 , year=

    Qwen Technical Report , author=. arXiv preprint arXiv:2309.16609 , year=

    arXiv

  41. [62]

    Proceedings of the AAAI conference on artificial intelligence , volume=

    Evolutionary large language model for automated feature transformation , author=. Proceedings of the AAAI conference on artificial intelligence , volume=

  42. [63]

    2018 , publisher=

    Feature engineering for machine learning and data analytics , author=. 2018 , publisher=

    2018

  43. [64]

    arXiv preprint arXiv:2510.25223 , year=

    FELA: A Multi-Agent Evolutionary System for Feature Engineering of Industrial Event Log Data , author=. arXiv preprint arXiv:2510.25223 , year=

    arXiv

  44. [65]

    arXiv preprint arXiv:2602.17641 , year=

    FAMOSE: A ReAct Approach to Automated Feature Discovery , author=. arXiv preprint arXiv:2602.17641 , year=

  45. [66]

    Workshop on automatic machine learning , pages=

    TPOT: A tree-based pipeline optimization tool for automating machine learning , author=. Workshop on automatic machine learning , pages=. 2016 , organization=

    2016

  46. [67]

    International Conference on Machine Learning , pages=

    Openfe: Automated feature generation with expert-level performance , author=. International Conference on Machine Learning , pages=. 2023 , organization=

    2023

  47. [68]

    AgentSquare: Automatic LLM Agent Search in Modular Design Space , author=

  48. [69]

    Evoagent: Towards automatic multi-agent generation via evolutionary algorithms , author=. Proceedings of the 2025 Conference of the Nations of the Americas Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 1: Long Papers) , pages=

    2025

  49. [70]

    Advances in neural information processing systems , volume=

    Reevo: Large language models as hyper-heuristics with reflective evolution , author=. Advances in neural information processing systems , volume=

  50. [71]

    Evolution of Heuristics: Towards Efficient Automatic Algorithm Design Using Large Language Model , author=

  51. [72]

    Transformers Can Do Bayesian Inference , author=

  52. [73]

    arXiv preprint arXiv:2510.14150 , year=

    Codeevolve: An open source evolutionary coding agent for algorithm discovery and optimization , author=. arXiv preprint arXiv:2510.14150 , year=

    arXiv

  53. [74]

    GEPA: Reflective Prompt Evolution Can Outperform Reinforcement Learning , author=

  54. [75]

    arXiv preprint arXiv:2509.19349 , year=

    Shinkaevolve: Towards open-ended and sample-efficient program evolution , author=. arXiv preprint arXiv:2509.19349 , year=

    arXiv

  55. [76]

    URL https://github

    Openevolve: an open-source evolutionary coding agent, 2025 , author=. URL https://github. com/codelion/openevolve , volume=

    2025

  56. [77]

    arXiv preprint arXiv:2506.13131 , year=

    Alphaevolve: A coding agent for scientific and algorithmic discovery , author=. arXiv preprint arXiv:2506.13131 , year=

    arXiv

  57. [78]

    Promptbreeder: Self-Referential Self-Improvement via Prompt Evolution , author=

  58. [79]

    Proceedings of the 19th Conference of the European Chapter of the Association for Computational Linguistics (Volume 1: Long Papers) , pages=

    Dynamic cheatsheet: Test-time learning with adaptive memory , author=. Proceedings of the 19th Conference of the European Chapter of the Association for Computational Linguistics (Volume 1: Long Papers) , pages=

  59. [80]

    Reddy , booktitle=

    Nikhil Abhyankar and Sanchit Kabra and Saaketh Desai and Chandan K. Reddy , booktitle=. 2026 , url=

    2026

  60. [81]

    arXiv preprint arXiv:2506.15196 , year=

    Heuragenix: Leveraging llms for solving complex combinatorial optimization challenges , author=. arXiv preprint arXiv:2506.15196 , year=

    arXiv

  61. [82]

    arXiv preprint arXiv:2603.24382 , year=

    MolEvolve: LLM-Guided Evolutionary Search for Interpretable Molecular Optimization , author=. arXiv preprint arXiv:2603.24382 , year=

  62. [83]

    arXiv preprint arXiv:2503.21155 , year=

    Embedding domain-specific knowledge from LLMs into the feature engineering pipeline , author=. arXiv preprint arXiv:2503.21155 , year=

    arXiv

  63. [84]

    arXiv e-prints , pages=

    Evoprompt: Connecting llms with evolutionary algorithms yields powerful prompt optimizers , author=. arXiv e-prints , pages=

  64. [85]

    LLM-SR: Scientific Equation Discovery via Programming with Large Language Models , author=

  65. [86]

    Algorithm Discovery With LLMs: Evolutionary Search Meets Reinforcement Learning , author=

  66. [87]

    International Workshop on Causality, Agents and Large Models , pages=

    Feature Generation Using LLMs: An Evolutionary Algorithm Approach , author=. International Workshop on Causality, Agents and Large Models , pages=. 2024 , organization=

    2024

  67. [88]

    International Conference on Machine Learning , pages=

    TabICL: A Tabular Foundation Model for In-Context Learning on Large Data , author=. International Conference on Machine Learning , pages=. 2025 , organization=

    2025

  68. [89]

    Exploring LLM Agents for Cleaning Tabular Machine Learning Datasets , author=

  69. [90]

    arXiv preprint arXiv:2302.02041 , year=

    REaLTabFormer: Generating Realistic Relational and Tabular Data using Transformers , author=. arXiv preprint arXiv:2302.02041 , year=

    arXiv

  70. [91]

    Language Models are Realistic Tabular Data Generators , author=

  71. [92]

    Making Pre-trained Language Models Great on Tabular Prediction , author=

  72. [93]

    International Conference on Machine Learning , pages=

    CARTE: Pretraining and Transfer for Tabular Learning , author=. International Conference on Machine Learning , pages=. 2024 , organization=

    2024

  73. [94]

    Proceedings of the ACM Web Conference 2024 , pages=

    Towards cross-table masked pretraining for web data mining , author=. Proceedings of the ACM Web Conference 2024 , pages=

    2024

  74. [95]

    Alan Arazi and Eilam Shapira and Roi Reichart , booktitle=. Tab. 2026 , url=

    2026

  75. [96]

    Proceedings of the 2023 conference on empirical methods in natural language processing , pages=

    Generative table pre-training empowers models for tabular prediction , author=. Proceedings of the 2023 conference on empirical methods in natural language processing , pages=

    2023

  76. [97]

    Advances in Neural Information Processing Systems , volume=

    Large scale transfer learning for tabular data via language modeling , author=. Advances in Neural Information Processing Systems , volume=

  77. [98]

    Towards Foundation Models for Learning on Tabular Data, October 2023 , author=

    2023

  78. [99]

    Advances in neural information processing systems , volume=

    Optimized feature generation for tabular data via llms with decision tree reasoning , author=. Advances in neural information processing systems , volume=

  79. [100]

    International conference on artificial intelligence and statistics , pages=

    Tabllm: Few-shot classification of tabular data with large language models , author=. International conference on artificial intelligence and statistics , pages=. 2023 , organization=

    2023

  80. [101]

    Advances in Neural Information Processing Systems , volume=

    Lift: Language-interfaced fine-tuning for non-language machine learning tasks , author=. Advances in Neural Information Processing Systems , volume=

  81. [102]

    arXiv preprint arXiv:2505.09388 , year=

    Qwen3 technical report , author=. arXiv preprint arXiv:2505.09388 , year=

    arXiv

  82. [103]

    International conference on machine learning , pages=

    Batch normalization: Accelerating deep network training by reducing internal covariate shift , author=. International conference on machine learning , pages=. 2015 , organization=

    2015

  83. [104]

    2017 , publisher=

    Multilayer perceptron (MLP) , booktitle=. 2017 , publisher=

    2017

  84. [105]

    Advances in neural information processing systems , volume=

    CatBoost: unbiased boosting with categorical features , author=. Advances in neural information processing systems , volume=

  85. [106]

    5-coder technical report , author=

    Qwen2. 5-coder technical report , author=. arXiv preprint arXiv:2409.12186 , year=

    arXiv

  86. [107]

    arXiv preprint arXiv:2410.21276 , year=

    Gpt-4o system card , author=. arXiv preprint arXiv:2410.21276 , year=

    arXiv

  87. [108]

    2007 , publisher=

    UCI machine learning repository , author=. 2007 , publisher=

    2007

  88. [109]

    arXiv preprint arXiv:2503.14434 , year=

    Llm-fe: Automated feature engineering for tabular data with llms as evolutionary optimizers , author=. arXiv preprint arXiv:2503.14434 , year=

    arXiv

  89. [110]

    ACM SIGKDD Explorations Newsletter , volume=

    OpenML: networked science in machine learning , author=. ACM SIGKDD Explorations Newsletter , volume=. 2014 , publisher=

    2014

  90. [111]

    Nature , year=

    Accurate predictions on small data with a tabular foundation model , author=. Nature , year=. doi:10.1038/s41586-024-08328-6 , publisher=

    doi

  91. [112]

    Forty-first International Conference on Machine Learning , year=

    Large language models can automatically engineer features for few-shot tabular learning , author=. Forty-first International Conference on Machine Learning , year=

  92. [113]

    Cresswell and Keyvan Golestan and Guangwei Yu and Anthony L

    Junwei Ma and Valentin Thomas and Rasa Hosseinzadeh and Alex Labach and Hamidreza Kamkari and Jesse C. Cresswell and Keyvan Golestan and Guangwei Yu and Anthony L. Caterini and Maksims Volkovs , booktitle=. Tab. 2025 , url=

    2025

  93. [114]

    Advances in Neural Information Processing Systems , volume=

    Large language models for automated data science: Introducing caafe for context-aware automated feature engineering , author=. Advances in Neural Information Processing Systems , volume=

Pith tools

Reviewed July 31, 2026 · model on record in the stance chip above.