Pith. sign in

REVIEW 8 cited by

Optimized Feature Generation for Tabular Data via LLMs with Decision Tree Reasoning

Not yet reviewed by Pith; the record is open.

Re-run · record.json Download PDF Read on arXiv ↗

This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.

SPECIMEN: schema-true, not a live event

T0 review · schema-true

One-sentence machine reading of the paper's core claim.

pith:XXXXXXXX · record.json · timestamp

arxiv 2406.08527 v2 pith:BCF5GBVQ submitted 2024-06-12 cs.LG cs.AI

Jaehyun Nam , Kyuyoung Kim , Seunghyuk Oh , Jihoon Tack , Jaehyung Kim , Jinwoo Shin This is my paper · ORCID
classification cs.LGcs.AI
keywords featurereasoningllmsdecisionengineeringexperimentsmodelsoctree
verification ladder T0 review T1 audit T2 compute T3 formal

Signed reviews

No signed human review yet.

0 comments
read the original abstract

In tabular prediction tasks, tree-based models combined with automated feature engineering methods often outperform deep learning approaches that rely on learned representations. While these feature engineering techniques are effective, they typically depend on a pre-defined search space and primarily use validation scores for feature selection, thereby missing valuable insights from previous experiments. To address these limitations, we propose a novel tabular learning framework that utilizes large language models (LLMs), termed Optimizing Column feature generator with decision Tree reasoning (OCTree). Our key idea is to leverage the reasoning capabilities of LLMs to identify effective feature generation rules without manually specifying the search space and provide language-based reasoning information highlighting past experiments as feedback for iterative rule improvements. We use decision trees to convey this reasoning information, as they can be easily represented in natural language, effectively providing knowledge from prior experiments (i.e., the impact of the generated features on performance) to the LLMs. Our empirical results demonstrate that OCTree consistently enhances the performance of various prediction models across diverse benchmarks, outperforming competing automated feature engineering methods. Code is available at https://github.com/jaehyun513/OCTree.

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 8 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. When Do Corrective Features Help? An Agent for Corrective Feature Discovery on Black-Box Forecasters

    cs.LG 2026-08 conditional novelty 6.0 of 10

    CRAFTER corrects frozen time-series forecasters by mining residual features with an MCTS search and an LLM, gating candidates by validation error, and fitting a small corrector; gains concentrate on weak backbones wit...

  2. DeepFeature: LLM-Empowered Context-aware Feature Generation for Wearable Biosignals

    cs.AI 2025-12 conditional novelty 6.0 of 10

    An LLM-based pipeline that generates context-aware features from raw wearable biosignals and iteratively refines them via performance feedback reports AUROC gains of 4-10% over baselines.

  3. SIFOTL: A Principled, Statistically-Informed Fidelity-Optimization Method for Tabular Learning

    cs.LG 2025-07 reject novelty 6.0 of 10

    A privacy-aware method combining LLM feature synthesis, twin XGBoost models, and a Pareto-weighted tree identifies shift-driving segments with F1 up to 0.96 on synthetic EHR data.

  4. A Survey on Evaluating Quality and Trustworthiness in LLM-Generated Data

    cs.AI 2026-01 conditional novelty 5.0 of 10

    A metric-oriented survey that classifies intrinsic quality and trustworthiness metrics for LLM-generated data across six modalities and documents systematic evaluation gaps in the current literature.

  5. Research Challenges in Relational Database Management Systems for LLM Queries

    cs.DB 2025-08 conditional novelty 5.0 of 10

    Open-source SQL-plus-LLM database integrations fail on most representative LLM queries and run orders of magnitude slower than enterprise APIs, due to missing structured outputs, poor batching, and plan-unaware LLM calls.

  6. Transformers Boost the Performance of Decision Trees on Tabular Data across Sample Sizes

    cs.CL 2025-02 conditional novelty 5.0 of 10

    Seeding gradient boosted trees with scaled transformer scores (LLM-Boost and PFN-Boost) improves average tabular classification performance over either component alone for most tested sample sizes.

  7. GPT-HTree: A Decision Tree Framework Integrating Hierarchical Clustering and Large Language Models for Explainable Classification

    cs.LG 2025-01 reject novelty 4.0 of 10

    GPT-HTree combines hierarchical clustering, per-cluster decision trees, and LLM-generated persona descriptions, and claims to identify VC founder clusters with up to 9x success rates, but the claim is not validated ou...

  8. Revisiting Data Analysis with Pre-trained Foundation Models

    cs.DB 2025-01 conditional novelty 3.0 of 10

    A comprehensive review of using pre-trained foundation models for data analysis, with a taxonomy of tasks, challenges, and a research roadmap.

Pith tools