REVIEW 6 cited by
LLM-Select: Feature Selection with Large Language Models
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
LLM-Select: Feature Selection with Large Language Models
read the original abstract
In this paper, we demonstrate a surprising capability of large language models (LLMs): given only input feature names and a description of a prediction task, they are capable of selecting the most predictive features, with performance rivaling the standard tools of data science. Remarkably, these models exhibit this capacity across various query mechanisms. For example, we zero-shot prompt an LLM to output a numerical importance score for a feature (e.g., "blood pressure") in predicting an outcome of interest (e.g., "heart failure"), with no additional context. In particular, we find that the latest models, such as GPT-4, can consistently identify the most predictive features regardless of the query mechanism and across various prompting strategies. We illustrate these findings through extensive experiments on real-world data, where we show that LLM-based feature selection consistently achieves strong performance competitive with data-driven methods such as the LASSO, despite never having looked at the downstream training data. Our findings suggest that LLMs may be useful not only for selecting the best features for training but also for deciding which features to collect in the first place. This could benefit practitioners in domains like healthcare and the social sciences, where collecting high-quality data comes at a high cost.
Forward citations
Cited by 6 Pith papers
-
Memory-Augmented LLM-based Multi-Agent System for Automated Feature Generation on Tabular Data
MALMAS is a memory-augmented multi-agent LLM system that generates diverse, high-quality features for tabular data via agent decomposition, routing, and iterative memory-guided refinement.
-
Mamba-SSM with LLM Reasoning for Feature Selection: Faithfulness-Aware Biomarker Discovery
LLM chain-of-thought filtering of Mamba saliency features on TCGA-BRCA data produces a 17-gene set with AUC 0.927 that beats both the raw 50-gene saliency list and a 5000-gene baseline while using far fewer features, ...
-
DeepFeature: LLM-Empowered Context-aware Feature Generation for Wearable Biosignals
An LLM-based pipeline that generates context-aware features from raw wearable biosignals and iteratively refines them via performance feedback reports AUROC gains of 4-10% over baselines.
-
Can Explanations Improve Recommendations? Evidence from Prediction-Informed Explanations
RecPIE jointly optimizes recommendation predictions and LLM-generated natural-language explanations via alternating training and reinforcement learning, yielding 3-4% accuracy gains and higher human preference on Goog...
-
Feature Generation Using LLMs: An Evolutionary Algorithm Approach
A funsearch-style evolutionary loop using LLaMA-3.1 7B-generated Python expressions creates new table features and improves F1 in 13 of 16 evaluated classification settings.
-
LLM-FS: Zero-Shot Feature Selection for Effective and Interpretable Malware Detection
Zero-shot LLM-based feature selection matches traditional statistical feature selection in accuracy on the EMBOD malware dataset while offering interpretability and stability advantages.
discussion (0)
Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.