Evangelidis91/llm-feature-engineering

This repository contains an empirical study that measures whether LLM-suggested features improve tabular ML models. It evaluates 10 LLMs (two generations) across 3 datasets, 2 prompt styles (including “with-stats”), and 3 model families (logistic/linear, random forest, XGBoost), using 5-fold CV and paired statistical tests. Primary artifacts include metrics_full.csv (ground-truth per-experiment scores), llm_suggestions.json (parsed feature proposals), raw LLM outputs, per-call cost/latency logs, validity rates, and figures for reproducibility. Key findings: frontier-tier and production-tier LLMs deliver statistically similar downstream performance; cost and validity diverge (e.g., GPT-5.5 had 100% validity but high cost and only moderate usefulness); Gemini 3.1 Pro had the highest win rate (most useful features); tree models (Random Forest, XGBoost) benefit most from LLM features; weaker models improve with column-stat prompts. The study catalogs common failure modes (hallucinated columns, numeric instability, type errors, stylistic verbosity) and emphasizes value-oriented evaluation (usefulness vs. code correctness). Intended users are ML practitioners and researchers exploring LLM-assisted feature engineering.

AI-related keywords found in this repo's description, topics, or README summary — grouped by category. Each badge links to the corresponding ranking detail page.