FlowETL uses LLMs and a small target dataset to automatically infer and apply data-cleaning transformations, reporting high data-quality scores across 14 datasets.
Data Engineering for Data Analytics: A Classification of the Issues, and Case Studies
1 Pith paper cite this work. Polarity classification is still indexing.
1
Pith paper citing it
abstract
Consider the situation where a data analyst wishes to carry out an analysis on a given dataset. It is widely recognized that most of the analyst's time will be taken up with \emph{data engineering} tasks such as acquiring, understanding, cleaning and preparing the data. In this paper we provide a description and classification of such tasks into high-levels groups, namely data organization, data quality and feature engineering. We also make available four datasets and example analyses that exhibit a wide variety of these problems, to help encourage the development of tools and techniques to help reduce this burden and push forward research towards the automation or semi-automation of the data engineering process.
citation-role summary
background 1
citation-polarity summary
fields
cs.SE 1years
2025 1verdicts
CONDITIONAL 1roles
background 1polarities
unclear 1representative citing papers
citing papers explorer
-
FlowETL: An Autonomous Example-Driven Pipeline for Data Engineering
FlowETL uses LLMs and a small target dataset to automatically infer and apply data-cleaning transformations, reporting high data-quality scores across 14 datasets.