RECORD
Deep Learning to Jointly Schema Match, Impute, and Transform Databases
Databases · Submitted 22 Jun 2022
abstract compiled · 2026-09-23 · no language-model review
The authors' main claim In synthetic and real-world experiments using two electronic health record databases, our algorithms outperform existing baselines for matching variable sets, while jointly learning to impute unshared or transformed variables.
The compiled record
The paper's own sentences, arranged. Each label is drawn from a closed vocabulary; the number beside it is the probability the classifier gave that label. Numbers in the tables are the authors', checked by code. No sentence in this record was written by a model.
Open the compiled record
The abstract, sentence by sentence
An applied problem facing all areas of data science is harmonizing data sources. Joining data from multiple origins with unmapped and only partially overlapping features is a prerequisite to developing and testing robust, generalizable algorithms, especially in health care. We approach this issue in the common but difficult case of numeric features such as nearly Gaussian and binary features, where unit changes and variable shift make simple matching of univariate summaries unsuccessful. We develop two novel procedures to address this problem. First, we demonstrate multiple methods of "fingerprinting" a feature based on its associations to other features. In the setting of even modest prior information, this allows most shared features to be accurately identified. Second, we demonstrate a deep learning algorithm for translation between databases. Unlike prior approaches, our algorithm takes advantage of discovered mappings while identifying surrogates for unshared features and learning transformations. In synthetic and real-world experiments using two electronic health record databases, our algorithms outperform existing baselines for matching variable sets, while jointly learning to impute unshared or transformed variables.
background contribution method result interpretation main claim
| # | sentence | role | the authors' own work | states the main conclusion | strength |
|---|---|---|---|---|---|
| s0 | An applied problem facing all areas of data science is harmonizing data sources. | background 1.00 | 0.04 | 0.02 | asserted 1.00 |
| s1 | Joining data from multiple origins with unmapped and only partially overlapping features is a prerequisite to developing and testing robust, generalizable algorithms, especially in health care. | background 1.00 | 0.05 | 0.03 | asserted 1.00 |
| s2 | We approach this issue in the common but difficult case of numeric features such as nearly Gaussian and binary features, where unit changes and variable shift make simple matching of univariate summaries unsuccessful. | background 0.86 | 0.12 | 0.02 | asserted 1.00 |
| s3 | We develop two novel procedures to address this problem. | contribution 0.99 | 0.87 | 0.04 | asserted 0.98 |
| s4 | First, we demonstrate multiple methods of "fingerprinting" a feature based on its associations to other features. | method 0.74 | 0.93 | 0.03 | asserted 1.00 |
| s5 | In the setting of even modest prior information, this allows most shared features to be accurately identified. | result 0.97 | 0.88 | 0.08 | asserted 0.94 |
| s6 | Second, we demonstrate a deep learning algorithm for translation between databases. | method 0.52 | 0.93 | 0.04 | asserted 1.00 |
| s7 | Unlike prior approaches, our algorithm takes advantage of discovered mappings while identifying surrogates for unshared features and learning transformations. | method 0.66 | 0.85 | 0.09 | asserted 1.00 |
| s8 | In synthetic and real-world experiments using two electronic health record databases, our algorithms outperform existing baselines for matching variable sets, while jointly learning to impute unshared or transformed variables. | result 1.00 | 0.97 | 0.86 | asserted 1.00 |
What was examined, and what was not
- arXiv listing abstract v1
- full text
- figures and tables
- references and citing sentences
- relations to other papers
- P abstract
- S sections
- T body sentences
- C citations
- R cross-paper relations
The paper
Original abstract
An applied problem facing all areas of data science is harmonizing data sources. Joining data from multiple origins with unmapped and only partially overlapping features is a prerequisite to developing and testing robust, generalizable algorithms, especially in health care. We approach this issue in the common but difficult case of numeric features such as nearly Gaussian and binary features, where unit changes and variable shift make simple matching of univariate summaries unsuccessful. We develop two novel procedures to address this problem. First, we demonstrate multiple methods of "fingerprinting" a feature based on its associations to other features. In the setting of even modest prior information, this allows most shared features to be accurately identified. Second, we demonstrate a deep learning algorithm for translation between databases. Unlike prior approaches, our algorithm takes advantage of discovered mappings while identifying surrogates for unshared features and learning transformations. In synthetic and real-world experiments using two electronic health record databases, our algorithms outperform existing baselines for matching variable sets, while jointly learning to impute unshared or transformed variables.
Human review
How the literature uses it
What this paper does with the arXiv papers it cites
What it cites · 2 sentences without a confident label
Cites as background 1 paper
-
For quantitative data, simple characteristics such as the mean, variance, mode, and fraction missing are commonly used [ref] .
Schema Matching using Machine Learning · Related work · 0.83
Discussion
Know something about this paper? Leave a comment, or highlight any passage above to comment on it.