Pith. sign in

REVIEW 3 major objections 4 minor 31 references

Metadata Reconstruction from Values Alone: Recovering Column Semantics in Undocumented Warehouses

T0 review · 3 major / 4 minor · reviewed 2026-08-12 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read A verification harness can recover column semantics from values alone, because it declines to guess without grounded evidence.

desk verdict A scrupulously measured system paper: the selection-over-prose finding holds, but the abstract over-credits the harness on ICD-9 decoding and the 'values alone' framing is vulnerable to a memorization confound that needs a contamination control. read the letter →

arxiv 2608.07946 v1 pith:JCLH7FLH submitted 2026-08-08 cs.DB cs.AI

Mike Helwig This is my paper · ORCID
classification cs.DBcs.AI
keywords metadatareconstructiontext-to-SQLabstentionselectivepredictioncolumnsemanticsdataprofilingverificationharnessschemaopacity
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper attacks the regime where text-to-SQL benchmarks do not: production warehouses with cryptic column identifiers and missing documentation. It claims that a column's meaning can be recovered from its values alone, provided a language model is placed inside a verification harness: a deterministic profiler decides whether the system speaks, and the model writes only when the evidence licenses it. Measured against human-authored documentation, the harness recovers more of what authors wrote (0.475 accurate on the 42% of columns it commits to, against 0.223 on 94% for the same model used directly), but the paper shows the gain is selection, not better prose. On the reconstructed catalog, the system answers selectively with calibrated abstention, converting a naive model's confident silent errors into refusals. If right, this reframes the bottleneck in querying undocumented warehouses as metadata quality rather than translation quality.

What carries the argument

The central object is the verification harness: a deterministic profiler extracts structural evidence (value fingerprints, a 26-pattern library, checksum verdicts), a language model proposes semantics conditioned on that evidence, and every fact carries provenance with a confidence capped by its evidence class. A code-enforced commit gate refuses prose claims whose grounding tier is too weak, and at query time the route (answer / confirm-first / refuse) is pure thresholded arithmetic over a feature vector in which the LLM contributes features but never decides the route.

What would settle it

Run the reconstruction comparison on a private, never-published warehouse, or on schemas constructed after the model's training cutoff, with identifiers destroyed and documentation withheld; if the harness no longer beats the direct model on recall-when-claimed, or if coverage no longer tracks deterministic evidence, the 'from values alone' claim is falsified.

Watch

Extended reading notes

Core claim

The paper claims that when a column is stripped of its name and documentation, the Rosetta architecture recovers more of what human authors wrote than either a statistical semantic-type detector or the same language model used directly, because it is the only one of the three that declines when it cannot ground an answer, and because that declining is driven by deterministic evidence rather than by the model's self-report. The paper is explicit that the advantage is selection, not prose: on the 283 columns where both arms speak, the harness writes no better metadata than the model alone, and on two facets measurably worse. Separately, over the reconstructed catalog, the system delivers calibrated abstention at query time, converting a naive model's confident silent errors into auditable refusals, with coverage that tracks difficulty. The paper also reports, at length, that its own authority ladder is not the mechanism behind the abstention; execution grounding and the deterministic threshold are.

Load-bearing premise

The headline results are measured on public benchmark data that a frontier language model may have memorized, so the claim that meanings are recovered 'from values alone' depends on the model not recalling the documentation or ontology mappings instead of deriving them from the values.

Editorial extensions

If this is right

  • Undocumented warehouses can be bootstrapped into query-answerable catalogs without human schema curation, with each reconstructed fact carrying an audit trail.
  • Deploying a naive text-to-SQL system on an opaque schema should be expected to fail silently on a majority of questions; an evidence-gated pipeline converts those silent errors into refusals.
  • The measurable value of a language-model harness in this setting is selecting which columns to describe, not writing better descriptions.
  • Abstention must be enforced in code rather than requested in a prompt, since prompt-requested abstention transfers poorly across model families.
  • A catalog that is 0.475 accurate on 42% of columns is more operationally useful than one that is 0.223 accurate on 94%, because the second kind cannot be trusted anywhere.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • If selection is the real gain, cheaper deterministic competence detectors might capture much of the benefit without an LLM; the paper's own evidence-split on BIRD columns suggests a testable extension of this kind.
  • The code-enforced commit gate result implies that any system claiming selective answering should register its refusals in code, not in prompt language, because model disposition varies across backbones.
  • A testable extension would apply the same value-decoding protocol to proprietary, registry-assigned code systems, where no world knowledge exists; the paper's NDC result predicts the gate abstains on all such keys.
  • The finding that execution grounding carries the routing signal suggests that query-time safety may be improved more by execution verification and result-sanity checks than by richer metadata confidence scores.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 4 minor

Summary. The paper proposes Rosetta, a system that places an LLM inside a deterministic verification harness to reconstruct column semantics (expanded names, descriptions, value decodes) from values alone, and then uses the reconstructed catalog for calibrated abstention in text-to-SQL. It evaluates against BIRD human documentation with identifiers destroyed, against a blind i2b2/Synthea warehouse for ICD-9 decoding, and against Spider for query-time routing, comparing a statistical arm, an LLM-direct arm, and the full harness. The headline result is a selection effect: Rosetta commits to 42% of columns at 0.475 recall-when-claimed versus 0.223 on 94% for the LLM-direct baseline, with no prose advantage on the paired common ground, and a code-enforced gate removes ungrounded claims. The paper also reports backbone-swap results, a validated lexical metric, an honest set of negative results, and a frozen experimental record.

Significance. If the attribution holds, the contribution is significant: a competence detector that converts a naive model's confident silent errors into auditable refusals, with label-free confidence and a deterministic route decision. The paper is unusually careful methodologically: external ground truth, cluster bootstraps over databases, a blinded judge, a second rater, backbone swaps, registered predictions, held-out databases, and explicit negative results are all strengths that go well beyond typical practice. However, two load-bearing issues need to be addressed before the central claims can be accepted at face value: possible memorization of public benchmarks undermines the 'from values alone' attribution, and the paper's own ablation and backbone-swap results contradict the claimed mechanism for abstention in the default configuration.

major comments (3)
  1. [5.2, 5.4.1, 5.2.10] The central claim that semantics are recovered 'from values alone' is not protected against the model's memorization of public benchmarks. BIRD mini-dev and i2b2/Synthea are public datasets, and Gemini 3.1 Pro is a frontier model whose training data plausibly includes these schemas, their human-authored documentation, and widely republished ICD-9 code-to-meaning tables. The only contamination caveat in the paper appears in Section 5.2.10, and it is applied to the names-visible join experiment ('these are public benchmark schemas a frontier model has plausibly seen in training'), not to the headline reconstruction study of Section 5.2 or the ICD-9 decode of Section 5.4.1. The prefix-stripping control (0.745 vs 0.740) rules out reading the answer off the 'ICD9:' prefix, but it does not rule out recalling memorized code-to-meaning tables; the identifier-destruction control only rules out use of column names. The held-out BIRD train databases in Section 5.2.8 are still public BIRD data. Since an ICD-9 code such as '493.90' contains no information about asthma, the 95.5% decode rate must be substantially attributed to parametric knowledge rather than to inference from the values. This does not necessarily destroy the relative selection advantage over an identical model used directly, but it weakens the 'from values alone' framing and the absolute recovery numbers. A concrete remedy would be an evaluation on a non-public corpus with novel value-to-meaning mappings, or at minimum an explicit disclosure that the headline results measure recovery under possible training-data contamination.
  2. [1, 5.2.1, 5.2.8, 5.8.1] The paper's central claim states that the system's declining behavior is 'driven by the deterministic evidence rather than by the model's self-report' (Section 1 and the abstract). The paper's own measurements contradict this for the default configuration. Section 5.2.8 reports that on Claude Sonnet 4.6 arm C's coverage rises from 0.422 to 0.823 and the evidence-tracking gap is no longer detectable (-0.089, interval spanning zero); Section 5.2.9 shows that all three abstention behaviors (column, table, value) collapse on a second backbone because abstention is requested in a prompt rather than enforced in code. Section 5.8.1 further shows that authority features add no incremental discrimination and that the ladder caps rarely bind. The code-enforced gate of Section 5.2.8 does make no-evidence coverage 0.000, but this is by construction rather than a measured property of the deterministic evidence, and it was implemented after the fact, even if under a registered protocol. The central claim should be restated so that the default configuration is described as model-dependent selection and the gated configuration as enforced abstention; the current phrasing overstates what is established.
  3. [Abstract, 5.4.1] The abstract credits Rosetta with decoding 95.5% of ICD-9 codes from values alone and abstaining on all 44 NDC drug codes. Section 5.4.1 states that 'the harness adds nothing here': the plain model achieves the same 0.745 overall decode rate and also declines all 44 NDC codes on the same backbone, and the abstention in the default configuration is prompt-requested rather than code-enforced. The headline therefore attributes to the architecture a behavior that the paper's own measurements attribute to the backbone. The abstract and the contribution list should be rephrased to credit the model for the decoding and to credit the harness only for provenance and the audit trail, or the gate-enabled variant should be used if the NDC refusal is to be presented as a system property.
minor comments (4)
  1. [5.2.10 (Table)] The destroyed-condition row in the join-graph table contains the slash-separated entry 'stems / naked model — / 0.000', which is confusing to read as a single cell; separate rows or explicit column headers would make the 0.000 precision/recall for both proposers clearer.
  2. [Abstract and 5.2] The abstract says 799 columns and Section 5.2 reports 680 paired columns across eleven databases; the relationship between the full mini-dev set and the paired subset is explained, but the abstract's '680 paired columns' should make clear that recovery numbers are conditional on the paired subset, since the unpaired columns affect coverage denominators.
  3. [5.3] The second-rater reliability check was performed on the ten-database judging round and not on the eleven-database round reported in the main text; this is disclosed, but the main-text presentation should state the limitation before the kappa value is quoted, not only in the subsequent paragraph.
  4. [5.2.8] The gate-ON no-evidence coverage of 0.000 is described as 'by construction,' which is correct; however, the phrase 'recall-when-claimed never degrades' in the same subsection could be misread as a general empirical guarantee when the measured differences are flat on Gemini and held-out data and positive only on the two speculating backbones.

Circularity Check

0 steps flagged · score 2.0 of 10

No circular derivation: the primary results are measured against external human-authored documentation and labels; the lone self-citation is a companion methodology paper that is not load-bearing.

full rationale

The paper's central derivation chain is self-contained against external benchmarks. The reconstruction claim (Section 5.2) is scored against BIRD's human-written column documentation after identifiers are destroyed; the three arms (statistical, LLM-direct, Rosetta) receive identical mangled inputs, and the paired common-ground analysis (Section 5.2.1) explicitly isolates selection from prose quality and reports the prose result as a null. The query-time abstention claim (Sections 5.1, 5.6, 5.7) is evaluated on 1,082 execution-labeled Spider questions and BIRD full-strip questions, with the isotonic calibrator fit out-of-fold, database-clustered intervals, and both a naive always-answerer and a naive-plus-abstention baseline. The authority ladder is not used as a self-justifying premise: Sections 5.8 and 5.8.1 report three independent analyses showing the ladder is not the mechanism, and Section 5.2.8 reports that the evidence-tracking selection behavior does not transfer to a second backbone, correcting the earlier claim. The code-enforced gate's no-evidence coverage of 0.000 is explicitly labelled by the authors as an enforced property rather than a measured finding: 'With the gate on, coverage on no-evidence columns is 0.000 on every backbone by construction. We registered before running that this half of the prediction would hold mechanically once the decision was in code, so we report it as an enforced property, not a finding.' This is therefore not a fitted parameter renamed as a prediction. The only self-citation is the companion methodology paper [12] in Section 4, cited as the place where design rules are distilled; every rule it refers to is code-verified and measured here, so the citation is not load-bearing. The memorization concern about public BIRD and i2b2 data is a data-contamination and external-validity risk rather than a circularity of derivation, and the paper itself discloses the contamination caveat for the names-visible join experiment (Section 5.2.10). No fitted parameter is renamed as a prediction, no uniqueness theorem is imported from the authors' prior work, and no known result is renamed as organization. Score 2 reflects only the presence of the minor companion self-citation; the derivation itself is not circular.

Assumptions & free parameters 6 free parameters · 4 assumptions · 2 invented entities

The paper's central claims rest on hand-chosen routing weights, thresholds, and tier caps, plus post-hoc isotonic calibration. The most load-bearing assumptions are that BIRD's documentation is unbiased ground truth, that mangled public schemas represent undocumented warehouses, and that the LLM is not simply recalling public benchmark content from training. The invented entities are internal system constructs with no independent falsifiable handle outside the paper.

free parameters (6)
  • L4 routing feature weights = execution_ok 0.20, link_calibrated_probability 0.20, catalog_entry_mean_confidence 0.18…
    Hand-pinned weights in prediction/layer4_result.py that determine the routing score. Section 4.3.
  • Route thresholds per query policy = DOCUMENTED_STRONG 0.60/0.20, AMBIGUOUS_DOC 0.60/0.05, CRYPTIC_LEGACY 0.65/0.20
    Thresholds in consumption/answer_formatter.py that convert the L4 score into answer, confirm-first, or refuse. Section 4.3.
  • Grounding-tier confidence caps = 0.50, 0.60, 0.70, 0.75, 0.85, 0.90, 0.95, 1.00 for tiers 0 through 7
    Hand-chosen caps in rosetta/grounding/tiers.py. Section 4.1. The paper's own ablation in Section 5.8.1 finds removing all caps changes one of 41 outcomes, so they are not the mechanism, but they are part of the shipped confidence accounting.
  • Pattern acceptance thresholds = checksum pass fraction 0.95, LLM verification regex fullmatch 0.90, regex-only cap 0.85, fingerprint cap 0.70
    Thresholds that decide whether a pattern or name proposal is accepted. Sections 4.2 and 5.2.3.
  • Isotonic calibration map = 51-point grid fitted to Spider execution labels
    Fit out-of-fold on 1,082 execution-labeled samples and used to produce calibrated probabilities, with ECE 0.026. This is a fitted post-hoc model, not a parameter-free derivation. Sections 4.3 and 5.1.
  • Auto-commit tier threshold = SIBLING_CONSISTENT, tier 3
    Determines which reconstructed facts commit to the catalog without human review. Section 4.4.
assumptions (4)
  • domain assumption BIRD's human-authored column documentation is an unbiased ground truth for column semantics.
    The headline reconstruction metrics score against BIRD's database_description CSV files. If that documentation is noisy, biased, or partly memorized by the model, the measured accuracy is not true semantic recovery. Section 5.2.
  • domain assumption A schema mangled at severity 1.0, with identifiers destroyed, faithfully represents an undocumented production warehouse.
    The mangler preserves values and data, but real warehouses have legacy quirks, stale documentation, and contradictions. The paper does not validate the mangled benchmark as a complete stand-in for production opacity. Section 5.1.
  • domain assumption Content-token recall and the LLM judge scores measure meaning recovery.
    Section 5.3 validates token recall against a blinded LLM judge with rho 0.642, but the judge is itself a model, the second rater is the author, and no human expert validation is reported.
  • domain assumption The LLM's outputs on public BIRD and i2b2 data are not substantially contaminated by memorized schemas or documentation.
    The paper only flags training-data contamination for the names-visible join experiment in Section 5.2.10. The primary column-description and ICD-9 decode results are equally exposed to benchmark memorization, yet this is not tested or controlled.
invented entities (2)
  • Grounding-tier ladder, a seven-rung authority ladder
    purpose: Caps fact confidence by evidence class and records provenance for every reconstructed fact.
    It is an internal construct of the system. The paper's own ablations in Sections 5.8 and 5.8.1 show it is not the mechanism behind delivered safety, and removing every cap changed one of 41 outcomes. No independent evidence outside the system validates the tier ordering.
  • Structural evidence instrument mapping profiler output to evidence tiers
    purpose: Deterministic mapper used by the commit gate to decide whether a prose claim is licensed.
    This is a design component. Its no-evidence coverage 0.000 under the gate is true by construction, not by measurement, and its coverage of dev columns, 50.3%, is a property of the instrument rather than an externally validated fact.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Metadata Reconstruction from Values Alone: Recovering Column Semantics in Undocumented Warehouses." pith.science (2026). https://pith.science/paper/JCLH7FLH

@misc{pith2026260807946,
  author       = {Pith},
  title        = {Pith review of: Metadata Reconstruction from Values Alone: Recovering Column Semantics in Undocumented Warehouses},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/JCLH7FLH}},
  note         = {Machine review of arXiv:2608.07946}
}
read the original abstract

Text-to-SQL benchmarks ship schemas whose column names already say what the columns mean. Production warehouses are the inverse: cryptic identifiers, partial or absent documentation. We address the problem they pose first: recovering what columns and values mean from the data itself. Rosetta places a language model inside a verification harness: a deterministic profiler extracts structural evidence (value fingerprints, a 26-pattern library, checksum verdicts), the model proposes semantics conditioned on that evidence, and every fact carries provenance and a confidence bounded by its evidence class. Against human documentation on 680 paired columns across eleven BIRD databases, identifiers destroyed, the harness delivers metadata that is 0.475 accurate on the 42% of columns it commits to, against 0.223 on 94% for the same model used directly. Restricted to the 283 columns where both arms speak, the harness writes no better prose than the model alone; the gain is selection: deterministic evidence governs whether the system speaks (coverage +0.257 [0.128, 0.378]), not how well. The deterministic layer is a competence detector, not a competence amplifier. A backbone swap bounds the claim: the prose finding reproduces, but prompt-requested abstention does not transfer; a code-enforced commit gate (predictions registered first; measured on a third backbone and held-out databases) makes no-evidence coverage 0.000 on every backbone. On a blind i2b2 clinical warehouse Rosetta decodes 95.5% of 134 real ICD-9 codes from values alone and abstains on all 44 NDC drug codes. The catalog supports calibrated abstention at query time: under full schema opacity a naive translator falls from 0.92 to 0.42 execution accuracy while our gate answers at 86% accuracy over 59% coverage. Negative results are reported plainly, including that our own authority ladder is not the mechanism behind the headline.

Figures

Figures reproduced from arXiv: 2608.07946 by the authors.

Figure 1
Figure 1. Calibrated abstention: as schema opacity rises and generation collapses, coverage collapses with it (left) while accuracy [PITH_FULL_IMAGE:figures/full_fig_p026_1.png] view at source ↗
Figure 4
Figure 4. Per-database accuracy-when-answered (the hon [PITH_FULL_IMAGE:figures/full_fig_p027_4.png] view at source ↗
Figure 3
Figure 3. The three-way gate is monotone in both confidence [PITH_FULL_IMAGE:figures/full_fig_p027_3.png] view at source ↗

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

31 extracted references · 8 canonical work pages

  1. [1]

    Ziawasch Abedjan, Lukasz Golab, and Felix Naumann. 2015. Profiling Relational Data: A Survey.The VLDB Journal24, 4 (2015), 557–581. doi:10.1007/s00778- 015-0389-y

    doi:10.1007/s00778- 2015

  2. [2]

    Kaiwen Chen, Yueting Chen, Xiaohui Yu, and Nick Koudas. 2025. Reliable Text- to-SQL with Adaptive Abstention.Proceedings of the ACM on Management of Data3, 1 (2025). arXiv:2501.10858 doi:10.1145/3709719

    arXiv 2025

  3. [3]

    C. K. Chow. 1970. On Optimum Recognition Error and Reject Tradeoff.IEEE Transactions on Information Theory16, 1 (1970), 41–46. doi: 10.1109/tit. 1970.1054406

    arXiv 1970

  4. [4]

    Ran El-Yaniv and Yair Wiener. 2011. Agnostic Selective Classification. InAdvances in Neural Information Processing Systems (NeurIPS)

    work page 2011

  5. [5]

    Raul Castro Fernandez, Ziawasch Abedjan, Famien Koko, Gina Yuan, Samuel Madden, and Michael Stonebraker. 2018. Aurum: A Data Discovery System. In Proceedings of the IEEE 34th International Conference on Data Engineering (ICDE). 1001–1012. doi:10.1109/ICDE.2018.00094

    arXiv 2018

  6. [6]

    Dawei Gao, Haibin Wang, Yaliang Li, Xiuyu Sun, Yichen Qian, Bolin Ding, and Jin- gren Zhou. 2024. Text-to-SQL Empowered by Large Language Models: A Bench- mark Evaluation.Proceedings of the VLDB Endowment(2024). arXiv:2308.15363

    arXiv 2024

  7. [7]

    Yonatan Geifman and Ran El-Yaniv. 2017. Selective Classification for Deep Neural Networks. InAdvances in Neural Information Processing Systems (NeurIPS). arXiv:1705.08500

    arXiv 2017

  8. [8]

    Yonatan Geifman and Ran El-Yaniv. 2019. SelectiveNet: A Deep Neural Net- work with an Integrated Reject Option. InProceedings of the 36th International Conference on Machine Learning (ICML). arXiv:1901.09192

    arXiv 2019

Show all 31 references
  1. [9]

    Weinberger

    Chuan Guo, Geoff Pleiss, Yu Sun, and Kilian Q. Weinberger. 2017. On Calibration of Modern Neural Networks. InProceedings of the 34th International Conference on Machine Learning (ICML). arXiv:1706.04599

    2017 arXiv

  2. [10]

    Jiaqi Guo, Zecheng Zhan, Yan Gao, Yan Xiao, Jian-Guang Lou, Ting Liu, and Dongmei Zhang. 2019. Towards Complex Text-to-SQL in Cross-Domain Database with Intermediate Representation. InProceedings of the 57th Annual Meeting of the Association for Computational Linguistics (ACL)...

    2019 arXiv

  3. [11]

    Noy, Christopher Olston, Neoklis Polyzotis, Sudip Roy, and Steven Euijong Whang

    Alon Halevy, Flip Korn, Natalya F. Noy, Christopher Olston, Neoklis Polyzotis, Sudip Roy, and Steven Euijong Whang. 2016. Goods: Organizing Google’s Datasets. InProceedings of the 2016 International Conference on Management of Data (SIGMOD). 795–806. doi:10.1145/2882903.2903730

    2016

  4. [12]

    Mike Helwig. 2026. Deterministic by Default: Design Principles for Language Models inside Verification Harnesses. Companion methodology paper. Artifact: https://github.com/mike-m6online/Rosetta

    2026

  5. [13]

    Bakker, Emanuel Zgraggen, Arvind Satyanarayan, Tim Kraska, Cagatay Demiralp, and Cesar Hidalgo

    Madelon Hulsebos, Kevin Zeng Hu, Michiel A. Bakker, Emanuel Zgraggen, Arvind Satyanarayan, Tim Kraska, Cagatay Demiralp, and Cesar Hidalgo. 2019. Sherlock: A Deep Learning Approach to Semantic Data Type Detection. In Proceedings of the 25th ACM SIGKDD International Conference ...

    2019 arXiv

  6. [14]

    Saurav Kadavath, Tom Conerly, Amanda Askell, Tom Henighan, Dawn Drain, Ethan Perez, Nicholas Schiefer, Zac Hatfield-Dodds, Nova DasSarma, Eli Tran- Johnson, et al. 2022. Language Models (Mostly) Know What They Know.arXiv preprint arXiv:2207.05221(2022). arXiv:2207.05221

    2022 arXiv

  7. [15]

    Haoyang Li, Jing Zhang, Cuiping Li, and Hong Chen. 2023. RESDSQL: Decoupling Schema Linking and Skeleton Parsing for Text-to-SQL. InProceedings of the AAAI Conference on Artificial Intelligence. arXiv:2302.05965

    2023 arXiv

  8. [16]

    Jinyang Li, Binyuan Hui, Ge Qu, Jiaxi Yang, Binhua Li, Bowen Li, Bailin Wang, Bowen Qin, Ruiying Geng, Nan Huo, Xuanhe Zhou, Chenhao Ma, Guoliang Li, Kevin Chang, Fei Huang, Reynold Cheng, and Yongbin Li. 2023. Can LLM Already Serve as a Database Interface? A BIg Bench for Lar...

    2023 arXiv

  9. [17]

    Terrance Liu, Shuyi Wang, Daniel Preotiuc-Pietro, Yash Chandarana, and Chirag Gupta. 2025. Calibrating LLMs for Text-to-SQL Parsing by Leveraging Sub- clause Frequencies. InProceedings of the 2025 Conference on Empirical Methods in Natural Language Processing. 16958–16982. arX...

    2025

  10. [18]

    Alexandru Niculescu-Mizil and Rich Caruana. 2005. Predicting Good Probabilities with Supervised Learning. InProceedings of the 22nd International Conference on Machine Learning (ICML). 625–632. doi:10.1145/1102351.1102430

    2005

  11. [19]

    Thorsten Papenbrock, Tanja Bergmann, Moritz Finke, Jakob Zwiener, and Fe- lix Naumann. 2015. Data Profiling with Metanome.Proceedings of the VLDB Endowment8, 12 (2015), 1860–1863. doi:10.14778/2824032.2824086

    2015

  12. [20]

    Thorsten Papenbrock, Sebastian Kruse, Jorge-Arnulfo Quiané-Ruiz, and Felix Naumann. 2015. Divide & Conquer-based Inclusion Dependency Discovery.Pro- ceedings of the VLDB Endowment8, 7 (2015), 774–785. doi:10.14778/2752939. 2752946

    2015 doi

  13. [21]

    John C. Platt. 1999. Probabilistic Outputs for Support Vector Machines and Comparisons to Regularized Likelihood Methods. InAdvances in Large Margin Classifiers. MIT Press, 61–74

    1999

  14. [22]

    Mohammadreza Pourreza and Davood Rafiei. 2023. DIN-SQL: Decomposed In- Context Learning of Text-to-SQL with Self-Correction. InAdvances in Neural Information Processing Systems (NeurIPS). arXiv:2304.11015

    2023 arXiv

  15. [23]

    Robert Richardson. 2026. What Predicts Correctness in Text-to-SQL? A Selective- Prediction Study.arXiv preprint arXiv:2607.06799(2026). arXiv:2607.06799

    2026 arXiv

  16. [24]

    Alexandra Rostin, Oliver Albrecht, Jana Bauckmann, Felix Naumann, and Ulf Leser. 2009. A Machine Learning Approach to Foreign Key Discovery. InPro- ceedings of the 12th International Workshop on the Web and Databases (WebDB)

    2009

  17. [25]

    Bailin Wang, Richard Shin, Xiaodong Liu, Oleksandr Polozov, and Matthew Richardson. 2020. RAT-SQL: Relation-Aware Schema Encoding and Linking for Text-to-SQL Parsers. InProceedings of the 58th Annual Meeting of the Association for Computational Linguistics (ACL). arXiv:1911.04942

    2020 arXiv

  18. [26]

    Bingbing Wen, Jihan Yao, Shangbin Feng, Chenjun Xu, Yulia Tsvetkov, Bill Howe, and Lucy Lu Wang. 2025. Know Your Limits: A Survey of Abstention in Large Language Models.Transactions of the Association for Computational Linguistics (TACL)(2025). arXiv:2407.18418

    2025 arXiv

  19. [27]

    Xiaojun Xu, Chang Liu, and Dawn Song. 2017. SQLNet: Generating Structured Queries from Natural Language without Reinforcement Learning.arXiv preprint arXiv:1711.04436(2017). arXiv:1711.04436

    2017 arXiv

  20. [28]

    Tao Yu, Rui Zhang, Kai Yang, Michihiro Yasunaga, Dongxu Wang, Zifan Li, James Ma, Irene Li, Qingning Yao, Shanelle Roman, Zilin Zhang, and Dragomir Radev. 2018. Spider: A Large-Scale Human-Labeled Dataset for Complex and Cross-Domain Semantic Parsing and Text-to-SQL Task. InPr...

    2018 arXiv

  21. [29]

    Bianca Zadrozny and Charles Elkan. 2002. Transforming Classifier Scores into Accurate Multiclass Probability Estimates. InProceedings of the Eighth ACM SIGKDD International Conference on Knowledge Discovery and Data Mining (KDD). 694–699. doi:10.1145/775047.775151

    2002

  22. [30]

    Procopiuc, and Divesh Srivastava

    Meihui Zhang, Marios Hadjieleftheriou, Beng Chin Ooi, Cecilia M. Procopiuc, and Divesh Srivastava. 2010. On Multi-Column Foreign Key Discovery.Proceedings of the VLDB Endowment3, 1–2 (2010), 805–814. doi: 10.14778/1920841. 1920944

    2010 doi

  23. [31]

    Victor Zhong, Caiming Xiong, and Richard Socher. 2017. Seq2SQL: Generating Structured Queries from Natural Language using Reinforcement Learning.arXiv preprint arXiv:1709.00103(2017). arXiv:1709.00103 37

    2017 arXiv

Pith tools

Reviewed August 12, 2026 · model on record in the stance chip above.