Exploiting LLMs for Automatic Hypothesis Assessment via a Logit-Based Calibrated Prior

Raul Castro Fernandez; Yue Gong

REVIEW 3 major objections 5 minor 33 references

Exploiting LLMs for Automatic Hypothesis Assessment via a Logit-Based Calibrated Prior

T0 review · 3 major / 5 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read A calibrated prior built from an LLM's raw logits predicts which real-world correlations deserve a second look.

desk verdict The logit-based prior is a real and reusable technique; the mode-based results are credible, but the 'calibrated' headline is mostly a tuned kernel width, not the logit distribution. read the letter →

arxiv 2506.03444 v1 pith:2S7LKQEN submitted 2025-06-03 cs.LG cs.CL

Yue Gong , Raul Castro Fernandez This is my paper

classification cs.LGcs.CL

keywords hypothesisassessmentcorrelationpriorLLMlogitscalibrationsurpriserankingelicitationPearsondatadiscovery

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper tries to show that an LLM's pretrained knowledge can be turned into a usable statistical prior for hypothesis assessment: given two variables and their context, the prior says what correlation an informed observer would expect, so an observed correlation far from that expectation counts as surprising and worth expert attention. It proposes the Logit-based Calibrated Prior (LCP), which reads the model's raw token logits when it answers a prompted correlation question, assembles the competing numeric answers into a discrete distribution, and smooths it into a continuous density over $[-1,1]$. On 2,096 real-world variable pairs LCP reaches 78.8% sign accuracy, mean absolute error 0.26, and 89.2% coverage of its 95% credible intervals, while cutting the average information content of observed correlations from 0.69 to 0.27. The authors also argue the prior generalizes to counterfactual contexts, so it reasons from context rather than memorized name pairs.

What carries the argument

The load-bearing object is the logit-derived discrete distribution over correlation values: at each decoding step of a structured numeric answer, the method keeps the top-$k$ token probabilities, enumerates all token sequences that form valid floats in $[-1,1]$, sums their log-probabilities, merges duplicate numeric values, and softmax-normalizes. This discrete distribution is then smoothed into the LCP density $f(r) = \frac{1}{Z} \sum_{j=1}^N p_j \mathcal{N}(r; r_j, \sigma^2)$ truncated to $[-1,1]$, with $\sigma = 0.4$ chosen by negative log-likelihood on a validation set. This machinery matters because it converts the LLM's raw belief state into a density $p(r \mid C_{X,Y})$ evaluable at any observed correlation, which is exactly what surprise scoring, information-content measurement, and credible-interval calibration require.

What would settle it

Compute LCP's 95% credible interval coverage on a fresh set of variable pairs from an unseen domain without retuning $\sigma$; if coverage falls far below the reported 89.2% (for example, below 80%), the transfer assumption fails. As a second check, if ranking a new pool of expert-flagged hypotheses by LCP likelihood does not beat magnitude-based and random rankings, the central utility claim is falsified.

Watch

Extended reading notes

Core claim

The central discovery claimed is that a non-parametric, logit-derived distribution, not a Gaussian or other parametric form, faithfully captures what an LLM believes about a correlation, and that once calibrated it behaves like a genuine predictive distribution for real data. LCP's construction aggregates the joint log-probabilities of all top-$k$ token sequences that parse to valid numbers in $[-1,1]$, merges sequences that map to the same value, and softmax-normalizes them; the resulting discrete prior is smoothed with truncated Gaussian kernels of width $\sigma = 0.4$, tuned on 300 validation pairs to minimize average negative log-likelihood. With that single width, the prior assigns 89.2% coverage to 95% credible intervals, outperforms both parametric and uncalibrated KDE baselines, beats a fine-tuned masked-language-model classifier on thresholded binary correlation prediction despite requiring no training itself, and ranks expert-flagged correlations from noisy urban data above magnitude-based or classifier-based rankings. The contextual-contradiction experiment, in which contrived contexts reverse 84 known relationships, reports 95.2% sign accuracy under the reversed priors, which the authors take as evidence of reasoning rather than memorization.

Load-bearing premise

The method works only if the token probabilities the LLM assigns to candidate numeric answers are a faithful expression of its belief, and if the single smoothing width $\sigma = 0.4$ tuned on 300 validation pairs transfers that calibration to every other variable pair and to the retrieval task.

Editorial extensions

If this is right

Surprise can be scored as $-\log p(r_{\mathrm{obs}})$, so a discovery system can rank thousands of variable pairs by how much they deviate from an informed prior rather than by $|r|$.
A single prior supports any correlation threshold without retraining, unlike a binary classifier that must be re-fit for each decision boundary.
The retrieval result in noisy urban data implies that LCP can surface expert-validated hypotheses even after joins, aggregation, and missing-value transformations.
The contextual-contradiction result implies that the prior updates with modified contexts, making it applicable to variables whose relationships are not in the pretraining corpus.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

If LCP's success reflects general world-knowledge calibration, the same logit-aggregation trick could elicit priors for other scalar relationship statistics, such as trend slopes, effect sizes, or risk ratios, wherever an LLM can answer a structured numeric prompt.
The single fixed $\sigma = 0.4$ is a strong transfer assumption; a natural extension the paper does not test is hierarchical or input-dependent kernel widths that adapt uncertainty to variable-pair familiarity.
The surprise score is relative to the LLM's knowledge, not to a domain expert's knowledge; in specialized fields an LLM may systematically call familiar relationships surprising, so field-specific recalibration is a testable next step.
The benchmark's variable pairs come from public tabular and urban open data; applying the same pipeline to scientific or medical variable pairs would test whether the coverage and ranking gains persist outside those distributions.

Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

The logit-based construction is a real contribution, and the mode-based results (sign accuracy, MAE) are believable. But the headline calibration numbers—89.2% coverage and information content 0.27—are mostly an artifact of the tuned kernel width σ=0.4, not evidence that the logit-derived discrete distribution is calibrated.

What the paper does well: it shows how to turn raw token logits over a numeric field into a continuous predictive distribution by enumerating top-k token sequences, aggregating probabilities, and smoothing with a tuned Gaussian kernel. That is a practical and reusable technique for eliciting continuous priors from LLMs, and it does not require the model to parameterize a fixed distribution. The comparison against Gaussian and KDE priors is informative, especially the analysis of why Gaussian elicitation fails: GPT-4o defaults to σ=0.1, apparently because it assumes a sample size of 100. That is a useful empirical finding. The Nexus retrieval experiment, though small, demonstrates real utility: LCP ranks expert-flagged correlations higher than ranking by magnitude or by a fine-tuned RoBERTa. The contextual contradiction experiment is a creative way to probe memorization, and the inclusion of a multi-hop failure case shows honest reporting.

The main soft spot is the calibration evidence. The paper tunes σ on 300 validation pairs to minimize negative log-likelihood, then reports coverage and information content on the evaluation set with that same σ. A prior that placed all of its mass at the LCP mode and smoothed with the same σ=0.4 would likely produce similar coverage and IC. So the wide kernel, not the shape of the logit distribution, is doing the calibration work. That does not invalidate the mode-based results, but it means the paper's claim of a "calibrated continuous predictive distribution" is overstated. The validation set's origin is also unspecified—it says 300 pairs disjoint from evaluation, but not where they are drawn from. That matters because the tuned σ carries the calibration story. The memorization probe is on a selected subset (84 pairs where the initial sign prediction was correct), so the 95.2% under contradictory contexts is encouraging but not a clean generalization result.

Overall, the core technical idea and the predictive-accuracy results are solid and worth publishing. The calibration claims should be softened and re-framed in revision, and the validation split should be documented. I would send this to peer review; it is the kind of paper that will elicit useful discussion about what "calibration" means for LLM-elicited priors.

Referee Report

3 major / 5 minor

Summary. The paper proposes the Logit-based Calibrated Prior (LCP), a method that converts GPT-4o's token logits under a correlation-prediction prompt into a discrete distribution over [-1,1], then smooths it with a Gaussian kernel whose width sigma is tuned on a 300-pair validation set. The authors evaluate this prior on 2,096 real-world variable pairs, reporting sign accuracy 78.8%, MAE 0.26, 95% credible interval coverage 89.2%, and average information content 0.27, and compare against uniform, Gaussian, KDE, and fine-tuned RoBERTa baselines. They also apply LCP to rank expert-flagged correlations in the Nexus urban-data corpus and test context sensitivity with counterfactual scenarios.

Significance. The paper addresses a timely problem—automatic triage of discovered correlations—and the mode-based results (sign accuracy, MAE, and the Nexus retrieval experiment) are plausible and interesting. The benchmark construction is transparent and the anonymous repository is a strength. However, the headline calibration evidence is currently dominated by the single fitted kernel width sigma, so the central claim that LCP is a 'calibrated continuous predictive distribution' is not established. The paper would be significantly strengthened by a control that isolates the contribution of the logit distribution shape.

major comments (3)

[Section 2, Eq. (1); Section 4, Fig. 3] The coverage and information-content results are not independent evidence of calibration because both are controlled by the fitted kernel width. sigma*=0.4 is selected on 300 validation pairs by minimizing average negative log-likelihood, and the same sigma fixes the width of the 95% credible intervals and the density p(r_obs) used for information content. A point-mass prior at the LCP mode smoothed with the same sigma would likely achieve nearly the same 89.2% coverage and 0.27 information content, given MAE=0.26. The comparison with KDE (59.9% coverage with median sigma=0.08) demonstrates only that a larger kernel width increases coverage, not that the shape of the logit distribution matters. Please add a control that replaces the logit mixture with a delta at the mode while keeping sigma=0.4, and report coverage across confidence levels rather than a single 95% number.
[Section 2, validation set] The validation set is underspecified. The text says Dval consists of 300 randomly sampled correlations, disjoint from our evaluation dataset, but Section 3 only describes the 2,096-pair benchmark and does not mention a separate validation set. It is unclear whether these 300 pairs are drawn from the same Kaggle/Cause-Effect sources and whether they share tables or variable pairs with the evaluation set. Because sigma is the only fitted parameter and the paper claims it transfers to all pairs and to Nexus, the sampling process and the level of overlap must be documented.
[Section 6 and Abstract] The claim that the prior 'generalizes to correlations not seen during LLM pretraining' is not supported by the contextual-contradiction experiment. That experiment uses 84 Cause-Effect pairs, replaces the context with a synthetic counterfactual generated by Gemini 2.5 Pro, and sets r_obs = -r_obs as an approximation. This tests whether predictions move in the expected direction, not whether the prior is accurate on unseen correlation values or domains. Please rephrase the claim or add a held-out evaluation based on a temporal or source-based split.

minor comments (5)

[Algorithm 1] The value of k in the top-k decoding (line 5) is never stated; please report it and include a sensitivity analysis for the resulting discrete distribution.
[Appendix A] The chi-square goodness-of-fit test treats the model-assigned probabilities p_j as pseudo-counts with M=1000, which is not a valid application of the chi-square test because these probabilities are not independent empirical frequencies; the normality-rejection claim should be supported by a different diagnostic or stated more cautiously.
[Section 5, Table 1] The retrieval comparison is based on a single pool of 115 items with 15 targets; please report bootstrap confidence intervals or a significance test for the Precision@K and average-rank differences.
[Section 5] Please clarify which correlation value is used as r_obs when computing p(r_obs) for the Nexus ranking, since the text says ground-truth correlation values are unavailable.
[Figure 4] The numbers printed above the boxes in Figure 4 are not defined in the caption; please state what they represent (apparently standard deviations or mean values).

Circularity Check

0 steps flagged · score 1.0 of 10

No significant circularity: sigma is tuned on a disjoint validation set and the reported coverage and information content are held-out evaluation metrics, not fits renamed as predictions.

full rationale

The derivation chain for LCP is self-contained and not circular. The discrete prior is constructed from LLM token logits (Algorithm 1) with no reference to the target correlation r_obs; the mode, sign accuracy, and mean absolute error are therefore independent of any fitted parameter. The kernel width sigma=0.4 is chosen on a held-out validation set, which Section 2 explicitly states is disjoint from the evaluation dataset, by minimizing average negative log-likelihood. Reporting 95% credible interval coverage and information content on that disjoint evaluation set is a standard held-out calibration check, not a reduction: tuning one scalar to maximize validation likelihood does not, by construction, force 89.2% coverage or 0.27 information content on unseen pairs, and those metrics could fail if the logit-derived modes were poor. The only self-citation is the reuse of the authors' Nexus system [8] as a source of expert-flagged correlations in Section 5; those labels are human judgments from the prior Nexus evaluation and are not derived from LCP, so the citation supplies an external benchmark rather than a load-bearing premise. No equation in the paper is equivalent to its own input by construction, and no fitted parameter is renamed as a prediction.

Assumptions & free parameters 2 free parameters · 5 assumptions · 0 invented entities

The central calibration claim rests on a single global bandwidth sigma=0.4 fit to 300 validation pairs, and on the assumption that LLM token logits represent genuine belief; no independent evidence is provided for either.

free parameters (2)

kernel width sigma = 0.4
Tuned on 300 validation correlations by minimizing average negative log-likelihood; controls the spread of the LCP density and therefore the calibration, coverage, and information content results.
top-k token count = not reported
Algorithm 1 enumerates top-k tokens at each decoding step; the value of k is not given in the paper, which affects the support of the discrete prior and is a choice made by the authors.

assumptions (5)

domain assumption LLM token logits reflect the model's epistemic belief about the correlation coefficient
The entire LCP construction interprets softmaxed log-probabilities over numeric token sequences as a valid prior distribution over r, invoked in Section 2 and Algorithm 1.
ad hoc to paper A single global Gaussian kernel width transfers across all variable pairs and datasets
The paper assumes bias in LLM uncertainty is stable across inputs, so sigma=0.4 need not be re-tuned within the evaluation; this is stated in Section 2 without independent evidence.
domain assumption The prompt template and variable descriptions provide sufficient context for correlation prediction
Results depend on prompt G.1 and on GPT-4o-generated or Kaggle descriptions; no analysis of prompt sensitivity is provided.
domain assumption The benchmark of 2,096 pairs is representative of real-world hypothesis assessment workloads
The benchmark mixes Cause-Effect pairs with filtered Kaggle correlations and stratified sampling by |r|; the Nexus retrieval pool is separate and small.
domain assumption Surprise defined as low prior likelihood maps to hypothesis worthiness
The paper equates low p(robs) under LCP with novelty and expert relevance, an assumption underlying the ranking evaluation in Section 5.

how reviews work

0 comments

Cite this review

Pith. "Pith review of Exploiting LLMs for Automatic Hypothesis Assessment via a Logit-Based Calibrated Prior." pith.science (2026). https://pith.science/paper/2S7LKQEN

@misc{pith2026250603444,
  author       = {Pith},
  title        = {Pith review of: Exploiting LLMs for Automatic Hypothesis Assessment via a Logit-Based Calibrated Prior},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/2S7LKQEN}},
  note         = {Machine review of arXiv:2506.03444}
}

read the original abstract

As hypothesis generation becomes increasingly automated, a new bottleneck has emerged: hypothesis assessment. Modern systems can surface thousands of statistical relationships-correlations, trends, causal links-but offer little guidance on which ones are novel, non-trivial, or worthy of expert attention. In this work, we study the complementary problem to hypothesis generation: automatic hypothesis assessment. Specifically, we ask: given a large set of statistical relationships, can we automatically assess which ones are novel and worth further exploration? We focus on correlations as they are a common entry point in exploratory data analysis that often serve as the basis for forming deeper scientific or causal hypotheses. To support automatic assessment, we propose to leverage the vast knowledge encoded in LLMs' weights to derive a prior distribution over the correlation value of a variable pair. If an LLM's prior expects the correlation value observed, then such correlation is not surprising, and vice versa. We propose the Logit-based Calibrated Prior, an LLM-elicited correlation prior that transforms the model's raw output logits into a calibrated, continuous predictive distribution over correlation values. We evaluate the prior on a benchmark of 2,096 real-world variable pairs and it achieves a sign accuracy of 78.8%, a mean absolute error of 0.26, and 95% credible interval coverage of 89.2% in predicting Pearson correlation coefficient. It also outperforms a fine-tuned RoBERTa classifier in binary correlation prediction and achieves higher precision@K in hypothesis ranking. We further show that the prior generalizes to correlations not seen during LLM pretraining, reflecting context-sensitive reasoning rather than memorization.

Figures

Figures reproduced from arXiv: 2506.03444 by the authors.

**Figure 2.** The Bias toward High Correlations in Kaggle dataset The Cause-Effect dataset contains 108 variable pairs with known causal relationships. We retain 96 pairs where the correlation is statistically significant (p < 0.05). The Kaggle dataset consists of correlations between variable pairs extracted from publicly available tables on Kaggle. The original dataset provides variable names but lacks variable descriptions. … view at source ↗

**Figure 3.** Accuracy vs. Calibration of Correlation Priors (IC=Information Content) [PITH_FULL_IMAGE:figures/full_fig_p006_3.png] view at source ↗

Figures from the paper (4 more)

**Figure 4.** Figure 4: Full Distribution of Metrics over Different Priors [PITH_FULL_IMAGE:figures/full_fig_p006_4.png]

**Figure 5.** Figure 5: Performance across ten bins of the true correlation [PITH_FULL_IMAGE:figures/full_fig_p007_5.png]

**Figure 6.** Figure 6: Classification Performance over Different Correlation thresholds [PITH_FULL_IMAGE:figures/full_fig_p007_6.png]

**Figure 7.** Figure 7: Distribution of kernel standard deviations for Gaussian and KDE priors. [PITH_FULL_IMAGE:figures/full_fig_p012_7.png]

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

33 extracted references · 18 canonical work pages

[1]

Krishnan, and Payam Barnaghi

Alexander Capstick, Rahul G. Krishnan, and Payam Barnaghi. Autoelicit: Using large language models for expert prior elicitation in predictive modelling, 2025. URL https://arxiv.org/ abs/2411.17284

arXiv 2025
[2]

Data polygamy: The many-many relationships among urban spatio-temporal data sets

Fernando Chirigati, Harish Doraiswamy, Theodoros Damoulas, and Juliana Freire. Data polygamy: The many-many relationships among urban spatio-temporal data sets. InProceedings of the 2016 International Conference on Management of Data, pages 1011–1025, 2016

work page 2016
[3]

Lmpriors: Pre-trained language models as task-specific priors.arXiv preprint arXiv:2210.12530, 2022

Kristy Choi, Chris Cundy, Sanjari Srivastava, and Stefano Ermon. Lmpriors: Pre-trained language models as task-specific priors.arXiv preprint arXiv:2210.12530, 2022

arXiv 2022
[4]

Exploratory data analysis.Secondary analysis of electronic health records, pages 185–203, 2016

MIT Critical Data, Matthieu Komorowski, Dominic C Marshall, Justin D Salciccioli, and Yves Crutain. Exploratory data analysis.Secondary analysis of electronic health records, pages 185–203, 2016

work page 2016
[5]

Aleatory or epistemic? does it matter?Structural Safety, 31(2):105–112, 2009

Armen Der Kiureghian and Ove Ditlevsen. Aleatory or epistemic? does it matter?Structural Safety, 31(2):105–112, 2009

work page 2009
[6]

Bert: Pre-training of deep bidirectional transformers for language understanding

Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. Bert: Pre-training of deep bidirectional transformers for language understanding. InProceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT), pages 4171–4186, 2019

2019
[7]

Riding tandem: Does cycling infrastructure investment mirror gentrification and privilege in portland, or and chicago, il? Research in Transportation Economics, 60:14–24, 2016

Elizabeth Flanagan, Ugo Lachapelle, and Ahmed El-Geneidy. Riding tandem: Does cycling infrastructure investment mirror gentrification and privilege in portland, or and chicago, il? Research in Transportation Economics, 60:14–24, 2016

work page 2016
[8]

Nexus: Correlation discovery over collections of spatio-temporal tabular data.Proc

Yue Gong, Sainyam Galhotra, and Raul Castro Fernandez. Nexus: Correlation discovery over collections of spatio-temporal tabular data.Proc. ACM Manag. Data, 2(3), May 2024. doi: 10.1145/3654957. URLhttps://doi.org/10.1145/3654957

work page doi:10.1145/3654957 2024

Show all 33 references

[9]

Shelf: the sheffield elicitation framework

John Paul Gosling. Shelf: the sheffield elicitation framework. InElicitation: The science and art of structuring judgement, pages 61–93. Springer, 2017

2017
[10]

Automated prior elicitation from large language models for bayesian logistic regression

Henry Gouk and Boyan Gao. Automated prior elicitation from large language models for bayesian logistic regression. InAutoML Conference 2024 (Workshop Track), 2024. URL https://openreview.net/forum?id=euLzlnU7gz

2024
[11]

Big data: A revolution that will transform how we live, work, and think, 2014

Saint John Walker. Big data: A revolution that will transform how we live, work, and think, 2014

2014
[12]

What uncertainties do we need in bayesian deep learning for computer vision? InNeurIPS, 2017

Alex Kendall and Yarin Gal. What uncertainties do we need in bayesian deep learning for computer vision? InNeurIPS, 2017

2017
[13]

Biograph: unsupervised biomedical knowledge discovery via automated hypothesis generation.Genome biology, 12:1–12, 2011

Anthony ML Liekens, Jeroen De Knijf, Walter Daelemans, Bart Goethals, Peter De Rijk, and Jurgen Del-Favero. Biograph: unsupervised biomedical knowledge discovery via automated hypothesis generation.Genome biology, 12:1–12, 2011

2011
[14]

Roberta: A robustly optimized bert pretraining approach.arXiv preprint arXiv:1907.11692, 2019

Yinhan Liu, Myle Ott, Naman Goyal, Jingfei Du, Mandar Joshi, Danqi Chen, Omer Levy, Mike Lewis, Luke Zettlemoyer, and Veselin Stoyanov. Roberta: A robustly optimized bert pretraining approach.arXiv preprint arXiv:1907.11692, 2019

1907 arXiv
[15]

Distinguishing cause from effect using observational data: methods and benchmarks.Journal of Machine Learning Research, 17(32):1–102, 2016

Joris M Mooij, Jonas Peters, Dominik Janzing, Jakob Zscheischler, and Bernhard Schölkopf. Distinguishing cause from effect using observational data: methods and benchmarks.Journal of Machine Learning Research, 17(32):1–102, 2016

2016
[16]

John Wiley & Sons, 2006

Anthony O’Hagan, Caitlin E Buck, Alireza Daneshkhah, J Richard Eiser, Paul H Garthwaite, David J Jenkinson, Jeremy E Oakley, and Tim Rakow.Uncertain judgements: eliciting experts’ probabilities. John Wiley & Sons, 2006

2006
[17]

OpenAI. Gpt-4o. https://platform.openai.com/docs/models/gpt-4o, 2024. Ac- cessed: 2025-05-15. 10

2024
[18]

Language models as knowledge bases?arXiv preprint arXiv:1909.01066, 2019

Fabio Petroni, Tim Rocktäschel, Patrick Lewis, Anton Bakhtin, Yuxiang Wu, Alexander H Miller, and Sebastian Riedel. Language models as knowledge bases?arXiv preprint arXiv:1909.01066, 2019

1909 arXiv
[19]

Chicago data portal, 2025

Chicago Data Portal. Chicago data portal, 2025. URL https://data.cityofchicago.org/

2025
[20]

Llm processes: Numerical predictive distributions conditioned on natural language.Advances in Neural Information Processing Systems, 37:109609–109671, 2024

James Requeima, John Bronskill, Dami Choi, Richard Turner, and David K Duvenaud. Llm processes: Numerical predictive distributions conditioned on natural language.Advances in Neural Information Processing Systems, 37:109609–109671, 2024

2024
[21]

Correla- tion sketches for approximate join-correlation queries

Aécio Santos, Aline Bessa, Fernando Chirigati, Christopher Musco, and Juliana Freire. Correla- tion sketches for approximate join-correlation queries. InProceedings of the 2021 International Conference on Management of Data, pages 1531–1544, 2021

2021
[22]

Efficiently estimating mutual information between attributes across tables

Aécio Santos, Flip Korn, and Juliana Freire. Efficiently estimating mutual information between attributes across tables. In2024 IEEE 40th International Conference on Data Engineering (ICDE), pages 193–206, 2024. doi: 10.1109/ICDE60146.2024.00022

2024
[23]

John Wiley & Sons, 2015

David W Scott.Multivariate density estimation: theory, practice, and visualization. John Wiley & Sons, 2015

2015
[24]

A mathematical theory of communication.The Bell system technical journal, 27(3):379–423, 1948

Claude E Shannon. A mathematical theory of communication.The Bell system technical journal, 27(3):379–423, 1948

1948
[25]

Directlingam: A direct method for learning a linear non-gaussian structural equation model.Journal of Machine Learning Research-JMLR, 12(Apr):1225–1248, 2011

Shohei Shimizu, Takanori Inazumi, Yasuhiro Sogawa, Aapo Hyvarinen, Yoshinobu Kawahara, Takashi Washio, Patrik O Hoyer, Kenneth Bollen, and Patrik Hoyer. Directlingam: A direct method for learning a linear non-gaussian structural equation model.Journal of Machine Learning Resea...

2011
[26]

Routledge, 2018

Bernard W Silverman.Density estimation for statistics and data analysis. Routledge, 2018

2018
[27]

Large language models encode clinical knowledge.Nature, 620(7972):172–180, 2023

Karan Singhal, Shekoofeh Azizi, Tao Tu, S Sara Mahdavi, Jason Wei, Hyung Won Chung, Nathan Scales, Ajay Tanwani, Heather Cole-Lewis, Stephen Pfohl, et al. Large language models encode clinical knowledge.Nature, 620(7972):172–180, 2023

2023
[28]

Statistical methods, 8thedn.Ames: Iowa State Univ

George W Snedecor and Witiiam G Cochran. Statistical methods, 8thedn.Ames: Iowa State Univ. Press Iowa, 54:71–82, 1989

1989
[29]

Wilkins, Benjamin J

Scott Spangler, Angela D. Wilkins, Benjamin J. Bachman, Meena Nagarajan, Tajhal Dayaram, Peter Haas, Sam Regenbogen, Curtis R. Pickering, Austin Comer, Jeffrey N. Myers, Ioana Stanoi, Linda Kato, Ana Lelescu, Jacques J. Labrie, Neha Parikh, Andreas Martin Lisewski, Lawrence Do...

2014
[30]

Can large language models predict data correlations from column names? Proc

Immanuel Trummer. Can large language models predict data correlations from column names? Proc. VLDB Endow., 16(13):4310–4323, September 2023. ISSN 2150-8097. doi: 10.14778/ 3625054.3625066. URLhttps://doi.org/10.14778/3625054.3625066

2023
[31]

Improving scientific hypothesis generation with knowledge grounded large language models, 2024

Guangzhi Xiong, Eric Xie, Amir Hassan Shariatmadari, Sikun Guo, Stefan Bekiranov, and Aidong Zhang. Improving scientific hypothesis generation with knowledge grounded large language models, 2024. URLhttps://arxiv.org/abs/2411.02382

2024 arXiv
[32]

Large language models for scientific discovery in molecular property prediction

Yizhen Zheng, Huan Yee Koh, Jiaxin Ju, Anh TN Nguyen, Lauren T May, Geoffrey I Webb, and Shirui Pan. Large language models for scientific discovery in molecular property prediction. Nature Machine Intelligence, pages 1–11, 2025

2025
[33]

coefficient

Yangqiaoyu Zhou, Haokun Liu, Tejes Srivastava, Hongyuan Mei, and Chenhao Tan. Hypoth- esis generation with large language models. InProceedings of the 1st Workshop on NLP for Science (NLP4Science), page 117–139. Association for Computational Linguistics, 2024. doi: 10.18653/v1...

2024 doi

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.

[1] [1]

Krishnan, and Payam Barnaghi

Alexander Capstick, Rahul G. Krishnan, and Payam Barnaghi. Autoelicit: Using large language models for expert prior elicitation in predictive modelling, 2025. URL https://arxiv.org/ abs/2411.17284

arXiv 2025

[2] [2]

Data polygamy: The many-many relationships among urban spatio-temporal data sets

Fernando Chirigati, Harish Doraiswamy, Theodoros Damoulas, and Juliana Freire. Data polygamy: The many-many relationships among urban spatio-temporal data sets. InProceedings of the 2016 International Conference on Management of Data, pages 1011–1025, 2016

work page 2016

[3] [3]

Lmpriors: Pre-trained language models as task-specific priors.arXiv preprint arXiv:2210.12530, 2022

Kristy Choi, Chris Cundy, Sanjari Srivastava, and Stefano Ermon. Lmpriors: Pre-trained language models as task-specific priors.arXiv preprint arXiv:2210.12530, 2022

arXiv 2022

[4] [4]

Exploratory data analysis.Secondary analysis of electronic health records, pages 185–203, 2016

MIT Critical Data, Matthieu Komorowski, Dominic C Marshall, Justin D Salciccioli, and Yves Crutain. Exploratory data analysis.Secondary analysis of electronic health records, pages 185–203, 2016

work page 2016

[5] [5]

Aleatory or epistemic? does it matter?Structural Safety, 31(2):105–112, 2009

Armen Der Kiureghian and Ove Ditlevsen. Aleatory or epistemic? does it matter?Structural Safety, 31(2):105–112, 2009

work page 2009

[6] [6]

Bert: Pre-training of deep bidirectional transformers for language understanding

Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. Bert: Pre-training of deep bidirectional transformers for language understanding. InProceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT), pages 4171–4186, 2019

2019

[7] [7]

Riding tandem: Does cycling infrastructure investment mirror gentrification and privilege in portland, or and chicago, il? Research in Transportation Economics, 60:14–24, 2016

Elizabeth Flanagan, Ugo Lachapelle, and Ahmed El-Geneidy. Riding tandem: Does cycling infrastructure investment mirror gentrification and privilege in portland, or and chicago, il? Research in Transportation Economics, 60:14–24, 2016

work page 2016

[8] [8]

Nexus: Correlation discovery over collections of spatio-temporal tabular data.Proc

Yue Gong, Sainyam Galhotra, and Raul Castro Fernandez. Nexus: Correlation discovery over collections of spatio-temporal tabular data.Proc. ACM Manag. Data, 2(3), May 2024. doi: 10.1145/3654957. URLhttps://doi.org/10.1145/3654957

work page doi:10.1145/3654957 2024