Pith. sign in

REVIEW 4 major objections 5 minor 68 references

Interestingness First Classifiers

T0 review · 4 major / 5 minor · reviewed 2026-08-05 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read EUREKA ranks features by LLM-judged interestingness and trains interpretable classifiers on the top-ranked ones; the resulting rules—humidity for occupancy, colons for citations—beat chance and significance tests on six benchmarks.

desk verdict A genuinely new objective and a clean pipeline, but 'interestingness' is validated only by the LLM itself, so the central claim is not yet supported. read the letter →

arxiv 2508.19780 v1 pith:X6GC7G22 submitted 2025-08-27 cs.LG stat.ML

Ryoma Sato This is my paper · ORCID
classification cs.LGstat.ML
keywords interestingness-firstclassifiersfeatureselectionbyLLMpairwisecomparisonrankingBordacountinterpretablehypothesisgenerationtabularbenchmarkssurprisingrules
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper tries to establish that classifiers can be built for surprise rather than accuracy, and that doing so yields rules that are both non-obvious and still predictive. The proposed framework, EUREKA, uses an LLM to compare features pairwise—asking which would make the more interesting prediction rule—then aggregates those judgments into a ranking and trains a simple logistic-regression classifier on the top-ranked features. Across six tabular benchmarks, the selected features are exactly the ones accuracy-driven methods ignore: humidity instead of light for office occupancy, colon-in-title instead of reference-list length for citations, capital-loss instead of capital-gain for income. The resulting classifiers exceed the majority-class baseline and pass a significance test, while remaining easy to state and interpret. If this is right, classification becomes a mechanism for hypothesis discovery: surprising signals worth investigating, not just predictions to be scored.

What carries the argument

The engine of the method is pairwise comparison by a large language model feeding a Borda count. For every pair of features, the LLM is asked which one would make the more interesting prediction rule for the target label; each win gives the feature one vote, and the features are ranked by total votes—a scheme the paper invokes as information-theoretically near-optimal for ranking from noisy comparisons. On top of this ranking, EUREKA builds a logistic-regression classifier using only the top-K interesting features, increasing K until the classifier first beats the baseline; interpretability of the base model is treated as a necessary condition, because an uninterpretable rule could not be co

What would settle it

Present the same pairwise 'which rule is more interesting?' prompts to a different LLM family and to a panel of human raters on the Occupancy Detection and Twin Papers tasks. If the Borda rankings across LLMs disagree substantially (Kendall τ < 0.5) or if the LLM ranking correlates no better than chance with human interestingness ratings, the claim that EUREKA captures interestingness rather than model-specific quirks would fail.

Watch

Extended reading notes

Core claim

The paper's central claim is that interestingness can be the primary objective in building a classifier, not an afterthought. Concretely, it claims that asking an LLM to compare features by 'which would make the more interesting rule' yields rankings whose top features are consistently non-obvious yet predictive: Humidity and HumidityRatio outrank Light and CO2 for occupancy; 'title contains a colon' outranks 'longer reference list' for citation outcome; capital-loss outranks capital-gain for income. The resulting rules achieve accuracy above the chance rate—85% for humidity-based occupancy, 52% for the colon rule on a 17,000-instance test set—and pass a likelihood-ratio significance test. T

Load-bearing premise

The entire method rests on the premise that an LLM's answer to 'which feature would make the more interesting rule?' is a trustworthy proxy for human interestingness; the paper only checks that the same LLM is stable across repeated runs, not that its rankings match what people actually find surprising.

Editorial extensions

If this is right

  • EUREKA is ready to apply to any labeled tabular dataset with meaningful column names: the pipeline returns a short, human-readable rule that is both non-obvious and significantly better than the majority-class baseline.
  • The K-selection procedure gives a principled stopping rule: it returns the smallest prefix of the interestingness ranking whose classifier becomes predictive, making clear which features are interesting-but-weak and which are both interesting and predictive.
  • The discovered rules are interpretable by construction, so they can be communicated to non-experts—e.g., 'high humidity means the room is occupied'—without needing post-hoc explanation.
  • The significance checks (likelihood-ratio test at α=0.05 with Bonferroni correction) mean that even a 2% edge over chance on 17,000 samples, as in the colon-in-title rule, is reported as a real signal rather than noise.
  • Because EUREKA avoids features that accuracy-driven baselines rank first (light, reference-list length, BI-RADS score, capital gain), it explores a complementary region of the feature space that conventional feature selection leaves untouched.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Editorial extension: EUREKA is effectively an automated hypothesis generator, but the paper stops at showing the rules are statistically detectable; a natural next step is to design confirmatory studies for rules like 'colon in title ⇒ more citations' before treating them as knowledge.
  • Editorial extension: because the LLM sees feature names and task descriptions, its 'surprise' judgments may come from prior beliefs about the world rather than from the data; comparing EUREKA's rankings with data-only surprisal measures would separate these sources.
  • Editorial extension: the framework could be extended to rank pairs of features or whole candidate classifiers, which the paper names as a limitation and future direction but does not test.
  • Editorial extension: the paper's tolerance of spurious correlations—justified by the Nicolas Cage example—suggests that an interestingness-first tool would benefit from an explicit 'spurious but fun' label so automated discoveries are not mistaken for causal findings.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. The paper introduces EUREKA, a framework for building 'interestingness-first' classifiers: features are ranked by LLM-based pairwise comparisons of how interesting each feature would make a prediction rule, and an interpretable logistic regression is trained on the top-K features. Experiments on six tabular datasets show that EUREKA selects features different from accuracy-driven baselines (e.g., humidity instead of light for occupancy, colon-in-title instead of reference-list length for citation counts) and that classifiers using these features achieve above-chance accuracy. The paper positions this as a new objective for supervised learning, prioritizing surprise and communicability over raw accuracy.

Significance. If the central claim holds, the paper introduces a genuinely different objective for classifier construction and a simple, reproducible pipeline. The method is easy to understand, the pairwise Borda-count ranking has a theoretical grounding in the ranking-from-pairwise-comparisons literature, and several of the discovered rules are concrete and thought-provoking. The predictive component is partially tested: selected features do yield above-chance accuracy on held-out data in several datasets, and the authors run a significance test against an intercept-only model. However, the paper's signature claim—that the selected rules are 'non-obvious yet still predictive'—is only established for the predictive half. The interestingness half is operationalized as 'whatever gpt-5-nano ranks highly' and is validated only by within-model stability, not by any human or independent rater. The paper's own Limitations section explicitly declines to use objective or human-grounded measures. As a result, the current evidence supports a weaker claim: LLM-judged feature rankings can be stable and can point to features that remain predictive. Whether these are interesting to humans remains unverified.

major comments (4)
  1. [§2.3, Table 1, §5] The central construct 'interestingness' has no external anchor. The only validation is stability of gpt-5-nano rankings across 20 runs (Table 1), but stability does not imply validity: a ranking can be stable and still arbitrary, or driven by the LLM's prior knowledge of benchmark datasets and their documentation. The paper's Limitations section says 'we deliberately refrain from relying on such objective metrics' and regards subjectivity as essential, but the abstract and Section 3 present examples as objectively 'non-obvious' and 'interesting'. Under the current operationalization, any feature ranked high by the LLM is, by definition, interesting, so no disconfirming evidence can arise on that axis. To support the abstract's claim, the authors should either (a) obtain human interestingness ratings for the top-ranked and bottom-ranked rules, (b) show agreement with an independent LLM or
  2. [§2.6, §3.3, Table 2] The K* selection rule is underspecified to the point of possible post-hoc selection. Section 2.6 says to increase K until the classifier is 'predictive enough', but no threshold is defined. In Table 2, when the top-ranked feature lacks predictive power, the authors 'also report the second-ranked feature'—e.g., HumidityRatio and Humidity for Occupancy, Density and Age for Mammographic Mass. This means the headline feature set is not the output of a single well-defined algorithm; it is selected after looking at performance. A reader cannot tell whether the reported features would have been chosen by a fixed rule. Please specify the criterion precisely (e.g., validation accuracy above chance plus a margin, or a p-value threshold on a validation likelihood-ratio test), apply it before inspecting test results, and report the automatically selected K* together with the accuracy at that K*. As
  3. [Figure 2, §3.3] The empirical support for 'consistently identifies features that are non-obvious yet still predictive' rests on a single stratified 80/20 split per dataset, with no error bars, confidence intervals, or multiple splits. This is especially problematic for the Twin Papers result, where the advantage over the chance rate is about 2 percentage points; even with 17,000 test instances, the reader needs a confidence interval and a description of how the significance stars were computed for each point. The 'chance rate' is defined as majority-class accuracy, which is a weak benchmark for class-imbalanced problems; a classifier that is statistically significant against an intercept-only model can still have test accuracy below the majority-class baseline (as the paper itself notes for HumidityRatio). Reporting balanced accuracy, AUC, or calibration alongside accuracy, and repeating the evaluation
  4. [§3.1, §2.3] The method is not fully reproducible as described. The exact prompt used for pairwise comparisons is not given, the chosen model is only 'gpt-5-nano', and no temperature, sampling parameters, or API settings are stated. Since the LLM's judgment is the entire interestingness signal, the prompt and sampling details are essential experimental configuration, not an implementation detail. Similarly, the validation-based baseline is described as 'split the training data into validation sets' without specifying the number of folds or the selection rule. Please include the full prompt and the exact validation protocol in an appendix or supplementary material.
minor comments (5)
  1. [§3.3] The sentence about HumidityRatio being 'statistically significant' but 'test accuracy does not reach the chance baseline' is confusing as written. Clarify how a significant effect can fall below the majority-class baseline, for example by reporting the effect direction and the class imbalance, and explain why the star notation in Figure 2 does not mean 'above chance'.
  2. [Figure 2] The Adult subplot's legend order appears inconsistent with the K values shown on the x-axis; the listed feature sets are not in monotone inclusion order. Please reorder the legend to correspond to the plotted curves.
  3. [§4.2] The related work discussion mentions InterFeat and HypoBench as 'concurrent studies' treating interestingness as an evaluation criterion, but there is no direct technical comparison or positioning against their metrics. A brief comparison of the underlying notion of interestingness would help.
  4. [§5] The spurious-correlations paragraph says 'we believe spurious correlations are not a major concern in our problem setting', yet immediately invokes the Nicolas Cage/pool-drowning example as a valuable discovery. That example is exactly a spurious correlation, so the paragraph undercuts its own point. Please rewrite to separate 'spurious as causal' from 'interesting as narrative'.
  5. [§2.3] The claim that pairwise Borda count is 'up to constant factors, an information-theoretically optimal procedure' relies on Shah and Wainwright's theorem, but the setting here uses LLM judgments with possible transitive inconsistency. A sentence noting the assumptions of that theorem would prevent overgeneralization.

Circularity Check

1 steps flagged · score 4.0 of 10

Interestingness is operationalized as the LLM's own pairwise preference, so the 'non-obvious' half of the central claim is self-referential; predictive accuracy remains externally tested.

  1. self definitional [Abstract; Sec. 2.3; Sec. 5 (Limitations)]
    "Across several benchmark datasets, EUREKA consistently identifies features that are non-obvious yet still predictive. ... To assess interestingness, we ask a large language model to compare two candidate features fi and fj: 'If one could predict the label (e.g., Room Occupancy) only from fi (e.g., Humidity) and only from fj (e.g., CO2), which prediction rule would be more interesting?' ... we deliberately refrain from relying on such objective metrics and instead emphasize subjective interestingness of classifiers."

    The 'non-obvious/interesting' component of the central claim is scored with the same LLM pairwise judgment that produces the ranking. Because interestingness is operationalized as 'whichever feature the LLM prefers,' the top-ranked features are interesting by construction, and the abstract's claim that they are 'non-obvious' restates the input rather than testing it. The Limitations section explicitly disclaims objective or human-grounded measures, so no external validation of interestingness exists. The predictive-accuracy half is not circular: it is evaluated on held-out test data against majority-class chance and an intercept-only null model. But the interestingness half of the claimed result reduces to the LLM's own preferences, making the joint claim partially self-fulfilling.

full rationale

EUREKA's pipeline (LLM pairwise comparisons -> Borda ranking -> logistic regression on top-K features) is a self-contained empirical procedure with no equation-level circularity in the ranking or classifier construction. The Borda counting is justified by an external theorem [49], and the comparison with active ranking is an independent methodological check. The predictive claim is externally grounded: accuracies are computed on held-out test data and compared with a majority-class chance rate, with likelihood-ratio tests against an intercept-only null model. The 'interestingness' claim, however, is not externally grounded: the only operationalization of interestingness is the LLM's answer to the pairwise prompt (Sec. 2.3), and the paper explicitly declines objective or human measures (Sec. 5). Thus when the abstract says EUREKA 'consistently identifies features that are non-obvious,' the non-obviousness is defined by the same judgments used to select the features; that component is self-referential. The self-citation [47] is only a dataset source and is not load-bearing. Overall, the central claim is partially circular in its interestingness dimension but retains independent content in its predictive dimension, yielding a moderate score.

Assumptions & free parameters 2 free parameters · 4 assumptions · 0 invented entities

The framework is simple with few numerical free parameters, but the load-bearing assumptions are the LLM's subjective judgments and the undefined K* criterion.

free parameters (2)
  • K (number of top-ranked features selected) = Per dataset: Occupancy 2, Twin Papers 1, Mammographic Mass 2, Breast Cancer Wisconsin 1, Adult 1, Website Phishing 1
    The method chooses the smallest K such that the classifier is 'predictive enough', but the threshold is not defined a priori; different datasets end up with different K.
  • LLM prompt for pairwise interestingness comparison = Not specified; described only as a natural-language question in Section 2.3
    The ranking depends on the exact wording and model parameters (e.g., temperature), neither of which is reported, so the output is not fully determined.
assumptions (4)
  • domain assumption The LLM's pairwise judgments of interestingness correspond to human interestingness.
    No human evaluation is performed; the method's purpose is to satisfy human curiosity, but the paper only shows internal consistency of one LLM.
  • domain assumption Feature names carry semantic meaning that the LLM can interpret.
    The method fails when columns are anonymized (Section 5), so this assumption is acknowledged as a limitation.
  • standard math The Borda count aggregation of pairwise LLM preferences yields a meaningful global ranking.
    Borda count is a standard method, but its use here assumes transitivity and comparability of LLM preferences.
  • domain assumption The smallest K that yields 'predictive enough' accuracy is chosen without using the test set.
    Section 2.6 does not specify the data split used for this stopping rule; if test accuracy informed K, the reported accuracies are biased.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Interestingness First Classifiers." pith.science (2026). https://pith.science/paper/X6GC7G22

@misc{pith2026250819780,
  author       = {Pith},
  title        = {Pith review of: Interestingness First Classifiers},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/X6GC7G22}},
  note         = {Machine review of arXiv:2508.19780}
}
read the original abstract

Most machine learning models are designed to maximize predictive accuracy. In this work, we explore a different goal: building classifiers that are interesting. An ``interesting classifier'' is one that uses unusual or unexpected features, even if its accuracy is lower than the best possible model. For example, predicting room congestion from CO2 levels achieves near-perfect accuracy but is unsurprising. In contrast, predicting room congestion from humidity is less accurate yet more nuanced and intriguing. We introduce EUREKA, a simple framework that selects features according to their perceived interestingness. Our method leverages large language models to rank features by their interestingness and then builds interpretable classifiers using only the selected interesting features. Across several benchmark datasets, EUREKA consistently identifies features that are non-obvious yet still predictive. For example, in the Occupancy Detection dataset, our method favors humidity over CO2 levels and light intensity, producing classifiers that achieve meaningful accuracy while offering insights. In the Twin Papers dataset, our method discovers the rule that papers with a colon in the title are more likely to be cited in the future. We argue that such models can support new ways of knowledge discovery and communication, especially in settings where moderate accuracy is sufficient but novelty and interpretability are valued.

Figures

Figures reproduced from arXiv: 2508.19780 by the authors.

Figure 1
Figure 1. Comparison of ranking methods. Mean absolute error (MAE) of estimated Borda scores against the ground truth, as a function of the number of pairwise comparisons N. Both methods achieve similar accuracy. Lower is better. 2.4 Comparison with Active Ranking An alternative is active ranking from pairwise comparison [17], which adaptively selects which items to compare based on confidence bounds, similar to successive el… view at source ↗
Figure 2
Figure 2. Test accuracy as a function of the number of interesting features [PITH_FULL_IMAGE:figures/full_fig_p008_2.png] view at source ↗

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

68 extracted references · 58 canonical work pages

  1. [1]

    Adebayo, J

    J. Adebayo, J. Gilmer, M. Muelly, I. J. Goodfellow, M. Hardt, and B. Kim. Sanity checks for saliency maps. InAdvances in Neural Information Processing Systems 31: Annual Conference on Neural Infor- mation Processing Systems 2018, NeurIPS, pages 9525–9536, 2018

    work page 2018

  2. [2]

    N. Ailon. An active learning algorithm for ranking from pairwise preferences with an almost optimal query complexity.J. Mach. Learn. Res., 13:137–164, 2012

    work page 2012

  3. [3]

    S. Bach, A. Binder, G. Montavon, F. Klauschen, K.-R. Müller, and W. Samek. On pixel-wise explana- tions for non-linear classifier decisions by layer-wise relevance propagation.PLOS ONE, 10(7):e0130140, 2015

    work page 2015

  4. [4]

    R. A. Bradley and M. E. Terry. Rank analysis of incomplete block designs: I. the method of paired comparisons. Biometrika, 39(3/4):324–345, 1952

    work page 1952

  5. [5]

    T. B. Brown, B. Mann, N. Ryder, M. Subbiah, J. Kaplan, P. Dhariwal, A. Neelakantan, P. Shyam, G. Sastry, A. Askell, S. Agarwal, A. Herbert-Voss, G. Krueger, T. Henighan, R. Child, A. Ramesh, D. M. Ziegler, J. Wu, C. Winter, C. Hesse, M. Chen, E. Sigler, M. Litwin, S. Gray, B. Chess, J. Clark, C. Berner, S. McCandlish, A. Radford, I. Sutskever, and D. Amod...

    work page 2020

  6. [6]

    Cattelan

    M. Cattelan. Models for paired comparison data: A review with emphasis on dependent data.Statistical Science, 27(3):412–433, 2012. doi: 10.1214/12-STS396

    doi:10.1214/12-sts396 2012

  7. [7]

    M. Chai, E. Herron, E. Cervantes, and T. Ghosal. Exploring scientific hypothesis generation with mamba. In Proceedings of the 1st Workshop on NLP for Science (NLP4Science), pages 197–207. Association for Computational Linguistics, Nov. 2024. URL https://aclanthology.org/2024. nlp4science-1.17/

    work page 2024

  8. [8]

    Doshi-Velez and B

    F. Doshi-Velez and B. Kim. Towards a rigorous science of interpretable machine learning. arXiv, abs/1702.08608, 2017. URL http://arxiv.org/abs/1702.08608

    arXiv 2017

Show all 68 references
  1. [9]

    Edwards and N

    T. Edwards and N. S. Holtzman. A meta-analysis of correlations between depression and first person singular pronoun use.Journal of Research in Personality, 68:63–68, 2017

    2017

  2. [10]

    Esteva, B

    A. Esteva, B. Kuprel, R. A. Novoa, J. Ko, S. M. Swetter, H. M. Blau, and S. Thrun. Dermatologist-level classification of skin cancer with deep neural networks.nature, 542(7639):115–118, 2017

    2017

  3. [11]

    A. A. Freitas. On rule interestingness measures.Knowl. Based Syst., 12(5-6):309–315, 1999

    1999

  4. [12]

    Guidotti, A

    R. Guidotti, A. Monreale, S. Ruggieri, F. Turini, F. Giannotti, and D. Pedreschi. A survey of methods for explaining black box models.ACM Comput. Surv., 51(5):93:1–93:42, 2019

    2019

  5. [13]

    Gulshan, L

    V. Gulshan, L. Peng, M. Coram, M. C. Stumpe, D. Wu, A. Narayanaswamy, S. Venugopalan, K. Widner, T. Madams, J. Cuadros, et al. Development and validation of a deep learning algorithm for detection of diabetic retinopathy in retinal fundus photographs.jama, 316(22):2402–2410, 2016

    2016

  6. [14]

    Guyon, J

    I. Guyon, J. Weston, S. Barnhill, and V. Vapnik. Gene selection for cancer classification using support vector machines.Machine learning, 46(1):389–422, 2002

    2002

  7. [15]

    He and Z

    K. He and Z. Chen. From reasoning to learning: A survey on hypothesis discovery and rule learning with large language models, 2025. URLhttps://doi.org/10.48550/arXiv.2505.21935

    2025 doi

  8. [16]

    X. He, L. Liao, H. Zhang, L. Nie, X. Hu, and T. Chua. Neural collaborative filtering. InProceedings of the 26th International Conference on World Wide Web, WWW, pages 173–182. ACM, 2017

    2017

  9. [17]

    Heckel, N

    R. Heckel, N. B. Shah, K. Ramchandran, and M. J. Wainwright. Active ranking from pairwise comparisons and the futility of parametric assumptions. arXiv, abs/1606.08842, 2016. URL http: //arxiv.org/abs/1606.08842

    2016 arXiv

  10. [18]

    Heckel, M

    R. Heckel, M. Simchowitz, K. Ramchandran, and M. J. Wainwright. Approximate ranking from pairwise comparisons. InProceedings of the 21st International Conference on Artificial Intelligence and Statistics, AISTATS, pages 1057–1066. PMLR, 2018. 11

    2018

  11. [19]

    Hinton, L

    G. Hinton, L. Deng, D. Yu, G. E. Dahl, A.-r. Mohamed, N. Jaitly, A. Senior, V. Vanhoucke, P. Nguyen, T. N. Sainath, et al. Deep neural networks for acoustic modeling in speech recognition: The shared views of four research groups.IEEE Signal processing magazine, 29(6):82–97, 2012

    2012

  12. [20]

    Aleatoryandepistemicuncertaintyinprobabilityelicitationwithanexamplefromhazardous waste management.Reliability engineering & system safety, 54(2-3):217–223, 1996

    S.C.Hora. Aleatoryandepistemicuncertaintyinprobabilityelicitationwithanexamplefromhazardous waste management.Reliability engineering & system safety, 54(2-3):217–223, 1996

    1996

  13. [21]

    Hüllermeier and W

    E. Hüllermeier and W. Waegeman. Aleatoric and epistemic uncertainty in machine learning: an intro- duction to concepts and methods.Mach. Learn., 110(3):457–506, 2021

    2021

  14. [22]

    Y. Ke, K. George, K. Pandya, G. Großmann, D. Blumenthal, M. Sprang, S. Vollmer, and D. A. Selby. Biodisco: Multi-agent hypothesis generation with dual-mode evidence, iterative feedback and temporal evaluation, 2025. URLhttp://arxiv.org/abs/2508.01285

    2025

  15. [23]

    B. Kim, M. Wattenberg, J. Gilmer, C. J. Cai, J. Wexler, F. B. Viégas, and R. Sayres. Interpretability beyondfeatureattribution: Quantitativetestingwithconceptactivationvectors(TCAV). In Proceedings of the 35th International Conference on Machine Learning, ICML, pages 2673–2682, 2018

    2018

  16. [24]

    Matrixfactorizationtechniquesforrecommendersystems

    Y.Koren, R.Bell, andC.Volinsky. Matrixfactorizationtechniquesforrecommendersystems. Computer, 42(8):30–37, 2009

    2009

  17. [25]

    Linden, B

    G. Linden, B. Smith, and J. York. Amazon. com recommendations: Item-to-item collaborative filtering. IEEE Internet computing, 7(1):76–80, 2003

    2003

  18. [26]

    Z. C. Lipton. The mythos of model interpretability.Commun. ACM, 61(10):36–43, 2018

    2018

  19. [27]

    H. Liu, S. Huang, J. Hu, Y. Zhou, and C. Tan. Hypobench: Towards systematic and principled benchmarking for hypothesis generation. arXiv, abs/2504.11524, 2025. URL https://doi.org/10. 48550/arXiv.2504.11524

    2025 doi

  20. [28]

    H. Liu, Y. Zhou, M. Li, C. Yuan, and C. Tan. Literature meets data: A synergistic approach to hypothesis generation. InProceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), ACL, pages 245–281. Association for Computati...

    2025

  21. [29]

    X. Liu, X. Dong, X. Gao, Y. Feng, and X. Pang. Improving research idea generation through data: An empirical investigation in social science.arXiv, abs/2505.21396, 2025. URLhttps://doi.org/10. 48550/arXiv.2505.21396

    2025 doi

  22. [30]

    R. D. Luce.Individual choice behavior: A theoretical analysis. John Wiley and Sons, 1959

    1959

  23. [31]

    S. M. Lundberg and S. Lee. A unified approach to interpreting model predictions. InAdvances in Neural Information Processing Systems 30: Annual Conference on Neural Information Processing Systems 2017, NeurIPS, pages 4765–4774, 2017

    2017

  24. [32]

    Meier, S

    L. Meier, S. Van De Geer, and P. Bühlmann. The group lasso for logistic regression.Journal of the Royal Statistical Society Series B: Statistical Methodology, 70(1):53–71, 2008

    2008

  25. [33]

    T. Miller. Explanation in artificial intelligence: Insights from the social sciences.Artificial Intelligence, 267:1–38, 2019

    2019

  26. [34]

    R. K. Mothilal, A. Sharma, and C. Tan. Explaining machine learning classifiers through diverse coun- terfactual explanations. In In Proceedings of the 2020 Conference on Fairness, Accountability, and Transparency, FAT*, pages 607–617. ACM, 2020

    2020

  27. [35]

    Movva, K

    R. Movva, K. Peng, N. Garg, J. Kleinberg, and E. Pierson. Sparse autoencoders for hypothesis genera- tion. In Proceedings of the 42nd International Conference on Machine Learning, ICML, 2025

    2025

  28. [36]

    Negahban, S

    S. Negahban, S. Oh, and D. Shah. Rank centrality: Ranking from pairwise comparisons.Oper. Res., 65(1):266–287, 2017

    2017

  29. [37]

    M. E. J. Newman. Efficient computation of rankings from pairwise comparisons.J. Mach. Learn. Res., 24:238:1–238:25, 2023

    2023

  30. [38]

    D. Ofer, M. Linial, and D. Shahaf. Interfeat: An automated pipeline for finding interesting hypotheses in structured biomedical data.arXiv, abs/2505.13534, 2025. URLhttps://doi.org/10.48550/arXiv. 2505.13534. 12

    2025 doi arXiv

  31. [39]

    O’Neill, T

    C. O’Neill, T. Ghosal, R. Raileanu, M. Walmsley, T. D. Bui, K. Schawinski, and I. Ciuca. Sparks of science: Hypothesis generation using structured paper data. arXiv, abs/2504.12976, 2025. URL https://doi.org/10.48550/arXiv.2504.12976

    2025 doi arXiv

  32. [40]

    Paulo, A

    G. Paulo, A. Mallen, C. Juang, and N. Belrose. Automatically interpreting millions of features in large language models. In Proceedings of the 42nd International Conference on Machine Learning, ICML, 2025

    2025

  33. [41]

    E. Paulson. A sequential procedure for selecting the population with the largest mean from k normal populations. The Annals of Mathematical Statistics, pages 174–180, 1964

    1964

  34. [42]

    L. R. Rabiner. A tutorial on hidden markov models and selected applications in speech recognition. Proceedings of the IEEE, 77(2):257–286, 2002

    2002

  35. [43]

    Radford, J

    A. Radford, J. W. Kim, T. Xu, G. Brockman, C. McLeavey, and I. Sutskever. Robust speech recogni- tion via large-scale weak supervision. InProceedings of the 40th International Conference on Machine Learning, ICML, volume 202, pages 28492–28518. PMLR, 2023

    2023

  36. [44]

    why should I trust you?

    M. T. Ribeiro, S. Singh, and C. Guestrin. "why should I trust you?": Explaining the predictions of any classifier. InProceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, KDD, pages 1135–1144. ACM, 2016

    2016

  37. [45]

    M. T. Ribeiro, S. Singh, and C. Guestrin. Anchors: High-precision model-agnostic explanations. In Proceedings of the 32nd AAAI Conference on Artificial Intelligence, AAAI, pages 1527–1535. AAAI Press, 2018

    2018

  38. [46]

    Rude, E.-M

    S. Rude, E.-M. Gortner, and J. Pennebaker. Language use of depressed and depression-vulnerable college students. Cognition & Emotion, 18(8):1121–1133, 2004

    2004

  39. [47]

    R. Sato, M. Yamada, and H. Kashima. Twin papers: A simple framework of causal inference for citations via coupling. In Proceedings of the 31st ACM International Conference on Information & Knowledge Management, CIKM, pages 4444–4448. ACM, 2022

    2022

  40. [48]

    R. R. Selvaraju, M. Cogswell, A. Das, R. Vedantam, D. Parikh, and D. Batra. Grad-cam: Visual expla- nations from deep networks via gradient-based localization. InProceedings of the IEEE International Conference on Computer Vision, ICCV, pages 618–626. IEEE Computer Society, 2017

    2017

  41. [49]

    N. B. Shah and M. J. Wainwright. Simple, robust and optimal ranking from pairwise comparisons.J. Mach. Learn. Res., 18:1–38, 2018

    2018

  42. [50]

    Shrikumar, P

    A. Shrikumar, P. Greenside, and A. Kundaje. Learning important features through propagating acti- vation differences. In Proceedings of the 34th International Conference on Machine Learning, ICML, volume 70, pages 3145–3153. PMLR, 2017

    2017

  43. [51]

    Simonyan, A

    K. Simonyan, A. Vedaldi, and A. Zisserman. Deep inside convolutional networks: Visualising image classification models and saliency maps. InProceedings of the 2nd International Conference on Learning Representations, ICLR, Workshop Track Proceedings, 2014

    2014

  44. [52]

    N. R. Smalheiser. Rediscovering don swanson: The past, present and future of literature-based discovery. J. Data Inf. Sci., 2(4):43–64, 2017

    2017

  45. [53]

    UsingARROWSMITH:Acomputer-assistedapproachtoformulat- ing and assessing scientific hypotheses.Computer Methods and Programs in Biomedicine, 57(3):149–153,

    N.R.SmalheiserandD.R.Swanson. UsingARROWSMITH:Acomputer-assistedapproachtoformulat- ing and assessing scientific hypotheses.Computer Methods and Programs in Biomedicine, 57(3):149–153,

  46. [54]

    W. S. Spangler, A. D. Wilkins, B. J. Bachman, M. Nagarajan, T. Dayaram, P. J. Haas, S. Regenbogen, C. R. Pickering, A. Comer, J. N. Myers, I. Stanoi, L. Kato, A. Lelescu, J. J. Labrie, N. Parikh, A. M. Lisewski, L. A. Donehower, Y. Chen, and O. Lichtarge. Automated hypothesis ...

    2014

  47. [55]

    Sundararajan, A

    M. Sundararajan, A. Taly, and Q. Yan. Axiomatic attribution for deep networks. InProceedings of the 34th International Conference on Machine Learning, ICML, volume 70, pages 3319–3328. PMLR, 2017

    2017

  48. [56]

    D. R. Swanson. Fish oil, raynaud’s syndrome, and undiscovered public knowledge. Perspectives in Biology and Medicine, 30(1):7–18, 1986. 13

    1986

  49. [57]

    Sybrandt, M

    J. Sybrandt, M. Shtutman, and I. Safro. MOLIERE: automatic biomedical hypothesis generation system. In Proceedings of the 23rd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, KDD, pages 1633–1642. ACM, 2017

    2017

  50. [58]

    L. L. Thurstone. A law of comparative judgment.Psychological Review, 34(4):273–286, 1927

    1927

  51. [59]

    T. Vigen. Spurious Correlations. Hachette Books, New York, 2015. ISBN 9780316339452

    2015

  52. [60]

    Wachter, B

    S. Wachter, B. D. Mittelstadt, and C. Russell. Counterfactual explanations without opening the black box: Automated decisions and the gdpr.Harvard Journal of Law & Technology, 31(2):841–887, 2017

    2017

  53. [61]

    Q. Wang, D. Downey, H. Ji, and T. Hope. SciMON: Scientific inspiration machines optimized for novelty. In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), ACL, pages 279–299. Association for Computational Linguist...

    2024 doi

  54. [62]

    Xiong, E

    G. Xiong, E. Xie, A. H. Shariatmadari, S. Guo, S. Bekiranov, and A. Zhang. Improving scientific hypothesis generation with knowledge grounded large language models.arXiv, abs/2411.02382, 2024. URL https://doi.org/10.48550/arXiv.2411.02382

    2024 doi arXiv

  55. [63]

    Xiong, E

    G. Xiong, E. Xie, C. Williams, M. Kim, A. H. Shariatmadari, S. Guo, S. Bekiranov, and A. Zhang. Toward reliable scientific hypothesis generation: Evaluating truthfulness and hallucination in large language models.arXiv, abs/2505.14599, 2025. URLhttps://doi.org/10.48550/arXiv.2...

    2025 doi arXiv

  56. [64]

    Y. Yang, J. Wu, and Y. Yue. Robust hypothesis generation: Llm-automated language bias for inductive logic programming. CoRR, abs/2505.21486, 2025. URL https://doi.org/10.48550/arXiv.2505. 21486

    2025 doi

  57. [65]

    Yuan and Y

    M. Yuan and Y. Lin. Model selection and estimation in regression with grouped variables.Journal of the Royal Statistical Society Series B: Statistical Methodology, 68(1):49–67, 2006

    2006

  58. [66]

    Zabaleta and J

    M. Zabaleta and J. Lehman. Simulating tabular datasets through llms to rapidly explore hypotheses about real-world entities. arXiv, abs/2411.18071, 2024. URL https://doi.org/10.48550/arXiv. 2411.18071

    2024 doi arXiv

  59. [67]

    Y. Zhou, H. Liu, T. Srivastava, H. Mei, and C. Tan. Hypothesis generation with large language models. In Proceedings of the 1st Workshop on NLP for Science (NLP4Science), pages 117–139. Association for Computational Linguistics, Nov. 2024. URLhttps://aclanthology.org/2024.nlp4...

    2024

  60. [1998]

    URL https://pubmed.ncbi.nlm.nih.gov/9822851/

    doi: 10.1016/S0169-2607(98)00033-9. URL https://pubmed.ncbi.nlm.nih.gov/9822851/

Pith tools

Reviewed August 5, 2026 · model on record in the stance chip above.