Pith. sign in

REVIEW 4 major objections 5 minor 28 references

QuEst: Enhancing Estimates of Quantile-Based Distributional Measures Using Model Predictions

T0 review · 4 major / 5 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read QuEst estimates any quantile-based distributional measure with valid confidence intervals, and proves its tuned estimator is asymptotically never worse than real-data-only inference.

desk verdict A genuine extension of PPI to quantile-based distributional measures, with sound asymptotics but a load-bearing shared-distribution assumption and a sketchy proof for step weights. read the letter →

arxiv 2507.05220 v1 pith:4P5TOJRN submitted 2025-07-07 cs.LG stat.ML

Zhun Deng , Thomas P Zollo , Benjamin Eyre , Amogh Inamdar , David Madras , Richard Zemel This is my paper
classification cs.LGstat.ML MSC 62G0562G1562G20
keywords quantile-baseddistributionalmeasuresprediction-poweredinferenceconfidenceintervalsL-statisticsCVaRvariancereductionLLMauto-evaluationhybridestimation
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

QuEst is a method for estimating quantile-based distributional measures — tail risk such as CVaR, segment averages such as quartiles, and individual quantiles — by combining a small sample of expensive observed measurements with a large pool of cheap model predictions. The paper generalizes prediction-powered inference beyond means and single quantiles to every functional of the form $Q_\psi(F) = \int_0^1 \psi(p) F^{-1}(p)\,dp$, and proves an asymptotic central limit theorem for the hybrid estimator using the classical theory of L-statistics. Its central theoretical result is a variance identity: with the variance-minimizing mixing weight, the asymptotic variance is the classical observed-only variance minus a non-negative correction, so the estimator is never worse than using real data alone and is strictly better whenever the imputed values carry information about the true values. Experiments on wealth modeling, gene expression, opinion polling, and LLM evaluation show the largest gains precisely where observed data are scarcest.

What carries the argument

The central object is the quantile-based distributional measure, a functional of the quantile function $Q_\psi(F) = \int_0^1 \psi(p) F^{-1}(p)\,dp$ with $\psi \geq 0$ and $\int_0^1 \psi = 1$; CVaR, interval-VaR, and plain quantiles correspond to particular choices of $\psi$ (an indicator on a tail, an indicator on an interval, and a Dirac delta). The argument is carried by the L-statistic representation $Q_\psi(F_n) = \sum_{i=1}^n \left[\int_{(i-1)/n}^{i/n} \psi(p)\,dp\right] M_{(i)}$, which turns the estimator into an asymptotically linear statistic so the standard central-limit machinery applies; the hybrid estimator inherits this normality with a cross-covariance term $\eta_\psi(F, \tilde{F})$ between the observed and imputed statistics. The optimal mixing weight $\lambda$ minimizes a quadratic variance function and has a closed form, and the no-worse-than-classical result follows from completing the square, so the whole procedure has no user-set hyperparameters.

What would settle it

Simulate a setting where the labeled and unlabeled input pools come from different distributions, holding the imputation model fixed, and measure the empirical coverage of the QuEst 95% intervals: coverage dropping below nominal would show the equal-distribution assumption is load-bearing. A second check targets the quantile case, where the variance formula contains $\beta(1-\beta)/f^2(F^{-1}(\beta))$: on a distribution whose density is zero or discontinuous at the target quantile, test whether the estimated standard error stays finite and whether the intervals remain valid.

Watch

Extended reading notes

Core claim

The paper claims that any quantile-based distributional measure can be estimated in the prediction-powered paradigm. Given $n$ observed pairs and $N \gg n$ imputed pairs whose inputs share a distribution, the estimator $\hat{Q}_\psi(\lambda) = \lambda Q_\psi(\tilde{F}^u_N) + Q_\psi(F_n) - \lambda Q_\psi(\tilde{F}_n)$ is asymptotically unbiased and asymptotically normal with variance $\rho^2_\psi(\lambda, F, \tilde{F}) = \lambda^2(1 + r)\sigma^2_\psi(\tilde{F}) + \sigma^2_\psi(F) - 2\lambda\eta_\psi(F, \tilde{F})$, where $r = \lim n/N$ and $\eta_\psi$ is the covariance between the observed and imputed statistics. Because this is quadratic in $\lambda$, the variance-minimizing weight has a closed form, and substituting the empirical CDFs yields $\rho^2_\psi(\hat\lambda, F_n, \tilde{F}_n, \tilde{F}^u_N) = \sigma^2_\psi(F_n) - \eta^2_\psi / ((1 + n/N)\sigma^2_\psi(\tilde{F}^u_N))$. The second term is non-negative, so the tuned QuEst estimator never has larger asymptotic variance than the classical observed-only estimator; the paper also extends the construction to joint inference over several measures or metrics, and to a parameterized imputed-side weighting function whose coefficients are chosen by convex optimization, for which a central limit theorem is proved.

Load-bearing premise

The load-bearing premise is that the unlabeled inputs used to build the large imputed pool come from the same distribution as the labeled inputs, so the imputed CDF estimated on the big pool matches the imputed CDF of the labeled sample; if that fails, the bias-correction term is computed on the wrong distribution and the confidence intervals can lose coverage.

Editorial extensions

If this is right

  • Any measure of the form $Q_\psi(F) = \int_0^1 \psi(p)F^{-1}(p)\,dp$ — CVaR, interval-VaR such as quartile averages, or a single quantile — can be estimated from $n$ observed plus $N$ imputed samples with a closed-form optimal mixing weight and no user-set hyperparameters.
  • The tuned estimator's asymptotic variance equals the classical observed-only variance minus a non-negative correction, so QuEst's confidence intervals are never wider than the observed-only ones and are strictly tighter whenever imputed values correlate with observed values.
  • Empirically, gains concentrate where data are scarce: with 100 gold-standard labels on GeneExpression, QuEst roughly halves both estimation error and interval width relative to observed-only inference while maintaining nominal coverage, and the advantage fades as the observed sample grows.
  • The multidimensional version yields joint confidence regions whose volume is smaller than that of the corresponding univariate constructions — 34% smaller than the classical univariate volume and 7% smaller than the classical multivariate volume in the news-summarization experiment.
  • The adaptive-weighting extension (QuEst-Opt) preserves the asymptotic guarantees while further reducing variance when imputation error is heteroskedastic, with the largest improvements in the few-observed-sample regime.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • A budget rule follows from the variance identity that the paper leaves implicit: the benefit of QuEst shrinks as the observed sample grows, so the method is best understood as a scarce-data instrument, and the identity itself says when additional gold-standard labels stop paying for themselves.
  • The convex weight-optimization trick behind the QuEst-Opt extension — choosing the imputed-side weighting function from a flexible basis to maximize covariance cancellation — is a generic recipe that should transfer to other hybrid estimators outside the quantile-functional family, a direction the paper opens but does not develop.
  • In model-selection settings, a testable consequence is that ranking accuracy per unit of labeling budget should rise as the observed–imputed correlation rises; the paper's toxicity experiments point in that direction without formalizing the trade-off.
  • Because every guarantee is asymptotic, the natural stress test for the low-$n$ regime where QuEst shows its largest gains is finite-sample calibration of the normal-based intervals, for example by comparing them with bootstrap intervals, which the paper does not report.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. The paper introduces QuEst, a framework for estimating quantile-based distributional measures (QBDMs) — such as CVaR, interval-VaR, and quantiles — by combining a small sample of ground-truth observations with a larger pool of model-imputed predictions. The estimator is a weighted combination of an imputed-data QBDM and a debiased observed-data QBDM, with the mixing weight λ chosen to minimize asymptotic variance. The authors derive asymptotic normality via L-statistics, provide a closed-form for the optimal λ, extend the approach to multidimensional settings, and propose QuEst-Opt, which optimizes a basis-function weighting over the imputed data. Experiments on PovertyMap, GeneExpression, OpinionQA, LLM toxicity red-teaming, and news summarization show improved point estimates and tighter confidence intervals relative to observed-only baselines.

Significance. If the theoretical claims hold, QuEst fills a real gap: existing prediction-powered inference tools target M-estimators, means, or single quantiles, while QBDMs such as CVaR and interval-VaR are central in economics, finance, and risk assessment. The variance-never-exceeds-classical result is elegant and follows cleanly from the quadratic form of the variance and Cauchy-Schwarz. The paper also releases code, which supports reproducibility, and the experiments span diverse and realistic applications. The main concerns are that the theory is not fully rigorous for the discontinuous weight functions used in the headline applications, and the central unbiasedness guarantee depends on an unvalidated same-distribution assumption for the unlabeled pool. If these gaps are addressed, the framework would be a strong contribution to hybrid inference.

major comments (4)
  1. [Section 3.1] The assumption that unlabeled inputs X^u_i are drawn from the same distribution as labeled inputs X_i is load-bearing. The estimator subtracts λ Qψ(\tilde F_n) while adding λ Qψ(\tilde F^u_N); under distribution shift these empirical CDFs converge to different limits, leaving an asymptotic bias λ*(Qψ(\tilde F^u) - Qψ(\tilde F)) that does not vanish. This invalidates the unbiasedness and confidence-interval guarantees in Theorem 2 and Corollary 3. The manuscript states the assumption but provides no sensitivity analysis, no diagnostic, and no fallback for the common case where the unlabeled pool is collected under different conditions (e.g., PovertyMap's cross-country test data). Please add a discussion of this failure mode and, ideally, a diagnostic or an alternative correction.
  2. [Appendix C.1.2] The proof of the CLT for step-function weight functions (CVaR, Interval-VaR) is only a sketch. The argument 'specify a class of smooth functions ψ_k and apply dominated convergence' does not control the remainder in the approximation of Qψ(F_n) by Qψ_k(F_n), nor does it verify that the variance functional σ^2_ψ(F) is continuous under this approximation. The bias calculation in Appendix C.1.1 uses a Taylor expansion of ψ that requires twice continuous differentiability, which fails exactly for the main examples. Since Theorem 2 and Corollary 3 are claimed for 'almost everywhere bounded and smooth functions with finite discontinuous points', the proof must be completed with a careful L-statistic argument (or a citation to classical results with explicit conditions).
  3. [Section 3.2.1, Corollary 3] The plug-in of \hat λ into the CLT is asserted via Slutsky's rule, but the required regularity conditions are not stated. In particular, one needs convergence of \hat λ to λ* with the denominator σ^2_ψ(\tilde F^u_N) bounded away from zero, consistency of the estimated variance ρ^2_ψ(\hat λ, F_n, \tilde F_n, \tilde F^u_N), and a uniform CLT in a neighborhood of λ*. Without these, the confidence intervals used in the experiments lack justification. Please provide the precise conditions and a proof.
  4. [Theorem 4 and Theorem 5] The multidimensional CLT (Theorem 4) is stated without proof; Appendix C.4 only lists covariance formulas. The QuEst-Opt CLT (Theorem 5) is proved in Appendix C.5 under the assumption that ψ is twice continuously differentiable almost everywhere, which excludes the step-function measures studied in the paper, and the proof's first step relies on \tilde F^u = \tilde F, inheriting the sensitivity of comment 1. The scope of these extensions should be clarified or the assumptions relaxed.
minor comments (5)
  1. [Section 3.2] The notation ηψ(F, \tilde F) = Cov(Qψ(F), Qψ(\tilde F)) is abusive because Qψ(F) is a parameter, not a random variable; the definition in Appendix C.2 is an asymptotic covariance. Please use notation such as ηψ(F, \tilde F) = lim n Cov(Qψ(F_n), Qψ(\tilde F_n)).
  2. [Section 3.2] The estimator definition and variance formulas mix arguments: ρ^2_ψ is defined with three arguments in Theorem 2 but with four arguments in the empirical version, and the λ formula uses σ^2_ψ(\tilde F^u_N) while the theorem uses σ^2_ψ(\tilde F). Please make the notation consistent.
  3. [Figure 2] The error bars are said to be too small to observe; report the standard errors or show them on a separate plot, and state the number of trials used to compute each average.
  4. [Appendix A.1] PovertyMap is described as having a train/test distribution shift with test data from separate countries; this should be tied to the same-distribution assumption in Section 3.1 and the sensitivity discussion in the paper.
  5. [Section 3.2.1 and Section 5] The statement that 'QuEst involves no hyperparameters' is not fully accurate because QuEst-Opt requires choosing basis functions and a regularization constant α, and clipping λ to [0,1] is a user choice. Please soften the claim accordingly.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: QuEst's hybrid estimator targets an external QBDM functional; lambda and xi are variance-minimization parameters, not fitted to reproduce the target, and the theoretical results rest on classical L-statistics rather than the authors' prior conclusions.

full rationale

QuEst's target Qpsi(F)=integral psi(p)F^{-1}(p)dp is defined independently of the estimator, and the estimator Qhat_psi(lambda)=lambda Qpsi(Ftilde_N^u)+Qpsi(F_n)-lambda Qpsi(Ftilde_n) is a debiasing combination of L-statistics. The mixing weight lambda is chosen by minimizing the asymptotic variance rho^2_psi(lambda,F_n,Ftilde_n,Ftilde_N^u), not by matching the observed value of Qpsi; the displayed identity rho^2_psi(lambda_hat,...)=sigma^2_psi(F_n)-eta^2/((1+n/N)sigma^2_psi(Ftilde_N^u)) is the exact minimum of a convex quadratic in lambda and is a derivation rather than a redefinition. The same holds for the Section 5 extension, where psi_xi=xi^T phi is a free basis expansion optimized for variance with a fixed alpha-regularizer; the CLT for the data-dependent xi is proved from first-order optimality and strong convexity, not imported from a prior QuEst paper. The L-statistic CLTs are cited to Van der Vaart (2000) and Aaronson et al. (1996), external sources; author self-citations (Eyre and Madras 2024, Deng et al. 2024, Zollo et al. 2023, Snell et al. 2022) are contextual and do not carry the derivation. The Section 3.1 assumption that unlabeled inputs share the label distribution is stated explicitly; if violated the bias no longer cancels, but that is a correctness and robustness risk, not circularity. Because the central estimator, variance formula, and CI validity are derived from classical asymptotic expansions and verified on external datasets against observed-only, imputed-only, and PPI baselines, no step reduces to its own inputs.

Assumptions & free parameters 3 free parameters · 4 assumptions · 0 invented entities

The central claim rests on standard L-statistics CLT and the i.i.d. same-distribution assumption for labeled and unlabeled inputs. No new entities are postulated. The method introduces a data-driven mixing weight λ (and, in the extension, a coefficient vector ξ and regularization constant α), which are tuning artifacts rather than parameters of the phenomenon; they do not create circularity because they are selected to minimize asymptotic variance, not to match the target measure.

free parameters (3)
  • λ (mixing weight) = closed form η/((1+n/N)σ²) estimated from data
    Chosen to minimize estimated asymptotic variance; this is a data-derived statistic, not a constant of the target distribution.
  • ξ (basis coefficients in QuEst-Opt) = 30-dimensional vector in experiments (sinusoidal basis)
    Optimized on the same data to minimize estimated variance with α-regularization; not used to fit the target QBDM.
  • α (regularization constant) = not specified in experiments
    Any fixed positive constant is allowed by theory; the chosen value is not reported.
assumptions (4)
  • standard math Classical L-statistics CLT (Van der Vaart Theorem 22.3)
    Invoked in Section 3.2 and Appendix C.1 to establish asymptotic normality of Qψ(Fn).
  • domain assumption Labeled and unlabeled inputs are i.i.d. from the same distribution
    Section 3.1: 'We assume that each unlabeled input X^u_i is drawn from the same distribution as the labeled inputs Xi.' This makes the imputed CDFs equal, essential for debiasing.
  • domain assumption Regularity conditions: bounded smooth ψ or ψ with finitely many discontinuities, finite second moments, and for the Dirac delta case a positive smooth density
    Stated in Theorems 2, 4, 5 as 'under certain regularity conditions' and in Appendix C.1; required for the CLT and variance formulas.
  • domain assumption Consistency of plug-in variance estimators (η, σ²)
    Corollary 3 requires the empirical variance functional to be consistent and the estimated λ to converge to a constant; no explicit proof beyond 'standard regularity conditions'.

how reviews work

0 comments
Cite this review

Pith. "Pith review of QuEst: Enhancing Estimates of Quantile-Based Distributional Measures Using Model Predictions." pith.science (2026). https://pith.science/paper/4P5TOJRN

@misc{pith2026250705220,
  author       = {Pith},
  title        = {Pith review of: QuEst: Enhancing Estimates of Quantile-Based Distributional Measures Using Model Predictions},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/4P5TOJRN}},
  note         = {Machine review of arXiv:2507.05220}
}
read the original abstract

As machine learning models grow increasingly competent, their predictions can supplement scarce or expensive data in various important domains. In support of this paradigm, algorithms have emerged to combine a small amount of high-fidelity observed data with a much larger set of imputed model outputs to estimate some quantity of interest. Yet current hybrid-inference tools target only means or single quantiles, limiting their applicability for many critical domains and use cases. We present QuEst, a principled framework to merge observed and imputed data to deliver point estimates and rigorous confidence intervals for a wide family of quantile-based distributional measures. QuEst covers a range of measures, from tail risk (CVaR) to population segments such as quartiles, that are central to fields such as economics, sociology, education, medicine, and more. We extend QuEst to multidimensional metrics, and introduce an additional optimization technique to further reduce variance in this and other hybrid estimators. We demonstrate the utility of our framework through experiments in economic modeling, opinion polling, and language model auto-evaluation.

Figures

Figures reproduced from arXiv: 2507.05220 by the authors.

Figure 1
Figure 1. Estimates derived from small gold-standard (observed) datasets may be noisy, while [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 2
Figure 2. Experimental results for estimating Interval-VaR and CVaR using three datasets (Pover [PITH_FULL_IMAGE:figures/full_fig_p011_2.png] view at source ↗
Figure 3
Figure 3. Experimental results for estimating VaR on two datasets (PovertyMap, GeneExpression). [PITH_FULL_IMAGE:figures/full_fig_p012_3.png] view at source ↗
Figures from the paper (6 more)
Figure 4
Figure 4. Figure 4: LLM auto-evaluation on a red-teaming toxicity task. [PITH_FULL_IMAGE:figures/full_fig_p013_4.png]
Figure 5
Figure 5. Figure 5: LLM auto-evaluation on a red-teaming toxicity task. Each point corresponds to one model; [PITH_FULL_IMAGE:figures/full_fig_p013_5.png]
Figure 6
Figure 6. Figure 6: Visualization of a two-dimensional 90% confidence region defined using different esti [PITH_FULL_IMAGE:figures/full_fig_p015_6.png]
Figure 7
Figure 7. Figure 7: QuEst-Opt outperforms QuEst on synthetic datasets that exhibit heteroskedasticity (varying [PITH_FULL_IMAGE:figures/full_fig_p017_7.png]
Figure 8
Figure 8. Figure 8: Coverage results across 3 datasets and using different numbers of observed data. [PITH_FULL_IMAGE:figures/full_fig_p025_8.png]
Figure 9
Figure 9. Figure 9: QuEst-Opt performs comparably to or better than QuEst on real-world datasets. [PITH_FULL_IMAGE:figures/full_fig_p026_9.png]

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

28 extracted references · 13 canonical work pages

  1. [1]

    Strong laws for l-and u-statistics

    J Aaronson, R Burton, H Dehling, D Gilat, T Hill, and B Weiss. Strong laws for l-and u-statistics. Transactions of the American Mathematical Society, 348 0 (7): 0 2845--2866, 1996

    work page 1996

  2. [2]

    Angelopoulos, Stephen Bates, Clara Fannjiang, Michael I

    Anastasios N. Angelopoulos, Stephen Bates, Clara Fannjiang, Michael I. Jordan, and Tijana Zrnic. Prediction-powered inference, 2023. URL https://arxiv.org/abs/2301.09633

    arXiv 2023

  3. [3]

    Angelopoulos, John C

    Anastasios N. Angelopoulos, John C. Duchi, and Tijana Zrnic. Ppi++: Efficient prediction-powered inference, 2024. URL https://arxiv.org/abs/2311.01453

    arXiv 2024

  4. [4]

    Argyle, Ethan C

    Lisa P. Argyle, Ethan C. Busby, Nancy Fulda, Joshua R. Gubler, Christopher Rytting, and David Wingate. Out of one, many: Using language models to simulate human samples. Political Analysis, 31 0 (3): 0 337–351, February 2023. ISSN 1476-4989. doi:10.1017/pan.2023.2. URL http://dx.doi.org/10.1017/pan.2023.2

    doi:10.1017/pan.2023.2 2023

  5. [5]

    Angelopoulos, Nir Yosef, Jitendra Malik, and Michael I

    Pierre Boyeau, Anastasios N. Angelopoulos, Nir Yosef, Jitendra Malik, and Michael I. Jordan. Autoeval done right: Using synthetic data for model evaluation, 2024. URL https://arxiv.org/abs/2403.07008

    arXiv 2024

  6. [6]

    Distribution-Free Statistical Dispersion Control for Societal Applications

    Zhun Deng, Thomas P. Zollo, Jake C. Snell, Toniann Pitassi, and Richard Zemel. Distribution-free statistical dispersion control for societal applications, 2024. URL https://arxiv.org/abs/2309.13786

    work page Pith review arXiv 2024

  7. [7]

    Questioning the survey responses of large language models, 2024

    Ricardo Dominguez-Olmedo, Moritz Hardt, and Celestine Mendler-Dünner. Questioning the survey responses of large language models, 2024. URL https://arxiv.org/abs/2306.07951

    arXiv 2024

  8. [8]

    Arias, Samuel Lantei Mills, and Albert Motivans

    Jessica Espey, Eric Swanson, Shaida Badiee, Zach Christensen, Alex Fischer, Marc Levy, Greg Yetman, Alex de Sherbinin, Robert Chen, Yue Qiu, Geoffrey Greenwell, Thilo Klein, Johannes Jütting, Morten Jerven, Grant Cameron, Ana Milena Aguilar Rivera, Victoriano C. Arias, Samuel Lantei Mills, and Albert Motivans. Data for development: A needs assessment for ...

    work page 2015

Show all 28 references
  1. [9]

    Auto-evaluation with few labels through post-hoc regression, 2024

    Benjamin Eyre and David Madras. Auto-evaluation with few labels through post-hoc regression, 2024. URL https://arxiv.org/abs/2411.12665

    2024 arXiv

  2. [10]

    Out of the ordinary: Spectrally adapting regression for covariate shift, 2023

    Benjamin Eyre, Elliot Creager, David Madras, Vardan Papyan, and Richard Zemel. Out of the ordinary: Spectrally adapting regression for covariate shift, 2023. URL https://arxiv.org/abs/2312.17463

    2023 arXiv

  3. [11]

    Alex Hofer, Bhuwan Dhingra, Amir Globerson, and William W

    Adam Fisch, Joshua Maynez, R. Alex Hofer, Bhuwan Dhingra, Amir Globerson, and William W. Cohen. Stratified prediction-powered inference for hybrid language model evaluation, 2024. URL https://arxiv.org/abs/2406.04291

    2024 arXiv

  4. [12]

    Red teaming language models to reduce harms: Methods, scaling behaviors, and lessons learned, 2022

    Deep Ganguli, Liane Lovitt, Jackson Kernion, Amanda Askell, Yuntao Bai, Saurav Kadavath, Ben Mann, Ethan Perez, Nicholas Schiefer, Kamal Ndousse, Andy Jones, Sam Bowman, Anna Chen, Tom Conerly, Nova DasSarma, Dawn Drain, Nelson Elhage, Sheer El-Showk, Stanislav Fort, Zac Hatfi...

    2022 arXiv

  5. [13]

    Alex Hofer, Joshua Maynez, Bhuwan Dhingra, Adam Fisch, Amir Globerson, and William W

    R. Alex Hofer, Joshua Maynez, Bhuwan Dhingra, Adam Fisch, Amir Globerson, and William W. Cohen. Bayesian prediction-powered inference, 2024. URL https://arxiv.org/abs/2405.06034

    2024 arXiv

  6. [14]

    John J. Horton. Large language models as simulated economic agents: What can we learn from homo silicus?, 2023. URL https://arxiv.org/abs/2301.07543

    2023

  7. [15]

    John M. Jumper, Richard Evans, Alexander Pritzel, Tim Green, Michael Figurnov, Olaf Ronneberger, Kathryn Tunyasuvunakool, Russ Bates, Augustin Ž \'i dek, Anna Potapenko, Alex Bridgland, Clemens Meyer, Simon A A Kohl, Andy Ballard, Andrew Cowie, Bernardino Romera-Paredes, Stani...

    2021

  8. [16]

    Earnshaw, Imran S

    Pang Wei Koh, Shiori Sagawa, Henrik Marklund, Sang Michael Xie, Marvin Zhang, Akshay Balsubramani, Weihua Hu, Michihiro Yasunaga, Richard Lanas Phillips, Irena Gao, Tony Lee, Etienne David, Ian Stavness, Wei Guo, Berton A. Earnshaw, Imran S. Haque, Sara Beery, Jure Leskovec, A...

    2021 arXiv

  9. [17]

    G-eval: Nlg evaluation using gpt-4 with better human alignment

    Yang Liu, Dan Iter, Yichong Xu, Shuohang Wang, Ruochen Xu, and Chenguang Zhu. G-eval: Nlg evaluation using gpt-4 with better human alignment. arXiv preprint arXiv:2303.16634, 2023

    2023 arXiv

  10. [18]

    Cohen, and Mirella Lapata

    Shashi Narayan, Shay B. Cohen, and Mirella Lapata. Don`t give me the details, just the summary! topic-aware convolutional neural networks for extreme summarization. In Ellen Riloff, David Chiang, Julia Hockenmaier, and Jun ' ichi Tsujii, editors, Proceedings of the 2018 Confer...

    2018 doi

  11. [19]

    Trends in income and wealth inequality, January 2020

    Pew Research Center . Trends in income and wealth inequality, January 2020. URL https://www.pewresearch.org/social-trends/2020/01/09/trends-in-income-and-wealth-inequality. Accessed: 2024-12-11

    2020

  12. [20]

    Conditional value-at-risk for general loss distributions

    R Tyrrell Rockafellar and Stanislav Uryasev. Conditional value-at-risk for general loss distributions. page 29, 2002

    2002

  13. [21]

    Snell, Thomas P

    Jake C. Snell, Thomas P. Zollo, Zhun Deng, Toniann Pitassi, and Richard Zemel. Quantile risk control: A flexible framework for bounding the probability of high-loss predictions, 2022. URL https://arxiv.org/abs/2212.13629

    2022 arXiv

  14. [22]

    Little, and Rajiv McCoy

    Dylan Taylor, Surya Chhetri, Michael Tassia, Arjun Biddanda, Stephanie Yan, Genevieve Wojcik, A. Little, and Rajiv McCoy. Sources of gene expression variation in a globally diverse human cohort. Nature, 632: 0 122--130, 07 2024. doi:10.1038/s41586-024-07708-2

    2024 doi

  15. [23]

    The evolution, evolvability and engineering of gene regulatory dna

    Eeshit Dhaval Vaishnav, Carl G de Boer, Jennifer Molinet, Moran Yassour, Lin Fan, Xian Adiconis, Dawn A Thompson, Joshua Z Levin, Francisco A Cubillos, and Aviv Regev. The evolution, evolvability and engineering of gene regulatory dna. Nature, 603 0 (7901): 0 455—463, March 20...

    2022 doi

  16. [24]

    Asymptotic statistics, volume 3

    Aad W Van der Vaart. Asymptotic statistics, volume 3. Cambridge university press, 2000

    2000

  17. [25]

    Lobell, Stefano Ermon, and M

    Christopher Yeh, Anthony Perez, Anne Driscoll, George Azzari, Zhongyi Tang, D. Lobell, Stefano Ermon, and M. Burke. Using publicly available satellite imagery and deep learning to understand economic well-being in africa. Nature Communications, 11, 2020. URL https://api.semant...

    2020

  18. [26]

    Benchmarking large language models for news summarization

    Tianyi Zhang, Faisal Ladhak, Esin Durmus, Percy Liang, Kathleen McKeown, and Tatsunori B Hashimoto. Benchmarking large language models for news summarization. Transactions of the Association for Computational Linguistics, 12: 0 39--57, 2024

    2024

  19. [27]

    Judging llm-as-a-judge with mt-bench and chatbot arena

    Lianmin Zheng, Wei-Lin Chiang, Ying Sheng, Siyuan Zhuang, Zhanghao Wu, Yonghao Zhuang, Zi Lin, Zhuohan Li, Dacheng Li, Eric Xing, et al. Judging llm-as-a-judge with mt-bench and chatbot arena. Advances in Neural Information Processing Systems, 36: 0 46595--46623, 2023

    2023

  20. [28]

    Prompt risk control: A rigorous framework for responsible deployment of large language models

    Thomas P Zollo, Todd Morrill, Zhun Deng, Jake C Snell, Toniann Pitassi, and Richard Zemel. Prompt risk control: A rigorous framework for responsible deployment of large language models. arXiv preprint arXiv:2311.13628, 2023

    2023 arXiv

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.