Assessing GPT Model Uncertainty in Mathematical OCR Tasks via Entropy Analysis

Alexei Kaltchenko

REVIEW 4 major objections 4 minor 27 references

Assessing GPT Model Uncertainty in Mathematical OCR Tasks via Entropy Analysis

T0 review · 4 major / 4 minor · reviewed 2026-08-12 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read The paper claims that GPT uncertainty in math OCR scales with image resolution, measurable by token entropy.

desk verdict The paper's empirical trend is real, but it's negative log-likelihood, not entropy, so the central claim is unsupported. read the letter →

arxiv 2412.01221 v2 pith:JYKIHIVV submitted 2024-12-02 cs.IT math.IT

Alexei Kaltchenko This is my paper · ORCID

classification cs.ITmath.IT MSC 94A17

keywords GPTentropyanalysismathematicalOCRLaTeXrecognitionimageresolutionuncertaintyquantificationconditionalmutualinformation

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper claims that the uncertainty of a GPT model extracting LaTeX from images of math equations can be read directly from the entropy of its generated tokens: higher-resolution images produce lower entropy, and lower-resolution images produce higher entropy and more errors. The authors model the OCR task as a noisy information channel, identify the sum of negative log probabilities of generated tokens with the conditional entropy H(Y|X) of the recognized text given the image, and support the resolution trend with the Data Processing Inequality and Fano's Inequality. In experiments with GPT-4o on a page converted at 300, 150, 96, and 72 dpi, total entropy rises from 7.77 bits to 38.91 bits while accuracy falls from perfect to severely degraded. The claim matters because it offers a token-level, ground-truth-free uncertainty signal for a practical OCR use case.

What carries the argument

The central object is the conditional entropy of the generated token sequence, H(y|x) = -Σ log2 P_i(y_i | x, y_{1:i-1}), summed over output positions, together with its normalized version dividing by n log2|V|. The paper identifies this empirically computed quantity with the theoretical conditional entropy H(Y|X) of the OCR channel, and uses three information-theoretic tools to explain the experiments: the identity I(X;Y) = H(Y) - H(Y|X) with H(Y) assumed constant for fixed text, the Data Processing Inequality applied to resolution reduction as a deterministic function, and Fano's Inequality H(Y|X) ≥ H(P_e) + P_e log(|Y|-1) linking conditional entropy to error probability. This machinery turns token-level log probabilities into a claimed measure of recognition uncertainty and error likelihood.

What would settle it

Compute the true next-token entropy at each position from the full vocabulary distribution (not just the sampled token's probability) for the same four images, and check whether total entropy still rises monotonically from 300 to 72 dpi; if it does not, the paper's trend is an artifact of the negative-log-likelihood approximation rather than a property of the model's uncertainty.

Watch

Extended reading notes

Core claim

The paper's central claim is that, for GPT-based mathematical OCR, the conditional Shannon entropy of the output token sequence is a faithful proxy for the model's uncertainty, and that this entropy is governed by input image resolution through the information-theoretic relation I(X;Y) = H(Y) - H(Y|X) with H(Y) approximately constant for a fixed text. Empirically, using the GPT-4o model with temperature zero and logprobs enabled, the paper reports a monotone increase in total entropy as resolution drops: 7.77 bits at 300 dpi, 12.28 at 150 dpi, 18.27 at 96 dpi, and 38.91 bits at 72 dpi, with normalized per-token entropy rising from 0.0171 to 0.082 bits. The authors take this as validating that lower resolution reduces mutual information between image and text, raising conditional entropy, and that higher conditional entropy implies higher error probability by Fano's Inequality. The contribution is the direct application of conditional entropy computed from token log-probabilities as a quantitative uncertainty measure in math-OCR.

Load-bearing premise

The argument collapses if the sum of negative log probabilities of the sampled tokens is not a good stand-in for the model's true conditional entropy: for example, if the model's reported per-token probabilities are miscalibrated or if the missing probability mass over the rest of the vocabulary is large.

Editorial extensions

If this is right

In deployed math-OCR systems, normalized token entropy can flag low-confidence extractions for human review without needing a reference transcription.
Image resolution becomes a tunable parameter with a measurable uncertainty cost: below roughly 100 dpi, the reported entropy rises sharply, so a quality floor can be set from entropy data.
Because the entropy is computed from a single generation, it offers a cheap, per-output uncertainty estimate for closed models that expose logprobs, no ensembles or internal states required.
The monotone relation between resolution and entropy gives a concrete validation target for other GPT-based OCR models: the same experimental protocol can compare uncertainty calibration across models.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

The paper does not establish whether the entropy-accuracy link is monotone at finer resolution gradations or across varied document layouts; a natural extension is testing multiple pages and fonts to see if a single normalized-entropy threshold predicts errors.
The reported total entropy mixes sequence length with per-token confidence, so a stricter test would recompute entropy from the full vocabulary distribution at each position; if the trend vanishes, the result is an artifact of negative log-likelihood rather than true entropy.
One implication the authors leave implicit is that entropy could drive an adaptive OCR pipeline, re-rendering an image at higher resolution whenever normalized entropy exceeds a threshold.
The Data Processing Inequality argument holds for deterministic downsampling; stochastic degradation such as blur or noise adds randomness that may break the simple monotone claim, a testable edge case.

Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

Colleague,

Here's my read of arXiv:2412.01221. The empirical observation is real: GPT-4o's summed negative log-likelihood over its greedy output for a fixed math PDF page increases as resolution drops (7.77 to 38.91 bits). That's a plausible, reproducible practical finding, and the GitHub repo and clear write-up are in its favor.

What's not real is the paper's framing. Section 3 develops theory for the conditional entropy H(Y|X). Section 4.6 asserts a 'direct relation' between that and the 'entropy' H(y|x) computed in the experiments. But the experiments use Eq (21): Hi = -log2 Pi(yi | ...), the self-information of the greedily chosen token, not the Shannon entropy of the next-token distribution from Eq (13). These are different: the latter is an expectation over the vocabulary. So the measured numbers are negative log-likelihood sums, not estimates of H(Y|X). The monotonic trend does not validate the entropy-based theory. This is the load-bearing flaw.

The theoretical derivation also has a soft spot: Eq (6), the claim that high-res increases mutual information, is asserted, not proved. DPI actually supports the direction of the inequality, but the paper's application is loose because the model's output sequence changes with resolution, so the comparison isn't a clean DPI setup. Fano's inequality is used in the direction low H implies low Pe, which is fine as a contrapositive, so I'd not count that as an additional error. But it doesn't rescue the entropy identification.

Other weaknesses: one page, four resolutions, no repeated trials or error bars, accuracy judged visually, no comparison across models or PDFs. The claimed 'gap' in the literature is asserted, not demonstrated; semantic entropy and related work already occupy that space.

If the authors reframed the paper as 'negative log-likelihood increases with downsampling for GPT-4o on one page,' it would be a modest empirical note. As written, the central claim about entropy is unsupported, and the theory section doesn't connect to the measurements. I'd desk reject it; it needs a complete rework to be a credible contribution. Not worth a serious referee slot in its current form.

Referee Report

4 major / 4 minor

Summary. The paper proposes to quantify the uncertainty of GPT models in converting images of mathematical equations into LaTeX code by using entropy measures. The theoretical part (Sections 2--3) models OCR as a noisy channel and argues that higher image resolution increases mutual information I(X;Y) and decreases conditional entropy H(Y|X), relying on the Data Processing Inequality and Fano's Inequality. The experimental part (Section 5) uses GPT-4o on one arXiv page rendered at 72, 96, 150, and 300 dpi, computes a quantity called total entropy as the sum of negative log-probabilities of the greedily generated tokens (Eq. 21), and reports that this quantity increases monotonically as resolution decreases. The paper concludes that entropy analysis empirically validates the theoretical framework and that entropy is a practical uncertainty measure for GPT-based mathematical OCR.

Significance. If the central claim were sound, the paper would provide a simple information-theoretic explanation and diagnostic for resolution-dependent failures of LLM-based math OCR, which is a practically relevant task. The paper ships code and uses a publicly available source document, which is a positive reproducibility feature. However, the load-bearing identification of the measured quantity with Shannon entropy is incorrect, and the theoretical derivation contains unsupported and misapplied inequalities. As presented, the paper does not establish either that entropy (in the Shannon sense) decreases with resolution or that the reported monotonic trend is an entropy phenomenon. The core contribution is therefore not supported, and the framing overstates what the experiments show.

major comments (4)

[Section 5.1.3, Eq. (21) and Section 4.1, Eq. (13)] The experimental quantity Hi = -log2 Pi(yi | x, y1:i-1) is the self-information of the greedily decoded token, not the Shannon entropy of the next-token distribution defined in Eq. (13). For a token with probability 0.6, Eq. (21) gives 0.737 bits, whereas Eq. (13) gives about 1.22 bits, as the paper's own example in Eq. (18) illustrates. The reported totals of 7.77, 12.28, 18.27, and 38.91 bits are sums of negative log-likelihoods along a single deterministic output path, not estimates of H(Y|X), which is an expectation over all possible outputs. Section 4.6 asserts a 'direct relation' without derivation, but the theoretical results of Section 3 apply to H(Y|X), not to the self-information of a single sample. Consequently, the experimental monotonic trend does not validate the entropy-based framework.
[Section 3.2.1, Eq. (6) and Section 3.2.3, Eq. (11)] The central inequality I_high-res(X;Y) > I_low-res(X;Y) is simply asserted without proof, and all subsequent claims about H(Y|X) in Eqs. (7) and (9) follow only from this assertion. The Data Processing Inequality is also misapplied: for the actual processing chain X -> f(X) -> Y, in which the OCR system observes only the downsampled image, DPI gives I(X; f(X)) >= I(X;Y), not I(f(X);Y) <= I(X;Y). In fact, if Y is generated solely from f(X), then I(X;Y) = I(f(X);Y), so the inequality in Eq. (11) does not establish that downsampling reduces the mutual information between the original image and the recognized text.
[Section 3.2.4, Eq. (12) and surrounding text] The paper states that 'the lower H(Y|X) implies the lower error rates Pe.' This is not a consequence of Fano's inequality, which gives the lower bound H(Y|X) >= H(Pe) + Pe log(|Y|-1). A smaller H(Y|X) does not imply a smaller Pe; the inequality only constrains achievable error probabilities from below. The argument that low entropy predicts high accuracy is therefore not supported by the cited inequality.
[Section 4.4, Eq. (17) vs Section 5.1.3, Eq. (21)] Section 4.4 explains that exact entropy requires the full next-token distribution over the vocabulary and that an approximation may be computed by summing over a subset of tokens as in Eq. (17). Section 5.1.3 instead uses only the negative log-probability of the single sampled token. This is not the subset approximation described in Eq. (17), because no sum over a subset of the vocabulary is performed. The manuscript thus defines two different quantities under the same symbol Hi without reconciling them.

minor comments (4)

[Section 5, Table 1] The experimental basis is a single page from one arXiv paper, one image per resolution, and a single generation with fixed seed and temperature 0. The strong claims in the abstract about a 'clear relationship' and 'reduced uncertainty' are not supported by any statistical repetition, error bars, or significance testing.
[Section 4.2, Eq. (16)] The notation H(y|x) is overloaded: it is used both for the total entropy defined in Eq. (14) and for the normalized entropy defined in Eq. (16). Using different symbols, such as H_norm or h_total, would avoid confusion.
[Section 5.1.3] The reported normalized entropy is labeled in 'bits' throughout Table 1, but a dimensionless quantity divided by n log2|V| cannot have units of bits; this labeling should be corrected.
[Section 6, first bullet] The stated limitation that entropy estimates 'may be approximations' if the model does not output the full distribution does not describe the actual computation in Eq. (21), which is not an approximation of Eq. (13) but a different quantity. This discrepancy should be acknowledged explicitly.

Circularity Check

2 steps flagged · score 6.0 of 10

The measured 'entropy' is negative log-likelihood of greedy tokens (Eq. 21), not Shannon conditional entropy (Eq. 13), so the experiment-to-theory validation is a definitional renaming; Section 3's resolution argument also assumes the ordering it derives.

self definitional [Section 3.2.1, Eqs. (5)-(7)]
"High-resolution images capture detailed features... These increased details affect mutual information I(X;Y) and conditional entropy H(Y|X) as follows: • Mutual Information Increases: a detailed input image X provides more specific information about the output text Y, thereby increasing I(X;Y): Ihigh-res(X;Y)>Ilow-res(X;Y) (6) • Conditional Entropy Decreases: Since H(Y) is constant, an increase in I(X;Y) directly leads toward a decrease in H(Y|X): Hhigh-res(Y|X)=H(Y)-Ihigh-res(X;Y)<Hlow-res(Y|X) (7)"

By Eq. (5), I(X;Y)=H(Y)-H(Y|X). With H(Y) 'approximately constant', the assertion I_high>I_low is mathematically equivalent to H_high<H_low. Thus the bullet 'Conditional Entropy Decreases' is not a consequence of the inequality but the same statement rewritten through an identity. The paper gives no independent derivation of Eq. (6); it is asserted from intuition about detail. Therefore Section 3's advertised 'demonstration' reduces to assuming what it claims to prove.
self definitional [Section 5.1.3, Eq. (21), and Section 4.6]
"For each output token yi, we calculated the negative log probability: Hi = − log2Pi(yi | x, y1:i−1) (21) The total conditional entropy H(y | x) is the sum of the individual entropies: H(y | x)=∑ni=1Hi (22) ... The entropy H(y|x) calculated here represents the sum of the uncertainties at each step in the generation of y by the GPT model, given the input x. It serves as an empirical approximation of the conditional entropy H(Y|X) in the OCR context."

Equation (13) defines H_i as the Shannon entropy of the full next-token distribution: -sum_{y in V} P_i(y) log P_i(y). Equation (21) defines the measured H_i as -log_2 P_i(y_i) for the single greedily decoded token, i.e., self-information (surprisal), not an expectation over V. These are different quantities except in degenerate cases. Section 4.6 asserts without derivation that Eq. (22) 'serves as an empirical approximation' of H(Y|X). Since all experimental entropy values and the resolution trend are computed from Eq. (21), the experiments do not measure the theoretical quantity whose behavior Section 3 predicts. The word 'entropy' is being applied by definition to a negative log-likelihood, so the validation loop is closed only by renaming.

full rationale

The paper has an independent experimental component: real GPT-4o calls at four resolutions and reported token log-probabilities. However, the central conceptual bridge is circular in two places. First, Section 3's derivation of the resolution-entropy relationship starts from the asserted inequality I_high-res > I_low-res (Eq. 6); because I(X;Y)=H(Y)-H(Y|X) and H(Y) is assumed constant, this inequality is already the conclusion that H(Y|X) is lower at high resolution. The apparent derivation is a restatement of its own assumption. Second, and more load-bearing, the experiments do not compute the Shannon conditional entropy defined in Eq. (13). They compute the negative log-probability of the greedily generated token (Eq. 21) and sum it (Eq. 22). Section 4.6 simply asserts that this sum 'serves as an empirical approximation' of H(Y|X), with no derivation and despite the two definitions differing structurally. Consequently, the observed monotonic trend in the reported numbers is a trend in token surprisal, not in H(Y|X), and cannot validate the theoretical framework. The paper contains no fitted parameters and no load-bearing self-citation chain, but the main claim is partially circular because the measured quantity is renamed as the theoretical entropy it is supposed to support.

Assumptions & free parameters 0 free parameters · 4 assumptions · 0 invented entities

The central claim depends on four unproven or misapplied premises: constancy of H(Y), monotonic increase of mutual information with resolution, the identification of negative log-likelihood with entropy, and the applicability of DPI to the experimental setup. No new entities or parameters are introduced; the reported normalization of entropy uses an incorrect divisor (n instead of n log|V|).

assumptions (4)

domain assumption The entropy of the output text H(Y) is constant for a given text (Section 3.1).
Used to argue that changes in conditional entropy are directly opposite to changes in mutual information. In the experiments, the output text changes with resolution, so constancy is not satisfied for the measured outputs.
domain assumption Higher-resolution images provide more information about the text, so I(X;Y) increases with resolution (Eq 6).
This inequality is asserted without proof and functions as the premise of the theoretical derivation. It is a plausible physical assumption but not a theorem, and the paper provides no formal argument.
ad hoc to paper Negative log-likelihood of the generated token sequence approximates conditional entropy H(Y|X) (Section 4.6).
This is the key bridge between theory and experiment. It is mathematically incorrect in general: the NLL of a single sampled sequence is an estimate of cross-entropy, not the entropy of the predictive distribution. The paper itself notes the distribution is only partially available.
standard math Data Processing Inequality applies to resolution reduction as a deterministic function of the image (Section 3.2.3).
DPI is a theorem, but its application here requires that the low-resolution image is a deterministic function of the high-resolution image and that the relevant Y is the true text, not the model output. The paper does not distinguish these cases.

how reviews work

0 comments

Cite this review

Pith. "Pith review of Assessing GPT Model Uncertainty in Mathematical OCR Tasks via Entropy Analysis." pith.science (2026). https://pith.science/paper/JYKIHIVV

@misc{pith2026241201221,
  author       = {Pith},
  title        = {Pith review of: Assessing GPT Model Uncertainty in Mathematical OCR Tasks via Entropy Analysis},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/JYKIHIVV}},
  note         = {Machine review of arXiv:2412.01221}
}

read the original abstract

This paper investigates the uncertainty of Generative Pre-trained Transformer (GPT) models in extracting mathematical equations from images of varying resolutions and converting them into LaTeX code. We employ concepts of entropy and mutual information to examine the recognition process and assess the model's uncertainty in this Optical Character Recognition (OCR) task. By analyzing the conditional entropy of the output token sequences, we provide both theoretical insights and practical measurements of the GPT model's performance given different image qualities. Our experimental results, obtained using a Python implementation available on GitHub, demonstrate a clear relationship between image resolution and GPT model uncertainty. Higher-resolution images lead to lower entropy values, indicating reduced uncertainty and improved accuracy in the recognized LaTeX code. Conversely, lower-resolution images result in increased entropy, reflecting higher uncertainty and a higher likelihood of recognition errors. These findings highlight the practical importance of considering image quality in GPT-based mathematical OCR applications and demonstrate how entropy analysis, grounded in information-theoretic concepts, can effectively quantify model uncertainty in real-world tasks.

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

27 extracted references · 9 canonical work pages

[1]

Abdar, M., Pourpanah, F., Hussain, S., Rezazadegan, D., Liu, L., Ghavamzadeh, M., Fieguth, P., Cao, X., Khosravi, A., Acharya, U.R., Makar enkov, V., Naha- vandi, S.: A review of uncertainty quantiﬁcation in deep lea rning: Techniques, applications and challenges. Inf. Fusion 76, 243–297 (2020)

work page 2020
[2]

Radford, A., Wu, J., Child, R., Luan, D., Amodei, D., Suts kever, I.: Language models are unsupervised multitask learners. (2019)

work page 2019
[3]

Kadavath, S., Conerly, T., Askell, A., Henighan, T., Dra in, D., Perez, E., Schiefer, N., Dodds, Z., Dassarma, N., Tran-Johnson, E., Johnston, S. , El-Showk, S., Jones, A., Elhage, N., Hume, T., Chen, A., Bai, Y., Bowman, S., Fort, S., Ganguli, D., Hernandez, D., Jacobson, J., Kernion, J., Kravec, S., Lovit t, L., Ndousse, K., Olsson, C., Ringer, S., Amod...

arXiv 2022
[4]

Lin, S.C., Hilton, J., Evans, O.: Teaching models to expr ess their uncertainty in words. Trans. Mach. Learn. Res. 2022 (2022) 19

work page 2022
[5]

Lin, Z., Trivedi, S., Sun, J.: Generating with conﬁdence : Uncertainty quantiﬁ- cation for black-box large language models. Trans. Mach. Le arn. Res. 2024 (2023)

work page 2023
[6]

ArXiv 2306.13063 (2023)

Xiong, M., Hu, Z., Lu, X., Li, Y., Fu, J., He, J., Hooi, B.: C an llms express their uncertainty? an empirical evaluation of conﬁdence el icitation in llms. ArXiv 2306.13063 (2023)

arXiv 2023
[7]

ArX iv 2405.13907 (2024)

Yang, A., Chen, C., Pitas, K.: Just rephrase it! uncertai nty estimation in closed- source language models via multiple rephrased queries. ArX iv 2405.13907 (2024)

arXiv 2024
[8]

ArXiv 2406.04370 (2024)

Pedapati, T., Dhurandhar, A., Ghosh, S., Dan, S., Sattig eri, P.: Large language model conﬁdence estimation via black-box access. ArXiv 2406.04370 (2024)

arXiv 2024

Show all 27 references

[9]

Tect onophysics 526, 207– 216 (2012)

Wellmann, J.F., Regenauer-Lieb, K.: Uncertainties hav e a meaning: Information entropy as a quality measure for 3-d geological models. Tect onophysics 526, 207– 216 (2012)

2012
[10]

Sun, L., Zhang, X., Qian, Y., Xu, J., Zhang, S.: Feature s election using neighbor- hood entropy-based uncertainty measures for gene expressi on data classiﬁcation. Inf. Sci. 502, 18–41 (2019)

2019
[11]

ArXiv 2302.09664 (2023)

Kuhn, L., Gal, Y., Farquhar, S.: Semantic uncertainty: Linguistic invariances for uncertainty estimation in natural language generation . ArXiv 2302.09664 (2023)

2023 arXiv
[12]

A rXiv 2406.15927 (2024)

Kossen, J., Han, J., Razzak, M., Schut, L., Malik, S.A., Gal, Y.: Semantic entropy probes: Robust and cheap hallucination detection in llms. A rXiv 2406.15927 (2024)

2024 arXiv
[13]

ArXiv 2405.13845 (2024)

Qiu, X., Miikkulainen, R.: Semantic density: Uncertai nty quantiﬁcation in semantic space for large language models. ArXiv 2405.13845 (2024)

2024 arXiv
[14]

ArXiv 2308.06942 (2023)

Huang, C.-Y., Xie, Y., Jiang, Z., Lin, J.J., Li, M.: Appr oximating human-like few-shot learning with gpt-based compression. ArXiv 2308.06942 (2023)

2023 arXiv
[15]

: Distinguishing the knowable from the unknowable with language models

Ahdritz, G., Qin, T., Vyas, N., Barak, B., Edelman, B.L. : Distinguishing the knowable from the unknowable with language models. ArXiv 2402.03563 (2024)

2024 arXiv
[16]

ArXiv 2406.03441 (2024)

Becker, E., Soatto, S.: Cycles of thought: Measuring ll m conﬁdence through stable explanations. ArXiv 2406.03441 (2024)

2024 arXiv
[17]

Plaut, B., Nguyen, K., Trinh, T.: Probabilities of chat llms are miscalibrated but still predict correctness on multiple-choice q&a (2024)

2024
[18]

Ni, S., Bi, K., Yu, L., Guo, J.: Are large language models more honest in their probabilistic or verbalized conﬁdence? ArXiv 2408.09773 (2024) 20

2024 arXiv
[19]

ArXiv 2402.04957 (2024)

Chen, L., Perez-Lebel, A., Suchanek, F.M., Varoquaux, G.: Reconﬁdencing llms from the grouping loss perspective. ArXiv 2402.04957 (2024)

2024 arXiv
[20]

, Ye, J.: Inside: Llms’ internal states retain the power of hallucination detectio n

Chen, C., Liu, K., Chen, Z., Gu, Y., Wu, Y., Tao, M., Fu, Z. , Ye, J.: Inside: Llms’ internal states retain the power of hallucination detectio n. ArXiv 2402.03744 (2024)

2024 arXiv
[21]

ArXiv 2408.06816 (2024)

Yang, Y., Yoo, H., Lee, H.: Maqa: Evaluating uncertaint y quantiﬁcation in llms regarding data uncertainty. ArXiv 2408.06816 (2024)

2024 arXiv
[22]

ArXiv 2407.00499 (2024)

Wang, Z., Duan, J., Cheng, L., Zhang, Y., Wang, Q., Shen, H., Zhu, X., Shi, X., Xu, K.: Conu: Conformal uncertainty in large language model s with correctness coverage guarantees. ArXiv 2407.00499 (2024)

2024 arXiv
[23]

ArXiv 2404.15993 (2024)

Liu, L., Pan, Y., Li, X., Chen, G.: Uncertainty estimati on and quantiﬁcation for llms: A simple supervised approach. ArXiv 2404.15993 (2024)

2024 arXiv
[24]

ArXiv 2406.02543 (2024)

Abbasi-Yadkori, Y., Kuzborskij, I., György, A., Szepe sv’ari, C.: To believe or not to believe your llm. ArXiv 2406.02543 (2024)

2024 arXiv
[25]

OpenAI: OpenAI Platform Documentation. (2024). Updat ed: 2024-11-30. https://platform.openai.com/docs/models

2024
[26]

arXiv arXiv:2303.08774 (2023)

OpenAI: Gpt-4 technical report. arXiv arXiv:2303.08774 (2023). Updated: 2024-11-30

2023 arXiv
[27]

ArXiv 2106.13823 (2021) 21

Zhou, S.: Quantum data compression and quantum cross en tropy. ArXiv 2106.13823 (2021) 21

2021 arXiv

Pith tools

Reviewed August 12, 2026 · model on record in the stance chip above.

[1] [1]

Abdar, M., Pourpanah, F., Hussain, S., Rezazadegan, D., Liu, L., Ghavamzadeh, M., Fieguth, P., Cao, X., Khosravi, A., Acharya, U.R., Makar enkov, V., Naha- vandi, S.: A review of uncertainty quantiﬁcation in deep lea rning: Techniques, applications and challenges. Inf. Fusion 76, 243–297 (2020)

work page 2020

[2] [2]

Radford, A., Wu, J., Child, R., Luan, D., Amodei, D., Suts kever, I.: Language models are unsupervised multitask learners. (2019)

work page 2019

[3] [3]

Kadavath, S., Conerly, T., Askell, A., Henighan, T., Dra in, D., Perez, E., Schiefer, N., Dodds, Z., Dassarma, N., Tran-Johnson, E., Johnston, S. , El-Showk, S., Jones, A., Elhage, N., Hume, T., Chen, A., Bai, Y., Bowman, S., Fort, S., Ganguli, D., Hernandez, D., Jacobson, J., Kernion, J., Kravec, S., Lovit t, L., Ndousse, K., Olsson, C., Ringer, S., Amod...

arXiv 2022

[4] [4]

Lin, S.C., Hilton, J., Evans, O.: Teaching models to expr ess their uncertainty in words. Trans. Mach. Learn. Res. 2022 (2022) 19

work page 2022

[5] [5]

Lin, Z., Trivedi, S., Sun, J.: Generating with conﬁdence : Uncertainty quantiﬁ- cation for black-box large language models. Trans. Mach. Le arn. Res. 2024 (2023)

work page 2023

[6] [6]

ArXiv 2306.13063 (2023)

Xiong, M., Hu, Z., Lu, X., Li, Y., Fu, J., He, J., Hooi, B.: C an llms express their uncertainty? an empirical evaluation of conﬁdence el icitation in llms. ArXiv 2306.13063 (2023)

arXiv 2023

[7] [7]

ArX iv 2405.13907 (2024)

Yang, A., Chen, C., Pitas, K.: Just rephrase it! uncertai nty estimation in closed- source language models via multiple rephrased queries. ArX iv 2405.13907 (2024)

arXiv 2024

[8] [8]

ArXiv 2406.04370 (2024)

Pedapati, T., Dhurandhar, A., Ghosh, S., Dan, S., Sattig eri, P.: Large language model conﬁdence estimation via black-box access. ArXiv 2406.04370 (2024)

arXiv 2024