Pith. sign in

REVIEW 4 major objections 6 minor 33 references

Can Multiple Responses from an LLM Reveal the Sources of Its Uncertainty?

T0 review · 4 major / 6 minor · reviewed 2026-08-05 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read Why an LLM is unsure can be read from the disagreement among its repeated answers.

desk verdict Useful, clearly-described pipeline for attributing LLM uncertainty to question ambiguity vs. knowledge gaps, but it never tests the central claim that the sampled responses are what carry the signal—a Q-only ablation is missing. read the letter →

arxiv 2509.04464 v1 pith:7E3ZIBNC submitted 2025-08-28 cs.CL cs.AI

Yang Nan , Pengfei He , Ravi Tandon , Han Xu This is my paper
classification cs.CLcs.AI
keywords LLMuncertaintyattributionquestionambiguityknowledgegapsself-consistencyShannonentropyanswerdisagreementblack-boxdiagnosis
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper tries to establish that the source of a language model's uncertainty is visible in the pattern of disagreement among its repeatedly sampled answers. It proposes a two-step pipeline: sample many answers, keep questions where answer entropy is high, then ask an auxiliary LLM to classify the disagreement as question ambiguity, a knowledge gap, or both, and to name the missing concept in the latter cases. On ambiguous QA, commonsense science QA, and expert-level physics and chemistry benchmarks, clarifying ambiguous questions reduces uncertainty most for the 'ambiguity' label, while injecting the extracted missing knowledge reduces uncertainty and boosts accuracy. If correct, this turns uncertainty from a number into a diagnosis that can guide targeted fixes without modifying the model or accessing its internals.

What carries the argument

The mechanism is the disagreement pattern of N sampled answers, summarized by Shannon entropy over the answer distribution. The paper adds a threshold filter for high-uncertainty questions, a third-person 'Tom' framing that makes the auxiliary LLM judge someone else's answers, and two prompted steps: first classify the source, then extract the specific missing knowledge. The entropy says the model is uncertain; the auxiliary LLM's reading of the answer set says why.

What would settle it

Build a labeled test set of high-uncertainty questions whose true uncertainty source is known by construction, for example by taking unambiguous expert questions and withholding one required concept from the target model's context to create knowledge gaps, while injecting two mutually exclusive valid readings to create ambiguities, then check whether the auxiliary model's labels match the constructed sources at well above chance.

Watch

Extended reading notes

Core claim

The central claim is that underspecified questions and missing knowledge leave different, legible imprints on a model's sampled answers. When a question has multiple readings, the answers tend to diverge along the axis of interpretation; when a fact or concept is absent, the answers differ around that absent principle. An auxiliary LLM, prompted in the third person to analyze 'Tom's' answers, labels each high-uncertainty question as Question Ambiguity, Knowledge Gaps, or Both, and for the knowledge-related labels extracts the precise missing concept. The evidence is behavioral: clarified questions produce the largest uncertainty drops for the 'Question Ambiguity' label, and web-retrieved exp

Load-bearing premise

The attribution labels are considered correct because uncertainty falls most after clarification for questions labeled 'ambiguous,' but there are no ground-truth labels and clarification also helps the 'knowledge gap' cases substantially, so the ranking signal may be too weak to prove the classification is right.

Editorial extensions

If this is right

  • Users can refine a question when the model attributes uncertainty to ambiguity, and developers can inject the extracted missing concept into context or training data when the model attributes it to a knowledge gap.
  • The method works on black-box APIs because it only uses sampled outputs and prompts, not weights or internal states.
  • Deployments that already sample multiple responses for self-consistency can obtain both an uncertainty score and a source diagnosis from the same samples.
  • The extracted missing knowledge can be fed back as context and, in the paper's experiments, improves accuracy on high-uncertainty questions, especially for stronger target models.
  • A single auxiliary LLM can serve as a general diagnostic layer across domains, from historical fact questions to physics and chemistry reasoning.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The same disagreement-pattern diagnostic may apply beyond question answering, such as to code generation or summarization, because ambiguity-versus-knowledge imprints are task-general; the paper only tests QA-style benchmarks.
  • The relative weakness of the paper's validation signal suggests a synthetic ground-truth test would settle the method's real accuracy: construct questions with known ambiguity or deliberately withheld concepts and check whether the auxiliary labels recover them.
  • Because clarification also reduces uncertainty for 'Knowledge Gap' samples by 24 to 30 percent, the attribution labels may be capturing a graded mixture rather than discrete categories; users may need to treat 'Both' as common rather than exceptional.
  • The third-person framing may be doing real work for attribution quality; an ablation comparing first-person and third-person prompts would clarify whether the benefit comes from framing or from the auxiliary model's reasoning strength.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 6 minor

Summary. The paper proposes that when an LLM is uncertain, the patterns of disagreement among multiple sampled responses contain clues about the source of that uncertainty. The authors build a two-phase pipeline: (i) generate N responses from a target LLM and filter high-uncertainty questions by response entropy; (ii) use an auxiliary LLM to classify the uncertainty source as Question Ambiguity, Knowledge Gaps, or Both, and, for knowledge-related cases, to extract the specific missing concept. The attribution module is validated by measuring uncertainty reduction after clarifying the question, and the knowledge-gap module is validated by prepending retrieved explanatory context and measuring accuracy and uncertainty changes. Experiments are run on AmbigQA, OpenBookQA, and Physics/Chemistry subsets of MMLU-Pro with several target models. The paper concludes that disagreement patterns reveal uncertainty sources and can guide targeted interventions.

Significance. The problem is well chosen: moving from uncertainty quantification to uncertainty diagnosis is a meaningful step, and a robust method would be practically useful because it operates on black-box models without modifying the input at diagnosis time. The paper has several strengths: the auxiliary model is not fitted to the outcome metrics; the authors include a same-model replication (Table 3) showing the pattern does not depend on using a much stronger auxiliary model; the prompt templates, algorithm pseudocode, and numerous worked examples are provided; and the Limitations section candidly acknowledges the absence of direct ground-truth evaluation. If the central claim is correct, the contribution would be a lightweight, general diagnostic tool for LLM unreliability. However, the evidence is currently indirect, and the key missing control—whether the auxiliary LLM needs the sampled responses at all—leaves the central claim underdetermined.

major comments (4)
  1. [§4.2, Algorithm 1, Figure 3] The central claim is that disagreement among multiple responses contains rich clues about uncertainty source. Yet in every experiment the auxiliary LLM is given both the question Q and the N answers; there is no condition where Q is presented alone. Because the auxiliary model is itself a strong LLM, it may be able to classify a question as ambiguous or knowledge-gapped from Q alone, and the reported validation in §5.1 would then look the same even if the answers were ignored. A Q-only ablation is load-bearing: it should compare attribution labels and downstream validation metrics with and without the response set.
  2. [§5.1, Table 2] The validation of Uncertainty Attribution relies on a monotone proxy: larger uncertainty drop after clarification implies more of the uncertainty was due to question ambiguity. This proxy is weak. Knowledge Gaps samples still show large reductions (24.3–30.2% on AmbigQA; 23.7–51.7% on OpenBookQA), and some rows are nearly tied (e.g., GPT-3.5 on OpenBookQA: Both 23.84% vs Knowledge Gaps 23.72%). There are no confidence intervals, no statistical test, and no control for repeated sampling without clarification. The paper's own Limitations section concedes that this validation cannot provide a precise measure of attribution accuracy. The reported ordering is suggestive but not sufficient to establish that the labels are correct.
  3. [§5.2, Table 4] Knowledge-Gap Extraction is validated by prepending retrieved context and observing accuracy/uncertainty improvements. There is no control condition with generic or randomly selected knowledge passages, or with the correct answer statement, so the improvements could reflect the benefit of any additional relevant information rather than the precision of the extracted concept. The manual inspection of 20+20 samples shows many residual errors, especially for multi-step quantitative problems. A controlled comparison is needed to show that the extracted concept—rather than the retrieval process or passage relevance—is what drives the gains.
  4. [§4.1, Eq. (1)] The uncertainty score is Shannon entropy over exact string matches of entire sampled answers. For open-ended answers, paraphrases that are semantically identical are counted as different answers, which can inflate entropy and distort both the threshold filter and the before/after clarification reductions. The related work cites semantic clustering methods, but the method does not use them. The paper should either define an equivalence criterion or report sensitivity of the main results to exact-match versus semantic clustering.
minor comments (6)
  1. [§5.1] The threshold τ = 0.89 is fixed without sensitivity analysis. The entropy scale depends on N and on the number of distinct answer strings, so a single threshold across datasets and models is not obviously appropriate.
  2. [§5.1] OpenBookQA experiments use the first 500 examples of the training set, not a held-out split. This selection is convenient but should be justified or replaced with a test/validation split.
  3. [Appendix B, Table 5] In the prompt-only synthesis ablations, Chemistry accuracy improvements are very small (0.38–3.93%), which is in tension with the text's claim of 'clear improvements.' Please reconcile or moderate the claim.
  4. [§5.1] The comparison with the baseline Hou et al. is only qualitative. If the paper claims high alignment, report a quantitative comparison or remove the implication.
  5. [Figure 4] The prompt template in Figure 4 has an apparent copy-and-paste formatting issue in option B. Please proofread the displayed prompts.
  6. [Tables 2–4] No sample counts are reported per label in Tables 2–4. Without the number of high-uncertainty questions per category, the reader cannot assess the reliability of the reported means.

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity: the diagnosis is produced by an independent auxiliary LLM and validated by external behavioral outcomes, not by fitting or by construction.

full rationale

The paper's central claim is that disagreement patterns among multiple sampled responses contain clues about the source of uncertainty. The pipeline is: (1) sample N responses from a target LLM and compute Shannon entropy U(Q); (2) feed Q and the N answers to an independent auxiliary LLM (o1-mini) to label the source as Question Ambiguity, Knowledge Gaps, or Both; (3) for Knowledge Gaps/Both, prompt the auxiliary LLM to identify a specific missing concept. No parameter is fitted to the validation outcomes. The threshold tau=0.89 is a hand-set constant, not optimized. The auxiliary model's labels are generated before the validation metrics are computed; the labels are not derived from the uncertainty reduction after clarification or from the accuracy gain after knowledge injection. The Section 5.1 validation uses an external behavioral signal: clarifying the input question reduces uncertainty most for Question Ambiguity labels, less for Both, and least for Knowledge Gaps. The Section 5.2 validation retrieves an external passage based on the extracted concept, prepends it to the question, and measures accuracy/uncertainty changes; these outcomes are not the quantities the pipeline optimizes. There are no load-bearing self-citations: the cited epistemic/aleatoric decomposition (Kendall and Gal, 2017; Hou et al., 2023) is background, and no uniqueness claim is imported from the authors' prior work. The paper's Limitations section honestly notes the lack of direct ground-truth metrics for the labels and extracted knowledge; that is an evaluation weakness and a validity concern, but not circularity. A missing Q-only ablation would test whether the answer set is necessary for diagnosis, but that is an empirical control issue, not a definitional or fitted reduction of the claimed result to its inputs. Overall, the derivation chain does not reduce to its own inputs.

Assumptions & free parameters 3 free parameters · 5 assumptions · 0 invented entities

The framework introduces no new physical or model entities; it reuses an auxiliary LLM and web search. The main assumptions are behavioral proxies and the informativeness of disagreement patterns.

free parameters (3)
  • Uncertainty threshold tau = 0.89
    Hand-chosen cutoff to filter high-uncertainty samples; the paper says 'chosen to exclude confident cases while retaining enough high-uncertainty samples for analysis.' It affects which samples are diagnosed.
  • Number of samples N = 10
    Chosen as a balance between reliable uncertainty estimation and cost; applies to both target sampling and uncertainty computation.
  • Sampling temperature / decoding parameters = not reported
    The paper says 'stochastic decoding' (Algorithm 1) but never reports temperature or other generation parameters, which directly shape the disagreement patterns being analyzed.
assumptions (5)
  • standard math Shannon entropy of the empirical answer distribution is a valid measure of model uncertainty
    Used in Eq. 2 to define U(Q); standard for self-consistency but assumes the samples are representative of the model's uncertainty.
  • domain assumption Disagreement patterns among sampled answers carry information about the source of uncertainty
    Central hypothesis of the paper (Abstract); not independently proven.
  • domain assumption The auxiliary LLM's attribution to Question Ambiguity, Knowledge Gaps, or Both is reliable
    No ground-truth labels exist; the paper validates only indirectly via behavioral proxies (Section 5.1).
  • domain assumption Uncertainty reduction after clarification is a monotone proxy for the fraction of uncertainty due to question ambiguity
    Used in Section 5.1; the paper itself notes knowledge-gap samples also show substantial reductions.
  • domain assumption Accuracy improvement after injecting retrieved knowledge confirms the extracted concept was the missing knowledge
    Used in Section 5.2; no control condition rules out generic context benefits.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Can Multiple Responses from an LLM Reveal the Sources of Its Uncertainty?." pith.science (2026). https://pith.science/paper/7E3ZIBNC

@misc{pith2026250904464,
  author       = {Pith},
  title        = {Pith review of: Can Multiple Responses from an LLM Reveal the Sources of Its Uncertainty?},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/7E3ZIBNC}},
  note         = {Machine review of arXiv:2509.04464}
}
read the original abstract

Large language models (LLMs) have delivered significant breakthroughs across diverse domains but can still produce unreliable or misleading outputs, posing critical challenges for real-world applications. While many recent studies focus on quantifying model uncertainty, relatively little work has been devoted to \textit{diagnosing the source of uncertainty}. In this study, we show that, when an LLM is uncertain, the patterns of disagreement among its multiple generated responses contain rich clues about the underlying cause of uncertainty. To illustrate this point, we collect multiple responses from a target LLM and employ an auxiliary LLM to analyze their patterns of disagreement. The auxiliary model is tasked to reason about the likely source of uncertainty, such as whether it stems from ambiguity in the input question, a lack of relevant knowledge, or both. In cases involving knowledge gaps, the auxiliary model also identifies the specific missing facts or concepts contributing to the uncertainty. In our experiment, we validate our framework on AmbigQA, OpenBookQA, and MMLU-Pro, confirming its generality in diagnosing distinct uncertainty sources. Such diagnosis shows the potential for relevant manual interventions that improve LLM performance and reliability.

Figures

Figures reproduced from arXiv: 2509.04464 by the authors.

Figure 1
Figure 1. Illustration of the framework and example. We generate multiple responses from a target LLM and use [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 2
Figure 2. An example of using LLM to diagnose un￾certainty for a sample question: ten responses were collected, with options B and D each selected five times. To illustrate, two representative responses are shown. selects “electrical -> chemical -> light” or “chemi￾cal -> electrical -> light”. Referring to the detailed analysis from the model as shown in [PITH_FULL_IMAGE:figures/full_fig_p004_2.png] view at source ↗
Figure 3
Figure 3. Framework of our pipeline for extracting the [PITH_FULL_IMAGE:figures/full_fig_p005_3.png] view at source ↗
Figures from the paper (10 more)
Figure 4
Figure 4. Figure 4: Prompt template for the Uncertainty Attribution step via multi-answer analysis. Tom has 10 answers to the same question. Here are the question and answers: Question: [Specific Question Here] Answer1: [Specific Answer1 Here] Answer2: [Specific Answer2 Here] … Answer10: …
Figure 5
Figure 5. Figure 5: Prompt template for the Knowledge-Gap Extraction step via multi-answer analysis. Concept: [Missing Knowledge (Specific Concept)] Your task is to generate a complete, self-contained knowledge snippet based solely on this concept. The snippet should include: - 1. Definit…
Figure 6
Figure 6. Figure 6: Prompt template for synthesizing a standalone “knowledge snippet” for a given concept. [PITH_FULL_IMAGE:figures/full_fig_p014_6.png]
Figure 7
Figure 7. Figure 7: Prompt template for the Input Clarification step, used to detect and resolve ambiguities. [PITH_FULL_IMAGE:figures/full_fig_p014_7.png]
Figure 8
Figure 8. Figure 8: Example from an AmbigQA sample where the [PITH_FULL_IMAGE:figures/full_fig_p015_8.png]
Figure 9
Figure 9. Figure 9: Example from an OpenBookQA sample where the [PITH_FULL_IMAGE:figures/full_fig_p015_9.png]
Figure 10
Figure 10. Figure 10: Example from an MMLU-Pro-Chemistry sample where the model makes a [PITH_FULL_IMAGE:figures/full_fig_p016_10.png]
Figure 11
Figure 11. Figure 11: Example from an MMLU-Pro-Physics sample where the model makes a [PITH_FULL_IMAGE:figures/full_fig_p017_11.png]
Figure 12
Figure 12. Figure 12: Example from an MMLU-Pro-Chemistry sample where the model makes a [PITH_FULL_IMAGE:figures/full_fig_p018_12.png]
Figure 13
Figure 13. Figure 13: Example from an MMLU-Pro-Physics sample where the model makes a [PITH_FULL_IMAGE:figures/full_fig_p019_13.png]

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

33 extracted references · 30 canonical work pages

  1. [1]

    Knowledge Gaps,

    Uncertainty Attribution Prompt (Figure 4): Frames the task around a fictional charac- ter (“Tom”) who offers multiple answers, asks the auxiliary LLM to compare these re- sponses, and choose among “Question Am- Dataset Model Before After Unc. Reduction Rate(%) Acc. Improvement Rate(%)Unc. Acc. (%) Unc. Acc. (%) MMLU-Pro-Physics Llama3-8B-Instruct 1.83 28....

    work page

  2. [2]

    The prompt explicitly breaks the task into two steps—difference analysis and knowl- edge identification—to ensure clarity and fo- cus

    Knowledge-Gap Extraction Prompt (Fig- ure 5): Instructs the auxiliary LLM to first summarize key differences across the sam- pled answers and then pinpoint the single, pre- cise piece of missing factual or conceptual knowledge that would resolve the inconsis- tency. The prompt explicitly breaks the task into two steps—difference analysis and knowl- edge i...

    work page

  3. [3]

    Knowledge Synthesis Prompt (Figure 6): Takes a concise knowledge keyword or con- cept identified in the previous step and in- structs o1-mini to generate a self-contained explanatory snippet. This snippet includes a clear definition, core explanation, and any critical conditions or formulas, formatted as a standalone block that can be prefixed to any ques...

    work page

  4. [4]

    knowledge snippet

    Input Clarification Prompt (Figure 7): Guides the model to detect real-world ambiguities in the original question—such as unspecified timeframes, locations, or referents—and, if needed, to inject succinct, fact-grounded clari- fications without altering the question’s intent. If the question is already clear, the prompt simply reproduces it unchanged. E E...

    work page

  5. [5]

    standard candles

    **Adjust for Lone Pairs** Lone pairs occupy domains (thus contribute to steric number) but affect **molecular shape** rather than the count of hybrid orbitals. Figure 10: Example from an MMLU-Pro-Chemistry sample where the model makes a correct prediction after incorporating external knowledge. Question: Approximately how far away is the Andromeda Galaxy?...

    work page 2012

  6. [6]

    Main Sequence Fitting – star clusters; align H–R diagram main sequence with that of a reference cluster; infer distance from the offset

    work page

  7. [7]

    Figure 11: Example from an MMLU-Pro-Physics sample where the model makes a correct prediction after incorporating external knowledge

    Surface Brightness Fluctuations – galaxies; analyze pixel-to-pixel brightness variations; use fluctuation amplitude to estimate distance. Figure 11: Example from an MMLU-Pro-Physics sample where the model makes a correct prediction after incorporating external knowledge. Question: A 50.0 mL sample of 0.0025 M HBr is mixed with 50.0 mL of 0.0023 M KOH. Wha...

    work page

  8. [8]

    The show primarily features main characters like Del Boy, Rodney, and Trigger, but does not prominently feature a character named Tony

    Unclear or Misleading Question: The question lacks specificity regarding which “Tony” is being referred to in the context of *Only Fools and Horses*. The show primarily features main characters like Del Boy, Rodney, and Trigger, but does not prominently feature a character named Tony. This ambiguity can lead to confusion, causing respondents to guess or m...

    work page

Show all 33 references
  1. [9]

    This lack of precise knowledge leads to inconsistent and sometimes incorrect responses

    Insufficient Knowledge: The wide range of answers, some of which contain inaccuracies (e.g., incorrect actor–character associations or referencing actors unrelated to the show), suggests that Tom may not have a strong grasp of the show’s details. This lack of precise knowledge...

  2. [10]

    Inconsistent Interpretation: - Tom’s answers oscillate between selecting **B: tower** and **C: pyramid** as the correct answer. - This inconsistency indicates that the question’s phrasing might be open to multiple interpretations, leading to different conclusions based on how ...

  3. [11]

    - A **pyramid** inherently has slopes as part of its geometric structure

    Ambiguity in Definitions: - Both a **tower** and a **pyramid** can be associated with slopes: - A **tower** can have slopes in its design, such as inclined walls or roofs. - A **pyramid** inherently has slopes as part of its geometric structure. - Without clear context or spec...

  4. [12]

    **Identify the Central Atom** Determine which atom you’re examining

  5. [13]

    **Count Electron Domains** - Each σ bond (single, double or triple bond each count as one) - Each lone pair counts as one domain

  6. [14]

    **Compute Steric Number** Steric number = (number of σ bonds) + (number of lone pairs)

  7. [15]

    **Map Steric Number to Hybridization** | Domains | Hybridization | Electron-Domain Geometry | |:-------:|:-------------:|:-----------------------------------------:| | 2 | sp | Linear | | 3 | sp² | Trigonal planar / Bent | | 4 | sp³ | Tetrahedral / Trigonal pyramidal / Bent | ...

  8. [17]

    Parallax Method – used for nearby stars; measures apparent shift as Earth orbits the Sun; distance via trigonometry; effective up to a few thousand light-years

  9. [18]

    Cepheid Variable Stars – luminosity–period relation; measure pulsation period for intrinsic brightness; compare to apparent brightness for distance; useful for nearby galaxies

  10. [19]

    Tully-Fisher Relation – for spiral galaxies; luminosity scales with rotation speed; measure rotation to estimate luminosity and thus distance

  11. [20]

    Type Ia Supernovae – consistent peak luminosity; compare observed to intrinsic brightness to calculate distance; important for distant galaxies

  12. [21]

    Redshift & Hubble’s Law – for very distant galaxies; measure redshift to infer recessional velocity; apply Hubble’s Law (v = H₀d) to find distance

  13. [24]

    Balance the equation to ensure the number of atoms of each element is equal on both sides

    **Write the Balanced Chemical Equation:** Identify the acid, base, and their products. Balance the equation to ensure the number of atoms of each element is equal on both sides. For example: \[ \mathrm{HCl} + \mathrm{NaOH} \rightarrow \mathrm{NaCl} + \mathrm{H_2O}\]

  14. [25]

    - For solutions: \( \text{moles} = \text{molarity} \times \text{volume (in liters)} \) - For solids: \( \text{moles} = \frac{\text{mass}}{\text{molar mass}} \)

    **Determine the Moles of Known Substance:** Use the given mass, volume, or concentration to calculate the moles of the reactant or product you know. - For solutions: \( \text{moles} = \text{molarity} \times \text{volume (in liters)} \) - For solids: \( \text{moles} = \frac{\te...

  15. [26]

    **Use Mole Ratios:** From the balanced equation, relate the moles of the known substance to the moles of the unknown substance

  16. [27]

    **Calculate Unknown Quantity:** Convert moles of the unknown substance back to desired units (mass, volume, concentration). **Example:** If you have 0.5 L of 0.1 M HCl, how many grams of NaOH are needed to neutralize it? - Calculate moles of HCl: \[ 0.1 \,\text{M} \times 0.5 \...

  17. [28]

    **Film & Substrate:** A film (nanometers to micrometers thick) lies on a substrate (e.g., oil on water or anti-reflective film on glass)

  18. [29]

    **Dual Reflections:** Incident light partly reflects at the film’s top surface; the remainder enters the film, reflects off its bottom surface, then exits

  19. [30]

    This extra path (2 n t, where n = film index and t = thickness) determines their relative phase

    **Optical Path Difference:** The two reflected beams traverse different distances. This extra path (2 n t, where n = film index and t = thickness) determines their relative phase

  20. [31]

    - Reflection into lower-n medium adds no phase shift

    **Phase Shifts on Reflection:** - Reflection into higher-n medium adds a half-wavelength (½ λ) phase shift. - Reflection into lower-n medium adds no phase shift

  21. [32]

    Destructive:** - Constructive when 2 n t = m λ (m = 0, 1, 2…), yielding bright maxima

    **Constructive vs. Destructive:** - Constructive when 2 n t = m λ (m = 0, 1, 2…), yielding bright maxima. - Destructive when 2 n t = (m + ½) λ, producing dark minima

  22. [33]

    Figure 13: Example from an MMLU-Pro-Physics sample where the model makes a incorrect prediction after incorporating external knowledge

    **Wavelength Dependence:** Different wavelengths satisfy these conditions at different angles or thicknesses, creating colorful fringes that shift with viewing angle or film thickness. Figure 13: Example from an MMLU-Pro-Physics sample where the model makes a incorrect predict...

  23. [2022]

    arXiv preprint arXiv:2212.07769

    Clam: Selective clarification for ambiguous questions with generative language models. arXiv preprint arXiv:2212.07769. Lorenz Kuhn, Yarin Gal, and Sebastian Farquhar. 2023. Semantic uncertainty: Linguistic invariances for un- certainty estimation in natural language generatio...

    2023 arXiv

  24. [2023]

    arXiv preprint arXiv:2307.10236

    Look before you leap: An exploratory study of uncertainty measurement for large language models. arXiv preprint arXiv:2307.10236. Fred Jelinek. 1990. Self-organized language modeling for speech recognition. Readings in speech recogni- tion, pages 450–506. Saurav Kadavath, Tom ...

    1990 arXiv

  25. [2024]

    x” Answer: “ ˆy

    Mmlu-pro: A more robust and challenging multi-task language understanding benchmark. In The Thirty-eight Conference on Neural Information Processing Systems Datasets and Benchmarks Track. Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Fei Xia, Ed Chi, Quoc V Le, Denny...

    2022 arXiv

Pith tools

Reviewed August 5, 2026 · model on record in the stance chip above.