Pith. sign in

REVIEW 4 major objections 5 minor 1 cited by

Random-Set Large Language Models

T0 review · 4 major / 5 minor · reviewed 2026-08-16 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read Making an LLM output belief functions over token groups—instead of a single probability vector—both improves its answers and gives it a built-in signal for when it is guessing or hallucinating.

desk verdict Extends random-set neural nets to LLM token prediction with a practical clustering budget, but the mass-recovery step breaks the belief-function semantics, so the uncertainty and hallucination claims are unsupported. read the letter →

arxiv 2504.18085 v1 pith:PWFMKP57 submitted 2025-04-25 cs.CL cs.AIcs.LG

Muhammad Mubashar , Shireen Kudukkil Manchingal , Fabio Cuzzolin This is my paper · ORCID
classification cs.CLcs.AIcs.LG
keywords random-setlargelanguagemodelsbelieffunctionscredalsetsepistemicuncertaintyhallucinationdetectionpignisticprobabilityhierarchicalclusteringquestionanswering
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper tries to establish that an LLM can be made more trustworthy by changing what its final layer predicts: instead of a probability vector over the next token, it predicts a belief function over a small budget of token groups. The authors argue that this random-set output carries richer information about what the model knows versus what it is merely unsure about, and that this representation can be trained efficiently with a hierarchical-clustering trick. If correct, the approach would give a single fine-tuned model better answer accuracy plus two uncertainty signals—pignistic entropy and credal-set width—the latter of which separates correct from incorrect answers and rises when the context is deliberately wrong. The paper backs this with experiments on CoQA and OBQA across three base models, where the random-set variants outperform their standard counterparts in accuracy and cosine similarity.

What carries the argument

The load-bearing object is a belief function over a budget of focal sets: a non-additive assignment of mass to subsets of tokens, where the belief in a set is the sum of masses of its subsets, and the associated convex set of probability distributions is the credal set. The budget of focal sets is constructed by hierarchical clustering over token embeddings, then unioned with all singleton tokens, making the exponential power set tractable. The machinery also includes a recursive mass-computation formula used in the loss, a pignistic transformation that turns the belief function into a sampling distribution, and credal-set width as the measure of epistemic uncertainty.

What would settle it

Train an RS-LLM on a small vocabulary where the full power set is enumerable, compute the exact Möbius-inverse masses and exact credal sets, and compare them with the recursive budget approximation; if the approximate pignistic probabilities or credal widths differ materially, or if many predicted masses are negative and must be zeroed out by post-processing, the central uncertainty claims are not supported.

Watch

Extended reading notes

Core claim

The central claim is that replacing the standard softmax next-token distribution with a predicted finite random set—a belief function over a curated budget of token subsets—improves both answer quality and the model's ability to express epistemic uncertainty. The belief function is trained with a binary cross-entropy loss over belief values plus regularization terms encouraging non-negative masses that sum to one. At generation time, the pignistic probability (the center of mass of the associated credal set) is used to sample the next token, while the size of the credal set, measured by the upper-minus-lower probability bound on the most likely token, acts as a second-level uncertainty estimate. The authors report that RS-LLMs outperform standard LLMs on both CoQA and OBQA across Llama2-7b, Mistral-7b, and Phi-2, and that credal width provides a usable hallucination-detection signal when the model is given incorrect context.

Load-bearing premise

The whole result rests on the assumption that the approximate formula used to convert the model's set-belief scores into mass values, which only uses the limited dictionary of token groups, still yields a mathematically valid belief function; if it does not, the pignistic probability and credal-width uncertainty signals built on it are not well-defined.

Editorial extensions

If this is right

  • Any autoregressive LLM can be converted into an RS-LLM by redefining the final layer; the rest of the architecture, including LoRA-based fine-tuning, stays intact.
  • Uncertainty and hallucination signals are obtained from a single forward pass, without sampling multiple continuations or maintaining an ensemble.
  • Credal width separates correct from incorrect predictions and increases when the context is wrong, giving a direct, trainable hallucination detector.
  • The hierarchical-clustering budget makes belief prediction feasible at vocabulary scales of 32K to 51K tokens, with an ablation showing that a medium budget works best.
  • The same recipe extends beyond question answering: on the XSum summarization task, RS-Mistral achieved higher cosine similarity than standard Mistral.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The accuracy gains may partly come from the implicit label smoothing and structured regularization induced by training on group memberships; a direct comparison against a label-smoothing baseline would say how much of the improvement is due specifically to the belief-function semantics.
  • Because credal width is computed only over the chosen focal-set budget, uncertainty hidden inside larger clusters remains invisible; a testable extension would adapt the budget dynamically, adding clusters where the model is least confident.
  • The belief-function representation could be combined with semantic-level uncertainty: instead of measuring width over token groups, one could measure credal width over paraphrased answer sets, which may yield an even stronger hallucination signal.
  • The paper fixes the number of focal sets K by hand and finds that both too-small and too-large budgets hurt; an automatic criterion based on cluster overlap or validation uncertainty could remove this hyperparameter and likely improve robustness.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. The paper proposes Random-Set Large Language Models (RS-LLMs), which replace the standard softmax next-token distribution with a belief function defined on a budget of focal sets: the singletons of the vocabulary plus K clusters obtained by hierarchical clustering of token embeddings. The model is trained with a binary cross-entropy loss on belief values plus two mass-regularization terms, and at generation time the predicted belief vector is converted into a mass function via a recursive subtraction (Eq. 11), then into a pignistic probability for token sampling. The authors also propose pignistic entropy and the width of the associated credal set as uncertainty measures, and apply them to hallucination detection. Experiments on CoQA and OBQA with Llama2-7b, Mistral-7b and Phi-2 report that RS-LLMs outperform standard fine-tuned models in accuracy and cosine similarity, and that credal width separates correct from incorrect responses.

Significance. If the claims hold, the paper would offer a single-model, fine-tuning-only method for extracting epistemic uncertainty and hallucination signals from LLMs, which is currently of high interest. The budgeting scheme (hierarchical clustering) is a practical contribution for scaling random-set output layers to large vocabularies, and the use of credal-set width as an uncertainty measure is conceptually appealing. The paper also makes an explicit empirical claim—accuracy improvements over standard fine-tuning on two QA benchmarks across three base architectures—which, if verified with proper statistics, would be a useful result. However, the significance is contingent on the mathematical validity of the predicted belief functions and on the statistical reliability of the reported gains; both need substantial strengthening.

major comments (4)
  1. [Sec. 4.3, Eq. (11)] The recursive mass computation over the budget O, which is not closed under subsets, does not produce the Möbius inverse of the predicted belief function. For a focal set A, the sum over B⊂A includes only sets present in O, so the masses of all missing proper subsets are silently absorbed into m(A) or produce negative masses. The subsequent post-processing (clipping negatives and adding a universal set) changes the belief values, so the loss L_BCE in Eq. (8) no longer describes the deployed model. Consequently, the predicted object is not a genuine belief function, the credal set in Eq. (6) is not the convex set whose center of mass is the pignistic probability of Eq. (7), and the claim in Sec. 4.4 that BetP(t) lies within [P(t), P(t)] is not guaranteed. This invalidates the pignistic entropy (Eq. 13) and credal width (Eq. 14) as measures of epistemic uncertainty, and therefore undermines the hallucination detection claims in Sec. 5.3 and Fig. 5. The footnote about improper belief functions used in the literature does not rescue these claims, because the quantitative uncertainty interpretation relies on genuine credal-set semantics.
  2. [Sec. 5.1, Table 1] The central empirical claim that RS-LLMs 'clearly outperform' standard LLMs rests on a single run over a 500-sample test set, with no error bars, significance tests, or multiple seeds. The CoQA cosine-similarity differences are small (0.69 vs 0.71, 0.67 vs 0.72, 0.72 vs 0.73) and may lie within run-to-run variation, especially since the underlying model checkpoints are not specified. The authors should report confidence intervals (e.g., bootstrap over the test set) or results over several seeds, and apply a significance test (e.g., McNemar for OBQA accuracy) to support the accuracy claims.
  3. [Sec. 5.3, Table 2 and Fig. 5f] The hallucination detection evidence is weak. For OBQA, the credal width under correct vs incorrect context is 0.00±0.04 vs 0.02±0.08, which is negligible in magnitude and likely not statistically significant; the histograms in Fig. 5f appear to overlap heavily. The statement that 'both models exhibit good separability' is not supported by these numbers. In addition, randomly replacing the question or answer choices is not a standard hallucination evaluation; the authors should either use a recognized hallucination benchmark or report a proper detection metric such as AUROC, and they should clarify what notion of hallucination this setup targets.
  4. [Sec. 4.3, Eq. (8) and Eq. (11)] The loss L_BCE trains each belief value Bel(A) independently, with no constraint enforcing monotonicity (A⊆C ⇒ Bel(A) ≤ Bel(C)). Even if the budget were subset-closed, the predicted vector would not generally be a belief function, so the mass regularization terms (Eqs. 9-10) and post-processing are essential but ad hoc. The trained objective therefore does not describe the deployed post-processed model. The authors should either reparameterize the output to guarantee a valid mass function, or add a projection step that is part of the training objective, so that the loss reflects the actual predictor.
minor comments (5)
  1. [Eq. (11)] The equation should read m(A) = Bel(A) - Σ_{B⊂A} m(B), not m(A). The current typesetting appears to contain a typo.
  2. [Abstract and Sec. 1] The abstract says 'high-quality tests' and the introduction says 'allows so efficiently'; both are likely typos for 'texts' and 'allows this efficiently', respectively.
  3. [Sec. 4.1] The text says a vanilla RS-LLM would have '2N outputs' where N appears to be the vocabulary size; this should be 2^N (or 2^T) outputs, and the notation for the power set P(T) should be defined consistently.
  4. [Sec. 5.2] The phrase 'RS-LLMs clearly outperform the standard LLMs model on both datasets across all datasets' is confusing; the main table reports only CoQA and OBQA, so 'all datasets' is an overstatement. Also, the text refers to 'both models' while three base models are evaluated.
  5. [Fig. 5] The subfigure captions contain inconsistent capitalization ('Standard LLama2' vs 'Random-set LLama2') and the OBQA entropy subfigures show no clear separation, which should at least be commented on in the text rather than described as 'no real trend' without discussion.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: accuracy claims are benchmarked externally and uncertainty metrics are empirical validations of standard belief-function definitions.

full rationale

The paper's central accuracy claim (RS-LLMs outperform standard LLMs on CoQA/OBQA) is evaluated against external datasets and standard baselines; it does not follow by construction from the model definition. The conversion of predicted belief values to masses (Eq. 11) is an algebraic rearrangement of the belief-function definition, and its use over a restricted budget may be an approximation or validity issue, but it is not a circular step: the resulting masses are not fitted to the accuracy or hallucination targets. The pignistic transform and the claim that BetP lies in the credal interval are classical results (Shafer/Smets) invoked for support; while several citations are to the authors' own prior work (Cuzzolin; Manchingal et al.), those results are parameter-free mathematical facts rather than unverified premises whose conclusions are assumed. The uncertainty measures are validated against correctness and against deliberately corrupted contexts, giving the 'epistemic uncertainty' and 'hallucination detection' claims independent empirical content rather than defining them into existence. The acknowledged manual choice of K is a hyperparameter limitation, not circularity. No step in the derivation reduces a predicted quantity to a fitted input or to a self-citation chain.

Assumptions & free parameters 2 free parameters · 4 assumptions · 1 invented entities

The central claim rests on the validity of the approximate mass reconstruction (Eq. 11), the semantic meaningfulness of the hierarchical clustering budget, and the interpretation of credal width as epistemic uncertainty. Free parameters K, alpha, and beta are tuned on the CoQA test set.

free parameters (2)
  • K (number of non-singleton focal sets) = 8000
    Chosen by ablation on CoQA test set (Table 4: 2000, 4000, 8000, 16000, Combined); K=8000 gives highest cosine similarity 0.71. Reported results use this tuned value.
  • alpha and beta (mass regularization weights) = alpha = beta = 0.01
    Appendix B.1 sweeps {0.1, 0.01, 0.001, 0.0001}; 0.01 and 0.0001 tie at 0.71 cosine similarity, and 0.01 is selected. Tuned on the CoQA test set.
assumptions (4)
  • domain assumption A belief function is equivalent to a convex credal set of probability distributions (Sec. 3, Eq. 6).
    The paper relies on this equivalence to interpret predicted belief values as credal sets and to define credal width as epistemic uncertainty. Standard in Dempster-Shafer theory but not proven in this paper.
  • ad hoc to paper Hierarchically clustered token embeddings produce focal sets that capture the model's likely confusions (Sec. 4.2).
    No theoretical or empirical justification beyond qualitative examples in Appendix A; the success of the method depends on this choice.
  • domain assumption Credal set width measures the extent of epistemic uncertainty (Sec. 4.4).
    Treated as established via citations to Hüllermeier and Waegeman and to Bronevich and Klir, but the paper provides no independent validation against a ground-truth uncertainty decomposition.
  • ad hoc to paper The recursive mass formula m(A)=Bel(A)-sum_{B subset A} m(B) is well-defined and yields a valid mass function on a budget not closed under subsets (Eq. 11, Sec. 4.3).
    Möbius inversion requires all subsets; the paper acknowledges this and uses a rearrangement, but gives no proof that the result is a valid belief function. Post-processing is added to force validity.
invented entities (1)
  • Budget of focal token subsets produced by hierarchical clustering
    purpose: Reduces the 2^V output space to K+V belief predictions, enabling the random-set output layer in LLMs.
    No independent evidence that these clusters correspond to the model's true epistemic confusion; validation is indirect through downstream accuracy and credal width. The 'universal set' added in post-processing is part of this construct.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Random-Set Large Language Models." pith.science (2026). https://pith.science/paper/PWFMKP57

@misc{pith2026250418085,
  author       = {Pith},
  title        = {Pith review of: Random-Set Large Language Models},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/PWFMKP57}},
  note         = {Machine review of arXiv:2504.18085}
}
read the original abstract

Large Language Models (LLMs) are known to produce very high-quality tests and responses to our queries. But how much can we trust this generated text? In this paper, we study the problem of uncertainty quantification in LLMs. We propose a novel Random-Set Large Language Model (RSLLM) approach which predicts finite random sets (belief functions) over the token space, rather than probability vectors as in classical LLMs. In order to allow so efficiently, we also present a methodology based on hierarchical clustering to extract and use a budget of "focal" subsets of tokens upon which the belief prediction is defined, rather than using all possible collections of tokens, making the method scalable yet effective. RS-LLMs encode the epistemic uncertainty induced in their generation process by the size and diversity of its training set via the size of the credal sets associated with the predicted belief functions. The proposed approach is evaluated on CoQA and OBQA datasets using Llama2-7b, Mistral-7b and Phi-2 models and is shown to outperform the standard model in both datasets in terms of correctness of answer while also showing potential in estimating the second level uncertainty in its predictions and providing the capability to detect when its hallucinating.

Figures

Figures reproduced from arXiv: 2504.18085 by the authors.

Figure 1
Figure 1. Training and generation flow of RS-LLM. Training is performed in a parallel fashion using the teacher forcing [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 2
Figure 2. A belief function measures the total belief (sum [PITH_FULL_IMAGE:figures/full_fig_p004_2.png] view at source ↗
Figure 3
Figure 3. Proposed budgeting method for RS-LLM. First, embeddings are computed for all the tokens in vocabulary. Then, [PITH_FULL_IMAGE:figures/full_fig_p006_3.png] view at source ↗
Figures from the paper (3 more)
Figure 4
Figure 4. Figure 4: Training examples from CoQA and OBQA datasets. The text in black highlights the actual question, while the [PITH_FULL_IMAGE:figures/full_fig_p007_4.png]
Figure 5
Figure 5. Figure 5: Behavior of uncertainty measures of Llama2 and RS-Llama2 with respect to the correctness and closeness to the [PITH_FULL_IMAGE:figures/full_fig_p008_5.png]
Figure 6
Figure 6. Figure 6: (a) Frequency distribution of centroid distances of the obtained budgeted focal sets. (b) Frequency distribution [PITH_FULL_IMAGE:figures/full_fig_p013_6.png]

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. Epistemic Artificial Intelligence is Essential for Machine Learning Models to Truly 'Know When They Do Not Know'

    cs.AI 2025-05 conditional novelty 3.0 of 10

    Machine learning should use second-order uncertainty measures, such as credal sets and random sets, so models can explicitly represent ignorance and avoid overconfident predictions on unfamiliar data.

Reference graph

Works this paper leans on

84 extracted references · 42 canonical work pages · cited by 1 Pith paper

  1. [1]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 global.max substring 't := if while FUNCTION format.date year duplicate empty "emp...

  2. [2]

    https://www.sbert.net/

    sentence-transformers. https://www.sbert.net/. Accessed: 31-01-2025

    2025

  3. [3]

    L., Almeida, D., Altenschmidt, J., Altman, S., Anadkat, S., et al

    Achiam, J., Adler, S., Agarwal, S., Ahmad, L., Akkaya, I., Aleman, F. L., Almeida, D., Altenschmidt, J., Altman, S., Anadkat, S., et al. Gpt-4 technical report. arXiv preprint arXiv:2303.08774, 2023

    arXiv 2023

  4. [4]

    R., Bl \"o mer, J., Kuntze, D., and Sohler, C

    Ackermann, M. R., Bl \"o mer, J., Kuntze, D., and Sohler, C. Analysis of agglomerative clustering. Algorithmica, 69: 0 184--215, 2014

    2014

  5. [5]

    M., Firat, O., Johnson, M., Lepikhin, D., Passos, A., Shakeri, S., Taropa, E., Bailey, P., Chen, Z., et al

    Anil, R., Dai, A. M., Firat, O., Johnson, M., Lepikhin, D., Passos, A., Shakeri, S., Taropa, E., Bailey, P., Chen, Z., et al. Palm 2 technical report. arXiv preprint arXiv:2305.10403, 2023

    arXiv 2023

  6. [6]

    and Cuzzolin, F

    Antonucci, A. and Cuzzolin, F. Credal sets approximation by lower probabilities: application to credal networks. In Computational Intelligence for Knowledge-Based Systems Design: 13th International Conference on Information Processing and Management of Uncertainty, IPMU 2010, Dortmund, Germany, June 28-July 2, 2010. Proceedings 13, pp.\ 716--725. Springer, 2010

    2010

  7. [7]

    Training a helpful and harmless assistant with reinforcement learning from human feedback

    Bai, Y., Jones, A., Ndousse, K., Askell, A., Chen, A., DasSarma, N., Drain, D., Fort, S., Ganguli, D., Henighan, T., et al. Training a helpful and harmless assistant with reinforcement learning from human feedback. arXiv preprint arXiv:2204.05862, 2022

    arXiv 2022

  8. [8]

    and Linander, H

    Balabanov, O. and Linander, H. Uncertainty quantification in fine-tuned llms using lora ensembles. arXiv preprint arXiv:2402.12264, 2024

    arXiv 2024

Show all 84 references
  1. [9]

    Benchmarking llm powered chatbots: methods and metrics

    Banerjee, D., Singh, P., Avadhanam, A., and Srivastava, S. Benchmarking llm powered chatbots: methods and metrics. arXiv preprint arXiv:2308.04624, 2023

    2023 arXiv

  2. [10]

    Weight uncertainty in neural network

    Blundell, C., Cornebise, J., Kavukcuoglu, K., and Wierstra, D. Weight uncertainty in neural network. In International conference on machine learning, pp.\ 1613--1622. PMLR, 2015

    2015

  3. [11]

    Bouckaert, R. R. Bayesian belief networks: from construction to inference. PhD thesis, 1995

    1995

  4. [12]

    and Klir, G

    Bronevich, A. and Klir, G. J. Axioms for uncertainty measures on belief functions and credal sets. In NAFIPS 2008-2008 Annual Meeting of the North American Fuzzy Information Processing Society, pp.\ 1--6. IEEE, 2008

    2008

  5. [13]

    Credal learning theory

    Caprio, M., Sultana, M., Elia, E., and Cuzzolin, F. Credal learning theory. arXiv preprint arXiv:2402.00957, 2024

    2024 arXiv

  6. [14]

    Inside: Llms' internal states retain the power of hallucination detection

    Chen, C., Liu, K., Chen, Z., Gu, Y., Wu, Y., Tao, M., Fu, Z., and Ye, J. Inside: Llms' internal states retain the power of hallucination detection. arXiv preprint arXiv:2402.03744, 2024

    2024 arXiv

  7. [15]

    Training verifiers to solve math word problems

    Cobbe, K., Kosaraju, V., Bavarian, M., Chen, M., Jun, H., Kaiser, L., Plappert, M., Tworek, J., Hilton, J., Nakano, R., et al. Training verifiers to solve math word problems. arXiv preprint arXiv:2110.14168, 2021

    2021 arXiv

  8. [16]

    On the credal structure of consistent probabilities

    Cuzzolin, F. On the credal structure of consistent probabilities. In European Workshop on Logics in Artificial Intelligence, pp.\ 126--139. Springer, 2008

    2008

  9. [17]

    Complexes of outer consonant approximations

    Cuzzolin, F. Complexes of outer consonant approximations. In Symbolic and Quantitative Approaches to Reasoning with Uncertainty: 10th European Conference, ECSQARU 2009, Verona, Italy, July 1-3, 2009. Proceedings 10, pp.\ 275--286. Springer, 2009

    2009

  10. [18]

    Credal semantics of Bayesian transformations in terms of probability intervals

    Cuzzolin, F. Credal semantics of Bayesian transformations in terms of probability intervals . IEEE Transactions on Systems, Man, and Cybernetics, Part B: Cybernetics, 40 0 (2): 0 421--432, 2010 a

    2010

  11. [19]

    Geometric conditioning of belief functions

    Cuzzolin, F. Geometric conditioning of belief functions. Proceedings of BELIEF, 10, 2010 b

    2010

  12. [20]

    The geometry of consonant belief functions: simplicial complexes of necessity measures

    Cuzzolin, F. The geometry of consonant belief functions: simplicial complexes of necessity measures. Fuzzy Sets and Systems, 161 0 (10): 0 1459--1479, 2010 c

    2010

  13. [21]

    Three alternative combinatorial formulations of the theory of evidence

    Cuzzolin, F. Three alternative combinatorial formulations of the theory of evidence. Intelligent Data Analysis, 14 0 (4): 0 439--464, 2010 d

    2010

  14. [22]

    On consistent approximations of belief functions in the mass space

    Cuzzolin, F. On consistent approximations of belief functions in the mass space. In European Conference on Symbolic and Quantitative Approaches to Reasoning and Uncertainty, pp.\ 287--298. Springer, 2011 a

    2011

  15. [23]

    Geometric conditional belief functions in the belief space

    Cuzzolin, F. Geometric conditional belief functions in the belief space. In Proceedings of the 7th International Symposium on Imprecise Probabilities and Their Applications (ISIPTA’11), 2011 b

    2011

  16. [24]

    Lp consonant approximations of belief functions

    Cuzzolin, F. Lp consonant approximations of belief functions. IEEE Transactions on Fuzzy Systems, 22 0 (2): 0 420--436, 2013

    2013

  17. [25]

    Belief functions: theory and applications

    Cuzzolin, F. Belief functions: theory and applications. Springer, 2014

    2014

  18. [26]

    Generalised max entropy classifiers

    Cuzzolin, F. Generalised max entropy classifiers. In Belief Functions: Theory and Applications: 5th International Conference, BELIEF 2018, Compi \`e gne, France, September 17-21, 2018, Proceedings 5 , pp.\ 39--47. Springer, 2018 a

    2018

  19. [27]

    Visions of a generalized probability theory

    Cuzzolin, F. Visions of a generalized probability theory. arXiv preprint arXiv:1810.10341, 2018 b

    2018 arXiv

  20. [28]

    The Geometry of Uncertainty: The Geometry of Imprecise Probabilities

    Cuzzolin, F. The Geometry of Uncertainty: The Geometry of Imprecise Probabilities. Artificial Intelligence: Foundations, Theory, and Algorithms. Springer International Publishing, 2020. ISBN 9783030631536. URL https://books.google.co.uk/books?id=jNQPEAAAQBAJ

    2020

  21. [29]

    Reasoning with random sets: An agenda for the future

    Cuzzolin, F. Reasoning with random sets: An agenda for the future. arXiv preprint arXiv:2401.09435, 2023

    2023 arXiv

  22. [30]

    Uncertainty measures: A critical survey

    Cuzzolin, F. Uncertainty measures: A critical survey. Information Fusion, pp.\ 102609, 2024

    2024

  23. [31]

    and Frezza, R

    Cuzzolin, F. and Frezza, R. Integrating feature spaces for object tracking. Proc. of MTNS2000 (21Y25 June 2000), 2000

    2000

  24. [32]

    and Frezza, R

    Cuzzolin, F. and Frezza, R. Geometric analysis of belief space and conditional subspaces. In ISIPTA, pp.\ 122--132, 2001

    2001

  25. [33]

    Dempster, A. P. Upper and lower probabilities induced by a multivalued mapping. In Classic works of the Dempster-Shafer theory of belief functions , pp.\ 57--72. Springer, 2008

    2008

  26. [34]

    Distributed combination of belief functions

    Denoeux, T. Distributed combination of belief functions. Information Fusion, 65: 0 179--191, 2021

    2021

  27. [35]

    Detecting hallucinations in large language models using semantic entropy

    Farquhar, S., Kossen, J., Kuhn, L., and Gal, Y. Detecting hallucinations in large language models using semantic entropy. Nature, 630 0 (8017): 0 625--630, 2024

    2024

  28. [36]

    and Ghahramani, Z

    Gal, Y. and Ghahramani, Z. Dropout as a Bayesian approximation: Representing model uncertainty in deep learning. In International Conference on Machine Learning, pp.\ 1050--1059. PMLR, 2016

    2016

  29. [37]

    C., Khan, S., Cuzzolin, F., and Lukasiewicz, T

    Giunchiglia, E., Stoian, M. C., Khan, S., Cuzzolin, F., and Lukasiewicz, T. Road-r: The autonomous driving dataset with logical requirements. Machine Learning, 112 0 (9): 0 3261--3291, 2023

    2023

  30. [38]

    Glushkova, T., Zerva, C., Rei, R., and Martins, A. F. Uncertainty-aware machine translation evaluation. arXiv preprint arXiv:2109.06352, 2021

    2021 arXiv

  31. [39]

    and Cuzzolin, F

    Gong, W. and Cuzzolin, F. A belief-theoretical approach to example-based pose estimation. IEEE Transactions on Fuzzy Systems, 26 0 (2): 0 598--611, 2017

    2017

  32. [40]

    trlx: A framework for large scale reinforcement learning from human feedback

    Havrilla, A., Zhuravinskyi, M., Phung, D., Tiwari, A., Tow, J., Biderman, S., Anthony, Q., and Castricato, L. trlx: A framework for large scale reinforcement learning from human feedback. In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing...

    2023

  33. [41]

    J., Shen, Y., Wallis, P., Allen-Zhu, Z., Li, Y., Wang, S., Wang, L., and Chen, W

    Hu, E. J., Shen, Y., Wallis, P., Allen-Zhu, Z., Li, Y., Wang, S., Wang, L., and Chen, W. Lora: Low-rank adaptation of large language models. arXiv preprint arXiv:2106.09685, 2021

    2021 arXiv

  34. [42]

    and Waegeman, W

    H \"u llermeier, E. and Waegeman, W. Aleatoric and epistemic uncertainty in machine learning: An introduction to concepts and methods. Machine learning, 110 0 (3): 0 457--506, 2021

    2021

  35. [43]

    Javaheripi, M., Bubeck, S., Abdin, M., Aneja, J., Bubeck, S., Mendes, C. C. T., Chen, W., Del Giorno, A., Eldan, R., Gopi, S., et al. Phi-2: The surprising power of small language models. Microsoft Research Blog, 1 0 (3): 0 3, 2023

    2023

  36. [44]

    Q., Sablayrolles, A., Mensch, A., Bamford, C., Chaplot, D

    Jiang, A. Q., Sablayrolles, A., Mensch, A., Bamford, C., Chaplot, D. S., de las Casas, D., Bressand, F., Lengyel, G., Lample, G., Saulnier, L., Lavaud, L. R., Lachaux, M.-A., Stock, P., Scao, T. L., Lavril, T., Wang, T., Lacroix, T., and Sayed, W. E. Mistral 7b, 2023. URL http...

    2023 arXiv

  37. [45]

    Language models (mostly) know what they know

    Kadavath, S., Conerly, T., Askell, A., Henighan, T., Drain, D., Perez, E., Schiefer, N., Hatfield-Dodds, Z., DasSarma, N., Tran-Johnson, E., et al. Language models (mostly) know what they know. arXiv preprint arXiv:2207.05221, 2022

    2022 arXiv

  38. [46]

    and Gal, Y

    Kendall, A. and Gal, Y. What uncertainties do we need in bayesian deep learning for computer vision? Advances in neural information processing systems, 30, 2017

    2017

  39. [47]

    Kolmogorov, A. N. Three approaches to the quantitative definition of information. Problems of information transmission, 1 0 (1): 0 1--7, 1965

    1965

  40. [48]

    Semantic uncertainty: Linguistic invariances for uncertainty estimation in natural language generation

    Kuhn, L., Gal, Y., and Farquhar, S. Semantic uncertainty: Linguistic invariances for uncertainty estimation in natural language generation. arXiv preprint arXiv:2302.09664, 2023

    2023 arXiv

  41. [49]

    The enterprise of knowledge: An essay on knowledge, credal probability, and chance

    Levi, I. The enterprise of knowledge: An essay on knowledge, credal probability, and chance . The MIT Press, Cambridge, Massachusetts, 1980

    1980

  42. [50]

    Solving quantitative reasoning problems with language models

    Lewkowycz, A., Andreassen, A., Dohan, D., Dyer, E., Michalewski, H., Ramasesh, V., Slone, A., Anil, C., Schlag, I., Gutman-Solo, T., et al. Solving quantitative reasoning problems with language models. Advances in Neural Information Processing Systems, 35: 0 3843--3857, 2022

    2022

  43. [51]

    Teaching models to express their uncertainty in words

    Lin, S., Hilton, J., and Evans, O. Teaching models to express their uncertainty in words. arXiv preprint arXiv:2205.14334, 2022

    2022 arXiv

  44. [52]

    K., Mubashar, M., Wang, K., Shariatmadar, K., and Cuzzolin, F

    Manchingal, S. K., Mubashar, M., Wang, K., Shariatmadar, K., and Cuzzolin, F. Random-set convolutional neural network (RS-CNN) for epistemic deep learning. arXiv preprint arXiv:2307.05772, 2023

    2023 arXiv

  45. [53]

    K., Mubashar, M., Wang, K., Shariatmadar, K., and Cuzzolin, F

    Manchingal, S. K., Mubashar, M., Wang, K., Shariatmadar, K., and Cuzzolin, F. Random-set neural networks. In The Thirteenth International Conference on Learning Representations, 2025. URL https://openreview.net/forum?id=pdjkikvCch

    2025

  46. [54]

    Random sets and integral geometry

    Matheron, G. Random sets and integral geometry. Wiley Series in Probability and Mathematical Statistics, New York, 1975

    1975

  47. [55]

    On faithfulness and factuality in abstractive summarization

    Maynez, J., Narayan, S., Bohnet, B., and McDonald, R. On faithfulness and factuality in abstractive summarization. arXiv preprint arXiv:2005.00661, 2020

    2005 arXiv

  48. [56]

    J., Szlam, A., Boureau, Y.-L., and Dinan, E

    Mielke, S. J., Szlam, A., Boureau, Y.-L., and Dinan, E. Linguistic calibration through metacognition: aligning dialogue agent responses with expected correctness. arXiv preprint arXiv:2012.14983, 11, 2020

    2012 arXiv

  49. [57]

    Can a suit of armor conduct electricity? a new dataset for open book question answering

    Mihaylov, T., Clark, P., Khot, T., and Sabharwal, A. Can a suit of armor conduct electricity? a new dataset for open book question answering. In EMNLP, 2018

    2018

  50. [58]

    Random sets and random functions

    Molchanov, I. Random sets and random functions. Theory of Random Sets, pp.\ 451--552, 2017

    2017

  51. [59]

    Molchanov, I. S. Theory of random sets, volume 19. Springer, 2005

    2005

  52. [60]

    Modern hierarchical, agglomerative clustering algorithms

    M \"u llner, D. Modern hierarchical, agglomerative clustering algorithms. arXiv preprint arXiv:1109.2378, 2011

    2011 arXiv

  53. [61]

    B., and Lapata, M

    Narayan, S., Cohen, S. B., and Lapata, M. Don't give me the details, just the summary! topic-aware convolutional neural networks for extreme summarization. arXiv preprint arXiv:1808.08745, 2018

    2018 arXiv

  54. [62]

    Nguyen, H. T. On random sets and belief functions. Journal of Mathematical Analysis and Applications, 65: 0 531--542, 1978

    1978

  55. [63]

    M., Van Roy, B., McAleese, N., Aslanides, J., and Irving, G

    Osband, I., Asghari, S. M., Van Roy, B., McAleese, N., Aslanides, J., and Irving, G. Fine-tuning language models via epistemic neural networks. arXiv preprint arXiv:2211.01568, 2022

    2022 arXiv

  56. [64]

    Softmax probabilities (mostly) predict large language model correctness on multiple-choice q&a

    Plaut, B., Nguyen, K., and Trinh, T. Softmax probabilities (mostly) predict large language model correctness on multiple-choice q&a. arXiv preprint arXiv:2402.13213, 2024

    2024 arXiv

  57. [65]

    Reddy, S., Chen, D., and Manning, C. D. C o QA : A conversational question answering challenge. Transactions of the Association for Computational Linguistics, 7: 0 249--266, 2019. doi:10.1162/tacl_a_00266. URL https://aclanthology.org/Q19-1016

    2019 doi

  58. [66]

    E., Adi, Y., Liu, J., Sauvestre, R., Remez, T., et al

    Roziere, B., Gehring, J., Gloeckle, F., Sootla, S., Gat, I., Tan, X. E., Adi, Y., Liu, J., Sauvestre, R., Remez, T., et al. Code llama: Open foundation models for code. arXiv preprint arXiv:2308.12950, 2023

    2023 arXiv

  59. [67]

    Is the volume of a credal set a good measure for epistemic uncertainty? In Uncertainty in Artificial Intelligence, pp.\ 1795--1804

    Sale, Y., Caprio, M., and H \"u llermeier, E. Is the volume of a credal set a good measure for epistemic uncertainty? In Uncertainty in Artificial Intelligence, pp.\ 1795--1804. PMLR, 2023

    2023

  60. [68]

    A mathematical theory of evidence, volume 42

    Shafer, G. A mathematical theory of evidence, volume 42. Princeton university press, 1976 a

    1976

  61. [69]

    A theory of statistical evidence

    Shafer, G. A theory of statistical evidence. In Harper, W. L. and Hooker, C. A. (eds.), Foundations of Probability Theory, Statistical Inference, and Statistical Theories of Science, volume 2, pp.\ 365--436. Reidel, Dordrecht, 1976 b

    1976

  62. [70]

    Decision making in the tbm: the necessity of the pignistic transformation

    Smets, P. Decision making in the tbm: the necessity of the pignistic transformation. International Journal of Approximate Reasoning, 38 0 (2): 0 133--147, 2005 a . ISSN 0888-613X. doi:https://doi.org/10.1016/j.ijar.2004.05.003. URL https://www.sciencedirect.com/science/article...

    2005 doi

  63. [71]

    Decision making in the TBM: the necessity of the pignistic transformation

    Smets, P. Decision making in the TBM: the necessity of the pignistic transformation . International Journal of Approximate Reasoning, 38 0 (2): 0 133--147, 2005 b

    2005

  64. [72]

    and Kennes, R

    Smets, P. and Kennes, R. The transferable belief model. Artificial intelligence, 66 0 (2): 0 191--234, 1994

    1994

  65. [73]

    Llama 2: Open foundation and fine-tuned chat models

    Touvron, H., Martin, L., Stone, K., Albert, P., Almahairi, A., Babaei, Y., Bashlykov, N., Batra, S., Bhargava, P., Bhosale, S., et al. Llama 2: Open foundation and fine-tuned chat models. arXiv preprint arXiv:2307.09288, 2023

    2023 arXiv

  66. [74]

    Credal deep ensembles for uncertainty quantification

    Wang, K., Cuzzolin, F., Shariatmadar, K., Moens, D., Hallez, H., et al. Credal deep ensembles for uncertainty quantification. Advances in Neural Information Processing Systems, 37: 0 79540--79572, 2024 a

    2024

  67. [75]

    K., Cuzzolin, F., Moens, D., and Hallez, H

    Wang, K., Shariatmadar, K., Manchingal, S. K., Cuzzolin, F., Moens, D., and Hallez, H. Creinns: Credal-set interval neural networks for uncertainty estimation in classification tasks. Neural Networks, pp.\ 107198, 2025

    2025

  68. [76]

    Uncertainty estimation and reduction of pre-trained models for text regression

    Wang, Y., Beck, D., Baldwin, T., and Verspoor, K. Uncertainty estimation and reduction of pre-trained models for text regression. Transactions of the Association for Computational Linguistics, 10: 0 680--696, 2022

    2022

  69. [77]

    Blob: Bayesian low-rank adaptation by backpropagation for large language models

    Wang, Y., Shi, H., Han, L., Metaxas, D., and Wang, H. Blob: Bayesian low-rank adaptation by backpropagation for large language models. arXiv preprint arXiv:2406.11675, 2024 b

    2024 arXiv

  70. [78]

    V., Zhou, D., et al

    Wei, J., Wang, X., Schuurmans, D., Bosma, M., Xia, F., Chi, E., Le, Q. V., Zhou, D., et al. Chain-of-thought prompting elicits reasoning in large language models. Advances in neural information processing systems, 35: 0 24824--24837, 2022

    2022

  71. [79]

    Williams, R. J. and Zipser, D. A learning algorithm for continually running fully recurrent neural networks. Neural computation, 1 0 (2): 0 270--280, 1989

    1989

  72. [80]

    A brief overview of chatgpt: The history, status quo and potential future development

    Wu, T., He, S., Liu, J., Sun, S., Liu, K., Han, Q.-L., and Tang, Y. A brief overview of chatgpt: The history, status quo and potential future development. IEEE/CAA Journal of Automatica Sinica, 10 0 (5): 0 1122--1136, 2023

    2023

  73. [81]

    Yager, R. R. Entropy and specificity in a mathematical theory of evidence. Classic works of the Dempster-Shafer theory of belief functions, pp.\ 291--310, 2008

    2008

  74. [82]

    X., Robeyns, M., Wang, X., and Aitchison, L

    Yang, A. X., Robeyns, M., Wang, X., and Aitchison, L. Bayesian low-rank adaptation for large language models. arXiv preprint arXiv:2308.13111, 2023

    2023 arXiv

  75. [83]

    H., Kolehmainen, J., Shivakumar, P

    Yu, Y., Yang, C.-H. H., Kolehmainen, J., Shivakumar, P. G., Gu, Y., Ren, S. R. R., Luo, Q., Gourav, A., Chen, I.-F., Liu, Y.-C., et al. Low-rank adaptation of large language model rescoring for parameter-efficient speech recognition. In 2023 IEEE Automatic Speech Recognition a...

    2023

  76. [84]

    and Fagiuoli, E

    Zaffalon, M. and Fagiuoli, E. Tree-based credal networks for classification. Reliable computing, 9 0 (6): 0 487--509, 2003

    2003

Pith tools

Reviewed August 16, 2026 · model on record in the stance chip above.