Pith. sign in

REVIEW 2 major objections 5 minor 65 references

Quantifying imperfect cognition via achieved information gain

T0 review · 2 major / 5 minor · reviewed 2026-08-08 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read The paper proposes achieved information gain (AIG), defined as the ideal information gain minus the remaining gain after an imperfect update, and argues it is the correct quantitative measure of how much information a cognitive operation…

desk verdict AIG is a clean and useful three-state information measure; the uniqueness claim leans too hard on the Locality axiom, but the definition and examples survive a fair reading. read the letter →

arxiv 2502.04088 v3 pith:6E7E6IIX submitted 2025-02-06 cs.IT math.IT

Torsten En{ss}lin This is my paper · ORCID
classification cs.ITmath.IT MSC 94A1594A1762B1062F15
keywords achievedinformationgainrelativeentropybeliefupdatingcognitivefidelityefficiencyscoringrulesBayesianinferencesustainablecomputing
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Relative entropy (the Kullback-Leibler divergence) rewards any update that makes a belief more definite, even if the new certainty is wrong. The paper proposes achieved information gain (AIG), which measures the information obtained in an imperfect cognitive update by subtracting what would still be needed to reach the ideal state from what an ideal update would have provided. Concretely, AIG is the expected log ratio of Bob's updated belief to his initial belief, averaged under Alice's ideal distribution. It is zero for no update, maximal for a perfect update, and negative when the update moves belief away from the truth. The paper derives AIG from axioms and shows how ratios of AIG define cognitive fidelity and cognitive efficiency for communication, inference, and memorization.

What carries the argument

The central object is the three-state information gain $D_S(I_A,I_B,I_0) = \langle \ln[P(s|I_B)/P(s|I_0)] \rangle_{s|I_A}$: an expected log-likelihood ratio of the updated belief to the initial belief, averaged over the ideal belief. It carries the argument because it makes direction matter: approaching the ideal belief yields positive gain, standing still yields zero, and moving away yields negative gain. The axiomatic derivation in Section 4 forces this logarithmic form by combining additivity over Alice's atomic beliefs with locality, which says that when Alice knows the exact state the gain depends only on Bob's probability for that state; properness then fixes the sign and calibration fixes the reference point.

What would settle it

Take a Bernoulli situation with true rate $p_A=0.6$, initial belief $p_0=0.5$, and update to $p_B=0.9$: apparent gain is positive while AIG is negative, so observe whether decision-makers acting on $p_B$ do better or worse than those acting on $p_0$; if the wrong update reliably improves downstream decisions, the claim that AIG measures achieved information is undermined. Alternatively, replace the locality axiom by a non-local scoring-rule condition and check whether a different gain satisfying the remaining axioms can be constructed.

Watch

Extended reading notes

Core claim

The central claim is that a belief update should be scored by where it ends relative to both where it started and where it should have ended, not by how much confidence it added. AIG, defined as $D_S(I_A,I_B,I_0) := D_S(I_A,I_0) - D_S(I_A,I_B) = \langle \ln[P(s|I_B)/P(s|I_0)] \rangle_{s|I_A}$, is that score. Under axioms of additivity, locality, analyticity, properness, and calibration, the paper shows that any such gain must be proportional to AIG, so the logarithmic form is not a choice but a consequence. AIG inherits useful structural properties: it is anti-symmetric in the two Bob states, path-additive, reduces to ordinary relative entropy for perfect updates, and separates across variables even when the ideal distribution is correlated. The paper works through Bernoulli, binomial, Poisson, Beta, and Gaussian updates, mean-field approximations, incomplete data usage, and sampling-based estimates for intractable posteriors, and uses AIG to define cognitive fidelity (the ratio of AIG to ideal gain) and cognitive efficiency (the ratio of AIG to cost).

Load-bearing premise

The entire uniqueness argument hinges on the assumption that, when the ideal belief is certain a particular state is true, the score of an update should depend only on how much probability the updated belief gives to that state and not on what it says about other states.

Editorial extensions

If this is right

  • Apparent information gain can be arbitrarily misleading; replacing it with AIG makes wrong-direction updates report negative information instead of positive confidence.
  • Cognitive fidelity and cognitive efficiency give a quantitative basis for choosing between cheap approximate methods and expensive accurate ones, including the trade-off between computation and data acquisition costs.
  • In repeated measurements AIG typically grows only logarithmically with data set size, so a low-fidelity method may need far more data to match the gain of a high-fidelity one.
  • For intractable posteriors, AIG can be estimated by sample averages when samples from the ideal posterior are available.
  • AIG is path-additive and anti-symmetric in the two belief states of the learner, so it behaves like a directed distance rather than a divergence.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The uniqueness result is only as strong as the locality axiom; a reader who prefers scoring rules that also reward correct probabilities assigned to non-truth states will get a different measure, so testing the appeal of locality is the natural way to probe the paper's central claim.
  • AIG gives an immediate testable prediction for human or machine learning agents: updates that agents report as informative should correlate positively with AIG computed against a trusted posterior, and confidently wrong updates should correlate negatively.
  • In practice, AIG measures information relative to whoever supplies the ideal state; without an agreed-upon reference belief, its numerical value is a statement about agreement with that reference, not about absolute truth.
  • Because AIG is path-additive, it could serve as an online monitoring signal: accumulate gains along a chain of updates and flag episodes where the cumulative gain drops below zero.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

2 major / 5 minor

Summary. The paper proposes the achieved information gain (AIG), defined as DS(IA,IB,I0) := DS(IA,I0) − DS(IA,IB) = ⟨ln(P(s|IB)/P(s|I0))⟩_{s|IA}, as a quantitative measure of the information obtained in an imperfect cognitive update from I0 to IB relative to an ideal state IA. The manuscript derives AIG axiomatically in Section 4 from additivity, locality, analyticity, properness, and calibration; relates it to KL divergence, mutual information, Rényi divergence, and scoring rules; works out closed-form AIG expressions for Bernoulli, binomial, Poisson, Beta, and Gaussian updates; analyzes mean-field and incomplete-data approximations; gives a Monte Carlo estimator for intractable posteriors; and defines cognitive fidelity and cognitive efficiency, applying the latter to sustainable computing decisions.

Significance. If the axiomatic claim is accepted, AIG is an attractive three-state information measure with a clear operational reading: the reduction in Bob's expected surprise from Alice's perspective. The paper's concrete contributions are valuable: the Bernoulli, binomial, Poisson, Beta, and Gaussian formulas check out; the mean-field calculation and the incomplete-data Wiener-filter example are correct and instructive; and the estimator in Section 5.9, together with the data-averaged identity in Eq. (94), is a practical tool for benchmarking approximate posteriors. The cognitive-efficiency discussion, especially the comparison in Eq. (100), gives a useful decision criterion. The axiomatic derivation is parameter-free up to the unit scale λ, and the relation to established scoring rules is clearly laid out. The main weakness is that the uniqueness theorem is conditional on the Locality axiom, which is asserted rather than independently justified.

major comments (2)
  1. [Sec. 4.1, Locality axiom and Eqs. (35)-(41)] The uniqueness theorem is driven entirely by the Locality axiom. A three-state gain G_Q(IA,IB,I0) = ⟨S(IB,s) − S(I0,s)⟩_{s|IA} with the strictly proper quadratic scoring rule S(P,s) = −Σ_k (δ_{s,k} − P_k)^2 satisfies Additivity (linearity in P(s|IA)), Analyticity, Properness (it is maximal at IB = IA), and Calibration (it vanishes for IB = I0), but it is not proportional to AIG because for atomic IA it depends on P(s''|IB) for s'' ≠ s', violating Locality. The paper asserts Locality without an independent normative argument that a cognitive gain should ignore probabilities assigned to wrong states; the observation in Section 3.4 that the paper's preferred scoring rules are local is not a derivation of that requirement. The central claim should therefore be qualified as characterizing AIG among local gains, or an independent justification of Locality should be supplied.
  2. [Sec. 4.1, Eqs. (37)-(38)] The transition from the stationarity condition to the global solution g(s,q,I0) = λ ln q + c(s,I0) is under-specified. Equation (37) is evaluated only at q~ = P(s'|IA), which is a single value for each s' for a fixed IA. To obtain an ordinary differential equation in q~, one must explicitly quantify Properness over all Alice beliefs and observe that every q~ in (0,1) can be realized as P(s'|IA) for some IA. The text instead invokes the Analytical axiom, but infinite differentiability (C^∞) is not real analyticity, and a pointwise stationarity condition does not extend to an open set merely by smoothness. The author should either supply the missing quantification over IA or state real analyticity and show how it applies to the local identity.
minor comments (5)
  1. [Sec. 3.1, Eq. (14)] The sentence 'out of which the AIG is build as DS(IA,IB,I0) = DS(IA,IB) − DS(IA,I0)' has the sign reversed; it contradicts Eq. (6), Eq. (15), and the immediately following expression ⟨ln(P(s|IB)/P(s|I0))⟩_{s|IA}.
  2. [Sec. 4.1, Eq. (36)] The Lagrange-multiplier term λ(1 − ∫ P(s''|IB) ds'') is non-local in P(s''|IB), so the literal statement that the gain 'should only depend on P(s'|IB), but not on any probability he assigns to other cases' is not consistent with the displayed functional. Please clarify that Locality is required only on the simplex of normalized distributions.
  3. [Sec. 6.3] The quantity fB = day/decade is a dimensionless fraction, approximately 2.7 × 10^-4, not '0.27 h'. The sentence 'fB = day/decade ≈ 0.27 h' mixes units, and the subsequent Euro estimate should be re-expressed with a correctly unitized fraction of facility time.
  4. [Sec. 5.9, Eq. (102)] The expression '⟨DS(IA(d),IB(d),I0⟩' has mismatched angle brackets; it should read '⟨DS(IA(d),IB(d),I0)⟩_{d|I0}'.
  5. [Sec. 4.1, Analytical axiom] The text says the gain should be 'infinitely differentiable' and then concludes 'this means it must be analytical.' Infinite differentiability does not imply real analyticity; the wording should be corrected to avoid a false mathematical implication.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the axiomatic characterization of AIG is self-contained and does not reduce to its inputs.

full rationale

Section 4 derives AIG from five axioms that do not presuppose the logarithmic formula. The Locality axiom restricts the atomic gain to depend on P(s'|IB); Properness yields the differential equation whose solution is the logarithm; Calibration fixes the additive constant. The derivation is performed in the paper itself, so the citation to [13] (same author group) only indicates a derivation strategy and is not load-bearing. The Locality axiom is an assumption, not a circular input; the uniqueness claim is conditional on all axioms. No parameter is fitted and renamed a prediction, and no external uniqueness theorem is imported from the authors' prior work. The worked examples follow from the derived formula rather than being used to fit it. The omitted attention-gain derivation is explicitly a brevity choice. Overall, the central claim has independent mathematical content beyond its own definition.

Assumptions & free parameters 1 free parameters · 6 assumptions · 0 invented entities

The central claim rests on five axioms introduced in Section 4.1 plus the background choice of relative entropy as the information measure. No physical entities are introduced and no data are fitted; the only undetermined constant is the unit scale λ. The most contestable axiom is locality, because it forces the logarithmic form.

free parameters (1)
  • λ (unit scale) = arbitrary positive constant
    Appears in Eq. (41) as the multiplicative factor left undetermined by the axioms; it sets the units of AIG and does not affect the ratios CF and CE.
assumptions (6)
  • domain assumption Relative entropy (KL divergence) is the accepted measure of information gain and loss between two knowledge states.
    Eq. 3 defines all gains in terms of D_S(IA,IB); the entire framework inherits this choice from information theory.
  • ad hoc to paper The gain measure is additive over Alice's hierarchical beliefs.
    Section 4.1, Additive axiom. It allows decomposing P(s|IA) into atoms and is a linearity requirement not derived from other principles.
  • ad hoc to paper The gain measure is local: for certain Alice it depends only on Bob's probability for the true state.
    Section 4.1, Locality axiom. This is the key assumption forcing the logarithmic form of the gain (Eq. 38).
  • ad hoc to paper The gain is analytic in Bob's posterior probabilities.
    Section 4.1, Analytical axiom. Used to extend the first-order solution to all densities; note that ln P is not analytic at P=0.
  • ad hoc to paper The gain is proper (maximal at the ideal update) and calibrated (zero for no update).
    Section 4.1, Proper and Calibration axioms. These set the semantics of a gain and determine the integration constant c(s,I0).
  • domain assumption Maximum entropy principle justifies Gaussian distributions from known mean and covariance.
    Section 5, Eq. 43; used to construct PDFs for the worked examples.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Quantifying imperfect cognition via achieved information gain." pith.science (2026). https://pith.science/paper/6E7E6IIX

@misc{pith2026250204088,
  author       = {Pith},
  title        = {Pith review of: Quantifying imperfect cognition via achieved information gain},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/6E7E6IIX}},
  note         = {Machine review of arXiv:2502.04088}
}
read the original abstract

Cognition, information processing in form of inference, communication, and memorization, is the central activity of any intelligence. Its physical realization in a brain, computer, or in any other intelligent system requires resources like time, energy, memory, bandwidth, money, and others. Due to limited resources, many real world intelligent systems perform only imperfect cognition. To understand the trade-off between accuracy and resource investments in existing systems, e.g. in biology, as well as for the resource-aware optimal design of information processing systems, like computer algorithms and artificial neural networks, a quantification of information obtained in an imperfect cognitive operation is desirable. To this end, we propose the concept of the achieved information gain (AIG) of a belief update, which is given by the amount of information obtained by updating from the initial state of knowledge to the ideal state, minus the amount that a change from the imperfect to the ideal state would yield. AIG has many desirable properties for quantifying imperfect cognition. The ratio of achieved to ideally obtainable information measures cognitive fidelity and that of AIG to the necessary cognitive effort measures cognitive efficiency. We provide an axiomatic derivation of AIG, relate it to other information measures, illustrate its application to common scenarios of posterior inaccuracies, and discuss the implication of cognitive efficiency for sustainable resource allocation in computational inference.

Figures

Figures reproduced from arXiv: 2502.04088 by the authors.

Figure 1
Figure 1. AIG as the information gain of the ideal update [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 2
Figure 2. Knowledge update on a Bernoulli distributed event with occurrence rate [PITH_FULL_IMAGE:figures/full_fig_p009_2.png] view at source ↗
Figure 3
Figure 3. Knowledge update on a Poisson distributed quantity with expectation [PITH_FULL_IMAGE:figures/full_fig_p010_3.png] view at source ↗
Figures from the paper (3 more)
Figure 4
Figure 4. Figure 4: AIG under incorrect mean (left) and variance (right) of a Gaussian distribution according to Eq. 66 as parame [PITH_FULL_IMAGE:figures/full_fig_p012_4.png]
Figure 5
Figure 5. Figure 5: AIG under incorrect mean and variance for a [PITH_FULL_IMAGE:figures/full_fig_p013_5.png]
Figure 7
Figure 7. Figure 7: Various information gains in case IB represents incomplete data usage, where the total dataset of IA has rA = 220 ≈ 106 measurements of a scalar quantity, each with Gaussian noise that has the variance of the prior uncertainty, σn = σs (a signal to noise ratio of one),…

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

65 extracted references · 50 canonical work pages

  1. [1]

    On Information and Sufficiency

    S. Kullback and R. A. Leibler. “On Information and Sufficiency.” In: The Annals of Mathematical Statis- tics 22.1 (1951), pp. 79–86. doi: 10 . 1214 / aoms / 1177729694. url: https : / / doi . org / 10 . 1214 / aoms/1177729694

    arXiv 1951

  2. [2]

    Thermodynamics.Vol.1.Long- mans, Green and Company, 1906

    JosiahWillardGibbs. Thermodynamics.Vol.1.Long- mans, Green and Company, 1906

    work page 1906

  3. [3]

    A Mathematical Theory of Communication

    Claude Elwood Shannon. “A Mathematical Theory of Communication.” In: The Bell System Techni- cal Journal 27 (1948), pp. 379–423. url: http : / / plan9.bell- labs.com/cm/ms/what/shannonday/ shannon1948.pdf (visited on 04/22/2003)

    work page 1948

  4. [4]

    Measures of the value of informa- tion

    John McCarthy. “Measures of the value of informa- tion.” In:Proceedings of the National Academy of Sci- ences 42.9 (1956), pp. 654–655

    work page 1956

  5. [6]

    “Good” probability assessors

    Robert L Winkler and Allan H Murphy. ““Good” probability assessors.” In: Journal of Applied Mete- orology and Climatology7.5 (1968), pp. 751–758

    work page 1968

  6. [7]

    Updating probabil- ities

    Ariel Caticha and Adom Giffin. “Updating probabil- ities.” In: AIP Conference Proceedings. Vol. 872. 1. American Institute of Physics. 2006, pp. 31–42

    work page 2006

  7. [8]

    Strictly proper scoring rules, prediction, and estimation

    Tilmann Gneiting and Adrian E Raftery. “Strictly proper scoring rules, prediction, and estimation.” In: Journal of the American statistical Association 102.477 (2007), pp. 359–378

    work page 2007

  8. [9]

    Foundations of inference

    Kevin H Knuth and John Skilling. “Foundations of inference.” In: Axioms 1.1 (2012), pp. 38–73

    work page 2012

Show all 65 references
  1. [10]

    Optimal Be- lief Approximation

    Reimar Leike and Torsten Enßlin. “Optimal Be- lief Approximation.” In: Entropy 19.8 (Aug. 2017), p. 402.doi: 10.3390/e19080402. arXiv:1610.09018 [math.ST]

    2017 arXiv

  2. [11]

    Divergence and Sufficiency for Convex Optimization

    Peter Harremoes. “Divergence and Sufficiency for Convex Optimization.” In: Entropy 19.206 (Jan. 2017). doi: 10.3390/e19050206

    2017 doi

  3. [12]

    Theoret- ical aspects on measures of directed information with simulations

    Thomas Gkelsinis and Alex Karagrigoriou. “Theoret- ical aspects on measures of directed information with simulations.” In:Mathematics 8.4 (2020), p. 587

    2020

  4. [13]

    Attention to Entropic Communication

    Torsten Enßlin, Carolin Weidinger, and Philipp Frank. “Attention to Entropic Communication.” In: Annalen der Physik 536.7, 2300334 (July 2024), p. 2300334. doi: 10.1002/andp.202300334 . arXiv: 2307.11423 [cs.IT]

    2024 arXiv

  5. [14]

    Information Theory and Statisti- cal Mechanics

    E. T. Jaynes. “Information Theory and Statisti- cal Mechanics.” In: Phys. Rev. 106 (4 May 1957), pp. 620–630. doi: 10.1103/PhysRev.106.620. url: https://link.aps.org/doi/10.1103/PhysRev. 106.620

    1957 doi

  6. [15]

    Information Theory and Statistical Mechanics. II

    E. T. Jaynes. “Information Theory and Statistical Mechanics. II.” In: Phys. Rev. 108 (2 Oct. 1957), pp. 171–190. doi: 10.1103/PhysRev.108.171. url: https://link.aps.org/doi/10.1103/PhysRev. 108.171

    1957 doi

  7. [16]

    Information theory and statisti- cal mechanics (notes by the lecturer)

    Edwin T Jaynes. “Information theory and statisti- cal mechanics (notes by the lecturer).” In:Statistical physics 3 (1963), p. 181

    1963

  8. [17]

    Prior probabilities

    Edwin T Jaynes. “Prior probabilities.” In: IEEE Transactions on systems science and cybernetics4.3 (1968), pp. 227–241

    1968

  9. [18]

    E. T. Jaynes. Probability theory: The logic of science. Cambridge: Cambridge University Press, 2003

    2003

  10. [19]

    Expectation propagation for ap- proximate Bayesian inference

    Thomas P Minka. “Expectation propagation for ap- proximate Bayesian inference.” In: arXiv preprint arXiv:1301.2294 (2013)

    2013 arXiv

  11. [20]

    Variational inference: A review for statisticians

    David M Blei, Alp Kucukelbir, and Jon D McAuliffe. “Variational inference: A review for statisticians.” In: Journal of the American statistical Association 112.518 (2017), pp. 859–877

    2017

  12. [21]

    Metric Gaussian variational inference

    Jakob Knollmüller and Torsten A Enßlin. “Metric Gaussian variational inference.” In: arXiv preprint arXiv:1901.11033 (2019)

    2019 arXiv

  13. [22]

    Geometric Variational Inference

    Philipp Frank, Reimar Leike, and Torsten A. Enßlin. “Geometric Variational Inference.” In:Entropy 23.7, 853 (July 2021), p. 853. doi: 10 . 3390 / e23070853. arXiv: 2105.10470 [stat.ME]

    2021 arXiv

  14. [23]

    Information geometry

    SI Amari. “Information geometry.” In:Contemporary Mathematics 203 (1997), pp. 81–96

    1997

  15. [24]

    Information geometry

    Nihat Ay et al. Information geometry. Vol. 64. Springer, 2017

    2017

  16. [25]

    MIT press, 2001

    Manfred Opper and David Saad.Advanced mean field methods: Theory and practice. MIT press, 2001

    2001

  17. [26]

    An Idiosyncratic Journey Be- yond Mean Field Theory

    Jonathan S. Yedidia. “An Idiosyncratic Journey Be- yond Mean Field Theory.” In:Advanced Mean Field Methods: Theory and Practice. The MIT Press, June

  18. [28]

    Information Geometry of Alpha- Projection in Mean Field Approximation

    Sun-ichi Amari, Shiro Ikeda, and Hidetochi Shimokawa. “Information Geometry of Alpha- Projection in Mean Field Approximation.” In: Advanced Mean Field Methods: Theory and Practice. The MIT Press, June 2001. isbn: 9780262281058. doi: 10 . 7551 / mitpress / 1100 . 003 . 0020. ep...

    2001 doi

  19. [29]

    Information Geometry of Mean- Field Approximation

    Toshiyuki Tanaka. “Information Geometry of Mean- Field Approximation.” In: Advanced Mean Field Methods: Theory and Practice. The MIT Press, June

  20. [30]

    Practical variational inference for neu- ral networks

    Alex Graves. “Practical variational inference for neu- ral networks.” In: Advances in neural information processing systems24 (2011)

    2011

  21. [31]

    eprint: https://direct.mit.edu/ book/chapter-pdf/180244/9780262281058\_caq

    isbn: 9780262281058.doi: 10.7551/mitpress/ 1100.003.0021. eprint: https://direct.mit.edu/ book/chapter-pdf/180244/9780262281058\_caq. pdf. url: https://doi.org/10.7551/mitpress/ 1100.003.0021

    doi

  22. [32]

    An in- troduction to variational autoencoders

    Diederik P Kingma, Max Welling, et al. “An in- troduction to variational autoencoders.” In:Founda- tions and Trends in Machine Learning12.4 (2019), pp. 307–392

    2019

  23. [33]

    Stochastic gradient VB and the variational auto-encoder

    Diederik P Kingma and Max Welling. “Stochastic gradient VB and the variational auto-encoder.” In: Second international conference on learning represen- tations, ICLR. Vol. 19. 2014, p. 121

    2014

  24. [34]

    Probabilistic Autoencoder Using Fisher In- formation

    Johannes Zacherl, Philipp Frank, and Torsten A Enßlin. “Probabilistic Autoencoder Using Fisher In- formation.” In: Entropy 23.12 (2021), p. 1640

    2021

  25. [35]

    Bayesian decomposition of the Galactic multi-frequency sky using probabilis- tic autoencoders

    Sara Milosevic et al. “Bayesian decomposition of the Galactic multi-frequency sky using probabilis- tic autoencoders.” In:Astronomy & Astrophysics650 (2021), A100

    2021

  26. [36]

    Data compression

    David Salomon. Data compression. Springer, 2002

    2002

  27. [37]

    Probability of error for optimal codes in a Gaussian channel

    Claude E Shannon. “Probability of error for optimal codes in a Gaussian channel.” In:Bell System Tech- nical Journal38.3 (1959), pp. 611–656

    1959

  28. [38]

    Model fusion with Kullback- Leibler divergence

    Sebastian Claici et al. “Model fusion with Kullback- Leibler divergence.” In: International conference on machine learning. PMLR. 2020, pp. 2038–2047

    2020

  29. [39]

    Toward bayesian data compression

    Johannes Harth-Kitzerow et al. “Toward bayesian data compression.” In: Annalen der Physik 533.3 (2021), p. 2000508

    2021

  30. [40]

    Methods and tools for causal discovery and causal inference

    Ana Rita Nogueira et al. “Methods and tools for causal discovery and causal inference.” In:Wiley in- terdisciplinary reviews: data mining and knowledge discovery 12.2 (2022), e1449

    2022

  31. [41]

    Mutual Infor- mationBasedMatchingforCausalInferencewithOb- servational Data

    Lei Sun and Alexander G. Nikolaev. “Mutual Infor- mationBasedMatchingforCausalInferencewithOb- servational Data.” In: Journal of Machine Learning Research17.199(2016),pp.1–31. url: http://jmlr. org/papers/v17/15-420.html

    2016

  32. [42]

    The free energy prin- ciple for action and perception: A mathematical re- view

    Christopher L Buckley et al. “The free energy prin- ciple for action and perception: A mathematical re- view.” In: Journal of mathematical psychology 81 (2017), pp. 55–79

    2017

  33. [43]

    Inference with minimal Gibbs free energy in information field theory

    Torsten A. Enßlin and Cornelius Weig. “Inference with minimal Gibbs free energy in information field theory.” In: Phys. Rev. Lett. E 82.5, 051112 (Nov. 2010), p. 051112. doi: 10 . 1103 / PhysRevE . 82 . 051112. arXiv: 1004.2868 [astro-ph.IM]

    2010 arXiv

  34. [44]

    Sophisticated Inference

    Karl Friston et al. “Sophisticated Inference.” In:Neu- ral Computation33.3 (Mar. 2021), pp. 713–763.issn: 0899-7667. doi: 10 . 1162 / neco _ a _ 01351. eprint: https : / / direct . mit . edu / neco / article - pdf / 33 / 3 / 713 / 1889421 / neco \ _a \ _01351 . pdf. url: https...

    2021 doi

  35. [45]

    Active inference on discrete state-spaces: A synthesis

    Lancelot Da Costa et al. “Active inference on discrete state-spaces: A synthesis.” In:Journal of Mathemat- ical Psychology99 (2020), p. 102447

    2020

  36. [46]

    Kullback-Leibler information as a basis for strong inference in ecological studies

    Kenneth P Burnham and David R Anderson. “Kullback-Leibler information as a basis for strong inference in ecological studies.” In:Wildlife research 28.2 (2001), pp. 111–119

    2001

  37. [47]

    Reframing the Expected Free Energy: Four Formulations and a Unification

    Théophile Champion et al. “Reframing the Expected Free Energy: Four Formulations and a Unification.” In: arXiv e-prints, arXiv:2402.14460 (Feb. 2024), arXiv:2402.14460. doi: 10 . 48550 / arXiv . 2402 . 14460. arXiv: 2402.14460 [cs.AI]

    2024 doi arXiv

  38. [48]

    A Reputation Game Simulation: Emergent Social Phenomena from Information Theory

    Torsten Enßlin, Viktoria Kainz, and Céline Boehm. “A Reputation Game Simulation: Emergent Social Phenomena from Information Theory.” In:Annalen der Physik 534.5 (May 2022), p. 2100277. doi: 10 . 1002 / andp . 202100277 . arXiv: 2106 . 05414 [physics.soc-ph]

    2022

  39. [49]

    The Cambridge handbook of computational psychology

    Ron Sun. The Cambridge handbook of computational psychology. Cambridge University Press, 2008

    2008

  40. [50]

    Resource-aware research on Uni- verse and Matter: call-to-action in digital transfor- mation

    Ben Bruers et al. “Resource-aware research on Uni- verse and Matter: call-to-action in digital transfor- mation.” In: European Physical Journal Special Top- ics (Dec. 2024). doi: 10.1140/epjs/s11734- 024- 01436-4. arXiv: 2311.01169 [physics.comp-ph]. 20

    2024 arXiv

  41. [51]

    Cognitive efficiency: A conceptual and methodological comparison

    Bobby Hoffman. “Cognitive efficiency: A conceptual and methodological comparison.” In: Learning and Instruction 22.2 (2012), pp. 133–144.issn: 0959-4752. doi: https://doi.org/10.1016/j.learninstruc. 2011.09.001. url: https://www.sciencedirect. com/science/article/pii/S095947521100079X

    2012 doi

  42. [52]

    Signal En- hancement as Minimization of Relevant Informa- tion Loss

    Bernhard C. Geiger and Gernot Kubin. “Signal En- hancement as Minimization of Relevant Informa- tion Loss.” In:arXiv e-prints, arXiv:1205.6935 (May 2012), arXiv:1205.6935.doi: 10.48550/arXiv.1205

    2012 doi arXiv

  43. [53]

    Expected Information as Ex- pected Utility

    José Miguel Bernardo. “Expected Information as Ex- pected Utility.” In:Annals of Statistics7 (May 1979), pp. 686–690. doi: 10.1214/aos/1176344689 . url: https : / / api . semanticscholar . org / CorpusID : 121507326

    1979

  44. [54]

    On the calculation of mutual in- formation

    Tyrone E Duncan. “On the calculation of mutual in- formation.” In:SIAM Journal on Applied Mathemat- ics 19.1 (1970), pp. 215–220

    1970

  45. [55]

    On measures of entropy and informa- tion

    Alfréd Rényi. “On measures of entropy and informa- tion.” In: Proceedings of the fourth Berkeley sympo- sium on mathematical statistics and probability, vol- ume 1: contributions to the theory of statistics. Vol. 4. University of California Press. 1961, pp. 547–562

    1961

  46. [56]

    Computation of the amount of information about a stochastic function contained in another such function

    Izrail Moiseevich Gel’fand and Akiva Moiseevich Ya- glom. “Computation of the amount of information about a stochastic function contained in another such function.” In: Uspekhi Matematicheskikh Nauk 12.1 (1957), pp. 3–52

    1957

  47. [57]

    Probabilism, entropies and strictly properscoringrules

    Jürgen Landes. “Probabilism, entropies and strictly properscoringrules.” In: International Journal of Ap- proximate Reasoning63 (2015), pp. 1–21

    2015

  48. [58]

    On the estima- tion of the binomial probability in multistage clinical trials

    Sin-Ho Jung and Kyung Mann Kim. “On the estima- tion of the binomial probability in multistage clinical trials.” In:Statistics in medicine23.6(2004),pp.881– 896

    2004

  49. [59]

    Axiomatic characterization of the quadratic scoring rule

    Reinhard Selten. “Axiomatic characterization of the quadratic scoring rule.” In:Experimental Economics 1 (1998), pp. 43–61

    1998

  50. [60]

    Information field theory for cosmolog- ical perturbation reconstruction and nonlinear sig- nal analysis

    Torsten A. Enßlin, Mona Frommert, and Francisco S. Kitaura. “Information field theory for cosmolog- ical perturbation reconstruction and nonlinear sig- nal analysis.” In: Phys. Rev. Lett. D 80.10, 105005 (Nov. 2009), p. 105005.doi: 10.1103/PhysRevD.80. 105005. arXiv: 0806.3474...

    2009 arXiv

  51. [61]

    Equation of State Calcu- lations by Fast Computing Machines

    Nicholas Metropolis et al. “Equation of State Calcu- lations by Fast Computing Machines.” In:J. Chem. Phys 21.6 (June 1953), pp. 1087–1092.doi: 10.1063/ 1.1699114

    1953

  52. [62]

    Beta distribution — Wikipedia, The Free Encyclopedia

    Wikipedia contributors. Beta distribution — Wikipedia, The Free Encyclopedia . https : / / en . wikipedia . org / w / index . php ? title = Beta _ distribution & oldid = 1284886665. [Online; accessed 14-April-2025]. 2025

    2025

  53. [63]

    HybridMonteCarlo

    SimonDuaneetal.“HybridMonteCarlo.” In: Physics Letters B 195.2 (Sept. 1987), pp. 216–222.doi: 10. 1016/0370-2693(87)91197-X

    1987

  54. [64]

    A Conceptual Introduction to Hamiltonian Monte Carlo

    Michael Betancourt. “A Conceptual Introduction to Hamiltonian Monte Carlo.” In: arXiv e-prints, arXiv:1701.02434 (Jan. 2017), arXiv:1701.02434.doi: 10.48550/arXiv.1701.02434 . arXiv: 1701.02434 [stat.ME]

    2017 doi arXiv

  55. [65]

    Monte Carlo Sampling Methods using Markov Chains and their Applications

    W. K. Hastings. “Monte Carlo Sampling Methods using Markov Chains and their Applications.” In: Biometrika 57.1 (Apr. 1970), pp. 97–109. doi: 10 . 1093/biomet/57.1.97

    1970

  56. [68]

    Metric Gaussian Variational Inference

    Jakob Knollmüller and Torsten A. Enßlin. “Metric Gaussian Variational Inference.” In: arXiv e-prints, arXiv:1901.11033 (Jan. 2019), arXiv:1901.11033.doi: 10.48550/arXiv.1901.11033 . arXiv: 1901.11033 [stat.ML]. 21

    1901 doi arXiv

  57. [6935]

    arXiv: 1205.6935 [cs.IT]

    arXiv

Pith tools

Reviewed August 8, 2026 · model on record in the stance chip above.