Pith. sign in

REVIEW 3 major objections 3 minor 59 references

Perspectives on Tsallis Statistics for Artificial Intelligence

T0 review · 3 major / 3 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read This paper argues that Tsallis statistics, a one-parameter family of entropies, is not a physics curiosity but a practical unifying design principle for AI, making many independent methods the same q-deformation of standard Boltzmann-Gibbs

desk verdict A useful, honest review that makes the q-dial synthesis explicit and ships working code; the Section 4.1 claim that heavy-tailed SGD noise is a nonextensive signature overreaches and needs support or downgrading. read the letter →

arxiv 2608.01223 v1 pith:KGWCKVAL submitted 2026-08-02 cs.AI cs.CE

Kleyton da Costa , Bernardo Modenesi This is my paper
classification cs.AIcs.CE MSC 82B3068T0794A17 PACS 05.90.+m89.70.Cf07.05.Mh
keywords Tsallisentropynonextensivestatisticalmechanicsentropicindexqsparseattentionentmaxmaximum-entropyreinforcementlearningq-Gaussianheavy-tailedgradientnoise
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper is trying to establish that Tsallis statistics—the one-parameter family of entropies that reduces to Shannon entropy at $q=1$—is not a physics curiosity but a working design principle for artificial intelligence. The unifying claim is that many independently invented methods, including sparse attention (sparsemax and $\alpha$-entmax), maximum-entropy and sparse reinforcement learning, heavy-tailed latent-variable models, robust losses, and focal-style reweighting, are all the same move: deform a Boltzmann–Gibbs–Shannon object by a single index $q$. The paper reads $q$ as a dial controlling the weight of rare versus frequent events, with $q=1$ recovering softmax, Gaussian, and cross-entropy behavior, and $q\neq 1$ producing sparsity, heavy tails, or robustness. It further claims that the heavy-tailed weight spectra and gradient noise already observed in deep networks are nonextensive signatures, and that $q$ should be learned from data rather than fixed. If this is right, one pattern and one learnable scalar organize a broad slice of AI, connect it to nonextensive statistical mechanics, and supply a shared language for robustness and sparse computation.

What carries the argument

The carrier of the argument is the deformed pair $\ln_q x = (x^{1-q}-1)/(1-q)$ and $\exp_q(x) = [1+(1-q)x]_+^{1/(1-q)}$, together with the Tsallis entropy $S_q(p) = (1 - \sum_i p_i^q)/(q-1)$. The Tsallis cut-off $[\cdot]_+$ in the $q$-exponential is what produces exact zeros and therefore sparsity; the power-law tails for $q>1$ produce heavy-tailed distributions and robustness; the maximum-entropy variational problem with $S_q$ yields $q$-exponentials and $q$-Gaussians; and the $q$-central limit theorem supplies the attractor statement that correlated aggregates converge to $q$-Gaussian limits. Superstatistics—$p(\varepsilon)=\int f(\beta)e^{-\beta\varepsilon}d\beta$ with $f(\beta)$ a gamma

What would settle it

Measure the per-mini-batch inverse-temperature or gradient-scale distribution over many batches of a trained Transformer and compare the predicted $q = 1 + \mathrm{Var}(\beta)/\langle\beta\rangle^2$ with the $q$ fitted to the marginal gradient noise; a mismatch, or a scale distribution that is not gamma-like, would falsify the superstatistical account. A sharper test: check whether the increments' tail lies in the finite-variance $q$-Gaussian regime or in the divergent-variance $(q,\alpha)$-stable regime that the paper itself flags.

Watch

Extended reading notes

Core claim

The paper asserts that the deep-learning toolbox is implicitly Tsallisian. Its central object is the q-dial: every standard Boltzmann–Gibbs–Shannon component—the softmax, the Shannon-entropy regularizer, the Gaussian prior or likelihood, the cross-entropy loss—can be deformed by an entropic index $q$, and many known methods are exactly those deformed objects. Sparsemax is the max-Tsallis-entropy projection onto the simplex at $q=2$; $\alpha$-entmax generalizes it; Tsallis-entropy-regularized Markov decision processes have $q$-exponential optimal policies, with $q=2$ giving sparse policies; Student-$t$ and other robust heavy-tailed models are $q$-Gaussians for $1<q<3$; and the Tsallis cross-e

Load-bearing premise

The load-bearing premise is that the fluctuating scale (per-mini-batch curvature and gradient magnitude) in deep learning is roughly gamma-distributed, so that averaging local Gaussian fluctuations yields exactly $q$-exponential statistics with $q = 1 + \mathrm{Var}(\beta)/\langle\beta\rangle^2$; if that hidden-scale premise fails, the claim that heavy-tailed spectra and gradient noise are nonextensive signatures loses its foundation, even though the survey's unification woul

Editorial extensions

If this is right

  • Sparse attention maps with exact zeros become a special case of max-Tsallis-entropy projection; $q$ can be learned per head or per layer, and efficient sparse kernels make this practical at scale.
  • Reinforcement-learning policies become $q$-exponential functions of $Q$-values, so a single $q$ tunes exploration, sparsity, and near-greedy commitment; a learned $q$ anneals exploration into exploitation.
  • For $q<1$, the $q$-cross-entropy is bounded and resists label noise, giving a principled robustness dial that recovers ordinary cross-entropy at $q=1$.
  • Heavy-tailed gradient noise and weight spectra place SGD outside the Gaussian basin; $q$-CLT and stable-law analysis become the natural convergence language for deep learning.
  • Because every deformed method collapses to its Boltzmann–Gibbs–Shannon counterpart at $q=1$, adopting a $q$-dial is backward compatible with existing baselines.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • A practical transfer the paper leaves implicit: the algebraic q-dial means machinery built for one deformed method, such as differentiable entmax projections, can be transplanted into another, such as Tsallis-regularized policy updates, with minimal changes.
  • The superstatistics link suggests a direct experimental check the paper does not run: fit the gamma scale distribution to empirical mini-batch gradient norms and see whether the predicted $q$ reproduces the fitted gradient-noise tail.
  • The paper's own caveat that divergent variance routes to $(q,\alpha)$-stable laws rather than $q$-Gaussians implies that some heavy-tailed SGD phenomena may require stable-law theory, which is a sharper prediction than generic $q$-Gaussian fitting.
  • If $q$ genuinely mirrors distributional properties of the data, a learned $q$ could double as an out-of-distribution detector: shifts in attention sparsity or fitted tail index across domains would signal non-stationarity.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 3 minor

Summary. This is a perspective/review paper arguing that Tsallis statistics offers a unifying design principle for AI. The authors review the mathematical toolkit (q-entropy, q-logarithm/exponential, q-CLT, q-Gaussians, superstatistics) and survey a broad set of applications—sparse attention (sparsemax, α-entmax), maximum-entropy and sparse RL, heavy-tailed generative models, robust losses, and optimization—claiming that these are all instances of a single 'q-dial' that deforms a Boltzmann–Gibbs–Shannon object. They further argue that the heavy-tailed weight spectra and gradient noise observed in deep networks are 'nonextensive signatures,' and that q should be treated as a learnable inductive bias. The paper includes two small reproducible experiments on learning q (heavy-tailed density estimation and label-noise-robust classification) and releases an open-source JAX library (qjax).

Significance. The paper makes a genuinely useful organizing contribution. The mathematical review is careful and appropriately caveated, including the triple-index qualification on the q-CLT and the non-smoothness of the Tsallis cut-off. The identification of a common q-deformation pattern across independently developed methods—sparse attention, Tsallis-regularized RL, q-Gaussian generative models, bounded losses—is plausible and well illustrated. The open-source qjax library and the reproducible minimal experiments are concrete strengths. However, the stronger interpretive claim of Section 4.1, that heavy-tailed weight spectra and gradient noise are 'nonextensive signatures,' is not established by the evidence presented and currently conflicts with the paper's own q-CLT review in the cited divergent-variance regime. The q-dial unification of existing methods does not depend on this stronger claim, but the abstract and Section 4.1 present it as part of the thesis. The paper is thus valuable as a perspective, but the nonextensive-signatures component needs either support or explicit downgrading.

major comments (3)
  1. [Section 4.1 (vs. Section 2.3)] The claim that empirically observed gradient noise with finite tail index α<2 (ref. [50], divergent variance) makes q-Gaussians the 'natural stationary and increment distributions' contradicts the paper's own review in Section 2.3. There it states that when the (2q−1)-variance diverges, the attractor is a (q,α)-stable law, not a q-Gaussian; α<2 is exactly such a regime. To sustain the 'nonextensive signatures' claim, the paper needs either a derivation connecting α<2 stable noise to the q-Gaussian/q-CLT framework, or an explicit statement that q-Gaussians are only a phenomenological approximation in this regime. As written, Section 4.1 is internally inconsistent with Sections 2.3 and 2.4.
  2. [Section 4.1 (superstatistics premise, Eq. 14)] The superstatistical mechanism is asserted in Section 4.1 but not measured. Section 2.5 is properly hedged ('to the extent that these fluctuations are gamma-like'), yet Section 4.1 states that per-mini-batch curvature and gradient scale fluctuations 'yield q-exponential statistics' as though this were established. No measurement of the distribution of the fluctuating scale in deep networks is provided, and the paper's own Section 4.5 warns that 'heavy tails should be demonstrated, not assumed.' Either supply measurements or downgrade the language to a testable hypothesis/analogy. This is load-bearing because the abstract and Section 4.1 present these signatures as part of the paper's central thesis.
  3. [Section 1, page 3] The introductory characterization of the entropic index is internally inconsistent with the rest of the manuscript. It says q<1 encourages 'heavy tails' and q>1 encourages 'sparsity,' but Section 2.2 and Section 2.4 establish that q-exponentials and q-Gaussians have power-law tails for q>1 and compact support for q<1, and Table 2 maps q>1 to sparse/peaked behavior. The robust q-loss regime is also q<1 (Section 4.4), not q>1. Since the 'q-dial' direction is central to the paper's framing, this paragraph should be corrected so that the mapping from q to tail behavior is unambiguous and consistent.
minor comments (3)
  1. [Section 3.1, footnote 1] The note about normalization conventions is useful, but the statement that 'the identifications below hold up to this rescaling of the scores' is easy to misread. Consider stating explicitly how α maps to q in Table 2 and in the comparisons of learned α values across papers.
  2. [Section 4.4, Eq. (19)] The text says the q-gradient of the Tsallis entropy has a finite limit as q→1; giving the explicit limit value (or a pointer to it) would make the differentiability claim more concrete.
  3. [Section 4.5, 'Empirical justification'] This pitfall is well stated and should be applied directly to the Section 4.1 signatures discussion. As written, the two sections are in tension.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the paper's unifications are read off explicit definitions; the only serious weakness is an unsupported (and internally tensioned) interpretive claim in Section 4.1, which is a correctness/evidence gap, not a circular reduction.

full rationale

This is a perspective/review paper. Its organizing move is to display known methods as instances of deforming a BGS object by q: α-entmax is defined as the Sq-regularized argmax (Eq. 15), the Tsallis max-ent policy is a q-exponential (Eq. 16), q-Gaussians are the max-Sq densities (Eq. 13), and the q-cross-entropy is defined via ln_q (Eq. 17). The 'q-dial' pattern is therefore read off definitions, not fitted to data, so it is not a case of a fitted input being renamed a prediction. The superstatistics step (Section 2.5, Eq. 14) is a mathematical mixture identity: a gamma-distributed β yields a q-exponential marginal with q = 1+Var(β)/⟨β⟩². Section 4.1 invokes this identity to interpret empirically heavy-tailed weight spectra and gradient noise, but the paper does not fit β and then claim a prediction; it explicitly defers the test ('fit q (or the tail index α) and test it against the superstatistical prediction'). The paper repeatedly discloses its own evidentiary limits: Section 4.5 warns that 'Heavy tails should be demonstrated, not assumed'; Section 4.4 labels its experiments 'deliberately small and illustrative'; Section 3.4 marks the entmax/over-smoothing benefit as a 'plausible and testable conjecture rather than a demonstrated benefit.' The genuine flaw is non-circular: Section 4.1 cites Simsekli et al. for α<2 (divergent variance) and then calls the natural attractors q-Gaussians, whereas Section 2.3 says divergent (2q−1)-variance leads to (q,α)-stable laws, not q-Gaussians. That is an unsupported mapping or internal inconsistency, not an output that reduces to an input. There are no load-bearing self-citations by the present authors; the cited q-CLT and superstatistics theorems are external results. Under the stated standard, no circular step can be exhibited, so the score is 0.

Assumptions & free parameters 3 free parameters · 5 assumptions · 0 invented entities

The paper introduces no new physical or mathematical entities: the 'q-dial' is a descriptive metaphor and 'nonextensive signatures' is a reframing of already-measured phenomena. The only fitted quantities appear in the two small illustrations and are disclosed as such. The superstatistical variance that would fix q for SGD noise is unmeasured, so the Section 4.1 reading of learning dynamics rests on an unestimated premise.

free parameters (3)
  • learned q in heavy-tailed density estimation illustration = q_hat = 1.49
    Fitted by maximum likelihood to synthetic data drawn from a q-Gaussian with q*=1.5 (Section 4.4, Table 3a, experiment 1). Disclosed as a self-consistency check of gradient flow into q, not as a prediction.
  • learned q in label-noise classification illustration = q_hat about 0.08-0.33 depending on noise rate
    Selected on a clean held-out validation set under a weak prior centered at q=1 (Section 4.4, Table 3b). The paper honestly reports that q is weakly identified at rho=0 (overlapping standard deviations).
  • superstatistical relative variance Var(beta)/<beta>^2 for SGD = not measured
    Sections 2.5 and 4.1 invoke q = 1 + Var(beta)/<beta>^2 for mini-batch SGD fluctuations, but the variance of the fluctuating scale is asserted as gamma-like rather than estimated. This unmeasured premise underlies the 'nonextensive signatures' reading of gradient noise.
assumptions (5)
  • domain assumption Umarov-Tsallis-Steinberg q-CLT (Theorem 1, informal)
    The claim that aggregates of correlated variables converge to q-Gaussians (Section 2.3, used in Section 4.1) rests on this specific generalization of the CLT, which the paper itself warns is frequently blurred in applied work due to the triple-index bookkeeping.
  • standard math Superstatistics with chi-squared mixing yields q-exponential marginals
    Equation (14) with a gamma f(beta) reproducing exp_q is a textbook-level result (Beck-Cohen [18]); used as the dynamical origin of q in Sections 2.5 and 4.1.
  • domain assumption Mini-batch SGD scale fluctuations are gamma-like
    Applied in Section 4.1 to explain gradient noise as q-exponential. The premise is explicitly conditional ('to the extent that these fluctuations are gamma-like', Section 2.5) and untested in the paper. Load-bearing for the 'nonextensive signatures' thesis.
  • standard math Concavity and maximizer properties of Sq (Furuichi)
    Used in Section 2.1 to justify that the Tsallis-entmax maximization in Eq. (15) is well-posed as a generalized max-entropy projection.
  • domain assumption Differentiability of entmax outputs through KKT conditions
    Section 4.4 asserts that differentiating through the entmax KKT system yields a usable gradient d(p*)/dq with only measure-zero non-smoothness at the Tsallis cut-off. Stated without a formal theorem, and load-bearing for the learnable-q proposal; the paper cites Vasylenko et al. [26] as the scale demonstration.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Perspectives on Tsallis Statistics for Artificial Intelligence." pith.science (2026). https://pith.science/paper/KGWCKVAL

@misc{pith2026260801223,
  author       = {Pith},
  title        = {Pith review of: Perspectives on Tsallis Statistics for Artificial Intelligence},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/KGWCKVAL}},
  note         = {Machine review of arXiv:2608.01223}
}
abstract

Tsallis statistics generalizes Boltzmann-Gibbs statistical mechanics through a single real parameter $q$ that controls the weight assigned to rare and frequent events. Originally proposed to describe physical systems with long-range correlations, multifractal geometry, and heavy-tailed fluctuations, the framework has become a recurring ingredient in modern artificial intelligence (AI): it underlies sparse attention mechanisms (\textsc{sparsemax} and $\alpha$-\textsc{entmax}), maximum-entropy reinforcement learning with controllable exploration, robust and heavy-tailed probabilistic models, and a family of generalized loss functions and regularizers. This paper offers a structured perspective on where Tsallis statistics meets AI. We first review the mathematical core: $q$-entropy and its variational (maximum-entropy) foundation, the $q$-exponential and $q$-logarithm, the $q$-central limit theorem, $q$-Gaussian distributions, and their dynamical origin in superstatistics, emphasizing the properties that matter for machine learning. We then survey applications across softmax generalization, reinforcement learning, sequential and graph neural models, generative and probabilistic modeling, loss design, and optimization, extracting the recurring design pattern in each case: a tunable interpolation between dense/uniform and sparse/peaked behavior governed by $q$. We further argue that the heavy-tailed weight spectra and gradient-noise statistics empirically observed in deep networks are themselves nonextensive signatures, placing modern learning dynamics within the scope of $q$-statistics. Finally, we discuss methodological pitfalls, the relationship to information geometry and $q$-exponential families, and open directions, arguing that $q$ should be treated as a learnable inductive bias rather than a fixed hyperparameter.

Figures

Figures reproduced from arXiv: 2608.01223 by the authors.

Figure 1
Figure 1. Tsallis statistics as an organizing field for artificial intelligence. A single [PITH_FULL_IMAGE:figures/full_fig_p003_1.png] view at source ↗
Figure 2
Figure 2. Constantino Tsallis, who introduced the generalized entropy Sq in 1988. This section reviews the mathematical core of Tsal￾lis statistics at the level of detail an ML researcher needs. Standard references are the founding paper [1], the monograph [4], and the broader literature on gen￾eralized thermostatistics [5]. We work throughout with discrete distributions p = (p1, . . . , pn) on a finite al￾phabet and with the… view at source ↗
Figure 3
Figure 3. Additivity is a property of the entropy functional; [PITH_FULL_IMAGE:figures/full_fig_p007_3.png] view at source ↗
Figures from the paper (3 more)
Figure 4
Figure 4. Figure 4: The q-Gaussian Gq(x) ∝ expq (−x 2 ) as a one-parameter bridge across classical distributions (curves normalized to unit peak; all three panels show the same curves). (a) On a linear scale, q < 1 yields compact support (the curve reaches exactly zero, via the Tsallis cu…
Figure 5
Figure 5. Figure 5: Generalizing the softmax via Tsallis entropy (Eq. [PITH_FULL_IMAGE:figures/full_fig_p016_5.png]
Figure 6
Figure 6. Figure 6: Empirical illustration of a learnable entropic index [PITH_FULL_IMAGE:figures/full_fig_p028_6.png]

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

59 extracted references · 39 canonical work pages

  1. [50]

    Şimşekli, L

    U. Şimşekli, L. Sagun, M. Gürbüzbalaban, A tail-index analysis of stochastic gradient noise in deep neural networks, in: Proceedings of the 36th International Conference on Machine Learning (ICML), 2019, pp. 5827–5837. URL: https://proceedings.mlr.press/v97/ simsekli19a.html

    work page 2019

  2. [1]

    Tsallis, Possible generalization of Boltzmann–Gibbs statistics, Journal of Statistical Physics 52 (1988) 479–487

    C. Tsallis, Possible generalization of Boltzmann–Gibbs statistics, Journal of Statistical Physics 52 (1988) 479–487. URL:https://doi.org/10. 1007/BF01016429

    work page 1988

  3. [2]

    S. A. Sepúlveda Fontaine, J. M. Amigó, Applications of entropy in data analysis and machine learning: A review, Entropy 26 (2024) 1126. URL: https://doi.org/10.3390/e26121126

    doi:10.3390/e26121126 2024

  4. [3]

    Entropy measures and their applications: A comprehensive review

    N. Kumar, A. Dixit, V. Vijay, Entropy measures and their applications: A comprehensive review, arXiv preprint arXiv:2503.15660 (2025). URL: https://arxiv.org/abs/2503.15660. 31

    work page Pith review arXiv 2025

  5. [4]

    Tsallis, Introduction to Nonextensive Statistical Mechanics: Ap- proaching a Complex World, Springer, New York, 2009

    C. Tsallis, Introduction to Nonextensive Statistical Mechanics: Ap- proaching a Complex World, Springer, New York, 2009. URL:https: //doi.org/10.1007/978-0-387-85359-8

    doi:10.1007/978-0-387-85359-8 2009

  6. [5]

    Naudts, Generalised Thermostatistics, Springer, London, 2011

    J. Naudts, Generalised Thermostatistics, Springer, London, 2011. URL: https://doi.org/10.1007/978-0-85729-355-8

    doi:10.1007/978-0-85729-355-8 2011

  7. [6]

    A. R. Plastino, A. Plastino, Stellar polytropes and Tsallis’ entropy, Physics Letters A 174 (1993) 384–386. URL: https://doi.org/10. 1016/0375-9601(93)90195-6

    work page 1993

  8. [7]

    Gell-Mann, C

    M. Gell-Mann, C. Tsallis (Eds.), Nonextensive Entropy: Interdisciplinary Applications, OxfordUniversityPress, 2004.URL: https://global.oup. com/academic/product/nonextensive-entropy-9780195159776

    work page 2004

Show all 59 references
  1. [8]

    Tsallis, Beyond Boltzmann–Gibbs–Shannon in physics and elsewhere, Entropy 21 (2019) 696

    C. Tsallis, Beyond Boltzmann–Gibbs–Shannon in physics and elsewhere, Entropy 21 (2019) 696. URL:https://doi.org/10.3390/e21070696

    2019 doi

  2. [9]

    Penrose, Foundations of Statistical Mechanics: A Deductive Treat- ment, Pergamon Press, Oxford, 1970

    O. Penrose, Foundations of Statistical Mechanics: A Deductive Treat- ment, Pergamon Press, Oxford, 1970

    1970

  3. [10]

    Furuichi, Information theoretical properties of Tsallis entropies, Jour- nal of Mathematical Physics 47 (2006) 023302

    S. Furuichi, Information theoretical properties of Tsallis entropies, Jour- nal of Mathematical Physics 47 (2006) 023302. URL:https://doi.org/ 10.1063/1.2165744

    2006 doi

  4. [11]

    Tsallis, R

    C. Tsallis, R. S. Mendes, A. R. Plastino, The role of constraints within generalized nonextensive statistics, Physica A: Statistical Mechanics and its Applications 261 (1998) 534–554. URL:https://doi.org/10.1016/ S0378-4371(98)00437-3

    1998

  5. [12]

    Nielsen, R

    F. Nielsen, R. Nock, A closed-form expression for the Sharma–Mittal entropy of exponential families, Journal of Physics A: Mathematical and Theoretical 45 (2012) 032003. URL:https://doi.org/10.1088/ 1751-8113/45/3/032003

    2012

  6. [13]

    Hanel, S

    R. Hanel, S. Thurner, A comprehensive classification of complex sta- tistical systems and an axiomatic derivation of their entropy and dis- tribution functions, Europhysics Letters (EPL) 93 (2011) 20006. URL: https://doi.org/10.1209/0295-5075/93/20006. 32

    2011 doi

  7. [14]

    E. M. F. Curado, C. Tsallis, Generalized statistical mechanics: connection with thermodynamics, Journal of Physics A: Mathematical and General 24 (1991) L69–L72. URL:https://doi.org/10.1088/0305-4470/24/ 2/004

    1991 doi

  8. [15]

    Umarov, C

    S. Umarov, C. Tsallis, S. Steinberg, On a q-central limit theorem consistent with nonextensive statistical mechanics, Milan Journal of Mathematics 76 (2008) 307–328. URL: https://doi.org/10.1007/ s00032-008-0087-y

    2008

  9. [16]

    E. P. Borges, A possible deformed algebra and calculus inspired in nonextensive thermostatistics, Physica A: Statistical Mechanics and its Applications 340 (2004) 95–101. URL:https://doi.org/10.1016/ j.physa.2004.03.082

    2004

  10. [17]

    Prato, C

    D. Prato, C. Tsallis, Nonextensive foundation of Lévy distributions, Physical Review E 60 (1999) 2398–2401. URL:https://doi.org/10. 1103/PhysRevE.60.2398

    1999

  11. [18]

    C. Beck, E. G. D. Cohen, Superstatistics, Physica A: Statistical Mechanics and its Applications 322 (2003) 267–275. URL: https: //doi.org/10.1016/S0378-4371(03)00019-0

    2003 doi

  12. [19]

    A. F. T. Martins, R. F. Astudillo, From softmax to sparsemax: A sparse model of attention and multi-label classification, in: Proceedings of the 33rd International Conference on Machine Learning (ICML), 2016, pp. 1614–1623. URL: https://proceedings.mlr.press/v48/martins16. html

    2016

  13. [20]

    Peters, V

    B. Peters, V. Niculae, A. F. T. Martins, Sparse sequence-to-sequence models, in: Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics (ACL), 2019, pp. 1504–1519. URL:https: //doi.org/10.18653/v1/P19-1146

    2019 doi

  14. [21]

    G. M. Correia, V. Niculae, A. F. T. Martins, Adaptively sparse trans- formers, in: Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing (EMNLP-IJCNLP), 2019, pp. 2174–2184. URL:https://doi.org/10.18653/v1/D19-1223. 33

    2019 doi

  15. [22]

    Blondel, A

    M. Blondel, A. F. T. Martins, V. Niculae, Learning with Fenchel–Young losses, Journal of Machine Learning Research 21 (2020) 1–69. URL: https://jmlr.org/papers/v21/19-021.html

    2020

  16. [23]

    A. F. T. Martins, A. Farinhas, M. Treviso, V. Niculae, P. M. Q. Aguiar, M. A. T. Figueiredo, Sparse and continuous attention mechanisms, in: Advances in Neural Information Processing Systems (NeurIPS), volume 33, 2020. URL:https://arxiv.org/abs/2006.07214

    2020 arXiv

  17. [24]

    A. F. T. Martins, M. Treviso, A. Farinhas, P. M. Q. Aguiar, M. A. T. Figueiredo, M. Blondel, V. Niculae, Sparse continuous distributions and Fenchel–Young losses, Journal of Machine Learning Research 23 (2022) 1–74. URL:https://arxiv.org/abs/2108.01988

    2022 arXiv

  18. [25]

    Gonçalves, M

    N. Gonçalves, M. Treviso, A. F. T. Martins, AdaSplash: Adaptive sparse flash attention, in: Proceedings of the 42nd International Conference on Machine Learning (ICML), 2025. URL:https://proceedings.mlr. press/v267/goncalves25a.html

    2025

  19. [26]

    Vasylenko, H

    P. Vasylenko, H. Pitorro, A. F. T. Martins, M. Treviso, Long-context generalization with sparse attention, in: International Conference on Learning Representations (ICLR), 2026. URL: https://arxiv.org/ abs/2506.16640

    2026

  20. [27]

    Haarnoja, A

    T. Haarnoja, A. Zhou, P. Abbeel, S. Levine, Soft actor-critic: Off-policy maximum entropy deep reinforcement learning with a stochastic actor, in: Proceedings of the 35th International Conference on Machine Learning (ICML), 2018, pp. 1861–1870. URL:https://proceedings.mlr.pres...

    2018

  21. [28]

    K. Lee, S. Choi, S. Oh, Sparse Markov decision processes with causal sparse Tsallis entropy regularization for reinforcement learning, IEEE Robotics and Automation Letters 3 (2018) 1466–1473. URL:https: //doi.org/10.1109/LRA.2018.2800085

    2018

  22. [29]

    K. Lee, S. Kim, S. Lim, S. Choi, S. Oh, Tsallis reinforcement learning: A unified framework for maximum entropy reinforcement learning, arXiv preprint arXiv:1902.00137 (2019). URL:https://arxiv.org/abs/1902. 00137. 34

    1902 arXiv

  23. [30]

    Y. Chow, O. Nachum, M. Ghavamzadeh, Path consistency learning in Tsallis entropy regularized MDPs, in: Proceedings of the 35th Inter- national Conference on Machine Learning (ICML), 2018, pp. 979–988. URL:https://proceedings.mlr.press/v80/chow18a.html

    2018

  24. [31]

    Geist, B

    M. Geist, B. Scherrer, O. Pietquin, A theory of regularized Markov decision processes, in: Proceedings of the 36th International Conference on Machine Learning (ICML), 2019, pp. 2160–2169. URL:https:// arxiv.org/abs/1901.11275

    2019 arXiv

  25. [32]

    Vieillard, O

    N. Vieillard, O. Pietquin, M. Geist, Munchausen reinforcement learning, in: Advances in Neural Information Processing Systems (NeurIPS), volume 33, 2020. URL:https://arxiv.org/abs/2007.14430

    2020 arXiv

  26. [33]

    Zimmert, Y

    J. Zimmert, Y. Seldin, Tsallis-INF: An optimal algorithm for stochastic and adversarial bandits, Journal of Machine Learning Research 22 (2021) 1–49. URL:https://arxiv.org/abs/1807.07623

    2021 arXiv

  27. [34]

    Zhang, S

    J. Zhang, S. P. Karimireddy, A. Veit, S. Kim, S. J. Reddi, S. Kumar, S. Sra, Why are adaptive methods good for attention models?, in: Ad- vances in Neural Information Processing Systems (NeurIPS), volume 33,

  28. [35]

    Treviso, A

    M. Treviso, A. Góis, P. Fernandes, E. Fonseca, A. F. T. Martins, Predicting attention sparsity in transformers, in: Proceedings of the Sixth Workshop on Structured Prediction for NLP, 2022. URL: https://arxiv.org/abs/2109.12188

    2022 arXiv

  29. [36]

    Y. Ye, S. Ji, Sparse graph attention networks, IEEE Transactions on Knowledge and Data Engineering 35 (2023) 905–916. URL:https: //arxiv.org/abs/1912.00552

    2023 arXiv

  30. [37]

    Q. Li, Z. Han, X.-M. Wu, Deeper insights into graph convolutional networks for semi-supervised learning, in: Proceedings of the 32nd AAAI Conference on Artificial Intelligence (AAAI), 2018. URL:https: //arxiv.org/abs/1801.07606

    2018 arXiv

  31. [38]

    K. Oono, T. Suzuki, Graph neural networks exponentially lose expressive power for node classification, in: International Conference on Learning Representations (ICLR), 2020. URL: https://arxiv.org/abs/1905. 10947. 35

    2020

  32. [39]

    Takahashi, T

    H. Takahashi, T. Iwata, Y. Yamanaka, M. Yamada, S. Yagi, Student-t variational autoencoder for robust density estimation, in: Proceedings of the 27th International Joint Conference on Artificial Intelligence (IJCAI), 2018, pp. 2696–2702. URL:https://doi.org/10.24963/ijcai.2018/ 374

    2018 doi

  33. [40]

    Okhotin, D

    A. Okhotin, D. Molchanov, V. Arkhipkin, G. Bartosh, V. Ohanesian, A. Alanov, D. Vetrov, Star-shaped denoising diffusion probabilistic mod- els, in: Advances in Neural Information Processing Systems (NeurIPS), volume 36, 2023. URL:https://arxiv.org/abs/2302.05259

    2023 arXiv

  34. [41]

    Pandey, J

    K. Pandey, J. Pathak, Y. Xu, S. Mandt, M. Pritchard, A. Vahdat, M. Mardani, Heavy-tailed diffusion models, in: International Conference on Learning Representations (ICLR), 2025. URL:https://arxiv.org/ abs/2410.14171

    2025 arXiv

  35. [42]

    Shariatian, U

    D. Shariatian, U. Şimşekli, A. Durmus, Heavy-tailed diffusion with denoising lévy probabilistic models, in: International Conference on Learning Representations (ICLR), 2025. URL: https://arxiv.org/ abs/2407.18609

    2025 arXiv

  36. [43]

    van der Maaten, G

    L. van der Maaten, G. Hinton, Visualizing data using t-SNE, Journal of Machine Learning Research 9 (2008) 2579–2605. URL:https://www. jmlr.org/papers/v9/vandermaaten08a.html

    2008

  37. [44]

    Zhang, M

    Z. Zhang, M. R. Sabuncu, Generalized cross entropy loss for train- ing deep neural networks with noisy labels, in: Advances in Neu- ral Information Processing Systems (NeurIPS), volume 31, 2018, pp. 8778–8788. URL:https://proceedings.neurips.cc/paper/2018/ hash/f2925f97bc13ad2...

    2018

  38. [45]

    E. Amid, M. K. Warmuth, S. Srinivasan, Two-temperature logistic regres- sion based on the Tsallis divergence, in: Proceedings of the 22nd Inter- national Conference on Artificial Intelligence and Statistics (AISTATS), 2019, pp. 2388–2396. URL:https://arxiv.org/abs/1705.07210

    2019 arXiv

  39. [46]

    E. Amid, M. K. Warmuth, R. Anil, T. Koren, Robust bi-tempered logistic loss based on Bregman divergences, in: Advances in Neural Information Processing Systems (NeurIPS), volume 32, 2019. URL:https://arxiv. org/abs/1906.03361. 36

    2019 arXiv

  40. [47]

    Tsallis, D

    C. Tsallis, D. A. Stariolo, Generalized simulated annealing, Physica A: Statistical Mechanics and its Applications 233 (1996) 395–406. URL: https://doi.org/10.1016/S0378-4371(96)00271-3

    1996 doi

  41. [48]

    A. C. Soterroni, R. L. Galski, F. M. Ramos, Theq-gradient method for global optimization, arXiv preprint arXiv:1209.2084 (2012). URL: https://arxiv.org/abs/1209.2084

    arXiv

  42. [49]

    C. H. Martin, M. W. Mahoney, Implicit self-regularization in deep neural networks: Evidence from random matrix theory and implications for learning, Journal of Machine Learning Research 22 (2021) 1–73. URL: https://jmlr.org/papers/v22/20-410.html

    2021

  43. [51]

    Gürbüzbalaban, U

    M. Gürbüzbalaban, U. Şimşekli, L. Zhu, The heavy-tail phenomenon in SGD, in: Proceedings of the 38th International Conference on Machine Learning (ICML), 2021, pp. 3964–3975. URL:https://arxiv.org/abs/ 2006.04740

    2021 arXiv

  44. [52]

    Hodgkinson, M

    L. Hodgkinson, M. W. Mahoney, Multiplicative noise and heavy tails in stochastic optimization, in: Proceedings of the 38th International Conference on Machine Learning (ICML), 2021, pp. 4262–4274. URL: https://arxiv.org/abs/2006.06293

    2021 arXiv

  45. [53]

    Şimşekli, O

    U. Şimşekli, O. Sener, G. Deligiannidis, M. A. Erdogdu, Hausdorff di- mension, heavy tails, and generalization in neural networks, in: Advances in Neural Information Processing Systems (NeurIPS), volume 33, 2020. URL:https://arxiv.org/abs/2006.09313

    2020 arXiv

  46. [54]

    Barsbey, M

    M. Barsbey, M. Sefidgaran, M. A. Erdogdu, G. Richard, U. Şimşekli, Heavy tails in SGD and compressibility of overparametrized neural net- works, in: AdvancesinNeuralInformationProcessingSystems(NeurIPS), volume 34, 2021. URL:https://arxiv.org/abs/2106.03795. 37

    2021 arXiv

  47. [55]

    Amari, Information Geometry and Its Applications, volume 194 ofApplied Mathematical Sciences, Springer, 2016

    S.-i. Amari, Information Geometry and Its Applications, volume 194 ofApplied Mathematical Sciences, Springer, 2016. URL:https://doi. org/10.1007/978-4-431-55978-8

    2016 doi

  48. [56]

    Amari, A

    S.-i. Amari, A. Ohara, Geometry ofq-exponential family of probability distributions, Entropy 13 (2011) 1170–1185. URL:https://doi.org/ 10.3390/e13061170

    2011 doi

  49. [57]

    Korbel, R

    J. Korbel, R. Hanel, S. Thurner, Information geometric duality ofϕ- deformed exponential families, Entropy 21 (2019) 112. URL:https: //doi.org/10.3390/e21020112

    2019 doi

  50. [58]

    w_key"]) @ params[

    E. Amid, R. Nock, M. K. Warmuth, Clustering above exponential families with tempered exponential measures, in: Proceedings of the 26th International Conference on Artificial Intelligence and Statistics (AISTATS), 2023, pp. 2994–3017. URL:https://arxiv.org/abs/2211. 02765. Appe...

    2023

  51. [2020]

    URL:https://arxiv.org/abs/1912.03194

    1912 arXiv

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.