REVIEW 2 major objections 2 minor 2 cited by
Bayesian inference for the learning rate in Generalised Bayesian inference
T0 review · 2 major / 2 minor · reviewed 2026-05-22 · grok-4.3
Pith's one-line read Bayesian inference on held-out data estimates the learning rate for generalised Bayesian inference.
desk verdict Paper gives a held-out Bayesian route to posteriors over the learning rate in GBI using two new utilities, but the whole thing rests on the existence of recoverable true hyperparameter values. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
Hyperparameter posterior defined using held-out data in Generalised Bayesian Inference, specifically the ELPPD-utility based posterior and the pseudo-true parameter covering posterior.
What would settle it
If applying the method to new simulated or real datasets shows that the GBI-posteriors do not outperform standard Bayesian inference or fail to select optimal hyperparameters, the practical utility would be questioned.
Extended reading notes
Core claim
In settings where unknown true values of the learning rate and loss hyperparameters exist, Bayesian inference with held-out data produces hyperparameter posteriors. Two such posteriors are defined: one maximising an ELPPD-utility and one targeting coverage of the pseudo-true parameter. This framework enables estimation and uncertainty quantification for multiple hyperparameters jointly, with asymptotic results provided for multi-modular Generalised Bayes posteriors used in the examples.
Load-bearing premise
There exist unknown true hyperparameter values about which it is meaningful to have prior belief.
Editorial extensions
If this is right
- GBI-posteriors outperform standard Bayesian inference on simulated test data.
- The method selects optimal or near-optimal hyperparameter values in large real problems like text analysis.
- Supports joint estimation and uncertainty quantification for multiple hyperparameters.
- Asymptotic results hold for special multi-modular Generalised Bayes posteriors.
Reading between the lines
- This approach may extend to other forms of robust or generalised inference where hyperparameters need tuning.
- Applying it to more complex models could reveal how uncertainty in hyperparameters propagates to predictions.
- Comparing the two defined posteriors on additional benchmarks might show which is preferable in different settings.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper proposes a Bayesian framework for inferring the learning rate and other loss hyperparameters in Generalised Bayesian Inference (GBI) by performing inference on held-out data. It defines two hyperparameter posteriors—one based on an ELPPD-utility and one targeting coverage of the pseudo-true parameter—supports joint estimation of multiple hyperparameters, provides asymptotic results for multi-modular GBI posteriors, and reports experiments showing outperformance over standard Bayesian inference on simulated data plus near-optimal hyperparameter selection in a large text-analysis application.
Significance. If the central assumptions hold, the work supplies a principled route to uncertainty quantification for GBI hyperparameters, especially valuable in multi-dataset settings where standard Bayesian inference is known to be fragile. The combination of asymptotic analysis for the multi-modular case and concrete experimental gains on both simulated and real problems would strengthen the practical case for GBI in misspecified or heterogeneous data regimes.
major comments (2)
- [Section 3 (definitions of the two hyperparameter posteriors)] The construction of both hyperparameter posteriors presupposes the existence of stable, identifiable 'true' hyperparameter values recoverable from held-out likelihoods. In the multi-modular GBI examples that combine heterogeneous data sets, model misspecification can render any such pseudo-true value non-unique, so that the resulting hyperparameter posterior may concentrate on an artifact rather than a quantity that demonstrably improves downstream GBI performance.
- [Section 5 (experiments)] The experimental claims rest on a held-out data protocol whose precise implementation (train/held-out split, post-hoc choices, and definition of the two posteriors) is not fully specified in the abstract or summary; without these details the reported outperformance on simulated test data and optimal selection in the text-analysis example cannot be independently verified.
minor comments (2)
- [Section 4 (asymptotics)] The asymptotic results for the special multi-modular posteriors are stated but their regularity conditions and the precise sense in which they justify the hyperparameter posteriors could be stated more explicitly.
- [Throughout] Notation for the learning rate, loss hyperparameters, and the two hyperparameter posteriors should be introduced once and used consistently to avoid reader confusion.
Simulated Author's Rebuttal
We thank the referee for their constructive and insightful comments. We address each of the major comments in turn below, providing clarifications and indicating where revisions will be made to strengthen the manuscript.
read point-by-point responses
-
Referee: [Section 3 (definitions of the two hyperparameter posteriors)] The construction of both hyperparameter posteriors presupposes the existence of stable, identifiable 'true' hyperparameter values recoverable from held-out likelihoods. In the multi-modular GBI examples that combine heterogeneous data sets, model misspecification can render any such pseudo-true value non-unique, so that the resulting hyperparameter posterior may concentrate on an artifact rather than a quantity that demonstrably improves downstream GBI performance.
Authors: We appreciate the referee raising this important point about potential non-uniqueness under misspecification. Our hyperparameter posteriors are defined directly via optimization of explicit held-out utilities (ELPPD for the first and pseudo-true coverage for the second), rather than assuming a unique identifiable truth a priori. In the multi-modular setting, the asymptotic results we provide establish concentration of the hyperparameter posterior around the value(s) that optimize the chosen utility, even when the underlying pseudo-true parameter is a set rather than a singleton. The experiments, including the heterogeneous data examples, show that the resulting GBI posteriors yield measurable improvements in predictive performance and calibration over standard Bayesian inference, indicating that the inferred hyperparameters are functionally useful rather than artifacts. We will add a brief discussion of this nuance to Section 3 in the revision. revision: partial
-
Referee: [Section 5 (experiments)] The experimental claims rest on a held-out data protocol whose precise implementation (train/held-out split, post-hoc choices, and definition of the two posteriors) is not fully specified in the abstract or summary; without these details the reported outperformance on simulated test data and optimal selection in the text-analysis example cannot be independently verified.
Authors: We agree that greater detail on the experimental protocol is essential for reproducibility. The current manuscript provides the high-level protocol in Section 5 and the supplementary material, but we acknowledge that the precise train/held-out splits, implementation of the two hyperparameter posteriors, and any post-hoc decisions are not described at the level of granularity needed for independent verification. In the revised manuscript we will expand Section 5 with a dedicated subsection and add an appendix that fully specifies the data partitioning, exact definitions and computational implementations of both hyperparameter posteriors, and all experimental choices for the simulated and text-analysis examples. revision: yes
Circularity Check
No significant circularity; hyperparameter posteriors derived from held-out data as external benchmark
full rationale
The paper defines hyperparameter posteriors for the learning rate in GBI via Bayesian updating on held-out data, using an ELPPD-utility and a pseudo-true parameter target. This relies on data splits independent of the main inference, avoiding self-definition or fitted-input-as-prediction. No load-bearing self-citations, uniqueness theorems from prior author work, or ansatz smuggling are required for the central construction; asymptotic results for multi-modular cases are stated separately. The framework is self-contained against external benchmarks (held-out likelihoods), so the derivation chain does not reduce to its inputs by construction.
Assumptions & free parameters
assumptions (1)
- domain assumption Existence of unknown true hyperparameter values about which prior belief is meaningful
Cite this review
Pith. "Pith review of Bayesian inference for the learning rate in Generalised Bayesian inference." pith.science (2026). https://pith.science/paper/3BMVUJ5X
@misc{pith2026250612532,
author = {Pith},
title = {Pith review of: Bayesian inference for the learning rate in Generalised Bayesian inference},
year = {2026},
howpublished = {\url{https://pith.science/paper/3BMVUJ5X}},
note = {Machine review of arXiv:2506.12532}
}
read the original abstract
In Generalised Bayesian Inference (GBI), the learning rate and hyperparameters of the loss must be estimated. These inference-hyperparameters can't be estimated jointly with the other parameters, from the data, by giving them a prior. However, in some settings there exist unknown ``true'' hyperparameter-values about which it is meaningful to have prior belief. It is then possible to use Bayesian inference with held-out data to get hyperparameter-posteriors. We define two hyperparameter posteriors, one based on an ELPPD-utility and one aiming to cover the pseudo-true parameter. The new framework supports estimation and uncertainty quantification for multiple hyperparameters jointly. Experiments show that the resulting GBI-posteriors out-perform Bayesian inference on simulated test data and select optimal or near optimal hyperparameter values in a large real problem of text analysis. Generalised Bayesian inference is particularly useful for combining multiple data sets and most of our examples belong to that setting. We also give asymptotic results for some of the special ``multi-modular'' Generalised Bayes posteriors which we use in our examples.
Lean theorems connected to this paper
-
IndisputableMonolith/Cost/FunctionalEquation.leanwashburn_uniqueness_aczel unclear?
unclearRelation between the paper passage and the cited Recognition theorem.
We define two hyperparameter posteriors, one based on an ELPPD-utility and one aiming to cover the pseudo-true parameter... ρ(s|y(J,m); x) ∝ ρ(s) ∏ ps(y(j)|x)
-
IndisputableMonolith/Foundation/ArithmeticFromLogic.leanabsolute_floor_iff_bare_distinguishability unclear?
unclearRelation between the paper passage and the cited Recognition theorem.
The posterior for the hyperparameters s = (η, β) in (6) may concentrate as we gather more calibration data y(J,m)
What do these tags mean?
- matches
- The paper's claim is directly supported by a theorem in the formal canon.
- supports
- The theorem supports part of the paper's argument, but the paper may add assumptions or extra steps.
- extends
- The paper goes beyond the formal theorem; the theorem is a base layer rather than the whole result.
- uses
- The paper appears to rely on the theorem as machinery.
- contradicts
- The paper's claim conflicts with a theorem or certificate in the canon.
- unclear
- Pith found a possible connection, but the passage is too broad, indirect, or ambiguous to say the theorem truly supports the claim.
Forward citations
Cited by 2 Pith papers
-
When can a posterior predictive check identify the learning rate? Exact degeneracy in Gaussian models and implications for Generalised Bayesian Inerence
Exact finite-sample analysis reveals that the PPC-based selector for the learning rate η is η-invariant with known variance and flat prior, and data-independent with unknown variance and reference prior, collapsing to...
-
Symmetrized Sinkhorn-Gibbs Inference for Oscillatory Inverse Problems
A symmetrized Sinkhorn divergence, averaging transport costs of a signal and its negation, yields better Gibbs posterior inference for oscillatory inverse problems.
Reviewed May 22, 2026 · model on record in the stance chip above.
Discussion (0). Sign in to comment.