REVIEW 4 major objections 5 minor 1 cited by
Can Generative AI Solve Your In-Context Learning Problem? A Martingale Perspective
T0 review · 4 major / 5 minor · reviewed 2026-08-11 · deepseek-v4-flash
Pith's one-line read A p-value computed from a model's own generations can predict whether it can solve an in-context learning task.
desk verdict Useful practical estimator with a clean Doob core, but the exchangeability assumption is untested for LLMs and the finite-N error is unquantified. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The load-bearing object is Doob's theorem for estimators, applied through a martingale argument. It says that under identifiability, the posterior expectation of a function $h(F)$ converges almost surely to $h(F)$ as observations accumulate; this lets the discrepancy $g_\theta(x, f)$, which depends on an unseen explanation $f$, be replaced by $g_\theta(x, x_\infty)$, which depends only on an infinite dataset and hence on the CGM's predictive distribution. The generative predictive p-value then replaces $x_\infty$ by a finite completion $x_N$, sampled ancestrally from $p_\theta(z, y \mid x_n)$, so that the discrepancy can be evaluated from generated completions and response log probabilities.
What would settle it
Construct an ICL problem whose context order is known to change the answer distribution (violating exchangeability), fit an explicit Bayesian model to the same data, and compare the generative predictive p-value to the exact posterior predictive p-value; systematic divergence as $N$ grows would show the paper's equality does not hold for that CGM.
Extended reading notes
Core claim
On the paper's own terms, the central discovery is Theorem 1: under standard regularity conditions (complete and separable spaces, a measurable family of likelihoods, identifiability, and an integrability condition), the posterior predictive p-value $p_{\mathrm{ppc}}$ equals the martingale predictive p-value $p_{\mathrm{mpc}}$, so posterior predictive checks can be carried out using only the predictive distribution $p_\theta(x \mid x_n)$. Since infinite completions are impossible, the paper defines the generative predictive p-value $p_{\mathrm{gpc}}$ (Equation 3), which uses $N$ ancestrally sampled completions of the observed context as stand-ins for latent explanations. The paper argues that for large enough $N$ this approximates the posterior predictive p-value, with estimation error left to future formal analysis, and it demonstrates empirically that the resulting statistic separates in-capability from out-of-capability ICL tasks.
Load-bearing premise
The argument assumes the conditional generative model's predictive distribution is exchangeable, or close enough to exchangeable that de Finetti's theorem supplies a latent explanation and Doob's martingale equality holds; the paper asserts this for ICL problems but does not test exchangeability for pretrained language models.
Editorial extensions
If this is right
- Posterior predictive checks become available for any conditional generative model that can output next-token probabilities, including LLMs, without specifying likelihood or posterior.
- The same p-value, under the negative log-likelihood discrepancy, also signals whether the number of in-context examples is sufficient: lower p-values correlate with higher response error.
- Varying the number of generated completions $N-n$ interpolates the p-value between the NLML and the ideal NLL posterior predictive p-values, so the approximation can be tuned by computational budget.
- Thresholding the generative predictive p-value at a significance level yields a binary predictor of model capability that is better than random on all tested synthetic tabular, LLM, and imaging ICL tasks.
- The procedure needs only generated queries and responses plus their log probabilities, so it applies broadly to current black-box generative models.
Reading between the lines
- Because the equality theorem rests on exchangeability, the method is on firmest ground for models whose predictive distribution is permutation invariant; for pretrained LLMs this is a working approximation, and a task where order materially changes the answer is where the test would be expected to mislead.
- The finite-$N$ gap between $p_{\mathrm{gpc}}$ and $p_{\mathrm{ppc}}$ is left unquantified; a practical diagnostic would be to compute the p-value at several $N$ values and check stability before trusting a low p-value.
- The same statistic could be used for model selection among several CGMs on a fixed task, by comparing their generative predictive p-values rather than thresholding a single model.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper addresses the problem of deciding when a conditional generative model (CGM) can solve an in-context learning (ICL) problem. It assumes a Bayesian interpretation of ICL in which the CGM's predictive distribution pθ(x | xn) is the posterior predictive of some latent-variable model pθ(x, f). The main theoretical result (Theorem 1, proved in Appendix C via Doob's theorem) states that, under measurability and identifiability conditions (Conditions 1-3) and an integrability condition, the posterior predictive p-value pppc (Eq. 1) equals a martingale predictive p-value pmpc (Eq. 2) that depends on an infinite dataset x∞ rather than on the latent explanation f. The paper then introduces the generative predictive p-value pgpc (Eq. 3), which truncates x∞ to a finite completion x_N, gives Algorithm 1 to estimate it, and evaluates it as a capability predictor on tabular, natural-language, and imaging ICL tasks with Llama-2 and Gemma-2. The central practical claim is that pgpc, computed only from CGM predictive samples and log-probabilities, accurately predicts whether the model can solve the ICL problem.
Significance. The elegance of the proposal is that it reduces Bayesian model criticism for black-box CGMs to operations that are available in practice: ancestral sampling from pθ and scoring with log-probabilities. The proof of Theorem 1 via Doob's martingale theorem is a clean extension of the martingale posterior literature (Fong et al., 2023; Lee et al., 2023; Falck et al., 2024), and the paper is honest about the finite-N gap in Section 6.2. If the exchangeability premise were established for the target CGMs, the method would be a useful diagnostic for assessing ICL suitability. However, the current manuscript does not establish that premise, does not quantify the finite-N error, and does not connect pgpc to the formal capability definition in Section B; the empirical evidence, while suggestive, is not yet strong enough to support the broad 'can it solve your ICL problem' claim.
major comments (4)
- [Section 3 and Theorem 1] The equality pppc = pmpc is proved for a Bayesian model (F, X1, X2, ...) with X_i conditionally i.i.d. given F. By de Finetti's theorem, such a representation exists only if the joint distribution of every finite collection of observations is exchangeable. For pretrained LLMs, pθ is not permutation invariant: changing the order of in-context examples changes next-token probabilities, and the joint probability of a set of examples is order-dependent. The paper's assertion in Section 3 that one can treat pθ(x, f) as 'a convenient abuse of notation' when pθ(x) approximates an exchangeable distribution is unsupported with respect to ICL; no evidence is given that transformer LMs have approximately exchangeable predictive distributions in the ICL settings of Section 7. Consequently, the latent explanation f in Condition 3 may not exist for the actual CGM, and the equality pppc = pmpc has no well-defined Bayesian target. This is load-bearing because it is the theoretical justification for interpreting pgpc as a posterior predictive check.
- [Section 6.2, Eq. (3), Algorithm 1] pgpc truncates the infinite dataset x∞ to a finite completion x_N, but no bound or rate is given for |pgpc - pppc|; the paper explicitly states in Section 6.2 that a formal analysis of this error is left to future work. The practical estimator also has Monte Carlo error from M replicates, and in the natural-language experiments N - n = 10 and M = 20. Without an error analysis, the empirical agreement in Figures 8 and 9 cannot be attributed to Theorem 1; the finite-N truncation is particularly delicate because Algorithm 1 generates completions sequentially from pθ(z, y | x_N), and this sequential procedure may not converge to the exchangeable posterior predictive at practical values of N when the model is not exchangeable.
- [Section B and Section 7] The paper defines model capability through Eq. (5) but never establishes a formal relationship between pgpc and this definition. The empirical ground truth in Section 7 is based on the model's own ICL error rate (Figure 4), so the reported 'accurate predictor' results measure correlation between two model-derived quantities. A low posterior predictive p-value indicates that the holdout data are surprising under the model's predictive distribution; it does not by itself imply that the model fails Definition 1. To support the central claim, the authors should either prove (or precisely state conditions under which) small pgpc implies low capability, or validate against an independent, non-model-derived capability measure with appropriate baselines.
- [Section 7.1, Figures 8 and 9] The natural-language and imaging evaluations contain only two in-capability and two out-of-capability tasks per model, and the reported precision, recall, F1, and accuracy curves are presented without error bars or confidence intervals. With such a small number of tasks, the curves in Figures 8 and 9 may be driven by task-specific effects; no baseline (e.g., random guessing or thresholding a simple model-confidence score) is reported. This weakens the empirical claim that pgpc accurately predicts model capability and should be addressed with more tasks, repeated subsampling, or at least a bootstrap analysis.
minor comments (5)
- [Sections 2-3] The symbol θ is used both for the CGM and for the Bayesian model; using distinct notation (e.g., θ for the CGM and M for the Bayesian model) would reduce confusion.
- [Section 5, Eq. (1)] The notation gθ(x, ·) is overloaded; the text does not specify which discrepancy function (NLML or NLL) is being used in the generic definition of pppc.
- [Appendix D] There is a typo: 'nuber' should be 'number' in the description of Algorithm 2.
- [Figure 2 caption] The caption says 'Transformer models (pink)' but the figure appears to show fitted Bayesian linear and polynomial models; the caption should be made consistent with the figure contents.
- [Appendix H] Gemma-2 9B results are reported only in an appendix; a one- or two-sentence summary in Section 7 would help readers understand the qualitative difference from Llama-2 7B without consulting the appendix.
Circularity Check
No significant circularity: Theorem 1 is an external Doob-martingale result, and the empirical p-value is benchmarked against external capability labels.
full rationale
The paper's central claim is Theorem 1, which states that the posterior predictive p-value pppc equals the martingale predictive p-value pmpc under Conditions 1-3. The proof in Appendix C invokes Doob's theorem for estimators, with the detailed proof attributed to Miller (2018), an external source; pmpc is not defined to equal pppc but rather replaces the latent explanation f with an infinite dataset x-infinity, so the equality is a nontrivial martingale result rather than a definitional identity. The finite-N estimator pgpc (Eq. 3) truncates x-infinity, and Section 6.2 explicitly states that a formal analysis of the resulting error is left to future work; this is an admitted limitation, not a circular step. Empirically, the capability predictor is evaluated against external labels: SST2 and AG News are treated as in-capability tasks, MQP and RTE as out-of-capability tasks, and SVHN/MNIST/CIFAR-10 as in/near/far OOD data, so the claimed predictive accuracy is not manufactured by fitting the p-value to the outcome it predicts. The exchangeability assumption for LLMs is asserted in Section 3 without direct verification ('if p-theta(x) approximates an exchangeable distribution p(x), as is the case with ICL problems'), which is a substantive correctness risk, but it is an unverified premise rather than a reduction of the result to its own inputs. Self-citations to Jesson et al. (2024) supply a definition ((1-epsilon)-likely set), a hallucination-rate estimator, and an experimental backbone model, but none of these is the load-bearing justification for Theorem 1 or for the benchmark separation; the theorem's proof and the external task labels carry those claims. Accordingly, no circular step can be exhibited, and the appropriate finding is no significant circularity.
Assumptions & free parameters
free parameters (3)
- M =
M=40 (tabular), 20 (natural language), 100 (imaging)
- N-n =
200 (tabular), 10 (natural language), 8 (imaging)
- alpha =
evaluated over {0.01, 0.05, 0.1, 0.2, 0.5}
assumptions (4)
- domain assumption The CGM predictive distribution is exchangeable or approximates an exchangeable distribution, so de Finetti's theorem guarantees a latent explanation f.
- standard math Doob's theorem conditions hold: complete separable metric spaces, a measurable family of probability measures, and identifiability.
- standard math Observations are conditionally independent given the explanation f, and the log-likelihood has finite prior expectation.
- domain assumption ICL examples are i.i.d. samples from the reference task distribution p(z, y | f*).
Cite this review
Pith. "Pith review of Can Generative AI Solve Your In-Context Learning Problem? A Martingale Perspective." pith.science (2026). https://pith.science/paper/XW6NOIER
@misc{pith2026241206033,
author = {Pith},
title = {Pith review of: Can Generative AI Solve Your In-Context Learning Problem? A Martingale Perspective},
year = {2026},
howpublished = {\url{https://pith.science/paper/XW6NOIER}},
note = {Machine review of arXiv:2412.06033}
}
abstract
This work is about estimating when a conditional generative model (CGM) can solve an in-context learning (ICL) problem. An in-context learning (ICL) problem comprises a CGM, a dataset, and a prediction task. The CGM could be a multi-modal foundation model; the dataset, a collection of patient histories, test results, and recorded diagnoses; and the prediction task to communicate a diagnosis to a new patient. A Bayesian interpretation of ICL assumes that the CGM computes a posterior predictive distribution over an unknown Bayesian model defining a joint distribution over latent explanations and observable data. From this perspective, Bayesian model criticism is a reasonable approach to assess the suitability of a given CGM for an ICL problem. However, such approaches -- like posterior predictive checks (PPCs) -- often assume that we can sample from the likelihood and posterior defined by the Bayesian model, which are not explicitly given for contemporary CGMs. To address this, we show when ancestral sampling from the predictive distribution of a CGM is equivalent to sampling datasets from the posterior predictive of the assumed Bayesian model. Then we develop the generative predictive $p$-value, which enables PPCs and their cousins for contemporary CGMs. The generative predictive $p$-value can then be used in a statistical decision procedure to determine when the model is appropriate for an ICL problem. Our method only requires generating queries and responses from a CGM and evaluating its response log probability. We empirically evaluate our method on synthetic tabular, imaging, and natural language ICL tasks using large language models.
Figures
Figures from the paper (17 more)
Forward citations
Cited by 1 Pith paper
-
LLMs are Bayesian, In Expectation, Not in Realization
Transformers can be Bayes-competitive in prequential log loss even when their predictive distributions are not invariant to example order, provided the cumulative predictive KL to the Bayesian reference stays small.
Reference graph
Works this paper leans on
-
[1]
∼.” It means “sampled according to
An ICL problem comprises a model θ, a dataset xn = {zi, yi}n i=1 ∼ p(xn | f ∗), and a task f ∗. Assume that valid responses y to user queries z ∼ p(z | f ∗) are distributed as p(y | z, f ∗) under the task. Finally, let A(z, f ∗) denote any set of responses satisfying P r(Y ∈ A(z, f ∗) | z, f ∗) ≥ 1 − ϵ. (4) The model θ is called capable of solving the ICL...
work page 2024
-
[2]
Doob’s Theorem for estimators. Let F ∼ Pθ and X1, X2, . . .i.i.d ∼ Pf θ. Assume Conditions 1 to 3 and a measurable function h : F →R such that R |h(f)|dPθ(f) < ∞, then lim n→∞ E[h(F) | X1, X2, . . . ,Xn] = h(F) a.s. [Mθ]. (7) Proof. Miller (2018) provides a detailed proof of this theorem. Lemma
work page 2018
-
[4]
A survey on in-context learning
Qingxiu Dong, Lei Li, Damai Dai, Ce Zheng, Zhiyong Wu, Baobao Chang, Xu Sun, Jingjing Xu, and Zhifang Sui. A survey on in-context learning. arXiv preprint arXiv:2301.00234,
-
[6]
Is in-context learning in large language models bayesian? a martingale perspective
Fabian Falck, Ziyu Wang, and Chris Holmes. Is in-context learning in large language models bayesian? a martingale perspective. arXiv preprint arXiv:2406.00793,
-
[7]
Trapping LLM hallucinations using tagged context prompts
Philip Feldman, James R Foulds, and Shimei Pan. Trapping LLM hallucinations using tagged context prompts. arXiv:2306.06085,
-
[8]
Language models (mostly) know what they know
Saurav Kadavath, Tom Conerly, Amanda Askell, Tom Henighan, Dawn Drain, Ethan Perez, Nicholas Schiefer, Zac Hatfield Dodds, Nova DasSarma, Eli Tran-Johnson, et al. Language models (mostly) know what they know. arXiv:2207.05221,
-
[10]
A detailed treatment of doob’s theorem
Jeffrey W Miller. A detailed treatment of doob’s theorem. arXiv preprint arXiv:1801.03122,
-
[12]
Baolin Peng, Michel Galley, Pengcheng He, Hao Cheng, Yujia Xie, Yu Hu, Qiuyuan Huang, Lars Liden, Zhou Yu, Weizhu Chen, et al. Check your facts and try again: Improving large language models with external knowledge and automated feedback. arXiv:2302.12813,
Show all 20 references
-
[13]
Steering Llama 2 via contrastive activation addition
Nina Rimsky, Nick Gabrieli, Julian Schulz, Meg Tong, Evan Hubinger, and Alexander Matt Turner. Steering Llama 2 via contrastive activation addition. arXiv:2312.06681,
-
[15]
Llama 2: Open foundation and fine-tuned chat models
Hugo Touvron, Louis Martin, Kevin Stone, Peter Albert, Amjad Almahairi, Yasmine Babaei, Nikolay Bashlykov, Soumya Batra, Prajjwal Bhargava, Shruti Bhosale, et al. Llama 2: Open foundation and fine-tuned chat models. arXiv:2307.09288,
-
[16]
A stitch in time saves nine: Detecting and mitigating hallucinations of LLMs by validating low-confidence generation
Neeraj Varshney, Wenlin Yao, Hongming Zhang, Jianshu Chen, and Dong Yu. A stitch in time saves nine: Detecting and mitigating hallucinations of LLMs by validating low-confidence generation. arXiv:2307.03987,
-
[17]
Pre-training and in-context learning is bayesian inference a la de finetti
Naimeng Ye, Hanming Yang, Andrew Siah, and Hongseok Namkoong. Pre-training and in-context learning is bayesian inference a la de finetti. arXiv preprint arXiv:2408.03307,
-
[18]
Mitigating language model hallucination with interactive question-knowledge alignment
Shuo Zhang, Liangming Pan, Junzhou Zhao, and William Yang Wang. Mitigating language model hallucination with interactive question-knowledge alignment. arXiv:2305.13669,
-
[1984]
Trusting your evidence: Hallucinate less with context-aware decoding
Weijia Shi, Xiaochuang Han, Mike Lewis, Yulia Tsvetkov, Luke Zettlemoyer, and Wen-tau Yih. Trusting your evidence: Hallucinate less with context-aware decoding. In Proceedings of the 2024 Conference of the North American Chapter of the Association for Computational Linguistics...
2024
-
[2006]
ISBN 978-3-540-33428-6
Springer Berlin Heidelberg. ISBN 978-3-540-33428-6. Shehzaad Dhuliawala, Mojtaba Komeili, Jing Xu, Roberta Raileanu, Xian Li, Asli Celikyilmaz, and Jason E Weston. Chain-of-verification reduces hallucination in large language models. In ICLR 2024 Workshop on Reliable and Respo...
2024
-
[2018]
Holdout predictive checks for [b]ayesian model criticism
Gemma E Moran, David M Blei, and Rajesh Ranganath. Holdout predictive checks for [b]ayesian model criticism. In 2023 IMS International Conference on Statistics and Data Science (ICSDS), pp. 94,
2023
-
[2021]
Halo: Estimation and reduction of hallucinations in open-source weak large language models
Mohamed Elaraby, Mengyin Lu, Jacob Dunn, Xueying Zhang, Yu Wang, and Shizhu Liu. Halo: Estimation and reduction of hallucinations in open-source weak large language models. arXiv:2308.11764,
-
[2022]
Semantic entropy probes: Robust and cheap hallucination detection in llms
Jannik Kossen, Jiatong Han, Muhammed Razzak, Lisa Schut, Shreshth Malik, and Yarin Gal. Semantic entropy probes: Robust and cheap hallucination detection in llms. arXiv preprint arXiv:2406.15927,
-
[2023]
Linguistic calibration of language models
Neil Band, Xuechen Li, Tengyu Ma, and Tatsunori Hashimoto. Linguistic calibration of language models. arXiv:2404.00474,
-
[2024]
Language models are few-shot learners
Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared D Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, et al. Language models are few-shot learners. Advances in neural information processing systems, 33:1877–1901,
1901
Reviewed August 11, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.