Pith. sign in

REVIEW 3 major objections 6 minor 21 references

A 2.8M-parameter transformer matches the Bayesian posterior over hypothesis classes to within 0.01 bits—even with opaque per-episode symbols—showing in-context learning genuinely performs model selection, not merely filtering.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review

2026-08-02 09:13 UTC pith:52ER2NM3

load-bearing objection Solid, honest extension of BWT to model selection; the closed-form math is right, but the fixed reciprocal-position evaluation leaves a positional shortcut open—the authors know this, and the paper needs a randomized-interrogation control before the strong claim holds. the 3 major comments →

arxiv 2607.19379 v1 pith:52ER2NM3 submitted 2026-07-01 cs.LG stat.ML

Bayesian Wind Tunnels for Model Selection

Siddhartha R Dalal , Vishal Misra , Abhay Parekh This is my paper
classification cs.LG stat.ML
keywords Bayesian model selectionin-context learningBayesian wind tunnelfixed-point-free involutionsOccam's razorperceptual access conditiontransformer calibrationcircuit complexity
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

This paper asks whether transformers can perform Bayesian model selection—choosing the right hypothesis class from data—rather than only Bayesian filtering within a known class. To test this, it constructs 'model-selection wind tunnels' in which the ground-truth posterior over hypothesis classes is available in closed form. The central result: a 2.8M-parameter transformer matches the Bayesian optimum to 0.01 bits of predictive entropy on involution-versus-bijection and non-nested involution-versus-3-cycle tasks, even when symbol tokens are remapped at the start of every episode. The capability, however, has a sharp boundary: it succeeds when the class-defining property is purely relational (f(f(x))=x), fails with opaque symbols when the property requires arithmetic, and fails entirely—even with integer tokens—when the arithmetic needs modular division or polynomial interpolation. If correct, this shows in-context learning implements Occam's razor with calibrated uncertainty, while exposing a circuit-depth limit on what gradient descent can compile.

Core claim

The paper's central claim is that a small transformer, trained only on next-token prediction over sequences drawn from a two-level generative process (first sample a hypothesis class, then sample a function from it), develops an implicit posterior over classes that matches the exact Bayesian posterior with bit-level precision. The cleanest demonstration uses fixed-point-free involutions on Z_16: the defining property f(f(x))=x is purely relational, so the model succeeds with both integer tokens and opaque symbols whose meanings change every episode (0.01-bit entropy MAE, class-posterior MAE 0.003–0.004). The same holds in a non-nested comparison—involutions versus 3-cycles on Z_12, disjoint

What carries the argument

The load-bearing machinery is the model-selection Bayesian wind tunnel: a generative process with a closed-form posterior P(M|D) over hypothesis classes, together with its entropy and predictive distributions. The paper's key conceptual device is the perceptual access condition—the requirement that a hypothesis class's defining property be evaluable from token identities alone, without arithmetic. The concrete testbed is the fixed-point-free involution class (f(f(x))=x, no fixed points), whose defining property is purely relational and hence label-invariant; the implicit class posterior is extracted at reciprocal positions where the involution makes a deterministic prediction, allowing a cle

Load-bearing premise

The evaluation assumes that the hand-designed input ordering—interleaving fresh inputs with reciprocal tests at predictable positions—is a fair test of model selection rather than a positional shortcut, since class posteriors are only extracted at these predictable reciprocal positions.

What would settle it

Randomize the interrogation schedule so reciprocal tests (presenting a previous output as the next input) appear at unpredictable positions, or extract the implicit class posterior at fresh-input positions using the full mixture formula, and check whether the 0.01-bit entropy agreement and class-posterior MAE under 0.001 survive. If performance collapses to the bijection-elimination baseline, the model was exploiting positional regularity rather than performing Bayesian model selection. The same test would falsify the paper's central claim if the agreement persists under randomization.

Watch this falsifier. Get emailed when new claim-graph text bears on it.

Share X Bluesky LinkedIn Reddit HN

If this is right

  • Gradient-trained transformers can implement the full Bayesian pipeline—model selection across hypothesis classes followed by within-class filtering—at bit-level precision, not merely Bayesian filtering within a known class.
  • Model selection generalizes to non-nested, disjoint hypothesis classes (involutions vs. 3-cycles), so the behavior cannot be explained by subset elimination or a simplicity prior alone.
  • The perceptual access condition gives a predictive rule of thumb for when a hypothesis class will be learnable: relational properties with any stable token encoding, shallow arithmetic only with stable integer semantics, and deeper arithmetic (division, interpolation) not at all.
  • The failure boundary is robust to 112x scaling and extended training, indicating it is a structural property of gradient-compiled inference rather than a capacity or optimization artifact.
  • Frontier language models show the qualitative Bayesian signatures (entropy decreasing with evidence, correct direction of class posteriors) but with a ~55x calibration gap that purpose-training on the exact task distribution eliminates.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • A testable extension the paper leaves open: randomizing the positions of reciprocal tests within each episode would determine whether the 0.01-bit agreement reflects genuine posterior tracking or an exploitable positional regularity; the paper itself flags this as the main protocol limitation.
  • The polynomial barrier suggests a conjectural analogue of formal circuit complexity: the learnability of a discriminative statistic may be governed by the minimum depth of its verification circuit (depth-1 addition learnable, depth-2 division/inversion not), which could be tested by architectures designed to implement shallow arithmetic more directly.
  • The stationarity control implies the bottleneck is dynamic semantic rebinding, not label identity. A possible remedy worth testing: meta-learning across a small set of fixed relabelings, or adding an auxiliary loss that predicts integer identities from opaque tokens, might unlock opaque-symbol arithmetic without architectural changes.
  • The ~55x LLM calibration gap, measured with lossy top-5 logprob and low-sample probes, is likely a lower bound; obtaining full-vocabulary log probabilities could reveal systematic over- or under-confidence in frontier models' implicit class posteriors.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

3 major / 6 minor

Summary. The paper extends Bayesian wind tunnels from within-class filtering to between-class model selection. It derives closed-form posteriors for several hypothesis-class comparisons on Z_n (rotations vs. bijections, involutions vs. bijections, involutions vs. 3-cycles, polynomials vs. bijections) and trains a 2.8M-parameter transformer on these generative processes. The central claims are: (i) the model achieves 0.01-bit entropy agreement with the Bayesian optimum for involution-vs-bijection model selection, even with opaque symbols; (ii) non-nested involution-vs-3-cycle selection achieves class-posterior MAE under 0.001; (iii) model selection over arithmetic structures (rotations, scalar multiplication) succeeds with integer tokens but fails with opaque symbols, a boundary that persists under 112x scaling; and (iv) polynomial detection fails for all degrees tested, including affine. The paper also probes frontier LLMs and finds qualitative Bayesian behavior with a large calibration gap. The mathematical derivations in Sections 3 and 5 are correct and clearly presented. The empirical protocols include multiple controls (scaling, stationarity, header ablation, extended training, falsification), and the negative results are honestly reported. However, the evaluation protocol for the involution and non-nested experiments relies on a fixed input ordering with reciprocal tests at predictable positions, which leaves open a positional-shortcut explanation for the headline results.

Significance. If the central claim holds, this would be an important extension of the BWT methodology: a quantitative testbed for Bayesian model selection, not just filtering, and a clean dissociation between relational and arithmetic structure in what gradient-compiled inference can implement. The closed-form posterior derivations (Propositions 1-5) are a useful resource, and the paper is genuinely reproducible in spirit, with on-the-fly data generation and a planned release of code. The negative results (opaque rotations, polynomial barrier, scaling wall) are valuable empirical boundary conditions. However, the significance of the paper hinges on whether the model's apparent model-selection behavior is genuine or an artifact of the predictable reciprocal-test schedule. The paper itself acknowledges this gap in the limitations section. With an additional randomized-schedule control or evaluation at arbitrary positions, the contribution would be solid; without it, the headline claims overstate what the current measurements establish.

major comments (3)
  1. [Section 5.6 and Section 6.3] The fixed input ordering ("after every 2 fresh inputs, we present a previous output") and the exclusive evaluation "at all reciprocal positions per sequence" permit a positional retrieval policy to reproduce the reported entropy MAE and extracted class posteriors without performing Bayesian model selection. At a reciprocal position, the Bayesian posterior P(Inv|D_k) is a deterministic function of k and of whether the reciprocal match holds, and the correct output is always the previously paired input. A model can learn a position-indexed confidence schedule plus a local match check, exactly matching the Bayesian predictive at those positions. This undermines the headline claim of 0.01-bit agreement and the non-nested MAE under 0.001. The paper's own limitation statement ("randomizing the interrogation schedule would rule out any positional heuristic") confirms that the current design doe
  2. [Section 5.6, Remark 6, Eq. (16)] The class-posterior extraction is restricted to reciprocal positions "where the mixture inversion is clean," and Eq. (16) assumes the model's predictive distribution has the exact Bayesian mixture form p_model = p_class * 1_{reciprocal} + (1-p_class)/(n-k) over the unused outputs. This assumption is not verified. If the model allocates probability mass differently at those positions (e.g., to tokens that are not unused outputs), the extracted class posterior can be biased regardless of the true class-tracking behavior. Please validate the mixture form on held-out data (e.g., by comparing the full predicted distribution to the Bayesian form) or extend the extraction to arbitrary positions using a less assumption-dependent method.
  3. [Section 6.3, Table 1] The non-nested involution-vs-3-cycle result reports class-posterior MAE under 0.001, but this is measured at reciprocal positions where a single observation deterministically falsifies one class (Section 6.2). A binary classifier on the reciprocal match would achieve near-zero MAE. The claim that the model "maintain[s] two competing structural hypotheses simultaneously" (Section 6.3) is therefore not supported by the presented metrics. Please provide evidence that the model tracks both classes at fresh positions or in a protocol where the two classes are not one-shot distinguishable, or soften the interpretation accordingly.
minor comments (6)
  1. [Section 4, Evaluation paragraph] The text refers to "the Bayesian optimum from Theorem 3," but the result is Proposition 3 (as in the proposition numbering). Please correct the cross-reference.
  2. [Section 5.6, first paragraph] The sentence "after every 2 fresh inputs (x_i, f(x_i)), we present a previous output y_j" is ambiguous: it could be read as two fresh pairs followed by a reciprocal query. Consider rephrasing to "after every two fresh input-output pairs, the next query is a previous output."
  3. [Section 7.2] The phrase "standard (random-input) protocol" is not defined earlier. Please specify the input ordering used in the polynomial experiments, since the rotation and involution experiments use different orderings.
  4. [Section 8, 'The dual-entropy ratio' paragraph] This paragraph introduces the dual-entropy framework and attributes it to Papers I and II, but the connection to the present paper's metrics is not fully spelled out. The notation (H_I, H_P, rho) appears only here; consider either integrating it more explicitly or moving it to related work.
  5. [Section 6.2] The condition "(and f(y_i) \neq y_i)" is redundant because both the involution and the 3-cycle classes are fixed-point-free; f(y_i)=y_i is impossible under either class. Minor, but the text could be simplified.
  6. [Section 5.3, Eq. (22)] The Bayes factor for a new-element observation is given as (n-j+1)/(r-1). The derivation would be clearer if it noted that the factor r-1 counts the possible outputs under the involution class (free elements other than x_j) and n-j+1 counts unused outputs under the bijection class.

Circularity Check

0 steps flagged

No load-bearing circularity; closed-form benchmarks are independent of the model, and the only caveats are acknowledged evaluation limits.

full rationale

The Bayesian benchmarks (Propositions 1-5) are closed-form likelihood computations under specified priors and do not use any transformer output; the model is trained only with next-token cross-entropy on sequences drawn from the same generative process. No parameter is fitted to the benchmark, and the reported entropy/class-posterior MAE is a comparison against an independent closed-form target. The main caveats—reciprocal-only posterior extraction and the fixed interleaved input schedule—are explicitly acknowledged in Section 5.6 and the Limitations; they concern whether a positional shortcut could mimic the numbers, not whether the derivation reduces to its inputs. The self-citations to [1] and [3] appear in the Discussion (dual-entropy ratio) as interpretive machinery and are not load-bearing for the core wind-tunnel result. Accordingly, no circular step is identified; score 2 reflects only minor non-load-bearing self-citation and the noted evaluation limitation, not derivation-level circularity.

Axiom & Free-Parameter Ledger

0 free parameters · 4 axioms · 0 invented entities

No physical free parameters or invented entities appear. The load-bearing assumptions are standard Bayesian calculation, the specific prior/architecture choices, and the evaluation protocol's designed reciprocal positions. The central claim does not depend on fitted constants, but does depend on the fairness of the reciprocal-position measurement.

axioms (4)
  • standard math Bayes' rule with uniform priors over functions within each class yields the closed-form posteriors and predictive distributions.
    Used throughout Sections 3.3-3.5 and 5.2 to define the Bayesian optimum against which models are measured.
  • domain assumption The data-generating process uses class prior π=0.5 and uniform within-class function priors.
    Section 3.2 fixes these priors for both training and evaluation; all reported benchmarks assume them, so generalization to extreme priors is untested.
  • ad hoc to paper The designed input ordering interleaving fresh inputs with reciprocal tests is part of the generative distribution.
    Section 5.6 states reciprocal tests are interleaved at predictable positions; this is required to make the posterior extraction clean, but it may allow positional heuristics.
  • ad hoc to paper The implicit class posterior extraction assumes the model's predictive distribution has the exact Bayesian mixture form (Eq. 16 for rotations, analogous inversion for involutions).
    Used to convert a single predictive probability into a class posterior; if the model's distribution deviates from this form, the extracted quantity is not a true posterior.

reviewed 2026-08-02 · how reviews work

0 comments
Cite this review

Pith. "Pith review of Bayesian Wind Tunnels for Model Selection." pith.science (2026). https://pith.science/paper/52ER2NM3

@misc{pith2026260719379,
  author       = {Pith},
  title        = {Pith review of: Bayesian Wind Tunnels for Model Selection},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/52ER2NM3}},
  note         = {Machine review of arXiv:2607.19379}
}
Share X Bluesky LinkedIn Reddit HN
read the original abstract

Prior work has shown that transformers can perform exact Bayesian filtering within a fixed hypothesis class. Can they also perform Bayesian model selection -- identifying the correct hypothesis class from data? We introduce model-selection Bayesian wind tunnels: controlled environments where ground-truth posteriors over hypothesis classes are available in closed form. Using fixed-point-free involutions -- whose defining property f(f(x))=x is purely relational -- a 2.8M-parameter transformer achieves 0.01-bit entropy agreement with the Bayesian optimum (3 seeds), with both integer tokens and opaque symbols whose meanings change every episode. This extends to non-nested comparisons: involutions vs. 3-cycles (where neither class is a subset of the other) achieve class-posterior MAE under 0.001, demonstrating genuine model selection beyond simplicity/subset bias. We then identify a sharp perceptual access condition: when the discriminative statistic requires arithmetic -- modular addition (rotations) or multiplication (f(x)=cx mod p) -- model selection succeeds with integer tokens but fails completely with opaque symbols, and this boundary persists under 112x scaling (2.8M to 316M parameters). A stationarity control confirms the operative factor: opaque tokens with a fixed relabeling succeed (0.009-bit MAE), showing that stable semantics, not integer identity, enable circuit compilation. Header subtask diagnostics localize the failure to the composition of header inversion with arithmetic rather than header parsing itself. Probing frontier LLMs on the same tasks shows qualitative Bayesian behavior but a large calibration gap (~55x), measured through lossy probes and therefore directional rather than exact.

Figures

Figures reproduced from arXiv: 2607.19379 by Abhay Parekh, Siddhartha R Dalal, Vishal Misra.

Figure 1
Figure 1. Figure 1: The scaling boundary. Entropy MAE vs. parameter count for integer tokens (blue) and opaque symbols [PITH_FULL_IMAGE:figures/full_fig_p009_1.png] view at source ↗
Figure 2
Figure 2. Figure 2: Attention to header positions in the 25M-parameter opaque-trained model, averaged over 500 episodes. [PITH_FULL_IMAGE:figures/full_fig_p010_2.png] view at source ↗
Figure 3
Figure 3. Figure 3: Entropy MAE vs. group size 𝑛 for opaque rotation model selection (3 seeds, mean ± std). The boundary persists across all group sizes tested. MAE rises with 𝑛 but remains below the uniform baseline (log2 𝑛, dashed), indicating the model learns bijection elimination but not model selection. 5.2 Posterior Computation The posterior computation requires tracking the set of constraints imposed by observations on… view at source ↗
Figure 4
Figure 4. Figure 4: Training loss curves for the involution BWT ( [PITH_FULL_IMAGE:figures/full_fig_p014_4.png] view at source ↗
Figure 5
Figure 5. Figure 5: Per-position predictive entropy for the involution BWT ( [PITH_FULL_IMAGE:figures/full_fig_p015_5.png] view at source ↗
Figure 6
Figure 6. Figure 6: Implicit class posterior 𝑃 (Inv | D𝑘 ) over positions (mean ± std over 3 seeds). Both models implement calibrated Occam’s razor: the posterior rises from ≈ 0.5 at 𝑘 = 1 toward 1.0 as reciprocal confirmations accumulate, tracking the Bayesian optimal posterior (dashed) with MAE < 0.004. 6.1 3-Cycle Permutations Definition 5 (3-Cycle Class). A fixed-point-free 3-cycle permutation on Z𝑛 (with 𝑛 divisible by 3… view at source ↗
Figure 7
Figure 7. Figure 7: Per-position entropy tracking for the rotation model-selection task with integer tokens at two scales. [PITH_FULL_IMAGE:figures/full_fig_p017_7.png] view at source ↗
Figure 8
Figure 8. Figure 8: Per-position predictive entropy for the non-nested BWT (involutions vs. 3-cycles, [PITH_FULL_IMAGE:figures/full_fig_p018_8.png] view at source ↗
Figure 9
Figure 9. Figure 9: Implicit class posterior 𝑃 (Inv | D𝑘 ) at reciprocal positions for the non-nested BWT (mean ± std over 3 seeds). Both models track the Bayesian optimal posterior (dashed) with MAE < 0.001, exploiting the one-shot discrimination property: a single reciprocal observation deterministically distinguishes involutions from 3-cycles. Integer tokens Opaque symbols Arithmetic: addition (rotations) 0.12 bits 1.28 bi… view at source ↗
Figure 10
Figure 10. Figure 10: Predictive entropy vs. position for the involution BWT ( [PITH_FULL_IMAGE:figures/full_fig_p021_10.png] view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

21 extracted references · 10 linked inside Pith

  1. [1]

    Dalal, and Vishal Misra

    Naman Agarwal, Siddhartha R. Dalal, and Vishal Misra. 2025. The Bayesian Geometry of Transformer Attention. arXiv:2512.22471 [cs.LG] https://arxiv.org/abs/2512.22471 Paper I of the Bayesian Attention Trilogy

    Pith/arXiv arXiv 2025

  2. [2]

    Dalal, and Vishal Misra

    Naman Agarwal, Siddhartha R. Dalal, and Vishal Misra. 2025. Geometric Scaling of Bayesian Inference in LLMs. arXiv:2512.23752 [cs.CL] https://arxiv.org/abs/2512.23752 Paper III of the Bayesian Attention Trilogy

    Pith/arXiv arXiv 2025

  3. [3]

    Dalal, and Vishal Misra

    Naman Agarwal, Siddhartha R. Dalal, and Vishal Misra. 2025. Gradient Dynamics of Attention: How Cross-Entropy Sculpts Bayesian Manifolds. arXiv:2512.22473 [cs.LG] https://arxiv.org/abs/2512.22473 Paper II of the Bayesian Attention Trilogy

    Pith/arXiv arXiv 2025

  4. [4]

    Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared D

    Tom B. Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared D. Kaplan, Prafulla Dhariwal, et al. 2020. Language Models are Few-Shot Learners.Advances in Neural Information Processing Systems33 (2020), 1877–1901

    2020

  5. [5]

    Bilal Chughtai, Lawrence Chan, and Neel Nanda. 2023. A Toy Model of Universality: Reverse Engineering How Networks Learn Group Operations. InInternational Conference on Machine Learning (ICML)

    2023

  6. [6]

    Liang, and Gregory Valiant

    Shivam Garg, Dimitris Tsipras, Percy S. Liang, and Gregory Valiant. 2022. What Can Transformers Learn In-Context? A Case Study of Simple Function Classes. InAdvances in Neural Information Processing Systems, Vol. 35. 29881–29895

    2022

  7. [7]

    Kass and Adrian E

    Robert E. Kass and Adrian E. Raftery. 1995. Bayes Factors.J. Amer. Statist. Assoc.90, 430 (1995), 773–795

    1995

  8. [8]

    Ash, Surbhi Goel, Akshay Krishnamurthy, and Cyril Zhang

    Bingbin Liu, Jordan T. Ash, Surbhi Goel, Akshay Krishnamurthy, and Cyril Zhang. 2023. Transformers Learn Shortcuts to Automata. InInternational Conference on Learning Representations (ICLR)

    2023

  9. [9]

    David J. C. MacKay. 1992. A Practical Bayesian Framework for Backpropagation Networks.Neural Computation4, 3 (1992), 448–472

    1992

  10. [10]

    William Merrill and Ashish Sabharwal. 2023. The Parallelism Tradeoff: Limitations of Log-Precision Transformers. Transactions of the Association for Computational Linguistics11 (2023), 531–545

    2023

  11. [11]

    Samuel Müller, Noah Hollmann, Sebastian Pineda Arango, Josif Grabocka, and Frank Hutter. 2023. Transformers Can Do Bayesian Inference. InInternational Conference on Learning Representations

    2023

  12. [12]

    Neel Nanda, Lawrence Chan, Tom Lieberum, Jess Smith, and Jacob Steinhardt. 2023. Progress Measures for Grokking via Mechanistic Interpretability.arXiv preprint arXiv:2301.05217(2023)

    Pith/arXiv arXiv 2023

  13. [13]

    Catherine Olsson, Nelson Elhage, Neel Nanda, Nicholas Joseph, Nova DasSarma, Tom Henighan, Ben Mann, Amanda Askell, Yuntao Bai, Anna Chen, Tom Conerly, Dawn Drain, Deep Ganguli, Zac Hatfield-Dodds, Danny Hernandez, Scott Johnston, Andy Jones, Jackson Kernion, Liane Lovitt, Kamal Ndousse, Dario Amodei, Tom Brown, Jared Clark, Jared Kaplan, Sam McCandlish, ...

    Pith/arXiv arXiv 2022

  14. [14]

    Madhur Panwar, Kabir Ahuja, and Navin Goyal. 2024. In-Context Learning Through the Bayesian Prism. InInternational Conference on Learning Representations. https://openreview.net/forum?id=HX5ujdsSon Hierarchical meta-ICL setup with unions of function families

    2024

  15. [15]

    Alethea Power, Yuri Burda, Harri Edwards, Igor Babuschkin, and Vedant Misra. 2022. Grokking: Generalization Beyond Overfitting on Small Algorithmic Datasets.arXiv preprint arXiv:2201.02177(2022)

    Pith/arXiv arXiv 2022

  16. [16]

    Arik Reuter, Tim G. J. Rudner, Vincent Fortuin, and David Rügamer. 2025. Can Transformers Learn Full Bayesian Infer- ence in Context?. InInternational Conference on Machine Learning. https://arxiv.org/abs/2501.16825 arXiv:2501.16825

    Pith/arXiv arXiv 2025

  17. [17]

    Johannes von Oswald, Eyvind Niklasson, Ettore Randazzo, João Sacramento, Alexander Mordvintsev, Andrey Zhmogi- nov, and Max Vladymyrov. 2023. Transformers Learn In-Context by Gradient Descent.arXiv preprint arXiv:2212.07677 (2023). Also appeared in ICML 2023

    Pith/arXiv arXiv 2023

  18. [18]

    Yuxin Wang, Botian Jiang, Yiran Guo, Quan Gan, David Wipf, Xuanjing Huang, and Xipeng Qiu. 2025. Prior-Fitted Networks Scale to Larger Datasets When Treated as Weak Learners.arXiv preprint arXiv:2503.01256(2025)

    Pith/arXiv arXiv 2025

  19. [19]

    Gail Weiss, Yoav Goldberg, and Eran Yahav. 2021. Thinking Like Transformers. InInternational Conference on Machine Learning (ICML)

    2021

  20. [20]

    Sang Michael Xie, Aditi Raghunathan, Percy Liang, and Tengyu Ma. 2022. An Explanation of In-Context Learning as Implicit Bayesian Inference. InInternational Conference on Learning Representations

    2022

  21. [21]

    Naimeng Ye and Hongseok Namkoong. 2024. Exchangeable Sequence Models Quantify Uncertainty Over Latent Concepts.arXiv preprint arXiv:2408.03307(2024). , Vol. 1, No. 1, Article . Publication date: July

    Pith/arXiv arXiv 2024

This paper was first reviewed by deepseek-v4-flash on August 2, 2026.