Pith. sign in

REVIEW 3 major objections 5 minor 41 references

The paper establishes that a two-step RAG policy's regret splits into candidate-set loss and within-candidate choice loss, and that the choice loss is controlled by nearest-neighbor prediction error, yielding explicit rates under a margin c

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · deepseek-v4-flash

2026-08-01 15:36 UTC pith:6L5NTIDC

load-bearing objection The RAG-as-nearest-neighbor-matching framing is genuinely new and the paper is candid about its limits, but the headline regret rates rest on an unmeasured \eta_n and the simulations don't isolate the retrieval mechanism. the 3 major comments →

arxiv 2607.18225 v1 pith:6L5NTIDC submitted 2026-07-20 econ.EM cs.LGmath.STstat.MEstat.MLstat.TH

Vector Search As Nearest Neighbor Matching: RAG-based Policy Learning in Causal Inference

Masahiro Kato , Taka Kato This is my paper
classification econ.EM cs.LGmath.STstat.MEstat.MLstat.TH
keywords policy learningretrieval-augmented generationnearest-neighbor matchingregret decompositionpotential outcomesvector searchmargin conditionconditional average treatment effect
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

This paper treats a RAG system that recommends actions as a policy-learning problem under the potential outcome framework. For the two-step variant—generate a finite candidate set, retrieve action-specific evidence, estimate expected outcomes, and pick the largest estimate—it proves that regret decomposes cleanly into the loss from missing good candidates and the loss from mis-ranking the candidates that were generated. It then shows the within-candidate regret is bounded by the prediction error of the outcome estimator, and when retrieval is modeled as action-specific k-nearest-neighbor matching, this yields explicit rates in terms of database size, embedding dimension, smoothness, and a margin condition. The point is to give RAG-based decision-making the same decision-theoretic footing as classical policy learning. A sympathetic reader would care because this is one of the few places where a widely used but largely heuristic architecture gets formal regret guarantees tied to its retrieval step.

Core claim

The central claim is that the action recommended by a two-step RAG policy—generate a finite candidate set g(x) from the query, retrieve action-specific evidence, estimate conditional expected outcomes, and choose the argmax—has regret R(batwo) = Rgen(g) + Rchoice(batwo;g). Theorem 4.2 bounds the within-candidate regret by 2E[max_a |bf(a,X) - f0(a,X)|]; under a margin condition this becomes C(2rf)^(1+kappa). When retrieval is k-nearest-neighbor matching in an embedding space, Theorem 5.1 supplies concentration and mean-squared-error bounds for the matched outcome estimator, yielding regret decaying at rate N^{-beta(1+kappa)/(2beta+d_phi)} in the binary-action case. The one-step method is eval

What carries the argument

The load-bearing objects are the regret decomposition identity, which separates total regret at the point where the candidate set is fixed; the margin condition on the gap between the best and second-best conditional expected outcomes, which converts prediction error into regret at an accelerated rate; and the action-specific k-nearest-neighbor matching estimator ef_k(a,x), defined as the average outcome of the k closest database cases under action a. The bias-variance tradeoff of this estimator, together with smoothness of the outcome function and a lower bound on local mass in the embedding space, supplies the explicit rates. The decomposition itself does not require candidate generation t

Load-bearing premise

The load-bearing premise is that the RAG generator's output stays uniformly close to the nearest-neighbor matched estimate on every action and context, and that the embedding loses no outcome-relevant covariate information; if either fails, the regret rates do not bind actual RAG behavior.

What would settle it

Take a binary-action retrieval database and a fixed query distribution, and measure the uniform gap D_n = max_{a,x} |bf(a,x) - ef_k(a,x)| between the RAG generator and the k-NN matched estimator while comparing actual regret with the bound using D_n. If D_n is large while the k-NN estimator is accurate, or if removing one covariate from the embedding reverses the sign of the conditional treatment effect on a set of positive probability, the claimed N^{-beta(1+kappa)/(2beta+d_phi)} rate fails.

Watch this falsifier — get emailed when new claim-graph text bears on it.

If this is right

  • When the candidate set contains an action within epsilon of the optimum, candidate-set regret is at most epsilon; candidate quality and estimation quality contribute additively to total regret.
  • If the expected-outcome estimator has uniform error r_f, binary regret is O(r_f^(1+kappa)) under the margin condition, so easy problems with few near-boundary contexts learn faster than the raw prediction error suggests.
  • Under k-NN retrieval with smooth outcome functions, choosing k proportional to N^{2beta/(2beta+d_phi)} balances bias and variance and yields the stated regret rate as the database grows.
  • The one-step method's regret is an upper bound on candidate-set regret when its output lies in the candidate set; the two-step method can beat the one-step method only when estimation or ranking within the candidate set is sufficiently accurate.
  • When the policy class is all measurable selectors, pointwise maximization of the estimated expected outcome is equivalent to maximizing the induced policy value over the class.
  • If the RAG generator differs from the k-NN benchmark by at most eta_n, then eta_n enters the regret bound additively, so the rate degrades gracefully as long as the generator tracks the matched estimator.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • Extension: the uniform-error assumption max_{a,x} |bf(a,x) - ef_k(a,x)| <= eta_n is never measured in the paper; a practical test would benchmark a generator against the k-NN estimate on held-out contexts. If eta_n is not small, the published rates should not be expected to bind for a trained RAG system.
  • Extension: the same decomposition suggests a generic recipe: any outcome estimator with a uniform error bound plugs directly into the regret formula, so the paper's contribution is a reduction from RAG policy learning to prediction-error control, not a bound unique to a specific generator.
  • Extension: because the margin condition is defined on the embedding space, the choice of embedding changes both the difficulty and the achievable rate; one could tune the embedding to maximize a proxy for the margin rather than raw retrieval similarity.
  • Extension: in the large-action simulation, candidate-set regret is not negligible, which points toward future work on learned candidate generation with coverage guarantees as a complement to the choice-stage analysis.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. The paper introduces RAG-PL, a decision-theoretic treatment of retrieval-augmented generation as policy learning under the Neyman–Rubin potential outcome model. It distinguishes one-step RAG-PL (policy RAG evaluated directly) from two-step RAG-PL (candidate action generation followed by expected-outcome estimation/ranking and action choice). The main theoretical contribution is the regret decomposition R(ba_two) = R_gen(g) + R_choice(ba_two;g) and bounds on the within-candidate term under margin conditions. For binary actions, Theorem 4.3 converts contrast estimation error into regret, and Theorem 5.1 supplies concentration and MSE bounds for k-nearest-neighbor matching in the embedding space, leading to the rate claims in Eqs. (5)--(6). The paper also discusses substituting transformer in-context learning bounds, with caveats, and reports simulations with GPT-5.4 mini comparing one- and two-step RAG-PL against a RAG baseline without covariates.

Significance. If the results are read as conditional statements, the paper makes a valuable connection between vector search and nearest-neighbor matching and gives a clean, internally correct regret decomposition (Theorems 4.2, 4.3, and 5.1 are sound under their stated assumptions). The paper is also commendably candid about several limitations, especially in Appendix F, where it states that the cited transformer bounds do not apply to retrieval-selected contexts. However, the abstract and conclusion frame the results as 'usable regret guarantees' for RAG. The actual RAG-specific rates in Eqs. (5)--(6) depend on two unverified conditions: the uniform approximation η_n between the RAG output and the k-NN matched estimator, and the embedding sufficiency condition b_{φ,a}=0. Neither is bounded theoretically nor measured in the simulation, so the rates are not established for the deployed RAG policy. If these gaps are addressed or the claims are appropriately qualified, the paper would be a useful contribution to the literature.

major comments (3)
  1. [§5.1, Eqs. (5)--(6)] The rate N^{-β(1+κ)/(2β+d_φ)} is derived from the assumption max_{a∈{0,1}} sup_x |bf(a,x) − ef_k(a,x)| ≤ η_n. Theorem 5.1 controls ef_k − f_0, but not bf − ef_k. Appendix F explicitly states that the transformer in-context learning bounds apply to examples sampled under the pretraining task model, not to retrieval-selected contexts, so they cannot be invoked to control η_n. The simulation section also does not measure η_n, even though the prompt supplies the exact k-NN means, making measurement straightforward. If η_n is of the same order as or larger than the nearest-neighbor rate, Eqs. (5)--(6) are vacuous for the actual RAG policy. The paper is honest about this condition, but the abstract's 'usable regret guarantees' overstates what has been established. Please either bound or evaluate η_n, or explicitly restrict the rate claims to the k-NN matched estimator benchmark and adjust the
  2. [§5.1 and Prop. C.3] The rates also require b_{φ,a}=0, i.e., the embedding retains all outcome-relevant information so that f_0(a,x)=m_a(φ(x)). Proposition C.3 proves that this condition, together with identification, implies the equality; it does not establish that any practical embedding satisfies it. If φ drops a covariate that enters E[Y(a)|X], the additive term b_φ enters Eq. (5) and the claimed rate is not obtained. This is a load-bearing modeling assumption that should be listed explicitly among the hypotheses of the main regret-rate theorem and should be audited in the simulation. As with η_n, the current presentation makes the rate claim appear more unconditional than the theory supports.
  3. [§4.5 and App. F] The contribution statement that regret is bounded 'using prediction-error guarantees for ... transformers' is only partially supported. The substitution in §4.5 uses MSE-type bounds from Kim et al. (2024), Oko et al. (2024), and Ching et al. (2026), but Appendix F states that the sampling model for these bounds (i.i.d. in-context examples under the pretraining task distribution) does not cover RAG's retrieval-selected examples. Unless an additional bridging assumption is introduced and verified, the transformer-based regret bound is not a theorem for RAG. This caveat should appear in the main text at the first point where the transformer substitution is made, and the contribution summary should be revised so that the transformer claim is not overstated.
minor comments (5)
  1. [§3.1] Typo: 'he second returns' should be 'The second returns'.
  2. [§5.2] The sentence 'The density ratio is therefore proportional to an inverse propensity score defined conditional on the embedding. The density ratio is proportional to an inverse propensity score.' repeats the same statement; delete the duplication.
  3. [§4.5] The sentence 'If this bound holds for both binary expected-outcome estimates, Theorem 4.3 below gives ...' is vague about which additional hypotheses of Theorem 4.3 are needed (e.g., boundedness of τ_0 and the margin constants). Spell out the full assumption set when substituting the q_ICL bound.
  4. [§5.1] The k-NN matching analysis is restricted to binary actions, but Section 3.1 discusses finite and continuous action spaces. The paper does note this restriction, but a sentence clarifying that Theorem 5.1 and Eqs. (5)--(6) do not cover the 24-action DGP 3 would prevent misreading of the simulation results as validating the rate theorem.
  5. [Table 1 and §7] The baseline 'RAG without covariates' differs from RAG-PL in both the input information and the retrieval rule. This is acknowledged in the text, but the table caption should state that the comparison is not a controlled ablation.

Circularity Check

0 steps flagged

No significant circularity: the regret bounds are derived from explicit assumptions and standard nonparametric results; the unquantified η_n condition is an applicability caveat, not a definitional reduction.

full rationale

The derivation chain is self-contained once its assumptions are granted. The regret decomposition (1) is an algebraic identity; Theorem 4.2 bounds within-candidate regret by 2Eδf using the defining argmax property; Theorem 4.3 is the standard plug-in classification margin argument. Section 5's Theorem 5.1 is a textbook k-NN bias/variance calculation, and equations (5)-(6) follow by combining it with Theorem 4.3 and the explicit assumption max_{a,x}|bf(a,x)-ef_k(a,x)|≤η_n. That assumption is not fitted from data or renamed as a prediction; the paper explicitly states that for a general document corpus a regret rate 'requires an assumption or an evaluation of the difference represented by η_n', and Appendix F admits the transformer in-context learning bounds do not directly transfer to retrieval-selected contexts. These are honest limitations, not circular steps. The self-citations (Kato 2025a,b) are contextual or auxiliary equivalences whose proofs are included or independent; they do not carry the main regret argument. No fitted parameter is presented as a prediction, and no external result is replaced by an author-forcing uniqueness claim.

Axiom & Free-Parameter Ledger

2 free parameters · 6 axioms · 0 invented entities

The theoretical results inherit standard causal and nonparametric assumptions. The two most fragile are that the embedding retains outcome-relevant information and that the LLM generator's estimates are close to a k-NN estimator; neither is validated in the experiments. The transformer-based bound is explicitly conditional on an unverified sampling model.

free parameters (2)
  • neighborhood size k = 6 (binary DGPs), 5 (DGP3 candidate evaluation)
    User-specified in simulations; the theory balances k ~ N^{2beta/(2beta+d_phi)}, but the simulations fix k without adapting it to the data.
  • margin constants C_M, kappa = unspecified
    Assumed, not estimated; standard in plug-in classification, but the rates depend on their values.
axioms (6)
  • domain assumption Potential outcomes: consistency, conditional exchangeability given X, positivity; E[Y|A=a,X=x]=E[Y(a)|X=x].
    Required for the causal interpretation; stated in §6.1 and Prop. C.1.
  • domain assumption Hölder smoothness and local mass: m_a is beta-Hölder and Q_a(B(h,r)) >= c_a r^{d_phi}.
    Theorem 5.1 relies on these standard kNN conditions.
  • domain assumption Margin condition: Pr(0<|tau_0(X)| <= t) <= C_M t^kappa.
    Eq. (4); standard fast-rate assumption for plug-in classifiers.
  • ad hoc to paper Embedding preserves outcome information: E[Y(a)|X]=m_a(phi(X)), so b_{phi,a}=0.
    Prop. C.3; not verified for the text embedding used in simulations.
  • ad hoc to paper Generator-retrieval discrepancy bounded: max_{a,x}|bf(a,x)-ef_k(a,x)| <= eta_n.
    §5.1; this is assumed, not measured, and the regret bounds collapse without it.
  • ad hoc to paper Transformer MSE bounds apply to RAG-selected in-context examples.
    App. F admits this is not guaranteed; the retrieval rule violates the i.i.d. sampling model used in the transformer theory.

pith-pipeline@v1.3.0-alltime-deepseek · 24078 in / 11603 out tokens · 98381 ms · 2026-08-01T15:36:38.920296+00:00 · methodology

0 comments
read the original abstract

We propose one-step and two-step methods for policy learning with retrieval-augmented generation (RAG). We formulate RAG-based action selection under the potential outcome framework. In the two-step method, vector search retrieves action-specific neighboring evidence in an embedding space, the generator estimates conditional expected outcomes or their contrasts, and a plug-in rule selects an action. This formulation connects action-specific vector search with nearest-neighbor matching in causal inference. We decompose the regret of the two-step method into candidate-generation regret and within-candidate choice regret, and we bound the latter using prediction-error guarantees for nearest-neighbor estimators and transformers. We evaluate the one-step method directly as a policy because its intermediate computation is unobserved.

Figures

Figures reproduced from arXiv: 2607.18225 by Masahiro Kato, Taka Kato.

Figure 1
Figure 1. Figure 1: Repetition-level mean regret in DGP 1. The diamond denotes the mean across [PITH_FULL_IMAGE:figures/full_fig_p021_1.png] view at source ↗
Figure 2
Figure 2. Figure 2: Repetition-level mean regret in DGP 2. The diamond denotes the mean across [PITH_FULL_IMAGE:figures/full_fig_p022_2.png] view at source ↗
Figure 3
Figure 3. Figure 3: Repetition-level mean regret in DGP 3. The diamond denotes the mean across [PITH_FULL_IMAGE:figures/full_fig_p023_3.png] view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

41 extracted references · 3 linked inside Pith

  1. [1]

    Alberto Abadie and Guido W. Imbens. Large sample properties of matching estimators for average treatment effects. Econometrica, 74 0 (1): 0 235--267, 2006

    2006

  2. [2]

    What learning algorithm is in-context learning? investigations with linear models

    Ekin Aky \"u rek, Dale Schuurmans, Jacob Andreas, Tengyu Ma, and Denny Zhou. What learning algorithm is in-context learning? investigations with linear models. In International Conference on Learning Representations (ICLR), 2023

    2023

  3. [3]

    Self- RAG : Learning to retrieve, generate, and critique through self-reflection

    Akari Asai, Zeqiu Wu, Yizhong Wang, Avirup Sil, and Hannaneh Hajishirzi. Self- RAG : Learning to retrieve, generate, and critique through self-reflection. In International Conference on Learning Representations (ICLR), 2024

    2024

  4. [4]

    Policy learning with observational data

    Susan Athey and Stefan Wager. Policy learning with observational data. Econometrica, 89 0 (1): 0 133--161, 2021

    2021

  5. [5]

    Tsybakov

    Jean-Yves Audibert and Alexandre B. Tsybakov. Fast learning rates for plug-in classifiers. The Annals of Statistics, 35 0 (2): 0 608--633, 2007

    2007

  6. [6]

    Transformers as statisticians: Provable in-context learning with in-context algorithm selection

    Yu Bai, Fan Chen, Huan Wang, Caiming Xiong, and Song Mei. Transformers as statisticians: Provable in-context learning with in-context algorithm selection. In Workshop on Efficient Systems for Foundation Models at ICML2023, 2023

    2023

  7. [7]

    Underwood, and Richard J

    Michelle Ching, Ioana Popescu, Nico Smith, Tianyi Ma, William G. Underwood, and Richard J. Samworth. Efficient and minimax optimal in-context nonparametric regression with transformers. In International Conference on Machine Learning (ICML), 2026

    2026

  8. [8]

    A semiparametric instrumental variable approach to optimal treatment regimes under endogeneity

    Yifan Cui and Eric Tchetgen Tchetgen. A semiparametric instrumental variable approach to optimal treatment regimes under endogeneity. Journal of the American Statistical Association, 116 0 (533): 0 162--173, 2021

    2021

  9. [9]

    Using embedding models to improve probabilistic race prediction, 2026

    Noah Dasanaike and Kosuke Imai. Using embedding models to improve probabilistic race prediction, 2026. a rXiv: 2604.22555

    Pith/arXiv arXiv 2026

  10. [10]

    What can transformers learn in-context? a case study of simple function classes

    Shivam Garg, Dimitris Tsipras, Percy Liang, and Gregory Valiant. What can transformers learn in-context? a case study of simple function classes. In International Conference on Neural Information Processing Systems (NeurIPS), 2022

    2022

  11. [11]

    Retrieval-augmented reinforcement learning

    Anirudh Goyal, Abram Friesen, Andrea Banino, Theophane Weber, Nan Rosemary Ke, Adri \`a Puigdom \`e nech Badia, Arthur Guez, Mehdi Mirza, Peter C Humphreys, Ksenia Konyushova, Michal Valko, Simon Osindero, Timothy Lillicrap, Nicolas Heess, and Charles Blundell. Retrieval-augmented reinforcement learning. In International Conference on Machine Learning (IC...

    2022

  12. [12]

    Realm: retrieval-augmented language model pre-training

    Kelvin Guu, Kenton Lee, Zora Tung, Panupong Pasupat, and Ming-Wei Chang. Realm: retrieval-augmented language model pre-training. In International Conference on Machine Learning (ICML), 2020

    2020

  13. [13]

    Understanding scaling laws with statistical and approximation theory for transformer neural networks on intrinsically low-dimensional data

    Alexander Havrilla and Wenjing Liao. Understanding scaling laws with statistical and approximation theory for transformer neural networks on intrinsically low-dimensional data. In Annual Conference on Neural Information Processing Systems (NeurIPS), 2024

    2024

  14. [14]

    Humphreys, Arthur Guez, Olivier Tieleman, Laurent Sifre, Th\' e ophane Weber, and Timothy Lillicrap

    Peter C. Humphreys, Arthur Guez, Olivier Tieleman, Laurent Sifre, Th\' e ophane Weber, and Timothy Lillicrap. Large-scale retrieval for reinforcement learning. In International Conference on Neural Information Processing Systems (NeurIPS), 2022

    2022

  15. [15]

    Genai-powered inference, 2025

    Kosuke Imai and Kentaro Nakamura. Genai-powered inference, 2025. a rXiv: 2507.03897

    Pith/arXiv arXiv 2025

  16. [16]

    Causal inference with generative artificial intelligence: Application to texts as treatments, 2026

    Kosuke Imai and Kentaro Nakamura. Causal inference with generative artificial intelligence: Application to texts as treatments, 2026. a rXiv: 2410.00903

    Pith/arXiv arXiv 2026

  17. [17]

    Leveraging passage retrieval with generative models for open domain question answering

    Gautier Izacard and Edouard Grave. Leveraging passage retrieval with generative models for open domain question answering. In Proceedings of the 16th Conference of the European Chapter of the Association for Computational Linguistics: Main Volume, 2021

    2021

  18. [18]

    Non-asymptotic uniform rates of consistency for k-nn regression

    Heinrich Jiang. Non-asymptotic uniform rates of consistency for k-nn regression. Proceedings of the AAAI Conference on Artificial Intelligence, 33 0 (01): 0 3999–4006, Jul. 2019

    2019

  19. [19]

    Dense passage retrieval for open-domain question answering

    Vladimir Karpukhin, Barlas Oguz, Sewon Min, Patrick Lewis, Ledell Wu, Sergey Edunov, Danqi Chen, and Wen-tau Yih. Dense passage retrieval for open-domain question answering. In Conference on Empirical Methods in Natural Language Processing (EMNLP), 2020

    2020

  20. [20]

    Bridging the gap between empirical welfare maximization and conditional average treatment effect estimation in policy learning, 2025 a

    Masahiro Kato. Bridging the gap between empirical welfare maximization and conditional average treatment effect estimation in policy learning, 2025 a . a rXiv: 2510.26723

    arXiv 2025

  21. [21]

    Nearest neighbor matching as least squares density ratio estimation and riesz regression, 2025 b

    Masahiro Kato. Nearest neighbor matching as least squares density ratio estimation and riesz regression, 2025 b . a rXiv: 2510.24433

    arXiv 2025

  22. [22]

    Generalization through memorization: Nearest neighbor language models

    Urvashi Khandelwal, Omer Levy, Dan Jurafsky, Luke Zettlemoyer, and Mike Lewis. Generalization through memorization: Nearest neighbor language models. In International Conference on Learning Representations (ICLR), 2020

    2020

  23. [23]

    Transformers are minimax optimal nonparametric in-context learners

    Juno Kim, Tai Nakamaki, and Taiji Suzuki. Transformers are minimax optimal nonparametric in-context learners. In Annual Conference on Neural Information Processing Systems (NeurIPS), 2024

    2024

  24. [24]

    Who should be treated? empirical welfare maximization methods for treatment choice

    Toru Kitagawa and Aleksey Tetenov. Who should be treated? empirical welfare maximization methods for treatment choice. Econometrica, 86 0 (2): 0 591--616, 2018

    2018

  25. [25]

    u ttler, Mike Lewis, Wen-tau Yih, Tim Rockt\

    Patrick Lewis, Ethan Perez, Aleksandra Piktus, Fabio Petroni, Vladimir Karpukhin, Naman Goyal, Heinrich K\" u ttler, Mike Lewis, Wen-tau Yih, Tim Rockt\" a schel, Sebastian Riedel, and Douwe Kiela. Retrieval-augmented generation for knowledge-intensive nlp tasks. In International Conference on Neural Information Processing Systems (NeurIPS), 2020

    2020

  26. [26]

    Estimation based on nearest neighbor matching: from density ratio to average treatment effect

    Zhexiao Lin, Peng Ding, and Fang Han. Estimation based on nearest neighbor matching: from density ratio to average treatment effect. Econometrica, 91 0 (6): 0 2187--2217, 2023

    2023

  27. [27]

    STRAP : Robot sub-trajectory retrieval for augmented policy learning

    Marius Memmel, Jacob Berg, Bingqing Chen, Abhishek Gupta, and Jonathan Francis. STRAP : Robot sub-trajectory retrieval for augmented policy learning. In CoRL 2024 Workshop on Mastering Robot Manipulation in a World of Abundant Data, 2024

    2024

  28. [28]

    Pretrained transformer efficiently learns low-dimensional target functions in-context

    Kazusato Oko, Yujin Song, Taiji Suzuki, and Denny Wu. Pretrained transformer efficiently learns low-dimensional target functions in-context. In Annual Conference on Neural Information Processing Systems (NeurIPS), 2024

    2024

  29. [29]

    Estimating optimal treatment rules with an instrumental variable: A partial identification learning approach

    Hongming Pu and Bo Zhang. Estimating optimal treatment rules with an instrumental variable: A partial identification learning approach. Journal of the Royal Statistical Society Series B: Statistical Methodology, 83 0 (2): 0 318--345, 2021

    2021

  30. [30]

    Kessler, and Alex Luedtke

    Hongxiang Qiu, Marco Carone, Ekaterina Sadikova, Maria Petukhova, Ronald C. Kessler, and Alex Luedtke. Optimal individualized decision rules using instrumental variable methods. Journal of the American Statistical Association, 116 0 (533): 0 174--191, 2021

    2021

  31. [31]

    Rosenbaum and Donald B

    Paul R. Rosenbaum and Donald B. Rubin. The central role of the propensity score in observational studies for causal effects. Biometrika, 70 0 (1): 0 41--55, 1983

    1983

  32. [32]

    Nonparametric regression using deep neural networks with ReLU activation function

    Johannes Schmidt-Hieber. Nonparametric regression using deep neural networks with ReLU activation function. Annals of Statistics, 48 0 (4): 0 1875--1897, 2020

    2020

  33. [33]

    REPLUG : Retrieval-augmented black-box language models

    Weijia Shi, Sewon Min, Michihiro Yasunaga, Minjoon Seo, Richard James, Mike Lewis, Luke Zettlemoyer, and Wen-tau Yih. REPLUG : Retrieval-augmented black-box language models. In Proceedings of the 2024 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 1: Long Papers), 2024

    2024

  34. [34]

    REGENT : A retrieval-augmented generalist agent that can act in-context in new environments

    Kaustubh Sridhar, Souradeep Dutta, Dinesh Jayaraman, and Insup Lee. REGENT : A retrieval-augmented generalist agent that can act in-context in new environments. In International Conference on Learning Representations (ICLR), 2025

    2025

  35. [35]

    Adaptivity of deep re LU network for learning in besov and mixed smooth besov spaces: optimal rate and curse of dimensionality

    Taiji Suzuki. Adaptivity of deep re LU network for learning in besov and mixed smooth besov spaces: optimal rate and curse of dimensionality. In International Conference on Learning Representations (ICLR), 2019

    2019

  36. [36]

    Deep learning is adaptive to intrinsic dimensionality of model smoothness in anisotropic besov space

    Taiji Suzuki and Atsushi Nitanda. Deep learning is adaptive to intrinsic dimensionality of model smoothness in anisotropic besov space. In International Conference on Neural Information Processing Systems (NeurIPS), 2021

    2021

  37. [37]

    Counterfactual risk minimization: learning from logged bandit feedback

    Adith Swaminathan and Thorsten Joachims. Counterfactual risk minimization: learning from logged bandit feedback. In International Conference on Machine Learning (ICML), 2015

    2015

  38. [38]

    Approximation and estimation ability of transformers for sequence-to-sequence functions with infinite dimensional input

    Shokichi Takakura and Taiji Suzuki. Approximation and estimation ability of transformers for sequence-to-sequence functions with infinite dimensional input. In International Conference on Machine Learning (ICML), 2023

    2023

  39. [39]

    Transformers learn in-context by gradient descent

    Johannes Von Oswald, Eyvind Niklasson, Ettore Randazzo, Jo\ a o Sacramento, Alexander Mordvintsev, Andrey Zhmoginov, and Max Vladymyrov. Transformers learn in-context by gradient descent. In International Conference on Machine Learning (ICML). JMLR.org, 2023

    2023

  40. [40]

    C ausal RAG : Integrating causal graphs into retrieval-augmented generation

    Nengbo Wang, Xiaotian Han, Jagdip Singh, Jing Ma, and Vipin Chaudhary. C ausal RAG : Integrating causal graphs into retrieval-augmented generation. In Findings of the Association for Computational Linguistics: ACL 2025, 2025

    2025

  41. [41]

    Reddi, and Sanjiv Kumar

    Chulhee Yun, Srinadh Bhojanapalli, Ankit Singh Rawat, Sashank J. Reddi, and Sanjiv Kumar. Are transformers universal approximators of sequence-to-sequence functions? In International Conference on Learning Representations (ICLR), 2020

    2020