Pith. sign in

REVIEW 4 major objections 5 minor 18 references

Multilinguality in LLM-Designed Reward Functions for Restless Bandits: Effects on Task Performance and Fairness

T0 review · 4 major / 5 minor · reviewed 2026-08-10 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read This paper claims that prompting an LLM reward designer in English instead of Hindi, Tamil, or Tulu changes both task performance and fairness in restless-bandit resource allocation.

desk verdict The paper's headline claim of significantly better English performance is not backed by its own statistics, but the question is real, the study is honest, and it deserves a rigorous peer review. read the letter →

arxiv 2501.13120 v1 pith:BO2KGRNC submitted 2025-01-20 cs.CL cs.AIcs.LGcs.MA

Ambreesh Parthasarathy , Chandrasekar Subramanian , Ganesh Senrayan , Shreyash Adappanavar , Aparna Taneja , Balaraman Ravindran , Milind Tambe This is my paper · ORCID
classification cs.CLcs.AIcs.LGcs.MA
keywords restlessmulti-armedbanditsrewardfunctiondesignlargelanguagemodelsmultilingualpromptslow-resourcelanguagesfairnessdemographicparityresourceallocation
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper tests what happens when the goal prompts that steer an LLM-designed reward function for restless multi-armed bandits are written in Hindi, Tamil, and Tulu instead of English. Using the DLM pipeline on a synthetic public-health resource-allocation environment, it finds that English prompts yield acceptable reward functions more often and produce allocations that match the user's stated priorities more closely. It also finds that semantically identical prompts with different phrasing lead to different allocations, and that more complex prompts degrade performance in every language, with English degrading less. On fairness, prompts in low-resource languages and prompts with more intended features are more likely to create demographic-parity violations on features the user did not intend to target. The paper's point is that language choice and prompt wording are not neutral: they change both the effectiveness and the fairness of automated resource allocation.

What carries the argument

The central object is the DLM (Decision-Language Model) pipeline: an LLM is given a chain-of-thought prompt with a goal and is asked to write a Python reward function, then an evolutionary search with an LLM-based reflection step refines proposals, and a Whittle-index policy solves the resulting restless multi-armed bandit, a sequential resource-allocation model where each arm's state evolves regardless of whether it is pulled. The paper varies the language of the goal prompt (English, Hindi, Tamil, Tulu) and the complexity and phrasing of the prompt, and measures outcomes with two instruments: the rate of 'acceptable' reward functions that use all and only the intended features, and demographic-parity variance across feature buckets, where high variance on unintended features counts as unfairness. The work these parts do is to separate language effects from prompt-complexity and phrasing effects in a controlled synthetic environment.

What would settle it

Have native speakers back-translate the eight Hindi, Tamil, and Tulu prompts into English without seeing the originals, then rerun the DLM pipeline with the back-translated prompts; if acceptable-reward rates and fairness metrics match the English-prompt results, the claimed language effects collapse, while if the gaps persist under faithful translations, they are confirmed.

Watch

Extended reading notes

Core claim

On the paper's own terms, the central discovery is that the DLM algorithm's LLM-proposed reward functions are significantly better when prompted in English than in Hindi, Tamil, or Tulu, and that this gap shows up both in task performance and in fairness. The intended reward features are identified correctly at a higher rate under English prompts, and the resulting Whittle-index allocations deviate less from what the prompt asks for. The paper further claims that the exact phrasing of a prompt alters allocations even when the meaning is unchanged, that explicit statements of the goal help, and that increasing prompt complexity hurts all languages but hurts lower-resource languages more. On the fairness side, low-resource languages and more complex prompts are both highly likely to create demographic-parity variance on unintended features, which the paper treats as unfairness. Because the system is aimed at grassroots public-health workers who may prefer local languages, the claim implies that deployment language alone could shift who gets calls.

Load-bearing premise

The load-bearing premise is that the Hindi, Tamil, and Tulu prompts are faithful translations of the English prompts; the paper provides no back-translation, native-speaker verification, or translation-quality scores, so the observed gaps could in principle be artifacts of translation quality rather than of language itself.

Editorial extensions

If this is right

  • English prompts yield significantly higher rates of acceptable reward functions than Hindi, Tamil, or Tulu prompts across the tested goal prompts.
  • Semantically equivalent rephrasings produce noticeably different allocations, so prompt wording is a performance variable, not a nuisance.
  • As prompts involve more intended features, success rates drop for all languages, and English remains more stable; past roughly three intended features, performance collapses regardless of language.
  • Low-resource-language prompts and more complex prompts are more likely to produce demographic-parity variance on unintended features, meaning unfair allocations along dimensions the user did not specify.
  • For deployed public-health allocation, these gaps imply that outcomes would differ across language communities unless the DLM pipeline is modified.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • A reasonable reading is that the performance gap may reflect the LLM's weaker command of Hindi, Tamil, and Tulu rather than any property of those languages; if so, improving multilingual instruction-following or adding a prompt-rewriting stage could close most of the gap.
  • The fairness result suggests a cheap testable safeguard: compute demographic-parity variance on all non-target features before deployment and reject reward functions that exceed a threshold on unintended dimensions.
  • One could extend the study by back-translating the translated prompts and comparing reward quality on the back-translations, which would separate translation fidelity from intrinsic language handling.
  • The collapse past roughly three intended features hints that the evolutionary search and reflection stage, not the LLM's language ability, may be the binding constraint for complex goals.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. The paper studies how the language of goal prompts affects the DLM algorithm, which uses an LLM (Gemini 1.0 Pro) to design reward functions for restless multi-armed bandits in a public-health-inspired resource allocation setting. The authors construct a synthetic environment with six features, run eight prompts (six increasing in complexity and two rephrased versions) in English, Hindi, Tamil, and Tulu, and evaluate three outcomes: the rate of 'acceptable' reward functions, task performance, and fairness measured by demographic-parity variance. The paper reports that English prompts produce better reward functions and task performance, that phrasing matters, that performance degrades with prompt complexity but less so for English, and that low-resource languages and complex prompts are more likely to induce unfairness along unintended features.

Significance. If fully supported, this would be a valuable early empirical result on language-dependent behavior of LLM-based reward design, with direct implications for equitable deployment of automated resource allocation in multilingual, low-resource settings. The paper addresses an understudied question—non-English prompts and fairness rather than performance alone—and includes a low-resource language (Tulu), multiple prompt complexities, and two correlation strengths. Its strengths include the explicit study of prompt phrasing and the use of a fairness metric beyond task success. However, the central comparisons are not yet supported by the evidence as presented: the 'significantly better' claim lacks inferential tests, the multilingual manipulation is not validated as translation-equivalent, and the task-success/fairness thresholds are not operationalized. These issues are fixable in revision and do not invalidate the research direction, but they block acceptance in the current form.

major comments (4)
  1. [Abstract and Section 3.1, Table 3] The claim that LLM-proposed reward functions are 'significantly better when prompted in English' is not supported by any significance test. In Table 3, the reported standard errors overlap for most language pairs: for Prompt 1, English is 0.65 ± 0.15 versus Hindi 0.60 ± 0.15; for Prompt 5, all four languages report 0.10. The paper provides no paired test across the 20 runs, no confidence intervals for differences, and no effect-size measure. Because this claim is the paper's headline result, the authors must report appropriate inferential statistics (for example, paired bootstrap or McNemar-style tests on the per-run acceptability outcomes) or soften the language to descriptive trends.
  2. [Section 2.2, Table 2] The English-versus-other-languages comparison presupposes that the Hindi, Tamil, and Tulu prompts are semantically equivalent translations of the English originals, but the paper gives no evidence for this. Table 2 lists only English prompts; the translated prompts, the translation procedure, translator qualifications, back-translations, and any native-speaker validation are absent. Without this information, observed performance and fairness gaps are confounded with translation quality and cannot be attributed to language per se. The authors should include the full translated prompt set and a validation protocol, and ideally a control condition that perturbs the English prompts to quantify sensitivity to wording.
  3. [Section 4.3 and Figure 6/7] The 'task success rate' used to support Result 4 is not operationally defined. The text says success occurs when 'the allocations have deviated from the relevant features to the point where the allocation is higher than what it would have been had the allocation been uniform across feature values,' but no threshold, comparison procedure, or code is provided. This makes the complexity-degradation and English-robustness claims non-reproducible. In addition, the authors' own Note in Section 4.3 states that the very poor performance for Prompts 2 and 3 is due to failure on ordered categorical features; because Prompt 2 (two features) is not systematically easier than Prompt 5 (two features plus an inference step) in Table 3, the monotonic complexity story is not supported by the raw acceptability rates. The analysis should separate feature-type effects from complexity effects and precisely define the success criterion.
  4. [Section 5, Figures 8–11] The fairness results depend on two unspecified choices. First, the 'certain threshold' used for the absolute DP-variance counts in Figures 9 and 10 is not reported, so the reader cannot tell whether the plotted differences are robust or an artifact of one threshold value. Second, the relative count (unintended-feature DP variance greater than intended-feature DP variance) may be trivially sensitive to whether the intended feature has high variance by construction. Furthermore, Result 5 is reported as strong for α = 0.2 but 'not as pronounced' for α = 0.8; the paper explains this by correlation propagation but does not provide a sensitivity analysis. The fairness claim 'low-resource languages create unfairness' is therefore conditional on one synthetic correlation structure and should be qualified accordingly, with thresholds reported and confidence intervals or error bars given for the counts.
minor comments (5)
  1. [Section 2.3, Equation (3)] Equation (3) defines P(Y=1) as the unweighted average of group-level allocation probabilities, not the overall allocation rate across arms; this should be clarified because the demographic-parity variance in Equation (2) then compares group probabilities against an unweighted mean rather than the population prevalence.
  2. [Section 3, Table 3] The caption calls the entries 'acceptable prompt rates,' but the rate is of acceptable reward functions, not prompts; please reword for clarity.
  3. [Section 2.2] The full set of translated prompts is not included in the paper or an appendix; providing them is essential for reproducibility and for any verification of translation equivalence.
  4. [Section 2.1, Table 1] The feature weight vector is described as 'inspired by' prior work, but the choice of signs and magnitudes is one of several plausible specifications; a sensitivity check over the weight vector would strengthen the claim that the results are not artifacts of the synthetic environment.
  5. [Section 4.3, Figures 6–7] The x-axis label 'increasing complexity' is not numerically defined; consider reporting the number of intended features and the reasoning step for each prompt directly in the figure.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the multilingual performance and fairness results are empirical measurements, not derivations that reduce to their inputs.

full rationale

This paper is an empirical evaluation, not a derivation. The LLM-generated reward functions are obtained from a fixed prompting pipeline and then evaluated against externally defined criteria (feature inclusion, allocation deviation, and DP variance). No parameter is fitted to the target results, and no claim is justified by defining the input in terms of the output. The authors rely on the DLM algorithm from Behari et al. (2024), and authors Taneja and Tambe overlap with that work and with Verma et al. (2024); however, the DLM algorithm is the object under study rather than the authority for the paper's conclusions, so the self-citation is not load-bearing. The 'acceptable reward function' and 'success' definitions are operationalizations of task adherence; the complexity trend is observed from allocations across prompts, not entailed by the definitions used. Translation equivalence is an unverified assumption and a validity threat, but a confound is not a circular derivation. No quoted step reduces an equation to itself by construction or renames a fitted quantity as a prediction. Hence the appropriate finding is no significant circularity.

Assumptions & free parameters 4 free parameters · 5 assumptions · 0 invented entities

The evaluation rests on a hand-built synthetic environment with hand-chosen structural equations, feature distributions, and weights, plus a fairness metric defined by the authors. None of these are fitted to data; they are assumptions. The paper relies on the cited DLM pipeline and Whittle index solver as given, and on the LLM's outputs without verification of translation fidelity.

free parameters (4)
  • alpha (strength of structural relationships) = 0.2 and 0.8
    Chosen by the authors to simulate low and high conditional correlation between features in the synthetic graphical model. Not derived from data.
  • Feature weight vector for transition probabilities = Age 0.8, Income 1.5, Language Spoken -0.3, Education 1.5, Phone Ownership -1.5, Times To Be Called 0.3
    Hand-chosen based on domain reasoning and 'inspired by' Verma et al. (Table 1). These weights determine the active transition probabilities and hence the reward environment.
  • DP variance threshold for fairness counts = not specified numerically
    Fairness plots (Figures 8-11) count prompts where DP variance in unintended features is greater than 'a certain threshold', but the threshold value is not stated in the text, making the metric ambiguous.
  • Task success threshold = not specified numerically
    Success is defined as allocations deviating from relevant features 'to the point where the allocation is higher than what it would have been had the allocation been uniform'. The precise threshold is not defined.
assumptions (5)
  • domain assumption Synthetic structural equations (Age to Income, Education to Income, Income to Phone Ownership, Phone Ownership to Times To Be Called, Language Spoken independent) reflect real-world causal structure.
    Section 2.1. The entire environment is built on these hand-specified equations, with no real-world data validation.
  • domain assumption Feature distributions are uniform in the population.
    Section 2.1 states 'we have assumed that the variables follow a uniform distribution as we do not have any information on the prior distribution of these variables.'
  • standard math Whittle index policy is an appropriate optimal or near-optimal solver for the RMAB.
    Section 2 uses a Whittle Index based solution, following Verma et al. and Whittle 1988, which is a standard method for restless bandits.
  • ad hoc to paper An 'acceptable reward function' is one that contains all relevant features and no spurious features, as defined by the authors.
    Section 3 defines acceptability in this way. The criterion is reasonable but author-defined and not validated against any external standard.
  • domain assumption The feature 'Language Spoken' does not influence likelihood of conversion.
    Section 2.1: 'In our structural equations, we assume Language Spoken to be independent of the rest' and in Table 1 notes 'Language Spoken does not influence the likelihood of conversion; thus, it can take a small positive/negative value.' This is a modeling choice that affects fairness analysis.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Multilinguality in LLM-Designed Reward Functions for Restless Bandits: Effects on Task Performance and Fairness." pith.science (2026). https://pith.science/paper/BO2KGRNC

@misc{pith2026250113120,
  author       = {Pith},
  title        = {Pith review of: Multilinguality in LLM-Designed Reward Functions for Restless Bandits: Effects on Task Performance and Fairness},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/BO2KGRNC}},
  note         = {Machine review of arXiv:2501.13120}
}
read the original abstract

Restless Multi-Armed Bandits (RMABs) have been successfully applied to resource allocation problems in a variety of settings, including public health. With the rapid development of powerful large language models (LLMs), they are increasingly used to design reward functions to better match human preferences. Recent work has shown that LLMs can be used to tailor automated allocation decisions to community needs using language prompts. However, this has been studied primarily for English prompts and with a focus on task performance only. This can be an issue since grassroots workers, especially in developing countries like India, prefer to work in local languages, some of which are low-resource. Further, given the nature of the problem, biases along population groups unintended by the user are also undesirable. In this work, we study the effects on both task performance and fairness when the DLM algorithm, a recent work on using LLMs to design reward functions for RMABs, is prompted with non-English language commands. Specifically, we run the model on a synthetic environment for various prompts translated into multiple languages. The prompts themselves vary in complexity. Our results show that the LLM-proposed reward functions are significantly better when prompted in English compared to other languages. We also find that the exact phrasing of the prompt impacts task performance. Further, as prompt complexity increases, performance worsens for all languages; however, it is more robust with English prompts than with lower-resource languages. On the fairness side, we find that low-resource languages and more complex prompts are both highly likely to create unfairness along unintended dimensions.

Figures

Figures reproduced from arXiv: 2501.13120 by the authors.

Figure 1
Figure 1. Plot showcasing the allocation percentages [PITH_FULL_IMAGE:figures/full_fig_p004_1.png] view at source ↗
Figure 2
Figure 2. Plot showcasing the allocation percentages for the [PITH_FULL_IMAGE:figures/full_fig_p004_2.png] view at source ↗
Figure 5
Figure 5. Plot showcasing the summed allocation percent [PITH_FULL_IMAGE:figures/full_fig_p005_5.png] view at source ↗
Figures from the paper (25 more)
Figure 4
Figure 4. Figure 4: Plot showcasing the allocation percentages for the [PITH_FULL_IMAGE:figures/full_fig_p005_4.png]
Figure 6
Figure 6. Figure 6: Plot measuring the success rate for different lan [PITH_FULL_IMAGE:figures/full_fig_p006_6.png]
Figure 7
Figure 7. Figure 7: Plot measuring the success rate for different lan [PITH_FULL_IMAGE:figures/full_fig_p006_7.png]
Figure 11
Figure 11. Figure 11: Average number of languages in each prompt [PITH_FULL_IMAGE:figures/full_fig_p007_11.png]
Figure 9
Figure 9. Figure 9: Average number of prompts where the DP variance of unintended features is greater than a threshold value (higher is more unfair) for α = 0.2 [PITH_FULL_IMAGE:figures/full_fig_p007_9.png]
Figure 10
Figure 10. Figure 10: Average number of prompts where the DP variance of unintended features is greater than a thresh￾old value (higher is more unfair) for α = 0.8 Improved reflection component Ideally, even if one of the multiple proposed reward functions is good, the LLM [PITH_FULL_IMAG…
Figure 12
Figure 12. Figure 12: Allocation for Prompt 1, α = 0.2 D Appendix: Complete Prompt An example of a full prompt that is given to the LLM is presented in [PITH_FULL_IMAGE:figures/full_fig_p008_12.png]
Figure 13
Figure 13. Figure 13: Allocation for Prompt 2, α = 0.2 [PITH_FULL_IMAGE:figures/full_fig_p009_13.png]
Figure 14
Figure 14. Figure 14: Allocation for Prompt 3, α = 0.2 [PITH_FULL_IMAGE:figures/full_fig_p009_14.png]
Figure 15
Figure 15. Figure 15: Allocation for Prompt 4, α = 0.2 [PITH_FULL_IMAGE:figures/full_fig_p010_15.png]
Figure 16
Figure 16. Figure 16: Allocation for Prompt 5, α = 0.2 [PITH_FULL_IMAGE:figures/full_fig_p010_16.png]
Figure 17
Figure 17. Figure 17: Allocation for Prompt 6, α = 0.2 [PITH_FULL_IMAGE:figures/full_fig_p011_17.png]
Figure 18
Figure 18. Figure 18: Allocation for Prompt 7, α = 0.2 [PITH_FULL_IMAGE:figures/full_fig_p011_18.png]
Figure 19
Figure 19. Figure 19: Allocation for Prompt 8, α = 0.2 [PITH_FULL_IMAGE:figures/full_fig_p012_19.png]
Figure 20
Figure 20. Figure 20: Allocation for Prompt 1, α = 0.8 [PITH_FULL_IMAGE:figures/full_fig_p012_20.png]
Figure 21
Figure 21. Figure 21: Allocation for Prompt 2, α = 0.8 [PITH_FULL_IMAGE:figures/full_fig_p013_21.png]
Figure 22
Figure 22. Figure 22: Allocation for Prompt 3, α = 0.8 [PITH_FULL_IMAGE:figures/full_fig_p013_22.png]
Figure 23
Figure 23. Figure 23: Allocation for Prompt 4, α = 0.8 [PITH_FULL_IMAGE:figures/full_fig_p014_23.png]
Figure 24
Figure 24. Figure 24: Allocation for Prompt 5, α = 0.8 [PITH_FULL_IMAGE:figures/full_fig_p014_24.png]
Figure 25
Figure 25. Figure 25: Allocation for Prompt 6, α = 0.8 [PITH_FULL_IMAGE:figures/full_fig_p015_25.png]
Figure 26
Figure 26. Figure 26: Allocation for Prompt 7, α = 0.8 [PITH_FULL_IMAGE:figures/full_fig_p015_26.png]
Figure 27
Figure 27. Figure 27: Allocation for Prompt 8, α = 0.8 [PITH_FULL_IMAGE:figures/full_fig_p016_27.png]
Figure 28
Figure 28. Figure 28: Phrasing for both bracket age α = 0.8 [PITH_FULL_IMAGE:figures/full_fig_p017_28.png]
Figure 29
Figure 29. Figure 29: Phrasing for first bracket age α = 0.8 [PITH_FULL_IMAGE:figures/full_fig_p017_29.png]
Figure 32
Figure 32. Figure 32: Phrasing for income α = 0.8 [PITH_FULL_IMAGE:figures/full_fig_p017_32.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

18 extracted references · 13 canonical work pages

  1. [1]

    E.; Fort, S.; Lanham, T.; Telleen-Lawton, T.; Conerly, T.; Henighan, T.; Hume, T.; Bowman, S

    Bai, Y.; Kadavath, S.; Kundu, S.; Askell, A.; Kernion, J.; Jones, A.; Chen, A.; Goldie, A.; Mirhoseini, A.; McKinnon, C.; Chen, C.; Olsson, C.; Olah, C.; Hernandez, D.; Drain, D.; Ganguli, D.; Li, D.; Tran-Johnson, E.; Perez, E.; Kerr, J.; Mueller, J.; Ladish, J.; Landau, J.; Ndousse, K.; Lukosuite, K.; Lovitt, L.; Sellitto, M.; Elhage, N.; Schiefer, N.; ...

    arXiv 2022

  2. [2]

    M.; and Tambe, M

    Behari, N.; Zhang, E.; ZHAO, Y.; Taneja, A.; Nagaraj, D. M.; and Tambe, M. 2024. A Decision-Language Model ( DLM ) for Dynamic Restless Multi-Armed Bandit Tasks in Public Health. In The Thirty-eighth Annual Conference on Neural Information Processing Systems

    work page 2024

  3. [3]

    Cao, Y.; Zhao, H.; Cheng, Y.; Shu, T.; Chen, Y.; Liu, G.; Liang, G.; Zhao, J.; Yan, J.; and Li, Y. 2024. Survey on Large Language Model-Enhanced Reinforcement Learning: Concept, Taxonomy, and Methods. IEEE Transactions on Neural Networks and Learning Systems, 1–21

    work page 2024

  4. [4]

    Demidova, A.; Atwany, H.; Rabih, N.; Sha ' ban, S.; and Abdul-Mageed, M. 2024. John vs. Ahmed: Debate-Induced Bias in Multilingual LLM s. In Habash, N.; Bouamor, H.; Eskander, R.; Tomeh, N.; Abu Farha, I.; Abdelali, A.; Touileb, S.; Hamed, I.; Onaizan, Y.; Alhafni, B.; Antoun, W.; Khalifa, S.; Haddad, H.; Zitouni, I.; AlKhamissi, B.; Almatham, R.; and Mri...

    work page 2024

  5. [5]

    Koller, D.; and Friedman, N. 2009. Probabilistic Graphical Models: Principles and Techniques - Adaptive Computation and Machine Learning. The MIT Press. ISBN 0262013193

    work page 2009

  6. [6]

    J.; Loftus, J.; Russell, C.; and Silva, R

    Kusner, M. J.; Loftus, J.; Russell, C.; and Silva, R. 2017. Counterfactual Fairness. In Guyon, I.; Luxburg, U. V.; Bengio, S.; Wallach, H.; Fergus, R.; Vishwanathan, S.; and Garnett, R., eds., Advances in Neural Information Processing Systems, volume 30. Curran Associates, Inc

    work page 2017

  7. [7]

    M.; Bullard, K.; and Sadigh, D

    Kwon, M.; Xie, S. M.; Bullard, K.; and Sadigh, D. 2023. Reward Design with Language Models. arXiv:2303.00001

    arXiv 2023

  8. [8]

    Lai, W.; Mesgar, M.; and Fraser, A. 2024. LLM s Beyond E nglish: Scaling the Multilingual Capability of LLM s with Cross-Lingual Feedback. In Ku, L.-W.; Martins, A.; and Srikumar, V., eds., Findings of the Association for Computational Linguistics: ACL 2024, 8186--8213. Bangkok, Thailand: Association for Computational Linguistics

    work page 2024

Show all 18 references
  1. [9]

    Liu, H.; Liu, K.; and Zhao, Q. 2013. Learning in a Changing World: Restless Multiarmed Bandit With Unknown Dynamics. IEEE Trans. Inf. Theor., 59(3): 1902–1916

    2013

  2. [10]

    J.; Liang, W.; Wang, G.; Huang, D.-A.; Bastani, O.; Jayaraman, D.; Zhu, Y.; Fan, L.; and Anandkumar, A

    Ma, Y. J.; Liang, W.; Wang, G.; Huang, D.-A.; Bastani, O.; Jayaraman, D.; Zhu, Y.; Fan, L.; and Anandkumar, A. 2024. Eureka: Human-Level Reward Design via Coding Large Language Models. In The Twelfth International Conference on Learning Representations

    2024

  3. [11]

    Pearl, J. 2009. Causality. Cambridge University Press, 2 edition

    2009

  4. [12]

    Sun, S.; Liu, R.; Lyu, J.; Yang, J.-W.; Zhang, L.; and Li, X. 2024. A Large Language Model-Driven Reward Design Framework via Dynamic Feedback for Reinforcement Learning. arXiv:2410.14660

    2024 arXiv

  5. [13]

    M.; Hauth, A.; Millican, K.; et al

    Team, G.; Anil, R.; Borgeaud, S.; Alayrac, J.-B.; Yu, J.; Soricut, R.; Schalkwyk, J.; Dai, A. M.; Hauth, A.; Millican, K.; et al. 2023. Gemini: a family of highly capable multimodal models. arXiv preprint arXiv:2312.11805

    2023 arXiv

  6. [14]

    Verma, S.; Boehmer, N.; Kong, L.; and Tambe, M. 2024. Balancing Act: Prioritization Strategies for LLM -Designed Restless Bandit Rewards. In Workshop on Socially Responsible Language Modelling Research

    2024

  7. [15]

    Whittle, P. 1988. Restless bandits: Activity allocation in a changing world. Journal of applied probability, 25(A): 287--298

    1988

  8. [16]

    G.; Chiang, H.-T

    Yu, W.; Gileadi, N.; Fu, C.; Kirmani, S.; Lee, K.-H.; Arenas, M. G.; Chiang, H.-T. L.; Erez, T.; Hasenclever, L.; Humplik, J.; Ichter, B.; Xiao, T.; Xu, P.; Zeng, A.; Zhang, T.; Heess, N.; Sadigh, D.; Tan, J.; Tassa, Y.; and Xia, F. 2023. Language to Rewards for Robotic Skill ...

    2023

  9. [17]

    , " * write output.state after.block = add.period write newline

    ENTRY address archivePrefix author booktitle chapter edition editor eid eprint howpublished institution isbn journal key month note number organization pages publisher school series title type volume year label extra.label sort.label short.list INTEGERS output.state before.all...

  10. [18]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 gl...

Pith tools

Reviewed August 10, 2026 · model on record in the stance chip above.