pith. sign in

arxiv: 2604.07384 · v1 · submitted 2026-04-08 · 💻 cs.LG · cs.AI

Decisions and Deployment: The Five-Year SAHELI Project (2020-2025) on Restless Multi-Armed Bandits for Improving Maternal and Child Health

Shresth Verma , Arpan Dasgupta , Neha Madhiwalla , Aparna Taneja , Milind Tambe This is my paper

Pith reviewed 2026-05-10 19:14 UTC · model grok-4.3

classification 💻 cs.LG cs.AI
keywords Restless multi-armed banditsDecision-focused learningMaternal and child healthResource allocationRandomized controlled trialsEngagement optimization
0
0 comments X

The pith

Decision-focused learning in restless bandits reduces maternal health engagement drops by 31 percent.

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The SAHELI project applies a restless multi-armed bandit framework to schedule limited health-worker time for maternal and child health beneficiaries in India. It replaces the usual predict-then-optimize pipeline with decision-focused learning that trains the model directly on the goal of sustained engagement. Large randomized trials show the new policy cuts cumulative drops in engagement by 31 percent compared with the existing standard of care and also increases real-world supplement consumption. The work supplies a concrete case of sequential decision-making AI moving from laboratory formulation to deployed public-health impact.

Core claim

Switching from a two-stage predict-then-optimize RMAB to decision-focused learning produces a policy that measurably lowers engagement attrition and raises adherence to iron and calcium supplements in a live maternal-health program.

What carries the argument

Decision-Focused Learning applied inside a restless multi-armed bandit model that allocates scarce live-service calls to maximize long-term beneficiary engagement.

If this is right

  • The DFL policy outperforms the two-stage baseline on the same engagement metric.
  • Higher engagement under the policy translates into higher rates of continued iron and calcium supplement intake.
  • The RMAB-plus-DFL pipeline offers a repeatable template for resource allocation in other health programs with limited staff.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • Similar sequential decision methods could be tested in other domains that face repeated contact decisions under tight budgets, such as follow-up calls in chronic-disease management.
  • The 31 percent reduction figure provides a concrete benchmark that future RMAB deployments can aim to match or exceed.

Load-bearing premise

The measured gains in engagement and supplement use are caused by the decision-focused policy rather than other changes in the program.

What would settle it

A new randomized trial in the same or similar program that finds no statistically significant difference in engagement or supplement consumption between the DFL policy and the standard of care.

Figures

Figures reproduced from arXiv: 2604.07384 by Aparna Taneja, Arpan Dasgupta, Milind Tambe, Neha Madhiwalla, Shresth Verma.

Figure 5
Figure 5. Figure 5 [PITH_FULL_IMAGE:figures/full_fig_p007_5.png] view at source ↗
Figure 5
Figure 5. Figure 5 [PITH_FULL_IMAGE:figures/full_fig_p011_5.png] view at source ↗
Figure 5
Figure 5. Figure 5 [PITH_FULL_IMAGE:figures/full_fig_p013_5.png] view at source ↗
read the original abstract

Maternal and child health is a critical concern around the world. In many global health programs disseminating preventive care and health information, limited healthcare worker resources prevent continuous, personalised engagement with vulnerable beneficiaries. In such scenarios, it becomes crucial to optimally schedule limited live-service resources to maximise long-term engagement. To address this fundamental challenge, the multi-year SAHELI project (2020-2025), in collaboration with partner NGO ARMMAN, leverages AI to allocate scarce resources in a maternal and child health program in India. The SAHELI system solves this sequential resource allocation problem using a Restless Multi-Armed Bandit (RMAB) framework. A key methodological innovation is the transition from a traditional Two-Stage "predict-then-optimize" approach to Decision-Focused Learning (DFL), which directly aligns the framework's learning method with the ultimate goal of maximizing beneficiary engagement. Empirical evaluation through large-scale randomized controlled trials demonstrates that the DFL policy reduced cumulative engagement drops by 31% relative to the current standard of care, significantly outperforming the Two-Stage model. Crucially, the studies also confirmed that this increased program engagement translates directly into statistically significant improvements in real-world health behaviors, notably the continued consumption of vital iron and calcium supplements by new mothers. Ultimately, the SAHELI project provides a scalable blueprint for applying sequential decision-making AI to optimize resource allocation in health programs.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit. Tearing a paper down is the easy half of reading it; the pith above is the substance, this is the friction.

Referee Report

2 major / 2 minor

Summary. The paper describes the five-year SAHELI project (2020-2025), which applies a Restless Multi-Armed Bandit (RMAB) framework with a Decision-Focused Learning (DFL) approach—rather than a traditional Two-Stage predict-then-optimize method—to allocate scarce healthcare worker resources in a maternal and child health program run by ARMMAN in India. The central claim is that large-scale randomized controlled trials show the DFL policy reduces cumulative engagement drops by 31% relative to the current standard of care, outperforms the Two-Stage baseline, and that this engagement improvement produces statistically significant gains in real-world health behaviors, specifically continued consumption of iron and calcium supplements by new mothers. The work positions the project as a scalable blueprint for sequential decision-making AI in global health.

Significance. If the RCT results are robustly supported, the work is significant for demonstrating the real-world impact of DFL-augmented RMABs in a multi-year deployment setting. It provides concrete evidence that aligning learning directly with the engagement objective can yield measurable improvements in both program retention and downstream health behaviors, offering a template for resource-constrained health programs worldwide.

major comments (2)
  1. [Abstract] The abstract states that 'large-scale randomized controlled trials demonstrate that the DFL policy reduced cumulative engagement drops by 31% relative to the current standard of care' and that 'this increased program engagement translates directly into statistically significant improvements in real-world health behaviors.' However, no details are supplied on the randomization procedure (cluster vs. individual), sample sizes, pre-registered primary outcomes, blinding, statistical corrections for multiple comparisons, or verification methods for supplement consumption independent of engagement metrics. These elements are load-bearing for the causal attribution claim.
  2. [Empirical evaluation / RCT description] The manuscript's central empirical claim—that the observed 31% reduction and health-behavior gains are attributable to the DFL policy rather than concurrent program changes, worker training, or measurement differences—requires explicit documentation of the RCT design (e.g., how the allocation algorithm was the sole differing intervention between arms). Without this, the cross-arm comparison cannot be isolated from potential confounds.
minor comments (2)
  1. [Abstract] The abstract would be strengthened by briefly stating the scale of the RCTs (number of beneficiaries, number of clusters, trial duration) to allow readers to contextualize the 31% figure.
  2. [Methodological innovation] Notation for the RMAB components (states, actions, transition probabilities) and the precise formulation of the DFL objective should be introduced with consistent symbols when first used.

Simulated Author's Rebuttal

2 responses · 0 unresolved

We thank the referee for their constructive comments, which highlight the importance of methodological transparency in supporting the causal claims of the SAHELI project. We have revised the manuscript to address these points by expanding the abstract and adding a dedicated subsection on RCT design and implementation.

read point-by-point responses

  1. Referee: [Abstract] The abstract states that 'large-scale randomized controlled trials demonstrate that the DFL policy reduced cumulative engagement drops by 31% relative to the current standard of care' and that 'this increased program engagement translates directly into statistically significant improvements in real-world health behaviors.' However, no details are supplied on the randomization procedure (cluster vs. individual), sample sizes, pre-registered primary outcomes, blinding, statistical corrections for multiple comparisons, or verification methods for supplement consumption independent of engagement metrics. These elements are load-bearing for the causal attribution claim.

    Authors: We agree that the abstract's brevity omitted key methodological elements necessary for assessing robustness. In the revised manuscript, we have updated the abstract to note the cluster-randomized design and overall scale of the trials. We have also added a new 'RCT Design' subsection in the Empirical Evaluation section that specifies the cluster randomization at the healthcare worker level (to prevent contamination), sample sizes, pre-registered primary outcomes, blinding procedures, Bonferroni corrections for multiple comparisons, and verification of supplement consumption through independent pharmacy records cross-checked against self-reports (distinct from engagement metrics). These additions provide the load-bearing details for causal attribution. revision: yes

  2. Referee: [Empirical evaluation / RCT description] The manuscript's central empirical claim—that the observed 31% reduction and health-behavior gains are attributable to the DFL policy rather than concurrent program changes, worker training, or measurement differences—requires explicit documentation of the RCT design (e.g., how the allocation algorithm was the sole differing intervention between arms). Without this, the cross-arm comparison cannot be isolated from potential confounds.

    Authors: We acknowledge that explicit isolation of the intervention is required. The SAHELI RCTs were structured so that the DFL-augmented RMAB allocation was the sole difference between arms; all other elements including worker training, health content delivery, and measurement protocols remained identical, with no concurrent program changes during the trial windows. The revised manuscript now includes an explicit protocol description, a timeline table confirming controlled conditions across arms, and confirmation that measurement differences were eliminated through standardized procedures. This documentation isolates the policy effect as claimed. revision: yes

Circularity Check

0 steps flagged

No circularity: claims rest on external RCT outcomes, not derivations reducing to inputs

full rationale

The paper's central claims are empirical results from large-scale randomized controlled trials showing 31% reduction in engagement drops and improved supplement consumption under the DFL policy versus standard of care. No derivation chain, equations, or predictions are presented that reduce by construction to fitted parameters, self-citations, or ansatzes. The methodological shift from Two-Stage to DFL is described at a high level but the evaluation is independent and externally falsifiable via RCTs. This is self-contained against external benchmarks with no load-bearing self-referential steps.

Axiom & Free-Parameter Ledger

0 free parameters · 0 axioms · 0 invented entities

Abstract provides no explicit free parameters, axioms, or invented entities; the work applies established RMAB and DFL techniques to a new setting with empirical validation.

pith-pipeline@v0.9.0 · 5587 in / 1078 out tokens · 30492 ms · 2026-05-10T19:14:54.517218+00:00 · methodology

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Lean theorems connected to this paper

Citations machine-checked in the Pith Canon. Every link opens the source theorem in the public Lean library.

What do these tags mean?
matches
The paper's claim is directly supported by a theorem in the formal canon.
supports
The theorem supports part of the paper's argument, but the paper may add assumptions or extra steps.
extends
The paper goes beyond the formal theorem; the theorem is a base layer rather than the whole result.
uses
The paper appears to rely on the theorem as machinery.
contradicts
The paper's claim conflicts with a theorem or certificate in the canon.
unclear
Pith found a possible connection, but the passage is too broad, indirect, or ambiguous to say the theorem truly supports the claim.

Reference graph

Works this paper leans on

43 extracted references · 43 canonical work pages

  1. [1]

    Are you still taking iron pills after delivery?

    “Are you still taking iron pills after delivery?”

    work page

  2. [2]

    Are you still taking calcium pills after delivery?

    “Are you still taking calcium pills after delivery?”

    work page

  3. [3]

    What was the baby’s weight at birth?

    “What was the baby’s weight at birth?”, there is statistically significant im- provement in score in the intervention group. Table 5.2 shows the scores and p-value for the three questions across control and intervention groups. The improved understanding and continued use of postnatal iron and calcium supplements for mothers in the intervention group is e...

    work page

  4. [4]

    Group maintenance: A restless bandits ap- proach.INFORMS J

    Abderrahmane Abbou and Viliam Makis. Group maintenance: A restless bandits ap- proach.INFORMS J. Comput., 31(4):719–731, 2019

    work page 2019

  5. [5]

    ARMMAN. Assessing the impact of mobile-based intervention on health literacy among pregnant women in urban india.https://armman.org/wp-content/uploads/ 2019/09/Sion-Study-Abstract.pdf, 2019. Accessed: 2022-08-12

    work page 2019

  6. [6]

    ARMMAN helping mothers and children.https://armman.org/, 2022

    ARMMAN. ARMMAN helping mothers and children.https://armman.org/, 2022. Accessed: 2022-05-19

    work page 2022

  7. [7]

    Prioritizing hepatitis c treatment in us prisons.Operations Research, 67(3):853– 873, 2019

    Turgay Ayer, Can Zhang, Anthony Bonifonte, Anne C Spaulding, and Jagpreet Chhat- wal. Prioritizing hepatitis c treatment in us prisons.Operations Research, 67(3):853– 873, 2019

    work page 2019

  8. [8]

    A decision-language model (dlm) for dynamic restless multi-armed bandit tasks in public health.Advances in Neural Information Processing Systems, 37:3964–4002, 2024

    Nikhil Behari, Edwin Zhang, Yunfan Zhao, Aparna Taneja, Dheeraj Nagaraj, and Milind Tambe. A decision-language model (dlm) for dynamic restless multi-armed bandit tasks in public health.Advances in Neural Information Processing Systems, 37:3964–4002, 2024

    work page 2024

  9. [9]

    Optimizing vital sign mon- itoring in resource-constrained maternal care: An rl-based restless bandit approach

    Niclas Boehmer, Yunfan Zhao, Guojun Xiong, Paula Rodriguez-Diaz, Paola Del Cueto Cibrian, Joseph Ngonzi, Adeline Boatin, and Milind Tambe. Optimizing vital sign mon- itoring in resource-constrained maternal care: An rl-based restless bandit approach. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 39, pages 28843– 28849, 2025

    work page 2025

  10. [10]

    Heart failure patient adherence: epidemiology, cause, and treatment.Heart failure clinics, 9(1):49–58, 2013

    Paul S Corotto, Melissa M McCarey, Suzanne Adams, Prateeti Khazanie, and David J Whellan. Heart failure patient adherence: epidemiology, cause, and treatment.Heart failure clinics, 9(1):49–58, 2013

    work page 2013

  11. [11]

    Beyond listenership: Ai-predicted interventions drive improvements in maternal health behaviours.arXiv preprint arXiv:2507.20755, 2025

    Arpan Dasgupta, Sarvesh Gharat, Neha Madhiwalla, Aparna Hegde, Milind Tambe, and Aparna Taneja. Beyond listenership: Ai-predicted interventions drive improvements in maternal health behaviours.arXiv preprint arXiv:2507.20755, 2025

    work page arXiv 2025

  12. [12]

    Use of mobile health (mhealth) technologies and interventions among community health workers globally: a scoping review.Health promotion practice, 20(6):805–817, 2019

    Jody Early, Carmen Gonzalez, Vanessa Gordon-Dseagu, and Laura Robles-Calderon. Use of mobile health (mhealth) technologies and interventions among community health workers globally: a scoping review.Health promotion practice, 20(6):805–817, 2019

    work page 2019

  13. [13]

    The law of attrition.J Med Internet Res, 7(1):e11, Mar 2005

    Gunther Eysenbach. The law of attrition.J Med Internet Res, 7(1):e11, Mar 2005

    work page 2005

  14. [14]

    The challenge of global health

    Laurie Garrett. The challenge of global health. InGlobal Health, pages 525–548. Routledge, 2017

    work page 2017

  15. [15]

    Factors in- fluencing adherence to mhealth apps for prevention or management of noncommunica- ble diseases: Systematic review.J Med Internet Res, 24(5):e35371, May 2022

    Robert Jakob, Samira Harperink, Aaron Maria Rudolf, Elgar Fleisch, Severin Haug, Jacqueline Louise Mair, Alicia Salamanca-Sanabria, and Tobias Kowatsch. Factors in- fluencing adherence to mhealth apps for prevention or management of noncommunica- ble diseases: Systematic review.J Med Internet Res, 24(5):e35371, May 2022

    work page 2022

  16. [16]

    Effectiveness of information technology–enabled ‘smart eating’ health pro- motion intervention: A cluster randomized controlled trial.PLOS ONE, 15:e0225892, 01 2020

    Jasvir Kaur, Manmeet Kaur, Venkatesan Chakrapani, Jacqui Webster, Joseph Santos, and Raj Kumar. Effectiveness of information technology–enabled ‘smart eating’ health pro- motion intervention: A cluster randomized controlled trial.PLOS ONE, 15:e0225892, 01 2020

    work page 2020

  17. [17]

    A call to action: the global failure to effectively tackle maternal mortality rates.The Lancet Global Health, 11(8):e1165–e1167, 2023

    Asma Khalil, Athina Samara, Pat O’Brien, Conrado Milani Coutinho, Silvana Maria Quintana, and Shamez N Ladhani. A call to action: the global failure to effectively tackle maternal mortality rates.The Lancet Global Health, 11(8):e1165–e1167, 2023

    work page 2023

  18. [18]

    Killian, Bryan Wilder, Amit Sharma, Vinod Choudhary, Bistra Dilkina, and Milind Tambe

    Jackson A. Killian, Bryan Wilder, Amit Sharma, Vinod Choudhary, Bistra Dilkina, and Milind Tambe. Learning to prescribe interventions for tuberculosis patients using digital adherence data.Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, Jul 2019

    work page 2019

  19. [19]

    Peng Liao, Kristjan Greenewald, Predrag Klasnja, and Susan Murphy. Personalized heartsteps: A reinforcement learning algorithm for optimizing physical activity.Pro- Verma et al.: SAHELI Project17 ceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies, 4(1):1–22, 2020

    work page 2020

  20. [20]

    mhealth and big-data integration: promises for healthcare system in india.BMJ health & care informatics, 26(1), 2019

    Samaneh Madanian, Dave T Parry, David Airehrour, and Marianne Cherrington. mhealth and big-data integration: promises for healthcare system in india.BMJ health & care informatics, 26(1), 2019

    work page 2019

  21. [21]

    Field study in deploy- ing restless multi-armed bandits: Assisting non-profits in improving maternal and child health

    Aditya Mate, Lovish Madaan, Aparna Taneja, Neha Madhiwalla, Shresth Verma, Gargi Singh, Aparna Hegde, Pradeep Varakantham, and Milind Tambe. Field study in deploy- ing restless multi-armed bandits: Assisting non-profits in improving maternal and child health. InProceedings of the AAAI Conference on Artificial Intelligence, volume 36, pages 12017–12025, 2022

    work page 2022

  22. [22]

    Rates of attrition and dropout in app-based interventions for chronic disease: Systematic review and meta-analysis.J Med Internet Res, 22(9):e20283, Sep 2020

    Gideon Meyerowitz-Katz, Sumathy Ravi, Leonard Arnolda, Xiaoqi Feng, Glen Maberly, and Thomas Astell-Burt. Rates of attrition and dropout in app-based interventions for chronic disease: Systematic review and meta-analysis.J Med Internet Res, 22(9):e20283, Sep 2020

    work page 2020

  23. [23]

    Markovian restless bandits and index policies: A review.Mathematics, 11(7):1639, 2023

    Jos ´e Ni˜no-Mora. Markovian restless bandits and index policies: A review.Mathematics, 11(7):1639, 2023

    work page 2023

  24. [24]

    mhealth intervention to improve diabetes risk behaviors in in- dia: A prospective, parallel group cohort study.Journal of Medical Internet Research, 18:e207, 08 2016

    Angela Pfammatter, Bonnie Spring, Nalini Saligram, Raj Dav ´e, Arun Gowda, Linelle Blais, Monika Arora, Harish Ranjani, Om Ganda, Donald Hedeker, Sethu Reddy, and Sandhya Ramalingam. mhealth intervention to improve diabetes risk behaviors in in- dia: A prospective, parallel group cohort study.Journal of Medical Internet Research, 18:e207, 08 2016

    work page 2016

  25. [25]

    Tulsky, Andrew R

    Louise Pilote, Jacqueline P. Tulsky, Andrew R. Zolopa, Judith A. Hahn, Gisela F. Schecter, and Andrew R. Moss. Tuberculosis Prophylaxis in the Homeless: A Trial to Improve Adherence to Referral.Archives of Internal Medicine, 156(2):161–165, 01 1996

    work page 1996

  26. [26]

    Pearl: A mobile robotic assistant for the elderly

    Martha E Pollack, Laura Brown, Dirk Colbry, Cheryl Orosz, Bart Peintner, Sailesh Ra- makrishnan, Sandra Engberg, Judith T Matthews, Jacqueline Dunbar-Jacob, Colleen E McCarthy, et al. Pearl: A mobile robotic assistant for the elderly. InAAAI workshop on automation as eldercare, volume 2002. AAAI, 2002, Edmonton, Alberta, Canada, 2002

    work page 2002

  27. [27]

    Restless poach- ers: Handling exploration-exploitation tradeoffs in security domains

    Yundi Qian, Chao Zhang, Bhaskar Krishnamachari, and Milind Tambe. Restless poach- ers: Handling exploration-exploitation tradeoffs in security domains. In Catholijn M. Jonker, Stacy Marsella, John Thangarajah, and Karl Tuyls, editors,Proceedings of the 2016 International Conference on Autonomous Agents & Multiagent Systems, Singa- pore, May 9-13, 2016, pa...

    work page 2016

  28. [28]

    Decision- focused learning without decision-making: Learning locally optimized decision losses

    Sanket Shah, Kai Wang, Bryan Wilder, Andrew Perrault, and Milind Tambe. Decision- focused learning without decision-making: Learning locally optimized decision losses. InAdvances in Neural Information Processing Systems, 2022

    work page 2022

  29. [29]

    Application of support vector machine for prediction of medication adherence in heart failure patients.Healthcare informatics research, 16(4):253–259, 2010

    Youn-Jung Son, Hong-Gee Kim, Eung-Hee Kim, Sangsup Choi, and Soo-Kyoung Lee. Application of support vector machine for prediction of medication adherence in heart failure patients.Healthcare informatics research, 16(4):253–259, 2010

    work page 2010

  30. [30]

    Michelle Stanton, Andrew Molineux, Charles Mackenzie, Louise Kelly-Hope, et al. Mo- bile technology for empowering health workers in underserved communities: new ap- proaches to facilitate the elimination of neglected tropical diseases.JMIR public health and surveillance, 2(1):e5064, 2016

    work page 2016

  31. [31]

    The indian telecom services performance indi- cators: April–june, 2025, September 2025

    Telecom Regulatory Authority of India. The indian telecom services performance indi- cators: April–june, 2025, September 2025. Accessed: 2026-01-26

    work page 2025

  32. [32]

    Fumaz, Ramon Bay ´es, Roger Paredes, David M

    Albert Tuldr `a, Ma Jos ´e Ferrer, Carmina R. Fumaz, Ramon Bay ´es, Roger Paredes, David M. Burger, and Bonaventura Clotet. Monitoring Adherence to HIV Therapy. 18RMAB for Maternal Health Archives of Internal Medicine, 159(12):1376–1377, 06 1999

    work page 1999

  33. [33]

    Decision-focused evaluation: Analyzing performance of deployed restless multi-arm bandits

    Paritosh Verma, Shresth Verma, Aditya Mate, Aparna Taneja, and Milind Tambe. Decision-focused evaluation: Analyzing performance of deployed restless multi-arm bandits. InProceedings of the International Conference on Autonomous Agents and Multiagent Systems (AAMAS), volume 22, 2023

    work page 2023

  34. [34]

    Leverag- ing ai to improve health information access in the world’s largest maternal mobile health program.AI Magazine, 45(4):526–536, 2024

    Shresth Verma, Arshika Lalan, Paula Rodriguez Diaz, Panayiotis Danassis, Amrita Ma- hale, Kumar Madhu Sudan, Aparna Hegde, Milind Tambe, and Aparna Taneja. Leverag- ing ai to improve health information access in the world’s largest maternal mobile health program.AI Magazine, 45(4):526–536, 2024

    work page 2024

  35. [35]

    Restless multi-armed bandits for maternal and child health: Results from decision-focused learning

    Shresth Verma, Aditya Mate, Kai Wang, Neha Madhiwalla, Aparna Hegde, Aparna Taneja, and Milind Tambe. Restless multi-armed bandits for maternal and child health: Results from decision-focused learning. InProceedings of the 2023 International Con- ference on Autonomous Agents and Multiagent Systems, pages 1312–1320, 2023

    work page 2023

  36. [36]

    Increasing impact of mobile health programs: Saheli for maternal and childcare

    Shresth Verma, Gargi Singh, Aditya Mate, Paritosh Verma, Sruthi Gorantala, Neha Mad- hiwalla, Aparna Hegde, Divy Thakkar, Manish Jain, Milind Tambe, and Aparna Taneja. Increasing impact of mobile health programs: Saheli for maternal and childcare. In Innovative Applications of Artificial Intelligence (IAAI), 2023

    work page 2023

  37. [37]

    Increasing impact of mobile health programs: Saheli for maternal and child care

    Shresth Verma, Gargi Singh, Aditya Mate, Paritosh Verma, Sruthi Gorantla, Neha Mad- hiwalla, Aparna Hegde, Divy Thakkar, Manish Jain, Milind Tambe, et al. Increasing impact of mobile health programs: Saheli for maternal and child care. InProceedings of the aaai conference on artificial intelligence, volume 37, pages 15594–15602, 2023

    work page 2023

  38. [38]

    Scalable game-focused learning of adversary models: Data-to-decisions in network security games

    Kai Wang, Andrew Perrault, Aditya Mate, and Milind Tambe. Scalable game-focused learning of adversary models: Data-to-decisions in network security games. InAAMAS, pages 1449–1457, 2020

    work page 2020

  39. [39]

    Learning mdps from features: Predict-then-optimize for sequential de- cision making by reinforcement learning.Advances in Neural Information Processing Systems, 34, 2021

    Kai Wang, Sanket Shah, Haipeng Chen, Andrew Perrault, Finale Doshi-Velez, and Milind Tambe. Learning mdps from features: Predict-then-optimize for sequential de- cision making by reinforcement learning.Advances in Neural Information Processing Systems, 34, 2021

    work page 2021

  40. [40]

    Scalable decision-focused learning in restless multi-armed bandits with application to maternal and child health

    Kai Wang*, Shresth Verma*, Aditya Mate, Sanket Shah, Aparna Taneja, Neha Madhi- walla, Aparna Hegde, and Milind Tambe. Scalable decision-focused learning in restless multi-armed bandits with application to maternal and child health. InProceedings of the AAAI Conference on Artificial Intelligence (AAAI), 2023

    work page 2023

  41. [41]

    Restless bandits: Activity allocation in a changing world.Journal of applied probability, 25(A):287–298, 1988

    Peter Whittle. Restless bandits: Activity allocation in a changing world.Journal of applied probability, 25(A):287–298, 1988

    work page 1988

  42. [42]

    Melding the data-decisions pipeline: Decision-focused learning for combinatorial optimization

    Bryan Wilder, Bistra Dilkina, and Milind Tambe. Melding the data-decisions pipeline: Decision-focused learning for combinatorial optimization. InProceedings of the AAAI Conference on Artificial Intelligence, volume 33, pages 1658–1665, 2019

    work page 2019

  43. [43]

    Towards a pretrained model for restless ban- dits via multi-arm generalization.arXiv preprint arXiv:2310.14526, 2023

    Yunfan Zhao, Nikhil Behari, Edward Hughes, Edwin Zhang, Dheeraj Nagaraj, Karl Tuyls, Aparna Taneja, and Milind Tambe. Towards a pretrained model for restless ban- dits via multi-arm generalization.arXiv preprint arXiv:2310.14526, 2023

    work page arXiv 2023