Pith. sign in

REVIEW 5 major objections 5 minor 56 references

Adaptive Robot-Assisted Feeding: An Online Learning Framework for Acquiring Previously Unseen Food Items

T0 review · 5 major / 5 minor · reviewed 2026-08-14 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read Contextual bandit algorithms (epsilon-greedy and LinUCB) with SPANet features let a feeding robot acquire previously unseen foods after about ten attempts per food.

desk verdict A clean bandit framework for adaptive feeding, but the paper's headline convergence claim is not actually backed by the plotted data. read the letter →

arxiv 1908.07088 v4 pith:4DUFRMCD submitted 2019-08-19 cs.RO cs.AIcs.LG

Ethan K. Gordon , Xiang Meng , Matt Barnes , Tapomayukh Bhattacharjee , Siddhartha S. Srinivasa This is my paper · ORCID
classification cs.ROcs.AIcs.LG
keywords fooditemsdifferentpreviouslyfeedingrobot-assistedsuccessfulacquisition
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper tackles a practical problem: a robot arm that helps people eat needs to pick up foods it may never have seen, from banana slices to grapes. Different foods require different skewering angles, and trying every strategy on every meal is slow and frustrating. The authors frame this as a 'contextual bandit' problem, a standard machine learning setup where at each step the robot sees an image of the food (the context), chooses one of six skewering strategies (the action), and gets a single bit of feedback: success or failure. Over time it should learn which strategy works best for each type of food.

The key idea is to reuse an existing neural network, SPANet, that was already trained on thousands of skewering attempts on other foods. The network's internal features serve as a compact description of each food. The robot then runs a simple linear learning rule on top of those features, using either random exploration (epsilon-greedy) or an uncertainty-aware rule (LinUCB). To avoid wasting robot time, the authors tune the small number of algorithm knobs in a simulation built from prior data, using a statistical trick called doubly robust estimation to correct for biased data.

In real-robot experiments, all algorithms converged to the best strategy within roughly ten failed attempts per new food. LinUCB was the most stable, especially when starting from a model pretrained on dissimilar foods. The authors are careful to note this works for discrete, solid foods and a limited action space; realistic plates with mixed or soft foods remain future work.

Extended reading notes

Core claim

The paper claims that, even starting with a model trained on thousands of skewering attempts on dissimilar previously seen food items, epsilon-greedy and LinUCB algorithms can quickly converge to the most successful manipulation strategy for previously unseen food items. The abstract states: 'we demonstrate empirically on a robot-assisted feeding system that, even starting with a model trained on thousands of skewering attempts on dissimilar previously seen food items, epsilon-greedy and LinUCB algorithms can quickly converge to the most successful manipulation strategy.' Section V adds that 'this convergence could happen within 10 attempts, even if the previously unseen food requires a completely new acquisition strategy.'

Load-bearing premise

The load-bearing premise is that the expected loss of each action is a linear function of the SPANet penultimate-layer features, and that this linear structure remains valid for foods whose optimal strategies are very different from the training distribution. This enters in Section III-B: 'justified by the success of SPANet, we assume a linear map from the Rd features to the expected cost of each action'. If linear realizability fails for a given food, the weighted least-squares oracle updates a linear theta that cannot represent the true action-value differences, so no amount of online data will converge to the correct policy.

Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

5 major / 5 minor

Summary. The paper proposes an online learning framework for robot-assisted feeding in which bite acquisition is cast as a contextual bandit problem. Food images are featurized by the penultimate layer of SPANet, and the expected loss of each of six manipulation strategies is modeled as a linear function of those features. The authors use an importance-weighted linear regression oracle, test epsilon-greedy and LinUCB exploration strategies, tune hyperparameters in a simulated environment built from prior data with a doubly robust loss estimator, and then evaluate on a real robot. Experiment 1 starts from a zero initial theta on apple, banana, and grape; Experiment 2 starts from a pretrained theta and tests banana and carrot. The central claim is that, even for previously unseen food items requiring very different strategies, the algorithms converge to the best strategy within about 10 attempts.

Significance. If validated, the framework would be a useful step toward data-efficient adaptation in assistive feeding, reducing the need to pre-collect food-specific training data. The paper has genuine strengths: the contextual-bandit formulation is clean, the use of a doubly robust estimator to build a simulation environment is appropriate, the algorithms are standard and clearly described, and the real-robot evaluation addresses a practically important problem. The main limitation is that the strongest quantitative claim, convergence within about 10 attempts for previously unseen foods, is not directly supported by the plotted real-robot data, and the experimental evidence is thin in several load-bearing places. The paper is therefore plausible but needs substantial strengthening of the empirical support before the central claim can be accepted.

major comments (5)
  1. [Section V / Figure 7] The claim that both experiments show convergence 'within 10 attempts' is not supported by the plotted Experiment 2 data. Figure 7 shows cumulative loss only for attempts 20 through 30, and the caption states that attempts 20 through 25 were performed on a previously seen food item. The first 20 attempts on the novel food, which is exactly the window in which the 'within 10 attempts' convergence would need to appear, are omitted from both the figure and the quantitative discussion. Section V's statement that 'both experiments suggest that this convergence could happen within 10 attempts' therefore overreaches; at most the data show stable behavior after attempt 25 on banana and after some unreported point on carrot.
  2. [Section IV-A / Figure 5] The hyperparameters (epsilon, alpha, lambda, and d) are tuned in simulation using banana, apple, and grape as the excluded foods, and the same three foods are then used in the real-robot Experiment 1. This means those foods are not genuinely 'previously unseen' at the system level: information about their success-rate structure has influenced the choice of exploration parameters and regularization. The abstract's claim about adapting to previously unseen foods is weakened because the tuner had access to the test foods. A cleaner protocol would tune on a separate held-out set of foods and reserve the three test foods entirely, or would report sensitivity of the real-robot results to the chosen hyperparameters.
  3. [Section III-B / Equation (2)] The linear realizability assumption, E[ct | at, xt] = theta_a^T phi(xt), is load-bearing: if the true expected loss is not linear in the SPANet features for an out-of-distribution food, no amount of online data can make the weighted least-squares oracle converge to the correct policy. The paper justifies this only by citing SPANet's success, but the entire point of the paper is to handle foods whose optimal strategies differ from the training distribution. The authors should provide a direct check, for example residual or calibration plots on held-out food data, or a comparison of the linear model against a nonparametric estimate for the foods used in the experiments. Without such evidence, the convergence claims rest on an untested functional form.
  4. [Section IV-B / Figures 6 and 7] The experimental section does not describe repeated trials for either experiment, despite Figure 7 displaying a '3-Trial Range' that is never explained in the text. If the range is over multiple runs, the number of runs and the protocol should be stated. If it is a within-run quantity such as a moving window, that should be clarified. As written, the reader cannot determine whether the reported cumulative-loss curves are single trajectories or averages, and the paper's comparative statements about greedy, epsilon-greedy, and LinUCB are therefore not backed by statistical evidence.
  5. [Algorithm 1 / Section III-A] Because the context is resampled only after a success, failures repeat the same context. In Experiment 2, where each trial uses a single food item and failures dominate early attempts, the algorithm may appear to converge for that specific food item without demonstrating any contextual generalization to unseen foods. This is worth acknowledging explicitly; as presented, the evidence supports per-item adaptation more strongly than it supports generalization across novel food categories.
minor comments (5)
  1. [Section III-B / Equation (2)] Equation (2) is written as a squared-loss objective but is not a complete definition of the weighted least-squares estimate; the notation should make clear that theta_hat minimizes this expression over all actions and that the sum runs over collected samples, not over the feature dimension.
  2. [Figure 6] The vertical line in Figure 6 is described only in the caption as the point after which 100% of strategies selected were among the best strategies. The text should define what 'best strategy set' means formally and explain how the line is computed, since the convergence claim in Section V depends on this definition.
  3. [Figure 7] The x-axis labels in Figure 7 read 'Banana Acquisition Attempt' and 'Carrot Acquisition Attempt', but the plotted range is 20-30 and includes five attempts on a different previously seen food. The axis and caption should be clarified to distinguish the test-food attempt index from the total attempt index.
  4. [Section IV-B(b)] The procedure states that 'we removed and replaced the food item only after a successful acquisition,' but Experiment 2's protocol of 20 attempts followed by 5 attempts on a previously seen food and then 5 more attempts is not described as repeated. Please clarify whether the food item is replaced after every attempt or only after success, and how the fixed 20-attempt count is maintained.
  5. [References] LinUCB is cited in different forms across the paper: [10] is used for the UCB formula, [45] for the original LinUCB paper, and [50] for the regret bound. Unifying these citations would help the reader trace the exact algorithm and guarantees being used.

Circularity Check

1 steps flagged · score 4.0 of 10

The 'previously unseen' foods in Experiment 1 are the same banana/apple/grape items used to tune the bandit hyperparameters in simulation, so part of the convergence claim is a fitted-input result.

  1. fitted input called prediction [Section IV-A 'Tuning in Simulation' and Section IV-B(c) 'Experiment 1'; abstract and Section V claim]
    "We tune these by constructing a simulated training environment using the data from [7]. Specifically, we exclude from SPANet three food items with very different success rate distributions over strategies. ... [Fig. 5 caption:] Hyper-parameter tuning in simulation, with banana, apple, and grapes excluded from SPANet. ... We cycle through 3 food items (apple, banana, then grape) 20 times, leading to 60 total attempts."

    The exploration hyperparameters (epsilon, alpha, lambda, d) are fit in simulation on the same three foods that are later presented as 'previously unseen' foods in Experiment 1. Consequently, the real-robot convergence on apple, banana, and grape is not an out-of-sample prediction of the framework's ability to handle unseen foods; the test foods were part of the tuning input. The abstract's claim that the algorithms 'quickly converge' when starting from a model trained on dissimilar previously seen items is therefore partially a statement about fitted inputs. The carrot condition of Experiment 2 is not among the tuned foods and provides independent evidence, but the paper's general 'within 10 attempts' claim does not isolate that condition.

full rationale

The paper's online-learning derivation is otherwise self-contained: SPANet provides image features, the linear-cost map is explicitly stated as an assumption justified by SPANet's architecture, and the real-robot experiments collect new bandit feedback that is not used in the prior training. The self-citation to [7] for SPANet and the training dataset is not load-bearing circularity because the framework's central convergence claim is evaluated on new physical robot attempts rather than derived from [7] alone. The main circularity concern is the overlap between the simulation-tuning foods and the Experiment 1 'unseen' test foods, which breaks the independence of that demonstration; the skeptic's additional observation that Experiment 2's cumulative-loss plots start at attempt 20 is an evidentiary gap (the claimed 10-attempt convergence is not plotted) rather than a circularity, and is noted here for completeness. Overall, there is partial contamination but not a derivation that reduces by construction, so the appropriate score is moderate.

Assumptions & free parameters 5 free parameters · 4 assumptions · 0 invented entities

The central claim rests on a small set of tuned hyperparameters (lambda, d, epsilon, alpha) and on the linear realizability of action costs in the SPANet feature space. No new physical entities are introduced. The main assumptions are modeling choices about the feature representation and the feedback protocol, plus the uniform-propensity condition for the doubly robust estimator.

free parameters (5)
  • L2 regularization lambda = 100
    Tuned in simulation (Figure 5a) to balance bias and variance for the linear regression oracle; lambda=100 chosen over 1000 based on greedy cumulative loss.
  • Feature dimension d = 2048
    Retained at SPANet's original dimension because reducing d reduced the best possible policy performance (pi*), despite improving regret bounds.
  • Exploration parameter epsilon = 0.1
    Tuned in simulation for epsilon-greedy, chosen at a local minimum in cumulative loss (Figure 5b).
  • LinUCB width alpha = 0.01
    Tuned in simulation, reached a slight minimum loss across a range of alpha values (Figure 5c).
  • Success criterion thresholds = 5 seconds; 2 of 4 tines
    Manual thresholds used to define binary success/failure in real-robot experiments; chosen by hand and not varied.
assumptions (4)
  • domain assumption The expected loss of each action is a linear function of the SPANet penultimate-layer features: E[c_t | a_t, x_t] = theta_a^T phi(x_t).
    Assumed in Section III-B, justified by SPANet's final linear layer, but untested for foods with very different optimal strategies.
  • standard math The propensity score during data collection was uniform, p(a_i | x_i) = 1/6, making the doubly robust estimator unbiased.
    Stated in Section IV-A; relies on random uniform action selection in the original data collection of [7].
  • domain assumption Context resampling only occurs after a success; on failure the context is kept, providing effectively better-than-bandit feedback.
    This repeated-context protocol is stated in Algorithm 1 and discussed in Section III-C; it is a departure from standard contextual bandit assumptions and may improve convergence.
  • domain assumption SPANet features capture the visual and physical differences needed to distinguish strategies across food items.
    Core to the featurizer; Experiment 1 tests this for three foods, but it is not guaranteed for arbitrary unseen foods.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Adaptive Robot-Assisted Feeding: An Online Learning Framework for Acquiring Previously Unseen Food Items." pith.science (2026). https://pith.science/paper/4DUFRMCD

@misc{pith2026190807088,
  author       = {Pith},
  title        = {Pith review of: Adaptive Robot-Assisted Feeding: An Online Learning Framework for Acquiring Previously Unseen Food Items},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/4DUFRMCD}},
  note         = {Machine review of arXiv:1908.07088}
}
abstract

A successful robot-assisted feeding system requires bite acquisition of a wide variety of food items. It must adapt to changing user food preferences under uncertain visual and physical environments. Different food items in different environmental conditions require different manipulation strategies for successful bite acquisition. Therefore, a key challenge is how to handle previously unseen food items with very different success rate distributions over strategy. Combining low-level controllers and planners into discrete action trajectories, we show that the problem can be represented using a linear contextual bandit setting. We construct a simulated environment using a doubly robust loss estimate from previously seen food items, which we use to tune the parameters of off-the-shelf contextual bandit algorithms. Finally, we demonstrate empirically on a robot-assisted feeding system that, even starting with a model trained on thousands of skewering attempts on dissimilar previously seen food items, $\epsilon$-greedy and LinUCB algorithms can quickly converge to the most successful manipulation strategy.

Figures

Figures reproduced from arXiv: 1908.07088 by the authors.

Figure 1
Figure 1. When faced with new foods, a robot-assisted feeding system [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. Generalization results for SPANet on select food items using [PITH_FULL_IMAGE:figures/full_fig_p002_2.png] view at source ↗
Figure 3
Figure 3. SPANet out-of-class success rate using data from [7], given different [PITH_FULL_IMAGE:figures/full_fig_p002_3.png] view at source ↗
Figures from the paper (5 more)
Figure 4
Figure 4. Figure 4: Online learning framework. SPANet is trained on previously seen food items, and then all but the last layer is frozen as a featurizer. The final [PITH_FULL_IMAGE:figures/full_fig_p003_4.png]
Figure 5
Figure 5. Figure 5: Hyper-parameter tuning in simulation, with banana, apple, and grapes excluded from SPANet. ( [PITH_FULL_IMAGE:figures/full_fig_p005_5.png]
Figure 6
Figure 6. Figure 6: Results of Experiment 1 (Left) using the Autonomous Dexterous Arm (ADA) (Right). SPANet was trained on 12 food types – excluding apples, bananas, and grapes – 3 types of food with significantly different success rate distributions over strategy. Initialized to θ0 = ~0,…
Figure 7
Figure 7. Figure 7: Empirical cumulative loss for each contextual bandit algorithm on an unseen food item when [PITH_FULL_IMAGE:figures/full_fig_p006_7.png]
Figure 8
Figure 8. Figure 8: Evolution of the internal upper confidence bound (UCB) estimate of the success rate of each action over time for one of the banana experiments. [PITH_FULL_IMAGE:figures/full_fig_p006_8.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

56 extracted references · 51 canonical work pages

  1. [1]

    Americans with disabilities: 2010,

    M. W. Brault, “Americans with disabilities: 2010,” Current population reports, vol. 7, pp. 70–131, 2012

    work page 2010

  2. [2]

    An electric wheelchair mounted robotic arm-a survey of potential users,

    S. D. Prior, “An electric wheelchair mounted robotic arm-a survey of potential users,” Journal of medical engineering & technology, vol. 14, no. 4, pp. 143–154, 1990

    work page 1990

  3. [3]

    Devices for assisting manipulation: a summary of user task priorities,

    C. A. Stanger, C. Anglin, W. S. Harwin, and D. P. Romilly, “Devices for assisting manipulation: a summary of user task priorities,” IEEE Transactions on rehabilitation Engineering, vol. 2, no. 4, pp. 256–265, 1994

    work page 1994

  4. [4]

    MySpoon, 2018, https://www.secom.co.jp/english/myspoon/food.html

    work page 2018

  5. [5]

    Obi, 2018, https://meetobi.com/

    work page 2018

  6. [6]

    Towards robotic feeding: Role of haptics in fork-based food manipulation,

    T. Bhattacharjee, G. Lee, H. Song, and S. S. Srinivasa, “Towards robotic feeding: Role of haptics in fork-based food manipulation,” IEEE Robotics and Automation Letters , 2019

    work page 2019

  7. [7]

    Robot-assisted feeding: Gener- alizing skewering strategies across food items on a realistic plate,

    R. Feng, Y . Kim, G. Lee, E. K. Gordon, M. Schmittle, S. Kumar, T. Bhattacharjee, and S. S. Srinivasa, “Robot-assisted feeding: Gener- alizing skewering strategies across food items on a realistic plate,” in International Symposium on Robotics Research , 2019

    work page 2019

  8. [8]

    Transfer depends on acquisition: Analyzing manipulation strategies for robotic feeding,

    D. Gallenberger, T. Bhattacharjee, Y . Kim, and S. Srinivasa, “Transfer depends on acquisition: Analyzing manipulation strategies for robotic feeding,” ACM/IEEE International Conference on Human-Robot In- teraction, 2019

    work page 2019

Show all 56 references
  1. [9]

    A contextual bandit bake-off,

    A. Bietti, A. Agarwal, and J. Langford, “A contextual bandit bake-off,” arXiv preprint 1802.04064 , Feb. 2018

    2018 arXiv

  2. [10]

    Contextual bandits with linear payoff functions,

    C. Wei, L. Li, L. Reyzin, and R. E. Schapire, “Contextual bandits with linear payoff functions,” inProceedings of the Fourteenth International Conference on Artificial Intelligence and Statistics, 2011, pp. 208–214

    2011

  3. [11]

    Robotic manipulation of food products–a review,

    P. Chua, T. Ilschner, and D. Caldwell, “Robotic manipulation of food products–a review,” Industrial Robot: An International Journal , vol. 30, no. 4, pp. 345–354, 2003

    2003

  4. [12]

    Meeting the need for robotic handling of food products,

    F. Erzincanli and J. Sharp, “Meeting the need for robotic handling of food products,” Food Control, vol. 8, no. 4, pp. 185–190, 1997

    1997

  5. [13]

    Soft robotic manipulation of onions and artichokes in the food industry,

    R. Morales, F. Badesa, N. Garcia-Aracil, J. Sabater, and L. Zollo, “Soft robotic manipulation of onions and artichokes in the food industry,” Advances in Mechanical Engineering , vol. 6, p. 345291, 2014

    2014

  6. [14]

    Research towards generalised robotic systems for handling non-rigid products,

    P. Brett, A. Shacklock, and K. Khodabendehloo, “Research towards generalised robotic systems for handling non-rigid products,” in Inter- national Conference on Advanced Robotics . IEEE, 1991, pp. 1530– 1533

    1991

  7. [15]

    Teaching from examples in assembly and manipulation of snack food ingredients by robot,

    T. G. Williams, J. J. Rowland, and M. H. Lee, “Teaching from examples in assembly and manipulation of snack food ingredients by robot,” in IEEE/RSJ International Conference on Intelligent Robots and Systems, vol. 4. IEEE, 2001, pp. 2300–2305

    2001

  8. [16]

    Technologies for robot grippers in pick and place operations for fresh fruits and vegetables,

    C. Blanes, M. Mellado, C. Ortiz, and A. Valera, “Technologies for robot grippers in pick and place operations for fresh fruits and vegetables,” Spanish Journal of Agricultural Research , vol. 9, no. 4, pp. 1130–1141, 2011

    2011

  9. [17]

    Inspection and grading of agricultural and food products by computer vision systems–a review,

    T. Brosnan and D.-W. Sun, “Inspection and grading of agricultural and food products by computer vision systems–a review,” Computers and Electronics in Agriculture , vol. 36, no. 2, pp. 193–213, 2002

    2002

  10. [18]

    Learning techniques used in computer vision for food quality evaluation: a review,

    C.-J. Du and D.-W. Sun, “Learning techniques used in computer vision for food quality evaluation: a review,” Journal of food engineering , vol. 72, no. 1, pp. 39–55, 2006

    2006

  11. [19]

    Shape feature extraction and classifi- cation of food material using computer vision,

    K. Ding and S. Gunasekaran, “Shape feature extraction and classifi- cation of food material using computer vision,” Transactions of the ASAE, vol. 37, no. 5, pp. 1537–1545, 1994

    1994

  12. [20]

    A chinese cooking robot for elderly and disabled people,

    W.-T. Ma, W.-X. Yan, Z. Fu, and Y .-Z. Zhao, “A chinese cooking robot for elderly and disabled people,” Robotica, vol. 29, no. 6, pp. 843–852, 2011

    2011

  13. [21]

    Cooking with robots: designing a household system working in open environments,

    Y . Sugiura, D. Sakamoto, A. Withana, M. Inami, and T. Igarashi, “Cooking with robots: designing a household system working in open environments,” in Proceedings of the SIGCHI Conference on Human Factors in Computing Systems . ACM, 2010, pp. 2427–2430

    2010

  14. [22]

    Bakebot: Baking cookies with the pr2,

    M. Bollini, J. Barry, and D. Rus, “Bakebot: Baking cookies with the pr2,” in The PR2 workshop: results, challenges and lessons learned in advancing robots with a common platform, IROS , 2011

    2011

  15. [23]

    Robotic roommates mak- ing pancakes,

    M. Beetz, U. Klank, I. Kresse, A. Maldonado, L. M ¨osenlechner, D. Pangercic, T. R ¨uhr, and M. Tenorth, “Robotic roommates mak- ing pancakes,” in IEEE-RAS International Conference on Humanoid Robots. IEEE, 2011, pp. 529–536

    2011

  16. [24]

    Oreo separator machines,

    “Oreo separator machines,” https://vimeo.com/63347829,[Online; Re- trieved on 1st February, 2018]

    2018

  17. [25]

    Learning haptic representation for manipulating deformable food objects,

    M. C. Gemici and A. Saxena, “Learning haptic representation for manipulating deformable food objects,” in IEEE/RSJ International Conference on Intelligent Robots and Systems . IEEE, 2014, pp. 638– 645

    2014

  18. [26]

    Towards assistive feeding with a General-Purpose mobile manipulator,

    D. Park, Y . K. Kim, Z. M. Erickson, and C. C. Kemp, “Towards assistive feeding with a General-Purpose mobile manipulator,” arXiv preprint arXiv:1605.07996, May 2016

    2016 arXiv

  19. [27]

    Algorithms, Implementation, and Studies on Eating with a Shared Control Robot Arm,

    L. V . Herlant, “Algorithms, Implementation, and Studies on Eating with a Shared Control Robot Arm,” Ph.D. dissertation, The Robotics Institute Carnegie Mellon University, 2016

    2016

  20. [28]

    Automatic ad format selection via contextual bandits,

    L. Tang, R. Rosales, A. Singh, and D. Agarwal, “Automatic ad format selection via contextual bandits,” in Proceedings of the 22nd ACM international conference on Information & Knowledge Management . ACM, 2013, pp. 1587–1594

    2013

  21. [29]

    Counter- factual reasoning and learning systems: The example of computational advertising,

    L. Bottou, J. Peters, J. Qui ˜nonero-Candela, D. X. Charles, D. M. Chickering, E. Portugaly, D. Ray, P. Simard, and E. Snelson, “Counter- factual reasoning and learning systems: The example of computational advertising,” The Journal of Machine Learning Research , vol. 14, no. ...

    2013

  22. [30]

    Microrandomized trials: An experimen- tal design for developing just-in-time adaptive interventions

    P. Klasnja, E. B. Hekler, S. Shiffman, A. Boruvka, D. Almirall, A. Tewari, and S. A. Murphy, “Microrandomized trials: An experimen- tal design for developing just-in-time adaptive interventions.” Health Psychology, vol. 34, no. S, p. 1220, 2015

    2015

  23. [31]

    Encouraging physical activity in patients with diabetes through automatic personalized feedback via reinforcement learning improves glycemic control,

    I. Hochberg, G. Feraru, M. Kozdoba, S. Mannor, M. Tennenholtz, and E. Yom-Tov, “Encouraging physical activity in patients with diabetes through automatic personalized feedback via reinforcement learning improves glycemic control,” Diabetes care, vol. 39, no. 4, pp. e59– e60, 2016

    2016

  24. [32]

    Informing sequential clinical decision-making through reinforcement learning: an empirical study,

    S. M. Shortreed, E. Laber, D. J. Lizotte, T. S. Stroup, J. Pineau, and S. A. Murphy, “Informing sequential clinical decision-making through reinforcement learning: an empirical study,”Machine learning, vol. 84, no. 1-2, pp. 109–136, 2011

    2011

  25. [33]

    Adaptive routing with end-to- end feedback: Distributed learning and geometric approaches,

    B. Awerbuch and R. D. Kleinberg, “Adaptive routing with end-to- end feedback: Distributed learning and geometric approaches,” in Proceedings of the thirty-sixth annual ACM symposium on Theory of computing. ACM, 2004, pp. 45–53

    2004

  26. [34]

    Offline policy evaluation across representations with applications to educa- tional games,

    T. Mandel, Y .-E. Liu, S. Levine, E. Brunskill, and Z. Popovic, “Offline policy evaluation across representations with applications to educa- tional games,” in Proceedings of the 2014 international conference on Autonomous agents and multi-agent systems. International Foundatio...

    2014

  27. [35]

    Exploration in inter- active personalized music recommendation: a reinforcement learning approach,

    X. Wang, Y . Wang, D. Hsu, and Y . Wang, “Exploration in inter- active personalized music recommendation: a reinforcement learning approach,” ACM Transactions on Multimedia Computing, Communi- cations, and Applications (TOMM) , vol. 11, no. 1, p. 7, 2014

    2014

  28. [36]

    Portfolio choices with orthogonal bandit learning,

    W. Shen, J. Wang, Y .-G. Jiang, and H. Zha, “Portfolio choices with orthogonal bandit learning,” in Twenty-Fourth International Joint Conference on Artificial Intelligence , 2015

    2015

  29. [37]

    Robust trajectory selection for rearrangement planning as a multi-armed bandit problem,

    M. C. Koval, J. E. King, N. S. Pollard, and S. S. Srinivasa, “Robust trajectory selection for rearrangement planning as a multi-armed bandit problem,” in 2015 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) . IEEE, 2015, pp. 2678–2685

    2015

  30. [38]

    Online learning of robot soccer free kick plans using a bandit approach,

    J. P. Mendoza, R. Simmons, and M. Veloso, “Online learning of robot soccer free kick plans using a bandit approach,” in Twenty-Sixth International Conference on Automated Planning and Scheduling , 2016

    2016

  31. [39]

    Bandit-based model selection for deformable object manipulation,

    D. McConachie and D. Berenson, “Bandit-based model selection for deformable object manipulation,” arXiv preprint arXiv:1703.10254 , 2017

    2017 arXiv

  32. [40]

    Grocery stores carry 40,000 more items than they did in the 1990s,

    A. Malito, “Grocery stores carry 40,000 more items than they did in the 1990s,” 2017

    2017

  33. [41]

    Finite-time analysis of the multiarmed bandit problem,

    P. Auer, N. Cesa-Bianchi, and P. Fischer, “Finite-time analysis of the multiarmed bandit problem,” Machine learning, vol. 47, no. 2-3, pp. 235–256, 2002

    2002

  34. [42]

    The non- stochastic multiarmed bandit problem,

    P. Auer, N. Cesa-Bianchi, Y . Freund, and R. E. Schapire, “The non- stochastic multiarmed bandit problem,” SIAM journal on computing , vol. 32, no. 1, pp. 48–77, 2002

    2002

  35. [43]

    The epoch-greedy algorithm for multi- armed bandits with side information,

    J. Langford and T. Zhang, “The epoch-greedy algorithm for multi- armed bandits with side information,” in Advances in neural informa- tion processing systems , 2008, pp. 817–824

    2008

  36. [44]

    Mostly exploration-free algorithms for contextual bandits,

    H. Bastani, M. Bayati, and K. Khosravi, “Mostly exploration-free algorithms for contextual bandits,” arXiv preprint arXiv:1704.09011 , 2017

    2017 arXiv

  37. [45]

    A Contextual- Bandit approach to personalized news article recommendation,

    L. Li, W. Chu, J. Langford, and R. E. Schapire, “A Contextual- Bandit approach to personalized news article recommendation,” arXiv preprint arXiv:1003.0146, Feb. 2010

    2010 arXiv

  38. [46]

    Practical contextual bandits with regression oracles,

    D. J. Foster, A. Agarwal, M. Dud ´ık, H. Luo, and R. E. Schapire, “Practical contextual bandits with regression oracles,” arXiv preprint arXiv:1803.01088, Mar. 2018

    2018 arXiv

  39. [47]

    Taming the monster: A fast and simple algorithm for contextual bandits,

    A. Agarwal, D. Hsu, S. Kale, J. Langford, L. Li, and R. E. Schapire, “Taming the monster: A fast and simple algorithm for contextual bandits,” arXiv preprint arXiv:1402.0555 , Feb. 2014

    2014 arXiv

  40. [48]

    Lattimore and C

    T. Lattimore and C. Szepesvari, Bandit Algorithms . Cambridge University Press, 2019

    2019

  41. [49]

    Focal loss for dense object detection,

    T. Lin, P. Goyal, R. Girshick, K. He, and P. Doll ´ar, “Focal loss for dense object detection,” in 2017 IEEE International Conference on Computer Vision (ICCV) , Oct. 2017, pp. 2999–3007

    2017

  42. [50]

    Improved algorithms for linear stochastic bandits,

    Y . Abbasi-yadkori, D. P ´al, and C. Szepesv ´ari, “Improved algorithms for linear stochastic bandits,” in Advances in Neural Information Processing Systems 24 , J. Shawe-Taylor, R. S. Zemel, P. L. Bartlett, F. Pereira, and K. Q. Weinberger, Eds. Curran Associates, Inc., 2011,...

    2011

  43. [51]

    Doubly robust policy evaluation and learning,

    M. Dudik, J. Langford, and L. Li, “Doubly robust policy evaluation and learning,” arXiv preprint arXiv:1103.4601 , Mar. 2011

    2011 arXiv

  44. [52]

    JACO, 2018, https://www.kinovarobotics.com/en/products/ robotic-armseries

    K. JACO, 2018, https://www.kinovarobotics.com/en/products/ robotic-armseries

    2018

  45. [53]

    A.-I. F.-T. Sensor, 2018, https://www.ati-ia.com/products/ft/ft models. aspx?id=Nano25

    2018

  46. [54]

    Is more autonomy always better? exploring preferences of users with mobility impairments in robot- assisted feeding,

    T. Bhattacharjee, E. Gordon, R. Scalise, M. Cabrera, A. Caspi, M. Cakmak, and S. Srinivasa, “Is more autonomy always better? exploring preferences of users with mobility impairments in robot- assisted feeding,” in ACM/IEEE International Conference on Human- Robot Interaction, 2020

    2020

  47. [55]

    Learning audio feedback for estimating amount and flow of granular material,

    S. Clarke, T. Rhodes, C. G. Atkeson, and O. Kroemer, “Learning audio feedback for estimating amount and flow of granular material,” in Proceedings of The 2nd Conference on Robot Learning , ser. Proceedings of Machine Learning Research, A. Billard, A. Dragan, J. Peters, and J. M...

    2018

  48. [56]

    Marginal posterior sampling for slate bandits,

    M. Dimakopoulou, N. Vlassis, and T. Jebara, “Marginal posterior sampling for slate bandits,” in Proceedings of the Twenty-Eighth International Joint Conference on Artificial Intelligence , 2019, pp. 2223–2229

    2019

Pith tools

Reviewed August 14, 2026 · model on record in the stance chip above.