Pith. sign in

REVIEW 5 major objections 4 minor 52 references

In Pursuit of Predictive Models of Human Preferences Toward AI Teammates

T0 review · 5 major / 4 minor · reviewed 2026-08-09 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read Human preferences toward AI teammates can be predicted from human-free measurements of agent behavior, and final team score predicts those preferences more weakly than action diversity, strategic dominance, and coordination metrics.

desk verdict Large, honest Hanabi study with a load-bearing statistics problem: the headline correlations treat eight agents as hundreds of independent observations. read the letter →

arxiv 2503.15516 v1 pith:SE46FZAF submitted 2025-01-31 cs.HC cs.AI

Ho Chit Siu , Jaime D. Peña , Yutai Zhou , Ross E. Allen This is my paper · ORCID
classification cs.HCcs.AI
keywords human-AIteamingHanabireinforcementlearninghumanpreferencesteamworkratingobjectivemetricsinformationtheorygame
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper asks whether an AI agent's behavior can be scored without any human involvement in a way that predicts how much human teammates will like working with it. Using the cooperative card game Hanabi, the authors measured a pool of AI agents on task-performance, information-theoretic, and game-theoretic metrics through AI-only games, then had 241 human participants play with a subset of those agents and rate each as a teammate. The central finding is that the final game score a human-AI team achieves is a weaker predictor of human preference than several AI-only metrics, including the diversity of the agent's actions, its instantaneous coordination with a partner, and how often it takes strategically dominated moves. If the correlations hold, objective metrics computable without humans in the loop could serve as reward signals for training collaborative AI, replacing expensive human feedback during training.

What carries the argument

The load-bearing machinery is a three-stage pipeline. First, AI-only game traces are compressed into three families of objective metrics: task performance (self-play, intra-algorithm cross-play, and inter-algorithm cross-play scores), information-theoretic measures (action-distribution entropy, action-response distribution entropy, instantaneous coordination, and context independence), and game-theoretic move frequencies (discarding a known-playable card, playing a known-unplayable card, and playing a known-playable card). Second, a subset of agents spanning this metric space is played by 241 human participants who rate each teammate on a summed Likert teamwork scale. Third, single-variable regressions with a Bonferroni-corrected significance threshold link each objective metric to the subjective rating, and it is the correlation table produced by this step that carries the argument's weight.

What would settle it

A held-out prediction test would settle the claim: use the fitted regressions from the eight evaluated agents to predict teamwork ratings for new agents, run the same human protocol, and check whether the predicted preference ordering matches the observed ratings. A cleaner experiment would train two agents matched on self-play score but differing in G1-dominated frequency, since the claim predicts the low-G1 agent earns higher teamwork ratings despite equal game scores.

Watch

Extended reading notes

Core claim

The paper's central claim is that subjective human preferences toward AI teammates are partially predictable from objective metrics that require no human interaction, and that task performance is not the most informative metric for this purpose. In Hanabi, the human-AI game score correlates with teamwork ratings at $r = 0.350$ including the random agent and $r = 0.203$ without it, while action-response distribution entropy ($r = 0.466$) and instantaneous coordination (nonlinear fit, $r = 0.517$) correlate more strongly. The game-theoretic measures show the sharpest practical signal: a 1-percent increase in the frequency of discarding a card known to be playable corresponds to a 9-point drop on the 35-point teamwork scale, whereas self-play and intra-algorithm cross-play scores are weak-to-moderate predictors and inter-algorithm cross-play score shows no significant correlation once the random agent is excluded. The authors read these results as refuting the common assumption that better self-play or cross-play performance implies a more preferred teammate, and they propose that human-collaborative agents should be trained with rewards that blend task score with information-theoretic and game-theoretic terms.

Load-bearing premise

The correlations assume that how an agent behaves next to another AI, including the actions it favors and the game lengths it produces, is representative of how it behaves next to a human, even though Hanabi strategies are strongly partner-dependent.

Editorial extensions

If this is right

  • Reward functions for training collaborative agents should blend task score with information-theoretic and game-theoretic terms rather than maximizing raw performance alone.
  • Self-play and cross-play scores, widely used as coordination benchmarks, would be demoted from proxies for human preference to weak indicators at best.
  • Designers could screen candidate AI teammates with cheap AI-AI evaluations before committing to expensive human-subject experiments.
  • Trainers should suppress single-step dominated moves even at very low frequencies, since a 1-percent increase in such moves predicts a 9-point teamwork-rating drop.
  • Because no metric reached strong predictive power even in Hanabi's stripped-down setting, a universal human-preference reward function is unlikely to exist across less-constrained teaming tasks.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The asymmetry between the steep negative slope for dominated moves and the gentler positive slope for dominant moves suggests preference is driven more by avoiding visible mistakes than by maximizing visible competence; a study that varies only G1-move frequency while holding game score constant could test this loss-aversion pattern directly.
  • The paper's transfer assumption, that AI-AI statistics represent human-AI behavior, could be checked by recomputing the objective metrics with a scripted human-modeled partner; if agent rankings shift, the correlations would need re-interpretation.
  • An engineering test would train a new agent with a reward penalty on G1-dominated moves and run a fresh human study to see whether it outranks a score-matched agent without the penalty.
  • Because the strongest predictors carry domain-specific interpretations, the same correlation pipeline may need to be rebuilt for each new teaming task, which argues against expecting a single portable human-preference metric.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

5 major / 4 minor

Summary. The paper asks whether objective, human-free measurements of an AI agent's behavior can predict human subjective preferences toward that agent as a teammate. The authors evaluate a pool of Hanabi AI agents on task-performance metrics (self-play, intra-XP, inter-XP), information-theoretic metrics (action entropy, action-response entropy, instantaneous coordination, context independence), and game-theoretic metrics (frequencies of dominated and dominant moves) using AI-only games. They then collect human teamwork ratings (N=241 participants, each playing with two of eight agents) and correlate the agent-level AI-only metric means with the participant-level teamwork ratings. The headline result is that human-AI game score is a weaker predictor of subjective preference than several AI-only metrics, including self-play score, ARD-entropy, and G1-dominated move frequency, with the stated long-term goal of using such metrics to shape RL reward functions. The paper also reports that the AI-only metrics correlate only weakly to moderately overall and that results are sensitive to inclusion of RandomBot.

Significance. If the central claim were statistically valid, the paper would make a genuinely useful contribution: it would offer a practical, human-free proxy signal for training human-collaborative AI and would empirically challenge the common assumption that self-play or cross-play performance predicts human preference. The study has notable strengths: a large human-subjects sample, a diverse set of publicly available agents, deliberate reporting of results with and without an outlier agent, and careful documentation of metric definitions and experiment procedures in the appendices. However, the statistical analysis at the core of the paper has a load-bearing flaw: the AI-only metrics are per-agent constants measured for only eight agents, while the regression treats hundreds of participant-level ratings as independent observations. This invalidates the reported p-values and makes the headline comparison between human-AI game score and AI-only metrics confounded by level of analysis. The instability of the correlations to the exclusion of a single agent further undermines the claim that these metrics are predictive.

major comments (5)
  1. [Section 3.2, Table 5; Appendix D] The correlation analysis in Section 3.2 regresses each participant's block-level teamwork rating (two ratings per participant, 482 total) on per-agent constant predictors that take only eight distinct values. This treats ratings as independent despite being nested within agents, and the effective sample size for each AI-only metric is at most eight agents, not 482. The Bonferroni correction in Appendix D addresses multiple comparisons but does not address clustering or the tiny number of independent units on the predictor side. Consequently, the p-values in Table 5 are artificially small, and the relative ordering of correlation strengths across metrics is not a valid basis for the paper's central claim. A mixed-effects model or cluster-robust inference with agents as the clustering unit would be required, and with eight agents the confidence intervals around the agent-level correlations would be very wide.
  2. [Section 3.2, Table 5; Section E.2] The headline comparison is confounded by level of analysis: the human-AI game score is a participant-level variable measured in the same games as the rating, whereas self-play, inter-XP, entropy, CI, and game-theoretic frequencies are agent-level means computed from a different set of games. Comparing Pearson correlations across these two levels conflates the unit of analysis with predictive utility. Even if the within-level correlations were computed correctly, the statement that AI-only metrics are 'more predictive' than human-AI game score cannot be supported unless both are evaluated on the same observational units or with a comparable aggregation scheme.
  3. [Section 3.2, Table 5; Section E.1, E.2] The results are not robust to the inclusion of a single agent. Excluding RandomBot flips AD-entropy from r=-0.086 to r=0.301, reduces inter-XP from r=0.370 (p<0.001) to r=0.047 (p=0.344), changes the sign of G2-dominated from 0.332 to 0.058, and drops self-play from 0.447 to 0.168. Since the analysis has only eight agents, these large swings show that the correlations are driven substantially by one outlier rather than by a stable relationship. The paper does report both analyses, which is transparent, but the instability undercuts the abstract's claim that the AI-only metrics are generally predictive of human preference.
  4. [Section 2.2, 3.1, Table 3; Appendix E.2] The transfer assumption that AI-only metrics computed from AI-AI games are representative of behavior with human partners is load-bearing and not justified. Hanabi behavior is strongly partner-dependent, as the paper itself demonstrates: Table 3 and Section E.2 explain RandomBot's low ARD-entropy and CI as artifacts of short AI-AI games, and the paper notes that OP+Aux and AP+Aux have high inter-XP scores yet low human ratings. Without validation that the AI-only metric distributions match those in human-AI games, the correlations in Table 5 may reflect game-length or partner-policy artifacts rather than stable agent traits. The paper should either compute the metrics from the human-AI game traces (where feasible) or provide evidence that the metric values are consistent across partner types.
  5. [Appendix C.2, Section E.2; Table 5] The interpretation of Context Independence is reversed after the fact: the paper asserts that lower CI is better in Hanabi because the number of concepts exceeds the number of actions, but RandomBot has the lowest CI and is the least preferred agent. The paper explains this by game-length effects, but that explanation relies on the same transfer assumption criticized above. Additionally, the Instantaneous Coordination analysis uses a parabolic fit with three free parameters (a,b,c) selected after viewing the data, so the reported r values (0.515 and 0.332) are not confirmatory correlations; they overstate the strength of the relationship. These issues should be addressed with a pre-specified model or a correction for model selection, and the CI reversal should be validated independently.
minor comments (4)
  1. [Appendix C.2] The Shannon entropy citation appears as a malformed placeholder '(?)Chp 2]cover2006elements' rather than a proper citation; the reference should be corrected.
  2. [Section 3.2, Table 5 note] The table note says IC 'had no m and p values,' but the text in Section E.2 reports r values for the nonlinear fit; it would be helpful to state explicitly that no p-value is computed because the fit is nonlinear and post-hoc.
  3. [Figure 8 and Section E.2] The discussion of game length (Figure 8) is very useful for interpreting the entropy and CI results, but it would be improved by reporting the mean and standard deviation of game length for each agent in the main text rather than only in the appendix.
  4. [General] There are several typographical and formatting issues, such as 'Jacques et al.' in Section C.2 (should be 'Jaques et al.') and inconsistent spacing in Table 3; a careful proofread is needed.

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity: the objective AI-only metrics and subjective human ratings are measured in separate experiments and correlated directly.

full rationale

The paper's central claim is an empirical correlation between AI-only objective metrics and subjective human teamwork ratings. The objective metrics are defined in Appendix C independently of the human ratings and are measured in separate AI-AI games (Section 2.2), while the subjective ratings are collected in separate human-AI games (Section 3.1). The correlations in Table 5 are computed directly from these two independent measurement streams. No parameter is fitted to the human ratings and then reused to predict those same ratings, and no objective metric is defined in terms of the teamwork score. The self-citations (Siu et al. 2021; Lucas & Allen 2022) are used for prior context, agent selection, or metric definitions that are restated in the appendices; none of the new claims depends on an unverified result imported from those citations. Post-hoc modeling choices, such as the parabolic fit for IC and the reversed interpretation of CI, are statistical or interpretive issues rather than derivation-circularity. Robustness concerns about the small number of agents and clustered p-values are validity concerns, not circularity. Therefore there is no significant circularity in the claimed derivation.

Assumptions & free parameters 3 free parameters · 5 assumptions · 0 invented entities

The central claim rests on the transferability of AI-only metrics to human-AI interaction, on the validity of a summed Likert scale as an interval outcome, and on a non-random, small sample of 8 agents. The IC parabola and the reversed CI interpretation are data-informed modeling choices. No new entities are introduced.

free parameters (3)
  • IC parabolic regression coefficients (a, b, c) = not reported
    Instantaneous coordination is fit with y = a(x+b)^2 + c, with a<0, to maximize correlation with teamwork rating; the coefficients are estimated from the human rating data and no values are given.
  • Bonferroni significance threshold alpha = 0.00278 (0.05/18)
    The correction is applied across 18 regressions, but not across the multiple with/without RandomBot exclusion variants, so the effective error rate is uncontrolled.
  • CI formula sampling budget = 500 formulas, tree depth 3
    Context independence requires sampling logical formulas; this hand-chosen budget affects the CI values for all agents.
assumptions (5)
  • domain assumption The summed Likert responses B3-B8 form an interval-level teamwork rating suitable for linear regression.
    The paper sums six 7-point items into a 0-35 score and uses linear regression without testing interval properties or the scale's unidimensionality (Section 3.1).
  • domain assumption AI-only metrics computed from AI-AI games are valid measures of the same agent's behavior in human-AI games.
    The predictors come from AI-AI play (Section 2.2) while the outcome comes from human-AI play (Section 3.1); Hanabi behavior depends on the partner's actions, so this transfer is assumed rather than established.
  • ad hoc to paper The 8 agents selected for human testing are a sufficient and representative sample for estimating correlations between objective metrics and human preference.
    Agents were down-selected to span the objective metric space (Section 2.2), not randomly or to cover human preference space; only 8 agents enter the correlation, and results are sensitive to removing one (RandomBot).
  • ad hoc to paper The reversed interpretation of Context Independence (lower is better) is valid in Hanabi.
    Section C.2 argues the usual CI interpretation should be reversed for Hanabi; this reversal aligns the metric direction with the observed correlation and is not validated against an external benchmark.
  • domain assumption A linear relationship (or a single parabola for IC) is the correct functional form for each objective metric's relation to teamwork rating.
    The paper performs single-variable linear regressions without testing nonlinear alternatives except for IC, where a parabola is assumed (Section D).

how reviews work

0 comments
Cite this review

Pith. "Pith review of In Pursuit of Predictive Models of Human Preferences Toward AI Teammates." pith.science (2026). https://pith.science/paper/SE46FZAF

@misc{pith2026250315516,
  author       = {Pith},
  title        = {Pith review of: In Pursuit of Predictive Models of Human Preferences Toward AI Teammates},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/SE46FZAF}},
  note         = {Machine review of arXiv:2503.15516}
}
read the original abstract

We seek measurable properties of AI agents that make them better or worse teammates from the subjective perspective of human collaborators. Our experiments use the cooperative card game Hanabi -- a common benchmark for AI-teaming research. We first evaluate AI agents on a set of objective metrics based on task performance, information theory, and game theory, which are measurable without human interaction. Next, we evaluate subjective human preferences toward AI teammates in a large-scale (N=241) human-AI teaming experiment. Finally, we correlate the AI-only objective metrics with the human subjective preferences. Our results refute common assumptions from prior literature on reinforcement learning, revealing new correlations between AI behaviors and human preferences. We find that the final game score a human-AI team achieves is less predictive of human preferences than esoteric measures of AI action diversity, strategic dominance, and ability to team with other AI. In the future, these correlations may help shape reward functions for training human-collaborative AI.

Figures

Figures reproduced from arXiv: 2503.15516 by the authors.

Figure 1
Figure 1. Letter-value plot of subjective teamwork rating statistics for each AI agent used in human-AI teaming experiments. rating of OP+Aux and AP+Aux agents. This tends to refute the assumption made by Lucas & Allen (2022) that agents with higher inter-XP scores will naturally be preferred by human teammates. Indeed, results in Section 3.2 will indi￾cate that inter-XP does not have a statistically significant correlation t… view at source ↗
Figure 2
Figure 2. Scores during human-AI games vs human teamwork ratings. There is a medium to strong statistically significant correlation. Blue is data with RandomBot, green is without. The data from both the x and y axes come from the same games, so there is no need to take the mean of one side as in most other plots. evaluations, since they would allow for the formation of algorithm-specific conventions (Lucas & Allen, 2022). Fin… view at source ↗
Figure 3
Figure 3. Teamwork rating vs self-play, intra-XP, and inter-XP scores. Correlation values match the colors of the associated lines (blue is with RandomBot, green is without). Intra-algorithm cross-play (intra-XP) cannot be evaluated on non-learning-based agents, thus rule-base bots (RandomBot, SmartBot, HolmesBot) do not appear in the second plot. Action-response distribution entropy ( [PITH_FULL_IMAGE:figures/full_fig_p018_3.png] view at source ↗
Figures from the paper (8 more)
Figure 4
Figure 4. Figure 4: Information-theoretic metrics vs teamwork rating. Note that while the y-axes and colorbar are the same across all plots, the x-axes are not. it finds itself are very limited, lowering its CI value ( [PITH_FULL_IMAGE:figures/full_fig_p019_4.png]
Figure 5
Figure 5. Figure 5: Frequency of discarding a card known to be presently playable (G1), playing a card known to be presently unplayable (G2), and playing a card known to be presently playable (G3) vs teamwork rating. G1 and G2 are dominated moves, while G3 is a dominant move. Note that wh…
Figure 6
Figure 6. Figure 6: Pairwise subject preferences between the first and second bots they played with. Scores are a sum of Likert items P1 to P7, where the neutral value is taken to be zero for ease of visualization. Data do not exist on the diagonal since no bot was played with twice by th…
Figure 7
Figure 7. Figure 7: AI-AI scores vs human-AI scores. Though significant, we see that the correlations (without RandomBot as an outlier) are generally small, and notably, that the spread of human-AI game scores in the highest-performing AI agents is substantial. In fact, AP and OP agents s…
Figure 8
Figure 8. Figure 8: Number of turns in AI-only games. 24 [PITH_FULL_IMAGE:figures/full_fig_p024_8.png]
Figure 9
Figure 9. Figure 9: Subjects’ self-rated familiarity with Hanabi (one of the demographic questions asked). There were a total of 241 subjects, of which 93 self-reported ≥ 4 on familiarity. 25 [PITH_FULL_IMAGE:figures/full_fig_p025_9.png]
Figure 10
Figure 10. Figure 10: Distribution of participants per bot type. HolmesBot, OP, and OBL were added in a later round of experiments, resulting in fewer interactions. 26 [PITH_FULL_IMAGE:figures/full_fig_p026_10.png]
Figure 11
Figure 11. Figure 11: Distribution of participants per bot pairing. 27 [PITH_FULL_IMAGE:figures/full_fig_p027_11.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

52 extracted references · 37 canonical work pages

  1. [1]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 global.max substring 't := if while FUNCTION format.date year duplicate empty "emp...

  2. [2]

    Emergent complexity via multi-agent competition

    Bansal, T., Pachocki, J., Sidor, S., Sutskever, I., and Mordatch, I. Emergent complexity via multi-agent competition. arXiv preprint arXiv:1710.03748, 2017

    arXiv 2017

  3. [3]

    N., Chandar, S., Burch, N., Lanctot, M., Song, H

    Bard, N., Foerster, J. N., Chandar, S., Burch, N., Lanctot, M., Song, H. F., Parisotto, E., Dumoulin, V., Moitra, S., Hughes, E., et al. The hanabi challenge: A new frontier for ai research. Artificial Intelligence, 280: 0 103216, 2020

    work page 2020

  4. [4]

    Dota 2 with large scale deep reinforcement learning

    Berner, C., Brockman, G., Chan, B., Cheung, V., Dkebiak, P., Dennison, C., Farhi, D., Fischer, Q., Hashme, S., Hesse, C., et al. Dota 2 with large scale deep reinforcement learning. arXiv preprint arXiv:1912.06680, 2019

    arXiv 1912

  5. [5]

    Emergence of Communication in an Interactive World with Consistent Speakers

    Bogin, B., Geva, M., and Berant, J. Emergence of communication in an interactive world with consistent speakers. arXiv preprint arXiv:1809.00549, 2018

    work page Pith review arXiv 2018

  6. [6]

    Generating and Adapting to Diverse Ad-Hoc Cooperation Agents in Hanabi

    Canaan, R., Gao, X., Togelius, J., Nealen, A., and Menzel, S. Generating and adapting to diverse ad-hoc cooperation agents in hanabi. arXiv preprint arXiv:2004.13710, 2020

    work page Pith review arXiv 2004

  7. [7]

    K., Griffiths, T., Seshia, S., Abbeel, P., and Dragan, A

    Carroll, M., Shah, R., Ho, M. K., Griffiths, T., Seshia, S., Abbeel, P., and Dragan, A. On the utility of learning about humans for human-ai coordination. Advances in Neural Information Processing Systems, 32, 2019

    work page 2019

  8. [8]

    F., Leike, J., Brown, T., Martic, M., Legg, S., and Amodei, D

    Christiano, P. F., Leike, J., Brown, T., Martic, M., Legg, S., and Amodei, D. Deep reinforcement learning from human preferences. Advances in neural information processing systems, 30, 2017

    2017

Show all 52 references
  1. [9]

    Cooke, N. J. and Shope, S. M. Synthetic task environments for teams: Certt’s uav-ste. In Handbook of human factors and ergonomics methods, pp.\ 476--483. CRC Press, Boca Raton, FL, USA, 2004

    2004

  2. [10]

    Cover, T. M. and Thomas, J. A. Elements of Information Theory Second Edition. Wiley-Interscience, USA, 2006. ISBN 0471241954

    2006

  3. [11]

    (FAIR)†, M. F. A. R. D. T., Bakhtin, A., Brown, N., Dinan, E., Farina, G., Flaherty, C., Fried, D., Goff, A., Gray, J., Hu, H., et al. Human-level play in the game of diplomacy by combining language models with strategic reasoning. Science, 378 0 (6624): 0 1067--1074, 2022

    2022

  4. [12]

    Human-level performance in no-press diplomacy via equilibrium search

    Gray, J., Lerer, A., Bakhtin, A., and Brown, N. Human-level performance in no-press diplomacy via equilibrium search. arXiv preprint arXiv:2010.02923, 2020

    2010 arXiv

  5. [13]

    Robot transparency and team orientation effects on human--robot teaming

    Guznov, S., Lyons, J., Pfahler, M., Heironimus, A., Woolley, M., Friedman, J., and Neimeier, A. Robot transparency and team orientation effects on human--robot teaming. International Journal of Human--Computer Interaction, 36 0 (7): 0 650--660, 2020

    2020

  6. [14]

    Hoff, K. A. and Bashir, M. Trust in automation: Integrating empirical evidence on factors that influence trust. Human factors, 57 0 (3): 0 407--434, 2015

    2015

  7. [15]

    Evaluating fluency in human--robot collaboration

    Hoffman, G. Evaluating fluency in human--robot collaboration. IEEE Transactions on Human-Machine Systems, 49 0 (3): 0 209--218, 2019

    2019

  8. [16]

    D., Porter, W

    Holcomb, S. D., Porter, W. K., Ault, S. V., Mao, G., and Wang, J. Overview on deepmind and its alphago zero ai. In Proceedings of the 2018 international conference on big data and education, pp.\ 67--71, 2018

    2018

  9. [17]

    and Foerster, J

    Hu, H. and Foerster, J. N. Simplified action decoder for deep multi-agent reinforcement learning. arXiv preprint arXiv:1912.02288, 2019

    1912 arXiv

  10. [18]

    other-play

    Hu, H., Lerer, A., Peysakhovich, A., and Foerster, J. “other-play” for zero-shot coordination. In International Conference on Machine Learning, pp.\ 4399--4410. PMLR, 2020

    2020

  11. [19]

    Off-belief learning

    Hu, H., Lerer, A., Cui, B., Pineda, L., Brown, N., and Foerster, J. Off-belief learning. In International Conference on Machine Learning, pp.\ 4369--4379. PMLR, 2021

    2021

  12. [20]

    J., Lerer, A., Foerster, J., and Brown, N

    Hu, H., Wu, D. J., Lerer, A., Foerster, J., and Brown, N. Human-ai coordination via human-regularized search and learning. arXiv preprint arXiv:2210.05125, 2022

    2022 arXiv

  13. [21]

    Z., and De Freitas, N

    Jaques, N., Lazaridou, A., Hughes, E., Gulcehre, C., Ortega, P., Strouse, D., Leibo, J. Z., and De Freitas, N. Social influence as intrinsic motivation for multi-agent deep reinforcement learning. In International Conference on Machine Learning, pp.\ 3040--3049. PMLR, 2019

    2019

  14. [22]

    Champion-level drone racing using deep reinforcement learning

    Kaufmann, E., Bauersfeld, L., Loquercio, A., M \"u ller, M., Koltun, V., and Scaramuzza, D. Champion-level drone racing using deep reinforcement learning. Nature, 620 0 (7976): 0 982--987, 2023

    2023

  15. [23]

    R., Sobh, I., Talpaert, V., Mannion, P., Al Sallab, A

    Kiran, B. R., Sobh, I., Talpaert, V., Mannion, P., Al Sallab, A. A., Yogamani, S., and P \'e rez, P. Deep reinforcement learning for autonomous driving: A survey. IEEE Transactions on Intelligent Transportation Systems, 23 0 (6): 0 4909--4926, 2021

    2021

  16. [24]

    Google research football: A novel reinforcement learning environment

    Kurach, K., Raichuk, A., Stanczyk, P., Zajkac, M., Bachem, O., Espeholt, L., Riquelme, C., Vincent, D., Michalski, M., Bousquet, O., et al. Google research football: A novel reinforcement learning environment. In Proceedings of the AAAI conference on artificial intelligence, v...

    2020

  17. [25]

    and Siu, H

    Leahy, K. and Siu, H. C. Tell me what you want (what you really, really want): Addressing the expectation gap for goal conveyance from humans to robots. arXiv preprint arXiv:2403.14344, 2024

    2024 arXiv

  18. [26]

    Improving policies via search in cooperative partially observable games

    Lerer, A., Hu, H., Foerster, J., and Brown, N. Improving policies via search in cooperative partially observable games. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 34:05, pp.\ 7187--7194, 2020

    2020

  19. [27]

    P., and Ungar, L

    Lin, D., Foster, D. P., and Ungar, L. H. Vif regression: a fast regression algorithm for large data. Journal of the American Statistical Association, 106 0 (493): 0 232--247, 2011

    2011

  20. [28]

    On the pitfalls of measuring emergent communication

    Lowe, R., Foerster, J., Boureau, Y.-L., Pineau, J., and Dauphin, Y. On the pitfalls of measuring emergent communication. arXiv preprint arXiv:1903.05168, 2019

    1903 arXiv

  21. [29]

    and Allen, R

    Lucas, K. and Allen, R. E. Any-play: An intrinsic augmentation for zero-shot coordination. In Proceedings of the 21st International Conference on Autonomous Agents and Multiagent Systems, pp.\ 853--861, 2022

    2022

  22. [30]

    Trajectory diversity for zero-shot coordination

    Lupu, A., Cui, B., Hu, H., and Foerster, J. Trajectory diversity for zero-shot coordination. In International conference on machine learning, pp.\ 7204--7213. PMLR, 2021

    2021

  23. [31]

    Playing atari with deep reinforcement learning

    Mnih, V., Kavukcuoglu, K., Silver, D., Graves, A., Antonoglou, I., Wierstra, D., and Riedmiller, M. Playing atari with deep reinforcement learning. arXiv preprint arXiv:1312.5602, 2013

    2013 arXiv

  24. [32]

    Continuous coordination as a realistic scenario for lifelong learning

    Nekoei, H., Badrinaaraayanan, A., Courville, A., and Chandar, S. Continuous coordination as a realistic scenario for lifelong learning. In International Conference on Machine Learning, pp.\ 8016--8024. PMLR, 2021

    2021

  25. [33]

    quuxplusone/hanabi: framework for writing bots that play hanabi

    O'Dwyer, A. quuxplusone/hanabi: framework for writing bots that play hanabi. https://github.com/Quuxplusone/Hanabi/, 2019

    2019

  26. [34]

    Human--autonomy teaming: A review and analysis of the empirical literature

    O’Neill, T., McNeese, N., Barron, A., and Schelble, B. Human--autonomy teaming: A review and analysis of the empirical literature. Human factors, 64 0 (5): 0 904--938, 2022

    2022

  27. [35]

    D., Ermon, S., and Finn, C

    Rafailov, R., Sharma, A., Mitchell, E., Manning, C. D., Ermon, S., and Finn, C. Direct preference optimization: Your language model is secretly a reward model. Advances in Neural Information Processing Systems, 36, 2024

    2024

  28. [36]

    Y., Peng, W., and Huh-Yoo, J

    Rheu, M., Shin, J. Y., Peng, W., and Huh-Yoo, J. Systematic review: Trust-building factors and implications for conversational agent design. International Journal of Human--Computer Interaction, 37 0 (1): 0 81--96, 2021

    2021

  29. [37]

    G., Lee, C

    Richens, J. G., Lee, C. M., and Johri, S. Improving the accuracy of medical diagnosis with causal machine learning. Nature communications, 11 0 (1): 0 3923, 2020

    2020

  30. [38]

    and Royzman, E

    Rozin, P. and Royzman, E. B. Negativity bias, negativity dominance, and contagion. Personality and social psychology review, 5 0 (4): 0 296--320, 2001

    2001

  31. [39]

    A., and Li, S

    Shah, A., Kamath, P., Shah, J. A., and Li, S. Bayesian inference of temporal task specifications from demonstrations. Advances in Neural Information Processing Systems, 31, 2018

    2018

  32. [40]

    A general reinforcement learning algorithm that masters chess, shogi, and go through self-play

    Silver, D., Hubert, T., Schrittwieser, J., Antonoglou, I., Lai, M., Guez, A., Lanctot, M., Sifre, L., Kumaran, D., Graepel, T., et al. A general reinforcement learning algorithm that masters chess, shogi, and go through self-play. Science, 362 0 (6419): 0 1140--1144, 2018

    2018

  33. [41]

    Simon, H. A. Bounded rationality. Utility and probability, pp.\ 15--18, 1990

    1990

  34. [42]

    C., Pe \ n a, J., Chen, E., Zhou, Y., Lopez, V., Palko, K., Chang, K., and Allen, R

    Siu, H. C., Pe \ n a, J., Chen, E., Zhou, Y., Lopez, V., Palko, K., Chang, K., and Allen, R. E. Evaluation of human-ai teams for learned and rule-based agents in hanabi. Advances in Neural Information Processing Systems, 34, 2021

    2021

  35. [43]

    Collaborating with humans without human data

    Strouse, D., McKee, K., Botvinick, M., Hughes, E., and Everett, R. Collaborating with humans without human data. Advances in Neural Information Processing Systems, 34: 0 14502--14515, 2021

    2021

  36. [44]

    Sutton, R. S. and Barto, A. G. Reinforcement learning: An introduction. MIT Press, Cambridge, MA, USA, 2018

    2018

  37. [45]

    Overview of deep learning in medical imaging

    Suzuki, K. Overview of deep learning in medical imaging. Radiological physics and technology, 10 0 (3): 0 257--273, 2017

    2017

  38. [46]

    Game theory: an introduction

    Tadelis, S. Game theory: an introduction. Princeton university press, USA, 2013

    2013

  39. [47]

    Tauer, J. M. and Harackiewicz, J. M. Winning isn't everything: Competition, achievement orientation, and intrinsic motivation. Journal of Experimental Social Psychology, 35 0 (3): 0 209--238, 1999

    1999

  40. [48]

    Thaler, R. H. and Sunstein, C. R. Nudge: Improving Decisions About Health, Wealth, and Happiness. Penguin, 2009

    2009

  41. [49]

    M., Mathieu, M., Dudzik, A., Chung, J., Choi, D

    Vinyals, O., Babuschkin, I., Czarnecki, W. M., Mathieu, M., Dudzik, A., Chung, J., Choi, D. H., Powell, R., Ewalds, T., Georgiev, P., et al. Grandmaster level in starcraft ii using multi-agent reinforcement learning. Nature, 575 0 (7782): 0 350--354, 2019

    2019

  42. [50]

    R., Barrett, S., Kawamoto, K., MacGlashan, J., Subramanian, K., Walsh, T

    Wurman, P. R., Barrett, S., Kawamoto, K., MacGlashan, J., Subramanian, K., Walsh, T. J., Capobianco, R., Devlic, A., Eckert, F., Fuchs, F., et al. Outracing champion gran turismo drivers with deep reinforcement learning. Nature, 602 0 (7896): 0 223--228, 2022

    2022

  43. [51]

    Winning is not everything: Enhancing game development with intelligent agents

    Zhao, Y., Borovikov, I., de Mesentier Silva, F., Beirami, A., Rupert, J., Somers, C., Harder, J., Kolen, J., Pinto, J., Pourabolghasem, R., et al. Winning is not everything: Enhancing game development with intelligent agents. IEEE Transactions on Games, 12 0 (2): 0 199--212, 2020

    2020

  44. [52]

    M., Stiennon, N., Wu, J., Brown, T

    Ziegler, D. M., Stiennon, N., Wu, J., Brown, T. B., Radford, A., Amodei, D., Christiano, P., and Irving, G. Fine-tuning language models from human preferences. arXiv preprint arXiv:1909.08593, 2019

    1909 arXiv

Pith tools

Reviewed August 9, 2026 · model on record in the stance chip above.