Pith. sign in

REVIEW 2 major objections 33 references

Co-Creativity at the Table: A Qualitative Analysis of Creative Interactions in the Podcast "Adventure AI"

T0 review · 2 major / 0 minor · reviewed 2026-06-26 · grok-4.3

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read Analysis of AI in D&D podcast shows AI succeeds in some game roles but not others

desk verdict Original thematic coding of one AI-D&D podcast but held back by missing methods and single-source limits. read the letter →

arxiv 2606.18010 v1 pith:YR6N7CCX submitted 2026-06-16 cs.HC

Hanna Dodd , Daniel G. Brown This is my paper
classification cs.HC
keywords qualitativeanalysistabletoprole-playinggamesartificialintelligenceDungeonsandDragonshuman-AIinteractionpodcastco-creativity
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper examines human-AI interactions in the Adventure AI podcast featuring Dungeons & Dragons play. It performs a qualitative thematic analysis across three seasons to identify patterns in AI roles, human roles, AI evaluations and failures, and the treatment of AI as a person or character at the table. A sympathetic reader would care because the work maps concrete places where AI supports collaborative storytelling and where it does not, supplying guidance for future decisions about AI in gaming.

What carries the argument

Thematic qualitative analysis of podcast episodes on roles of AI, roles of humans, evaluations and failures of AI, and treatment of AI as person and character

What would settle it

A separate collection of human-AI Dungeons & Dragons sessions in which AI fails in every area the analysis marks as successful would undermine the reported distinctions between appropriate and inappropriate uses.

Watch

Extended reading notes

Core claim

The analysis reveals that artificial intelligence succeeds in many aspects of the game while proving less appropriate in others. This supplies a basis for future work on where artificial intelligence should and should not be used in gaming spaces.

Load-bearing premise

The recorded interactions in the podcast serve as a sufficient and representative sample for drawing general conclusions about AI suitability in tabletop role-playing games.

Editorial extensions

If this is right

  • AI can be assigned to certain content-generation and support tasks during play.
  • Human participants remain central for oversight and complex narrative choices.
  • Failures in AI performance point to limits in handling social or interpretive elements.
  • Design choices for future AI gaming tools can draw on the observed success and failure patterns.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The same thematic approach could be applied to other collaborative creative settings such as collaborative writing or world-building sessions.
  • Controlled live-game experiments could test whether the podcast patterns hold outside recorded formats.
  • AI system builders might use the failure categories to set priorities for improving specific capabilities.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

2 major / 0 minor

Summary. The manuscript reports a qualitative thematic analysis of three seasons (2023–2025) of the Adventure AI podcast, which records human players interacting with an AI in Dungeons & Dragons sessions. It identifies four overarching themes—roles of AI, roles of humans, evaluations and failures of AI, and treatment of AI as person and character—and concludes that AI succeeds in some aspects of TTRPG play but is less appropriate in others, thereby providing a basis for future work on suitable uses of AI in gaming spaces.

Significance. If the thematic analysis is methodologically transparent, the work could usefully document concrete patterns of success and failure in a real-world human-AI co-creative setting. The choice of a publicly available podcast as data source is a strength, as it permits scrutiny and potential reuse by other researchers studying collaborative creativity in HCI.

major comments (2)
  1. [Abstract] Abstract and methods description: the manuscript states that a qualitative analysis was completed but supplies no information on coding procedure, number of coders, inter-rater checks, or how episodes were selected, so the link between data and reported themes cannot be evaluated.
  2. [Abstract] Abstract, final sentence: the claim that the analysis 'gives a basis for future work on where artificial intelligence should and should not be used in gaming spaces' treats the recorded sessions as sufficient for identifying general success/failure patterns, yet no comparison to other groups, AIs, or non-podcast TTRPG play is reported.

Simulated Author's Rebuttal

2 responses · 0 unresolved

We thank the referee for their constructive feedback on methodological transparency and the scope of our conclusions. We address each major comment below and indicate where revisions will be made.

read point-by-point responses

  1. Referee: [Abstract] Abstract and methods description: the manuscript states that a qualitative analysis was completed but supplies no information on coding procedure, number of coders, inter-rater checks, or how episodes were selected, so the link between data and reported themes cannot be evaluated.

    Authors: We agree that greater methodological transparency is needed. The full manuscript contains a methods section, but it does not sufficiently detail episode selection criteria, the coding process, number of coders, or inter-rater procedures. In the revised version we will expand this section to include these details (e.g., how the three seasons were sampled, the thematic analysis steps following Braun & Clarke, coder count, and any reliability checks), allowing readers to evaluate the link between data and themes. revision: yes

  2. Referee: [Abstract] Abstract, final sentence: the claim that the analysis 'gives a basis for future work on where artificial intelligence should and should not be used in gaming spaces' treats the recorded sessions as sufficient for identifying general success/failure patterns, yet no comparison to other groups, AIs, or non-podcast TTRPG play is reported.

    Authors: The sentence is deliberately phrased as providing 'a basis for future work' rather than asserting generalizable patterns. The study is an in-depth qualitative examination of one publicly available podcast; we do not claim the findings apply beyond this case or that the sessions are representative. This positioning is standard for exploratory qualitative work in HCI and does not require comparative data to be valid. We will, however, add a sentence in the discussion clarifying the exploratory, case-specific nature of the contribution. revision: partial

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity: descriptive qualitative analysis with no derivations or self-referential reductions

full rationale

The paper conducts thematic analysis on three seasons of a single podcast to identify roles of AI/humans, evaluations/failures, and person/character treatment. No equations, parameters, predictions, or fitted inputs exist. No self-citations, uniqueness theorems, or ansatzes are invoked. The claim that the analysis 'gives a basis for future work' is a direct summary of observed themes from the source material rather than a reduction to any prior input by construction. This is a standard self-contained qualitative report.

Assumptions & free parameters 0 free parameters · 1 assumptions · 0 invented entities

The central claim rests on the assumption that standard qualitative thematic analysis applied to podcast transcripts yields reliable, generalizable insights about AI use in gaming; no free parameters or invented entities are introduced.

assumptions (1)
  • domain assumption Qualitative thematic analysis of media content can identify meaningful patterns about technology use that support design recommendations.
    Invoked implicitly by performing the analysis and drawing design implications from the resulting themes.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Co-Creativity at the Table: A Qualitative Analysis of Creative Interactions in the Podcast "Adventure AI"." pith.science (2026). https://pith.science/paper/YR6N7CCX

@misc{pith2026260618010,
  author       = {Pith},
  title        = {Pith review of: Co-Creativity at the Table: A Qualitative Analysis of Creative Interactions in the Podcast "Adventure AI"},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/YR6N7CCX}},
  note         = {Machine review of arXiv:2606.18010}
}
read the original abstract

Tabletop role-playing games provide a unique environment for interaction with artificial intelligence (AI) due to their complex and collaborative nature. We analyze Adventure AI, a podcast featuring human-AI interactions in Dungeons & Dragons play, to examine how AI is and can be used in tabletop role-playing gaming and how players perceive this use. We complete a qualitative analysis of three seasons of this podcast, from 2023 to 2025, reporting on the overarching themes of roles of AI, roles of humans, the evaluations and failures of AI, and its treatment as a person and character at the table. There are many aspects of the game where artificial intelligence succeeds, while there are others where it is less appropriate. This analysis gives a basis for future work on where artificial intelligence should and should not be used in gaming spaces.

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

33 extracted references · 20 canonical work pages

  1. [1]

    Devi Acharya, Michael Mateas, and Noah Wardrip-Fruin. 2021. Story Improvisation in Tabletop Roleplaying Games: Towards a Computational Assistant for Game Masters. In2021 IEEE Conference on Games (CoG). 01–08. doi:10.1109/CoG52621.2021.9619006

    work page doi:10.1109/cog52621.2021.9619006 2021

  2. [2]

    Adventure AI. [n. d.]. Adventure AI. https://www.youtube.com/channel/UC9wSqJTcUtWUCZvLo5Bu0Lw

  3. [3]

    Karl Bergström. 2011. Framing Storytelling with Games. InInteractive Storytelling. Springer, Berlin, Heidelberg, 170–181. doi:10.1007/978-3-642- 25289-1_19

    work page doi:10.1007/978-3-642- 2011

  4. [4]

    David G. W. Birch. 2024. Hallucinations And Hope: Dungeons And Dragons, AI And The New Economy. https://www.forbes.com/sites/davidbirch/2024/12/27/hallucinations-and-hope-dungeons-and-dragons-ai-and-the-new-economy/

    2024

  5. [5]

    Boyd Branch, Piotr Mirowski, Kory Mathewson, Sophia Ppali, and Alexandra Covaci. 2024. Designing and Evaluating Dialogue LLMs for Co-Creative Improvised Theatre. arXiv:2405.07111

    work page arXiv 2024

  6. [6]

    2022.Thematic analysis: a practical guide

    Virginia Braun and Victoria Clarke. 2022.Thematic analysis: a practical guide. SAGE, London

    2022

  7. [7]

    Joanna Brooks, Serena McCluskey, Emma Turley, and Nigel King. 2015. The Utility of Template Analysis in Qualitative Psychology Research. Qualitative Research in Psychology12, 2 (April 2015), 202–222. doi:10.1080/14780887.2014.955224

    work page doi:10.1080/14780887.2014.955224 2015

  8. [8]

    Martin, Daphne Ippolito, Suma Bailis, and David Reitter

    Chris Callison-Burch, Gaurav Singh Tomar, Lara J. Martin, Daphne Ippolito, Suma Bailis, and David Reitter. 2022. Dungeons and Dragons as a Dialog Challenge for Artificial Intelligence. InProceedings of the 2022 Conference on Empirical Methods in Natural Language Processing. 9379–9393. arXiv:2210.07109 [cs] doi:10.18653/v1/2022.emnlp-main.637

    work page doi:10.18653/v1/2022.emnlp-main.637 2022

Show all 33 references
  1. [9]

    2010.The Creation of Narrative in Tabletop Role-Playing Games

    Jennifer Grouling Cover. 2010.The Creation of Narrative in Tabletop Role-Playing Games. McFarland & Company, Inc., Publishers, Jefferson, North Carolina London

    2010

  2. [10]

    Wesley Schneider, Ray Winninger, James Wyatt, Makenzie De Armas, and Inc Wizards of the Coast

    Jeremy Crawford, Christopher Perkins, Ben Petrisor, F. Wesley Schneider, Ray Winninger, James Wyatt, Makenzie De Armas, and Inc Wizards of the Coast. 2024.Dungeons & Dragons 2024 Player’s Handbook. Wizards of the Coast LLC, Renton, WA

    2024

  3. [11]

    Anders Drachen, Michael Hitchens, Ruth Aylett, and Sandy Louchart. 2009. Modeling the Story Facilitation of Game Masters in Multi-Player Role-Playing Games. InIntelligent Narrative Technologies II

    2009

  4. [12]

    Eladhari, Philip L

    Mirjam P. Eladhari, Philip L. Lopes, and Georgios N. Yannakakis. 2014. Interweaving Story Coherence and Player Creativity through Story- Making Games. InInteractive Storytelling, Alex Mitchell, Clara Fernández-Vara, and David Thue (Eds.). Springer International Publishing, 73–...

    2014 doi

  5. [13]

    1983.Shared Fantasy : Role-Playing Games as Social Worlds

    Gary Alan Fine. 1983.Shared Fantasy : Role-Playing Games as Social Worlds. University of Chicago Press, Chicago

    1983

  6. [14]

    Santiago Góngora, Luis Chiruzzo, Gonzalo Méndez, and Pablo Gervás. 2024. Skill Check: Some Considerations on the Evaluation of Gamemastering Models for Role-Playing Games. InGames and Learning Alliance, Pierpaolo Dondio, Mariana Rocha, Attracta Brennan, Avo Schönbohm, Francesc...

    2024 doi

  7. [15]

    Tuomas Harviainen

    J. Tuomas Harviainen. 2012. Ritualistic Games, Boundary Control, and Information Uncertainty.Simulation & Gaming43, 4 (Aug. 2012), 506–527. doi:10.1177/1046878111435395

    2012 doi

  8. [16]

    Hodhod and Brian S

    Rania A. Hodhod and Brian S. Magerko. 2014. Computational Creativity: Improv Agents and Conceptual Blending.International Journal of Cognitive Informatics and Natural Intelligence8, 2 (April 2014), 1–14. doi:10.4018/IJCINI.2014040101

    2014 doi

  9. [17]

    It’s Not Fair!

    Akrivi Katifori, Dimitra Petousi, Pantelis Sakellariadis, and Yannis Ioannidis. 2024. “It’s Not Fair!” - Exploring Game Master Unfairness in Tabletop Role Playing Games. InProceedings of the 19th International Conference on the Foundations of Digital Games (FDG ’24). Associati...

    2024 doi

  10. [18]

    Nicolas Legewie and Anne Nassauer. 2018. YouTube, Google, Facebook: 21st Century Online Video Research and Research Ethics.Forum Qualitative Sozialforschung / Forum: Qualitative Social Research19, 3 (Sept. 2018). doi:10.17169/fqs-19.3.3130

    2018 doi

  11. [19]

    Adventure AI

    Antonios Liapis, Georgios N. Yannakakis, Mark J. Nelson, Mike Preuss, and Rafael Bidarra. 2019. Orchestrating Game Generation.IEEE Transactions on Games11, 1 (March 2019), 48–68. doi:10.1109/TG.2018.2870876 Co-Creativity at the Table: A Qualitative Analysis of Creative Interac...

    2019 doi

  12. [20]

    2024.Technology Design Recommendations Informed by Observations of Videos of Popular Musicians Teaching and Learning Songs By Ear

    Christopher Liscio. 2024.Technology Design Recommendations Informed by Observations of Videos of Popular Musicians Teaching and Learning Songs By Ear. Ph. D. Dissertation. University of Waterloo, Waterloo, Ontario

    2024

  13. [21]

    2001.The Fantasy Role-Playing Game: A New Performing Art

    Daniel Mackay, Brooks McNamara, and Marshall Blonsky. 2001.The Fantasy Role-Playing Game: A New Performing Art. McFarland & Company, Inc., Publishers, Jefferson, N.C. London

    2001

  14. [22]

    Lara J Martin, Srijan Sood, and Mark O Riedl. 2018. Dungeons and DQNs: Toward Reinforcement Learning Agents That Play Tabletop Roleplaying Games. InINT/WICED@AIIDE

    2018

  15. [23]

    Michael Mateas and Andrew Stern. 2005. Structuring Content in the Façade Interactive Drama Architecture.Proceedings of the AAAI Conference on Artificial Intelligence and Interactive Digital Entertainment1, 1 (June 2005), 93–98. doi:10.1609/aiide.v1i1.18722

    2005 doi

  16. [24]

    Rakesh K. Maurya. 2024. Using AI Based Chatbot ChatGPT for Practicing Counseling Skills Through Role-Play.Journal of Creativity in Mental Health19, 4 (2024), 513–528. doi:10.1080/15401383.2023.2297857

    2024 doi

  17. [25]

    Manticore Media. [n. d.]. Adventure AI Powered D&D One-Shots

  18. [26]

    Ulrich Hoppe

    Julia Othlinghaus-Wulhorst, Anne Mainz, and H. Ulrich Hoppe. 2019. Training Customer Complaint Management in a Virtual Role-Playing Game: A User Study. InTransforming Learning with Meaningful Technologies, Maren Scheffel, Julien Broisin, Viktoria Pammer-Schindler, Andri Ioanno...

    2019 doi

  19. [27]

    2024.Dungeon Master’s Guide(fifth edition ed.)

    Christopher Perkins and James Wyatt. 2024.Dungeon Master’s Guide(fifth edition ed.). Wizards of the Coast, Renton, WA

    2024

  20. [28]

    Jonathan Strugnell, Marsha Berry, Fabio Zambetta, and Stefan Greuter. 2018. Narrative Improvisation: Simulating Game Master Choices. In Interactive Storytelling, Rebecca Rouse, Hartmut Koenitz, and Mads Haahr (Eds.). Springer International Publishing, Cham, 428–441. doi:10.100...

    2018 doi

  21. [29]

    David Thue, Vadim Bulitko, Marcia Spetch, and Eric Wasylishen. 2007. Interactive Storytelling: A Player Modelling Approach.Proceedings of the AAAI Conference on Artificial Intelligence and Interactive Digital Entertainment3, 1 (June 2007), 43–48. doi:10.1609/aiide.v3i1.18780

    2007 doi

  22. [30]

    Susanna Vartinen, Perttu Hamalainen, and Christian Guckelsberger. 2024. Generating Role-Playing Game Quests With GPT Language Models.IEEE Transactions on Games16, 1 (2024), 127–139. doi:10.1109/TG.2022.3228480

    2024 doi

  23. [31]

    Xiao You, Pittawat Taveekitworachai, Siyuan Chen, Mustafa Can Gursesli, Xiaoxu Li, Yi Xia, and Ruck Thawonmas. 2024. Dungeons, Dragons, and Emotions: A Preliminary Study of Player Sentiment in LLM-driven TTRPGs. InProceedings of the 19th International Conference on the Foundat...

    2024 doi

  24. [32]

    Li Zhang, Marco Gillies, Kulwant Dhaliwal, Amanda Gower, Dale Robertson, and Barry Crabtree. 2009. E-Drama: Facilitating Online Role-play Using an AI Actor and Emotionally Expressive Characters.International Journal of Artificial Intelligence in Education19, 1 (Jan. 2009), 5–38

    2009

  25. [33]

    Andrew Zhu, Lara Martin, Andrew Head, and Chris Callison-Burch. 2023. CALYPSO: LLMs as Dungeon Master’s Assistants.Proceedings of the AAAI Conference on Artificial Intelligence and Interactive Digital Entertainment19, 1 (Oct. 2023), 380–390. doi:10.1609/aiide.v19i1.27534

    2023 doi

Pith tools

Reviewed June 26, 2026 · model on record in the stance chip above.