A modular framework for automated evaluation of procedural content generation in serious games with deep reinforcement learning agents

Eleftherios Kalafatis; Konstantia Zarkogianni; Konstantina Nikita; Konstantinos Mitsis; Maria Athanasiou

REVIEW 3 major objections 6 minor 48 references

A modular framework for automated evaluation of procedural content generation in serious games with deep reinforcement learning agents

T0 review · 3 major / 6 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read Deep reinforcement learning agents can serve as automated playtesters that rank procedurally generated content in serious games, with agents trained on adaptive content reaching a 97% peak win rate.

desk verdict A solid modular framework for PCG evaluation, with a modest case study whose statistical and proxy-validation gaps are real but fixable; worth serious review, not rejection. read the letter →

arxiv 2505.16801 v2 pith:46TGMYRN submitted 2025-05-22 cs.LG cs.AI

Eleftherios Kalafatis , Konstantinos Mitsis , Konstantia Zarkogianni , Maria Athanasiou , Konstantina Nikita This is my paper · ORCID

classification cs.LGcs.AI

keywords seriousgamesproceduralcontentgenerationdeepreinforcementlearningautomatedgametestinggeneticalgorithmdynamicdifficultyadjustmentplayerexperience

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper tries to establish that a modular framework using deep reinforcement learning (DRL) agents can produce meaningful data for evaluating procedurally generated content (PCG) in serious games. The framework trains DRL playtesters against different versions of a card-based serious game about obstructive sleep apnea: one version generates non-player characters randomly, and two versions generate them with a genetic algorithm that adapts content to the player's performance. In the validation experiment, agents trained on the two adaptive versions reached a 97% peak win rate and achieved statistically significant higher win rates than agents trained on the random version, which peaked at 94%. The authors argue this shows the framework can distinguish between PCG techniques and reveal properties such as dynamic difficulty adjustment without requiring human playtesters. If valid, automated evaluation could lower the cost of developing and tuning personalized serious games.

What carries the argument

The load-bearing mechanism is the four-space modular framework itself. It is built from a Serious Game space, an Interaction Interpreter space that converts game mechanics into vectorized observations and actions, a Deep Reinforcement Learning space that trains PPO agents, and a Testing space that runs two evaluation protocols: a Comprehensive Test that exhaustively pits agents against all content combinations, and a Scenario-Based Test that pits trained agents against generated NPCs under regular gameplay conditions. All spaces communicate through API interfaces, so swapping the game only requires rewriting the interpreter module. Within the experiment, the genetic-algorithm PCG is the content generator under evaluation; it maintains a population of 20 NPC profiles, selects the five fittest as parents, and applies crossover and 5% mutation to produce the next generation. The framework's claim to meaningful evaluation rests on comparing agent win rates and training dynamics across PCG versions.

What would settle it

Run the same three game versions with a different DRL algorithm, or with human players, and compare the resulting ranking of versions with the PPO win-rate ranking; if the ranking flips or human measures disagree, the framework's evaluation signal is not a reliable proxy. A concrete observation: if human players show no learning-gain or competence advantage on Versions 2 and 3 over Version 1, or if agents trained with a second algorithm rank Version 1 above the genetic-algorithm versions, the central claim would be refuted.

Watch

Extended reading notes

Core claim

On its own terms, the paper's central discovery is that the performance of DRL game-testing agents is a usable evaluation signal for PCG integration in serious games. The authors trained twenty PPO agents on each of three versions of the "Wake Up for the Future" game for one million steps, saving checkpoints every ten thousand steps. In the comprehensive test, agents trained on the two genetic-algorithm versions (Versions 2 and 3) peaked at a 97% max win rate, while the random version (Version 1) peaked at 94%. In the scenario-based test, Kruskal-Wallis tests found a statistically significant effect of training version on win rate (p = 0.007 for randomly generated opponents, p = 0.008 for PCG-generated opponents), and Mann-Whitney U tests localized the difference to comparisons between the random version and each genetic-algorithm version. The paper takes this as evidence that the framework can rank PCG techniques and that it produces data on attribute selection that tracks the GA's intended dynamic difficulty adjustment.

Load-bearing premise

The whole evaluation rests on the assumption that how often a DRL agent wins after one million training steps is a valid proxy for the quality or effectiveness of procedurally generated content in a serious game for humans.

Editorial extensions

If this is right

PCG techniques for serious games could be tuned and compared without recruiting human playtesters, reducing the cost of early-stage game development.
The framework can distinguish between closely related PCG implementations, as evidenced by the measured differences between the unconstrained and normalized genetic-algorithm versions.
Agents trained on adaptive, difficulty-adjusting content reach higher win rates faster, suggesting dynamic difficulty adjustment aids both human and artificial players.
The attribute-frequency logs provide an audit trail for PCG behavior, showing whether a generator converges to a restricted content distribution or maintains diversity near the theoretical expectation.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

A direct test of the framework's validity would be to run the same three game versions with human players and check whether their learning outcomes or self-reported competence match the agent win-rate ordering; the paper does not make this comparison.
Because only PPO was used, the evaluation signal may be algorithm-dependent; training the same agents with a different DRL algorithm could reverse the ranking between PCG versions.
Win rate conflates content quality with content exploitability: a generator that produces weaker opponents would inflate agent win rates without necessarily improving the serious game's learning outcomes.
The GA's mutation rate of 5% and parent-selection scheme are not varied in the study, so the framework's sensitivity to PCG parameter changes remains untested.

Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

The headline: this is a legitimate framework paper, not a breakthrough. The four-space architecture (Serious Game, Interaction Interpreter, DRL, Testing) with GII/DEI interfaces is a genuinely useful way to organize automated evaluation of PCG in serious games. The writing is clear, and the case study is honest: three versions of an existing OSA game, PPO agents, 20 seeds, two test protocols. The finding that GA-based content leads to higher agent win rates than random content replicates the authors' earlier human-tester result [40], which is a meaningful sanity check.

What the paper does well: it identifies a gap—no existing framework targets PCG evaluation in serious games—and fills it with a modular design that should be adaptable to other games. The distinction between Comprehensive and Scenario-Based tests is sensible. The authors also report policy collapse in v1/v3, which shows they're not just cherry-picking the headline win rates.

The soft spots are all in the validation. The IQR outlier removal drops 6 of 60 agent instances and paired seeds are removed with them, but there's no sensitivity analysis showing the result holds without that step. The Kruskal-Wallis and Mann-Whitney tests are run on seed-paired data as if the groups were independent, which is wrong; the seeds are matched across versions, so a paired test is more appropriate. The Fig. 2 label "Stable Diffusion 3" is a typo (should be Stable-Baselines3), minor but sloppy. No code or data artifacts are provided, which makes the 60-agent experiment hard to audit.

The bigger conceptual issue, which the stress-test note raises, is the proxy question: agent win rate is treated as evidence of PCG quality, but the paper never compares agent metrics to human outcomes within the framework. The citation to [19] (PPO attention correlates with human attention) is suggestive, not evidence for this specific game. Still, I don't think this is fatal. The paper explicitly frames itself as a framework producing data for human evaluation, and it defers qualitative human interviews to future work. The claim that needs softening is the abstract's "capability to produce meaningful data"—for now, the evidence shows the framework can discriminate between PCG variants in one game, with the discrimination aligned with a prior human study. That's a reasonable start, not a fully validated substitute for human testing.

My verdict: conditional accept for the framework, with requested revisions on statistical rigor, sensitivity analysis, artifacts, and claim calibration. This deserves serious peer review; it's a solid engineering paper with a clearly identified niche. I'd bring it to reading group as a good example of how to structure an evaluation pipeline for adaptive content.

Recommend: send to review, expect heavy revision but not rejection.

Referee Report

3 major / 6 minor

Summary. The paper proposes a modular four-space framework (Serious Game, Interaction Interpreter, Deep Reinforcement Learning, Testing) for using DRL game-testing agents to evaluate procedural content generation in serious games. The framework is validated on a card-based serious game about obstructive sleep apnea, comparing three content-generation versions: random NPC generation (Version 1) and two genetic-algorithm variants with unnormalized versus normalized weight updates (Versions 2 and 3). Twenty PPO agents per version are trained for one million steps, and evaluation is conducted through a Comprehensive Test and a Scenario-Based Test. The reported results show higher win rates for agents trained on Versions 2 and 3 (peaking at 97%) than on Version 1 (peaking at 94%), with statistically significant Kruskal-Wallis differences in the Scenario-Based Test (p=0.007 for random NPCs, p=0.008 for PCG-generated NPCs). The paper concludes that the framework can produce meaningful data for evaluating PCG in serious games.

Significance. If the central claim is valid, the framework addresses a genuine gap: few systematic, modular tools exist for automated evaluation of PCG in serious games, where human playtesting is costly and content spaces are large. The paper's strengths include a clearly described modular architecture, the use of 20 random seeds per condition, explicit nonparametric statistical tests, and a comparison involving both exhaustive and scenario-based evaluation protocols. The reported results are also falsifiable in the narrow sense that the hypothesis 'GA-based content yields better agent metrics than random content' is tested and supported. However, the broader significance hinges entirely on whether DRL agent win rate is a valid proxy for player-relevant content quality, and this premise is not validated within the framework. The paper's claims are therefore plausible but not yet established at the level stated.

major comments (3)

[§III, §VI] The central claim that the framework produces 'meaningful data for the evaluation of procedurally generated content' depends on the unvalidated assumption that a PPO agent's win rate after 1M training steps is a valid proxy for player-relevant content quality or player experience. The paper's support is external: citation [19] reports correlation between PPO attention and human attention, and the results are said to align with the authors' earlier human-tester study [40]. Neither source provides a within-framework validation that agent win rates correspond to human outcomes in these exact SG versions. This matters because Versions 2 and 3 use a genetic algorithm that adapts content to the interacting agent; a PPO agent's play patterns and reward structure (+100/-2) may trigger different GA responses than a human player would, so the observed win-rate differences could reflect PPO-specific learnability under adaptive content rather than content quality for human players. The paper itself defers human validation to future work in §VI, so the evaluative conclusion is stronger than the evidence. I would ask for either a direct human-agent comparison on the same versions, an independent validation of the agent metric against known player-experience outcomes, or a substantial softening of the 'meaningful data' claim.
[§IV.D, §V.B] The Scenario-Based Test procedure introduces potential selection bias that is not fully reported. The authors select the 'best trained instance' of each agent based on win rate during the Comprehensive Test, then apply IQR-based outlier removal and also remove agents sharing seeds with outliers, yet they do not report the final number of agents per group after these steps. Selecting the best checkpoint for each agent is a form of peeking that can inflate apparent differences between versions, and removing outliers without sensitivity analysis can change the Kruskal-Wallis conclusions. The statistical claims (p=0.007 and p=0.008) need the final group sizes, a description of how many seeds were removed in each arm, and a re-analysis either without outlier removal or with a robust alternative such as rank-based tests on the full data.
[§V.A, §VI] The paper overstates the training-time superiority of Versions 2 and 3 relative to Version 1. Training metadata in Table II show overlapping average win rates (0.51±0.09 versus 0.57±0.06 and 0.57±0.07) and the Kruskal-Wallis test on total wins is not significant (p=0.08). The Discussion states that 'win rates achieved in the first 200k SGAs indicate faster training' but no statistical test is reported for that early-training window, and multiple-comparison corrections are absent. Additionally, the Discussion says the results from 'both tests' support the superiority of Versions 2 and 3, but no significance tests are reported for the Comprehensive Test. These claims should be qualified to the specific test and time window that actually produced significant results, or supported with appropriate statistical analyses.

minor comments (6)

[Fig. 2 caption] The caption refers to the 'DRL Model (Stable Diffusion 3, PPO)'; this appears to be an error, as Stable Diffusion 3 is a text-to-image model and the text elsewhere refers to Stable-Baselines3 for PPO. The caption should be corrected to avoid confusion.
[Abstract] The abstract reports 'p=0009'; this should read p=0.009, and it would be clearer to specify that this p-value refers to the Scenario-Based Test rather than training performance.
[§VI] The Discussion states that agents trained on Version 3 'played a total of 24,023 less duels than those trained in version 2 (Table 1)'; the reference should be to Table II, not Table I.
[§IV.A] The text says SG Version 1 'does not incorporate PCG' while also describing it as a version with random NPC creation and referring to three versions of PCG; this terminology should be clarified, for example by defining Version 1 as a baseline with random content generation rather than no PCG at all.
[§V.C, Fig. 6] The interpretation that 'changes in the selected attributes by the PCG correspond to changes in the agent's win rate' is based on visual inspection of one agent's curves; without a quantitative lag or correlation analysis, this should be presented as an illustrative observation rather than as evidence.
[Fig. 4 caption] The caption says dashed lines show average win rate and regular lines show max win rate, but the surrounding text in §V.B describes the opposite assignment; please verify and align the caption with the text.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity; the paper's claims are empirical validations of the framework rather than derivations that reduce to their inputs.

full rationale

The central derivation chain is an experiment: DRL agents are trained against three PCG variants and their measured win rates are compared. No equation or definition makes the outcome equivalent to an input. The experiment hypothesis that GA-based versions lead to better agent metrics is a prediction tested on fresh data, not a parameter fitted to those data; hyperparameters were screened on Version 1 and then applied across all versions, so the reported Version 2/3 win-rate advantage is not a fitted artifact. The appeal to the authors' prior human study [40] is used as convergent evidence that agent behavior 'comes in line with' human experience; that human study is an externally falsifiable, published result, so it does not make the argument circular. The framework's main limitation, that agent win rate has not been directly validated against human player outcomes in this paper, is a validity gap rather than a circularity: Section VI explicitly defers qualitative human interviews and additional showcase games to future work. Similarly, the observed attribute-rotation behavior in Fig. 6 is a measured correspondence, not an assumption used to construct the outcome. The fact that the GA is designed to adapt content explains why the hypothesis was plausible, but the observed win rates are still measurements of agent-environment interaction, not consequences imposed by the framework's definitions. Accordingly, no load-bearing step reduces to its own inputs, and no self-citation chain is used to forbid alternative interpretations.

Assumptions & free parameters 4 free parameters · 4 assumptions · 0 invented entities

The central claim does not introduce new physical or conceptual entities. The framework's software modules (GII, DEI, Metrics Logger) are architectural components, not invented entities in the sense of new forces or dimensions. The free parameters are standard RL and GA settings, with the reward function being the most consequential hand-tuned choice.

free parameters (4)

PPO hyperparameters = n_steps=128, batch_size=4, vf_coef=0.5, clip_range=0.2
Chosen via a grid search on Version 1 to optimize learning speed and performance (Section IV-C); applied to all versions, so tuning on the baseline could bias comparisons, though the direction is unexamined.
Reward weights = +100 for impactful actions and duel wins, -2 for other actions
Hand-designed in Section IV-C to emphasize impactful moves while avoiding heavy punishment. These values shape agent behavior and therefore the win-rate metric that drives all conclusions.
GA parameters = population=20, parents=5, mutation probability=5%
Inherited from the authors' prior work [40]; these control the dynamics of Versions 2 and 3 and affect how quickly agents learn, but they are not fitted to the current experiment's outcome.
Training length = 1,000,000 SGAs
Set by trial and error in Section IV-D as the point where more SGAs do not improve performance; it defines the agent instances used in all evaluations.

assumptions (4)

domain assumption A PPO agent's win rate is a valid proxy for the quality or player experience of procedurally generated content.
Used throughout to compare PCG versions; referenced to [19] and [40] but not directly validated within this framework (Section III, VI).
domain assumption The three SG versions differ only in their NPC generation method, with all other game mechanics and content identical.
Needed for a clean comparison between versions; stated in Section IV-A but not exhaustively verified in the paper.
domain assumption The serious game 'Wake Up for the Future' is representative of serious games with PCG.
The paper generalizes the framework's value beyond this one game, but only this game is tested (Section IV-A, Section VII).
domain assumption Using Mann-Whitney U tests on seed-paired agent samples is statistically valid despite the pairing.
The same 20 random seeds initialize agents for every version, creating paired data; the paper uses unpaired tests in Section V-B, which is a questionable choice.

how reviews work

0 comments

Cite this review

Pith. "Pith review of A modular framework for automated evaluation of procedural content generation in serious games with deep reinforcement learning agents." pith.science (2026). https://pith.science/paper/46TGMYRN

@misc{pith2026250516801,
  author       = {Pith},
  title        = {Pith review of: A modular framework for automated evaluation of procedural content generation in serious games with deep reinforcement learning agents},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/46TGMYRN}},
  note         = {Machine review of arXiv:2505.16801}
}

read the original abstract

Serious Games (SGs) are nowadays shifting focus to include procedural content generation (PCG) in the development process as a means of offering personalized and enhanced player experience. However, the development of a framework to assess the impact of PCG techniques when integrated into SGs remains particularly challenging. This study proposes a methodology for automated evaluation of PCG integration in SGs, incorporating deep reinforcement learning (DRL) game testing agents. To validate the proposed framework, a previously introduced SG featuring card game mechanics and incorporating three different versions of PCG for nonplayer character (NPC) creation has been deployed. Version 1 features random NPC creation, while versions 2 and 3 utilize a genetic algorithm approach. These versions are used to test the impact of different dynamic SG environments on the proposed framework's agents. The obtained results highlight the superiority of the DRL game testing agents trained on Versions 2 and 3 over those trained on Version 1 in terms of win rate (i.e. number of wins per played games) and training time. More specifically, within the execution of a test emulating regular gameplay, both Versions 2 and 3 peaked at a 97% win rate and achieved statistically significant higher (p=0009) win rates compared to those achieved in Version 1 that peaked at 94%. Overall, results advocate towards the proposed framework's capability to produce meaningful data for the evaluation of procedurally generated content in SGs.

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

48 extracted references · 33 canonical work pages

[40]

Procedural content generation based on a genetic algorithm in a serious game for obstructive sleep apnea,

K. Mitsis, E. Kalafatis, K. Zarkogianni, G. Mourkousis, and K. S. Nikita, “Procedural content generation based on a genetic algorithm in a serious game for obstructive sleep apnea,” in 2020 IEEE Conference on Games (CoG), Aug. 2020, pp. 694–697. doi: 10.1109/CoG47356.2020.9231785

arXiv 2020
[1]

However, the development of a framework to assess the impact of PCG techniques when integrated into SGs remains particularly challenging

ToG-2024-0198.R3 1 Abstract— Serious Games (SGs) are nowadays shifting focus to include procedural content generation (PCG) in the development process as a means of offering personalized and enhanced player experience. However, the development of a framework to assess the impact of PCG techniques when integrated into SGs remains particularly challenging. ...

work page 2024
[2]

The agents are then trained against three SG versions and checkpoints are saved throughout training

Training Process Pipeline; the DRL Model (Stable Diffusion 3, PPO) is initialized with 20 random seeds to create the agents. The agents are then trained against three SG versions and checkpoints are saved throughout training. This results in a set of trained instances for each agent and each SG version. ToG-2024-0198.R3 5 NPCs that possess 3 out of 7 avai...

work page 2024
[3]

ToG-2024-0198.R3 7 The Scenario-Based test pits the best agent trained instances from all versions against NPCs generated randomly (version

The maximum, median and minimum win rate achieved by agents for (a) Version 1 (b) Version 2 (c) Version 3 and cumulative reward gathered by agents for (d) Version 1 (e) Version 2 (f) Version 3 calculated every 500 SGAs. ToG-2024-0198.R3 7 The Scenario-Based test pits the best agent trained instances from all versions against NPCs generated randomly (version

work page 2024
[4]

Wake Up for the Future

The Evaluation Framework contains four spaces; Serious Game, Interaction Interpreter, Deep Reinforcement Learning and Testing. The Metrics Logger collects information from the Serious Game Deep Reinforcement Learning and Testing spaces and parses it for human evaluation. ToG-2024-0198.R3 4 Comprehensive Test and the Scenario-Based Test. The Comprehensive ...

work page 2024
[5]

Rising to the Challenge: An Emotion-Driven Approach Toward Adaptive Serious Games,

C. Schrader, J. Brich, J. Frommel, V. Riemer, and K. Rogers, “Rising to the Challenge: An Emotion-Driven Approach Toward Adaptive Serious Games,” in Serious Games and Edutainment Applications : Volume II, M. Ma and A. Oikonomou, Eds., Cham: Springer International Publishing, 2017, pp. 3–28. doi: 10.1007/978-3-319-51645-5_1

work page doi:10.1007/978-3-319-51645-5_1 2017
[6]

Wake Up for the Future

PCG Performance – Version 2 (a) Appearance rate of attributes for agent 0 (b) Win rate of agent 0 over SGAs of training. ToG-2024-0198.R3 9 increase overall competence of the agents trained on all versions, as the content provided through PCG during testing doesn’t affect the apparent win rate that these agents experience. This is possibly achieved by con...

work page 2024
[8]

The expected benefits of the procedurally generated game environments trade off against the great challenge of evaluating them [9]

tackles narrative generation, with emphasis on the scenario’s conclusion, as a way to improve personalization on SGs by exploring interactive digital narratives, player experience modelling and experience-driven PCG. The expected benefits of the procedurally generated game environments trade off against the great challenge of evaluating them [9]. Game tes...

work page 2024

Show all 48 references

[9]

Dashed lines show the average win rate and regular lines show the max win rate of the agent instances trained for the respective number of SGAs

Comprehensive Test. Dashed lines show the average win rate and regular lines show the max win rate of the agent instances trained for the respective number of SGAs. ToG-2024-0198.R3 8 performance over version 1 in almost all trained instances, as expected from the initial expe...

2024
[10]

What is procedural content generation? Mario on the borderline,

J. Togelius, E. Kastbjerg, D. Schedl, and G. N. Yannakakis, “What is procedural content generation? Mario on the borderline,” in Proceedings of the 2nd International Workshop on Procedural Content Generation in Games, in PCGames ’11. New York: Association for Computing Machine...

2011
[11]

Experience-Driven Procedural Content Generation,

G. N. Yannakakis and J. Togelius, “Experience-Driven Procedural Content Generation,” IEEE Trans. Affect. Comput., vol. 2, no. 3, pp. 147–161, Jul. 2011, doi: 10.1109/T-AFFC.2011.6

2011 doi
[12]

Integrated Approach to Personalized Procedural Map Generation Using Evolutionary Algorithms,

W. L. Raffe, F. Zambetta, X. Li, and K. O. Stanley, “Integrated Approach to Personalized Procedural Map Generation Using Evolutionary Algorithms,” IEEE Trans. Comput. Intell. AI Games, vol. 7, no. 2, pp. 139–155, Jun. 2015, doi: 10.1109/TCIAIG.2014.2341665

2015
[13]

Classifying serious games: the G/P/S model,

D. Djaouti, J. Alvarez, and J.-P. Jessel, “Classifying serious games: the G/P/S model,” in Handbook of research on improving learning and motivation through educational games: Multidisciplinary approaches, IGI global, 2011, pp. 118–136

2011
[14]

Deep learning, reinforcement learning, and world models,

Y. Matsuo et al., “Deep learning, reinforcement learning, and world models,” Neural Netw., vol. 152, pp. 267–275, Aug. 2022, doi: 10.1016/j.neunet.2022.03.037

2022 doi
[15]

An Ontology for Personalization in Serious Games for Assessment,

B. Said, L. Cheniti-Belcadhi, and G. El Khayat, “An Ontology for Personalization in Serious Games for Assessment,” in 2019 IEEE Second International Conference on Artificial Intelligence and Knowledge Engineering (AIKE), Jun. 2019, pp. 148–154. doi: 10.1109/AIKE.2019.00035

2019
[16]

2010, pp

ToG-2024-0198.R3 10 Games Based Learning, Copenhagen, Denmark: Academic Publishing Limited, Oct. 2010, pp. 412–417

2024
[17]

Serious Interactive Digital Narrative: Explorations in Personalization and Player Experience Enrichment,

A. Moradi-Karkaj, “Serious Interactive Digital Narrative: Explorations in Personalization and Player Experience Enrichment,” in 2021 International Serious Games Symposium (ISGS), Nov. 2021, pp. 35–42. doi: 10.1109/ISGS54702.2021.9685015

2021
[18]

Synchronizing Game and AI Design in PCG-Based Game Prototypes,

H. Bomström, M. Kelanti, J. Lappalainen, E. Annanperä, and K. Liukkunen, “Synchronizing Game and AI Design in PCG-Based Game Prototypes,” in International Conference on the Foundations of Digital Games, Bugibba Malta: ACM, Sep. 2020, pp. 1–8. doi: 10.1145/3402942.3402989

2020
[20]

Deep learning for procedural content generation,

J. Liu, S. Snodgrass, A. Khalifa, S. Risi, G. N. Yannakakis, and J. Togelius, “Deep learning for procedural content generation,” Neural Comput. Appl., vol. 33, no. 1, pp. 19–37, Jan. 2021, doi: 10.1007/s00521-020-05383-8

2021 doi
[21]

Human-Like Playtesting with Deep Learning,

S. F. Gudmundsson et al., “Human-Like Playtesting with Deep Learning,” in 2018 IEEE Conference on Computational Intelligence and Games (CIG), Aug. 2018, pp. 1–8. doi: 10.1109/CIG.2018.8490442

2018
[22]

Systematic Review of Dynamic Difficulty Adaption for Serious Games: The Importance of Diverse Approaches,

A. Seyderhelm and K. Blackmore, “Systematic Review of Dynamic Difficulty Adaption for Serious Games: The Importance of Diverse Approaches,” SSRN Electron. J., Jan. 2021, doi: 10.2139/ssrn.3982971

2021 doi
[23]

Wuji: Automatic Online Combat Game Testing Using Evolutionary Deep Reinforcement Learning,

Y. Zheng et al., “Wuji: Automatic Online Combat Game Testing Using Evolutionary Deep Reinforcement Learning,” in 2019 34th IEEE/ACM International Conference on Automated Software Engineering (ASE), Nov. 2019, pp. 772–784. doi: 10.1109/ASE.2019.00077

2019
[24]

Proximal Policy Optimization Algorithms,

J. Schulman, F. Wolski, P. Dhariwal, A. Radford, and O. Klimov, “Proximal Policy Optimization Algorithms,” Aug. 28, 2017, arXiv: arXiv:1707.06347. doi: 10.48550/arXiv.1707.06347

2017 doi arXiv
[25]

Automatic Parameter Optimization Using Genetic Algorithm in Deep Reinforcement Learning for Robotic Manipulation Tasks,

A. Sehgal, N. Ward, H. La, and S. Louis, “Automatic Parameter Optimization Using Genetic Algorithm in Deep Reinforcement Learning for Robotic Manipulation Tasks,” Nov. 01, 2022, arXiv: arXiv:2204.03656. doi: 10.48550/arXiv.2204.03656

2022 doi arXiv
[26]

Human-level control through deep reinforcement learning,

V. Mnih et al., “Human-level control through deep reinforcement learning,” Nature, vol. 518, no. 7540, Art. no. 7540, Feb. 2015, doi: 10.1038/nature14236

2015 doi
[27]

Machine versus Human Attention in Deep Reinforcement Learning Tasks,

S. (Sihang) Guo et al., “Machine versus Human Attention in Deep Reinforcement Learning Tasks,” in Advances in Neural Information Processing Systems, Curran Associates, Inc., 2021, pp. 25370–25385. Accessed: Jan. 12,

2021
[28]

Review of Intrinsic Motivation in Simulation-based Game Testing,

S. Roohi, J. Takatalo, C. Guckelsberger, and P. Hämäläinen, “Review of Intrinsic Motivation in Simulation-based Game Testing,” in Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems, in CHI ’18. New York, NY, USA: Association for Computing Machinery, A...

2018
[29]

Inspector: Pixel-Based Automated Game Testing via Exploration, Detection, and Investigation,

G. Liu et al., “Inspector: Pixel-Based Automated Game Testing via Exploration, Detection, and Investigation,” in 2022 IEEE Conference on Games (CoG), Aug. 2022, pp. 237–244. doi: 10.1109/CoG51982.2022.9893630

2022
[30]

Serious Game

with success. On the other hand, a study employing interactive evaluation involved educational games and recruited 150 students to evaluate the accuracy of employing support vector machines and GA for PCG [31]. Another example of interactive evaluation is a study of procedural...

2024
[33]

Personalized Dynamic Difficulty Adjustment Imitation Learning Meets Reinforcement Learning,

R. Fuchs, R. Gieseke, and A. Dockhorn, “Personalized Dynamic Difficulty Adjustment Imitation Learning Meets Reinforcement Learning,” in 2024 IEEE Conference on Games (CoG), Aug. 2024, pp. 1–2. doi: 10.1109/CoG60054.2024.10645659

2024
[34]

Expanding Expressive Range: Evaluation Methodologies for Procedural Content Generation,

A. Summerville, “Expanding Expressive Range: Evaluation Methodologies for Procedural Content Generation,” Proc. AAAI Conf. Artif. Intell. Interact. Digit. Entertain., vol. 14, no. 1, Art. no. 1, Sep. 2018, doi: 10.1609/aiide.v14i1.13012

2018 doi
[35]

Tools for Landscape Analysis of Optimisation Problems in Procedural Content Generation for Games,

V. Volz, B. Naujoks, P. Kerschke, and T. Tušar, “Tools for Landscape Analysis of Optimisation Problems in Procedural Content Generation for Games,” Appl. Soft Comput., vol. 136, p. 110121, Mar. 2023, doi: 10.1016/j.asoc.2023.110121

2023
[36]

Beyond Playing to Win: Creating a Team of Agents With Distinct Behaviors for Automated Gameplay,

C. Guerrero-Romero, S. Lucas, and D. Perez-Liebana, “Beyond Playing to Win: Creating a Team of Agents With Distinct Behaviors for Automated Gameplay,” IEEE Trans. Games, vol. 15, no. 3, pp. 469–482, Sep. 2023, doi: 10.1109/TG.2023.3241864

2023
[37]

Automated Playtesting With Procedural Personas Through MCTS With Evolved Heuristics,

C. Holmgård, M. C. Green, A. Liapis, and J. Togelius, “Automated Playtesting With Procedural Personas Through MCTS With Evolved Heuristics,” IEEE Trans. Games, vol. 11, no. 4, pp. 352–362, Dec. 2019, doi: 10.1109/TG.2018.2808198

2019
[38]

Automated Play-Testing through RL Based Human-Like Play-Styles Generation,

P. L. P. de Woillemont, R. Labory, and V. Corruble, “Automated Play-Testing through RL Based Human-Like Play-Styles Generation,” Proc. AAAI Conf. Artif. Intell. Interact. Digit. Entertain., vol. 18, no. 1, Art. no. 1, Oct. 2022, doi: 10.1609/aiide.v18i1.21958

2022 doi
[39]

A data-driven procedural-content-generation approach for educational games,

D. Hooshyar, M. Yousefi, M. Wang, and H. Lim, “A data-driven procedural-content-generation approach for educational games,” J. Comput. Assist. Learn., vol. 34, no. 6, pp. 731–739, 2018, doi: 10.1111/jcal.12280

2018 doi
[41]

General Video Game AI: Competition, Challenges and Opportunities,

D. Perez-Liebana, S. Samothrakis, J. Togelius, T. Schaul, and S. Lucas, “General Video Game AI: Competition, Challenges and Opportunities,” Proc. AAAI Conf. Artif. Intell., vol. 30, no. 1, Art. no. 1, Mar. 2016, doi: 10.1609/aaai.v30i1.9869

2016 doi
[42]

learn” and “predict

that features robustness and flexibility in complex DRL policies. These traits are of paramount importance to navigate the procedurally generated SG environments required to produced adequate testing data. SB3, an open-source framework, addresses this challenge by implementing...

2024
[43]

General Video Game AI: A Multitrack Framework for Evaluating Agents, Games, and Content Generation Algorithms,

D. Perez-Liebana, J. Liu, A. Khalifa, R. D. Gaina, J. Togelius, and S. M. Lucas, “General Video Game AI: A Multitrack Framework for Evaluating Agents, Games, and Content Generation Algorithms,” IEEE Trans. Games, vol. 11, no. 3, pp. 195–214, Sep. 2019, doi: 10.1109/TG.2019.2901021

2019
[44]

MAP-Elites to Generate a Team of Agents that Elicits Diverse Automated Gameplay,

C. Guerrero-Romero and D. Perez-Liebana, “MAP-Elites to Generate a Team of Agents that Elicits Diverse Automated Gameplay,” in 2021 IEEE Conference on Games (CoG), Aug. 2021, pp. 1–8. doi: 10.1109/CoG52621.2021.9619142

2021
[45]

Artificial Players in the Design Process: Developing an Automated Testing Tool for Game Level and World Design,

S. Stahlke, A. Nova, and P. Mirza-Babaei, “Artificial Players in the Design Process: Developing an Automated Testing Tool for Game Level and World Design,” in Proceedings of the Annual Symposium on Computer-Human Interaction in Play, Virtual Event Canada: ACM, Nov. 2020, pp. 2...

2020
[46]

Automated game testing using computer vision methods,

C. Paduraru, M. Paduraru, and A. Stefanescu, “Automated game testing using computer vision methods,” in 2021 36th IEEE/ACM International Conference on Automated Software Engineering Workshops (ASEW), Nov. 2021, pp. 65–72. doi: 10.1109/ASEW52652.2021.00024

2021
[47]

Game Description,

M. Genesereth and M. Thielscher, “Game Description,” in General Game Playing, M. Genesereth and M. Thielscher, Eds., in Synthesis Lectures on Artificial Intelligence and Machine Learning. , Cham: Springer International Publishing, 2014, pp. 13–29. doi: 10.1007/978-3-031-01569-4_2

2014 doi
[49]

Gymnasium: A Standard Interface for Reinforcement Learning Environments,

M. Towers et al., “Gymnasium: A Standard Interface for Reinforcement Learning Environments,” Nov. 08, 2024, arXiv: arXiv:2407.17032. doi: 10.48550/arXiv.2407.17032

2024 doi arXiv
[50]

DLR-RM/stable-baselines3. (Dec. 03, 2024). Python. DLR-RM. Accessed: Dec. 03,

2024
[51]

Creating Competitive Opponents for Serious Games through Dynamic Difficulty Adjustment,

D. Kristan, P. Bessa, R. Costa, and C. Vaz de Carvalho, “Creating Competitive Opponents for Serious Games through Dynamic Difficulty Adjustment,” Information, vol. 11, no. 3, Art. no. 3, Mar. 2020, doi: 10.3390/info11030156

2020 doi
[2022]

doi: 10.1007/978-981-19-0638-1

doi
[2023]

Available: https://proceedings.neurips.cc/paper/2021/hash/d58e2f077670f4de9cd7963c857f2534-Abstract.html

[Online]. Available: https://proceedings.neurips.cc/paper/2021/hash/d58e2f077670f4de9cd7963c857f2534-Abstract.html

2021

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.

[1] [40]

Procedural content generation based on a genetic algorithm in a serious game for obstructive sleep apnea,

K. Mitsis, E. Kalafatis, K. Zarkogianni, G. Mourkousis, and K. S. Nikita, “Procedural content generation based on a genetic algorithm in a serious game for obstructive sleep apnea,” in 2020 IEEE Conference on Games (CoG), Aug. 2020, pp. 694–697. doi: 10.1109/CoG47356.2020.9231785

arXiv 2020

[2] [1]

However, the development of a framework to assess the impact of PCG techniques when integrated into SGs remains particularly challenging

ToG-2024-0198.R3 1 Abstract— Serious Games (SGs) are nowadays shifting focus to include procedural content generation (PCG) in the development process as a means of offering personalized and enhanced player experience. However, the development of a framework to assess the impact of PCG techniques when integrated into SGs remains particularly challenging. ...

work page 2024

[3] [2]

The agents are then trained against three SG versions and checkpoints are saved throughout training

Training Process Pipeline; the DRL Model (Stable Diffusion 3, PPO) is initialized with 20 random seeds to create the agents. The agents are then trained against three SG versions and checkpoints are saved throughout training. This results in a set of trained instances for each agent and each SG version. ToG-2024-0198.R3 5 NPCs that possess 3 out of 7 avai...

work page 2024

[4] [3]

ToG-2024-0198.R3 7 The Scenario-Based test pits the best agent trained instances from all versions against NPCs generated randomly (version

The maximum, median and minimum win rate achieved by agents for (a) Version 1 (b) Version 2 (c) Version 3 and cumulative reward gathered by agents for (d) Version 1 (e) Version 2 (f) Version 3 calculated every 500 SGAs. ToG-2024-0198.R3 7 The Scenario-Based test pits the best agent trained instances from all versions against NPCs generated randomly (version

work page 2024

[5] [4]

Wake Up for the Future

The Evaluation Framework contains four spaces; Serious Game, Interaction Interpreter, Deep Reinforcement Learning and Testing. The Metrics Logger collects information from the Serious Game Deep Reinforcement Learning and Testing spaces and parses it for human evaluation. ToG-2024-0198.R3 4 Comprehensive Test and the Scenario-Based Test. The Comprehensive ...

work page 2024

[6] [5]

Rising to the Challenge: An Emotion-Driven Approach Toward Adaptive Serious Games,

C. Schrader, J. Brich, J. Frommel, V. Riemer, and K. Rogers, “Rising to the Challenge: An Emotion-Driven Approach Toward Adaptive Serious Games,” in Serious Games and Edutainment Applications : Volume II, M. Ma and A. Oikonomou, Eds., Cham: Springer International Publishing, 2017, pp. 3–28. doi: 10.1007/978-3-319-51645-5_1

work page doi:10.1007/978-3-319-51645-5_1 2017

[7] [6]

Wake Up for the Future

PCG Performance – Version 2 (a) Appearance rate of attributes for agent 0 (b) Win rate of agent 0 over SGAs of training. ToG-2024-0198.R3 9 increase overall competence of the agents trained on all versions, as the content provided through PCG during testing doesn’t affect the apparent win rate that these agents experience. This is possibly achieved by con...

work page 2024

[8] [8]

The expected benefits of the procedurally generated game environments trade off against the great challenge of evaluating them [9]

tackles narrative generation, with emphasis on the scenario’s conclusion, as a way to improve personalization on SGs by exploring interactive digital narratives, player experience modelling and experience-driven PCG. The expected benefits of the procedurally generated game environments trade off against the great challenge of evaluating them [9]. Game tes...

work page 2024