{"id":"91efa729-7793-48cf-9f14-b9c4c95e7f88","arxiv_id":"2501.15770","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":0,"one_line_summary":"ProcrastiMate, a serious game with a counselor role and customized stories, helped 27 college students learn about procrastination causes and coping strategies while maintaining psychological distance.","lead":"Researchers built a text adventure game called ProcrastiMate that lets players act as counselors helping fictional students overcome procrastination. A two-week field study with 27 participants found the game helped players reflect on their own procrastination and try new coping strategies in a psychologically safe way.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The central claim that ProcrastiMate caused learning and reflection rests on self-report and no control condition; the observed effects could be demand characteristics or the pre-interview, and the customization of Level 2 stories makes the relatability finding partly circular.","rationale":"The reader's weakest_assumption identifies precisely the same load-bearing concern: no control condition and reliance on self-report leave the causal link between the game and the observed learning/emotion changes unestablished. My stress-test concurs and sharpens the point in two ways. First, the learning measure is entirely qualitative self-report coded from post-interviews; there is no pre-test of procrastination knowledge, so the claim of 'learning' cannot be distinguished from participants simply re-describing their own pre-existing views or from demand characteristics. Second, the customization pipeline (Section 5.1.3, Section 6.2 Stage 2) means Level 2 stories were built from each participant's own pre-interview elicitations; a post-interview statement like 'that is exactly me' (Section 7.3.1) is then not an independent signal of the game's narrative design, but a consequence of the input content. This circularity does not invalidate the design idea, but it does mean the relatability finding cannot stand as evidence that the game, rather than the participant's own story, generated the reflection. The paper is a well-conducted qualitative study with transparent methods and a plausible contribution, which is why I would not move the verdict to reject or unverified. The existing CONDITIONAL verdict already accommodates the need for a controlled follow-up, so I recommend no change to the reader's verdict. The concrete test I propose is a straightforward RCT with an active control and an objective knowledge measure; that single experiment would settle whether the concern lands.","tokens_in":27019,"tokens_out":4599,"duration_ms":47772,"concrete_test":"Run a three-arm randomized controlled trial with N≈30 per arm: (1) ProcrastiMate, (2) an active-control text-adventure game of matched length and structure but without procrastination content, and (3) a no-intervention condition. Administer a validated 20-item objective knowledge test on Steel's four causes of procrastination and matching coping strategies, plus the NPEAPS and a daily task-completion log, at pre-test, post-test, and 4-week follow-up. If the ProcrastiMate arm does not show significantly larger knowledge gains and reductions in negative procrastination-related emotions than the active-control arm, the central claim fails; if it does, the concern is resolved.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The paper's central claim is that the two-week field study revealed that ProcrastiMate facilitated learning and self-reflection while maintaining psychological distance, and motivated integration of knowledge into daily life (Abstract, Section 7). The evidence for this claim is (a) a pre/post reduction in negative emotions measured by NPEAPS (Section 7.1, t(26)=2.869, p<0.01) and (b) qualitative themes from post-interviews coded into Kolb's Experiential Learning Cycle (Section 7.2). The load-bearing assumption is that these outcomes are caused by the game itself. This assumption is not secured: there is no control condition, no baseline measure of procrastination knowledge, and no objective proxy for learning or behavior. Participants knew the study aimed to help them learn about procrastination through a game, so post-interview reports of learning, reflection, and real-world experimentation are vulnerable to social desirability and experimenter demand. The pre/post NPEAPS change could also reflect the normalizing effect of the pre-interview discussion about procrastination, regression to the mean, or simply the passage of two weeks. Additionally, the customization of Level 2 stories directly from each participant's pre-interview disclosures (Section 5.1.3) means that findings of strong relatability and self-reflection (Section 7.3.1) may be recognition of one's own narrative rather than evidence that the game's design features produced learning. The authors acknowledge the lack of a control group only partially in Section 8.4, but the central causal claim remains unquantified. If these self-reports are not caused by the game, the paper's contribution as a serious game intervention is not established.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper presents ProcrastiMate, a text-adventure serious game intended to help college students learn about procrastination's psychological causes and coping strategies. The design is grounded in Piers Steel's four-cause framework and a 40-card strategy deck, and it implements three design considerations: customized stories (Level 2 built from each participant's pre-interview), a counselor role, and adaptive GPT-4-driven dialogues. A formative Wizard-of-Oz study with 5 players and 2 experts led to three design goals, and the final game was evaluated in a two-week field study with 27 self-selected participants using pre/post interviews, the NPEAPS emotion scale, and descriptive learning/engagement/empathy scales. The findings report significant pre/post reduction in negative emotions and qualitative themes mapped to Kolb's Experiential Learning Cycle, plus evidence of relatability, psychological safety, and real-life strategy experimentation. The paper contributes a design rationale and empirical insights for using serious games for procrastination education.","tokens_in":27299,"tokens_out":4270,"duration_ms":41115,"significance":"If the results hold, the paper makes a useful contribution to HCI by shifting procrastination interventions from purely behavioral tools toward experiential learning, and it articulates a concrete design principle—balancing personal relevance with psychological distance—that could inform serious games for other sensitive psychological topics. The strengths include the theory-grounded game content (Steel's framework and 40 strategy cards), the clear articulation of design goals, the rich qualitative quotes from participants, and the detailed supplementary materials (prompt templates and interview outlines) that improve reproducibility. However, the evidence is limited by the absence of a control condition, the lack of any objective learning measure, the self-selected and homogeneous sample, and the partly circular design of the personalization; these limitations bound the strength of the causal claims that can be drawn.","major_comments":[{"comment":"The central claim that ProcrastiMate 'facilitated learning and self-reflection' rests on a within-subjects field study with no control condition. The significant reduction in negative emotions (t(26)=2.869, p<0.01) could be produced by the pre-interview discussion about procrastination, demand characteristics, or regression to the mean. The paper should either add a comparison condition (e.g., an information-only or non-personalized game arm) or explicitly temper the causal claims and discuss these alternative explanations with reference to the data.","section":"Section 6.2 and 7.1"},{"comment":"The finding that players found the stories highly relatable is partly circular because the Level 2 stories were generated directly from each participant's own pre-interview disclosures. The quotes in 7.3.1 (e.g., 'mirrored my own experiences exactly') may reflect recognition of one's own narrative rather than a general property of the game design. To support the claim that the customization design is effective, the paper should compare relatability between the personalized Level 2 cases and the non-personalized Level 1 cases, or at least explicitly discuss this confound.","section":"Section 5.1.3 and 7.3.1"},{"comment":"The post-interview was scheduled when participants reached 'the midpoint of Level 2,' meaning learning and experience measures were collected before the game was completed. This timing should be reported as a limitation and its potential effect on the observed outcomes discussed, since partial exposure may understate or alter the learning effects relative to full playthrough.","section":"Section 6.2 (Stage 3)"},{"comment":"The limitations section omits three threats that directly qualify the main claims: the lack of a control group, the self-selected and single-university sample with mostly high procrastination tendencies, and the reliance on self-report without any objective measure of learning or behavior change. These should be added and their implications for the claims in the abstract and Discussion should be acknowledged.","section":"Section 8.4"}],"minor_comments":[{"comment":"The descriptive statistics for the Experiencing Scale, EGameFlow, and State Empathy are reported as means without inferential statistics or comparison benchmarks; consider clarifying that these are auxiliary descriptive indicators rather than evidence of effectiveness.","section":"Section 7.1"},{"comment":"The figure caption is missing a period, and the figure is referenced in the text without a clear pointer to its full content; also the callouts 'Figure 3-b' and 'Figure 3-c' could be introduced more explicitly in Section 4.2.","section":"Figure 3"},{"comment":"The three design goals are listed without explicitly tying each goal to the finding from Section 4.3 that motivated it; adding a sentence of connection would improve readability.","section":"Section 4.4"},{"comment":"Please report effect sizes for the NPEAPS pre/post comparisons to help readers gauge the magnitude of the change.","section":"Section 7.1"},{"comment":"The description of the GPT-4 integration is clear, but the paper should mention whether any guardrails or validation were used to ensure the generated feedback is faithful to the theory-grounded content; this is relevant to reproducibility.","section":"Section 5.5"}],"recommendation":"major_revision","confidential_remarks":"This is a well-written design study with a compelling prototype and rich qualitative data, and I believe it can become a publishable CHI paper. The main issue is the gap between the strong causal claim in the abstract ('revealed that ProcrastiMate facilitated learning') and the evidence provided. The omissions in Section 8.4 are surprising given that the design has clear methodological limitations; a careful revision that tempers the claims and adds the missing limitations discussion would address the core concern. I would also encourage the editor to weigh whether a two-week, no-control field study can support the word 'facilitated' without further qualification."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Here's my read on arXiv:2501.15770. This is a solid, well-documented exploratory study: a text-adventure serious game that teaches procrastination causes and coping strategies, with a formative study feeding into design and a 27-participant two-week field study. The genuinely new thing is the design approach — casting the player as a counselor helping NPCs whose stories are partly customized from the player's own pre-interview disclosures — and the associated insight that balancing personal relevance with psychological distance is what makes the learning feel safe and effective. That is a real contribution to the serious-games-for-psychoeducation space, and the qualitative data (Kolb-cycle coding, quotes) are rich and credible. The authors also include a thoughtful limitations section and report tensions (e.g., participants worrying that understanding causes could rationalize procrastination), which speaks well of their honesty.\n\nThe main soft spot is the one everyone will see: no control condition, no objective learning or behavior measure, and the central claim that the game caused the observed changes rests on self-report and one pre/post emotion scale (NPEAPS). There is also a real circularity in the relatability finding: Level 2 stories are built from each participant's own pre-interview, so 'that's exactly me' is partly by construction. The stress-test note has it right. That said, this is an exploratory field study, not an RCT, and the authors partially acknowledge the lack of a control group in Section 8.4. My concern is more about framing: the abstract and discussion lean on 'revealed' and 'facilitated' in ways that imply causation. That should be toned down to 'reported' / 'may support' / 'suggests.' I'd also like to see a more careful separation of the NPEAPS result from the qualitative claims, since regression to the mean is a live alternative.\n\nWho is this for? HCI researchers working on serious games, psychoeducation, and procrastination interventions. It deserves a serious referee: the design reasoning is systematic, the artifact is clearly described, and the empirical material is useful even if it cannot bear the full causal weight. I'd send it to review but ask for revision that (a) reframes claims as exploratory, (b) addresses the circularity risk explicitly, and (c) ideally adds a comparison arm or at least a delayed self-report to strengthen the evidence. I would not desk-reject it.","headline":"A solid exploratory study of a serious game for procrastination learning with a genuinely useful design insight, but the evaluation oversells the causal claim; worth reviewing with requests for tempering and further validation.","tokens_in":27868,"tokens_out":2235,"would_cite":false,"duration_ms":20597,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"The paper claims that ProcrastiMate, a text-adventure game where players act as counselors for procrastinating non-player characters, supports experiential learning about procrastination, reduces negative emotions tied to it, and…","keywords":["Procrastination","Serious games","Learning","Reflection","Text adventure game","Psychological distance","Experiential learning","Coping strategies"],"falsifier":"A randomized controlled trial in which one group plays ProcrastiMate and a control group receives an equally engaging non-procrastination game or a neutral discussion, followed by a validated knowledge test and an objective behavioral measure such as actual time-to-start on a real assignment, would settle the causal claim; if the control group shows the same emotion and behavior changes, the game is not the cause.","tokens_in":26846,"feed_emoji":"🎮","tokens_out":5876,"duration_ms":52516,"temperature":0.7,"pith_summary":"This paper argues that the best way to intervene on procrastination may be to teach people about it, not just help them manage time. It presents ProcrastiMate, a text-adventure game in which college students play a counselor helping fictional students overcome procrastination, with stories customized from players' own experiences and adaptive dialogue feedback. A two-week field study with 27 self-identified procrastinators found that the game supported learning and self-reflection, significantly lowered negative procrastination-related emotions, and led players to experiment with new coping strategies in their daily lives. The authors' central proposal is that balancing personal relevance with psychological distance—letting players engage with familiar scenarios through another character's perspective—makes this educational approach safe and effective.","feed_headline":"Text-adventure game cuts negative emotions tied to procrastination","feed_subtitle":"In a two-week field study, 27 students playing as counselors reported learning, self-reflection, and real-life strategy use.","key_machinery":"The load-bearing mechanism is the counselor role combined with customized narratives and adaptive dialogue. Players are assigned the role of a procrastination counselor who helps non-player characters (NPCs) solve procrastination cases, where each case is built from Piers Steel's four causes of procrastination and the correct coping strategy cards map to those causes. Level 1 uses stories drawn from a shared pool of college procrastination scenarios, Level 2 personalizes stories from each player's pre-interview, and GPT-4-generated NPC feedback explains in red or green text why a selected strategy does or does not work. This machinery creates the psychological distance that lets players reflect on self-relevant content without feeling judged, which the paper identifies as the key to safe and effective learning.","core_discovery":"The central discovery is a design mechanism: casting players as counselors for in-game characters whose procrastination stories mirror their own lets players learn the causes and coping strategies of procrastination without triggering the defensiveness that direct \"you are procrastinating\" framing causes. The paper grounds the game content in Piers Steel's four-cause model of procrastination (low self-efficacy, low perceived task value, high impulsiveness, and distant delay) and a set of 40 coping strategy cards, then evaluates the game through Kolb's experiential learning cycle. In the field study, participants moved through concrete experience, reflective observation, abstract conceptualization, and active experimentation, reporting that they began analyzing their own triggers in the four-cause framework, reconsidered old strategies, and tried new ones in real life. The quantitative finding that negative emotions about procrastination decreased after play is offered as evidence that learning reduced guilt and self-blame.","pith_inferences":["A head-to-head comparison between the counselor-framed game and a direct second-person version would isolate whether psychological distance, rather than the game's content, is the active ingredient; the paper's design cannot separate these.","Because the adaptive dialogues were generated by GPT-4, a simpler scripted-dialogue condition would reveal how much of the scaffolding effect depends on LLM-driven responsiveness versus well-written feedback content.","The study measured emotion and self-reported application only at two weeks; an objective follow-up such as assignment submission times or task-start latency would test whether the reduced negative emotion and reported experimentation translate into sustained behavioral change."],"forward_implications":["If the finding holds, HCI interventions for procrastination can target understanding and emotional reframing rather than only time management and task tracking.","Players' reports of applying new strategies in daily life suggest that game-acquired coping knowledge can transfer beyond the virtual environment, even without a built-in behavior-change mechanism.","The counselor-role pattern—indirect, other-directed engagement with self-relevant scenarios—can be reused for other psychologically sensitive topics such as emotional eating or impostor syndrome.","The significant drop in negative procrastination-related emotion implies that educational gameplay may interrupt the guilt-anxiety-procrastination cycle at the emotional level, complementing behavioral tools."],"supporting_citations":[{"why":"Supplies the four-cause procrastination framework that structures the game's chapters, cases, and coping strategy cards.","marker":"[72]"},{"why":"Justifies the educational approach by arguing that understanding procrastination is the key to overcoming it.","marker":"[73]"},{"why":"Provides the Experiential Learning Cycle model used to code and interpret participants' learning processes.","marker":"[40]"},{"why":"Supplies the NPEAPS scale used to measure the pre/post change in negative and positive procrastination-related emotions.","marker":"[42]"},{"why":"Provides the thematic analysis method used to analyze the qualitative interview data.","marker":"[11]"},{"why":"Supplies the General Procrastination Scale used to characterize participants' trait procrastination tendencies.","marker":"[44]"}],"fun_headline_variants":["Text adventure game cuts procrastination guilt via role-play","Counseling avatars in a game eases procrastination distress","Role-playing as counselor in a game reduces procrastination guilt","Game lets you walk in others' shoes to cut procrastination guilt","Playing counselor in a game lowers procrastination self-blame"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The findings assume that the game itself—not the pre-interview, the act of participating in a study about procrastination, or a desire to please the researchers—produced the measured drop in negative emotion and the reported learning and behavior changes, because there was no control group and learning was only self-reported.","fun_headline_variants_meta":{"raw":{"variants":["Text adventure game cuts procrastination guilt via role-play","Counseling avatars in a game eases procrastination distress","Role-playing as counselor in a game reduces procrastination guilt","Game lets you walk in others' shoes to cut procrastination guilt","Playing counselor in a game lowers procrastination self-blame"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.001166,"raw_usage":{"total_tokens":4789,"prompt_tokens":875,"completion_tokens":3914,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":491,"completion_tokens_details":{"reasoning_tokens":3830}},"tokens_in":491,"tokens_out":3914,"duration_ms":25838,"temperature":1.0,"reasoning_tokens":3830,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-10T13:56:47.733011+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"A randomized controlled trial in which one group plays ProcrastiMate and a control group receives an equally engaging non-procrastination game or a neutral discussion, followed by a validated knowledge test and an objective behavioral measure such as actual time-to-start on a real assignment, would settle the causal claim; if the control group shows the same emotion and behavior changes, the game is not the cause.","supporting_citations":[{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Supplies the four-cause procrastination framework that structures the game's chapters, cases, and coping strategy cards."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Justifies the educational approach by arguing that understanding procrastination is the key to overcoming it."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Supplies the NPEAPS scale used to measure the pre/post change in negative and positive procrastination-related emotions."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Supplies the General Procrastination Scale used to characterize participants' trait procrastination tendencies."}],"review_version":1}