{"id":"c9ef5606-39db-4e71-bd82-0445353f2177","arxiv_id":"2505.03807","paper_version":1,"verdict":"CONDITIONAL","confidence":"HIGH","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":4,"one_line_summary":"A multi-agent system with VLM and RAG lets users talk with stage-aware Harry Potter characters and customize scenes, with a user study reporting enhanced engagement.","lead":"This paper presents an interactive system that lets viewers chat with AI-simulated characters from video stories and customize new scenes. It combines a vision-language model, retrieval-augmented generation, and multiple AI agents to create characters that appear to grow as the story advances.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The claim of emergent character growth is not yet distinguished from stage-dependent retrieval: growth is guaranteed by Equations 3-4, and the case studies show exactly the variation the design produces, so the central claim needs a control condition before it can be accepted as being about the…","rationale":"The paper's contribution is a system prototype and design space, and much of what it claims about usability and interaction is plausibly supported by the formative interviews and the 18-participant user study. The weak point is specifically the 'emergent character growth' claim in the Abstract and Conclusion. The load-bearing assumption is that stage-varying responses constitute psychological growth rather than retrieval-induced variation; this assumption is the same one the reader identified, and it is the least secure part of the central claim. The concern is addressable with control conditions, a baseline, and released prompts/code, and the paper itself flags hallucination and the need to study perceived growth in Section 8.3.3. There is no basis to move to REJECT, because the design, formative study, and user feedback are genuine evidence for the system's interactive value even if the growth interpretation is not yet validated. I therefore leave the reader's CONDITIONAL verdict unchanged.","tokens_in":26131,"tokens_out":2489,"duration_ms":24850,"concrete_test":"Run a paired computational experiment on a fixed query bank: for each query, generate responses at stage 1 and stage 5 under four conditions: (A) the current system, (B) stage-1 context with stage-5 retrieved chunks, (C) stage-5 context with stage-1 retrieved chunks, and (D) no RAG. Blind-rate the responses for stage appropriateness and for psychological growth indicators (self-awareness, relationship awareness, other-awareness). If condition B responses resemble stage-5 responses, or condition C resembles stage-1 responses, the observed 'growth' is driven by retrieval context rather than character evolution. Also probe questions deliberately orthogonal to plot facts (values, moral dilemmas, preferences) and check whether stage-dependent shifts persist; if they vanish, the growth claim reduces to factual knowledge accumulation.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The strongest claim is that the system 'effectively portrays emergent character social behavior and growth' (Abstract; Conclusion). The load-bearing requirement is that stage-to-stage variation in character responses reflects the characters' psychological development, not merely a swapping of retrieved context. Equations 3 and 4 make stage-dependence a design feature: C_stage_i is built by retrieving stage-specific chunks, and O_character_i = LLM(C_stage_i + Q_user + P_character_i + I_MAS_i). Equation 2 defines d(growth) = f'(stage) * d(stage), so any change in retrieved context yields nonzero d(growth) by construction. The Section 6.2 case studies (Figure 10) illustrate exactly the variation the architecture guarantees: 'Who is Dobby?' gets 'We haven't encountered him yet' in stage 1 and a detailed answer in stage 5 because the stage-5 context contains Dobby's introduction. This demonstrates knowledge accumulation in the retrieval corpus, not necessarily growth in the character. Section 7.4.1 reports that perceived growth is most evident for users who have never seen the film, and that the user's pre-knowledge and stage order affect gains; Section 8.3.3 concedes that 'user's perception of character growth still needs attention.' The user evaluation has no baseline (no-RAG, shuffled-stage, or same-stage repeated-question condition) and no statistical comparison, so the study cannot distinguish 'the character has developed' from 'the stage-5 context contains the answer.' This is a validity concern about the central claim, not an internal inconsistency, and it is addressable with control conditions.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper describes an interactive video-story system that combines a vision-language model (VLM) for video understanding, retrieval-augmented generation (RAG), and a multi-agent system (MAS) to let users chat with characters at different narrative stages and to customize story scenes. The system is applied to the first two Harry Potter films, with narrative data divided into five stages. The authors report a formative interview study with 12 experts, a user study with 18 participants, and case examples intended to show emergent character growth and social behavior. The central claim, stated in the abstract and conclusion, is that the system effectively portrays emergent character social behavior and growth and enhances the interactive experience.","tokens_in":26426,"tokens_out":2869,"duration_ms":32494,"significance":"If the central claim were fully supported, the system would make a useful contribution to interactive storytelling and human-computer interaction by integrating machine video understanding, retrieval-augmented generation, and multi-agent role-play in a single workflow. The formative study, the concrete system implementation, and the qualitative user feedback are valuable design resources, and the paper is commendably explicit about several limitations, including model hallucination, limited VLM visual understanding, and the need for further work on perceived character growth. However, the evidence presented does not yet distinguish system-designed stage-dependent retrieval from genuine, emergent character development, which is the load-bearing part of the claimed contribution.","major_comments":[{"comment":"The claim of emergent character growth is largely guaranteed by construction. Equation 3 defines the context C_stage_i as chunks retrieved for a specific story stage, Equation 4 makes the character output O_character_i depend directly on C_stage_i, and Equation 2 defines d(growth) = f'(stage) * d(stage). Any change in retrieved context therefore produces a change in output that the paper labels as growth. The case study in Section 6.2 (Figure 10), such as the stage-1 versus stage-5 answer to \"Who is Dobby?\", demonstrates knowledge accumulation in the retrieval corpus, not necessarily psychological development of the character. To support the abstract and conclusion claims, the authors should add a control condition that separates retrieval-driven knowledge updating from character-level developmental change, for example by comparing against a no-RAG condition, a shuffled-stage retrieval condition, or repeated questions within the same stage.","section":"Section 5.2.2, Eqs. (2)-(4)"},{"comment":"The user evaluation does not include any baseline condition or statistical comparison. With 18 participants, a 7-point Likert questionnaire, and follow-up interviews, the ratings shown in Figures 14-15 can support qualitative usability insights but cannot by themselves establish that the system \"effectively portrays emergent character social behavior and growth.\" The paper also reports in Section 7.3 that model hallucination tends to make character responses converge or include irrelevant information, and Figure 15 shows a large standard deviation for the \"difference in persona of different characters is noticeable\" item. These observations weaken the strong claim and should be addressed either with additional comparative data or with a more circumscribed statement of what the user study actually demonstrates.","section":"Section 7.2-7.3, Table 1 and Figure 15"},{"comment":"The paper's own analysis shows that perceived character growth depends on the user's prior familiarity with the story and on the order of stages experienced. Specifically, users who had only read part of the story \"often struggle to discern the characters' growth,\" users who had never seen the film mainly perceive knowledge accumulation, and users who were almost completely familiar perceive self-awareness and other-awareness. This indicates that the reported growth may be substantially driven by the user's external knowledge and the stage-ordering of the interface, rather than by an intrinsic, emergent property of the agent. The authors should either control for familiarity and stage order in the analysis or qualify the growth claim accordingly.","section":"Section 7.4.1"},{"comment":"The classification of character responses into knowledge accumulation, other-awareness, self-awareness, and relationship awareness is presented as evidence of social behavior, but the coding procedure is not described. No coding scheme, inter-rater reliability measure, or systematic sampling method is given, so it is unclear how the categories were applied and whether they are reproducible. Since this classification is the main evidence for the \"social behavior\" component of the central claim, the authors should provide a transparent coding protocol or reframe the four categories as illustrative observations rather than measured outcomes.","section":"Section 6.2"}],"minor_comments":[{"comment":"There is a grammatical error in the sentence \"The dialogue extracted from raw movie By applying the 'moviepy' library,\" and the audio extraction step should be described in a complete, proofread sentence.","section":"Section 4.2"},{"comment":"Several bullets under \"Task\" contain incomplete expressions with missing symbols, such as \"Characters & Conflict + ,\" and \"Plot + ,\" which makes the mapping between tasks and narrative elements difficult to follow. These should be fixed before publication.","section":"Section 3.2"},{"comment":"The function f(stage) appearing in Eq. (2) is not formally defined. The authors should state what f represents, how it is computed, and whether it is a theoretical device or an implemented quantity in the system.","section":"Section 5.2.2, Eq. (2)"},{"comment":"Equation 3 writes C_stage_i as a sum over Chunk_j but does not specify the range of j or the relationship between chunks and stages. Clarifying this notation would help readers understand how retrieval exactly selects stage-specific context.","section":"Section 5.2.2, Eq. (3)"},{"comment":"The text says the right side shows mean and standard deviation for each question, but the figure does not display numerical values or error bars, and the main text does not report the actual means and standard deviations. Reporting these numbers in a table would make the evaluation results concrete.","section":"Figure 15"},{"comment":"The sentence \"all 8 users mentioned\" appears inconsistent with the stated participant count of 18. Please check whether this refers to a subset and clarify the wording.","section":"Section 8.1.2"}],"recommendation":"major_revision","confidential_remarks":"The core system contribution is credible as an integration of existing techniques, but the paper's headline claim of emergent character growth needs substantially stronger evidence or more modest wording. If the authors can add a control condition and a more rigorous analysis of the user study, the manuscript could become acceptable. The current evidence is primarily qualitative and, as the paper itself notes in several places, the perception of growth is fragile. I would also encourage the authors to check the formatting of equations and bullets in Section 3.2, as several placeholder symbols appear to have been lost in the PDF conversion."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Colleague,\n\nQuick read on arXiv:2505.03807. The real contribution is the design space and prototype, not the 'emergent character growth' finding. The paper does solid work mapping user intentions (formative study, 12 professionals) into a three-stage system that combines VLM-based video understanding, RAG retrieval, and multi-agent role-playing. The 'interaction space' idea, trans-temporal sharing, and scene customization are genuinely useful additions to interactive storytelling for fixed video content. The interface design is thoughtful, and the user study, while small, is used honestly.\n\nThe soft spot is exactly what the stress-test flags: the central claim that characters 'grow' is built into the retrieval design. Equation 3 retrieves stage-specific chunks, Equation 4 makes the output a function of that context, and Equation 2 defines growth as a function of stage. So the case studies in Figure 10 demonstrate that the system returns different answers at different stages—which the architecture guarantees. 'Who is Dobby?' gets 'haven't met him' at stage 1 and a biography at stage 5 because the stage-5 context contains Dobby. That is knowledge accumulation in the corpus, not evidence of psychological growth. The authors even concede in 7.4.1 that perceived growth is strongest for viewers unfamiliar with the film, and in 8.3.3 that user perception of growth needs attention.\n\nThis isn't a fatal flaw for the systems contribution, but it means the paper's strongest sentence—'effectively portrays emergent character social behavior and growth'—is not supported by the evidence as presented. The evaluation has no baseline (no-RAG, shuffled-stage, or repeated-question control) and no statistical comparison. That is addressable. A modest revision that adds one control condition and rephrases 'emergent growth' as 'stage-dependent variation' would make the paper considerably more honest.\n\nWho should read it: people building LLM-based interactive narrative systems, and anyone interested in how to (and how not to) evaluate 'emergence' in LLM agents. The design space in Section 3 is worth reading; the evaluation section should be read with the equations in hand.\n\nRecommendation: send it to peer review. It deserves referee time, but the referees should push for a control condition and language that distinguishes designed stage-dependence from emergent psychological growth.\n\nBest.","headline":"A genuinely useful design space and prototype for interactive video storytelling, but the headline 'emergent character growth' claim is largely a product of the stage-dependent retrieval design and needs a control condition before it can be taken at face value.","tokens_in":26998,"tokens_out":2313,"would_cite":true,"duration_ms":23726,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"A multi-agent system lets viewers chat with video-story characters that appear to grow and evolve across story stages.","keywords":["video story interaction","multi-agent system","retrieval-augmented generation","vision language model","character growth","interactive narrative","scene customization","Harry Potter"],"falsifier":"A reader could test this by asking the same question at two stages and swapping the retrieved stage contexts: if responses still look stage-coherent, or if users cannot identify which stage a response came from, then the growth effect is driven by retrieval prompts rather than by the characters' accumulated narrative state.","tokens_in":25897,"feed_emoji":"🎬","tokens_out":6340,"duration_ms":63440,"temperature":0.7,"pith_summary":"This paper claims that a fixed, pre-shot video story can support genuinely interactive, personalized exploration if the machine first understands the story across vision, audio, and plot, and then lets users talk with characters who are driven by a multi-agent system grounded in the current narrative stage. The intended payoff is that characters show growth potential: the same question asked at an early stage and a late stage gets answers reflecting different knowledge, self-concept, and relationships, so users feel they are walking alongside evolving people rather than selecting pre-written branches. The paper reports a formative interview study with 12 professionals, a system applied to the first two Harry Potter movies, and a user study with 18 participants in which the growth-capable characters and scene customization were rated the most useful functions. This matters because interactive video so far has been limited to choice trees and specially designed narratives; the paper's route would in principle work on existing films.","feed_headline":"Chat with Harry Potter as he grows across the story","feed_subtitle":"Video-story system pairs language-model agents with retrieval so characters answer differently at every stage.","key_machinery":"The load-bearing mechanism is the stage-conditioned generation loop defined by Equations 3 and 4: a RAGAgent computes stage-specific context $C_{stage_i}$ as a chunked retrieval from vision, audio, and plot text, then each character agent produces $O_{character_i} = LLM(C_{stage_i}+Q_{User}+P_{character_i}+I_{MAS_i})$. The interaction memory stream records each user query and each character's reply so that later rounds are conditioned on prior dialogue. The growth potential is conceptually defined by Equation 2, $d(growth)=f'(stage)\\cdot d(stage)$, making character change a function of narrative stage by construction.","core_discovery":"The central claim is that retrieval-augmented generation combined with a multi-agent system lets a fixed video story support open-ended, stage-aware dialogue with characters that appear to grow. Each narrative stage has its own interaction space; a RAGAgent retrieves stage-specific chunks of plot, dialogue, and visual description for the user's question, and character agents answer from that context, so the same question asked in stage 1 and stage 5 draws on different knowledge and yields different self-descriptions. The paper interprets these differences as emergent social behavior and growth, using objective self-awareness theory to classify them as knowledge accumulation, awareness of others, self-awareness, and relationship awareness. It reports that 18 users rated the character-growth and scene-customization modules highest and that participants familiar with the story observed stage-consistent changes in Harry's and Hermione's responses.","pith_inferences":["If the growth effect is mostly a retrieval effect, the architecture should reproduce perceived character growth on any serialized story, so a direct test is to run the same design on a non-Harry-Potter corpus and measure whether users report growth without the authors' chosen story.","A stronger causal test than the paper reports would scramble stage labels: asking the same question with stage-1 context mislabeled as stage-5 and vice versa, then asking users to judge which stage each response belongs to.","The interaction memory stream is session-local; the paper does not give characters persistent memory of one user across days, so a natural extension is to make growth user-specific: a character that remembers prior conversations with the same viewer and changes accordingly.","The same stage-conditioned character design could transfer to educational or biographical interaction, where 'stage' is a curriculum or a life period, though the paper only gestures at children's learning in the discussion."],"forward_implications":["Open-ended questions can replace fixed choice points: users ask what they want, and the RAGAgent retrieves the stage-specific context needed for an in-character answer.","Characters diverge: the same question produces different answers from the same character at different stages and from different characters at the same stage, which users read as growth and personality.","Dialogue can drive visualization: a scene mentioned in chat can be expanded, re-perspectived, or given biography in WebVR, so the user's own assumptions become visible.","The approach works on existing films with fixed plots, such as Harry Potter, without rewriting the narrative.","Stage-wise interaction spaces let users experience the story as a progression and notice changes between stages, which was the most highly rated function in the user study."],"supporting_citations":[{"why":"Provides the character-centered video story understanding benchmark the system builds on.","marker":"[13]"},{"why":"Shows role-playing agents trained on character data, the approach that cannot generalize across stories.","marker":"[56]"},{"why":"Supplies the lifelong-learning agent concept that motivates agents with growth potential.","marker":"[64]"},{"why":"Provides retrieval-augmented generation, the technique behind the RAGAgent's stage-specific context.","marker":"[21]"},{"why":"Demonstrates multi-agent dialogue chains used to motivate multi-character discussions.","marker":"[50]"},{"why":"Supplies the self-awareness theory used to interpret response changes as character growth.","marker":"[35]"},{"why":"A Harry Potter dialogue-agent benchmark the paper contrasts with and extends toward stage-aware growth.","marker":"[10]"}],"fun_headline_variants":["Multi-agent systems bring video stories to life","RAG-powered character chats that change per story stage","Ask video characters anything, at any point in the plot","From any video, a chatty cast that remembers the story"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that the stage-dependent variation in the characters' answers is genuine evidence of psychological growth rather than an effect of swapping which story facts are placed in front of the language model.","fun_headline_variants_meta":{"raw":{"variants":["Multi-agent systems bring video stories to life","RAG-powered character chats that change per story stage","Ask video characters anything, at any point in the plot","From any video, a chatty cast that remembers the story"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000263,"raw_usage":{"total_tokens":1560,"prompt_tokens":864,"completion_tokens":696,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":480,"completion_tokens_details":{"reasoning_tokens":632}},"tokens_in":480,"tokens_out":696,"duration_ms":8001,"temperature":1.0,"reasoning_tokens":632,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-16T04:25:45.434874+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"A reader could test this by asking the same question at two stages and swapping the retrieved stage contexts: if responses still look stage-coherent, or if users cannot identify which stage a response came from, then the growth effect is driven by retrieval prompts rather than by the characters' accumulated narrative state.","supporting_citations":[{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Provides the character-centered video story understanding benchmark the system builds on."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Supplies the lifelong-learning agent concept that motivates agents with growth potential."}],"review_version":1}