{"id":"28a7f158-3b70-4173-bf10-5282ad372331","arxiv_id":"2411.18049","paper_version":2,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":0,"one_line_summary":"An empirical study of 23 web data stories identifies six spatial immersion design patterns, and a 600-person comparison finds immersive variants more interesting and persuasive but not more understandable or trustworthy.","lead":"This paper studies web articles that use 3D scenes, camera movement, and interactive navigation to make readers feel inside the data. It finds these immersive stories are judged more interesting and persuasive, but not easier to understand or more trustworthy.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The paper never measures experienced spatial immersion, so the headline effects could stem from 3D novelty or animation rather than immersion; a presence manipulation check is needed before the central claim is accepted.","rationale":"The reader's weakest assumption correctly identifies the unmeasured construct of spatial immersion as the central vulnerability. My read of Sections IV, V, and VI-D confirms that the immersive variants are defined by the six design patterns rather than by any direct measure of experienced spatial immersion, and the authors explicitly defer immersion measurement to future work. This is more fundamental than the multiple-comparisons issue: even if all the reported p-values survive correction, the interpretation of the effect as 'spatial immersion' rather than '3D visual novelty' remains unsupported without a manipulation check. The concern does not warrant rejection because the paper's claims could be rephrased as being about design patterns for spatial immersion, or supported by a targeted follow-up study. The conditional verdict remains appropriate, so no adjustment is needed: the manuscript should be accepted only if the authors either add such a manipulation check or substantially soften the attribution to spatial immersion.","tokens_in":24610,"tokens_out":5219,"duration_ms":50389,"concrete_test":"Conduct a follow-up experiment using the same four stories and three variants, adding a validated spatial-presence measure (e.g., the spatial presence subscale of ITC-SOPI or the Witmer-Singer Presence Questionnaire) immediately after each reading condition, plus a perceived-novelty rating as a control. Fit a mixed-effects model with condition as a fixed effect, story and participant as random effects, and verify the immersive-versus-animated contrast on spatial presence. If the immersive condition does not score significantly higher on presence than the animated condition, or if a mediation model shows novelty rather than presence explaining the interest and persuasion effects, the central claim is not supported.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The load-bearing assumption is that the 'immersive' condition actually evokes spatial immersion. Section IV constructs the four immersive variants directly from the six design patterns, and Section V measures interest, understanding, persuasiveness, trustworthiness, and curiosity—but no manipulation check of spatial immersion or presence is included. The authors explicitly defer such measurement to future work in Section VI-D ('Future work could measure the degree of immersion by gathering eye movements or task completion time after immersive experience'). Without a presence or immersion measure, the observed advantages in interest (DS1, DS2, DS4) and persuasion (DS1, DS2) may be attributable to 3D visual novelty, perceived 'fanciness', or animation, rather than to spatial immersion per se. Participants in Section V-E describe the immersive variants with phrases like 'fancy 3D visual', 'almost an adventure', and 'up and close, personally looking into the statistics'; these comments indicate engagement and novelty, not verified spatial presence. The animated variant was designed to control for animation, but it does not control for 3D appearance or novelty, nor does it confirm that the immersive variant increases the intended construct. Consequently, the central claim cannot be distinguished from a claim about visual spectacle unless perceived spatial immersion is measured. This is a construct-validity gap, not a cosmetic omission: the paper's title and RQ2 ask about the impact of spatial immersion, and the abstract attributes the effects to 'design patterns for spatial immersion'.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"This paper investigates spatial immersion in web data stories. The authors collect 23 real-world data stories, identify six design patterns that purportedly promote spatial immersion, and then construct four new data stories, each in three variants (static, animated, immersive). A crowdsourced within-subjects study (N=600) compares the variants on five self-reported factors: interest, ease of understanding, persuasiveness, trustworthiness, and curiosity. The authors report that immersive variants received significantly more interest votes in DS1, DS2, and DS4, and more persuasiveness votes in DS1 and DS2, while no variant was consistently more understandable or trustworthy. Open-ended comments are analyzed qualitatively into four themes, leading to design implications and suggestions for authoring tools.","tokens_in":24846,"tokens_out":5453,"duration_ms":51063,"significance":"If the empirical claims withstand scrutiny, the paper offers a useful design vocabulary for immersive web data storytelling and one of the few controlled comparisons of static, animated, and immersive variants. The strengths are the systematic collection of 23 stories, the construction of four matched stimulus sets, the counterbalanced within-subjects design, and the qualitative analysis of participant comments. The authors also provide an openly accessible demo site, which supports reproducibility of the materials. However, the current analysis does not yet support the central causal interpretation about spatial immersion because the study never measures experienced immersion and the statistical reporting is incomplete; these issues need to be addressed before the paper's main claims can be accepted.","major_comments":[{"comment":"The quantitative results rest on 60 one-sample Wilcoxon signed-rank tests (5 factors × 4 stories × 3 pairwise comparisons) with no multiple-comparison correction. At α = 0.05, one would expect roughly three false positives by chance, so some of the highlighted significant results may be spurious. The paper also reports only significance stars rather than exact p-values, effect sizes, or confidence intervals. I recommend reporting exact test statistics and p-values, adding effect sizes (e.g., rank-biserial correlation), and applying a correction such as Benjamini–Hochberg, or at least justifying the uncorrected analysis.","section":"V-D, Fig. 6"},{"comment":"The procedure states that participants answered three knowledge-check questions after reading the first condition, but the paper never reports or analyzes these data. Since ease of understanding is measured only through a self-reported Likert item, the objective knowledge-check responses could provide important convergent evidence about comprehension. The omission is especially noticeable because the authors explicitly excluded memorability due to carryover concerns, suggesting they were aware of objective measures; the collected knowledge-check data should be analyzed and reported, or their absence should be explained.","section":"V-B and V-D"},{"comment":"The immersive condition is assumed to evoke spatial immersion because it implements the six design patterns, but no manipulation check of perceived spatial immersion or presence is reported. The animated condition controls for animation and dynamic transitions, but it does not control for 3D appearance or visual novelty. Participant comments quoted in V-E describe the immersive variants as 'fancy 3D visual' and 'almost an adventure,' which suggests engagement and novelty, not verified spatial presence. The paper's own limitation section (VI-D) defers measuring immersion to future work ('Future work could measure the degree of immersion by gathering eye movements or task completion time'), which is insufficient for the central RQ2 claim that spatial immersion affects the reader's experience. Without a presence or immersion measure, the observed effects on interest and persuasion could be attributed to 3D visual spectacle rather than to spatial immersion per se.","section":"IV, V-D, VI-D"},{"comment":"The abstract and conclusion state that 'data stories with the design patterns for spatial immersion' are more interesting and persuasive, but the experiment only compares three holistic variants, with each immersive story bundling two to four patterns. No pattern-level decomposition or manipulation check isolates the effect of the six-pattern set, and the patterns themselves were derived from the same collection of stories that guided the construction of the immersive stimuli. The causal attribution to the six patterns is therefore broader than what the experiment directly supports; the paper should either soften this claim or add an analysis that addresses the pattern-level contribution.","section":"Abstract, IV-V"}],"minor_comments":[{"comment":"There is a typo in 'cognitive absorbtion' in the related work section; the reference title uses 'Absorption'.","section":"II-A"},{"comment":"The caption says the bar charts are 'linearly concatenated,' which is unclear; please explain the arrangement and explicitly define what the asterisk denotes.","section":"Fig. 6 caption"},{"comment":"The claim that 'More than half of the participants in DS1, DS2, and DS3 rated extremely positive to the immersive stories when compared to the static' is not quantified; please provide exact counts or percentages.","section":"V-D"},{"comment":"The 'Kenny and Becker's VR content' reference appears only as a footnote URL; a formal citation would be more appropriate.","section":"IV-C"},{"comment":"The source name 'Visual Ciannamon' appears to be a typo for 'Visual Cinnamon.'","section":"Table I"},{"comment":"Eye movements and task completion time are behavioral proxies, not direct measures of experienced immersion; in future work, a validated presence questionnaire would be a more direct manipulation check.","section":"VI-D"}],"recommendation":"major_revision","confidential_remarks":"The manuscript addresses a timely and interesting topic, and the empirical materials and study design are a solid foundation. The main risk is construct validity: the paper uses 'spatial immersion' as the explanatory construct but never measures it, and the statistical reporting does not yet support the strength of the claims. These issues are fixable within a revision—for example, by adding a presence manipulation check, reporting corrected and effect-sized statistics, analyzing the collected knowledge-check data, and softening or re-scoping the causal claims. I do not see a reason for rejection, but the revision needs to be substantive rather than cosmetic."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"I read this one because the topic is timely: web data stories increasingly use 3D and camera movement, and we have little comparative evidence on what those choices do to readers. The paper gives us that evidence in a controlled, within-subjects study of 600 participants across four stories, each in static, animated, and immersive versions. That's a real contribution. The six design patterns—cinematic camera, intuitive encoding, natural movement, direct manipulation, realism, dynamic dimension—aren't novel in themselves, and the authors say so, but the collected 23-story corpus and the way the patterns are used to construct stimuli are useful for future work. The qualitative coding is thoughtful and the public demos are a plus.\n\nThe soft spots are real. The biggest is construct validity: the immersive condition is assumed to evoke spatial immersion because it was built from the patterns, but the study never measures presence or perceived immersion. Participants say things like \"fancy 3D visual,\" \"almost an adventure,\" \"drawn in,\" which could just be novelty or visual spectacle. The authors themselves list measuring immersion as future work. Without a manipulation check, the title and RQ2 claim about spatial immersion isn't directly supported; the effects could be about 3D graphics or animation richness.\n\nThen the statistics: 60 one-sample Wilcoxon tests with no multiple-comparison correction, no effect sizes or confidence intervals. That's likely to produce a few false positives among the significant results. Also, the knowledge-check questions collected in the procedure are never analyzed, which is a missed opportunity. And the abstract's \"more interesting and persuasive\" overstates the persuasiveness result—it was significant for only two of four stories, and the paper's own text says no condition consistently won on that factor.\n\nThe circularity concern is moderate: patterns are extracted from the 23 stories and then used to build the stimuli that \"validate\" them. That's acceptable if framed as design-pattern evaluation rather than taxonomy validation, but an independent measure of immersion would break the loop.\n\nWho benefits: data storytelling researchers, visualization designers, and anyone building web-based narrative visualizations. I'd send this to peer review with major revision—the empirical core is worth keeping, but it needs a manipulation check, corrected statistics, reported effect sizes, and a more careful abstract. If those are addressed, it's a solid reference for the field.","headline":"Solid empirical study of immersive web data stories, but the central claim outruns the data: no manipulation check and uncorrected multiple tests.","tokens_in":25379,"tokens_out":2911,"would_cite":true,"duration_ms":26552,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Web data stories that feel spatially immersive are more interesting and persuasive than static or animated versions, but they are not more understandable or trustworthy.","keywords":["data storytelling","spatial immersion","immersive analytics","design patterns","scrollytelling","crowdsourced user study","reader engagement","persuasiveness"],"falsifier":"Run the same four stories with an added condition: one built with the immersive design patterns, and a separate 'decorative 3D' condition that uses the same 3D rendering but omits spatial cues such as camera motion or depth coherence. If readers rate both conditions equally interesting and persuasive, the spatial patterns are not the active ingredient; if perceived presence scores predict the interest advantage, immersion itself carries the effect. Measuring presence with a short questionnaire in the original design would also test the assumption directly.","tokens_in":24431,"feed_emoji":"📊","tokens_out":4179,"duration_ms":34619,"temperature":0.7,"pith_summary":"This paper asks whether web data stories that give readers a feeling of moving through a 3D data space actually change how readers respond. Drawing on a collection of 23 existing stories, the authors identify six recurring design patterns for evoking spatial immersion, then build four of their own stories in static, animated, and immersive versions. In a crowdsourced within-subjects study of 600 readers, the immersive versions were rated significantly more interesting and persuasive in several stories, and also more curiosity-provoking, but no condition was consistently easier to understand or more trustworthy. The paper's point is that spatial immersion is a usable design lever with real but selective effects: it buys engagement and persuasion while risking clarity and perceived honesty.","feed_headline":"Immersive data stories sway readers on interest and persuasion","feed_subtitle":"A 600-reader study finds 3D-designed stories beat static and animated ones at engagement, but not at clarity or trust.","key_machinery":"The central object is a set of six design patterns (DP1–DP6) distilled from 23 spatially immersive web data stories: cinematic camera shots and transitions, intuitive encoding between phenomenon and representation, smoothly moving elements, direct manipulation of camera or visualization, realistic appearance, and dynamic dimension (switching between 2D and 3D views). These patterns are what the authors mean by spatial immersion in this web context — not head-mounted displays, but author-controlled virtual cameras, depth cues, and motion that make readers feel present in a data space. The patterns carry the argument twice: they are the taxonomy claimed to describe how practitioners create immersion, and they are the ingredient list used to build the four immersive experimental stories.","core_discovery":"The paper claims that the deliberate use of six design patterns — cinematic camera shots and transitions, intuitive data representations, realism, naturally moving elements, direct manipulation of camera or visualization, and dynamic dimension — reliably evokes spatial immersion in web data stories, and that this immersion shifts reader experience in a specific way: it makes stories more interesting, more persuasive, and more likely to spark curiosity, without improving (and sometimes hurting) understanding and trust. The claim rests on a controlled comparison in which four data stories were each built in three variants that share text and data but differ only in visual design. Across the five cognitive factors measured, immersive variants received significantly more interest votes in DS1, DS2, and DS4 and more persuasiveness votes in DS1 and DS2, with no consistent winner on ease of understanding or trustworthiness.","pith_inferences":["The study does not measure whether readers actually felt spatially immersed, so if the interest and persuasion gains come from 3D novelty or animation rather than spatial immersion itself, the proposed mechanism would need revision.","A direct test is possible: add a presence or immersion questionnaire, or include a condition with 3D visuals that deliberately omit the spatial patterns, and see whether the measured advantage tracks perceived immersion.","The selective results suggest a cost-benefit model: immersion raises attention and emotional engagement while also raising reading cost and suspicion, so its net value depends on the story's goal rather than on immersion alone.","The pattern list is a snapshot of pre-2022 practice, and newer techniques such as VR-compatible scrollytelling suggest that the taxonomy will likely expand."],"forward_implications":["Designers who want readers to find a story engaging and persuasive can add camera-driven 3D scenes, realistic textures, and natural motion without sacrificing the underlying text content.","For facts that must be understood precisely or trusted fully, static or animated 2D presentations remain the safer choice, and immersion should be reserved for topics where comprehension is already easy.","Hybrid designs that alternate clear 2D views with immersive 3D scenes may combine the engagement gains with the understanding and trust of simpler presentations.","Future authoring tools could expose these patterns as high-level primitives, since the authors estimate that building the immersive variant accounted for roughly 90 percent of development time.","The six patterns give researchers a vocabulary for describing and comparing immersive web stories beyond the specific examples collected."],"supporting_citations":[{"why":"Supplies the genre-and-tactics design space for narrative visualization that the collection review extends.","marker":"[18]"},{"why":"Provides the immersion-as-user-experience versus immersion-as-technology framing that locates the paper's approach.","marker":"[8]"},{"why":"Defines spatial immersion as a distinct concept and supplies criteria for judging whether a story evokes it.","marker":"[12]"},{"why":"Provides the definition of visual data stories used to filter collected stories for data facts, message, and narrative-visualization interplay.","marker":"[17]"},{"why":"Models the crowdsourced study approach for measuring reader engagement with visual narrative flow.","marker":"[11]"},{"why":"Frames immersive visual data stories as a research area and motivates the focus on immersion in data storytelling.","marker":"[38]"}],"fun_headline_variants":["Immersive data stories are more interesting and persuasive","Immersive design boosts interest but not clarity or trust","3D-designed stories beat static and animated on engagement","Spatial immersion makes web stories persuasive, not clearer","Immersive stories: more persuasive, not more trustworthy"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The immersive variants are treated as spatially immersive because they implement the six design patterns, but the study never measures whether readers actually experienced spatial immersion or presence, so the observed effects might stem from 3D novelty or visual fanciness rather than immersion itself.","fun_headline_variants_meta":{"raw":{"variants":["Immersive data stories are more interesting and persuasive","Immersive design boosts interest but not clarity or trust","3D-designed stories beat static and animated on engagement","Spatial immersion makes web stories persuasive, not clearer","Immersive stories: more persuasive, not more trustworthy"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000589,"raw_usage":{"total_tokens":2722,"prompt_tokens":857,"completion_tokens":1865,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":473,"completion_tokens_details":{"reasoning_tokens":1800}},"tokens_in":473,"tokens_out":1865,"duration_ms":12160,"temperature":1.0,"reasoning_tokens":1800,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-12T11:32:50.474789+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run the same four stories with an added condition: one built with the immersive design patterns, and a separate 'decorative 3D' condition that uses the same 3D rendering but omits spatial cues such as camera motion or depth coherence. If readers rate both conditions equally interesting and persuasive, the spatial patterns are not the active ingredient; if perceived presence scores predict the interest advantage, immersion itself carries the effect. Measuring presence with a short questionnaire in the original design would also test the assumption directly.","supporting_citations":[{"cited_title":"Narrative visualization: Telling stories with data,","cited_arxiv_id":null,"evidence_quote":"Supplies the genre-and-tactics design space for narrative visualization that the collection review extends."},{"cited_title":"Immersion revisited: A review of existing definitions of immersion and their relation to different theories of presence,","cited_arxiv_id":null,"evidence_quote":"Provides the immersion-as-user-experience versus immersion-as-technology framing that locates the paper's approach."},{"cited_title":"Spatial immersion versus emotional immersion, which is more immersive?","cited_arxiv_id":null,"evidence_quote":"Defines spatial immersion as a distinct concept and supplies criteria for judging whether a story evokes it."},{"cited_title":"More than telling a story: Transforming data into visually shared stories,","cited_arxiv_id":null,"evidence_quote":"Provides the definition of visual data stories used to filter collected stories for data facts, message, and narrative-visualization interplay."},{"cited_title":"Visual narrative flow: Exploring factors shaping data visualization story reading experiences,","cited_arxiv_id":null,"evidence_quote":"Models the crowdsourced study approach for measuring reader engagement with visual narrative flow."}],"review_version":1}