pith. sign in

arxiv: 2606.07794 · v1 · pith:DHGQA426new · submitted 2026-06-05 · 💻 cs.HC

The Choreography of Augmented Reality Timelines: Studying the Relative Position, Chronology, & Situatedness of Event Sequences

Isabelle Kwan , Jessica Ziyu Chen , Matthew Brehmer This is my paper

Pith reviewed 2026-06-27 20:43 UTC · model grok-4.3

classification 💻 cs.HC
keywords timeline visualizationaugmented realityspatial timelinestemporal mental modelschoreographic designevent sequencesimmersive storytellingformative study
0
0 comments X

The pith

A choreographic approach to arranging events in space supports flexible augmented reality timelines.

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

Traditional timeline tools enforce strict chronological order, which can flatten the irregular, context-rich nature of personal and historical stories. The paper reports a study in which twelve participants freely placed event cards in a physical room, revealing varied strategies that incorporated relative position, pacing, and situational details. From these observations the authors derive a choreographic method for timeline construction and demonstrate it with a tablet-based AR prototype that lets users draw and view timelines in physical space. If the approach holds, it would allow timeline stories to preserve lived irregularity while remaining legible to others who move through the same space.

Core claim

The authors claim that strategies observed when participants arranged events in a physical environment can be synthesized into a choreographic approach to timeline creation. This approach treats timeline construction as the design of movement paths and viewpoints rather than the imposition of a single linear sequence. They implemented the approach in a proof-of-concept tablet AR application that supports spatial drawing and viewing of event sequences, and they reflect on how chronology, pacing, and spatial context can be encoded together in immersive timeline stories.

What carries the argument

The choreographic approach, which treats timeline creation as the arrangement of events in physical space to encode relative position, chronology, and situated context.

If this is right

  • Timeline visualizations can simultaneously encode strict chronology and irregular personal pacing without forcing one to dominate the other.
  • Immersive AR interfaces can treat the viewer's physical movement as an active part of story comprehension rather than passive consumption.
  • Designers gain concrete tactics for placing events so that spatial proximity signals narrative relationships beyond mere time order.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • The same spatial-arrangement logic could be tested in non-AR settings such as large physical installations or projected room-scale displays.
  • Future work might measure whether choreographic timelines improve recall accuracy or emotional engagement compared with conventional linear tools.
  • The method may extend to collaborative storytelling where multiple people contribute events to a shared physical space.

Load-bearing premise

Observations from twelve participants freely arranging events can be turned into general design principles for AR timeline systems.

What would settle it

A controlled test in which new users attempt to interpret and navigate timelines created with the choreographic method and report systematic confusion about event order or context.

Figures

Figures reproduced from arXiv: 2606.07794 by Isabelle Kwan, Jessica Ziyu Chen, Matthew Brehmer.

Figure 1
Figure 1. Figure 1: (A—C) In our formative study, participants placed events (dates + descriptions) as sticky notes around a room, which [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. Figure 2: These timeline sketches show participants’ placement of sticky notes in Part 1 of our formative study. Icons in the [PITH_FULL_IMAGE:figures/full_fig_p004_2.png] view at source ↗
Figure 4
Figure 4. Figure 4: Sketches contrasting the placement of personal [PITH_FULL_IMAGE:figures/full_fig_p005_4.png] view at source ↗
Figure 3
Figure 3. Figure 3: We annotate a subset of sketches from Figure 2 to [PITH_FULL_IMAGE:figures/full_fig_p005_3.png] view at source ↗
Figure 5
Figure 5. Figure 5: Our prototype has three viewing modes corresponding to different treatments of chronology, where visual connec [PITH_FULL_IMAGE:figures/full_fig_p007_5.png] view at source ↗
read the original abstract

Timelines are effective ways to tell historical and personal stories. However, most timeline visualization tools impose an inflexible model of time prioritizing chronological clarity. On the other hand, unconstrained representations can better capture the irregular and contextual nature of lived time, but often at the cost of interpretability. In this work, we explore this continuum with a study of how historical and personal timelines could manifest in physical spaces. We conducted a formative study (N=12) in which participants freely arranged events within a physical environment. We observed a diversity of strategies reflecting the personal and context-dependent nature of temporal mental models. We also invited participants to consider how others could move through their timelines. Our analysis led to a choreographic approach to timeline creation, as well as a proof-of-concept tablet-based augmented reality (AR) application that supports spatial timeline drawing and viewing. Finally, we reflect on the design implications of encoding chronology, pacing, and spatial context in immersive timeline stories.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit. Tearing a paper down is the easy half of reading it; the pith above is the substance, this is the friction.

Referee Report

2 major / 2 minor

Summary. The manuscript reports a formative study (N=12) in which participants freely arranged historical and personal events within a physical environment. Observations of diverse, personal, and context-dependent temporal strategies, along with considerations for how others might move through the arrangement, lead the authors to propose a 'choreographic approach' to timeline creation. They present a proof-of-concept tablet-based AR application supporting spatial timeline drawing and viewing, and reflect on design implications for encoding chronology, pacing, and situatedness in immersive timeline stories.

Significance. If the physical-space strategies translate to AR without substantial confounding, the work could usefully expand timeline visualization research beyond rigid linear models toward situated, multi-perspective representations that account for viewer movement and personal mental models. The PoC demonstrates technical feasibility of spatial drawing in AR. The absence of any evaluation of the PoC, however, leaves the practical utility of the derived principles speculative.

major comments (2)
  1. [Formative study and analysis] Formative study and analysis sections: The central claim that observed physical arrangement strategies (personal temporal models, consideration of others' movement) directly inform effective AR timeline encodings is load-bearing, yet the manuscript provides no empirical test of these strategies inside an AR interface; potential AR-specific factors such as occlusion, registration error, or tablet ergonomics are not addressed as possible confounds.
  2. [Proof-of-concept application] Proof-of-concept application section: The tablet AR PoC is presented as realizing the choreographic approach, but no user evaluation, usability test, or comparison against baseline timeline tools is reported; without such validation the design implications for chronology/pacing/situatedness cannot be substantiated.
minor comments (2)
  1. [Abstract] The abstract states the study led to both the choreographic approach and the PoC but does not indicate whether any participants interacted with the AR prototype.
  2. [Related work] Related-work discussion could more explicitly contrast the proposed choreographic framing with prior spatial or embodied timeline systems in AR/VR.

Simulated Author's Rebuttal

2 responses · 0 unresolved

We thank the referee for the detailed and constructive report. The comments correctly identify that our work is positioned as formative rather than evaluative. Below we respond to each major comment, clarifying the intended scope of the study and prototype while acknowledging where additional discussion of limitations is warranted.

read point-by-point responses

  1. Referee: [Formative study and analysis] Formative study and analysis sections: The central claim that observed physical arrangement strategies (personal temporal models, consideration of others' movement) directly inform effective AR timeline encodings is load-bearing, yet the manuscript provides no empirical test of these strategies inside an AR interface; potential AR-specific factors such as occlusion, registration error, or tablet ergonomics are not addressed as possible confounds.

    Authors: We agree that the manuscript contains no empirical test of the observed strategies inside an AR interface and does not address AR-specific confounds such as occlusion or registration error. The study was explicitly formative: its purpose was to surface diverse physical arrangement practices and participant considerations for others' movement in order to derive a choreographic design approach. The manuscript does not claim that these strategies have been shown to be effective once transferred to AR; it presents them as generative insights that motivated the subsequent prototype. We will revise the discussion and limitations sections to state this scope explicitly and to list AR-specific factors as open questions for future controlled studies. revision: partial

  2. Referee: [Proof-of-concept application] Proof-of-concept application section: The tablet AR PoC is presented as realizing the choreographic approach, but no user evaluation, usability test, or comparison against baseline timeline tools is reported; without such validation the design implications for chronology/pacing/situatedness cannot be substantiated.

    Authors: The PoC is offered only as a technical demonstration that spatial timeline drawing and viewing are feasible on a tablet AR platform; it is not presented as a validated implementation. Because the paper's contribution centers on the formative observations and the resulting design approach, we did not include a user study of the prototype. We will add a short paragraph in the discussion that labels the design implications as reflective and provisional, and that identifies a comparative evaluation against conventional timeline tools as necessary future work. revision: partial

Circularity Check

0 steps flagged

No circularity: qualitative empirical study with no derivations or fitted parameters

full rationale

The paper reports a formative study (N=12) in which participants arranged events in physical space, from which the authors derive design observations and implement a proof-of-concept AR application. No equations, parameters, uniqueness theorems, or self-citations are invoked as load-bearing steps in any derivation chain. All central claims rest directly on the observed participant strategies rather than reducing to inputs by construction, renaming, or self-referential modeling. This is the expected outcome for a non-mathematical HCI formative study.

Axiom & Free-Parameter Ledger

0 free parameters · 1 axioms · 0 invented entities

The central claims rest on the domain assumption that physical arrangements reveal participants' temporal mental models and that these can inform AR design. No free parameters or invented entities are introduced.

axioms (1)
  • domain assumption Participants' free arrangements in physical space reflect personal and context-dependent temporal mental models.
    Invoked to justify moving from observed strategies to a choreographic design approach.

pith-pipeline@v0.9.1-grok · 5706 in / 1109 out tokens · 29369 ms · 2026-06-27T20:43:12.626539+00:00 · methodology

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

44 extracted references · 26 canonical work pages

  1. [1]

    Fereshteh Amini, Nathalie Henry Riche, Bongshin Lee, Christophe Hurter, and Pourang Irani. 2015. Understanding data videos: Looking at narrative visualiza- tion through the cinematography lens. InACM Conf. Human Factors in Computing Systems (CHI). doi:10.1145/2702123.2702431

    work page doi:10.1145/2702123.2702431 2015

  2. [2]

    Benjamin Bach, Conglei Shi, Nicolas Heulot, Tara Madhyastha, Tom Grabowski, and Pierre Dragicevic. 2015. Time curves: Folding time to visualize patterns of temporal evolution in data.IEEE Trans. Visualization & Computer Graphics (TVCG)22, 1 (2015). doi:10.1109/TVCG.2015.2467851

    work page doi:10.1109/tvcg.2015.2467851 2015

  3. [3]

    Matthew Brehmer. 2025. On data visualization and interior design. InProc. IEEE VIS alt.VIS Workshop. altvis.github.io/papers/2025/abstracts/interior-design.pdf

    2025

  4. [4]

    Matthew Brehmer, Bongshin Lee, Benjamin Bach, Nathalie Henry Riche, and Tamara Munzner. 2016. Timelines revisited: A design space and considerations for expressive storytelling.IEEE Trans. Visualization & Computer Graphics (TVCG) 23, 9 (2016). doi:10.1109/TVCG.2016.2614803

    work page doi:10.1109/tvcg.2016.2614803 2016

  5. [5]

    Lydia Byrne, Daniel Angus, and Janet Wiles. 2019. Figurative frames: A critical vocabulary for images in information visualization.Information Visualization18, 1 (2019). doi:10.1177/1473871617724212

    work page doi:10.1177/1473871617724212 2019

  6. [6]

    2026.Data through movement: Designing embodied human-data interaction for informal learning

    Francesco Cafaro and Jessica Roberts. 2026.Data through movement: Designing embodied human-data interaction for informal learning. Springer Nature

    2026

  7. [7]

    Nabila Chowdhury, Anna Offenwanger, Theophanis Tsandilas, Fanny Chevalier, and Syed Ishtiaque Ahmed. 2025. MomentsVR: Accessible immersive spaces for authoring personal narration. InExtended Abstracts ACM Conf. Computer- Supported Cooperative Work and Social Computing (CSCW): Companion Publica- tion. doi:10.1145/3715070.3749270

    work page doi:10.1145/3715070.3749270 2025

  8. [8]

    Michael DiBenigno, Mehmet Kosa, and Mina C Johnson-Glenberg. 2021. Flow Immersive: A multiuser, multidimensional, multiplatform interactive COVID-19 data visualization tool.Frontiers in Psychology12 (2021). doi:10.3389/fpsyg.2021. 661613

    work page doi:10.3389/fpsyg.2021 2021

  9. [9]

    Gwendal Fouché, Ferran Argelaguet Sanz, Emmanuel Faure, and Charles Kervrann. 2022. Timeline design space for immersive exploration of time-varying spatial 3d data. InACM Symp. Virtual Reality Software & Technology (VRST). doi:10.1145/3562939.3565612

    work page doi:10.1145/3562939.3565612 2022

  10. [10]

    Google. 2014. Cardboard. arvr.google.com/cardboard/

    2014

  11. [11]

    Google. 2018. ARCore. developers.google.com/ar

    2018

  12. [12]

    Joseph Graham, William Newman, and John Stacy. 2008. The geologic time spiral: A path to the past. pubs.usgs.gov/gip/2008/58

    2008

  13. [13]

    2012.Time Warped: Unlocking the Mysteries of Time Perception

    Claudia Hammond. 2012.Time Warped: Unlocking the Mysteries of Time Perception. House of Anansi

    2012

  14. [14]

    Chang Han and Andrew McNutt. 2025. The Feng Shui of visualization: Design the path to success and good fortune. doi:10.48550/arXiv.2510.00344 Proc. IEEE VIS alt.VIS Workshop

    work page doi:10.48550/arxiv.2510.00344 2025

  15. [15]

    Franconeri

    Steve Haroz, Robert Kosara, and Steven L. Franconeri. 2016. The connected scatterplot for presenting paired time series.IEEE Trans. Visualization & Computer Graphics (TVCG)22, 9 (2016). doi:10.1109/TVCG.2015.2502587

    work page doi:10.1109/tvcg.2015.2502587 2016

  16. [16]

    Christophe Hurter, Bernice Rogowitz, Guillaume Truong, Tiffany Andry, Hugo Romat, Ludovic Gardy, Fereshteh Amini, and Nathalie Henry Riche. 2024. Memory recall for data visualizations in mixed reality, virtual reality, 3D and 2D.IEEE Trans. Visualization & Computer Graphics (TVCG)30, 10 (2024). doi:10.1109/TVCG.2023.3336588

    work page doi:10.1109/tvcg.2023.3336588 2024

  17. [17]

    Radhika Pankaj Jain, Adam Drogemuller, Kadek Ananta Satriadi, Ross Smith, and Andrew Cunningham. 2025. Strollytelling: Coupling animation with physical locomotion to explore immersive data stories. InACM Conf. Human Factors in Computing Systems (CHI). doi:10.1145/3706598.3713132 The Choreography of Augmented Reality Timelines GI 26, June 09–12, 2026, Kitc...

    work page doi:10.1145/3706598.3713132 2025

  18. [18]

    Roger Kenny and Ana Asnes Becker. 2015. Is the Nasdaq in Another Bubble? The Wall Street Journal(2015). graphics.wsj.com/3d-nasdaq/

    2015

  19. [19]

    Omar Khan, Zaid Ahmed, Hyeongil Nam, and Kangsoo Kim. 2025. Tangible- Moments: Embedding XR memories onto physical objects. InIEEE Conf. Virtual Reality and 3D User Interfaces (VRW). doi:10.1109/VRW66409.2025.00227

    work page doi:10.1109/vrw66409.2025.00227 2025

  20. [20]

    Ian Li, Anind Dey, and Jodi Forlizzi. 2010. A stage-based model of personal informatics systems. InACM Conf. Human Factors in Computing Systems (CHI). doi:10.1145/1753326.1753409

    work page doi:10.1145/1753326.1753409 2010

  21. [21]

    Jiazhou Liu, Kadek Ananta Satriadi, Barrett Ens, and Tim Dwyer. 2024. Inves- tigating the effects of physical landmarks on spatial memory for information visualisation in augmented reality. InIntl. Symp. Mixed and Augmented Reality (ISMAR). doi:10.1109/ISMAR62088.2024.00043

    work page doi:10.1109/ismar62088.2024.00043 2024

  22. [22]

    2024.Storytelling for Spatial Computing and Mixed Reality: The Art of Augmenting Imagination

    Rob Morgan. 2024.Storytelling for Spatial Computing and Mixed Reality: The Art of Augmenting Imagination. CRC Press

    2024

  23. [23]

    2023.Saving Time: Discovering a Life Beyond the Clock

    Jenny Odell. 2023.Saving Time: Discovering a Life Beyond the Clock. Random House

    2023

  24. [24]

    Anna Offenwanger, Matthew Brehmer, Fanny Chevalier, and Theophanis Tsandi- las. 2023. Timesplines: Sketch-based authoring of flexible and idiosyncratic timelines.IEEE Trans. Visualization & Computer Graphics (TVCG)30, 1 (2023). doi:10.1109/TVCG.2023.3326520

    work page doi:10.1109/tvcg.2023.3326520 2023

  25. [25]

    Heike Otten, Lennart Hildebrand, Till Nagel, Marian Dörk, and Boris Müller

  26. [26]

    In IEEE VIS Arts Program (VISAP)

    Shifted Maps: Revealing spatio-temporal topologies in movement data. In IEEE VIS Arts Program (VISAP). doi:10.1109/VISAP45312.2018.9046054

    work page doi:10.1109/visap45312.2018.9046054 2018

  27. [27]

    Gary Perelman, Emmanuel Dubois, and Marcos Serrano. 2024. Dual-thumb ointing and command selection techniques for tablets: Enhancing distant inter- action with large displays using a two-handed tablet.International Journal of Human-Computer Studies184 (2024). doi:10.1016/j.ijhcs.2023.103203

    work page doi:10.1016/j.ijhcs.2023.103203 2024

  28. [28]

    Charles Perin. 2017. The symmetry of my life: An autobiographical visual- ization. InIEEE Conf. Visualization & Visual Analytics (VIS) Poster Proceedings. inria.hal.science/hal-01587944

    2017

  29. [29]

    Laura J Perovich and Nicole Zizzi. 2024. Feeling data through movement: Design- ing somatic data experiences with dancers. InACM Intl. Conf. Tangible, Embedded, and Embodied Interaction (TEI). doi:10.1145/3623509.3633371

    work page doi:10.1145/3623509.3633371 2024

  30. [30]

    Ken Pfeuffer, Ken Hinckley, Michel Pahud, and Bill Buxton. 2017. Thumb+ pen interaction on tablets. InACM Conf. Human Factors in Computing Systems (CHI). doi:10.1145/3025453.3025567

    work page doi:10.1145/3025453.3025567 2017

  31. [31]

    Catherine Plaisant, Brett Milash, Anne Rose, Seth Widoff, and Ben Shneiderman

  32. [32]

    InACM Conf

    LifeLines: visualizing personal histories. InACM Conf. Human Factors in Computing Systems (CHI). doi:10.1145/238386.238493

    work page doi:10.1145/238386.238493

  33. [33]

    Dario Rodighiero, Lins Derry, Douglas Duhaime, Jordan Kruguer, Maximilian C Mueller, Christopher Pietsch, Jeffrey T Schnapp, and Jeff Steward. 2022. Surprise machines: revealing Harvard Art museums’ image collection.Information Design Journal27, 1 (2022). doi:10.1075/idj.22013.rod

    work page doi:10.1075/idj.22013.rod 2022

  34. [34]

    2010.Cartographies of Time: A History of the Timeline

    Daniel Rosenberg and Anthony Grafton. 2010.Cartographies of Time: A History of the Timeline. Princeton Architectural Press

    2010

  35. [35]

    Ross Institute and Moebio Labs. 2015. Ross spiral curriculum. spi- ral.rosslearningsystem.org/spiral

    2015

  36. [36]

    Book Sadprasid, Carl Gutwin, and Scott Bateman. 2024. Improving video nav- igation for spatial task tutorials by spatially segmenting and situating how-to videos. InACM Symp. Spatial User Interaction (SUI). doi:10.1145/3677386.3682103

    work page doi:10.1145/3677386.3682103 2024

  37. [37]

    Wai Tong, Kento Shigyo, Lin-Ping Yuan, Mingming Fan, Ting-Chuen Pong, Huamin Qu, and Meng Xia. 2024. VisTellAR: Embedding data visualization to short-form videos using mobile augmented reality.IEEE Trans. Visualization & Computer Graphics (TVCG)31, 3 (2024). doi:10.1109/TVCG.2024.3372104

    work page doi:10.1109/tvcg.2024.3372104 2024

  38. [38]

    Camilla Torna. 2011. Visualizing time. InInfinite Instances: Studies and Images of Time, Olga Ast (Ed.). Mark Batty Publisher, 42–51

    2011

  39. [39]

    Mark Twain. 1914. How to make history dates stick.Harper’s Monthly Magazine 130, 775 (1914). http://www.twainquotes.com/HistoryDates/HistoryDates.html

    1914

  40. [40]

    Unity Technologies 2025. Unity 6. unity.com/releases/unity-6

    2025

  41. [41]

    Veronica Vu, Yogesh Rai, and Haeyong Chung. 2025. Walking through time: A hybrid immersive system for spatial and expressive timeline authoring. InIEEE Conf. Visualization & Visual Analytics (VIS). doi:10.1109/VIS60296.2025.00076

    work page doi:10.1109/vis60296.2025.00076 2025

  42. [42]

    James Walker, Rita Borgo, and Mark W Jones. 2015. Timenotes: A study on effective chart visualization and interaction techniques for time-series data.IEEE Trans. Visualization & Computer Graphics (TVCG)22, 1 (2015). doi:10.1109/TVCG. 2015.2467751

    work page doi:10.1109/tvcg 2015

  43. [43]

    Webalon. 2010. Tiki-Toki. tiki-toki.com

    2010

  44. [44]

    Chen Zhu-Tian, Yijia Su, Yifang Wang, Qianwen Wang, Huamin Qu, and Yingcai Wu. 2019. MARVisT: Authoring glyph-based visualization in mobile augmented reality.IEEE Trans. Visualization & Computer Graphics (TVCG)26, 8 (2019). doi:10.1109/TVCG.2019.2892415

    work page doi:10.1109/tvcg.2019.2892415 2019