pith. sign in

arxiv: 2606.11986 · v1 · pith:JNCFC37Vnew · submitted 2026-06-10 · 💻 cs.HC

Channels and Substrates: Distributed Cognition as an Interaction Model for Ubiquitous Analytics

Niklas Elmqvist , Panagiotis D. Ritsos , Peter W. S. Butcher This is my paper

Pith reviewed 2026-06-27 08:21 UTC · model grok-4.3

classification 💻 cs.HC
keywords ubiquitous analyticsdistributed cognitioncross-device interactioninput channelsoutput channelsvisual channelssituated analyticsrepresentational state
0
0 comments X

The pith

Ubiquitous analytics interaction is modeled as propagation of representational state across substrates using distributed cognition rather than traffic through a single interface.

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper establishes that traditional HCI models assuming a single monolithic interface and stable sensorimotor loop fit poorly with cross-device and ubiquitous analytics, where sensemaking unfolds across multiple devices, artifacts, and people. Instead, it proposes modeling interaction as propagation of representational state across substrates including minds, speech, bodies, artifacts, and devices. Input and output channels are introduced as generalizations of visual channels, carrying representational state through these substrates with availability, suitability, and preferability depending on context. This framework is demonstrated through reanalysis of several ubiquitous, immersive, and situated analytics systems. A sympathetic reader would care because it offers a foundation for designing systems that match real-world distributed settings instead of forcing everything through one device.

Core claim

The paper claims that interaction in ubiquitous analytics can be modeled using distributed cognition as propagation of representational state across substrates—minds, speech, bodies, artifacts, and devices—rather than as traffic through a single interface. On this basis input and output channels are introduced as generalizations of the visual channels from data visualization, carrying representational state through substrates whose availability, suitability, and preferability depend on context. The channels and substrates framework is demonstrated by reanalyzing several ubiquitous, immersive, and situated analytics systems.

What carries the argument

input and output channels as generalizations of visual channels that carry representational state through substrates in a distributed cognition model

If this is right

  • Design of ubiquitous analytics systems can be guided by evaluating channel availability, suitability, and preferability across different substrates in a given context.
  • Existing systems can be reanalyzed to reveal how they already distribute representational state across multiple substrates rather than relying on one interface.
  • Input and output channels provide a way to generalize visual channel concepts to non-visual substrates such as speech, touch, or physical artifacts.
  • Cross-device setups become analyzable as coordinated propagation of state instead of separate interfaces competing for user attention.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • The model could extend to collaborative sensemaking scenarios beyond analytics, where multiple people and artifacts coordinate state propagation.
  • New evaluation methods might be developed that measure how effectively a system supports channel switching or state handoff across substrates.
  • Toolkits could be built that explicitly expose channel properties to help designers select and combine substrates during development.

Load-bearing premise

Distributed cognition theory supplies a sufficiently precise and operationalizable foundation for defining input and output channels whose availability, suitability, and preferability can guide concrete system design in cross-device analytics.

What would settle it

Apply the channels and substrates framework to redesign an existing cross-device analytics system and observe whether the resulting design decisions differ meaningfully from those produced by traditional single-interface models, or identify an observed interaction sequence that cannot be expressed as propagation of representational state across substrates.

Figures

Figures reproduced from arXiv: 2606.11986 by Niklas Elmqvist, Panagiotis D. Ritsos, Peter W. S. Butcher.

Figure 1
Figure 1. Figure 1: Interaction as distributed cognition across input channels and substrates. [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. Figure 2: Input channels as chains of substrate transformations. [PITH_FULL_IMAGE:figures/full_fig_p006_2.png] view at source ↗
Figure 3
Figure 3. Figure 3: Channel availability and suitability shift across contexts. [PITH_FULL_IMAGE:figures/full_fig_p007_3.png] view at source ↗
Figure 4
Figure 4. Figure 4: Input and output channel design space. Each of the four analytical dimensions used to characterize input and output channels. Substrate distinguishes six propagation media (this is a non-exhaustive list): air, light, contact/touch, proprioception, acoustic, and digital. Bandwidth represents the rate at which representational state can pass through the channel. Precision represents the granularity at which … view at source ↗
Figure 5
Figure 5. Figure 5: Put That There [12]. The MIT Media Room and its SDMS combined gestural and voice interaction system. The left image (a) shows the conceptual sketch, the right (b) shows the system in actual use. pointed-at location, and the visual feedback is immediate and co￾located with the action. Input channels. The system uses two input channels in concert: W Pointing and ?Voice. 2 The pointing channel operates via a … view at source ↗
Figure 6
Figure 6. Figure 6: ImAxes [25]. One of the first immersive analytics system, ImAxes supports multidimensional visual exploration using an intuitive 3D manipulation interaction accessed using 3D motion controllers in immersive Virtual Reality. The controllers do provide a brief haptic buzz when an axis is pulled past the duplication threshold on the attribute shelf, but this is feedback on the input action, not a channel for … view at source ↗
Figure 7
Figure 7. Figure 7: Wizualization [4]. The system uses a combination of § Gesture (using ASL), 3D W Pointing, and ?Voice commands to enable remote users to co-create visualizations and co-analyze them in AR space. Unlike ImAxes, the channel profile is a bit broader as there are two input modalities that are being used. Moreover, due to the AR nature of the output, and the collaborative support, the co-presence of users and th… view at source ↗
Figure 8
Figure 8. Figure 8: MARVIS [51]. This system combines mobile devices with AR HMDs to create a collaborative visualization space. Visualizations are displayed via the AR HMD adjacent or superimposed to the mobile devices to augment the display capabilities of the latter. devices combine with W Pointing on a shared display, distribut￾ing input across device boundaries while keeping output purely øVisual. 6 Discussion This paper… view at source ↗
read the original abstract

Traditional HCI interaction models assume a single monolithic interface and a stable sensorimotor loop. These models fit poorly with cross-device (XVA) and ubiquitous analytics (UA), where interactive data sensemaking unfolds across multiple devices, artifacts, and people in disparate settings from the office to the factory floor. In this paper, we show how interaction in ubiquitous analytics can be modeled using distributed cognition as propagation of representational state across substrates -- minds, speech, bodies, artifacts, and devices -- rather than as traffic through a single interface. On this basis we introduce input and output channels as generalizations of the visual channels from data visualization: just as visual channels carry data through properties of the visual substrate, input and output channels carry representational state through substrates whose availability, suitability, and preferability depend on context. We demonstrate the channels and substrates framework by reanalyzing several ubiquitous, immersive, and situated analytics systems.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit. Tearing a paper down is the easy half of reading it; the pith above is the substance, this is the friction.

Referee Report

2 major / 2 minor

Summary. The paper argues that traditional HCI models assuming a single monolithic interface and stable sensorimotor loop are inadequate for cross-device (XVA) and ubiquitous analytics (UA), where sensemaking spans multiple devices, artifacts, and people. It proposes modeling UA interaction via distributed cognition as the propagation of representational state across substrates (minds, speech, bodies, artifacts, devices) rather than traffic through one interface. Input and output channels are introduced as generalizations of visual channels from data visualization, with properties of availability, suitability, and preferability that depend on context. The framework is demonstrated through reanalysis of several existing ubiquitous, immersive, and situated analytics systems.

Significance. If the operationalizability claim holds, the work could supply a principled way to reason about multi-substrate interaction in UA that extends beyond standard interface-centric models, potentially informing design choices in settings like factories or collaborative environments. The reanalysis approach reuses established distributed-cognition theory without introducing new free parameters or ad-hoc axioms, which is a strength for conceptual clarity, but the absence of any derived design recommendation or falsifiable prediction that differs from conventional HCI analysis limits immediate applicability.

major comments (2)
  1. [Abstract, paragraph 3; reanalysis sections] Abstract, paragraph 3 and the reanalysis sections: the central claim asserts that channel availability/suitability/preferability can operationally guide concrete UA design decisions and generalize visual channels. However, the demonstration consists solely of retrofitting existing system features into the new terminology; no section derives even one concrete design recommendation or falsifiable prediction from the channel/substrate properties that would have altered a prior design choice or differs from standard HCI analysis. This leaves the asserted operationalizability unsupported.
  2. [Demonstration / reanalysis] The weakest assumption identified in the reader's note is load-bearing: without at least one worked example showing how the framework would change a design decision (e.g., preferring one substrate over another in a specific context), the generalization from visual channels remains terminological rather than prescriptive.
minor comments (2)
  1. [Introduction / framework definition] Clarify whether the substrates list (minds, speech, bodies, artifacts, devices) is intended to be exhaustive or extensible, and provide a brief example of how a new substrate would be incorporated.
  2. [Framework section] The paper would benefit from a short table contrasting the new channel/substrate properties with conventional visual-channel properties to make the generalization explicit.

Simulated Author's Rebuttal

2 responses · 0 unresolved

We thank the referee for the constructive review. The comments correctly identify that our demonstration relies on reanalysis rather than forward-looking design guidance. We respond to each major comment below.

read point-by-point responses

  1. Referee: [Abstract, paragraph 3; reanalysis sections] Abstract, paragraph 3 and the reanalysis sections: the central claim asserts that channel availability/suitability/preferability can operationally guide concrete UA design decisions and generalize visual channels. However, the demonstration consists solely of retrofitting existing system features into the new terminology; no section derives even one concrete design recommendation or falsifiable prediction from the channel/substrate properties that would have altered a prior design choice or differs from standard HCI analysis. This leaves the asserted operationalizability unsupported.

    Authors: We agree the manuscript currently demonstrates the framework through reanalysis of existing systems to illustrate its descriptive utility across diverse UA settings. This approach reuses distributed cognition without new axioms, but does not include a forward design example. To strengthen the operationalizability claim, we will revise by adding a brief hypothetical design scenario in the demonstration section showing how channel properties (e.g., preferability in a hands-busy context) could alter a substrate choice. revision: yes

  2. Referee: [Demonstration / reanalysis] The weakest assumption identified in the reader's note is load-bearing: without at least one worked example showing how the framework would change a design decision (e.g., preferring one substrate over another in a specific context), the generalization from visual channels remains terminological rather than prescriptive.

    Authors: We acknowledge the point: the current reanalysis validates applicability but stops short of prescriptive guidance. The added scenario in revision will address this by walking through a concrete context (e.g., factory-floor analytics) where suitability and availability properties lead to preferring one channel/substrate combination over another, distinguishing it from standard single-interface analysis. revision: yes

Circularity Check

0 steps flagged

No significant circularity; applies external distributed cognition theory to UA via conceptual mapping and reanalysis

full rationale

The paper's derivation applies the pre-existing theory of distributed cognition (external to this work) to model interaction in ubiquitous analytics as propagation of representational state across substrates. Input/output channels are introduced as a generalization of visual channels from data visualization. This is demonstrated solely through reanalysis of existing systems, with no equations, fitted parameters, self-citations as load-bearing premises, or reductions where outputs equal inputs by construction. The framework remains self-contained against external benchmarks.

Axiom & Free-Parameter Ledger

0 free parameters · 1 axioms · 2 invented entities

The model introduces two new conceptual entities (channels and substrates) and rests on one domain assumption without free parameters or empirical fitting.

axioms (1)
  • domain assumption Distributed cognition theory provides an appropriate and operational basis for modeling interaction across multiple heterogeneous substrates in analytics settings.
    Invoked in the abstract as the foundation for the propagation-of-state model.
invented entities (2)
  • input and output channels no independent evidence
    purpose: Generalization of visual channels to carry representational state through any substrate.
    New construct introduced to unify interaction across devices and modalities.
  • substrates no independent evidence
    purpose: Minds, speech, bodies, artifacts, and devices as carriers of representational state.
    Core organizing concept of the framework.

pith-pipeline@v0.9.1-grok · 5694 in / 1318 out tokens · 22489 ms · 2026-06-27T08:21:52.807754+00:00 · methodology

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

92 extracted references · 61 canonical work pages

  1. [1]

    Sriram Karthik Badam, Fereshteh Amini, Niklas Elmqvist, and Pourang Irani

  2. [2]

    InProceedings of the IEEE Conference on Visual Analytics Sci- ence and Technology

    Supporting Visual Exploration for Multiple Users in Large Display Environments. InProceedings of the IEEE Conference on Visual Analytics Sci- ence and Technology. IEEE Computer Society, Los Alamitos, CA, USA, 1–10. doi:10.1109/vast.2016.7883506

    work page doi:10.1109/vast.2016.7883506 2016

  3. [3]

    Sriram Karthik Badam, Eli Fisher, and Niklas Elmqvist. 2015. Munin: A Peer- to-Peer Middleware for Ubiquitous Analytics and Visualization Spaces.IEEE Transactions on Visualization and Computer Graphics21, 2 (2015), 215–228. doi:10. 1109/TVCG.2014.2337337

    arXiv 2015

  4. [4]

    Robert Ball, Chris North, and Doug A. Bowman. 2007. Move to Improve: Pro- moting Physical Navigation to Increase User Performance with Large Displays. InProceedings of the ACM Conference on Human Factors in Computing Systems. ACM, New York, NY, USA, 191–200. doi:10.1145/1240624.1240656

    work page doi:10.1145/1240624.1240656 2007

  5. [5]

    Andrea Batch, Peter W. S. Butcher, Panagiotis D. Ritsos, and Niklas Elmqvist

  6. [6]

    Hard Magic

    Wizualization: A "Hard Magic" Visualization System for Immersive and Ubiquitous Analytics.IEEE Transactions on Visualization and Computer Graphics 30, 1 (2024), 507–517. doi:10.1109/TVCG.2023.3326580

    work page doi:10.1109/tvcg.2023.3326580 2024

  7. [7]

    Thomas, and Kim Marriott

    Andrea Batch, Andrew Cunningham, Maxime Cordeil, Niklas Elmqvist, Tim Dwyer, Bruce H. Thomas, and Kim Marriott. 2020. There Is No Spoon: Evaluating Performance, Space Use, and Presence with Expert Domain Users in Immersive Analytics.IEEE Transactions on Visualization and Computer Graphics26, 1 (2020), 536–546. doi:10.1109/TVCG.2019.2934803

    work page doi:10.1109/tvcg.2019.2934803 2020

  8. [8]

    Andrea Batch, Biswaksen Patnaik, Moses Akazue, and Niklas Elmqvist. 2020. Scents and Sensibility: Evaluating Information Olfactation. InProceedings of the ACM Conference on Human Factors in Computing Systems. ACM, New York, NY, USA, 1–14. doi:10.1145/3313831.3376733

    work page doi:10.1145/3313831.3376733 2020

  9. [9]

    Michel Beaudouin-Lafon. 2000. Instrumental Interaction: an Interaction Model for Designing Post-WIMP User Interfaces. InProceedings of the ACM Conference on Human Factors in Computing Systems. ACM, New York, NY, USA, 446–453. doi:10.1145/332040.332473

    work page doi:10.1145/332040.332473 2000

  10. [10]

    Michel Beaudouin-Lafon. 2004. Designing Interaction, Not Interfaces. InPro- ceedings of the ACM Conference on Advanced Visual Interfaces. ACM, New York, NY, USA, 15–22. doi:10.1145/989863.989865

    work page doi:10.1145/989863.989865 2004

  11. [11]

    Joanna Bergström and Kasper Hornbæk. 2025. DIRA: A model of the user interface.International Journal of Human-Computer Studies193 (2025), 103381. doi:10.1016/J.IJHCS.2024.103381

    work page doi:10.1016/j.ijhcs.2024.103381 2025

  12. [12]

    1967.Sémiologie graphique : Les diagrammes - Les réseaux - Les cartes(les réimpressions ed.)

    Jacques Bertin. 1967.Sémiologie graphique : Les diagrammes - Les réseaux - Les cartes(les réimpressions ed.). Editions de l’Ecole des Hautes Etudes en Sciences, Paris, France

    1967

  13. [13]

    Susanne Bødker. 2006. When second wave HCI meets third wave challenges. In Proceedings of the Nordic Conference on Human-Computer Interaction. ACM, New York, NY, USA, 1–8. doi:10.1145/1182475.1182476

    work page doi:10.1145/1182475.1182476 2006

  14. [14]

    Put-That-There

    Richard A. Bolt. 1980. “Put-That-There”: Voice and gesture at the graphics interface.Computer Graphics14, 3 (1980), 262–270. doi:10.1145/965105.807503

    work page doi:10.1145/965105.807503 1980

  15. [15]

    Marcel Borowski, Peter W. S. Butcher, Janus Bager Kristensen, Jonas Oxenbøll Petersen, Panagiotis D. Ritsos, Clemens N. Klokmose, and Niklas Elmqvist. 2025. DashSpace: A Live Collaborative Platform for Immersive and Ubiquitous Ana- lytics.IEEE Transactions on Visualization and Computer Graphics31, 10 (2025), 7034–7047. doi:10.1109/TVCG.2025.3537679

    work page doi:10.1109/tvcg.2025.3537679 2025

  16. [16]

    Matthew Brehmer and Tamara Munzner. 2013. A multi-level typology of abstract visualization tasks.IEEE Transactions on Visualization and Computer Graphics 19, 12 (2013), 2376–2385. doi:10.1109/TVCG.2013.124

    work page doi:10.1109/tvcg.2013.124 2013

  17. [17]

    William Buxton. 1983. Lexical and pragmatic considerations of input structures. SIGGRAPH Computer Graphics17, 1 (Jan. 1983), 31–37. doi:10.1145/988584.988586

    work page doi:10.1145/988584.988586 1983

  18. [18]

    Card, Jock D

    Stuart K. Card, Jock D. Mackinlay, and George G. Robertson. 1990. The design space of input devices. InProceedings of the ACM Conference on Human Factors in Computing Systems. New York, NY, USA, ACM, 117–124. doi:10.1145/97243.97263

    work page doi:10.1145/97243.97263 1990

  19. [19]

    Card, Jock D

    Stuart K. Card, Jock D. Mackinlay, and George G. Robertson. 1991. A Morpho- logical Analysis of the Design Space of Input Devices.ACM Transactions on Information Systems9, 2 (1991), 99–122. doi:10.1145/123078.128726

    work page doi:10.1145/123078.128726 1991

  20. [20]

    Card, Jock D

    Stuart K. Card, Jock D. Mackinlay, and Ben Shneiderman (Eds.). 1999.Readings in Information Visualization: Using Vision to Think. Morgan Kaufmann Publishers, San Francisco, CA, USA

    1999

  21. [21]

    Tom Chandler, Maxime Cordeil, Tobias Czauderna, Tim Dwyer, Jaroslaw Glowacki, Cagatay Goncu, Matthias Klapperstueck, Karsten Klein, Falk Schreiber, and Elliot Wilson. 2015. Immersive Analytics. InProceedings of the Interna- tional Symposium on Big Data Visual Analytics. IEEE, Piscataway, NJ, USA, 1–8. doi:10.1109/BDVA.2015.7314296

    work page doi:10.1109/bdva.2015.7314296 2015

  22. [22]

    Jinho Choi, Sanghun Jung, Deok Gun Park, Jaegul Choo, and Niklas Elmqvist

  23. [23]

    Yenpure, S

    Visualizing for the Non-Visual: Enabling the Visually Impaired to Use Visualization.Computer Graphics Forum38, 3 (2019), 249–260. doi:10.1111/CGF. 13686

    work page doi:10.1111/cgf 2019

  24. [24]

    Pramod Chundury, Biswaksen Patnaik, Yasmin Reyazuddin, Christine Tang, Jonathan Lazar, and Niklas Elmqvist. 2022. Towards Understanding Sensory Substitution for Accessible Visualization: An Interview Study.IEEE Transactions on Visualization and Computer Graphics28, 1 (2022), 1084–1094. doi:10.1109/ Channels and Substrates: Distributed Cognition as an Inte...

    arXiv 2022

  25. [25]

    Andy Clark and David Chalmers. 1998. The Extended Mind.Analysis58, 1 (1998), 7–19. doi:10.1093/analys/58.1.7

    work page doi:10.1093/analys/58.1.7 1998

  26. [26]

    Cleveland and Robert McGill

    William S. Cleveland and Robert McGill. 1984. Graphical Perception: Theory, Experimentation and Application to the Development of Graphical Methods.J. Amer. Statist. Assoc.79, 387 (Sept. 1984), 531–554. doi:10.2307/2288400

    work page doi:10.2307/2288400 1984

  27. [27]

    Cook and Janice Miller Polgar

    Albert M. Cook and Janice Miller Polgar. 2014.Assistive Technologies: Principles and Practice(4th ed.). Elsevier Health Sciences, St. Louis, MO, USA

    2014

  28. [28]

    Thomas, and Kim Marriott

    Maxime Cordeil, Andrew Cunningham, Tim Dwyer, Bruce H. Thomas, and Kim Marriott. 2017. ImAxes: Immersive Axes as Embodied Affordances for Interactive Multivariate Data Visualisation. InProceedings of the ACM Symposium on User Interface Software and Technology. ACM, New York, NY, USA, 71–83. doi:10.1145/3126594.3126613

    work page doi:10.1145/3126594.3126613 2017

  29. [29]

    Evanthia Dimara and Charles Perin. 2020. What is Interaction for Data Visual- ization?IEEE Transactions on Visualization and Computer Graphics26, 1 (2020), 119–129. doi:10.1109/TVCG.2019.2934283

    work page doi:10.1109/tvcg.2019.2934283 2020

  30. [30]

    2001.Where the Action Is: The Foundations of Embodied Interaction

    Paul Dourish. 2001.Where the Action Is: The Foundations of Embodied Interaction. MIT Press, Cambridge, MA, USA

    2001

  31. [31]

    Niklas Elmqvist. 2011. Embodied Human-Data Interaction. InProceedings of the ACM CHI Workshop on Embodied Interaction: Theory and Practice in HCI. ACM, New York, NY, USA

    2011

  32. [32]

    Niklas Elmqvist. 2023. Data Analytics Anywhere and Everywhere.Commun. ACM66, 12 (2023), 52–63. doi:10.1145/3584858

    work page doi:10.1145/3584858 2023

  33. [33]

    Niklas Elmqvist. 2023. Visualization for the Blind.Interactions30, 1 (2023), 52–56. doi:10.1145/3571737

    work page doi:10.1145/3571737 2023

  34. [34]

    Niklas Elmqvist and Pourang Irani. 2013. Ubiquitous Analytics: Interacting with Big Data Anywhere, Anytime.IEEE Computer46, 4 (2013), 86–89. doi:10.1109/ mc.2013.147

    2013

  35. [35]

    Neven A. M. ElSayed, Bruce H. Thomas, Kim Marriott, Julia Piantadosi, and Ross T. Smith. 2016. Situated Analytics: Demonstrating immersive analytical tools with Augmented Reality.Journal of Visual Languages & Computing36 (2016), 13–23. doi:10.1016/j.jvlc.2016.07.006

    work page doi:10.1016/j.jvlc.2016.07.006 2016

  36. [36]

    2014.Learning by Expanding: An Activity-Theoretical Approach to Developmental Research(2nd ed.)

    Yrjö Engeström. 2014.Learning by Expanding: An Activity-Theoretical Approach to Developmental Research(2nd ed.). Cambridge University Press, Cambridge, UK

    2014

  37. [37]

    Saul Greenberg, Nicolai Marquardt, Till Ballendat, Rob Diaz-Marino, and Miaosen Wang. 2011. Proxemic interactions: the new ubicomp?Interactions18, 1 (2011), 42–50. doi:10.1145/1897239.1897250

    work page doi:10.1145/1897239.1897250 2011

  38. [38]

    Mark Hancock, Sheelagh Carpendale, and Andy Cockburn. 2007. Shallow- Depth 3D Interaction: Design and Evaluation of One-, Two- and Three-Touch Techniques. InProceedings of the ACM Conference on Human Factors in Computing Systems. ACM, New York, NY, USA, 1147–1156. doi:10.1145/1240624.1240798

    work page doi:10.1145/1240624.1240798 2007

  39. [39]

    Neuhoff (Eds.)

    Thomas Hermann, Andy Hunt, and John G. Neuhoff (Eds.). 2011.The Sonification Handbook. Logos Verlag, Berlin, Germany

    2011

  40. [40]

    Juan David Hincapié-Ramos, Xiang Guo, Paymahn Moghadasian, and Pourang Irani. 2014. Consumed Endurance: A Metric to Quantify Arm Fatigue of Mid- Air Interactions. InProceedings of the ACM Conference on Human Factors in Computing Systems. ACM, New York, NY, USA, 1063–1072. doi:10.1145/2556288. 2557130

    work page doi:10.1145/2556288 2014

  41. [41]

    Hollan, Edwin L

    James D. Hollan, Edwin L. Hutchins, and David Kirsh. 2000. Distributed cogni- tion: toward a new foundation for human-computer interaction research.ACM Transactions on Computer-Human Interaction7, 2 (2000), 174–196. doi:10.1145/ 353485.353487

    arXiv 2000

  42. [42]

    Kasper Hornbæk and Antti Oulasvirta. 2017. What Is Interaction?. InProceedings of the ACM Conference on Human Factors in Computing Systems. ACM, New York, NY, USA, 5040–5052. doi:10.1145/3025453.3025765

    work page doi:10.1145/3025453.3025765 2017

  43. [43]

    1995.Cognition in the Wild

    Edwin Hutchins. 1995.Cognition in the Wild. MIT Press, Cambridge, MA, USA

    1995

  44. [44]

    Edwin Hutchins. 1995. How a Cockpit Remembers Its Speeds.Cognitive Science 19, 3 (1995), 265–288. doi:10.1207/s15516709cog1903_1

    work page doi:10.1207/s15516709cog1903_1 1995

  45. [45]

    Hiroshi Ishii and Brygg Ullmer. 1997. Tangible Bits: Towards Seamless Interfaces between People, Bits and Atoms. InProceedings of the ACM Conference on Human Factors in Computing Systems. ACM, New York, NY, USA, 234–241. doi:10.1145/ 258549.258715

    arXiv 1997

  46. [46]

    Robert J. K. Jacob. 1991. The use of eye movements in human-computer in- teraction techniques: what you look at is what you get.ACM Transactions on Information Systems9, 2 (April 1991), 152–169. doi:10.1145/123078.128728

    work page doi:10.1145/123078.128728 1991

  47. [47]

    Jakobsen, Yonas Sahlemariam Haile, Soren Knudsen, and Kasper Horn- bæk

    Mikkel R. Jakobsen, Yonas Sahlemariam Haile, Soren Knudsen, and Kasper Horn- bæk. 2013. Information Visualization and Proxemics: Design Opportunities and Empirical Findings.IEEE Transactions on Visualization and Computer Graphics 19, 12 (2013), 2386–2395. doi:10.1109/tvcg.2013.166

    work page doi:10.1109/tvcg.2013.166 2013

  48. [48]

    Yvonne Jansen, Pierre Dragicevic, Petra Isenberg, Jason Alexander, Abhijit Karnik, Johan Kildal, Sriram Subramanian, and Kasper Hornbæk. 2015. Opportunities and Challenges for Data Physicalization. InProceedings of the ACM Conference on Human Factors in Computing Systems. ACM, New York, NY, USA, 3227–3236. doi:10.1145/2702123.2702180

    work page doi:10.1145/2702123.2702180 2015

  49. [49]

    KyungTae Kim and Niklas Elmqvist. 2012. Embodied lenses for collaborative visual queries on tabletop displays.Information Visualization11, 4 (2012), 319–338. doi:10.1177/1473871612441874

    work page doi:10.1177/1473871612441874 2012

  50. [50]

    David Kirsh. 1995. The Intelligent Use of Space.Artificial Intelligence73, 1-2 (1995), 31–68. doi:10.1016/0004-3702(94)00017-U

    work page doi:10.1016/0004-3702(94)00017-u 1995

  51. [51]

    Stefanie Klum, Petra Isenberg, Ricardo Langner, Jean-Daniel Fekete, and Raimund Dachselt. 2012. Stackables: combining tangibles for faceted browsing. InProceed- ings of the ACM Conference on Advanced Visual Interfaces. ACM, New York, NY, USA, 241–248. doi:10.1145/2254556.2254600

    work page doi:10.1145/2254556.2254600 2012

  52. [53]

    Heidi Lam. 2008. A Framework of Interaction Costs in Information Visualization. IEEE Transactions on Visualization and Computer Graphics14, 6 (2008), 1149–1156. doi:10.1109/TVCG.2008.109

    work page doi:10.1109/tvcg.2008.109 2008

  53. [54]

    Ricardo Langner, Marc Satkowski, Wolfgang Büschel, and Raimund Dachselt

  54. [55]

    Srinivasan, N

    MARVIS: Combining Mobile Devices and Augmented Reality for Visual Data Analysis. InProceedings of the ACM Conference on Human Factors in Com- puting Systems. ACM, New York, NY, USA, 468:1–468:17. doi:10.1145/3411764. 3445593

    work page doi:10.1145/3411764

  55. [56]

    Kim, Ali Parsaei, Jean-Daniel Fekete, Pierre Drag- icevic, and Sean Follmer

    Mathieu Le Goc, Lawrence H. Kim, Ali Parsaei, Jean-Daniel Fekete, Pierre Drag- icevic, and Sean Follmer. 2016. Zooids: Building Blocks for Swarm User Interfaces. InProceedings of the ACM Symposium on User Interface Software and Technology. ACM, New York, NY, USA, 97–109. doi:10.1145/2984511.2984547

    work page doi:10.1145/2984511.2984547 2016

  56. [57]

    SK Lee, William Buxton, and K. C. Smith. 1985. A Multi-Touch Three Dimensional Touch-Sensitive Tablet. InProceedings of the ACM Conference on Human Factors in Computing Systems. ACM, New York, NY, USA, 21–25. doi:10.1145/317456.317461

    work page doi:10.1145/317456.317461 1985

  57. [58]

    Nersessian, and John T

    Zhicheng Liu, Nancy J. Nersessian, and John T. Stasko. 2008. Distributed Cognition as a Theoretical Framework for Information Visualization.IEEE Transactions on Visualization and Computer Graphics14, 6 (2008), 1173–1180. doi:10.1109/TVCG.2008.121

    work page doi:10.1109/tvcg.2008.121 2008

  58. [59]

    Wendy E. Mackay. 1999. Is Paper Safer? The Role of Paper Flight Strips in Air Traffic Control.ACM Transactions on Computer-Human Interaction6, 4 (1999), 311–340. doi:10.1145/331490.331491

    work page doi:10.1145/331490.331491 1999

  59. [60]

    Mackay and Michel Beaudouin-Lafon

    Wendy E. Mackay and Michel Beaudouin-Lafon. 2025. Interaction Substrates: Combining Power and Simplicity in Interactive Systems. InProceedings of the ACM Conference on Human Factors in Computing Systems. ACM, New York, NY, USA, 6871:1–687:16. doi:10.1145/3706598.3714006

    work page doi:10.1145/3706598.3714006 2025

  60. [61]

    Jock Mackinlay. 1986. Automating the Design of Graphical Presentations of Relational Information.ACM Transactions on Graphics5, 2 (April 1986), 110–141. doi:10.1145/22949.22950

    work page doi:10.1145/22949.22950 1986

  61. [62]

    Karon E. MacLean. 2008. Haptic Interaction Design for Everyday Interfaces. Reviews of Human Factors and Ergonomics4, 1 (2008), 149–194. doi:10.1518/ 155723408X342826

    2008

  62. [63]

    Thomas (Eds.)

    Kim Marriott, Falk Schreiber, Tim Dwyer, Karsten Klein, Nathalie Henry Riche, Takayuki Itoh, Wolfgang Stuerzlinger, and Bruce H. Thomas (Eds.). 2018.Im- mersive Analytics. Lecture Notes in Computer Science, Vol. 11190. Springer International Publishing, Berlin, Germany. doi:10.1007/978-3-030-01388-2

    work page doi:10.1007/978-3-030-01388-2 2018

  63. [64]

    2014.Visualization Analysis and Design

    Tamara Munzner. 2014.Visualization Analysis and Design. A K Peters (CRC Press), Boca Raton, FL, USA

    2014

  64. [65]

    Sundar Murugappan, Vinayak, Niklas Elmqvist, and Karthik Ramani. 2012. Ex- tended multitouch: recovering touch posture and differentiating users using a depth camera. InProceedings of the ACM Symposium on User Interface Software and Technology. ACM, New York, NY, USA, 487–496. doi:10.1145/2380116.2380177

    work page doi:10.1145/2380116.2380177 2012

  65. [66]

    Nardi (Ed.)

    Bonnie A. Nardi (Ed.). 1995.Context and Consciousness: Activity Theory and Human-Computer Interaction. MIT Press, Cambridge, MA, USA

    1995

  66. [67]

    Donald A. Norman. 1986. Cognitive Engineering. InUser Centered System Design: New Perspectives on Human-Computer Interaction, Donald A. Norman and Stephen W. Draper (Eds.). Lawrence Erlbaum Associates, Hillsdale, NJ, USA, 31–61

    1986

  67. [68]

    Donald A. Norman. 1988.The Design of Everyday Things. Basic Books, New York, NY, USA

    1988

  68. [69]

    Panëels and Jonathan C

    Sabrina A. Panëels and Jonathan C. Roberts. 2010. Review of Designs for Haptic Data Visualization.IEEE Transactions on Haptics3, 2 (2010), 119–137. doi:10. 1109/TOH.2009.44

    2010

  69. [70]

    Biswaksen Patnaik, Andrea Batch, and Niklas Elmqvist. 2019. Information Olfactation: Harnessing Scent to Convey Data.IEEE Transactions on Visualization and Computer Graphics25, 1 (2019), 726–736. doi:10.1109/TVCG.2018.2865237

    work page doi:10.1109/tvcg.2018.2865237 2019

  70. [71]

    Ken Pfeuffer, Jason Alexander, Ming Ki Chong, and Hans Gellersen. 2014. Gaze- touch: combining gaze with multi-touch for interaction on the same surface. In Proceedings of the ACM Symposium on User Interface Software and Technology. ACM, New York, NY, USA, 509–518. doi:10.1145/2642918.2647397

    work page doi:10.1145/2642918.2647397 2014

  71. [72]

    Pike, John Stasko, Remco Chang, and Theresa A

    William A. Pike, John Stasko, Remco Chang, and Theresa A. O’Connell. 2009. The Science of Interaction.Information Visualization8, 4 (Dec. 2009), 263–274. doi:10.1057/ivs.2009.22

    work page doi:10.1057/ivs.2009.22 2009

  72. [73]

    Potter, Linda J

    Richard L. Potter, Linda J. Weldon, and Ben Shneiderman. 1988. Improving the Accuracy of Touch Screens: An Experimental Evaluation of Three Strategies. InProceedings of the ACM Conference on Human Factors in Computing Systems. ACM, New York, NY, USA, 27–32. doi:10.1145/57167.57171

    work page doi:10.1145/57167.57171 1988

  73. [74]

    Michael Scaife and Yvonne Rogers. 1996. External cognition: how do graphical representations work?International Journal of Human-Computer Studies45, 2 Niklas Elmqvist, Panagiotis D. Ritsos, and Peter W. S. Butcher (1996), 185–213. doi:10.1006/ijhc.1996.0048

    work page doi:10.1006/ijhc.1996.0048 1996

  74. [75]

    Orit Shaer and Eva Hornecker. 2010. Tangible User Interfaces: Past, Present, and Future Directions.Foundations and Trends in Human-Computer Interaction3, 1–2 (2010), 1–137. doi:10.1561/1100000026

    work page doi:10.1561/1100000026 2010

  75. [76]

    Lawrence A. Shapiro. 2011.Embodied Cognition. Routledge, New York, NY, USA

    2011

  76. [77]

    Ritsos, and Niklas Elmqvist

    Sungbok Shin, Andrea Batch, Peter William Scott Butcher, Panagiotis D. Ritsos, and Niklas Elmqvist. 2024. The Reality of the Situation: A Survey of Situated Analytics.IEEE Transactions on Visualization and Computer Graphics30, 8 (2024), 5147–5164. doi:10.1109/TVCG.2023.3285546

    work page doi:10.1109/tvcg.2023.3285546 2024

  77. [78]

    Ben Shneiderman. 1983. Direct Manipulation: A Step Beyond Programming Languages.Computer16, 8 (1983), 57–69. doi:10.1109/MC.1983.1654471

    work page doi:10.1109/mc.1983.1654471 1983

  78. [79]

    Cohen, Steven Jacobs, and Niklas Elmqvist

    Ben Shneiderman, Catherine Plaisant, Maxine S. Cohen, Steven Jacobs, and Niklas Elmqvist. 2016.Designing the User Interface - Strategies for Effective Human-Computer Interaction(6th edition ed.). Pearson, Hoboken, NJ, USA

    2016

  79. [80]

    David Simkin and Reid Hastie. 1987. An Information-Processing Analysis of Graph Perception.J. Amer. Statist. Assoc.82, 398 (1987), 454–465. doi:10.1080/ 01621459.1987.10478448

    arXiv 1987

  80. [81]

    Martin Spindler, Christian Tominski, Heidrun Schumann, and Raimund Dachselt

Showing first 80 references.