Pith. sign in

REVIEW 3 major objections 4 minor 94 references

An Interaction Design Toolkit for Physical Task Guidance with Artificial Intelligence and Mixed Reality

T0 review · 3 major / 4 minor · reviewed 2026-08-11 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read This paper proposes MixITS-Kit, an interaction design toolkit that distills eight low-fidelity AI+MR task-guidance prototypes into six design considerations, 36 design patterns, and an Interaction Canvas.

desk verdict A genuinely useful eight-gulf framework and 36-pattern catalog for MixITS design, backed by honest but thin evidence from one student course; worth refereeing, not desk rejecting. read the letter →

arxiv 2412.16892 v1 pith:37KL4KNI submitted 2024-12-22 cs.HC

Arthur Caetano , Alejandro Aponte , Misha Sra This is my paper · ORCID
classification cs.HC
keywords mixitsphysicaltaskguidancemixedrealityhuman-AIinteractiondesigntoolkitpatternscanvaslow-fidelityprototyping
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Physical skills often need live instruction and feedback, but a human expert is not always available. The paper argues that systems combining artificial intelligence with mixed reality can fill that role, and that designing such systems is hard in a specific, discoverable way: problems arise at the junctions between user, AI, and the physical environment. To capture those problems, it proposes MixITS-Kit, a toolkit made of six high-level design considerations, 36 lower-level design patterns with example solutions, and an Interaction Canvas for analyzing interaction breakdowns. The toolkit was derived from eight low-fidelity prototypes built by graduate teams over a ten-week course, and a small evaluation suggests that newcomers can use it to analyze a design problem and generate concrete solutions. The paper's central claim is that these structured tools fill a gap left by design guidance that treats AI or mixed reality separately.

What carries the argument

The load-bearing mechanism is the Interaction Canvas built on the Gulfs of Execution and Evaluation model, which measures the gap between a user's intentions and the actions a system supports, and the gap between the system's displayed state and the user's interpretation. The model is applied not to a single user and system but to three entities: the human user, the AI-MR system, and the real environment. This yields eight named gulfs—human execution and evaluation toward AI and environment, and AI execution and evaluation toward human and environment—and each design pattern is assigned to one of these gulfs. The canvas asks the designer to specify, for a given interaction, who the actor is, what the target is, whether the actor achieved the goal, and whether the actor correctly interpreted the target's feedback. That same labeling process was used to sort the 63 functionalities extracted from the eight prototypes into the final 36 patterns, which gives the catalog its structure and provides designers with a shared vocabulary for naming breakdowns.

What would settle it

An observational study of independently built MixITS systems during user testing would settle the catalog's completeness: if a substantial share of interaction breakdowns cannot be assigned to any of the 36 pattern labels, the claim that these are the recurring design problems of the domain is falsified.

Watch

Extended reading notes

Core claim

On the paper's own terms, the central discovery is that the design problems of Mixed Reality Intelligent Task Support (MixITS)—AI-driven instruction and feedback for tasks in the real world—can be elicited and organized into a reusable interaction design toolkit. The authors claim that MixITS-Kit offers a structured set of tools that fulfill gaps in existing design aids that consider only AI or MR alone and tackle the unique challenges of task support situated in the real environment at different levels of abstraction. The toolkit consists of six design considerations (Teaching and Directing, Interaction Timing, Error Handling, Sensors and Actuators, Evolving Context, and Building Trust), 36 design patterns organized by eight interaction gulfs, and an Interaction Canvas that guides designers through execution and evaluation questions for each pair of actors. In an evaluation, eight participants with mixed AI and MR experience used the toolkit to diagnose fictional user issues, match them to patterns, and revise their solutions; most reported that the toolkit was learnable, supported creative solutions, and offered a shared vocabulary, while two of eight initially chose a pattern from a different gulf than expected.

Load-bearing premise

The load-bearing premise is that the design challenges and solutions seen in eight low-fidelity student projects from one 10-week course taught by the same instructors also show up in real physical-task-guidance systems built by working professionals.

Editorial extensions

If this is right

  • Novice designers can move from a reported user problem to a concrete, sketched solution in about half an hour: median completion time was 27 minutes for the first task and 8.5 minutes for the revision task.
  • Because every pattern is labeled with a gulf, the toolkit gives design teams a shared vocabulary: in the peer-interpretation task, all eight participants correctly identified the actor and target of a solution, and half identified the exact pattern.
  • Designers are nudged to balance proactive and reactive AI interventions, preserve user agency when errors occur, and build trust through transparent explanations and error reporting, rather than treating guidance as one-way instruction.
  • The high-level considerations and low-level patterns are meant to be used together, so reflecting on a consideration such as Building Trust can revise and enrich an already proposed pattern solution.
  • The pattern catalog is explicitly a starting point, not a closed list, and the paper expects it to grow as more MixITS systems are built and studied.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The eight-gulf scheme is essentially an interaction graph with human, AI, and environment as nodes; that structure could be turned into a generative design probe or automated checklist for other situated AI systems, which the paper only gestures at in its future-work section.
  • The paper deliberately omits pattern frequencies, so the catalog currently cannot tell a novice which problems are most common; collecting frequency data from deployed systems would be a natural next dataset and would turn the taxonomy into an evidence-ranked resource.
  • Because the elicitation method relies on course artifacts, the most direct test of the toolkit's value is replication with professional practitioners; the paper asks for this explicitly, but it has not been done.
  • The paper speculates that taking the design considerations early reduces rework, but does not measure this; a longitudinal comparison of design iteration counts with and without the toolkit would put that claim on firmer ground.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 4 minor

Summary. The paper proposes MixITS-Kit, a design toolkit for Mixed Reality Intelligent Task Support (MixITS) systems, comprising six high-level design considerations, 36 design patterns grouped under eight interaction gulfs, and an Interaction Canvas. The toolkit is derived through reflexive thematic analysis and design-pattern elicitation on eight low-fidelity prototypes built by 25 graduate students in a 10-week author-taught course. The authors evaluate MixITS-Kit in an asynchronous take-home study with eight participants, who used the toolkit to analyze and solve design problems for a fictional rock-climbing guidance app, and report generally positive self-ratings and moderate pattern-recognition accuracy. The paper concludes that the toolkit can serve as a valuable resource for practitioners and researchers.

Significance. If the elicited content transfers beyond the classroom setting, MixITS-Kit addresses a real gap: the existing design guidance covers AI and MR separately or only partly at their intersection, while little structured support exists for the combined design space of physical task guidance. The manuscript's strengths include the concrete and inspectable artifact in Table 1 (the pattern catalog), the anchoring of the analysis in Norman's Gulfs of Execution and Evaluation, the use of an established evaluation framework for HCI toolkits (Ledo et al.), and a transparent acknowledgment of the main limitations, including curricular bias and the need for data from experienced professionals. The central risk is that the practitioner-directed claim rests on an unverified assumption about the representativeness of eight student-team prototypes, and the evaluation does not provide strong evidence about transferability or comparative value. The contribution is best characterized as an initial, structured resource that needs further validation before broad practitioner guidance claims are warranted.

major comments (3)
  1. [§3, §8.1, §9] The MixITS-Kit is elicited exclusively from eight low-fidelity prototypes developed by 25 graduate students in a single 10-week course taught by the authors (§3, §4.2), and §8.1 acknowledges that the teaching approach and selected materials may have introduced biases. The abstract's claim that the toolkit 'can serve as a valuable resource for practitioners' and §9's claim that it 'offers a structured set of tools that fulfill gaps in existing design aids' depend on these classroom artifacts being representative of real-world MixITS design problems. No evidence is presented for such transferability, e.g., a coverage analysis of the published MixITS systems cited in §2.2 or design data from professional practitioners. Since §8.1 explicitly defers the collection of professional data to future work, the central claim currently exceeds the evidence; the paper should either add an external-validity check or temper the abstract and conclusion to describe a catalog for early-career designers with its external validity still open.
  2. [§6.4–§6.6] In Task 2, each participant's 'correct' identification of a peer's design pattern is scored against the authors' own labeling (the 'originally intended ones,' §6.4). Because the solutions being classified were produced with the same catalog and the reference labels come from the authors' Table 1, the exact-match rate of half of the participants measures agreement with the authors' interpretation rather than an independent demonstration of shared vocabulary. In addition, the study has no baseline condition, so the favorable self-reports in Figure 6 (e.g., 'It would take me longer to solve the task without the toolkit,' M=4.5) cannot be compared against working without the toolkit or against using the underlying AI/MR guideline sets separately. The paper should either add a comparison condition or explicitly restrict the claim to perceived usefulness in a single session with novice designers.
  3. [§6.6, §7.1] The results section reports that in Task 1 two of eight participants applied patterns from non-anticipated gulfs with justifications that did not align with the gulf concepts, and in Task 2 only four of eight identified the exact design pattern and five of eight identified the correct gulf. Section 7.1 nonetheless concludes there was a 'high level of performance' and calls the results promising. With a 50% exact-match rate and no baseline, the evidence is more naturally read as showing that the toolkit is learnable but that pattern recognition is unreliable; the interpretation should be calibrated to these numbers.
minor comments (4)
  1. [§6.4–§6.6] The recognition counts in §6.6 are reported in a way that does not add up for the eight participants: 'Four participants successfully recognized the correct design pattern, while five identified the correct gulf. One participant misidentified the pattern, and three incorrectly identified the gulf.' Please clarify whether these categories are mutually exclusive and how the remaining participants (if any) are classified, or report full contingency counts.
  2. [§6.1, §6.3] There are typographical and phrasing issues in the evaluation section: 'We instructed participants to to work directly' (§6.3), 'experinece' (§6.1), and 'we consider this participants are representative' (§6.1). These should be corrected.
  3. [References] Reference [5] appears garbled ('BURTON R. R. and ITS. International Conference. 1982. Diagnosing Bugs in a Simple Procedural Skill, Sleeman.'); please fix the citation entry.
  4. [§6.6] The statement in §6.6 that 'Participants identified the most challenging aspects of learning the canvas and design patterns' does not indicate how these challenges were measured or aggregated; please clarify whether these are open-ended responses or quantitative items.

Circularity Check

2 steps flagged · score 4.0 of 10

Closed-loop evaluation: Task 2 recognition and Task 1 'mistakes' are scored against the same Table 1 labels and eight gulfs used to build the toolkit, so the validation measures internal consistency rather than external value.

  1. self definitional [Section 6.3 (Task 2) and Section 6.6 (Results)]
    "Given only the solution text description and sketch image, they had to identify which design pattern from Table 1 their peer applied... This exercise aimed to assess whether participants could recognize patterns in their peer's work, indicating the potential of MixITS-Kit as a shared language for MixITS designers and developers."

    The criterion for 'correct' recognition in Task 2 is the pattern label that the Task 1 participant selected from the same Table 1 that the Task 2 reviewer was given. Success therefore measures whether two people, presented with the same closed label set, choose the same label for the same text and sketch. That is a label-consistency check inside the toolkit's own vocabulary; it does not test whether the 36 patterns correspond to real MixITS design problems. The 'shared language' claim is verified by the very language it defines, so the result is partly definitional rather than independent evidence of value.

  2. fitted input called prediction [Section 6.3 (Task 1) and Section 6.6 (Results)]
    "We created errors contextualized in the experimental scenario and based on one of the eight MixITS interaction gulfs according to the condition... In Task 1, two of the eight participants applied design patterns from gulfs different than anticipated and provided justifications that did not align with the defined gulf concepts, indicating a mistake."

    The Task 1 scenarios are constructed from the same eight-gulf taxonomy that the toolkit encodes, and a participant 'mistake' is defined as choosing a gulf different from the authors' anticipated one. The authors' anticipation is the same classification used to label the 63 triplets and build the 36-pattern catalog. Thus the evaluation's success metric is agreement with the toolkit's own internal categories on inputs generated from those categories. This is a closed-loop consistency test, not an external validation that the toolkit improves MixITS design outcomes.

full rationale

The derivation of the toolkit artifacts is not circular: the six design considerations and 36 patterns were elicited from eight low-fidelity student prototypes through reflexive thematic analysis and pattern elicitation, and the Interaction Canvas is an application of Norman's Gulfs, an external framework. The course-derived corpus is an empirical input, and the paper explicitly documents the elicitation procedure (Sections 4.1-4.2). There is no load-bearing self-citation: references to the authors' own prior work ([7], [52], [75]) are background technical examples and are not used to justify the toolkit's content or uniqueness. The circularity burden sits in the evaluation section. Task 1 builds fictional errors from the eight gulfs and scores 'mistakes' against the authors' anticipated gulf; Task 2 asks participants to recognize which Table 1 pattern a peer applied, where the ground truth is the label chosen from the same Table 1. Both tasks therefore measure whether participants can reproduce the authors' labels on stimuli generated from those labels, which is an internal consistency check. The paper's own Section 8.1 concedes that the course may have introduced biases and that future work should add data from experienced professionals, which further limits the external force of the practitioner-directed claim. This is partial circularity in the evaluation claim, not in the pattern-elicitation derivation; the toolkit may still be useful, but the reported evidence does not independently establish that value.

Assumptions & free parameters 0 free parameters · 4 assumptions · 2 invented entities

The central claims rest on the assumption that student low-fidelity prototypes capture the design space, that the environment is passive, and that self-report measures reflect value. No numeric free parameters are fitted; the artifact contributions are conceptual. The main burden is the transferability and evaluation assumptions.

assumptions (4)
  • domain assumption Low-fidelity prototypes built by 25 graduate students in a 10-week course surface representative MixITS design challenges.
    Underpins the entire derivation of design considerations and patterns in Sections 3 and 4; the authors flag curricular bias in Section 8.1 but do not validate transferability.
  • domain assumption The physical environment can be treated as a passive entity affected by user and AI actions.
    Explicit in Section 5.2: 'We treat the physical environment as a passive entity affected by user and AI actions, rather than as an active participant.' This simplification defines the eight-gulf model.
  • domain assumption Single-analyst reflexive thematic analysis without inter-rater reliability yields valid themes.
    Section 4.1 invokes Braun and Clarke to justify a single analyst; acceptable in qualitative HCI but an assumption about interpretive validity.
  • domain assumption Participant Likert self-reports and pattern-recognition accuracy measure toolkit usefulness.
    The evaluation in Section 6 relies on these measures; no behavioral outcome or design-quality metric is used.
invented entities (2)
  • Eight MixITS Interaction Gulfs
    purpose: Analytical vocabulary to classify execution and evaluation gaps between human, AI-MR system, and environment.
    A conceptual framework introduced in Section 5.2; it is used to label the 36 patterns and to structure the Canvas, and the evaluation asks participants to identify these gulfs, so evidence is internal.
  • MixITS-Kit
    purpose: Toolkit for designing MixITS systems, composed of the Canvas, Design Considerations, and Design Patterns.
    The paper's main artifact; its utility is assessed through self-report and internal consistency tasks rather than external outcomes.

how reviews work

0 comments
Cite this review

Pith. "Pith review of An Interaction Design Toolkit for Physical Task Guidance with Artificial Intelligence and Mixed Reality." pith.science (2026). https://pith.science/paper/37KL4KNI

@misc{pith2026241216892,
  author       = {Pith},
  title        = {Pith review of: An Interaction Design Toolkit for Physical Task Guidance with Artificial Intelligence and Mixed Reality},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/37KL4KNI}},
  note         = {Machine review of arXiv:2412.16892}
}
read the original abstract

Physical skill acquisition, from sports techniques to surgical procedures, requires instruction and feedback. In the absence of a human expert, Physical Task Guidance (PTG) systems can offer a promising alternative. These systems integrate Artificial Intelligence (AI) and Mixed Reality (MR) to provide realtime feedback and guidance as users practice and learn skills using physical tools and objects. However, designing PTG systems presents challenges beyond engineering complexities. The intricate interplay between users, AI, MR interfaces, and the physical environment creates unique interaction design hurdles. To address these challenges, we present an interaction design toolkit derived from our analysis of PTG prototypes developed by eight student teams during a 10-week-long graduate course. The toolkit comprises Design Considerations, Design Patterns, and an Interaction Canvas. Our evaluation suggests that the toolkit can serve as a valuable resource for practitioners designing PTG systems and researchers developing new tools for human-AI interaction design.

Figures

Figures reproduced from arXiv: 2412.16892 by the authors.

Figure 1
Figure 1. Low-fidelity prototyping allowed students to iterate on the design of MixITS systems with a small budget. Students created [PITH_FULL_IMAGE:figures/full_fig_p008_1.png] view at source ↗
Figure 2
Figure 2. Six MixITS Design Considerations produced by a reflexive thematic analysis of data collected in the design course. Section 5 presents the considerations in detail. 5 MIXIT-KIT DESIGN TOOLKIT From the analysis of the extensive documentation of the interactive design process of MixITS prototypes during the course, we distilled six design considerations for the design of future MixITS systems. We encourage readers to c… view at source ↗
Figure 3
Figure 3. When applying Don Norman’s Gulfs of Execution and Evaluation [ [PITH_FULL_IMAGE:figures/full_fig_p016_3.png] view at source ↗
Figures from the paper (3 more)
Figure 4
Figure 4. Figure 4: The MixITS Interaction Canvas can help designers identify interaction problems leading to the Gulfs of Execution and Evaluation between the AI system, Human users, and the Environment. Designers can use this visual tool by filling out the blanks, in gray text for both …
Figure 5
Figure 5. Figure 5: (a) In Task 1, participants used Interaction Canvases to identify a Gulf on fictional-user issues. (b) They then referred to [PITH_FULL_IMAGE:figures/full_fig_p019_5.png]
Figure 6
Figure 6. Figure 6: Participant agreement levels (5-point Likert scale) with statements about the Design Toolkit. [PITH_FULL_IMAGE:figures/full_fig_p022_6.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

94 extracted references · 66 canonical work pages

  1. [1]

    Christopher Alexander, Sara Ishikawa, Murray Silverstein, Max Jacobson, Ingrid Fiksdahl-King, and Angel Shlomo. 1977. A pattern language : towns, buildings, construction. Oxford University Press, New York

    1977

  2. [2]

    James E Allen, Curry I Guinn, and Eric Horvtz. 1999. Mixed-initiative interaction. IEEE Intelligent Systems and their Applications 14, 5 (1999), 14–23

    1999

  3. [3]

    Saleema Amershi, Dan Weld, Mihaela Vorvoreanu, Adam Fourney, Besmira Nushi, Penny Collisson, Jina Suh, Shamsi Iqbal, Paul N Bennett, Kori Inkpen, et al. 2019. Guidelines for human-AI interaction. In Proceedings of the 2019 chi conference on human factors in computing systems . 1–13. https://doi.org/10.1145/3290605.3300233

    arXiv 2019

  4. [4]

    Fraser Anderson, Tovi Grossman, Justin Matejka, and George Fitzmaurice. 2013. YouMove: enhancing movement training with an augmented reality mirror. In Proceedings of the 26th annual ACM symposium on User interface software and technology . 311–320

    2013

  5. [5]

    John R Anderson, C Franklin Boyle, and Brian J Reiser. 1985. Intelligent tutoring systems. Science 228, 4698 (1985), 456–462. https://doi.org/10.1126/ science.228.4698.456

    1985

  6. [6]

    Sean Andrist, Dan Bohus, and Ashley Feniello. 2019. Demonstrating a framework for rapid development of physically situated interactive systems. In 2019 14th ACM/IEEE International Conference on Human-Robot Interaction (HRI) . IEEE, 668–668. 25 Caetano et al

    2019

  7. [7]

    Alejandro Aponte, Arthur Caetano, Yunhao Luo, and Misha Sra. 2024. GraV: Grasp Volume Data for the Design of One-Handed XR Interfaces. In Proceedings of the 2024 ACM Designing Interactive Systems Conference . 151–167

    2024

  8. [8]

    Apple. 2024. Designing for visionOS. Retrieved February 5, 2024 from https://developer.apple.com/design/human-interface-guidelines/designing- for-visionos

    2024

Show all 94 references
  1. [9]

    Valentino Artizzu, Kris Luyten, Gustavo Rovelo Ruiz, and Lucio Davide Spano. 2024. ViRgilites: Multilevel Feedforward for Multimodal Interaction in VR. Proceedings of the ACM on Human-Computer Interaction 8, EICS (2024), 1–24

    2024

  2. [10]

    Narges Ashtari, Andrea Bunt, Joanna McGrenere, Michael Nebeling, and Parmit K Chilana. 2020. Creating augmented and virtual reality applications: Current practices, challenges, and opportunities. In Proceedings of the 2020 CHI conference on human factors in computing systems . 1–13

    2020

  3. [11]

    Guillermo Bernal, Nelson Hidalgo, Conor Russomanno, and Pattie Maes. 2022. Galea: A physiological sensing system for behavioral research in Virtual Environments. In 2022 IEEE Conference on Virtual Reality and 3D User Interfaces (VR) . IEEE, 66–76

    2022

  4. [12]

    Dan Bohus, Sean Andrist, Ashley Feniello, Nick Saw, Mihai Jalobeanu, Patrick Sweeney, Anne Loomis Thompson, and Eric Horvitz. 2021. Platform for situated intelligence. arXiv preprint arXiv:2103.15975 (2021)

    2021 arXiv

  5. [13]

    Dan Bohus, Sean Andrist, Nick Saw, Ann Paradiso, Ishani Chakraborty, and Mahdi Rad. 2024. SIGMA: An Open-Source Interactive System for Mixed-Reality Task Assistance Research – Extended Abstract. 2024 IEEE Conference on Virtual Reality and 3D User Interfaces Abstracts and Works...

    2024

  6. [14]

    Jan O Borchers. 2000. A pattern approach to interaction design. In Proceedings of the 3rd conference on Designing interactive systems: processes, practices, methods, and techniques . 369–378

    2000

  7. [15]

    Virginia Braun and Victoria Clarke. 2006. Using thematic analysis in psychology. Qualitative research in psychology 3, 2 (2006), 77–101

    2006

  8. [16]

    2024.A range of ways of approaching (reflexive) TA

    Virginia Braun and Victoria Clarke. 2024.A range of ways of approaching (reflexive) TA. Retrieved February 5, 2024 from https://www.thematicanalysis. net/understanding-ta/

    2024

  9. [17]

    Colin Burns, Eric Dishman, William Verplank, and Bud Lassiter. 1994. Actors, hairdos & videotape—informance design. In Conference companion on Human factors in computing systems . 119–120

    1994

  10. [18]

    Richard Byrne, Joe Marshall, and Florian’Floyd’ Mueller. 2016. Balance ninja: towards the design of digital vertigo games via galvanic vestibular stimulation. In Proceedings of the 2016 Annual Symposium on Computer-Human Interaction in Play . 159–170

    2016

  11. [19]

    Donald T Campbell. 1986. Relabeling internal and external validity for applied social scientists. New Directions for Program Evaluation 1986, 31 (1986), 67–77

    1986

  12. [20]

    Sonia Castelo, Joao Rulff, Erin McGowan, Bea Steers, Guande Wu, Shaoyu Chen, Iran Roman, Roque Lopez, Ethan Brewer, Chen Zhao, et al. 2023. Argus: Visualization of ai-assisted task guidance in ar. IEEE Transactions on Visualization and Computer Graphics (2023)

    2023

  13. [21]

    Ishan Chatterjee, Tadeusz Pforte, Aspen Tng, Farshid Salemi Parizi, Chaoran Chen, and Shwetak Patel. 2022. ARDW: An augmented reality workbench for printed circuit board debugging. In Proceedings of the 35th Annual ACM Symposium on User Interface Software and Technology . 1–16

    2022

  14. [22]

    Hwan-Hee Choi, Jeroen JG Van Merriënboer, and Fred Paas. 2014. Effects of the physical environment on cognitive load and learning: Towards a new model of cognitive load. Educational psychology review 26 (2014), 225–244

    2014

  15. [23]

    Sven Coppers, Kris Luyten, Davy Vanacken, David Navarre, Philippe Palanque, and Christine Gris. 2019. Fortunettes: feedforward about the future state of GUI widgets. Proceedings of the ACM on Human-Computer Interaction 3, EICS (2019), 1–20

    2019

  16. [24]

    Design Council. 2024. The Double Diamond. Retrieved February 5, 2024 from https://www.designcouncil.org.uk/our-resources/the-double-diamond/

    2024

  17. [25]

    Dan Curtis, David Mizell, Peter Gruenbaum, and Adam Janin. 1999. Several devils in the details: making an AR application work in the airplane factory. In Proc. Int’l Workshop Augmented Reality. 47–60

    1999

  18. [26]

    Marco De Sá and Elizabeth Churchill. 2012. Mobile augmented reality: exploring design and prototyping techniques. In Proceedings of the 14th international conference on Human-computer interaction with mobile devices and services . 221–230

    2012

  19. [27]

    Mats Ole Ellenberg, Marc Satkowski, Weizhou Luo, and Raimund Dachselt. 2023. Spatiality and Semantics-Towards Understanding Content Placement in Mixed Reality. In Extended Abstracts of the 2023 CHI Conference on Human Factors in Computing Systems . 1–8

    2023

  20. [28]

    Chris Elsden, Ella Tallyn, and Bettina Nissen. 2020. When Do Design Workshops Work (or Not)?. In Companion Publication of the 2020 ACM Designing Interactive Systems Conference . 245–250. https://doi.org/10.1145/3393914.3395856

    2020

  21. [29]

    Amir Erez and Alice M Isen. 2002. The influence of positive affect on the components of expectancy motivation. Journal of Applied psychology 87, 6 (2002), 1055

    2002

  22. [30]

    K Anders Ericsson and Herbert A Simon. 1980. Verbal reports as data. Psychological review 87, 3 (1980), 215

    1980

  23. [31]

    Anke Eyck, Kelvin Geerlings, Dina Karimova, Bernt Meerbeek, Lu Wang, Wijnand IJsselsteijn, Yvonne De Kort, Michiel Roersma, and Joyce Westerink. 2006. Effect of a virtual coach on athletes’ motivation. In Persuasive Technology: First International Conference on Persuasive Tech...

    2006

  24. [32]

    Steven Feiner, Blair MacIntyre, and Doree Seligmann. 1993. Knowledge-based augmented reality. Commun. ACM 36, 7 (1993), 53–62

    1993

  25. [33]

    KJ Kevin Feng, Maxwell James Coppock, and David W McDonald. 2023. How Do UX Practitioners Communicate AI as a Design Material? Artifacts, Conceptions, and Propositions. In Proceedings of the 2023 ACM Designing Interactive Systems Conference . 2263–2280. https://doi.org/10.1145...

    2023 doi

  26. [34]

    John H Flavell. 1976. Metacognitive aspects of problem solving. In The nature of intelligence . Routledge, 231–236. 26 An Interaction Design Toolkit for Physical Task Guidance with Artificial Intelligence and Mixed Reality

    1976

  27. [35]

    Gabriel Freitas, Marcio Sarroglia Pinho, Milene Selbach Silveira, and Frank Maurer. 2020. A systematic review of rapid prototyping tools for augmented reality. In 2020 22nd Symposium on Virtual and Augmented Reality (SVR) . IEEE, 199–209

    2020

  28. [36]

    Steffen Haesler, Kangsoo Kim, Gerd Bruder, and Greg Welch. 2018. Seeing is believing: improving the perceived trust in visually embodied Alexa in augmented reality. In 2018 IEEE International Symposium on Mixed and Augmented Reality Adjunct (ISMAR-Adjunct) . IEEE, 204–205

    2018

  29. [37]

    Shannon F Harp and Richard E Mayer. 1997. The role of interest in learning from scientific text and illustrations: On the distinction between emotional interest and cognitive interest. Journal of educational psychology 89, 1 (1997), 92

    1997

  30. [38]

    Steven Henderson and Steven Feiner. 2010. Exploring the benefits of augmented reality documentation for maintenance and repair.IEEE transactions on visualization and computer graphics 17, 10 (2010), 1355–1368

    2010

  31. [39]

    Teresa Hirzle, Florian Müller, Fiona Draxler, Martin Schmitz, Pascal Knierim, and Kasper Hornbæk. 2023. When xr and ai meet-a scoping review on extended reality and artificial intelligence. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems . 1–45

    2023

  32. [40]

    Kasper Hornbæk and Antti Oulasvirta. 2017. What is interaction?. In Proceedings of the 2017 CHI conference on human factors in computing systems . 5040–5052

    2017

  33. [41]

    Eric Horvitz. 1999. Principles of mixed-initiative user interfaces. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (Pittsburgh, Pennsylvania, USA) (CHI ’99). Association for Computing Machinery, New York, NY, USA, 159–166. https://doi.org/10.1145...

    1999 doi

  34. [42]

    Edwin L Hutchins, James D Hollan, and Donald A Norman. 1985. Direct manipulation interfaces. Human–computer interaction 1, 4 (1985), 311–338

    1985

  35. [43]

    John F Kelley. 1983. An empirical methodology for writing user-friendly natural language computer applications. In Proceedings of the SIGCHI conference on Human Factors in Computing Systems . 193–196

    1983

  36. [44]

    Kenneth R Koedinger and Vincent Aleven. 2007. Exploring the assistance dilemma in experiments with cognitive tutors. Educational Psychology Review 19 (2007), 239–264

    2007

  37. [45]

    Diana Laurillard. 2013. Teaching as a design science: Building pedagogical patterns for learning and technology . Routledge

    2013

  38. [46]

    Joseph LaViola. 2017. 3D User Interfaces (2nd edition ed.). Addison-Wesley Professional

    2017

  39. [47]

    David Ledo, Steven Houben, Jo Vermeulen, Nicolai Marquardt, Lora Oehlberg, and Saul Greenberg. 2018. Evaluation strategies for HCI toolkit research. In Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems . 1–17

    2018

  40. [48]

    Germán Leiva, Cuong Nguyen, Rubaiat Habib Kazi, and Paul Asente. 2020. Pronto: Rapid augmented reality video prototyping using sketches and enaction. In Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems . 1–13

    2020

  41. [49]

    Tianyi Li, Mihaela Vorvoreanu, Derek DeBellis, and Saleema Amershi. 2023. Assessing human-AI interaction early through factorial surveys: a study on the guidelines for human-AI interaction. ACM Transactions on Computer-Human Interaction 30, 5 (2023), 1–45

    2023

  42. [50]

    Xingyu Bruce Liu, Jiahao Nick Li, David Kim, Xiang’Anthony’ Chen, and Ruofei Du. 2024. Human I/O: Towards a Unified Approach to Detecting Situational Impairments. In Proceedings of the CHI Conference on Human Factors in Computing Systems . 1–18

    2024

  43. [51]

    Maria Luce Lupetti and Dave Murray-Rust. 2024. (Un) making AI Magic: A Design Taxonomy. In Proceedings of the CHI Conference on Human Factors in Computing Systems . 1–21

    2024

  44. [52]

    Sara Mandic, Rhys Tracy, and Misha Sra. 2023. ARFit: Pose-based Exercise Feedback with Mobile AR. In Proceedings of the 2023 ACM Symposium on Spatial User Interaction. 1–3

    2023

  45. [53]

    Joseph A Maxwell. 2010. Using numbers in qualitative research. Qualitative inquiry 16, 6 (2010), 475–482

    2010

  46. [54]

    Meta. 2024. Designing for Mixed Reality . Retrieved February 5, 2024 from https://developer.oculus.com/resources/mr-design-guideline/

    2024

  47. [55]

    Microsoft. 2024. Thinking Differently for Mixed Reality . Retrieved February 5, 2024 from https://learn.microsoft.com/en-us/windows/mixed- reality/discover/case-study-expanding-the-design-process-for-mixed-reality

    2024

  48. [56]

    Paul Milgram, Haruo Takemura, Akira Utsumi, and Fumio Kishino. 1995. Augmented reality: A class of displays on the reality-virtuality continuum. In Telemanipulator and telepresence technologies, Vol. 2351. Spie, 282–292

    1995

  49. [57]

    Andreea Muresan, Jess McIntosh, and Kasper Hornbæk. 2023. Using feedforward to reveal interaction possibilities in virtual reality.ACM Transactions on Computer-Human Interaction 30, 6 (2023), 1–47

    2023

  50. [58]

    Romain Nith, Yun Ho, and Pedro Lopes. 2024. SplitBody: Reducing Mental Workload while Multitasking via Muscle Stimulation. In Proceedings of the CHI Conference on Human Factors in Computing Systems . 1–11

    2024

  51. [59]

    D. Norman. 2013. The Design of Everyday Things: Revised and Expanded Edition . Basic Books. https://books.google.com/books?id=nVQPAAAAQBAJ

    2013

  52. [60]

    Donald A Norman. 1981. Categorization of action slips. Psychological review 88, 1 (1981), 1

    1981

  53. [61]

    Donald A Norman. 1986. User-centered System Design: New Perspectives on Human–computer Interaction

    1986

  54. [62]

    Antti Oulasvirta, Esko Kurvinen, and Tomi Kankainen. 2003. Understanding contexts by being there: case studies in bodystorming. Personal and ubiquitous computing 7 (2003), 125–134

    2003

  55. [63]

    Long Ouyang, Jeffrey Wu, Xu Jiang, Diogo Almeida, Carroll Wainwright, Pamela Mishkin, Chong Zhang, Sandhini Agarwal, Katarina Slama, Alex Ray, et al. 2022. Training language models to follow instructions with human feedback. Advances in Neural Information Processing Systems 35...

    2022

  56. [64]

    Denise F Polit and Cheryl Tatano Beck. 2010. Generalization in quantitative and qualitative research: Myths and strategies. International journal of nursing studies 47, 11 (2010), 1451–1458

    2010

  57. [65]

    BURTON R. R. and ITS. International Conference. 1982. Diagnosing Bugs in a Simple Procedural Skill, Sleeman. Intelligent Tutoring Systems (1982). 27 Caetano et al

    1982

  58. [66]

    Sebastian Felix Rauh, Cristian Bogdan, Gerrit Meixner, and Andrii Matviienko. 2024. Navigating the Virtuality-Reality Clash: Reflection and Design Patterns for Industrial Mixed Reality Applications. In Proceedings of the 2024 ACM Designing Interactive Systems Conference . 2247–2266

    2024

  59. [67]

    Symeon Retalis, Petros Georgiakakis, and Yannis Dimitriadis. 2006. Eliciting design patterns for e-learning systems. Computer Science Education 16, 2 (2006), 105–118

    2006

  60. [68]

    Benjamin Rheault, Shivvrat Arya, Akshay Vyas, Jikai Wang, Rohith Peddi, Brett Bendall, Vibhav Gogate, Nicholas Ruozzi, Yu Xiang, and Eric D Ragan. 2024. Predictive Task Guidance with Artificial Intelligence in Augmented Reality. In 2024 IEEE Conference on Virtual Reality and 3...

    2024

  61. [69]

    Liam Rigby, Burkhard C Wünsche, and Alex Shaw. 2020. piARno-an augmented reality piano tutor. In Proceedings of the 32nd Australian Conference on Human-Computer Interaction. 481–491

    2020

  62. [70]

    Björn Schwerdtfeger, Troels Frimor, Daniel Pustka, and Gudrun Klinker. 2006. Mobile information presentation schemes for supra-adaptive logistics applications. In Advances in Artificial Reality and Tele-Existence: 16th International Conference on Artificial Reality and Telexis...

    2006

  63. [71]

    Ambika Shahu, Sonja Dorfbauer, Philipp Wintersberger, and Florian Michahelles. 2023. Skillab - A Multimodal Augmented Reality Environment for Learning Manual Tasks. In Human-Computer Interaction – INTERACT 2023 , José Abdelnour Nocera, Marta Kristín Lárusdóttir, Helen Petrie, ...

    2023

  64. [72]

    Scott A Shappell and Douglas A Wiegmann. 2000. The human factors analysis and classification system–HFACS. (2000)

    2000

  65. [73]

    Adwait Sharma, Michael A Hedderich, Divyanshu Bhardwaj, Bruno Fruchard, Jess McIntosh, Aditya Shekhar Nittala, Dietrich Klakow, Daniel Ashbrook, and Jürgen Steimle. 2021. SoloFinger: Robust microgestures while grasping everyday objects. In Proceedings of the 2021 CHI conferenc...

    2021

  66. [74]

    Tomasz Sosnowski, Teodor Stoev, Thomas Kirste, and Kristina Yordanova. 2023. Challenges in Modelling Cooking Task Execution for User Assistance. In Proceedings of the 8th international Workshop on Sensor-Based Activity Recognition and Artificial Intelligence . 1–4

    2023

  67. [75]

    Misha Sra, Xuhai Xu, and Pattie Maes. 2017. GalVR: a novel collaboration interface using GVS. In Proceedings of the 23rd ACM Symposium on Virtual Reality Software and Technology. 1–2

    2017

  68. [76]

    Amant and Paul R Cohen

    Robert St. Amant and Paul R Cohen. 1997. Interaction with a mixed-initiative system for exploratory data analysis. In Proceedings of the 2nd international conference on Intelligent user interfaces . 15–22

    1997

  69. [77]

    Dag Svanaes and Gry Seland. 2004. Putting the users center stage: role playing and low-fi prototyping enable end users to design mobile systems. In Proceedings of the SIGCHI conference on Human factors in computing systems . 479–486

    2004

  70. [78]

    Arthur Tang, Charles Owen, Frank Biocca, and Weimin Mou. 2003. Comparative effectiveness of augmented reality in object assembly. InProceedings of the SIGCHI conference on Human factors in computing systems . 73–80

    2003

  71. [79]

    Michael Terry, Chinmay Kulkarni, Martin Wattenberg, Lucas Dixon, and Meredith Ringel Morris. 2023. AI Alignment in the Design of Interactive AI: Specification Alignment, Process Alignment, and Evaluation Support. arXiv preprint arXiv:2311.00710 (2023)

    2023 arXiv

  72. [80]

    P CAUDELL Thomas and WM David. 1992. Augmented reality: An application of heads-up display technology to manual manufacturing processes. In Hawaii international conference on system sciences , Vol. 2. ACM SIGCHI Bulletin, 659–669

    1992

  73. [81]

    Marcel Tiator, Christian Geiger, Bastian Dewitz, Ben Fischer, Laurin Gerhardt, David Nowottnik, and Hendrik Preu. 2018. Venga! climbing in mixed reality. In Proceedings of the First Superhuman Sports Design Challenge: First International Symposium on Amplifying Capabilities an...

    2018

  74. [82]

    ME Tresselt and MS Mayzner. 1960. A study of incidental learning. The journal of psychology 50, 2 (1960), 339–347

    1960

  75. [83]

    Ultraleap. 2024. XR Design Guidelines. Retrieved February 5, 2024 from https://docs.ultraleap.com/xr-guidelines/

    2024

  76. [84]

    Marieke van Asselen, Eva Fritschy, and Albert Postma. 2006. The influence of intentional and incidental learning on acquiring spatial knowledge during navigation. Psychological Research 70 (2006), 151–156

    2006

  77. [85]

    Jo Vermeulen, Kris Luyten, Elise van den Hoven, and Karin Coninx. 2013. Crossing the bridge over Norman’s Gulf of Execution: revealing feedforward’s true identity. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems . 1931–1940

    2013

  78. [86]

    Peng Wang, Xiaoliang Bai, Mark Billinghurst, Shusheng Zhang, Xiangyu Zhang, Shuxia Wang, Weiping He, Yuxiang Yan, and Hongyu Ji. 2021. AR/MR remote collaboration on physical tasks: a review. Robotics and Computer-Integrated Manufacturing 72 (2021), 102071

    2021

  79. [87]

    Kimberly A Weaver, Hannes Baumann, Thad Starner, Hendrick Iben, and Michael Lawo. 2010. An empirical task analysis of warehouse order picking using head-mounted displays. In Proceedings of the SIGCHI conference on human factors in computing systems . 1695–1704

    2010

  80. [88]

    Niall Winters and Yishay Mor. 2009. Dealing with abstraction: Case study generalisation as a method for eliciting design patterns. Computers in Human Behavior 25, 5 (2009), 1079–1088

    2009

  81. [89]

    Guande Wu, Jing Qian, Sonia Castelo Quispe, Shaoyu Chen, João Rulff, and Claudio Silva. 2024. ARTiST: Automated Text Simplification for Task Guidance in Augmented Reality. In Proceedings of the CHI Conference on Human Factors in Computing Systems . 1–24

    2024

  82. [90]

    Xuhai Xu, Anna Yu, Tanya R Jonker, Kashyap Todi, Feiyu Lu, Xun Qian, João Marcelo Evangelista Belo, Tianyi Wang, Michelle Li, Aran Mun, et al

  83. [91]

    Qian Yang, Aaron Steinfeld, Carolyn Rosé, and John Zimmerman. 2020. Re-examining whether, why, and how human-AI interaction is uniquely difficult to design. In Proceedings of the 2020 chi conference on human factors in computing systems . 1–13. https://doi.org/10.1145/3313831....

    2020

  84. [92]

    Nur Yildirim, Alex Kass, Teresa Tung, Connor Upton, Donnacha Costello, Robert Giusti, Sinem Lacin, Sara Lovic, James M O’Neill, Rudi O’Reilly Meehan, et al. 2022. How experienced designers of enterprise applications engage AI as a design material. In Proceedings of the 2022 CH...

    2022

  85. [93]

    Dong Woo Yoo, Hamid Tarashiyoun, and Mohsen Moghaddam. 2023. Modeling gaze behavior for real-time estimation of visual attention and expertise level in augmented reality. In2023 IEEE International Symposium on Mixed and Augmented Reality Adjunct (ISMAR-Adjunct). IEEE, 487–492....

    2023

  86. [2023]

    In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems

    XAIR: A Framework of Explainable AI in Augmented Reality. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–30

    2023

Pith tools

Reviewed August 11, 2026 · model on record in the stance chip above.