pith. sign in

arxiv: 2605.21635 · v1 · pith:MMXD6NCLnew · submitted 2026-05-20 · 💻 cs.HC · cs.AI· cs.CY

Addressing the Synergy Gap: The Six Elements of the Design Space

Tommaso Turchi , Ben Wilson , Matt Roach , Alan Dix , Alessio Malizia This is my paper

Pith reviewed 2026-05-22 09:10 UTC · model grok-4.3

classification 💻 cs.HC cs.AIcs.CY
keywords synergy gaphuman-AI collaborationdesign spacesociotechnical contextdecision frameworksholistic evaluationAI assistancehybrid systems
0
0 comments X

The pith

Closing the human-AI synergy gap requires explicit design across six interconnected elements instead of narrow focus on interfaces or trust alone.

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper defines a synergy gap in which AI assistance rarely produces combined results better than humans or AI working separately. Current efforts center on making AI interpretable, calibrating trust, or refining interfaces, yet these address only fragments of what makes combinations succeed. The authors map a wider design space through six linked elements: the surrounding sociotechnical context, the frameworks that structure decisions, the specific human participants, the AI's actual capabilities, the forms of interaction, and methods of holistic evaluation. Treating these elements as interconnected shows how choices in one area shape outcomes in others. The resulting vocabulary and lens aim to guide practitioners toward systems that reliably achieve genuine synergy.

Core claim

Genuine human-AI synergy remains rare because existing work treats combination as a narrow engineering task; the authors argue that closing the gap instead demands engagement with a design space of six interconnected elements—sociotechnical context, decision-making frameworks, human decision participants, AI capabilities, interaction, and holistic evaluation—each of which shapes the others and together supply the shared vocabulary, analytical lens, and evaluation criteria needed for hybrid systems to outperform either party alone.

What carries the argument

The six interconnected elements of the design space—sociotechnical context, decision-making frameworks, human decision participants, AI capabilities, interaction, and holistic evaluation—that together determine whether human-AI combination produces synergy.

If this is right

  • Practitioners gain a shared vocabulary to design hybrid systems that target synergy rather than isolated improvements.
  • Researchers obtain an analytical lens for studying patterns across different human-AI combination attempts.
  • Evaluators can measure the full quality of decision-making instead of accuracy metrics alone.
  • Design choices in any one element must account for effects on the other five to avoid unintended shortfalls in performance.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • Testing the framework in one domain such as medical diagnosis could identify which single element most often blocks synergy when left unaddressed.
  • The elements may help explain why some human-AI teams achieve synergy in high-stakes settings while similar teams do not.
  • Extending the map to multi-human multi-AI groups could reveal additional interaction patterns not covered by the current six.

Load-bearing premise

That engaging explicitly with these six elements will close the synergy gap where narrower approaches focused on interpretability, trust, or interfaces have not.

What would settle it

A side-by-side study of human-AI teams in the same decision task, one built with only current narrow methods and one incorporating all six elements, showing no measurable increase in synergy rates for the broader design.

Figures

Figures reproduced from arXiv: 2605.21635 by Alan Dix, Alessio Malizia, Ben Wilson, Matt Roach, Tommaso Turchi.

Figure 1
Figure 1. Figure 1: The synergy gap (exemplified) in human-AI decision-making. Different studies show varying degrees of this [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 2
Figure 2. Figure 2: The six elements of the design space for human-AI decision systems. [PITH_FULL_IMAGE:figures/full_fig_p006_2.png] view at source ↗
read the original abstract

AI is now embedded in healthcare, finance, policy, and many other domains, yet genuine human-AI synergy - combined performance that exceeds what either party achieves alone - is uncommon. Meta-analyses show that AI assistance tends to improve human performance compared to working alone, but studies finding true synergy are scarce. We call this persistent shortfall the synergy gap. Most current work treats human-AI combination as an engineering problem and concentrates on interpretability, trust calibration, or interface design. These matter, but they cover only part of what determines whether combination works. Closing the synergy gap, we argue, requires explicit engagement with a wider design space. We map that space through six interconnected elements: sociotechnical context, decision-making frameworks, human decision participants, AI capabilities, interaction, and holistic evaluation. For each element, we describe what it covers, how it shapes the others in practice, and what it implies for design. The result is a shared vocabulary for practitioners building hybrid systems, an analytical lens for researchers studying combination patterns, and a starting point for evaluators interested in the full quality of human-AI decision-making rather than accuracy alone.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit. Tearing a paper down is the easy half of reading it; the pith above is the substance, this is the friction.

Referee Report

1 major / 2 minor

Summary. The manuscript identifies a 'synergy gap' in human-AI decision making, where meta-analyses show that AI assistance typically improves human performance relative to working alone but true synergy (combined performance exceeding either alone) remains rare. It argues that prevailing efforts centered on interpretability, trust calibration, and interface design address only part of the problem. The authors map a wider design space through six interconnected elements—sociotechnical context, decision-making frameworks, human decision participants, AI capabilities, interaction, and holistic evaluation—describing for each what it covers, how it shapes the others, and its design implications. The framework is positioned as a shared vocabulary for practitioners, an analytical lens for researchers, and a starting point for evaluators focused on the full quality of hybrid decision making.

Significance. If the framework is taken up, it could usefully broaden research and practice in human-AI systems by shifting attention from narrow technical interventions to the full set of sociotechnical and evaluative factors that determine whether synergy occurs. The integrative structure drawing on existing meta-analyses provides a coherent organizing lens that may help practitioners avoid piecemeal solutions in domains such as healthcare and finance. Its value as an explicit starting point rather than a tested model means immediate impact will depend on subsequent empirical work that applies or refines the six-element mapping.

major comments (1)
  1. The section defining the six elements: the claim that these six elements comprehensively cover the remaining factors determining human-AI synergy (beyond interpretability, trust, and interfaces) is central to the argument yet rests on descriptive assertion rather than a systematic gap analysis or comparison against alternative partitions of the design space; this leaves open the possibility that important factors are omitted or that the mapping is not minimal.
minor comments (2)
  1. The abstract and introduction would benefit from one concrete, worked example showing how the six elements interact in a single domain (e.g., medical diagnosis) to illustrate the claimed interconnections.
  2. Notation for the elements is clear, but a summary table or diagram listing each element, its key sub-factors, and cross-element influences would improve readability and support the 'shared vocabulary' goal.

Simulated Author's Rebuttal

1 responses · 0 unresolved

We thank the referee for the constructive feedback, which helps us clarify the foundations and scope of the proposed framework. We address the single major comment below.

read point-by-point responses

  1. Referee: The section defining the six elements: the claim that these six elements comprehensively cover the remaining factors determining human-AI synergy (beyond interpretability, trust, and interfaces) is central to the argument yet rests on descriptive assertion rather than a systematic gap analysis or comparison against alternative partitions of the design space; this leaves open the possibility that important factors are omitted or that the mapping is not minimal.

    Authors: We acknowledge that the six elements are advanced as a synthesized mapping of the design space rather than the direct product of a formal, exhaustive gap analysis or comparison to alternative partitions. The selection integrates recurring factors identified across meta-analyses and empirical studies of human-AI decision making in domains such as healthcare and finance, with the explicit goal of extending attention beyond interpretability, trust calibration, and interface design. We do not claim the partition is unique, minimal, or exhaustive; the manuscript already positions the framework as a shared vocabulary and starting point for further work. To address the concern directly, we will revise the relevant section to include a concise rationale for the element selection, drawing on the literature synthesis, and to note that other coherent organizations of the space remain possible. This will temper any implication of comprehensive coverage while preserving the integrative value of the six-element structure. revision: yes

Circularity Check

0 steps flagged

No significant circularity: conceptual framework proposal

full rationale

The paper is a descriptive and prescriptive conceptual framework that maps a design space for addressing the human-AI synergy gap. It defines the gap via cited meta-analyses and proposes six interconnected elements as an analytical lens and shared vocabulary. No equations, derivations, fitted parameters, or quantitative predictions exist. The elements are introduced by explicit description rather than by reduction to prior inputs or self-referential definitions. Interconnections are presented qualitatively as implications for design, not as forced outputs of any model. The central claim rests on the analytical assertion that existing work (interpretability, trust, interfaces) is partial, which is supported by external literature rather than internal self-citation chains or constructed equivalences. This is a self-contained HCI framework paper with no load-bearing steps that reduce to the paper's own inputs.

Axiom & Free-Parameter Ledger

0 free parameters · 1 axioms · 1 invented entities

The paper rests on background meta-analyses about AI assistance effects and introduces the synergy gap term plus the six-element structure as a new organizing framework without new empirical data or formal derivations.

axioms (1)
  • domain assumption Meta-analyses show that AI assistance tends to improve human performance compared to working alone, but studies finding true synergy are scarce.
    Stated directly in the abstract as the foundation for identifying the synergy gap.
invented entities (1)
  • synergy gap no independent evidence
    purpose: To label the persistent shortfall where genuine human-AI synergy is uncommon
    Defined by the authors based on the scarcity of synergy findings in meta-analyses; no independent falsifiable handle provided in abstract.

pith-pipeline@v0.9.0 · 5738 in / 1372 out tokens · 41407 ms · 2026-05-22T09:10:45.288405+00:00 · methodology

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

40 extracted references · 40 canonical work pages

  1. [1]

    Decision Making in Action: Models and Methods , shorttitle =

    work page

  2. [2]

    2011 , series =

    Thinking, Fast and Slow , author =. 2011 , series =

    work page 2011

  3. [3]

    Vroom, V. H. and Jago, A. G. , title =. American Psychologist , year =

    work page

  4. [4]

    Kurtz, C. F. and Snowden, D. J. , title =. IBM Syst. J. , month = jul, pages =. 2003 , issue_date =. doi:10.1147/sj.423.0462 , abstract =

    work page doi:10.1147/sj.423.0462 2003

  5. [5]

    2025 , address =

    Turchi, Tommaso and Dix, Alan and Roach, Matt and Malizia, Alessio and Wilson, Ben , booktitle =. 2025 , address =

    work page 2025

  6. [6]

    Proceedings of the 2024 International Conference on Advanced Visual Interfaces , articleno =

    Dix, Alan and Roach, Matt and Turchi, Tommaso and Malizia, Alessio and Wilson, Ben , title =. Proceedings of the 2024 International Conference on Advanced Visual Interfaces , articleno =. 2024 , isbn =. doi:10.1145/3656650.3660537 , abstract =

    work page doi:10.1145/3656650.3660537 2024

  7. [7]

    Proceedings of the AAAI Conference on Human Computation and Crowdsourcing , author=

    A Taxonomy of Human and ML Strengths in Decision-Making to Investigate Human-ML Complementarity , volume=. Proceedings of the AAAI Conference on Human Computation and Crowdsourcing , author=. 2023 , month=. doi:10.1609/hcomp.v11i1.27554 , abstractNote=

    work page doi:10.1609/hcomp.v11i1.27554 2023

  8. [8]

    Ferrario, Andrea and Facchini, Alessandro and Dur'an, J. M. , journal =. Epistemology Gives a Future to Complementarity in Human-

    work page

  9. [9]

    AI Reliance and Decision Quality: Fundamentals, Interdependence, and the Effects of Interventions , year =

    Schoeffer, Jakob and Jakubik, Johannes and V\". AI Reliance and Decision Quality: Fundamentals, Interdependence, and the Effects of Interventions , year =. J. Artif. Int. Res. , month = jun, pages =. doi:10.1613/jair.1.15873 , abstract =

    work page doi:10.1613/jair.1.15873

  10. [10]

    Proceedings of the 28th International Conference on Intelligent User Interfaces , pages =

    Schemmer, Max and Kuehl, Niklas and Benz, Carina and Bartos, Andrea and Satzger, Gerhard , title =. Proceedings of the 28th International Conference on Intelligent User Interfaces , pages =. 2023 , isbn =. doi:10.1145/3581641.3584066 , abstract =

    work page doi:10.1145/3581641.3584066 2023

  11. [11]

    Proxy tasks and subjective measures can be misleading in evaluating explainable AI systems , year =

    Bu. Proxy tasks and subjective measures can be misleading in evaluating explainable AI systems , year =. Proceedings of the 25th International Conference on Intelligent User Interfaces , pages =. doi:10.1145/3377325.3377498 , abstract =

    work page doi:10.1145/3377325.3377498

  12. [12]

    Computer Graphics Forum , volume =

    Holter, Steffen and El-Assady, Mennatallah , title =. Computer Graphics Forum , volume =. doi:https://doi.org/10.1111/cgf.15107 , url =. https://onlinelibrary.wiley.com/doi/pdf/10.1111/cgf.15107 , abstract =

    work page doi:10.1111/cgf.15107

  13. [13]

    Frontiers in Computer Science , VOLUME=

    Gomez, Catalina and Cho, Sue Min and Ke, Shichang and Huang, Chien-Ming and Unberath, Mathias , TITLE=. Frontiers in Computer Science , VOLUME=. 2025 , URL=. doi:10.3389/fcomp.2024.1521066 , ISSN=

    work page doi:10.3389/fcomp.2024.1521066 2025

  14. [14]

    2026 , eprint=

    Explaining and Improving Information Complementarities in Multi-Agent Decision-making , author=. 2026 , eprint=

    work page 2026

  15. [15]

    Who Should

    Shuai Ma and Ying Lei and Xinru Wang and Chengbo Zheng and Chuhan Shi and Ming Yin and Xiaojuan Ma , bibsource =. Who Should. Proceedings of the 2023. doi:10.1145/3544548.3581058 , editor =

    work page doi:10.1145/3544548.3581058 2023

  16. [16]

    Vera and Bellamy, Rachel K

    Zhang, Yunfeng and Liao, Q. Vera and Bellamy, Rachel K. E. , title =. Proceedings of the 2020 Conference on Fairness, Accountability, and Transparency , pages =. 2020 , isbn =. doi:10.1145/3351095.3372852 , abstract =

    work page doi:10.1145/3351095.3372852 2020

  17. [17]

    Proceedings of the AAAI Conference on Human Computation and Crowdsourcing , author=

    Beyond Accuracy: The Role of Mental Models in Human-AI Team Performance , volume=. Proceedings of the AAAI Conference on Human Computation and Crowdsourcing , author=. 2019 , month=. doi:10.1609/hcomp.v7i1.5285 , abstractNote=

    work page doi:10.1609/hcomp.v7i1.5285 2019

  18. [18]

    Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems , articleno =

    Gagan Bansal and Tongshuang Wu and Joyce Zhou and Raymond Fok and Besmira Nushi and Ece Kamar and Marco T. Does the Whole Exceed its Parts? The Effect of. doi:10.1145/3411764.3445717 , editor =

    work page doi:10.1145/3411764.3445717

  19. [19]

    Proceedings of the AAAI Conference on Artificial Intelligence , author=

    Align When They Want, Complement When They Need! Human-Centered Ensembles for Adaptive Human-AI Collaboration , volume=. Proceedings of the AAAI Conference on Artificial Intelligence , author=. 2026 , month=. doi:10.1609/aaai.v40i21.38786 , abstractNote=

    work page doi:10.1609/aaai.v40i21.38786 2026

  20. [20]

    Proceedings of the Extended Abstracts of the CHI Conference on Human Factors in Computing Systems , articleno =

    Farmer, Torrence S and Ho, Chien-Ju , title =. Proceedings of the Extended Abstracts of the CHI Conference on Human Factors in Computing Systems , articleno =. 2025 , isbn =. doi:10.1145/3706599.3719998 , abstract =

    work page doi:10.1145/3706599.3719998 2025

  21. [21]

    Proceedings of the AAAI Conference on Human Computation and Crowdsourcing , author=

    Mix and Match: Characterizing Heterogeneous Human Behavior in AI-assisted Decision Making , volume=. Proceedings of the AAAI Conference on Human Computation and Crowdsourcing , author=. 2024 , month=. doi:10.1609/hcomp.v12i1.31604 , number=

    work page doi:10.1609/hcomp.v12i1.31604 2024

  22. [22]

    Advancing Human-AI Complementarity: The Impact of User Expertise and Algorithmic Tuning on Joint Decision Making , year =

    Inkpen, Kori and Chappidi, Shreya and Mallari, Keri and Nushi, Besmira and Ramesh, Divya and Michelucci, Pietro and Mandava, Vani and Vep. Advancing Human-AI Complementarity: The Impact of User Expertise and Algorithmic Tuning on Joint Decision Making , year =. ACM Trans. Comput.-Hum. Interact. , month = sep, articleno =. doi:10.1145/3534561 , abstract =

    work page doi:10.1145/3534561

  23. [23]

    Perspectives on Psychological Science , volume =

    Mark Steyvers and Aakriti Kumar , title =. Perspectives on Psychological Science , volume =. 2024 , doi =. https://doi.org/10.1177/17456916231181102 , abstract =

    work page doi:10.1177/17456916231181102 2024

  24. [24]

    A Bayesian Framework for Human-

    Amin, Saurabh and Bennouna, Amine and Huttenlocher, Daniel and Kong, Dingwen and Lyu, Liang and Ozdaglar, Asu , journal =. A Bayesian Framework for Human-

    work page

  25. [25]

    and Singh, Ambuj K

    Askarisichani, Omid and Bullo, Francesco and Friedkin, Noah E. and Singh, Ambuj K. , title =. Annals of the New York Academy of Sciences , volume =. doi:https://doi.org/10.1111/nyas.14783 , url =. https://nyaspubs.onlinelibrary.wiley.com/doi/pdf/10.1111/nyas.14783 , abstract =

    work page doi:10.1111/nyas.14783

  26. [26]

    Proceedings of the 32nd ACM Conference on User Modeling, Adaptation and Personalization , pages =

    Salimzadeh, Sara and Gadiraju, Ujwal , title =. Proceedings of the 32nd ACM Conference on User Modeling, Adaptation and Personalization , pages =. 2024 , isbn =. doi:10.1145/3627043.3659567 , abstract =

    work page doi:10.1145/3627043.3659567 2024

  27. [27]

    Systems , VOLUME =

    Bao, Ying and Gong, Wankun and Yang, Kaiwen , TITLE =. Systems , VOLUME =. 2023 , NUMBER =

    work page 2023

  28. [28]

    Puranam, Phanish , date =. Human–. 2021 , journal =. doi:10.1007/s41469-021-00095-2 , url =

    work page doi:10.1007/s41469-021-00095-2 2021

  29. [29]

    European Journal of Information Systems , volume =

    Patrick Hemmer and Max Schemmer and Niklas Kühl and Michael Vössing and Gerhard Satzger , title =. European Journal of Information Systems , volume =. 2025 , publisher =. doi:10.1080/0960085X.2025.2475962 , URL =

    work page doi:10.1080/0960085x.2025.2475962 2025

  30. [30]

    and Burton, Jason W

    Berger, J. and Burton, Jason W. and Hertwig, Ralph and Kosch, Thomas and Kurvers, R. and Kurzenberger, Benito and Lazik, Christopher and Onnasch, L. and Rieger, Tobias and Thoma, Anna I. and Wulff, Dirk U. and Herzog, Stefan M. , journal =. Fostering Human Learning Is Crucial for Boosting Human-

    work page

  31. [31]

    Vera and Tan, Chenhao , title =

    Lai, Vivian and Chen, Chacha and Smith-Renner, Alison and Liao, Q. Vera and Tan, Chenhao , title =. Proceedings of the 2023 ACM Conference on Fairness, Accountability, and Transparency , pages =. 2023 , isbn =. doi:10.1145/3593013.3594087 , abstract =

    work page doi:10.1145/3593013.3594087 2023

  32. [32]

    When Combinations of Humans and

    Vaccaro, Michelle and Almaatouq, Abdullah and Malone, Thomas , date =. When Combinations of Humans and. 2024 , journal =. doi:10.1038/s41562-024-02024-1 , url =

    work page doi:10.1038/s41562-024-02024-1 2024

  33. [33]

    Licklider, J. C. R. , journal=. Man-Computer Symbiosis , year=

    work page

  34. [34]

    and McCoy, Thomas H

    Jacobs, Maia and Pradier, Melanie F. and McCoy, Thomas H. and Perlis, Roy H. and Doshi-Velez, Finale and Gajos, Krzysztof Z. , date =. How Machine-Learning Recommendations Influence Clinician Treatment Selections: The Example of Antidepressant Selection , shorttitle =. 2021 , journal =. doi:10.1038/s41398-021-01224-x , url =

    work page doi:10.1038/s41398-021-01224-x 2021

  35. [35]

    1993 , publisher =

    Decision Making in Action: Models and Methods , shorttitle =. 1993 , publisher =

    work page 1993

  36. [36]

    and Raiffa, Howard and Tversky, Amos , editor =

    Bell, David E. and Raiffa, Howard and Tversky, Amos , editor =. Decision. 1988 , edition =. doi:10.1017/CBO9780511598951.003 , url =

    work page doi:10.1017/cbo9780511598951.003 1988

  37. [37]

    2024 , journal =

    Sarkar, Advait , date =. 2024 , journal =. doi:10.1145/3649404 , url =

    work page doi:10.1145/3649404 2024

  38. [38]

    Skov, and Jesper Kjeldskov

    Van Berkel, Niels and Skov, Mikael B. and Kjeldskov, Jesper , date =. Human-. 2021 , journal =. doi:10.1145/3486941 , url =

    work page doi:10.1145/3486941 2021

  39. [39]

    and Galatolo, Federico Andrea and La Mantia, Gaetano and Campisi, Giuseppina and Di Fede, Olga , date =

    Turchi, Tommaso and Mikhaylova, Daria and Troccoli, Miriana and Malizia, Alessio and Cimino, Mario Giovanni C.A. and Galatolo, Federico Andrea and La Mantia, Gaetano and Campisi, Giuseppina and Di Fede, Olga , date =. Ecological. Proceedings of the 16th. 2025 , pages =. doi:10.1145/3750069.3750072 , url =

    work page doi:10.1145/3750069.3750072 2025

  40. [40]

    Dimensions of

    Wilson, Ben and Natali, Chiara and Roach, Matt and Scott, Darren and Rahat, Alma and Rawlinson, David and Cabitza, Federico , date =. Dimensions of. 2025 , journal =. doi:10.1007/s10606-025-09514-4 , url =

    work page doi:10.1007/s10606-025-09514-4 2025