Pith. sign in

REVIEW 4 major objections 5 minor 42 references

Identifying Explanation Needs: Towards a Catalog of User-based Indicators

T0 review · 4 major / 5 minor · reviewed 2026-08-15 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read Software can recognize when a user needs an explanation by watching for 17 behavior patterns, 8 system events, and 14 emotional or physical reactions.

desk verdict A useful exploratory catalog of self-reported indicators, but the abstract overclaims runtime-trigger readiness since the study lacks any no-need baseline. read the letter →

arxiv 2506.16997 v1 pith:G3J4542A submitted 2025-06-20 cs.SE

Hannah Deters , Laura Reinhardt , Jakob Droste , Martin Obaidi , Kurt Schneider This is my paper · ORCID
classification cs.SE
keywords explainabilityexplanationneedsuser-basedindicatorsruntimedetectionuserbehaviorrequirementselicitationtelemetryself-reportedstudy
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper aims to give software a way to know when a user is confused and needs an explanation, without asking the user directly. From an online study in which 66 participants recalled moments of needing explanations in software they had recently used, the authors built a catalog of 17 behavior-based indicators, 8 system-event indicators, and 14 emotion or physical-reaction indicators. They also connect these indicators to types of explanation needs, showing for instance that back-and-forth navigation is associated with interaction, system behavior, and user-interface needs. If the indicators hold up in practice, systems could use them in prototypes, telemetry from deployed apps, and runtime triggers to deliver explanations at the moment they are wanted.

What carries the argument

The central object is a four-part taxonomy of runtime-detectable indicators derived from coded self-reports. It carries the argument by giving engineers a modular checklist: behavior-based indicators can be logged from interaction data, event-based indicators from system internals, and emotion and physical indicators would require additional sensors. The paper's contribution is not a fully validated detector but the catalog itself, offered as a foundation for choosing and implementing indicators in specific systems.

What would settle it

Record users interacting with a software system, log the proposed behavior-based indicators (back-and-forth navigation, canceled actions, click spamming, inactivity) together with system-event indicators (errors, loading times, design deviations), and interrupt users at random moments to ask whether they currently need an explanation; if the logged signals are no more frequent during self-reported need than at other times, the catalog fails.

Watch

Extended reading notes

Core claim

The central claim is that the need for an explanation leaves observable traces that can be captured at runtime, and that these traces can be organized into a reusable catalog. The study identifies 17 behavior-based indicators (such as back-and-forth navigation, canceled actions, repetitive actions, click spamming, and inactivity), 8 system-event indicators (such as system errors, loading times, and design deviations), and 14 indicators covering emotions and physical reactions, with facial expressions the most commonly reported physical sign and annoyance the most common emotion. The authors further report that workflow interruptions, especially back-and-forth navigation and canceled actions, are the most frequently described behavioral signals, and that privacy, security, and domain-knowledge needs are harder to detect from behavior alone.

Load-bearing premise

The catalog rests on the assumption that participants' retrospective self-reports accurately capture what they actually did and felt at the moment of needing an explanation, since the indicators are built entirely from those recollections.

Editorial extensions

If this is right

  • Requirements engineers can instrument high-fidelity prototypes to record the behavior indicators and identify where test users need explanations.
  • Deployed applications can collect telemetry matching the behavior-based indicators to locate explanation needs after release.
  • At runtime, source-code instrumentation can evaluate indicators and trigger explanations at appropriate moments.
  • The indicator-to-need-type mappings let engineers select targeted indicators, for example back-and-forth navigation for interaction or interface needs.
  • The catalog gives explainability research more objective, observation-based measures of whether a need for explanation has been met.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • A practical detector could likely be built from common product-analytics events alone, since session paths, clicks on non-interactive elements, and session duration already approximate several behavior-based indicators.
  • If these indicators are validated, the same signals could feed adaptive user interfaces that not only explain but also adjust navigation or undo support, because several indicators point to usability problems rather than explanation needs specifically.
  • A testable extension would combine behavior-based indicators with the reported emotions through facial-expression or biometric sensing; the paper lists the reactions, but the catalog gives a target list for training such detectors.
  • The frequency distribution suggests a practical starting point: workflow-interruption indicators appeared most often and are among the cheapest to log, so early adopters should begin with them.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. This paper reports an online survey study (N=66) in which participants recalled up to five explanation needs for each of three recently used software systems and described their behaviors, emotions, and physical reactions. The responses were coded into four taxonomies: 17 behavior-based, 8 event-based, 6 physical-reaction, and 8 emotional-state indicators. The authors also tabulate associations between these indicators and the need types of their earlier taxonomy (Droste et al.), and propose uses in requirements engineering, post-deployment telemetry, and runtime explanation triggering.

Significance. If validated, the catalog would be a practical aid for eliciting explanation requirements and for detecting explanation needs from usage data. The authors report interrater agreement (B&P kappa 0.59-0.91), share coding guidelines and pseudonymized data, and openly acknowledge several validity threats. The strength of the paper is its descriptive taxonomy: a systematic first step with transparent coding. However, the paper's central contribution as stated in the abstract—that the indicators can trigger explanations at runtime—is not supported by the study design, which relies entirely on retrospective self-reports and lacks a no-need baseline. The descriptive results are therefore valuable as hypotheses, not as validated detectors.

major comments (4)
  1. [Abstract and Section VI.C.d] The runtime-trigger claim in the abstract is contradicted by the conclusion-validity section. Section VI.C.d states 'we cannot draw any conclusions about the actual applicability of the indicators.' Since the abstract claims the indicators 'can be used to trigger explanations at appropriate moments during the runtime,' this is a load-bearing overclaim. Either the abstract and the 'Real-time Explanation Triggers' use case (Section VI.B.3) must be reworded to describe a future research goal, or a validation study must be added.
  2. [Section III.B and Section IV.A] The study collects only episodes in which a need was present (Section III.B, Q2.1-Q4.1); it never samples moments without explanation needs. Therefore the catalog cannot support claims about specificity or false-positive rates. For example, 'back-and-forth navigation' (44 reports) and 'click spamming' (15) are plausible in normal usage without any explanation need. Without a control condition, RQ1 'Which runtime indicators exist that signal a need for explanation' cannot be answered beyond identifying candidate correlates. Please add a baseline condition or explicitly restrict the claim to hypotheses.
  3. [Section V, Tables II and III] The mapping between indicators and need types in Section V (Tables II and III) is an internal association: the same participants reported both the need and the behavior in the same questionnaire, using prompts derived from the authors' own taxonomy [4]. This does not externally validate that the indicator causes or reliably accompanies the need; it may reflect post-hoc rationalization. The paper should present this as exploratory and state that external validation is required.
  4. [Sections III.C, III.E, and VI.C.b] Although framed as 'captured at runtime,' all indicators were elicited as retrospective free-text memories (Sections III.B, III.C, VI.C.b). No usage logs, clickstreams, sensor data, or system telemetry were recorded. Consequently, even the behavioral indicators are not shown to be detectable by software or sensors; physical reactions are explicitly described as 'not yet ready-to-use methods' (Section VI.B.1). Please make the gap between self-report and runtime measurability explicit in the central claims.
minor comments (5)
  1. [Section III.G] Typo: 'pseundonomized' should be 'pseudonymized'.
  2. [Section IV.D] Typo: 'beeing' should be 'being'.
  3. [Section IV.B] Typo: 'posessecurity' should be 'poses security'.
  4. [Figure 6 and Section IV.D] The figure labels 'Concern' with 14, but the text says 'The emotion of concern ... was indicated 30 times.' The number 30 appears in the figure under 'Confusion,' so the text and figure appear to disagree about which category has which count.
  5. [Section IV.C] Grammar: 'These indicators may not yet ready for direct use' should be 'may not yet be ready for direct use.'

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity: the catalog is a self-report survey result, and the runtime-trigger claim is explicitly unvalidated rather than derived from the inputs by construction.

full rationale

The paper makes no numerical predictions from fitted parameters and contains no equations whose output is equivalent to an input. The indicator catalog is built from participants' retrospective self-reports: participants recalled explanation needs and then described behaviors and feelings (Section III.B). Coding used the authors' prior taxonomy of explanation needs (Droste et al. [4]) for labeling the needs and associating indicators with need types. This self-citation is not circular in a load-bearing sense: the taxonomy is used to structure and categorize the data, not to prove that the indicators work. The claimed indicator-to-need associations are presented as observed frequencies in the same self-reported episodes, not as validated detectors. Moreover, the paper itself disclaims the runtime-trigger inference: Section VI.C.d states 'we cannot draw any conclusions about the actual applicability of the indicators,' and Section VI.B.1 states that physical reactions 'only provide rough directions, but not yet ready-to-use methods.' The abstract's stronger phrasing about triggering explanations at runtime is an aspirational framing that the paper's own validity section qualifies. The main methodological weakness—absence of a no-need baseline and reliance on recall—is a validity threat, not a circular reduction. No self-definitional, fitted-input, imported-uniqueness, or ansatz-smuggling pattern is present.

Assumptions & free parameters 0 free parameters · 4 assumptions · 0 invented entities

No numerical parameters are fitted; the catalog is qualitative. The main assumptions concern the reliability of self-reports, the validity of the authors' own need taxonomy, sample sufficiency, and the future feasibility of sensor-based detection. No new theoretical entities are introduced.

assumptions (4)
  • domain assumption Participants' retrospective self-reports of behavior and emotions accurately reflect their actual behavior and internal states at the moment an explanation need arose.
    The entire catalog is derived from these self-reports (Section III.E), and the authors themselves note recall and availability bias as threats (Section VI.C.b).
  • domain assumption The Droste et al. taxonomy of explanation needs is a valid and complete coding scheme.
    It is used as the basis for Round 1 labeling (Section III.E) and was developed by the same research group, so its validity is assumed rather than independently checked.
  • domain assumption A convenience sample of 66 adults is sufficient to identify the most prominent explanation-need indicators.
    External validity is limited; the authors state the sample likely did not capture all indicators (Section VI.C.c).
  • domain assumption Emotional states and physical reactions could be captured by additional devices such as cameras, microphones, or biometric sensors.
    The paper proposes these indicators for future research and notes they are not yet ready for direct use (Section IV.C).

how reviews work

0 comments
Cite this review

Pith. "Pith review of Identifying Explanation Needs: Towards a Catalog of User-based Indicators." pith.science (2026). https://pith.science/paper/G3J4542A

@misc{pith2026250616997,
  author       = {Pith},
  title        = {Pith review of: Identifying Explanation Needs: Towards a Catalog of User-based Indicators},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/G3J4542A}},
  note         = {Machine review of arXiv:2506.16997}
}
read the original abstract

In today's digitalized world, where software systems are becoming increasingly ubiquitous and complex, the quality aspect of explainability is gaining relevance. A major challenge in achieving adequate explanations is the elicitation of individual explanation needs, as it may be subject to severe hypothetical or confirmation biases. To address these challenges, we aim to establish user-based indicators concerning user behavior or system events that can be captured at runtime to determine when a need for explanations arises. In this work, we conducted explorative research in form of an online study to collect self-reported indicators that could indicate a need for explanation. We compiled a catalog containing 17 relevant indicators concerning user behavior, 8 indicators concerning system events and 14 indicators concerning emotional states or physical reactions. We also analyze the relationships between these indicators and different types of need for explanation. The established indicators can be used in the elicitation process through prototypes, as well as after publication to gather requirements from already deployed applications using telemetry and usage data. Moreover, these indicators can be used to trigger explanations at appropriate moments during the runtime.

Figures

Figures reproduced from arXiv: 2506.16997 by the authors.

Figure 1
Figure 1. Structure of the survey systems the participant used. Finally, demographic data such as age and field of work were collected (Q6). We provided supporting examples and information for the questions to enable users to answer the questions as accurately as possible. For example, when eliciting explanation needs, it has proven to be effective to use a taxonomy that the participant can use to navigate [4], [25]. For this… view at source ↗
Figure 2
Figure 2. Data analysis procedure 2www.maxqda.com TABLE I INTERRATER AGREEMENT VALUES DURING LABELING Questions Taxonomy B & P κ Round 1 Q2.1, Q3.1, Q4.1 Types of explanation needs [4] 0.68 Round 2 Q2.2, Q3.2, Q4.2 Behavior-based and event￾based indicators 0.61 Round 3 Q5 Behavior-based and event￾based indicators 0.59 Round 4 Q2.3, Q3.3, Q4.3 Emotional state and phys￾ical reaction indicators 0.91 Round 1: We used the explaina… view at source ↗
Figure 3
Figure 3. shows behavior-based indicators that were mentioned by participants in the survey. Usage Error. This category of indicators relates to incorrect user input and was reported nine times. The responses from the participants revealed that the need for explanation arises at different error rates depending on the use case. In some cases, the need for explanation occurred in real time when an erroneous input was entered (r… view at source ↗
Figures from the paper (3 more)
Figure 4
Figure 4. Figure 4: Event-based Indicators System Anomaly. This category includes unusual system behavior. Eleven participants stated that they wish to receive an explanation of what is happening during loading times. In addition, deviations from usual design principles or design changes …
Figure 5
Figure 5. Figure 5: Physical Reaction Indicators be used. Finally, specific eye movement patterns could also provide information about the need for explanation, which can be recorded using eye trackers. D. Emotional State The indicators for emotional state work in a similar way to those f…
Figure 6
Figure 6. Figure 6: Emotional State Indicators Negative Emotions. 143 responses from participants re￾ported negative emotions related to the need for explanation. The emotion of annoyance includes impatience or displea￾sure, which was mentioned 47 times. The emotion of feeling overwhelmed…

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

42 extracted references · 37 canonical work pages

  1. [4]

    Explanations in Everyday Software Systems: Towards a Taxonomy for Explainability Needs

    J. Droste, H. Deters, M. Obaidi, and K. Schneider, “Explanations in everyday software systems: Towards a taxonomy for explainability needs,”arXiv preprint arXiv:2404.16644, 2024

    work page Pith review arXiv 2024

  2. [1]

    Exploring explainability: a definition, a model, and a knowledge catalogue,

    L. Chazette, W. Brunotte, and T. Speith, “Exploring explainability: a definition, a model, and a knowledge catalogue,” in2021 IEEE 29th international requirements engineering conference (RE). IEEE, 2021, pp. 197–208

    work page 2021

  3. [2]

    Peeking inside the black-box: a survey on explainable artificial intelligence (xai),

    A. Adadi and M. Berrada, “Peeking inside the black-box: a survey on explainable artificial intelligence (xai),”IEEE access, vol. 6, pp. 52 138– 52 160, 2018

    work page 2018

  4. [3]

    How do ml practitioners perceive explainability? an interview study of practices and challenges,

    U.-e. Habiba, M. K. Habib, J. Bogner, J. Fritzsch, and S. Wagner, “How do ml practitioners perceive explainability? an interview study of practices and challenges,”Empirical Software Engineering, vol. 30, no. 1, p. 18, 2025

    work page 2025

  5. [5]

    Explanations on demand - a technique for eliciting the actual need for explanations,

    H. Deters, J. Droste, M. Fechner, and J. Kl ¨under, “Explanations on demand - a technique for eliciting the actual need for explanations,” in2023 IEEE 31st International Requirements Engineering Conference Workshops (REW), 2023, pp. 345–351

    work page 2023

  6. [6]

    Context, content, consent- how to design user-centered privacy explanations (s)

    W. Brunotte, J. Droste, and K. Schneider, “Context, content, consent- how to design user-centered privacy explanations (s).” inSEKE, 2023, pp. 86–89

    work page 2023

  7. [7]

    Privacy explanations–a means to end-user trust,

    W. Brunotte, A. Specht, L. Chazette, and K. Schneider, “Privacy explanations–a means to end-user trust,”Journal of Systems and Soft- ware, vol. 195, p. 111545, 2023

    work page 2023

  8. [8]

    Designing end-user personas for explainability requirements using mixed methods research,

    J. Droste, H. Deters, J. Puglisi, and J. Kl ¨under, “Designing end-user personas for explainability requirements using mixed methods research,” in2023 IEEE 31st International Requirements Engineering Conference Workshops (REW), 2023, pp. 129–135

    work page 2023

Show all 42 references
  1. [9]

    Modeling and evaluating per- sonas with software explainability requirements,

    H. Ramos, M. Fonseca, and L. Ponciano, “Modeling and evaluating per- sonas with software explainability requirements,” inHuman-Computer Interaction: 7th Iberoamerican Workshop, HCI-COLLAB 2021, Sao Paulo, Brazil, September 8–10, 2021, Proceedings 7. Springer, 2021, pp. 136–149

    2021

  2. [10]

    Personalized explanation in machine learning: A conceptualization,

    J. Schneider and J. Handali, “Personalized explanation in machine learning: A conceptualization,” inTwenty-Seventh European Conference on Information Systems (ECIS2019), 2019

    2019

  3. [11]

    Explanation in artificial intelligence: In- sights from the social sciences,

    T. Miller, “Explanation in artificial intelligence: In- sights from the social sciences,”Artificial Intelli- gence, vol. 267, pp. 1–38, 2019. [Online]. Available: https://www.sciencedirect.com/science/article/pii/S0004370218305988

    2019

  4. [12]

    The x factor: On the relationship between user experi- ence and explainability,

    H. Deters, J. Droste, A. Hess, V . Kl ¨os, K. Schneider, T. Speith, and A. V ogelsang, “The x factor: On the relationship between user experi- ence and explainability,” inProceedings of the 13th Nordic Conference on Human-Computer Interaction, 2024, pp. 1–12

    2024

  5. [13]

    A stressful explanation: The dual effect of explainable artificial intelligence in personal health manage- ment,

    M. Gr ¨uning, T. Wolf, and M. Trenz, “A stressful explanation: The dual effect of explainable artificial intelligence in personal health manage- ment,” inProceedings of the 57th Hawaii International Conference on System Sciences, 2024

    2024

  6. [14]

    A systematic review and taxonomy of expla- nations in decision support and recommender systems,

    I. Nunes and D. Jannach, “A systematic review and taxonomy of expla- nations in decision support and recommender systems,”User Modeling and User-Adapted Interaction, vol. 27, pp. 393–444, 2017

    2017

  7. [15]

    How much information? effects of transparency on trust in an algorithmic interface,

    R. F. Kizilcec, “How much information? effects of transparency on trust in an algorithmic interface,” inProceedings of the 2016 CHI Conference on Human Factors in Computing Systems, ser. CHI ’16. New York, NY , USA: Association for Computing Machinery, 2016, p. 2390–2395. [Onl...

    2016

  8. [16]

    Chapter 81 experimental evidence on the existence of hypothetical bias in value elicitation methods,

    G. W. Harrison and E. E. Rutstr ¨om, “Chapter 81 experimental evidence on the existence of hypothetical bias in value elicitation methods,” in Handbook of Experimental Economics Results, C. R. Plott and V . L. Smith, Eds. Amsterdam: Elsevier, 2008, vol. 1, pp. 752–767

    2008

  9. [17]

    Explainability as a non-functional requirement: challenges and recommendations,

    L. Chazette and K. Schneider, “Explainability as a non-functional requirement: challenges and recommendations,”Requirements Engineer- ing, vol. 25, no. 4, pp. 493–514, 2020

    2020

  10. [18]

    Explainability as a non-functional requirement,

    M. A. K ¨ohl, K. Baum, M. Langer, D. Oster, T. Speith, and D. Bohlender, “Explainability as a non-functional requirement,” in2019 IEEE 27th International Requirements Engineering Conference (RE). IEEE, 2019, pp. 363–368

    2019

  11. [19]

    Towards public understanding of software through mod- eling,

    R. Jongeling, “Towards public understanding of software through mod- eling,” inProceedings of the ACM/IEEE 27th International Conference on Model Driven Engineering Languages and Systems, 2024, pp. 665– 669

    2024

  12. [20]

    Explainable software systems: from requirements analysis to system evaluation,

    L. Chazette, W. Brunotte, and T. Speith, “Explainable software systems: from requirements analysis to system evaluation,”Requirements Engi- neering, vol. 27, no. 4, pp. 457–487, 2022

    2022

  13. [21]

    Explainable security,

    L. Vigano and D. Magazzeni, “Explainable security,” in2020 IEEE European Symposium on Security and Privacy Workshops (EuroS&PW). IEEE, 2020, pp. 293–300

    2020

  14. [22]

    Explain- ability as a requirement for hardware: Introducing explainable hardware (xhw),

    T. Speith, J. Speith, S. Becker, Y . Zou, A. Biega, and C. Paar, “Explain- ability as a requirement for hardware: Introducing explainable hardware (xhw),” in2024 IEEE 32nd International Requirements Engineering Conference (RE). IEEE, 2024, pp. 354–362

    2024

  15. [23]

    The role of a mental model in learning to operate a device,

    D. E. Kieras and S. Bovair, “The role of a mental model in learning to operate a device,”Cognitive science, vol. 8, no. 3, pp. 255–273, 1984

    1984

  16. [24]

    Mental models: concepts for human- computer interaction research,

    N. Staggers and A. F. Norcio, “Mental models: concepts for human- computer interaction research,”International Journal of Man-machine studies, vol. 38, no. 4, pp. 587–605, 1993

    1993

  17. [25]

    Automating explanation need management in app reviews: A case study from the navigation app industry,

    M. Obaidi, N. V oß, J. Droste, H. Deters, M. Herrmann, J. Fischbach, and K. Schneider, “Automating explanation need management in app reviews: A case study from the navigation app industry,”arXiv preprint arXiv:2501.08087, 2025

    2025 arXiv

  18. [26]

    On the pulse of requirements elicitation: Physiological triggers and explainability needs,

    H. Deters, J. Droste, and K. Schneider, “On the pulse of requirements elicitation: Physiological triggers and explainability needs,” inREFSQ Workshops. CEUR Workshop Proceedings, 2024

    2024

  19. [27]

    Emotions and perceived productivity of software developers at the workplace,

    D. Girardi, F. Lanubile, N. Novielli, and A. Serebrenik, “Emotions and perceived productivity of software developers at the workplace,”IEEE Transactions on Software Engineering, vol. 48, no. 9, pp. 3326–3341, 2022

    2022

  20. [28]

    Wearable- based affect recognition—a review,

    P. Schmidt, A. Reiss, R. D ¨urichen, and K. Van Laerhoven, “Wearable- based affect recognition—a review,”Sensors, vol. 19, no. 19, p. 4079, 2019

    2019

  21. [29]

    Requirements elicitation techniques: a systematic literature review based on the maturity of the techniques,

    C. Pacheco, I. Garc ´ıa, and M. Reyes, “Requirements elicitation techniques: a systematic literature review based on the maturity of the techniques,”IET Software, vol. 12, no. 4, p. 365–378, Aug. 2018. [Online]. Available: https://doi.org/10.1049/iet-sen.2017.0144

    2018

  22. [30]

    Successful requirement elicita- tion by combining requirement engineering techniques,

    D. Mishra, A. Mishra, and A. Yazici, “Successful requirement elicita- tion by combining requirement engineering techniques,” in2008 First International Conference on the Applications of Digital Information and Web Technologies (ICADIWT), 2008, pp. 258–263

    2008

  23. [31]

    Requirements elicitation using a combina- tion of prototypes and scenarios,

    M. Mannio and U. Nikula, “Requirements elicitation using a combina- tion of prototypes and scenarios,” inTelecom Business Research Center Lappeenranta Research Report 5, pp. 283–296

  24. [32]

    Prototypes as assets, not toys. why and how to extract knowledge from prototypes. (experience report),

    K. Schneider, “Prototypes as assets, not toys. why and how to extract knowledge from prototypes. (experience report),” inProceedings of IEEE 18th International Conference on Software Engineering, 1996, pp. 522–531

    1996

  25. [33]

    Software prototypes: Enhancing the quality of requirements engineering process,

    B. Suranto, “Software prototypes: Enhancing the quality of requirements engineering process,” in2015 International Symposium on Technology Management and Emerging Technologies (ISTMET), 2015, pp. 148–153

    2015

  26. [34]

    Supplementary material for research paper “identi- fying explanation needs: Towards a catalog of user-based indica- tors

    Anonymous, “Supplementary material for research paper “identi- fying explanation needs: Towards a catalog of user-based indica- tors”,” 2025, our supplementary material is available online at: https://figshare.com/s/3190fae7cce453011646

    2025

  27. [35]

    Supplementary Material -

    J. Droste, H. Deters, M. Obaidi, and K. Schneider, “Supplementary Material - ”Explanations in Everyday Software Systems: Towards a Taxonomy for Explainability Needs” (RE’24),” Mar. 2024

    2024

  28. [36]

    Coefficient kappa: Some uses, misuses, and alternatives,

    R. L. Brennan and D. J. Prediger, “Coefficient kappa: Some uses, misuses, and alternatives,” vol. 41, no. 3, pp. 687–699, 1981

    1981

  29. [37]

    A coefficient of agreement for nominal scales,

    J. Cohen, “A coefficient of agreement for nominal scales,”Educational and Psychological Measurement, vol. 20, no. 1, pp. 37–46, 1960

    1960

  30. [38]

    The measurement of observer agreement for categorical data,

    J. R. Landis and G. G. Koch, “The measurement of observer agreement for categorical data,”Biometrics, vol. 33, no. 1, pp. 159–174, 1977

    1977

  31. [39]

    A model relating pupil diameter to mental workload and lighting conditions,

    B. Pfleging, D. K. Fekety, A. Schmidt, and A. L. Kun, “A model relating pupil diameter to mental workload and lighting conditions,” in Proceedings of the 2016 CHI conference on human factors in computing systems, 2016, pp. 5776–5788

    2016

  32. [40]

    Comparison of blood volume pulse and skin conductance responses to mental and af- fective stimuli at different anatomical sites,

    A. Kushki, J. Fairley, S. Merja, G. King, and T. Chau, “Comparison of blood volume pulse and skin conductance responses to mental and af- fective stimuli at different anatomical sites,”Physiological measurement, vol. 32, no. 10, p. 1529, 2011

    2011

  33. [41]

    Do users’ explainability needs in software change with mood?

    M. Obaidi, J. Droste, H. Deters, M. Herrmann, J. Kl ¨under, and K. Schneider, “Do users’ explainability needs in software change with mood?” 2025. [Online]. Available: https://arxiv.org/abs/2502.06546

    2025 arXiv

  34. [42]

    Wohlin, P

    C. Wohlin, P. Runeson, M. H ¨ost, M. C. Ohlsson, B. Regnell, and A. Wessl´en,Experimentation in software engineering. Springer Science & Business Media, 2012

    2012

Pith tools

Reviewed August 15, 2026 · model on record in the stance chip above.