pith. sign in

arxiv: 2605.20206 · v1 · pith:73K5BYAJnew · submitted 2026-04-08 · 💻 cs.HC · cs.AI· cs.SE

PrivacyAkinator: Articulating Key Privacy Design Decisions by Answering LLM-Generated Multiple-choice Questions

Qiyu Li , Yuen Sum Wong , Yuen Kei Wong , Longxuan Yu , Haojian Jin This is my paper

Pith reviewed 2026-05-21 10:06 UTC · model grok-4.3

classification 💻 cs.HC cs.AIcs.SE
keywords privacy design decisionsLLM-generated questionsuser studyPrivacyAkinatorPRAMnovice developersdata flowsrisk assessment
0
0 comments X

The pith

PrivacyAkinator lets developers identify 47 percent more key privacy decisions in 73 percent less time than PRAM by answering LLM-generated questions.

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper aims to show that novice developers can articulate important privacy design decisions more effectively with an interactive tool than with NIST's expert-oriented Privacy Risk Assessment Methodology. PrivacyAkinator generates multiple-choice questions from a universal representation of privacy as data flows and stakeholder interactions, drawing on a design space mined from privacy news articles and using dynamic prioritization. A study with 24 participants found the approach surfaces 47 percent more decisions while cutting time by 73 percent. This matters because current privacy frameworks demand specialized knowledge that most software teams lack, leaving privacy considerations incomplete during early design stages. If the results hold, teams could embed privacy thinking into routine development without hiring separate experts for every project.

Core claim

PrivacyAkinator introduces three innovations: a universal privacy representation that abstracts design decisions into data flows and stakeholder interactions, a domain-aware design space mined from 10,000 privacy-related news articles, and a dynamic question-generation workflow that prioritizes relevant LLM-created multiple-choice questions. Developers using the tool articulated key privacy decisions more completely and quickly than with PRAM in observational and controlled studies.

What carries the argument

PrivacyAkinator's universal privacy representation that abstracts decisions into data flows and stakeholder interactions, combined with its dynamic LLM question-generation workflow.

If this is right

  • Novice developers can surface privacy design decisions without deep prior expertise in risk frameworks.
  • Early-stage privacy articulation becomes feasible inside ordinary development timelines rather than requiring separate expert reviews.
  • The gap between structured privacy methods and everyday coding practice narrows through automated question guidance.
  • Teams can document and revisit privacy choices in a structured yet lightweight format during iterative design.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • The same question-generation pattern could apply to other hard-to-articulate domains such as accessibility or security trade-offs if suitable news or documentation corpora exist.
  • Embedding the tool directly into IDEs or issue trackers might make privacy documentation a natural byproduct of feature work rather than a separate task.
  • Over-reliance on news-derived examples risks under-representing privacy concerns that appear first in technical standards or internal company data rather than public reporting.
  • Longitudinal use across multiple projects could reveal whether repeated exposure to the questions improves developers' unaided privacy reasoning over time.

Load-bearing premise

The LLM-generated questions and the universal privacy representation drawn from news articles accurately capture the privacy decisions that matter most in actual development work without missing key issues or introducing bias.

What would settle it

A follow-up study in which expert reviewers or post-release privacy audits identify important decisions that PrivacyAkinator users systematically overlooked, or where time savings disappear on larger codebases.

Figures

Figures reproduced from arXiv: 2605.20206 by Haojian Jin, Longxuan Yu, Qiyu Li, Yuen Kei Wong, Yuen Sum Wong.

Figure 1
Figure 1. Figure 1: PrivacyAkinator helps developers articulate key privacy design decisions by answering LLM-generated multiple-choice [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. Figure 2: Data Flow Diagram defines data actions that occur across the data lifecycle, indicates the key entities involved in the [PITH_FULL_IMAGE:figures/full_fig_p008_2.png] view at source ↗
Figure 3
Figure 3. Figure 3: Privacy Storyboard illustrates data actions across the data lifecycle with stakeholder roles. While more aligned with [PITH_FULL_IMAGE:figures/full_fig_p009_3.png] view at source ↗
Figure 4
Figure 4. Figure 4: Our final multi-layer graphical representation adopts a three-layer representation with data flow, stakeholder [PITH_FULL_IMAGE:figures/full_fig_p009_4.png] view at source ↗
Figure 5
Figure 5. Figure 5: We organize our domain-aware privacy design space by data actions and stakeholder interactions, categorizing privacy [PITH_FULL_IMAGE:figures/full_fig_p010_5.png] view at source ↗
Figure 6
Figure 6. Figure 6: PrivacyAkinator Design Goal Panel [PITH_FULL_IMAGE:figures/full_fig_p020_6.png] view at source ↗
Figure 7
Figure 7. Figure 7: PrivacyAkinator functional Requirement Panel [PITH_FULL_IMAGE:figures/full_fig_p020_7.png] view at source ↗
Figure 8
Figure 8. Figure 8: PrivacyAkinator Workflow & Questions Panel [PITH_FULL_IMAGE:figures/full_fig_p021_8.png] view at source ↗
Figure 9
Figure 9. Figure 9: PrivacyAkinator Access System Design Panel [PITH_FULL_IMAGE:figures/full_fig_p021_9.png] view at source ↗
Figure 10
Figure 10. Figure 10: PrivacyAkinator Generated Worksheet B Data Actions and Stakeholder Interactions [PITH_FULL_IMAGE:figures/full_fig_p022_10.png] view at source ↗
read the original abstract

NIST's Privacy Risk Assessment Methodology (PRAM) provides a structured framework for privacy experts to assess privacy risks. However, its complexity and reliance on expert knowledge make it difficult for novice developers to use effectively. This paper explores methods to lower these barriers. We first performed an observational study with 12 participants using PRAM in real-world scenarios, and found that novice developers struggled most with articulating privacy-related design decisions. We then developed PrivacyAkinator, an interactive tool that helps developers articulate key privacy decisions by answering LLM-generated multiple-choice questions. PrivacyAkinator introduces three innovations: a universal privacy representation that abstracts privacy-related design decisions into data flows and stakeholder interactions; a domain-aware design space mined from 10K privacy-related news articles; and a dynamic question-generation workflow to prioritize relevant questions. Our user study with 24 participants suggests that developers using PrivacyAkinator identified 47% more key decisions in 73% less time compared to PRAM.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit. Tearing a paper down is the easy half of reading it; the pith above is the substance, this is the friction.

Referee Report

2 major / 1 minor

Summary. The paper introduces PrivacyAkinator, an interactive tool that assists novice developers in articulating key privacy design decisions via LLM-generated multiple-choice questions. It builds on a universal privacy representation abstracting data flows and stakeholder interactions, mined from 10K privacy-related news articles, along with a dynamic question-generation workflow. An observational study with 12 participants using PRAM identified struggles with decision articulation; a subsequent user study with 24 participants reports that PrivacyAkinator users identified 47% more key decisions in 73% less time compared to PRAM.

Significance. If the quantitative results hold under rigorous evaluation, the work could meaningfully lower barriers to privacy-by-design practices for non-expert developers, an important gap in HCI and usable privacy. The combination of news-derived design space mining with LLM-driven dynamic questioning is a concrete technical contribution that could generalize to other decision-support domains. The initial observational study provides useful grounding for the tool's motivation.

major comments (2)
  1. User study (abstract and evaluation section): The headline claim of 47% more key decisions identified in 73% less time rests on the n=24 comparison, yet the manuscript supplies no information on how 'key decisions' were defined with objective, pre-registered criteria, no independent gold-standard inventory per scenario, no blinded adjudication, and no inter-rater reliability metric. This measurement choice is load-bearing for the central superiority claim and creates potential circularity because the LLM questions are generated from the same news-derived representation used to surface decisions.
  2. User study (abstract and evaluation section): No details are provided on statistical tests, task selection, scenario balancing, or controls for learning/order effects between the PrivacyAkinator and PRAM arms. Without these, the reported time and decision-count differences cannot be confidently attributed to the tool rather than confounds.
minor comments (1)
  1. The description of the dynamic question-generation workflow would benefit from an explicit algorithm or pseudocode listing the prioritization steps.

Simulated Author's Rebuttal

2 responses · 0 unresolved

We thank the referee for their constructive feedback. We address each major comment below and will revise the manuscript to incorporate additional methodological details where appropriate.

read point-by-point responses

  1. Referee: User study (abstract and evaluation section): The headline claim of 47% more key decisions identified in 73% less time rests on the n=24 comparison, yet the manuscript supplies no information on how 'key decisions' were defined with objective, pre-registered criteria, no independent gold-standard inventory per scenario, no blinded adjudication, and no inter-rater reliability metric. This measurement choice is load-bearing for the central superiority claim and creates potential circularity because the LLM questions are generated from the same news-derived representation used to surface decisions.

    Authors: We acknowledge that the current manuscript lacks sufficient detail on the decision identification process. In the revision we will expand the evaluation section to define 'key decisions' as those mapping directly to elements of the universal privacy representation (data flows and stakeholder interactions) derived from the news-mined design space. The criteria were developed from the preceding observational study and applied consistently to both conditions. We did not use a separate gold-standard inventory or blinded adjudication; decisions were articulated by participants and then mapped to the representation by the research team. We will report inter-rater reliability if multiple coders performed the mapping. Regarding circularity, the shared representation was deliberately chosen to provide an objective, comparable basis for counting decisions across PrivacyAkinator and PRAM rather than introducing subjective judgment; we will clarify this rationale. revision: yes

  2. Referee: User study (abstract and evaluation section): No details are provided on statistical tests, task selection, scenario balancing, or controls for learning/order effects between the PrivacyAkinator and PRAM arms. Without these, the reported time and decision-count differences cannot be confidently attributed to the tool rather than confounds.

    Authors: We agree these details are necessary. The revised manuscript will describe the statistical tests applied to the decision counts and completion times, the process for selecting and balancing the privacy scenarios across conditions, and the measures taken to control for order and learning effects (including counterbalancing of tool order). These additions will strengthen the attribution of the observed differences to the tool. revision: yes

Circularity Check

0 steps flagged

No significant circularity; empirical user study benchmarks against external PRAM

full rationale

The paper reports an observational study (n=12) identifying novice struggles with PRAM, followed by development of PrivacyAkinator using a news-derived universal privacy representation and LLM question generation, then a comparative user study (n=24) measuring time and number of articulated decisions against the external NIST PRAM baseline. No equations, parameter fitting, self-definitional loops, or load-bearing self-citations appear. The success metric (decisions identified) is participant-reported and compared to an independent method rather than defined by the tool's representation. This qualifies as self-contained against external benchmarks with no reduction of claims to inputs by construction.

Axiom & Free-Parameter Ledger

0 free parameters · 0 axioms · 0 invented entities

No free parameters, mathematical axioms, or invented physical entities are introduced; the work rests on the empirical validity of the observational study findings and the assumption that the mined design space and LLM outputs align with real privacy risks.

pith-pipeline@v0.9.0 · 5718 in / 1100 out tokens · 58488 ms · 2026-05-21T10:06:40.499563+00:00 · methodology

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

122 extracted references · 122 canonical work pages · 1 internal anchor

  1. [1]

    Saeema Ahmed, Ken M Wallace, and Lucienne T Blessing. 2003. Understanding the differences between how novice and experienced designers approach design tasks.Research in engineering design14, 1 (2003), 1–11. doi:10.1007/s00163-002- 0023-z

    work page doi:10.1007/s00163-002- 2003

  2. [2]

    Abdulrahman Alhazmi and Nalin Asanka Gamagedara Arachchilage. 2021. I’m all ears! Listening to software developers on putting GDPR principles into software development practice.Personal and Ubiquitous Computing25, 5 (2021), 879–892. doi:10.1007/s00779-021-01544-1

    work page doi:10.1007/s00779-021-01544-1 2021

  3. [3]

    Nada Alhirabi, Stephanie Beaumont, Jose Tomas Llanos, Dulani Meedeniya, Omer Rana, and Charith Perera. 2023. PARROT: Interactive Privacy-Aware Internet of Things Application Design Tool.Proc. ACM Interact. Mob. Wearable Ubiquitous Technol.7, 1, Article 1 (March 2023), 37 pages. doi:10.1145/3580880

    work page doi:10.1145/3580880 2023

  4. [4]

    Majed Alshammari and Andrew Simpson. 2018. Towards an effective privacy impact and risk assessment methodology: risk assessment. InInternational Conference on Trust and Privacy in Digital Business. Springer, 85–99. doi:10.1007/ 978-3-319-98385-1_7

    work page 2018

  5. [5]

    Jenna Amatulli. 2020. Zoom can track who’s not paying attention in your video call. here’s how. https://www.huffpost.com/entry/zoom-tracks-not-paying- attention-video-call_l_5e7b96b5c5b6b7d80959ea96. Accessed on 05/10/2025

    work page 2020

  6. [6]

    Waleed Ammar, Shomir Wilson, Norman Sadeh, and Noah A Smith. 2012. Au- tomatic categorization of privacy policies: A pilot study.School of Computer Science, Language Technology Institute, Technical Report CMU-LTI-12-019(2012)

    work page 2012

  7. [7]

    Oshrat Ayalon, Eran Toch, Irit Hadar, and Michael Birnhack. 2017. How Devel- opers Make Design Decisions about Users’ Privacy: The Place of Professional Communities and Organizational Climate. InCompanion of the 2017 ACM Con- ference on Computer Supported Cooperative Work and Social Computing(Portland, Oregon, USA)(CSCW ’17 Companion). Association for Com...

    work page doi:10.1145/3022198.3026326 2017

  8. [8]

    Ero Balsa. 2023. Technocracy, pseudoscience and performative compliance: the risks of privacy risk assessments. Lessons from NIST’s Privacy Risk Assessment Methodology. arXiv:2310.05936 [cs.CR] https://arxiv.org/abs/2310.05936

    work page arXiv 2023

  9. [9]

    Vinayshekhar Bannihatti Kumar, Roger Iyengar, Namita Nisal, Yuanyuan Feng, Hana Habib, Peter Story, Sushain Cherivirala, Margaret Hagan, Lorrie Cranor, Shomir Wilson, Florian Schaub, and Norman Sadeh. 2020. Finding a Choice in a Haystack: Automatic Extraction of Opt-Out Statements from Privacy Policy Text. InProceedings of The Web Conference 2020(Taipei, ...

    work page arXiv 2020

  10. [10]

    Mitra Bokaei Hosseini, Rocky Slavin, Travis Breaux, Xiaoyin Wang, and Jianwei Niu. 2020. Disambiguating requirements through syntax-driven semantic anal- ysis of information types. InRequirements Engineering: Foundation for Software Quality: 26th International Working Conference, REFSQ 2020, Pisa, Italy, March 24–27, 2020, Proceedings 26. Springer, 97–115...

    work page doi:10.1007/978-3-030-44429-7_7 2020

  11. [11]

    John D Bransford, Ann L Brown, and Rodney R Cocking. 2000. How experts differ from novices.How people learn: Brain, mind, experience, and school(2000), 31–50. doi:10.17226/9853

    work page doi:10.17226/9853 2000

  12. [12]

    Sean Brooks, Michael Garcia, Naomi Lefkovitz, Suzanne Lightman, and Ellen Nadeau. 2017. An Introduction to Privacy Engineering and Risk Management in Federal Information Systems. doi:10.6028/NIST.IR.8062

    work page doi:10.6028/nist.ir.8062 2017

  13. [13]

    Lucas Brutschy, Pietro Ferrara, and Peter Müller. 2014. Static analysis for independent app developers.SIGPLAN Not.49, 10 (Oct. 2014), 847–860. doi:10. 1145/2714064.2660219

    work page arXiv 2014

  14. [14]

    Jean-Marie Burkhardt, Françoise Détienne, and Susan Wiedenbeck. 2002. Object- oriented program comprehension: Effect of expertise, task and phase.Empirical Software Engineering7, 2 (2002), 115–156. doi:10.1023/A:1015297914742

    work page doi:10.1023/a:1015297914742 2002

  15. [15]

    Nicholas Carlson. 2010. Warning: Google buzz has a huge privacy flaw-business insider.Retrieved December18 (2010), 2018

    work page 2010

  16. [16]

    Yu, Qiang Yang, and Xing Xie

    Yupeng Chang, Xu Wang, Jindong Wang, Yuan Wu, Linyi Yang, Kaijie Zhu, Hao Chen, Xiaoyuan Yi, Cunxiang Wang, Yidong Wang, Wei Ye, Yue Zhang, Yi Chang, Philip S. Yu, Qiang Yang, and Xing Xie. 2024. A Survey on Evaluation of Large Language Models.ACM Trans. Intell. Syst. Technol.15, 3, Article 39 (March 2024), 45 pages. doi:10.1145/3641289

    work page doi:10.1145/3641289 2024

  17. [17]

    Chaoran Chen, Daodao Zhou, Yanfang Ye, Toby Jia-Jun Li, and Yaxing Yao. 2025. CLEAR: Towards Contextual LLM-Empowered Privacy Policy Analysis and Risk Generation for Large Language Model Applications. InProceedings of the 30th International Conference on Intelligent User Interfaces (IUI ’25). Association for Computing Machinery, New York, NY, USA, 277–297...

    work page doi:10.1145/3708359 2025

  18. [18]

    Amit Chowdhry. 2016. Uber: Users are more likely to pay surge pricing if their phone battery is low

    work page 2016

  19. [19]

    Roger Clarke. 2009. Privacy impact assessment: Its origins and development. Computer Law & Security Review25, 2 (2009), 123–135. doi:10.1016/j.clsr.2009. 02.002

    work page doi:10.1016/j.clsr.2009 2009

  20. [20]

    Nancy J. Cooke. 1994. Varieties of knowledge elicitation techniques.Interna- tional Journal of Human-Computer Studies41, 6 (1994), 801–849. doi:10.1006/ ijhc.1994.1083

    work page arXiv 1994

  21. [21]

    Jason Cronk and Stuart S

    R. Jason Cronk and Stuart S. Shapiro. 2021. Quantitative Privacy Risk Analysis. In 2021 IEEE European Symposium on Security and Privacy Workshops (EuroS&PW). 340–350. doi:10.1109/EuroSPW54576.2021.00043

    work page doi:10.1109/eurospw54576.2021.00043 2021

  22. [22]

    Hao Cui, Rahmadi Trimananda, Athina Markopoulou, and Scott Jordan. 2023. PoliGraph: Automated Privacy Policy Analysis using Knowledge Graphs. In32nd USENIX Security Symposium (USENIX Security 23). USENIX Association, Ana- heim, CA, 1037–1054. https://www.usenix.org/conference/usenixsecurity23/ presentation/cui

    work page 2023

  23. [23]

    Sourya Joyee De and Daniel Le Métayer. 2016. PRIAM: a privacy risk anal- ysis methodology. InData Privacy Management and Security Assurance: 11th International Workshop, DPM 2016 and 5th International Workshop, QASA 2016, Heraklion, Crete, Greece, September 26-27, 2016, Proceedings 11. Springer, 221–229. doi:10.1007/978-3-319-47072-6_15

    work page doi:10.1007/978-3-319-47072-6_15 2016

  24. [24]

    Mina Deng, Kim Wuyts, Riccardo Scandariato, Bart Preneel, and Wouter Joosen

    work page

  25. [25]

    doi:10.1007/s00766-010-0115-7

    A privacy threat analysis framework: supporting the elicitation and fulfillment of privacy requirements.Requirements Engineering16, 1 (2011), 3–32. doi:10.1007/s00766-010-0115-7

    work page doi:10.1007/s00766-010-0115-7 2011

  26. [26]

    Thomas Dohmke, Marco Iansiti, and Greg Richards. 2023. Sea Change in Soft- ware Development: Economic and Productivity Analysis of the AI-Powered De- veloper Lifecycle. arXiv:2306.15033 [econ.GN] https://arxiv.org/abs/2306.15033

    work page arXiv 2023

  27. [27]

    W Dou, DH Jeong, F Stukes, W Ribarsky, HR Lipford, and R Chang. 2009. Comparing Usage Patterns of Domain Experts and Novices in Visual Analytical Tasks. InSensemaking Workshop

    work page 2009

  28. [28]

    Moemen Ebrahim, Shawkat Guirguis, and Christine Basta. 2025. Enhancing Software Requirements Engineering with Language Models and Prompting Tech- niques: Insights from the Current Research and Future Directions. InProceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Vol- ume 4: Student Research Workshop). Association fo...

    work page doi:10.18653/v1/2025.acl-srw.31 2025

  29. [29]

    Bernstein

    Ethan Fast, William McGrath, Pranav Rajpurkar, and Michael S. Bernstein. 2016. Augur: Mining Human Behaviors from Fiction to Power Interactive Systems. In Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems (San Jose, California, USA)(CHI ’16). Association for Computing Machinery, New York, NY, USA, 237–247. doi:10.1145/2858036.2858528

    work page doi:10.1145/2858036.2858528 2016

  30. [30]

    Yuanyuan Feng, Abhilasha Ravichander, Yaxing Yao, Shikun Zhang, and Rex Chen. 2024. Understanding How to Inform Blind and Low-Vision Users about Data Privacy through Privacy Question Answering Assistants. In33rd USENIX Security Symposium (USENIX Security 24). USENIX Association, Philadelphia, PA, 2065–2082. https://www.usenix.org/conference/usenixsecurity...

    work page 2024

  31. [31]

    Yuanyuan Feng, Yaxing Yao, and Norman Sadeh. 2021. A Design Space for Privacy Choices: Towards Meaningful Privacy Control in the Internet of Things. InProceedings of the 2021 CHI Conference on Human Factors in Computing Systems (Yokohama, Japan)(CHI ’21). Association for Computing Machinery, New York, NY, USA, Article 64, 16 pages. doi:10.1145/3411764.3445148

    work page doi:10.1145/3411764.3445148 2021

  32. [32]

    David Flaherty. 2000. Privacy impact assessments: an essential tool for data pro- tection.Privacy Law & Policy Reporter5 (2000), 85. doi:au/journals/PrivLawPRpr/ 2000/45.html

    work page 2000

  33. [33]

    You Don’t Need a University Degree to Comprehend Data Protection This Way

    Vincent Freiberger, Arthur Fleig, and Erik Buchmann. 2025. "You Don’t Need a University Degree to Comprehend Data Protection This Way": LLM-Powered Interactive Privacy Policy Assessment. InProceedings of the Extended Abstracts of the CHI Conference on Human Factors in Computing Systems (CHI EA ’25). Association for Computing Machinery, New York, NY, USA, ...

    work page doi:10.1145/3706599.3719816 2025

  34. [34]

    Xun Ge, Ching-Huei Chen, and Kendrick A. Davis. 2005. Scaffolding Novice Instructional Designers’ Problem-Solving Processes Using Question Prompts in CHI ’26, April 13–17, 2026, Barcelona, Spain Li et al. a Web-Based Learning Environment.Journal of Educational Computing Research 33, 2 (2005), 219–248. doi:10.2190/5F6J-HHVF-2U2B-8T3G

    work page doi:10.2190/5f6j-hhvf-2u2b-8t3g 2005

  35. [35]

    The Guardian. 2025. Privacy. https://www.theguardian.com/world/privacy. Accessed on 05/10/2025

    work page 2025

  36. [36]

    Komal Gupta and Aditya Shrivastava. 2025. Zero Data Retention in LLM-based Enterprise AI Assistants: A Comparative Study of Market Leading Agentic AI Products. arXiv:2510.11558 [cs.AI] https://arxiv.org/abs/2510.11558

    work page arXiv 2025

  37. [37]

    Seda Gürses, Carmela Troncoso, and Claudia Diaz. 2011. Engineering privacy by design.Computers, Privacy & Data Protection14, 3 (2011), 25

    work page 2011

  38. [38]

    Irit Hadar, Tomer Hasson, Oshrat Ayalon, Eran Toch, Michael Birnhack, Sofia Sherman, and Arod Balissa. 2018. Privacy by designers: software developers’ privacy mindset.Empirical Software Engineering23, 1 (2018), 259–289. doi:10. 1007/s10664-017-9517-1

    work page 2018

  39. [39]

    Shin, and Karl Aberer

    Hamza Harkous, Kassem Fawaz, Rémi Lebret, Florian Schaub, Kang G. Shin, and Karl Aberer. 2018. Polisis: Automated Analysis and Presentation of Privacy Policies Using Deep Learning. In27th USENIX Security Symposium (USENIX Security 18). USENIX Association, Baltimore, MD, 531–548. https://www.usenix. org/conference/usenixsecurity18/presentation/harkous

    work page 2018

  40. [40]

    Shin, and Karl Aberer

    Hamza Harkous, Kassem Fawaz, Kang G. Shin, and Karl Aberer. 2016. PriBots: Conversational Privacy with Chatbots. InTwelfth Symposium on Usable Privacy and Security (SOUPS 2016). USENIX Association, Denver, CO. https://www.usenix.org/conference/soups2016/workshop-program/wfpn/ presentation/harkous

    work page 2016

  41. [41]

    Hart and Lowell E

    Sandra G. Hart and Lowell E. Staveland. 1988. Development of NASA-TLX (Task Load Index): Results of Empirical and Theoretical Research. InHuman Mental Workload, Peter A. Hancock and Najmedin Meshkati (Eds.). Advances in Psychology, Vol. 52. North-Holland, 139–183. doi:10.1016/S0166-4115(08)62386- 9

    work page doi:10.1016/s0166-4115(08)62386- 1988

  42. [42]

    Kashmir Hill. 2024. How target figured out a teen girl was pregnant before her father did. https://www.forbes.com/sites/kashmirhill/2012/02/16/how-target- figured-out-a-teen-girl-was-pregnant-before-her-father-did/. Accessed on 05/10/2025

    work page 2024

  43. [43]

    James Hollan, Edwin Hutchins, and David Kirsh. 2000. Distributed cognition: toward a new foundation for human-computer interaction research.ACM Trans. Comput.-Hum. Interact.7, 2 (June 2000), 174–196. doi:10.1145/353485.353487

    work page doi:10.1145/353485.353487 2000

  44. [44]

    Breaux, Rocky Slavin, Jianwei Niu, and Xiaoyin Wang

    Mitra Bokaei Hosseini, Travis D. Breaux, Rocky Slavin, Jianwei Niu, and Xiaoyin Wang. 2021. Analyzing privacy policies through syntax-driven semantic analysis of information types.Information and Software Technology138 (2021), 106608. doi:10.1016/j.infsof.2021.106608

    work page doi:10.1016/j.infsof.2021.106608 2021

  45. [45]

    Mitra Bokaei Hosseini, John Heaps, Rocky Slavin, Jianwei Niu, and Travis Breaux. 2021. Ambiguity and Generality in Natural Language Privacy Policies. In2021 IEEE 29th International Requirements Engineering Conference (RE). 70–81. doi:10.1109/RE51729.2021.00014

    work page doi:10.1109/re51729.2021.00014 2021

  46. [46]

    Cory Hymel and Hiroe Johnson. 2025. Analysis of LLMs vs Human Experts in Requirements Engineering. arXiv:2501.19297 [cs.SE] https://arxiv.org/abs/2501. 19297

    work page arXiv 2025

  47. [47]

    Leonardo Horn Iwaya, Ala Sarah Alaqra, Marit Hansen, and Simone Fischer- Hübner. 2024. Privacy impact assessments in the wild: A scoping review.Array 23 (2024), 100356. doi:10.1016/j.array.2024.100356

    work page doi:10.1016/j.array.2024.100356 2024

  48. [48]

    Haojian Jin, Hong Shen, Mayank Jain, Swarun Kumar, and Jason I. Hong. 2021. Lean Privacy Review: Collecting Users’ Privacy Concerns of Data Practices at a Low Cost.ACM Trans. Comput.-Hum. Interact.28, 5, Article 34 (Aug. 2021), 55 pages. doi:10.1145/3463910

    work page doi:10.1145/3463910 2021

  49. [49]

    Samantha Katcher, Ben Ballard, Cara Bloom, Katie Isaacson, Julie McEwen, Stuart Shapiro, Shelby Slotter, Mark Paes, and Ryan Xu. 2024. The PANOPTIC™ Privacy Threat Model. InTwentieth Symposium on Usable Privacy and Security (SOUPS)

    work page 2024

  50. [50]

    alien traces

    Dilara Keküllüoğlu and Yasemin Acar. 2023. "We are a startup to the core": A qualitative interview study on the security and privacy development practices in Turkish software startups. In2023 IEEE Symposium on Security and Privacy (SP). 2015–2031. doi:10.1109/SP46215.2023.10179339

    work page doi:10.1109/sp46215.2023.10179339 2023

  51. [51]

    Patrick Gage Kelley, Joanna Bresee, Lorrie Faith Cranor, and Robert W. Reeder

    work page

  52. [52]

    nutrition label

    A "nutrition label" for privacy. InProceedings of the 5th Symposium on Usable Privacy and Security(Mountain View, California, USA)(SOUPS ’09). Association for Computing Machinery, New York, NY, USA, Article 4, 12 pages. doi:10.1145/1572532.1572538

    work page doi:10.1145/1572532.1572538

  53. [53]

    Patrick Gage Kelley, Lucian Cesca, Joanna Bresee, and Lorrie Faith Cranor. 2010. Standardizing privacy notices: an online study of the nutrition label approach. InProceedings of the SIGCHI Conference on Human Factors in Computing Systems (Atlanta, Georgia, USA)(CHI ’10). Association for Computing Machinery, New York, NY, USA, 1573–1582. doi:10.1145/175332...

    work page doi:10.1145/1753326.1753561 2010

  54. [54]

    Shaymaa Mamdouh Khalil, Hayretdin Bahsi, and Tarmo Korõtko. 2024. Threat modeling of industrial control systems: A systematic literature review.Comput- ers & Security136 (2024), 103543. doi:10.1016/j.cose.2023.103543

    work page doi:10.1016/j.cose.2023.103543 2024

  55. [55]

    Jeong-Dong Kim, Jiseong Son, and Doo-Kwon Baik. 2012. CA5W1HOnto: On- tological Context-Aware Model Based on 5W1H.International Journal of Dis- tributed Sensor Networks8, 3 (2012), 247346. doi:10.1155/2012/247346

    work page doi:10.1155/2012/247346 2012

  56. [56]

    Konrad Kollnig, Anastasia Shuba, Max Van Kleek, Reuben Binns, and Nigel Shadbolt. 2022. Goodbye Tracking? Impact of iOS App Tracking Trans- parency and Privacy Labels. InProceedings of the 2022 ACM Conference on Fairness, Accountability, and Transparency(Seoul, Republic of Korea)(FAccT ’22). Association for Computing Machinery, New York, NY, USA, 508–520....

    work page doi:10.1145/3531146.3533116 2022

  57. [57]

    Madhava Krishna, Bhagesh Gaur, Arsh Verma, and Pankaj Jalote. 2024. Using LLMs in Software Requirements Specifications: An Empirical Evaluation. In 2024 IEEE 32nd International Requirements Engineering Conference (RE). 475–483. doi:10.1109/RE59067.2024.00056

    work page doi:10.1109/re59067.2024.00056 2024

  58. [58]

    Naomi Lefkovitz. 2020. NIST Privacy Framework: The Implementation Chal- lenges. https://www.bankinfosecurity.com/interviews/nist-privacy-framework- implementation-challenges-i-4594. (Accessed: 2025-05-23)

    work page 2020

  59. [59]

    2009.Data Flow Diagram

    Qing Li and Yu-Liu Chen. 2009.Data Flow Diagram. Springer Berlin Heidelberg, Berlin, Heidelberg, 85–97. doi:10.1007/978-3-540-89556-5_4

    work page doi:10.1007/978-3-540-89556-5_4 2009

  60. [60]

    Tianshi Li, Yuvraj Agarwal, and Jason I. Hong. 2018. Coconut: An IDE Plugin for Developing Privacy-Friendly Apps.Proc. ACM Interact. Mob. Wearable Ubiquitous Technol.2, 4, Article 178 (Dec. 2018), 35 pages. doi:10.1145/3287056

    work page doi:10.1145/3287056 2018

  61. [61]

    Tianshi Li, Lorrie Faith Cranor, Yuvraj Agarwal, and Jason I. Hong. 2024. Matcha: An IDE Plugin for Creating Accurate Privacy Nutrition Labels.Proc. ACM Interact. Mob. Wearable Ubiquitous Technol.8, 1, Article 33 (March 2024), 38 pages. doi:10.1145/3643544

    work page doi:10.1145/3643544 2024

  62. [62]

    Tianshi Li, Elizabeth Louie, Laura Dabbish, and Jason I. Hong. 2021. How Developers Talk About Personal Data and What It Means for User Privacy: A Case Study of a Developer Forum on Reddit.Proc. ACM Hum.-Comput. Interact. 4, CSCW3, Article 220 (Jan. 2021), 28 pages. doi:10.1145/3432919

    work page doi:10.1145/3432919 2021

  63. [63]

    Tianshi Li, Kayla Reiman, Yuvraj Agarwal, Lorrie Faith Cranor, and Jason I. Hong. 2022. Understanding Challenges for Developers to Create Accurate Privacy Nutrition Labels. InProceedings of the 2022 CHI Conference on Human Factors in Computing Systems(New Orleans, LA, USA)(CHI ’22). Association for Computing Machinery, New York, NY, USA, Article 588, 24 p...

    work page arXiv 2022

  64. [64]

    Tony W Li, Arshia Arya, and Haojian Jin. 2024. Redesigning Privacy with User Feedback: The Case of Zoom Attendee Attention Tracking. InProceedings of the 2024 CHI Conference on Human Factors in Computing Systems(Honolulu, HI, USA)(CHI ’24). Association for Computing Machinery, New York, NY, USA, Article 237, 14 pages. doi:10.1145/3613904.3642594

    work page doi:10.1145/3613904.3642594 2024

  65. [65]

    Newman, Jason I

    James Lin, Mark W. Newman, Jason I. Hong, and James A. Landay. 2000. DENIM: finding a tighter fit between tools and practice for Web site design. InProceedings of the SIGCHI Conference on Human Factors in Computing Systems(The Hague, The Netherlands)(CHI ’00). Association for Computing Machinery, New York, NY, USA, 510–517. doi:10.1145/332040.332486

    work page doi:10.1145/332040.332486 2000

  66. [66]

    Fei Liu, Rohan Ramanath, Norman Sadeh, and Noah A. Smith. 2014. A Step To- wards Usable Privacy Policy: Automatic Alignment of Privacy Statements. InPro- ceedings of COLING 2014, the 25th International Conference on Computational Lin- guistics: Technical Papers. Dublin City University and Association for Computa- tional Linguistics, Dublin, Ireland, 884–8...

    work page 2014

  67. [67]

    Yinhan Liu, Myle Ott, Naman Goyal, Jingfei Du, Mandar Joshi, Danqi Chen, Omer Levy, Mike Lewis, Luke Zettlemoyer, and Veselin Stoyanov. 2019. RoBERTa: A Robustly Optimized BERT Pretraining Approach. arXiv:1907.11692 [cs.CL] https://arxiv.org/abs/1907.11692

    work page internal anchor Pith review Pith/arXiv arXiv 2019

  68. [68]

    Jesus Luna, Neeraj Suri, and Ioannis Krontiris. 2012. Privacy-by-design based on quantitative threat modeling. In2012 7th International Conference on Risks and Security of Internet and Systems (CRiSIS). 1–8. doi:10.1109/CRISIS.2012.6378941

    work page doi:10.1109/crisis.2012.6378941 2012

  69. [69]

    Stephen MacNeil, Zijian Ding, Kexin Quan, Thomas j Parashos, Yajie Sun, and Steven P. Dow. 2021. Framing Creative Work: Helping Novices Frame Better Problems through Interactive Scaffolding. InProceedings of the 13th Conference on Creativity and Cognition(Virtual Event, Italy)(C&C ’21). Association for Computing Machinery, New York, NY, USA, Article 30, 1...

    work page arXiv 2021

  70. [70]

    Giles Turner Matt Day and Natalia Drozdiak / Bloomberg. 2019. Thousands of Amazon workers listen to Alexa conversations. https://time.com/5568815/ amazon-workers-listen-to-alexa/. Accessed on 05/10/2025

    work page arXiv 2019

  71. [71]

    Mary L McHugh. 2012. Interrater reliability: the kappa statistic.Biochemia medica22, 3 (2012), 276–282

    work page 2012

  72. [72]

    McKnight and Julius Najab

    Patrick E. McKnight and Julius Najab. 2010.Mann-Whitney U Test. John Wiley & Sons, Ltd, 1–1. doi:10.1002/9780470479216.corpsy0524

    work page doi:10.1002/9780470479216.corpsy0524 2010

  73. [73]

    Liz Mineo. 2025. What happens to your data if 23andMe col- lapses? https://news.harvard.edu/gazette/story/2025/03/what-happens-to- your-genetic-data-if-23andme-collapses/. Accessed on 06/01/2025

    work page 2025

  74. [74]

    Yair Neuman, Dan Assaf, Yohai Cohen, Mark Last, Shlomo Argamon, New- ton Howard, and Ophir Frieder. 2013. Metaphor Identification in Large Texts Corpora.PLOS ONE8, 4 (04 2013), 1–9. doi:10.1371/journal.pone.0062343

    work page doi:10.1371/journal.pone.0062343 2013

  75. [75]

    Ngoon, C

    Tricia J. Ngoon, C. Ailie Fraser, Ariel S. Weingarten, Mira Dontcheva, and Scott Klemmer. 2018. Interactive Guidance Techniques for Improving Creative Feed- back. InProceedings of the 2018 CHI Conference on Human Factors in Computing Systems(Montreal QC, Canada)(CHI ’18). Association for Computing Machinery, New York, NY, USA, 1–11. doi:10.1145/3173574.31...

    work page doi:10.1145/3173574.3173629 2018

  76. [76]

    Helen Nissenbaum. 2004. Privacy as contextual integrity.Wash. L. Rev.79 (2004),

    work page 2004

  77. [77]

    https://digitalcommons.law.uw.edu/wlr/vol79/iss1/10

    work page

  78. [78]

    NIST. 2020. NIST privacy framework: a tool for improving privacy through enterprise risk management. doi:10.6028/NIST.CSWP.01162020

    work page doi:10.6028/nist.cswp.01162020 2020

  79. [79]

    Donald A. Norman. 1993.Things that make us smart: defending human attributes in the age of the machine. Addison-Wesley Longman Publishing Co., Inc., USA. doi:10.5555/200550

    work page doi:10.5555/200550 1993

  80. [80]

    2017.Unified Modeling Language 2.5.1

    Object Management Group Standards Development Organization (OMG SDO). 2017.Unified Modeling Language 2.5.1. Object Management Group Standards Development Organization (OMG SDO). https://www.omg.org/spec/UML/2.5. 1/PDF#page=681.07 OMG Document Number formal/2017-12-05, Chapter 18

    work page 2017

Showing first 80 references.