REVIEW 3 major objections 5 minor 43 references
Making GDPR Usable: A Model to Support Usability Evaluations of Privacy
T0 review · 3 major / 5 minor · reviewed 2026-08-14 · deepseek-v4-flash
Pith's one-line read A three-axis cube, the UP Cube, aims to turn the GDPR's usability requirements into measurable privacy-evaluation criteria.
desk verdict A well-structured conceptual framework that maps GDPR usability goals onto a cube model, but the central measurability claim outruns what is actually specified. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The load-bearing object is the Usable Privacy Cube, a three-axis evaluative model. Two base axes are EuroPriSe's criteria reorganized into data-protection principles (lawfulness, purpose limitation, data minimization, transparency, security, accountability, and so on) and data-subject rights (information, access, erasure, objection, and the rest); the third axis is made of the new UP criteria, oriented by usability goals extracted from the GDPR. Each UP criterion is modular, ordered on the axis, and broken into subcriteria labeled [Effectiveness], [Efficiency], or [Satisfaction] with sublabels for objective/perceived measures and TEFM (time, effort, financial, material) resources. The cube's work is to make every privacy requirement an intersection of a principle, a right, and a measurable usability question.
What would settle it
Have two independent evaluators apply the same UP criteria (for example, UPC.2) to the same privacy notice in the same context of use; if the resulting effectiveness and efficiency scores diverge widely, or if the scores do not move when the notice is rewritten in obviously plainer language, then the claim that the criteria produce measurable outcomes fails.
Extended reading notes
Core claim
The central discovery is that the GDPR's scattered usability phrasings—clear and plain language, easily accessible information, easy withdrawal of consent, easily exercised rights—can be consolidated into a single evaluative structure. The authors extract 30 usable privacy goals from the regulation, derive 24 usable privacy criteria with subcriteria from them, and show that the existing EuroPriSe criteria can be redescribed as just two axes: the principles controllers must follow and the rights data subjects hold. Each UP subcriterion is classified as measuring effectiveness, efficiency, or satisfaction and as capturing either objective or perceived outcomes; the intended outputs are counts and frequencies (errors, completeness), resource measures (time, effort, financial, material), and satisfaction-scale values. The paper's contention is that these outputs measure the level of usability with which a privacy goal is reached, on a scale, in a specified context of use.
Load-bearing premise
The load-bearing premise is construct validity: the answers people give to questions such as "how much time and effort do you need to access the information?" really measure the usability of privacy, and those answers can be turned into scores without a fully specified scale or aggregation rule.
Editorial extensions
If this is right
- A certification body could extend an existing scheme like EuroPriSe with the UP criteria on an article-by-article basis, starting with Article 12 because it already contains five usability goals.
- Evaluation output can be translated into visual labels, such as traffic-light scales, that let data subjects quickly assess the level of data protection of a product or service.
- Companies that already meet mandatory GDPR compliance could use usability-of-privacy scores to differentiate their products in the market.
- Privacy evaluations would need to specify a context of use—users, goals, tasks, resources, and environment—so the same criterion can yield different results for different user groups, as ISO 9241-11 requires.
- The reclassification of EuroPriSe into principles and rights makes the two perspectives explicit: what controllers must do and what data subjects can demand, including their interaction points.
Reading between the lines
- A natural next test is whether the UP criteria discriminate between two GDPR-compliant services; if their scores do not differ where users clearly find one notice easier, the scale has little diagnostic value.
- The same subcriteria could be reused earlier in the design process as a usability checklist, before certification, since the questions identify where a privacy interface is likely to fail.
- The cube's emphasis on TEFM suggests a concrete cost-of-privacy measure—how much time and effort a data subject spends to understand or exercise a right—that could be compared across products as a kind of privacy price.
- The principles–rights split may expose trade-offs in measurable terms, such as transparency versus data minimization, where scoring high on one axis could lower the other; the cube does not yet say how these tensions should be weighted.
Signed reviews
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper introduces the Usable Privacy Cube (UP Cube), a three-axis model for privacy evaluation. Two base axes come from EuroPriSe certification criteria reorganized into data protection principles and data subjects' rights; the third axis consists of new 'usable privacy criteria' (UP criteria) derived from usability goals extracted from the GDPR text. The paper claims that the UP criteria are 'always measurable' and produce measures of effectiveness, efficiency, and satisfaction (both objective and perceived), which could support privacy labels and a future certification methodology for the usability of GDPR compliance. It also sketches interactions between the axes and reports plans for three IoT use-case validations.
Significance. If the measurement claim could be substantiated, the contribution would be genuinely useful: it offers a systematic, traceable mapping from specific GDPR recitals and articles to evaluative usability criteria, integrates with an existing certification scheme, and addresses a real gap in privacy certification, which currently focuses on legal compliance rather than on how usable that compliance is for data subjects. The paper is also honest about its limitations and explicitly names the missing validation steps. Its strengths are the systematic extraction of 30 UP goals and 24 UP criteria with explicit outcome labels, and the transparent reorganization of EuroPriSe criteria into the two base axes.
major comments (3)
- [Section 6 and Section 8] The central claim that 'the proposed criteria are always measurable' is not substantiated. The paper provides no response scales, anchors, elicitation instruments, or units of analysis for subcriteria such as UPC.2.7 ('To what degree do the data subjects perceive the information as concise?'), UPC.2.4 ('How much of the information were the data subjects able to access?'), or UPC.20.2 (accuracy of remembering risks and rights). Section 8 concedes exactly this gap: 'one needs to investigate which existing HCI methods for usability testing should be used for each of the UP criteria, and in what way.' As it stands, the UP criteria are a set of evaluative questions rather than a measurement instrument, so the claim that they produce measurements of effectiveness, efficiency, and satisfaction is premature.
- [Section 6.1] The process for combining subcriteria into a main UP criterion score is unspecified. The text states that 'the score for a main UP criterion is established based on evaluations of more specific UP criteria,' but it gives no aggregation function, weighting scheme, normalization rule, or treatment of conflicting subcriteria. Without this, the model cannot yield the overall effectiveness/efficiency/satisfaction measures needed for the proposed privacy labels or for comparisons between products.
- [Section 6.2] Several subcriteria presuppose reference standards that are never defined. For example, UPC.2.4-UPC.2.6 measure how much 'the information' the data subject could access and understand, but the paper does not specify which information constitutes the target set or what counts as complete access/understanding; UPC.6.5 and UPC.7.2 require judging whether a data subject can express the 'correct and intended meaning' without defining how that meaning is established. These missing operational definitions are load-bearing because they determine whether two evaluators would obtain comparable measurements from the same product.
minor comments (5)
- [Introduction] The sentence 'there is not other work that extends privacy certification schemes with usability criteria' contains a grammatical error and should read 'there is no other work'.
- [Table 1] The checkmarks in Table 1 are ambiguous for mixed rows: the row for 'C. Target of Evaluation (ToE)' and the rows for '2.3.2 Internal Data Disclosure' and '2.3.3 Disclosure of Data to Third Parties' show ticks in more than one column, but the accompanying text does not explain whether this means the whole row is classified as mixed or whether different subcriteria within the row belong to different categories.
- [Section 6.2, UPC.20.2] UPC.20.2 asks how accurately data subjects can remember risks, rules, safeguards, and rights, but it is labeled [Ey:Cognitive responses]; accuracy of recall is an effectiveness measure, not an efficiency measure. This labeling appears inconsistent with the notation introduced in Section 6.1.
- [Section 7, item 5] The phrase 'extracted from the the Recital (39)' contains a duplicated article and should be corrected.
- [Section 6.2, UPC.10.4] The word 'conse nt' in the phrase 'become aware of the fact that, and the extent to which, conse nt is given' is a typographical error and should read 'consent'.
Circularity Check
No circularity: the UP criteria are operational restatements of the GDPR goals and EuroPriSe criteria, not a prediction derived from them by construction.
full rationale
The paper explicitly builds the UP Cube from two external inputs: the GDPR text (from which 30 UP goals are extracted) and the EuroPriSe criteria (which are reorganized into rights, principles, and context). The UP criteria in Section 6 are keyed to the goals by design, e.g., UPC.2 is the question form of UPG.18, so the criteria necessarily track the goals; this is standard rubric design, not a circular reduction, because the paper makes no quantitative claim whose output is forced by a fitted parameter. The central measurability claim, 'The proposed criteria are always measurable,' is not derived; it is an assertion, and Section 8 (Further Work) concedes that 'one needs to investigate which existing HCI methods for usability testing should be used for each of the UP criteria, and in what way.' That is an under-specification and validation gap, flagged by the paper itself, not circularity. The only self-citation is the footnote identifying [20] as the authors' own short version; it is descriptive and not load-bearing. No uniqueness theorem, ansatz, or fitted-value prediction is imported from the authors' prior work, so the derivation chain remains self-contained.
Assumptions & free parameters
assumptions (4)
- domain assumption Usability, as defined in ISO 9241-11:2018 (effectiveness, efficiency, satisfaction), can be meaningfully applied to privacy protection.
- domain assumption The GDPR text contains identifiable usability goals that can be extracted and operationalized.
- domain assumption EuroPriSe criteria are a valid representation of GDPR compliance and can be reorganized into rights and principles without loss.
- domain assumption Established HCI usability testing methods can produce valid measurements for the proposed constructs.
Cite this review
Pith. "Pith review of Making GDPR Usable: A Model to Support Usability Evaluations of Privacy." pith.science (2026). https://pith.science/paper/BQNGPA7C
@misc{pith2026190803503,
author = {Pith},
title = {Pith review of: Making GDPR Usable: A Model to Support Usability Evaluations of Privacy},
year = {2026},
howpublished = {\url{https://pith.science/paper/BQNGPA7C}},
note = {Machine review of arXiv:1908.03503}
}
read the original abstract
We introduce a new model for evaluating privacy that builds on the criteria proposed by the EuroPriSe certification scheme by adding usability criteria. Our model is visually represented through a cube, called Usable Privacy Cube (or UP Cube), where each of its three axes of variability captures, respectively: rights of the data subjects, privacy principles, and usable privacy criteria. We slightly reorganize the criteria of EuroPriSe to fit with the UP Cube model, i.e., we show how EuroPriSe can be viewed as a combination of only rights and principles, forming the two axes at the basis of our UP Cube. In this way we also want to bring out two perspectives on privacy: that of the data subjects and, respectively, that of the controllers/processors. We define usable privacy criteria based on usability goals that we have extracted from the whole text of the General Data Protection Regulation. The criteria are designed to produce measurements of the level of usability with which the goals are reached. Precisely, we measure effectiveness, efficiency, and satisfaction, considering both the objective and the perceived usability outcomes, producing measures of accuracy and completeness, of resource utilization (e.g., time, effort, financial), and measures resulting from satisfaction scales. In the long run, the UP Cube is meant to be the model behind a new certification methodology capable of evaluating the usability of privacy, to the benefit of common users. For industries, considering also the usability of privacy would allow for greater business differentiation, beyond GDPR compliance.
Figures
Reference graph
Works this paper leans on
-
[1]
Standard ISO/IEC 29100:2011 (2011)
Information technology – Security techniques – Privacy f ramework. Standard ISO/IEC 29100:2011 (2011)
work page 2011
-
[2]
Official Journal of the European Union L 119/1 (2016)
Regulation (EU) 2016/679 of the European Parliament and o f the Council of 27 April 2016 on the protection of natural persons with regard t o the processing of personal data and on the free movement of such data, and repea ling Directive 95/46/EC. Official Journal of the European Union L 119/1 (2016)
work page 2016
-
[3]
uk/pa/ld201516/ldselect/ldeucom/129/12909.htm# idTextAnchor235
The House of Lords EU Committee, European Union Committees report on Online Platforms and the Digital Single Market (2016), https://publications.parliament. uk/pa/ld201516/ldselect/ldeucom/129/12909.htm# idTextAnchor235
work page 2016
-
[4]
EuroPriSe Criteria for the certification of IT products an d IT-based services – v201701. Tech. rep. (2017), https://www.european-privacy-seal.eu/AppFile/ GetFile/6a29f2ca-f918-4fdf-a1a8-7ec186b2e78a
work page 2017
-
[5]
Standard ISO 9241-11:2018 (2018)
Ergonomics of human-system interaction – Part 11: Usabil ity: Definitions and con- cepts. Standard ISO 9241-11:2018 (2018)
work page 2018
-
[6]
Ackerman, M.S., Mainwaring, S.D.: Privacy Issues and Hum an-Computer Interac- tion. In: Cranor, L., Garfinkel, S. (eds.) Security and usabi lity: designing secure systems that people can use, pp. 381–399. O’Reilly (2005)
work page 2005
-
[7]
Communica tions of the ACM 42(12), 41–46 (1999)
Adams, A., Sasse, M.A.: Users are not the enemy. Communica tions of the ACM 42(12), 41–46 (1999)
work page 1999
-
[8]
In: Privacy and Data Prot ection Seals, pp
Balboni, P., Dragan, T.: Controversies and challenges of trustmarks: Lessons for privacy and data protection seals. In: Privacy and Data Prot ection Seals, pp. 83–
Show all 43 references
-
[9]
Usability evaluation in industry 189(194), 4–7 (1996)
Brooke, J.: SUS – A quick and dirty usability scale. Usability evaluation in industry 189(194), 4–7 (1996)
1996
-
[10]
In: Privacy and Data Protection Seals, pp
Cavoukian, A., Chibba, M.: Privacy seals in the USA, Euro pe, Japan, Canada and Australia. In: Privacy and Data Protection Seals, pp. 59–82 . Springer (2018)
2018
-
[11]
Sams Publishing (2004)
Cooper, A.: The Inmates Are Running the Asylum – Why High- Tech Products Drive Us Crazy and How to Restore the Sanity. Sams Publishing (2004)
2004
-
[12]
CHI EA ’18, ACM (2018)
Cranor, L.F.: SIGCHI Social Impact Award Talk – Making Pr ivacy and Security More Usable. CHI EA ’18, ACM (2018). https://doi.org/10.1145/3170427.3185061
2018
-
[13]
O’Reilly (2005)
Cranor, L.F., Garfinkel, S.: Security and usability: des igning secure systems that people can use. O’Reilly (2005)
2005
-
[14]
Intellect Books, Revised edn
Dumas, J.S., Redish, J.C.: A Practical Guide to Usabilit y Testing. Intellect Books, Revised edn. (1999) A Model to Support Usability Evaluations of Privacy 35
1999
-
[15]
Electronic Commerce Research and Applications 10(1), 17–25 (2011)
Edelman, B.: Adverse selection in online ”trust” certifi cations and search results. Electronic Commerce Research and Applications 10(1), 17–25 (2011)
2011
-
[16]
Luxembourg: Publications Offi ce of the European Union (2018)
European Union Agency for Fundamental Rights: Handbook on European data protection law – 2018 edition. Luxembourg: Publications Offi ce of the European Union (2018)
2018
-
[17]
In: Proceedings of the SIGCHI conference on Human factors in computing systems
Good, N.S., Krekelberg, A.: Usability and privacy: a stu dy of Kazaa P2P file- sharing. In: Proceedings of the SIGCHI conference on Human factors in computing systems. pp. 137–144. ACM (2003)
2003
-
[18]
In: Privacy and Data Protection Seals, pp
Hansen, M.: The Schleswig-Holstein data protection sea l. In: Privacy and Data Protection Seals, pp. 35–48. Springer (2018)
2018
-
[19]
Foun- dations and Trends in Human-Computer Interaction 1(1), 1–137 (2007)
Iachello, G., Hong, J.: End-user Privacy in Human-Compu ter Interaction. Foun- dations and Trends in Human-Computer Interaction 1(1), 1–137 (2007)
2007
-
[20]
In: Friedewald, M., ¨Onen, M., Lievens, E., Krenn, S., Fricker, S
Johansen, J., Fischer-H¨ ubner, S.: Making GDPR Usable: A Model to Support Usability Evaluations of Privacy. In: Friedewald, M., ¨Onen, M., Lievens, E., Krenn, S., Fricker, S. (eds.) Privacy and Identity Manageme nt. Data for Bet- ter Living: AI and Privacy, IFIP Advances in I...
2020 doi
-
[21]
In: Privacy and Data Protection Seals, pp
Kamara, I., De Hert, P.: Data protection certification in the EU: Possibilities, Ac- tors and Building Blocks in a reformed landscape. In: Privacy and Data Protection Seals, pp. 7–34. Springer (2018)
2018
-
[22]
In: Cranor, L., Garfinkel, S
Karat, C.M., Brodie, C., Karat, J.: Usability design and evaluation for privacy and security solutions. In: Cranor, L., Garfinkel, S. (eds.) Security and usability: designing secure systems that people can use, pp. 47–74. O’R eilly (2005)
2005
-
[23]
The Human- Computer Interaction Handbook pp
Karat, C.M., Karat, J., Brodie, C.: Privacy Security and Trust: Human-Computer Interaction Challenges and Opportunities at their Interse ction. The Human- Computer Interaction Handbook pp. 669–700 (2012)
2012
-
[24]
In: International Conference on Ubiquitous Compu ting
Langheinrich, M.: Privacy by design – Principles of priv acy-aware ubiquitous systems. In: International Conference on Ubiquitous Compu ting. pp. 273–291. Springer (2001)
2001
-
[25]
, Gasser, U., O’Brien, D.R., Steinke, T., Vadhan, S.: Bridging the gap between computer science and legal approaches to privacy
Nissim, K., Bembenek, A., Wood, A., Bun, M., Gaboardi, M. , Gasser, U., O’Brien, D.R., Steinke, T., Vadhan, S.: Bridging the gap between computer science and legal approaches to privacy. Harvard Journal of Law & Technology 31(2), 687 (Spring 2018)
2018
-
[26]
In: Pri- vacy and Data Protection Seals, pp
Papakonstantinou, V.: Introduction: Privacy and Data P rotection Seals. In: Pri- vacy and Data Protection Seals, pp. 1–6. Springer (2018)
2018
-
[27]
In: I nternational Workshop on Privacy Enhancing Technologies
Patrick, A.S., Kenny, S.: From privacy legislation to interface design: Implementing information privacy in human-computer interactions. In: I nternational Workshop on Privacy Enhancing Technologies. pp. 107–124. Springer ( 2003)
2003
-
[28]
In: Handbook for Privacy and Privacy-Enhancing Tec hnologies: The case of Intelligent Software Agents, chap
Patrick, A.S., Kenny, S., Holmes, C., van Breukelen, M.: Human Computer Inter- action. In: Handbook for Privacy and Privacy-Enhancing Tec hnologies: The case of Intelligent Software Agents, chap. 12, pp. 249–290 (2003 )
2003
-
[29]
John Wiley & Sons (2015)
Preece, J., Rogers, Y., Sharp, H.: Interaction design: b eyond human-computer interaction. John Wiley & Sons (2015)
2015
-
[30]
WW Norton & Company (2015)
Schneier, B.: Data and Goliath: The hidden battles to collect your data and control your world. WW Norton & Company (2015)
2015
-
[31]
Computer networks 76, 146–164 (2015)
Sicari, S., Rizzardi, A., Grieco, L.A., Coen-Porisini, A.: Security, privacy and trust in Internet of Things: The road ahead. Computer networks 76, 146–164 (2015)
2015
-
[32]
IEEE Internet of Things Journal 1(1), 3–9 (2014) 36 J
Stankovic, J.A.: Research directions for the internet o f things. IEEE Internet of Things Journal 1(1), 3–9 (2014) 36 J. Johansen & S. Fischer-H¨ ubner
2014
-
[33]
Computer (10), 71–72 (1993)
Weiser, M.: Ubiquitous computing. Computer (10), 71–72 (1993)
1993
-
[34]
In: USENIX Security Symposium
Whitten, A., Tygar, J.D.: Why Johnny Can’t Encrypt: A Usa bility Evaluation of PGP 5.0. In: USENIX Security Symposium. vol. 348 (1999) A Model to Support Usability Evaluations of Privacy 37 9 Annexes 9.1 Annex A: A list of the Recitals and Articles of GDPR, in ful l text, from...
1999
-
[35]
Any part of such a declaration which constitutes an infringement of this Regulation sha ll not be binding
If the data subject’s consent is given in the context of a written declaration which also concerns other matters, the request for consent sha ll be presented in a manner which is clearly distinguishable from the other matters, in an in telligible and easily accessible form, us...
-
[36]
The withdrawal of consent shall not affect the lawfulness of processing based on consent before its withdrawal
The data subject shall have the right to withdraw his or her cons ent at any time. The withdrawal of consent shall not affect the lawfulness of processing based on consent before its withdrawal. Prior to giving consent, th e data subject shall be informed thereof. It shall be a...
-
[37]
The infor mation shall be provided in writing, or by other means, including, where appr opriate, by electronic means
The controller shall take appropriate measures to provide any in formation referred to in Articles 13 and 14 and any communication under Article s 15 to 22 and 34 relating to processing to the data subject in a concise, tr ansparent, intelligible and easily accessible form, us...
-
[38]
The controller shall facilitate the exercise of data subject right s under Articles 15 to 22. In the cases referred to in Article 11(2), the con troller shall not refuse to act on the request of the data subject for exercis ing his or her rights under Articles 15 to 22, unless...
-
[39]
Where the icons are presented electr onically they shall be machine-readable
The information to be provided to data subjects pursuant to Ar ticles 13 and 14 may be provided in combination with standardised icons in order to give in an easily visible, intelligible and clearly legible manner a meaningful overv iew of the intended processing. Where the ic...
-
[40]
Section 3
The data subject shall have the right to obtain from the contro ller con- firmation as to whether or not personal data concerning him or her are being processed, and, where that is the case, access to the personal data and the fol- lowing information: (h) the existence of autom...
-
[41]
Section 4
Where the controller has made the personal data public and is oblig ed pur- suant to paragraph 1 to erase the personal data, the controller , taking account of available technology and the cost of implementation, shall take reas onable steps, including technical measures, to i...
-
[42]
Article 22
At the latest at the time of the first communication with the data s ubject, the right referred to in paragraphs 1 and 2 shall be explicitly brough t to the attention of the data subject and shall be presented clearly and s eparately from any other information. Article 22. Auto...
-
[43]
The data subject shall have the right not to be subject to a dec ision based solely on automated processing, including profiling, which produces le gal effects concerning him or her or similarly significantly affects him or her
Reviewed August 14, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.