{"id":"6988ab54-74fb-4ff1-a296-9fa950ab27f1","arxiv_id":"1908.06814","paper_version":4,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":0,"one_line_summary":"A systematic review of privacy policies shows that natural language, graphical, and machine-readable forms each serve one community, and no single form can be simultaneously legal, understandable, and enforceable.","lead":"Privacy policies come in three forms: long legal texts, icons and labels, and computer-readable rules, and each form serves a different community but fails the others. This survey maps all three forms, shows that no single form is legal, understandable, and enforceable at once, and argues future policies should combine them.","discovery_kind":"review","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Field-level absence claims rest on a deliberately representative corpus; Section 6.2's 'no existing solution' needs a completeness argument or a softened wording.","rationale":"Agree with the reader's weakest-assumption analysis. The paper is honest about its sampling strategy in Section 2, but does not carry the qualification through to the conclusions: Sections 6.2 and 6.3 shift from 'representative work' to 'no existing solution' and quantitative percentages. Since the SoK's contribution is precisely a gap analysis, the central claim is load-bearing; it cannot be saved by the fact that the individual facet studies are accurate. The Section 6.1 argument provides good reasons why current mono-faceted formats fail at least one requirement, and it is plausible that the three requirements are in tension, but no formal or systematic argument rules out a single artifact that combines the needed expressiveness. A single counterexample outside the corpus would falsify the strongest claim. The paper's internal evidence also contains a small inconsistency: Section 3.1 identifies legal basis as common in natural-language policies, while Section 6.3 says legal basis is absent from all studied work and Figure 5 reports coverage only for graphical and machine-readable policies; this reinforces the need to scope the claims. This is not an integrity issue and the survey remains valuable; the appropriate disposition is to keep the conditional verdict, with the revision request to either justify corpus completeness or weaken the absence claims. No machine-checked artifact or independent reproduction supports the negative conclusions.","tokens_in":26096,"tokens_out":8964,"duration_ms":88699,"concrete_test":"Conduct a forward citation snowball starting from the paper's own references: collect all works in Google Scholar or Scopus (2015–2020) that cite P3P, Polisis, DaPIS, PILOT, LPL, or SIMPL and screen them for a single policy artifact that simultaneously provides natural-language text, graphical icons, and machine-readable syntax (e.g., legal basis, retention, and DS rights expressed in all three forms). If such an artifact exists, the Section 6.2 'no existing solution encompasses...' claim is false; if a documented search protocol returns no such artifact, the absence claim can be upgraded from a sample observation to a defeasible field-level finding.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The load-bearing step is the inference from a deliberately representative corpus to field-level absence claims. Section 2 explicitly says an exhaustive review is 'unfeasible and undesirable' and that the authors focus on 'representative work.' Yet Section 6.2 states that 'no existing solution encompasses the requirements for legal compliance, understandability, and enforceability,' and Section 6.3 reports coverage percentages (e.g., Legal basis 5%/0%/95% for graphical and 0%/0%/100% for machine-readable) as if they described the whole solution space. A representative sample supports 'none found in the reviewed corpus,' not 'none exists.' The only argument for the stronger 'single facet cannot' claim is Section 6.1's illustrative comparison of a Facebook privacy-policy excerpt, a Privacy Tech icon, and an APPEL-P3P fragment; these examples show current representational trade-offs, not an impossibility result over the design space. Because the paper's main recommendation—only multi-faceted policies can satisfy legal validity, understandability, and enforceability—depends on this absence claim, any omitted counterexample weakens the central conclusion. The quantitative support is also less robust than presented: Figure 5 omits natural-language coverage, while Section 3.1 says legal basis is 'regularly found' in natural-language policies, making the Section 6.3 statement that legal basis is 'absent from all the studied work' at least ambiguous.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper systematizes privacy-policy research into three 'facets': natural-language, graphical, and machine-readable policies. It adapts an existing taxonomy (adding legal basis and grouping items), surveys representative works in each facet, and categorizes them in Tables 2 and 3. It argues that each facet serves a different audience and that mono-faceted policies cannot simultaneously satisfy legal validity, understandability, and enforceability/auditability. The paper then discusses multi-faceted policies, proposes unified and compound design approaches, and quantifies taxonomy-item coverage by facet in Figure 5 and Section 6.3.","tokens_in":26360,"tokens_out":7320,"duration_ms":68496,"significance":"The paper's conceptual framework and comparative tables are valuable for researchers and regulators: the three-facet decomposition is clear, the taxonomy mapping to GDPR/FIPPs/CCPA/HIPAA/COPPA is a useful reference, and the explicit study of coverage gaps (e.g., legal basis and policy change) is a concrete contribution. The manuscript is transparent about its representative scope, and the detailed tables make the authors' classifications checkable against the cited sources. However, the central absence claims and the quantitative coverage analysis need to be aligned with that scope; with those revisions, the paper would be a solid systematization contribution.","major_comments":[{"comment":"The claim that 'no existing solution encompasses the requirements for legal compliance, understandability, and enforceability' is a field-level absence claim, but the paper explicitly restricts its scope to representative work and states that exhaustive analysis is 'unfeasible and undesirable' (Section 2). A representative corpus supports 'none found in the reviewed corpus,' not 'none exists.' Because this absence claim is load-bearing for the paper's main recommendation and for Section 6.1's 'single facet cannot' statement, the authors should either add a systematic search and inclusion protocol that can support the stronger claim or soften the conclusion to the reviewed corpus.","section":"Section 6.2 (see also Section 2)"},{"comment":"The heat-map percentages (e.g., Legal basis 5%/0%/95% for graphical policies) are presented as quantitative results, but the manuscript does not document the coding procedure, sample sizes, or inter-rater reliability used to classify solutions as complete, partial, or absent. These numbers underpin the 'forgotten items' discussion. Moreover, the text statement that 'legal basis and policy change are absent from all the studied work' is inconsistent with Figure 5, which shows 5% complete coverage of legal basis for graphical policies, and with Section 3.1, which says legal basis is 'regularly found' in natural-language policies. The figure should include a natural-language column (or its omission should be justified), and the text claim should be scoped accordingly.","section":"Section 6.3 and Figure 5"},{"comment":"The categorical statement 'A single facet cannot cover all the requirements of privacy policies' is not established by the illustrative example of a Facebook excerpt, a Privacy Tech icon, and an APPEL-P3P fragment. That example demonstrates limitations of three particular instances, not an impossibility result over the design space of each facet. The claim should be reframed as an absence claim about the reviewed corpus, or supported by a general argument showing why any mono-faceted policy must fail at least one of the three requirements.","section":"Section 6.1"}],"minor_comments":[{"comment":"'HIPPA' should be 'HIPAA' throughout the paper.","section":"Section 3.1 and Table 1"},{"comment":"The sentence 'They present the fine-grained information in a table such as nutrition labels observed on food packaging.' is a verbatim repetition of the preceding sentence and should be removed.","section":"Section 4.1"},{"comment":"'Cunche et al. [71]' is inconsistent with the reference list, where [71] is authored by Morel, Cunche, and Le Métayer; change to 'Morel et al.'","section":"Section 5.3"},{"comment":"The citation [24] for Rei's Prolog semantics is a general Prolog textbook; the Rei language definition [58] should be cited instead.","section":"Section 5.2"},{"comment":"The legend symbols (e.g., 'We use  to denote') do not render in the manuscript text; ensure the glyphs appear in the final PDF.","section":"Tables 2 and 3"},{"comment":"'Coarsed grained' should be 'coarse-grained.'","section":"Appendix A, Table 4"},{"comment":"The survey uses two of the authors' own works ([71,75]) as evidence for the benefits of machine-readable and multi-faceted policies and as a representative example in Table 3. In a SoK, this conflict of interest should be acknowledged and, where possible, supplemented by independent sources.","section":"Sections 5 and 6"}],"recommendation":"major_revision","confidential_remarks":"The paper's scope and venue are appropriate for a journal or a SoK-style publication, but the mismatch between the declared representative corpus and the field-level conclusions is the central risk. The quantitative claims in Section 6.3 need methodological grounding; without this, they could mislead readers. The self-citation pattern is not disqualifying but should be disclosed."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"You should know two things up front. First, this is a usable and honest systematization: the three-facet split (natural language, graphical, machine-readable) is a real frame, not a repackaging, and it will save people time. Second, the paper overstates what its own method can support when it says \"a single facet cannot cover all the requirements\" and \"no existing solution encompasses\" all three. That is a strong absence claim built on a deliberately representative sample, and the authors themselves say exhaustive review is \"unfeasible and undesirable.\"\n\nThe paper earns its keep in the middle. The tables are detailed and the coverage heat map in Figure 5 is a handy artifact. I especially like how it connects each facet to a community and then shows why each mono-faceted solution leaves other communities underserved. The guidelines for unied versus compound multi-faceted policies are concrete and sensible. The survey also situates itself well against Schaub et al. and Cranor; the emphasis on enforcement and auditability in machine-readable policies is a genuine addition. The self-citations are modest and do not drive the argument.\n\nNow the soft spots. The biggest is the step from \"representative work\" to \"no existing solution\" and the coverage percentages in Section 6.3. The percentages come from the authors' own classification of a non-exhaustive set, without a documented coding procedure or inter-rater reliability. That is ok for a scoping survey, but not for field-level conclusions like \"legal basis is absent from all the studied work.\" That sentence is also internally inconsistent: Figure 5 shows legal basis fully covered in 5% of graphical policies (DaPIS), and Section 3.1 says legal basis is \"regularly found\" in natural language policies, while Figure 5 does not even include a natural-language column. The authors should fix that contradiction and soften the absence claims to \"none found in the reviewed corpus.\" The central analytic claim—that no single representational medium can simultaneously serve lawyers, lay-users, and machines—is plausible and well argued with the Facebook/Privacy-Tech/APPEL example, but it is an argument about representational trade-offs, not an impossibility proof. I would want the wording to match that.\n\nBottom line: this is a serious, competently constructed survey. The right editorial outcome is peer review, not desk rejection, with a request to tighten the claims and correct the legal-basis inconsistency. I would cite it and would be glad to see a revised version in print.\n\nRecommendation: send to reviewers, but tell the authors to fix the Section 6.3 wording and add a short completeness or sampling justification.","headline":"A genuinely useful SoK of privacy-policy representations, with a clever three-facet frame; the field-level absence claims outrun the deliberately representative corpus, and one quantitative section contradicts the main text.","tokens_in":26887,"tokens_out":1332,"would_cite":true,"duration_ms":16079,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Privacy policies cannot be expressed in a single format: natural language is needed for legal validity, graphical icons for lay understanding, and machine-readable code for enforcement and audit, so all three facets must be combined.","keywords":["privacy policies","natural language","graphical privacy policies","machine-readable privacy policies","multi-faceted privacy policies","legal compliance","usability","enforcement"],"falsifier":"Find any published or deployed privacy-policy system that simultaneously satisfies all three requirements—recognized as legally valid by a regulator or court, comprehensible to lay users in a controlled study, and machine-enforced with audit logs—and the claim that a single facet cannot cover all requirements is disproved; conversely, surveying a larger corpus and finding only two-facet combinations would confirm it.","tokens_in":25894,"feed_emoji":"🔒","tokens_out":4843,"duration_ms":48069,"temperature":0.7,"pith_summary":"Privacy policies must satisfy three requirements at once: they must be legally valid, understandable by all parties, and enforceable by machines. This paper surveys the three established ways of expressing policies—natural language, graphical icons and labels, and machine-readable policy languages—and argues that each format is tailored to one audience and therefore fails at least one of the three requirements. The central claim is that no single facet can cover all requirements, so future policies must combine all three, for example by generating graphical and machine-readable versions from a legally grounded natural-language core. If the claim is right, the practical path to compliant, usable, enforceable transparency is not choosing a format but building consistency across formats.","feed_headline":"One privacy-policy format can't be legal, readable, and enforced","feed_subtitle":"Survey of text, icon, and code policies: only combining all three can satisfy legal, readability, and audit needs.","key_machinery":"The organizing device is the facet taxonomy: a categorization of privacy-policy expression into natural language, graphical, and machine-readable formats, overlaid on an adapted version of a prior privacy-policy taxonomy (with legal basis added as an item). This lets the authors compare what each format can express, who it serves, and what it omits, and it grounds the argument that the formats are complementary rather than interchangeable.","core_discovery":"The paper's core discovery is a systematized map of privacy-policy expression, organized into three facets: natural language (the only format with legal value), graphical representations (designed for lay-user comprehension), and machine-readable privacy languages (designed for automatic enforcement and auditing). Surveying representative work in each facet through a common taxonomy of policy items—first and third party collection, legal basis, data-subject rights, retention, security, policy change, and other—it finds that each facet covers at least one requirement well but neglects others. In particular, no surveyed solution combines all three facets; existing multi-faceted efforts combine at most two. The paper concludes that a single facet cannot cover all requirements, and proposes guidelines for multi-faceted policies in two styles—unified, where one core facet generates the others, and compound, where existing policies are used together with automated consistency checking.","pith_inferences":["A tri-faceted policy standard could resemble a nutrition label for privacy plus a machine-readable appendix, where regulators audit the code against the text.","The same taxonomy-based gap analysis could be applied to emerging formats such as privacy dashboards, browser-based consent managers, or successors to earlier machine-readable standards to see which items they cover.","The paper's consistency challenge suggests a testable benchmark: automatically checking whether a generated graphical or machine-readable policy preserves the meaning of the natural-language original, which could be formalized as a semantic-equivalence problem.","If legal basis and policy change remain absent because they are hard to represent, regulators may need to prescribe standard phrasing or icons for those items rather than leaving expression open."],"forward_implications":["Mono-faceted policies—text-only, icon-only, or code-only—cannot simultaneously be legally valid, understandable, and enforceable, so organizations must adopt multi-faceted policies to meet all three requirements.","A unified multi-faceted policy should take natural language as the core facet, since it is legally mandatory, and generate machine-readable and graphical versions from it, preserving each facet's distinctive details.","Consistency between facets is the central engineering challenge: the machine-readable version must faithfully represent the legal text, and current manual checking cannot scale without tool support.","Two taxonomy items are almost entirely uncovered—legal basis and policy change—so new policy languages and icon sets should target these gaps.","Existing multi-faceted solutions combine at most two facets; designing a solution covering all three remains an open research direction."],"supporting_citations":[{"why":"Supplies the privacy-policy taxonomy that structures the comparison of content across all facets.","marker":"[109]"},{"why":"Defines the legal requirements that natural-language policies must meet, including transparency and legal basis.","marker":"[39]"},{"why":"Provides regulator guidelines that support the use of icons in privacy notices, anchoring the graphical facet's requirements.","marker":"[111]"},{"why":"Combines natural language and graphical facets, serving as an example of a two-facet solution that still lacks machine enforcement.","marker":"[50]"},{"why":"Combines natural language with formal machine-readable semantics, illustrating a two-facet approach without graphical representation.","marker":"[65]"},{"why":"Combines natural language and machine-readable policies with analysis tools, another two-facet example with formal semantics.","marker":"[75]"},{"why":"Combines natural language and machine-readable policies while supporting data retention obligations, supporting the claim about two-facet combinations.","marker":"[45]"},{"why":"Presents the privacy nutrition label, a canonical graphical standardized notice used to show the understandability benefit of the graphical facet.","marker":"[60]"},{"why":"Demonstrates a recent icon set that covers legal basis and data-subject rights, showing partial progress within the graphical facet.","marker":"[90]"}],"fun_headline_variants":["Privacy policies need three faces: text, icons, and code","No single privacy-policy format works: combine three","Survey: text, icons, and code must merge for privacy policies","One privacy-policy format fails; three facets together win","Privacy policies: three facets, but none alone suffices"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The paper's field-level conclusion rests on the assumption that the representative set of surveyed works is enough to establish that no existing privacy-policy solution covers all three requirements; if a tri-faceted solution exists outside that corpus, the central claim weakens.","fun_headline_variants_meta":{"raw":{"variants":["Privacy policies need three faces: text, icons, and code","No single privacy-policy format works: combine three","Survey: text, icons, and code must merge for privacy policies","One privacy-policy format fails; three facets together win","Privacy policies: three facets, but none alone suffices"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000205,"raw_usage":{"total_tokens":1359,"prompt_tokens":878,"completion_tokens":481,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":494,"completion_tokens_details":{"reasoning_tokens":412}},"tokens_in":494,"tokens_out":481,"duration_ms":5405,"temperature":1.0,"reasoning_tokens":412,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-14T12:33:24.265622+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Find any published or deployed privacy-policy system that simultaneously satisfies all three requirements—recognized as legally valid by a regulator or court, comprehensible to lay users in a controlled study, and machine-enforced with audit logs—and the claim that a single facet cannot cover all requirements is disproved; conversely, surveying a larger corpus and finding only two-facet combinations would confirm it.","supporting_citations":[{"cited_title":"Cameron Russell, et al","cited_arxiv_id":null,"evidence_quote":"Supplies the privacy-policy taxonomy that structures the comparison of content across all facets."},{"cited_title":"Guidelines on transparency under Regulation 2016/679, 2017-12-15","cited_arxiv_id":null,"evidence_quote":"Provides regulator guidelines that support the use of icons in privacy notices, anchoring the graphical facet's requirements."},{"cited_title":"A formal privacy management framework","cited_arxiv_id":null,"evidence_quote":"Combines natural language with formal machine-readable semantics, illustrating a two-facet approach without graphical representation."},{"cited_title":"Analysis of privacy policies to enhance informed consent","cited_arxiv_id":null,"evidence_quote":"Combines natural language and machine-readable policies with analysis tools, another two-facet example with formal semantics."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Presents the privacy nutrition label, a canonical graphical standardized notice used to show the understandability benefit of the graphical facet."},{"cited_title":"DaPIS: An Ontology-Based Data Pro- tection Icon Set","cited_arxiv_id":null,"evidence_quote":"Demonstrates a recent icon set that covers legal basis and data-subject rights, showing partial progress within the graphical facet."}],"review_version":1}