{"id":"b10f5aa1-ddf4-4e1a-a0d2-b611047cf298","arxiv_id":"2411.18380","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":2,"one_line_summary":"A survey of 192 UK smartphone users found that out-of-device privacy concerns and technology affinity correlate with preference for personalized shoulder surfing protections, while age, gender, and smartphone OS showed no significant effects.","lead":"This paper maps how personal traits like privacy priorities and technology affinity relate to which shoulder surfing protection mechanisms people prefer. It finds that users favor simple alerting mechanisms like icon overlays and often prefer non-digital options such as covering the screen with a hand.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"ATI correlations in §6.6 are internally inconsistent: Perceived Usefulness p=0.032 is called non-significant, and τ=0.105 is called 'strong'; this undercuts the central ATI-personalisation claim.","rationale":"The reader's weakest_assumption was the privacy-paradox gap between stated preferences and real adoption, which is a general limitation acknowledged in §7.3. My concern is more immediate and internal: the ATI analysis in §6.6 contains a statistical self-contradiction. The paper reports p=0.032 as non-significant while also describing τ=0.105 as 'strong', and then concludes ATI does not affect Perceived Usefulness despite the reported p-value indicating otherwise. Since the abstract's second highlighted driver of personalisation preference is ATI, this error undermines one of the two core correlational claims. The proposed re-analysis with multiple-comparison correction would settle whether the ATI–personalisation correlation survives; if it does not, the central claim loses its ATI support. The other main correlation, ODPS, is less affected, so the overall verdict remains CONDITIONAL rather than REJECT. This is the same condition the reader identified but with a different, more specific technical basis.","tokens_in":15432,"tokens_out":3513,"duration_ms":31390,"concrete_test":"Recompute Kendall's tau and p-values for ATI vs Perceived Usefulness and ATI vs Personalisation from the raw survey responses; verify the reported values (τ=0.105, p=0.037 and τ=0.109, p=0.032), then apply a Benjamini-Hochberg correction across all correlations reported in §6.6. If the ATI–Perceived Usefulness p-value is indeed 0.032, the conclusion that ATI does not impact Perceived Usefulness is contradicted; if the ATI–Personalisation p-value loses significance after correction, the central ATI-personalisation claim fails.","verdict_should_be":"UNCHANGED","load_bearing_attack":"In §6.6, the Affinity for Technology Interaction (ATI) analysis contains a self-contradiction that directly affects one of the paper's central claims. The text reports: 'a weak association was observed with Perceived Usefulness which was not found to be statistically significant (τ = .109, p = 0.032).' Since p = 0.032 is less than the conventional 0.05 threshold, this association is statistically significant, contradicting the conclusion in the same paragraph: 'the ATI score does not impact Perceived Usefulness.' Additionally, the paper describes a 'strong positive correlation between Personalisation and ATI' with τ = 0.105, but a Kendall's tau of 0.105 is conventionally a weak correlation, not strong. Because the abstract and key takeaways highlight that users with high ATI favour personalisation, the reliability of the entire ATI-related finding is questionable. The error could stem from a misreported coefficient, a mis-assigned p-value, or a failure to apply multiple-comparison correction across the many correlations reported in §6.6. Without corrected statistics, the direction and significance of the ATI–personalisation relationship—one of only two personal-attribute correlations emphasized in the abstract—cannot be trusted as written.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"This SoK paper presents a systematic literature review that identifies ten categories of content-based shoulder surfing protection mechanisms, and then reports an online survey (N=192 UK participants) that measured users' perceptions of video prototypes of these mechanisms. The paper correlates personal attributes (age, gender, smartphone OS, out-of-device privacy scale (ODPS), and affinity for technology interaction (ATI)) with two components derived from the perception responses: Perceived Usefulness and Personalisation. The main claims are that users perceive the mechanisms as protective but prefer non-digital alternatives, that icon overlay is the most preferred mechanism, that ODPS correlates positively with both components (strongly with Personalisation, weakly with Perceived Usefulness), and that ATI correlates with Personalisation but not Perceived Usefulness. The paper concludes with design takeaways for future shoulder surfing protection mechanisms.","tokens_in":15668,"tokens_out":5990,"duration_ms":51419,"significance":"If the results hold, the paper contributes a structured taxonomy of shoulder surfing mechanisms and an initial empirical mapping of personal attributes to preferences, which is valuable for designing user-tailored privacy mechanisms. The use of an externally validated ODPS scale and video-based prototypes to improve participant comprehension are strengths. The ODPS correlations are clearly reported and large in effect size (τ = 0.549 and τ = 0.300), and the paper's acknowledgment of the privacy paradox limitation in Section 7.3 is appropriate. However, the ATI correlations in Section 6.6 are internally inconsistent in both direction and significance, and the effect sizes are small, which undermines one of the two personal-attribute correlations highlighted in the abstract. The paper's central design takeaways therefore require a corrected re-analysis of the ATI results.","major_comments":[{"comment":"The ATI results are internally inconsistent. The text calls τ = 0.105 a 'strong positive correlation' between Personalisation and ATI, but a Kendall's tau of 0.105 is conventionally weak. It then reports 'a weak association was observed with Perceived Usefulness which was not found to be statistically significant (τ = .109, p = 0.032)', yet p = 0.032 is below the conventional 0.05 threshold. This contradicts the next sentence 'the ATI score does not impact Perceived Usefulness' and also contradicts the Introduction's claim that high-ATI users scored low on perceived usefulness. Because the abstract highlights ATI as a predictor of personalisation, the direction and significance of these correlations must be corrected and re-reported.","section":"§6.6"},{"comment":"No multiple-comparison correction is reported for the many Kendall's tau correlations computed (ODPS, ATI, age, gender, OS, each against Personalisation and Perceived Usefulness). With roughly ten correlations, a Bonferroni threshold would be about 0.005, which would make the ATI-personalisation p = 0.037 non-significant. The authors should report corrected p-values or explicitly justify why uncorrected tests are appropriate.","section":"§6.6"},{"comment":"The PCA factor scores for 'Perceived Usefulness' and 'Personalisation' are used in the correlations, but the paper does not state how the scores were computed (e.g., regression, Bartlett, or summed items) or whether the factor structure was stable across the ten mechanism categories. This information is needed to interpret the magnitude and reproducibility of the reported correlations with ODPS and ATI.","section":"§6.5"}],"minor_comments":[{"comment":"The reference to the survey platform is missing: 'Qualtrics [ ?]' should be resolved to a proper citation or removed.","section":"§5.1"},{"comment":"'Principle component analysis' should be 'Principal component analysis' throughout.","section":"§6.5"},{"comment":"In the ODPS results, 'τ = 0.300, = p < 0.001' contains an extraneous '='; it should read 'τ = 0.300, p < 0.001'.","section":"§6.6"},{"comment":"In the post hoc analysis, items (1) and (4) list the same comparisons for icon overlay; one of them is redundant and should be removed.","section":"§6.4"},{"comment":"The table displays empty cells in the Personalisation column and includes a statement highlighted as removed; the formatting should be cleaned to show the final factor loadings only.","section":"Table 5"},{"comment":"The exclusion threshold of 'less than half the average time' is arbitrary; a brief justification or sensitivity analysis would strengthen the reporting of data quality.","section":"§6.1"},{"comment":"In the bullet 'Increased Personalisation with Increased Affinity for Technology Interaction', the phrase 'technology-preferred mechanisms' is awkward and likely a typo; consider rephrasing to 'preferred mechanisms' or 'technology-oriented mechanisms'.","section":"§7.2"}],"recommendation":"major_revision","confidential_remarks":"The ATI inconsistency in Section 6.6 is serious but appears fixable with a re-analysis; the ODPS correlations and the preference rankings are more solid and likely to survive correction. Note that one of the authors is a co-author of the ODPS scale, though the scale is externally validated, so this is not a circularity problem. The literature review is scoped to Google Scholar's top-10 venues, which is a limitation for an SoK but is partially mitigated by forward/backward search. The paper is within scope for a usable security journal; the main concern is statistical credibility of the ATI claim."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Here is my take. This is a competent SoK with a genuinely new dataset: 192 UK users rated and ranked ten categories of shoulder-surfing protection mechanisms, and the authors correlate those preferences with the ODPS scale, ATI, age, gender, and OS. The strongest result is the ODPS correlation with Personalisation (tau=0.549), which is large and makes sense: people who care about out-of-device privacy want protections they can tailor. The mechanism categorization is clean, the videos are a nice touch, and the survey has pilot testing, attention checks, and comprehension checks. The finding that users prefer non-digital alternatives and favor icon overlay, haptic, and tangible mechanisms over filters is consistent with prior work and useful for designers.\n\nThe soft spots are real but fixable. The ATI analysis in §6.6 is internally contradictory. The text calls tau=0.105 a 'strong positive correlation' when that is weak by any convention, and it describes p=0.032 as 'not found to be statistically significant.' Since the abstract highlights the ATI-personalisation link, this needs to be corrected—either the coefficient, the p-value, or the multiple-comparison story. As written, that finding cannot be trusted. The paper also relies entirely on self-reported reactions to video prototypes; the authors acknowledge in §7.3 that wild studies are needed to overcome the privacy paradox. That is a genuine limitation for an SoK that wants to guide design, but not a fatal one for an initial mapping.\n\nA few smaller issues: gender is analyzed with Kendall's tau on a binary variable, which is unconventional; a Mann-Whitney or point-biserial would be cleaner. The conclusion sentence about 'no significant differences between male and female iOS and Android users' conflates two separate comparisons. And the PCA would benefit from reporting eigenvalues and cross-loadings, though the two-factor solution with alphas above .7 is reasonable.\n\nThe circularity worry is mild: ODPS is the authors' own scale, but it was validated in a separate CHI paper, not fit to this dataset. That is acceptable. This paper deserves a serious referee. The data is useful, the ODPS finding is likely robust, and the design takeaways are concrete. I would send it to peer review and ask for a corrected statistical section and a more careful discussion of null results, not a desk reject.","headline":"Useful new survey data on shoulder-surfing protection preferences, but the ATI analysis in §6.6 has a self-contradictory statistical report that must be fixed before the headline claim can be trusted.","tokens_in":16188,"tokens_out":2603,"would_cite":false,"duration_ms":22369,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Users prefer alert-style shoulder-surfing shields over screen filters, and privacy-minded, tech-savvy users want them personalised.","keywords":["shoulder surfing","privacy preferences","protection mechanisms","personal attributes","out-of-device privacy","affinity for technology","systematization of knowledge","usable security"],"falsifier":"A longitudinal field study that installs a shoulder-surfing protection app on participants' phones and logs whether icon-overlay alerts are actually kept and used, checking whether high out-of-device-privacy and technology-affinity scores predict those choices, would settle the claim; if adoption does not track the stated preferences or the correlations vanish, the central mapping fails.","tokens_in":15226,"feed_emoji":"🛡","tokens_out":7244,"duration_ms":61577,"temperature":0.7,"pith_summary":"This paper maps what different kinds of people want from protections against shoulder surfing, the practice of bystanders reading a phone screen without permission. The authors systematise existing mechanisms into ten categories, show video prototypes to 192 UK participants, and find that people regard the mechanisms as privacy-protective but would rather fall back on non-digital moves such as covering the screen with a hand. Among the device-based options, an icon overlay that alerts the user to a bystander ranks first, followed by haptic, tangible, and screen-brightness mechanisms. The paper's key correlation claim is that people who care strongly about out-of-device privacy and people with high technology affinity favour personalised protection mechanisms, while age, gender, and smartphone operating system make no measurable difference. If these correlations hold, designers can tailor shoulder-surfing protections to user profiles and ship alert-style mechanisms as defaults.","feed_headline":"Users prefer alert-style shields over screen filters","feed_subtitle":"Icon-overlay alerts rank first; privacy-concerned, tech-savvy users want personalised designs.","key_machinery":"The load-bearing objects are the ten mechanism categories (display color change, screen brightness, selective visibility, distortion, blurry, replacement by protective text, replacement by meaningful text, physical tangible component, icon overlay, haptic, and combination) derived from the literature review, the video prototypes used to present them, the two validated scales called the Out-of-Device Privacy Scale and the Affinity for Technology Interaction scale, and the two-factor principal component solution separating Perceived Usefulness from Personalisation. The categorisation by information display makes comparable mechanisms that come from different papers, and Kendall's tau correlations tie the personal-attribute scales to the preference components. This machinery turns scattered prototypes into a single preference space.","core_discovery":"On the paper's own terms, the discovery is a first empirical mapping between personal attributes and preferences for content-based shoulder-surfing protection. After a systematic literature review, the authors classified 27 mechanisms into ten categories by how information is presented on screen, and surveyed N=192 participants with video prototypes. Responses split into two components, which a principal component analysis names Perceived Usefulness and Personalisation. Out-of-device privacy concern correlated positively with both components, and affinity for technology interaction correlated positively with personalisation; no significant differences appeared for age, gender, or operating system. The authors conclude that protection mechanisms should be unobtrusive alerts available by default, and that designers can use privacy profiles to tailor them.","pith_inferences":["The authors stop at stated preferences; a step they leave implicit is that an adaptive privacy assistant could use out-of-device privacy and technology-affinity scores to recommend or configure mechanisms, provided real adoption data confirm the stated preferences.","The Euclidean distances between mechanism categories suggest several filter-style mechanisms are near-substitutes in users' minds, so designers might consolidate them; the paper does not test whether users would accept such consolidation in an actual choice.","Because participants preferred non-digital alternatives overall, the strongest practical competitor to any device mechanism may remain the simplest one, a hand or screen cover, and designs should complement that habit rather than try to replace it.","The UK-only, content-only scope leaves open whether the same attribute mapping holds for authentication shoulder surfing or in other cultures, which a direct replication survey could test."],"forward_implications":["Future shoulder-surfing protections will more likely succeed if they ship pre-installed, since participants said they would not go out of their way to install them.","Alert-style mechanisms (icon overlay, haptic, tangible) should be prioritised over content-obfuscation filters, because they were the only categories participants ranked favourably.","A user's out-of-device privacy score can flag who will value both usefulness and personalisation in a mechanism, enabling profile-based recommendations.","Tech-savvy users form a distinct segment that wants adjustable, customisable mechanisms even though they do not rate usefulness higher.","Age, gender, and smartphone operating system do not need to be design variables for these protections, simplifying the design space."],"supporting_citations":[{"why":"Supplies the proxemics-aware framing and several mechanism categories, including display color change, screen brightness, selective visibility, and distortion.","marker":"[9]"},{"why":"Supplies the haptic, icon overlay, and tangible mechanisms that ranked highest in the survey.","marker":"[19]"},{"why":"Supplies the Out-of-Device Privacy Scale whose correlations with perceived usefulness and personalisation are the paper's main quantitative result.","marker":"[30]"},{"why":"Supplies the Affinity for Technology Interaction scale used to show that tech-savvy users favour personalisation.","marker":"[40]"},{"why":"Provides earlier diary-study evidence of user concern about shoulder surfing and preference for alerting mechanisms, which this survey extends.","marker":"[3]"},{"why":"Provides evidence that users want to signal to bystanders that they have been caught, informing the mechanism properties assessed.","marker":"[8]"},{"why":"Supplies selective visibility and replacement-by-meaningful-text mechanisms that feed into the categorisation.","marker":"[10]"},{"why":"Supplies the replacement-by-protective-text mechanism category used in the survey.","marker":"[18]"}],"fun_headline_variants":["Icon overlays win in shoulder-surfing privacy protection survey","Tech-savvy, privacy-minded users favor tailored screen shields","Non-digital shields preferred: personalization key for screen privacy","Users prefer unobtrusive alerts over filters for screen privacy","Privacy profile determines shield preference, not age or OS"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that people's Likert ratings and rankings of video prototypes reflect how they would actually behave with a real mechanism installed, even though the paper itself flags that a privacy paradox could separate stated preferences from real adoption.","fun_headline_variants_meta":{"raw":{"variants":["Icon overlays win in shoulder-surfing privacy protection survey","Tech-savvy, privacy-minded users favor tailored screen shields","Non-digital shields preferred: personalization key for screen privacy","Users prefer unobtrusive alerts over filters for screen privacy","Privacy profile determines shield preference, not age or OS"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.001044,"raw_usage":{"total_tokens":4359,"prompt_tokens":885,"completion_tokens":3474,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":501,"completion_tokens_details":{"reasoning_tokens":3391}},"tokens_in":501,"tokens_out":3474,"duration_ms":27667,"temperature":1.0,"reasoning_tokens":3391,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-12T11:15:22.649967+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"A longitudinal field study that installs a shoulder-surfing protection app on participants' phones and logs whether icon-overlay alerts are actually kept and used, checking whether high out-of-device-privacy and technology-affinity scores predict those choices, would settle the claim; if adoption does not track the stated preferences or the correlations vanish, the central mapping fails.","supporting_citations":[{"cited_title":"Enhancing mobile content privacy with proxemics aware notifications and protection,","cited_arxiv_id":null,"evidence_quote":"Supplies the proxemics-aware framing and several mechanism categories, including display color change, screen brightness, selective visibility, and distortion."},{"cited_title":"Communicating shoulder surfing attacks to users,","cited_arxiv_id":null,"evidence_quote":"Supplies the haptic, icon overlay, and tangible mechanisms that ranked highest in the survey."},{"cited_title":"Out-of-device privacy unveiled: Designing and validating the out-of-device privacy scale (odps),","cited_arxiv_id":null,"evidence_quote":"Supplies the Out-of-Device Privacy Scale whose correlations with perceived usefulness and personalisation are the paper's main quantitative result."},{"cited_title":"A personal resource for tech- nology interaction: development and validation of the affinity for technology interaction (ati) scale,","cited_arxiv_id":null,"evidence_quote":"Supplies the Affinity for Technology Interaction scale used to show that tech-savvy users favour personalisation."},{"cited_title":"Shoulder surfing through the social lens: A longitudinal investigation & insights from an exploratory diary study,","cited_arxiv_id":null,"evidence_quote":"Provides earlier diary-study evidence of user concern about shoulder surfing and preference for alerting mechanisms, which this survey extends."},{"cited_title":"The interplay between personal relationships & shoulder surfing mitigation,","cited_arxiv_id":null,"evidence_quote":"Provides evidence that users want to signal to bystanders that they have been caught, informing the mechanism properties assessed."},{"cited_title":"Eyespot: Leveraging gaze to protect private text content on mobile devices from shoulder surfing,","cited_arxiv_id":null,"evidence_quote":"Supplies selective visibility and replacement-by-meaningful-text mechanisms that feed into the categorisation."},{"cited_title":"Cashtags: Protecting the input and display of sensitive data,","cited_arxiv_id":null,"evidence_quote":"Supplies the replacement-by-protective-text mechanism category used in the survey."}],"review_version":1}