{"id":"546ab37b-7112-4615-8e0a-4eab11b989b0","arxiv_id":"1908.09210","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":0,"one_line_summary":"A 20-interview study finds that users prefer fictitious security-question profiles that are configurable, relatable, interesting, and memorable.","lead":"This paper interviews 20 people about fake profiles designed to answer security questions. It finds they want profiles that feel relatable and can be customized, and argues websites need new question types to support them.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The central design recommendation assumes configurability improves relatability and memorability, but the interview data never test that link.","rationale":"The reader correctly flags external validity as a weakness, but the more load-bearing issue is internal: the interview protocol never tests whether configurability yields the desired attributes. This is not a question of sample size only; the construct validity of the main recommendation is missing. The reported counts are internally consistent (11+9=20 for configurability level, 11+8+1=20 for potential use), and the paper is clearly written, so I see no evidence of fabrication or circularity. However, the central recommendation is a synthesis of separate preference questions, not a tested relationship. This does not change the overall verdict: the paper remains a plausible, exploratory qualitative study whose design recommendations should be treated as hypotheses requiring behavioral validation. A follow-up experiment with recall and subjective measures would settle whether the causal assumption holds.","tokens_in":5275,"tokens_out":5232,"duration_ms":53476,"concrete_test":"Conduct a between-subjects experiment with three conditions: (1) static system-generated profile, (2) user-configured profile, and (3) user-configured profile with anti-self-matching and answer-space constraints. After a standardized exposure and a 1-week delay, measure recall accuracy of profile attributes and self-reported relatability, interestingness, and memorability. If condition (2) does not significantly outperform condition (1) on recall and the subjective scales, the paper's central design recommendation is not supported. If condition (3) also fails, the recommendation conflicts with the security constraints the paper itself proposes.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim is that fictitious profiles should be configurable so users can make them 'relatable, interesting and memorable.' To support this, the paper would need evidence that configurability actually produces those qualities. The Methodology section describes three separate probes: selection of one of two static profiles, attribute keep/remove/add marking, and a direct question about desired configurability. The Results report these separately: relatability, memorability, and interestingness emerged from profile-selection justifications, while configurability preferences came from a different question. No question asked whether configurability increases relatability, interest, or memorability, and no recall or comprehension measure was taken. The first paragraph of 'Discussion and Recommendations' nevertheless fuses these into 'users should be given the option to configure the profiles to make them relatable, interesting and memorable.' Within the sample the support for even the 'highly configurable' framing is thin: 11/20 preferred high configurability and 9/20 preferred low, while 14/20 wanted some configurability. The load-bearing inference from 'users want configurability' and 'users like relatable/interesting/memorable profiles' to 'configurability causes relatability/interest/memorability' is therefore not directly supported by the data as reported.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper reports a qualitative study of 20 structured interviews investigating how users would want system-generated fictitious profiles to be designed for answering security questions. Participants chose between two static example profiles, marked attributes to keep/remove/add, and answered questions about desired configurability, availability, and willingness to adopt the approach. The authors report that relatability, memorability, and interesting attributes drive profile selection; that participants prefer configurable profiles; and that slightly more than half of participants would consider using fictitious profiles. They derive design recommendations favoring highly configurable profiles with safeguards, enhanced security questions, and stronger protection for stored profiles, and they call for future work to evaluate memorability empirically.","tokens_in":5407,"tokens_out":2607,"duration_ms":27741,"significance":"If the findings hold, the paper offers a useful user-centered design direction for an under-explored approach to mitigating the known memorability and security limitations of system-generated answers to security questions. The qualitative method is recognizable, with independent coding by two researchers and tie-breaking by a third, and the paper is honest in its 'Further Research' section that the actual usability benefits have not yet been empirically validated. The main strength is that it addresses a genuine gap in the usable security literature. However, the study's scope is small, the data are attitudinal rather than behavioral, and the central design recommendation goes beyond what the interview questions were able to test.","major_comments":[{"comment":"The statement that users 'should be given the option to configure the profiles to make them relatable, interesting and memorable' fuses two separate findings into a causal claim. The interview protocol asked (a) which of two static profiles participants would select and why, (b) which attributes to keep/remove/add, and (c) what level of configurability they desired. No question asked whether configurability increases relatability, interest, or memorability, and no recall or comprehension measure was taken. The data show 14/20 participants wanted configurability and 11/20 wanted high configurability, while 9/20 preferred low configurability, but they do not show that configurability produces the three qualities named in the abstract. The recommendation should be rephrased as two independent findings, or additional evidence for the link should be provided.","section":"Discussion and Recommendations, 'Improving the design of fictitious profiles'"},{"comment":"The text repeatedly refers to Table 1 ('The main finding from Table 1 is...'), but no Table 1 appears anywhere in the manuscript. Since the attribute keep/remove/add findings rest entirely on this table, the results as reported cannot be checked. The table must be included, or the references to it must be removed and the findings reported in full text.","section":"Results, 'Are there any preferred attribute categories?'"},{"comment":"The adoption finding is based on 20 participants' stated intentions after a brief session with two static example profiles, with no prototype, no long-term exposure, and no behavioral measure. The paper itself acknowledges in 'Further Research' that a direct usability evaluation of memorability has not yet been done. The claim that fictitious profiles 'seem to have been well received' overstates what the data can support; the relevant results should be framed as hypothetical preferences only, and external-validity limitations should be acknowledged in the text.","section":"Methodology and 'Would users use fictitious profiles and why?'"},{"comment":"All interviews were conducted by a single researcher, and no interview transcripts, coding sheets, or inter-rater agreement statistics are provided. Because the study is purely qualitative and small, the absence of this material makes it difficult for a reader to assess the reliability of the theme extraction, despite the described dual-coding procedure. The authors should provide at least an excerpt of the coding scheme or an appendix with illustrative coded responses.","section":"Methodology"}],"minor_comments":[{"comment":"11/20 is 55% of the participants, so describing this as 'Almost half the participants' is incorrect; the text should say 'more than half' or '11 of 20 reported...'.","section":"Results, 'Would users use fictitious profiles and why?'"},{"comment":"Reference [14] (Trewin et al., 'Biometric authentication on a mobile device') does not appear to support the sentence 'using system-generated information has usability limitations (mainly memorability) [1,14]'; the authors should verify that this citation is appropriate.","section":"References"},{"comment":"Figure 2 is described as 'Example of attributes marked by participants' but the caption and text do not explain the marking legend (e.g., what symbols indicate keep, remove, and add), which makes the figure hard to interpret independently.","section":"Methodology"},{"comment":"The sentence 'Our findings also reveal that users would prefer fictitious profiles to be available all the time' is slightly too strong given the underlying result, since 4/20 participants preferred limited availability; consider stating '16 of 20 participants preferred...'.","section":"Discussion and Recommendations, 'Availability vs security'"},{"comment":"The manuscript would benefit from a short related-work paragraph linking to systems that generate random answers or avatars for authentication, because the current introduction moves quickly from prior work to the interview study and leaves the novelty claim somewhat implicit.","section":"Throughout"}],"recommendation":"major_revision","confidential_remarks":"The manuscript reads like a condensed workshop-style report, and its central contribution currently rests on an unsupported causal link between configurability preferences and the qualities of relatability, interest, and memorability. The missing Table 1 is a concrete verifiability problem that must be fixed. If the authors are willing to reframe the contribution as a set of user preferences rather than a validated design recommendation, and to include the missing materials, the paper could become acceptable for a venue that accepts small qualitative studies. I would not recommend reject, because the topic is relevant and the data, though limited, are reported with reasonable transparency."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Quick read on 1908.09210: small, honest qualitative study, but the headline recommendation fuses two separate findings in a way the data don't support.\n\nWhat's new: this is the first empirical work on what users want from fictitious profiles for security questions, following the authors' own earlier proposal. The interview protocol is straightforward and the reported counts are internally consistent. The preference for relatability, memorability, and interesting attributes is plausible, and the suggestion to constrain configurability so users don't recreate their own info is sensible. Credit where due: the authors clearly flag the memorability limit and call for further work.\n\nSoft spots: the sample is 20 convenience recruits, 15 male, 15 postgraduates, single interviewer, no transcripts or codebook, and Table 1 is referenced but absent from the text I had. That caps generalizability and makes the thematic analysis unverifiable. More importantly, the central design claim goes a step beyond the data. Participants were asked which profile they liked and why, and separately whether they wanted configuration. The results show 14/20 wanted some configuration, but 11/20 wanted high and 9/20 wanted low. No question tied configuration to relatability, interest, or memorability, and no recall or comprehension measure was taken. So 'users should be able to configure profiles to make them relatable, interesting, memorable' is a plausible design hypothesis, not a finding. The Discussion presents it as a finding.\n\nThe adoption finding is also softer than the abstract suggests: 11/20 would consider, 8/20 prefer their own answers, 1 unsure. That's not a majority endorsement of the concept overall, just a plurality in one direction.\n\nBottom line: worth taking seriously as an exploratory study. A careful referee should ask for the missing table, raw notes or coding materials, and a rewording of the causal inference. For a desk editor, this deserves peer review rather than a desk reject, with the expectation of revision.","headline":"Useful exploratory study; the main design recommendation overreaches the data.","tokens_in":5971,"tokens_out":1947,"would_cite":false,"duration_ms":19451,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Security questions become usable when users can tweak fictitious profiles, this interview study finds.","keywords":["usable security","security questions","fictitious profiles","fallback authentication","account recovery","user configurability","memorability","qualitative interviews"],"falsifier":"Give users a real fictitious-profile system, let them configure profiles, and observe recoveries weeks later; if most users either cannot recall their configured answers, or configure answers that coincide with their real personal facts and are guessable by close contacts, the design premise fails.","tokens_in":5009,"feed_emoji":"🎭","tokens_out":6192,"duration_ms":60501,"temperature":0.7,"pith_summary":"This paper tackles a long-standing trade-off in account recovery: security questions are vulnerable to guessing and disclosure, while system-generated answers are hard to remember. It tries to establish that the middle path—system-generated fictitious profiles that users can partly configure—can be both usable and safe, and it derives design recommendations from 20 structured interviews. The central claim is that configurability is the key: users want to adjust profiles so they feel relatable, interesting, and memorable, and the system should stop them from recreating their own identity or choosing low-entropy answers. The paper also claims that existing security-question sets, built around names, places, and favourites, would need to be extended to cover profile-style attributes. A sympathetic reader would care because it gives concrete design guidance for an account-recovery mechanism that avoids the worst failure modes of both real personal questions and random secrets.","feed_headline":"User-tweaked fake profiles make security questions usable","feed_subtitle":"20 interviews say configurable, relatable personas could cut guessing and memorability pain—if safeguards stop self-matching.","key_machinery":"The central object is the fictitious profile: a system-generated persona whose details—name, age, gender, characteristics, favourites, and similar fields—serve as the answers to security questions. The mechanism carrying the argument is user configurability: the paper's interviewees reported that being able to adjust profile attributes makes the persona relatable, interesting, and memorable, which is what makes the system-generated answer usable over time. The paper treats configurability as a double-edged lever: it must be wide enough to support these three qualities, but constrained enough that users cannot recreate their own identity or define answers with a tiny guessable space, for example by checking configured attributes against the user's social-networking profile.","core_discovery":"The paper's central discovery, on its own terms, is that a fictitious profile is usable as a security-question answer only when the user can make it 'theirs' without making it true. Most participants wanted profiles they could configure—11 of 20 wanted detailed configurability, 14 of 20 wanted at least some—and the reasons they gave for choosing or editing profiles clustered around relatability, memorability, and interesting attributes. They preferred text-based profile fields such as basic information, characteristics, and favourites, and disliked numeric attributes like finance. The paper further reports that 16 of 20 users wanted the profile available at all times, and 11 of 20 said they would actually use a fictitious profile to answer security questions, mostly because it would be more secure than their own answers; the 8 who would not cited memorability. From these findings the paper draws design requirements: let users configure, prevent self-matching and small answer spaces, protect always-available profiles, and broaden the set of security questions so profile attributes have corresponding questions.","pith_inferences":["A further implication, not drawn in the paper: if 'relatable' means 'close to my own life,' then acquaintances who know the user may still guess configured answers; the social-network check addresses exact self-matching but not close-guess risk.","A testable extension the paper does not run: measure the effective answer-space entropy of user-configured profiles; the security argument stands or falls on whether configured answers are harder to guess than real personal facts.","The same configurability mechanism could transfer beyond account recovery, such as letting users generate fictional personas for privacy-conscious registration, which the authors list only as future work."],"forward_implications":["Fictitious-profile systems should expose configuration of text-based fields (basic info, characteristics, favourites) rather than numeric or financial fields.","Accounts that offer fictitious profiles need a check that configured attributes do not match the user's real social-networking data, otherwise the security gain is lost.","Websites using security questions would need new question types that cover profile attributes, because the standard name/place/favourite sets do not fit a fictitious persona.","Because users want the profile available at all times, the service must store it more securely (e.g., encryption and anonymization) to offset the increased exposure.","Memorability remains unresolved: a substantial minority preferred their own answers, so further techniques to make profile answers easier to recall are needed before wide adoption."],"supporting_citations":[{"why":"Proposes the original idea of using system-generated fictitious profiles for security questions, which this study extends.","marker":"[9]"},{"why":"Shows system-generated information is more secure than user-chosen answers but has memorability limits, motivating the usability investigation.","marker":"[1,14]"},{"why":"Documents statistical guessing attacks and limited answer spaces for personal knowledge questions, defining the security problem.","marker":"[5]"},{"why":"Documents the categories of current security questions and how social networks expose their answers.","marker":"[12]"},{"why":"Measures how easily partners and acquaintances guess security-question answers, providing the insider-attack rationale for system-generated answers.","marker":"[13]"},{"why":"Documents why major services moved away from security questions, defining the account-recovery context.","marker":"[4]"}],"fun_headline_variants":["Let users configure fake profiles to answer security questions","User-tuned fake profiles improve security question usability","Fake profiles need user tweaks to answer security questions","Give users fake profiles to personalize for security answers"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The study assumes that what 20 people say after briefly reading two printed fictitious profiles predicts how real users would configure and remember such profiles in actual, long-term account recovery.","fun_headline_variants_meta":{"raw":{"variants":["Let users configure fake profiles to answer security questions","User-tuned fake profiles improve security question usability","Fake profiles need user tweaks to answer security questions","Give users fake profiles to personalize for security answers"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000854,"raw_usage":{"total_tokens":3687,"prompt_tokens":896,"completion_tokens":2791,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":512,"completion_tokens_details":{"reasoning_tokens":2730}},"tokens_in":512,"tokens_out":2791,"duration_ms":21298,"temperature":1.0,"reasoning_tokens":2730,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-14T11:17:55.304537+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Give users a real fictitious-profile system, let them configure profiles, and observe recoveries weeks later; if most users either cannot recall their configured answers, or configure answers that coincide with their real personal facts and are guessable by close contacts, the design premise fails.","supporting_citations":[{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Proposes the original idea of using system-generated fictitious profiles for security questions, which this study extends."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Documents statistical guessing attacks and limited answer spaces for personal knowledge questions, defining the security problem."},{"cited_title":"Bernheim Brush, and Serge Egelman","cited_arxiv_id":null,"evidence_quote":"Measures how easily partners and acquaintances guess security-question answers, providing the insider-attack rationale for system-generated answers."}],"review_version":1}