REVIEW 4 major objections 5 minor 20 references
That's Not Me! Designing Fictitious Profiles to Answer Security Questions
T0 review · 4 major / 5 minor · reviewed 2026-08-14 · deepseek-v4-flash
Pith's one-line read Security questions become usable when users can tweak fictitious profiles, this interview study finds.
desk verdict Useful exploratory study; the main design recommendation overreaches the data. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The central object is the fictitious profile: a system-generated persona whose details—name, age, gender, characteristics, favourites, and similar fields—serve as the answers to security questions. The mechanism carrying the argument is user configurability: the paper's interviewees reported that being able to adjust profile attributes makes the persona relatable, interesting, and memorable, which is what makes the system-generated answer usable over time. The paper treats configurability as a double-edged lever: it must be wide enough to support these three qualities, but constrained enough that users cannot recreate their own identity or define answers with a tiny guessable space, for example by checking configured attributes against the user's social-networking profile.
What would settle it
Give users a real fictitious-profile system, let them configure profiles, and observe recoveries weeks later; if most users either cannot recall their configured answers, or configure answers that coincide with their real personal facts and are guessable by close contacts, the design premise fails.
Extended reading notes
Core claim
The paper's central discovery, on its own terms, is that a fictitious profile is usable as a security-question answer only when the user can make it 'theirs' without making it true. Most participants wanted profiles they could configure—11 of 20 wanted detailed configurability, 14 of 20 wanted at least some—and the reasons they gave for choosing or editing profiles clustered around relatability, memorability, and interesting attributes. They preferred text-based profile fields such as basic information, characteristics, and favourites, and disliked numeric attributes like finance. The paper further reports that 16 of 20 users wanted the profile available at all times, and 11 of 20 said they would actually use a fictitious profile to answer security questions, mostly because it would be more secure than their own answers; the 8 who would not cited memorability. From these findings the paper draws design requirements: let users configure, prevent self-matching and small answer spaces, protect always-available profiles, and broaden the set of security questions so profile attributes have corresponding questions.
Load-bearing premise
The study assumes that what 20 people say after briefly reading two printed fictitious profiles predicts how real users would configure and remember such profiles in actual, long-term account recovery.
Editorial extensions
If this is right
- Fictitious-profile systems should expose configuration of text-based fields (basic info, characteristics, favourites) rather than numeric or financial fields.
- Accounts that offer fictitious profiles need a check that configured attributes do not match the user's real social-networking data, otherwise the security gain is lost.
- Websites using security questions would need new question types that cover profile attributes, because the standard name/place/favourite sets do not fit a fictitious persona.
- Because users want the profile available at all times, the service must store it more securely (e.g., encryption and anonymization) to offset the increased exposure.
- Memorability remains unresolved: a substantial minority preferred their own answers, so further techniques to make profile answers easier to recall are needed before wide adoption.
Reading between the lines
- A further implication, not drawn in the paper: if 'relatable' means 'close to my own life,' then acquaintances who know the user may still guess configured answers; the social-network check addresses exact self-matching but not close-guess risk.
- A testable extension the paper does not run: measure the effective answer-space entropy of user-configured profiles; the security argument stands or falls on whether configured answers are harder to guess than real personal facts.
- The same configurability mechanism could transfer beyond account recovery, such as letting users generate fictional personas for privacy-conscious registration, which the authors list only as future work.
Signed reviews
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper reports a qualitative study of 20 structured interviews investigating how users would want system-generated fictitious profiles to be designed for answering security questions. Participants chose between two static example profiles, marked attributes to keep/remove/add, and answered questions about desired configurability, availability, and willingness to adopt the approach. The authors report that relatability, memorability, and interesting attributes drive profile selection; that participants prefer configurable profiles; and that slightly more than half of participants would consider using fictitious profiles. They derive design recommendations favoring highly configurable profiles with safeguards, enhanced security questions, and stronger protection for stored profiles, and they call for future work to evaluate memorability empirically.
Significance. If the findings hold, the paper offers a useful user-centered design direction for an under-explored approach to mitigating the known memorability and security limitations of system-generated answers to security questions. The qualitative method is recognizable, with independent coding by two researchers and tie-breaking by a third, and the paper is honest in its 'Further Research' section that the actual usability benefits have not yet been empirically validated. The main strength is that it addresses a genuine gap in the usable security literature. However, the study's scope is small, the data are attitudinal rather than behavioral, and the central design recommendation goes beyond what the interview questions were able to test.
major comments (4)
- [Discussion and Recommendations, 'Improving the design of fictitious profiles'] The statement that users 'should be given the option to configure the profiles to make them relatable, interesting and memorable' fuses two separate findings into a causal claim. The interview protocol asked (a) which of two static profiles participants would select and why, (b) which attributes to keep/remove/add, and (c) what level of configurability they desired. No question asked whether configurability increases relatability, interest, or memorability, and no recall or comprehension measure was taken. The data show 14/20 participants wanted configurability and 11/20 wanted high configurability, while 9/20 preferred low configurability, but they do not show that configurability produces the three qualities named in the abstract. The recommendation should be rephrased as two independent findings, or additional evidence for the link should be provided.
- [Results, 'Are there any preferred attribute categories?'] The text repeatedly refers to Table 1 ('The main finding from Table 1 is...'), but no Table 1 appears anywhere in the manuscript. Since the attribute keep/remove/add findings rest entirely on this table, the results as reported cannot be checked. The table must be included, or the references to it must be removed and the findings reported in full text.
- [Methodology and 'Would users use fictitious profiles and why?'] The adoption finding is based on 20 participants' stated intentions after a brief session with two static example profiles, with no prototype, no long-term exposure, and no behavioral measure. The paper itself acknowledges in 'Further Research' that a direct usability evaluation of memorability has not yet been done. The claim that fictitious profiles 'seem to have been well received' overstates what the data can support; the relevant results should be framed as hypothetical preferences only, and external-validity limitations should be acknowledged in the text.
- [Methodology] All interviews were conducted by a single researcher, and no interview transcripts, coding sheets, or inter-rater agreement statistics are provided. Because the study is purely qualitative and small, the absence of this material makes it difficult for a reader to assess the reliability of the theme extraction, despite the described dual-coding procedure. The authors should provide at least an excerpt of the coding scheme or an appendix with illustrative coded responses.
minor comments (5)
- [Results, 'Would users use fictitious profiles and why?'] 11/20 is 55% of the participants, so describing this as 'Almost half the participants' is incorrect; the text should say 'more than half' or '11 of 20 reported...'.
- [References] Reference [14] (Trewin et al., 'Biometric authentication on a mobile device') does not appear to support the sentence 'using system-generated information has usability limitations (mainly memorability) [1,14]'; the authors should verify that this citation is appropriate.
- [Methodology] Figure 2 is described as 'Example of attributes marked by participants' but the caption and text do not explain the marking legend (e.g., what symbols indicate keep, remove, and add), which makes the figure hard to interpret independently.
- [Discussion and Recommendations, 'Availability vs security'] The sentence 'Our findings also reveal that users would prefer fictitious profiles to be available all the time' is slightly too strong given the underlying result, since 4/20 participants preferred limited availability; consider stating '16 of 20 participants preferred...'.
- [Throughout] The manuscript would benefit from a short related-work paragraph linking to systems that generate random answers or avatars for authentication, because the current introduction moves quickly from prior work to the interview study and leaves the novelty claim somewhat implicit.
Circularity Check
No circularity: empirical interview study with self-contained findings; self-citations are background, not load-bearing.
full rationale
This paper is an empirical, interview-based study rather than a derivation chain; there is no fitted parameter, prediction equation, or imported uniqueness theorem that reduces to its inputs. The central recommendation—that fictitious profiles should be configurable to be relatable, interesting, and memorable—is grounded in the reported themes from 20 structured interviews: selection justifications elicited relatability/memorability/interestingness, and a separate question elicited configurability preferences. The authors' own prior work (Micallef & Just [9]) is cited only as background for the fictitious-profile concept, not as evidence for the present findings; the 'potential adoption' discussion cites [7,16] only as related work on gamified memorability, not to justify the results. The closest issue is that the interview protocol directly asked about the design dimensions the authors proposed, which can invite socially desirable answers, but that is a validity/interpretation concern, not circularity by construction. No equation or result in the paper is equivalent to its input by definition, so the circularity score is 0.
Assumptions & free parameters
assumptions (4)
- domain assumption System-generated information for security questions is more secure than users' own answers.
- domain assumption The 20 interviewees are a sufficient basis for design recommendations.
- ad hoc to paper The two example profiles and the interview explanation adequately convey the fictitious profile concept.
- domain assumption Self-reported willingness to use fictitious profiles predicts actual adoption.
Cite this review
Pith. "Pith review of That's Not Me! Designing Fictitious Profiles to Answer Security Questions." pith.science (2026). https://pith.science/paper/TFFXJ4BQ
@misc{pith2026190809210,
author = {Pith},
title = {Pith review of: That's Not Me! Designing Fictitious Profiles to Answer Security Questions},
year = {2026},
howpublished = {\url{https://pith.science/paper/TFFXJ4BQ}},
note = {Machine review of arXiv:1908.09210}
}
read the original abstract
Although security questions are still widely adopted, they still have several limitations. Previous research found that using system-generated information to answer security questions could be more secure than users' own answers. However, using system-generated information has usability limitations. To improve usability, previous research proposed the design of system-generated fictitious profiles. The information from these profiles would be used to answer security questions. However, no research has studied the elements that could influence the design of fictitious profiles or systems that use them to answer security questions. To address this research gap, we conducted an empirical investigation through 20 structured interviews. Our main findings revealed that to improve the design of fictitious profiles, users should be given the option to configure the profiles to make them relatable, interesting and memorable. We also found that the security questions currently provided by websites would need to be enhanced to cater for fictitious profiles.
Figures
Reference graph
Works this paper leans on
-
[1]
Mahdi Nasrullah Al-Ameen, Matthew Wright, and Shannon Scielzo. 2015. Towards Making Random Passwords Memorable. Proceedings of the 33rd Annual ACM Conference on Human Factors in Computing Systems - CHI ’15, ACM Press, 2315–2324. http://doi.org/10.1145/2702123.2702241
arXiv 2015
-
[2]
Yusuf Albayram and Mohammad Maifi Hasan Khan
-
[3]
Lynne Baillie. 2002. The home workshop: a method for investigating the home. Retrieved Sept 2, 2015 from http://researchrepository.napier.ac.uk/3858/
work page 2002
-
[4]
Joseph Bonneau, Elie Bursztein, Ilan Caron, Rob Jackson, and Mike Williamson. 2015. Secrets, Lies, and Account Recovery. Proceedings of the 24th International Conference on World Wide Web - WWW ’15, ACM Press, 141–150. http://doi.org/10.1145/2736277.2741691
arXiv 2015
-
[5]
Joseph Bonneau, Mike Just, and Greg Matthews. 2010. What’s in a Name? Evaluating Statistical Attacks on Personal Knowledge Questions. International Conference on Financial Cryptography and Data Security, Springer, Berlin, Heidelberg, 98–113. http://doi.org/10.1007/978-3-642-14577-3_10
-
[6]
Barney G Glaser, Anselm L Strauss, and Elizabeth Strutzel. 1968. The Discovery of Grounded Theory; Strategies for Qualitative Research. Nursing Research 17, 4
work page 1968
-
[7]
Nicholas Micallef and Nalin Asanka Gamagedara Arachchilage. 2017. A Gamified Approach to Improve Users’ Memorability of Fall-back Authentication. Thirteenth Symposium on Usable Privacy and Security (SOUPS 2017), USENIX Association
work page 2017
-
[8]
Nicholas Micallef, Lynne Baillie, and Stephen Uzor
Show all 20 references
-
[9]
Nicholas Micallef and Mike Just. 2011. Using Avatars for Improved Authentication with Challenge Questions. SECURWARE 2011, The Fifth International Conference on Emerging Security Information, Systems and Technologies, 121–124
2011
-
[10]
Proceedings of the 18th international conference on Human-computer interaction with mobile devices and services MobileHCI ’16, ACM Press, 112–123
Time to exercise!: an aide-memoire stroke app for post-stroke arm rehabilitation. Proceedings of the 18th international conference on Human-computer interaction with mobile devices and services MobileHCI ’16, ACM Press, 112–123. http://doi.org/10.1145/2935334.2935338
-
[11]
Marisca Milikowski and Jan J. Elshout. 1995. What makes a number easy to remember? British Journal of Psychology 86, 4: 537–547. http://doi.org/10.1111/j.2044-8295.1995.tb02571.x
1995
-
[12]
Nicholas Micallef, Mike Just, Lynne Baillie, Martin Halvey, and Hilmi Gunes Kayacik. 2015. Why aren’t users using protection? Investigating the usability of smartphone locking. Proceedings of the 17th international conference on Human-computer interaction with mobile devices a...
2015
-
[13]
Bernheim Brush, and Serge Egelman
Stuart Schechter, A.J. Bernheim Brush, and Serge Egelman. 2009. It’s No Secret. Measuring the Security and Reliability of Authentication via “Secret” Questions. 2009 30th IEEE Symposium on Security and Privacy, IEEE, 375–390. http://doi.org/10.1109/SP.2009.11
2009 doi
-
[14]
Ariel Rabkin and Ariel. 2008. Personal knowledge questions for fallback authentication:security questions in the era of Facebook. Proceedings of the 4th symposium on Usable privacy and security - SOUPS ’08, ACM Press, 13. http://doi.org/10.1145/1408664.1408667
2008
-
[15]
Peng Zhao, Kaigui Bian, Tong Zhao, et al. 2017. Understanding Smartphone Sensor and App Data for Enhancing the Security of Secret Questions. IEEE Transactions on Mobile Computing 16, 2: 552–565. http://doi.org/10.1109/TMC.2016.2546245
2017
-
[16]
Shari Trewin, Cal Swart, Larry Koved, Jacquelyn Martino, Kapil Singh, and Shay Ben-David. 2012. Biometric authentication on a mobile device: A Study of User Effort, Error and Task Disruption. Proceedings of the 28th Annual Computer Security Applications Conference on - ACSAC ’...
2012
-
[17]
Phishing threat avoidance behaviour: An empirical investigation
Arachchilage, Nalin Asanka Gamagedara, Steve Love, and Konstantin Beznosov. "Phishing threat avoidance behaviour: An empirical investigation." Computers in Human Behavior 60 (2016): 185-197
2016
-
[18]
Security questions education: exploring gamified features and functionalities
Micallef, Nicholas, and Nalin Asanka Gamagedara Arachchilage. "Security questions education: exploring gamified features and functionalities." Information & Computer Security 26, no. 3 (2018): 365-378
2018
-
[20]
Security awareness of computer users: A game based learning approach
Arachchilage, Gamagedara, and Nalin Asanka. Security awareness of computer users: A game based learning approach. Diss. Brunel University, School of Information Systems, Computing and Mathematics, 2012
2012
-
[2016]
Human-centric Computing and Information Sciences 6, 1: 16
Evaluating smartphone-based dynamic security questions for fallback authentication: a field study. Human-centric Computing and Information Sciences 6, 1: 16. http://doi.org/10.1186/s13673-016-0072-3
Reviewed August 14, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.