Routing Cybersecurity Awareness Training by FFM Personality Trait: A Quasi-Experimental Evaluation
Pith reviewed 2026-06-30 13:11 UTC · model grok-4.3
The pith
Personality-conditional cybersecurity training raises post-assessment scores and pass rates over standard video sessions.
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
Core claim
The personality-conditional group scored significantly higher on the post-assessment (M = 35.88, SD = 5.00 vs M = 30.75, SD = 10.23; Welch's t(58.5) = 2.81, p = .007; Cohen's d = 0.62; 95% CI [1.47, 8.79] marks), with a pass-rate of 100% versus 77.5% (Fisher's exact p < .01).
What carries the argument
The TailoredSec mobile application, which administers the BFI-10 to determine a dominant FFM trait and routes the user to one of four training modules (with Conscientiousness and Neuroticism sharing a module).
Load-bearing premise
The BFI-10 accurately identifies a dominant personality trait that can be used to route training content in a way that produces better learning outcomes, and the non-random quasi-experimental allocation did not introduce selection bias between groups.
What would settle it
A fully randomized trial with a larger sample that finds no significant difference in post-assessment scores or pass rates between the personality-routed condition and the standard video condition.
Figures
read the original abstract
Cybersecurity awareness training has historically adopted a one-size-fits-all approach, despite established individual differences in how users process and retain security information. Personality has been proposed as one axis along which training content might be tailored; yet no prior study has implemented and empirically evaluated a complete personality-conditional system end-to-end. This paper reports the design, implementation, and quasi-experimental evaluation of \emph{TailoredSec}, a mobile cybersecurity awareness application that routes training content based on a user's dominant Five-Factor Model (FFM) personality trait, as measured by the ten-item Big Five Inventory (BFI-10). Seventy-four UK-based adults were allocated to a traditional video-training condition ($n = 40$) or a personality-conditional condition ($n = 34$). Both groups completed a four-item scenario-based pre-assessment (scored 0--40), a single training session, and an equivalent post-assessment. The personality-conditional group additionally completed the BFI-10 (Big Five Inventory-10) and was routed to one of four training modules covering five FFM traits (Conscientiousness and Neuroticism share a module). Pre-assessment scores did not differ between groups ($t(69.1) = 0.43$, $p = .67$), confirming baseline equivalence. The personality-conditional group scored significantly higher on the post-assessment ($M = 35.88$, $SD = 5.00$ vs $M = 30.75$, $SD = 10.23$; Welch's $t(58.5) = 2.81$, $p = .007$; Cohen's $d = 0.62$; 95\% CI $[1.47, 8.79]$ marks), with a pass-rate of 100\% versus 77.5\% (Fisher's exact $p < .01$). These results offer preliminary support for personality-conditional content routing as a feasible design principle for cybersecurity awareness training.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The manuscript reports the design and quasi-experimental evaluation of TailoredSec, a mobile application for cybersecurity awareness training that routes content based on a user's dominant Five-Factor Model (FFM) personality trait as assessed by the BFI-10. In a study with 74 UK adults, participants were allocated to either a traditional video-based training condition (n=40) or a personality-conditional condition (n=34) where they completed the BFI-10 and received one of four tailored modules. Both groups showed equivalent pre-assessment scores, but the personality-conditional group demonstrated significantly higher post-assessment scores (M=35.88, SD=5.00 vs. M=30.75, SD=10.23; Welch's t(58.5)=2.81, p=.007, d=0.62) and a higher pass rate (100% vs 77.5%, Fisher's exact p<.01), providing preliminary evidence for the efficacy of personality-based routing in cybersecurity training.
Significance. If replicated under a design that isolates the routing mechanism, this would be the first end-to-end empirical evaluation of personality-conditional cybersecurity awareness training and could inform more effective, individualized security education programs. The moderate effect size (d=0.62) and use of standard statistical tests provide a concrete starting point, though the quasi-experimental nature limits causal strength.
major comments (2)
- [Abstract / Design] The quasi-experimental allocation (n=40 traditional vs n=34 personality-conditional) is non-random, and only the personality arm completes the BFI-10 before training. This introduces potential confounds from selection bias or procedural effects (e.g., demand characteristics from the extra questionnaire) that are not controlled, so the post-assessment difference cannot be attributed specifically to trait-based content routing (abstract).
- [Abstract] The BFI-10 is used to identify a dominant trait for routing to one of four modules (Conscientiousness and Neuroticism share a module), but the manuscript provides no validation of this mapping's reliability or appropriateness for isolating a single dominant trait within the sample, leaving the routing mechanism's internal validity untested.
minor comments (2)
- [Abstract] The four-item scenario-based pre- and post-assessments (scored 0-40) are described only at a high level; including example items or full scoring rubric would improve reproducibility.
- [Abstract] The abstract reports baseline equivalence via t(69.1)=0.43, p=.67 but does not specify whether the full methods section includes power analysis or handling of the small total sample (n=74) for the reported effect sizes and CIs.
Simulated Author's Rebuttal
We thank the referee for their thoughtful and constructive comments on the manuscript. We address each major comment below, acknowledging the inherent limitations of the quasi-experimental design and outlining specific revisions to the abstract and methods sections.
read point-by-point responses
-
Referee: [Abstract / Design] The quasi-experimental allocation (n=40 traditional vs n=34 personality-conditional) is non-random, and only the personality arm completes the BFI-10 before training. This introduces potential confounds from selection bias or procedural effects (e.g., demand characteristics from the extra questionnaire) that are not controlled, so the post-assessment difference cannot be attributed specifically to trait-based content routing (abstract).
Authors: We agree that the non-random allocation and the procedural difference (BFI-10 completion only in one arm) introduce potential confounds, including possible demand characteristics, that prevent strong causal attribution to the routing mechanism alone. Although pre-assessment equivalence supports baseline comparability, we cannot fully isolate the effect. We will revise the abstract to more explicitly state these limitations and describe the results as preliminary evidence. revision: yes
-
Referee: [Abstract] The BFI-10 is used to identify a dominant trait for routing to one of four modules (Conscientiousness and Neuroticism share a module), but the manuscript provides no validation of this mapping's reliability or appropriateness for isolating a single dominant trait within the sample, leaving the routing mechanism's internal validity untested.
Authors: The dominant trait was identified via the highest BFI-10 subscale score, with Conscientiousness and Neuroticism sharing a module based on their overlapping implications for security-related caution and compliance. We did not perform additional validation of this mapping or its reliability for single-trait isolation in the sample. We will revise the methods section to clarify the rationale for the mapping and explicitly note the lack of validation as a study limitation. revision: yes
Circularity Check
No circularity: empirical quasi-experimental evaluation with direct statistical comparisons
full rationale
This paper reports a quasi-experimental study comparing traditional video training against personality-conditional routing using BFI-10 scores and scenario-based assessments. Outcomes rest on measured pre/post test scores, Welch's t-tests, Cohen's d, and Fisher's exact tests with no mathematical derivations, fitted parameters renamed as predictions, or self-referential equations. The central claim (higher post-assessment scores in the personality arm) is supported by direct empirical data collection rather than any derivation chain that reduces to its own inputs by construction. No load-bearing self-citations, ansatzes, or uniqueness theorems are invoked. The design is self-contained against external benchmarks.
Axiom & Free-Parameter Ledger
axioms (2)
- domain assumption The BFI-10 validly and reliably measures dominant Five-Factor Model traits for routing purposes
- standard math Standard assumptions for Welch's t-test and Fisher's exact test hold for the group comparisons
Reference graph
Works this paper leans on
-
[1]
Verizon,2023DataBreachInvestigationsReport,2023.URL:https: //www.verizon.com/business/resources/reports/dbir/, [Online; ac- cessed 2024-01-10]
2023
-
[2]
N.Mashtalyar,U.Rupasinghe,S.Bhullar, Socialengineeringattacks: Recent advances and challenges, in: HCI for Cybersecurity, Privacy and Trust: Third International Conference, HCI-CPT 2021, Springer,
2021
-
[3]
doi:10.1007/978-3-030-77392-2_27
-
[4]
Babcock, Human error causes 60% of data breaches: How to protect your organization | bit- warden, 2026
K. Babcock, Human error causes 60% of data breaches: How to protect your organization | bit- warden, 2026. URL:https://bitwarden.com/blog/ how-to-protect-your-organization-from-human-error-and-data-breaches/, [Online; accessed 2026-05-12]
2026
-
[5]
Gupta, C
M. Gupta, C. Akiri, K. Aryal, E. Parker, L. Praharaj, From ChatGPT to ThreatGPT: Impact of generative AI in cybersecurity and privacy, IEEE Access 11 (2023) 80218–80245
2023
-
[6]
Aslam, Ai and cybersecurity: an ever-evolving landscape, Inter- national Journal of Advanced Engineering Technologies and Innova- tions 1 (2024) 52–71
M. Aslam, Ai and cybersecurity: an ever-evolving landscape, Inter- national Journal of Advanced Engineering Technologies and Innova- tions 1 (2024) 52–71
2024
-
[7]
Khando, S
K. Khando, S. Gao, S. M. Islam, A. Salman, Enhancing employees information security awareness in private and public organisations: A systematic literature review, Computers & Security 106 (2021) 102267
2021
-
[8]
Aldawood, G
H. Aldawood, G. Skinner, Reviewing cyber security social engineer- ing training and awareness programs—pitfalls and ongoing issues, Future Internet 11 (2019) 73
2019
-
[9]
M. Workman, Wisecrackers: A theory-grounded investigation of phishing and pretext social engineering threats to information secu- rity, Journal of the American Society for Information Science and Technology 59 (2008) 662–674
2008
-
[10]
Bullée, L
J. Bullée, L. Montoya, W. Pieters, M. Junger, P. Hartel, On the anatomyofsocialengineeringattacks:Aliterature-baseddissectionof successful attacks, Journal of Investigative Psychology and Offender Profiling 15 (2018) 20–45
2018
-
[11]
P. Kim, J. Homan, R. Metzer, How long do employees remember information security training programs? a study of knowledge acqui- sition and retention, Issues in Information Systems 17 (2016)
2016
-
[12]
H. El-Sabagh, Adaptive e-learning environment based on learning styles and its impact on development students’ engagement, Inter- national Journal of Educational Technology in Higher Education 18 (2021) 1–24
2021
-
[13]
R. R. McCrae, O. P. John, An introduction to the five-factor model and its applications, Journal of Personality 60 (1992) 175–215
1992
-
[14]
A. T. Shappie, C. A. Dawson, S. M. Debb, Personality as a predictor of cybersecurity behavior, Psychology of Popular Media Culture 9 (2019) 475–480. Okwata and Razzaque:Preprint submitted to ElsevierPage 12 of 14 Routing Cybersecurity Awareness Training by FFM Personality Trait
2019
-
[15]
Gratian, S
M. Gratian, S. Bandi, M. Cukier, J. Dykstra, A. Ginther, Correlating human traits and cyber security behaviour intentions, Computers & Security 73 (2018) 345–358
2018
-
[16]
Baltuttis, T
D. Baltuttis, T. Teubner, M. T. Adam, A typology of cybersecurity behavior among knowledge workers, Computers & Security 140 (2024) 103741
2024
-
[17]
S. M. Kennison, E. Chan-Tin, Taking risks with cybersecurity: Usingknowledgeandpersonalcharacteristicstopredictself-reported cybersecurity behaviors, Frontiers in Psychology 11 (2020) 546546
2020
-
[18]
Kalhoro, R
S. Kalhoro, R. K. Ayyasamy, A. Jebna, A. Kalhoro, K. Krishnan, S.Nodeson,Howpersonalitytraitsimpactoncybersecuritybehaviors of SME employees, in: 2022 International Conference on Innova- tion and Intelligence for Informatics, Computing, and Technologies (3ICT), IEEE, 2022, pp. 635–641
2022
-
[19]
S. M. Albladi, G. R. Weir, Personality traits and cyber-attack victimisation: Multiple mediation analysis, in: 2017 Internet of Things Business Models, Users, and Networks, IEEE, 2017, pp. 1–6. doi:10.1109/CTTE.2017.8260932
-
[20]
T.Halevi,J.Lewis,N.Memon, Culturalandpsychologicalfactorsin cyber-security, in: Proceedings of the 18th International Conference onInformationIntegrationandWeb-basedApplicationsandServices, ACM, 2016, pp. 318–324
2016
-
[21]
S.T.Lawson,M.Yeo,H.Hansen,E.Pearson, Phishingforlongtails: Examining organizational repeat-clickers and non-clickers following a phishing simulation campaign, Computers & Security 99 (2020) 102064
2020
-
[22]
S. Uebelacker, S. Quiel, The social engineering personality frame- work, in: 2014 Workshop on Socio-Technical Aspects in Security and Trust, IEEE, 2014, pp. 24–30. doi:10.1109/STAST.2014.12
-
[23]
R.Cialdini,Influence:ThePsychologyofPersuasion,HarperCollins, New York, 2009
2009
-
[24]
R. Gianotti, S. Cazella, P. Behar, A model for integrating personality traits into an educational recommender system, in: 2019 IEEE 19th International Conference on Advanced Learning Technologies (ICALT), 2019. doi:10.1109/ICALT.2019.00119
-
[25]
Thorp, L
S. Thorp, L. Rimol, S. Grassini, Association of the Big Five personality traits with training effectiveness, sense of presence, and cybersickness in virtual reality, Multimodal Technologies and Inter- action 7 (2023)
2023
-
[26]
F. Mouton, M. M. Malan, L. Leenen, H. Venter, Social engineering attack framework, in: 2014 Information Security for South Africa, 2014, pp. 1–9. doi:10.1109/ISSA.2014.6950510
-
[27]
Hadnagy, Social engineering: The art of human hacking (2011)
C. Hadnagy, Social engineering: The art of human hacking (2011)
2011
-
[28]
M.Zwilling,G.Klien,D.Lesjak,Ł.Wiechetek,F.Cetin,H.N.Basim, Cyber security awareness, knowledge and behavior: A comparative study, Journal of Computer Information Systems 62 (2022) 82–97
2022
-
[29]
L. A. Tawalbeh, F. Muheidat, Factors that motivate defense against social engineering attacks across organizations, Procedia Computer Science 224 (2023) 75–82
2023
-
[30]
Parsons, D
K. Parsons, D. Calic, M. Pattinson, M. Butavicius, A. McCormac, T. Zwaans, The development of the Human Aspects of Information Security Questionnaire (HAIS-Q): Testing reliability and validity, Computers & Security 69 (2017) 506–517
2017
-
[31]
Furnham, Personality and learning style: A study of three instru- ments, Personality and Individual Differences 13 (1992) 429–438
A. Furnham, Personality and learning style: A study of three instru- ments, Personality and Individual Differences 13 (1992) 429–438
1992
-
[32]
Komarraju, S
M. Komarraju, S. J. Karau, R. R. Schmeck, A. Avdic, The Big Five personality traits, learning styles, and academic achievement, Personality and Individual Differences 51 (2011) 472–477
2011
-
[33]
doi:10.1109/ICCSE.2017.8085483
J.Du,etal.,Ananalysisofinfluencefactorsforacademicperformance about personality traits and thinking styles of students, in: 2017 12th International Conference on Computer Science and Education (ICCSE), 2017. doi:10.1109/ICCSE.2017.8085483
-
[34]
Halevi, J
T. Halevi, J. Lewis, N. Memon, Spear-phishing in the wild: A real- worldstudy ofpersonality,phishing self-efficacyandvulnerability to spear-phishing attacks, Social Science Research Network (2013)
2013
-
[35]
Rammstedt, O
B. Rammstedt, O. P. John, Measuring personality in one minute or less:A10-itemshortversionoftheBigFiveInventoryinEnglishand German, Journal of Research in Personality 41 (2007) 203–212
2007
-
[36]
D. Papatsaroucha, Y. Nikoloudakis, I. Kefaloukos, E. Pallis, E. K. Markakis, A survey on human and personality vulnerability assess- ment in cyber-security: Challenges, approaches, and open issues, 2021.arXiv:2106.11625
-
[37]
Alseadoon, M
I. Alseadoon, M. Othman, S. Tang, Who is more susceptible to phishing emails? a Saudi Arabian study, Jurnal Teknologi 64 (2013)
2013
-
[38]
Roberts, C
B. Roberts, C. Lejuez, R. Krueger, J. Richards, P. Hill, What is conscientiousness and how can it be assessed?, Developmental Psychology 50 (2012) 1315–1330
2012
-
[39]
Bansal, F
G. Bansal, F. Zahedi, D. Gefen, The impact of personal dispositions on information sensitivity, privacy concern and trust in disclosing healthinformationonline, DecisionSupportSystems49(2010)138– 150
2010
-
[40]
S.Lai,etal., Automaticpersonalityidentificationusingstudents’on- line learning behavior, IEEE Transactions on Learning Technologies 13 (2020) 26–37
2020
-
[41]
F. Giannakas, G. Kambourakis, S. Gritzalis, CyberAware: A mobile game-based app for cybersecurity education and awareness, in: 2015InternationalConferenceonInteractiveMobileCommunication Technologies and Learning (IMCL), 2015. doi:10.1109/IMCTL.2015. 7359553
-
[42]
S. Sudha, et al., Impact of smartphone-based interactive learning modules on cybersecurity learning at the high-school level, in: 2023 IEEE Global Engineering Education Conference (EDUCON), 2023. doi:10.1109/EDUCON54358.2023.10125124
-
[43]
O.P.John,E.M.Donahue,R.Kentle,BigFiveInventory(BFI),1991. doi:10.1037/t07550-000
-
[44]
Rammstedt, C
B. Rammstedt, C. Beierlein, Can’t we make it any shorter? the limits of personality assessment and ways to overcome them, Journal of Individual Differences 35 (2014) 212–220
2014
-
[45]
Ahmad, Analysis of cross-platform mobile application develop- mentframeworks,InternationalJournalofInnovativeTechnologyand Exploring Engineering (2023)
M. Ahmad, Analysis of cross-platform mobile application develop- mentframeworks,InternationalJournalofInnovativeTechnologyand Exploring Engineering (2023)
2023
-
[46]
FlutterFlow,FlutterFlow:Buildhighquality,customisedappsquickly,
-
[47]
URL:https://www.flutterflow.io/, [Online; accessed 2025- 06-17]
2025
-
[48]
URL:https://firebase.google
Google, Firestore | firebase, 2025. URL:https://firebase.google. com/docs/firestore, [Online; accessed 2025-06-17]
2025
-
[49]
Hidayanti, R
P. Hidayanti, R. Handayani, B. Rifai, UI/UX design of online tickets for situ pasir maung tourism using the Figma application, SinkrOn 8 (2023) 1051–1063
2023
-
[50]
I. Blau, O. Weiser, Y. Eshet-Alkalai, Face-to-face versus one-way and two-way videoconferencing: How medium naturalness and per- sonalitytraitsinfluenceachievementandperceivedlearning, in:2016 11th Iberian Conference on Information Systems and Technologies (CISTI), 2016. doi:10.1109/CISTI.2016.7521581
-
[51]
Isoaho, P
J. Isoaho, P. Nikander, Cybersecurity education: Bridging the gap between theory and practice, IEEE Security & Privacy 19 (2021) 70–76. A. Consent Notification Text The following text was displayed on the application’s onboarding screen prior to commencement of the pre- assessment: Dear Participant, Thank you for considering participating in this survey. ...
2021
-
[52]
(Extraversion, R)
is reserved. (Extraversion, R)
-
[53]
(Agreeableness)
is generally trusting. (Agreeableness)
-
[54]
(Conscientiousness, R)
tends to be lazy. (Conscientiousness, R)
-
[55]
(Neuroticism, R)
is relaxed, handles stress well. (Neuroticism, R)
-
[56]
(Openness, R)
has few artistic interests. (Openness, R)
-
[57]
(Extraversion)
is outgoing, sociable. (Extraversion)
-
[58]
(Agreeableness, R)
tends to find fault with others. (Agreeableness, R)
-
[59]
(Conscientiousness)
does a thorough job. (Conscientiousness)
-
[60]
(Neuroticism)
gets nervous easily. (Neuroticism)
-
[61]
Employee Bonus Details
has an active imagination. (Openness) C. Assessment Quiz Questions Pre-Assessment Scenarios (four items) Q1.You find a USB drive in your mailbox with a note stating it contains sensitive business data your company needs. What should you do? a. Immediately plug the USB drive into your computer to access the data b.Inform your company’s IT department about ...
discussion (0)
Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.