REVIEW 4 major objections 5 minor 1 cited by
Talking Like a Phisher: LLM-Based Attacks on Voice Phishing Classifiers
T0 review · 4 major / 5 minor · reviewed 2026-08-06 · deepseek-v4-flash
Pith's one-line read This paper reports that commercial LLMs can rewrite vishing transcripts so that ML-based detectors misclassify them as benign, with GPT-4o lowering accuracy by up to 30.96% while BERTScore stays high.
desk verdict The threat model is real and the question is timely, but the headline accuracy drops are computed on mismatched test sets, so the central quantitative claim doesn't hold as stated. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The load-bearing mechanism is the attack pipeline specified in Algorithm 1: a combined prompt, $P_{combined} = P_{rephrase} \oplus P_{noise}$, instructs the LLM to rephrase the scammer speech and insert as many filler sentences as possible built from a list of benign Korean words, while avoiding suspicious keywords like account, card, or bank. The generated transcript is then cleaned, tokenized with the MeCab-ko morphological analyzer, converted to TF-IDF feature vectors, and scored by ten classical ML classifiers (logistic regression, decision tree, random forest, AdaBoost, gradient boosting, histogram gradient boosting, XGBoost, LightGBM, CatBoost, and linear SVM). The paper's explanation for why the evasion works is that TF-IDF classifiers rely on token occurrence patterns, uniqueness, and transcript length, and both rephrasing and benign injection alter those statistical features. Statistical testing (Wilcoxon signed-rank, Friedman, Nemenyi) and BERTScore complete the pipeline, respectively asserting that the drops are consistent across classifiers and that the original meaning is preserved.
What would settle it
Re-run the evaluation on the same balanced test set for both original and adversarial transcripts and compare per-class false-positive and false-negative rates. If, on a balanced set, the accuracy drops shrink to near zero or the Wilcoxon test is no longer significant, the claimed evasion effect as measured does not hold.
Extended reading notes
Core claim
The paper's central claim is that LLM-generated adversarial transcripts are both practically and statistically effective against ML-based vishing classifiers. Using a prompt that combines rephrasing (to reduce linguistic similarity to scam patterns) with benign-context injection (to dilute scam indicators inside a friendly conversation), the authors generate adversarial versions of real vishing scripts with GPT-4o, GPT-4o mini, Gemini 2.0, and Qwen2.5. They report that every tested LLM produces a statistically significant accuracy drop across the ten classifiers (Wilcoxon one-tailed p-values below 0.05), and that GPT-4o offers the best trade-off between evasion and semantic preservation, with BERTScore F1 values around 0.72 to 0.75 and an accuracy drop of up to 30.96% on the full vishing set. The paper also claims that these attacks are economically and computationally cheap, and that none of the LLMs triggered content filters or refused the request.
Load-bearing premise
The central claim assumes the original and adversarial accuracies are measured on comparable evaluation sets with the same class balance; the paper uses a balanced 244-sample test set for original accuracy but a vishing-only 146-sample or 609-sample set for adversarial accuracy, so the reported drops and significance tests may not measure what they claim.
Editorial extensions
If this is right
- If the central claim holds, any deployment that classifies vishing solely from TF-IDF features of transcripts can be degraded by LLM-rewritten scripts without the attacker needing to modify the call content.
- The sub-cent cost and sub-nine-second generation time mean the attack is scalable enough for bulk campaigns, not just isolated calls.
- Because GPT-4o keeps BERTScore high while evading classifiers, the threat is not only detection bypass: the rewritten scripts remain plausible enough to be used in real calls.
- The reported Wilcoxon, Friedman, and Nemenyi results imply the evasion effect is consistent across an ensemble of ten classifiers, so retraining on one classifier alone may not be sufficient.
- All four tested LLMs complied with the malicious prompt, implying that current commercial guardrails do not block this particular abuse.
Reading between the lines
- The adversarial accuracy numbers in Tables 2 and 3 are computed only on vishing transcripts (146 and 609 samples respectively), while the original accuracies are on a balanced 244-sample test set, so the reported accuracy drops mix evasion with the absence of benign samples; a balanced re-evaluation is needed to confirm the size of the evasion effect.
- A natural extension would test the same pipeline against transformer-based detectors (for example, fine-tuned BERT classifiers) and against classifiers that use acoustic or caller-reputation features; the paper only evaluates TF-IDF plus classical ML models, and those alternative detectors may be more robust.
- The prompt explicitly forbids suspicious keywords and demands euphemisms, which suggests the attack targets lexical surface patterns; probing refusal behavior with safety-tuned LLMs or system-level guardrails could show whether this pipeline can be blocked without losing legitimate security-training uses.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper proposes and evaluates an LLM-based attack pipeline that rewrites real-world Korean vishing transcripts using four commercial LLMs (GPT-4o, GPT-4o mini, Gemini 2.0, Qwen2.5) and tests whether the rewritten transcripts evade a set of ten classical ML classifiers trained on the KorCCViD dataset. The central claim is that the rewrites substantially reduce classifier accuracy---by up to 30.96% for GPT-4o on the full vishing set---while preserving semantic content as measured by BERTScore, and that these attacks are statistically significant, cheap, and fast. The authors also provide a threat model, a prompt-engineering strategy, a cost/time analysis, and a single worked transcript example.
Significance. If the quantitative claims were supported, the paper would be a practically relevant demonstration that cheap commercial-LLM rewrites can evade current transcript-based vishing classifiers while preserving deceptive intent. The paper has real strengths: it uses a real-world Korean vishing corpus, evaluates four commercial LLMs, includes BERTScore-based semantic checks, reports cost and latency, and gives a concrete worked example. The qualitative direction---that LLM rewrites degrade the recall of TF-IDF-based classifiers---is plausible. However, the headline accuracy drops and the statistical significance claims are not supported by the reported numbers because the original and adversarial accuracies are measured on different evaluation populations. The paper also does not release code or generated transcripts, which limits auditability.
major comments (4)
- [§5.3, Eq. (4); Tables 2 and 3] The accuracy drop defined in Eq. (4) is not computed on comparable evaluation sets. Table 1 reports original accuracy on the 244-sample balanced test set described in §5.1, while the adversarial accuracies in Table 2 are computed on a vishing-only set whose numerical values imply a denominator of 146 (and whose caption says 100), and the adversarial accuracies in Table 3 are computed on the 609 vishing-only transcripts. On a one-class set, "accuracy" equals recall, so each reported drop mixes a balanced-set accuracy with a vishing-only recall; the observed drops are therefore partly a class-balance artifact and do not measure the attack's effect on a deployed mixed-traffic classifier. The experiments should be re-run on the same 244-instance test set (with adversarial rewrites replacing the vishing test inputs and the benign inputs unchanged), or the claims should be restated as recall drops on vishing-only data.
- [§6.1, Table 2] The Wilcoxon signed-rank test is applied to ten classifier-level accuracy pairs drawn from different evaluation sets, so it does not support the abstract's claim that the attacks are "statistically effective" at the transcript level. In addition, for MiniGPT-4o two classifiers (HistGradientBoosting and LGBM) have adversarial accuracy higher than original accuracy; the reported one-tailed p-value of 0.0098 is not consistent with the exact signed-rank distribution for n=10 and W=14 (one-sided p is approximately 0.097). A paired transcript-level test such as McNemar's test on the same evaluation instances should replace the classifier-level test, and the p-values should be recomputed.
- [§6.2, Table 3 and Fig. 5] The headline 30.96% accuracy drop for GradientBoostingClassifier is the difference between a balanced-set accuracy (0.954918) and a one-class recall (0.645320 on 609 vishing transcripts); the same confounding affects every row. The ROC/AUC comparison in Fig. 5 is also between different populations, so the reported AUC declines (e.g., DecisionTree from 0.99 to 0.87) do not establish a like-for-like degradation. These claims should be re-derived from a common test set.
- [Algorithm 1, Phase 4] The pseudocode is ambiguous about whether `D_original` and `D_adversarial` are the same evaluation instances: line 16 evaluates the original accuracy on the balanced test set, while line 17 evaluates on an unspecified adversarial set. The definitions of these datasets must be made explicit and identical in composition except for the adversarial rewriting.
minor comments (5)
- [Table 2 caption] The caption states "100 vishing samples" but the reported adversarial accuracies are consistent with a denominator of 146; this inconsistency must be resolved.
- [§6.3, Table 4] The case study is described as demonstrating why the attack works; a single transcript is illustrative, not demonstrative, and the wording should be softened accordingly.
- [Fig. 2] The prompt contains a typo ("addtional") and the Korean word list appears as "??" placeholders; these should be corrected.
- [§6.1, Fig. 4] The claim that GPT-4o maintains BERTScore precision, recall, and F1 values ranging from 0.72 to 0.75 should be accompanied by distribution statistics and sample sizes rather than only density plots.
- [Reproducibility] The paper does not state whether the code or generated transcripts will be released; given that the adversarial outputs are the core evidence, a release statement or supplement would aid auditability.
Circularity Check
No circularity found: the central claim is an external empirical measurement, not a derivation from fitted parameters or load-bearing self-citations.
full rationale
The paper's central claim is an empirical result: LLM-rewritten vishing transcripts lower the accuracy of ten ML classifiers trained on KorCCViD. This is measured against an externally sourced transcript dataset and standard classifiers, not derived from a fitted parameter or from the authors' prior work. The accuracy-drop definition in Eq. (4) directly compares measured accuracies; no quantity is defined in terms of the claimed outcome. Self-citations ([2], [3], [13]) appear only in related-work and survey context and do not carry the argument. The attack prompt, BERTScore evaluation, and statistical tests are all external measurements. A separate validity concern, not a circularity, is that Tables 2 and 3 compute adversarial accuracy on vishing-only subsets while 'Original Acc.' comes from the balanced 244-sample test set, so the reported drop mixes denominators; this affects whether the drop is a like-for-like accuracy comparison, but it does not make the result true by construction.
Assumptions & free parameters
assumptions (5)
- domain assumption KorCCViD v1.3 transcripts and labels are accurate ground truth for real-world Korean vishing.
- domain assumption TF-IDF features plus the ten listed classical ML classifiers represent deployed vishing detection.
- domain assumption BERTScore is a valid proxy for preservation of malicious intent.
- ad hoc to paper Original and adversarial accuracies are computed on comparable evaluation sets.
- domain assumption Commercial LLM outputs observed in the experiments generalize over time and across API versions and do not trigger safety refusals.
Cite this review
Pith. "Pith review of Talking Like a Phisher: LLM-Based Attacks on Voice Phishing Classifiers." pith.science (2026). https://pith.science/paper/WJSHANAM
@misc{pith2026250716291,
author = {Pith},
title = {Pith review of: Talking Like a Phisher: LLM-Based Attacks on Voice Phishing Classifiers},
year = {2026},
howpublished = {\url{https://pith.science/paper/WJSHANAM}},
note = {Machine review of arXiv:2507.16291}
}
read the original abstract
Voice phishing (vishing) remains a persistent threat in cybersecurity, exploiting human trust through persuasive speech. While machine learning (ML)-based classifiers have shown promise in detecting malicious call transcripts, they remain vulnerable to adversarial manipulations that preserve semantic content. In this study, we explore a novel attack vector where large language models (LLMs) are leveraged to generate adversarial vishing transcripts that evade detection while maintaining deceptive intent. We construct a systematic attack pipeline that employs prompt engineering and semantic obfuscation to transform real-world vishing scripts using four commercial LLMs. The generated transcripts are evaluated against multiple ML classifiers trained on a real-world Korean vishing dataset (KorCCViD) with statistical testing. Our experiments reveal that LLM-generated transcripts are both practically and statistically effective against ML-based classifiers. In particular, transcripts crafted by GPT-4o significantly reduce classifier accuracy (by up to 30.96%) while maintaining high semantic similarity, as measured by BERTScore. Moreover, these attacks are both time-efficient and cost-effective, with average generation times under 9 seconds and negligible financial cost per query. The results underscore the pressing need for more resilient vishing detection frameworks and highlight the imperative for LLM providers to enforce stronger safeguards against prompt misuse in adversarial social engineering contexts.
Figures
Forward citations
Cited by 1 Pith paper
-
Evaluating AI Models' Capability to Automate Voice Phishing Attacks
AI voice models already drive self-reported vishing compliance up to 36% and make automated attacks economically viable at U.S. scale while human operators are not.
Reference graph
Works this paper leans on
-
[1]
Pujara, P., Chaudhari, M.: Phishing website detection using machine learning: a review. International Journal of Scientific Research in Computer Science, Engi- neering and Information Technology 3(7), 395–399 (2018)
work page 2018
-
[2]
IEEE Access 12, 191586– 191602 (2024)
Li, W., Ul Arfeen Laghari, S., Manickam, S., Chong, Y.W., Li, B.: Machine learning-enabled attacks on anti-phishing blacklists. IEEE Access 12, 191586– 191602 (2024). https://doi.org/10.1109/ACCESS.2024.3516754
-
[3]
LI, W., LAGHARI, S.U.A., MANICKAM, S., and, Y.W.C.: Exploration and eval- uation of human-centric cloaking techniques in phishing websites. KSII Trans- Talking Like a Phisher - Accepted by ICDF2C 2025 17 actions on Internet and Information Systems 19(1), 232–258 (January 2025). https://doi.org/10.3837/tiis.2025.01.011
-
[4]
European Journal of Information Systems 34(3), 460–474 (2025)
Tian, C.A., Jensen, M.L., Bott, G., and, X.R.L.: The influence of affective process- ing on phishing susceptibility. European Journal of Information Systems 34(3), 460–474 (2025). https://doi.org/10.1080/0960085X.2024.2351442
-
[5]
The Jour- nal of the Korea Contents Association 12(7), 176–182 (2012)
Cho, H.D.: Voice phishing occurrence and counterplan. The Jour- nal of the Korea Contents Association 12(7), 176–182 (2012). https://doi.org/10.5392/JKCA.2012.12.07.176
-
[6]
Se- curity Journal 30, 454–466 (2017)
Choi, K., Lee, J.l., Chun, Y.t.: Voice phishing fraud and its modus operandi. Se- curity Journal 30, 454–466 (2017). https://doi.org/10.1057/sj.2014.49
-
[7]
Ray, A., Saha, S., Chakrabarty, K., Collins, L., Lafata, K., Emami-Naeini, P.: Exploring the impact of ethnicity on susceptibility to voice phishing, https://www.usenix.org/system/files/soups2023-poster85_ray_ abstract_final.pdf
-
[8]
IEEE Access 13, 37946–37965 (2025)
Boussougou, M.K.M., Hamandawana, P., Park, D.J.: Enhancing voice phishing detection using multilingual back-translation and smote: An empirical study. IEEE Access 13, 37946–37965 (2025). https://doi.org/10.1109/ACCESS.2025.3545250
Show all 30 references
-
[9]
ACM Comput
Goyal, S., Doddapaneni, S., Khapra, M.M., Ravindran, B.: A survey of adver- sarial defenses and robustness in nlp. ACM Comput. Surv. 55(14s) (Jul 2023). https://doi.org/10.1145/3593042
2023 doi
-
[10]
Gallagher, S., Gelman, B., Taoufiq, S., V¨ or¨ os, T., Lee, Y., Kyadige, A., Bergeron, S.: Phishing and Social Engineering in the Age of LLMs, pp. 81–86. Springer Nature Switzerland, Cham (2024). https://doi.org/10.1007/978-3-031-54827-7 8
2024 doi
-
[11]
Electronics 12(21) (2023)
Thakur, K., Ali, M.L., Obaidat, M.A., Kamruzzaman, A.: A systematic re- view on deep-learning-based phishing email detection. Electronics 12(21) (2023). https://doi.org/10.3390/electronics12214545
2023 doi
-
[12]
European Journal of Applied Science, Engineering and Technology 3(2), 41–54 (Mar 2025)
Chinta, P.C.R., Moore, C.S., Karaka, L.M., Sakuru, M., Bodepudi, V., Maka, S.R.: Building an intelligent phishing email detection system using machine learning and feature engineering. European Journal of Applied Science, Engineering and Technology 3(2), 41–54 (Mar 2025). http...
2025 doi
-
[13]
IEEE Access 12, 187976–188012 (2024)
Li, W., Manickam, S., Chong, Y.W., Leng, W., Nanda, P.: A state-of-the-art review on phishing website detection techniques. IEEE Access 12, 187976–188012 (2024). https://doi.org/10.1109/ACCESS.2024.3514972
2024
-
[14]
In: 2024 11th International Conference on Com- puting for Sustainable Global Development (INDIACom)
Kawale, M., Maru, B., Dagu, S., Varghese, M., Gupta, V.: Machine learning based phishing website detection. In: 2024 11th International Conference on Com- puting for Sustainable Global Development (INDIACom). pp. 833–837 (2024). https://doi.org/10.23919/INDIACom61295.2024.10498854
2024
-
[15]
In: 2024 1st International Conference on Innova- tive Engineering Sciences and Technological Research (ICIESTR)
Abdul Samad, S.R., Ganesan, P., S, T., Balasubramaniyan, S., Rajiakodi, S., Rashid Al Kaabi, A.S.: Sms-shield: A lightweight approach for smishing detec- tion using machine learning. In: 2024 1st International Conference on Innova- tive Engineering Sciences and Technological R...
2024
-
[16]
Procedia Computer Sci- ence 244, 248–259 (2024)
Saidat, M.R.A., Yerima, S.Y., Shaalan, K.: Advancements of sms spam detec- tion: A comprehensive survey of nlp and ml techniques. Procedia Computer Sci- ence 244, 248–259 (2024). https://doi.org/10.1016/j.procs.2024.10.198, 6th Inter- national Conference on AI in Computational...
2024 doi
-
[17]
Journal of Ambient Intelligence and Humanized Computing 14(7), 8173–8184 (2023)
Lee, M., Park, E.: Real-time korean voice phishing detection based on machine learning approaches. Journal of Ambient Intelligence and Humanized Computing 14(7), 8173–8184 (2023). https://doi.org/10.1007/s12652-021-03587-x 18 W. Li et al
2023 doi
-
[18]
In: 2024 8th Cy- ber Security in Networking Conference (CSNet)
Phang, Z.H., Tan, W.M., Xiong Choo, J.S., Ong, Z.K., Isaac Tan, W.H., Guo, H.: Vishguard: Defending against vishing. In: 2024 8th Cy- ber Security in Networking Conference (CSNet). pp. 108–115 (2024). https://doi.org/10.1109/CSNet64211.2024.10851764
2024
-
[19]
In: Proceedings of the 2021 ACM Workshop on Security and Privacy Analytics
Derakhshan, A., Harris, I.G., Behzadi, M.: Detecting telephone-based so- cial engineering attacks using scam signatures. In: Proceedings of the 2021 ACM Workshop on Security and Privacy Analytics. p. 67–73. IWSPA ’21, Association for Computing Machinery, New York, NY, USA (202...
2021
-
[20]
Human-centric Computing and Information Sciences 11(2) (2021)
Kim, J.W., Hong, G.W., Chang, H.: Voice recognition and document classification- based data analysis for voice phishing detection. Human-centric Computing and Information Sciences 11(2) (2021). https://doi.org/10.22967/HCIS.2021.11.002
2021 doi
-
[21]
arXiv preprint arXiv:2110.13980 (2021)
Alsmadi, I., Ahmad, K., Nazzal, M., Alam, F., Al-Fuqaha, A., Khreishah, A., Algosaibi, A.: Adversarial attacks and defenses for social network text processing applications: Techniques, challenges and future research directions. arXiv preprint arXiv:2110.13980 (2021)
2021 arXiv
-
[22]
arXiv preprint arXiv:1907.11932 (2019)
Jin, D., Jin, Z., Zhou, J.T., Szolovits, P.: Is bert really robust? natural language at- tack on text classification and entailment. arXiv preprint arXiv:1907.11932 (2019)
2019 arXiv
-
[23]
arXiv preprint arXiv:2410.14569 (2024)
Kim, H., Song, M., Na, S.H., Shin, S., Lee, K.: When llms go online: The emerging threat of web-enabled llms. arXiv preprint arXiv:2410.14569 (2024)
2024 arXiv
-
[24]
In: 2024 IEEE Symposium on Security and Privacy (SP)
Roy, S.S., Thota, P., Naragam, K.V., Nilizadeh, S.: From chatbots to phish- bots?: Phishing scam generation in commercial large language models. In: 2024 IEEE Symposium on Security and Privacy (SP). pp. 36–54 (2024). https://doi.org/10.1109/SP54263.2024.00182
2024
-
[25]
In: 2024 ASU International Conference in Emerging Technologies for Sus- tainability and Intelligent Systems (ICETSIS)
Alotaibi, L., Seher, S., Mohammad, N.: Cyberattacks using chatgpt: Ex- ploring malicious content generation through prompt engineering. In: 2024 ASU International Conference in Emerging Technologies for Sus- tainability and Intelligent Systems (ICETSIS). pp. 1304–1311 (2024). ...
2024
-
[26]
https: //taku910.github.io/mecab/ (2005), accessed: 2025-05-25
Kudo, T.: Mecab: Yet another part-of-speech and morphological analyzer. https: //taku910.github.io/mecab/ (2005), accessed: 2025-05-25
2005
-
[27]
Mathematics 11, 3217 (07 2023)
Moussavou Boussougou, M.K., Park, D.J.: Attention-based 1d cnn-bilstm hybrid model enhanced with fasttext word embedding for korean voice phishing detection. Mathematics 11, 3217 (07 2023). https://doi.org/10.3390/math11143217
2023 doi
-
[28]
OpenAI: Gpt-4 technical report (2023), https://arxiv.org/abs/2303. 08774
2023
-
[29]
DeepMind, G.: Gemini: Advanced general-purpose ai (2023), https:// deepmind.google/gemini
2023
-
[30]
Bai, C., et al.: Qwen technical report (2023), https://arxiv.org/abs/2309. 16609
2023
Reviewed August 6, 2026 · model on record in the stance chip above.
Discussion (0). Sign in to comment.