{"id":"fa55a261-b148-4c38-af72-9c1356393e77","arxiv_id":"2411.13874","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":4.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":4,"one_line_summary":"LLM-rephrased phishing emails evade current email detectors more often than original ones, and training on LLM-generated variants partly restores detection.","lead":"The paper measures how well email phishing detectors catch emails that large language models rewrote to sound harmless. Across Gmail, SpamAssassin, Proofpoint, and three machine learning models, detection accuracy drops for rephrased emails, and the authors propose training on LLM-augmented data to recover some of the lost accuracy.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The headline recall drops are arithmetically impossible: Table I/III keep original TP counts for rephrased emails while increasing FN, so TP+FN exceeds the number of phishing emails and all reported declines are unreliable.","rationale":"The reader's weakest assumption was that rephrased emails preserve phishing intent. That is a valid secondary concern, but the more load-bearing problem is that the reported results are internally inconsistent: in every table, TP is carried over from the original emails while FN increases, making TP+FN exceed the total number of phishing emails. The reported recall values exactly match a formula that divides the original TP by the sum of original TP and rephrased FN, which is not a real recall on any single corpus. This means the headline declines—the paper's main contribution—are not grounded in valid measurements. Since the direction of the effect is plausible and likely survives correction, the appropriate disposition remains conditional rather than outright rejection: the authors should re-run and correct the tables, then re-evaluate whether the quantitative findings still hold. No ad hominem is intended; this is a technical flaw in the reported metrics that must be fixed before the central claim can be trusted.","tokens_in":12596,"tokens_out":4609,"duration_ms":51599,"concrete_test":"Recompute Table I few-shot rows from raw per-email detector outputs, without reusing original TP counts: for each detector, count TP and FN only among the 600 few-shot rephrased Nazario emails, verify TP+FN=600, and recalculate recall, precision, accuracy, and F1. Repeat for zero-shot rows and for Table III on the Nigerian Fraud dataset (400 phishing emails). If the corrected recall differs from the published value for any detector (e.g., Gmail corrected recall is 80.5% rather than 83.04% if TP=483), the reported performance declines are invalid and all tables need regeneration before the central claim can be assessed.","verdict_should_be":"CONDITIONAL","load_bearing_attack":"The paper's central claim is that LLM-rephrased phishing emails evade detectors, with recall dropping from 98.12% to 83.04% for Gmail and from 94.43% to 75.44% for Naive Bayes (Table I). But the underlying counts cannot be correct. In Table I, for every detector, the TP column is identical across original, zero-shot, and few-shot rows, while FN increases. For Gmail few-shot, TP=573 and FN=117, giving TP+FN=690 against only 600 phishing emails in the dataset. The same impossibility appears in zero-shot rows (573+41=614) and across all detectors, including Table III (e.g., Gmail zero-shot on Nigerian Fraud: 387+49=436 vs. 400 phishing emails). The reported few-shot recall values equal TP_original/(TP_original+FN_rephrased), e.g., 573/(573+117)=83.04% for Gmail and 559/(559+182)=75.44% for Naive Bayes. This indicates the recalls are computed by mixing the original TP counts with the rephrased FN counts instead of recomputing TP on the rephrased corpus. If the actual TP among rephrased emails were, say, 483 for Gmail few-shot (since FN=117), the true recall would be 80.5%, not 83.04%; similarly Naive Bayes true recall would be 69.7%, not 75.44%. The direction of the decline may survive correction, but the magnitude and all precise numbers are unverified. Because the empirical claim is built entirely on these tables, the study cannot be accepted as quantitatively reliable without re-running the evaluations and correcting every metric.","agreement_with_reader":"disagree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper evaluates how well traditional phishing detectors (Gmail, SpamAssassin, Proofpoint), machine learning classifiers (Naive Bayes, SVM, Logistic Regression), and five LLMs detect phishing emails after the emails are rephrased by GPT-4o and Llama 3 using zero-shot and few-shot prompting. It reports notable recall declines on rephrased emails and proposes using LLM-rephrased emails as data augmentation to improve phishing detection. The authors introduce a new dataset, LLM-Nazario, for this purpose.","tokens_in":12950,"tokens_out":4448,"duration_ms":45506,"significance":"The research question is timely and practically important: if LLM-rephrased phishing emails systematically evade current detectors, this is a serious security concern, and the proposed data-augmentation direction could be valuable for defense. The paper also covers a broad set of detectors and two phishing corpora, which is commendable. However, the quantitative evidence is undermined by fundamental arithmetic inconsistencies in the confusion-matrix tables, a missing verification that rephrased emails preserve the phishing payload, and a circular data-augmentation evaluation. Because the central empirical claim rests entirely on the flawed tables, the results as presented are not reliable. If corrected and validated, the paper could make a meaningful contribution, but in its current form the findings do not support the conclusions.","major_comments":[{"comment":"The confusion-matrix counts are arithmetically impossible. The paper states that the Nazario test set contains 600 phishing and 600 legitimate emails, so TP+FN must equal 600 for any fixed test set. Yet the few-shot Gmail row reports TP=573 and FN=117, giving TP+FN=690, and the zero-shot row gives 614; the same problem appears in Table III (e.g., Gmail zero-shot on Nigerian Fraud gives TP+FN=436 versus the stated 400 phishing emails). The reported recalls such as 83.04% equal TP_original/(TP_original+FN_rephrased), which mixes counts from different evaluation corpora. All metrics derived from these tables are therefore invalid, and the paper's central claim that detection accuracy declines on rephrased emails is unsupported by the reported data.","section":"Section IV-A, Table I"},{"comment":"The paper does not verify that the LLM-rephrased emails preserve the phishing payload. The zero-shot prompt instructs the model to \"ask for the same information the original email is asking for,\" but there is no check that the malicious link, credential request, or harmful instruction survives rewriting. If the rewrites drop the payload, then the observed detection gap measures style change rather than a real security risk. The authors should provide evidence—either manual inspection or automated checks—that the rephrased emails still constitute phishing attempts before claiming a security-relevant evasion.","section":"Section III-C"},{"comment":"The data-augmentation evaluation is circular. The models are trained on GPT-4- and Llama-3-rephrased emails and then tested on LLM-rephrased emails generated by the same model family, leading to improvements such as SVM's 12.86% accuracy gain. With no held-out distribution or cross-model testing, the observed gains may simply reflect train/test overlap in wording and style, not improved generalization to novel LLM-generated attacks. A valid evaluation should test on rephrased emails from a different LLM family or on independently generated phishing variants.","section":"Section IV-C and Figure 5"},{"comment":"The experimental setup lacks essential statistical and methodological detail. The paper reports a majority vote over three iterations but gives no variance, confidence intervals, or significance testing, which is insufficient for a claim based on differences between 98.12% and 83.04% recall. In addition, Section IV-B states that machine learning models were trained on 1500 emails from the original datasets, which is inconsistent with the dataset sizes described in Section III-A (1200 for Nazario, 800 for Nigerian Fraud); the train/test split and any overlap between training and evaluation subsets are not clearly described. These omissions prevent the reader from assessing the reliability of the reported results.","section":"Section III-B and Section IV-B"}],"minor_comments":[{"comment":"The caption of Table II says \"Nigerian Fraud Dataset,\" but Section IV-A describes Table II as showing LLM performance on the Nazario dataset; this inconsistency should be corrected.","section":"Table II caption"},{"comment":"References [34] and [35] are the same paper (Kojima et al.) and should be merged.","section":"References"},{"comment":"The Naive Bayes equation is written with a single word w_i as the conditioning event, but the preceding text indicates the probability is conditioned on a word's presence; the notation should be clarified. The logistic regression formula also introduces w and x without explicitly defining the feature vector for an email.","section":"Section IV"},{"comment":"There are several typos and grammar errors, including \"sophisticated models has already outperformed\" (Abstract/Introduction), \"the the emails\" (Section III-B), and \"significantl\" (Conclusion). A careful language edit is needed.","section":"Whole paper"}],"recommendation":"major_revision","confidential_remarks":"The paper's headline numbers are not credible as reported; even the original rows in Table I do not sum to the stated class counts (e.g., Gmail original TP+FN=584, not 600). The authors need to re-run all evaluations and recompute every metric from consistent confusion matrices. The data-augmentation section also needs a more rigorous experimental design with held-out generators. I would not recommend acceptance in the current form, but the research direction is sound and a corrected version could be worth reviewing."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"The headline results are arithmetically impossible. In Table I, every detector shows the same TP for original, zero-shot, and few-shot rows while FN increases, so TP+FN exceeds the 600 phishing emails in the Nazario dataset. For example, Gmail few-shot has TP=573, FN=117, summing to 690. The same pattern appears in Table III and in the LLM tables. The recall values are computed by dividing the original TP by the inflated TP+FN, so they are not real measurements on the rephrased corpus. That is a load-bearing flaw: the paper's central quantitative claim rests on these tables.\n\nWhat the paper does well is scope: it tests a broader set of detectors than prior work, including Gmail, SpamAssassin, Proofpoint, three ML models, and five LLMs, and it introduces the LLM-Nazario augmentation dataset. The idea of using LLM-generated variants for training is reasonable and worth exploring. The zero-shot and few-shot prompts are described, which helps reproducibility.\n\nThe soft spots are not minor. Besides the arithmetic contradiction, there is no verification that the rephrased emails preserve the phishing payload; the prompt asks for the same information, but the paper never checks that the link or credential request survives. The data augmentation experiment likely has train/test overlap since both are drawn from the same LLM family, so the accuracy gains may be inflated. There are no error bars or statistical tests, and the ML training split is vague. Also, Table II is captioned as Nigerian Fraud but appears to report Nazario results.\n\nThe direction of the result, that LLM rephrasing reduces detector recall, is plausible and already supported by the cited work of Heiding et al. and Hazell. So the paper is not conceptually wrong, but this version cannot be trusted quantitatively. I would not cite its specific numbers. I would send it to review only with the expectation that the authors re-run all evaluations and correct the metrics; the current text is not ready. The dataset, if released, might be a useful resource, but I would not count on it.","headline":"The paper's core numbers don't add up: TP+FN exceeds the total number of phishing emails, so the headline recall drops are internally inconsistent, even though the general direction matches prior work.","tokens_in":624,"tokens_out":840,"would_cite":false,"duration_ms":29946,"reading_group":"maybe","serious_thinker":"no","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"The paper claims that current phishing defenses—commercial filters, classical machine-learning classifiers, and LLM-based detectors—lose substantial accuracy against phishing emails rewritten by LLMs, and that retraining on LLM-augmented…","keywords":["phishing detection","large language models","zero-shot prompting","few-shot prompting","semantic evasion","data augmentation","email security","adversarial rewriting"],"falsifier":"Manually inspect the 600 rephrased Nazario emails (or a random sample) and check whether each still contains a working phishing URL or a concrete request for credentials; recompute the detectors' recall on only those emails whose payload survived, and if the gap narrows to near zero, the paper's central claim would be refuted as an artifact of rewriting.","tokens_in":12406,"feed_emoji":"🎣","tokens_out":8044,"duration_ms":58616,"temperature":0.7,"pith_summary":"This paper tries to establish that current phishing defenses—commercial filters, classic machine-learning classifiers, and even LLM-based detectors—lose significant accuracy when phishing emails are rewritten by an LLM to sound more legitimate. Using GPT-4o with zero-shot and few-shot prompts to rephrase emails from two public datasets, the authors measure recall drops across every detector, with the largest drops on few-shot rephrased emails (e.g., Gmail recall from 98.12% to 83.04% on the Nazario dataset). The paper also claims a remedy: training models on an LLM-augmented dataset of 5,000 emails improves detection of rephrased attacks by roughly 10 to 13 accuracy points. If right, this means simple prompt-based rewriting is enough to lower the bar for phishing attacks, and current defenses need updating or augmentation to keep up.","feed_headline":"Simple LLM rewrites cut recall of every tested detector","feed_subtitle":"Phishing recall drops from 98% to as low as 75%; training on LLM-augmented data partly restores it.","key_machinery":"The machinery is the rephrasing-plus-evaluation pipeline. Zero-shot prompting hands GPT-4o a single instruction to rephrase the email with the same sender, receiver, subject, and body, avoiding urgency, deadlines, generic greetings, and money amounts while still asking for the same information; few-shot prompting adds three crafted examples of rephrased phishing emails. The rephrased emails are then passed through the same detector battery as the originals—Gmail Spam Filter, SpamAssassin, Proofpoint, SVM, logistic regression, naive Bayes, and five LLMs—and the resulting accuracy, precision, recall, and F1 scores are compared. The same LLM-generated outputs are used to construct the LLM-Nazario augmented training set, turning the attack generator into a data-augmentation tool.","core_discovery":"Stated on the paper's own terms, the discovery is that a single prompt-based rewriting step—asking GPT-4o to rephrase a phishing email so it sounds more legitimate while keeping the same core topic—is enough to push phishing emails across the detection boundary of every system tested. On the Nazario dataset, Gmail's recall on phishing falls from 98.12% to 83.04% for few-shot rephrased emails, and naive Bayes falls from 94.43% to 75.44%; the same pattern holds on the Nigerian Fraud dataset, where Gemini's recall drops to 75.64%. The paper attributes this to a shift in vocabulary: suspicious markers like 'urgent' and 'large payment' are replaced by neutral terms like 'account update' and 'credentials', narrowing the decision boundary between phishing and legitimate mail. It then claims that retraining SVM, naive Bayes, and logistic regression on an LLM-augmented dataset (LLM-Nazario, 5,000 emails generated by GPT-4 and Llama 3) improves accuracy on rephrased emails by 12.86, 10.38, and 10.90 percentage points respectively.","pith_inferences":["A direct check the paper leaves undone is verifying that the rewritten emails retain the original phishing payload; if many rewrites drop the link or credential request, the reported recall gap overstates the security risk, and a filtered re-analysis on payload-preserving rewrites would settle this.","The decision-boundary mechanism suggests an arms race: as detectors train on the current set of LLM-rephrased variants, attackers can iterate prompts to generate new neutral-sounding phrasings, so evaluations should include adaptive, closed-loop rewriting rather than a static test set.","The results are based on English text from two public datasets with sender, receiver, subject, and body features; they may not transfer to non-English phishing, image-based emails, or messages where the malicious content lives in attachments or links rather than the body.","If LLM-generated bulk mail were required to carry model provenance metadata, detectors could use that signal—but attackers could simply strip it, so the regulatory value depends on enforcement at the email-service level rather than on content analysis alone."],"forward_implications":["Commercial filters and machine-learning classifiers will miss a larger share of LLM-rephrased phishing than of conventional phishing, so security teams should treat style-based keyword signals as a weak link.","Training the same classical classifiers on LLM-augmented data recovers a meaningful share of the lost accuracy, giving defenders a low-cost update path that does not require changing the model family.","LLM-based detectors also lose recall on rephrased email, which means the detection gap is not specific to keyword-based tools; any detector that relies on surface text is vulnerable to style rewriting.","Because few-shot rewriting consistently produces the largest recall drops, an attacker with only a handful of examples can make phishing emails substantially harder to catch without fine-tuning.","The released LLM-Nazario dataset is intended to serve as a shared resource for fine-tuning and evaluating future phishing detectors."],"supporting_citations":[{"why":"Supplies the Nazario and Nigerian Fraud phishing corpora used for all experiments.","marker":"[28]"},{"why":"Justifies the choice of zero-shot and few-shot prompting as effective for new tasks without training data.","marker":"[6]"},{"why":"Establishes that LLM-generated phishing emails can evade traditional detection, motivating the comparison.","marker":"[20]"},{"why":"Shows LLMs can generate realistic spear-phishing at scale, supporting the threat model.","marker":"[15]"},{"why":"Provides an example of LLM-based phishing detection in use, serving as a comparison point.","marker":"[21]"},{"why":"Provides the classical machine-learning approach that the SVM, logistic regression, and naive Bayes baselines extend.","marker":"[8]"},{"why":"Identifies GPT-4, the model family used for rephrasing in this study.","marker":"[19]"}],"fun_headline_variants":["LLM rephrasing evades every phishing filter tested","Phishing recall crashes to 75% after simple LLM rewrite","One prompt rewrite evades all tested phishing detectors","Naive Bayes drops to 75% on LLM-crafted phishing","LLM phishing bypasses Gmail, Proofpoint, and SpamAssassin"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The rephrased emails still contain the same phishing payload—the malicious link or the request for credentials—so the measured detection gap reflects evasion of real attacks rather than a change in content.","fun_headline_variants_meta":{"raw":{"variants":["LLM rephrasing evades every phishing filter tested","Phishing recall crashes to 75% after simple LLM rewrite","One prompt rewrite evades all tested phishing detectors","Naive Bayes drops to 75% on LLM-crafted phishing","LLM phishing bypasses Gmail, Proofpoint, and SpamAssassin"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000408,"raw_usage":{"total_tokens":2156,"prompt_tokens":1022,"completion_tokens":1134,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":638,"completion_tokens_details":{"reasoning_tokens":1044}},"tokens_in":638,"tokens_out":1134,"duration_ms":10042,"temperature":1.0,"reasoning_tokens":1044,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-12T15:46:47.070108+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Manually inspect the 600 rephrased Nazario emails (or a random sample) and check whether each still contains a working phishing URL or a concrete request for credentials; recompute the detectors' recall on only those emails whose payload survived, and if the gap narrows to near zero, the paper's central claim would be refuted as an artifact of rewriting.","supporting_citations":[{"cited_title":"In: 3rd IEEE International Conference on Computing and Machine Intelligence (ICMI), pp","cited_arxiv_id":null,"evidence_quote":"Supplies the Nazario and Nigerian Fraud phishing corpora used for all experiments."},{"cited_title":"IEEE Access (2024)","cited_arxiv_id":null,"evidence_quote":"Establishes that LLM-generated phishing emails can evade traditional detection, motivating the comparison."},{"cited_title":"arXiv 2305 (2023)","cited_arxiv_id":null,"evidence_quote":"Shows LLMs can generate realistic spear-phishing at scale, supporting the threat model."},{"cited_title":"In: Proceedings of the 16th International Conference on World Wide Web, pp","cited_arxiv_id":null,"evidence_quote":"Provides the classical machine-learning approach that the SVM, logistic regression, and naive Bayes baselines extend."}],"review_version":1}