Advancing Email Spam Detection: Leveraging Zero-Shot Learning and Large Language Models

Ghazaleh Shirvani; Saeid Ghasemshirazi

REVIEW 4 major objections 6 minor 1 cited by

Advancing Email Spam Detection: Leveraging Zero-Shot Learning and Large Language Models

T0 review · 4 major / 6 minor · reviewed 2026-08-16 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read The paper claims a BERT-summarizer plus FLAN-T5 zero-shot classifier detects spam at 72 percent accuracy without labeled training data or retraining, because semantic alignment replaces learned class patterns.

desk verdict An honestly written but thin empirical note: the evaluation is on SMS, not email, the headline claim is unsupported, and the reported micro-F1 is internally inconsistent. read the letter →

arxiv 2505.02362 v1 pith:AHUADXE3 submitted 2025-05-05 cs.CR cs.AIcs.LG

Ghazaleh SHirvani , Saeid Ghasemshirazi This is my paper · ORCID

classification cs.CRcs.AIcs.LG

keywords spamdetectionzero-shotlearningFLAN-T5BERTlargelanguagemodelsconceptdrifttextsummarizationSMS

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper is trying to establish that a spam filter can be built without a labeled training set by combining BERT as a summarizer and FLAN-T5 as a zero-shot classifier. It argues that this pairing removes the need for large labeled datasets, adapts to evolving spam patterns, and scales better than supervised systems. On the Spam SMS Detection dataset the reported accuracy is 72%, with micro-precision 0.65, micro-recall 0.54, and micro-F1 0.52. The motivation is that if the approach works, deployment becomes a matter of prompt design rather than data collection, and spam filters could track new campaign language without retraining. The authors themselves identify low recall as the main remaining weakness.

What carries the argument

The load-bearing mechanism is zero-shot semantic alignment: FLAN-T5 embeds the summarized email text and the class labels 'spam' and 'ham' into a shared semantic space and classifies by contextual alignment. BERT's summarization step is the noise filter; it strips duplicates, stopwords, and non-alphabetic characters and distills the message to its crucial content, so the classifier works on a clean condensed representation rather than raw text. FLAN-T5's extensive pretraining across diverse tasks is what supplies the generalization that makes the alignment work on patterns the model has not been fine-tuned on.

What would settle it

Run the same BERT-summarization plus FLAN-T5 zero-shot pipeline on a genuine email corpus with longer business-style messages and phishing examples, using the same prompts, and compare accuracy and recall with the reported 72% and 0.54; if the numbers drop substantially, the paper's email-spam claim is not supported.

Watch

Extended reading notes

Core claim

The central claim is that a two-stage pipeline, BERT condensing raw content into concise summaries followed by FLAN-T5 classifying those summaries as spam or ham in a zero-shot manner, can achieve viable spam detection without any labeled examples. The paper reports an accuracy of 72%, micro-precision 0.65, micro-recall 0.54, and micro-F1 0.52 on the Spam SMS Detection dataset. In the authors' framing, this demonstrates that zero-shot learning with a large pretrained language model is a practical route to spam detection and that BERT-based summarization sharpens the signal the classifier sees.

Load-bearing premise

The load-bearing assumption is that performance on the Spam SMS Detection dataset transfers to email spam, yet the paper never tests the system on actual email content, so the title's central claim rests on an untested domain shift.

Editorial extensions

If this is right

Spam filters could be pointed at a new domain or language by rephrasing the 'spam' and 'ham' labels, with no labeled corpus to collect.
The system should track new spam styles without retraining, since classification depends on semantic alignment rather than a fixed training distribution.
Operational cost shifts from data labeling and periodic retraining to per-message inference, since every email passes through BERT summarization and FLAN-T5 classification.
The reported recall of 0.54 means that even in the paper's own evaluation many spam messages are missed, so a practical deployment would need a confidence threshold or a complementary filter.
The same summarization-plus-zero-shot structure could be applied to phishing, fraud, and misinformation detection, as the paper suggests in its discussion.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

Editorial inference: because the evaluation uses SMS messages, the settled empirical claim is about short informal text; whether the numbers extend to longer, formatted email is untested and should be checked before real-world email deployment.
Editorial inference: the precision-recall balance (0.65 versus 0.54) suggests the model is conservative, which is useful when false positives are costly but dangerous if missed spam is the greater harm; a hybrid filter with a small supervised fallback would be a natural next test.
Editorial inference: the zero-shot property implies a cheap adaptation experiment: re-run the same pipeline with updated label prompts as a new spam campaign appears and measure whether accuracy holds, which would directly test the concept-drift claim the paper relies on.
Editorial inference: a direct comparison with a simple supervised classifier on the same data would calibrate the reported 72% against familiar baselines and show how much the zero-shot design trades away.

Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

Short version: this is a preliminary study, not a research contribution. The central claim about email spam detection is never tested. The paper reports 72% accuracy on an SMS dataset and then concludes that a BERT+FLAN-T5 zero-shot pipeline provides 'robust' email spam detection. That is a load-bearing mismatch. SMS spam and email spam differ in length, structure, vocabulary, and adversarial tactics, and the paper itself concedes in the Discussion that only the Spam SMS Detection dataset was used. No email corpus appears anywhere in the evaluation.

What the paper does well is stay honest in its limitations section. It acknowledges the low recall, the computational overhead, and the need for broader datasets. The writing is clear and the architecture is described simply enough to follow. That said, none of the components are new: BERT summarization and FLAN-T5 zero-shot classification are both established techniques, and the cited literature already includes LLM-based spam detectors (ChatSpamDetector, ChatGPT-based detection). The paper offers no comparison with those or with any supervised baseline, which makes the 72% number nearly impossible to interpret.

Beyond the domain mismatch, there are real soft spots in the numbers. The paper reports micro-precision 0.65, micro-recall 0.54, and micro-F1 0.52. Under the standard definition, micro-F1 for those two values is about 0.59, not 0.52. That arithmetic inconsistency suggests the metrics were not carefully checked. There are also no implementation details: no prompt, no model version, no train/test split, no error bars, no random seed. For a zero-shot claim, the exact prompt is not a minor detail; it largely determines the result.

The conclusion overstates the evidence, calling the system 'robust' and 'a significant step forward' when recall is 0.54 and the evaluation domain is wrong. The reader's take and the stress-test note have this right. I disagree with anyone who would let the SMS evaluation stand as a proxy for email; the burden is on the authors to justify that transfer, and they do not.

Who is this paper for? Maybe a workshop poster or an undergraduate project report. It is not ready for peer review as a serious research contribution. If it came across my desk, I'd desk-reject it, with encouragement to resubmit after adding an email dataset, baselines, full configuration details, and corrected metrics. A serious referee would spend most of their time pointing out what is missing, not evaluating what is there.

Referee Report

4 major / 6 minor

Summary. The paper proposes a zero-shot email spam detection pipeline that first uses BERT to summarize message content and then uses FLAN-T5 to classify the summaries as spam or ham. The authors claim that this approach removes the need for large labeled datasets, adapts to concept drift, and scales better than traditional supervised spam detectors. The only empirical evaluation reported is on the Spam SMS Detection dataset, where the system is said to achieve 72% accuracy, micro-precision 0.65, micro-recall 0.54, and micro-F1 0.52. The paper concludes that the system provides robust email spam detection in adversarial environments.

Significance. If the proposed pipeline were convincingly validated, the idea of using a summarization model followed by a zero-shot instruction-tuned model for spam detection would be a reasonable contribution to the growing literature on LLM-based spam filtering. The paper's framing directly addresses data scarcity and concept drift, which are real operational concerns. However, the significance is severely limited by the evaluation: the central claim concerns email, but the experiments use an SMS dataset, and the reported metrics are internally inconsistent with the standard micro-F1 definition. The paper also omits key experimental details such as the prompt template, the exact model variants, the data split, and any baseline comparison. These omissions mean the empirical contribution, as presented, does not support the claimed conclusions.

major comments (4)

[Section 4 Evaluation; Section 5 Discussion] The title, abstract, and conclusion all claim email spam detection, but the only evaluation is on the Spam SMS Detection dataset. No email corpus is used, and the paper provides no argument that SMS spam is a valid proxy for email spam in terms of message length, structure, vocabulary, metadata, or adversarial tactics. The Discussion explicitly admits 'this study focuses on the Spam SMS Detection dataset' and recommends evaluation on phishing and business email compromise datasets. This admitted limitation directly undercuts the conclusion that the system provides robust email spam detection.
[Section 4 Evaluation] The reported metrics are internally inconsistent: with micro-precision 0.65 and micro-recall 0.54, the standard micro-averaged F1 would be approximately 0.59 (2 * 0.65 * 0.54 / (0.65 + 0.54)), not the reported 0.52. The paper does not explain whether the metrics are micro-averaged, macro-averaged, per-class, or computed differently. This inconsistency needs to be resolved before the results can be interpreted.
[Section 3 Methodology; Section 4 Evaluation] The evaluation is not reproducible because the paper omits essential experimental details: the exact FLAN-T5 variant (e.g., base, large, or xxl), the BERT model and summarization procedure, the prompt template used for zero-shot classification, the train/test split (or whether the full dataset was used without any split), the number of samples, and the inference configuration. Without these details, the reported 72% accuracy and the micro-metrics cannot be verified, and the zero-shot nature of the classification cannot be assessed.
[Section 4 Evaluation] No baseline or comparison is provided. The claim that the 72% accuracy is 'promising' is uninterpretable without comparing against standard supervised spam detectors, other zero-shot LLM classifiers, or prior results on the Spam SMS Detection dataset. The paper also reports no error bars or statistical significance, so it is unclear whether the observed performance reflects a stable property of the system.

minor comments (6)

[Section 2.1] There are typographical and spacing errors in the list of limitations, such as 'Theyrequirelarge, labeleddatasets' and 'Theyare unable'.
[Section 2.3] The bullet item for BERT is formatted as 'BER T' instead of 'BERT'.
[Figure 1] Figure 1's caption appears garbled ('v Extract Summary Preprocssing') and the figure itself is not referenced in the text with a meaningful explanation of its components.
[Section 4 Evaluation] The Spam SMS Detection dataset is not cited, so readers cannot identify the exact dataset version or its license.
[Abstract] The abstract uses the phrase 'NLPs' where 'NLP' or 'NLP techniques' would be more standard.
[Section 6 Conclusion] The conclusion states that the system demonstrates 'robustness to concept drift,' but no temporal or evolving-data evaluation was conducted; this claim goes beyond the evidence presented.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the system is evaluated empirically on an external benchmark, and its components are independent pretrained models.

full rationale

The paper's pipeline (BERT summarization followed by FLAN-T5 zero-shot classification) is not derived from the evaluation results; it is an empirical proposal tested on the Spam SMS Detection dataset. No parameter is fitted to the test data, no equation defines the target metric in terms of an input, and no load-bearing premise is justified by a self-citation chain. The pretrained models come from external sources, and the reported 72% accuracy, micro-precision 0.65, micro-recall 0.54, and micro-F1 0.52 are measured outcomes rather than constructions. The main weakness is external validity: the paper claims email spam detection but evaluates on SMS data, and the paper itself acknowledges this limitation in the Discussion ('this study focuses on the Spam SMS Detection dataset,' recommending evaluation on 'a wider range of datasets, including diverse languages, domains, and spam types'). That is a correctness or evidence gap, not circular reasoning, because the evaluation could have failed and the claim does not reduce to its inputs by definition. The internal inconsistency between micro-F1 and micro-precision/recall is also a correctness concern, not a circularity concern. Accordingly, no circular step is exhibited and the score is 0.

Assumptions & free parameters 0 free parameters · 3 assumptions · 0 invented entities

The paper introduces no new fitted parameters or invented entities. It rests on three domain assumptions about the transferability of pretrained language models to spam classification and the representativeness of the SMS dataset for email spam. These assumptions are not tested.

assumptions (3)

domain assumption FLAN-T5's zero-shot semantic alignment between the summarized text and the labels 'spam' and 'ham' is a valid classifier without task-specific training.
The entire method depends on the pretrained model's ability to map a short summary into the correct label space; this is asserted in Methodology and never validated against a supervised baseline.
domain assumption BERT summarization preserves the information needed to distinguish spam from ham.
The paper removes noise and condenses content before classification; if the summary discards spam indicators, classification cannot succeed. No analysis of summary quality or information retention is provided.
domain assumption The Spam SMS Detection dataset is representative of email spam.
The paper claims email spam detection but evaluates on SMS messages (Section 4); the transferability of results across modalities is assumed without evidence.

how reviews work

0 comments

Cite this review

Pith. "Pith review of Advancing Email Spam Detection: Leveraging Zero-Shot Learning and Large Language Models." pith.science (2026). https://pith.science/paper/AHUADXE3

@misc{pith2026250502362,
  author       = {Pith},
  title        = {Pith review of: Advancing Email Spam Detection: Leveraging Zero-Shot Learning and Large Language Models},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/AHUADXE3}},
  note         = {Machine review of arXiv:2505.02362}
}

read the original abstract

Email spam detection is a critical task in modern communication systems, essential for maintaining productivity, security, and user experience. Traditional machine learning and deep learning approaches, while effective in static settings, face significant limitations in adapting to evolving spam tactics, addressing class imbalance, and managing data scarcity. These challenges necessitate innovative approaches that reduce dependency on extensive labeled datasets and frequent retraining. This study investigates the effectiveness of Zero-Shot Learning using FLAN-T5, combined with advanced Natural Language Processing (NLP) techniques such as BERT for email spam detection. By employing BERT to preprocess and extract critical information from email content, and FLAN-T5 to classify emails in a Zero-Shot framework, the proposed approach aims to address the limitations of traditional spam detection systems. The integration of FLAN-T5 and BERT enables robust spam detection without relying on extensive labeled datasets or frequent retraining, making it highly adaptable to unseen spam patterns and adversarial environments. This research highlights the potential of leveraging zero-shot learning and NLPs for scalable and efficient spam detection, providing insights into their capability to address the dynamic and challenging nature of spam detection tasks.

Figures

Figures reproduced from arXiv: 2505.02362 by the authors.

**Figure 1.** The Proposed Methodology Architecture that only relevant content remains, and improves the quality of the input data. Following this, the cleaned email content is processed through BERT, which generates concise summaries by distilling the most relevant information while preserving the semantic meaning. This step reduces the complexity of the input data, allowing the classification model to focus on critical details … view at source ↗

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

Ensemble of Unsupervised Deep Learning for Clustering Imbalanced Tabular Data
cs.LG 2026-07 conditional novelty 5.0 of 10

Ensemble variants of deep clustering methods (embedding-averaging and majority voting) achieve the best average ACC, NMI, and ARI on 16 imbalanced binary tabular datasets.

Reference graph

Works this paper leans on

13 extracted references · 10 canonical work pages · cited by 1 Pith paper

[1]

Explainability with semantic concept composition and zero- shot learning for anomaly detection,

N. S. Bendre, “Explainability with semantic concept composition and zero- shot learning for anomaly detection,” Ph.D. dissertation, The University of Texas at San Antonio, 2021

work page 2021
[2]

Survey of review spam detection using machine learning techniques,

M. Crawford, T. M. Khoshgoftaar, J. D. Prusa, A. N. Richter, and H. Al Na- jada, “Survey of review spam detection using machine learning techniques,” Journal of Big Data, vol. 2, pp. 1–24, 2015

work page 2015
[3]

Applying lazy learning algorithms to tackle concept drift in spam filtering,

F. Fdez-Riverola, E. L. Iglesias, F. Díaz, J. R. Méndez, and J. M. Corchado, “Applying lazy learning algorithms to tackle concept drift in spam filtering,” Expert Systems with Applications, vol. 33, no. 1, pp. 36–48, 2007

work page 2007
[4]

Chatspamdetector: Lever- aging large language models for effective phishing email detection,

T. Koide, N. Fukushi, H. Nakano, and D. Chiba, “Chatspamdetector: Lever- aging large language models for effective phishing email detection,”arXiv preprint arXiv:2402.18093, 2024

arXiv 2024
[5]

Spamdam: Towards privacy- preserving and adversary-resistant sms spam detection,

Y. Li, R. Zhang, W. Rong, and X. Mi, “Spamdam: Towards privacy- preserving and adversary-resistant sms spam detection,”arXiv preprint arXiv:2404.09481, 2024

arXiv 2024
[6]

Prompt-learning and zero-shot text classification with domain- specific textual data,

H. Luo, “Prompt-learning and zero-shot text classification with domain- specific textual data,” 2023

work page 2023
[7]

Deceptive opinion spam detection approaches: a literature survey,

S. K. Maurya, D. Singh, and A. K. Maurya, “Deceptive opinion spam detection approaches: a literature survey,” Applied intelligence, vol. 53, no. 2, pp. 2189–2234, 2023. 7

work page 2023
[8]

A review on social spam detec- tion: Challenges, open issues, and future directions,

S. Rao, A. K. Verma, and T. Bhatia, “A review on social spam detec- tion: Challenges, open issues, and future directions,”Expert Systems with Applications, vol. 186, p. 115742, 2021

work page 2021

Show all 13 references

[9]

Evaluating the performance of chatgpt for spam email detection,

S. Si, Y. Wu, L. Tang, Y. Zhang, and J. Wosik, “Evaluating the performance of chatgpt for spam email detection,”arXiv preprint arXiv:2402.15537, 2024

2024 arXiv
[10]

Large language models for forecasting and anomaly detection: A systematic literature review,

J. Su, C. Jiang, X. Jin, Y. Qiao, T. Xiao, H. Ma, R. Wei, Z. Jing, J. Xu, and J. Lin, “Large language models for forecasting and anomaly detection: A systematic literature review,”arXiv preprint arXiv:2402.10350, 2024

2024 arXiv
[11]

Machine learning for spam detec- tion,

P. Teja Nallamothu and M. Shais Khan, “Machine learning for spam detec- tion,” Asian Journal of Advances in Research, vol. 6, no. 1, pp. 167–179, 2023

2023
[12]

Chatgpt: A threat to spam filtering systems,

A. Utaliyeva, M. Pratiwi, H. Park, and Y.-H. Choi, “Chatgpt: A threat to spam filtering systems,” in2023 IEEE International Conference on High Performance Computing & Communications, Data Science & Systems, Smart City & Dependability in Sensor, Cloud & Big Data Systems & Appli...

2023
[13]

Semantics-aware bert for language understanding,

Z. Zhang, Y. Wu, H. Zhao, Z. Li, S. Zhang, X. Zhou, and X. Zhou, “Semantics-aware bert for language understanding,” inProceedings of the AAAI conference on artificial intelligence, vol. 34, no. 05, 2020, pp. 9628– 9635. 8

2020

Pith tools

Reviewed August 16, 2026 · model on record in the stance chip above.

[1] [1]

Explainability with semantic concept composition and zero- shot learning for anomaly detection,

N. S. Bendre, “Explainability with semantic concept composition and zero- shot learning for anomaly detection,” Ph.D. dissertation, The University of Texas at San Antonio, 2021

work page 2021

[2] [2]

Survey of review spam detection using machine learning techniques,

M. Crawford, T. M. Khoshgoftaar, J. D. Prusa, A. N. Richter, and H. Al Na- jada, “Survey of review spam detection using machine learning techniques,” Journal of Big Data, vol. 2, pp. 1–24, 2015

work page 2015

[3] [3]

Applying lazy learning algorithms to tackle concept drift in spam filtering,

F. Fdez-Riverola, E. L. Iglesias, F. Díaz, J. R. Méndez, and J. M. Corchado, “Applying lazy learning algorithms to tackle concept drift in spam filtering,” Expert Systems with Applications, vol. 33, no. 1, pp. 36–48, 2007

work page 2007

[4] [4]

Chatspamdetector: Lever- aging large language models for effective phishing email detection,

T. Koide, N. Fukushi, H. Nakano, and D. Chiba, “Chatspamdetector: Lever- aging large language models for effective phishing email detection,”arXiv preprint arXiv:2402.18093, 2024

arXiv 2024

[5] [5]

Spamdam: Towards privacy- preserving and adversary-resistant sms spam detection,

Y. Li, R. Zhang, W. Rong, and X. Mi, “Spamdam: Towards privacy- preserving and adversary-resistant sms spam detection,”arXiv preprint arXiv:2404.09481, 2024

arXiv 2024

[6] [6]

Prompt-learning and zero-shot text classification with domain- specific textual data,

H. Luo, “Prompt-learning and zero-shot text classification with domain- specific textual data,” 2023

work page 2023

[7] [7]

Deceptive opinion spam detection approaches: a literature survey,

S. K. Maurya, D. Singh, and A. K. Maurya, “Deceptive opinion spam detection approaches: a literature survey,” Applied intelligence, vol. 53, no. 2, pp. 2189–2234, 2023. 7

work page 2023

[8] [8]

A review on social spam detec- tion: Challenges, open issues, and future directions,

S. Rao, A. K. Verma, and T. Bhatia, “A review on social spam detec- tion: Challenges, open issues, and future directions,”Expert Systems with Applications, vol. 186, p. 115742, 2021

work page 2021