Pith. sign in

REVIEW 4 major objections 5 minor 17 references

Send to which account? Evaluation of an LLM-based Scambaiting System

T0 review · 4 major / 5 minor · reviewed 2026-08-15 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read This paper reports that an LLM-based scambaiting system, run against real scammers over five months, obtained mule-account details in about 32% of conversations that got a reply, and that human review made disclosures faster.

desk verdict A genuinely new field dataset, but the headline IDR is internally inconsistent and the disclosure labels are unvalidated; major revision before it can be trusted. read the letter →

arxiv 2509.08493 v1 pith:HQ7MXOUY submitted 2025-09-10 cs.CR cs.AI

Hossein Siadati , Haadi Jafarian , Sima Jafarikhah This is my paper · ORCID
classification cs.CRcs.AI
keywords scambaitinglargelanguagemodelsconversationalhoneypotsfinancialfraudmuleaccountsthreatintelligencehuman-in-the-loopinformationdisclosure
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper reports the first large-scale, real-world test of an automated scambaiting system: an LLM that impersonates a potential victim in email conversations with actual scammers. Over 154 days the system seeded 2,638 conversations and logged 18,797 messages, and the authors measure how often scammers hand over useful financial details. The central result is an Information Disclosure Rate of about 31.74% among conversations that matured, meaning roughly one in three active baiting chats ended with the scammer sending mule-account information (bank accounts used to collect scam proceeds). In the human-in-the-loop mode, the Human Acceptance Rate was 69.02%, meaning human reviewers sent most LLM drafts unedited, and human-in-the-loop review shortened time-to-disclosure relative to fully automated operation. If these rates hold up, they are evidence that conversational LLM agents can generate actionable financial threat intelligence at scale rather than only in controlled simulations.

What carries the argument

The load-bearing object is the deployed scambaiting pipeline: spam-honeypot email addresses feed threads in which a ChatGPT-based model, driven by a single system prompt, proposes the next reply in a victim persona, and a human operator reviews and either approves or edits that reply before it is sent. Around this pipeline the paper wraps a metric suite: Information Disclosure Rate (share of engagements in which the scammer discloses financial account details), Information Disclosure Speed (message turns and days until first disclosure), Human Acceptance Rate (share of model drafts sent without edits), Takeoff Ratio (share of seeded threads that receive at least one scammer reply), and response-latency statistics. These metrics convert a production message log into the paper's quantitative claims about what works in real scambaiting.

What would settle it

An independent audit of the raw message log should reproduce the paper's own inconsistent counts: 1,285 matured engagements in the dataset section versus 1,599 multi-message matured engagements in the endurance analysis, and 466 successful disclosures overall versus 509 in the endurance analysis. If these numbers cannot be reconciled, or if blind re-annotation of the flagged financial-account messages does not agree with the database flags, the 31.74% IDR claim is not established.

Watch

Extended reading notes

Core claim

The paper's central claim is that an operational LLM-powered scambaiting system, which talks to real scammers under the guise of a victim persona, can extract mule bank-account details at a meaningful rate in production. Across 2,638 seeded conversations and 18,797 messages, 1,285 conversations received at least one scammer reply, and 466 of those were flagged by the database as ending in financial-account disclosure: an Information Disclosure Rate of 17.66% over all seeds and 31.74% over matured conversations. The authors also report a Human Acceptance Rate of 69.02%, an average disclosure time of 10.3 message turns or 7.4 days, and faster disclosure in the human-in-the-loop mode (median 3.2 days to disclosure, 90% by 7.5 days) than in the fully automated mode (90% by 25.3 days). These numbers are presented as the first large-scale evidence that LLM-driven conversational honeypots can generate actionable financial threat intelligence rather than merely prolonging scammer time.

Load-bearing premise

The headline rates rest on the accuracy of the database flags that mark which scammer messages contain financial account information, and the paper never describes how those flags were created or validated; if the flags are wrong or incomplete, every IDR and IDS figure shifts.

Editorial extensions

If this is right

  • Operators can expect roughly one disclosure per three active conversations, so the volume of flagged mule accounts scales with how many seeds can be sent rather than with conversation length alone.
  • A 28-day no-response cutoff is defensible: only 5% of engagements remain alive past 28 days, so keeping threads open longer mostly wastes resources.
  • First-message design is the highest-leverage intervention: openings around 301-500 characters had the best takeoff, and openings averaging 551 characters significantly hurt reply rates.
  • Human-in-the-loop operation improves speed without sacrificing depth: Mode II reached 90% of disclosures by 7.5 days versus 25.3 days in Mode I, with comparable message counts.
  • Scammer response latency is a usable triage signal: successful engagements had median reply times of 2.37 hours versus 6.90 hours for unsuccessful ones.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • If the disclosure labels are accurate, roughly one in three conversations that receive even a single reply yields a mule account, so takeoff improvement (currently 48.7%) is the main lever for total intelligence volume; improving elicitation further would matter less at the margin.
  • The paper's success signal is mule-account disclosure in email threads; the same pipeline could be tested for cryptocurrency wallet addresses, romance scams, or investment scams, where the disclosure signal and conversation dynamics differ.
  • The correlation between human acceptance and engagement success suggests a control loop the paper does not propose: use HAR as a live engagement-health score to decide when to let the LLM run without edits and when to escalate to human rewriting.
  • The takeoff findings are observational, so a randomized A/B test of seed-message length and send day (e.g., Monday and Wednesday peaks versus Sunday troughs) would be the natural next experiment to establish causality.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. This paper evaluates a production LLM-based scambaiting system by analyzing a database dump of 2,638 email engagements with suspected scammers over five months. It defines a suite of metrics—Information Disclosure Rate and Speed, Human Acceptance Rate, Message Freshness, Takeoff Ratio, Engagement Endurance, and Response Invocation—and reports headline results: an IDR of approximately 32%, a HAR of 69.02%, and a takeoff ratio of 48.7%. The paper also compares an LLM-only mode with a human-in-the-loop mode and derives seven operational insights about engagement persistence, response latency, and human oversight.

Significance. If correct, these results would constitute the first large-scale field measurement of LLM-driven scambaiting and would provide a useful starting point for future deployments and metric standardization. The scale of the corpus (18,797 messages, 2,638 seeds) and the longitudinal design are genuine strengths, and the proposed metrics (IDS, takeoff, death pulse) are well motivated. However, the quantitative foundation is not currently reliable: the central IDR is unreproducible from the paper's own counts, the disclosure labels are unvalidated, and the mode comparisons are confounded by deployment phase. The paper thus cannot yet support its headline contribution, although the qualitative observations may survive a corrected analysis.

major comments (4)
  1. [§VI.A.1; §VI.C.2; Table II] The headline IDR is not reproducible. §IV.B and Table II fix matured engagements at 1,285 out of 2,638 seeds, and §VI.A.1 reports 466 disclosures, giving 466/2,638 = 17.66% overall. The same paragraph states that restricting to matured engagements gives 31.74%, but 466/1,285 = 36.26%, not 31.74%. §VI.C.2 then states 'Among the 1,599 multi-message matured engagements, 509 were successful' and reports 1,090 unsuccessful, which sums to 1,599; 509/1,599 = 31.83%, close to 31.74%, but 1,599 contradicts the 1,285 matured count and 509 contradicts the 466 disclosures from §VI.A.1. Table V repeats 31.74%, and the Abstract says 'approximately 32%'. No consistent choice of numerator and denominator yields the reported matured IDR. Since the paper's central claim is that the system extracts actionable mule-account intelligence at this rate, the authors must reconcile these counts and state exactly which engagements constitute the denominator and which messages constitute a disclosure.
  2. [§IV.A; §IV.C] The disclosure labels are not validated. The dataset description says only that the database contains 'flags indicating whether a scammer's message contains financial account information (e.g., bank details)'. The paper never defines 'mule account', describes how the flags were produced (automated detector, operator annotation, string matching), or reports any precision/recall or inter-annotator agreement. The IDR numerator depends entirely on these flags, and the intelligence-sharing claim in §IV.C ('collected threat intelligence was responsibly and promptly shared with financial institutions') presupposes their accuracy. Without a validation protocol or a sample audit, the reader cannot distinguish a true mule-account disclosure from any message that merely mentions a bank account. This is a load-bearing gap, not a presentation issue.
  3. [§IV.A; §VI.A.2; §VII Insights I, II, VI] The Mode I versus Mode II comparisons are confounded by deployment phase. Mode I was active for 120 days and Mode II for 34 days, in distinct operational phases with different periods, personas, and evolving prompts. The paper attributes faster disclosure (§VI.A.2), higher HAR (§VI.B.1), and improved efficiency (§VI.C.2) to human-in-the-loop oversight, but these outcomes could equally be due to seasonality, different scam cohorts, prompt revisions, or operator learning. The data do not support causal claims about HITL; the design is observational. The authors should either temper the causal language in Insights I, II, and VI or provide evidence that the two modes were otherwise comparable.
  4. [§VI.C.2; §VI.C.3; §IV.B] Additional internal inconsistencies in engagement-level statistics prevent the reader from trusting any of the derived rates. §IV.B reports matured engagements averaging 13.4 messages (median 9.0), while §VI.C.2 reports 12.2 (median 7.0) for the same population. §VI.C.3 refers to 'the 33 successful engagements' in the survival analysis, conflicting with the 466 (or 509) successful engagements used elsewhere. Also, the counts in §IV.B do not sum: 1,170 non-responders plus 1,285 matured engagements leaves 183 seeded engagements unaccounted for. These discrepancies may be typographical, but they compound the unreliability of the headline metrics and must be corrected throughout the paper.
minor comments (5)
  1. [Table III] The Takeoff Ratio description in Table III says 'Percentage of matured engagements that receive at least one scammer response'; this should be 'percentage of seeded engagements that receive at least one scammer response', matching §V.C and §VI.C.1.
  2. [§V.A] The IDR formula uses |Total Engagements| without specifying whether it means seeded or matured engagements. §VI.A.1 uses both senses, which is confusing; the formula and the table should use consistent notation.
  3. [§IV.B] The percentage for 1,170 non-responders is given as 46%, but 1,170/2,638 is 44.4%. The text also says 'approximately half of these unresponsive email addresses were inactive', but no data are provided to support the 50% claim.
  4. [§VI.C.1; Fig. 4] The text reports the highest takeoff rate on Monday at 52.66%, while Fig. 4 states Monday's rate is 55.1%; these numbers should be reconciled.
  5. [§I] There is a typo, 'a gen model', in the Introduction.

Circularity Check

0 steps flagged · score 0.0 of 10

Observational evaluation with no circular derivation; reported rates are operational measurements, and internal count inconsistencies are correctness risks, not circularity.

full rationale

The paper is an observational field evaluation of an existing LLM scambaiting platform; it does not derive a predictive result from first principles. IDR, IDS, HAR, takeoff ratio, and engagement endurance are all defined operationally in Section V and then measured on the collected 18,797-message dataset, so there is no fitted input that is later renamed as a prediction. The closest self-referential element is HAR, which measures whether the same human operators who review LLM suggestions accepted them unchanged; however, HAR is explicitly presented as a measured proportion, and the claim that acceptance correlates with successful engagements is an empirical comparison rather than an equation-level reduction. The paper's central percentages (e.g., 17.66% and 31.74% IDR) are direct tallies of database flags, not outputs of a model fitted to those flags. Some self-citations appear in related-work context (references [8] and [9]) but they are not load-bearing for the headline rates. The manuscript does contain serious reproducibility and correctness concerns: the counts conflict (466 disclosures among 1,285 matured engagements would be 36.26%, not 31.74%, while 509/1,599 equals 31.83%), and the disclosure flags lack any labeling or validation protocol; but these are data-validity and internal-consistency issues, not circularity. No step in the paper reduces, by construction, to its own input.

Assumptions & free parameters 0 free parameters · 5 assumptions · 0 invented entities

The central metrics rest on data-quality assumptions about scammer identity, disclosure labeling, and mode comparability. No free parameters are fitted, but the paper introduces hand-defined categories and metrics that are not validated. The absence of released data prevents external checks of these assumptions.

assumptions (5)
  • domain assumption Database flags marking messages as containing financial account information are accurate and complete.
    IDR computation in Section VI.A.1 relies on these flags, but no labeling protocol, validation, or inter-annotator reliability is described.
  • domain assumption Targeted email addresses obtained from spam honeypots belong to genuine scammers.
    Section IV.C asserts high confidence but provides no independent verification; if many addresses are inactive or innocent, takeoff and IDR would be biased.
  • domain assumption Operator acceptance and edits are a valid proxy for message quality and conversational alignment.
    HAR is used in Section VI.B as a quality metric and predictor of success, but operator behavior may reflect safety, style, or compliance preferences rather than quality.
  • domain assumption Mode I and Mode II are comparable despite different deployment windows and evolving prompts.
    Mode I ran 120 days and Mode II 34 days, with organic changes to prompt templates and human interventions over time (Section IV.A), so observed differences may be time effects rather than configuration effects.
  • domain assumption The provided database dump is complete and free of selection or logging errors.
    The analysis depends entirely on a dump supplied by the platform administrators (Section IV.A); completeness and correctness are not independently verified.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Send to which account? Evaluation of an LLM-based Scambaiting System." pith.science (2026). https://pith.science/paper/HQ7MXOUY

@misc{pith2026250908493,
  author       = {Pith},
  title        = {Pith review of: Send to which account? Evaluation of an LLM-based Scambaiting System},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/HQ7MXOUY}},
  note         = {Machine review of arXiv:2509.08493}
}
read the original abstract

Scammers are increasingly harnessing generative AI(GenAI) technologies to produce convincing phishing content at scale, amplifying financial fraud and undermining public trust. While conventional defenses, such as detection algorithms, user training, and reactive takedown efforts remain important, they often fall short in dismantling the infrastructure scammers depend on, including mule bank accounts and cryptocurrency wallets. To bridge this gap, a proactive and emerging strategy involves using conversational honeypots to engage scammers and extract actionable threat intelligence. This paper presents the first large-scale, real-world evaluation of a scambaiting system powered by large language models (LLMs). Over a five-month deployment, the system initiated over 2,600 engagements with actual scammers, resulting in a dataset of more than 18,700 messages. It achieved an Information Disclosure Rate (IDR) of approximately 32%, successfully extracting sensitive financial information such as mule accounts. Additionally, the system maintained a Human Acceptance Rate (HAR) of around 70%, indicating strong alignment between LLM-generated responses and human operator preferences. Alongside these successes, our analysis reveals key operational challenges. In particular, the system struggled with engagement takeoff: only 48.7% of scammers responded to the initial seed message sent by defenders. These findings highlight the need for further refinement and provide actionable insights for advancing the design of automated scambaiting systems.

Figures

Figures reproduced from arXiv: 2509.08493 by the authors.

Figure 1
Figure 1. System architecture of the scambaiting platform that [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 2
Figure 2. Information Disclosure Speed: Cumulative timeline of [PITH_FULL_IMAGE:figures/full_fig_p007_2.png] view at source ↗
Figure 3
Figure 3. Human Acceptance Rate: Cumulative distribution [PITH_FULL_IMAGE:figures/full_fig_p008_3.png] view at source ↗
Figures from the paper (3 more)
Figure 4
Figure 4. Figure 4: Takeoff Ratio: Monday exhibited the highest takeoff [PITH_FULL_IMAGE:figures/full_fig_p009_4.png]
Figure 5
Figure 5. Figure 5: Response Invocation: Faster scammer responses are [PITH_FULL_IMAGE:figures/full_fig_p010_5.png]
Figure 6
Figure 6. Figure 6: Survival rate of scammer responses over time. Engage [PITH_FULL_IMAGE:figures/full_fig_p011_6.png]

Review history (2 revisions) →

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

17 extracted references · 16 canonical work pages

  1. [1]

    New FTC data show big jump in reported losses to fraud: $10 billion in 2023,

    Federal Trade Commission, “New FTC data show big jump in reported losses to fraud: $10 billion in 2023,” Mar. 2025, accessed: 2025-07-12. [Online]. Avail- able: https://www.ftc.gov/news-events/news/press-releases/2025/03/ new-ftc-data-show-big-jump-reported-losses-fraud-125-billion-2024

    work page 2023

  2. [2]

    Report links chatgpt to 1265% rise in phishing emails,

    InfoSecurity Magazine Staff, “Report links chatgpt to 1265% rise in phishing emails,”InfoSecurity Magazine, Oct. 2023, reported from the SlashNext State of Phishing Report 2023, noting a 1,265% increase in malicious phishing emails :contentReference[oaicite:1]index=1. [Online]. Available: https: //www.infosecurity-magazine.com/news/chatgpt-linked-rise-phishing/

    work page 2023

  3. [3]

    Ai tools such as chatgpt are generating a mammoth increase in mali- cious phishing emails,

    CNBC Technology Executive Council, “Ai tools such as chatgpt are generating a mammoth increase in mali- cious phishing emails,” https://www.cnbc.com/2023/11/28/ ai-like-chatgpt-is-creating-huge-increase-in-malicious-phishing-email. html, Nov. 2023, accessed: 2025-07-12

    work page 2023

  4. [4]

    Discussion paper: Ex- ploiting llms for scam automation: A looming threat,

    G. Gressel, R. Pankajakshan, and Y . Mirsky, “Discussion paper: Ex- ploiting llms for scam automation: A looming threat,” inProceedings of the 3rd ACM Workshop on the Security Implications of Deepfakes and Cheapfakes, 2024, pp. 20–24

    work page 2024

  5. [5]

    Abusegpt: Abuse of generative ai chatbots to create smishing campaigns,

    A. M. Shibli, M. M. A. Pritom, and M. Gupta, “Abusegpt: Abuse of generative ai chatbots to create smishing campaigns,” in2024 12th International Symposium on Digital Forensics and Security (ISDFS), San Antonio, TX, USA, 2024, pp. 1–6

    work page 2024

  6. [6]

    Money mules,

    Federal Bureau of Investigation, “Money mules,” n.d., accessed: 2025- 07-12. [Online]. Available: https://www.fbi.gov/how-we-can-help-you/ scams-and-safety/common-frauds-and-scams/money-mules

    work page 2025

  7. [7]

    Phishing and money mules,

    D. Flor ˆencio and C. Herley, “Phishing and money mules,” in2010 ieee international workshop on information forensics and security. IEEE, 2010, pp. 1–5

    work page 2010

  8. [8]

    A framework for analysis attackers’ accounts,

    H. Siadati, J. Koven, C. F. da Silvaet al., “A framework for analysis attackers’ accounts,” inSecurity, Privacy and User Interaction, ser. Lecture Notes in Computer Science, M. R. de Castellarnau, A. Markopoulou, and F. Perich, Eds. Springer International Publishing, 2020, vol. 12229, pp. 49–65. [Online]. Available: https://link.springer. com/chapter/10.1...

    work page doi:10.1007/978-3-030-43754-1 2020

Show all 17 references
  1. [9]

    Prevention, detection, and reaction to cyber impersonation attacks,

    H. Siadati, “Prevention, detection, and reaction to cyber impersonation attacks,” Ph.D. dissertation, New York University Tandon School of Engineering, 2019, proQuest Dissertations & Theses, Document ID: 13426039. Accessed: 2025-07-12. [Online]. Available: https: //www.proques...

    2019

  2. [10]

    Bot wars evolved: Orchestrating competing llms in a counterstrike against phone scams,

    N. Basta, C. Atkins, and D. Kaafar, “Bot wars evolved: Orchestrating competing llms in a counterstrike against phone scams,” inPacific-Asia Conference on Knowledge Discovery and Data Mining. Springer, 2025, pp. 338–350

    2025

  3. [11]

    Puppeteer: Leveraging a large language model for scambaiting,

    P. Charnsethikul, B. Crotty, J. Mirkovic, J. Liu, R. Saiya, and G. Bartlett, “Puppeteer: Leveraging a large language model for scambaiting,” 2025

    2025

  4. [12]

    Re:scam – AI tool to waste scammers’ time (official site),

    Netsafe, “Re:scam – AI tool to waste scammers’ time (official site),” https://netsafe.org.nz/rescam, 2024, accessed 2025

    2024

  5. [13]

    Re:scam2.0 ai email engagement bot,

    Netsafe, “Re:scam2.0 ai email engagement bot,” https://www.rescam. org, 2024, design led by Maria Slade / Elliot Oxborough; relaunched 2024

    2024

  6. [14]

    Send scam emails to this chatbot and it’ll waste their time for you,

    J. Vincent, “Send scam emails to this chatbot and it’ll waste their time for you,”The Verge, Nov. 10 2017, accessed: 2025-07-11. [Online]. Available: https://www.theverge.com/2017/11/10/16632724/ scam-chatbot-ai-email-rescam-netsafe

    2017

  7. [15]

    Scamgpt-j: Inside the scammer’s mind, a generative ai-based approach toward combating messaging scams,

    X. W. Tan, K. See, and S. Kok, “Scamgpt-j: Inside the scammer’s mind, a generative ai-based approach toward combating messaging scams,” arXiv preprint arXiv:2412.13528, 2024

    2024 arXiv

  8. [16]

    Into the gray zone: The private sector and active defense against cyber threats,

    D. C. Blair, M. Chertoff, F. J. Cilluffo, and N. O’Connor, “Into the gray zone: The private sector and active defense against cyber threats,”Center for Cyber and Homeland Security. https://cchs. gwu. edu/sites/cchs. gwu. edu/files/downloads/CCHS-ActiveDefenseReportFINAL. pdf, 2016

    2016

  9. [17]

    The enterprise strikes back: Conceptualizing the hackbot-reversing social engineering in the cyber defense context

    M. Lundie, K. Lindke, A. Amos-Binks, M. P. Aiken, and D. Janosek, “The enterprise strikes back: Conceptualizing the hackbot-reversing social engineering in the cyber defense context.” inHICSS, 2024, pp. 984–993. Category Metric Value Disclosure Success Information Disclosure R...

    2024

Pith tools

Reviewed August 15, 2026 · model on record in the stance chip above.