Pith. sign in

REVIEW 5 major objections 4 minor 1 cited by

FAA Framework: A Large Language Model-Based Approach for Credit Card Fraud Investigations

T0 review · 5 major / 4 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read A team of LLM agents can run a credit card fraud investigation from alert to report, and the reports support accurate fraud decisions.

desk verdict Plausible LLM-agent framework for fraud investigation, but the evaluation is self-referential and under-powered; deserves a major-revision review, not acceptance as-is. read the letter →

arxiv 2506.11635 v1 pith:CSVI73IZ submitted 2025-06-13 cs.CR cs.AI

Shaun Shuster , Eyal Zaloof , Asaf Shabtai , Rami Puzis This is my paper
classification cs.CRcs.AI
keywords creditcardfraudinvestigationlargelanguagemodelsLLMagentsautomatedevidencequalityevaluationdetectionexplainableAI
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper claims that a team of large language model agents can carry out a complete credit card fraud investigation, from a suspicious transaction number to a written report, and that the resulting reports are good enough to support a final fraud decision. On 500 synthetic cases across two datasets, a detective agent reading only the generated reports reached an F1 score of 98-99 percent. The authors also report that most collected evidence was relevant, logically consistent, and had high impact on fraud suspicion, while investigations averaged about seven steps. If these results hold, the framework could take over a large share of the repetitive alert work that currently causes analyst fatigue. The authors position it as an assistant: human analysts still handle client interviews and formal processing of each case.

What carries the argument

The FAA framework: a loop in which a multimodal LLM plans the next investigative step, executes it by writing and running Python code against a transaction database and generating visualizations, reads those visualizations with a vision agent, integrates the results into the case narrative, and repeats until it decides enough evidence exists. A report-generating agent then distills the evidence, and a detective agent makes the fraud call. The evaluation relies on a custom evidence quality score that uses an LLM judge to rate each piece of evidence on impact, relevance, new knowledge, and logical alignment.

What would settle it

Give the unfiltered reports from the 500 investigations to experienced fraud analysts and ask them to rate each evidence item on the same four-point aspects; if human-LLM agreement is close to chance or the humans rate large shares of the evidence as irrelevant or low-impact, the paper's quality claims fail. A complementary check is running the framework on real-world alerts with confirmed outcomes, where an F1 drop far below 98 percent would show the synthetic evaluation overstates reliability.

Watch

Extended reading notes

Core claim

The central claim is that a fully automated pipeline of planning, information gathering, analysis, report generation, and final classification can produce credit card fraud investigations whose evidence supports reliable decisions. In the evaluation, the framework with vision achieved an F1 of 0.9801 on Sparkov and 0.99 on CCTD, with precision and recall near 98-99 percent, and the evidence-quality evaluation showed 71-72 percent of evidence with high or very high impact on suspicion, all evidence relevant, and no logical inconsistencies. The paper presents this as evidence that LLM-based investigation can automate large parts of the fraud analyst workload.

Load-bearing premise

The evidence quality ratings come from an LLM judge that the paper never validates against human fraud analysts, so if the LLM's self-assessment does not match expert judgment, the reported evidence quality and reliability conclusions do not stand.

Editorial extensions

If this is right

  • Analysts would receive a draft investigation and verdict for each alert, letting them focus on complex cases rather than routine screening.
  • The evidence quality score offers a way to check that automated investigations produce useful and coherent evidence, not only correct final labels.
  • Including the vision agent raises detection F1 on both datasets (e.g., 0.972 to 0.980 on Sparkov) while roughly doubling token consumption.
  • Because the framework is not trained on the benchmark distributions, its performance is not dataset-specific, though it does not beat every tuned baseline on a static benchmark.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • A natural next test is to have human fraud analysts rate the same reports; the current evidence-quality claims rest entirely on an LLM judging itself.
  • The roughly 200k input tokens per vision-enabled investigation means real-world deployment is an economic question about LLM pricing versus analyst time saved.
  • The plan-gather-analyze loop is generic enough to extend to other investigation workloads such as money-laundering suspicious activity reports or chargeback disputes.
  • Since the framework has no memory of past cases, it may be slow to pick up new fraud patterns; adding case memory or periodic fine-tuning is a concrete way to test this.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

5 major / 4 minor

Summary. The paper introduces the FAA framework, a multi-agent LLM system for automating credit card fraud investigations. The system uses GPT-4o to plan investigation steps, execute Python code against transaction databases, interpret visualizations through a vision agent, and generate reports via a report-generating agent. A detective agent then classifies the transaction as fraudulent or legitimate based on the generated report. The authors evaluate the framework on 500 transactions from the Sparkov and CCTD datasets, reporting evidence-quality ratings (impact, relevance, novelty, logical alignment) supplied by an LLM judge, efficiency measures (number of steps, token counts, minimal trajectory ratio), and detective-agent F1 scores of 0.972–0.99. The paper concludes that the FAA framework produces reliable and efficient investigations and can automate large parts of the fraud analyst workload.

Significance. If the empirical claims were well supported, this would be a useful contribution to automated fraud investigation: the system design is clearly described, the prompts are provided in the appendix, and the work addresses a practical problem with an end-to-end pipeline. The authors also make a reasonable effort to probe dataset memorization and to define a multi-aspect evidence-quality measure. However, the central evaluation is self-referential: the same LLM family generates the evidence, produces the report, and rates the quality of the evidence, with no human validation, no grounding audit against the underlying database, and no uncertainty quantification. As a result, the headline numbers (98–99% F1, 71–76% high-impact evidence) do not yet establish that the framework reliably investigates transactions or that the generated reports are trustworthy for fraud analysts.

major comments (5)
  1. [Section 4.3.3, Figure 4] The evidence-quality score is produced by a GPT-4o judge on reports generated by GPT-4o agents, and the paper treats these ratings as ground truth for relevance, logical alignment, and new knowledge. There is no validation against human fraud analysts and no check that the described evidence actually corresponds to entries in the transaction database. The prompt in Appendix Listing 7 instructs the judge to be critical, but this does not validate the rating scale. Consequently, the RQ1 conclusion that all evidence was relevant and logically consistent, and that 71–76% of evidence had high impact, is unsupported.
  2. [Section 4.5, Table 3] The detective-agent F1 of 98–99% does not certify the reliability of the investigation process. The detective agent sees only the LLM-generated report, not the raw data, so a strong LLM could infer the correct label from a hallucinated or misleading report. The paper reports no control condition, no grounding audit, and no human comparison of report content, so the F1 measures correlation between the report text and the dataset label rather than the soundness of the investigation.
  3. [Section 4.2, Table 1] The memorization check is underpowered. With 50 total samples and 29 correct predictions, the accuracy of 0.58 is statistically indistinguishable from chance (approximate 95% confidence interval 0.44–0.70), so the paper's claim that the LLM 'is outputting predictions randomly' is not justified. Moreover, the check only tests direct raw-feature prediction of the is-fraud column; it does not rule out memorization of dataset patterns that could influence an investigation built on the same database.
  4. [Section 4.3.2, Table 2] The minimal trajectory ratio is not reproducible because the manuscript never specifies how the set of 'supporting decision steps' (SDS) is determined. Without a concrete criterion or a demonstrated reliable method for identifying these steps, the efficiency claims based on the minimal trajectory ratio cannot be assessed.
  5. [Tables 2 and 3] No error bars, confidence intervals, or significance tests are reported for any of the main metrics. The differences attributed to the vision agent (for example, 0.972 vs. 0.9801 F1 on Sparkov) may be within noise, so the RQ4 conclusion that the vision agent improves the framework is not statistically supported.
minor comments (4)
  1. [Global] There are numerous typos and inconsistencies, including 'costumers' for 'customers', 'an strategy' for 'a strategy', 'The workflow the of FAA framework', 'framework framework' in Section 5, and Table 3 labeling the proposed framework 'FFA' instead of 'FAA'.
  2. [Abstract, Section 4.4, Table 2] The abstract states that most investigations include 20–40% excess steps compared to a minimal trajectory, but Table 2 reports minimal trajectory ratios of 0.76–0.83, corresponding to 17–24% excess steps. These numbers should be reconciled.
  3. [Section 4.4] The sentence 'the distribution in the categories... does not include neither agree nor disagree' is ambiguous. Since relevance and logical alignment are described as binary aspects, the text should say explicitly that the ratings were only 'agree' or 'strongly agree' on those aspects.
  4. [Section 4.5] The phrase 'establishing a new state-of-the-art for this dataset' is an overclaim, because the FAA framework is an investigation-and-reporting system rather than a detection model in the same sense as the baseline classifiers; the comparisons should be explicitly framed as task-different.

Circularity Check

1 steps flagged · score 6.0 of 10

Evidence quality is scored by GPT-4o judging GPT-4o's own reports, so RQ1's 'reliable evidence' claim reduces to LLM self-agreement; the detective F1 is label-based and not circular.

  1. self definitional [Section 4.3.3 (Evidence Quality Score); Listing 7 (Appendix)]
    "Therefore, in our evaluation, we use the unfiltered report. In addition, we implemented this measure using a few-shot prompting technique to prompt an LLM to provide a rating on a five-point Likert scale for each piece of evidence on 4 different aspects as follows:"

    The paper's RQ1 conclusion that the FAA produces 'reliable and efficient investigations' with 'high-quality evidence' is measured by asking an LLM to rate evidence in a report produced by the same LLM family. Section 4.3.3 defines evidence quality as whatever the judge LLM assigns on a Likert scale, and Listing 7 supplies the grading prompt. There is no human-analyst ground truth, no check that the rated evidence corresponds to actual database contents, and no external audit of relevance or logical consistency. Thus the claims 'all evidence was relevant' and 'no logical inconsistencies' are the model's self-assessment of the model's own output, not an independent measurement. The evidence-quality metric reduces by construction to LLM self-agreement.

full rationale

The only circular step I can exhibit is the evidence-quality measure (Section 4.3.3 / Listing 7): 'high-quality evidence' is operationalized as ratings assigned by an LLM to a report generated by the same model, with no human or database ground truth. The paper's RQ1 conclusions—all evidence relevant, no logical inconsistencies, 71-76% high impact—are therefore the model agreeing with itself. The detective-agent F1 (Section 4.5) is not formally circular: it compares the LLM detective's decision against dataset labels, an external criterion; weaknesses there (no grounding audit, possible dataset artifacts) are correctness risks, not circularity. The memorization check (0.58 accuracy, n=50) is statistically weak but not circular. No load-bearing self-citation or imported uniqueness theorem appears. Because one of the two pillars of the 'reliable investigations' claim is self-referential while the other is externally benchmarked, the overall circularity is partial.

Assumptions & free parameters 1 free parameters · 4 assumptions · 2 invented entities

The framework relies on proprietary LLM behavior and self-referential evaluation. No quantitative parameters are fitted, but the methodology makes strong assumptions about the reliability of LLM-generated evidence and LLM-based quality assessment. The invented metrics lack independent validation.

free parameters (1)
  • None
    The paper does not fit any numerical parameters to data. The framework hyperparameters, such as the stopping condition and the evidence quality Likert scale, are qualitative choices rather than estimated parameters.
assumptions (4)
  • domain assumption GPT-4o produces reliable investigative reasoning, code execution, and chart interpretation.
    The entire framework depends on the proprietary model behaving correctly; no open alternative or independent verification is provided.
  • ad hoc to paper LLM-based evaluation of evidence quality correlates with human expert judgment.
    Section 4.3.3 uses an LLM to rate evidence on four aspects without validating the ratings against human fraud analysts.
  • domain assumption The Sparkov and CCTD datasets adequately represent real-world credit card transactions.
    Both datasets are synthetic and may not capture the complexity of real fraud patterns; the paper does not test on real transaction data.
  • ad hoc to paper The memorization check with 50 samples is sufficient to rule out data memorization.
    Section 4.2 uses only 50 transactions, which yields wide confidence intervals and is underpowered to detect memorization reliably.
invented entities (2)
  • Evidence Quality Score
    purpose: Rates each piece of evidence on four aspects (impact, relevance, new knowledge, logical alignment) using an LLM judge.
    The score is not validated against human expert ratings; it is an internal LLM judgment and lacks external falsifiable grounding.
  • Minimal Trajectory Ratio
    purpose: Measures investigation efficiency as the fraction of steps that support the final decision.
    The assessment of which steps are 'supporting' is done retrospectively without a pre-specified criterion and is not validated against human expert judgments.

how reviews work

0 comments
Cite this review

Pith. "Pith review of FAA Framework: A Large Language Model-Based Approach for Credit Card Fraud Investigations." pith.science (2026). https://pith.science/paper/CSVI73IZ

@misc{pith2026250611635,
  author       = {Pith},
  title        = {Pith review of: FAA Framework: A Large Language Model-Based Approach for Credit Card Fraud Investigations},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/CSVI73IZ}},
  note         = {Machine review of arXiv:2506.11635}
}
read the original abstract

The continuous growth of the e-commerce industry attracts fraudsters who exploit stolen credit card details. Companies often investigate suspicious transactions in order to retain customer trust and address gaps in their fraud detection systems. However, analysts are overwhelmed with an enormous number of alerts from credit card transaction monitoring systems. Each alert investigation requires from the fraud analysts careful attention, specialized knowledge, and precise documentation of the outcomes, leading to alert fatigue. To address this, we propose a fraud analyst assistant (FAA) framework, which employs multi-modal large language models (LLMs) to automate credit card fraud investigations and generate explanatory reports. The FAA framework leverages the reasoning, code execution, and vision capabilities of LLMs to conduct planning, evidence collection, and analysis in each investigation step. A comprehensive empirical evaluation of 500 credit card fraud investigations demonstrates that the FAA framework produces reliable and efficient investigations comprising seven steps on average. Thus we found that the FAA framework can automate large parts of the workload and help reduce the challenges faced by fraud analysts.

Figures

Figures reproduced from arXiv: 2506.11635 by the authors.

Figure 1
Figure 1. A typical fraud detection framework workflow. The process begins [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 2
Figure 2. 3.2.2. Tools. Code Execution Tool. Code Interpreter allows the FAA to write and run Python code in a sandboxed execution environment. This tool can process files such as the transaction database and output the execution’s result. Code Interpreter allows the FAA to run code iteratively to resolve the investigation. When the FAA writes code that fails to run, it can iterate on this code by attempting to run different … view at source ↗
Figure 2
Figure 2. The FAA framework for automating fraud investigation. This includes performing a sequence of investigation steps until the FAA determines [PITH_FULL_IMAGE:figures/full_fig_p006_2.png] view at source ↗
Figures from the paper (3 more)
Figure 3
Figure 3. Figure 3: The communication and workflow between the FAA framework components: the FAA, tools, and agents. [PITH_FULL_IMAGE:figures/full_fig_p006_3.png]
Figure 5
Figure 5. Figure 5: Categories of the investigation steps performed by the FAA [PITH_FULL_IMAGE:figures/full_fig_p009_5.png]
Figure 6
Figure 6. Figure 6: Comparison of the quality of evidence in investigations performed without vision agent and with vision agent across four aspects. [PITH_FULL_IMAGE:figures/full_fig_p011_6.png]

Discussion (0). Sign in to comment.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. Operational Evidence Gaps for LLMs in Fraud Detection and Trust-and-Safety Workflows

    cs.CR 2026-07 conditional novelty 5.0 of 10

    A survey of 49 LLM fraud and trust-and-safety papers finds that fraud work reports almost no per-decision latency, cost, or calibration evidence, while moderation work reports more.

Reference graph

Works this paper leans on

57 extracted references · 50 canonical work pages · cited by 1 Pith paper

  1. [1]

    Understanding credit card frauds,

    T. P. Bhatla, V . Prabhu, and A. Dua, “Understanding credit card frauds,” Cards business review, vol. 1, no. 6, pp. 1–15, 2003

    work page 2003

  2. [2]

    Identity fraud victimization: a critical review of the literature of the past two decades,

    Y . Irvin-Erickson, “Identity fraud victimization: a critical review of the literature of the past two decades,” Crime Science, vol. 13, no. 1, p. 3, 2024

    work page 2024

  3. [3]

    Credit card fraud: aware- ness and prevention,

    K. J. Barker, J. D’amato, and P. Sheridon, “Credit card fraud: aware- ness and prevention,” Journal of financial crime , vol. 15, no. 4, pp. 398–410, 2008

    work page 2008

  4. [4]

    Man-in-the-middle attack on contactless payment over nfc communications: design, implementation, experiments and detection,

    S. Akter, S. Chellappan, T. Chakraborty, T. A. Khan, A. Rahman, and A. A. Al Islam, “Man-in-the-middle attack on contactless payment over nfc communications: design, implementation, experiments and detection,” IEEE Transactions on Dependable and Secure Computing, vol. 18, no. 6, pp. 3012–3023, 2020

    work page 2020

  5. [5]

    Data breaches: What the underground world of

    K. K. Peretti, “Data breaches: What the underground world of” carding” reveals,” Santa Clara Computer & High Tech. LJ , vol. 25, p. 375, 2008

    work page 2008

  6. [6]

    Identity theft, malware, and social engineering in dealing with cybercrime,

    A. Mishra, B. B. Gupta, and D. Gupta, “Identity theft, malware, and social engineering in dealing with cybercrime,” in Computer and Cyber Security. Auerbach Publications, 2018, pp. 627–648

    work page 2018

  7. [7]

    Payment card fraud losses reach 32.34 billion,

    S. BARBARA, “Payment card fraud losses reach 32.34 billion,” 2022, accessed: 2024-07-25. [Online]. Available: https://www.globenewswire.com/news-release/2022/12/22/2578877/ 0/en/Payment-Card-Fraud-Losses-Reach-32-34-Billion.html

    work page 2022

  8. [8]

    Effective detection of sophisticated online banking fraud on extremely imbalanced data,

    W. Wei, J. Li, L. Cao, Y . Ou, and J. Chen, “Effective detection of sophisticated online banking fraud on extremely imbalanced data,” World Wide Web, vol. 16, pp. 449–475, 2013

    work page 2013

Show all 57 references
  1. [9]

    Fintech forensics: Criminal investigation and digital evidence in financial technologies,

    B. Nikkel, “Fintech forensics: Criminal investigation and digital evidence in financial technologies,” Forensic Science International: Digital Investigation, vol. 33, p. 200908, 2020

    2020

  2. [10]

    Digital forensic research: current state of the art,

    S. Raghavan, “Digital forensic research: current state of the art,” Csi Transactions on ICT , vol. 1, pp. 91–114, 2013

    2013

  3. [11]

    An examination of digital forensic models,

    M. Reith, C. Carr, and G. Gunsch, “An examination of digital forensic models,” International Journal of digital evidence , vol. 1, no. 3, pp. 1–12, 2002

    2002

  4. [12]

    The investigation team and their roles: The important role of investigation team,

    F. C. Academy, “The investigation team and their roles: The important role of investigation team,” 2024, accessed: 2024-07-27. [Online]. Available: https://financialcrimeacademy.org/ the-investigation-team-and-their-roles/

    2024

  5. [13]

    Credit card fraud detection: A realistic modeling and a novel learn- ing strategy,

    A. Dal Pozzolo, G. Boracchi, O. Caelen, C. Alippi, and G. Bontempi, “Credit card fraud detection: A realistic modeling and a novel learn- ing strategy,” IEEE Transactions on Neural Networks and Learning Systems, vol. 29, no. 8, pp. 3784–3797, 2018

    2018

  6. [14]

    Exploring llm-based agents for root cause analysis,

    D. Roy, X. Zhang, R. Bhave, C. Bansal, P. Las-Casas, R. Fonseca, and S. Rajmohan, “Exploring llm-based agents for root cause analysis,” in Companion Proceedings of the 32nd ACM International Conference on the Foundations of Software Engineering , 2024, pp. 208–219

    2024

  7. [15]

    Automatic root cause analysis via large lan- guage models for cloud incidents,

    Y . Chen, H. Xie, M. Ma, Y . Kang, X. Gao, L. Shi, Y . Cao, X. Gao, H. Fan, M. Wen et al., “Automatic root cause analysis via large lan- guage models for cloud incidents,” in Proceedings of the Nineteenth European Conference on Computer Systems , 2024, pp. 674–688

    2024

  8. [16]

    Cyber sentinel: Exploring conversational agents in streamlining security tasks with gpt-4,

    M. Kaheh, D. K. Kholgh, and P. Kostakos, “Cyber sentinel: Exploring conversational agents in streamlining security tasks with gpt-4,” arXiv preprint arXiv:2309.16422, 2023

    2023 arXiv

  9. [17]

    A framework for llm- assisted smart policing system,

    P. Sarzaeim, Q. H. Mahmoud, and A. Azim, “A framework for llm- assisted smart policing system,” IEEE Access, 2024

    2024

  10. [18]

    Sparkov data generation,

    N. Brandon, “Sparkov data generation,” GitHub repository, 2016. [Online]. Available: https://github.com/namebrandon/Sparkov Data Generation

    2016

  11. [19]

    Credit card transactions,

    E. Altman, “Credit card transactions,” 2019, apache License 2.0. [Online]. Available: https://www.kaggle.com/datasets/ealtman2019/ credit-card-transactions

    2019

  12. [20]

    Graph neural network for fraud detection via spatial-temporal attention,

    D. Cheng, X. Wang, Y . Zhang, and L. Zhang, “Graph neural network for fraud detection via spatial-temporal attention,” IEEE Transactions on Knowledge and Data Engineering , vol. 34, no. 8, pp. 3800–3813, 2020

    2020

  13. [21]

    Rule-based credit card fraud detection using user’s keystroke behavior,

    J. Kumar and V . Saxena, “Rule-based credit card fraud detection using user’s keystroke behavior,” in Soft Computing: Theories and Applications: Proceedings of SoCTA 2021. Springer, 2022, pp. 469– 480

    2021

  14. [22]

    Fraud detection of credit card payment system by genetic algorithm,

    K. RamaKalyani and D. UmaDevi, “Fraud detection of credit card payment system by genetic algorithm,” International Journal of Sci- entific & Engineering Research , vol. 3, no. 7, pp. 1–6, 2012

    2012

  15. [23]

    Credit card fraud detection in the era of disruptive tech- nologies: A systematic review,

    A. Cherif, A. Badhib, H. Ammar, S. Alshehri, M. Kalkatawi, and A. Imine, “Credit card fraud detection in the era of disruptive tech- nologies: A systematic review,” Journal of King Saud University- Computer and Information Sciences , vol. 35, no. 1, pp. 145–174, 2023

    2023

  16. [24]

    Performance evaluation of machine learning methods for credit card fraud detection using smote and adaboost,

    E. Ileberi, Y . Sun, and Z. Wang, “Performance evaluation of machine learning methods for credit card fraud detection using smote and adaboost,” IEEE Access, vol. 9, pp. 165 286–165 294, 2021

    2021

  17. [25]

    A machine learning based credit card fraud detection using the ga algorithm for feature selection,

    ——, “A machine learning based credit card fraud detection using the ga algorithm for feature selection,” Journal of Big Data, vol. 9, no. 1, p. 24, 2022. [Online]. Available: https: //doi.org/10.1186/s40537-022-00573-8

    2022 doi

  18. [26]

    Semi-supervised credit card fraud detection via attribute- driven graph representation,

    S. Xiang, M. Zhu, D. Cheng, E. Li, R. Zhao, Y . Ouyang, L. Chen, and Y . Zheng, “Semi-supervised credit card fraud detection via attribute- driven graph representation,” in Proceedings of the AAAI Conference on Artificial Intelligence , vol. 37, no. 12, 2023, pp. 14 557–14 565

    2023

  19. [27]

    Enhancing credit card fraud detection through advanced ensemble learning tech- niques and deep learning integration,

    M. Dhandore, C. Agrawal, M. Meena, and I. Journal, “Enhancing credit card fraud detection through advanced ensemble learning tech- niques and deep learning integration,” vol. 5, pp. 2582–6948, 09 2024

    2024

  20. [28]

    Identifying fraudulent credit card transactions using ensemble learning,

    J. Jemai, A. Zarrad, and A. Daud, “Identifying fraudulent credit card transactions using ensemble learning,” IEEE Access , vol. 12, pp. 54 893–54 900, 2024

    2024

  21. [29]

    Explainable ai: current status and future directions,

    P. Gohel, P. Singh, and M. Mohanty, “Explainable ai: current status and future directions,” 2021

    2021

  22. [30]

    xfraud: explainable fraud transaction detection,

    S. X. Rao, S. Zhang, Z. Han, Z. Zhang, W. Min, Z. Chen, Y . Shan, Y . Zhao, and C. Zhang, “xfraud: explainable fraud transaction detection,” Proceedings of the VLDB Endowment , vol. 15, no. 3, p. 427–436, Nov. 2021. [Online]. Available: http://dx.doi.org/10.14778/3494124.3494128

    2021

  23. [31]

    Heterogeneous graph trans- former,

    Z. Hu, Y . Dong, K. Wang, and Y . Sun, “Heterogeneous graph trans- former,” 2020

    2020

  24. [32]

    Gnnex- plainer: Generating explanations for graph neural networks,

    R. Ying, D. Bourgeois, J. You, M. Zitnik, and J. Leskovec, “Gnnex- plainer: Generating explanations for graph neural networks,” 2019

    2019

  25. [33]

    of Certified Fraud Examiners, Fraud Examiners Manual

    A. of Certified Fraud Examiners, Fraud Examiners Manual. ACFE, 2024, accessed: 2024-07-17. [Online]. Available: https://www.acfe.com/training-events-and-products/ cfe-exam-preparation/cfe-exam-prep---study-on-your-own-options/ fraud-examiners-manual

    2024

  26. [34]

    Credit card fraud investigation: How it works & why it’s important,

    Chargebacks911, “Credit card fraud investigation: How it works & why it’s important,” 2023, accessed: 12 September 2024. [Online]. Available: https://chargebacks911.com/ credit-card-fraud-investigation/

    2023

  27. [35]

    W. V . P. Bart Baesens, V ´eronique Van Vlasselaer, FRAUD ANALYTICS . Wiley, 2015, accessed: 2024-09-12. [Online]. Available: https://onlinelibrary.wiley.com/doi/book/10.1002/ 9781119146841?msockid=34b4c9be4ab86f9a36b2dbee4bc56ef9

    2015

  28. [36]

    Investigation report example: How to write an investigative report,

    C. IQ, “Investigation report example: How to write an investigative report,” https://www.caseiq.com/resources/ ultimate-guide-to-writing-investigation-reports/, 2024, accessed: 2024-07-28

    2024

  29. [37]

    A framework for integrated digital forensic investigation employing autogen ai agents,

    A. Wickramasekara and M. Scanlon, “A framework for integrated digital forensic investigation employing autogen ai agents,” in 2024 12th International Symposium on Digital Forensics and Security (ISDFS), 2024, pp. 01–06

    2024

  30. [38]

    An llm- driven approach to gain cybercrime insights with evidence networks,

    H. Zhou, W. Xu, J. Dehlinger, S. Chakraborty, and L. Deng, “An llm- driven approach to gain cybercrime insights with evidence networks,” usnix, 2024

    2024

  31. [39]

    Chatids: Explainable cybersecurity using generative ai,

    V . J ¨uttner, M. Grimmer, and E. Buchmann, “Chatids: Explainable cybersecurity using generative ai,” 2023

    2023

  32. [40]

    Unleashing the poten- tial of prompt engineering in large language models: a comprehensive review,

    B. Chen, Z. Zhang, N. Langren ´e, and S. Zhu, “Unleashing the poten- tial of prompt engineering in large language models: a comprehensive review,” arXiv preprint arXiv:2310.14735 , 2023

    2023 arXiv

  33. [41]

    How to conduct an effective fraud in- vestigation: Your complete guide,

    C. IQ, “How to conduct an effective fraud in- vestigation: Your complete guide,” accessed: 2024- 08-07. [Online]. Available: https://www.caseiq.com/resources/ how-to-conduct-a-fraud-investigation-the-complete-guide/#analyze

    2024

  34. [42]

    Openai assistants overview,

    OpenAI, “Openai assistants overview,” 2024, accessed: 2024-08-

    2024

  35. [43]

    Available: https://platform.openai.com/docs/assistants/ overview

    [Online]. Available: https://platform.openai.com/docs/assistants/ overview

  36. [44]

    Hello gpt-4o,

    ——, “Hello gpt-4o,” accessed: 14-Aug-2024. [Online]. Available: https://openai.com/index/hello-gpt-4o/

    2024

  37. [45]

    Elephants never forget: Testing language models for memorization of tabular data,

    S. Bordt, H. Nori, and R. Caruana, “Elephants never forget: Testing language models for memorization of tabular data,” arXiv preprint arXiv:2403.06644, 2024

    2024 arXiv

  38. [46]

    Towards optimizing the costs of llm usage,

    S. Shekhar, T. Dubey, K. Mukherjee, A. Saxena, A. Tyagi, and N. Kotla, “Towards optimizing the costs of llm usage,” arXiv preprint arXiv:2402.01742, 2024

    2024 arXiv

  39. [47]

    Gptcache: An open-source semantic cache for llm appli- cations enabling faster answers and cost savings,

    F. Bang, “Gptcache: An open-source semantic cache for llm appli- cations enabling faster answers and cost savings,” in Proceedings of the 3rd Workshop for Natural Language Processing Open Source Software (NLP-OSS 2023) , 2023, pp. 212–218

    2023

  40. [48]

    Hybrid llm: Cost-efficient and quality-aware query routing,

    D. Ding, A. Mallick, C. Wang, R. Sim, S. Mukherjee, V . Ruhle, L. V . Lakshmanan, and A. H. Awadallah, “Hybrid llm: Cost-efficient and quality-aware query routing,” arXiv preprint arXiv:2404.14618, 2024

    2024 arXiv

  41. [49]

    Routellm: Learning to route llms with preference data,

    I. Ong, A. Almahairi, V . Wu, W.-L. Chiang, T. Wu, J. E. Gonzalez, M. W. Kadous, and I. Stoica, “Routellm: Learning to route llms with preference data,” arXiv preprint arXiv:2406.18665 , 2024

    2024 arXiv

  42. [50]

    tiktoken,

    OpenAI, “tiktoken,” 2024, accessed: 2024-08-14. [Online]. Available: https://github.com/openai/tiktoken

    2024

  43. [51]

    Enhancing credit card fraud detection through advanced ensemble learning techniques and deep learning integration

    M. D. Dhandore, M. C. Agrawal, and M. P. Meena, “Enhancing credit card fraud detection through advanced ensemble learning techniques and deep learning integration.”

  44. [52]

    Identifying fraudulent credit card transactions using ensemble learning,

    J. Jemai, A. Zarrad, and A. Daud, “Identifying fraudulent credit card transactions using ensemble learning,” IEEE Access, 2024

    2024

  45. [53]

    Leveraging graph-based learning for credit card fraud detection: a comparative study of clas- sical, deep learning and graph-based approaches,

    S. Harish, C. Lakhanpal, and A. H. Jafari, “Leveraging graph-based learning for credit card fraud detection: a comparative study of clas- sical, deep learning and graph-based approaches,” Neural Computing and Applications, vol. 36, no. 34, pp. 21 873–21 883, 2024. Appendix

    2024

  46. [54]

    This is the transaction number you need to investigate: trans_num (str): ’{trans_num}.’ Important: You must stop after each step of investigation! Listing 1

    Prompts A new alert for a transaction we suspect might be fraudulent has come to our system. This is the transaction number you need to investigate: trans_num (str): ’{trans_num}.’ Important: You must stop after each step of investigation! Listing 1. The message is given to th...

  47. [55]

    Planning Phase: Use your current evidence and domain knowledge to generate new ideas to determine the next steps in assessing whether the transaction is fraudulent or not

  48. [56]

    This may include creating a plot and retrieving relevant data

    Information-Gathering Phase: Implement and execute the code necessary to carry out the steps outlined in the planning phase. This may include creating a plot and retrieving relevant data

  49. [57]

    report_evaluation

    Analysis Phase: This phase involves analyzing and interpreting the newly gathered evidence. This evidence is then integrated with information from previous steps to derive meaningful insights about the case. At this point, the LLM may choose to stop if it determines that enoug...

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.