Pith. sign in

REVIEW 4 major objections 5 minor 2 cited by

Agentic AI Systems Applied to tasks in Financial Services: Modeling and model risk management crews

T0 review · 4 major / 5 minor · reviewed 2026-08-08 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read Two multi-agent “crews” of LLM specialists, coordinated by a human expert and a judge agent, can carry out financial model development and independent model risk management on standard credit datasets.

desk verdict A sincere, well-documented HITL system demo that deserves peer review, but the claimed agentic effectiveness is under-supported because the human expert supplies the core modeling decisions. read the letter →

arxiv 2502.05439 v2 pith:P6JHRJOB submitted 2025-02-08 cs.AI cs.CEcs.CLcs.LG

Izunna Okpala , Ashkan Golgoon , Arjun Ravi Kannan This is my paper · ORCID
classification cs.AIcs.CEcs.CLcs.LG
keywords LargeLanguageModelsMulti-AgentSystemsAgenticCollaborationHuman-in-the-LoopModelRiskManagementFinancialModelingCredit
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper tries to show that a small team of specialized AI agents—an “agentic crew”—can perform two jobs that banks normally assign to separate groups: building a machine learning model and independently checking that model for risk. The modeling crew runs exploratory data analysis, feature engineering, model selection, training, and evaluation; the model risk management crew replicates the model, checks documentation against a modeling guide, examines conceptual soundness, and stress-tests the model on shifted and adversarial inputs. A human expert orchestrates the workflow, with a judge agent reviewing the other agents’ code and output. On credit card fraud, credit card approval, and portfolio credit risk datasets, the crews produce models whose metrics are comparable to AutoML and Kaggle baselines, while the risk-management tests reveal where the models degrade under distribution shift.

What carries the argument

The crews themselves are the central mechanism: modular teams of role-playing LLM agents organized in a hierarchical human-agent-task-tool structure. Each agent carries a persona and backstory, uses a memory stream to pass task context and results to the next agent, and activates specialized tools (Python code execution, EDA analysis, cache-augmented generation for compliance checking); a judge agent audits the others’ code and output, and the human expert delegates tasks, adds instructions, and corrects errors. This combination of role specialization, memory chaining, and human oversight is what the paper credits for keeping outputs aligned with financial modeling standards.

What would settle it

Run the modeling and MRM crews on a new, unseen financial dataset without any human feedback or code corrections, and compare the autonomously produced metrics to those from a standard AutoML run; if the agents require frequent human fixes, silently produce wrong results, or cannot reproduce their own metrics, the claimed effectiveness and robustness is not supported.

Watch

Extended reading notes

Core claim

The central claim is that LLM-powered multi-agent crews, built on a role-playing collaboration framework with a human-in-the-loop as orchestrator, can effectively collaborate on complex financial modeling and model risk management tasks. Each agent has a persona (data analyst, data scientist, machine learning engineer, model validation analyst, or documentation writer), a shared memory stream that chains outputs between stages, and tools such as a Python code executor, an exploratory-data-analysis tool, and a cache-augmented-generation checker. The judge agent reviews the actions and code of the other agents and reports to the human, who can inject feedback and corrections at any step. The paper reports that on the three datasets the agentic pipeline selects competitive models (CatBoost and XGBoost), produces evaluation metrics on par with AutoML and popular Kaggle solutions, and generates model risk reports that expose significant performance drops under shifted inputs while remaining robust to outlier inputs.

Load-bearing premise

The load-bearing premise is that the LLM agents generate correct-enough Python code and reasoning on their own most of the time, so a human expert can catch the residual failures; the paper itself concedes a 1-10% chance that outputs do not match expectations and notes that the authors manually verified every output, so this reliability assumption has not been tested outside the authors' controlled runs.

Editorial extensions

If this is right

  • Financial model development can be compressed into a sequence of delegated agent tasks, with the human expert reviewing handoffs rather than writing every step by hand.
  • Independent model risk management can be run as a parallel crew, reproducing the model, checking documentation compliance, and stress-testing under shifted and adversarial inputs.
  • Agentic results are competitive with automated machine learning baselines, so model quality need not be sacrificed for automation.
  • MRM stress tests can surface real vulnerabilities—for example, accuracy holding while precision and F1 collapse under input shift—making model risk reporting a systematic byproduct of the workflow.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The same crew architecture could be transplanted to other regulated modeling tasks—insurance pricing, anti-money-laundering screening, or climate-risk scoring—where documentation and independent validation are mandatory.
  • The compliance-check step is only as authoritative as the modeling guide it checks; because the paper’s organizational guide is AI-generated and illustrative, the demonstration establishes workflow feasibility, not regulatory approval.
  • As LLM code reliability improves and the judge agent gets better at spotting subtle errors, the human role could shift from checking every artifact to managing exceptions, which would change how much staff time these workflows actually save.
  • A cost-sensitive deployment test comparing total human audit hours and residual agent failure rates against manual modeling would quantify the practical benefit that this paper demonstrates qualitatively.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. The paper proposes two CrewAI-based agentic crews: a modeling crew (data extraction, EDA, feature engineering, meta-tuning, model training, model evaluation, documentation, and a judge agent) and a model risk management crew (documentation compliance, model replication, conceptual soundness, outcome analysis, documentation, and a judge agent). A human expert acts as system orchestrator, delegating tasks and supplying corrective feedback. The framework is demonstrated on three public financial datasets: credit card fraud detection, credit card approval, and portfolio credit risk. The reported model metrics are competitive with external Kaggle solutions and H2O AutoML baselines, and the MRM crew produces replication, feature-importance, and stress-test reports. The central claim is that such crews can effectively and robustly collaborate on modeling and MRM tasks with human oversight.

Significance. If the central claim were established with proper controls, this would be a useful demonstration of LLM multi-agent workflows in a regulated financial-services setting. The paper has real strengths: it benchmarks against external Kaggle and H2O AutoML solutions, includes detailed interaction logs (Logs 1-4), uses public datasets, and candidly reports manual verification (§3.5.5) and a 1-10% residual-error estimate (§3.1). These features make the workflow transparent at the narrative level and would serve as a reproducible case study if code and prompts were released. However, the evidence does not yet separate the human expert's substantive modeling decisions from agent contributions, and the robustness and compliance evidence is not fully specified. The contribution is best read as an architectural demonstration until those gaps are closed.

major comments (4)
  1. [§3.1, Logs 1-4] The human expert, not the agents, supplies the core modeling methodology. In Log 2 the human specifies the preprocessing recipe (KNN imputation, standard scaler, ordinal encoding, KBest), and in Log 3 the human specifies the tuning protocol (use the preprocessor, 40% sample, CV=5, roc_auc). These are substantive modeling decisions, not the residual 1-10% corrections estimated in §3.1. Because the paper reports no ablation or control condition that removes or randomizes such human direction, the results are compatible with the human performing the complex modeling while agents act as code-generating and executing tools. Please add an ablation (e.g., generic task prompts without human-specified recipes, a single-LLM control given the same instructions, or a quantitative ledger of human decisions versus agent decisions) to support the claimed agentic effectiveness.
  2. [§3.4, Appendix B] The Outcome Analyzer's perturbation procedure is not specified enough to support the robustness claim. §3.4 says shifted inputs are regenerated by multiplication or addition of fixed/randomized values, and Appendix B reports only the final code outcome; no seed, perturbation magnitudes, feature selection rule, or generation algorithm is given. All reported metrics in §3.5 are single-run point estimates with no confidence intervals or repeated trials, and the 1-10% failure estimate in §3.1 is not derived from observed error counts. Please provide the exact perturbation algorithm and parameters, report variability across runs or seeds, and give empirical failure counts so the robustness results are reproducible and statistically meaningful.
  3. [§3.4, footnote 1] The compliance check is performed against an AI-generated modeling guide created for this paper. Footnote 1 states the guide 'is AI-generated and intended for illustrative purposes only' and is not the actual internal guide of the employer. The MRM claim that the modeling crew 'adheres to the organization's guidelines' is therefore self-referential: it compares one LLM output against another LLM-generated standard. Please evaluate the documentation against an independent, authoritative standard (e.g., published regulatory guidance or an externally provided modeling checklist) or explicitly re-scope the claim as an internal consistency check.
  4. [§3.5.3] The class-imbalance statement appears internally inconsistent. The text says '78.18% of instances belonging to the majority class (‘loan_status’ = 1)' after defining loan_status=1 as default. In the cited Kaggle credit-risk dataset, the majority class is non-default (loan_status=0) at roughly 78%. If the paper's statement is correct, the dataset is pathological; if incorrect, the data description and conclusions about imbalance are wrong. Please verify the target encoding and correct the text, because the model's precision/recall tradeoff and the MRM stress-test interpretation depend on which class is the majority.
minor comments (5)
  1. [Eq. (2.1)] Equation (2.1) is typeset incorrectly: the traditional softmax denominator is not shown as a sum over j, and the temperature-adjusted formula lacks parentheses. Please rewrite it with proper LaTeX.
  2. [Abstract and §1] The abstract says the crews 'can effectively collaborate' but the paper never defines success criteria for effectiveness; please state quantitative acceptance criteria before the experiments.
  3. [Logs and general formatting] The log transcripts contain many typographical artifacts (e.g., 'spliting', 'ana lys is', 'e xte rn al'), which should be cleaned; the Keywords line also has a double colon.
  4. [Reproducibility] No code repository or full prompt templates are provided; consider releasing them in a supplementary artifact to support reproducibility of the claimed workflow.
  5. [§3.5.2] The phrase 'record-high metrics' for the Kaggle benchmark is vague and could be misread as a state-of-the-art claim; please rephrase it as 'high on the leaderboard' or give the specific leaderboard context.

Circularity Check

1 steps flagged · score 3.0 of 10

Core modeling results are externally benchmarked, but the MRM compliance check validates against a self-generated AI guide rather than an independent standard.

  1. other [Section 3.4, Documentation Compliance Checker agent description and footnote 1; Log 4 in Appendix B.]
    "This organizational modeling guide is AI-generated and intended for illustrative purposes only. It is not affiliated with, endorsed by, or reflective of the actual internal modeling guide of the Discover Financial Services."

    The MRM crew's compliance validation is performed by comparing the modeling crew's documentation against an 'organizational modeling guide' that the paper itself generated for illustrative purposes. Log 4 shows the guide is a generic 'Comprehensive Guide to Machine Learning Pipeline Development' whose enumerated stages (EDA, Feature Engineering, Model Selection, Hyperparameter Tuning, Model Training, Model Evaluation) are exactly the same stage decomposition as the modeling crew's agents. The checker's positive finding is therefore a comparison of two artifacts constructed from the same checklist, not validation against an external regulatory or institutional standard.

full rationale

The paper's central modeling-claim evidence is not circular: the agentic modeling results are benchmarked against external H2O AutoML runs and public Kaggle solutions, and the reported metrics (accuracy, precision, recall, F1, AUC) come from held-out test data. No self-citation chain is load-bearing, and there is no uniqueness theorem or fitted-parameter-as-prediction pattern. The human-in-the-loop orchestration shown in Logs 1-4 is a validity limitation rather than circularity: the human supplies substantive modeling decisions (e.g., drop Time, 80/20 split, KNN imputation, ordinal encoding, KBest, CV=5, roc_auc), so the demonstration does not isolate agentic capability, but the paper's claim explicitly includes a human-in-the-loop module, so this is under-support rather than a derivation that reduces to its own inputs. The one genuinely self-referential component is the MRM Documentation Compliance Checker: the 'organizational modeling guide' used as the compliance benchmark is AI-generated for this paper, and the guide's stage list matches the modeling crew's own task decomposition, making the compliance finding a comparison of two artifacts built from the same checklist. This affects only the compliance portion of the MRM demonstration, not the externally benchmarked modeling results, so the overall circularity score is moderate.

Assumptions & free parameters 6 free parameters · 6 assumptions · 0 invented entities

The paper's conclusions rest on the reliability of LLM code generation, the adequacy of CrewAI as an orchestration layer, the validity of the chosen baseline benchmarks, and the validity of the AI-generated modeling guide as a compliance reference. There is no formal derivation; the claim is empirical.

free parameters (6)
  • train_test_split_ratio = 0.8/0.2
    Chosen by hand in Log 1 and used for all datasets; affects all reported metrics.
  • meta_tuning_subsample_fraction = 0.4 (40% of training set)
    Selected to speed up grid search; changes which hyperparameters are selected.
  • llm_temperature = 0.2
    Chosen in §3.1 to balance creativity and precision; controls stochasticity of every agent's output.
  • shifted_input_generation = Unspecified random reassignment of non-categorical features
    Central to MRM stress test; no distribution or magnitude is given (§3.4, Log 4).
  • outlier_input_generation = Unspecified multiplication/addition beyond original range
    Central to MRM stress test; no magnitude is specified (§3.4, Log 4).
  • model_hyperparameter_grids = Not reported
    GridSearchCV ranges for XGBoost, CatBoost, LightGBM, and others are not listed, so model selection is not reproducible.
assumptions (6)
  • domain assumption CrewAI supplies working memory, role playing, tool use, and delegation as described.
    The whole pipeline is built on CrewAI (§2.1, §3.2); if CrewAI's behavior differs, the results change.
  • domain assumption LLM agents generate correct Python code for EDA, feature engineering, model training, and evaluation.
    Agents rely on a code execution tool for these tasks (§3.3); errors require human correction, as shown in Log 4.
  • domain assumption Public Kaggle datasets and baseline results are valid external benchmarks.
    Models are compared to existing Kaggle solutions and H2O AutoML (§3.5).
  • ad hoc to paper The AI-generated modeling guide is a sufficient compliance reference.
    Footnote 1 states the guide is AI-generated and illustrative, not affiliated with Discover; the compliance check in Log 4 asserts adherence to this guide.
  • domain assumption Standard ML metrics are appropriate for class-imbalanced data.
    Accuracy, precision, recall, F1, and AUC are used without discussion of class-imbalance caveats.
  • domain assumption Human audit catches all critical agent failures.
    The paper relies on human-in-the-loop and manual verification (§3.1, §3.5.5) to ensure that agent errors do not corrupt the final model.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Agentic AI Systems Applied to tasks in Financial Services: Modeling and model risk management crews." pith.science (2026). https://pith.science/paper/P6JHRJOB

@misc{pith2026250205439,
  author       = {Pith},
  title        = {Pith review of: Agentic AI Systems Applied to tasks in Financial Services: Modeling and model risk management crews},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/P6JHRJOB}},
  note         = {Machine review of arXiv:2502.05439}
}
read the original abstract

The advent of large language models has ushered in a new era of agentic systems, where artificial intelligence programs exhibit remarkable autonomous decision-making capabilities across diverse domains. This paper explores agentic system workflows in the financial services industry. In particular, we build agentic crews with human-in-the-loop module that can effectively collaborate to perform complex modeling and model risk management (MRM) tasks. The modeling crew consists of a judge agent and multiple agents who perform specific tasks such as exploratory data analysis, feature engineering, model selection/hyperparameter tuning, model training, model evaluation, and writing documentation. The MRM crew consists of a judge agent along with specialized agents who perform tasks such as checking compliance of modeling documentation, model replication, conceptual soundness, analysis of outcomes, and writing documentation. We demonstrate the effectiveness and robustness of modeling and MRM crews by presenting a series of numerical examples applied to credit card fraud detection, credit card approval, and portfolio credit risk modeling datasets.

Figures

Figures reproduced from arXiv: 2502.05439 by the authors.

Figure 1
Figure 1. General components of an LLM-based agent (adapted from [58]) Another way to tackle the issue of safety and harm in agentic systems is the use of guardrails. Guardrails are a set of rules that ensure operational safety and ethical practices in machine learning applications. Their implementation can be in the form of a layered protection model, system prompts, retrieval-augmented gen￾eration (RAG) [29] architectures, … view at source ↗
Figure 2
Figure 2. Agentic system collaboration structure: Horizontal Collaboration (left), Hierarchical Collabora￾tion (middle), Nested Collaboration (right) (adapted from [14]) selection. In Equation 2.1, xi is the raw score (logit) for class i, T is the temperature parameter, and j is the index that iterates over all classes in the set of possible classes. (2.1) T raditional Sof tmax e xi Pe xj .... T emperature − adjusted sof tmax… view at source ↗
Figure 3
Figure 3. Mind-map demo of the agentic system 3.2. Financial crews system architecture. The proposed system comprises two interconnected crews and tools. Within the crews, we have several autonomous agents, each responsible for distinct tasks within the pipeline. The architecture is designed to promote modularity, allowing agents to operate independently and collaboratively. The key crews and agents include: (1) Modeling Crew… view at source ↗
Figures from the paper (7 more)
Figure 4
Figure 4. Figure 4: Memory, delegation and information retrieval 3.2.1. Memory, delegation and information retrieval. The memory property is most effective when individual agents store their inputs and outputs in memory, allowing for effective transfer of knowledge to the other agents. Si…
Figure 5
Figure 5. Figure 5: Visual representation of the modeling crew (6) Model Evaluation Agent: This agent is responsible for evaluating the trained model using the test data. It uses the same LLM engine as the meta-tuning and model training agent. The generated code calculates the accuracy sc…
Figure 6
Figure 6. Figure 6: Visual representation of the Model Risk Management (MRM) crew this agent using the organizational modeling guide.1 This guide shows the steps to be adhered to while training or building machine learning models. The agent utilizes Cache-Augmented Generation (CAG) framew…
Figure 7
Figure 7. Figure 7: Pictorial representation of the Human-Agent-Task-Tool integration 3.5.1. Credit Card Fraud Detection Dataset. We present the performance metrics derived from parsing the credit card fraud detection dataset [22] through the agentic system and highlight several subtletie…
Figure 8
Figure 8. Figure 8: Performance report on the Credit Card Fraud Detection dataset 3.5.2. Credit Card Approval Prediction Dataset. This section provides an overview of evaluating the perfor￾mance of the agentic system when applied to the credit card approval dataset [25]. The dataset consi…
Figure 9
Figure 9. Figure 9: Performance report for credit card approval 3.5.3. Portfolio Credit Risk Dataset. The result of using agentic programming on the portfolio credit risk dataset [24] presents further arguments to underscore the importance of model risk management. Refer to Appendix A for…
Figure 10
Figure 10. Figure 10: Performance report for portfolio credit risk A). The AutoML solution produced comparable metrics to both the agentic system and the Kaggle solution, achieving an accuracy of 92.9%, precision of 94.4%, recall of 72.4%, and an f1-score of 81.9% using Dis￾tributed Random…

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 2 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. Evaluating the Sensitivity of LLMs to Prior Context

    cs.CL 2025-05 conditional novelty 6.0 of 10

    Prior conversational context, especially from a different knowledge domain, can sharply reduce LLM multiple-choice accuracy, and repeating the task near the query mitigates the drop.

  2. Governing Generative AI Across Financial Institutions: A Framework for Generative AI Risk Control

    q-fin.RM 2026-07 unverdicted novelty 4.0 of 10

    GAICF maps SR 26-2 model-risk principles into approved-use gates, risk tiers, evidence checks, and output monitoring for generative AI outside the formal model boundary.

Reference graph

Works this paper leans on

117 extracted references · 29 canonical work pages · cited by 2 Pith papers

  1. [1]

    A. Abid, M. Farooqi, and J. Zou. Persistent anti-muslim bias in large language models. In Proceedings of the 2021 AAAI/ACM Conference on AI, Ethics, and Society, pages 298–306, 2021

    2021

  2. [2]

    S. G. Ayyamperumal and L. Ge. Current state of llm risks and ai guardrails.arXiv preprint arXiv:2406.12934, 2024

    arXiv 2024

  3. [3]

    Bahoo, M

    S. Bahoo, M. Cucculelli, X. Goga, and J. Mondolo. Artificial intelligence in finance: a comprehensive review through bibliometric and content analysis.SN Business & Economics, 4(2):23, 2024

    2024

  4. [4]

    Besta, N

    M. Besta, N. Blach, A. Kubicek, R. Gerstenberger, M. Podstawski, L. Gianinazzi, J. Gajda, T. Lehmann, H. Niewiadomski, P. Nyczyk, et al. Graph of thoughts: Solving elaborate problems with large language models. InProceedings of the AAAI Conference on Artificial Intelligence, volume 38, pages 17682–17690, 2024

    2024

  5. [5]

    M. Caesar. Credit Card Approval Prediction (Cleaned Version). https://www.kaggle.com/code/caesarmario/ credit-card-approval-prediction-w-pycaret/notebook , June 2022. [Online; accessed 17. Jan. 2025]

    2022

  6. [6]

    A. Chan, R. Salganik, A. Markelius, C. Pang, N. Rajkumar, D. Krasheninnikov, L. Langosco, Z. He, Y. Duan, M. Carroll, et al. Harms from increasingly agentic algorithmic systems. InProceedings of the 2023 ACM Conference on Fairness, Accountability, and Transparency, pages 651–666, 2023

    2023

  7. [7]

    C.-M. Chan, W. Chen, Y. Su, J. Yu, W. Xue, S. Zhang, J. Fu, and Z. Liu. Chateval: Towards better llm-based evaluators through multi-agent debate.arXiv preprint arXiv:2308.07201, 2023

    arXiv 2023

  8. [8]

    G. Chen, L. Fan, Z. Gong, N. Xie, Z. Li, Z. Liu, C. Li, Q. Qu, S. Ni, and M. Yang. Agentcourt: Simulating court with adversarial evolvable lawyer agents.arXiv preprint arXiv:2408.08089, 2024

    arXiv 2024

Show all 117 references
  1. [9]

    W. Chen, Y. Su, J. Zuo, C. Yang, C. Yuan, C. Qian, C.-M. Chan, Y. Qin, Y. Lu, R. Xie, et al. Agentverse: Facilitating multi-agent collaboration and exploring emergent behaviors in agents.arXiv preprint arXiv:2308.10848, 2023

    2023 arXiv

  2. [10]

    Cheng, C

    Y. Cheng, C. Zhang, Z. Zhang, X. Meng, S. Hong, W. Li, Z. Wang, Z. Wang, F. Yin, J. Zhao, et al. Exploring large language model based intelligent agents: Definitions, methods, and prospects.arXiv preprint arXiv:2401.03428, 2024

    2024 arXiv

  3. [11]

    Multi AI agent systems with crewAI - deeplearning.ai, 2024

    CrewAI. Multi AI agent systems with crewAI - deeplearning.ai, 2024. [Online; accessed 12. Aug. 2024]. AGENTIC AI SYSTEMS FOR FINANCIAL SER VICES 23

    2024

  4. [12]

    crewAI: Cutting-edge framework for orchestrating role-playing, autonomous AI agents.https://github.com/ crewAIInc/crewAI/, 2024

    crewAIInc. crewAI: Cutting-edge framework for orchestrating role-playing, autonomous AI agents.https://github.com/ crewAIInc/crewAI/, 2024

    2024

  5. [13]

    Y. Du, S. Li, A. Torralba, J. B. Tenenbaum, and I. Mordatch. Improving factuality and reasoning in language models through multiagent debate.arXiv preprint arXiv:2305.14325, 2023

    2023 arXiv

  6. [14]

    S. Han, Q. Zhang, Y. Yao, W. Jin, Z. Xu, and C. He. Llm multi-agent systems: Challenges and open problems.arXiv preprint arXiv:2402.03578, 2024

    2024 arXiv

  7. [15]

    J. He, C. Treude, and D. Lo. Llm-based multi-agent systems for software engineering: Vision and the road ahead.arXiv preprint arXiv:2404.04834, 2024

    2024 arXiv

  8. [16]

    J. J. Horton. Large language models as simulated economic agents: What can we learn from homo silicus? Technical report, National Bureau of Economic Research, 2023

    2023

  9. [17]

    W. Hua, X. Yang, Z. Li, C. Wei, and Y. Zhang. Trustagent: Towards safe and trustworthy llm-based agents through agent constitution. arXiv preprint arXiv:2402.01586, 2024

    2024 arXiv

  10. [18]

    Huang, Q

    D. Huang, Q. Bu, J. M. Zhang, M. Luck, and H. Cui. Agentcoder: Multi-agent-based code generation with iterative testing and optimisation.arXiv preprint arXiv:2312.13010, 2023

    2023 arXiv

  11. [19]

    Huang, C

    Y. Huang, C. Zhou, K. Cui, and X. Lu. A multi-agent reinforcement learning framework for optimizing financial trading strategies based on timesnet.Expert Systems with Applications, 237:121502, 2024

    2024

  12. [20]

    Jingrong, H

    H. Jingrong, H. Shan, C. Zhaobin, L. Yu, L. Yingying, et al. Ai-driven digital transformation in banking: A new perspective on operational efficiency and risk management.Information Systems and Economics, 5(1):82–90, 2024

    2024

  13. [21]

    Jinxin, Z

    S. Jinxin, Z. Jiabao, W. Yilei, W. Xingjiao, L. Jiawen, and H. Liang. Cgmi: Configurable general multi-agent interaction framework. arXiv preprint arXiv:2308.12503, 2023

    2023 arXiv

  14. [22]

    Credit Card Fraud Detection Dataset.https://www.kaggle.com/datasets/mlg-ulb/creditcardfraud/data, Mar

    Kaggle. Credit Card Fraud Detection Dataset.https://www.kaggle.com/datasets/mlg-ulb/creditcardfraud/data, Mar

  15. [23]

    Credit Fraud, Dealing with Imbalanced Datasets

    Kaggle. Credit Fraud, Dealing with Imbalanced Datasets. https://www.kaggle.com/code/janiobachmann/ credit-fraud-dealing-with-imbalanced-datasets , Mar. 2005. [Online; accessed 22. Jan. 2025]

    2005

  16. [24]

    Credit Risk Dataset.https://www.kaggle.com/datasets/laotse/credit-risk-dataset/data, 2020

    Kaggle. Credit Risk Dataset.https://www.kaggle.com/datasets/laotse/credit-risk-dataset/data, 2020. [Online; ac- cessed 3. Oct. 2024]

    2020

  17. [25]

    Credit Card Approval.https://www.kaggle.com/datasets/rikdifos/credit-card-approval-prediction, 2021

    Kaggle. Credit Card Approval.https://www.kaggle.com/datasets/rikdifos/credit-card-approval-prediction, 2021. [Online; accessed 3. Oct. 2024]

    2021

  18. [26]

    Credit Risk Prediction Training and EDA

    Kaggle. Credit Risk Prediction Training and EDA. https://www.kaggle.com/code/anshtanwar/ credit-risk-prediction-training-and-eda , 2024. [Online; accessed 22. Jan. 2025]

    2024

  19. [27]

    Kanaparthi

    V. Kanaparthi. Transformational application of artificial intelligence and machine learning in financial technologies and financial services: A bibliometric review.arXiv preprint arXiv:2401.15710, 2024

    2024 arXiv

  20. [28]

    K. J. Koa, Y. Ma, R. Ng, and T.-S. Chua. Learning to generate explainable stock predictions using self-reflective large language models. InProceedings of the ACM on Web Conference 2024, pages 4304–4315, 2024

    2024

  21. [29]

    Lewis, E

    P. Lewis, E. Perez, A. Piktus, F. Petroni, V. Karpukhin, N. Goyal, H. Küttler, M. Lewis, W.-t. Yih, T. Rocktäschel, et al. Retrieval-augmented generation for knowledge-intensive nlp tasks.Advances in Neural Information Processing Systems, 33:9459–9474, 2020

    2020

  22. [30]

    C. Li, R. Yang, T. Li, M. Bafarassat, K. Sharifi, D. Bergemann, and Z. Yang. Stride: A tool-assisted llm agent framework for strategic and interactive decision-making.arXiv preprint arXiv:2405.16376, 2024

    2024 arXiv

  23. [31]

    G. Li, H. Hammoud, H. Itani, D. Khizbullin, and B. Ghanem. Camel: Communicative agents for" mind" exploration of large language model society.Advances in Neural Information Processing Systems, 36:51991–52008, 2023

    2023

  24. [32]

    Preprint, 2024

    N.Li, C.Gao, M.Li, Y.Li, andQ.Liao.Econagent: Largelanguagemodel-empoweredagentsforsimulatingmacroeconomic activities. Preprint, 2024

    2024

  25. [33]

    R. Li, X. Wang, and H. Yu. Exploring llm multi-agents for icd coding.arXiv preprint arXiv:2406.15363, 2024

    2024 arXiv

  26. [34]

    Y. Li, Y. Yu, H. Li, Z. Chen, and K. Khashanah. Tradinggpt: Multi-agent system with layered memory and distinct characters for enhanced financial trading performance.arXiv preprint arXiv:2309.03736, 2023

    2023 arXiv

  27. [35]

    Liang, Z

    T. Liang, Z. He, W. Jiao, X. Wang, Y. Wang, R. Wang, Y. Yang, Z. Tu, and S. Shi. Encouraging divergent thinking in large language models through multi-agent debate.arXiv preprint arXiv:2305.19118, 2023

    2023 arXiv

  28. [36]

    Y. Ling, X. Jiang, and Y. Kim. Mallm-gan: Multi-agent large language model as generative adversarial network for synthesizing tabular data.arXiv preprint arXiv:2406.10521, 2024

    2024 arXiv

  29. [37]

    X. Liu, B. Lan, Z. Hu, Y. Liu, Z. Zhang, W. Zhou, F. Wang, and M. Shieh. Codexgraph: Bridging large language models and code repositories via code graph databases.arXiv preprint arXiv:2408.03910, 2024

    2024 arXiv

  30. [38]

    Mitra, L

    A. Mitra, L. Del Corro, G. Zheng, S. Mahajan, D. Rouhana, A. Codas, Y. Lu, W.-g. Chen, O. Vrousgos, C. Rosset, et al. Agentinstruct: Toward generative teaching with agentic flows.arXiv preprint arXiv:2407.03502, 2024

    2024 arXiv

  31. [39]

    Mündler, M

    N. Mündler, M. N. Müller, J. He, and M. Vechev. Code agents are state of the art software testers.arXiv preprint arXiv:2406.12952, 2024. 24 OKPALA, GOLGOON, RA VI KANNAN

    2024 arXiv

  32. [40]

    M. Nasr, N. Carlini, J. Hayase, M. Jagielski, A. F. Cooper, D. Ippolito, C. A. Choquette-Choo, E. Wallace, F. Tramèr, and K. Lee. Scalable extraction of training data from (production) language models.arXiv preprint arXiv:2311.17035, 2023

    2023 arXiv

  33. [41]

    M.H.Nguyen, T.P.Chau, P.X.Nguyen, andN.D.Bui.Agilecoder: Dynamiccollaborativeagentsforsoftwaredevelopment based on agile methodology.arXiv preprint arXiv:2406.11912, 2024

    2024 arXiv

  34. [42]

    Y. Nie, Y. Kong, X. Dong, J. M. Mulvey, H. V. Poor, Q. Wen, and S. Zohren. A survey of large language models for financial applications: Progress, prospects and challenges.arXiv preprint arXiv:2406.11903, 2024

    2024 arXiv

  35. [43]

    Pandey, A

    H. Pandey, A. Amod, et al. Advancing healthcare automation: Multi-agent systems for medical necessity justification. arXiv preprint arXiv:2404.17977, 2024

    2024 arXiv

  36. [44]

    J. S. Park, J. O’Brien, C. J. Cai, M. R. Morris, P. Liang, and M. S. Bernstein. Generative agents: Interactive simulacra of human behavior. InProceedings of the 36th annual acm symposium on user interface software and technology, pages 1–22, 2023

    2023

  37. [45]

    T. Park. Enhancing anomaly detection in financial markets with an llm-based multi-agent framework.arXiv preprint arXiv:2403.19735, 2024

    2024 arXiv

  38. [46]

    C. Qian, X. Cong, C. Yang, W. Chen, Y. Su, J. Xu, Z. Liu, and M. Sun. Communicative agents for software development. arXiv preprint arXiv:2307.07924, 6, 2023

    2023 arXiv

  39. [47]

    Rasheed, M

    Z. Rasheed, M. Waseem, M. Saari, K. Systä, and P. Abrahamsson. Codepori: Large scale model for autonomous software development by using multi-agents.arXiv preprint arXiv:2402.01411, 2024

    2024 arXiv

  40. [48]

    Schwartz, A

    S. Schwartz, A. Yaeli, and S. Shlomov. Enhancing trust in llm-based ai automation agents: New considerations and future challenges. arXiv preprint arXiv:2308.05391, 2023

    2023 arXiv

  41. [49]

    Shamsujjoha, Q

    M. Shamsujjoha, Q. Lu, D. Zhao, and L. Zhu. Towards ai-safety-by-design: A taxonomy of runtime guardrails in foundation model based systems.arXiv preprint arXiv:2408.02205, 2024

    2024 arXiv

  42. [50]

    Shanahan, K

    M. Shanahan, K. McDonell, and L. Reynolds. Role play with large language models.Nature, 623(7987):493–498, 2023

    2023

  43. [51]

    Shinn, F

    N. Shinn, F. Cassano, A. Gopinath, K. Narasimhan, and S. Yao. Reflexion: Language agents with verbal reinforcement learning. Advances in Neural Information Processing Systems, 36, 2024

    2024

  44. [52]

    Sudarshan, S

    M. Sudarshan, S. Shih, E. Yee, A. Yang, J. Zou, C. Chen, Q. Zhou, L. Chen, C. Singhal, and G. Shih. Agentic llm workflows for generating patient-friendly medical reports.arXiv preprint arXiv:2408.01112, 2024

    2024 arXiv

  45. [53]

    J. Sun, C. Dai, Z. Luo, Y. Chang, and Y. Li. Lawluo: A chinese law firm co-run by llm agents. arXiv preprint arXiv:2407.16252, 2024

    2024 arXiv

  46. [54]

    Talebirad and A

    Y. Talebirad and A. Nadiri. Multi-agent collaboration: Harnessing the power of intelligent llm agents.arXiv preprint arXiv:2306.03314, 2023

    2023 arXiv

  47. [55]

    D. Tang, Z. Chen, K. Kim, Y. Song, H. Tian, S. Ezzini, Y. Huang, and J. K. T. F. Bissyande. Collaborative agents for software engineering.arXiv preprint arXiv:2402.02172, 2024

    2024 arXiv

  48. [56]

    X. Tang, Q. Jin, K. Zhu, T. Yuan, Y. Zhang, W. Zhou, M. Qu, Y. Zhao, J. Tang, Z. Zhang, et al. Prioritizing safeguarding over autonomy: Risks of llm agents for science.arXiv preprint arXiv:2402.04247, 2024

    2024 arXiv

  49. [57]

    N.Vadori, L.Ardon, S.Ganesh, T.Spooner, S.Amrouni, J.Vann, M.Xu, Z.Zheng, T.Balch, andM.Veloso.Towardsmulti- agent reinforcement learning-driven over-the-counter market simulations.Mathematical Finance, 34(2):262–347, 2024

    2024

  50. [58]

    Varshney

    T. Varshney. NVIDIA Generaitve AI Technical Blog: Introduction to LLM Agents.https://developer.nvidia.com/blog/ introduction-to-llm-agents/, 2023

    2023

  51. [59]

    X. Wan, H. Deng, K. Zou, and S. Xu. Enhancing the efficiency and accuracy of underlying asset reviews in structured finance: The application of multi-agent framework.arXiv preprint arXiv:2405.04294, 2024

    2024 arXiv

  52. [60]

    H. Wang, S. Zhao, Z. Qiang, N. Xi, B. Qin, and T. Liu. Beyond direct diagnosis: Llm-based multi-specialist agent consultation for automatic diagnosis.arXiv preprint arXiv:2401.16107, 2024

    2024 arXiv

  53. [61]

    S. Wang, H. Yuan, L. M. Ni, and J. Guo. Quantagent: Seeking holy grail in trading by self-improving large language model. arXiv preprint arXiv:2402.03755, 2024

    2024 arXiv

  54. [62]

    S. Wang, H. Yuan, L. Zhou, L. M. Ni, H.-Y. Shum, and J. Guo. Alpha-gpt: Human-ai interactive alpha mining for quantitative investment.arXiv preprint arXiv:2308.00016, 2023

    2023

  55. [63]

    Z. Wang, D. J. Kim, and T.-H. Chen. Identifying performance-sensitive configurations in software systems through code analysis with llm agents.arXiv preprint arXiv:2406.12806, 2024

    2024 arXiv

  56. [64]

    J. Wei, X. Wang, D. Schuurmans, M. Bosma, F. Xia, E. Chi, Q. V. Le, D. Zhou, et al. Chain-of-thought prompting elicits reasoning in large language models.Advances in neural information processing systems, 35:24824–24837, 2022

    2022

  57. [65]

    Weidinger, J

    L. Weidinger, J. Uesato, M. Rauh, C. Griffin, P.-S. Huang, J. Mellor, A. Glaese, M. Cheng, B. Balle, A. Kasirzadeh, et al. Taxonomy of risks posed by language models. InProceedings of the 2022 ACM Conference on Fairness, Accountability, and Transparency, pages 214–229, 2022

    2022

  58. [66]

    M. Wu, Y. Yuan, G. Haffari, and L. Wang. (perhaps) beyond human translation: Harnessing multi-agent collaboration for translating ultra-long literary texts.arXiv preprint arXiv:2405.11804, 2024. AGENTIC AI SYSTEMS FOR FINANCIAL SER VICES 25

    2024 arXiv

  59. [67]

    Q. Wu, G. Bansal, J. Zhang, Y. Wu, S. Zhang, E. Zhu, B. Li, L. Jiang, X. Zhang, and C. Wang. Autogen: Enabling next-gen llm applications via multi-agent conversation framework.arXiv preprint arXiv:2308.08155, 2023

    2023 arXiv

  60. [68]

    C. Xie, C. Chen, F. Jia, Z. Ye, K. Shu, A. Bibi, Z. Hu, P. Torr, B. Ghanem, and G. Li. Can large language model agents simulate human trust behaviors?arXiv preprint arXiv:2402.04559, 2024

    2024 arXiv

  61. [69]

    F. Xing. Designing heterogeneous llm agents for financial sentiment analysis.arXiv preprint arXiv:2401.05799, 2024

    2024 arXiv

  62. [70]

    S. Yao, J. Zhao, D. Yu, N. Du, I. Shafran, K. Narasimhan, and Y. Cao. React: Synergizing reasoning and acting in language models. arXiv preprint arXiv:2210.03629, 2022

    2022 arXiv

  63. [71]

    Y. Yu, H. Li, Z. Chen, Y. Jiang, Y. Li, D. Zhang, R. Liu, J. W. Suchow, and K. Khashanah. Finmem: A performance- enhanced llm trading agent with layered memory and character design. InProceedings of the AAAI Symposium Series, volume 3, pages 595–597, 2024

    2024

  64. [72]

    Y. Yu, Z. Yao, H. Li, Z. Deng, Y. Cao, Z. Chen, J. W. Suchow, R. Liu, Z. Cui, D. Zhang, et al. Fincon: A synthe- sized llm multi-agent system with conceptual verbal reinforcement for enhanced financial decision making.arXiv preprint arXiv:2407.06567, 2024

    2024 arXiv

  65. [73]

    H. Yuan, S. Wang, and J. Guo. Alpha-gpt 2.0: Human-in-the-loop ai for quantitative investment. arXiv preprint arXiv:2402.09746, 2024

    2024 arXiv

  66. [74]

    D. Zan, A. Yu, W. Liu, D. Chen, B. Shen, W. Li, Y. Yao, Y. Gong, X. Chen, B. Guan, et al. Codes: Natural language to code repository via multi-layer sketch.arXiv preprint arXiv:2403.16443, 2024

    2024 arXiv

  67. [75]

    Zhang, X

    C. Zhang, X. Liu, M. Jin, Z. Zhang, L. Li, Z. Wang, W. Hua, D. Shu, S. Zhu, X. Jin, et al. When ai meets finance (stocka- gent): Large language model-based stock trading in simulated real-world environments.arXiv preprint arXiv:2407.18957, 2024

    2024 arXiv

  68. [76]

    Zhang, X

    J. Zhang, X. Xu, and S. Deng. Exploring collaboration mechanisms for llm agents: A social psychology view.arXiv preprint arXiv:2310.02124, 2023

    2023 arXiv

  69. [77]

    Zhang, L

    W. Zhang, L. Zhao, H. Xia, S. Sun, J. Sun, M. Qin, X. Li, Y. Zhao, Y. Zhao, X. Cai, et al. Finagent: A multimodal foundation agent for financial trading: Tool-augmented, diversified, and generalist. arXiv preprint arXiv:2402.18485, 2024

    2024 arXiv

  70. [78]

    Zhang, H

    Y. Zhang, H. Ruan, Z. Fan, and A. Roychoudhury. Autocoderover: Autonomous program improvement.arXiv preprint arXiv:2404.05427, 2024

    2024 arXiv

  71. [79]

    H. Zhao, Z. Liu, Z. Wu, Y. Li, T. Yang, P. Shu, S. Xu, H. Dai, L. Zhao, G. Mai, et al. Revolutionizing finance with llms: An overview of applications and insights.arXiv preprint arXiv:2401.11641, 2024

    2024

  72. [80]

    Q. Zhao, J. Wang, Y. Zhang, Y. Jin, K. Zhu, H. Chen, and X. Xie. Competeai: Understanding the competition behaviors in large language model-based agents.arXiv preprint arXiv:2310.17512, 2023. Appendix A. Agentic collaboration in Financial modeling - credit risk use case In thi...

    2023 arXiv

  73. [85]

    : " ###

    Model E v a l u a t i o n Agent Enter the a s s o c i a t e d Agent n u m b e r : 1 W o r k i n g Agent : Data A n a l y s t S t a r t i n g Task : Extract the data in the Kaggle loc at ion - ‘ laotse / credit - risk - dataset , ’ split ↪→ it into train and test set and get 40...

  74. [89]

    E x p l o r a t o r y Data A na lys is Tool

    Model E v a l u a t i o n Agent Enter the a s s o c i a t e d Agent n u m b e r : 2 W o r k i n g Agent : Data S c i e n t i s t - EDA S t a r t i n g Task : Conduct a d eta il ed e x p l o r a t o r y data a na lys is on the pro vi ded dataset ↪→ e x t r a c t e d by the data...

  75. [93]

    g e n e r a t e d _ c o d e

    Model E v a l u a t i o n Agent Enter the a s s o c i a t e d Agent n u m b e r : 3 W o r k i n g Agent : S e n i o r Data S c i e n t i s t - F e a t u r e E n g i n e e r i n g S t a r t i n g Task : Create a feature e n g i n e e r i n g pip el ine using the train data p r ...

  76. [97]

    Model E v a l u a t i o n Agent Enter the a s s o c i a t e d Agent n u m b e r : 4 W o r k i n g Agent : S e n i o r M a c h i n e L e a r n i n g E n g i n e e r - Meta - T u n i n g S t a r t i n g Task : Perform h y p e r p a r a m e t e r tuning using G r i d S e a r c h ...

  77. [101]

    g e n e r a t e d _ c o d e

    Model E v a l u a t i o n Agent Enter the a s s o c i a t e d Agent n u m b e r : 5 AGENTIC AI SYSTEMS FOR FINANCIAL SER VICES 29 W o r k i n g Agent : S e n i o r M a c h i n e L e a r n i n g E n g i n e e r - Model T r a i n i n g S t a r t i n g Task : Train the the mod el...

  78. [102]

    Data E x t r a c t i o n Agent

  79. [103]

    Feature E n g i n e e r i n g Agent

  80. [104]

    Model Tr ai nin g Agent

  81. [105]

    g e n e r a t e d _ c o d e

    Model E v a l u a t i o n Agent Enter the a s s o c i a t e d Agent n u m b e r : 6 W o r k i n g Agent : S e n i o r M a c h i n e L e a r n i n g E n g i n e e r - Model E v a l u a t i o n S t a r t i n g Task : Load and ev al uat e the trained model using test dataset . Hu...

  82. [106]

    t r u n c a t e d

    Data E x t r a c t i o n Task : - The data has been s u c c e s s f u l l y e x t r a c t e d from Kaggle , split into train and test sets , ↪→ and a 40% sample has been e x t r a c t e d from the train dataset . . . . . . . t r u n c a t e d

  83. [107]

    - It i ncl ud ed i n f o r m a t i o n on the data ove rv iew

    EDA Task : - The E x p l o r a t o r y Data Ana ly sis ( EDA ) report pr ov ide d d eta il ed i nsi gh ts into the dataset . - It i ncl ud ed i n f o r m a t i o n on the data ove rv iew . . . . . . . t r u n c a t e d

  84. [108]

    - It i ncl ud ed i n f o r m a t i o n on the transformers , imputers , encoders , feature s e l e c t i o n ↪→ methods , and

    Feature E n g i n e e r i n g Task : - The Feature E n g i n e e r i n g report ou tli ne d the p ipe li ne used for feature t r a n s f o r m a t i o n ↪→ and s e l e c t i o n . - It i ncl ud ed i n f o r m a t i o n on the transformers , imputers , encoders , feature s e l ...

  85. [109]

    - The Senior Machine Le ar nin g E ngi ne er s u c c e s s f u l l y c o n d u c t e d

    Meta - Tune Task : - The Model S e l e c t i o n report de ta ile d the h y p e r p a r a m e t e r tuning process and i d e n t i f i e d ↪→ XGBoost as the best p e r f o r m i n g model . - The Senior Machine Le ar nin g E ngi ne er s u c c e s s f u l l y c o n d u c t e d ...

  86. [110]

    - The Senior Machine Le ar nin g E ngi ne er e f f e c t i v e l y r e t r a i n e d the model

    Model Tr ai nin g Task : - The best p e r f o r m i n g e s t i m a t o r was r e t r a i n e d using the X_train and y_train data . - The Senior Machine Le ar nin g E ngi ne er e f f e c t i v e l y r e t r a i n e d the model . . . . . . . t r u n c a t e d

  87. [111]

    Documentation Compliance Checker Agent,

    Model E v a l u a t i o n Task : - The e v a l u a t i o n metrics i n c l u d i n g Accuracy , Precision , Recall , F1 - score , and AUC were ↪→ c a l c u l a t e d for the trained model using the test dataset . - The trained model was . . . . . . . t r u n c a t e d AGENTIC ...

  88. [115]

    pdf _p ath

    Outcome A na lyz er Agent Enter the a s s o c i a t e d Agent n u m b e r : 1 32 OKPALA, GOLGOON, RA VI KANNAN W o r k i n g Agent : Data S c i e n t i s t - D o c u m e n t a t i o n C o m p l i a n c e Check S t a r t i n g Task : Verify the m ode li ng d o c u m e n t a t i...

  89. [117]

    Model R e p l i c a t i o n Agent AGENTIC AI SYSTEMS FOR FINANCIAL SER VICES 33

  90. [119]

    g e n e r a t e d _ c o d e

    Outcome A na lyz er Agent Enter the a s s o c i a t e d Agent n u m b e r : 2 W o r k i n g Agent : M a c h i n e L e a r n i n g E n g i n e e r i n g - Model r e p l i c a t i o n S t a r t i n g Task : Create and train a machine l ea rni ng pip el ine using the p a r a m e ...

  91. [123]

    g e n e r a t e d _ c o d e

    Outcome A na lyz er Agent Enter the a s s o c i a t e d Agent n u m b e r : 3 W o r k i n g Agent : S e n i o r Model V a l i d a t i o n A n a l y s t - C o n c e p t u a l S o u n d n e s s S t a r t i n g Task : Check feature i m p o r t a n c e and e x p l a i n a b i l i ...

  92. [124]

    3 0 1 8 8 8 0 8

    l o a n _ p e r c e n t _ i n c o m e - 0 . 3 0 1 8 8 8 0 8

  93. [125]

    2 4 0 6 8 6 8 8

    l o a n _ g r a d e - 0 . 2 4 0 6 8 6 8 8

  94. [126]

    2 4 0 0 7 7 2 4

    l o a n _ a m n t - 0 . 2 4 0 0 7 7 2 4

  95. [127]

    0 8 8 2 7 8 4 1

    l o a n _ i n t _ r a t e - 0 . 0 8 8 2 7 8 4 1

  96. [128]

    0 5 2 0 3 2 8 4 3

    p e r s o n _ i n c o m e - 0 . 0 5 2 0 3 2 8 4 3

  97. [129]

    0 3 5 0 3 7 6 3 7

    p e r s o n _ h o m e _ o w n e r s h i p - 0 . 0 3 5 0 3 7 6 3 7

  98. [130]

    0 1 8 3 0 0 7 0 8

    l o a n _ i n t e n t - 0 . 0 1 8 3 0 0 7 0 8

  99. [131]

    0 1 2 3 9 5 9 9 4

    p e r s o n _ e m p _ l e n g t h - 0 . 0 1 2 3 9 5 9 9 4

  100. [132]

    0 1 1 3 0 2 2 4 8 The feature i m p o r t a n c e values i ndi ca te the c o n t r i b u t i o n of each feature towards the model ’ s ↪→ p r e d i c t i v e p e r f o r m a n c e

    c b _ p e r s o n _ d e f a u l t _ o n _ f i l e - 0 . 0 1 1 3 0 2 2 4 8 The feature i m p o r t a n c e values i ndi ca te the c o n t r i b u t i o n of each feature towards the model ’ s ↪→ p r e d i c t i v e p e r f o r m a n c e . In this case , the top fea tu res such ...

  101. [133]

    D o c u m e n t a t i o n C o m p l i a n c e Agent

  102. [134]

    Model R e p l i c a t i o n Agent

  103. [135]

    C o n c e p t u a l S o u n d n e s s Agent

  104. [136]

    g e n e r a t e d _ c o d e

    Outcome A na lyz er Agent Enter the a s s o c i a t e d Agent n u m b e r : 4 W o r k i n g Agent : Model V a l i d a t i o n A n a l y s t - O u t c o m e A n a l y z e r S t a r t i n g Task : Conduct a d d i t i o n a l testing of the model located at ‘ credit / b e s t _ m...

  105. [137]

    t r u n c a t e d

    D o c u m e n t a t i o n C o m p l i a n c e Task : - Action : Cache - A u g m e n t e d G e n e r a t i o n ( CAG ) Tool - The m od eli ng d o c u m e n t a t i o n pr ovi de d by the mo de lin g team closely adheres to the ↪→ p r o c e s s e s d e s c r i b e d in the o r g...

  106. [138]

    t r u n c a t e d

    Model R e p l i c a t i o n Task : - Action : Python Code E xe cut or Tool - The model r e p l i c a t i o n was successful , and the e v a l u a t i o n metrics ..... t r u n c a t e d

  107. [139]

    ↪→ t r u n c a t e d

    C o n c e p t u a l S o u n d n e s s Task : - Action : Python Code E xe cut or Tool - The top 10 f ea tur es that c o n t r i b u t e d to the model ’ s result were i d e n t i f i e d ..... ↪→ t r u n c a t e d

  108. [140]

    t r u n c a t e d The mo de lin g risk m a n a g e m e n t agents have d e m o n s t r a t e d p r o f i c i e n c y in their tasks

    Outcome A na lyz er Task : - Action : Python Code E xe cut or Tool - The outcome a na lys is report i ncl ud es the e v a l u a t i o n ..... t r u n c a t e d The mo de lin g risk m a n a g e m e n t agents have d e m o n s t r a t e d p r o f i c i e n c y in their tasks . T...

  109. [2005]

    [Online; accessed 3. Oct. 2024]

    2024

Pith tools

Reviewed August 8, 2026 · model on record in the stance chip above.