{"id":"b41904b5-260c-4bf4-b389-292d257a89dd","arxiv_id":"2504.18369","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":4.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":0,"one_line_summary":"The authors propose an LLM-and-RAG-based threat modeling tool for LLM-integrated applications and report one early, unvalidated ChatGPT pilot as preliminary motivation.","lead":"This paper proposes ThreMoLIA, a tool that uses large language models and retrieval-augmented generation to automate threat modeling for applications that integrate LLMs. It is a vision paper with one preliminary ChatGPT test and a planned industrial evaluation, not a validated result.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The paper's own Quality Assurance section concedes that no metrics exist to measure threat-model quality; the central 'high-quality' claim and the industrial evaluation both depend on metrics the authors have not yet validated.","rationale":"The reader's weakest_assumption identifies exactly the same load-bearing premise: unvalidated quality metrics. I agree with that identification, and I do not find a more fundamental flaw. The paper is honest about the limitation and frames the work as a vision; however, the abstract's strong 'ensure high-quality' wording makes the metric gap decisive. The proposed fix — a calibration study using expert inter-rater agreement — is inexpensive and would de-risk the entire evaluation plan. The early pilot is appropriately labeled preliminary, so I would not reject the paper; the conditional verdict stands.","tokens_in":8097,"tokens_out":2460,"duration_ms":24433,"concrete_test":"Run a calibration study before the industrial case study: have two independent security experts score a fixed set of threat models (e.g., 10 generated by the ThreMoLIA prototype and 10 produced manually) using the Table 2 metrics, and compute inter-rater agreement (Cohen's kappa or ICC). Also compute the health score for each model. If inter-rater reliability is below 0.6, or if the health score diverges from expert rankings on a deliberately weakened threat model (e.g., one with a removed threat), the metrics are not yet fit to support the 'high-quality' claim. This single check would settle whether the QA component can function as an oracle.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central assertion in the abstract — that ThreMoLIA 'prioritizes the quality of threat modeling reports by implementing comprehensive data checks to ensure the accuracy and reliability of the output' — is only meaningful if threat-model quality can be measured. Section 3.1 (Quality Assurance Challenges) explicitly states: 'Currently, established metrics to systematically quantify and evaluate the quality of a threat model are lacking.' The QA component's metamorphic-relationship oracle requires a definition of correctness for generated threat models; without it, the 'health score' is ungrounded. Section 3.2 then proposes to create a benchmark by extracting metrics from the literature and validating them in focus groups, and to use those same metrics in the industrial comparison. This is circular: the metrics are both the instrument and the object being validated. The early ChatGPT-3.5 pilot (Section 4.1) does not exercise the QA component at all — it is a zero-shot prompt with no RAG, data checks, or health score — so it provides no evidence that 'high quality' can be ensured. If the metrics cannot be established, the primary quality claim is unsupported, though the time-saving claim might survive independently.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper presents ThreMoLIA, a vision for an LLM-based threat modeling approach for LLM-integrated applications (LIAs). The proposed architecture combines retrieval-augmented generation (RAG) with data aggregation from design documents, architecture repositories, and existing threat models, and includes a prompting component and a quality assurance component that checks syntactic correctness, applies metamorphic-relationship-based tests, and reports a health score. The authors map MITRE ATLAS techniques to the OWASP Top 10 for LLMs, outline an evaluation plan involving an industry partner, and report a preliminary pilot in which ChatGPT-3.5 Turbo was asked to perform zero-shot threat modeling on a simple LIA. The paper states that this pilot 'encouraged us to proceed.' The central claims are that automating threat modeling with an LLM accelerates development while keeping threat models up to date, and that quality is ensured through comprehensive data checks.","tokens_in":8407,"tokens_out":3064,"duration_ms":32746,"significance":"If realized as described, ThreMoLIA would address a timely and practically important problem: adapting threat modeling to the dynamic, nondeterministic components introduced by LLM integration. The proposed integration of RAG, continuous threat model updates, and an explicit quality assurance component is a reasonable design direction, and the collaboration with an industry partner gives the work potential practical relevance. Table 1's mapping of MITRE ATLAS techniques to OWASP Top 10 for LLM categories is a useful synthesis. The authors are also transparent about the absence of established threat-model quality metrics. However, the manuscript is currently a vision/position paper rather than a validated tool paper; the quality assurance and evaluation components are described as future work, and the only reported evidence is a single anecdotal pilot. The claimed ability to 'ensure the accuracy and reliability' of generated threat models is not supported by the evidence presented.","major_comments":[{"comment":"The paper's central quality claim is undercut by its own statement that 'Currently, established metrics to systematically quantify and evaluate the quality of a threat model are lacking.' The proposed health score and the metamorphic-relationship oracles both presuppose a definition of correctness for a generated threat model. Without such metrics, the 'comprehensive data checks' described in the abstract and Section 3.1 are ungrounded. The authors should either present a concrete, validated operationalization of threat-model quality or explicitly reframe the quality-assurance claims as an open design problem rather than a delivered capability.","section":"§3.1, Quality Assurance Challenges"},{"comment":"The evaluation plan is circular with respect to metrics. Section 3.2 proposes to create a benchmark by extracting metrics from the literature, validating them in focus groups, and then using those same metrics in the industrial comparison of ThreMoLIA against existing tools. This conflates the validation of the measurement instrument with the validation of the tool under test. The authors need an independent reference point—for example, expert judgment on a held-out set, a known planted-vulnerability corpus, or inter-rater agreement against manually produced threat models—so that the metrics themselves are not the only basis for judging whether ThreMoLIA produces high-quality outputs.","section":"§3.2, Evaluation Plan"},{"comment":"The reported pilot does not exercise any of the proposed ThreMoLIA components. It is a single zero-shot ChatGPT-3.5 session on a simple LIA, with a deliberately vague prompt, no RAG, no data checks, no health score, no baseline, and no quantitative or qualitative error analysis. Consequently, it cannot support the abstract's claim that the approach 'prioritizes the quality of threat modeling reports by implementing comprehensive data checks to ensure the accuracy and reliability of the output.' The pilot may be reported as anecdotal motivational evidence, but the paper should clearly state that it provides no evidence about the effectiveness or quality assurance of ThreMoLIA itself.","section":"§4.1, Initial Prototype"},{"comment":"Even as a preliminary feasibility check, the pilot lacks the measurement needed to justify the authors' conclusion that the results 'encouraged us to proceed.' There is no description of how the generated threat model was assessed, no comparison to a manually produced threat model for the same system, and no discussion of omissions or errors in the ChatGPT output. Without such an assessment, the pilot offers little more than the observation that an LLM can produce threat-model-like text. The authors should specify what would count as success in the planned evaluation and report at least a minimal check (e.g., expert rating, coverage of expected threat categories) for the pilot.","section":"§4.1, evaluation quality"}],"minor_comments":[{"comment":"The text states that STRIDE 'organizes possible threats across five categories,' but the list immediately following contains six categories (Spoofing, Tampering, Repudiation, Information Disclosure, Denial of Service, Elevation of Privilege). The count should be corrected.","section":"§2.1, STRIDE description"},{"comment":"There is an inconsistency in the reported number of specialists: Section 3.2 mentions 'two security specialists,' while Section 4.2 says 'a focus group consisting of three security researchers.' Please align these statements.","section":"§3.2 vs. §4.2"},{"comment":"Table 2 lists many metrics but does not indicate which are objective, measurable quantities versus subjective ratings. For example, 'Accuracy' as defined (true positives and true negatives divided by total classification outcomes) requires a ground truth, while 'Engineer Friendly' is a subjective satisfaction measure. The authors should clarify which metrics are expected to be computable without human judgment.","section":"§4.2, Table 2"},{"comment":"The paper would benefit from a explicit statement distinguishing 'proposed design' from 'implemented functionality.' The abstract and introduction use language that implies existing capability (e.g., 'ensures accuracy and reliability'), but the body repeatedly describes planned components and future evaluations. A consistent tense and framing would set accurate expectations.","section":"Throughout"}],"recommendation":"major_revision","confidential_remarks":"This manuscript is best understood as a vision or position paper. If the target venue does not accept vision papers, the current evidence is far too thin for acceptance as a full technical contribution. The paper's own admissions about missing metrics and the anecdotal pilot are honest, but they leave the central quality-assurance claim unsupported. A major revision that reframes the contribution as a proposal, tightens the evaluation plan to avoid circular metric validation, and adds even a minimal expert-based assessment of the pilot would make the paper suitable for a workshop or short-paper venue. No concerns about authorship or citation ethics beyond the heavy reliance on the authors' own prior work, which is appropriately cited."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"ThreMoLIA is a vision paper for an LLM+RAG tool that generates and updates threat models for LLM-integrated applications. What's actually new is modest: the combination of RAG, data aggregation, prompting, and a quality-assurance component aimed at LIA threat modeling, plus the claim of being co-developed with an industry partner. The paper is strongest in its architecture description and its candid enumeration of known challenges, and it's honest that no established metrics exist for threat-model quality. The mapping from MITRE ATLAS to OWASP Top 10 LLM is a useful summary.\n\nThe soft spots are real. The central quality claim in the abstract is not backed by evidence: the early pilot is a zero-shot ChatGPT session with no RAG, no QA, no health score, and no baseline, and the paper's own Section 3.1 says established metrics for threat-model quality are lacking. The evaluation plan is also a bit circular: metrics will be mined from the literature, validated in focus groups, and then used as the instrument to evaluate the tool. That's a workable plan for a vision paper, but it means the 'high-quality threat modeling' claim is currently unsupported. The time-savings claim may survive, but there is no measurement yet.\n\nNone of this is fatal for what this paper is—a proposal. The authors are clear about the open problems. The main issue is the gap between the assertiveness of the abstract and the evidence in the body. If this goes to review, I'd want the authors to soften the quality claim, report the pilot honestly as a feasibility check, and specify how the QA oracle will be grounded before the industrial study.","headline":"A plausible vision paper for LLM-based threat modeling whose central quality claim is not yet supported by evidence, but which is honest about its open problems.","tokens_in":8837,"tokens_out":1431,"would_cite":false,"duration_ms":13270,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"ThreMoLIA proposes an LLM-powered pipeline that generates and continuously updates threat models for LLM-integrated applications, grounding them in project-specific context and validating output through quality checks.","keywords":["threat modeling","large language models","LLM-integrated applications","retrieval-augmented generation","quality assurance","software security","secure software engineering"],"falsifier":"A concrete falsifier would be an industrial case study where ThreMoLIA-generated threat models for a real LIA are checked against a vulnerability list obtained from an independent red-team exercise, showing the tool reliably omits a threat category such as indirect prompt-injection chains that a human analyst would have caught.","tokens_in":7839,"feed_emoji":"🛡️","tokens_out":7105,"duration_ms":68755,"temperature":0.7,"pith_summary":"LLM-integrated applications (LIAs) expand the attack surface with new, nondeterministic failure modes, but threat modeling—the standard way to find and mitigate these risks—is slow and depends on scarce security expertise. The paper's central proposal is ThreMoLIA, an LLM-driven pipeline that generates and continuously updates threat models for LIAs by feeding architecture documents, requirements, existing threat models, and runtime monitoring data into a retrieval-augmented generator, then checking the output through a dedicated quality-assurance layer. If it works, security teams would need fewer expert hours for threat modeling and could keep the threat model current as the application evolves, instead of producing a one-off artifact. The paper reports an early zero-shot pilot with a general-purpose chatbot on a simple LIA that produced encouraging output and argues that a systematic evaluation with an industry partner is the next step.","feed_headline":"LLM pipeline automates threat modeling for AI-backed apps","feed_subtitle":"Generated threat models stay current as architecture changes, grounded in project documents and earlier models.","key_machinery":"The mechanism that carries the argument is retrieval-augmented generation (RAG): a pattern in which the LLM does not answer from memory alone but receives a set of relevant documents retrieved by vector similarity from a database filled by a data-aggregation component. RAG is what grounds the generated threat model in the actual architecture and in accumulated organizational knowledge such as earlier threat models. The second load-bearing component is the quality-assurance layer, which is supposed to make the nondeterministic output trustworthy: it validates the requested output format, extracts aspects of the model and compares them through metamorphic relationships—test oracles derived from expected relations between model versions—and assigns a health score that lets stakeholders decide whether to refine their prompt. Together these two pieces turn an off-the-shelf generative model into a context-aware, continuously updatable threat-modeling service.","core_discovery":"The paper sets out the claim that LLMs can support the threat modeling of LIAs well enough to be useful in industrial practice if the LLM is grounded in project-specific context and its output is checked. Concretely, it proposes a composable tool: a data-aggregation layer collects system descriptions, requirement specs, architectural diagrams, prior threat models, and post-release monitoring data; a retrieval-augmented generation (RAG) component vectorizes these sources and injects the relevant ones into the prompt; a prompting layer provides a system prompt, user prompt, and reasoning strategy such as chain of thought; and a quality-assurance layer checks syntactic correctness, runs metamorphic tests on the extracted content, and summarizes the result in a health score. The paper reports a preliminary evaluation in which a zero-shot chatbot prompted with a simple LIA architecture consulted two established LLM threat taxonomies and produced a threat model; the authors describe the results as encouraging and frame the work as the first LLM-supported threat modeling approach for LIAs developed with an industry partner.","pith_inferences":["The paper leaves implicit but directly testable: two teams model the same LIA, one using ThreMoLIA and one using manual methods, and the resulting threat lists are compared against a ground truth assembled through red-team attack simulation. If the automated approach misses a class of attacks that the manual team consistently finds, the central claim fails.","The paper's own admission that threat-model quality metrics do not yet exist suggests the health score may initially measure internal consistency rather than security correctness; validating the health score against real attacks is the first step that the evaluation plan should prioritize.","The proposed RAG-plus-quality-assurance architecture could generalize beyond security, for example to keep compliance documents or safety cases in sync with a changing system, since the underlying problem of grounding a generative model in evolving project artifacts and checking its output is general."],"forward_implications":["Threat modeling for AI-backed applications can begin at the architecture and design phase and be regenerated automatically when the architecture changes, rather than being a one-time manual exercise.","Organizations can reuse their accumulated threat models as a knowledge base, since existing models of similar applications become retrieved context for new work.","LLM-specific threats such as prompt injection and excessive agency would be covered systematically because the retrieval pipeline draws on LLM-specific threat taxonomies rather than only classical software threat categories.","The effort of security experts can shift from producing threat models from scratch to reviewing and refining machine-drafted ones, reducing required participation while keeping human oversight.","The planned industrial evaluation would produce the first benchmark for threat-model quality that is grounded in a systematic literature study and validated with practitioners."],"supporting_citations":[{"why":"Supplies the three-stage RAG design (indexing, retrieval, generation) that ThreMoLIA's context component follows.","marker":"[5]"},{"why":"Establishes prompt injection as a real LIA-specific threat, motivating why a dedicated LLM-aware threat modeling approach is needed.","marker":"[7]"},{"why":"Provides the standard definition and purpose of threat modeling, grounding the paper's argument that early modeling identifies and mitigates threats.","marker":"[20]"},{"why":"Supplies the four-question process framework that organizes the proposed threat modeling workflow.","marker":"[21]"},{"why":"Documents the low maturity of current threat modeling tools and the lack of quality metrics, motivating automation and the evaluation plan.","marker":"[31]"},{"why":"Contributes a machine-learning security maturity model that seeds the proposed quality-assurance health score.","marker":"[10]"}],"fun_headline_variants":["LLM tool automates threat modeling for AI-integrated apps","RAG-powered threat modeling cuts need for security experts","Automated threat modeling for LLM apps grounded in context","ThreMoLIA: Using LLMs to continuously update threat models","Early tests show LLM threat modeling for AI apps is promising"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The approach depends on being able to measure whether a generated threat model is complete and correct, but the paper states both that established quality metrics are lacking and that no agreed-upon benchmark exists to evaluate a threat model.","fun_headline_variants_meta":{"raw":{"variants":["LLM tool automates threat modeling for AI-integrated apps","RAG-powered threat modeling cuts need for security experts","Automated threat modeling for LLM apps grounded in context","ThreMoLIA: Using LLMs to continuously update threat models","Early tests show LLM threat modeling for AI apps is promising"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000724,"raw_usage":{"total_tokens":3259,"prompt_tokens":970,"completion_tokens":2289,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":586,"completion_tokens_details":{"reasoning_tokens":2204}},"tokens_in":586,"tokens_out":2289,"duration_ms":14746,"temperature":1.0,"reasoning_tokens":2204,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-16T10:17:25.134202+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"A concrete falsifier would be an industrial case study where ThreMoLIA-generated threat models for a real LIA are checked against a vulnerability list obtained from an independent red-team exercise, showing the tool reliably omits a threat category such as indirect prompt-injection chains that a human analyst would have caught.","supporting_citations":[{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Establishes prompt injection as a real LIA-specific threat, motivating why a dedicated LLM-aware threat modeling approach is needed."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Provides the standard definition and purpose of threat modeling, grounding the paper's argument that early modeling identifies and mitigates threats."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Supplies the four-question process framework that organizes the proposed threat modeling workflow."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Documents the low maturity of current threat modeling tools and the lack of quality metrics, motivating automation and the evaluation plan."},{"cited_title":"MLSMM: Machine Learning Security Maturity Model","cited_arxiv_id":"2306.16127","evidence_quote":"Contributes a machine-learning security maturity model that seeds the proposed quality-assurance health score."}],"review_version":1}