The Hidden Risks of LLM-Generated Web Application Code: A Security-Centric Evaluation of Code Generation Capabilities in Large Language Models

Deven Lunkad; Naziya Aslam; Sandeep Kumar Shukla; S. Venkatesan; Swaroop Dora

REVIEW 5 major objections 5 minor 17 references

The Hidden Risks of LLM-Generated Web Application Code: A Security-Centric Evaluation of Code Generation Capabilities in Large Language Models

T0 review · 5 major / 5 minor · reviewed 2026-08-16 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read No tested AI model fully meets web-code security best practices.

desk verdict Plausible multi-model LLM security comparison with a useful cross-model ranking, but missing artifacts and an internally inconsistent scoring table make the specific results unverifiable as written. read the letter →

arxiv 2504.20612 v1 pith:VCL33ZRG submitted 2025-04-29 cs.CR cs.AIcs.ET

Swaroop Dora , Deven Lunkad , Naziya Aslam , S. Venkatesan , Sandeep Kumar Shukla This is my paper · ORCID

classification cs.CRcs.AIcs.ET

keywords LLMcodegenerationsecuritywebapplicationauthenticationvulnerabilitiessessionmanagementHTTPheaderscompliancechecklistriskassessmentAI-assisteddevelopment

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper tries to establish that LLM-generated web application code is not production-ready on security grounds. Using identical prompts for an e-commerce authentication system, the paper tests five models—ChatGPT, DeepSeek, Claude, Gemini, and Grok—against a 48-parameter checklist spanning authentication, input validation, session security, secure storage, error handling, and HTTP security headers. The result is that every model falls short of industry best practice: none implements multi-factor authentication or any of the twelve HTTP security headers, and gaps appear in session cookie flags, CORS policy, rate limiting, and error-message disclosure. The paper concludes that human expertise or an automated security assessment framework is required before such code is deployed, and that the models themselves need to be improved to default to secure patterns. The paper argues this matters because developers increasingly rely on LLM-generated code without the security expertise to audit it, so the observed gaps would land directly in real applications.

What carries the argument

The load-bearing object is the security evaluation checklist, a 48-parameter rubric in six broader categories, each parameter judged Yes, No, or NA against generated code. Paired with the rubric is a risk assessment formula, Risk = Likelihood × Impact, that assigns each parameter a severity level from Very Low to Extreme and lets the paper convert raw compliance gaps into risk scores. The checklist does the argument's work: it turns qualitative code inspection into per-model coverage scores (for example, Gemini 8/8 in session security, Grok 3/11 in authentication, and all models 0/12 in HTTP security headers), which is what makes the 'none fully aligns' conclusion specific and comparable across models.

What would settle it

Run the same four prompts on the same five model versions, audit the generated code against the same checklist with two independent reviewers and a static-analysis scanner, and compare the resulting table; if most models implement multi-factor authentication and most of the twelve HTTP security headers in that run, the paper's central claim would be contradicted. A single generated codebase from the study that sets CSP, X-Frame-Options, HSTS, and multi-factor authentication under the published prompts would falsify the relevant row on its own.

Watch

Extended reading notes

Core claim

The paper's central discovery is a systematic, reproducible pattern of non-compliance rather than a single model failure. With the same four prompts, all five models used parameterized SQL queries and escaped special characters, and four of five regenerated session IDs and hashed passwords with salt; ChatGPT, Gemini, and Grok set Secure, HttpOnly, and SameSite cookie flags. Beyond those bright spots, none of the models enabled multi-factor authentication, enforced a secure CORS policy, or set any of the twelve HTTP security headers, and only isolated models added lockout, rate limiting, email verification, CSRF tokens, or session timeout. The compliance table also records model-specific issues: DeepSeek and Gemini allow JavaScript execution and HTML injection, Gemini reveals username existence in errors, and Claude does not hash passwords at all. The paper reads these results as evidence that no evaluated LLM fully aligns with industry best practices across the six security domains.

Load-bearing premise

The conclusion depends on the manual Yes/No/NA ratings in the compliance table being accurate; because the generated code is not shown and the ratings are not cross-checked by a second reviewer or an automated tool, a misclassification in any row could change a model's compliance score and shift the overall finding.

Editorial extensions

If this is right

Authentication code generated by these models should be treated as unaudited until checked against a security checklist, because the default output omits controls such as multi-factor authentication, security headers, and sometimes cookie flags.
Reviewers can prioritise the recurring weak spots—HTTP security headers, multi-factor authentication, CORS configuration, and rate limiting—since those fail across all or most models.
Users who want secure output must enumerate security requirements in the prompt, but the paper notes this is not a substitute for testing because many users will not know every requirement.
Deploying LLM-generated web code without a security gate carries concrete risks: account takeover via missing lockout, session hijacking via missing cookie flags, and cross-site scripting where input filtering is absent.
The paper's checklist and risk matrix give a template for a standardised security assessment of LLM code that could be automated or integrated into deployment pipelines.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

Because the prompts already nudged the models toward secure practice, unprompted everyday usage would likely produce even weaker security, so the observed gaps should be read as a lower bound on real-world risk.
The Yes/No/NA compliance table could be turned into an automated regression test: rerunning the same prompts after model updates and diffing the matrix would expose security regressions in new versions.
A natural follow-up experiment is to generate several samples per model and prompt to measure run-to-run variability, since a single generation per model cannot distinguish a systemic weakness from a one-off lapse.
A testable extension is to run static-analysis security scanners on the same generated code; agreement between the scanner and the manual ratings would show which checklist items can be checked automatically.

Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

Here's the quick read on arXiv:2504.20612. The paper compares five commercial LLMs—ChatGPT 4o, DeepSeek v3, Claude 3.5 Sonnet, Gemini 2.0 Flash, and Grok 3—on the security of generated PHP/HTML authentication code, using a 39-item checklist built from NIST and OWASP guidelines. The headline claim, that none of the models fully implement industry best practices, is entirely plausible and lines up with earlier work by Khoury et al. and Toth et al. The useful new piece is the direct cross-model comparison across six security domains. That's worth having if the data behind it can be trusted.

Credit where due: the checklist is sensibly organized, the prompts are realistic, and the authors map their findings to recognized standards. The discussion even includes a decent nuance, that in-band MFA can be weak. These are not the marks of a sloppy group; the design is coherent.

The problems are in the evidence. First, the generated code, the model outputs, and the scoring rubric are all absent. You cannot check whether the Yes/No/NA entries in Table IV are right. Second, Table IV is internally inconsistent. The note says 'Yes' means the model implements the security feature, but in the XSS rows DeepSeek and Gemini are marked 'Yes' and the text reads that as a vulnerability. Under the stated convention, 'Yes' would mean they are protected, which is the opposite of what the authors claim. Third, Table V's category counts don't reproduce from Table IV. Claude's Input Validation score is listed as 8/10, but a direct count from Table IV yields at most 6 or 7, depending on how you interpret the ambiguous XSS rows. That's not a minor cosmetic issue; the compliance scores are the paper's main result.

The methodological setup is also under-specified. Each model appears to have been run once, with no mention of temperature, number of samples, or seed. LLM outputs are stochastic, so a single run is enough to say 'this prompt produced vulnerable code once,' but not enough to support a general capability claim like 'these models do not generate secure code.' The authors partially acknowledge this when they say that prompting for every security requirement could improve results, but the abstract and conclusion are not tuned to that caveat.

Finally, the risk analysis is a black box. Table III gives qualitative likelihood and impact ratings, and Figure 1 shows radar charts for risk levels, but there's no aggregation formula or numeric scores. You can't audit how 'extreme' or 'very high' risk counts were derived.

Who this is for: researchers or practitioners who want a quick comparative snapshot of current commercial models on web auth security. Right now I'd treat the specific rankings as indicative, not verified.

Recommendation: send to peer review, not desk reject. The topic is timely, the multi-model comparison is a real step beyond single-model studies, and the defects are correctable. A serious referee should require the authors to release the code and prompt outputs, fix the Table IV semantics, make Table V reproducible, specify sampling parameters, and show the risk aggregation. That's heavy revision, but the paper is worth engaging.

Referee Report

5 major / 5 minor

Summary. The paper compares five LLMs (GPT-4o, DeepSeek v3, Claude 3.5 Sonnet, Gemini 2.0 Flash Experimental, Grok 3) on the task of generating a secure authentication system for an e-commerce website, using four prompts specified in Table II. The authors manually evaluate the generated code against a 48-item security checklist organized into six domains (authentication, input validation, session security, secure storage, error handling, HTTP security headers), assign risk levels based on a likelihood-impact matrix, and report per-model compliance counts in Table V. The central claim is that none of the evaluated models fully align with industry best practices, especially in authentication, session management, and HTTP security headers, and that human or automated security review is therefore necessary before deploying LLM-generated web application code.

Significance. The manuscript addresses a timely and practically important question: whether LLMs produce deployable secure authentication code. If the central claim were rigorously established, the paper would provide a valuable caution to developers who rely on LLM-generated web code and would strengthen the case for security assessment frameworks. The comparison of five widely used models on the same prompt set, and the use of a checklist aligned with NIST and OWASP, are useful starting points. However, because the generated code is not disclosed, the sampling design is unreported, and the coding scheme in Table IV is internally inconsistent, the evidence as presented does not yet support the strong universal claim. The study is best characterized as an exploratory pilot that, with substantial revision and artifact release, could become a reproducible benchmark.

major comments (5)

[§IV-A, Table IV (Note) and §IV-A4] The coding semantics for Table IV are internally inconsistent. The note states that 'Yes' denotes the LLM implementing that security feature, but for the rows 'JavaScript execution inside input fields' and 'HTML tag injection possible', a 'Yes' entry is interpreted in the text as the presence of the vulnerability (e.g., DeepSeek and Gemini are described as vulnerable). Under the note's stated semantics, those entries would mean the model implements JavaScript execution, which contradicts the analysis. This inconsistency also prevents the reproduction of Table V: Claude's 8/10 score for Input Validation can only be obtained by counting a 'No' in the vulnerability rows as a pass. Please provide a per-row polarity rule (for flaw parameters, 'No' is the secure outcome) or rephrase the note, and ensure Table V follows that rule consistently.
[§III and Table IV] No sampling or generation configuration is reported. The paper states only that each model was given identical prompts; Table IV contains a single value per model and parameter. Because LLM outputs are stochastic, a single draw cannot establish that a model 'never' produces compliant code or that non-compliance is typical. The conclusion 'none fully align with industry best practices' is therefore under-supported. Please report the number of independent generations, temperature, and seed (or another sampling protocol). If only one run was used, either add repeated sampling to characterize variability or soften the central claim to describe the specific single generated solutions that were examined.
[§IV-B and Table V] The scoring denominators include parameters marked 'NA' for all models (e.g., 'Type of MFA' and 'Backup codes' in Authentication, 'Handling of multiple identical parameters' in Input Validation). Since 'NA' means the concept is not implemented, it is unclear whether these rows are excluded from the denominator or counted as non-compliant. The reported 'x/11' or 'x/10' fractions overstate the number of applicable security features, and the treatment of partial implementations (e.g., 'Only Length', 'Length+letters+numbers', 'Error Code') is not specified. Clarify the treatment of NAs and partial values, and update all fractions in Table V accordingly under the clarified rule.
[§IV-A1 vs Table IV] The text in Section IV-A1 states that both ChatGPT and Gemini enforce a minimum password length, but Table IV lists Gemini as 'No' for the password complexity parameter. This contradiction directly affects the Authentication Security count in Table V (Gemini is credited with 2/11, which is not reproducible from Table IV under any consistent counting rule). Correct the table or the text, and provide a transparent mapping from Table IV entries to the counts in Table V.
[§IV.A and supplementary materials] The generated code itself is not included in the paper nor linked in a repository. Since the entire compliance evaluation is performed by manual inspection, the absence of the generated artifacts makes the results impossible to audit. Please provide the full output for each model and prompt (or a public repository) and, if feasible, the completed evaluation worksheet showing how each cell of Table IV was assigned.

minor comments (5)

[Throughout] Several stylistic issues appear, e.g., 'minimizing the time and effort' should be 'reducing the time and effort', and 'LLMs generated web application code' should be 'LLM-generated web application code' in multiple places.
[Table III] The table contains a typo 'Almost cer tain' and inconsistent capitalization (e.g., 'insignificant' vs 'Insignificant'). The risk matrix citation could also be more explicit about which likelihood/impact levels are standard.
[References and text] The text and references use the spacing 'OW ASP' and 'OW ASP Top 10' in several places; the correct form is 'OWASP'.
[Figure 1] The caption describes radar charts with 'red-shaded regions', but the figure is not visible in the manuscript. Ensure the actual figure is present and legible, and that the subfigure references (1a-1f) match the actual layout.
[Section V] The statement 'The LLMs can generate the secure code by avoiding the identified risk if the prompt specifically mentions every security requirement' is not evaluated or supported by the experiments in this paper; either add a small prompt-ablation experiment or remove this claim to avoid overclaiming.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the security evaluation is an empirical scoring of generated code against external standards.

full rationale

The paper's central claim is an empirical result about five LLM outputs, scored with a checklist grounded in NIST guidelines, OWASP Top 10, and a risk-matrix method cited from prior work [16]. There is no fitted parameter that is later renamed as a prediction; the checklist categories are input assumptions, not outputs of the analysis. The risk formula (Risk = Likelihood × Impact) is a standard definition adopted from an external source and is used to tabulate consequences, not to manufacture the compliance findings. Table IV's Yes/No entries record the authors' reading of the generated code; whatever their reliability, the reading is not equivalent to the conclusion by construction. No load-bearing claim relies on a self-citation: the reference list contains no prior work by these authors, and the 'uniqueness' of the framework is not imported from the authors' own publications. The main threats to the paper's validity are sampling (one generation per model) and coding consistency in Table IV, both of which are empirical-validity concerns rather than circularity. Therefore the circularity score is 0.

Assumptions & free parameters 0 free parameters · 4 assumptions · 0 invented entities

The central claim rests on the manual security evaluation and the chosen checklist and risk matrix. No numeric free parameters are fitted; the subjective likelihood/impact ratings are treated as axioms. No new entities are introduced.

assumptions (4)

domain assumption Risk matrix with ordinal likelihood and impact scales (Risk = Likelihood x Impact) is a valid basis for estimating security risk.
Used in Section III-B to derive risk categories; the specific likelihood/impact values in Table III are chosen by the authors, not empirically measured.
domain assumption The security checklist parameters are the relevant industry best practices for web authentication systems.
Stated in Section III-A, based on NIST and OWASP guidelines; the completeness and applicability of this checklist to all contexts is assumed.
domain assumption The five LLM outputs, generated once with the prompts in Table II, are representative of each model's typical secure-coding behavior.
The paper does not report repeated runs, sampling parameters, or variability; Section III states only that identical prompts were used.
domain assumption Manual inspection of generated code by the authors accurately determines whether each security parameter is satisfied.
No code is shown and no rubric beyond Table III parameters is given, so this is an unverified premise in Section IV-A.

how reviews work

0 comments

Cite this review

Pith. "Pith review of The Hidden Risks of LLM-Generated Web Application Code: A Security-Centric Evaluation of Code Generation Capabilities in Large Language Models." pith.science (2026). https://pith.science/paper/VCL33ZRG

@misc{pith2026250420612,
  author       = {Pith},
  title        = {Pith review of: The Hidden Risks of LLM-Generated Web Application Code: A Security-Centric Evaluation of Code Generation Capabilities in Large Language Models},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/VCL33ZRG}},
  note         = {Machine review of arXiv:2504.20612}
}

read the original abstract

The rapid advancement of Large Language Models (LLMs) has enhanced software development processes, minimizing the time and effort required for coding and enhancing developer productivity. However, despite their potential benefits, code generated by LLMs has been shown to generate insecure code in controlled environments, raising critical concerns about their reliability and security in real-world applications. This paper uses predefined security parameters to evaluate the security compliance of LLM-generated code across multiple models, such as ChatGPT, DeepSeek, Claude, Gemini and Grok. The analysis reveals critical vulnerabilities in authentication mechanisms, session management, input validation and HTTP security headers. Although some models implement security measures to a limited extent, none fully align with industry best practices, highlighting the associated risks in automated software development. Our findings underscore that human expertise is crucial to ensure secure software deployment or review of LLM-generated code. Also, there is a need for robust security assessment frameworks to enhance the reliability of LLM-generated code in real-world applications.

Figures

Figures reproduced from arXiv: 2504.20612 by the authors.

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

17 extracted references · 15 canonical work pages

[1]

Large language model assisted software engineering: prospects, challenges, and a case study,

L. Belzner, T. Gabor, and M. Wirsing, “Large language model assisted software engineering: prospects, challenges, and a case study,” in In- ternational Conference on Bridging the Gap between AI and Reality , pp. 355–374, Springer, 2023

work page 2023
[2]

[Accessed 20-03-2025]

OpenAI, “Openai.” https://www.openai.com/. [Accessed 20-03-2025]

work page 2025
[3]

Meet Claude — anthropic.com

Claude, “Meet Claude — anthropic.com.” https://www.anthropic.com/ claude. [Accessed 20-03-2025]

work page 2025
[4]

Llama — llama.meta.com

Llama, “Llama — llama.meta.com.” https://llama.meta.com/. [Accessed 20-03-2025]

work page 2025
[5]

Survey reveals ai’s impact on the developer experience

I. Shani, “Survey reveals ai’s impact on the developer experience.” https://github.blog/ 2023-06-13-survey-reveals-ais-impact-on-the-developer-experience/,

work page 2023
[6]

Do users write more insecure code with ai assistants?,

N. Perry, M. Srivastava, D. Kumar, and D. Boneh, “Do users write more insecure code with ai assistants?,” in Proceedings of the 2023 ACM SIGSAC conference on computer and communications security , pp. 2785–2799, 2023

work page 2023
[7]

Se- curity weaknesses of copilot generated code in github,

Y . Fu, P. Liang, A. Tahir, Z. Li, M. Shahin, J. Yu, and J. Chen, “Se- curity weaknesses of copilot generated code in github,” arXiv preprint arXiv:2310.02059, 2023

arXiv 2023
[8]

What is the Log4j Vulnerability? — IBM — ibm.com

log4j, “What is the Log4j Vulnerability? — IBM — ibm.com.” https: //www.ibm.com/think/topics/log4j. [Accessed 20-03-2025]

work page 2025

Show all 17 references

[9]

Llms in web development: Evaluat- ing llm-generated php code unveiling vulnerabilities and limitations,

R. T ´oth, T. Bisztray, and L. Erd˝odi, “Llms in web development: Evaluat- ing llm-generated php code unveiling vulnerabilities and limitations,” in International Conference on Computer Safety, Reliability, and Security , pp. 425–437, Springer, 2024

2024
[10]

How secure is code generated by chatgpt?,

R. Khoury, A. R. Avila, J. Brunelle, and B. M. Camara, “How secure is code generated by chatgpt?,” in 2023 IEEE international conference on systems, man, and cybernetics (SMC) , pp. 2445–2451, IEEE, 2023

2023
[11]

deepseek

deepseek, “deepseek.” https://www.deepseek.com/. [Accessed 20-03- 2025]

2025
[12]

[Accessed 20-03- 2025]

Gemini, “Gemini.” https://gemini.google.com/app. [Accessed 20-03- 2025]

2025
[13]

[Accessed 20-03-2025]

Grok, “Grok.” https://x.ai/. [Accessed 20-03-2025]

2025
[14]

[Accessed 20-03-2025]

NIST, “Nist.” https://www.nist.gov/news-events/news/2024/02/ nist-releases-version-20-landmark-cybersecurity-framework. [Accessed 20-03-2025]

2024
[15]

[Accessed 20-03-2025]

OW ASP, “Owasp.” https://owasp.org/www-project-top-ten/. [Accessed 20-03-2025]

2025
[16]

Application of the matrix approach in risk assessment,

N. Kova ˇcevi´c, A. Stojiljkovi´c, and M. Kovaˇc, “Application of the matrix approach in risk assessment,” Operational Research in Engineering Sciences: Theory and Applications , vol. 2, no. 3, pp. 55–64, 2019

2019
[2023]

[Accessed 20-03-2025]

2025

Pith tools

Reviewed August 16, 2026 · model on record in the stance chip above.

[1] [1]

Large language model assisted software engineering: prospects, challenges, and a case study,

L. Belzner, T. Gabor, and M. Wirsing, “Large language model assisted software engineering: prospects, challenges, and a case study,” in In- ternational Conference on Bridging the Gap between AI and Reality , pp. 355–374, Springer, 2023

work page 2023

[2] [2]

[Accessed 20-03-2025]

OpenAI, “Openai.” https://www.openai.com/. [Accessed 20-03-2025]

work page 2025

[3] [3]

Meet Claude — anthropic.com

Claude, “Meet Claude — anthropic.com.” https://www.anthropic.com/ claude. [Accessed 20-03-2025]

work page 2025

[4] [4]

Llama — llama.meta.com

Llama, “Llama — llama.meta.com.” https://llama.meta.com/. [Accessed 20-03-2025]

work page 2025

[5] [5]

Survey reveals ai’s impact on the developer experience

I. Shani, “Survey reveals ai’s impact on the developer experience.” https://github.blog/ 2023-06-13-survey-reveals-ais-impact-on-the-developer-experience/,

work page 2023

[6] [6]

Do users write more insecure code with ai assistants?,

N. Perry, M. Srivastava, D. Kumar, and D. Boneh, “Do users write more insecure code with ai assistants?,” in Proceedings of the 2023 ACM SIGSAC conference on computer and communications security , pp. 2785–2799, 2023

work page 2023

[7] [7]

Se- curity weaknesses of copilot generated code in github,

Y . Fu, P. Liang, A. Tahir, Z. Li, M. Shahin, J. Yu, and J. Chen, “Se- curity weaknesses of copilot generated code in github,” arXiv preprint arXiv:2310.02059, 2023

arXiv 2023

[8] [8]

What is the Log4j Vulnerability? — IBM — ibm.com

log4j, “What is the Log4j Vulnerability? — IBM — ibm.com.” https: //www.ibm.com/think/topics/log4j. [Accessed 20-03-2025]

work page 2025