Infecting Generative AI With Viruses

David Noever; Forrest McKee

REVIEW 4 major objections 5 minor 23 references

Infecting Generative AI With Viruses

T0 review · 4 major / 5 minor · reviewed 2026-08-10 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read JPEGs can carry a harmless EICAR test file into GPT-4o's code sandbox and have it extracted on request.

desk verdict A plausible red-team note on VLM file handling whose evidence is chat quotes, not artifacts—worth commenting on, but only after asking for hard evidence. read the letter →

arxiv 2501.05542 v1 pith:LAR3HNP4 submitted 2025-01-09 cs.CR

David Noever , Forrest McKee This is my paper · ORCID

classification cs.CR

keywords EICARtestfilestegomalwareJPEGappendedpayloadvision-languagemodelsecurityLLMcodeinterpretermalwaretransportbase64obfuscationcloudsandbox

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper claims that a harmless test-virus string called EICAR can be hidden inside an ordinary JPEG by appending it after the image's end marker, and that vision-language models such as GPT-4o will neither flag the image nor refuse to manipulate it. When asked, the model's Python code interpreter splits the file at the EICAR prefix and saves the payload as a separate downloadable file in the hosted workspace. The authors report four GPT-4o protocols: direct appended EICAR, split extraction, reversed-string obfuscation, and base64-encoded transport, and they note that other major chatbots can describe the blank image and write equivalent split code but do not expose the same in-place file operations. The significance is that image files, which vision-language models are built to accept, can become a low-skill delivery path for malware surrogates into hosted AI environments.

What carries the argument

The carrying object is the JPEG container combined with the model's code interpreter. JPEG readers stop at the end-of-image marker 0xFFD9 and ignore trailing data, so appending the EICAR string to a one-pixel JPEG preserves normal image display while hiding the payload. The LLM's Python environment then reads the raw byte stream, splits on the EICAR prefix, reverses it, or base64-decodes it, and writes the result as a file, making the surrogate virus available for shell commands such as cat and md5sum.

What would settle it

Re-run the four protocols in a fresh GPT-4o session with the same one-pixel JPEG plus EICAR, capture vendor-side scan logs and the produced file's hash; if the platform refuses the upload, logs a detection, or produces a file whose MD5 differs from the canonical EICAR hash, the paper's transport and extraction claim fails.

Watch

Extended reading notes

Core claim

On the paper's own terms, the central discovery is that the EICAR test string can ride inside a JPEG without changing how the image displays, and that a hosted model's code interpreter will perform the extraction on request. GPT-4o, after being asked to split the uploaded image on the marker 'X5O!', returned downloadable part1.bin and part2.bin files, and the same manipulation worked when the string was reversed or base64-encoded before upload. Copilot, Gemini, and Claude recognized the blank image and could write Python split code but did not offer equivalent sandbox file operations. The authors conclude that the traditional 'drop an EICAR' penetration test can be completed against an LLM workspace, and that current vision-language file inspection misses post-end-of-file and metadata payloads.

Load-bearing premise

The claim rests on trusting the chatbot's screen text as evidence that the hidden EICAR went undetected by the vendor and that the split file was really saved; the paper provides no vendor logs, file hashes, or reproducible workspace artifacts to back that up.

Editorial extensions

If this is right

A user can deliver a malware surrogate to an LLM workspace simply by uploading a JPEG and asking the model to split it, with no special tooling required.
Vision-language file inspection that relies on normal image decoding will not see payloads appended after the JPEG end marker.
Signature checks on the uploaded image give a false sense of safety because string reversal and base64 encoding are trivially reversible inside the model's Python sandbox.
The model's willingness to write and run the extraction code is what the paper calls an 'uplift' risk: the AI lowers the skill needed to complete a multi-stage malware-activation chain.
Any vendor that exposes a code interpreter over uploaded images should treat post-end-of-file and metadata sections as untrusted input, not just the decoded image.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

A natural next experiment is to repeat the four protocols across every model that exposes a code interpreter and record whether the delivered file's hash survives; the paper's qualitative chat quotes leave this quantitative replication open.
The same container trick likely generalizes to PNG, GIF, ZIP, and EXIF payloads, so the vulnerability class is broader than JPEG tails.
The practical risk may not be executing code inside the ephemeral sandbox, which is disposable, but that the AI generates a downloadable malicious artifact that the user can then propagate through ordinary channels; the paper gestures at this risk but does not test propagation.
If vendors retain the same chat logs a researcher sees, 'undetected' could mean only that no scan result was surfaced to the user; the stronger claim of no vendor-side detection would require access to vendor logs or independent instrumentation.

Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

Short red-team note worth a look if you care about VLM file handling, but the core claim is only as strong as the chat quotes they print. The genuinely interesting bit is 'uplift': GPT-4o writes the Python to split the JPEG at the EICAR marker and offers the halves for download. That is a clean demonstration that a vision model's code interpreter can be guided to perform a classic stego extraction. The paper is honest about mixed results across platforms—Copilot, Gemini, and Claude recognize the image but can't parse it in place, and Grok just says 'Oops'—and it uses EICAR properly as a benign surrogate under Microsoft's own pen-test rules.

The soft spot is exactly where the stress-test puts it: the extraction evidence is user-facing chat text. No vendor tool logs, no file hashes, no actual download URLs, no generated Python code, no reproducible artifact. The authors even admit the opacity ('one can only speculate that standard fail-safes apply'). For the central claim—that the model really wrote part2.bin to the sandbox and offered it for download—that is thin. A model can produce an affirmative sentence without making a tool call, and cloud platforms can sanitize or simulate responses. So 'undetected' and 'extracted' are inferred from the model's own description, not verified. That's a load-bearing gap, though not a fatal one: the behavior is plausible for OpenAI's sandbox, and the paper doesn't claim more than 'potentially executed' in the abstract. But the title 'Infecting Generative AI With Viruses' oversells it; nothing here infects a model.

The novelty is low—appending data after the JPEG EOI marker is decades old, and EICAR drops are standard pen-testing practice. The contribution is an application note plus the uplift framing. The experiments are not fully reproducible as reported, but a competent grad student could recreate them in an afternoon. The citation pattern is fine; self-citation is not an issue here.

Who is this for? Red-teamers and VLM vendors who want a quick checklist, not researchers looking for a new attack class. It deserves a serious referee because the observation is real and vendors should see it, but the authors should be asked to supply the actual tool output and hashes, or the claim should be softened to 'model output suggests.' I would not block publication over it, but I would not cite it in its current evidence state either.

Referee Report

4 major / 5 minor

Summary. The paper reports a short penetration-testing study in which EICAR test strings are appended to JPEG images and uploaded to vision-language/LLM chat platforms. The authors claim that GPT-4o's code interpreter can split a JPEG at the EICAR marker, reverse or base64-decode the payload, and make the EICAR file available in the workspace; they interpret this as 'dropping EICAR' and as an 'uplift' vulnerability. Additional platforms (Microsoft Copilot, Google Gemini 1.5 Pro, Anthropic Claude 3.5 Sonnet, and Grok-2) are tested with mixed results. The paper includes a command table for shell-based analysis and discusses historical stegomalware.

Significance. If substantiated with tool-call logs, file hashes, and filesystem evidence, the result would be a useful, low-risk demonstration that hosted LLM code interpreters can reconstruct a test payload from post-EOF image data, extending Microsoft's 'drop EICAR' rules of engagement to generative AI. The use of EICAR rather than live malware is responsible, and the paper's acknowledgment of environmental opacity is a strength. However, as it stands the central security claim rests on user-facing chat quotes rather than verifiable artifacts, and the cross-platform claim is overstated, so the significance is conditional on the evidence being supplied.

major comments (4)

[Example 2 Protocol (GPT-4o result) and discussion] The central claim that the EICAR file was 'delivered to the LLM unmodified' and 'potentially executed' is supported only by GPT-4o's user-facing reply, 'The file has been successfully split into two parts... Download part1.bin. Download part2.bin.' The manuscript provides no API tool-call transcript, no filesystem listing from the sandbox, no hash of the produced part2.bin, and no evidence that the file was actually executed. Since an LLM can produce affirmative statements without invoking tools, and since the paper itself concedes that 'one can only speculate that standard fail-safes apply to these cloud instances,' this quote alone cannot distinguish a real sandbox write from a simulated or hallucinated response. Please add exact prompts, tool-call logs, file hashes, and filesystem or process evidence, or reword the central conclusion to state that the model generated a script that would split and save the payload.
[Abstract and 'Other Foundational LLM Results'] The abstract claims 'We successfully executed four distinct protocols across multiple LLM platforms, including OpenAI GPT-4o, Microsoft Copilot, Google Gemini 1.5 Pro, and Anthropic Claude 3.5 Sonnet.' In the body, all four Example protocols are GPT-4o runs, and the 'Other Foundational LLM Results' paragraph reports that Copilot does not flag the JPEG but cannot parse it in place, Gemini and Claude write scripts but 'their cloud file storage does not enable it to parse the image in place or offer it to download,' and Grok returns only 'Oops.' These are mixed results, not successful execution of four protocols across four platforms. Please revise the abstract to state that the extraction-and-download result was observed only on GPT-4o, and clearly label the other platforms as partial or failed.
[EICAR test-string definition and Figure 2] The paper prints the EICAR string as 'X5O!P%@AP[4\PZX54(P)7CC)7}$EICAR-STANDARD-ANTIVIRUS-TEST-FILE!--ETC' and says '--ETC is included as placeholder so as not to trigger detection inside the research paper itself.' It is therefore unclear whether the experiments used the canonical EICAR byte string or a truncated variant; Figure 2's caption adds that the computed hash 'differs by a space at the end from the standard hash.' Because the paper's claims about VirusTotal detection rates (63/66) and about 'dropping EICAR' depend on the exact payload, please specify the exact byte string used, provide its SHA-256, and cite a specific VirusTotal report rather than the general service URL. If the tested file was not canonical EICAR, the detection-rate claims should be re-evaluated.
[Title, abstract, and 'Example 2 Protocol' conclusion] The title and several conclusions use execution language ('Infecting Generative AI With Viruses,' 'successfully executed four distinct protocols,' 'parses out the malware to an executable file on the virtual machine'), but the only direct evidence is the model's description of splitting a file. Supplemental Material 1 lists commands such as 'Emulate File Behavior' and 'Simulate Execution (Trace)' as tasks that could be performed, not as outputs that were observed. Unless the authors include an observed execution of the split file (for example, running it and observing a system effect), the manuscript should consistently use 'the model generated commands/scripts for extraction and execution' rather than 'executed' or 'potentially executed' as a validated result.

minor comments (5)

[Throughout] The text contains typos and grammar errors, including 'techni ques' in the abstract, 'sur rogate,' 'a n EICAR file,' and 'ins ide'; please copyedit the manuscript.
[Example 1 Protocol] The base image is described only as a 'one-pixel white blank' JPEG; for reproducibility, please provide the exact image bytes or the precise construction command (including the append method) and the pixel dimensions.
[References] References [9], [11], [18]-[20], and [23] are general URLs without access dates; because the evidence consists of chat outputs, please include the query date and, if possible, archived transcripts for each platform interaction.
[Discussion (uplift)] The term 'uplift' is introduced in the discussion after the experiments; define it in the introduction or avoid using it as if it were an established metric, since the paper does not measure uplift against a baseline.
[Supplemental Material 2] Supplemental Material 2 is titled 'Downloading Remote Infected Macro in Office Excel' but the content appears to be missing from the submitted version; either include the demonstration (screenshot or transcript) or remove the reference to it.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the findings are empirical observations against an external EICAR/VirusTotal standard, with no fitted parameters or load-bearing self-citation.

full rationale

The paper reports a set of exploratory security experiments rather than a derivation. Its central claims—that a JPEG with an appended EICAR string can be uploaded, that LLM code interpreters can split or reverse/base64-decode it, and that the resulting file is flagged by more VirusTotal engines—are empirical observations benchmarked against the external EICAR standard and VirusTotal, not consequences of any fitted parameter or self-referential definition. I found no load-bearing self-citation: the cited Microsoft pen-test rules, EICAR literature, steganography surveys, and previous LLM-security papers are used as background or as the external rule-of-engagement standard, and none of the authors' earlier works is invoked to justify the experimental conclusion. The closest thing to a self-referential element is the paper's definition of its own success criterion ('whether a documented set of prompts and file construction can place an EICAR file on the virtual machine'), but the reported result is an observed chat/tool response against that criterion, not the criterion itself. The manuscript also admits an evidentiary limitation: 'Given the closed operations of foundational LLMs, one can only speculate that standard fail-safes apply to these cloud instances...' This is a transparency/evidence-quality caveat, not a circularity. Likewise, the reader's concern that extraction is inferred from chat text rather than filesystem hashes is a validity risk, not a reduction of the conclusion to its inputs by construction. Therefore no circular step meets the evidentiary standard required by the rubric.

Assumptions & free parameters 0 free parameters · 4 assumptions · 0 invented entities

No free parameters or fitted constants appear. The paper depends on domain assumptions about EICAR as a malware surrogate, JPEG trailing-byte behavior, and the accuracy of model-generated evidence. No new physical entities are introduced; 'uplift' is a descriptive label for model assistance, not an invented mechanism.

assumptions (4)

domain assumption EICAR test string is an adequate surrogate for real malware.
The paper uses a benign ASCII file to stand in for a virus, but EICAR is not executable on Linux and no real payload was run; conclusions about 'virus transport' depend on this analogy.
domain assumption JPEG viewers and VLM image parsers ignore bytes appended after the 0xFFD9 end-of-image marker while byte-level tools can still read them.
The whole attack relies on trailing bytes being invisible to image analysis but accessible to Python file reads; this is standard behavior but is assumed without testing on each platform.
domain assumption The models' textual responses accurately describe their detection status and file-system actions.
No vendor-side logs or independent artifacts are provided; 'not detected' and 'file split' are taken from chat replies, which may not reflect actual scanning results.
domain assumption Microsoft's Penetration Testing Rules of Engagement permits and encourages dropping EICAR, making the test an authorized benchmark.
The ethical framing depends on this external policy; the paper does not discuss whether each vendor's terms allow this testing.

how reviews work

0 comments

Cite this review

Pith. "Pith review of Infecting Generative AI With Viruses." pith.science (2026). https://pith.science/paper/LAR3HNP4

@misc{pith2026250105542,
  author       = {Pith},
  title        = {Pith review of: Infecting Generative AI With Viruses},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/LAR3HNP4}},
  note         = {Machine review of arXiv:2501.05542}
}

read the original abstract

This study demonstrates a novel approach to testing the security boundaries of Vision-Large Language Model (VLM/ LLM) using the EICAR test file embedded within JPEG images. We successfully executed four distinct protocols across multiple LLM platforms, including OpenAI GPT-4o, Microsoft Copilot, Google Gemini 1.5 Pro, and Anthropic Claude 3.5 Sonnet. The experiments validated that a modified JPEG containing the EICAR signature could be uploaded, manipulated, and potentially executed within LLM virtual workspaces. Key findings include: 1) consistent ability to mask the EICAR string in image metadata without detection, 2) successful extraction of the test file using Python-based manipulation within LLM environments, and 3) demonstration of multiple obfuscation techniques including base64 encoding and string reversal. This research extends Microsoft Research's "Penetration Testing Rules of Engagement" framework to evaluate cloud-based generative AI and LLM security boundaries, particularly focusing on file handling and execution capabilities within containerized environments.

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

23 extracted references · 16 canonical work pages

[1]

Harley, D., Myers, L., & Willems, E. (2011). Test Files and Product Evaluation: The Case for and Against Malware Simulation. In AVAR Conference

work page 2011
[2]

Dunham, K. (2004). EICAR Test File Security Considerations. Inf. Secur. J. A Glob. Perspect., 12(6), 7-11

2004
[3]

(1999, September)

Abrams, R. (1999, September). Giving the EICAR test file some teeth. In Virus Bulletin 1999 Conference Proceedings

work page 1999
[4]

(2024, June)

Knöchel, M., & Karius, S. (2024, June). Text Steganography Methods and their Influence in Malware: A Comprehensive Overview and Evaluation. In Proceedings of the 2024 ACM Workshop on Information Hiding and Multimedia Security (pp. 113-124)

work page 2024
[5]

Williams, O. C. (2019). What are the cybersecurity risks of artificial intelligence generated steganography? (Master's thesis, Utica College)

work page 2019
[6]

MSRC (2025), Penetration Testing Rules of Engagement, Microsoft Re search Center, https://www.microsoft.com/en-us/msrc/pentest-rules-of-engagement

work page 2025
[7]

(2019), EICAR Standard Antivirus Test Files, https://github.com/fire1ce/eicar-standard- antivirus-test-files

Yakobov,S. (2019), EICAR Standard Antivirus Test Files, https://github.com/fire1ce/eicar-standard- antivirus-test-files

work page 2019
[8]

Salem, A., Banescu, S., & Pretschner, A. (2021). Maat: Automatically analyzing virustotal for accurate labeling and effective malware detection. ACM Transactions on Privacy and Security (TOPS) , 24(4), 1-35

work page 2021

Show all 23 references

[9]

https://www.virustotal.com/

VirusTotal (2025), Analyse suspicious files, domains, IPs and URLs to detect malware and other breaches, automatically share them with the security community. https://www.virustotal.com/

2025
[10]

SuperUser (2015), Appending one file to the end of another, https://superuser.com/questions/705126/appending -one-file-to-the-end-of-another

2015
[11]

Microsoft CoPilot (2025), https://copilot.microsoft.com/

2025
[12]

Chaganti, R., Ravi, V., Alazab, M., & Pham, T. D. (2021). Stegomalware: A Systematic Survey of Malware Hiding and Detection in Images, Machine Learning Models and Research Challenges. arXiv preprint arXiv:2110.02504

2021 arXiv
[13]

OpenAI (2024), Building an early warning system for LLM -aided biological threat creation , https://openai.com/index/building-an-early-warning-system-for-llm-aided-biological-threat-creation

2024
[14]

Chan, A., Bucknall, B., Bradley, H., & Krueger, D. (2023). Hazards from Increasingly Accessible Fine - Tuning of Downloadable Foundation Models. arXiv preprint arXiv:2312.14751

2023 arXiv
[15]

Jones, E., Dragan, A., & Steinhardt, J. (2024). Adversaries can misuse combinations of safe models. arXiv preprint arXiv:2406.14595

2024 arXiv
[16]

Open AI, (2024), OpenAI o1 System Card, https://openai.com/index/openai-o1-system-card/

2024
[17]

Open AI (2024), OpenAI o1 preview System Card, https://cdn.openai.com/o1-preview-system-card- 20240917.pdf

2024
[18]

Google Gemini (2025), Advanced Pro 1.5, https://gemini.google.com/

2025
[19]

Anthropic Claude 3.5 Sonnet (2025), https://claude.ai/

2025
[20]

Open AI GPT-4o (2025), https://chatgpt.com/

2025
[21]

Gridinsoft (2025), https://gridinsoft.com/

2025
[22]

(2024, December)

Liu, T., Deng, Z., Meng, G., Li, Y., & Chen, K. (2024, December). Demystifying RCE vulnerabilities in LLM-integrated apps. In Proceedings of the 2024 on ACM SIGSAC Conference on Computer and Communications Security (pp. 1716-1730)

2024
[23]

Executable Scenarios Command/Task Description Reason to Perform It Inspect File Type Use a file equivalent to determine the file type and format

Grok-2 (2025) X.com, https://x.com/i/grok Supplemental Material 1. Executable Scenarios Command/Task Description Reason to Perform It Inspect File Type Use a file equivalent to determine the file type and format. Verify the nature of the file (e.g., binary, executable, text) a...

2025

Pith tools

Reviewed August 10, 2026 · model on record in the stance chip above.

[1] [1]

Harley, D., Myers, L., & Willems, E. (2011). Test Files and Product Evaluation: The Case for and Against Malware Simulation. In AVAR Conference

work page 2011

[2] [2]

Dunham, K. (2004). EICAR Test File Security Considerations. Inf. Secur. J. A Glob. Perspect., 12(6), 7-11

2004

[3] [3]

(1999, September)

Abrams, R. (1999, September). Giving the EICAR test file some teeth. In Virus Bulletin 1999 Conference Proceedings

work page 1999

[4] [4]

(2024, June)

Knöchel, M., & Karius, S. (2024, June). Text Steganography Methods and their Influence in Malware: A Comprehensive Overview and Evaluation. In Proceedings of the 2024 ACM Workshop on Information Hiding and Multimedia Security (pp. 113-124)

work page 2024

[5] [5]

Williams, O. C. (2019). What are the cybersecurity risks of artificial intelligence generated steganography? (Master's thesis, Utica College)

work page 2019

[6] [6]

MSRC (2025), Penetration Testing Rules of Engagement, Microsoft Re search Center, https://www.microsoft.com/en-us/msrc/pentest-rules-of-engagement

work page 2025

[7] [7]

(2019), EICAR Standard Antivirus Test Files, https://github.com/fire1ce/eicar-standard- antivirus-test-files

Yakobov,S. (2019), EICAR Standard Antivirus Test Files, https://github.com/fire1ce/eicar-standard- antivirus-test-files

work page 2019

[8] [8]

Salem, A., Banescu, S., & Pretschner, A. (2021). Maat: Automatically analyzing virustotal for accurate labeling and effective malware detection. ACM Transactions on Privacy and Security (TOPS) , 24(4), 1-35

work page 2021