REVIEW 3 major objections 6 minor 99 references
This paper argues that imperceptible changes to user-shared images can corrupt the long-term memory of multimodal AI agents, with 61.6% poisoning and 58.4% injection success across five memory backends.
Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →
T0 review · deepseek-v4-flash
2026-08-01 22:39 UTC pith:UCCZANFW
load-bearing objection LUCID's headline image-only claim is contradicted by its own appendix: the attack also rewrites image_caption, so the ASR numbers conflate pixels with planted text. the 3 major comments →
Do Agents Dream of False Memories? Black-box Visual Attacks on Long-term Memory in Multimodal AI Agents
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
Core claim
On its own terms, the discovery is that adversarial perturbations built against a surrogate ensemble of publicly available contrastive vision-language encoders transfer to victim memory systems the attacker never sees, and that adding a text-alignment term—matching the perturbed image to the target caption in a joint embedding space—is what steers the stored description toward the attacker's claim. LUCID works in two modes: poisoning replaces the image in an existing turn whose text already anchors a real event, so the stored entry is internally inconsistent but textually invisible; injection plants a new turn on a fresh topic, where the false concept lives only in the visual embedding and n
What carries the argument
The load-bearing mechanism is the three-stage LUCID pipeline. Stage 1 scores candidate image-caption pairs on retrieval displacement, semantic contradiction of the correct answer, and overlap with the ground truth, selecting a target concept. Stage 2 assembles either a poisoned turn or an injected turn. Stage 3 maximizes a composite loss: optimal-transport feature alignment plus global cosine alignment across a surrogate ensemble, extended with a text-alignment objective that pushes the perturbed image toward the target caption's embedding, using projected signed-gradient updates at an l-infinity budget. The text-alignment term is what ties the pixel change to the language the memory system
Load-bearing premise
The load-bearing premise is that the pixel perturbation alone drives the reported success; the implementation also swaps the stored image caption—Appendix B.3.1 lists image_caption among the modified fields—and no condition varies the caption independently to isolate the image's contribution.
What would settle it
Re-run the poisoning and injection evaluations under four conditions: clean, oracle target, perturbed image with the original caption, and original image with the planted caption. If the perturbed-image-with-original-caption condition drops to near-baseline success while the planted-caption condition retains high success, the attack is caption-driven, not image-driven; that settles the image-only claim.
If this is right
- Memory backends that accept images and captions unconditionally at write time absorb poisoned entries in every tested system; the paper reports 100% write-through with no deduplication or rejection.
- Text-side defenses underperform: an LLM-as-judge write filter did not reduce injection success, and retrieval-time anomaly scoring left success unchanged, because planted captions are fluent and probe queries align with the fabricated content.
- Simple image preprocessing (Gaussian blur, JPEG compression) roughly halves average injection success with little effect on question-answering quality, but the paper notes these may not stop more recent stealthy adversarial image methods.
- Retrieval and generation are separable robustness dimensions: some models retrieve adversarial entries at high rates yet resist acting on them, while others act on them more readily than on the unperturbed oracle image.
- Placing an injection far from the probe does not reliably defuse it; in graph-structured memory the attack persists across intervening turns, and in one architecture success rises with temporal distance.
Where Pith is reading between the lines
- The paper frames the attack as image-only, but Appendix B.3.1 lists the stored image caption among the modified fields and no experiment holds captions fixed; the image-only claim would stand only if a perturbed-image-with-original-caption condition reproduces the success.
- If the perturbation alone is sufficient, then the practical countermeasure is not better filtering of retrieved text but write-time provenance and cross-modal consistency checks that verify the image against the stored text and the user's history.
- The oracle condition—an unperturbed target image with a planted caption—achieves results close to the adversarial condition, suggesting the bottleneck is the memory system's inability to verify visual authenticity rather than the subtlety of the pixel noise.
- A natural testable extension is to run the same pipeline on video or audio memory channels; similar transfer would indicate the vulnerability is about unconditional trust in multimodal content, not an artifact of still images.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper proposes LUCID, a black-box adversarial framework for corrupting the persistent long-term memory of multimodal AI agents. The claimed threat model is strictly image-bounded: the attacker only perturbs user-shared images and never writes text, with no access to the target MLLM, retrieval encoder, or memory store. LUCID operates in three stages — target selection from ShareGPT4V, payload construction in two modes (in-context poisoning and out-of-context injection), and transferable perturbation via a CLIP-based surrogate ensemble with a text-alignment loss. The evaluation uses the Mem-Gallery benchmark across five memory backends (MuRAG, NGMemory, AUGUSTUS, UniversalRAG, Mem0) and five MLLMs, reporting 61.6% poisoning ASR and 58.4% injection ASR, plus ablations over encoder, topic, injection count, temporal distance, and defenses.
Significance. If the stated image-only threat model were cleanly established, this would be a notable contribution: it is the first claimed trigger-free, image-only attack on persistent episodic memory in multimodal agents, evaluated across architecturally diverse backends including a production-oriented system. The paper has real strengths: evaluation over five memory architectures and five MLLMs, cross-encoder transfer experiments (including GME), a defense study, and detailed appendices that make the pipeline largely reproducible. However, the central claim is currently not supported because the implementation changes the image caption alongside the pixels and no ablation isolates the visual perturbation. The significance is therefore conditional on a load-bearing fix: an image-only condition that removes the caption confound. If that fix succeeds, the paper could be an important red-team result; if it fails, the contribution reduces to a less novel joint image-plus-caption attack. The paper also provides useful empirical observations about retrieval-stage versus generation-stage robustness, regardless of the main threat-model issue.
major comments (3)
- [§4.2, §5.2, Appendix B.3.1, Eq. (15), Table 10] The core 'strictly image-bounded, text-free' claim is contradicted by the implementation. Appendix B.3.1 states that 'the only fields modified are input_image, image_caption, and (optionally) image_id.' For in-context poisoning, _build_poison_caption writes an explicit false claim into the caption (e.g., 'A Brittany Spaniel dog...'); for out-of-context injection, the target caption is used as image_caption and the adversarial condition in Table 10 is described as pairing the perturbed image with an 'adversarial image caption.' No condition compares the perturbed image with the original/benign caption. Thus the reported 61.6% poisoning ASR and 58.4% injection ASR conflate pixel perturbations with attacker-written captions. The defense results in Table 5 do not isolate the caption contribution. A minimal ablation — adversarial image + original caption versus adversarial image + target capt
- [§6 'Attack Procedure' and Table 10] The 'oracle' condition is not a visual-only upper bound: it changes both the image and the caption relative to the clean condition, and the adversarial condition also changes both. The oracle–adversarial comparison therefore does not isolate the contribution of the perturbation. Additionally, §5.2 claims for out-of-context injection that 'the target caption c* never appears in any text field' and 'the attacker writes no text,' yet the implementation stores the target caption as image_caption and uses category-specific user primitives. This internal inconsistency affects the interpretation of every ASR table and should be resolved explicitly in the revised manuscript.
- [§6 'Evaluation Metrics' and §5.1 Eq. (8)] The poisoning metric 'ASR (VS)' measures visual similarity between the agent's response and the attacker-chosen target concept. The paper does not specify which embedding is used for this similarity, but the attack itself is optimized against CLIP image-text embeddings (Eq. 8 and Eq. 21 use CLIP-ViT-B/32), and the surrogate ensemble is CLIP-based. If ASR(VS) is computed with the same CLIP family, the metric can be high even when the MLLM's response is not semantically target-like to an independent judge. Please specify the similarity model and report agreement with an independent LLM or human judge; otherwise the headline success rate is vulnerable to circularity.
minor comments (6)
- [Title] The title contains a typo: 'F alse Memories' has an extra space.
- [Appendix B.1] There is a typo: 'PT-4o-mini' should be 'GPT-4o-mini'.
- [Section 6] The list of memory backends reads '(i) MuRAG, (ii) NGMemory, (iii) AUGUSTUS, (iv) UniversalRAG, and (vi) Mem0' — the numbering skips (v).
- [Tables 14–15] In Tables 14 and 15, ASR, C-ASR, and Ret. Rate are annotated with downward arrows and described as 'lower is better (↓).' For attack-success metrics, higher values are favorable to the attacker; the arrows and caption are reversed and should be corrected.
- [Figure 3] Figure 3 appears to be a low-resolution screenshot with values such as '45 80 35 50' scattered without clear axis labels or legend structure; it is difficult to read as rendered.
- [Notation] The paper alternates between 'Lucid' and 'LUCID' (e.g., Figure 1 labels versus algorithm text); please use one consistent name.
Circularity Check
Reported ASR conflates pixel perturbation with attacker-modified image_caption; 'image-bounded' holds only by excluding caption from 'text'.
specific steps
-
self definitional
[Abstract/Sec. 5.2 vs. Appendix B.3.1]
"Both mechanisms leave every word of the user/assistant text intact; the only fields modified are input_image, image_caption, and (optionally) image_id."
The paper's central threat model is 'strictly image-bounded' with 'no access to ... the text channel.' The appendix defines the text channel as only 'user/assistant text' and freely modifies image_caption, a text field that is stored in memory and later read by the MLLM. The predicate 'image-bounded' therefore holds by definition rather than by evidence: any attack outcome obtained with a modified caption qualifies as image-only, regardless of whether the pixel perturbation contributed. The claim 'success depends on the visual perturbation; the text channel provides no assistance' is thus enforced by excluding the one text field the attack actually edits.
-
fitted input called prediction
[Sec. 6 Evaluation Metrics; Appendix B.3.3 Step 1; Table 10]
"Constructs the attack turn with the category’s user_primitive as the user message, an empty assistant response, the source image as input_image, and the target caption as image_caption. ... 'ASR' measures the fraction of probes where the agent’s response contains the injected false claim."
The target caption—the literal false claim used as the success criterion—is inserted verbatim into the stored turn as image_caption in both oracle and adversarial conditions (Table 10: 'adversarial image caption'). ASR then counts responses containing that same planted claim. The reported 61.6% poisoning and 58.4% injection success rates therefore measure the presence of attacker-written text in memory, not the effect of the image perturbation. No condition keeps the original caption with the perturbed image, so the numbers are consistent with a text-injection attack and cannot be attributed to the visual perturbation.
-
self definitional
[Sec. 5.2 vs. Appendix B.3.3]
"The target caption c∗ never appears in any text field of t+. ... Constructs the attack turn with the category’s user_primitive as the user message, an empty assistant response, the source image as input_image, and the target caption as image_caption."
The paper's argument that out-of-context injection is undetectable rests on c* having 'no textual trace at storage time.' The implementation places c* directly into image_caption, which is a text field stored in the memory entry. The detection-asymmetry conclusion is therefore true only if image_caption is redefined as non-text—the same self-definitional move that sustains the 'strictly image-bounded' threat model. The central security claim is thus reduced to a definitional exclusion rather than an empirical property of the attack.
full rationale
The paper's novel contribution is the claim that LUCID is a strictly image-bounded, trigger-free attack requiring no access to the text channel. That claim is not established by the implemented conditions: the appendix shows that image_caption, a text field stored in memory and read by the MLLM, is modified to explicitly contain the target lie, and the injection condition places the target caption directly as image_caption. The ASR metric then checks whether the response contains that same false claim, so the headline success rates are compatible with a text-injection attack. There is partial independent evidence—transfer across heterogeneous retrieval encoders and the defense table showing image preprocessing reduces ASR—indicating the perturbation has some real effect. However, the central threat-model claim reduces by construction: 'image-bounded' is defined to exclude image_caption from 'text,' and the attack-success metric is coupled to the attacker-planted caption. This is partial circularity rather than a fully empty derivation, so the score is 6 rather than higher. No load-bearing self-citation or imported uniqueness theorem was found.
Axiom & Free-Parameter Ledger
free parameters (3)
- target scoring weights α, β, γ =
α=1.0, β=2.0, γ=0.5
- text-alignment loss weight λ_txt =
not reported
- perturbation hyperparameters =
ε=16/255, step=0.5/255, T=1000, K=10
axioms (3)
- domain assumption Attacker can substitute images at the upload path (CDN/SDK/webpage) without detection.
- domain assumption Victim memory backends store and later present retrieved images without semantic integrity verification.
- domain assumption Adversarial examples transfer from CLIP-based surrogates to target encoders (GME, SigLIP) and MLLMs.
read the original abstract
Multimodal AI agents increasingly rely on persistent long-term memory to ground generation in past visual and textual episodes. We show that unconditional trust in visual data creates a critical vulnerability. We propose Lucid, a black-box adversarial framework that compromises multimodal memory pipelines under a strictly image-bounded threat model, requiring no access to the target MLLM, target retrieval encoder, or the text channel. Lucid crafts imperceptible perturbations to enable two distinct failure modes based on the availability of historical context: (1) Memory poisoning, an in-context attack where the adversarial image replaces a benign one whose content is reinforced by prior textual context, reliably corrupting visual recall and steering the agent toward attacker-chosen narratives; (2) Memory injection, an out-of-context attack where the adversarial image replaces a benign one in a conversation turn devoid of prior textual grounding, causing the agent to generate attacker-influenced responses with no corrective signal from memory. We evaluate Lucid across various conversation domains and five black-box memory architectures, including graph-structured, LLM-summarized, and commercially deployed systems. Lucid achieves 61.6% ASR on poisoning and 58.4% ASR on injection, exposing a structural vulnerability in multimodal memory pipelines.
Figures
Reference graph
Works this paper leans on
-
[1]
https://www.anthropic.com/claude/haiku
Claude Haiku 4.5 — anthropic.com. https://www.anthropic.com/claude/haiku. [Ac- cessed 06-05-2026]
2026
-
[2]
https://docs.mem0.ai/open-source/ features/multimodal-support
Multimodal Support - Mem0 — docs.mem0.ai. https://docs.mem0.ai/open-source/ features/multimodal-support. [Accessed 04-05-2026]
2026
-
[3]
https://platform
text-embedding-3-small Model | OpenAI API — platform.openai.com. https://platform. openai.com/docs/models/text-embedding-3-small. [Accessed 03-05-2026]
2026
-
[4]
Mahyar Abbasian, Iman Azimi, Amir M Rahmani, and Ramesh Jain. Conversational health agents: A personalized llm-powered agent framework.arXiv preprint arXiv:2310.02374, 2023
Pith/arXiv arXiv 2023
-
[5]
Gpt-4 technical report.arXiv preprint arXiv:2303.08774, 2023
Josh Achiam, Steven Adler, Sandhini Agarwal, Lama Ahmad, Ilge Akkaya, Florencia Leoni Aleman, Diogo Almeida, Janko Altenschmidt, Sam Altman, Shyamal Anadkat, et al. Gpt-4 technical report.arXiv preprint arXiv:2303.08774, 2023
Pith/arXiv arXiv 2023
-
[6]
Eugene Bagdasaryan, Tsung-Yin Hsieh, Ben Nassi, and Vitaly Shmatikov. Abusing images and sounds for indirect instruction injection in multi-modal llms.arXiv preprint arXiv:2307.10490, 2023
Pith/arXiv arXiv 2023
-
[7]
Yuanchen Bei, Tianxin Wei, Xuying Ning, Yanjun Zhao, Zhining Liu, Xiao Lin, Yada Zhu, Hendrik Hamann, Jingrui He, and Hanghang Tong. Mem-gallery: Benchmarking multimodal long-term conversational memory for mllm agents.arXiv preprint arXiv:2601.03515, 2026
arXiv 2026
-
[8]
Poisoning attacks against support vector machines.arXiv preprint arXiv:1206.6389, 2012
Battista Biggio, Blaine Nelson, and Pavel Laskov. Poisoning attacks against support vector machines.arXiv preprint arXiv:1206.6389, 2012
Pith/arXiv arXiv 2012
-
[9]
Improving language models by retrieving from trillions of tokens
Sebastian Borgeaud, Arthur Mensch, Jordan Hoffmann, Trevor Cai, Eliza Rutherford, Katie Millican, George Bm Van Den Driessche, Jean-Baptiste Lespiau, Bogdan Damoc, Aidan Clark, et al. Improving language models by retrieving from trillions of tokens. InInternational conference on machine learning, pages 2206–2240. PMLR, 2022
2022
-
[10]
Language models are few-shot learners.Advances in neural information processing systems, 33:1877–1901, 2020
Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared D Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, et al. Language models are few-shot learners.Advances in neural information processing systems, 33:1877–1901, 2020
1901
-
[11]
Chunliang Chen, Ming Guan, Xiao Lin, Jiaxu Li, Luxi Lin, Qiyi Wang, Xiangyu Chen, Jixiang Luo, Changzhi Sun, Dell Zhang, et al. Telemem: Building long-term and multimodal memory for agentic ai.arXiv preprint arXiv:2601.06037, 2025
arXiv 2025
-
[12]
Less is more: Empowering gui agent with context- aware simplification
Gongwei Chen, Xurui Zhou, Rui Shao, Yibo Lyu, Kaiwen Zhou, Shuai Wang, Wentao Li, Yinchuan Li, Zhongang Qi, and Liqiang Nie. Less is more: Empowering gui agent with context- aware simplification. InProceedings of the IEEE/CVF International Conference on Computer Vision, pages 5901–5911, 2025
2025
-
[13]
Sharegpt4v: Improving large multi-modal models with better captions
Lin Chen, Jinsong Li, Xiaoyi Dong, Pan Zhang, Conghui He, Jiaqi Wang, Feng Zhao, and Dahua Lin. Sharegpt4v: Improving large multi-modal models with better captions. InEuropean Conference on Computer Vision, pages 370–387. Springer, 2024
2024
-
[14]
Murag: Multimodal retrieval-augmented generator for open question answering over images and text
Wenhu Chen, Hexiang Hu, Xi Chen, Pat Verga, and William Cohen. Murag: Multimodal retrieval-augmented generator for open question answering over images and text. InProceedings of the 2022 Conference on Empirical Methods in Natural Language Processing, pages 5558– 5570, 2022
2022
-
[15]
Xinyun Chen, Chang Liu, Bo Li, Kimberly Lu, and Dawn Song. Targeted backdoor attacks on deep learning systems using data poisoning.arXiv preprint arXiv:1712.05526, 2017
Pith/arXiv arXiv 2017
-
[16]
Agentpoison: Red- teaming llm agents via poisoning memory or knowledge bases.Advances in Neural Information Processing Systems, 37:130185–130213, 2024
Zhaorun Chen, Zhen Xiang, Chaowei Xiao, Dawn Song, and Bo Li. Agentpoison: Red- teaming llm agents via poisoning memory or knowledge bases.Advances in Neural Information Processing Systems, 37:130185–130213, 2024. 10
2024
-
[17]
Zirui Cheng, Jikai Sun, Anjun Gao, Yueyang Quan, Zhuqing Liu, Xiaohua Hu, and Minghong Fang. Secure retrieval-augmented generation against poisoning attacks.arXiv preprint arXiv:2510.25025, 2025
arXiv 2025
-
[18]
Prateek Chhikara, Dev Khant, Saket Aryan, Taranjeet Singh, and Deshraj Yadav. Mem0: Building production-ready ai agents with scalable long-term memory.arXiv preprint arXiv:2504.19413, 2025
Pith/arXiv arXiv 2025
-
[19]
Gheorghe Comanici, Eric Bieber, Mike Schaekermann, Ice Pasupat, Noveen Sachdeva, Inderjit Dhillon, Marcel Blistein, Ori Ram, Dan Zhang, Evan Rosen, et al. Gemini 2.5: Pushing the frontier with advanced reasoning, multimodality, long context, and next generation agentic capabilities.arXiv preprint arXiv:2507.06261, 2025
Pith/arXiv arXiv 2025
-
[20]
Towards large language models with human-like episodic memory.Trends in Cognitive Sciences, 2025
Cody V Dong, Qihong Lu, Kenneth A Norman, and Sebastian Michelmann. Towards large language models with human-like episodic memory.Trends in Cognitive Sciences, 2025
2025
-
[21]
Shen Dong, Shaochen Xu, Pengfei He, Yige Li, Jiliang Tang, Tianming Liu, Hui Liu, and Zhen Xiang. Memory injection attacks on llm agents via query-only interaction.arXiv preprint arXiv:2503.03704, 2025
arXiv 2025
-
[22]
How robust is google’s bard to adversarial image attacks? InR0-FoMo: Robustness of Few-shot and Zero-shot Learning in Large Foundation Models
Yinpeng Dong, Huanran Chen, Jiawei Chen, Zhengwei Fang, Xiao Yang, Yichi Zhang, Yu Tian, Hang Su, and Jun Zhu. How robust is google’s bard to adversarial image attacks? InR0-FoMo: Robustness of Few-shot and Zero-shot Learning in Large Foundation Models
-
[23]
Videoagent: A memory-augmented multimodal agent for video understanding
Yue Fan, Xiaojian Ma, Rujie Wu, Yuntao Du, Jiaqi Li, Zhi Gao, and Qing Li. Videoagent: A memory-augmented multimodal agent for video understanding. InEuropean Conference on Computer Vision, pages 75–92. Springer, 2024
2024
-
[24]
Neural graph memory: A structured approach to long-term memory in multimodal agents
Matthew Fisher. Neural graph memory: A structured approach to long-term memory in multimodal agents. 2025
2025
-
[25]
Embodied one-shot video recognition: Learning from actions of a virtual embodied agent
Yuqian Fu, Chengrong Wang, Yanwei Fu, Yu-Xiong Wang, Cong Bai, Xiangyang Xue, and Yu-Gang Jiang. Embodied one-shot video recognition: Learning from actions of a virtual embodied agent. InProceedings of the 27th ACM international conference on multimedia, pages 411–419, 2019
2019
-
[26]
Assistgpt: A general multi-modal assistant that can plan, execute, inspect, and learn
Difei Gao, Lei Ji, Luowei Zhou, Kevin Qinghong Lin, Joya Chen, Zihan Fan, and Mike Zheng Shou. Assistgpt: A general multi-modal assistant that can plan, execute, inspect, and learn. arXiv preprint arXiv:2306.08640, 2023
Pith/arXiv arXiv 2023
-
[27]
How to train long-context language models (effectively)
Tianyu Gao, Alexander Wettig, Howard Yen, and Danqi Chen. How to train long-context language models (effectively). InProceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 7376–7399, 2025
2025
-
[28]
Yunfan Gao, Yun Xiong, Xinyu Gao, Kangxiang Jia, Jinliu Pan, Yuxi Bi, Yixin Dai, Jiawei Sun, Haofen Wang, Haofen Wang, et al. Retrieval-augmented generation for large language models: A survey.arXiv preprint arXiv:2312.10997, 2(1):32, 2023
Pith/arXiv arXiv 2023
-
[29]
Llm-based framework for administrative task automation in healthcare
Senay A Gebreab, Khaled Salah, Raja Jayaraman, Muhammad Habib ur Rehman, and Samer Ellaham. Llm-based framework for administrative task automation in healthcare. In2024 12th International Symposium on Digital Forensics and Security (ISDFS), pages 1–7. IEEE, 2024
2024
-
[30]
Adversar- ial embedding: A robust and elusive steganography and watermarking technique
Salah GHAMIZI, Maxime CORDY , Mike PAPADAKIS, and Yves LE TRAON. Adversar- ial embedding: A robust and elusive steganography and watermarking technique. InIEEE Symposium on Security and Privacy, 2020
2020
-
[31]
Explaining and harnessing adversar- ial examples.arXiv preprint arXiv:1412.6572, 2014
Ian J Goodfellow, Jonathon Shlens, and Christian Szegedy. Explaining and harnessing adversar- ial examples.arXiv preprint arXiv:1412.6572, 2014
Pith/arXiv arXiv 2014
-
[32]
Tianyu Gu, Brendan Dolan-Gavitt, and Siddharth Garg. Badnets: Identifying vulnerabilities in the machine learning model supply chain.arXiv preprint arXiv:1708.06733, 2017
Pith/arXiv arXiv 2017
-
[33]
Agent smith: A single image can jailbreak one million multimodal llm agents exponentially fast
Xiangming Gu, Xiaosen Zheng, Tianyu Pang, Chao Du, Qian Liu, Ye Wang, Jing Jiang, and Min Lin. Agent smith: A single image can jailbreak one million multimodal llm agents exponentially fast. InInternational Conference on Machine Learning, pages 16647–16672. PMLR, 2024. 11
2024
-
[34]
Intelligent agents with llm-based process automation
Yanchu Guan, Dong Wang, Zhixuan Chu, Shiyu Wang, Feiyue Ni, Ruihua Song, and Chenyi Zhuang. Intelligent agents with llm-based process automation. InProceedings of the 30th ACM SIGKDD conference on knowledge discovery and data mining, pages 5018–5027, 2024
2024
-
[35]
Qi Guo, Shanmin Pang, Xiaojun Jia, Yang Liu, and Qing Guo. Efficient generation of targeted and transferable adversarial examples for vision-language models via diffusion models.IEEE Transactions on Information Forensics and Security, 20:1333–1348, 2024
2024
-
[36]
Hyeonjeong Ha, Qiusi Zhan, Jeonghwan Kim, Dimitrios Bralios, Saikrishna Sanniboina, Nanyun Peng, Kai-Wei Chang, Daniel Kang, and Heng Ji. Mm-poisonrag: Disrupting multi- modal rag with local and global poisoning attacks.arXiv preprint arXiv:2502.17832, 2025
Pith/arXiv arXiv 2025
-
[37]
Google’s cloud vision api is not robust to noise.arXiv preprint arXiv:1704.05051, 2017
Hossein Hosseini, Baicen Xiao, and Radha Poovendran. Google’s cloud vision api is not robust to noise.arXiv preprint arXiv:1704.05051, 2017
Pith/arXiv arXiv 2017
-
[38]
Mem-pal: Towards memory-based personalized dialogue assistants for long- term user-agent interaction
Zhaopei Huang, Qifeng Dai, Guozheng Wu, Xiaopeng Wu, Xubin Li, Tiezheng Ge, Wenxuan Wang, and Qin Jin. Mem-pal: Towards memory-based personalized dialogue assistants for long- term user-agent interaction. InProceedings of the AAAI Conference on Artificial Intelligence, volume 40, pages 31229–31237, 2026
2026
-
[39]
Augustus: An llm-driven multimodal agent system with contextualized user memory
Jitesh Jain, Shubham Maheshwari, Ning Yu, Wen-mei Hwu, and Humphrey Shi. Augustus: An llm-driven multimodal agent system with contextualized user memory. InNeurIPS 2025 Workshop on Bridging Language, Agent, and World Models for Reasoning and Planning
2025
-
[40]
Enabling chatbots with eyes and ears: An immersive multimodal conversation system for dynamic interactions
Jihyoung Jang, Minwook Bae, Minji Kim, Dilek Hakkani-Tur, and Hyounghun Kim. Enabling chatbots with eyes and ears: An immersive multimodal conversation system for dynamic interactions. InProceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 31481–31512, 2025
2025
-
[41]
Adversarial attacks against closed-source mllms via feature optimal alignment
Xiaojun Jia, Sensen Gao, Simeng Qin, Tianyu Pang, Chao Du, Yihao Huang, Xinfeng Li, Yiming Li, Bo Li, and Yang Liu. Adversarial attacks against closed-source mllms via feature optimal alignment. InThe Thirty-ninth Annual Conference on Neural Information Processing Systems
-
[42]
Adv-watermark: A novel watermark perturbation for adversarial examples
Xiaojun Jia, Xingxing Wei, Xiaochun Cao, and Xiaoguang Han. Adv-watermark: A novel watermark perturbation for adversarial examples. InProceedings of the 28th ACM international conference on multimedia, pages 1579–1587, 2020
2020
-
[43]
Memory os of ai agent
Jiazheng Kang, Mingming Ji, Zhe Zhao, and Ting Bai. Memory os of ai agent. InProceedings of the 2025 Conference on Empirical Methods in Natural Language Processing, pages 25972– 25981, 2025
2025
-
[44]
Dense passage retrieval for open-domain question answering
Vladimir Karpukhin, Barlas Oguz, Sewon Min, Patrick Lewis, Ledell Wu, Sergey Edunov, Danqi Chen, and Wen-tau Yih. Dense passage retrieval for open-domain question answering. InProceedings of the 2020 conference on empirical methods in natural language processing (EMNLP), pages 6769–6781, 2020
2020
-
[45]
Sangyeop Kim, Yohan Lee, Sanghwa Kim, Hyunjong Kim, and Sungzoon Cho. Pre-storage reasoning for episodic memory: Shifting inference burden to memory for personalized dialogue. arXiv preprint arXiv:2509.10852, 2025
Pith/arXiv arXiv 2025
-
[46]
Retrieval-augmented generation for knowledge-intensive nlp tasks.Advances in neural information processing systems, 33:9459–9474, 2020
Patrick Lewis, Ethan Perez, Aleksandra Piktus, Fabio Petroni, Vladimir Karpukhin, Naman Goyal, Heinrich Küttler, Mike Lewis, Wen-tau Yih, Tim Rocktäschel, et al. Retrieval-augmented generation for knowledge-intensive nlp tasks.Advances in neural information processing systems, 33:9459–9474, 2020
2020
-
[47]
Hello again! llm-powered personalized agent for long-term dialogue
Hao Li, Chenghao Yang, An Zhang, Yang Deng, Xiang Wang, and Tat-Seng Chua. Hello again! llm-powered personalized agent for long-term dialogue. InProceedings of the 2025 Conference of the Nations of the Americas Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 1: Long Papers), pages 5259–5276, 2025
2025
-
[48]
Jiaqi Li, Mengmeng Wang, Zilong Zheng, and Muhan Zhang. Loogle: Can long-context language models understand long contexts? InProceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 16304–16333, 2024. 12
2024
-
[49]
Unsupervised corpus poisoning attacks in continuous space for dense retrieval
Yongkang Li, Panagiotis Eustratiadis, Simon Lupart, and Evangelos Kanoulas. Unsupervised corpus poisoning attacks in continuous space for dense retrieval. InProceedings of the 48th International ACM SIGIR Conference on Research and Development in Information Retrieval, pages 2452–2462, 2025
2025
-
[50]
A frustratingly simple yet highly effective attack baseline: Over 90% success rate against the strong black- box models of gpt-4.5/4o/o1
Zhaoyi Li, Xiaohan Zhao, Dong-Dong Wu, Jiacheng Cui, and Zhiqiang Shen. A frustratingly simple yet highly effective attack baseline: Over 90% success rate against the strong black- box models of gpt-4.5/4o/o1. InThe Thirty-ninth Annual Conference on Neural Information Processing Systems
-
[51]
Retrieval augmented generation or long-context llms? a comprehensive study and hybrid approach
Zhuowan Li, Cheng Li, Mingyang Zhang, Qiaozhu Mei, and Michael Bendersky. Retrieval augmented generation or long-context llms? a comprehensive study and hybrid approach. In Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing: Industry Track, pages 881–893, 2024
2024
-
[52]
Memverse: Multimodal memory for lifelong learning agents.arXiv preprint arXiv:2512.03627, 2025
Junming Liu, Yifei Sun, Weihua Cheng, Haodong Lei, Yirong Chen, Licheng Wen, Xuemeng Yang, Daocheng Fu, Pinlong Cai, Nianchen Deng, et al. Memverse: Multimodal memory for lifelong learning agents.arXiv preprint arXiv:2512.03627, 2025
Pith/arXiv arXiv 2025
-
[53]
Lost in the middle: How language models use long contexts.Transactions of the association for computational linguistics, 12:157–173, 2024
Nelson F Liu, Kevin Lin, John Hewitt, Ashwin Paranjape, Michele Bevilacqua, Fabio Petroni, and Percy Liang. Lost in the middle: How language models use long contexts.Transactions of the association for computational linguistics, 12:157–173, 2024
2024
-
[54]
Seeing, listening, remembering, and reasoning: A multimodal agent with long-term memory
Lin Long, Yichen He, Wentao Ye, Yiyuan Pan, Yuan Lin, Hang Li, Junbo Zhao, and Wei Li. Seeing, listening, remembering, and reasoning: A multimodal agent with long-term memory. arXiv preprint arXiv:2508.09736, 2025
arXiv 2025
-
[55]
Preemptive image robustification for protecting users against man-in-the-middle adversarial attacks
Seungyong Moon, Gaon An, and Hyun Oh Song. Preemptive image robustification for protecting users against man-in-the-middle adversarial attacks. InProceedings of the AAAI Conference on Artificial Intelligence, volume 36, pages 7823–7830, 2022
2022
-
[56]
Zach Nussbaum, John X Morris, Brandon Duderstadt, and Andriy Mulyar. Nomic embed: Training a reproducible long context text embedder.arXiv preprint arXiv:2402.01613, 2024
Pith/arXiv arXiv 2024
-
[57]
Memgpt: towards llms as operating systems
Charles Packer, Vivian Fang, Shishir_G Patil, Kevin Lin, Sarah Wooders, and Joseph_E Gonza- lez. Memgpt: towards llms as operating systems. 2023
2023
-
[58]
Generative agents: Interactive simulacra of human behavior
Joon Sung Park, Joseph O’Brien, Carrie Jun Cai, Meredith Ringel Morris, Percy Liang, and Michael S Bernstein. Generative agents: Interactive simulacra of human behavior. InProceed- ings of the 36th annual acm symposium on user interface software and technology, pages 1–22, 2023
2023
-
[59]
Jiachen Qian. Visual inception: Compromising long-term planning in agentic recommenders via multimodal memory poisoning.arXiv preprint arXiv:2604.16966, 2026
Pith/arXiv arXiv 2026
-
[60]
Learning transferable visual models from natural language supervision
Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, et al. Learning transferable visual models from natural language supervision. InInternational conference on machine learning, pages 8748–8763. PmLR, 2021
2021
-
[61]
Meminsight: Autonomous memory augmentation for llm agents
Rana Salama, Jason Cai, Michelle Yuan, Anna Currey, Monica Sunkara, Yi Zhang, and Yassine Benajiba. Meminsight: Autonomous memory augmentation for llm agents. InProceedings of the 2025 Conference on Empirical Methods in Natural Language Processing, pages 33124–33140, 2025
2025
-
[62]
Visual memory injection attacks for multi-turn conversations.arXiv preprint arXiv:2602.15927, 2026
Christian Schlarmann and Matthias Hein. Visual memory injection attacks for multi-turn conversations.arXiv preprint arXiv:2602.15927, 2026
arXiv 2026
-
[63]
Pmg: Personalized multimodal generation with large language models
Xiaoteng Shen, Rui Zhang, Xiaoyan Zhao, Jieming Zhu, and Xi Xiao. Pmg: Personalized multimodal generation with large language models. InProceedings of the ACM Web Conference 2024, pages 3833–3843, 2024. 13
2024
-
[64]
Ezzeldin Shereen, Dan Ristea, Shae McFadden, Burak Hasircioglu, Vasilios Mavroudis, and Chris Hicks. One pic is all it takes: Poisoning visual document retrieval augmented generation with a single image.arXiv preprint arXiv:2504.02132, 2025
arXiv 2025
-
[65]
Scaling up memory for robotic control via experience retrieval
Ajay Sridhar, Jennifer Pan, Satvik Sharma, and Chelsea Finn. Scaling up memory for robotic control via experience retrieval. InThe Fourteenth International Conference on Learning Representations
-
[66]
User- in-the-loop evaluation of multimodal llms for activity assistance
Mrinal Verghese, Brian Chen, Hamid Eghbalzadeh, Tushar Nagarajan, and Ruta P Desai. User- in-the-loop evaluation of multimodal llms for activity assistance. InProceedings of the Winter Conference on Applications of Computer Vision, pages 1144–1154, 2025
2025
-
[67]
Badagent: Inserting and activating backdoor attacks in llm agents
Yifei Wang, Dizhan Xue, Shengjie Zhang, and Shengsheng Qian. Badagent: Inserting and activating backdoor attacks in llm agents. InProceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 9811–9827, 2024
2024
-
[68]
Mirix: Multi-agent memory system for llm-based agents.arXiv preprint arXiv:2507.07957, 2025
Yu Wang and Xi Chen. Mirix: Multi-agent memory system for llm-based agents.arXiv preprint arXiv:2507.07957, 2025
Pith/arXiv arXiv 2025
-
[69]
Emergent abilities of large language models.arXiv preprint arXiv:2206.07682, 2022
Jason Wei, Yi Tay, Rishi Bommasani, Colin Raffel, Barret Zoph, Sebastian Borgeaud, Dani Yogatama, Maarten Bosma, Denny Zhou, Donald Metzler, et al. Emergent abilities of large language models.arXiv preprint arXiv:2206.07682, 2022
Pith/arXiv arXiv 2022
-
[70]
Tianxin Wei, Noveen Sachdeva, Benjamin Coleman, Zhankui He, Yuanchen Bei, Xuying Ning, Mengting Ai, Yunzhe Li, Jingrui He, Ed H Chi, et al. Evo-memory: Benchmarking llm agent test-time learning with self-evolving memory.arXiv preprint arXiv:2511.20857, 2025
Pith/arXiv arXiv 2025
-
[71]
Yaxiong Wu, Sheng Liang, Chen Zhang, Yichao Wang, Yongyue Zhang, Huifeng Guo, Ruiming Tang, and Yong Liu. From human memory to ai memory: A survey on memory mechanisms in the era of llms.arXiv preprint arXiv:2504.15965, 2025
Pith/arXiv arXiv 2025
-
[72]
Sedm: Scalable self-evolving distributed memory for agents
Haoran Xu, Jiacong Hu, ZHANG Ke, Lei Yu, Yuxin Tang, Xinyuan Song, Yiqun Duan, Lynn Ai, and TIANYU SHI. Sedm: Scalable self-evolving distributed memory for agents. InWorkshop on Scaling Environments for Agents
-
[73]
A-mem: Agentic memory for llm agents.arXiv preprint arXiv:2502.12110, 2025
Wujiang Xu, Zujie Liang, Kai Mei, Hang Gao, Juntao Tan, and Yongfeng Zhang. A-mem: Agentic memory for llm agents.arXiv preprint arXiv:2502.12110, 2025
Pith/arXiv arXiv 2025
-
[74]
Mmrc: A large-scale benchmark for understanding multimodal large language model in real-world conversation
Haochen Xue, Feilong Tang, Ming Hu, Yexin Liu, Qidong Huang, Yulong Li, Chengzhi Liu, Zhongxing Xu, Chong Zhang, Chun-Mei Feng, et al. Mmrc: A large-scale benchmark for understanding multimodal large language model in real-world conversation. InProceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers),...
2025
-
[75]
Sikuan Yan, Xiufeng Yang, Zuchao Huang, Ercong Nie, Zifeng Ding, Zonggen Li, Xiaowen Ma, Jinhe Bi, Kristian Kersting, Jeff Z Pan, et al. Memory-r1: Enhancing large language model agents to manage and utilize memories via reinforcement learning.arXiv preprint arXiv:2508.19828, 2025
Pith/arXiv arXiv 2025
-
[76]
Transferable and stealthy adversarial attacks on large vision-language models
Zhewen Yao, Yao Zhu, and Shiliang Zhang. Transferable and stealthy adversarial attacks on large vision-language models. InThe Fourteenth International Conference on Learning Representations
-
[77]
Hanrong Ye, Haotian Zhang, Erik Daxberger, Lin Chen, Zongyu Lin, Yanghao Li, Bowen Zhang, Haoxuan You, Dan Xu, Zhe Gan, et al. Mm-ego: Towards building egocentric multimodal llms for video qa.arXiv preprint arXiv:2410.07177, 2024
Pith/arXiv arXiv 2024
-
[78]
Univer- salrag: Retrieval-augmented generation over corpora of diverse modalities and granularities
Woongyeong Yeo, Kangsan Kim, Soyeong Jeong, Jinheon Baek, and Sung Ju Hwang. Univer- salrag: Retrieval-augmented generation over corpora of diverse modalities and granularities. arXiv preprint arXiv:2504.20734, 2025
Pith/arXiv arXiv 2025
-
[79]
Woongyeong Yeo, Kangsan Kim, Jaehong Yoon, and Sung Ju Hwang. Worldmm: Dynamic multimodal memory agent for long video reasoning.arXiv preprint arXiv:2512.02425, 2025. 14
arXiv 2025
-
[80]
Poisoned- eye: Knowledge poisoning attack on retrieval-augmented generation based large vision-language models
Chenyang Zhang, Xiaoyu Zhang, Jian Lou, Kai Wu, Zilong Wang, and Xiaofeng Chen. Poisoned- eye: Knowledge poisoning attack on retrieval-augmented generation based large vision-language models. InForty-second International Conference on Machine Learning, 2025
2025
discussion (0)
Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.