{"work":{"id":"71c901c4-3c83-4e10-af54-3daef7fff397","openalex_id":null,"doi":null,"arxiv_id":"2407.01489","raw_key":null,"title":"Agentless: Demystifying LLM-based Software Engineering Agents","authors":null,"authors_text":"Chunqiu Steven Xia, Yinlin Deng, Soren Dunn, Lingming Zhang","year":2024,"venue":"cs.SE","abstract":"Recent advancements in large language models (LLMs) have significantly advanced the automation of software development tasks, including code synthesis, program repair, and test generation. More recently, researchers and industry practitioners have developed various autonomous LLM agents to perform end-to-end software development tasks. These agents are equipped with the ability to use tools, run commands, observe feedback from the environment, and plan for future actions. However, the complexity of these agent-based approaches, together with the limited abilities of current LLMs, raises the following question: Do we really have to employ complex autonomous software agents? To attempt to answer this question, we build Agentless -- an agentless approach to automatically solve software development problems. Compared to the verbose and complex setup of agent-based approaches, Agentless employs a simplistic three-phase process of localization, repair, and patch validation, without letting the LLM decide future actions or operate with complex tools. Our results on the popular SWE-bench Lite benchmark show that surprisingly the simplistic Agentless is able to achieve both the highest performance (32.00%, 96 correct fixes) and low cost ($0.70) compared with all existing open-source software agents! Furthermore, we manually classified the problems in SWE-bench Lite and found problems with exact ground truth patch or insufficient/misleading issue descriptions. As such, we construct SWE-bench Lite-S by excluding such problematic issues to perform more rigorous evaluation and comparison. Our work highlights the current overlooked potential of a simple, interpretable technique in autonomous software development. We hope Agentless will help reset the baseline, starting point, and horizon for autonomous software agents, and inspire future work along this crucial direction.","external_url":"https://arxiv.org/abs/2407.01489","cited_by_count":null,"metadata_source":"pith","metadata_fetched_at":"2026-05-23T00:52:19.367662+00:00","pith_arxiv_id":"2407.01489","created_at":"2026-05-10T00:19:46.447144+00:00","updated_at":"2026-05-23T00:52:19.367662+00:00","title_quality_ok":true,"display_title":"Agentless: Demystifying LLM-based Software Engineering Agents","render_title":"Agentless: Demystifying LLM-based Software Engineering Agents"},"hub":{"state":{"work_id":"71c901c4-3c83-4e10-af54-3daef7fff397","tier":"hub","tier_reason":"10+ Pith inbound or 1,000+ external citations","pith_inbound_count":58,"external_cited_by_count":null,"distinct_field_count":9,"first_pith_cited_at":"2024-11-07T06:36:19+00:00","last_pith_cited_at":"2026-05-21T04:54:55+00:00","author_build_status":"not_needed","summary_status":"needed","contexts_status":"needed","graph_status":"needed","ask_index_status":"not_needed","reader_status":"not_needed","recognition_status":"not_needed","updated_at":"2026-05-28T04:38:08.568640+00:00","tier_text":"hub"},"tier":"hub","role_counts":[{"context_role":"background","n":9},{"context_role":"baseline","n":2},{"context_role":"method","n":1}],"polarity_counts":[{"context_polarity":"background","n":9},{"context_polarity":"baseline","n":2},{"context_polarity":"use_method","n":1}],"runs":{"context_extract":{"job_type":"context_extract","status":"succeeded","result":{"enqueued_papers":25},"error":null,"updated_at":"2026-05-14T18:20:26.917329+00:00"},"graph_features":{"job_type":"graph_features","status":"succeeded","result":{"co_cited":[{"title":"Evaluating Large Language Models Trained on Code","work_id":"042493e9-b26f-4b4e-bbde-382072ca9b08","shared_citers":14},{"title":"SWE-bench: Can Language Models Resolve Real-World GitHub Issues?","work_id":"d0effe15-a689-441a-8e3f-ea35f1c4e4b1","shared_citers":10},{"title":"OpenHands: An Open Platform for AI Software Developers as Generalist Agents","work_id":"f1762ea0-e382-4f38-a28c-adc643789859","shared_citers":9},{"title":"Program Synthesis with Large Language Models","work_id":"fd241a05-03b9-4de2-9588-9d77ce176125","shared_citers":9},{"title":"Code Llama: Open Foundation Models for Code","work_id":"e73bffa4-7620-47ac-9327-259a60db52ca","shared_citers":8},{"title":"AutoGen: Enabling Next-Gen LLM Applications via Multi-Agent Conversation","work_id":"92b7eb9c-c3d8-4518-a376-06fa15dd895b","shared_citers":7},{"title":"SWE-agent: Agent-Computer Interfaces Enable Automated Software Engineering","work_id":"01826cd9-a652-403c-a2ec-531da9fe2b6a","shared_citers":7},{"title":"Jimenez, Alexander Wettig, Kabir Khandpur, Yanzhe Zhang, Binyuan Hui, Ofir Press, Ludwig Schmidt, and Diyi Yang","work_id":"6a906763-2e4e-4cea-a19c-d7a169c9376b","shared_citers":5},{"title":"arXiv preprint arXiv:2404.05427 (2024)","work_id":"fa93e782-fee4-4c22-9486-d9d5d62f7b93","shared_citers":4},{"title":"DeepSeek-Coder: When the Large Language Model Meets Programming -- The Rise of Code Intelligence","work_id":"f22dae5a-27e2-41d0-a061-c4286418dee3","shared_citers":4},{"title":"MemGPT: Towards LLMs as Operating Systems","work_id":"2698f5ad-c84c-40ca-b839-0912dae10ba2","shared_citers":4},{"title":"MetaGPT: Meta Programming for A Multi-Agent Collaborative Framework","work_id":"891b9780-a800-4e3c-bba0-53597ab8dc98","shared_citers":4},{"title":"Qwen2.5-Coder Technical Report","work_id":"09ba463d-6377-4017-9801-444ffb94b056","shared_citers":4},{"title":"ReAct: Synergizing Reasoning and Acting in Language Models","work_id":"407a2351-25f1-497d-b611-f77d0292a8e6","shared_citers":4},{"title":"SWE-Bench Pro: Can AI Agents Solve Long-Horizon Software Engineering Tasks?","work_id":"a561c78a-4b02-4053-a92a-bc5c7c5f6b9b","shared_citers":4},{"title":"arXiv preprint arXiv:2410.20285 , year=","work_id":"e26e6ec9-37e3-47d8-b494-e0a227e77e36","shared_citers":3},{"title":"ChatDev: Communicative Agents for Software Development","work_id":"5d8a3650-ab78-4991-b0d3-5309b59c690f","shared_citers":3},{"title":"DeepSeek-V3 Technical Report","work_id":"57d2791d-2219-4c31-a077-afc04b12a75c","shared_citers":3},{"title":"Islem Bouzenia, Premkumar Devanbu, and Michael Pradel","work_id":"eba62f4c-8d54-47b2-bb17-30063b120462","shared_citers":3},{"title":"Large language model-based agents for software engineering: A survey","work_id":"65bf4f26-a45c-4639-841f-9f865ef1030e","shared_citers":3},{"title":"Lingma SWE-GPT : An open development-process-centric language model for automated software improvement","work_id":"466826e3-e7ad-48fe-b71a-3d746eb77231","shared_citers":3},{"title":"Proximal Policy Optimization Algorithms","work_id":"240c67fe-d14d-4520-91c1-38a4e272ca19","shared_citers":3},{"title":"Reflexion: Language Agents with Verbal Reinforcement Learning","work_id":"778f739e-5f55-4961-8a2a-e4736a2757f4","shared_citers":3},{"title":"Self-Refine: Iterative Refinement with Self-Feedback","work_id":"59181e7f-e58e-45d3-8146-4477a9f53d5a","shared_citers":3}],"time_series":[{"n":3,"year":2025},{"n":31,"year":2026}],"dependency_candidates":[]},"error":null,"updated_at":"2026-05-14T18:19:58.193613+00:00"},"identity_refresh":{"job_type":"identity_refresh","status":"succeeded","result":{"items":[{"title":"Qwen3 Technical Report","outcome":"unchanged","work_id":"25a4e30c-1232-48e7-9925-02fa12ba7c9e","resolver":"local_arxiv","confidence":0.98,"old_work_id":"25a4e30c-1232-48e7-9925-02fa12ba7c9e"}],"counts":{"fixed":0,"merged":0,"unchanged":1,"quarantined":0,"needs_external_resolution":0},"errors":[],"attempted":1},"error":null,"updated_at":"2026-05-14T18:19:25.043503+00:00"},"summary_claims":{"job_type":"summary_claims","status":"succeeded","result":{"title":"Agentless: Demystifying LLM-based Software Engineering Agents","claims":[{"claim_text":"Recent advancements in large language models (LLMs) have significantly advanced the automation of software development tasks, including code synthesis, program repair, and test generation. More recently, researchers and industry practitioners have developed various autonomous LLM agents to perform end-to-end software development tasks. These agents are equipped with the ability to use tools, run commands, observe feedback from the environment, and plan for future actions. However, the complexity of these agent-based approaches, together with the limited abilities of current LLMs, raises the fo","claim_type":"abstract","evidence_strength":"source_metadata"}],"why_cited":"Pith tracks Agentless: Demystifying LLM-based Software Engineering Agents because it crossed a citation-hub threshold.","role_counts":[]},"error":null,"updated_at":"2026-05-14T18:19:29.716443+00:00"}},"summary":{"title":"Agentless: Demystifying LLM-based Software Engineering Agents","claims":[{"claim_text":"Recent advancements in large language models (LLMs) have significantly advanced the automation of software development tasks, including code synthesis, program repair, and test generation. More recently, researchers and industry practitioners have developed various autonomous LLM agents to perform end-to-end software development tasks. These agents are equipped with the ability to use tools, run commands, observe feedback from the environment, and plan for future actions. However, the complexity of these agent-based approaches, together with the limited abilities of current LLMs, raises the fo","claim_type":"abstract","evidence_strength":"source_metadata"}],"why_cited":"Pith tracks Agentless: Demystifying LLM-based Software Engineering Agents because it crossed a citation-hub threshold.","role_counts":[]},"graph":{"co_cited":[{"title":"Evaluating Large Language Models Trained on Code","work_id":"042493e9-b26f-4b4e-bbde-382072ca9b08","shared_citers":14},{"title":"SWE-bench: Can Language Models Resolve Real-World GitHub Issues?","work_id":"d0effe15-a689-441a-8e3f-ea35f1c4e4b1","shared_citers":10},{"title":"OpenHands: An Open Platform for AI Software Developers as Generalist Agents","work_id":"f1762ea0-e382-4f38-a28c-adc643789859","shared_citers":9},{"title":"Program Synthesis with Large Language Models","work_id":"fd241a05-03b9-4de2-9588-9d77ce176125","shared_citers":9},{"title":"Code Llama: Open Foundation Models for Code","work_id":"e73bffa4-7620-47ac-9327-259a60db52ca","shared_citers":8},{"title":"AutoGen: Enabling Next-Gen LLM Applications via Multi-Agent Conversation","work_id":"92b7eb9c-c3d8-4518-a376-06fa15dd895b","shared_citers":7},{"title":"SWE-agent: Agent-Computer Interfaces Enable Automated Software Engineering","work_id":"01826cd9-a652-403c-a2ec-531da9fe2b6a","shared_citers":7},{"title":"Jimenez, Alexander Wettig, Kabir Khandpur, Yanzhe Zhang, Binyuan Hui, Ofir Press, Ludwig Schmidt, and Diyi Yang","work_id":"6a906763-2e4e-4cea-a19c-d7a169c9376b","shared_citers":5},{"title":"arXiv preprint arXiv:2404.05427 (2024)","work_id":"fa93e782-fee4-4c22-9486-d9d5d62f7b93","shared_citers":4},{"title":"DeepSeek-Coder: When the Large Language Model Meets Programming -- The Rise of Code Intelligence","work_id":"f22dae5a-27e2-41d0-a061-c4286418dee3","shared_citers":4},{"title":"MemGPT: Towards LLMs as Operating Systems","work_id":"2698f5ad-c84c-40ca-b839-0912dae10ba2","shared_citers":4},{"title":"MetaGPT: Meta Programming for A Multi-Agent Collaborative Framework","work_id":"891b9780-a800-4e3c-bba0-53597ab8dc98","shared_citers":4},{"title":"Qwen2.5-Coder Technical Report","work_id":"09ba463d-6377-4017-9801-444ffb94b056","shared_citers":4},{"title":"ReAct: Synergizing Reasoning and Acting in Language Models","work_id":"407a2351-25f1-497d-b611-f77d0292a8e6","shared_citers":4},{"title":"SWE-Bench Pro: Can AI Agents Solve Long-Horizon Software Engineering Tasks?","work_id":"a561c78a-4b02-4053-a92a-bc5c7c5f6b9b","shared_citers":4},{"title":"arXiv preprint arXiv:2410.20285 , year=","work_id":"e26e6ec9-37e3-47d8-b494-e0a227e77e36","shared_citers":3},{"title":"ChatDev: Communicative Agents for Software Development","work_id":"5d8a3650-ab78-4991-b0d3-5309b59c690f","shared_citers":3},{"title":"DeepSeek-V3 Technical Report","work_id":"57d2791d-2219-4c31-a077-afc04b12a75c","shared_citers":3},{"title":"Islem Bouzenia, Premkumar Devanbu, and Michael Pradel","work_id":"eba62f4c-8d54-47b2-bb17-30063b120462","shared_citers":3},{"title":"Large language model-based agents for software engineering: A survey","work_id":"65bf4f26-a45c-4639-841f-9f865ef1030e","shared_citers":3},{"title":"Lingma SWE-GPT : An open development-process-centric language model for automated software improvement","work_id":"466826e3-e7ad-48fe-b71a-3d746eb77231","shared_citers":3},{"title":"Proximal Policy Optimization Algorithms","work_id":"240c67fe-d14d-4520-91c1-38a4e272ca19","shared_citers":3},{"title":"Reflexion: Language Agents with Verbal Reinforcement Learning","work_id":"778f739e-5f55-4961-8a2a-e4736a2757f4","shared_citers":3},{"title":"Self-Refine: Iterative Refinement with Self-Feedback","work_id":"59181e7f-e58e-45d3-8146-4477a9f53d5a","shared_citers":3}],"time_series":[{"n":3,"year":2025},{"n":31,"year":2026}],"dependency_candidates":[]},"authors":[]}}