{"paper":{"title":"Optimizing PyTorch Inference with LLM-Based Multi-Agent Systems","license":"http://arxiv.org/licenses/nonexclusive-distrib/1.0/","headline":"Multi-agent LLM systems optimize PyTorch code for 2.88x faster inference than eager execution on H100 GPUs.","cross_cats":["cs.AI","cs.DC"],"primary_cat":"cs.MA","authors_text":"Costin Iancu, Kirill Nagaitsev, Luka Grbcic, Samuel Williams","submitted_at":"2025-11-21T05:37:38Z","abstract_excerpt":"Maximizing performance on available GPU hardware is an ongoing challenge for modern AI inference systems. Traditional approaches include writing custom GPU kernels and using specialized model compilers to tune high-level code for specific GPU targets. Recent work shows that LLM-based multi-agent systems can effectively perform such tuning, often outperforming existing compilers and eliminating the need for manual kernel development. However, the dynamics of multi-agent systems for this task remain unexplored. In this work, we present a logical framework for comparing multi-agent PyTorch optimi"},"claims":{"count":4,"items":[{"kind":"strongest_claim","text":"The best implementation achieves an average 2.88x speedup over PyTorch Eager (1.85x over torch.compile) on an H100 GPU across diverse tasks in KernelBench.","source":"verdict.strongest_claim","status":"machine_extracted","claim_id":"C1","attestation":"unclaimed"},{"kind":"weakest_assumption","text":"That LLM-based multi-agent systems can reliably generate correct, bug-free optimized code at scale and that results on the KernelBench suite generalize to production inference workloads.","source":"verdict.weakest_assumption","status":"machine_extracted","claim_id":"C2","attestation":"unclaimed"},{"kind":"one_line_summary","text":"An exploit-heavy multi-agent LLM system with error-fixing agents delivers 2.88x average speedup over PyTorch Eager and 1.85x over torch.compile on H100 GPUs across KernelBench tasks.","source":"verdict.one_line_summary","status":"machine_extracted","claim_id":"C3","attestation":"unclaimed"},{"kind":"headline","text":"Multi-agent LLM systems optimize PyTorch code for 2.88x faster inference than eager execution on H100 GPUs.","source":"verdict.pith_extraction.headline","status":"machine_extracted","claim_id":"C4","attestation":"unclaimed"}],"snapshot_sha256":"94eb08b7e71295b5620a6816d68a435978e6ab7a766c505fb1a033f4959806c5"},"source":{"id":"2511.16964","kind":"arxiv","version":2},"verdict":{"id":"ad3137c4-1107-4219-9ed4-ab9604e54428","model_set":{"reader":"grok-4.3"},"created_at":"2026-05-17T20:59:11.566462Z","strongest_claim":"The best implementation achieves an average 2.88x speedup over PyTorch Eager (1.85x over torch.compile) on an H100 GPU across diverse tasks in KernelBench.","one_line_summary":"An exploit-heavy multi-agent LLM system with error-fixing agents delivers 2.88x average speedup over PyTorch Eager and 1.85x over torch.compile on H100 GPUs across KernelBench tasks.","pipeline_version":"pith-pipeline@v0.9.0","weakest_assumption":"That LLM-based multi-agent systems can reliably generate correct, bug-free optimized code at scale and that results on the KernelBench suite generalize to production inference workloads.","pith_extraction_headline":"Multi-agent LLM systems optimize PyTorch code for 2.88x faster inference than eager execution on H100 GPUs."},"references":{"count":5,"sample":[{"doi":"10.1145/3703412.3703416","year":2025,"title":"Accessed: 2025-10-17","work_id":"a6173c86-049f-4167-a4fb-a0022b7502c3","ref_index":1,"cited_arxiv_id":"","is_internal_anchor":false},{"doi":"","year":2025,"title":"AlphaEvolve: A coding agent for scientific and algorithmic discovery","work_id":"76a0f850-d490-4e4f-ab98-8d25df82cd23","ref_index":2,"cited_arxiv_id":"2506.13131","is_internal_anchor":true},{"doi":"","year":2019,"title":"Anjiang Wei, Tianran Sun, Yogesh Seenichamy, Hang Song, Anne Ouyang, Azalia Mirhoseini, Ke Wang, and Alex Aiken","work_id":"d2dbaa77-84ac-4089-831b-fbbc4af827dc","ref_index":3,"cited_arxiv_id":"","is_internal_anchor":false},{"doi":"","year":2025,"title":"Zhang, Z., Bajaj, A","work_id":"03fe7bc1-abe5-47bf-affe-5b84aa98fa39","ref_index":4,"cited_arxiv_id":"","is_internal_anchor":false},{"doi":"","year":2025,"title":"ISBN 9798331314385","work_id":"3e41f374-6f38-4cec-baca-12faade549ba","ref_index":5,"cited_arxiv_id":"","is_internal_anchor":false}],"resolved_work":5,"snapshot_sha256":"d4a47fa2f75d1d75d924a93446a58b856d64d12d86e566bcb3b910a69dceba2a","internal_anchors":1},"formal_canon":{"evidence_count":1,"snapshot_sha256":"31063d0985ae57706c90f375287da9de005a80bd14f3b474a89f2e8760c957c1"},"author_claims":{"count":0,"strong_count":0,"snapshot_sha256":"258153158e38e3291e3d48162225fcdb2d5a3ed65a07baac614ab91432fd4f57"},"builder_version":"pith-number-builder-2026-05-17-v1"}