{"paper":{"title":"Prompt reinforcing for long-term planning of large language models","license":"http://creativecommons.org/licenses/by-sa/4.0/","headline":"A reinforcement-learning-inspired method rewrites task prompts using feedback and experience replay to improve long-term planning in LLM agents.","cross_cats":["cs.LG"],"primary_cat":"cs.CL","authors_text":"Benjamin Matthias Ruppik, Carel van Niekerk, Chia-Hao Shen, Hsien-Chin Lin, Michael Heck, Milica Ga\\v{s}i\\'c, Nurul Lubis, Renato Vukovic, Shutong Feng","submitted_at":"2025-10-07T13:30:18Z","abstract_excerpt":"Large language models (LLMs) have achieved remarkable success in a wide range of natural language processing tasks and can be adapted through prompting. However, they remain suboptimal in multi-turn interactions, often relying on incorrect early assumptions and failing to track user goals over time, which makes such tasks particularly challenging. Prior works in dialogue systems have shown that long-term planning is essential for handling interactive tasks. In this work, we propose a prompt optimisation framework inspired by reinforcement learning, which enables such planning to take place by "},"claims":{"count":4,"items":[{"kind":"strongest_claim","text":"By generating turn-by-turn feedback and leveraging experience replay for prompt rewriting, our proposed method shows significant improvement in multi-turn tasks such as text-to-SQL and task-oriented dialogue.","source":"verdict.strongest_claim","status":"machine_extracted","claim_id":"C1","attestation":"unclaimed"},{"kind":"weakest_assumption","text":"That rewriting only the task instruction prompt via feedback and experience replay is sufficient to produce effective long-term planning behavior inside an unmodified LLM-based agent.","source":"verdict.weakest_assumption","status":"machine_extracted","claim_id":"C2","attestation":"unclaimed"},{"kind":"one_line_summary","text":"A prompt optimization method using turn-by-turn feedback and experience replay improves LLM performance on multi-turn tasks such as text-to-SQL and task-oriented dialogue.","source":"verdict.one_line_summary","status":"machine_extracted","claim_id":"C3","attestation":"unclaimed"},{"kind":"headline","text":"A reinforcement-learning-inspired method rewrites task prompts using feedback and experience replay to improve long-term planning in LLM agents.","source":"verdict.pith_extraction.headline","status":"machine_extracted","claim_id":"C4","attestation":"unclaimed"}],"snapshot_sha256":"46675fa8749b8906dd69dff1099b0e233e9122e9a1ee30baf523bd7426ec30b2"},"source":{"id":"2510.05921","kind":"arxiv","version":3},"verdict":{"id":"bf57d9cd-7b2d-40ae-b899-ff27a3f0d1fe","model_set":{"reader":"grok-4.3"},"created_at":"2026-05-18T09:22:08.880249Z","strongest_claim":"By generating turn-by-turn feedback and leveraging experience replay for prompt rewriting, our proposed method shows significant improvement in multi-turn tasks such as text-to-SQL and task-oriented dialogue.","one_line_summary":"A prompt optimization method using turn-by-turn feedback and experience replay improves LLM performance on multi-turn tasks such as text-to-SQL and task-oriented dialogue.","pipeline_version":"pith-pipeline@v0.9.0","weakest_assumption":"That rewriting only the task instruction prompt via feedback and experience replay is sufficient to produce effective long-term planning behavior inside an unmodified LLM-based agent.","pith_extraction_headline":"A reinforcement-learning-inspired method rewrites task prompts using feedback and experience replay to improve long-term planning in LLM agents."},"integrity":{"clean":true,"summary":{"advisory":0,"critical":0,"by_detector":{},"informational":0},"endpoint":"/pith/2510.05921/integrity.json","findings":[],"available":true,"detectors_run":[],"snapshot_sha256":"c28c3603d3b5d939e8dc4c7e95fa8dfce3d595e45f758748cecf8e644a296938"},"references":{"count":0,"sample":[],"resolved_work":0,"snapshot_sha256":"258153158e38e3291e3d48162225fcdb2d5a3ed65a07baac614ab91432fd4f57","internal_anchors":0},"formal_canon":{"evidence_count":2,"snapshot_sha256":"11d16f7479d73a472ef6f468430269f3ac2ee03ac35671fa7d57abb966440ddb"},"author_claims":{"count":0,"strong_count":0,"snapshot_sha256":"258153158e38e3291e3d48162225fcdb2d5a3ed65a07baac614ab91432fd4f57"},"builder_version":"pith-number-builder-2026-05-17-v1"}