{"id":"de7c46b8-b2b3-451c-b04b-0ca248018f0b","arxiv_id":"2504.21769","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"high","formal_verification":"none","parameter_count":2,"one_line_summary":"A language model can act as an interactive imitation learning teacher by generating a code policy and giving corrective or evaluative feedback based on angular similarity to the agent's actions.","lead":"This paper introduces LLM-iTeach, an interactive imitation learning method in which a large language model writes a Python policy from a task description and then automatically supplies corrective and evaluative feedback to a robot-learning agent, replacing the human teacher. The method outperforms behavior cloning and roughly matches a human-teacher method on simulated manipulation tasks, though the human-teacher comparison is not fully controlled.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The headline comparison to CEILing is uncontrolled: CEILing's 91±8% is borrowed at 100 human-feedback episodes while LLM-iTeach's 93±3% uses 400; at matched 100 episodes LLM-iTeach is 85±5% and tied with BC.","rationale":"The reader's weakest_assumption highlights that Eq. (4) labels 'good' as agreement with the CodePolicy rather than with task success. This is a real limitation and is partially self-acknowledged in Section V-B, but it is not the most load-bearing issue for the central claim: an imperfect teacher can still teach, and the ablations show the interactive loop adds value beyond the warm-start. The comparison to CEILing is more directly load-bearing because the abstract's central sentence is a comparative claim about human-level teaching. Table I compares LLM-iTeach at 400 episodes with CEILing at 100 episodes, and the CEILing numbers are not rerun in this paper's own simulator version. At matched 100 episodes, LLM-iTeach's average equals BC and is numerically below CEILing. Thus the current evidence cannot distinguish 'the LLM teacher is as good as a human teacher' from 'more training episodes compensate for a weaker teacher.' A secondary concern is that the beta threshold is selected by peaking at the evaluation success rate in Section V-D, which can inflate the reported number; fixing the comparison is the priority. The verdict remains conditional because the method is plausible and the repository is provided, but the claim needs a matched comparison and softer wording.","tokens_in":12384,"tokens_out":6079,"duration_ms":59965,"concrete_test":"Run CEILing under the identical RLBench/CoppeliaSim version, warm-start procedure, random seeds, and 100-episode evaluation protocol used for LLM-iTeach, and also train LLM-iTeach for exactly 100 episodes; report mean success with confidence intervals and a paired or bootstrap test. If at 100 episodes CEILing is significantly above LLM-iTeach, the equality claim in the abstract fails, and the paper should claim only that LLM-generated feedback is a cost-effective substitute at higher episode counts.","verdict_should_be":"UNCHANGED","load_bearing_attack":"Table I's headline comparison to CEILing is not controlled. CEILing's 91±8% average is imported from the original paper and uses 100 human-feedback episodes, whereas LLM-iTeach's best average (93±3%) uses 400 episodes. At the matched 100-episode count, LLM-iTeach averages 85±5%, below CEILing's 91±8% and tied with BC at 85±5%. The central claim that LLMs teach as effectively as human teachers therefore rests on comparing different training budgets rather than on teacher quality. The fact that LLM-iTeach improves when given four times as many episodes supports scalability, not equivalence: a cheaper teacher with more data may compensate for lower per-episode teaching quality, but that is a different claim. The same table also shows 'surpasses BC' only at 400 episodes; at 100 episodes the two methods are tied. A matched comparison, or at least a statistical test on the existing 100-episode rows, is needed before the equality claim can stand.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes LLM-iTeach, an interactive imitation learning framework in which a large language model (Llama3-70b) is prompted hierarchically to generate a Python CodePolicy, which then provides corrective and evaluative feedback to train a Gaussian stochastic policy for robotic manipulation. The agent is warm-started with ten LLM-generated demonstrations and trained on RLBench tasks in simulation. The authors compare LLM-iTeach against behavior cloning (BC) and against CEILing, a human-teacher IIL method, on four shared tasks, and additionally evaluate LLM-iTeach on four further tasks. The central claim is that LLM-iTeach surpasses BC and achieves or outscores CEILing, indicating that LLMs can serve as cost-effective, human-like interactive teachers.","tokens_in":12613,"tokens_out":3944,"duration_ms":42948,"significance":"If the central claim is supported, the paper would make a useful contribution: it demonstrates a concrete pipeline for replacing a human teacher with an LLM in interactive imitation learning, and it provides reproducible code and prompts. The hierarchical prompting design and the ablation studies are informative, and the authors are transparent about the limitations of the LLM teacher. However, the headline comparison to CEILing is not controlled at equal training budgets, and a key hyperparameter is selected from the evaluation curve; both issues must be resolved before the claim of human-equivalent teaching is justified.","major_comments":[{"comment":"The headline comparison to CEILing is not controlled: CEILing's 91±8% average is imported from its original publication and uses 100 human-feedback episodes, while LLM-iTeach's best average of 93±3% uses 400 episodes. At the matched 100-episode budget, LLM-iTeach averages 85±5%, identical to BC and below CEILing's 91±8%. The abstract's claim of 'achieves or even outscores CEILing' therefore conflates a larger training budget with higher teacher quality; the data support scalability, not equivalence. Please provide a matched comparison at equal episode counts (ideally with CEILing rerun under identical conditions), or at least a statistical test on the existing 100-episode rows, and revise the claims accordingly.","section":"Section V-B, Table I"},{"comment":"The threshold β=20 was selected because it produced the highest evaluation success rate on the same four tasks used for the main results, with no held-out validation or multiple-testing correction. The reported 93±3% figure is therefore an optimistic selection on the test data, and reporting it as the method's expected performance is not justified. Please use a validation split or pre-register β, and report performance across β values with confidence intervals so readers can assess sensitivity.","section":"Section V-D, Fig. 7"},{"comment":"Evaluative feedback is defined as angular agreement with the CodePolicy action, not as progress toward task success. As the paper itself states for CloseMicrowave and for inverse-kinematics failures in Section V-B, the CodePolicy can be wrong about physical constraints, so the teacher can label actions as 'good' that do not lead to task success. Because the final success rates are measured independently by RLBench, this is not a purely circular evaluation, but the training signal is only as good as the CodePolicy. The paper should quantify the correlation between Eq. (4) agreement and task success, or at least report the rate of incorrect positive feedback, and discuss how this affects the claim that the LLM teaches as effectively as a human.","section":"Section IV-A(b), Eq. (4)"}],"minor_comments":[{"comment":"The MDP tuple is written as <S,A,T,R,γ> but the text says 'with four components'; this is a five-tuple and should be corrected.","section":"Section II(a)"},{"comment":"The notation N(fθ(s,θ); σ^2) uses θ both as the policy parameters and inside the function f, which is confusing and likely a typo; please clarify the intended mean function.","section":"Section IV-B, Eq. (6)"},{"comment":"The statement that each experiment is repeated at least 20 times per task is vague about whether the reported standard deviations are across these repeats or across tasks; please specify the exact aggregation procedure.","section":"Section V-A"},{"comment":"CEILing correction rates are reported without standard deviations, and no significance tests accompany the statements about 'similar ratio' or 'comparable success rates'; adding such information would strengthen the comparison.","section":"Table I"},{"comment":"The text contains a typo: 'trough scaling' should be 'through scaling'.","section":"Section V-B"},{"comment":"The sentence beginning 'Oriented to that LLM-iTeach uses...' is ungrammatical and should be rephrased.","section":"Section IV-B"}],"recommendation":"major_revision","confidential_remarks":"The paper is within scope and the idea is timely, but the uncontrolled CEILing comparison and the test-set selection of β are load-bearing issues that need to be fixed before publication. I see no integrity concerns; the authors are transparent about their limitations, and the code release is a strength."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Quick take: this is a sensible, honestly written paper that builds an LLM-only interactive teacher by combining Code-as-Policies-style hierarchical prompting with a similarity rule for corrective/evaluative feedback. The code and prompts are public. I believe the authors when they say no prior IIL method removes the human entirely. That is the real novelty, and the ablations (feedback types, warm-start, beta sweep) plus four additional tasks give a reasonably complete picture. Credit where due: they also state their own limitations clearly — restricted observation, reliance on ground truth, inverse kinematics failures — rather than hiding them.\n\nThe soft spot is the headline comparison. Table I borrows CEILing's numbers from the original paper at 100 human-feedback episodes, while LLM-iTeach's best average (93±3) uses 400 episodes. At the matched 100-episode row, LLM-iTeach is 85±5, tied with BC and below CEILing. So \"achieves or outscores CEILing\" is true only when the LLM teacher gets four times the data. That is a real result — cheap teacher scales — but it is not evidence of teacher equivalence. The paper's own scaling curve shows BC also plateaus; the comparison should be re-run with CEILing under identical evaluation code and episode counts, or the claim should be softened to \"comparable at higher data budgets.\"\n\nSecond issue: the evaluative feedback in Eq. (4) labels an action \"good\" when its angle to the LLM's CodePolicy action is below beta. That is self-consistency, not task success. Final success rates are measured by RLBench, so the method isn't circular end-to-end, but the training signal can reinforce wrong behavior when the CodePolicy is wrong, as they admit for CloseMicrowave. Also, beta is chosen from the evaluation curve in Fig. 7, which is mild test-set tuning; a validation split would be cleaner.\n\nNone of this kills the contribution. The integration is clean, the writing is clear, and the artifacts are there. I'd send it to review, with the expectation of major revision: controlled CEILing comparison, validation-based beta selection, and a more measured abstract.","headline":"Useful integration of LLM code policies with interactive feedback, but the headline claim of matching human teachers rests on an uncontrolled episode-budget comparison and a self-referential success signal.","tokens_in":13132,"tokens_out":2241,"would_cite":true,"duration_ms":23047,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"An LLM can replace the human teacher in interactive imitation learning for robot manipulation, matching state-of-the-art results without human labor.","keywords":["Large Language Models","Interactive Imitation Learning","Robotic Manipulation","Hierarchical Prompting","CodePolicy","Similarity-based Feedback","RLBench"],"falsifier":"Construct a manipulation task where the LLM's CodePolicy is known to be wrong (e.g., it always aims for the microwave handle while any push on the door suffices, as the paper itself describes), train LLM-iTeach to completion, and check whether the agent's success rate stays at or below the LLM Teacher's. If the agent fails to exceed the teacher on this task and reproduces the handle-aiming behavior, the claim that evaluative feedback yields exploration gains beyond the teacher is falsified; the same holds if replacing the LLM's feedback with an oracle that labels actions by true task progress changes the outcome.","tokens_in":12120,"feed_emoji":"🤖","tokens_out":3563,"duration_ms":34121,"temperature":0.7,"pith_summary":"The paper tries to establish that a large language model can play the role of the interactive human teacher in imitation learning for robotic manipulation, providing both corrective and evaluative feedback during training without human labor. It introduces LLM-iTeach, which first asks the LLM to write a Python CodePolicy from a task description, then compares the agent's actions with the CodePolicy's actions; small angular deviation yields positive evaluative feedback, otherwise the CodePolicy's action is given as correction. In four RLBench manipulation tasks, LLM-iTeach reaches 93±3 percent average success at 400 episodes, matching CEILing's 91±8 percent with a human teacher and beating behavior cloning; it also transfers to four additional tasks with a simple description. The claim matters because it suggests the costly human-in-the-loop component of interactive imitation learning can be replaced by a readily available LLM.","feed_headline":"LLM teacher matches human instructor in robot training","feed_subtitle":"In four manipulation tasks, LLM-iTeach hits 93% success without a human in the loop.","key_machinery":"The central mechanism is the CodePolicy: a Python program generated by hierarchical prompting (two levels, where a planner prompt breaks the task into steps and action and check functions implement each step), executed at 20 Hz to produce the LLM's preferred action from API-provided object positions and robot state. The similarity-based feedback rule (Eq. 4) converts this preferred action into either evaluative or corrective feedback; the weighted negative log-likelihood loss (Eqs. 7–8) trains the agent, and episodes exceeding a time limit are aborted with their state-action pairs excluded from learning. This machinery carries the argument by defining the teacher signal entirely from the LLM code without human involvement.","core_discovery":"The central claim is that an LLM, prompted hierarchically to emit an executable policy, can serve as the teacher in interactive imitation learning and teach a stochastic policy agent to solve robotic manipulation tasks as effectively as a human teacher (CEILing baseline) and better than behavior cloning. The teaching signal is derived from a similarity check: if the angle between the agent's action and the CodePolicy's action is below threshold $\\beta$, evaluative 'good' feedback is given; otherwise the CodePolicy's action replaces the agent's action. With warm-start demonstrations (10 from the LLM teacher) and a weighting scheme that upweights corrected state-action pairs, LLM-iTeach achieves 93±3 percent success on four tasks at 400 episodes, exceeding CEILing's reported 91±8 percent at 100 episodes; ablations show both feedback types are needed and $\\beta = 20^\\circ$ is best. The paper also shows that the LLM teacher lacks physical and inverse-kinematics knowledge, causing failures in tasks like CloseMicrowave, yet the learned agent still outperforms the teacher through evaluative-feedback-driven exploration.","pith_inferences":["If LLM teaching works as described, the true constraint becomes the LLM's physical world model; pairing the teacher with a vision-language model or simulator-derived physical priors could close the gap on tasks like CloseMicrowave (this extends the paper's own limitation note).","The similarity threshold $\\beta$ acts as a trust knob between agent and teacher; one could anneal $\\beta$ over training to gradually reduce corrections, a strategy the paper does not explore.","A testable prediction follows: on tasks where the CodePolicy is systematically wrong (e.g., wrong grasp orientation), LLM-iTeach should plateau at the warm-start level or learn the wrong behavior, since positive feedback only signals agreement with the LLM, not task success."],"forward_implications":["Interactive imitation learning can be run at scale without human supervision, limited only by compute and API cost.","New manipulation tasks can be added by writing a one-sentence task description and re-prompting; the method transfers to long-horizon tasks with little engineering.","The agent can surpass the LLM teacher's direct success because evaluative feedback induces exploration, suggesting distillation-plus-exploration beats pure code execution.","The bottleneck shifts from human teaching effort to the quality of the LLM's CodePolicy and the completeness of the state information provided to it."],"supporting_citations":[{"why":"Supplies the CEILing baseline, the state-of-the-art interactive imitation learning method with a human teacher, whose experimental setup and results LLM-iTeach is compared against.","marker":"[8]"},{"why":"Supplies the hierarchical prompting strategy ('Code as Policies') that LLM-iTeach adapts to generate the CodePolicy.","marker":"[21]"},{"why":"Supplies the RLBench simulation environment and tasks used for all experiments.","marker":"[34]"},{"why":"Supplies the weighting scheme for corrected state-action pairs that LLM-iTeach adopts in its loss function.","marker":"[35]"},{"why":"Defines imitation learning and behavior cloning, the primary baseline and the problem setting being extended.","marker":"[3]"},{"why":"Provides the no-regret reduction of imitation learning that motivates the interactive feedback loop by addressing distribution shift.","marker":"[6]"},{"why":"Supplies the survey framing of interactive imitation learning and the taxonomy of corrective versus evaluative feedback.","marker":"[7]"},{"why":"Identifies the specific LLM (Llama 3 70B) used as the teacher in the experiments.","marker":"[36]"}],"fun_headline_variants":["LLM teacher trains robots to 93% success, no human","AI coach matches human expertise in robot skill learning","Robots learn from LLM, outclassing human-taught peers","LLM-driven imitation learning hits 93% task completion"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The entire training signal rests on the assumption that the LLM-generated CodePolicy's actions are good enough to serve as the standard of correctness; if the CodePolicy acts wrongly, the agent receives positive feedback for matching that wrong behavior.","fun_headline_variants_meta":{"raw":{"variants":["LLM teacher trains robots to 93% success, no human","AI coach matches human expertise in robot skill learning","Robots learn from LLM, outclassing human-taught peers","LLM-driven imitation learning hits 93% task completion"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000606,"raw_usage":{"total_tokens":2892,"prompt_tokens":1080,"completion_tokens":1812,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":696,"completion_tokens_details":{"reasoning_tokens":1742}},"tokens_in":696,"tokens_out":1812,"duration_ms":14847,"temperature":1.0,"reasoning_tokens":1742,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-16T04:54:27.778645+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Construct a manipulation task where the LLM's CodePolicy is known to be wrong (e.g., it always aims for the microwave handle while any push on the door suffices, as the paper itself describes), train LLM-iTeach to completion, and check whether the agent's success rate stays at or below the LLM Teacher's. If the agent fails to exceed the teacher on this task and reproduces the handle-aiming behavior, the claim that evaluative feedback yields exploration gains beyond the teacher is falsified; the same holds if replacing the LLM's feedback with an oracle that labels actions by true task progress changes the outcome.","supporting_citations":[{"cited_title":"Correct me if i am wrong: Interactive learning for robotic manipula- tion,","cited_arxiv_id":null,"evidence_quote":"Supplies the CEILing baseline, the state-of-the-art interactive imitation learning method with a human teacher, whose experimental setup and results LLM-iTeach is compared against."},{"cited_title":"Code as policies: Language model programs for embodied control,","cited_arxiv_id":null,"evidence_quote":"Supplies the hierarchical prompting strategy ('Code as Policies') that LLM-iTeach adapts to generate the CodePolicy."},{"cited_title":"RLBench: The robot learning benchmark & learning environment,","cited_arxiv_id":null,"evidence_quote":"Supplies the RLBench simulation environment and tasks used for all experiments."},{"cited_title":"A reduction of imitation learning and structured prediction to no-regret online learning,","cited_arxiv_id":null,"evidence_quote":"Provides the no-regret reduction of imitation learning that motivates the interactive feedback loop by addressing distribution shift."},{"cited_title":"Interactive imitation learning in robotics: A survey,","cited_arxiv_id":null,"evidence_quote":"Supplies the survey framing of interactive imitation learning and the taxonomy of corrective versus evaluative feedback."}],"review_version":1}