{"id":"5331fb9e-d1e5-482f-ae13-43559963d16b","arxiv_id":"2412.19159","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":3.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":4,"one_line_summary":"A simulated mobile robot learns multi-step household instructions better when training is staged from short sub-goals to full instructions, but the supporting experiments lack quantitative comparison.","lead":"This paper tests whether a robot in a simulated kitchen learns multi-step spoken instructions better when training is broken into stages, teaching one sub-goal at a time. The authors report that staged curriculum training beats learning the full instruction at once, but the evidence is mostly learning curves without error bars or success rates.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Baseline comparison in Fig. 6 is confounded by unequal training budget; ICL's advantage may reflect more episodes, not curriculum structure.","rationale":"The reader's weakest_assumption focused on the hand-decomposed curriculum failing to generalize to unseen instructions, which is a real limitation for the broader generalization claim. However, the paper's strongest claim, as quoted, is about outperforming the no-curriculum baseline, and that claim is directly undermined by the confounded comparison in Fig. 6: the ICL agent has strictly more training experience than the baseline at the point of comparison. The reader's rationale did mention a 'potentially confounded baseline,' so there is partial agreement, but the reader did not elevate it to the primary weakest assumption. I chose this because it is the single most load-bearing condition for the central claim: if the baseline is given matched compute, the claimed ICL advantage may vanish. The concern is concrete and testable with a straightforward re-run, and it does not change the overall CONDITIONAL verdict, since the flaw is fixable by reporting proper matched-budget comparisons and quantitative success rates.","tokens_in":9218,"tokens_out":4442,"duration_ms":43964,"concrete_test":"Run a matched-compute baseline: train a fresh agent directly on the full stage-4 instruction for the same total number of episodes as the full 4-stage ICL curriculum (e.g., 48,000 if each stage uses 12,000), and separately train a no-ICL agent on the stage-2 instruction for the same number of episodes as the ICL agent accumulated before the Fig. 6 comparison (stage 1 + stage 2). Report mean success rates over at least 3 seeds. If the matched baseline matches or exceeds ICL's reward/success, the curriculum advantage disappears.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central empirical claim—that ICL-trained agents outperform agents trained without curriculum learning—rests entirely on the learning curves in Fig. 6 (Section V-A). That comparison is confounded by unequal training budgets. The paper states that training 'initiated from stage 1... progressed through 12,000 episodes' and that the ICL agent reaches stage 2 only after completing stage 1, whereas the baseline is described only as being 'trained without ICL' on the stage-2 instruction. No statement is made that the baseline received the same number of episodes, timesteps, or gradient updates as the ICL agent at the point of comparison. If the baseline simply has fewer environment interactions, the higher reward shown in Fig. 6 is an expected consequence of more training, not of curriculum structure. Because the paper reports no quantitative success rates, no error bars, and no seed variance, the 'outperform' claim is not established by the presented evidence.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes incremental curriculum learning (ICL) for a multimodal deep Q-network (MDQN) that navigates a simulated mobile robot in AI2-THOR from RGB images and text instructions. The curriculum decomposes a compound instruction into four prefix stages (Table II) and trains with stage-dependent rewards. Experiments compare ICL with a no-curriculum baseline (Fig. 6), compare positive versus neutral reward designs (Fig. 7), test retraining on 9 objects (Figs. 8-9), and perform sensitivity analysis on maxtime, epsilon, and reward scaling (Fig. 10). The central claim is that robots trained with ICL outperform those without curriculum learning and generalize to more objects.","tokens_in":9389,"tokens_out":3249,"duration_ms":32016,"significance":"If the central claim were established with controlled experiments, the paper would provide a useful demonstration that task decomposition into prefix stages can stabilize deep reinforcement learning for instruction-following mobile robots. The use of AI2-THOR, a public simulator, and the simple MDQN architecture make the approach reproducible in principle. However, the evidence as presented is insufficient: the key comparison lacks a matched training budget, the generalization experiment retrains from random initialization, and no quantitative metrics or seed variance are reported. The authors are transparent about excluding attention mechanisms and about listing unseen-instruction handling as future work, which is honest, but it also confirms that the paper does not yet deliver on its generalization claim. The contribution is therefore currently at the level of a promising idea rather than a validated method.","major_comments":[{"comment":"The baseline comparison is confounded by unequal training budget. The text states that the ICL agent 'progressed through 12,000 episodes' and reached stage 2 only after completing stage 1, but it does not state how many episodes, timesteps, or gradient updates the no-ICL baseline received at the point of comparison. If the baseline was trained for fewer environment interactions, the higher reward of ICL in Figure 6 is an expected consequence of more training rather than of curriculum structure. The authors must match the training budget (or report learning curves as a function of environment steps for both methods) and report quantitative success rates.","section":"Section V-A, Figure 6"},{"comment":"The generalization claim is not supported by the experiment as described. The text says 'The training process restarts, with the model's weights and biases randomly initialized' when moving from 3 to 9 objects, so the experiment measures whether the same architecture can learn a harder task from scratch, not whether curriculum-learned skills transfer. To support generalization, the authors should test on held-out instructions or objects without retraining, or at minimum fine-tune from the pretrained weights and compare with random initialization.","section":"Section V-C"},{"comment":"The curriculum stages are hand-designed as prefixes of the exact test instruction 'find the bread, take it, go to the fridge, and place the bread inside', so the reported success on this instruction is partly built into the training setup. The paper provides no mechanism for generating a curriculum for a novel instruction, and the Conclusions confirm this by listing 'recognizing unseen instructions' as future work. The authors should evaluate on instructions not seen during curriculum design, or state clearly that the method requires a manually pre-decomposed curriculum.","section":"Section IV-C, Table II"},{"comment":"No quantitative success rates, error bars, or number of random seeds are reported anywhere in the experiments. The claims that ICL 'outperforms' the baseline (Abstract), that stage 4 was learned (Figure 5), and that positive reward slightly enhances convergence (Figure 7) all rest on visual inspection of single learning curves. The authors should report mean and variance over at least 3-5 seeds for all conditions, with task success rates at the end of training.","section":"Section V (overall)"}],"minor_comments":[{"comment":"Figure references are inconsistent in capitalization and formatting: 'figure 1', 'figure 2', and 'figure 7' appear alongside 'Figure 3' and 'Figure 6'; unify the citation style.","section":"General"},{"comment":"The learning curves lack axis labels and legends identifying which curve corresponds to ICL and which to the baseline; add clear labels for readability.","section":"Figures 6-10"},{"comment":"The action space includes 'throwing an object', but the example tasks involve placing objects (e.g., 'place the bread inside'); clarify whether placement is modeled as throwing and whether this affects task evaluation.","section":"Section IV-D"},{"comment":"There is a typo 'learnin g' in the Conclusions, and reference [1] contains 'V olume' with a stray space; the manuscript should be proofread for such issues.","section":"Conclusions and References"}],"recommendation":"major_revision","confidential_remarks":"The stress-test concern about unequal training budgets is valid and is the main empirical weakness; the authors must address it with a controlled comparison. The generalization section also promises more than the experiment delivers, and this needs substantive revision rather than a simple text change. The paper would benefit from releasing code and data, as none are currently mentioned, to support reproducibility."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Colleague,\n\nYou asked about arXiv:2412.19159. The short version: it's a workmanlike application of curriculum learning to instruction-following navigation in AI2-THOR, and the main empirical claim is under-supported. The paper shows that a four-stage curriculum, built by taking prefixes of one test instruction, lets a DQN-based agent learn to carry out 'find the bread, take it, go to the fridge, place it inside.' That's not a new algorithm—curriculum learning for RL is well established—and the staged prefix idea is narrow. But it does show the method can work in a realistic simulator, and the sensitivity analysis on maxtime, epsilon, and reward scaling is a useful practical check.\n\nWhat the paper does well: the setup is transparent, the model is standard (ResNet + LSTM + DQN), and the authors are not overselling the architecture. They also acknowledge that attention is out of scope, and they explicitly discuss catastrophic forgetting. That honesty is appreciated.\n\nThe soft spots are the ones you'd expect. Figure 6 compares ICL against a non-curriculum baseline, but the paper never states that both agents received the same number of episodes or gradient updates. The stress-test note is right: ICL is trained through stage 1 before moving to stage 2, so it gets more total interactions. The advantage in the learning curve could just be more training, not curriculum structure. Second, the 'generalization' section is mislabeled—the agent is retrained from random init on 9 objects, not tested on unseen instructions. That's not transfer. Third, there are no quantitative success rates, no error bars, no seeds. The learning curves are the only evidence, and they're visually inspected. Fourth, the curriculum stages are hand-designed from the very instruction used for evaluation, which makes the success partly built into the setup.\n\nThe core direction is plausible and the flaws are fixable. If the authors added a properly matched baseline (same episode budget per stage), a genuine transfer test to unseen instructions, and quantitative results with multiple seeds, this could be a decent workshop-level paper. As it stands, I wouldn't cite it, but it's not incoherent. It deserves a careful referee rather than a desk reject, because the empirical setup is easy to fix and the idea, while incremental, is legitimate.\n\nMy call: engage with it as a potential minor-revision paper at a workshop, not a strong journal submission.","headline":"Modest, honest application of curriculum learning to instruction-following navigation, but the baseline comparison is confounded and the generalization claim is mislabeled.","tokens_in":9910,"tokens_out":2689,"would_cite":false,"duration_ms":25767,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"A mobile robot can learn long task instructions when training is broken into sentence prefixes, the paper argues.","keywords":["incremental curriculum learning","deep reinforcement learning","mobile robot navigation","vision-and-language navigation","multimodal deep Q-network","AI2-THOR","reward shaping","catastrophic forgetting"],"falsifier":"Run the curriculum-trained agent on a held-out instruction from the paper's own Table I that was never used in training, such as \"Search for the cup, take it, go to the sink, and fill the cup with water,\" and compare success rates against the no-curriculum baseline. If the ICL agent does not clearly outperform the baseline on such an unseen multi-step instruction, the claimed generalization to task-based human instructions fails.","tokens_in":9018,"feed_emoji":"🤖","tokens_out":6126,"duration_ms":56695,"temperature":0.7,"pith_summary":"The paper claims that a mobile robot can learn to carry out multi-step household instructions, such as \"find the bread, take it, go to the fridge, and place the bread inside,\" if training is organized as an incremental curriculum: the full instruction is broken into progressively longer prefixes, and the robot learns each stage before moving to the next. The authors put a multimodal deep Q-network in the AI2-THOR kitchen simulator, with a frozen ResNet processing RGB images and an LSTM over GloVe embeddings processing the text, and reward completion with a stage-scaled positive reward. They report that this curriculum-trained agent succeeds on the four-stage instruction and generalizes from 3 to 9 objects, while a baseline trained without curriculum fails to learn even an intermediate stage. The wider point is that structured learning progressions, not just more data or bigger networks, may be the missing ingredient for instruction-following robots.","feed_headline":"Robot masters multi-step commands via staged training","feed_subtitle":"Incremental curriculum lets a robot learn \"find the bread, take it, place it\" where unscaled DRL fails.","key_machinery":"Incremental curriculum learning (ICL) is the hand-built schedule of four instruction stages, each a prefix of the final command, trained sequentially with transfer from the previous stage. The agent itself is a Multimodal Deep Q Network (MDQN) that concatenates a frozen ResNet18 image embedding with an LSTM text embedding built from GloVe word vectors and maps the combined vector through fully connected layers to Q-values; a goal-reaching reward growing from 5.0 to 20.0 across stages plus a small per-step penalty drives the DQN update. Stage-by-stage training and incremental reward scaling together carry the argument.","core_discovery":"The central discovery is that incremental curriculum learning is sufficient to make a deep reinforcement learning agent follow a compound task instruction in a photorealistic indoor simulator. Trained stage by stage, first \"find the bread,\" then \"find the bread, take it,\" and so on, the Multimodal Deep Q Network (MDQN) acquires the full behavior \"find the bread, take it, go to the fridge, and place the bread inside,\" whereas the same architecture trained directly on the multi-step instruction does not achieve the task. The paper further claims that the staged reward design, with larger completion rewards at later stages, reduces catastrophic forgetting of earlier skills and that the learned policy transfers to a kitchen with 9 objects without architectural changes. It frames this as evidence that curriculum learning addresses both training efficiency and generalization for vision-and-language navigation.","pith_inferences":["The paper's generalization test changes object count but not instruction structure; a harder hold-out would be a completely unseen instruction template from its own Table I, such as \"Spot the coffee machine, turn it on, find a mug,\" which the curriculum never staged.","The curriculum is constructed by hand from the exact test command; an automatic curriculum generator based on parsing dependencies or planning would be needed to claim the method generalizes to arbitrary novel instructions.","The stage-4 reward is also the largest, so part of the reported benefit may come from reward magnitude rather than the order of task decomposition; ablating reward scale while keeping a fixed order would separate these factors.","Because the text encoder is a plain LSTM over GloVe embeddings with stop-word filtering, compositionality is limited; richer language models or attention would be a natural next step to handle paraphrases and unseen verbs."],"forward_implications":["If ICL works as claimed, multi-step instruction following no longer requires solving the whole task at once: decomposing instructions into sequentially learned sub-goals is enough.","The same frozen visual encoder and small LSTM can handle a wider object vocabulary, from 3 to 9 objects, so scaling object sets does not force a new architecture or pretrained language model.","Incremental reward scaling is a practical lever: larger completion rewards at later stages outperform normalized or divided rewards, so practitioners can set reward values intuitively.","Curriculum learning plus reward shaping mitigates catastrophic forgetting, so earlier navigation skills persist after later stages are trained.","The approach provides a baseline for household instruction-following in AI2-THOR against which attention-based and memory-augmented models can be compared."],"supporting_citations":[{"why":"Supplies the original curriculum learning idea that tasks should be learned in increasing order of difficulty.","marker":"[3]"},{"why":"Frames curriculum design for reinforcement-learning domains, the methodological basis for the staged training schedule.","marker":"[4]"},{"why":"Provides the AI2-THOR simulator used for training and evaluation.","marker":"[17]"},{"why":"Gives the DQN algorithm that underlies the Q-learning updates of the MDQN.","marker":"[24]"},{"why":"Supplies the pretrained ResNet18 visual feature extractor used in the multimodal network.","marker":"[25]"},{"why":"Provides the GloVe word embeddings that map instruction words to vectors for the LSTM.","marker":"[26]"},{"why":"Supports the claim that reward shaping and curriculum help overcome catastrophic forgetting during later training stages.","marker":"[29]"}],"fun_headline_variants":["Staged training lets robots follow compound commands","Curriculum learning makes robot instruction mastery possible","Incremental steps teach robots multi-part tasks","Robot learns 'find, take, place' via staged training"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that the exact instruction to be executed is known ahead of time and can be manually split into progressively longer, learnable prefixes; if a robot meets an instruction whose decomposition is unavailable or whose structure was never staged, the curriculum gives it no mechanism to learn or generalize.","fun_headline_variants_meta":{"raw":{"variants":["Staged training lets robots follow compound commands","Curriculum learning makes robot instruction mastery possible","Incremental steps teach robots multi-part tasks","Robot learns 'find, take, place' via staged training"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000124,"raw_usage":{"total_tokens":1038,"prompt_tokens":815,"completion_tokens":223,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":431,"completion_tokens_details":{"reasoning_tokens":164}},"tokens_in":431,"tokens_out":223,"duration_ms":2861,"temperature":1.0,"reasoning_tokens":164,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-11T00:52:29.867323+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run the curriculum-trained agent on a held-out instruction from the paper's own Table I that was never used in training, such as \"Search for the cup, take it, go to the sink, and fill the cup with water,\" and compare success rates against the no-curriculum baseline. If the ICL agent does not clearly outperform the baseline on such an unseen multi-step instruction, the claimed generalization to task-based human instructions fails.","supporting_citations":[{"cited_title":"Bengio, J","cited_arxiv_id":null,"evidence_quote":"Supplies the original curriculum learning idea that tasks should be learned in increasing order of difficulty."},{"cited_title":"Narvekar, B","cited_arxiv_id":null,"evidence_quote":"Frames curriculum design for reinforcement-learning domains, the methodological basis for the staged training schedule."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Supplies the pretrained ResNet18 visual feature extractor used in the multimodal network."},{"cited_title":"Pennington, R","cited_arxiv_id":null,"evidence_quote":"Provides the GloVe word embeddings that map instruction words to vectors for the LSTM."}],"review_version":1}