{"id":"bb203c41-ed6d-42fa-a97b-52867ffe04ee","arxiv_id":"1908.03627","paper_version":2,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":5,"one_line_summary":"The paper shows that adding depth and segmentation prediction auxiliary tasks, supervised pre-training, and a complexity curriculum to A2C speeds up target-image navigation in realistic simulators.","lead":"This paper presents A2CAT-VN, a deep reinforcement learning agent that navigates toward a target image in simulated indoor environments by using extra training signals like depth maps and image segmentations. The authors report faster convergence than prior goal-oriented navigation methods on AI2-THOR and SUNCG scenes.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Claimed advantage over [3] on AI2-THOR rests on an uncontrolled comparison: the baseline is run only on bigger, harder scenes, and the SUNCG ablation co-varies pre-training and curriculum with the new auxiliary tasks.","rationale":"The paper's contribution is a method, not a theorem, so its core claim is empirical: the proposed architecture and auxiliary tasks make visual navigation training substantially faster than existing methods. The AI2-THOR comparison against [3] is the headline evidence for this claim, and it is also the least controlled experiment in the paper. The authors themselves supply the critical fact: the baseline was run on environments described as bigger and more difficult than the ones used in [3]. That makes it possible that the observed non-convergence of [3] reflects task difficulty rather than a deficiency of the baseline method. The SUNCG ablation is similarly inconclusive for the auxiliary-task attribution because pre-training and curriculum vary with the treatment. I am not claiming the method does not work; the public code and the qualitative results in continuous AI2-THOR are real evidence. But the quantitative headline claim needs a matched comparison before it can be accepted at face value. The reader's conditional verdict already captures this, so I do not propose changing the verdict.","tokens_in":10776,"tokens_out":7962,"duration_ms":88885,"concrete_test":"Run a matched head-to-head experiment on AI2-THOR: train A2CAT-VN and [3] on the same set of scenes (ideally both the four larger rooms used in Fig. 9 and the original smaller single-room scenes from [3]) with the same action set, reward threshold, maximum episode length, number of parallel environments, and frame budget of at least 7 million steps; report average episode length versus frame count for at least 5 seeds. If [3] converges on the original scenes or reaches comparable episode lengths on the matched scenes, the order-of-magnitude claim should be downgraded, and the comparison must be presented as scenario-specific rather than method-level.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The most load-bearing weakness is that the paper's central empirical claim, that A2CAT-VN converges an order of magnitude faster than the target-driven baseline [3] on AI2-THOR, is not tested in a controlled way. In Section IV-E the authors state that the environments they chose 'were bigger and more difficult to navigate than those used in [3]'. The baseline is therefore only evaluated on scenes outside the regime where it was originally validated; no experiment trains both methods on identical scenes, with identical reward thresholds, allowed actions, and evaluation protocols. If the baseline is simply slower on harder scenes, the comparison would not establish that A2CAT-VN is better. The supporting SUNCG ablation in Section IV-G has the same structural problem: A2CAT-VN is compared with A2C+UNREAL after the full method has received supervised pre-training and the environment-complexity curriculum (0.3 to 1.0), while the baseline is not described as receiving either, so the factor-of-two frame savings cannot be attributed to the proposed auxiliary tasks alone. The paper itself lists an ablation of the individual auxiliary tasks as future work, which is a reasonable admission of this gap. A minor but real inconsistency is that Table I lists the pre-training dataset size as 2e5 images while Section IV-G reports approximately 20,000.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes A2CAT-VN, a reinforcement learning architecture for vision-based navigation in which the batched A2C algorithm is augmented with auxiliary tasks: depth-map prediction, observation image segmentation, and target image segmentation. The auxiliary tasks are also used for supervised pre-training of the convolutional base. An environment-complexity curriculum that gradually increases the distance between initial and target states is introduced. The method is evaluated in DeepMind Lab, AI2-THOR (including a continuous variant), and House3D with SUNCG. The central empirical claims are that A2CAT-VN converges roughly an order of magnitude faster than the target-driven navigation baseline of Zhu et al. (ICRA 2017) on AI2-THOR, and that the visual-navigation auxiliary tasks reduce the number of training frames by about a factor of two on SUNCG.","tokens_in":11031,"tokens_out":3674,"duration_ms":36961,"significance":"If the reported results hold under controlled comparison, the paper would make a useful contribution to deep RL for navigation: it demonstrates a concrete architecture with three navigation-specific auxiliary tasks, an LSTM-based solution to partial observability, and a simple pre-training strategy, all validated in realistic indoor simulators. The authors release their implementation and framework on GitHub, which strengthens reproducibility. However, the experimental support for the headline claims is weakened by uncontrolled comparisons. The work is potentially publishable after a major revision that isolates the contributions of the auxiliary tasks, pre-training, and curriculum, and that replays the AI2-THOR baseline on matched environments.","major_comments":[{"comment":"The comparison with the baseline of Zhu et al. [3] is uncontrolled. The authors state that the environments chosen for this experiment \"were bigger and more difficult to navigate than those used in [3]\" but came from the same AI2-THOR simulator. Because the baseline is evaluated only on scenes outside the regime where it was originally validated, the faster convergence of A2CAT-VN (about 5e5 frames to a solution versus no convergence for [3] in 7e6 steps) cannot be unambiguously attributed to the proposed algorithm. To support the claim \"outperforms state-of-the-art,\" the authors need to run both methods on identical scenes with identical action sets, reward definitions, and evaluation protocols, or otherwise demonstrate that the difficulty mismatch does not account for the gap. As written, the result is consistent with an alternative explanation: the baseline may simply be slower on harder scenes.","section":"Section IV-E, Fig. 9"},{"comment":"The SUNCG ablation conflates at least three independent variables: the three visual-navigation auxiliary tasks, the supervised pre-training of the network, and the environment-complexity curriculum. The text reports that A2CAT-VN was pre-trained on approximately 20,000 images and that the environment complexity was linearly increased from 0.3 at step 5M to 1.0 at step 10M, while the comparison baseline A2C+UNREAL is not described as receiving either. The factor-of-two improvement in frames to reach an average episode length of 200 (3e6 vs. 8e6) may therefore be due partly or entirely to pre-training or the curriculum, rather than to the proposed auxiliary tasks. The paper itself lists an ablation study of the individual auxiliary tasks as future work; given that the central claim of Section IV-G is about the auxiliary tasks, this ablation is necessary for the claim to be supported. A controlled experiment should train all variants with identical pre-training, curriculum, and hyperparameters, and ideally ablate each auxiliary task separately.","section":"Section IV-G, Fig. 11"},{"comment":"There is a numerical inconsistency in the pre-training dataset size. Table I lists the pre-training dataset size as 2e5 images, while Section IV-G states \"The number of images we used for pre-training was approximately 20,000.\" This order-of-magnitude discrepancy directly affects the interpretability of the SUNCG ablation (the contribution of pre-training may be much smaller than implied by the table) and must be corrected for reproducibility. Please state the exact dataset size and reconcile the table with the text.","section":"Table I and Section IV-G"}],"minor_comments":[{"comment":"The conclusion that LSTM \"clearly outperforms\" frame stacking is based on a single training run. Reporting multiple seeds with error bars (e.g., standard deviation or confidence intervals) would make the claim more robust, especially because RL training curves are noisy.","section":"Section IV-D, Fig. 8"},{"comment":"The statement that A2CAT-VN \"found the optimal solution after approximately 5e5 frames\" is not accompanied by a quantitative success criterion. Please define \"optimal\" (for example, an episode-length or success-rate threshold) in the text or figure caption.","section":"Section IV-E, Fig. 9"},{"comment":"The continuous AI2-THOR experiment uses a single scene and no comparison baseline. The claim that the agent can \"navigate in non-stationary environments\" would be strengthened by showing results on more scenes or by comparing to a non-continuous baseline in the same setting.","section":"Section IV-F, Fig. 10"},{"comment":"The abstract states that the method \"outperforms state-of-the-art goal-oriented visual navigation methods from the literature,\" but the only direct comparison on AI2-THOR is to [3]. Please qualify the claim to refer to the specific baseline and setting evaluated.","section":"Abstract and Section IV-E"},{"comment":"The learning-rate schedule is written as 7e-4 * (fmax - f)/fmax without a description of when f reaches fmax or whether the rate is annealed to zero. A brief sentence clarifying the schedule would help reproducibility.","section":"Section IV-C, Table I"}],"recommendation":"major_revision","confidential_remarks":"The paper's headline results rely on comparisons where the baseline is not matched in environment difficulty, pre-training, or curriculum. The authors' own statement that an ablation study is future work is a candid admission of the central gap. I believe the work is salvageable with a controlled revision, and the open-source code is a strength. No concerns about novelty or scope beyond the experimental controls."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Take a look at this if you work on target-driven visual navigation, but don't quote the numbers without reading the experimental section carefully. The paper combines A2C with UNREAL-style auxiliary tasks and adds three visual-navigation heads (depth, observation segmentation, target segmentation), with a shared two-stream conv base for observation and target, an LSTM, supervised pre-training, and a curriculum. The combination is new; the pieces mostly are not. What's genuinely useful is the target-segmentation head, which forces the network to attend to the goal image, and the public code.\n\nThe writing is clear, the architecture is sensible, and the authors are honest that an ablation of the individual auxiliary tasks remains future work.\n\nThe main problem is that the two headline comparisons are not controlled. On AI2-THOR, the baseline [3] is trained only on scenes the authors themselves call 'bigger and more difficult' than those used in [3]; the baseline wasn't run on the same scenes with the same protocol, so 'our method converges in 5e5 frames while the baseline never converges in 7e6' is not a fair test. On SUNCG, the A2C+UNREAL baseline does not get the supervised pre-training or the curriculum that A2CAT-VN gets, so the factor-of-two savings in frames can't be attributed to the new auxiliary tasks. The paper acknowledges this second point in the conclusions, which is to its credit, but it undercuts the claim that the auxiliary tasks are the cause.\n\nMinor issues: Table I lists the pre-training dataset as 2e5 images while Section IV-G says about 20,000; one of those is wrong. Training curves are single runs, no error bars or multiple seeds.\n\nWho is this for? Anyone building sample-efficient navigation agents in indoor simulators. The architecture is a reasonable starting point and the code is public. But the empirical claims need a matched baseline and a proper ablation before they can be cited as evidence.\n\nRecommendation: Yes, send it to peer review. The work is serious and reproducible enough to warrant referee time, but a referee should ask for controlled comparisons and a corrected table.","headline":"A useful architecture for target-driven visual navigation with public code, but the headline speedups don't survive scrutiny because the baselines were not trained under matched conditions.","tokens_in":11587,"tokens_out":2420,"would_cite":false,"duration_ms":24693,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Adding depth-map and segmentation prediction as auxiliary tasks to batched A2C makes target-image navigation in indoor scenes converge in roughly 500,000 frames on AI2-THOR and roughly 3,000,000 frames on SUNCG houses.","keywords":["visual navigation","deep reinforcement learning","auxiliary tasks","actor-critic","A2C","target-image navigation","indoor navigation","AI2-THOR"],"falsifier":"Run the same SUNCG training with the same pre-training and complexity schedule while deleting only the three visual-navigation auxiliary heads (keeping the UNREAL tasks); if the agent still reaches average episode length 200 in roughly 3,000,000 frames, the claimed speedup is not caused by the new tasks. A second check: retrain the target-driven baseline on the exact same AI2-THOR environments and action set as A2CAT-VN; if it converges comparably fast, the reported order-of-magnitude difference disappears.","tokens_in":10567,"feed_emoji":"🧭","tokens_out":9630,"duration_ms":87803,"temperature":0.7,"pith_summary":"The paper sets out to show that an image-driven reinforcement-learning agent can navigate to a pictured target in realistic indoor environments much faster if the network is trained with auxiliary visual tasks alongside the main navigation objective. The proposed A2CAT-VN method extends the batched A2C actor-critic algorithm with the two standard UNREAL auxiliary tasks and adds three navigation-specific ones: predicting the depth map of the current observation, segmenting the observed image, and segmenting the target image. These side tasks make it possible to pre-train most of the network by supervised learning before reinforcement learning, and a curriculum that gradually starts episodes farther from the target is also used. If this is right, the practical cost of learning visual navigation drops from millions of environment frames to roughly hundreds of thousands on the tested simulators, and the same compact architecture can handle several targets and several rooms at once.","feed_headline":"Three side tasks cut visual-navigation training time","feed_subtitle":"Depth, segmentation, and target prediction cut needed frames from millions to hundreds of thousands.","key_machinery":"The load-bearing mechanism is the shared network with auxiliary prediction heads: a two-stream convolutional base (one stream for the observation image, one for the target image) feeds an LSTM, and the LSTM features drive the actor, critic, pixel-control head, reward-prediction head, and three visual-navigation heads. The visual-navigation heads reconstruct the depth map, segment the observed scene, and segment the target image through deconvolutional layers attached to the last convolutional layer. Because these tasks are supervised, they permit the bottom of the network to be pre-trained on cheap pre-collected observations, and they keep the target image actively used during training. The LSTM addresses partial observability, which the paper shows outperforms feeding the past four frames.","core_discovery":"On the paper's own terms, the central discovery is that adding visual auxiliary tasks to a batched A2C agent yields a large and fast convergence improvement for target-image navigation. In AI2-THOR, A2CAT-VN reaches a near-optimal policy after roughly 500,000 frames, while the target-driven navigation baseline from the literature does not converge within 7,000,000 steps in the same (larger, harder) environments. In SUNCG houses, the method reaches an average episode length of 200 in about 3,000,000 frames, compared with about 8,000,000 frames when only the UNREAL auxiliary tasks are used. The paper interprets the result as evidence that depth-map, observation-segmentation, and target-segmentation prediction give the shared convolutional layers good features early, and that supervised pre-training plus gradual environment-complexity increase further shortens training.","pith_inferences":["The paper does not isolate the contribution of the three new auxiliary tasks from the supervised pre-training and the complexity curriculum; a controlled ablation that varies only the auxiliary-task heads would settle that, and the paper itself lists such an ablation as future work.","If the advantage over the target-driven baseline depends on the baseline being run on larger and harder environments than it was designed for, the headline speedup could shrink in an exactly matched comparison; this is an open question, not something the current experiments answer.","A practical consequence, if the auxiliary tasks are what matters, is that any simulator offering depth and segmentation can be used to bootstrap a navigation policy before real-world deployment, but real robots without those labels would need a substitute or a transfer step."],"forward_implications":["On the AI2-THOR comparison, A2CAT-VN reaches a near-optimal policy after roughly 500,000 frames, whereas the target-driven navigation baseline has not converged after 7,000,000 frames.","On SUNCG houses, the full method reaches an average episode length of 200 after about 3,000,000 frames, versus about 8,000,000 frames with only the UNREAL auxiliary tasks.","The three visual-navigation auxiliary tasks make supervised pre-training possible for most of the network, which is what the paper credits for the reduced training time.","The environment-complexity curriculum, which starts episodes close to the target and gradually allows farther starts, is presented as a further stabilising factor.","The resulting architecture navigates from raw images alone, without hand-crafted features or an explicit map, and can pursue multiple targets in multiple environments."],"supporting_citations":[{"why":"Provides the target-driven visual navigation baseline that A2CAT-VN is compared against in the AI2-THOR experiment.","marker":"[3]"},{"why":"Supplies the UNREAL auxiliary tasks (reward prediction and pixel control) that the method extends, and the base training setup.","marker":"[7]"},{"why":"Introduces the House3D/SUNCG setting of finding a room type from an observation, used for the auxiliary-task comparison and pre-training.","marker":"[5]"},{"why":"Provides the SUNCG dataset of indoor houses that the method is trained and evaluated on through House3D.","marker":"[6]"},{"why":"Supplies the AI2-THOR simulator used for the main comparison and the multi-target experiments.","marker":"[10]"},{"why":"Provides DeepMind Lab, used for the partial-observability comparison and for pre-training the network.","marker":"[9]"},{"why":"Demonstrates auxiliary tasks and LSTM-based navigation in complex 3D environments, motivating the architecture choices.","marker":"[8]"},{"why":"Describes the batched A2C algorithm that this method extends with auxiliary tasks.","marker":"[11]"}],"fun_headline_variants":["Auxiliary tasks make visual navigation train 14x faster","Depth and segmentation pre-training speed up nav by 10x","Curriculum learning sharpens deep RL navigation training","Three visual side tasks cut training frames by an order of magnitude","Fast visual navigation via auxiliary predictions and curriculum"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that the faster convergence comes from the new visual auxiliary tasks, but the current experiments vary pre-training, curriculum, and environment difficulty at the same time, so the credited mechanism is not actually isolated from those other changes.","fun_headline_variants_meta":{"raw":{"variants":["Auxiliary tasks make visual navigation train 14x faster","Depth and segmentation pre-training speed up nav by 10x","Curriculum learning sharpens deep RL navigation training","Three visual side tasks cut training frames by an order of magnitude","Fast visual navigation via auxiliary predictions and curriculum"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000289,"raw_usage":{"total_tokens":1668,"prompt_tokens":893,"completion_tokens":775,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":509,"completion_tokens_details":{"reasoning_tokens":697}},"tokens_in":509,"tokens_out":775,"duration_ms":8172,"temperature":1.0,"reasoning_tokens":697,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-14T14:24:58.437792+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run the same SUNCG training with the same pre-training and complexity schedule while deleting only the three visual-navigation auxiliary heads (keeping the UNREAL tasks); if the agent still reaches average episode length 200 in roughly 3,000,000 frames, the claimed speedup is not caused by the new tasks. A second check: retrain the target-driven baseline on the exact same AI2-THOR environments and action set as A2CAT-VN; if it converges comparably fast, the reported order-of-magnitude difference disappears.","supporting_citations":[{"cited_title":"Target-driven visual navigation in indoor scenes using deep reinforcement learning,","cited_arxiv_id":null,"evidence_quote":"Provides the target-driven visual navigation baseline that A2CAT-VN is compared against in the AI2-THOR experiment."},{"cited_title":"Reinforcement learning with unsupervised auxiliary tasks,","cited_arxiv_id":null,"evidence_quote":"Supplies the UNREAL auxiliary tasks (reward prediction and pixel control) that the method extends, and the base training setup."},{"cited_title":"Building generalizable agents with a realistic and rich 3d environment,","cited_arxiv_id":null,"evidence_quote":"Introduces the House3D/SUNCG setting of finding a room type from an observation, used for the auxiliary-task comparison and pre-training."},{"cited_title":"Semantic scene completion from a single depth image,","cited_arxiv_id":null,"evidence_quote":"Provides the SUNCG dataset of indoor houses that the method is trained and evaluated on through House3D."},{"cited_title":"AI2-THOR: An Interactive 3D Environment for Visual AI,","cited_arxiv_id":null,"evidence_quote":"Supplies the AI2-THOR simulator used for the main comparison and the multi-target experiments."},{"cited_title":"Deepmind lab,","cited_arxiv_id":null,"evidence_quote":"Provides DeepMind Lab, used for the partial-observability comparison and for pre-training the network."},{"cited_title":"Learning to navigate in complex environments,","cited_arxiv_id":null,"evidence_quote":"Demonstrates auxiliary tasks and LSTM-based navigation in complex 3D environments, motivating the architecture choices."},{"cited_title":"Scalable trust-region method for deep reinforcement learning using Kronecker-factored approximation","cited_arxiv_id":"1708.05144","evidence_quote":"Describes the batched A2C algorithm that this method extends with auxiliary tasks."}],"review_version":1}