{"id":"cafc033d-7115-4007-8134-bb78ffdf7c3f","arxiv_id":"2506.16623","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":4,"one_line_summary":"Adding a history of recent actions to a vision-language model's prompt during frontier-based exploration yields 46% success and 24.8% SPL on HM3D zero-shot object navigation, on par with prior methods.","lead":"This paper presents a zero-shot object navigation method that feeds a vision-language model the robot's recent action history along with the current camera view, then uses the model's action scores to steer frontier-based exploration. On the HM3D benchmark it reports a 46% success rate and 24.8% SPL, comparable to earlier zero-shot methods, though the evaluation uses a small self-selected episode subset.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The comparative headline claim rests on 50 self-selected HM3D episodes compared against baselines evaluated on different episode sets; the resulting SR estimate is too noisy to establish 'comparable to state-of-the-art.'","rationale":"The central claim is explicitly comparative, so the evaluation protocol is the load-bearing element. The reader's weakest assumption identified exactly this: the 50 reported episodes may not be representative of the validation set, and baseline comparisons come from different papers and possibly different episode distributions. My analysis agrees. I considered other candidate concerns, such as the unspecified mapping of VLM action scores (including a 'go backward' action not present in the agent's action space) onto the value map, and the small measured benefit of action history in the ablation. These are real weaknesses in completeness and evidence, but they are secondary: even if the pipeline were fully specified, the comparative claim would remain unsupported without a reproducible, same-protocol evaluation. The paper's own Section III.C acknowledges VLM reasoning inconsistency and instance-level evaluation tension, which further underscores the fragility of the reported numbers. A concrete rerun on a fixed, published episode set with same-episode baseline comparisons would settle whether the concern lands. Since this matches the reader's conditional verdict, I recommend no change to the verdict.","tokens_in":7415,"tokens_out":4243,"duration_ms":49793,"concrete_test":"Re-run the full pipeline on the complete standard HM3D ObjectNav validation split, or at minimum on a fixed, published random 50-episode subset with a stated seed and episode IDs, and compute SR/SPL with Wilson confidence intervals. Then, using the same episode set, run or obtain results for L3MVN, VLFM, GAMap, and ImagineNav (or cite official challenge numbers only if those numbers are defined on the same episodes). If the full-split or same-subset SR/SPL remain within the confidence intervals of the leading baselines, the comparability claim stands; if the SR drops materially below 40% or the intervals no longer overlap the better baselines, the headline claim fails.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The abstract and Section III.A claim that a 46% SR and 24.8% SPL are 'comparable to state-of-the-art zero-shot methods.' The load-bearing premise is that these numbers were obtained under the same evaluation protocol as the baselines. Section III states that results are 'averaged over 50 episodes, selected from the full validation set,' while Table I says baseline results are 'from respective papers.' The selection rule for those 50 episodes is not specified. If the episodes were chosen for convenience or scene coverage rather than sampled uniformly, the point estimate is biased relative to the validation distribution used by L3MVN, VLFM, GAMap, and ImagineNav. Even if the 50 episodes are a uniform random sample, the statistical resolution is low: 23 successes out of 50 gives a 95% Wilson interval of roughly 32% to 61% for SR. Against that uncertainty, the reported gaps from VLFM (52.5/30.4) and GAMap (53.1/26.0) are not meaningful. The history ablation in Section III.B shows only a 2-point SR drop (46 to 44) and a 1.1-point SPL drop (24.8 to 23.7) on the same 50 episodes, with no error bars, so it does not independently secure the central contribution. Thus the evidence as reported does not yet establish that the history-augmented prompting strategy performs at the level of leading zero-shot methods on HM3D.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes a zero-shot Object Goal Navigation (ObjectNav) framework that integrates a vision-language model (LLaVA-1.6) into frontier-based exploration. The key idea is to give the VLM a prompt containing the current egocentric RGB view, the target object category, and a record of recent actions, so that it outputs probability scores for forward, backward, left, and right actions; these scores are projected into a 2D semantic value map that re-weights frontier selection. A final approach phase uses detection (YOLOv7/Grounding-DINO), VLM verification, Mobile-SAM segmentation, and KNN clustering to generate supplementary waypoints. The method is evaluated in Habitat on episodes from the HM3D dataset, reporting 46% SR and 24.8% SPL, which the authors claim are comparable to state-of-the-art zero-shot methods. An ablation study reports that removing the action-history mechanism decreases SR from 46% to 44% and SPL from 24.8% to 23.7%.","tokens_in":7708,"tokens_out":4199,"duration_ms":42413,"significance":"If the reported results were obtained under the same protocol as the baselines and the ablation were statistically supported, the paper would make a useful contribution by showing that explicitly providing action history to a VLM can mitigate oscillation in zero-shot ObjectNav and by presenting a modular system built from open-source components. The authors are transparent about the computational constraints that led to a 50-episode evaluation, and they use only publicly available models, which aids reproducibility. However, the evaluation as presented does not yet support the headline claim of state-of-the-art comparability: the episode subset is not specified, no error bars are given, and the ablation difference is within sampling noise. The core idea is plausible and the system is clearly described, but the evidence is currently insufficient to establish the central claims.","major_comments":[{"comment":"The headline comparison is not statistically grounded. The method is evaluated on 50 episodes 'selected from the full validation set' with no description of the selection procedure, while baseline SR/SPL values are taken from the original papers, which typically report on the full validation split. If the selected episodes are not representative of the same distribution, the point estimates are not comparable to those baselines. Even under uniform random sampling, 23 successes out of 50 gives a 95% Wilson interval of roughly 33% to 60% for SR, which overlaps the reported values for VLFM (52.5%), GAMap (53.1%), and ImagineNav (53.0%). To support the 'comparable to state-of-the-art' claim, the authors should evaluate on the same standard episode set as the baselines, or provide a clearly defined stratified sampling procedure, report confidence intervals, and make the episode list or seed available. The absence of error bars throughout Tables I and II makes the performance differences uninterpretable.","section":"Section III.A and Table I"},{"comment":"There is an internal inconsistency in the action space. Section II defines the agent's action space as 'move forward', 'turn left', 'turn right', 'look up'/'look down', and 'stop'. However, Section II-B states that the VLM produces probability scores for 'forward, backward, left, right', and Listing 1 explicitly asks the VLM to score 'go backward'. The paper does not explain how the 'go backward' score is used: is it discarded, mapped to a 180-degree turn, or projected onto the value map in some other way? Without this clarification, the value-map construction and the resulting navigation behavior are not reproducible. The authors should either remove 'go backward' from the prompt or specify exactly how its score is processed.","section":"Section II-B and Listing 1"},{"comment":"The ablation study does not support the claim that action history is crucial. Removing the history mechanism changes SR from 46% to 44% (a difference of one success out of 50 episodes) and SPL from 24.8% to 23.7%. With no error bars, no multiple seeds, and no significance test, this difference is within the sampling noise of a 50-episode evaluation and cannot be described as a 'substantial performance drop' or as confirming the critical role of the history component. The authors should provide confidence intervals, run multiple episode subsets or seeds, and temper the language accordingly.","section":"Section III.B and Table II"}],"minor_comments":[{"comment":"The history length is given only as 'e.g., N=10'; please provide the exact value used in the final experiments and justify it, ideally with a sensitivity analysis.","section":"Section II-C"},{"comment":"The hyperparameters λ, the detection confidence threshold (>0.8), and the semantic value threshold (>0.8) are free choices; please state how they were selected and whether the results are sensitive to their values.","section":"Section II-D and Eq. (1)"},{"comment":"Reference [13] is cited in the text as 'PaliGemma' but the cited paper is 'Pali' (arXiv:2209.06794); either cite the correct PaliGemma paper or fix the name.","section":"References"},{"comment":"The name 'LLaVA-1.6' appears as 'LLaV A-1.6' with an extraneous space; please correct this typo consistently.","section":"Throughout"},{"comment":"The paper notes that the VLM sometimes suggests actions inconsistent with its own scene interpretation, but it does not quantify how often this occurs or how it affects navigation metrics; a brief quantitative analysis would strengthen the evaluation.","section":"Section III.C"},{"comment":"The claim 'We are among the first to explicitly use history-augmented prompts' is not substantiated with a concrete comparison to prior work on memory- or history-based prompting in VLM navigation; please provide a more precise positioning.","section":"Section I"},{"comment":"Table I would benefit from a column indicating the number of episodes used in each baseline evaluation, since cross-paper comparisons are only meaningful if the evaluation protocols and episode distributions match.","section":"Table I"}],"recommendation":"major_revision","confidential_remarks":"The paper addresses a timely problem and the system design is thoughtful, with a clear modular architecture and the use of open-source components. However, the evaluation is the main weakness: the 50-episode self-selected subset, the cross-paper baseline comparison, the lack of error bars, and the statistically insignificant ablation all prevent the paper from supporting its central claims. These issues are fixable within the manuscript's scope by running on a standard episode set or a properly sampled subset with statistical reporting, and by clarifying the backward-action handling. I therefore recommend major revision rather than rejection."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"You should know this paper for one reason: it adds action history to the VLM prompt in zero-shot ObjectNav, which is a genuinely new trick relative to the cited baselines (L3MVN, VLFM, GAMap, ImagineNav). The system is clearly described, uses open-weights LLaVA-1.6, and the authors deserve credit for keeping the pipeline reproducible in principle. They also openly acknowledge real issues in Section III.C (VLM inconsistent with its own scene interpretation, class-level vs instance-level evaluation mismatch).\n\nBut the evidence is not strong enough to carry the central claim. The 46% SR and 24.8% SPL come from 50 episodes selected by the authors from the HM3D validation split, with baseline numbers taken from other papers that likely used different episode sets. Even if the selection were uniform, 23 successes out of 50 gives a wide confidence interval (roughly 32-61%), so the small gaps to VLFM (52.5 SR) and GAMap (53.1 SR) are within noise. The ablation that supposedly secures the core contribution is a 2-point SR drop (46 to 44) and 1.1-point SPL drop—that is not a 'substantial' drop, and there are no error bars. I also note the paper says the VLM outputs scores for 'go backward' when the action space defined earlier contains no backward action; the mapping of that score to the value map is never specified. The fallback mechanism (repeat last non-turn action) might be doing the work that the prompt is credited for, and it is not ablated on its own.\n\nThis is a legitimate system-paper idea with a plausible pipeline, but it is not at the level of a rigorous empirical claim. I would send it to peer review, but only with the expectation of major revisions: standard episode set with known distribution, error bars or at least multiple seeds, clarification of the action-score mapping, and a more careful ablations that separate the history prompt from the fallback.\n\nWho gets value from this? Researchers working on VLM-based navigation and prompt engineering for embodied AI. It is a decent workshop paper or a related-work citation, but I would not cite the headline numbers in my own work yet.\n\nFinal recommendation: engage with it, but treat the results as preliminary.","headline":"Likely a useful incremental idea (history-augmented VLM prompting for frontier-based ObjectNav), but the 50-episode self-selected evaluation and cross-paper baseline comparison are too weak to back the 'comparable to state-of-the-art' headline.","tokens_in":8251,"tokens_out":1967,"would_cite":false,"duration_ms":21979,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"A history-augmented vision-language model, prompted with its own recent actions, can guide frontier-based exploration to find objects in unseen indoor environments at success rates comparable to state-of-the-art zero-shot methods.","keywords":["Object Goal Navigation","Zero-shot navigation","Vision-language models","History-aware prompting","Frontier-based exploration","Semantic value map","HM3D","Habitat simulator"],"falsifier":"Run the same framework on the full official HM3D ObjectNav validation split, or on the exact episode list used by each baseline, and recompute SR and SPL; if the numbers fall materially below 46% and 24.8%, or below the baselines rerun on the same episodes, the paper's claim of state-of-the-art comparability would be refuted.","tokens_in":7188,"feed_emoji":"🤖","tokens_out":7157,"duration_ms":67698,"temperature":0.7,"pith_summary":"The paper sets out to show that zero-shot Object Goal Navigation does not need a trained navigation policy or expensive closed models: a frozen 7-billion-parameter open vision-language model can make the key decisions if its prompt includes what the robot has just done. The proposed system feeds the last ten actions into the VLM's prompt, asks for probability scores on forward, backward, left, and right, projects those scores onto a 2D semantic value map, and uses that map to pick among geometric frontiers. The authors report a 46% success rate and 24.8% SPL on 50 HM3D episodes, placing the method near leading alternatives that use larger or closed models. If this holds, the practical upshot is that robust object search in unseen homes can be achieved by prompt engineering around an accessible open VLM, with the added benefit of resisting the repetitive loops that plague memoryless navigators.","feed_headline":"History-aware VLM prompts push zero-shot navigation to 46% success","feed_subtitle":"Feeding a frozen 7B VLM its action history cuts loop errors and rivals top zero-shot navigators on HM3D.","key_machinery":"The core object is the history-augmented VLM prompt: a short, strictly formatted request to LLaVA-1.6 that includes the target name, the current view, and the last $N$ actions, and demands probability scores in $[0,1]$ for go forward, go backward, turn right, and turn left. Those scores are projected through a viewing-uncertainty model $c(d,\\theta)=e^{-\\lambda d}\\cos^2\\left(\\frac{\\theta}{\\theta_{\\mathrm{fov}}/2}\\cdot\\frac{\\pi}{2}\\right)$ into a semantic value map, fused across re-observations by confidence-weighted averaging, and combined with geometric frontiers so the agent heads to the highest-value frontier. Action history plus a fallback that repeats the last valid non-turn action is what stops oscillation. The last-piece mechanism is VLM-verified detection followed by Mobile-SAM segmentation and KNN clustering of high-value pixels to create waypoints for the low-level point-navigator VER. In short, the argument runs on a single loop: observe, prompt with history, score actions, fuse scores into a map, move; the loop's memory is what makes the VLM a stable navigator.","core_discovery":"On the paper's own terms, the discovery is that a VLM's reasoning ability, not just its embedding similarity, can be recruited for navigation by making the prompt temporally aware. At each decision step, LLaVA-1.6 receives the current RGB view, the target category, and a rolling record of the last ten actions, and is instructed to score each candidate action while avoiding repetitions. Those scores are converted into a confidence-weighted semantic value map and overlaid on a frontier map, so exploration moves toward regions the VLM finds semantically promising. A second VLM pass verifies detected objects and, with Mobile-SAM segmentation and KNN clustering of high-value pixels, generates extra waypoints for the final approach. The paper concludes that this history-augmented prompting strategy is the reason the framework reaches 46% SR and 24.8% SPL, levels it describes as comparable to state-of-the-art zero-shot methods.","pith_inferences":["With only 50 episodes, the reported 2-point SR improvement from action history has no confidence interval; a direct extension would be to rerun on the full validation split and report per-episode variance, which would tell readers whether the gain is stable.","Because the method only changes the prompt and the score-fusion layer, the same history-augmented prompting could be grafted onto other zero-shot navigators that already produce frontier scores, letting readers isolate the contribution of temporal context from the contribution of frontier exploration.","Combining history-aware prompting with scene-imagination-style reasoning, where the VLM is prompted to visualize likely object locations, is a plausible path past the roughly 50% SR ceiling shared by current zero-shot methods."],"forward_implications":["Frozen, open-source VLMs can be turned into ObjectNav planners by prompt design alone: no weights are updated and no navigation policy is learned.","Explicit action history reduces degenerate loop behavior; in the paper's 50-episode ablation, removing it drops SR from 46% to 44% and SPL from 24.8% to 23.7%.","The semantic value map lets the VLM's directional preferences be fused with geometric frontiers, so the approach inherits the coverage of frontier exploration while adding semantic bias.","Because the VLM choice is an engineering decision (LLaVA-1.6 was selected over PaliGemma and VILA), the prompting method should transfer to other open VLMs as they improve."],"supporting_citations":[{"why":"Supplies the frozen LLaVA-1.6 vision-language model that performs all semantic scoring and verification.","marker":"[12]"},{"why":"Provides the frontier-based exploration algorithm that generates geometric waypoints and the obstacle map.","marker":"[15]"},{"why":"Provides the VER low-level navigator that executes point-to-point movement between waypoints.","marker":"[16]"},{"why":"VLFM is the closest zero-shot VLM baseline; the paper compares its success rate and SPL to establish parity.","marker":"[6]"},{"why":"L3MVN is a language-model frontier-scoring baseline whose reported numbers anchor the comparison.","marker":"[22]"},{"why":"ImagineNav is a GPT-4o-mini baseline near the top of the comparison table and defines the performance level the paper aims to match.","marker":"[23]"},{"why":"GAMap supplies the strongest SPL baseline among the comparisons and a CLIP-plus-GPT-4 approach against which the method is measured.","marker":"[7]"},{"why":"Defines the Habitat ObjectNav 2023 protocol, action space, and HM3D validation episodes used in evaluation.","marker":"[11]"},{"why":"Defines SPL and the PointNav pretraining task used by VER, and supplies the evaluation metric convention.","marker":"[17]"}],"fun_headline_variants":["History-aware VLM prompts slash loop errors, hit 46% SR","Zero-shot navigation gets smarter with VLM action history","VLM reasoning with temporal prompts rivals top zero-shot navigators","46% success: VLM history prompts boost zero-shot ObjectNav","Feeding VLM action history curbs loops, lifts ObjectNav"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The central claim of matching state-of-the-art results depends on the assumption that the 50 HM3D episodes chosen for evaluation fairly represent the full validation set, and that success rates quoted from other papers were measured on comparable episodes under the same stopping conditions.","fun_headline_variants_meta":{"raw":{"variants":["History-aware VLM prompts slash loop errors, hit 46% SR","Zero-shot navigation gets smarter with VLM action history","VLM reasoning with temporal prompts rivals top zero-shot navigators","46% success: VLM history prompts boost zero-shot ObjectNav","Feeding VLM action history curbs loops, lifts ObjectNav"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000682,"raw_usage":{"total_tokens":3094,"prompt_tokens":939,"completion_tokens":2155,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":555,"completion_tokens_details":{"reasoning_tokens":2067}},"tokens_in":555,"tokens_out":2155,"duration_ms":14092,"temperature":1.0,"reasoning_tokens":2067,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-15T19:21:40.427729+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run the same framework on the full official HM3D ObjectNav validation split, or on the exact episode list used by each baseline, and recompute SR and SPL; if the numbers fall materially below 46% and 24.8%, or below the baselines rerun on the same episodes, the paper's claim of state-of-the-art comparability would be refuted.","supporting_citations":[{"cited_title":"Frontier based exploration for autonomous robot,","cited_arxiv_id":null,"evidence_quote":"Provides the frontier-based exploration algorithm that generates geometric waypoints and the obstacle map."},{"cited_title":"GAMap: Zero-Shot Object Goal Navigation with Multi-Scale Geometric-Affordance Guidance","cited_arxiv_id":"2410.23978","evidence_quote":"GAMap supplies the strongest SPL baseline among the comparisons and a CLIP-plus-GPT-4 approach against which the method is measured."}],"review_version":2}