{"id":"64e9e8b8-13f5-4eae-91fd-5ae6482f21c2","arxiv_id":"2603.27577","paper_version":3,"verdict":"REJECT","confidence":"MODERATE","novelty_score":4.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":6,"one_line_summary":"A 0.6B language model navigates by reading grid-structured text descriptions of depth, object class, and color instead of images, with reported R2R-CE/RxR-CE scores near the top of the leaderboard.","lead":"SOL-Nav turns each robot camera view into a grid of short text lines — average depth, object type, and color per cell — and asks a 0.6B language model to read those lines plus the instruction and output movement commands. The paper reports top competitive scores on R2R-CE and RxR-CE, but the benchmark results may lean on ground-truth semantic maps from the simulator, and no code is released.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Benchmark results may rely on ground-truth semantic segmentation from the simulator, making the claimed 0.6B pure-language advantage untested against real perception error.","rationale":"The reader's weakest assumption — that Tables 1-2 may use ground-truth semantic segmentation from the simulator — is exactly the load-bearing concern I identify. The paper's own Section 3.2 invites this reading by saying the segmentation map 'can be obtained directly from the datasets,' and Section 4 never clarifies which source was used in the benchmarks. The real-world section shows the authors do have a learned segmenter, but no benchmark results are reported with it. Since the central claim is about matching or beating multimodal rivals with a small, pure-language model, an oracle perception channel would invalidate the comparison. This is not an ad hominem or a disagreement with the approach; the structured-observation idea could still work, but the manuscript as written does not rule out the oracle-segmentation explanation. The concrete test I propose would settle the question: rerun the benchmark with a learned segmenter and compare. Until that is done, REJECT is appropriate. Internal numerical inconsistencies (e.g., Table 2 text says NE=6.95 but the table shows 6.87) are secondary and do not change this assessment.","tokens_in":13618,"tokens_out":4991,"duration_ms":51797,"concrete_test":"Run the R2R-CE val-unseen benchmark with semantic labels generated by a fixed, released SegFormer model (the same one used for real-world deployment, or a standard pretrained SegFormer), rather than using the simulator's ground-truth semantic sensor. Keep all other SOL-Nav settings identical and report NE/OS/SR/SPL. If SR or SPL drops by more than a few points relative to Table 1 (e.g., below NaVILA's 49.7/45.5), the reported results depend on oracle segmentation. Also verify that the published numbers reproduce when the simulator's semantic sensor is enabled.","verdict_should_be":"REJECT","load_bearing_attack":"The central claim is that SOL-Nav, a 0.6B pure-language policy, matches or beats much larger multimodal VLN-CE models on R2R/RxR val-unseen without extra data or a waypoint predictor. The load-bearing weakness is the semantic-perception channel. Section 3.2 states that 'the semantic segmentation map of the visual observation can be obtained directly from the datasets or using pre-trained semantic segmentation model' — i.e., the benchmark pipeline may use Habitat's ground-truth semantic sensor. Tables 1-2 list the observation modality 'S.RGB' without defining it or disclosing whether it is the simulator's semantic ground truth or a learned segmenter's output. If the former, SOL-Nav has an oracle semantic channel that most baselines (NaVid, NaVILA, UniNaVid, etc.) do not have; the claimed generalization and efficiency are then not evaluated under realistic perception noise, and the comparison is not apples-to-apples. If the latter, the specific segmenter, checkpoint, and integration are never specified, so the results are not reproducible. The real-world section does describe fine-tuning a SegFormer, but that variant is never evaluated on benchmark splits. Thus the strongest empirical claim — that a language-only 0.6B model reaches SR 53.6 / SPL 49.2 on R2R-CE val-unseen — rests on an undisclosed perception input that could be oracle information. This is a correctness risk, not a matter of consensus: the result could be true with ground-truth semantics and false with a real segmenter.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes SOL-Nav, a VLN framework that converts egocentric RGB-D observations into structured textual descriptions via a multi-resolution grid (current 6×6, short-term history 4×4, long-term history 2×2), extracting per-cell depth, dominant semantic class, and color name. These structured observations are concatenated with the navigation instruction and a system description into a pure-language prompt, which is fed to a 0.6B Qwen3-Embedding encoder with per-step classification heads that predict an action chunk of four discrete actions. The authors report state-of-the-art or competitive results on R2R-CE and RxR-CE val-unseen splits and describe real-world deployment on a Unitree Go2 robot. The central claim is that a small, language-only policy can match or surpass much larger multimodal VLN models without extra training data or a waypoint predictor, while being more parameter-efficient and generalizable.","tokens_in":14056,"tokens_out":7070,"duration_ms":71307,"significance":"If the empirical results were fully substantiated, SOL-Nav would be a notable contribution: it would demonstrate that structured semantic-language observations can replace learned visual encoders in VLN, enabling a 0.6B model to compete with 7B+ multimodal systems and potentially improving robustness to lighting/texture variation. The idea of converting visual input into an explicit semantic grid is interesting and could inspire further work on language-only embodied agents. However, the current manuscript does not yet establish this significance: the source of the semantic segmentation maps used in the benchmark experiments is undisclosed, several internal numerical and textual contradictions appear, and the real-world evaluation is purely qualitative. The claimed efficiency and generalization advantages cannot be assessed until these issues are resolved.","major_comments":[{"comment":"The paper does not specify how the semantic segmentation maps were obtained for the R2R-CE and RxR-CE experiments. Section 3.2 states that the map 'can be obtained directly from the datasets or using pre-trained semantic segmentation model', and the observation columns in Tables 1-2 list an undefined 'S.RGB' input. If 'directly from the datasets' means the Habitat/MP3D ground-truth semantic sensor, then SOL-Nav is granted an oracle perception channel that is not available to RGB-based baselines such as NaVid, NaVILA, or UniNaVid. This would make the comparison unfair and would mean the claimed generalization to unseen environments is not tested against real perception error. If a learned segmenter was used, its identity, checkpoint, and any fine-tuning must be disclosed. This ambiguity is load-bearing for the paper's central claim and must be resolved either by using a realistic segmente","section":"Section 3.2, Tables 1-2"},{"comment":"There are internal inconsistencies between the text and the tables. In the RxR-CE paragraph, the text reports NE=6.95 and names the metrics 'Objective Success', 'SP', and 'Success weighted by Path Length (SPL)', while Table 2 shows NE=6.87 and correctly lists OS, SR, and SPL. In the R2R-CE paragraph, the text says SOL-Nav 'performs slightly worse than InternVLA-N1 on OS, SR, and SPL', but Table 1 shows SOL-Nav's OS (72.9) is substantially higher than InternVLA-N1's OS (63.3); the very next sentence then claims SOL-Nav surpasses InternVLA-N1 on OS. These contradictions undermine confidence in the reported empirical results and must be corrected, with the tables and prose made consistent.","section":"Section 4.1, Tables 1-2"},{"comment":"The claim that 'our model is more than 50 times smaller than these counterparts' is not supported by any model-size figures in the paper. The baselines (NaVILA, UniNaVid, InternVLA-N1) are not given parameter counts, and typical 7B-scale models would be only about 10x larger than the 0.6B SOL-Nav. Please provide the actual model sizes from the cited papers or remove this claim.","section":"Section 4.1"},{"comment":"The real-world deployment section reports no quantitative results—no success rate, path length, or task completion metrics—only the statement that 'the results confirm SOL-Nav's real-time performance and robustness'. Moreover, the real-world variant uses a fine-tuned SegFormer, but this version is not evaluated on the R2R/RxR benchmark splits, so it is unclear whether the benchmark numbers and the deployed system use the same perception pipeline. Please add quantitative real-world evaluation or explicitly limit the claims to a feasibility demonstration.","section":"Section 4.3"}],"minor_comments":[{"comment":"There is a duplicated word: 'Specifically, Specifically, we first use'.","section":"Section 1"},{"comment":"The observation column heading 'S.RGB' is never defined. The text should explain whether this denotes the structured semantic-RGB-description input or something else.","section":"Tables 1-2"},{"comment":"The notation for grid resolutions is introduced as N_curr=6, N_short=4, N_long=2, but the values are not explicitly marked as grid sizes in the text before the 'N×N grid' formula; consider stating clearly that these are grid dimensions for current, short-term, and long-term observations.","section":"Section 3.2"},{"comment":"The phrases 'second column' and 'third column' in the R2R comparison are ambiguous because Table 1 does not have explicit columns beyond Method/Observation/metrics. It would be clearer to refer to row groups (e.g., 'methods without waypoint predictor' and 'methods with additional training data').","section":"Section 4.1"},{"comment":"No code or checkpoint is provided, which further limits reproducibility. Even a provisional link would be helpful.","section":"General"}],"recommendation":"major_revision","confidential_remarks":"The reader's concern about oracle semantic segmentation is well-founded and is the most serious issue in the paper. The ambiguity in Section 3.2 must be resolved before the benchmark comparisons can be taken at face value. The internal numerical and textual contradictions (NE 6.95 vs 6.87; OS 72.9 claimed both worse and better than InternVLA-N1's 63.3) also suggest the manuscript was not carefully proofread. I recommend major revision with a request that the authors either (a) clearly state whether ground-truth semantic labels were used and, if so, re-run the main experiments with a real segmenter and a discussion of the sensitivity, or (b) provide full details of the segmenter used. Without this, the central claim of generalization is not established."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Short version: SOL-Nav is worth a look for the idea — turning egocentric RGB-D into grid-structured text and feeding it to a 0.6B language model for action-chunk prediction — but the evaluation as written does not support the headline claim that a pure-language policy beats much larger multimodal systems fairly.\n\nWhat's new: the per-cell serialization of depth, dominant semantic class, and HSV-mapped color into a time-ordered text prompt is a specific mechanism I haven't seen in NavGPT, MapGPT, or NavCoT. The ablations are informative: dropping depth hurts most (NE 7.98 vs 5.11), history next, grid resolution third. If the numbers hold, this is a reasonable extension of language-as-observation VLN.\n\nWhere it falls down. Table 1's text contradicts itself: it first says SOL-Nav gets the best overall results on all four metrics, then says it performs slightly worse than InternVLA-N1 on OS, SR, and SPL, then says it surpasses InternVLA-N1 on OS by more than 15%. Table 2 quotes NE 6.95 while the table shows 6.87. These inconsistencies sit exactly where the main claim is made.\n\nThe bigger problem is the semantic-segmentation channel. Section 3.2 says the semantic map 'can be obtained directly from the datasets or using a pre-trained semantic segmentation model.' Tables 1–2 list 'S.RGB' without defining it. If the simulator's ground-truth semantic sensor was used during benchmark evaluation, SOL-Nav has an oracle perception channel that NaVid, NaVILA, and UniNaVid do not have. If instead a learned segmenter was used, the paper never says which one and the results are not reproducible. The real-world section does fine-tune a SegFormer, but that variant is never evaluated on the benchmark splits. So the central claim — 0.6B pure language reaching SR 53.6 / SPL 49.2 — rests on an undisclosed perception input.\n\nMinor issues: no error bars, no code release, and the 'more than 50 times smaller' statement overstates the 10x difference in Table 1. The baseline annotation groups (waypoint predictor, extra training data) are applied inconsistently across rows.\n\nWho it's for: people working on language-based observation representations for embodied agents. The idea is worth testing. But as submitted, the experiments do not back the abstract. A revision that discloses the segmentation source, either reruns with a learned segmenter or clearly labels oracle-semantic results, fixes the numerical inconsistencies, and releases code would deserve serious review.\n\nRecommendation: as a reviewer I would reject the current version, but I would not desk-reject it. Send it to peer review (or request a major revision) with the perception-channel and consistency questions as the first order of business.","headline":"A clever language-only VLN idea whose central benchmark claim is undercut by an undisclosed semantic-segmentation channel and by internal numerical contradictions.","tokens_in":14514,"tokens_out":4681,"would_cite":false,"duration_ms":42242,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"A 0.6B language model that reads grid-text descriptions of RGB-D views matches or beats much larger multimodal systems on vision-language navigation benchmarks.","keywords":["Vision-Language Navigation","Structured Observation Language","Large Language Models","RGB-D perception","Action chunk prediction","Generalizable Navigation","Robot Action Prediction"],"falsifier":"Run SOL-Nav on R2R-CE val-unseen with a learned segmenter in place of dataset-provided semantic maps; a significant drop from the reported SR/SPL 53.6/49.2 would show the headline depends on the oracle perception channel. Alternatively, check the evaluation code to see whether the semantic input in Tables 1-2 is ground-truth or predicted.","tokens_in":13531,"feed_emoji":"🤖","tokens_out":7357,"duration_ms":69128,"temperature":0.7,"pith_summary":"The paper aims to show that an embodied navigation agent does not need to process raw pixels at all: egocentric RGB-D frames can be compressed into a grid of short text strings describing depth, dominant semantic category, and color per cell, and this text, concatenated with the instruction, is enough for a 0.6B-parameter pre-trained language model to predict navigation actions. If true, it would mean that a large fraction of vision-language navigation can be done as pure language reasoning, with a model more than ten times smaller than current multimodal baselines, no waypoint predictor, and no extra training data. The authors report state-of-the-art results on the R2R-CE validation-unseen split and competitive results on RxR-CE, plus a real-world deployment on a quadruped robot. A sympathetic reader would care because the approach directly targets the cost and generalization bottlenecks of VLN: it avoids scratch visual encoders and is insensitive to lighting and texture changes.","feed_headline":"A 0.6B text-only policy matches 7B vision-language navigators","feed_subtitle":"It reads grid-text descriptions of RGB-D views instead of raw images, cutting model size and training data.","key_machinery":"The load-bearing object is the structured observation language prompt: a time-ordered sequence of observation matrices, each cell of which is a string concatenation of average depth, dominant semantic class, and HSV-mapped color name. The current observation uses a 6x6 grid, short-term history uses 4x4 grids, and long-term history uses 2x2 grids; these are concatenated with a system description and the instruction into one text sequence. The argument runs through this representation: because the input is pure text, the navigation policy can be implemented entirely in a pre-trained language model with lightweight linear classification heads, and the model's commonsense reasoning is reused dir","core_discovery":"The central claim: an egocentric RGB-D frame can be compressed into a grid of short text strings—each cell holding its average depth, dominant semantic class, and HSV-derived color name—and this text, arranged over multi-resolution grids with history and paired with the instruction, is enough for a 0.6B pre-trained language model to predict four-step action chunks. Reported on R2R-CE val-unseen: NE 5.11, OS 72.9, SR 53.6, SPL 49.2, best among methods without a waypoint predictor or extra training data; RxR-CE results are competitive. Ablations identify depth as the most critical input (SR drops from 53.6 to 21.6 without it), and the paper adds a real-world quadruped deployment at about 0.8s","pith_inferences":["The benchmark comparison is probably not perception-fair as reported: the paper notes segmentation maps 'can be obtained directly from the datasets,' which in the simulator means oracle semantics; if Tables 1-2 used those, part of the gain over RGB-only baselines comes from free perfect perception rather than the textual representation itself.","A direct test would be to train and evaluate the same pipeline with a fixed off-the-shelf segmenter in the loop; the real-world section already uses such a segmenter, so the oracle-to-learned gap is measurable.","If the claim holds, it suggests VLN reasoning is largely text-level spatial reasoning; a useful probe would be perturbing depth strings or color names and watching which channels the policy relies on.","The recipe—convert sensor data into compact discrete text before a language model—may transfer to other embodied tasks like manipulation, where the main cost is modal alignment."],"forward_implications":["If the results hold, a 0.6B pure-language policy is sufficient for strong VLN-CE performance, making visual encoders and multimodal fusion optional rather than necessary.","Because structured text discards lighting, texture, and raw appearance, the policy should transfer to unseen environments with less sim-to-real gap than pixel-based policies.","Training cost drops: the model is fine-tuned with LoRA on 10 epochs with standard benchmark data, no auxiliary waypoint predictor, and no extra datasets.","Real-world deployment becomes practical: inference at about 0.8s on an edge device with a quadruped robot, using a fine-tuned segmentation model and 50 real navigation samples.","Ablations imply that depth information is the most critical channel; any practical instantiation must preserve reliable depth, while color and fine-grained shape are less essential."],"fun_headline_variants":["0.6B text-only policy beats 7B VLN models on R2R-CE","Convert RGB-D views to grid text: tiny PLM navigates unseen scenes","Depth critical: dropping it tanks SR from 53.6% to 21.6%","SOL-Nav: 0.6B LM steers with structured language observations","Grid-text descriptions enable 0.6B model to match 7B navigators"],"cache_read_input_tokens":2304,"weakest_assumption_plain":"The benchmark tables likely rely on semantic segmentation maps that the simulator can supply directly from the dataset ground truth; if so, the claimed efficiency and generalization have not been tested against real perception error.","fun_headline_variants_meta":{"raw":{"variants":["0.6B text-only policy beats 7B VLN models on R2R-CE","Convert RGB-D views to grid text: tiny PLM navigates unseen scenes","Depth critical: dropping it tanks SR from 53.6% to 21.6%","SOL-Nav: 0.6B LM steers with structured language observations","Grid-text descriptions enable 0.6B model to match 7B navigators"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000806,"raw_usage":{"total_tokens":3380,"prompt_tokens":753,"completion_tokens":2627,"prompt_tokens_details":{"cached_tokens":256},"prompt_cache_hit_tokens":256,"prompt_cache_miss_tokens":497,"completion_tokens_details":{"reasoning_tokens":2514}},"tokens_in":497,"tokens_out":2627,"duration_ms":17240,"temperature":1.0,"reasoning_tokens":2514,"cache_read_input_tokens":256,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-04T05:38:23.288458+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run SOL-Nav on R2R-CE val-unseen with a learned segmenter in place of dataset-provided semantic maps; a significant drop from the reported SR/SPL 53.6/49.2 would show the headline depends on the oracle perception channel. Alternatively, check the evaluation code to see whether the semantic input in Tables 1-2 is ground-truth or predicted.","supporting_citations":[],"review_version":2}