{"id":"8e993a74-c21c-4edd-ae4a-93489c83897b","arxiv_id":"2607.19633","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":4,"one_line_summary":"A vision-language-model-based prune-and-merge abstraction improves success and runtime for TAMP, contact-implicit MPC, and a VLA policy in cluttered tabletop manipulation.","lead":"LENS is a robot-system component that uses a vision-language AI to decide which objects in a cluttered scene to ignore or group together before planning a task. The authors report that it improves success rates and keeps runtimes steady for three different robot control approaches, while releasing no code or data.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"C3+ success rates may be computed over mismatched trial sets, making the claimed improvement uninterpretable.","rationale":"The reader's weakest assumption focuses on VLM reliability and the adequacy of failure feedback. This is a legitimate concern, especially for the VLA stack where no feedback loop exists. However, I argue that a more immediate, concrete threat is the apparent mismatch in trial counts for the C3+ success-rate comparison. If baseline and LENS were evaluated on different object-count distributions, the quantitative success-rate claim for model-based control is unsupported, regardless of how well the VLM performs. This concern is directly checkable by requesting per-count data, and it reinforces the reader's call for code/data release. I do not believe this indicates misconduct—it may be a reporting ambiguity—but it is load-bearing because the central claim aggregates across three modules. The reader's CONDITIONAL verdict already asks for transparency, so I do not change the verdict; I only sharpen the condition by specifying the per-count breakdown as a required artifact.","tokens_in":13403,"tokens_out":8150,"duration_ms":89112,"concrete_test":"Request from the authors the per-object-count success counts and trial totals for both LENS-C3+ and baseline in the Section 5.2 simulation experiments. Restrict both methods to the same set of object counts and recompute success rates. If code/data are released, rerun the comparison on the common subset and report whether LENS still improves. If the denominators differ across methods, the aggregate success-rate claim is not interpretable and must be corrected.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim that LENS improves success rates across all three modules rests on the C3+ success-rate comparison. In Section 5.2, the paper reports 'LENS-C3+ and baseline succeeded on 39/45 and 17/30 trials respectively' while stating that 'results for each object are averaged across five trials.' If the object-count ablation used the same number of object values for both methods, the total number of trials should be equal (e.g., 5 trials × N object counts). The reported totals of 45 and 30 imply that baseline was evaluated on a different, likely easier, set of object counts (e.g., missing the highest clutter levels where it is known to time out). If so, the baseline success rate of 17/30 is inflated relative to LENS's 39/45, and the claimed improvement may vanish on a matched evaluation. This is a concrete experimental-validity threat to one of the three central pillars, independent of VLM reliability. The paper does not provide per-object-count success data, so the aggregate numbers cannot be interpreted as they stand.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"This paper introduces LENS, a vision-language-model-driven scene abstraction layer that sits in front of existing manipulation planners and controllers. Given a task description and a scene, GPT-4o outputs a list of task-relevant objects and merges them into sublists; LENS prunes the rest and, where appropriate, fuses grouped geometry. When the downstream module times out or errors, LENS re-queries the VLM with failure feedback and revises the abstraction. The authors instantiate LENS for PDDLStream-based TAMP, C3+ contact-implicit MPC, and the pi0.5 vision-language-action model, and report evaluations in simulation and on hardware on cluttered tabletop tasks. The paper claims that LENS improves success rates and runtimes across all three modules.","tokens_in":13628,"tokens_out":8670,"duration_ms":88190,"significance":"If the claims hold, LENS offers an appealingly generic interventional layer that could reduce combinatorial search in TAMP and contact-implicit MPC and reduce distribution shift for VLAs. Its strengths include the breadth of downstream integration, the explicit prompts and implementation details in the appendices, hardware experiments for both C3+ and VLA, and a C3+ comparison against geometric pruning baselines. However, as presented, the quantitative evidence is not yet fully convincing: the C3+ success-rate comparison appears to be computed over mismatched trial sets, and the closed-loop retry structure is not matched across conditions. The hardware VLA results also rest on very small per-condition samples without confidence intervals. These issues need to be addressed before the central claims are fully supported.","major_comments":[{"comment":"The text reports 'LENS-C3+ and baseline succeeded on 39/45 and 17/30 trials respectively' and states each object count is averaged over five trials. Thus LENS was evaluated on 9 object-count conditions and baseline on 6. The conditions are not matched; the baseline appears to omit the highest clutter levels (7+ objects), where it is reported to require over 4000 s. Consequently, the aggregate success rates 39/45 vs 17/30 are not comparable and do not support the claim that LENS improves C3+ success. Report per-object-count success counts and compare on the same trial set, or use a valid conditional comparison.","section":"Section 5.2, Figure 5"},{"comment":"LENS can retry up to N times and stops at first success. No statement indicates whether Baseline-C3+ was given the same number of attempts; the TAMP section explicitly says 'Baseline is rerun twice for fairness,' but no analogous statement appears for C3+. If the baseline is single-shot per trial, the reported success advantage may reflect the retry budget, not scene abstraction. Report per-attempt LENS success, or run the baseline with an identical retry budget and report cumulative success. Appendix C's Figure 10 should also clarify whether the distance-based baselines receive iterative feedback.","section":"Algorithm 1, Section 5.2"},{"comment":"Hardware VLA results use n=10 per fruit, with baseline success 0.0 for three of four fruits and 0.2 for the fourth. Raw counts and confidence intervals (or an appropriate hypothesis test) are needed to support the statement that LENS 'consistently improves' per task; the aggregate comparison may be significant but the per-fruit values are not demonstrated robust. Please report exact binomial CIs or raw counts and a combined analysis.","section":"Section 5.3, Table 1"}],"minor_comments":[{"comment":"Stacked success contributions by iteration are reported without error bars or raw counts; specify the per-environment number of episodes and variability.","section":"Section 5.1, Fig. 4a"},{"comment":"Phrase 'Results for each object are averaged across five trials' should read 'for each object count'.","section":"Section 5.2"},{"comment":"Typo: 'my have spurious selections' should be 'may have spurious selections'.","section":"Appendix B"},{"comment":"Typo: 'complimentarity' should be 'complementarity'.","section":"Appendix E"},{"comment":"Ground-truth segmentation in simulation gives LENS privileged perception; state this as an explicit limitation or include an ablation with a detector-based pipeline.","section":"Section 5.3"},{"comment":"VLM query time (1.76 s) is excluded from runtime comparisons; report end-to-end time including these queries for LENS.","section":"Section 5"},{"comment":"Clarify what 'Iteration 0/1/2' means for the distance-based baselines; if they do not receive feedback, label accordingly.","section":"Appendix C, Fig. 10"},{"comment":"Prompt constraints such as 'AT MOST 4 SUBLISTS TOTAL' are free parameters; no sensitivity analysis is provided. At least a brief discussion would help assess robustness.","section":"Appendix E"}],"recommendation":"major_revision","confidential_remarks":"Recommendation: major revision. The core idea is timely and the integrations are valuable, but the experimental reporting—especially the C3+ success-rate comparison and retry matching—must be tightened before the central claims can be accepted. I would be willing to review a revised version."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Quick take: this is a genuinely useful idea — a VLM-based prune/merge front-end that iteratively simplifies a scene for planning and control — with a real closed-loop twist. The evidence is consistent across three very different stacks, but the quantitative support is thinner than the abstract suggests, and the C3+ success-rate comparison in the paper is mismatched.\n\nWhat's new: LENS reformulates scene abstraction as a task-conditioned VLM query with failure feedback. Pruning removes irrelevant objects, merging groups coupled ones into a single body, and the closed-loop re-query lets the abstraction adapt to execution failures. I've seen VLM-based pruning before, but the merge operation and the retry mechanism are a step beyond — and the paper does the work of instantiating the idea for TAMP, contact-implicit MPC (C3+), and a π0.5 VLA. That breadth is the contribution, and it's credible. The authors include full prompts and are honest about failure modes.\n\nWhat works: the TAMP results are clean (50 episodes per condition, two feedback iterations). The runtime scaling plot for C3+ is striking — LENS stays roughly flat while baseline grows by orders of magnitude. The VLA simulation uses ground-truth segmentation, but the authors acknowledge that, and the hardware results with GroundingDINO/SAM/LaMa persist. The distance-based pruning baseline in Appendix C is a good check.\n\nThe soft spots:\n\n- The C3+ success-rate numbers don't add up as reported. Section 5.2 says LENS succeeded on 39/45 trials and baseline on 17/30. Those denominators are different, which implies baseline was not evaluated on the same object-count settings. If, as the runtime data suggests, baseline was only run on the easier clutter levels, its 17/30 is inflated relative to LENS's 39/45. The paper gives no per-object-count success data, so the success-rate claim for the model-based control pillar is uninterpretable as stated. The runtime plot is fine; the success-rate sentence is not.\n\n- No error bars or confidence intervals anywhere. The hardware VLA results are n=10 per fruit. You can see a signal, but not bound it.\n\n- The prompts are heavily hand-tuned. I don't fault the authors — that's how VLM-guided robotics works — but it undercuts the 'task-agnostic' claim. 'AT MOST 4 SUBLISTS TOTAL!!!!!!' is task-specific engineering.\n\n- No code or data released. For a closed-loop method whose behavior depends on prompt details and failure feedback, that's a significant reproducibility gap.\n\nWho this is for: robotics researchers working on manipulation in clutter, especially with TAMP or contact-implicit methods. It's a solid systems paper, not a foundational result. It deserves a serious referee — the method is worth engaging with, and the mismatched comparison is fixable in review if the authors report per-condition trial counts.\n\nRecommendation: send to peer review. Require matched success-rate reporting, per-condition data or error bars, and ideally code/data release before acceptance.","headline":"A genuinely useful VLM prune/merge front-end for manipulation, but the C3+ success-rate comparison is mismatched and the quantitative evidence is thinner than the abstract claims.","tokens_in":14145,"tokens_out":3490,"would_cite":true,"duration_ms":34122,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"A vision-language model can pre-filter cluttered scenes to improve planners, controllers, and learned policies.","keywords":["robot manipulation","scene abstraction","large language models","vision-language models","task and motion planning","contact-implicit MPC","vision-language-action models","cluttered environments"],"falsifier":"Replace the VLM's object selection with a random baseline (or a nearest-neighbour geometric heuristic) on the same cluttered scenes, keeping every other component identical; if the random baseline achieves comparable success rates, the VLM's semantic reasoning is not the causal mechanism. Alternatively, construct a scene where a critical object is visually similar to a distractor, causing the VLM to prune it, and where the downstream planner completes with a wrong but non-timeout plan—this would expose a silent failure the feedback loop cannot detect.","tokens_in":13272,"feed_emoji":"🤖","tokens_out":3786,"duration_ms":40155,"temperature":0.7,"pith_summary":"LENS is a plug-and-play front-end that automatically simplifies a cluttered robot scene by pruning task-irrelevant objects and merging coupled ones into single bodies, using a vision-language model (VLM) guided by task descriptions and failure feedback. The paper claims this closed-loop abstraction improves success rates and reduces computation time for three very different downstream systems: classical task-and-motion planning, contact-implicit model-predictive control, and a vision-language-action (VLA) policy. This matters because real-world manipulation scenes with many objects cause all three approaches to degrade sharply in speed and reliability, and manual scene-abstraction engineering does not scale. If LENS works, it offers a general, task-agnostic way to make existing robot stacks viable in clutter without retraining them.","feed_headline":"One VLM front-end improves three robot control stacks in clutter","feed_subtitle":"Pruning distractors and merging coupled objects before planning keeps task performance up as scenes grow messy.","key_machinery":"The prune-and-merge loop: a VLM is prompted with a task description, a scene representation (object list, poses, masses, or an image with bounding boxes), and a goal; it outputs a list of lists of task-relevant objects. Excluded objects are pruned—removed from decision variables, collision/contact pairs, or visual input—while each sublist's objects are merged into a single composite entity with a fused geometry and co-moving dynamics. Feedback is triggered by downstream timeout or error codes, appending a corrective prompt that encourages broader selection, then re-queries the VLM. This closed loop is what lets the abstraction adapt to hallucinations or evolving scenes.","core_discovery":"The central discovery is that task-relevant scene abstraction, generated by querying a frozen multimodal LLM (GPT-4o) and refined through closed-loop failure feedback, can serve as a universal front-end for robot manipulation. LENS constructs a reduced scene by removing objects that can be ignored and merging objects that are functionally or dynamically coupled into a single rigid body (or image-inpainted region for the VLA). When the downstream planner or controller times out or returns an error, the previous abstraction and a feedback message are appended to the prompt, and the VLM re-queries to produce a corrected scene. Experiments across TAMP, contact-implicit MPC (C3+), and a VLA (π0.5","pith_inferences":["The reliance on timeouts and error codes means tasks that fail silently—where the robot executes a wrong but non-erroneous action—would require richer feedback signals for LENS to correct itself; this is a testable boundary of the method.","Because the VLM is frozen and prompts are hand-designed, the prune/merge decisions could be distilled into a small learned model that runs in real time, making the abstraction layer cheaper at deployment.","The merging operation, which treats loosely coupled objects as a single dynamic body, may be a useful inductive bias beyond manipulation—for example in navigation or assembly, where piles or groups of objects can be reasoned about as units.","The paper's results suggest that 'task relevance' in manipulation is largely recoverable from a general VLM without task-specific training, but it remains to be seen whether this holds for long-horizon tasks where relevance evolves over time."],"forward_implications":["TAMP success rates improve in heavy clutter and stacked-object scenes because the planner only searches over task-relevant objects, avoiding distractor enumeration.","Contact-implicit MPC (C3+) maintains roughly constant execution time as object count increases (40–135 seconds across 2–8 objects), whereas the baseline grows by orders of magnitude (over 4000 seconds at 7 objects).","VLA success in cluttered LIBERO scenes rises from 0.5 to 0.69 in simulation, with hardware trials showing gains for each of four fruit-picking tasks.","The same abstraction component works across three fundamentally different downstream systems, suggesting it is a general, task-agnostic front-end.","Speedups are achieved without changing the downstream algorithms, so LENS can be dropped into existing pipelines."],"fun_headline_variants":["LLM simplifies clutter to boost three robot control stacks","Scene pruning by LLM lifts three robot stacks in clutter","LLM-driven scene edits improve three robot stacks","Closed-loop LLM scene abstraction boosts three robot methods","LLM scene pruning improves TAMP, MPC, and VLA in clutter"],"cache_read_input_tokens":2304,"weakest_assumption_plain":"The main thing that must hold is that a frozen vision-language model can correctly judge, from an image and a text goal, which objects matter for a task, and that any wrong pruning will be caught by a downstream timeout or error code rather than failing silently.","fun_headline_variants_meta":{"raw":{"variants":["LLM simplifies clutter to boost three robot control stacks","Scene pruning by LLM lifts three robot stacks in clutter","LLM-driven scene edits improve three robot stacks","Closed-loop LLM scene abstraction boosts three robot methods","LLM scene pruning improves TAMP, MPC, and VLA in clutter"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.001422,"raw_usage":{"total_tokens":5575,"prompt_tokens":744,"completion_tokens":4831,"prompt_tokens_details":{"cached_tokens":256},"prompt_cache_hit_tokens":256,"prompt_cache_miss_tokens":488,"completion_tokens_details":{"reasoning_tokens":4760}},"tokens_in":488,"tokens_out":4831,"duration_ms":27993,"temperature":1.0,"reasoning_tokens":4760,"cache_read_input_tokens":256,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-01T12:09:25.958086+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Replace the VLM's object selection with a random baseline (or a nearest-neighbour geometric heuristic) on the same cluttered scenes, keeping every other component identical; if the random baseline achieves comparable success rates, the VLM's semantic reasoning is not the causal mechanism. Alternatively, construct a scene where a critical object is visually similar to a distractor, causing the VLM to prune it, and where the downstream planner completes with a wrong but non-timeout plan—this would expose a silent failure the feedback loop cannot detect.","supporting_citations":[],"review_version":1}